2024-09-21T03:18:38.507 INFO:root:teuthology version: 1.2.2.dev2+g3752d38 2024-09-21T03:18:38.530 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-21T03:18:38.611 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/pdonnell-2024-09-20_23:48:22-fs-wip-pdonnell-testing-20240920.212106-debug-distro-default-smithi/7913466 branch: wip-pdonnell-testing-20240920.212106-debug description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/fuse objectstore-ec/bluestore-comp-ec-root omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/random export-check n/3 replication/default} standby-replay tasks/{0-subvolume/{with-no-extra-options} 1-check-counter 2-scrub/no 3-snaps/no 4-flush/yes 5-quiesce/with-quiesce 6-workunit/suites/fsync-tester}} email: pdonnell@redhat.com first_in_suite: false job_id: '7913466' kernel: kdb: 1 sha1: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi name: pdonnell-2024-09-20_23:48:22-fs-wip-pdonnell-testing-20240920.212106-debug-distro-default-smithi no_nested_subset: false openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-pdonnell-testing-20240920.212106-debug ceph: cephfs: ec_profile: - m=2 - k=2 - crush-failure-domain=osd max_mds: 3 session_timeout: 300 standby_replay: true cluster-conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 mds: debug mds: 20 debug mds balancer: 20 debug ms: 2 mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: client mount timeout: 30 debug client: 20 debug mgr: 20 debug ms: 1 mon warn on pool no app: false mon: mon op complaint time: 120 osd: osd op complaint time: 180 conf: mds: mds_export_ephemeral_random_max: 0.1 mgr: cephsqlite lock renewal timeout: 900000 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 osd: bluestore block size: 96636764160 bluestore compression mode: aggressive bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd shutdown pgref assert: true osd_max_omap_entries_per_request: 10000 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - fs.*is degraded - filesystem is degraded - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - filesystem is offline - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - insufficient standby MDS daemons available - MDS_UP_LESS_THAN_MAX - online, but wants - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - do not have an application enabled - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability - Degraded data redundancy sha1: 1843221895adccfb3b692060251dd6a0183bc410 subvols: create: 2 ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} ceph-fuse: client.0: mount_subvol_num: 0 check-counter: counters: mds: - mds.exported - mds.imported dry_run: true install: ceph: flavor: default sha1: 1843221895adccfb3b692060251dd6a0183bc410 kclient: client.0: mount_subvol_num: 1 selinux: allowlist: - scontext=system_u:system_r:logrotate_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-pdonnell-testing-20240920.212106-debug sha1: 1843221895adccfb3b692060251dd6a0183bc410 owner: scheduled_pdonnell@teuthology priority: 90 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.x - mds.a - mds.d - mds.g - mds.j - osd.0 - osd.3 - osd.6 - osd.9 - client.0 - - mon.b - mgr.y - mds.b - mds.e - mds.h - mds.k - osd.1 - osd.4 - osd.7 - osd.10 - - mon.c - mgr.z - mds.c - mds.f - mds.i - mds.l - osd.2 - osd.5 - osd.8 - osd.11 seed: 7864 sha1: 1843221895adccfb3b692060251dd6a0183bc410 sleep_before_teardown: 0 subset: 67/512 suite: fs suite_branch: wip-pdonnell-testing-20240920.212106-debug suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_1843221895adccfb3b692060251dd6a0183bc410/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: 1843221895adccfb3b692060251dd6a0183bc410 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi089.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBOaXISx0zPpL91TIyeZjsoahmSMyqTgHludfI86bgH6xaFbDVTyUqnsgha/w66tS5E7UaeO0sme7ke+t6zPcT7M= smithi110.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBHs1TwFKoIYwfchkfIkB8qZtiSoyZhUzh/AfkBe5qkrlGYuRGa9zHk5aHqKWPGih2NmfVbNSQTmu6jus5UoG4lE= smithi114.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBOaEbGkiKAWi55ZPkpOBrJIfKvVE9X9+Z8igB5SH1a1GCQUGW92V+YOpDJ4bAP6A/Izab5ZjWgXI+pO3F7BUB2o= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - cephadm: roleless: false - cephadm.shell: mon.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: mon.a: - ceph fs dump - ceph osd dump - fs.ready: timeout: 300 - ceph-fuse: null - exec: mon.a: - ceph fs set cephfs balance_automate false - ceph fs subvolumegroup pin cephfs qa random 0.10 - check-counter: {} - background_exec: mon.a: - while sleep 13; do ceph tell mds.cephfs:0 flush journal; done - quiescer: null - workunit: clients: all: - suites/fsync-tester.sh teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return false end - local function is_kdistro() return false end - local function is_fuse() return true end - local function syntax_version() return '' end teuthology_branch: main teuthology_sha1: 3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a timestamp: 2024-09-20_23:48:22 tube: smithi user: pdonnell verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2902 2024-09-21T03:18:38.612 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_1843221895adccfb3b692060251dd6a0183bc410/qa; will attempt to use it 2024-09-21T03:18:38.612 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_1843221895adccfb3b692060251dd6a0183bc410/qa/tasks 2024-09-21T03:18:38.612 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-09-21T03:18:38.614 INFO:teuthology.task.internal:Checking packages... 2024-09-21T03:18:38.634 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '1843221895adccfb3b692060251dd6a0183bc410' 2024-09-21T03:18:38.634 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-09-21T03:18:38.634 INFO:teuthology.packaging:ref: None 2024-09-21T03:18:38.635 INFO:teuthology.packaging:tag: None 2024-09-21T03:18:38.635 INFO:teuthology.packaging:branch: wip-pdonnell-testing-20240920.212106-debug 2024-09-21T03:18:38.635 INFO:teuthology.packaging:sha1: 1843221895adccfb3b692060251dd6a0183bc410 2024-09-21T03:18:38.635 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-pdonnell-testing-20240920.212106-debug 2024-09-21T03:18:38.782 INFO:teuthology.task.internal:Found packages for ceph version 19.3.0-5120.g18432218 2024-09-21T03:18:38.783 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-09-21T03:18:38.789 INFO:teuthology.task.internal:no buildpackages task found 2024-09-21T03:18:38.789 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-09-21T03:18:38.795 INFO:teuthology.task.internal:Saving configuration 2024-09-21T03:18:38.813 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-09-21T03:18:38.819 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-09-21T03:18:38.844 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi089.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-09-20_23:48:22-fs-wip-pdonnell-testing-20240920.212106-debug-distro-default-smithi/7913466', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-21 03:13:39.010155', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBOaXISx0zPpL91TIyeZjsoahmSMyqTgHludfI86bgH6xaFbDVTyUqnsgha/w66tS5E7UaeO0sme7ke+t6zPcT7M='} 2024-09-21T03:18:38.862 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi110.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-09-20_23:48:22-fs-wip-pdonnell-testing-20240920.212106-debug-distro-default-smithi/7913466', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-21 03:13:39.011475', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBHs1TwFKoIYwfchkfIkB8qZtiSoyZhUzh/AfkBe5qkrlGYuRGa9zHk5aHqKWPGih2NmfVbNSQTmu6jus5UoG4lE='} 2024-09-21T03:18:38.880 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi114.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-09-20_23:48:22-fs-wip-pdonnell-testing-20240920.212106-debug-distro-default-smithi/7913466', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-21 03:13:39.008617', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBOaEbGkiKAWi55ZPkpOBrJIfKvVE9X9+Z8igB5SH1a1GCQUGW92V+YOpDJ4bAP6A/Izab5ZjWgXI+pO3F7BUB2o='} 2024-09-21T03:18:38.880 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-09-21T03:18:38.885 INFO:teuthology.task.internal:roles: ubuntu@smithi089.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'mds.a', 'mds.d', 'mds.g', 'mds.j', 'osd.0', 'osd.3', 'osd.6', 'osd.9', 'client.0'] 2024-09-21T03:18:38.886 INFO:teuthology.task.internal:roles: ubuntu@smithi110.front.sepia.ceph.com - ['mon.b', 'mgr.y', 'mds.b', 'mds.e', 'mds.h', 'mds.k', 'osd.1', 'osd.4', 'osd.7', 'osd.10'] 2024-09-21T03:18:38.886 INFO:teuthology.task.internal:roles: ubuntu@smithi114.front.sepia.ceph.com - ['mon.c', 'mgr.z', 'mds.c', 'mds.f', 'mds.i', 'mds.l', 'osd.2', 'osd.5', 'osd.8', 'osd.11'] 2024-09-21T03:18:38.886 INFO:teuthology.run_tasks:Running task console_log... 2024-09-21T03:18:38.969 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f97a236c160>, signals=[15]) 2024-09-21T03:18:38.969 INFO:teuthology.run_tasks:Running task internal.connect... 2024-09-21T03:18:38.976 INFO:teuthology.task.internal:Opening connections... 2024-09-21T03:18:38.976 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi089.front.sepia.ceph.com 2024-09-21T03:18:38.978 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi089.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T03:18:39.054 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi110.front.sepia.ceph.com 2024-09-21T03:18:39.054 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi110.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T03:18:39.131 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi114.front.sepia.ceph.com 2024-09-21T03:18:39.131 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi114.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T03:18:39.207 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-09-21T03:18:39.212 DEBUG:teuthology.orchestra.run.smithi089:> uname -m 2024-09-21T03:18:39.231 INFO:teuthology.orchestra.run.smithi089.stdout:x86_64 2024-09-21T03:18:39.231 DEBUG:teuthology.orchestra.run.smithi089:> cat /etc/os-release 2024-09-21T03:18:39.289 INFO:teuthology.orchestra.run.smithi089.stdout:NAME="CentOS Stream" 2024-09-21T03:18:39.290 INFO:teuthology.orchestra.run.smithi089.stdout:VERSION="9" 2024-09-21T03:18:39.290 INFO:teuthology.orchestra.run.smithi089.stdout:ID="centos" 2024-09-21T03:18:39.290 INFO:teuthology.orchestra.run.smithi089.stdout:ID_LIKE="rhel fedora" 2024-09-21T03:18:39.290 INFO:teuthology.orchestra.run.smithi089.stdout:VERSION_ID="9" 2024-09-21T03:18:39.290 INFO:teuthology.orchestra.run.smithi089.stdout:PLATFORM_ID="platform:el9" 2024-09-21T03:18:39.290 INFO:teuthology.orchestra.run.smithi089.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-21T03:18:39.290 INFO:teuthology.orchestra.run.smithi089.stdout:ANSI_COLOR="0;31" 2024-09-21T03:18:39.290 INFO:teuthology.orchestra.run.smithi089.stdout:LOGO="fedora-logo-icon" 2024-09-21T03:18:39.290 INFO:teuthology.orchestra.run.smithi089.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-21T03:18:39.290 INFO:teuthology.orchestra.run.smithi089.stdout:HOME_URL="https://centos.org/" 2024-09-21T03:18:39.290 INFO:teuthology.orchestra.run.smithi089.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-21T03:18:39.290 INFO:teuthology.orchestra.run.smithi089.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-21T03:18:39.290 INFO:teuthology.orchestra.run.smithi089.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-21T03:18:39.291 INFO:teuthology.lock.ops:Updating smithi089.front.sepia.ceph.com on lock server 2024-09-21T03:18:39.307 DEBUG:teuthology.orchestra.run.smithi110:> uname -m 2024-09-21T03:18:39.326 INFO:teuthology.orchestra.run.smithi110.stdout:x86_64 2024-09-21T03:18:39.326 DEBUG:teuthology.orchestra.run.smithi110:> cat /etc/os-release 2024-09-21T03:18:39.384 INFO:teuthology.orchestra.run.smithi110.stdout:NAME="CentOS Stream" 2024-09-21T03:18:39.384 INFO:teuthology.orchestra.run.smithi110.stdout:VERSION="9" 2024-09-21T03:18:39.384 INFO:teuthology.orchestra.run.smithi110.stdout:ID="centos" 2024-09-21T03:18:39.384 INFO:teuthology.orchestra.run.smithi110.stdout:ID_LIKE="rhel fedora" 2024-09-21T03:18:39.384 INFO:teuthology.orchestra.run.smithi110.stdout:VERSION_ID="9" 2024-09-21T03:18:39.385 INFO:teuthology.orchestra.run.smithi110.stdout:PLATFORM_ID="platform:el9" 2024-09-21T03:18:39.385 INFO:teuthology.orchestra.run.smithi110.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-21T03:18:39.385 INFO:teuthology.orchestra.run.smithi110.stdout:ANSI_COLOR="0;31" 2024-09-21T03:18:39.385 INFO:teuthology.orchestra.run.smithi110.stdout:LOGO="fedora-logo-icon" 2024-09-21T03:18:39.385 INFO:teuthology.orchestra.run.smithi110.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-21T03:18:39.385 INFO:teuthology.orchestra.run.smithi110.stdout:HOME_URL="https://centos.org/" 2024-09-21T03:18:39.385 INFO:teuthology.orchestra.run.smithi110.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-21T03:18:39.385 INFO:teuthology.orchestra.run.smithi110.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-21T03:18:39.385 INFO:teuthology.orchestra.run.smithi110.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-21T03:18:39.385 INFO:teuthology.lock.ops:Updating smithi110.front.sepia.ceph.com on lock server 2024-09-21T03:18:39.407 DEBUG:teuthology.orchestra.run.smithi114:> uname -m 2024-09-21T03:18:39.425 INFO:teuthology.orchestra.run.smithi114.stdout:x86_64 2024-09-21T03:18:39.425 DEBUG:teuthology.orchestra.run.smithi114:> cat /etc/os-release 2024-09-21T03:18:39.482 INFO:teuthology.orchestra.run.smithi114.stdout:NAME="CentOS Stream" 2024-09-21T03:18:39.482 INFO:teuthology.orchestra.run.smithi114.stdout:VERSION="9" 2024-09-21T03:18:39.482 INFO:teuthology.orchestra.run.smithi114.stdout:ID="centos" 2024-09-21T03:18:39.483 INFO:teuthology.orchestra.run.smithi114.stdout:ID_LIKE="rhel fedora" 2024-09-21T03:18:39.483 INFO:teuthology.orchestra.run.smithi114.stdout:VERSION_ID="9" 2024-09-21T03:18:39.483 INFO:teuthology.orchestra.run.smithi114.stdout:PLATFORM_ID="platform:el9" 2024-09-21T03:18:39.483 INFO:teuthology.orchestra.run.smithi114.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-21T03:18:39.483 INFO:teuthology.orchestra.run.smithi114.stdout:ANSI_COLOR="0;31" 2024-09-21T03:18:39.483 INFO:teuthology.orchestra.run.smithi114.stdout:LOGO="fedora-logo-icon" 2024-09-21T03:18:39.483 INFO:teuthology.orchestra.run.smithi114.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-21T03:18:39.483 INFO:teuthology.orchestra.run.smithi114.stdout:HOME_URL="https://centos.org/" 2024-09-21T03:18:39.483 INFO:teuthology.orchestra.run.smithi114.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-21T03:18:39.483 INFO:teuthology.orchestra.run.smithi114.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-21T03:18:39.483 INFO:teuthology.orchestra.run.smithi114.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-21T03:18:39.484 INFO:teuthology.lock.ops:Updating smithi114.front.sepia.ceph.com on lock server 2024-09-21T03:18:39.499 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-09-21T03:18:39.508 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-09-21T03:18:39.513 INFO:teuthology.task.internal:Checking for old test directory... 2024-09-21T03:18:39.513 DEBUG:teuthology.orchestra.run.smithi089:> test '!' -e /home/ubuntu/cephtest 2024-09-21T03:18:39.516 DEBUG:teuthology.orchestra.run.smithi110:> test '!' -e /home/ubuntu/cephtest 2024-09-21T03:18:39.518 DEBUG:teuthology.orchestra.run.smithi114:> test '!' -e /home/ubuntu/cephtest 2024-09-21T03:18:39.538 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-09-21T03:18:39.544 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-09-21T03:18:39.544 DEBUG:teuthology.orchestra.run.smithi089:> test -z $(ls -A /var/lib/ceph) 2024-09-21T03:18:39.573 DEBUG:teuthology.orchestra.run.smithi110:> test -z $(ls -A /var/lib/ceph) 2024-09-21T03:18:39.575 DEBUG:teuthology.orchestra.run.smithi114:> test -z $(ls -A /var/lib/ceph) 2024-09-21T03:18:39.686 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-09-21T03:18:39.758 INFO:teuthology.run_tasks:Running task kernel... 2024-09-21T03:18:40.187 INFO:teuthology.task.kernel:normalize config orig: {'kdb': 1, 'sha1': 'distro'} 2024-09-21T03:18:40.187 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': 1, 'sha1': 'distro'}, 'mon.b': {'kdb': 1, 'sha1': 'distro'}, 'mon.c': {'kdb': 1, 'sha1': 'distro'}}, timeout 300 2024-09-21T03:18:40.187 DEBUG:teuthology.orchestra.run.smithi089:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-21T03:18:40.188 DEBUG:teuthology.orchestra.run.smithi110:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-21T03:18:40.188 DEBUG:teuthology.orchestra.run.smithi114:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-21T03:18:40.205 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T03:18:40.205 DEBUG:teuthology.orchestra.run.smithi110:> uname -r 2024-09-21T03:18:40.206 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T03:18:40.206 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T03:18:40.207 DEBUG:teuthology.orchestra.run.smithi089:> uname -r 2024-09-21T03:18:40.207 DEBUG:teuthology.orchestra.run.smithi114:> uname -r 2024-09-21T03:18:40.263 INFO:teuthology.orchestra.run.smithi110.stdout:5.14.0-503.el9.x86_64 2024-09-21T03:18:40.263 INFO:teuthology.task.kernel:Running kernel on smithi110: 5.14.0-503.el9.x86_64 2024-09-21T03:18:40.263 DEBUG:teuthology.orchestra.run.smithi110:> sudo yum install -y kernel 2024-09-21T03:18:40.265 INFO:teuthology.orchestra.run.smithi114.stdout:5.14.0-503.el9.x86_64 2024-09-21T03:18:40.265 INFO:teuthology.task.kernel:Running kernel on smithi114: 5.14.0-503.el9.x86_64 2024-09-21T03:18:40.265 DEBUG:teuthology.orchestra.run.smithi114:> sudo yum install -y kernel 2024-09-21T03:18:40.266 INFO:teuthology.orchestra.run.smithi089.stdout:5.14.0-503.el9.x86_64 2024-09-21T03:18:40.266 INFO:teuthology.task.kernel:Running kernel on smithi089: 5.14.0-503.el9.x86_64 2024-09-21T03:18:40.266 DEBUG:teuthology.orchestra.run.smithi089:> sudo yum install -y kernel 2024-09-21T03:18:42.645 INFO:teuthology.orchestra.run.smithi114.stdout:CentOS Stream 9 - BaseOS 13 MB/s | 8.3 MB 00:00 2024-09-21T03:18:42.725 INFO:teuthology.orchestra.run.smithi089.stdout:CentOS Stream 9 - BaseOS 13 MB/s | 8.3 MB 00:00 2024-09-21T03:18:42.912 INFO:teuthology.orchestra.run.smithi110.stdout:CentOS Stream 9 - BaseOS 11 MB/s | 8.3 MB 00:00 2024-09-21T03:18:44.580 INFO:teuthology.orchestra.run.smithi110.stdout:CentOS Stream 9 - AppStream 36 MB/s | 20 MB 00:00 2024-09-21T03:18:48.972 INFO:teuthology.orchestra.run.smithi110.stdout:CentOS Stream 9 - CRB 10 MB/s | 6.5 MB 00:00 2024-09-21T03:18:50.431 INFO:teuthology.orchestra.run.smithi110.stdout:CentOS Stream 9 - Extras packages 87 kB/s | 19 kB 00:00 2024-09-21T03:18:51.202 INFO:teuthology.orchestra.run.smithi110.stdout:Extra Packages for Enterprise Linux 35 MB/s | 23 MB 00:00 2024-09-21T03:18:56.541 INFO:teuthology.orchestra.run.smithi110.stdout:lab-extras 29 kB/s | 1.7 kB 00:00 2024-09-21T03:18:58.439 INFO:teuthology.orchestra.run.smithi110.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-09-21T03:18:58.439 INFO:teuthology.orchestra.run.smithi110.stdout:Package kernel-5.14.0-503.el9.x86_64 is already installed. 2024-09-21T03:18:58.524 INFO:teuthology.orchestra.run.smithi110.stdout:Dependencies resolved. 2024-09-21T03:18:58.531 INFO:teuthology.orchestra.run.smithi110.stdout:================================================================================ 2024-09-21T03:18:58.531 INFO:teuthology.orchestra.run.smithi110.stdout: Package Architecture Version Repository Size 2024-09-21T03:18:58.532 INFO:teuthology.orchestra.run.smithi110.stdout:================================================================================ 2024-09-21T03:18:58.532 INFO:teuthology.orchestra.run.smithi110.stdout:Installing: 2024-09-21T03:18:58.532 INFO:teuthology.orchestra.run.smithi110.stdout: kernel x86_64 5.14.0-508.el9 baseos 36 k 2024-09-21T03:18:58.532 INFO:teuthology.orchestra.run.smithi110.stdout:Installing dependencies: 2024-09-21T03:18:58.532 INFO:teuthology.orchestra.run.smithi110.stdout: kernel-core x86_64 5.14.0-508.el9 baseos 16 M 2024-09-21T03:18:58.532 INFO:teuthology.orchestra.run.smithi110.stdout: kernel-modules x86_64 5.14.0-508.el9 baseos 35 M 2024-09-21T03:18:58.532 INFO:teuthology.orchestra.run.smithi110.stdout: kernel-modules-core x86_64 5.14.0-508.el9 baseos 29 M 2024-09-21T03:18:58.532 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-21T03:18:58.532 INFO:teuthology.orchestra.run.smithi110.stdout:Transaction Summary 2024-09-21T03:18:58.532 INFO:teuthology.orchestra.run.smithi110.stdout:================================================================================ 2024-09-21T03:18:58.532 INFO:teuthology.orchestra.run.smithi110.stdout:Install 4 Packages 2024-09-21T03:18:58.532 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-21T03:18:58.533 INFO:teuthology.orchestra.run.smithi110.stdout:Total download size: 79 M 2024-09-21T03:18:58.533 INFO:teuthology.orchestra.run.smithi110.stdout:Installed size: 126 M 2024-09-21T03:18:58.533 INFO:teuthology.orchestra.run.smithi110.stdout:Downloading Packages: 2024-09-21T03:18:58.892 INFO:teuthology.orchestra.run.smithi110.stdout:(1/4): kernel-5.14.0-508.el9.x86_64.rpm 342 kB/s | 36 kB 00:00 2024-09-21T03:18:59.209 INFO:teuthology.orchestra.run.smithi110.stdout:(2/4): kernel-core-5.14.0-508.el9.x86_64.rpm 37 MB/s | 16 MB 00:00 2024-09-21T03:18:59.535 INFO:teuthology.orchestra.run.smithi110.stdout:(3/4): kernel-modules-core-5.14.0-508.el9.x86_6 45 MB/s | 29 MB 00:00 2024-09-21T03:18:59.868 INFO:teuthology.orchestra.run.smithi110.stdout:(4/4): kernel-modules-5.14.0-508.el9.x86_64.rpm 32 MB/s | 35 MB 00:01 2024-09-21T03:18:59.868 INFO:teuthology.orchestra.run.smithi110.stdout:-------------------------------------------------------------------------------- 2024-09-21T03:18:59.869 INFO:teuthology.orchestra.run.smithi110.stdout:Total 59 MB/s | 79 MB 00:01 2024-09-21T03:19:00.555 INFO:teuthology.orchestra.run.smithi110.stdout:Running transaction check 2024-09-21T03:19:00.921 INFO:teuthology.orchestra.run.smithi110.stdout:Transaction check succeeded. 2024-09-21T03:19:00.921 INFO:teuthology.orchestra.run.smithi110.stdout:Running transaction test 2024-09-21T03:19:01.867 INFO:teuthology.orchestra.run.smithi110.stdout:Transaction test succeeded. 2024-09-21T03:19:01.867 INFO:teuthology.orchestra.run.smithi110.stdout:Running transaction 2024-09-21T03:19:02.124 INFO:teuthology.orchestra.run.smithi089.stdout:CentOS Stream 9 - AppStream 1.1 MB/s | 20 MB 00:18 2024-09-21T03:19:02.894 INFO:teuthology.orchestra.run.smithi114.stdout:CentOS Stream 9 - AppStream 1.1 MB/s | 20 MB 00:19 2024-09-21T03:19:04.282 INFO:teuthology.orchestra.run.smithi110.stdout: Preparing : 1/1 2024-09-21T03:19:04.979 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : kernel-modules-core-5.14.0-508.el9.x86_64 1/4 2024-09-21T03:19:05.062 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : kernel-core-5.14.0-508.el9.x86_64 2/4 2024-09-21T03:19:07.804 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: kernel-core-5.14.0-508.el9.x86_64 2/4 2024-09-21T03:19:07.939 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : kernel-modules-5.14.0-508.el9.x86_64 3/4 2024-09-21T03:19:08.445 INFO:teuthology.orchestra.run.smithi114.stdout:CentOS Stream 9 - CRB 3.6 MB/s | 6.5 MB 00:01 2024-09-21T03:19:10.666 INFO:teuthology.orchestra.run.smithi089.stdout:CentOS Stream 9 - CRB 1.4 MB/s | 6.5 MB 00:04 2024-09-21T03:19:10.887 INFO:teuthology.orchestra.run.smithi114.stdout:CentOS Stream 9 - Extras packages 16 kB/s | 19 kB 00:01 2024-09-21T03:19:11.699 INFO:teuthology.orchestra.run.smithi114.stdout:Extra Packages for Enterprise Linux 33 MB/s | 23 MB 00:00 2024-09-21T03:19:12.475 INFO:teuthology.orchestra.run.smithi089.stdout:CentOS Stream 9 - Extras packages 44 kB/s | 19 kB 00:00 2024-09-21T03:19:12.747 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: kernel-modules-5.14.0-508.el9.x86_64 3/4 2024-09-21T03:19:12.847 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : kernel-5.14.0-508.el9.x86_64 4/4 2024-09-21T03:19:13.421 INFO:teuthology.orchestra.run.smithi089.stdout:Extra Packages for Enterprise Linux 28 MB/s | 23 MB 00:00 2024-09-21T03:19:17.064 INFO:teuthology.orchestra.run.smithi114.stdout:lab-extras 35 kB/s | 1.7 kB 00:00 2024-09-21T03:19:17.665 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: kernel-modules-core-5.14.0-508.el9.x86_64 4/4 2024-09-21T03:19:18.702 INFO:teuthology.orchestra.run.smithi089.stdout:lab-extras 39 kB/s | 1.7 kB 00:00 2024-09-21T03:19:18.912 INFO:teuthology.orchestra.run.smithi114.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-09-21T03:19:18.912 INFO:teuthology.orchestra.run.smithi114.stdout:Package kernel-5.14.0-503.el9.x86_64 is already installed. 2024-09-21T03:19:19.013 INFO:teuthology.orchestra.run.smithi114.stdout:Dependencies resolved. 2024-09-21T03:19:19.019 INFO:teuthology.orchestra.run.smithi114.stdout:================================================================================ 2024-09-21T03:19:19.020 INFO:teuthology.orchestra.run.smithi114.stdout: Package Architecture Version Repository Size 2024-09-21T03:19:19.020 INFO:teuthology.orchestra.run.smithi114.stdout:================================================================================ 2024-09-21T03:19:19.020 INFO:teuthology.orchestra.run.smithi114.stdout:Installing: 2024-09-21T03:19:19.020 INFO:teuthology.orchestra.run.smithi114.stdout: kernel x86_64 5.14.0-508.el9 baseos 36 k 2024-09-21T03:19:19.020 INFO:teuthology.orchestra.run.smithi114.stdout:Installing dependencies: 2024-09-21T03:19:19.020 INFO:teuthology.orchestra.run.smithi114.stdout: kernel-core x86_64 5.14.0-508.el9 baseos 16 M 2024-09-21T03:19:19.020 INFO:teuthology.orchestra.run.smithi114.stdout: kernel-modules x86_64 5.14.0-508.el9 baseos 35 M 2024-09-21T03:19:19.020 INFO:teuthology.orchestra.run.smithi114.stdout: kernel-modules-core x86_64 5.14.0-508.el9 baseos 29 M 2024-09-21T03:19:19.021 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-21T03:19:19.021 INFO:teuthology.orchestra.run.smithi114.stdout:Transaction Summary 2024-09-21T03:19:19.021 INFO:teuthology.orchestra.run.smithi114.stdout:================================================================================ 2024-09-21T03:19:19.021 INFO:teuthology.orchestra.run.smithi114.stdout:Install 4 Packages 2024-09-21T03:19:19.021 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-21T03:19:19.021 INFO:teuthology.orchestra.run.smithi114.stdout:Total download size: 79 M 2024-09-21T03:19:19.021 INFO:teuthology.orchestra.run.smithi114.stdout:Installed size: 126 M 2024-09-21T03:19:19.021 INFO:teuthology.orchestra.run.smithi114.stdout:Downloading Packages: 2024-09-21T03:19:19.865 INFO:teuthology.orchestra.run.smithi114.stdout:(1/4): kernel-5.14.0-508.el9.x86_64.rpm 64 kB/s | 36 kB 00:00 2024-09-21T03:19:20.583 INFO:teuthology.orchestra.run.smithi089.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-09-21T03:19:20.583 INFO:teuthology.orchestra.run.smithi089.stdout:Package kernel-5.14.0-503.el9.x86_64 is already installed. 2024-09-21T03:19:20.678 INFO:teuthology.orchestra.run.smithi089.stdout:Dependencies resolved. 2024-09-21T03:19:20.687 INFO:teuthology.orchestra.run.smithi089.stdout:================================================================================ 2024-09-21T03:19:20.687 INFO:teuthology.orchestra.run.smithi089.stdout: Package Architecture Version Repository Size 2024-09-21T03:19:20.687 INFO:teuthology.orchestra.run.smithi089.stdout:================================================================================ 2024-09-21T03:19:20.687 INFO:teuthology.orchestra.run.smithi089.stdout:Installing: 2024-09-21T03:19:20.687 INFO:teuthology.orchestra.run.smithi089.stdout: kernel x86_64 5.14.0-508.el9 baseos 36 k 2024-09-21T03:19:20.687 INFO:teuthology.orchestra.run.smithi089.stdout:Installing dependencies: 2024-09-21T03:19:20.687 INFO:teuthology.orchestra.run.smithi089.stdout: kernel-core x86_64 5.14.0-508.el9 baseos 16 M 2024-09-21T03:19:20.687 INFO:teuthology.orchestra.run.smithi089.stdout: kernel-modules x86_64 5.14.0-508.el9 baseos 35 M 2024-09-21T03:19:20.688 INFO:teuthology.orchestra.run.smithi089.stdout: kernel-modules-core x86_64 5.14.0-508.el9 baseos 29 M 2024-09-21T03:19:20.688 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-09-21T03:19:20.688 INFO:teuthology.orchestra.run.smithi089.stdout:Transaction Summary 2024-09-21T03:19:20.688 INFO:teuthology.orchestra.run.smithi089.stdout:================================================================================ 2024-09-21T03:19:20.688 INFO:teuthology.orchestra.run.smithi089.stdout:Install 4 Packages 2024-09-21T03:19:20.688 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-09-21T03:19:20.688 INFO:teuthology.orchestra.run.smithi089.stdout:Total download size: 79 M 2024-09-21T03:19:20.689 INFO:teuthology.orchestra.run.smithi089.stdout:Installed size: 126 M 2024-09-21T03:19:20.689 INFO:teuthology.orchestra.run.smithi089.stdout:Downloading Packages: 2024-09-21T03:19:20.795 INFO:teuthology.orchestra.run.smithi089.stdout:(1/4): kernel-5.14.0-508.el9.x86_64.rpm 423 kB/s | 36 kB 00:00 2024-09-21T03:19:20.865 INFO:teuthology.orchestra.run.smithi114.stdout:(2/4): kernel-core-5.14.0-508.el9.x86_64.rpm 10 MB/s | 16 MB 00:01 2024-09-21T03:19:21.312 INFO:teuthology.orchestra.run.smithi089.stdout:(2/4): kernel-core-5.14.0-508.el9.x86_64.rpm 26 MB/s | 16 MB 00:00 2024-09-21T03:19:21.582 INFO:teuthology.orchestra.run.smithi114.stdout:(3/4): kernel-modules-5.14.0-508.el9.x86_64.rpm 15 MB/s | 35 MB 00:02 2024-09-21T03:19:21.637 INFO:teuthology.orchestra.run.smithi089.stdout:(3/4): kernel-modules-5.14.0-508.el9.x86_64.rpm 37 MB/s | 35 MB 00:00 2024-09-21T03:19:22.012 INFO:teuthology.orchestra.run.smithi089.stdout:(4/4): kernel-modules-core-5.14.0-508.el9.x86_6 23 MB/s | 29 MB 00:01 2024-09-21T03:19:22.013 INFO:teuthology.orchestra.run.smithi089.stdout:-------------------------------------------------------------------------------- 2024-09-21T03:19:22.013 INFO:teuthology.orchestra.run.smithi089.stdout:Total 59 MB/s | 79 MB 00:01 2024-09-21T03:19:22.241 INFO:teuthology.orchestra.run.smithi114.stdout:(4/4): kernel-modules-core-5.14.0-508.el9.x86_6 12 MB/s | 29 MB 00:02 2024-09-21T03:19:22.241 INFO:teuthology.orchestra.run.smithi114.stdout:-------------------------------------------------------------------------------- 2024-09-21T03:19:22.241 INFO:teuthology.orchestra.run.smithi114.stdout:Total 24 MB/s | 79 MB 00:03 2024-09-21T03:19:22.693 INFO:teuthology.orchestra.run.smithi089.stdout:Running transaction check 2024-09-21T03:19:22.923 INFO:teuthology.orchestra.run.smithi114.stdout:Running transaction check 2024-09-21T03:19:23.034 INFO:teuthology.orchestra.run.smithi089.stdout:Transaction check succeeded. 2024-09-21T03:19:23.034 INFO:teuthology.orchestra.run.smithi089.stdout:Running transaction test 2024-09-21T03:19:23.270 INFO:teuthology.orchestra.run.smithi114.stdout:Transaction check succeeded. 2024-09-21T03:19:23.271 INFO:teuthology.orchestra.run.smithi114.stdout:Running transaction test 2024-09-21T03:19:24.009 INFO:teuthology.orchestra.run.smithi089.stdout:Transaction test succeeded. 2024-09-21T03:19:24.009 INFO:teuthology.orchestra.run.smithi089.stdout:Running transaction 2024-09-21T03:19:24.243 INFO:teuthology.orchestra.run.smithi114.stdout:Transaction test succeeded. 2024-09-21T03:19:24.243 INFO:teuthology.orchestra.run.smithi114.stdout:Running transaction 2024-09-21T03:19:26.406 INFO:teuthology.orchestra.run.smithi089.stdout: Preparing : 1/1 2024-09-21T03:19:26.673 INFO:teuthology.orchestra.run.smithi114.stdout: Preparing : 1/1 2024-09-21T03:19:27.062 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : kernel-modules-core-5.14.0-508.el9.x86_64 1/4 2024-09-21T03:19:27.155 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : kernel-core-5.14.0-508.el9.x86_64 2/4 2024-09-21T03:19:27.352 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : kernel-modules-core-5.14.0-508.el9.x86_64 1/4 2024-09-21T03:19:27.441 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : kernel-core-5.14.0-508.el9.x86_64 2/4 2024-09-21T03:19:29.914 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: kernel-core-5.14.0-508.el9.x86_64 2/4 2024-09-21T03:19:30.050 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : kernel-modules-5.14.0-508.el9.x86_64 3/4 2024-09-21T03:19:30.248 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: kernel-core-5.14.0-508.el9.x86_64 2/4 2024-09-21T03:19:30.384 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : kernel-modules-5.14.0-508.el9.x86_64 3/4 2024-09-21T03:19:34.857 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: kernel-modules-5.14.0-508.el9.x86_64 3/4 2024-09-21T03:19:34.951 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : kernel-5.14.0-508.el9.x86_64 4/4 2024-09-21T03:19:35.226 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: kernel-modules-5.14.0-508.el9.x86_64 3/4 2024-09-21T03:19:35.267 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : kernel-5.14.0-508.el9.x86_64 4/4 2024-09-21T03:19:39.795 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: kernel-modules-core-5.14.0-508.el9.x86_64 4/4 2024-09-21T03:19:40.127 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: kernel-modules-core-5.14.0-508.el9.x86_64 4/4 2024-09-21T03:19:55.615 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: kernel-core-5.14.0-508.el9.x86_64 4/4 2024-09-21T03:19:55.655 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: kernel-modules-5.14.0-508.el9.x86_64 4/4 2024-09-21T03:19:56.031 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: kernel-5.14.0-508.el9.x86_64 4/4 2024-09-21T03:19:56.031 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : kernel-5.14.0-508.el9.x86_64 1/4 2024-09-21T03:19:56.031 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : kernel-core-5.14.0-508.el9.x86_64 2/4 2024-09-21T03:19:56.032 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : kernel-modules-5.14.0-508.el9.x86_64 3/4 2024-09-21T03:19:56.903 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : kernel-modules-core-5.14.0-508.el9.x86_64 4/4 2024-09-21T03:19:56.903 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-21T03:19:56.903 INFO:teuthology.orchestra.run.smithi110.stdout:Installed: 2024-09-21T03:19:56.903 INFO:teuthology.orchestra.run.smithi110.stdout: kernel-5.14.0-508.el9.x86_64 2024-09-21T03:19:56.903 INFO:teuthology.orchestra.run.smithi110.stdout: kernel-core-5.14.0-508.el9.x86_64 2024-09-21T03:19:56.904 INFO:teuthology.orchestra.run.smithi110.stdout: kernel-modules-5.14.0-508.el9.x86_64 2024-09-21T03:19:56.904 INFO:teuthology.orchestra.run.smithi110.stdout: kernel-modules-core-5.14.0-508.el9.x86_64 2024-09-21T03:19:56.904 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-21T03:19:56.904 INFO:teuthology.orchestra.run.smithi110.stdout:Complete! 2024-09-21T03:19:57.258 DEBUG:teuthology.orchestra.run.smithi110:> rpm -q kernel | sort -rV | head -n 1 2024-09-21T03:19:57.352 INFO:teuthology.orchestra.run.smithi110.stdout:kernel-5.14.0-508.el9.x86_64 2024-09-21T03:19:57.352 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-508.el9.x86_64 2024-09-21T03:19:57.352 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-503.el9.x86_64 Expected: 5.14.0-508.el9.x86_64 2024-09-21T03:19:57.352 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-21T03:19:57.352 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-21T03:19:57.352 INFO:teuthology.task.kernel:Installing distro kernel on mon.b... 2024-09-21T03:19:57.352 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi110.front.sepia.ceph.com, path=None, version=distro) 2024-09-21T03:19:57.353 DEBUG:teuthology.orchestra.run.smithi110:> rpm -q kernel | sort -rV | head -n 1 2024-09-21T03:19:57.411 INFO:teuthology.orchestra.run.smithi110.stdout:kernel-5.14.0-508.el9.x86_64 2024-09-21T03:19:57.411 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-508.el9.x86_64 2024-09-21T03:19:57.411 DEBUG:teuthology.orchestra.run.smithi110:> sudo rpm -qi grub2-tools 2024-09-21T03:19:57.443 INFO:teuthology.orchestra.run.smithi110.stdout:Name : grub2-tools 2024-09-21T03:19:57.443 INFO:teuthology.orchestra.run.smithi110.stdout:Epoch : 1 2024-09-21T03:19:57.443 INFO:teuthology.orchestra.run.smithi110.stdout:Version : 2.06 2024-09-21T03:19:57.443 INFO:teuthology.orchestra.run.smithi110.stdout:Release : 82.el9 2024-09-21T03:19:57.443 INFO:teuthology.orchestra.run.smithi110.stdout:Architecture: x86_64 2024-09-21T03:19:57.443 INFO:teuthology.orchestra.run.smithi110.stdout:Install Date: Wed 28 Aug 2024 09:06:27 PM UTC 2024-09-21T03:19:57.443 INFO:teuthology.orchestra.run.smithi110.stdout:Group : Unspecified 2024-09-21T03:19:57.443 INFO:teuthology.orchestra.run.smithi110.stdout:Size : 8274670 2024-09-21T03:19:57.443 INFO:teuthology.orchestra.run.smithi110.stdout:License : GPLv3+ 2024-09-21T03:19:57.444 INFO:teuthology.orchestra.run.smithi110.stdout:Signature : RSA/SHA256, Mon 01 Jul 2024 07:46:42 AM UTC, Key ID 05b555b38483c65d 2024-09-21T03:19:57.444 INFO:teuthology.orchestra.run.smithi110.stdout:Source RPM : grub2-2.06-82.el9.src.rpm 2024-09-21T03:19:57.444 INFO:teuthology.orchestra.run.smithi110.stdout:Build Date : Fri 28 Jun 2024 09:24:44 AM UTC 2024-09-21T03:19:57.444 INFO:teuthology.orchestra.run.smithi110.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-09-21T03:19:57.444 INFO:teuthology.orchestra.run.smithi110.stdout:Packager : builder@centos.org 2024-09-21T03:19:57.444 INFO:teuthology.orchestra.run.smithi110.stdout:Vendor : CentOS 2024-09-21T03:19:57.444 INFO:teuthology.orchestra.run.smithi110.stdout:URL : http://www.gnu.org/software/grub/ 2024-09-21T03:19:57.444 INFO:teuthology.orchestra.run.smithi110.stdout:Summary : Support tools for GRUB. 2024-09-21T03:19:57.444 INFO:teuthology.orchestra.run.smithi110.stdout:Description : 2024-09-21T03:19:57.444 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-21T03:19:57.444 INFO:teuthology.orchestra.run.smithi110.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-09-21T03:19:57.444 INFO:teuthology.orchestra.run.smithi110.stdout:customizable bootloader with modular architecture. It supports a rich 2024-09-21T03:19:57.444 INFO:teuthology.orchestra.run.smithi110.stdout:variety of kernel formats, file systems, computer architectures and 2024-09-21T03:19:57.444 INFO:teuthology.orchestra.run.smithi110.stdout:hardware devices. 2024-09-21T03:19:57.444 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-21T03:19:57.445 INFO:teuthology.orchestra.run.smithi110.stdout:This subpackage provides tools for support of all platforms. 2024-09-21T03:19:57.445 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-09-21T03:19:57.445 INFO:teuthology.task.kernel:Updating grub on smithi110 to boot 5.14.0-508.el9.x86_64 2024-09-21T03:19:57.446 DEBUG:teuthology.orchestra.run.smithi110:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-09-21T03:19:58.089 INFO:teuthology.orchestra.run.smithi110.stderr:Generating grub configuration file ... 2024-09-21T03:19:59.475 INFO:teuthology.orchestra.run.smithi110.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-09-21T03:19:59.513 INFO:teuthology.orchestra.run.smithi110.stderr:done 2024-09-21T03:19:59.516 DEBUG:teuthology.orchestra.run.smithi110:> mktemp 2024-09-21T03:19:59.532 INFO:teuthology.orchestra.run.smithi110.stdout:/tmp/tmp.AwYzju6MQB 2024-09-21T03:19:59.532 DEBUG:teuthology.orchestra.run.smithi110:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.AwYzju6MQB 2024-09-21T03:19:59.611 DEBUG:teuthology.orchestra.run.smithi110:> sudo chmod 0666 /tmp/tmp.AwYzju6MQB 2024-09-21T03:19:59.763 DEBUG:teuthology.orchestra.remote:smithi110:/tmp/tmp.AwYzju6MQB is 6KB 2024-09-21T03:19:59.787 DEBUG:teuthology.orchestra.run.smithi110:> rm -fr /tmp/tmp.AwYzju6MQB 2024-09-21T03:19:59.804 DEBUG:teuthology.orchestra.run.smithi110:> sudo /bin/ls /boot/loader/entries || true 2024-09-21T03:19:59.869 INFO:teuthology.orchestra.run.smithi110.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-0-rescue.conf 2024-09-21T03:19:59.869 INFO:teuthology.orchestra.run.smithi110.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-239.el9.x86_64.conf 2024-09-21T03:19:59.869 INFO:teuthology.orchestra.run.smithi110.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-503.el9.x86_64.conf 2024-09-21T03:19:59.869 INFO:teuthology.orchestra.run.smithi110.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-508.el9.x86_64.conf 2024-09-21T03:19:59.870 DEBUG:teuthology.orchestra.run.smithi110:> sudo grub2-set-default fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-508.el9.x86_64 2024-09-21T03:20:00.014 DEBUG:teuthology.orchestra.run.smithi110:> sudo shutdown -r now 2024-09-21T03:20:00.104 ERROR:paramiko.transport:Socket exception: Connection reset by peer (104) 2024-09-21T03:20:18.155 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: kernel-core-5.14.0-508.el9.x86_64 4/4 2024-09-21T03:20:18.198 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: kernel-modules-5.14.0-508.el9.x86_64 4/4 2024-09-21T03:20:18.361 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: kernel-core-5.14.0-508.el9.x86_64 4/4 2024-09-21T03:20:18.405 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: kernel-modules-5.14.0-508.el9.x86_64 4/4 2024-09-21T03:20:18.568 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: kernel-5.14.0-508.el9.x86_64 4/4 2024-09-21T03:20:18.568 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : kernel-5.14.0-508.el9.x86_64 1/4 2024-09-21T03:20:18.568 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : kernel-core-5.14.0-508.el9.x86_64 2/4 2024-09-21T03:20:18.568 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : kernel-modules-5.14.0-508.el9.x86_64 3/4 2024-09-21T03:20:18.770 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: kernel-5.14.0-508.el9.x86_64 4/4 2024-09-21T03:20:18.770 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : kernel-5.14.0-508.el9.x86_64 1/4 2024-09-21T03:20:18.770 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : kernel-core-5.14.0-508.el9.x86_64 2/4 2024-09-21T03:20:18.770 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : kernel-modules-5.14.0-508.el9.x86_64 3/4 2024-09-21T03:20:19.422 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : kernel-modules-core-5.14.0-508.el9.x86_64 4/4 2024-09-21T03:20:19.422 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-09-21T03:20:19.423 INFO:teuthology.orchestra.run.smithi089.stdout:Installed: 2024-09-21T03:20:19.423 INFO:teuthology.orchestra.run.smithi089.stdout: kernel-5.14.0-508.el9.x86_64 2024-09-21T03:20:19.423 INFO:teuthology.orchestra.run.smithi089.stdout: kernel-core-5.14.0-508.el9.x86_64 2024-09-21T03:20:19.423 INFO:teuthology.orchestra.run.smithi089.stdout: kernel-modules-5.14.0-508.el9.x86_64 2024-09-21T03:20:19.423 INFO:teuthology.orchestra.run.smithi089.stdout: kernel-modules-core-5.14.0-508.el9.x86_64 2024-09-21T03:20:19.423 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-09-21T03:20:19.423 INFO:teuthology.orchestra.run.smithi089.stdout:Complete! 2024-09-21T03:20:19.701 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : kernel-modules-core-5.14.0-508.el9.x86_64 4/4 2024-09-21T03:20:19.701 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-21T03:20:19.701 INFO:teuthology.orchestra.run.smithi114.stdout:Installed: 2024-09-21T03:20:19.701 INFO:teuthology.orchestra.run.smithi114.stdout: kernel-5.14.0-508.el9.x86_64 2024-09-21T03:20:19.701 INFO:teuthology.orchestra.run.smithi114.stdout: kernel-core-5.14.0-508.el9.x86_64 2024-09-21T03:20:19.701 INFO:teuthology.orchestra.run.smithi114.stdout: kernel-modules-5.14.0-508.el9.x86_64 2024-09-21T03:20:19.701 INFO:teuthology.orchestra.run.smithi114.stdout: kernel-modules-core-5.14.0-508.el9.x86_64 2024-09-21T03:20:19.701 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-21T03:20:19.701 INFO:teuthology.orchestra.run.smithi114.stdout:Complete! 2024-09-21T03:20:19.778 DEBUG:teuthology.orchestra.run.smithi089:> rpm -q kernel | sort -rV | head -n 1 2024-09-21T03:20:19.867 INFO:teuthology.orchestra.run.smithi089.stdout:kernel-5.14.0-508.el9.x86_64 2024-09-21T03:20:19.867 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-508.el9.x86_64 2024-09-21T03:20:19.867 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-503.el9.x86_64 Expected: 5.14.0-508.el9.x86_64 2024-09-21T03:20:19.868 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-21T03:20:19.868 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-21T03:20:19.868 INFO:teuthology.task.kernel:Installing distro kernel on mon.a... 2024-09-21T03:20:19.868 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi089.front.sepia.ceph.com, path=None, version=distro) 2024-09-21T03:20:19.868 DEBUG:teuthology.orchestra.run.smithi089:> rpm -q kernel | sort -rV | head -n 1 2024-09-21T03:20:19.927 INFO:teuthology.orchestra.run.smithi089.stdout:kernel-5.14.0-508.el9.x86_64 2024-09-21T03:20:19.927 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-508.el9.x86_64 2024-09-21T03:20:19.927 DEBUG:teuthology.orchestra.run.smithi089:> sudo rpm -qi grub2-tools 2024-09-21T03:20:19.961 INFO:teuthology.orchestra.run.smithi089.stdout:Name : grub2-tools 2024-09-21T03:20:19.961 INFO:teuthology.orchestra.run.smithi089.stdout:Epoch : 1 2024-09-21T03:20:19.961 INFO:teuthology.orchestra.run.smithi089.stdout:Version : 2.06 2024-09-21T03:20:19.961 INFO:teuthology.orchestra.run.smithi089.stdout:Release : 82.el9 2024-09-21T03:20:19.961 INFO:teuthology.orchestra.run.smithi089.stdout:Architecture: x86_64 2024-09-21T03:20:19.962 INFO:teuthology.orchestra.run.smithi089.stdout:Install Date: Wed 28 Aug 2024 09:06:27 PM UTC 2024-09-21T03:20:19.962 INFO:teuthology.orchestra.run.smithi089.stdout:Group : Unspecified 2024-09-21T03:20:19.962 INFO:teuthology.orchestra.run.smithi089.stdout:Size : 8274670 2024-09-21T03:20:19.962 INFO:teuthology.orchestra.run.smithi089.stdout:License : GPLv3+ 2024-09-21T03:20:19.962 INFO:teuthology.orchestra.run.smithi089.stdout:Signature : RSA/SHA256, Mon 01 Jul 2024 07:46:42 AM UTC, Key ID 05b555b38483c65d 2024-09-21T03:20:19.962 INFO:teuthology.orchestra.run.smithi089.stdout:Source RPM : grub2-2.06-82.el9.src.rpm 2024-09-21T03:20:19.962 INFO:teuthology.orchestra.run.smithi089.stdout:Build Date : Fri 28 Jun 2024 09:24:44 AM UTC 2024-09-21T03:20:19.962 INFO:teuthology.orchestra.run.smithi089.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-09-21T03:20:19.962 INFO:teuthology.orchestra.run.smithi089.stdout:Packager : builder@centos.org 2024-09-21T03:20:19.962 INFO:teuthology.orchestra.run.smithi089.stdout:Vendor : CentOS 2024-09-21T03:20:19.962 INFO:teuthology.orchestra.run.smithi089.stdout:URL : http://www.gnu.org/software/grub/ 2024-09-21T03:20:19.962 INFO:teuthology.orchestra.run.smithi089.stdout:Summary : Support tools for GRUB. 2024-09-21T03:20:19.962 INFO:teuthology.orchestra.run.smithi089.stdout:Description : 2024-09-21T03:20:19.962 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-09-21T03:20:19.963 INFO:teuthology.orchestra.run.smithi089.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-09-21T03:20:19.963 INFO:teuthology.orchestra.run.smithi089.stdout:customizable bootloader with modular architecture. It supports a rich 2024-09-21T03:20:19.963 INFO:teuthology.orchestra.run.smithi089.stdout:variety of kernel formats, file systems, computer architectures and 2024-09-21T03:20:19.963 INFO:teuthology.orchestra.run.smithi089.stdout:hardware devices. 2024-09-21T03:20:19.963 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-09-21T03:20:19.963 INFO:teuthology.orchestra.run.smithi089.stdout:This subpackage provides tools for support of all platforms. 2024-09-21T03:20:19.964 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-09-21T03:20:19.964 INFO:teuthology.task.kernel:Updating grub on smithi089 to boot 5.14.0-508.el9.x86_64 2024-09-21T03:20:19.964 DEBUG:teuthology.orchestra.run.smithi089:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-09-21T03:20:20.058 DEBUG:teuthology.orchestra.run.smithi114:> rpm -q kernel | sort -rV | head -n 1 2024-09-21T03:20:20.140 INFO:teuthology.orchestra.run.smithi114.stdout:kernel-5.14.0-508.el9.x86_64 2024-09-21T03:20:20.140 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-508.el9.x86_64 2024-09-21T03:20:20.140 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-503.el9.x86_64 Expected: 5.14.0-508.el9.x86_64 2024-09-21T03:20:20.140 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-21T03:20:20.140 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-21T03:20:20.140 INFO:teuthology.task.kernel:Installing distro kernel on mon.c... 2024-09-21T03:20:20.140 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi114.front.sepia.ceph.com, path=None, version=distro) 2024-09-21T03:20:20.140 DEBUG:teuthology.orchestra.run.smithi114:> rpm -q kernel | sort -rV | head -n 1 2024-09-21T03:20:20.198 INFO:teuthology.orchestra.run.smithi114.stdout:kernel-5.14.0-508.el9.x86_64 2024-09-21T03:20:20.198 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-508.el9.x86_64 2024-09-21T03:20:20.199 DEBUG:teuthology.orchestra.run.smithi114:> sudo rpm -qi grub2-tools 2024-09-21T03:20:20.231 INFO:teuthology.orchestra.run.smithi114.stdout:Name : grub2-tools 2024-09-21T03:20:20.231 INFO:teuthology.orchestra.run.smithi114.stdout:Epoch : 1 2024-09-21T03:20:20.231 INFO:teuthology.orchestra.run.smithi114.stdout:Version : 2.06 2024-09-21T03:20:20.231 INFO:teuthology.orchestra.run.smithi114.stdout:Release : 82.el9 2024-09-21T03:20:20.231 INFO:teuthology.orchestra.run.smithi114.stdout:Architecture: x86_64 2024-09-21T03:20:20.231 INFO:teuthology.orchestra.run.smithi114.stdout:Install Date: Wed 28 Aug 2024 09:06:27 PM UTC 2024-09-21T03:20:20.231 INFO:teuthology.orchestra.run.smithi114.stdout:Group : Unspecified 2024-09-21T03:20:20.231 INFO:teuthology.orchestra.run.smithi114.stdout:Size : 8274670 2024-09-21T03:20:20.231 INFO:teuthology.orchestra.run.smithi114.stdout:License : GPLv3+ 2024-09-21T03:20:20.231 INFO:teuthology.orchestra.run.smithi114.stdout:Signature : RSA/SHA256, Mon 01 Jul 2024 07:46:42 AM UTC, Key ID 05b555b38483c65d 2024-09-21T03:20:20.231 INFO:teuthology.orchestra.run.smithi114.stdout:Source RPM : grub2-2.06-82.el9.src.rpm 2024-09-21T03:20:20.231 INFO:teuthology.orchestra.run.smithi114.stdout:Build Date : Fri 28 Jun 2024 09:24:44 AM UTC 2024-09-21T03:20:20.231 INFO:teuthology.orchestra.run.smithi114.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-09-21T03:20:20.232 INFO:teuthology.orchestra.run.smithi114.stdout:Packager : builder@centos.org 2024-09-21T03:20:20.232 INFO:teuthology.orchestra.run.smithi114.stdout:Vendor : CentOS 2024-09-21T03:20:20.232 INFO:teuthology.orchestra.run.smithi114.stdout:URL : http://www.gnu.org/software/grub/ 2024-09-21T03:20:20.232 INFO:teuthology.orchestra.run.smithi114.stdout:Summary : Support tools for GRUB. 2024-09-21T03:20:20.232 INFO:teuthology.orchestra.run.smithi114.stdout:Description : 2024-09-21T03:20:20.232 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-21T03:20:20.232 INFO:teuthology.orchestra.run.smithi114.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-09-21T03:20:20.232 INFO:teuthology.orchestra.run.smithi114.stdout:customizable bootloader with modular architecture. It supports a rich 2024-09-21T03:20:20.232 INFO:teuthology.orchestra.run.smithi114.stdout:variety of kernel formats, file systems, computer architectures and 2024-09-21T03:20:20.232 INFO:teuthology.orchestra.run.smithi114.stdout:hardware devices. 2024-09-21T03:20:20.232 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-21T03:20:20.232 INFO:teuthology.orchestra.run.smithi114.stdout:This subpackage provides tools for support of all platforms. 2024-09-21T03:20:20.233 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-09-21T03:20:20.233 INFO:teuthology.task.kernel:Updating grub on smithi114 to boot 5.14.0-508.el9.x86_64 2024-09-21T03:20:20.233 DEBUG:teuthology.orchestra.run.smithi114:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-09-21T03:20:20.711 INFO:teuthology.orchestra.run.smithi089.stderr:Generating grub configuration file ... 2024-09-21T03:20:20.939 INFO:teuthology.orchestra.run.smithi114.stderr:Generating grub configuration file ... 2024-09-21T03:20:22.086 INFO:teuthology.orchestra.run.smithi089.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-09-21T03:20:22.128 INFO:teuthology.orchestra.run.smithi089.stderr:done 2024-09-21T03:20:22.130 DEBUG:teuthology.orchestra.run.smithi089:> mktemp 2024-09-21T03:20:22.145 INFO:teuthology.orchestra.run.smithi089.stdout:/tmp/tmp.FgkTrwL2fa 2024-09-21T03:20:22.145 DEBUG:teuthology.orchestra.run.smithi089:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.FgkTrwL2fa 2024-09-21T03:20:22.213 DEBUG:teuthology.orchestra.run.smithi089:> sudo chmod 0666 /tmp/tmp.FgkTrwL2fa 2024-09-21T03:20:22.285 INFO:teuthology.orchestra.run.smithi114.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-09-21T03:20:22.317 INFO:teuthology.orchestra.run.smithi114.stderr:done 2024-09-21T03:20:22.319 DEBUG:teuthology.orchestra.run.smithi114:> mktemp 2024-09-21T03:20:22.335 INFO:teuthology.orchestra.run.smithi114.stdout:/tmp/tmp.gvdahuDAxB 2024-09-21T03:20:22.336 DEBUG:teuthology.orchestra.run.smithi114:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.gvdahuDAxB 2024-09-21T03:20:22.362 DEBUG:teuthology.orchestra.remote:smithi089:/tmp/tmp.FgkTrwL2fa is 6KB 2024-09-21T03:20:22.403 DEBUG:teuthology.orchestra.run.smithi114:> sudo chmod 0666 /tmp/tmp.gvdahuDAxB 2024-09-21T03:20:22.411 DEBUG:teuthology.orchestra.run.smithi089:> rm -fr /tmp/tmp.FgkTrwL2fa 2024-09-21T03:20:22.426 DEBUG:teuthology.orchestra.run.smithi089:> sudo /bin/ls /boot/loader/entries || true 2024-09-21T03:20:22.488 INFO:teuthology.orchestra.run.smithi089.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-0-rescue.conf 2024-09-21T03:20:22.488 INFO:teuthology.orchestra.run.smithi089.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-239.el9.x86_64.conf 2024-09-21T03:20:22.488 INFO:teuthology.orchestra.run.smithi089.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-503.el9.x86_64.conf 2024-09-21T03:20:22.489 INFO:teuthology.orchestra.run.smithi089.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-508.el9.x86_64.conf 2024-09-21T03:20:22.490 DEBUG:teuthology.orchestra.run.smithi089:> sudo grub2-set-default fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-508.el9.x86_64 2024-09-21T03:20:22.543 DEBUG:teuthology.orchestra.remote:smithi114:/tmp/tmp.gvdahuDAxB is 6KB 2024-09-21T03:20:22.563 DEBUG:teuthology.orchestra.run.smithi114:> rm -fr /tmp/tmp.gvdahuDAxB 2024-09-21T03:20:22.577 DEBUG:teuthology.orchestra.run.smithi114:> sudo /bin/ls /boot/loader/entries || true 2024-09-21T03:20:22.643 INFO:teuthology.orchestra.run.smithi114.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-0-rescue.conf 2024-09-21T03:20:22.643 INFO:teuthology.orchestra.run.smithi114.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-239.el9.x86_64.conf 2024-09-21T03:20:22.643 INFO:teuthology.orchestra.run.smithi114.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-503.el9.x86_64.conf 2024-09-21T03:20:22.643 INFO:teuthology.orchestra.run.smithi114.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-508.el9.x86_64.conf 2024-09-21T03:20:22.644 DEBUG:teuthology.orchestra.run.smithi089:> sudo shutdown -r now 2024-09-21T03:20:22.645 DEBUG:teuthology.orchestra.run.smithi114:> sudo grub2-set-default fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-508.el9.x86_64 2024-09-21T03:20:22.771 DEBUG:teuthology.orchestra.run.smithi114:> sudo shutdown -r now 2024-09-21T03:20:30.022 INFO:teuthology.task.kernel:Checking client mon.b for new kernel version... 2024-09-21T03:20:30.023 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi110.front.sepia.ceph.com' 2024-09-21T03:20:30.023 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi110.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T03:20:48.371 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.110 2024-09-21T03:20:52.648 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2024-09-21T03:20:52.649 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi089.front.sepia.ceph.com' 2024-09-21T03:20:52.649 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi089.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T03:20:52.774 INFO:teuthology.task.kernel:Checking client mon.c for new kernel version... 2024-09-21T03:20:52.774 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi114.front.sepia.ceph.com' 2024-09-21T03:20:52.775 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi114.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T03:20:57.376 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi110.front.sepia.ceph.com' 2024-09-21T03:20:57.376 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi110.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T03:21:11.151 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.89 2024-09-21T03:21:11.151 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.114 2024-09-21T03:21:20.158 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi089.front.sepia.ceph.com' 2024-09-21T03:21:20.159 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi089.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T03:21:20.160 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi114.front.sepia.ceph.com' 2024-09-21T03:21:20.161 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi114.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T03:21:23.219 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.89 2024-09-21T03:21:23.219 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.114 2024-09-21T03:21:35.230 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi089.front.sepia.ceph.com' 2024-09-21T03:21:35.231 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi089.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T03:21:35.233 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi114.front.sepia.ceph.com' 2024-09-21T03:21:35.233 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi114.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T03:21:53.647 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.89 2024-09-21T03:21:53.647 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.114 2024-09-21T03:21:57.378 DEBUG:teuthology.orchestra.remote:timed out 2024-09-21T03:22:08.658 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi089.front.sepia.ceph.com' 2024-09-21T03:22:08.659 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi089.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T03:22:08.661 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi114.front.sepia.ceph.com' 2024-09-21T03:22:08.661 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi114.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T03:22:09.075 DEBUG:teuthology.orchestra.run.smithi114:> true 2024-09-21T03:22:09.132 DEBUG:teuthology.orchestra.run.smithi089:> true 2024-09-21T03:22:09.360 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi114.front.sepia.ceph.com' 2024-09-21T03:22:09.360 INFO:teuthology.task.kernel:Checking kernel version of mon.c, want "5.14.0-508.el9.x86_64"... 2024-09-21T03:22:09.361 DEBUG:teuthology.orchestra.run.smithi114:> uname -r 2024-09-21T03:22:09.375 INFO:teuthology.orchestra.run.smithi114.stdout:5.14.0-508.el9.x86_64 2024-09-21T03:22:09.375 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-508.el9.x86_64 vs 5.14.0-508.el9.x86_64 2024-09-21T03:22:09.375 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-21T03:22:09.375 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-09-21T03:22:09.379 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi110.front.sepia.ceph.com' 2024-09-21T03:22:09.380 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi110.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T03:22:09.560 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi089.front.sepia.ceph.com' 2024-09-21T03:22:09.560 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "5.14.0-508.el9.x86_64"... 2024-09-21T03:22:09.560 DEBUG:teuthology.orchestra.run.smithi089:> uname -r 2024-09-21T03:22:09.574 INFO:teuthology.orchestra.run.smithi089.stdout:5.14.0-508.el9.x86_64 2024-09-21T03:22:09.575 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-508.el9.x86_64 vs 5.14.0-508.el9.x86_64 2024-09-21T03:22:09.575 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-21T03:22:09.575 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-09-21T03:22:09.643 DEBUG:teuthology.orchestra.run.smithi110:> true 2024-09-21T03:22:10.025 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi110.front.sepia.ceph.com' 2024-09-21T03:22:10.025 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "5.14.0-508.el9.x86_64"... 2024-09-21T03:22:10.025 DEBUG:teuthology.orchestra.run.smithi110:> uname -r 2024-09-21T03:22:10.042 INFO:teuthology.orchestra.run.smithi110.stdout:5.14.0-508.el9.x86_64 2024-09-21T03:22:10.042 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-508.el9.x86_64 vs 5.14.0-508.el9.x86_64 2024-09-21T03:22:10.042 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-21T03:22:10.042 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-09-21T03:22:10.376 INFO:teuthology.task.kernel:Enabling kdb on mon.c... 2024-09-21T03:22:10.376 DEBUG:teuthology.orchestra.run.smithi114:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-09-21T03:22:10.471 INFO:teuthology.orchestra.run.smithi114.stdout:ttyS1 2024-09-21T03:22:10.498 DEBUG:teuthology.parallel:result is None 2024-09-21T03:22:10.575 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2024-09-21T03:22:10.576 DEBUG:teuthology.orchestra.run.smithi089:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-09-21T03:22:10.693 INFO:teuthology.orchestra.run.smithi089.stdout:ttyS1 2024-09-21T03:22:10.724 DEBUG:teuthology.parallel:result is None 2024-09-21T03:22:11.043 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2024-09-21T03:22:11.043 DEBUG:teuthology.orchestra.run.smithi110:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-09-21T03:22:11.391 INFO:teuthology.orchestra.run.smithi110.stdout:ttyS1 2024-09-21T03:22:11.411 DEBUG:teuthology.parallel:result is None 2024-09-21T03:22:11.412 INFO:teuthology.run_tasks:Running task internal.base... 2024-09-21T03:22:11.420 INFO:teuthology.task.internal:Creating test directory... 2024-09-21T03:22:11.420 DEBUG:teuthology.orchestra.run.smithi089:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-21T03:22:11.422 DEBUG:teuthology.orchestra.run.smithi110:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-21T03:22:11.425 DEBUG:teuthology.orchestra.run.smithi114:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-21T03:22:11.464 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-09-21T03:22:11.475 INFO:teuthology.run_tasks:Running task internal.archive... 2024-09-21T03:22:11.484 INFO:teuthology.task.internal:Creating archive directory... 2024-09-21T03:22:11.484 DEBUG:teuthology.orchestra.run.smithi089:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-21T03:22:11.486 DEBUG:teuthology.orchestra.run.smithi110:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-21T03:22:11.488 DEBUG:teuthology.orchestra.run.smithi114:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-21T03:22:11.564 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-09-21T03:22:11.571 INFO:teuthology.task.internal:Enabling coredump saving... 2024-09-21T03:22:11.572 DEBUG:teuthology.orchestra.run.smithi089:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-21T03:22:11.574 DEBUG:teuthology.orchestra.run.smithi110:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-21T03:22:11.576 DEBUG:teuthology.orchestra.run.smithi114:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-21T03:22:11.620 INFO:teuthology.orchestra.run.smithi110.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-21T03:22:11.632 INFO:teuthology.orchestra.run.smithi110.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-21T03:22:11.645 INFO:teuthology.orchestra.run.smithi114.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-21T03:22:11.652 INFO:teuthology.orchestra.run.smithi089.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-21T03:22:11.664 INFO:teuthology.orchestra.run.smithi089.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-21T03:22:11.665 INFO:teuthology.orchestra.run.smithi114.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-21T03:22:11.667 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-09-21T03:22:11.675 INFO:teuthology.task.internal:Configuring sudo... 2024-09-21T03:22:11.675 DEBUG:teuthology.orchestra.run.smithi089:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-21T03:22:11.678 DEBUG:teuthology.orchestra.run.smithi110:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-21T03:22:11.680 DEBUG:teuthology.orchestra.run.smithi114:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-21T03:22:11.739 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-09-21T03:22:11.750 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-09-21T03:22:11.750 DEBUG:teuthology.orchestra.run.smithi089:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-21T03:22:11.752 DEBUG:teuthology.orchestra.run.smithi110:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-21T03:22:11.755 DEBUG:teuthology.orchestra.run.smithi114:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-21T03:22:11.803 DEBUG:teuthology.orchestra.run.smithi089:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-21T03:22:11.852 DEBUG:teuthology.orchestra.run.smithi089:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-21T03:22:11.901 DEBUG:teuthology.orchestra.run.smithi089:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-21T03:22:11.983 DEBUG:teuthology.orchestra.run.smithi089:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-21T03:22:12.024 DEBUG:teuthology.orchestra.run.smithi089:> set -ex 2024-09-21T03:22:12.024 DEBUG:teuthology.orchestra.run.smithi089:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-21T03:22:12.092 DEBUG:teuthology.orchestra.run.smithi110:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-21T03:22:12.137 DEBUG:teuthology.orchestra.run.smithi110:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-21T03:22:12.199 DEBUG:teuthology.orchestra.run.smithi110:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-21T03:22:12.243 DEBUG:teuthology.orchestra.run.smithi110:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-21T03:22:12.283 DEBUG:teuthology.orchestra.run.smithi110:> set -ex 2024-09-21T03:22:12.284 DEBUG:teuthology.orchestra.run.smithi110:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-21T03:22:12.349 DEBUG:teuthology.orchestra.run.smithi114:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-21T03:22:12.391 DEBUG:teuthology.orchestra.run.smithi114:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-21T03:22:12.580 DEBUG:teuthology.orchestra.run.smithi114:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-21T03:22:12.618 DEBUG:teuthology.orchestra.run.smithi114:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-21T03:22:12.664 DEBUG:teuthology.orchestra.run.smithi114:> set -ex 2024-09-21T03:22:12.665 DEBUG:teuthology.orchestra.run.smithi114:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-21T03:22:12.732 DEBUG:teuthology.orchestra.run.smithi089:> sudo service rsyslog restart 2024-09-21T03:22:12.734 DEBUG:teuthology.orchestra.run.smithi110:> sudo service rsyslog restart 2024-09-21T03:22:12.736 DEBUG:teuthology.orchestra.run.smithi114:> sudo service rsyslog restart 2024-09-21T03:22:12.781 INFO:teuthology.orchestra.run.smithi089.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-21T03:22:12.781 INFO:teuthology.orchestra.run.smithi110.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-21T03:22:12.828 INFO:teuthology.orchestra.run.smithi114.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-21T03:22:13.295 INFO:teuthology.run_tasks:Running task internal.timer... 2024-09-21T03:22:13.304 INFO:teuthology.task.internal:Starting timer... 2024-09-21T03:22:13.304 INFO:teuthology.run_tasks:Running task pcp... 2024-09-21T03:22:13.392 INFO:teuthology.run_tasks:Running task selinux... 2024-09-21T03:22:13.401 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:logrotate_t:s0']} 2024-09-21T03:22:13.402 DEBUG:teuthology.orchestra.run.smithi089:> sudo service auditd rotate 2024-09-21T03:22:13.447 INFO:teuthology.orchestra.run.smithi089.stdout:Rotating logs: 2024-09-21T03:22:13.449 DEBUG:teuthology.orchestra.run.smithi110:> sudo service auditd rotate 2024-09-21T03:22:13.508 INFO:teuthology.orchestra.run.smithi110.stdout:Rotating logs: 2024-09-21T03:22:13.510 DEBUG:teuthology.orchestra.run.smithi114:> sudo service auditd rotate 2024-09-21T03:22:13.572 INFO:teuthology.orchestra.run.smithi114.stdout:Rotating logs: 2024-09-21T03:22:13.574 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-09-21T03:22:13.574 DEBUG:teuthology.orchestra.run.smithi089:> /usr/sbin/getenforce 2024-09-21T03:22:13.603 INFO:teuthology.orchestra.run.smithi089.stdout:Permissive 2024-09-21T03:22:13.603 DEBUG:teuthology.orchestra.run.smithi110:> /usr/sbin/getenforce 2024-09-21T03:22:13.633 INFO:teuthology.orchestra.run.smithi110.stdout:Permissive 2024-09-21T03:22:13.634 DEBUG:teuthology.orchestra.run.smithi114:> /usr/sbin/getenforce 2024-09-21T03:22:13.655 INFO:teuthology.orchestra.run.smithi114.stdout:Permissive 2024-09-21T03:22:13.655 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi089.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi110.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi114.front.sepia.ceph.com': 'permissive'} 2024-09-21T03:22:13.656 DEBUG:teuthology.orchestra.run.smithi089:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 2024-09-21T03:22:13.684 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T03:22:13.684 DEBUG:teuthology.orchestra.run.smithi110:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 2024-09-21T03:22:13.714 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T03:22:13.714 DEBUG:teuthology.orchestra.run.smithi114:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 2024-09-21T03:22:13.738 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T03:22:13.739 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-09-21T03:22:13.739 DEBUG:teuthology.orchestra.run.smithi089:> sudo /usr/sbin/setenforce permissive 2024-09-21T03:22:13.775 DEBUG:teuthology.orchestra.run.smithi110:> sudo /usr/sbin/setenforce permissive 2024-09-21T03:22:13.814 DEBUG:teuthology.orchestra.run.smithi114:> sudo /usr/sbin/setenforce permissive 2024-09-21T03:22:13.847 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-09-21T03:22:13.856 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2024-09-21T03:22:14.093 INFO:teuthology.repo_utils:Fetching git.ceph.com_ceph-cm-ansible_main from origin 2024-09-21T03:22:14.203 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-09-21T03:22:14.213 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-09-21T03:22:14.214 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi089.front.sepia.ceph.com,smithi110.front.sepia.ceph.com,smithi114.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-09-21T03:26:05.468 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi089.front.sepia.ceph.com'), Remote(name='ubuntu@smithi110.front.sepia.ceph.com'), Remote(name='ubuntu@smithi114.front.sepia.ceph.com')] 2024-09-21T03:26:05.469 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi089.front.sepia.ceph.com' 2024-09-21T03:26:05.470 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi089.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T03:26:05.546 DEBUG:teuthology.orchestra.run.smithi089:> true 2024-09-21T03:26:05.626 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi089.front.sepia.ceph.com' 2024-09-21T03:26:05.627 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi110.front.sepia.ceph.com' 2024-09-21T03:26:05.627 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi110.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T03:26:05.702 DEBUG:teuthology.orchestra.run.smithi110:> true 2024-09-21T03:26:05.781 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi110.front.sepia.ceph.com' 2024-09-21T03:26:05.781 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi114.front.sepia.ceph.com' 2024-09-21T03:26:05.782 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi114.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T03:26:05.863 DEBUG:teuthology.orchestra.run.smithi114:> true 2024-09-21T03:26:05.950 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi114.front.sepia.ceph.com' 2024-09-21T03:26:05.951 INFO:teuthology.run_tasks:Running task clock... 2024-09-21T03:26:05.961 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-09-21T03:26:05.962 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-21T03:26:05.962 DEBUG:teuthology.orchestra.run.smithi089:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-21T03:26:05.965 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-21T03:26:05.965 DEBUG:teuthology.orchestra.run.smithi110:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-21T03:26:05.968 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-21T03:26:05.968 DEBUG:teuthology.orchestra.run.smithi114:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-21T03:26:05.994 INFO:teuthology.orchestra.run.smithi110.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-21T03:26:05.995 INFO:teuthology.orchestra.run.smithi089.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-21T03:26:06.008 INFO:teuthology.orchestra.run.smithi110.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-21T03:26:06.010 INFO:teuthology.orchestra.run.smithi089.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-21T03:26:06.021 INFO:teuthology.orchestra.run.smithi114.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-21T03:26:06.033 INFO:teuthology.orchestra.run.smithi110.stderr:sudo: ntpd: command not found 2024-09-21T03:26:06.036 INFO:teuthology.orchestra.run.smithi114.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-21T03:26:06.036 INFO:teuthology.orchestra.run.smithi089.stderr:sudo: ntpd: command not found 2024-09-21T03:26:06.044 INFO:teuthology.orchestra.run.smithi110.stdout:506 Cannot talk to daemon 2024-09-21T03:26:06.048 INFO:teuthology.orchestra.run.smithi089.stdout:506 Cannot talk to daemon 2024-09-21T03:26:06.058 INFO:teuthology.orchestra.run.smithi110.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-21T03:26:06.061 INFO:teuthology.orchestra.run.smithi114.stderr:sudo: ntpd: command not found 2024-09-21T03:26:06.063 INFO:teuthology.orchestra.run.smithi089.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-21T03:26:06.071 INFO:teuthology.orchestra.run.smithi110.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-21T03:26:06.072 INFO:teuthology.orchestra.run.smithi114.stdout:506 Cannot talk to daemon 2024-09-21T03:26:06.077 INFO:teuthology.orchestra.run.smithi089.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-21T03:26:06.087 INFO:teuthology.orchestra.run.smithi114.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-21T03:26:06.101 INFO:teuthology.orchestra.run.smithi114.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-21T03:26:06.127 INFO:teuthology.orchestra.run.smithi110.stderr:bash: line 1: ntpq: command not found 2024-09-21T03:26:06.130 INFO:teuthology.orchestra.run.smithi110.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-21T03:26:06.131 INFO:teuthology.orchestra.run.smithi110.stdout:=============================================================================== 2024-09-21T03:26:06.131 INFO:teuthology.orchestra.run.smithi110.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-21T03:26:06.131 INFO:teuthology.orchestra.run.smithi110.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-21T03:26:06.131 INFO:teuthology.orchestra.run.smithi110.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-21T03:26:06.131 INFO:teuthology.orchestra.run.smithi110.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-21T03:26:06.133 INFO:teuthology.orchestra.run.smithi089.stderr:bash: line 1: ntpq: command not found 2024-09-21T03:26:06.136 INFO:teuthology.orchestra.run.smithi089.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-21T03:26:06.136 INFO:teuthology.orchestra.run.smithi089.stdout:=============================================================================== 2024-09-21T03:26:06.136 INFO:teuthology.orchestra.run.smithi089.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-21T03:26:06.136 INFO:teuthology.orchestra.run.smithi089.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-21T03:26:06.136 INFO:teuthology.orchestra.run.smithi089.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-21T03:26:06.136 INFO:teuthology.orchestra.run.smithi089.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-21T03:26:06.165 INFO:teuthology.orchestra.run.smithi114.stderr:bash: line 1: ntpq: command not found 2024-09-21T03:26:06.168 INFO:teuthology.orchestra.run.smithi114.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-21T03:26:06.168 INFO:teuthology.orchestra.run.smithi114.stdout:=============================================================================== 2024-09-21T03:26:06.168 INFO:teuthology.orchestra.run.smithi114.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-21T03:26:06.168 INFO:teuthology.orchestra.run.smithi114.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-21T03:26:06.168 INFO:teuthology.orchestra.run.smithi114.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-21T03:26:06.168 INFO:teuthology.orchestra.run.smithi114.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-21T03:26:06.169 INFO:teuthology.run_tasks:Running task install... 2024-09-21T03:26:06.177 DEBUG:teuthology.task.install:project ceph 2024-09-21T03:26:06.177 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '1843221895adccfb3b692060251dd6a0183bc410'}} 2024-09-21T03:26:06.177 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '1843221895adccfb3b692060251dd6a0183bc410'} 2024-09-21T03:26:06.177 INFO:teuthology.task.install:Using flavor: default 2024-09-21T03:26:06.182 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-09-21T03:26:06.182 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-09-21T03:26:06.183 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '1843221895adccfb3b692060251dd6a0183bc410', 'tag': None, 'wait_for_package': False} 2024-09-21T03:26:06.183 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=1843221895adccfb3b692060251dd6a0183bc410 2024-09-21T03:26:06.185 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '1843221895adccfb3b692060251dd6a0183bc410', 'tag': None, 'wait_for_package': False} 2024-09-21T03:26:06.185 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=1843221895adccfb3b692060251dd6a0183bc410 2024-09-21T03:26:06.186 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '1843221895adccfb3b692060251dd6a0183bc410', 'tag': None, 'wait_for_package': False} 2024-09-21T03:26:06.186 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=1843221895adccfb3b692060251dd6a0183bc410 2024-09-21T03:26:06.328 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/flavors/default/ 2024-09-21T03:26:06.328 INFO:teuthology.task.install.rpm:Package version is 19.3.0-5120.g18432218 2024-09-21T03:26:06.341 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/flavors/default/ 2024-09-21T03:26:06.341 INFO:teuthology.task.install.rpm:Package version is 19.3.0-5120.g18432218 2024-09-21T03:26:06.352 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/flavors/default/ 2024-09-21T03:26:06.352 INFO:teuthology.task.install.rpm:Package version is 19.3.0-5120.g18432218 2024-09-21T03:26:06.444 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-21T03:26:06.444 DEBUG:teuthology.orchestra.run.smithi089:> set -ex 2024-09-21T03:26:06.444 DEBUG:teuthology.orchestra.run.smithi089:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-21T03:26:06.456 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-21T03:26:06.456 DEBUG:teuthology.orchestra.run.smithi114:> set -ex 2024-09-21T03:26:06.456 DEBUG:teuthology.orchestra.run.smithi114:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-21T03:26:06.461 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-21T03:26:06.461 DEBUG:teuthology.orchestra.run.smithi110:> set -ex 2024-09-21T03:26:06.461 DEBUG:teuthology.orchestra.run.smithi110:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-21T03:26:06.470 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-09-21T03:26:06.470 DEBUG:teuthology.orchestra.run.smithi089:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/1843221895adccfb3b692060251dd6a0183bc410/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-21T03:26:06.482 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-09-21T03:26:06.483 DEBUG:teuthology.orchestra.run.smithi114:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/1843221895adccfb3b692060251dd6a0183bc410/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-21T03:26:06.486 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-09-21T03:26:06.486 DEBUG:teuthology.orchestra.run.smithi110:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/1843221895adccfb3b692060251dd6a0183bc410/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-21T03:26:06.535 DEBUG:teuthology.orchestra.run.smithi089:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-21T03:26:06.547 DEBUG:teuthology.orchestra.run.smithi114:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-21T03:26:06.552 DEBUG:teuthology.orchestra.run.smithi110:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-21T03:26:06.610 DEBUG:teuthology.orchestra.run.smithi089:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-21T03:26:06.622 DEBUG:teuthology.orchestra.run.smithi114:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-21T03:26:06.627 DEBUG:teuthology.orchestra.run.smithi110:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-21T03:26:06.674 INFO:teuthology.orchestra.run.smithi089.stdout:check_obsoletes = 1 2024-09-21T03:26:06.676 DEBUG:teuthology.orchestra.run.smithi089:> sudo yum clean all 2024-09-21T03:26:06.686 INFO:teuthology.orchestra.run.smithi114.stdout:check_obsoletes = 1 2024-09-21T03:26:06.687 DEBUG:teuthology.orchestra.run.smithi114:> sudo yum clean all 2024-09-21T03:26:06.691 INFO:teuthology.orchestra.run.smithi110.stdout:check_obsoletes = 1 2024-09-21T03:26:06.693 DEBUG:teuthology.orchestra.run.smithi110:> sudo yum clean all 2024-09-21T03:26:06.918 INFO:teuthology.orchestra.run.smithi110.stdout:45 files removed 2024-09-21T03:26:06.919 INFO:teuthology.orchestra.run.smithi089.stdout:45 files removed 2024-09-21T03:26:06.921 INFO:teuthology.orchestra.run.smithi114.stdout:45 files removed 2024-09-21T03:26:06.944 DEBUG:teuthology.orchestra.run.smithi110:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-09-21T03:26:06.946 DEBUG:teuthology.orchestra.run.smithi089:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-09-21T03:26:06.951 DEBUG:teuthology.orchestra.run.smithi114:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-09-21T03:26:07.500 INFO:teuthology.orchestra.run.smithi089.stdout:ceph packages for x86_64 312 kB/s | 84 kB 00:00 2024-09-21T03:26:07.503 INFO:teuthology.orchestra.run.smithi114.stdout:ceph packages for x86_64 316 kB/s | 84 kB 00:00 2024-09-21T03:26:07.512 INFO:teuthology.orchestra.run.smithi110.stdout:ceph packages for x86_64 296 kB/s | 84 kB 00:00 2024-09-21T03:26:07.779 INFO:teuthology.orchestra.run.smithi114.stdout:ceph noarch packages 104 kB/s | 24 kB 00:00 2024-09-21T03:26:07.786 INFO:teuthology.orchestra.run.smithi089.stdout:ceph noarch packages 100 kB/s | 24 kB 00:00 2024-09-21T03:26:07.813 INFO:teuthology.orchestra.run.smithi110.stdout:ceph noarch packages 94 kB/s | 24 kB 00:00 2024-09-21T03:26:08.037 INFO:teuthology.orchestra.run.smithi114.stdout:ceph source packages 8.6 kB/s | 1.9 kB 00:00 2024-09-21T03:26:08.054 INFO:teuthology.orchestra.run.smithi089.stdout:ceph source packages 8.3 kB/s | 1.9 kB 00:00 2024-09-21T03:26:08.088 INFO:teuthology.orchestra.run.smithi110.stdout:ceph source packages 8.0 kB/s | 1.9 kB 00:00 2024-09-21T03:26:08.571 INFO:teuthology.orchestra.run.smithi089.stdout:CentOS Stream 9 - BaseOS 17 MB/s | 8.3 MB 00:00 2024-09-21T03:26:09.213 INFO:teuthology.orchestra.run.smithi110.stdout:CentOS Stream 9 - BaseOS 7.6 MB/s | 8.3 MB 00:01 2024-09-21T03:26:09.954 INFO:teuthology.orchestra.run.smithi114.stdout:CentOS Stream 9 - BaseOS 4.4 MB/s | 8.3 MB 00:01 2024-09-21T03:26:10.672 INFO:teuthology.orchestra.run.smithi089.stdout:CentOS Stream 9 - AppStream 20 MB/s | 20 MB 00:01 2024-09-21T03:26:10.906 INFO:teuthology.orchestra.run.smithi110.stdout:CentOS Stream 9 - AppStream 32 MB/s | 20 MB 00:00 2024-09-21T03:26:15.165 INFO:teuthology.orchestra.run.smithi089.stdout:CentOS Stream 9 - CRB 9.8 MB/s | 6.5 MB 00:00 2024-09-21T03:26:15.524 INFO:teuthology.orchestra.run.smithi110.stdout:CentOS Stream 9 - CRB 7.5 MB/s | 6.5 MB 00:00 2024-09-21T03:26:16.047 INFO:teuthology.orchestra.run.smithi114.stdout:CentOS Stream 9 - AppStream 4.1 MB/s | 20 MB 00:05 2024-09-21T03:26:16.749 INFO:teuthology.orchestra.run.smithi089.stdout:CentOS Stream 9 - Extras packages 91 kB/s | 19 kB 00:00 2024-09-21T03:26:17.008 INFO:teuthology.orchestra.run.smithi110.stdout:CentOS Stream 9 - Extras packages 73 kB/s | 19 kB 00:00 2024-09-21T03:26:17.737 INFO:teuthology.orchestra.run.smithi089.stdout:Extra Packages for Enterprise Linux 26 MB/s | 23 MB 00:00 2024-09-21T03:26:17.846 INFO:teuthology.orchestra.run.smithi110.stdout:Extra Packages for Enterprise Linux 32 MB/s | 23 MB 00:00 2024-09-21T03:26:21.872 INFO:teuthology.orchestra.run.smithi114.stdout:CentOS Stream 9 - CRB 3.3 MB/s | 6.5 MB 00:02 2024-09-21T03:26:23.357 INFO:teuthology.orchestra.run.smithi114.stdout:CentOS Stream 9 - Extras packages 80 kB/s | 19 kB 00:00 2024-09-21T03:26:23.609 INFO:teuthology.orchestra.run.smithi089.stdout:lab-extras 3.1 kB/s | 1.7 kB 00:00 2024-09-21T03:26:23.685 INFO:teuthology.orchestra.run.smithi110.stdout:lab-extras 3.3 kB/s | 1.7 kB 00:00 2024-09-21T03:26:24.161 INFO:teuthology.orchestra.run.smithi114.stdout:Extra Packages for Enterprise Linux 33 MB/s | 23 MB 00:00 2024-09-21T03:26:25.828 INFO:teuthology.orchestra.run.smithi089.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-21T03:26:25.829 INFO:teuthology.orchestra.run.smithi089.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-21T03:26:25.840 INFO:teuthology.orchestra.run.smithi089.stdout:Package NetworkManager-1:1.48.8-1.el9.x86_64 is already installed. 2024-09-21T03:26:25.841 INFO:teuthology.orchestra.run.smithi089.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-09-21T03:26:25.842 INFO:teuthology.orchestra.run.smithi089.stdout:Package util-linux-2.37.4-20.el9.x86_64 is already installed. 2024-09-21T03:26:25.843 INFO:teuthology.orchestra.run.smithi089.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-09-21T03:26:25.845 INFO:teuthology.orchestra.run.smithi089.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-09-21T03:26:25.847 INFO:teuthology.orchestra.run.smithi089.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-09-21T03:26:25.848 INFO:teuthology.orchestra.run.smithi089.stdout:Package xfsprogs-6.4.0-4.el9.x86_64 is already installed. 2024-09-21T03:26:25.905 INFO:teuthology.orchestra.run.smithi110.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-21T03:26:25.906 INFO:teuthology.orchestra.run.smithi110.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-21T03:26:25.910 INFO:teuthology.orchestra.run.smithi089.stdout:Dependencies resolved. 2024-09-21T03:26:25.919 INFO:teuthology.orchestra.run.smithi110.stdout:Package NetworkManager-1:1.48.8-1.el9.x86_64 is already installed. 2024-09-21T03:26:25.920 INFO:teuthology.orchestra.run.smithi110.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-09-21T03:26:25.921 INFO:teuthology.orchestra.run.smithi110.stdout:Package util-linux-2.37.4-20.el9.x86_64 is already installed. 2024-09-21T03:26:25.922 INFO:teuthology.orchestra.run.smithi110.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-09-21T03:26:25.924 INFO:teuthology.orchestra.run.smithi110.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-09-21T03:26:25.925 INFO:teuthology.orchestra.run.smithi089.stdout:======================================================================================= 2024-09-21T03:26:25.925 INFO:teuthology.orchestra.run.smithi089.stdout: Package Arch Version Repository Size 2024-09-21T03:26:25.925 INFO:teuthology.orchestra.run.smithi089.stdout:======================================================================================= 2024-09-21T03:26:25.925 INFO:teuthology.orchestra.run.smithi089.stdout:Installing: 2024-09-21T03:26:25.925 INFO:teuthology.orchestra.run.smithi089.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-09-21T03:26:25.925 INFO:teuthology.orchestra.run.smithi089.stdout: ceph x86_64 2:19.3.0-5120.g18432218.el9 ceph 6.6 k 2024-09-21T03:26:25.925 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-base x86_64 2:19.3.0-5120.g18432218.el9 ceph 5.5 M 2024-09-21T03:26:25.925 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-fuse x86_64 2:19.3.0-5120.g18432218.el9 ceph 873 k 2024-09-21T03:26:25.925 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-immutable-object-cache x86_64 2:19.3.0-5120.g18432218.el9 ceph 148 k 2024-09-21T03:26:25.926 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-mgr x86_64 2:19.3.0-5120.g18432218.el9 ceph 1.5 M 2024-09-21T03:26:25.926 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-mgr-cephadm noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 154 k 2024-09-21T03:26:25.926 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-mgr-dashboard noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 14 M 2024-09-21T03:26:25.926 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-mgr-diskprediction-local noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 7.4 M 2024-09-21T03:26:25.926 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-mgr-rook noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 49 k 2024-09-21T03:26:25.926 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-radosgw x86_64 2:19.3.0-5120.g18432218.el9 ceph 10 M 2024-09-21T03:26:25.926 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-test x86_64 2:19.3.0-5120.g18432218.el9 ceph 40 M 2024-09-21T03:26:25.926 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-volume noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 285 k 2024-09-21T03:26:25.926 INFO:teuthology.orchestra.run.smithi089.stdout: cephadm noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 784 k 2024-09-21T03:26:25.926 INFO:teuthology.orchestra.run.smithi089.stdout: cephfs-mirror x86_64 2:19.3.0-5120.g18432218.el9 ceph 226 k 2024-09-21T03:26:25.926 INFO:teuthology.orchestra.run.smithi089.stdout: cephfs-shell x86_64 2:19.3.0-5120.g18432218.el9 ceph 27 k 2024-09-21T03:26:25.926 INFO:teuthology.orchestra.run.smithi089.stdout: cephfs-top noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 18 k 2024-09-21T03:26:25.926 INFO:teuthology.orchestra.run.smithi089.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-09-21T03:26:25.927 INFO:teuthology.orchestra.run.smithi089.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-09-21T03:26:25.927 INFO:teuthology.orchestra.run.smithi089.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-09-21T03:26:25.927 INFO:teuthology.orchestra.run.smithi089.stdout: libcephfs-devel x86_64 2:19.3.0-5120.g18432218.el9 ceph 32 k 2024-09-21T03:26:25.927 INFO:teuthology.orchestra.run.smithi089.stdout: libcephfs2 x86_64 2:19.3.0-5120.g18432218.el9 ceph 766 k 2024-09-21T03:26:25.927 INFO:teuthology.orchestra.run.smithi089.stdout: librados-devel x86_64 2:19.3.0-5120.g18432218.el9 ceph 127 k 2024-09-21T03:26:25.927 INFO:teuthology.orchestra.run.smithi089.stdout: libtool x86_64 2.4.6-46.el9 appstream 581 k 2024-09-21T03:26:25.927 INFO:teuthology.orchestra.run.smithi089.stdout: libuuid-devel x86_64 2.37.4-20.el9 appstream 24 k 2024-09-21T03:26:25.927 INFO:teuthology.orchestra.run.smithi089.stdout: openssl-devel x86_64 1:3.2.2-6.el9 appstream 4.4 M 2024-09-21T03:26:25.927 INFO:teuthology.orchestra.run.smithi089.stdout: postgresql x86_64 13.16-1.el9 appstream 1.6 M 2024-09-21T03:26:25.927 INFO:teuthology.orchestra.run.smithi089.stdout: postgresql-contrib x86_64 13.16-1.el9 appstream 910 k 2024-09-21T03:26:25.927 INFO:teuthology.orchestra.run.smithi089.stdout: postgresql-server x86_64 13.16-1.el9 appstream 5.8 M 2024-09-21T03:26:25.927 INFO:teuthology.orchestra.run.smithi089.stdout: python3-cephfs x86_64 2:19.3.0-5120.g18432218.el9 ceph 161 k 2024-09-21T03:26:25.927 INFO:teuthology.orchestra.run.smithi089.stdout: python3-rados x86_64 2:19.3.0-5120.g18432218.el9 ceph 321 k 2024-09-21T03:26:25.928 INFO:teuthology.orchestra.run.smithi089.stdout: python3-rbd x86_64 2:19.3.0-5120.g18432218.el9 ceph 307 k 2024-09-21T03:26:25.928 INFO:teuthology.orchestra.run.smithi089.stdout: python3-rgw x86_64 2:19.3.0-5120.g18432218.el9 ceph 99 k 2024-09-21T03:26:25.928 INFO:teuthology.orchestra.run.smithi089.stdout: rbd-fuse x86_64 2:19.3.0-5120.g18432218.el9 ceph 86 k 2024-09-21T03:26:25.928 INFO:teuthology.orchestra.run.smithi089.stdout: rbd-mirror x86_64 2:19.3.0-5120.g18432218.el9 ceph 3.1 M 2024-09-21T03:26:25.928 INFO:teuthology.orchestra.run.smithi089.stdout: rbd-nbd x86_64 2:19.3.0-5120.g18432218.el9 ceph 170 k 2024-09-21T03:26:25.928 INFO:teuthology.orchestra.run.smithi089.stdout: xfsprogs-devel x86_64 6.4.0-4.el9 appstream 96 k 2024-09-21T03:26:25.928 INFO:teuthology.orchestra.run.smithi089.stdout:Upgrading: 2024-09-21T03:26:25.928 INFO:teuthology.orchestra.run.smithi089.stdout: NetworkManager x86_64 1:1.51.0-1.el9 baseos 2.3 M 2024-09-21T03:26:25.928 INFO:teuthology.orchestra.run.smithi089.stdout: NetworkManager-libnm x86_64 1:1.51.0-1.el9 baseos 1.8 M 2024-09-21T03:26:25.928 INFO:teuthology.orchestra.run.smithi089.stdout: NetworkManager-team x86_64 1:1.51.0-1.el9 baseos 41 k 2024-09-21T03:26:25.928 INFO:teuthology.orchestra.run.smithi089.stdout: NetworkManager-tui x86_64 1:1.51.0-1.el9 baseos 249 k 2024-09-21T03:26:25.928 INFO:teuthology.orchestra.run.smithi089.stdout: librados2 x86_64 2:19.3.0-5120.g18432218.el9 ceph 3.4 M 2024-09-21T03:26:25.928 INFO:teuthology.orchestra.run.smithi089.stdout: librbd1 x86_64 2:19.3.0-5120.g18432218.el9 ceph 3.2 M 2024-09-21T03:26:25.929 INFO:teuthology.orchestra.run.smithi089.stdout: openssl x86_64 1:3.2.2-6.el9 baseos 1.4 M 2024-09-21T03:26:25.929 INFO:teuthology.orchestra.run.smithi089.stdout: openssl-libs x86_64 1:3.2.2-6.el9 baseos 2.4 M 2024-09-21T03:26:25.929 INFO:teuthology.orchestra.run.smithi089.stdout:Installing dependencies: 2024-09-21T03:26:25.929 INFO:teuthology.orchestra.run.smithi089.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-09-21T03:26:25.929 INFO:teuthology.orchestra.run.smithi089.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-09-21T03:26:25.929 INFO:teuthology.orchestra.run.smithi089.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-21T03:26:25.929 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-common x86_64 2:19.3.0-5120.g18432218.el9 ceph 20 M 2024-09-21T03:26:25.929 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-grafana-dashboards noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 34 k 2024-09-21T03:26:25.929 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-mds x86_64 2:19.3.0-5120.g18432218.el9 ceph 2.3 M 2024-09-21T03:26:25.929 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-mgr-modules-core noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 292 k 2024-09-21T03:26:25.929 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-mon x86_64 2:19.3.0-5120.g18432218.el9 ceph 5.0 M 2024-09-21T03:26:25.929 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-osd x86_64 2:19.3.0-5120.g18432218.el9 ceph 18 M 2024-09-21T03:26:25.929 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-prometheus-alerts noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 16 k 2024-09-21T03:26:25.929 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-selinux x86_64 2:19.3.0-5120.g18432218.el9 ceph 25 k 2024-09-21T03:26:25.930 INFO:teuthology.orchestra.run.smithi089.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-21T03:26:25.930 INFO:teuthology.orchestra.run.smithi089.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-21T03:26:25.930 INFO:teuthology.orchestra.run.smithi089.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-21T03:26:25.930 INFO:teuthology.orchestra.run.smithi089.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-21T03:26:25.930 INFO:teuthology.orchestra.run.smithi089.stdout: gperftools-libs x86_64 2.9.1-3.el9 epel 308 k 2024-09-21T03:26:25.930 INFO:teuthology.orchestra.run.smithi089.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-09-21T03:26:25.930 INFO:teuthology.orchestra.run.smithi089.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-21T03:26:25.930 INFO:teuthology.orchestra.run.smithi089.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-21T03:26:25.930 INFO:teuthology.orchestra.run.smithi089.stdout: libcephsqlite x86_64 2:19.3.0-5120.g18432218.el9 ceph 158 k 2024-09-21T03:26:25.930 INFO:teuthology.orchestra.run.smithi089.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-21T03:26:25.930 INFO:teuthology.orchestra.run.smithi089.stdout: libnbd x86_64 1.20.2-2.el9 appstream 171 k 2024-09-21T03:26:25.930 INFO:teuthology.orchestra.run.smithi089.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-21T03:26:25.930 INFO:teuthology.orchestra.run.smithi089.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-21T03:26:25.930 INFO:teuthology.orchestra.run.smithi089.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-21T03:26:25.930 INFO:teuthology.orchestra.run.smithi089.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-21T03:26:25.931 INFO:teuthology.orchestra.run.smithi089.stdout: libradosstriper1 x86_64 2:19.3.0-5120.g18432218.el9 ceph 507 k 2024-09-21T03:26:25.931 INFO:teuthology.orchestra.run.smithi089.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-21T03:26:25.931 INFO:teuthology.orchestra.run.smithi089.stdout: librgw2 x86_64 2:19.3.0-5120.g18432218.el9 ceph 5.3 M 2024-09-21T03:26:25.931 INFO:teuthology.orchestra.run.smithi089.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-21T03:26:25.931 INFO:teuthology.orchestra.run.smithi089.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-09-21T03:26:25.931 INFO:teuthology.orchestra.run.smithi089.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-21T03:26:25.931 INFO:teuthology.orchestra.run.smithi089.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-09-21T03:26:25.931 INFO:teuthology.orchestra.run.smithi089.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-09-21T03:26:25.931 INFO:teuthology.orchestra.run.smithi089.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-09-21T03:26:25.931 INFO:teuthology.orchestra.run.smithi089.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-21T03:26:25.931 INFO:teuthology.orchestra.run.smithi089.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-21T03:26:25.931 INFO:teuthology.orchestra.run.smithi089.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-21T03:26:25.931 INFO:teuthology.orchestra.run.smithi089.stdout: postgresql-private-libs x86_64 13.16-1.el9 appstream 137 k 2024-09-21T03:26:25.931 INFO:teuthology.orchestra.run.smithi089.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-09-21T03:26:25.931 INFO:teuthology.orchestra.run.smithi089.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-09-21T03:26:25.932 INFO:teuthology.orchestra.run.smithi089.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-21T03:26:25.932 INFO:teuthology.orchestra.run.smithi089.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-09-21T03:26:25.932 INFO:teuthology.orchestra.run.smithi089.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-21T03:26:25.932 INFO:teuthology.orchestra.run.smithi089.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-21T03:26:25.932 INFO:teuthology.orchestra.run.smithi089.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-21T03:26:25.932 INFO:teuthology.orchestra.run.smithi089.stdout: python3-ceph-argparse x86_64 2:19.3.0-5120.g18432218.el9 ceph 45 k 2024-09-21T03:26:25.932 INFO:teuthology.orchestra.run.smithi089.stdout: python3-ceph-common x86_64 2:19.3.0-5120.g18432218.el9 ceph 148 k 2024-09-21T03:26:25.932 INFO:teuthology.orchestra.run.smithi089.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-21T03:26:25.932 INFO:teuthology.orchestra.run.smithi089.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-21T03:26:25.932 INFO:teuthology.orchestra.run.smithi089.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-21T03:26:25.932 INFO:teuthology.orchestra.run.smithi089.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-21T03:26:25.932 INFO:teuthology.orchestra.run.smithi089.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-09-21T03:26:25.932 INFO:teuthology.orchestra.run.smithi089.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-09-21T03:26:25.932 INFO:teuthology.orchestra.run.smithi089.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-21T03:26:25.932 INFO:teuthology.orchestra.run.smithi089.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-21T03:26:25.933 INFO:teuthology.orchestra.run.smithi089.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-21T03:26:25.933 INFO:teuthology.orchestra.run.smithi089.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-09-21T03:26:25.933 INFO:teuthology.orchestra.run.smithi089.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-09-21T03:26:25.933 INFO:teuthology.orchestra.run.smithi089.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-21T03:26:25.933 INFO:teuthology.orchestra.run.smithi089.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-21T03:26:25.933 INFO:teuthology.orchestra.run.smithi089.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-09-21T03:26:25.933 INFO:teuthology.orchestra.run.smithi089.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-21T03:26:25.933 INFO:teuthology.orchestra.run.smithi089.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-21T03:26:25.933 INFO:teuthology.orchestra.run.smithi089.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-21T03:26:25.933 INFO:teuthology.orchestra.run.smithi089.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-21T03:26:25.933 INFO:teuthology.orchestra.run.smithi089.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-21T03:26:25.933 INFO:teuthology.orchestra.run.smithi089.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-21T03:26:25.933 INFO:teuthology.orchestra.run.smithi089.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-21T03:26:25.933 INFO:teuthology.orchestra.run.smithi089.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-21T03:26:25.934 INFO:teuthology.orchestra.run.smithi089.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-21T03:26:25.934 INFO:teuthology.orchestra.run.smithi089.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-21T03:26:25.934 INFO:teuthology.orchestra.run.smithi089.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-21T03:26:25.934 INFO:teuthology.orchestra.run.smithi089.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-21T03:26:25.934 INFO:teuthology.orchestra.run.smithi089.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-21T03:26:25.934 INFO:teuthology.orchestra.run.smithi089.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-21T03:26:25.934 INFO:teuthology.orchestra.run.smithi089.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-21T03:26:25.934 INFO:teuthology.orchestra.run.smithi089.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-21T03:26:25.934 INFO:teuthology.orchestra.run.smithi089.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-21T03:26:25.934 INFO:teuthology.orchestra.run.smithi089.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-21T03:26:25.934 INFO:teuthology.orchestra.run.smithi089.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-21T03:26:25.934 INFO:teuthology.orchestra.run.smithi089.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-09-21T03:26:25.934 INFO:teuthology.orchestra.run.smithi089.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-21T03:26:25.934 INFO:teuthology.orchestra.run.smithi089.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-21T03:26:25.934 INFO:teuthology.orchestra.run.smithi089.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-21T03:26:25.934 INFO:teuthology.orchestra.run.smithi089.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-21T03:26:25.935 INFO:teuthology.orchestra.run.smithi089.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-09-21T03:26:25.935 INFO:teuthology.orchestra.run.smithi089.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-21T03:26:25.935 INFO:teuthology.orchestra.run.smithi089.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-21T03:26:25.935 INFO:teuthology.orchestra.run.smithi089.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-21T03:26:25.935 INFO:teuthology.orchestra.run.smithi089.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-21T03:26:25.935 INFO:teuthology.orchestra.run.smithi089.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-21T03:26:25.935 INFO:teuthology.orchestra.run.smithi089.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-21T03:26:25.935 INFO:teuthology.orchestra.run.smithi089.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-21T03:26:25.935 INFO:teuthology.orchestra.run.smithi089.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-21T03:26:25.935 INFO:teuthology.orchestra.run.smithi089.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-21T03:26:25.935 INFO:teuthology.orchestra.run.smithi089.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-21T03:26:25.935 INFO:teuthology.orchestra.run.smithi089.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-21T03:26:25.935 INFO:teuthology.orchestra.run.smithi089.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-09-21T03:26:25.935 INFO:teuthology.orchestra.run.smithi089.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-21T03:26:25.936 INFO:teuthology.orchestra.run.smithi089.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-21T03:26:25.936 INFO:teuthology.orchestra.run.smithi089.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-21T03:26:25.936 INFO:teuthology.orchestra.run.smithi089.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-21T03:26:25.936 INFO:teuthology.orchestra.run.smithi089.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-09-21T03:26:25.936 INFO:teuthology.orchestra.run.smithi089.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-09-21T03:26:25.936 INFO:teuthology.orchestra.run.smithi089.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-21T03:26:25.936 INFO:teuthology.orchestra.run.smithi089.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-21T03:26:25.936 INFO:teuthology.orchestra.run.smithi089.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-21T03:26:25.936 INFO:teuthology.orchestra.run.smithi089.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-09-21T03:26:25.936 INFO:teuthology.orchestra.run.smithi089.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-21T03:26:25.936 INFO:teuthology.orchestra.run.smithi089.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-09-21T03:26:25.936 INFO:teuthology.orchestra.run.smithi089.stdout:Installing weak dependencies: 2024-09-21T03:26:25.936 INFO:teuthology.orchestra.run.smithi089.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-09-21T03:26:25.936 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-09-21T03:26:25.936 INFO:teuthology.orchestra.run.smithi089.stdout:Transaction Summary 2024-09-21T03:26:25.937 INFO:teuthology.orchestra.run.smithi089.stdout:======================================================================================= 2024-09-21T03:26:25.937 INFO:teuthology.orchestra.run.smithi089.stdout:Install 149 Packages 2024-09-21T03:26:25.937 INFO:teuthology.orchestra.run.smithi089.stdout:Upgrade 8 Packages 2024-09-21T03:26:25.937 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-09-21T03:26:25.937 INFO:teuthology.orchestra.run.smithi110.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-09-21T03:26:25.937 INFO:teuthology.orchestra.run.smithi110.stdout:Package xfsprogs-6.4.0-4.el9.x86_64 is already installed. 2024-09-21T03:26:25.938 INFO:teuthology.orchestra.run.smithi089.stdout:Total download size: 228 M 2024-09-21T03:26:25.938 INFO:teuthology.orchestra.run.smithi089.stdout:Downloading Packages: 2024-09-21T03:26:25.991 INFO:teuthology.orchestra.run.smithi110.stdout:Dependencies resolved. 2024-09-21T03:26:26.010 INFO:teuthology.orchestra.run.smithi110.stdout:======================================================================================= 2024-09-21T03:26:26.010 INFO:teuthology.orchestra.run.smithi110.stdout: Package Arch Version Repository Size 2024-09-21T03:26:26.010 INFO:teuthology.orchestra.run.smithi110.stdout:======================================================================================= 2024-09-21T03:26:26.010 INFO:teuthology.orchestra.run.smithi110.stdout:Installing: 2024-09-21T03:26:26.010 INFO:teuthology.orchestra.run.smithi110.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-09-21T03:26:26.010 INFO:teuthology.orchestra.run.smithi110.stdout: ceph x86_64 2:19.3.0-5120.g18432218.el9 ceph 6.6 k 2024-09-21T03:26:26.010 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-base x86_64 2:19.3.0-5120.g18432218.el9 ceph 5.5 M 2024-09-21T03:26:26.010 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-fuse x86_64 2:19.3.0-5120.g18432218.el9 ceph 873 k 2024-09-21T03:26:26.011 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-immutable-object-cache x86_64 2:19.3.0-5120.g18432218.el9 ceph 148 k 2024-09-21T03:26:26.011 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-mgr x86_64 2:19.3.0-5120.g18432218.el9 ceph 1.5 M 2024-09-21T03:26:26.011 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-mgr-cephadm noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 154 k 2024-09-21T03:26:26.011 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-mgr-dashboard noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 14 M 2024-09-21T03:26:26.011 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-mgr-diskprediction-local noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 7.4 M 2024-09-21T03:26:26.011 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-mgr-rook noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 49 k 2024-09-21T03:26:26.011 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-radosgw x86_64 2:19.3.0-5120.g18432218.el9 ceph 10 M 2024-09-21T03:26:26.011 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-test x86_64 2:19.3.0-5120.g18432218.el9 ceph 40 M 2024-09-21T03:26:26.011 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-volume noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 285 k 2024-09-21T03:26:26.011 INFO:teuthology.orchestra.run.smithi110.stdout: cephadm noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 784 k 2024-09-21T03:26:26.011 INFO:teuthology.orchestra.run.smithi110.stdout: cephfs-mirror x86_64 2:19.3.0-5120.g18432218.el9 ceph 226 k 2024-09-21T03:26:26.011 INFO:teuthology.orchestra.run.smithi110.stdout: cephfs-shell x86_64 2:19.3.0-5120.g18432218.el9 ceph 27 k 2024-09-21T03:26:26.011 INFO:teuthology.orchestra.run.smithi110.stdout: cephfs-top noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 18 k 2024-09-21T03:26:26.011 INFO:teuthology.orchestra.run.smithi110.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-09-21T03:26:26.012 INFO:teuthology.orchestra.run.smithi110.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-09-21T03:26:26.012 INFO:teuthology.orchestra.run.smithi110.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-09-21T03:26:26.012 INFO:teuthology.orchestra.run.smithi110.stdout: libcephfs-devel x86_64 2:19.3.0-5120.g18432218.el9 ceph 32 k 2024-09-21T03:26:26.012 INFO:teuthology.orchestra.run.smithi110.stdout: libcephfs2 x86_64 2:19.3.0-5120.g18432218.el9 ceph 766 k 2024-09-21T03:26:26.012 INFO:teuthology.orchestra.run.smithi110.stdout: librados-devel x86_64 2:19.3.0-5120.g18432218.el9 ceph 127 k 2024-09-21T03:26:26.012 INFO:teuthology.orchestra.run.smithi110.stdout: libtool x86_64 2.4.6-46.el9 appstream 581 k 2024-09-21T03:26:26.012 INFO:teuthology.orchestra.run.smithi110.stdout: libuuid-devel x86_64 2.37.4-20.el9 appstream 24 k 2024-09-21T03:26:26.012 INFO:teuthology.orchestra.run.smithi110.stdout: openssl-devel x86_64 1:3.2.2-6.el9 appstream 4.4 M 2024-09-21T03:26:26.012 INFO:teuthology.orchestra.run.smithi110.stdout: postgresql x86_64 13.16-1.el9 appstream 1.6 M 2024-09-21T03:26:26.012 INFO:teuthology.orchestra.run.smithi110.stdout: postgresql-contrib x86_64 13.16-1.el9 appstream 910 k 2024-09-21T03:26:26.012 INFO:teuthology.orchestra.run.smithi110.stdout: postgresql-server x86_64 13.16-1.el9 appstream 5.8 M 2024-09-21T03:26:26.012 INFO:teuthology.orchestra.run.smithi110.stdout: python3-cephfs x86_64 2:19.3.0-5120.g18432218.el9 ceph 161 k 2024-09-21T03:26:26.012 INFO:teuthology.orchestra.run.smithi110.stdout: python3-rados x86_64 2:19.3.0-5120.g18432218.el9 ceph 321 k 2024-09-21T03:26:26.012 INFO:teuthology.orchestra.run.smithi110.stdout: python3-rbd x86_64 2:19.3.0-5120.g18432218.el9 ceph 307 k 2024-09-21T03:26:26.013 INFO:teuthology.orchestra.run.smithi110.stdout: python3-rgw x86_64 2:19.3.0-5120.g18432218.el9 ceph 99 k 2024-09-21T03:26:26.013 INFO:teuthology.orchestra.run.smithi110.stdout: rbd-fuse x86_64 2:19.3.0-5120.g18432218.el9 ceph 86 k 2024-09-21T03:26:26.013 INFO:teuthology.orchestra.run.smithi110.stdout: rbd-mirror x86_64 2:19.3.0-5120.g18432218.el9 ceph 3.1 M 2024-09-21T03:26:26.013 INFO:teuthology.orchestra.run.smithi110.stdout: rbd-nbd x86_64 2:19.3.0-5120.g18432218.el9 ceph 170 k 2024-09-21T03:26:26.013 INFO:teuthology.orchestra.run.smithi110.stdout: xfsprogs-devel x86_64 6.4.0-4.el9 appstream 96 k 2024-09-21T03:26:26.013 INFO:teuthology.orchestra.run.smithi110.stdout:Upgrading: 2024-09-21T03:26:26.013 INFO:teuthology.orchestra.run.smithi110.stdout: NetworkManager x86_64 1:1.51.0-1.el9 baseos 2.3 M 2024-09-21T03:26:26.013 INFO:teuthology.orchestra.run.smithi110.stdout: NetworkManager-libnm x86_64 1:1.51.0-1.el9 baseos 1.8 M 2024-09-21T03:26:26.013 INFO:teuthology.orchestra.run.smithi110.stdout: NetworkManager-team x86_64 1:1.51.0-1.el9 baseos 41 k 2024-09-21T03:26:26.013 INFO:teuthology.orchestra.run.smithi110.stdout: NetworkManager-tui x86_64 1:1.51.0-1.el9 baseos 249 k 2024-09-21T03:26:26.013 INFO:teuthology.orchestra.run.smithi110.stdout: librados2 x86_64 2:19.3.0-5120.g18432218.el9 ceph 3.4 M 2024-09-21T03:26:26.013 INFO:teuthology.orchestra.run.smithi110.stdout: librbd1 x86_64 2:19.3.0-5120.g18432218.el9 ceph 3.2 M 2024-09-21T03:26:26.013 INFO:teuthology.orchestra.run.smithi110.stdout: openssl x86_64 1:3.2.2-6.el9 baseos 1.4 M 2024-09-21T03:26:26.013 INFO:teuthology.orchestra.run.smithi110.stdout: openssl-libs x86_64 1:3.2.2-6.el9 baseos 2.4 M 2024-09-21T03:26:26.014 INFO:teuthology.orchestra.run.smithi110.stdout:Installing dependencies: 2024-09-21T03:26:26.014 INFO:teuthology.orchestra.run.smithi110.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-09-21T03:26:26.014 INFO:teuthology.orchestra.run.smithi110.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-09-21T03:26:26.014 INFO:teuthology.orchestra.run.smithi110.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-21T03:26:26.014 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-common x86_64 2:19.3.0-5120.g18432218.el9 ceph 20 M 2024-09-21T03:26:26.014 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-grafana-dashboards noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 34 k 2024-09-21T03:26:26.014 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-mds x86_64 2:19.3.0-5120.g18432218.el9 ceph 2.3 M 2024-09-21T03:26:26.014 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-mgr-modules-core noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 292 k 2024-09-21T03:26:26.014 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-mon x86_64 2:19.3.0-5120.g18432218.el9 ceph 5.0 M 2024-09-21T03:26:26.014 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-osd x86_64 2:19.3.0-5120.g18432218.el9 ceph 18 M 2024-09-21T03:26:26.014 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-prometheus-alerts noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 16 k 2024-09-21T03:26:26.014 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-selinux x86_64 2:19.3.0-5120.g18432218.el9 ceph 25 k 2024-09-21T03:26:26.014 INFO:teuthology.orchestra.run.smithi110.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-21T03:26:26.014 INFO:teuthology.orchestra.run.smithi110.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-21T03:26:26.015 INFO:teuthology.orchestra.run.smithi110.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-21T03:26:26.015 INFO:teuthology.orchestra.run.smithi110.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-21T03:26:26.015 INFO:teuthology.orchestra.run.smithi110.stdout: gperftools-libs x86_64 2.9.1-3.el9 epel 308 k 2024-09-21T03:26:26.015 INFO:teuthology.orchestra.run.smithi110.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-09-21T03:26:26.015 INFO:teuthology.orchestra.run.smithi110.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-21T03:26:26.015 INFO:teuthology.orchestra.run.smithi110.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-21T03:26:26.015 INFO:teuthology.orchestra.run.smithi110.stdout: libcephsqlite x86_64 2:19.3.0-5120.g18432218.el9 ceph 158 k 2024-09-21T03:26:26.015 INFO:teuthology.orchestra.run.smithi110.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-21T03:26:26.015 INFO:teuthology.orchestra.run.smithi110.stdout: libnbd x86_64 1.20.2-2.el9 appstream 171 k 2024-09-21T03:26:26.015 INFO:teuthology.orchestra.run.smithi110.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-21T03:26:26.015 INFO:teuthology.orchestra.run.smithi110.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-21T03:26:26.015 INFO:teuthology.orchestra.run.smithi110.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-21T03:26:26.015 INFO:teuthology.orchestra.run.smithi110.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-21T03:26:26.015 INFO:teuthology.orchestra.run.smithi110.stdout: libradosstriper1 x86_64 2:19.3.0-5120.g18432218.el9 ceph 507 k 2024-09-21T03:26:26.015 INFO:teuthology.orchestra.run.smithi110.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-21T03:26:26.016 INFO:teuthology.orchestra.run.smithi110.stdout: librgw2 x86_64 2:19.3.0-5120.g18432218.el9 ceph 5.3 M 2024-09-21T03:26:26.016 INFO:teuthology.orchestra.run.smithi110.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-21T03:26:26.016 INFO:teuthology.orchestra.run.smithi110.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-09-21T03:26:26.016 INFO:teuthology.orchestra.run.smithi110.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-21T03:26:26.016 INFO:teuthology.orchestra.run.smithi110.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-09-21T03:26:26.016 INFO:teuthology.orchestra.run.smithi110.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-09-21T03:26:26.016 INFO:teuthology.orchestra.run.smithi110.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-09-21T03:26:26.016 INFO:teuthology.orchestra.run.smithi110.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-21T03:26:26.016 INFO:teuthology.orchestra.run.smithi110.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-21T03:26:26.016 INFO:teuthology.orchestra.run.smithi110.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-21T03:26:26.016 INFO:teuthology.orchestra.run.smithi110.stdout: postgresql-private-libs x86_64 13.16-1.el9 appstream 137 k 2024-09-21T03:26:26.016 INFO:teuthology.orchestra.run.smithi110.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-09-21T03:26:26.016 INFO:teuthology.orchestra.run.smithi110.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-09-21T03:26:26.016 INFO:teuthology.orchestra.run.smithi110.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-21T03:26:26.016 INFO:teuthology.orchestra.run.smithi110.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-09-21T03:26:26.017 INFO:teuthology.orchestra.run.smithi110.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-21T03:26:26.017 INFO:teuthology.orchestra.run.smithi110.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-21T03:26:26.017 INFO:teuthology.orchestra.run.smithi110.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-21T03:26:26.017 INFO:teuthology.orchestra.run.smithi110.stdout: python3-ceph-argparse x86_64 2:19.3.0-5120.g18432218.el9 ceph 45 k 2024-09-21T03:26:26.017 INFO:teuthology.orchestra.run.smithi110.stdout: python3-ceph-common x86_64 2:19.3.0-5120.g18432218.el9 ceph 148 k 2024-09-21T03:26:26.017 INFO:teuthology.orchestra.run.smithi110.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-21T03:26:26.017 INFO:teuthology.orchestra.run.smithi110.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-21T03:26:26.017 INFO:teuthology.orchestra.run.smithi110.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-21T03:26:26.017 INFO:teuthology.orchestra.run.smithi110.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-21T03:26:26.017 INFO:teuthology.orchestra.run.smithi110.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-09-21T03:26:26.017 INFO:teuthology.orchestra.run.smithi110.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-09-21T03:26:26.017 INFO:teuthology.orchestra.run.smithi110.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-21T03:26:26.017 INFO:teuthology.orchestra.run.smithi110.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-21T03:26:26.017 INFO:teuthology.orchestra.run.smithi110.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-21T03:26:26.018 INFO:teuthology.orchestra.run.smithi110.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-09-21T03:26:26.018 INFO:teuthology.orchestra.run.smithi110.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-09-21T03:26:26.018 INFO:teuthology.orchestra.run.smithi110.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-21T03:26:26.018 INFO:teuthology.orchestra.run.smithi110.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-21T03:26:26.018 INFO:teuthology.orchestra.run.smithi110.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-09-21T03:26:26.018 INFO:teuthology.orchestra.run.smithi110.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-21T03:26:26.018 INFO:teuthology.orchestra.run.smithi110.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-21T03:26:26.018 INFO:teuthology.orchestra.run.smithi110.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-21T03:26:26.018 INFO:teuthology.orchestra.run.smithi110.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-21T03:26:26.018 INFO:teuthology.orchestra.run.smithi110.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-21T03:26:26.018 INFO:teuthology.orchestra.run.smithi110.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-21T03:26:26.018 INFO:teuthology.orchestra.run.smithi110.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-21T03:26:26.018 INFO:teuthology.orchestra.run.smithi110.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-21T03:26:26.018 INFO:teuthology.orchestra.run.smithi110.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-21T03:26:26.019 INFO:teuthology.orchestra.run.smithi110.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-21T03:26:26.019 INFO:teuthology.orchestra.run.smithi110.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-21T03:26:26.019 INFO:teuthology.orchestra.run.smithi110.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-21T03:26:26.019 INFO:teuthology.orchestra.run.smithi110.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-21T03:26:26.019 INFO:teuthology.orchestra.run.smithi110.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-21T03:26:26.019 INFO:teuthology.orchestra.run.smithi110.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-21T03:26:26.019 INFO:teuthology.orchestra.run.smithi110.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-21T03:26:26.019 INFO:teuthology.orchestra.run.smithi110.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-21T03:26:26.019 INFO:teuthology.orchestra.run.smithi110.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-21T03:26:26.019 INFO:teuthology.orchestra.run.smithi110.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-21T03:26:26.019 INFO:teuthology.orchestra.run.smithi110.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-09-21T03:26:26.019 INFO:teuthology.orchestra.run.smithi110.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-21T03:26:26.019 INFO:teuthology.orchestra.run.smithi110.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-21T03:26:26.019 INFO:teuthology.orchestra.run.smithi110.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-21T03:26:26.019 INFO:teuthology.orchestra.run.smithi110.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-21T03:26:26.020 INFO:teuthology.orchestra.run.smithi110.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-09-21T03:26:26.020 INFO:teuthology.orchestra.run.smithi110.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-21T03:26:26.020 INFO:teuthology.orchestra.run.smithi110.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-21T03:26:26.020 INFO:teuthology.orchestra.run.smithi110.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-21T03:26:26.020 INFO:teuthology.orchestra.run.smithi110.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-21T03:26:26.020 INFO:teuthology.orchestra.run.smithi110.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-21T03:26:26.020 INFO:teuthology.orchestra.run.smithi110.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-21T03:26:26.020 INFO:teuthology.orchestra.run.smithi110.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-21T03:26:26.020 INFO:teuthology.orchestra.run.smithi110.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-21T03:26:26.020 INFO:teuthology.orchestra.run.smithi110.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-21T03:26:26.020 INFO:teuthology.orchestra.run.smithi110.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-21T03:26:26.020 INFO:teuthology.orchestra.run.smithi110.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-21T03:26:26.020 INFO:teuthology.orchestra.run.smithi110.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-09-21T03:26:26.020 INFO:teuthology.orchestra.run.smithi110.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-21T03:26:26.021 INFO:teuthology.orchestra.run.smithi110.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-21T03:26:26.021 INFO:teuthology.orchestra.run.smithi110.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-21T03:26:26.021 INFO:teuthology.orchestra.run.smithi110.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-21T03:26:26.021 INFO:teuthology.orchestra.run.smithi110.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-09-21T03:26:26.021 INFO:teuthology.orchestra.run.smithi110.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-09-21T03:26:26.021 INFO:teuthology.orchestra.run.smithi110.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-21T03:26:26.021 INFO:teuthology.orchestra.run.smithi110.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-21T03:26:26.021 INFO:teuthology.orchestra.run.smithi110.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-21T03:26:26.021 INFO:teuthology.orchestra.run.smithi110.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-09-21T03:26:26.021 INFO:teuthology.orchestra.run.smithi110.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-21T03:26:26.021 INFO:teuthology.orchestra.run.smithi110.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-09-21T03:26:26.021 INFO:teuthology.orchestra.run.smithi110.stdout:Installing weak dependencies: 2024-09-21T03:26:26.021 INFO:teuthology.orchestra.run.smithi110.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-09-21T03:26:26.021 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-21T03:26:26.021 INFO:teuthology.orchestra.run.smithi110.stdout:Transaction Summary 2024-09-21T03:26:26.022 INFO:teuthology.orchestra.run.smithi110.stdout:======================================================================================= 2024-09-21T03:26:26.022 INFO:teuthology.orchestra.run.smithi110.stdout:Install 149 Packages 2024-09-21T03:26:26.022 INFO:teuthology.orchestra.run.smithi110.stdout:Upgrade 8 Packages 2024-09-21T03:26:26.022 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-21T03:26:26.022 INFO:teuthology.orchestra.run.smithi110.stdout:Total download size: 228 M 2024-09-21T03:26:26.022 INFO:teuthology.orchestra.run.smithi110.stdout:Downloading Packages: 2024-09-21T03:26:26.227 INFO:teuthology.orchestra.run.smithi089.stdout:(1/157): ceph-19.3.0-5120.g18432218.el9.x86_64. 47 kB/s | 6.6 kB 00:00 2024-09-21T03:26:26.453 INFO:teuthology.orchestra.run.smithi089.stdout:(2/157): ceph-fuse-19.3.0-5120.g18432218.el9.x8 3.8 MB/s | 873 kB 00:00 2024-09-21T03:26:26.553 INFO:teuthology.orchestra.run.smithi089.stdout:(3/157): ceph-immutable-object-cache-19.3.0-512 1.5 MB/s | 148 kB 00:00 2024-09-21T03:26:26.753 INFO:teuthology.orchestra.run.smithi089.stdout:(4/157): ceph-base-19.3.0-5120.g18432218.el9.x8 8.3 MB/s | 5.5 MB 00:00 2024-09-21T03:26:26.804 INFO:teuthology.orchestra.run.smithi110.stdout:(1/157): ceph-19.3.0-5120.g18432218.el9.x86_64. 30 kB/s | 6.6 kB 00:00 2024-09-21T03:26:27.037 INFO:teuthology.orchestra.run.smithi089.stdout:(5/157): ceph-common-19.3.0-5120.g18432218.el9. 21 MB/s | 20 MB 00:00 2024-09-21T03:26:27.045 INFO:teuthology.orchestra.run.smithi110.stdout:(2/157): ceph-fuse-19.3.0-5120.g18432218.el9.x8 3.5 MB/s | 873 kB 00:00 2024-09-21T03:26:27.079 INFO:teuthology.orchestra.run.smithi089.stdout:(6/157): ceph-mgr-19.3.0-5120.g18432218.el9.x86 4.7 MB/s | 1.5 MB 00:00 2024-09-21T03:26:27.114 INFO:teuthology.orchestra.run.smithi110.stdout:(3/157): ceph-base-19.3.0-5120.g18432218.el9.x8 11 MB/s | 5.5 MB 00:00 2024-09-21T03:26:27.129 INFO:teuthology.orchestra.run.smithi089.stdout:(7/157): ceph-mds-19.3.0-5120.g18432218.el9.x86 4.0 MB/s | 2.3 MB 00:00 2024-09-21T03:26:27.137 INFO:teuthology.orchestra.run.smithi110.stdout:(4/157): ceph-immutable-object-cache-19.3.0-512 1.6 MB/s | 148 kB 00:00 2024-09-21T03:26:27.213 INFO:teuthology.orchestra.run.smithi110.stdout:(5/157): ceph-mds-19.3.0-5120.g18432218.el9.x86 23 MB/s | 2.3 MB 00:00 2024-09-21T03:26:27.363 INFO:teuthology.orchestra.run.smithi089.stdout:(8/157): ceph-mon-19.3.0-5120.g18432218.el9.x86 15 MB/s | 5.0 MB 00:00 2024-09-21T03:26:27.413 INFO:teuthology.orchestra.run.smithi089.stdout:(9/157): ceph-selinux-19.3.0-5120.g18432218.el9 505 kB/s | 25 kB 00:00 2024-09-21T03:26:27.438 INFO:teuthology.orchestra.run.smithi110.stdout:(6/157): ceph-mon-19.3.0-5120.g18432218.el9.x86 22 MB/s | 5.0 MB 00:00 2024-09-21T03:26:27.530 INFO:teuthology.orchestra.run.smithi110.stdout:(7/157): ceph-mgr-19.3.0-5120.g18432218.el9.x86 3.9 MB/s | 1.5 MB 00:00 2024-09-21T03:26:28.547 INFO:teuthology.orchestra.run.smithi110.stdout:(8/157): ceph-osd-19.3.0-5120.g18432218.el9.x86 16 MB/s | 18 MB 00:01 2024-09-21T03:26:28.605 INFO:teuthology.orchestra.run.smithi110.stdout:(9/157): ceph-selinux-19.3.0-5120.g18432218.el9 432 kB/s | 25 kB 00:00 2024-09-21T03:26:28.855 INFO:teuthology.orchestra.run.smithi089.stdout:(10/157): ceph-osd-19.3.0-5120.g18432218.el9.x8 10 MB/s | 18 MB 00:01 2024-09-21T03:26:28.921 INFO:teuthology.orchestra.run.smithi089.stdout:(11/157): cephfs-mirror-19.3.0-5120.g18432218.e 3.3 MB/s | 226 kB 00:00 2024-09-21T03:26:28.988 INFO:teuthology.orchestra.run.smithi089.stdout:(12/157): cephfs-shell-19.3.0-5120.g18432218.el 406 kB/s | 27 kB 00:00 2024-09-21T03:26:29.031 INFO:teuthology.orchestra.run.smithi110.stdout:(10/157): ceph-radosgw-19.3.0-5120.g18432218.el 6.9 MB/s | 10 MB 00:01 2024-09-21T03:26:29.047 INFO:teuthology.orchestra.run.smithi089.stdout:(13/157): libcephfs-devel-19.3.0-5120.g18432218 543 kB/s | 32 kB 00:00 2024-09-21T03:26:29.089 INFO:teuthology.orchestra.run.smithi110.stdout:(11/157): cephfs-mirror-19.3.0-5120.g18432218.e 3.8 MB/s | 226 kB 00:00 2024-09-21T03:26:29.173 INFO:teuthology.orchestra.run.smithi110.stdout:(12/157): cephfs-shell-19.3.0-5120.g18432218.el 325 kB/s | 27 kB 00:00 2024-09-21T03:26:29.197 INFO:teuthology.orchestra.run.smithi089.stdout:(14/157): libcephfs2-19.3.0-5120.g18432218.el9. 5.0 MB/s | 766 kB 00:00 2024-09-21T03:26:29.231 INFO:teuthology.orchestra.run.smithi110.stdout:(13/157): libcephfs-devel-19.3.0-5120.g18432218 544 kB/s | 32 kB 00:00 2024-09-21T03:26:29.273 INFO:teuthology.orchestra.run.smithi089.stdout:(15/157): libcephsqlite-19.3.0-5120.g18432218.e 2.1 MB/s | 158 kB 00:00 2024-09-21T03:26:29.357 INFO:teuthology.orchestra.run.smithi089.stdout:(16/157): librados-devel-19.3.0-5120.g18432218. 1.5 MB/s | 127 kB 00:00 2024-09-21T03:26:29.373 INFO:teuthology.orchestra.run.smithi110.stdout:(14/157): libcephfs2-19.3.0-5120.g18432218.el9. 5.3 MB/s | 766 kB 00:00 2024-09-21T03:26:29.449 INFO:teuthology.orchestra.run.smithi110.stdout:(15/157): libcephsqlite-19.3.0-5120.g18432218.e 2.1 MB/s | 158 kB 00:00 2024-09-21T03:26:29.498 INFO:teuthology.orchestra.run.smithi089.stdout:(17/157): ceph-radosgw-19.3.0-5120.g18432218.el 4.4 MB/s | 10 MB 00:02 2024-09-21T03:26:29.507 INFO:teuthology.orchestra.run.smithi110.stdout:(16/157): librados-devel-19.3.0-5120.g18432218. 2.1 MB/s | 127 kB 00:00 2024-09-21T03:26:29.525 INFO:teuthology.orchestra.run.smithi114.stdout:lab-extras 49 kB/s | 1.7 kB 00:00 2024-09-21T03:26:29.549 INFO:teuthology.orchestra.run.smithi089.stdout:(18/157): libradosstriper1-19.3.0-5120.g1843221 2.6 MB/s | 507 kB 00:00 2024-09-21T03:26:29.624 INFO:teuthology.orchestra.run.smithi089.stdout:(19/157): python3-ceph-argparse-19.3.0-5120.g18 601 kB/s | 45 kB 00:00 2024-09-21T03:26:29.641 INFO:teuthology.orchestra.run.smithi110.stdout:(17/157): libradosstriper1-19.3.0-5120.g1843221 3.7 MB/s | 507 kB 00:00 2024-09-21T03:26:29.683 INFO:teuthology.orchestra.run.smithi089.stdout:(20/157): python3-ceph-common-19.3.0-5120.g1843 2.5 MB/s | 148 kB 00:00 2024-09-21T03:26:29.775 INFO:teuthology.orchestra.run.smithi089.stdout:(21/157): python3-cephfs-19.3.0-5120.g18432218. 1.7 MB/s | 161 kB 00:00 2024-09-21T03:26:29.925 INFO:teuthology.orchestra.run.smithi089.stdout:(22/157): python3-rados-19.3.0-5120.g18432218.e 2.1 MB/s | 321 kB 00:00 2024-09-21T03:26:29.976 INFO:teuthology.orchestra.run.smithi089.stdout:(23/157): python3-rbd-19.3.0-5120.g18432218.el9 6.0 MB/s | 307 kB 00:00 2024-09-21T03:26:30.026 INFO:teuthology.orchestra.run.smithi089.stdout:(24/157): python3-rgw-19.3.0-5120.g18432218.el9 1.9 MB/s | 99 kB 00:00 2024-09-21T03:26:30.258 INFO:teuthology.orchestra.run.smithi110.stdout:(18/157): ceph-test-19.3.0-5120.g18432218.el9.x 24 MB/s | 40 MB 00:01 2024-09-21T03:26:30.308 INFO:teuthology.orchestra.run.smithi110.stdout:(19/157): python3-ceph-argparse-19.3.0-5120.g18 907 kB/s | 45 kB 00:00 2024-09-21T03:26:30.401 INFO:teuthology.orchestra.run.smithi110.stdout:(20/157): python3-ceph-common-19.3.0-5120.g1843 1.6 MB/s | 148 kB 00:00 2024-09-21T03:26:30.418 INFO:teuthology.orchestra.run.smithi089.stdout:(25/157): ceph-test-19.3.0-5120.g18432218.el9.x 13 MB/s | 40 MB 00:03 2024-09-21T03:26:30.443 INFO:teuthology.orchestra.run.smithi089.stdout:(26/157): rbd-fuse-19.3.0-5120.g18432218.el9.x8 207 kB/s | 86 kB 00:00 2024-09-21T03:26:30.484 INFO:teuthology.orchestra.run.smithi110.stdout:(21/157): librgw2-19.3.0-5120.g18432218.el9.x86 6.3 MB/s | 5.3 MB 00:00 2024-09-21T03:26:30.509 INFO:teuthology.orchestra.run.smithi110.stdout:(22/157): python3-cephfs-19.3.0-5120.g18432218. 1.5 MB/s | 161 kB 00:00 2024-09-21T03:26:30.535 INFO:teuthology.orchestra.run.smithi110.stdout:(23/157): python3-rados-19.3.0-5120.g18432218.e 6.3 MB/s | 321 kB 00:00 2024-09-21T03:26:30.543 INFO:teuthology.orchestra.run.smithi089.stdout:(27/157): librgw2-19.3.0-5120.g18432218.el9.x86 5.1 MB/s | 5.3 MB 00:01 2024-09-21T03:26:30.560 INFO:teuthology.orchestra.run.smithi110.stdout:(24/157): python3-rbd-19.3.0-5120.g18432218.el9 6.0 MB/s | 307 kB 00:00 2024-09-21T03:26:30.569 INFO:teuthology.orchestra.run.smithi089.stdout:(28/157): rbd-nbd-19.3.0-5120.g18432218.el9.x86 1.3 MB/s | 170 kB 00:00 2024-09-21T03:26:30.585 INFO:teuthology.orchestra.run.smithi110.stdout:(25/157): python3-rgw-19.3.0-5120.g18432218.el9 1.9 MB/s | 99 kB 00:00 2024-09-21T03:26:30.594 INFO:teuthology.orchestra.run.smithi089.stdout:(29/157): ceph-grafana-dashboards-19.3.0-5120.g 684 kB/s | 34 kB 00:00 2024-09-21T03:26:30.602 INFO:teuthology.orchestra.run.smithi110.stdout:(26/157): rbd-fuse-19.3.0-5120.g18432218.el9.x8 2.0 MB/s | 86 kB 00:00 2024-09-21T03:26:30.619 INFO:teuthology.orchestra.run.smithi089.stdout:(30/157): ceph-mgr-cephadm-19.3.0-5120.g1843221 3.0 MB/s | 154 kB 00:00 2024-09-21T03:26:30.653 INFO:teuthology.orchestra.run.smithi110.stdout:(27/157): rbd-nbd-19.3.0-5120.g18432218.el9.x86 3.3 MB/s | 170 kB 00:00 2024-09-21T03:26:30.695 INFO:teuthology.orchestra.run.smithi089.stdout:(31/157): rbd-mirror-19.3.0-5120.g18432218.el9. 11 MB/s | 3.1 MB 00:00 2024-09-21T03:26:30.703 INFO:teuthology.orchestra.run.smithi110.stdout:(28/157): ceph-grafana-dashboards-19.3.0-5120.g 686 kB/s | 34 kB 00:00 2024-09-21T03:26:30.745 INFO:teuthology.orchestra.run.smithi089.stdout:(32/157): ceph-mgr-modules-core-19.3.0-5120.g18 5.7 MB/s | 292 kB 00:00 2024-09-21T03:26:30.761 INFO:teuthology.orchestra.run.smithi110.stdout:(29/157): ceph-mgr-cephadm-19.3.0-5120.g1843221 2.6 MB/s | 154 kB 00:00 2024-09-21T03:26:30.804 INFO:teuthology.orchestra.run.smithi089.stdout:(33/157): ceph-mgr-rook-19.3.0-5120.g18432218.e 848 kB/s | 49 kB 00:00 2024-09-21T03:26:30.871 INFO:teuthology.orchestra.run.smithi089.stdout:(34/157): ceph-prometheus-alerts-19.3.0-5120.g1 246 kB/s | 16 kB 00:00 2024-09-21T03:26:30.887 INFO:teuthology.orchestra.run.smithi110.stdout:(30/157): rbd-mirror-19.3.0-5120.g18432218.el9. 10 MB/s | 3.1 MB 00:00 2024-09-21T03:26:30.946 INFO:teuthology.orchestra.run.smithi089.stdout:(35/157): ceph-volume-19.3.0-5120.g18432218.el9 3.7 MB/s | 285 kB 00:00 2024-09-21T03:26:31.055 INFO:teuthology.orchestra.run.smithi089.stdout:(36/157): cephadm-19.3.0-5120.g18432218.el9.noa 7.1 MB/s | 784 kB 00:00 2024-09-21T03:26:31.147 INFO:teuthology.orchestra.run.smithi089.stdout:(37/157): ceph-mgr-diskprediction-local-19.3.0- 14 MB/s | 7.4 MB 00:00 2024-09-21T03:26:31.163 INFO:teuthology.orchestra.run.smithi089.stdout:(38/157): cephfs-top-19.3.0-5120.g18432218.el9. 169 kB/s | 18 kB 00:00 2024-09-21T03:26:31.387 INFO:teuthology.orchestra.run.smithi110.stdout:(31/157): ceph-mgr-dashboard-19.3.0-5120.g18432 23 MB/s | 14 MB 00:00 2024-09-21T03:26:31.438 INFO:teuthology.orchestra.run.smithi110.stdout:(32/157): ceph-mgr-modules-core-19.3.0-5120.g18 5.7 MB/s | 292 kB 00:00 2024-09-21T03:26:31.479 INFO:teuthology.orchestra.run.smithi110.stdout:(33/157): ceph-mgr-rook-19.3.0-5120.g18432218.e 1.2 MB/s | 49 kB 00:00 2024-09-21T03:26:31.521 INFO:teuthology.orchestra.run.smithi110.stdout:(34/157): ceph-prometheus-alerts-19.3.0-5120.g1 395 kB/s | 16 kB 00:00 2024-09-21T03:26:31.572 INFO:teuthology.orchestra.run.smithi110.stdout:(35/157): ceph-volume-19.3.0-5120.g18432218.el9 5.6 MB/s | 285 kB 00:00 2024-09-21T03:26:31.630 INFO:teuthology.orchestra.run.smithi089.stdout:(39/157): fuse-2.9.9-16.el9.x86_64.rpm 165 kB/s | 80 kB 00:00 2024-09-21T03:26:31.664 INFO:teuthology.orchestra.run.smithi110.stdout:(36/157): ceph-mgr-diskprediction-local-19.3.0- 9.5 MB/s | 7.4 MB 00:00 2024-09-21T03:26:31.695 INFO:teuthology.orchestra.run.smithi114.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-21T03:26:31.696 INFO:teuthology.orchestra.run.smithi114.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-21T03:26:31.697 INFO:teuthology.orchestra.run.smithi110.stdout:(37/157): cephadm-19.3.0-5120.g18432218.el9.noa 6.1 MB/s | 784 kB 00:00 2024-09-21T03:26:31.709 INFO:teuthology.orchestra.run.smithi114.stdout:Package NetworkManager-1:1.48.8-1.el9.x86_64 is already installed. 2024-09-21T03:26:31.710 INFO:teuthology.orchestra.run.smithi114.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-09-21T03:26:31.711 INFO:teuthology.orchestra.run.smithi114.stdout:Package util-linux-2.37.4-20.el9.x86_64 is already installed. 2024-09-21T03:26:31.712 INFO:teuthology.orchestra.run.smithi114.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-09-21T03:26:31.714 INFO:teuthology.orchestra.run.smithi114.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-09-21T03:26:31.714 INFO:teuthology.orchestra.run.smithi110.stdout:(38/157): cephfs-top-19.3.0-5120.g18432218.el9. 368 kB/s | 18 kB 00:00 2024-09-21T03:26:31.717 INFO:teuthology.orchestra.run.smithi114.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-09-21T03:26:31.718 INFO:teuthology.orchestra.run.smithi114.stdout:Package xfsprogs-6.4.0-4.el9.x86_64 is already installed. 2024-09-21T03:26:31.780 INFO:teuthology.orchestra.run.smithi114.stdout:Dependencies resolved. 2024-09-21T03:26:31.795 INFO:teuthology.orchestra.run.smithi114.stdout:======================================================================================= 2024-09-21T03:26:31.796 INFO:teuthology.orchestra.run.smithi114.stdout: Package Arch Version Repository Size 2024-09-21T03:26:31.796 INFO:teuthology.orchestra.run.smithi114.stdout:======================================================================================= 2024-09-21T03:26:31.796 INFO:teuthology.orchestra.run.smithi114.stdout:Installing: 2024-09-21T03:26:31.796 INFO:teuthology.orchestra.run.smithi114.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-09-21T03:26:31.796 INFO:teuthology.orchestra.run.smithi114.stdout: ceph x86_64 2:19.3.0-5120.g18432218.el9 ceph 6.6 k 2024-09-21T03:26:31.796 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-base x86_64 2:19.3.0-5120.g18432218.el9 ceph 5.5 M 2024-09-21T03:26:31.796 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-fuse x86_64 2:19.3.0-5120.g18432218.el9 ceph 873 k 2024-09-21T03:26:31.796 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-immutable-object-cache x86_64 2:19.3.0-5120.g18432218.el9 ceph 148 k 2024-09-21T03:26:31.796 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-mgr x86_64 2:19.3.0-5120.g18432218.el9 ceph 1.5 M 2024-09-21T03:26:31.796 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-mgr-cephadm noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 154 k 2024-09-21T03:26:31.796 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-mgr-dashboard noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 14 M 2024-09-21T03:26:31.796 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-mgr-diskprediction-local noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 7.4 M 2024-09-21T03:26:31.796 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-mgr-rook noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 49 k 2024-09-21T03:26:31.796 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-radosgw x86_64 2:19.3.0-5120.g18432218.el9 ceph 10 M 2024-09-21T03:26:31.797 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-test x86_64 2:19.3.0-5120.g18432218.el9 ceph 40 M 2024-09-21T03:26:31.797 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-volume noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 285 k 2024-09-21T03:26:31.797 INFO:teuthology.orchestra.run.smithi114.stdout: cephadm noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 784 k 2024-09-21T03:26:31.797 INFO:teuthology.orchestra.run.smithi114.stdout: cephfs-mirror x86_64 2:19.3.0-5120.g18432218.el9 ceph 226 k 2024-09-21T03:26:31.797 INFO:teuthology.orchestra.run.smithi114.stdout: cephfs-shell x86_64 2:19.3.0-5120.g18432218.el9 ceph 27 k 2024-09-21T03:26:31.797 INFO:teuthology.orchestra.run.smithi114.stdout: cephfs-top noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 18 k 2024-09-21T03:26:31.797 INFO:teuthology.orchestra.run.smithi114.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-09-21T03:26:31.797 INFO:teuthology.orchestra.run.smithi114.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-09-21T03:26:31.797 INFO:teuthology.orchestra.run.smithi114.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-09-21T03:26:31.797 INFO:teuthology.orchestra.run.smithi114.stdout: libcephfs-devel x86_64 2:19.3.0-5120.g18432218.el9 ceph 32 k 2024-09-21T03:26:31.797 INFO:teuthology.orchestra.run.smithi114.stdout: libcephfs2 x86_64 2:19.3.0-5120.g18432218.el9 ceph 766 k 2024-09-21T03:26:31.797 INFO:teuthology.orchestra.run.smithi114.stdout: librados-devel x86_64 2:19.3.0-5120.g18432218.el9 ceph 127 k 2024-09-21T03:26:31.797 INFO:teuthology.orchestra.run.smithi114.stdout: libtool x86_64 2.4.6-46.el9 appstream 581 k 2024-09-21T03:26:31.798 INFO:teuthology.orchestra.run.smithi114.stdout: libuuid-devel x86_64 2.37.4-20.el9 appstream 24 k 2024-09-21T03:26:31.798 INFO:teuthology.orchestra.run.smithi114.stdout: openssl-devel x86_64 1:3.2.2-6.el9 appstream 4.4 M 2024-09-21T03:26:31.798 INFO:teuthology.orchestra.run.smithi114.stdout: postgresql x86_64 13.16-1.el9 appstream 1.6 M 2024-09-21T03:26:31.798 INFO:teuthology.orchestra.run.smithi114.stdout: postgresql-contrib x86_64 13.16-1.el9 appstream 910 k 2024-09-21T03:26:31.798 INFO:teuthology.orchestra.run.smithi114.stdout: postgresql-server x86_64 13.16-1.el9 appstream 5.8 M 2024-09-21T03:26:31.798 INFO:teuthology.orchestra.run.smithi114.stdout: python3-cephfs x86_64 2:19.3.0-5120.g18432218.el9 ceph 161 k 2024-09-21T03:26:31.798 INFO:teuthology.orchestra.run.smithi114.stdout: python3-rados x86_64 2:19.3.0-5120.g18432218.el9 ceph 321 k 2024-09-21T03:26:31.798 INFO:teuthology.orchestra.run.smithi114.stdout: python3-rbd x86_64 2:19.3.0-5120.g18432218.el9 ceph 307 k 2024-09-21T03:26:31.798 INFO:teuthology.orchestra.run.smithi114.stdout: python3-rgw x86_64 2:19.3.0-5120.g18432218.el9 ceph 99 k 2024-09-21T03:26:31.798 INFO:teuthology.orchestra.run.smithi114.stdout: rbd-fuse x86_64 2:19.3.0-5120.g18432218.el9 ceph 86 k 2024-09-21T03:26:31.798 INFO:teuthology.orchestra.run.smithi114.stdout: rbd-mirror x86_64 2:19.3.0-5120.g18432218.el9 ceph 3.1 M 2024-09-21T03:26:31.798 INFO:teuthology.orchestra.run.smithi114.stdout: rbd-nbd x86_64 2:19.3.0-5120.g18432218.el9 ceph 170 k 2024-09-21T03:26:31.798 INFO:teuthology.orchestra.run.smithi114.stdout: xfsprogs-devel x86_64 6.4.0-4.el9 appstream 96 k 2024-09-21T03:26:31.798 INFO:teuthology.orchestra.run.smithi114.stdout:Upgrading: 2024-09-21T03:26:31.799 INFO:teuthology.orchestra.run.smithi114.stdout: NetworkManager x86_64 1:1.51.0-1.el9 baseos 2.3 M 2024-09-21T03:26:31.799 INFO:teuthology.orchestra.run.smithi114.stdout: NetworkManager-libnm x86_64 1:1.51.0-1.el9 baseos 1.8 M 2024-09-21T03:26:31.799 INFO:teuthology.orchestra.run.smithi114.stdout: NetworkManager-team x86_64 1:1.51.0-1.el9 baseos 41 k 2024-09-21T03:26:31.799 INFO:teuthology.orchestra.run.smithi114.stdout: NetworkManager-tui x86_64 1:1.51.0-1.el9 baseos 249 k 2024-09-21T03:26:31.799 INFO:teuthology.orchestra.run.smithi114.stdout: librados2 x86_64 2:19.3.0-5120.g18432218.el9 ceph 3.4 M 2024-09-21T03:26:31.799 INFO:teuthology.orchestra.run.smithi114.stdout: librbd1 x86_64 2:19.3.0-5120.g18432218.el9 ceph 3.2 M 2024-09-21T03:26:31.799 INFO:teuthology.orchestra.run.smithi114.stdout: openssl x86_64 1:3.2.2-6.el9 baseos 1.4 M 2024-09-21T03:26:31.799 INFO:teuthology.orchestra.run.smithi114.stdout: openssl-libs x86_64 1:3.2.2-6.el9 baseos 2.4 M 2024-09-21T03:26:31.799 INFO:teuthology.orchestra.run.smithi114.stdout:Installing dependencies: 2024-09-21T03:26:31.799 INFO:teuthology.orchestra.run.smithi114.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-09-21T03:26:31.799 INFO:teuthology.orchestra.run.smithi114.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-09-21T03:26:31.799 INFO:teuthology.orchestra.run.smithi114.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-21T03:26:31.799 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-common x86_64 2:19.3.0-5120.g18432218.el9 ceph 20 M 2024-09-21T03:26:31.799 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-grafana-dashboards noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 34 k 2024-09-21T03:26:31.799 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-mds x86_64 2:19.3.0-5120.g18432218.el9 ceph 2.3 M 2024-09-21T03:26:31.800 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-mgr-modules-core noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 292 k 2024-09-21T03:26:31.800 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-mon x86_64 2:19.3.0-5120.g18432218.el9 ceph 5.0 M 2024-09-21T03:26:31.800 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-osd x86_64 2:19.3.0-5120.g18432218.el9 ceph 18 M 2024-09-21T03:26:31.800 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-prometheus-alerts noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 16 k 2024-09-21T03:26:31.800 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-selinux x86_64 2:19.3.0-5120.g18432218.el9 ceph 25 k 2024-09-21T03:26:31.800 INFO:teuthology.orchestra.run.smithi114.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-21T03:26:31.800 INFO:teuthology.orchestra.run.smithi114.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-21T03:26:31.800 INFO:teuthology.orchestra.run.smithi114.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-21T03:26:31.800 INFO:teuthology.orchestra.run.smithi114.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-21T03:26:31.800 INFO:teuthology.orchestra.run.smithi114.stdout: gperftools-libs x86_64 2.9.1-3.el9 epel 308 k 2024-09-21T03:26:31.800 INFO:teuthology.orchestra.run.smithi114.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-09-21T03:26:31.800 INFO:teuthology.orchestra.run.smithi114.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-21T03:26:31.800 INFO:teuthology.orchestra.run.smithi114.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-21T03:26:31.800 INFO:teuthology.orchestra.run.smithi114.stdout: libcephsqlite x86_64 2:19.3.0-5120.g18432218.el9 ceph 158 k 2024-09-21T03:26:31.800 INFO:teuthology.orchestra.run.smithi114.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-21T03:26:31.800 INFO:teuthology.orchestra.run.smithi114.stdout: libnbd x86_64 1.20.2-2.el9 appstream 171 k 2024-09-21T03:26:31.801 INFO:teuthology.orchestra.run.smithi114.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-21T03:26:31.801 INFO:teuthology.orchestra.run.smithi114.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-21T03:26:31.801 INFO:teuthology.orchestra.run.smithi114.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-21T03:26:31.801 INFO:teuthology.orchestra.run.smithi114.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-21T03:26:31.801 INFO:teuthology.orchestra.run.smithi114.stdout: libradosstriper1 x86_64 2:19.3.0-5120.g18432218.el9 ceph 507 k 2024-09-21T03:26:31.801 INFO:teuthology.orchestra.run.smithi114.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-21T03:26:31.801 INFO:teuthology.orchestra.run.smithi114.stdout: librgw2 x86_64 2:19.3.0-5120.g18432218.el9 ceph 5.3 M 2024-09-21T03:26:31.801 INFO:teuthology.orchestra.run.smithi114.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-21T03:26:31.801 INFO:teuthology.orchestra.run.smithi114.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-09-21T03:26:31.801 INFO:teuthology.orchestra.run.smithi114.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-21T03:26:31.801 INFO:teuthology.orchestra.run.smithi114.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-09-21T03:26:31.801 INFO:teuthology.orchestra.run.smithi114.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-09-21T03:26:31.801 INFO:teuthology.orchestra.run.smithi114.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-09-21T03:26:31.802 INFO:teuthology.orchestra.run.smithi114.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-21T03:26:31.802 INFO:teuthology.orchestra.run.smithi114.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-21T03:26:31.802 INFO:teuthology.orchestra.run.smithi114.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-21T03:26:31.802 INFO:teuthology.orchestra.run.smithi114.stdout: postgresql-private-libs x86_64 13.16-1.el9 appstream 137 k 2024-09-21T03:26:31.802 INFO:teuthology.orchestra.run.smithi114.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-09-21T03:26:31.802 INFO:teuthology.orchestra.run.smithi114.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-09-21T03:26:31.802 INFO:teuthology.orchestra.run.smithi114.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-21T03:26:31.802 INFO:teuthology.orchestra.run.smithi114.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-09-21T03:26:31.802 INFO:teuthology.orchestra.run.smithi114.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-21T03:26:31.802 INFO:teuthology.orchestra.run.smithi114.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-21T03:26:31.802 INFO:teuthology.orchestra.run.smithi114.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-21T03:26:31.802 INFO:teuthology.orchestra.run.smithi114.stdout: python3-ceph-argparse x86_64 2:19.3.0-5120.g18432218.el9 ceph 45 k 2024-09-21T03:26:31.802 INFO:teuthology.orchestra.run.smithi114.stdout: python3-ceph-common x86_64 2:19.3.0-5120.g18432218.el9 ceph 148 k 2024-09-21T03:26:31.803 INFO:teuthology.orchestra.run.smithi114.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-21T03:26:31.803 INFO:teuthology.orchestra.run.smithi114.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-21T03:26:31.803 INFO:teuthology.orchestra.run.smithi114.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-21T03:26:31.803 INFO:teuthology.orchestra.run.smithi114.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-21T03:26:31.803 INFO:teuthology.orchestra.run.smithi114.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-09-21T03:26:31.803 INFO:teuthology.orchestra.run.smithi114.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-09-21T03:26:31.803 INFO:teuthology.orchestra.run.smithi114.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-21T03:26:31.803 INFO:teuthology.orchestra.run.smithi114.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-21T03:26:31.803 INFO:teuthology.orchestra.run.smithi114.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-21T03:26:31.803 INFO:teuthology.orchestra.run.smithi114.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-09-21T03:26:31.803 INFO:teuthology.orchestra.run.smithi114.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-09-21T03:26:31.803 INFO:teuthology.orchestra.run.smithi114.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-21T03:26:31.803 INFO:teuthology.orchestra.run.smithi114.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-21T03:26:31.804 INFO:teuthology.orchestra.run.smithi114.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-09-21T03:26:31.804 INFO:teuthology.orchestra.run.smithi114.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-21T03:26:31.804 INFO:teuthology.orchestra.run.smithi114.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-21T03:26:31.804 INFO:teuthology.orchestra.run.smithi114.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-21T03:26:31.804 INFO:teuthology.orchestra.run.smithi114.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-21T03:26:31.804 INFO:teuthology.orchestra.run.smithi114.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-21T03:26:31.804 INFO:teuthology.orchestra.run.smithi114.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-21T03:26:31.804 INFO:teuthology.orchestra.run.smithi114.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-21T03:26:31.804 INFO:teuthology.orchestra.run.smithi114.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-21T03:26:31.804 INFO:teuthology.orchestra.run.smithi114.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-21T03:26:31.805 INFO:teuthology.orchestra.run.smithi114.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-21T03:26:31.805 INFO:teuthology.orchestra.run.smithi114.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-21T03:26:31.805 INFO:teuthology.orchestra.run.smithi114.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-21T03:26:31.805 INFO:teuthology.orchestra.run.smithi114.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-21T03:26:31.805 INFO:teuthology.orchestra.run.smithi114.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-21T03:26:31.805 INFO:teuthology.orchestra.run.smithi114.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-21T03:26:31.805 INFO:teuthology.orchestra.run.smithi114.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-21T03:26:31.805 INFO:teuthology.orchestra.run.smithi114.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-21T03:26:31.805 INFO:teuthology.orchestra.run.smithi114.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-21T03:26:31.805 INFO:teuthology.orchestra.run.smithi114.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-21T03:26:31.805 INFO:teuthology.orchestra.run.smithi114.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-09-21T03:26:31.805 INFO:teuthology.orchestra.run.smithi114.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-21T03:26:31.805 INFO:teuthology.orchestra.run.smithi114.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-21T03:26:31.805 INFO:teuthology.orchestra.run.smithi114.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-21T03:26:31.806 INFO:teuthology.orchestra.run.smithi114.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-21T03:26:31.806 INFO:teuthology.orchestra.run.smithi114.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-09-21T03:26:31.806 INFO:teuthology.orchestra.run.smithi114.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-21T03:26:31.806 INFO:teuthology.orchestra.run.smithi114.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-21T03:26:31.806 INFO:teuthology.orchestra.run.smithi114.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-21T03:26:31.806 INFO:teuthology.orchestra.run.smithi114.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-21T03:26:31.806 INFO:teuthology.orchestra.run.smithi114.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-21T03:26:31.806 INFO:teuthology.orchestra.run.smithi114.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-21T03:26:31.806 INFO:teuthology.orchestra.run.smithi114.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-21T03:26:31.806 INFO:teuthology.orchestra.run.smithi114.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-21T03:26:31.806 INFO:teuthology.orchestra.run.smithi114.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-21T03:26:31.806 INFO:teuthology.orchestra.run.smithi114.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-21T03:26:31.806 INFO:teuthology.orchestra.run.smithi114.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-21T03:26:31.806 INFO:teuthology.orchestra.run.smithi114.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-09-21T03:26:31.807 INFO:teuthology.orchestra.run.smithi114.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-21T03:26:31.807 INFO:teuthology.orchestra.run.smithi114.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-21T03:26:31.807 INFO:teuthology.orchestra.run.smithi114.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-21T03:26:31.807 INFO:teuthology.orchestra.run.smithi114.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-21T03:26:31.807 INFO:teuthology.orchestra.run.smithi114.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-09-21T03:26:31.807 INFO:teuthology.orchestra.run.smithi114.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-09-21T03:26:31.807 INFO:teuthology.orchestra.run.smithi114.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-21T03:26:31.807 INFO:teuthology.orchestra.run.smithi114.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-21T03:26:31.807 INFO:teuthology.orchestra.run.smithi114.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-21T03:26:31.807 INFO:teuthology.orchestra.run.smithi114.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-09-21T03:26:31.807 INFO:teuthology.orchestra.run.smithi114.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-21T03:26:31.807 INFO:teuthology.orchestra.run.smithi114.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-09-21T03:26:31.807 INFO:teuthology.orchestra.run.smithi114.stdout:Installing weak dependencies: 2024-09-21T03:26:31.807 INFO:teuthology.orchestra.run.smithi114.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-09-21T03:26:31.808 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-21T03:26:31.808 INFO:teuthology.orchestra.run.smithi114.stdout:Transaction Summary 2024-09-21T03:26:31.808 INFO:teuthology.orchestra.run.smithi114.stdout:======================================================================================= 2024-09-21T03:26:31.808 INFO:teuthology.orchestra.run.smithi114.stdout:Install 149 Packages 2024-09-21T03:26:31.808 INFO:teuthology.orchestra.run.smithi114.stdout:Upgrade 8 Packages 2024-09-21T03:26:31.808 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-21T03:26:31.808 INFO:teuthology.orchestra.run.smithi114.stdout:Total download size: 228 M 2024-09-21T03:26:31.808 INFO:teuthology.orchestra.run.smithi114.stdout:Downloading Packages: 2024-09-21T03:26:31.823 INFO:teuthology.orchestra.run.smithi110.stdout:(39/157): fuse-2.9.9-16.el9.x86_64.rpm 637 kB/s | 80 kB 00:00 2024-09-21T03:26:31.890 INFO:teuthology.orchestra.run.smithi110.stdout:(40/157): libgfortran-11.5.0-2.el9.x86_64.rpm 4.5 MB/s | 801 kB 00:00 2024-09-21T03:26:31.915 INFO:teuthology.orchestra.run.smithi110.stdout:(41/157): libquadmath-11.5.0-2.el9.x86_64.rpm 2.0 MB/s | 191 kB 00:00 2024-09-21T03:26:31.922 INFO:teuthology.orchestra.run.smithi089.stdout:(40/157): libquadmath-11.5.0-2.el9.x86_64.rpm 655 kB/s | 191 kB 00:00 2024-09-21T03:26:31.941 INFO:teuthology.orchestra.run.smithi110.stdout:(42/157): python3-cffi-1.14.5-5.el9.x86_64.rpm 4.9 MB/s | 253 kB 00:00 2024-09-21T03:26:31.981 INFO:teuthology.orchestra.run.smithi089.stdout:(41/157): libgfortran-11.5.0-2.el9.x86_64.rpm 981 kB/s | 801 kB 00:00 2024-09-21T03:26:31.982 INFO:teuthology.orchestra.run.smithi110.stdout:(43/157): python3-ply-3.11-14.el9.noarch.rpm 2.5 MB/s | 106 kB 00:00 2024-09-21T03:26:32.024 INFO:teuthology.orchestra.run.smithi110.stdout:(44/157): python3-cryptography-36.0.1-4.el9.x86 11 MB/s | 1.2 MB 00:00 2024-09-21T03:26:32.041 INFO:teuthology.orchestra.run.smithi110.stdout:(45/157): python3-pycparser-2.20-6.el9.noarch.r 2.3 MB/s | 135 kB 00:00 2024-09-21T03:26:32.067 INFO:teuthology.orchestra.run.smithi110.stdout:(46/157): python3-requests-2.25.1-8.el9.noarch. 2.9 MB/s | 125 kB 00:00 2024-09-21T03:26:32.100 INFO:teuthology.orchestra.run.smithi110.stdout:(47/157): python3-urllib3-1.26.5-6.el9.noarch.r 3.6 MB/s | 215 kB 00:00 2024-09-21T03:26:32.142 INFO:teuthology.orchestra.run.smithi110.stdout:(48/157): automake-1.16.2-8.el9.noarch.rpm 8.9 MB/s | 689 kB 00:00 2024-09-21T03:26:32.176 INFO:teuthology.orchestra.run.smithi110.stdout:(49/157): bison-3.7.4-5.el9.x86_64.rpm 12 MB/s | 943 kB 00:00 2024-09-21T03:26:32.201 INFO:teuthology.orchestra.run.smithi110.stdout:(50/157): boost-program-options-1.75.0-8.el9.x8 1.8 MB/s | 107 kB 00:00 2024-09-21T03:26:32.206 INFO:teuthology.orchestra.run.smithi089.stdout:(42/157): ceph-mgr-dashboard-19.3.0-5120.g18432 8.8 MB/s | 14 MB 00:01 2024-09-21T03:26:32.227 INFO:teuthology.orchestra.run.smithi110.stdout:(51/157): elfutils-libelf-devel-0.191-4.el9.x86 431 kB/s | 22 kB 00:00 2024-09-21T03:26:32.231 INFO:teuthology.orchestra.run.smithi089.stdout:(43/157): python3-cffi-1.14.5-5.el9.x86_64.rpm 820 kB/s | 253 kB 00:00 2024-09-21T03:26:32.252 INFO:teuthology.orchestra.run.smithi110.stdout:(52/157): flex-2.6.4-9.el9.x86_64.rpm 6.1 MB/s | 313 kB 00:00 2024-09-21T03:26:32.268 INFO:teuthology.orchestra.run.smithi114.stdout:(1/157): ceph-19.3.0-5120.g18432218.el9.x86_64. 46 kB/s | 6.6 kB 00:00 2024-09-21T03:26:32.277 INFO:teuthology.orchestra.run.smithi110.stdout:(53/157): flexiblas-3.0.4-8.el9.x86_64.rpm 653 kB/s | 33 kB 00:00 2024-09-21T03:26:32.319 INFO:teuthology.orchestra.run.smithi110.stdout:(54/157): flexiblas-openblas-openmp-3.0.4-8.el9 437 kB/s | 18 kB 00:00 2024-09-21T03:26:32.324 INFO:teuthology.orchestra.run.smithi089.stdout:(44/157): python3-ply-3.11-14.el9.noarch.rpm 911 kB/s | 106 kB 00:00 2024-09-21T03:26:32.378 INFO:teuthology.orchestra.run.smithi110.stdout:(55/157): flexiblas-netlib-3.0.4-8.el9.x86_64.r 24 MB/s | 3.0 MB 00:00 2024-09-21T03:26:32.403 INFO:teuthology.orchestra.run.smithi110.stdout:(56/157): libaio-devel-0.3.111-13.el9.x86_64.rp 133 kB/s | 11 kB 00:00 2024-09-21T03:26:32.416 INFO:teuthology.orchestra.run.smithi089.stdout:(45/157): python3-cryptography-36.0.1-4.el9.x86 2.8 MB/s | 1.2 MB 00:00 2024-09-21T03:26:32.428 INFO:teuthology.orchestra.run.smithi110.stdout:(57/157): libnbd-1.20.2-2.el9.x86_64.rpm 3.3 MB/s | 171 kB 00:00 2024-09-21T03:26:32.441 INFO:teuthology.orchestra.run.smithi089.stdout:(46/157): python3-requests-2.25.1-8.el9.noarch. 1.0 MB/s | 125 kB 00:00 2024-09-21T03:26:32.445 INFO:teuthology.orchestra.run.smithi110.stdout:(58/157): libpmemobj-1.12.1-1.el9.x86_64.rpm 3.7 MB/s | 160 kB 00:00 2024-09-21T03:26:32.462 INFO:teuthology.orchestra.run.smithi110.stdout:(59/157): librabbitmq-0.11.0-7.el9.x86_64.rpm 1.3 MB/s | 45 kB 00:00 2024-09-21T03:26:32.496 INFO:teuthology.orchestra.run.smithi110.stdout:(60/157): librdkafka-1.6.1-102.el9.x86_64.rpm 13 MB/s | 662 kB 00:00 2024-09-21T03:26:32.521 INFO:teuthology.orchestra.run.smithi110.stdout:(61/157): libtool-2.4.6-46.el9.x86_64.rpm 9.7 MB/s | 581 kB 00:00 2024-09-21T03:26:32.533 INFO:teuthology.orchestra.run.smithi089.stdout:(47/157): python3-urllib3-1.26.5-6.el9.noarch.r 1.8 MB/s | 215 kB 00:00 2024-09-21T03:26:32.535 INFO:teuthology.orchestra.run.smithi114.stdout:(2/157): ceph-fuse-19.3.0-5120.g18432218.el9.x8 3.2 MB/s | 873 kB 00:00 2024-09-21T03:26:32.538 INFO:teuthology.orchestra.run.smithi110.stdout:(62/157): libuuid-devel-2.37.4-20.el9.x86_64.rp 585 kB/s | 24 kB 00:00 2024-09-21T03:26:32.555 INFO:teuthology.orchestra.run.smithi110.stdout:(63/157): libzstd-devel-1.5.1-2.el9.x86_64.rpm 1.4 MB/s | 47 kB 00:00 2024-09-21T03:26:32.580 INFO:teuthology.orchestra.run.smithi110.stdout:(64/157): lttng-ust-2.12.0-6.el9.x86_64.rpm 6.8 MB/s | 292 kB 00:00 2024-09-21T03:26:32.606 INFO:teuthology.orchestra.run.smithi110.stdout:(65/157): lua-5.4.4-4.el9.x86_64.rpm 3.7 MB/s | 188 kB 00:00 2024-09-21T03:26:32.610 INFO:teuthology.orchestra.run.smithi114.stdout:(3/157): ceph-base-19.3.0-5120.g18432218.el9.x8 11 MB/s | 5.5 MB 00:00 2024-09-21T03:26:32.623 INFO:teuthology.orchestra.run.smithi110.stdout:(66/157): openblas-0.3.26-2.el9.x86_64.rpm 921 kB/s | 38 kB 00:00 2024-09-21T03:26:32.636 INFO:teuthology.orchestra.run.smithi114.stdout:(4/157): ceph-immutable-object-cache-19.3.0-512 1.4 MB/s | 148 kB 00:00 2024-09-21T03:26:32.700 INFO:teuthology.orchestra.run.smithi089.stdout:(48/157): python3-pycparser-2.20-6.el9.noarch.r 289 kB/s | 135 kB 00:00 2024-09-21T03:26:32.715 INFO:teuthology.orchestra.run.smithi110.stdout:(67/157): openblas-openmp-0.3.26-2.el9.x86_64.r 46 MB/s | 4.9 MB 00:00 2024-09-21T03:26:32.719 INFO:teuthology.orchestra.run.smithi114.stdout:(5/157): ceph-mds-19.3.0-5120.g18432218.el9.x86 21 MB/s | 2.3 MB 00:00 2024-09-21T03:26:32.733 INFO:teuthology.orchestra.run.smithi089.stdout:(49/157): automake-1.16.2-8.el9.noarch.rpm 2.3 MB/s | 689 kB 00:00 2024-09-21T03:26:32.790 INFO:teuthology.orchestra.run.smithi110.stdout:(68/157): openssl-devel-3.2.2-6.el9.x86_64.rpm 27 MB/s | 4.4 MB 00:00 2024-09-21T03:26:32.792 INFO:teuthology.orchestra.run.smithi089.stdout:(50/157): elfutils-libelf-devel-0.191-4.el9.x86 372 kB/s | 22 kB 00:00 2024-09-21T03:26:32.841 INFO:teuthology.orchestra.run.smithi110.stdout:(69/157): postgresql-13.16-1.el9.x86_64.rpm 13 MB/s | 1.6 MB 00:00 2024-09-21T03:26:32.859 INFO:teuthology.orchestra.run.smithi089.stdout:(51/157): flex-2.6.4-9.el9.x86_64.rpm 4.6 MB/s | 313 kB 00:00 2024-09-21T03:26:32.884 INFO:teuthology.orchestra.run.smithi089.stdout:(52/157): boost-program-options-1.75.0-8.el9.x8 581 kB/s | 107 kB 00:00 2024-09-21T03:26:32.895 INFO:teuthology.orchestra.run.smithi110.stdout:(70/157): postgresql-contrib-13.16-1.el9.x86_64 8.5 MB/s | 910 kB 00:00 2024-09-21T03:26:32.910 INFO:teuthology.orchestra.run.smithi089.stdout:(53/157): bison-3.7.4-5.el9.x86_64.rpm 2.4 MB/s | 943 kB 00:00 2024-09-21T03:26:32.924 INFO:teuthology.orchestra.run.smithi110.stdout:(71/157): postgresql-private-libs-13.16-1.el9.x 1.6 MB/s | 137 kB 00:00 2024-09-21T03:26:32.935 INFO:teuthology.orchestra.run.smithi089.stdout:(54/157): flexiblas-3.0.4-8.el9.x86_64.rpm 432 kB/s | 33 kB 00:00 2024-09-21T03:26:32.960 INFO:teuthology.orchestra.run.smithi089.stdout:(55/157): flexiblas-openblas-openmp-3.0.4-8.el9 363 kB/s | 18 kB 00:00 2024-09-21T03:26:32.969 INFO:teuthology.orchestra.run.smithi114.stdout:(6/157): ceph-mgr-19.3.0-5120.g18432218.el9.x86 4.5 MB/s | 1.5 MB 00:00 2024-09-21T03:26:32.975 INFO:teuthology.orchestra.run.smithi110.stdout:(72/157): protobuf-3.14.0-13.el9.x86_64.rpm 20 MB/s | 1.0 MB 00:00 2024-09-21T03:26:32.985 INFO:teuthology.orchestra.run.smithi089.stdout:(56/157): libaio-devel-0.3.111-13.el9.x86_64.rp 222 kB/s | 11 kB 00:00 2024-09-21T03:26:33.019 INFO:teuthology.orchestra.run.smithi089.stdout:(57/157): libnbd-1.20.2-2.el9.x86_64.rpm 2.9 MB/s | 171 kB 00:00 2024-09-21T03:26:33.036 INFO:teuthology.orchestra.run.smithi114.stdout:(7/157): ceph-mon-19.3.0-5120.g18432218.el9.x86 16 MB/s | 5.0 MB 00:00 2024-09-21T03:26:33.044 INFO:teuthology.orchestra.run.smithi089.stdout:(58/157): libpmemobj-1.12.1-1.el9.x86_64.rpm 2.7 MB/s | 160 kB 00:00 2024-09-21T03:26:33.050 INFO:teuthology.orchestra.run.smithi110.stdout:(73/157): postgresql-server-13.16-1.el9.x86_64. 37 MB/s | 5.8 MB 00:00 2024-09-21T03:26:33.070 INFO:teuthology.orchestra.run.smithi089.stdout:(59/157): librabbitmq-0.11.0-7.el9.x86_64.rpm 903 kB/s | 45 kB 00:00 2024-09-21T03:26:33.075 INFO:teuthology.orchestra.run.smithi110.stdout:(74/157): python3-attrs-20.3.0-7.el9.noarch.rpm 874 kB/s | 87 kB 00:00 2024-09-21T03:26:33.117 INFO:teuthology.orchestra.run.smithi110.stdout:(75/157): python3-colorama-0.4.6-3.el9.noarch.r 1.4 MB/s | 61 kB 00:00 2024-09-21T03:26:33.162 INFO:teuthology.orchestra.run.smithi089.stdout:(60/157): libtool-2.4.6-46.el9.x86_64.rpm 6.2 MB/s | 581 kB 00:00 2024-09-21T03:26:33.195 INFO:teuthology.orchestra.run.smithi089.stdout:(61/157): librdkafka-1.6.1-102.el9.x86_64.rpm 4.3 MB/s | 662 kB 00:00 2024-09-21T03:26:33.201 INFO:teuthology.orchestra.run.smithi110.stdout:(76/157): python3-babel-2.9.1-2.el9.noarch.rpm 40 MB/s | 6.0 MB 00:00 2024-09-21T03:26:33.221 INFO:teuthology.orchestra.run.smithi089.stdout:(62/157): libuuid-devel-2.37.4-20.el9.x86_64.rp 417 kB/s | 24 kB 00:00 2024-09-21T03:26:33.226 INFO:teuthology.orchestra.run.smithi110.stdout:(77/157): python3-devel-3.9.19-8.el9.x86_64.rpm 2.2 MB/s | 245 kB 00:00 2024-09-21T03:26:33.246 INFO:teuthology.orchestra.run.smithi089.stdout:(63/157): libzstd-devel-1.5.1-2.el9.x86_64.rpm 943 kB/s | 47 kB 00:00 2024-09-21T03:26:33.251 INFO:teuthology.orchestra.run.smithi110.stdout:(78/157): python3-jinja2-2.11.3-6.el9.noarch.rp 4.8 MB/s | 248 kB 00:00 2024-09-21T03:26:33.277 INFO:teuthology.orchestra.run.smithi110.stdout:(79/157): python3-jsonpatch-1.21-16.el9.noarch. 510 kB/s | 26 kB 00:00 2024-09-21T03:26:33.287 INFO:teuthology.orchestra.run.smithi114.stdout:(8/157): ceph-common-19.3.0-5120.g18432218.el9. 17 MB/s | 20 MB 00:01 2024-09-21T03:26:33.302 INFO:teuthology.orchestra.run.smithi110.stdout:(80/157): python3-jsonpointer-2.0-4.el9.noarch. 383 kB/s | 19 kB 00:00 2024-09-21T03:26:33.327 INFO:teuthology.orchestra.run.smithi110.stdout:(81/157): python3-mako-1.1.4-6.el9.noarch.rpm 3.4 MB/s | 172 kB 00:00 2024-09-21T03:26:33.329 INFO:teuthology.orchestra.run.smithi114.stdout:(9/157): ceph-selinux-19.3.0-5120.g18432218.el9 609 kB/s | 25 kB 00:00 2024-09-21T03:26:33.353 INFO:teuthology.orchestra.run.smithi110.stdout:(82/157): python3-markupsafe-1.1.1-12.el9.x86_6 690 kB/s | 35 kB 00:00 2024-09-21T03:26:33.363 INFO:teuthology.orchestra.run.smithi089.stdout:(64/157): lua-5.4.4-4.el9.x86_64.rpm 1.6 MB/s | 188 kB 00:00 2024-09-21T03:26:33.388 INFO:teuthology.orchestra.run.smithi089.stdout:(65/157): lttng-ust-2.12.0-6.el9.x86_64.rpm 1.7 MB/s | 292 kB 00:00 2024-09-21T03:26:33.395 INFO:teuthology.orchestra.run.smithi110.stdout:(83/157): python3-numpy-f2py-1.20.1-5.el9.x86_6 5.9 MB/s | 250 kB 00:00 2024-09-21T03:26:33.414 INFO:teuthology.orchestra.run.smithi089.stdout:(66/157): openblas-0.3.26-2.el9.x86_64.rpm 765 kB/s | 38 kB 00:00 2024-09-21T03:26:33.462 INFO:teuthology.orchestra.run.smithi110.stdout:(84/157): python3-numpy-1.20.1-5.el9.x86_64.rpm 38 MB/s | 5.1 MB 00:00 2024-09-21T03:26:33.487 INFO:teuthology.orchestra.run.smithi110.stdout:(85/157): python3-oauthlib-3.1.1-5.el9.noarch.r 2.4 MB/s | 222 kB 00:00 2024-09-21T03:26:33.512 INFO:teuthology.orchestra.run.smithi110.stdout:(86/157): python3-packaging-20.9-5.el9.noarch.r 1.5 MB/s | 77 kB 00:00 2024-09-21T03:26:33.538 INFO:teuthology.orchestra.run.smithi110.stdout:(87/157): python3-prettytable-0.7.2-27.el9.noar 842 kB/s | 42 kB 00:00 2024-09-21T03:26:33.545 INFO:teuthology.orchestra.run.smithi114.stdout:(10/157): ceph-radosgw-19.3.0-5120.g18432218.el 20 MB/s | 10 MB 00:00 2024-09-21T03:26:33.563 INFO:teuthology.orchestra.run.smithi110.stdout:(88/157): python3-protobuf-3.14.0-13.el9.noarch 5.2 MB/s | 269 kB 00:00 2024-09-21T03:26:33.588 INFO:teuthology.orchestra.run.smithi110.stdout:(89/157): python3-pyasn1-0.4.8-6.el9.noarch.rpm 3.1 MB/s | 159 kB 00:00 2024-09-21T03:26:33.604 INFO:teuthology.orchestra.run.smithi114.stdout:(11/157): cephfs-mirror-19.3.0-5120.g18432218.e 3.8 MB/s | 226 kB 00:00 2024-09-21T03:26:33.613 INFO:teuthology.orchestra.run.smithi110.stdout:(90/157): python3-pyasn1-modules-0.4.8-6.el9.no 5.4 MB/s | 279 kB 00:00 2024-09-21T03:26:33.622 INFO:teuthology.orchestra.run.smithi089.stdout:(67/157): flexiblas-netlib-3.0.4-8.el9.x86_64.r 4.1 MB/s | 3.0 MB 00:00 2024-09-21T03:26:33.630 INFO:teuthology.orchestra.run.smithi110.stdout:(91/157): python3-pytz-2021.1-5.el9.noarch.rpm 1.2 MB/s | 51 kB 00:00 2024-09-21T03:26:33.647 INFO:teuthology.orchestra.run.smithi110.stdout:(92/157): python3-requests-oauthlib-1.3.0-12.el 1.6 MB/s | 54 kB 00:00 2024-09-21T03:26:33.671 INFO:teuthology.orchestra.run.smithi114.stdout:(12/157): cephfs-shell-19.3.0-5120.g18432218.el 406 kB/s | 27 kB 00:00 2024-09-21T03:26:33.681 INFO:teuthology.orchestra.run.smithi110.stdout:(93/157): python3-toml-0.10.2-6.el9.noarch.rpm 1.2 MB/s | 42 kB 00:00 2024-09-21T03:26:33.715 INFO:teuthology.orchestra.run.smithi110.stdout:(94/157): python3-wcwidth-0.2.5-8.el9.noarch.rp 1.3 MB/s | 44 kB 00:00 2024-09-21T03:26:33.738 INFO:teuthology.orchestra.run.smithi114.stdout:(13/157): libcephfs-devel-19.3.0-5120.g18432218 476 kB/s | 32 kB 00:00 2024-09-21T03:26:33.757 INFO:teuthology.orchestra.run.smithi110.stdout:(95/157): qatlib-24.02.0-1.el9.x86_64.rpm 5.2 MB/s | 221 kB 00:00 2024-09-21T03:26:33.791 INFO:teuthology.orchestra.run.smithi110.stdout:(96/157): qatlib-service-24.02.0-1.el9.x86_64.r 1.0 MB/s | 36 kB 00:00 2024-09-21T03:26:33.838 INFO:teuthology.orchestra.run.smithi114.stdout:(14/157): libcephfs2-19.3.0-5120.g18432218.el9. 7.5 MB/s | 766 kB 00:00 2024-09-21T03:26:33.913 INFO:teuthology.orchestra.run.smithi114.stdout:(15/157): libcephsqlite-19.3.0-5120.g18432218.e 2.1 MB/s | 158 kB 00:00 2024-09-21T03:26:33.949 INFO:teuthology.orchestra.run.smithi110.stdout:(97/157): python3-scipy-1.6.2-8.el9.x86_64.rpm 51 MB/s | 16 MB 00:00 2024-09-21T03:26:33.966 INFO:teuthology.orchestra.run.smithi110.stdout:(98/157): qatzip-libs-1.2.0-1.el9.x86_64.rpm 271 kB/s | 47 kB 00:00 2024-09-21T03:26:33.989 INFO:teuthology.orchestra.run.smithi114.stdout:(16/157): librados-devel-19.3.0-5120.g18432218. 1.7 MB/s | 127 kB 00:00 2024-09-21T03:26:34.000 INFO:teuthology.orchestra.run.smithi110.stdout:(99/157): socat-1.7.4.1-6.el9.x86_64.rpm 6.0 MB/s | 304 kB 00:00 2024-09-21T03:26:34.017 INFO:teuthology.orchestra.run.smithi110.stdout:(100/157): uuid-1.6.2-55.el9.x86_64.rpm 1.1 MB/s | 58 kB 00:00 2024-09-21T03:26:34.042 INFO:teuthology.orchestra.run.smithi110.stdout:(101/157): xfsprogs-devel-6.4.0-4.el9.x86_64.rp 2.2 MB/s | 96 kB 00:00 2024-09-21T03:26:34.059 INFO:teuthology.orchestra.run.smithi110.stdout:(102/157): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.5 MB/s | 64 kB 00:00 2024-09-21T03:26:34.072 INFO:teuthology.orchestra.run.smithi114.stdout:(17/157): libradosstriper1-19.3.0-5120.g1843221 5.9 MB/s | 507 kB 00:00 2024-09-21T03:26:34.076 INFO:teuthology.orchestra.run.smithi110.stdout:(103/157): zlib-devel-1.2.11-41.el9.x86_64.rpm 1.3 MB/s | 45 kB 00:00 2024-09-21T03:26:34.343 INFO:teuthology.orchestra.run.smithi110.stdout:(104/157): lua-devel-5.4.4-4.el9.x86_64.rpm 79 kB/s | 22 kB 00:00 2024-09-21T03:26:34.414 INFO:teuthology.orchestra.run.smithi089.stdout:(68/157): postgresql-13.16-1.el9.x86_64.rpm 2.0 MB/s | 1.6 MB 00:00 2024-09-21T03:26:34.419 INFO:teuthology.orchestra.run.smithi114.stdout:(18/157): ceph-osd-19.3.0-5120.g18432218.el9.x8 12 MB/s | 18 MB 00:01 2024-09-21T03:26:34.523 INFO:teuthology.orchestra.run.smithi114.stdout:(19/157): librgw2-19.3.0-5120.g18432218.el9.x86 12 MB/s | 5.3 MB 00:00 2024-09-21T03:26:34.548 INFO:teuthology.orchestra.run.smithi114.stdout:(20/157): python3-ceph-argparse-19.3.0-5120.g18 349 kB/s | 45 kB 00:00 2024-09-21T03:26:34.551 INFO:teuthology.orchestra.run.smithi110.stdout:(105/157): ceph-common-19.3.0-5120.g18432218.el 2.5 MB/s | 20 MB 00:07 2024-09-21T03:26:34.574 INFO:teuthology.orchestra.run.smithi114.stdout:(21/157): python3-ceph-common-19.3.0-5120.g1843 2.9 MB/s | 148 kB 00:00 2024-09-21T03:26:34.599 INFO:teuthology.orchestra.run.smithi114.stdout:(22/157): python3-cephfs-19.3.0-5120.g18432218. 3.1 MB/s | 161 kB 00:00 2024-09-21T03:26:34.602 INFO:teuthology.orchestra.run.smithi110.stdout:(106/157): abseil-cpp-20211102.0-3.el9.x86_64.r 2.1 MB/s | 548 kB 00:00 2024-09-21T03:26:34.619 INFO:teuthology.orchestra.run.smithi110.stdout:(107/157): grpc-data-1.46.7-10.el9.noarch.rpm 1.2 MB/s | 19 kB 00:00 2024-09-21T03:26:34.624 INFO:teuthology.orchestra.run.smithi114.stdout:(23/157): python3-rados-19.3.0-5120.g18432218.e 6.3 MB/s | 321 kB 00:00 2024-09-21T03:26:34.644 INFO:teuthology.orchestra.run.smithi110.stdout:(108/157): gperftools-libs-2.9.1-3.el9.x86_64.r 3.3 MB/s | 308 kB 00:00 2024-09-21T03:26:34.649 INFO:teuthology.orchestra.run.smithi114.stdout:(24/157): python3-rbd-19.3.0-5120.g18432218.el9 6.0 MB/s | 307 kB 00:00 2024-09-21T03:26:34.661 INFO:teuthology.orchestra.run.smithi110.stdout:(109/157): libarrow-doc-9.0.0-11.el9.noarch.rpm 1.5 MB/s | 26 kB 00:00 2024-09-21T03:26:34.675 INFO:teuthology.orchestra.run.smithi114.stdout:(25/157): python3-rgw-19.3.0-5120.g18432218.el9 1.9 MB/s | 99 kB 00:00 2024-09-21T03:26:34.678 INFO:teuthology.orchestra.run.smithi110.stdout:(110/157): liboath-2.6.7-2.el9.x86_64.rpm 2.9 MB/s | 49 kB 00:00 2024-09-21T03:26:34.700 INFO:teuthology.orchestra.run.smithi114.stdout:(26/157): rbd-fuse-19.3.0-5120.g18432218.el9.x8 1.7 MB/s | 86 kB 00:00 2024-09-21T03:26:34.703 INFO:teuthology.orchestra.run.smithi110.stdout:(111/157): libunwind-1.6.2-1.el9.x86_64.rpm 2.6 MB/s | 67 kB 00:00 2024-09-21T03:26:34.728 INFO:teuthology.orchestra.run.smithi110.stdout:(112/157): luarocks-3.9.2-5.el9.noarch.rpm 6.0 MB/s | 151 kB 00:00 2024-09-21T03:26:34.767 INFO:teuthology.orchestra.run.smithi114.stdout:(27/157): rbd-nbd-19.3.0-5120.g18432218.el9.x86 2.5 MB/s | 170 kB 00:00 2024-09-21T03:26:34.770 INFO:teuthology.orchestra.run.smithi110.stdout:(113/157): parquet-libs-9.0.0-11.el9.x86_64.rpm 20 MB/s | 839 kB 00:00 2024-09-21T03:26:34.812 INFO:teuthology.orchestra.run.smithi110.stdout:(114/157): python3-asyncssh-2.13.2-5.el9.noarch 13 MB/s | 548 kB 00:00 2024-09-21T03:26:34.834 INFO:teuthology.orchestra.run.smithi114.stdout:(28/157): rbd-mirror-19.3.0-5120.g18432218.el9. 20 MB/s | 3.1 MB 00:00 2024-09-21T03:26:34.846 INFO:teuthology.orchestra.run.smithi110.stdout:(115/157): protobuf-compiler-3.14.0-13.el9.x86_ 1.1 MB/s | 863 kB 00:00 2024-09-21T03:26:34.851 INFO:teuthology.orchestra.run.smithi114.stdout:(29/157): ceph-grafana-dashboards-19.3.0-5120.g 411 kB/s | 34 kB 00:00 2024-09-21T03:26:34.871 INFO:teuthology.orchestra.run.smithi110.stdout:(116/157): python3-bcrypt-3.2.2-1.el9.x86_64.rp 740 kB/s | 43 kB 00:00 2024-09-21T03:26:34.884 INFO:teuthology.orchestra.run.smithi114.stdout:(30/157): ceph-mgr-cephadm-19.3.0-5120.g1843221 3.0 MB/s | 154 kB 00:00 2024-09-21T03:26:34.905 INFO:teuthology.orchestra.run.smithi110.stdout:(117/157): python3-certifi-2023.05.07-4.el9.noa 537 kB/s | 14 kB 00:00 2024-09-21T03:26:34.930 INFO:teuthology.orchestra.run.smithi110.stdout:(118/157): python3-cachetools-4.2.4-1.el9.noarc 384 kB/s | 32 kB 00:00 2024-09-21T03:26:34.955 INFO:teuthology.orchestra.run.smithi110.stdout:(119/157): python3-cheroot-8.6.0-4.el9.noarch.r 3.5 MB/s | 172 kB 00:00 2024-09-21T03:26:34.956 INFO:teuthology.orchestra.run.smithi089.stdout:(69/157): postgresql-contrib-13.16-1.el9.x86_64 1.6 MB/s | 910 kB 00:00 2024-09-21T03:26:35.014 INFO:teuthology.orchestra.run.smithi110.stdout:(120/157): libarrow-9.0.0-11.el9.x86_64.rpm 11 MB/s | 4.4 MB 00:00 2024-09-21T03:26:35.039 INFO:teuthology.orchestra.run.smithi110.stdout:(121/157): python3-cmd2-2.4.2-2.el9.noarch.rpm 3.5 MB/s | 303 kB 00:00 2024-09-21T03:26:35.073 INFO:teuthology.orchestra.run.smithi110.stdout:(122/157): python3-cherrypy-18.6.1-2.el9.noarch 2.5 MB/s | 358 kB 00:00 2024-09-21T03:26:35.081 INFO:teuthology.orchestra.run.smithi089.stdout:(70/157): postgresql-private-libs-13.16-1.el9.x 1.1 MB/s | 137 kB 00:00 2024-09-21T03:26:35.107 INFO:teuthology.orchestra.run.smithi110.stdout:(123/157): python3-google-auth-2.34.0-1.el9.noa 2.3 MB/s | 220 kB 00:00 2024-09-21T03:26:35.132 INFO:teuthology.orchestra.run.smithi110.stdout:(124/157): python3-jaraco-8.2.1-3.el9.noarch.rp 430 kB/s | 11 kB 00:00 2024-09-21T03:26:35.157 INFO:teuthology.orchestra.run.smithi110.stdout:(125/157): python3-grpcio-tools-1.46.7-10.el9.x 1.7 MB/s | 144 kB 00:00 2024-09-21T03:26:35.174 INFO:teuthology.orchestra.run.smithi089.stdout:(71/157): openssl-devel-3.2.2-6.el9.x86_64.rpm 2.5 MB/s | 4.4 MB 00:01 2024-09-21T03:26:35.182 INFO:teuthology.orchestra.run.smithi110.stdout:(126/157): python3-jaraco-classes-3.2.1-5.el9.n 350 kB/s | 18 kB 00:00 2024-09-21T03:26:35.208 INFO:teuthology.orchestra.run.smithi110.stdout:(127/157): python3-jaraco-collections-3.0.0-8.e 461 kB/s | 23 kB 00:00 2024-09-21T03:26:35.233 INFO:teuthology.orchestra.run.smithi110.stdout:(128/157): python3-jaraco-functools-3.5.0-2.el9 387 kB/s | 19 kB 00:00 2024-09-21T03:26:35.258 INFO:teuthology.orchestra.run.smithi110.stdout:(129/157): python3-jaraco-text-3.2.0-6.el9.noar 392 kB/s | 20 kB 00:00 2024-09-21T03:26:35.284 INFO:teuthology.orchestra.run.smithi110.stdout:(130/157): python3-logutils-0.3.5-21.el9.noarch 1.8 MB/s | 46 kB 00:00 2024-09-21T03:26:35.309 INFO:teuthology.orchestra.run.smithi110.stdout:(131/157): python3-more-itertools-8.12.0-2.el9. 3.1 MB/s | 79 kB 00:00 2024-09-21T03:26:35.335 INFO:teuthology.orchestra.run.smithi110.stdout:(132/157): python3-natsort-7.1.1-5.el9.noarch.r 2.3 MB/s | 58 kB 00:00 2024-09-21T03:26:35.360 INFO:teuthology.orchestra.run.smithi114.stdout:(31/157): ceph-test-19.3.0-5120.g18432218.el9.x 20 MB/s | 40 MB 00:02 2024-09-21T03:26:35.368 INFO:teuthology.orchestra.run.smithi110.stdout:(133/157): python3-pecan-1.4.2-3.el9.noarch.rpm 8.0 MB/s | 272 kB 00:00 2024-09-21T03:26:35.382 INFO:teuthology.orchestra.run.smithi089.stdout:(72/157): openblas-openmp-0.3.26-2.el9.x86_64.r 2.5 MB/s | 4.9 MB 00:01 2024-09-21T03:26:35.402 INFO:teuthology.orchestra.run.smithi110.stdout:(134/157): python3-kubernetes-26.1.0-2.el9.noar 6.1 MB/s | 1.0 MB 00:00 2024-09-21T03:26:35.410 INFO:teuthology.orchestra.run.smithi114.stdout:(32/157): ceph-mgr-modules-core-19.3.0-5120.g18 5.8 MB/s | 292 kB 00:00 2024-09-21T03:26:35.466 INFO:teuthology.orchestra.run.smithi089.stdout:(73/157): python3-attrs-20.3.0-7.el9.noarch.rpm 1.0 MB/s | 87 kB 00:00 2024-09-21T03:26:35.490 INFO:teuthology.orchestra.run.smithi110.stdout:(135/157): python3-portend-3.1.0-2.el9.noarch.r 135 kB/s | 16 kB 00:00 2024-09-21T03:26:35.510 INFO:teuthology.orchestra.run.smithi114.stdout:(33/157): ceph-mgr-diskprediction-local-19.3.0- 12 MB/s | 7.4 MB 00:00 2024-09-21T03:26:35.527 INFO:teuthology.orchestra.run.smithi110.stdout:(136/157): python3-pyperclip-1.8.2-2.el9.noarch 766 kB/s | 28 kB 00:00 2024-09-21T03:26:35.536 INFO:teuthology.orchestra.run.smithi114.stdout:(34/157): ceph-mgr-rook-19.3.0-5120.g18432218.e 394 kB/s | 49 kB 00:00 2024-09-21T03:26:35.553 INFO:teuthology.orchestra.run.smithi110.stdout:(137/157): python3-pyOpenSSL-21.0.0-1.el9.noarc 598 kB/s | 90 kB 00:00 2024-09-21T03:26:35.561 INFO:teuthology.orchestra.run.smithi114.stdout:(35/157): ceph-prometheus-alerts-19.3.0-5120.g1 328 kB/s | 16 kB 00:00 2024-09-21T03:26:35.578 INFO:teuthology.orchestra.run.smithi110.stdout:(138/157): python3-repoze-lru-0.7-16.el9.noarch 613 kB/s | 31 kB 00:00 2024-09-21T03:26:35.586 INFO:teuthology.orchestra.run.smithi114.stdout:(36/157): ceph-volume-19.3.0-5120.g18432218.el9 5.6 MB/s | 285 kB 00:00 2024-09-21T03:26:35.603 INFO:teuthology.orchestra.run.smithi110.stdout:(139/157): python3-rsa-4.9-2.el9.noarch.rpm 2.3 MB/s | 59 kB 00:00 2024-09-21T03:26:35.620 INFO:teuthology.orchestra.run.smithi114.stdout:(37/157): cephadm-19.3.0-5120.g18432218.el9.noa 13 MB/s | 784 kB 00:00 2024-09-21T03:26:35.628 INFO:teuthology.orchestra.run.smithi110.stdout:(140/157): python3-tempora-5.0.0-2.el9.noarch.r 1.4 MB/s | 36 kB 00:00 2024-09-21T03:26:35.645 INFO:teuthology.orchestra.run.smithi114.stdout:(38/157): cephfs-top-19.3.0-5120.g18432218.el9. 315 kB/s | 18 kB 00:00 2024-09-21T03:26:35.654 INFO:teuthology.orchestra.run.smithi110.stdout:(141/157): python3-routes-2.5.1-5.el9.noarch.rp 1.8 MB/s | 188 kB 00:00 2024-09-21T03:26:35.679 INFO:teuthology.orchestra.run.smithi110.stdout:(142/157): python3-typing-extensions-4.12.2-2.e 1.4 MB/s | 74 kB 00:00 2024-09-21T03:26:35.696 INFO:teuthology.orchestra.run.smithi110.stdout:(143/157): python3-websocket-client-1.2.3-2.el9 5.3 MB/s | 90 kB 00:00 2024-09-21T03:26:35.721 INFO:teuthology.orchestra.run.smithi110.stdout:(144/157): python3-webob-1.8.8-2.el9.noarch.rpm 3.4 MB/s | 230 kB 00:00 2024-09-21T03:26:35.738 INFO:teuthology.orchestra.run.smithi110.stdout:(145/157): python3-zc-lockfile-2.0-10.el9.noarc 1.2 MB/s | 20 kB 00:00 2024-09-21T03:26:35.749 INFO:teuthology.orchestra.run.smithi089.stdout:(74/157): protobuf-3.14.0-13.el9.x86_64.rpm 1.8 MB/s | 1.0 MB 00:00 2024-09-21T03:26:35.780 INFO:teuthology.orchestra.run.smithi110.stdout:(146/157): python3-grpcio-1.46.7-10.el9.x86_64. 2.8 MB/s | 2.0 MB 00:00 2024-09-21T03:26:35.806 INFO:teuthology.orchestra.run.smithi110.stdout:(147/157): re2-20211101-20.el9.x86_64.rpm 2.8 MB/s | 191 kB 00:00 2024-09-21T03:26:35.808 INFO:teuthology.orchestra.run.smithi089.stdout:(75/157): python3-colorama-0.4.6-3.el9.noarch.r 1.0 MB/s | 61 kB 00:00 2024-09-21T03:26:35.831 INFO:teuthology.orchestra.run.smithi110.stdout:(148/157): python3-werkzeug-2.0.3-3.el9.1.noarc 3.1 MB/s | 427 kB 00:00 2024-09-21T03:26:35.837 INFO:teuthology.orchestra.run.smithi114.stdout:(39/157): ceph-mgr-dashboard-19.3.0-5120.g18432 14 MB/s | 14 MB 00:00 2024-09-21T03:26:35.871 INFO:teuthology.orchestra.run.smithi114.stdout:(40/157): fuse-2.9.9-16.el9.x86_64.rpm 319 kB/s | 80 kB 00:00 2024-09-21T03:26:35.881 INFO:teuthology.orchestra.run.smithi110.stdout:(149/157): thrift-0.15.0-2.el9.x86_64.rpm 16 MB/s | 1.6 MB 00:00 2024-09-21T03:26:35.938 INFO:teuthology.orchestra.run.smithi114.stdout:(41/157): libgfortran-11.5.0-2.el9.x86_64.rpm 2.7 MB/s | 801 kB 00:00 2024-09-21T03:26:35.950 INFO:teuthology.orchestra.run.smithi089.stdout:(76/157): python3-devel-3.9.19-8.el9.x86_64.rpm 1.7 MB/s | 245 kB 00:00 2024-09-21T03:26:35.971 INFO:teuthology.orchestra.run.smithi114.stdout:(42/157): python3-cffi-1.14.5-5.el9.x86_64.rpm 2.5 MB/s | 253 kB 00:00 2024-09-21T03:26:35.997 INFO:teuthology.orchestra.run.smithi114.stdout:(43/157): libquadmath-11.5.0-2.el9.x86_64.rpm 1.2 MB/s | 191 kB 00:00 2024-09-21T03:26:36.047 INFO:teuthology.orchestra.run.smithi114.stdout:(44/157): python3-ply-3.11-14.el9.noarch.rpm 1.4 MB/s | 106 kB 00:00 2024-09-21T03:26:36.056 INFO:teuthology.orchestra.run.smithi110.stdout:(150/157): NetworkManager-1.51.0-1.el9.x86_64.r 13 MB/s | 2.3 MB 00:00 2024-09-21T03:26:36.072 INFO:teuthology.orchestra.run.smithi114.stdout:(45/157): python3-pycparser-2.20-6.el9.noarch.r 1.8 MB/s | 135 kB 00:00 2024-09-21T03:26:36.092 INFO:teuthology.orchestra.run.smithi089.stdout:(77/157): python3-jinja2-2.11.3-6.el9.noarch.rp 1.7 MB/s | 248 kB 00:00 2024-09-21T03:26:36.106 INFO:teuthology.orchestra.run.smithi114.stdout:(46/157): python3-cryptography-36.0.1-4.el9.x86 7.3 MB/s | 1.2 MB 00:00 2024-09-21T03:26:36.124 INFO:teuthology.orchestra.run.smithi110.stdout:(151/157): librbd1-19.3.0-5120.g18432218.el9.x8 11 MB/s | 3.2 MB 00:00 2024-09-21T03:26:36.131 INFO:teuthology.orchestra.run.smithi114.stdout:(47/157): python3-requests-2.25.1-8.el9.noarch. 1.5 MB/s | 125 kB 00:00 2024-09-21T03:26:36.142 INFO:teuthology.orchestra.run.smithi089.stdout:(78/157): python3-jsonpatch-1.21-16.el9.noarch. 512 kB/s | 26 kB 00:00 2024-09-21T03:26:36.156 INFO:teuthology.orchestra.run.smithi114.stdout:(48/157): python3-urllib3-1.26.5-6.el9.noarch.r 2.5 MB/s | 215 kB 00:00 2024-09-21T03:26:36.173 INFO:teuthology.orchestra.run.smithi110.stdout:(152/157): NetworkManager-libnm-1.51.0-1.el9.x8 15 MB/s | 1.8 MB 00:00 2024-09-21T03:26:36.192 INFO:teuthology.orchestra.run.smithi089.stdout:(79/157): python3-jsonpointer-2.0-4.el9.noarch. 382 kB/s | 19 kB 00:00 2024-09-21T03:26:36.233 INFO:teuthology.orchestra.run.smithi110.stdout:(153/157): librados2-19.3.0-5120.g18432218.el9. 7.9 MB/s | 3.4 MB 00:00 2024-09-21T03:26:36.256 INFO:teuthology.orchestra.run.smithi114.stdout:(49/157): boost-program-options-1.75.0-8.el9.x8 1.0 MB/s | 107 kB 00:00 2024-09-21T03:26:36.258 INFO:teuthology.orchestra.run.smithi110.stdout:(154/157): NetworkManager-team-1.51.0-1.el9.x86 305 kB/s | 41 kB 00:00 2024-09-21T03:26:36.292 INFO:teuthology.orchestra.run.smithi110.stdout:(155/157): NetworkManager-tui-1.51.0-1.el9.x86_ 2.2 MB/s | 249 kB 00:00 2024-09-21T03:26:36.367 INFO:teuthology.orchestra.run.smithi089.stdout:(80/157): python3-mako-1.1.4-6.el9.noarch.rpm 1.6 MB/s | 172 kB 00:00 2024-09-21T03:26:36.367 INFO:teuthology.orchestra.run.smithi089.stdout:(81/157): python3-markupsafe-1.1.1-12.el9.x86_6 694 kB/s | 35 kB 00:00 2024-09-21T03:26:36.368 INFO:teuthology.orchestra.run.smithi114.stdout:(50/157): elfutils-libelf-devel-0.191-4.el9.x86 520 kB/s | 22 kB 00:00 2024-09-21T03:26:36.368 INFO:teuthology.orchestra.run.smithi114.stdout:(51/157): automake-1.16.2-8.el9.noarch.rpm 2.9 MB/s | 689 kB 00:00 2024-09-21T03:26:36.368 INFO:teuthology.orchestra.run.smithi114.stdout:(52/157): bison-3.7.4-5.el9.x86_64.rpm 3.9 MB/s | 943 kB 00:00 2024-09-21T03:26:36.391 INFO:teuthology.orchestra.run.smithi114.stdout:(53/157): flexiblas-3.0.4-8.el9.x86_64.rpm 651 kB/s | 33 kB 00:00 2024-09-21T03:26:36.416 INFO:teuthology.orchestra.run.smithi114.stdout:(54/157): flex-2.6.4-9.el9.x86_64.rpm 2.6 MB/s | 313 kB 00:00 2024-09-21T03:26:36.425 INFO:teuthology.orchestra.run.smithi110.stdout:(156/157): openssl-libs-3.2.2-6.el9.x86_64.rpm 14 MB/s | 2.4 MB 00:00 2024-09-21T03:26:36.441 INFO:teuthology.orchestra.run.smithi114.stdout:(55/157): flexiblas-openblas-openmp-3.0.4-8.el9 363 kB/s | 18 kB 00:00 2024-09-21T03:26:36.467 INFO:teuthology.orchestra.run.smithi110.stdout:(157/157): openssl-3.2.2-6.el9.x86_64.rpm 5.8 MB/s | 1.4 MB 00:00 2024-09-21T03:26:36.467 INFO:teuthology.orchestra.run.smithi114.stdout:(56/157): libaio-devel-0.3.111-13.el9.x86_64.rp 222 kB/s | 11 kB 00:00 2024-09-21T03:26:36.472 INFO:teuthology.orchestra.run.smithi110.stdout:-------------------------------------------------------------------------------- 2024-09-21T03:26:36.472 INFO:teuthology.orchestra.run.smithi110.stdout:Total 22 MB/s | 228 MB 00:10 2024-09-21T03:26:36.509 INFO:teuthology.orchestra.run.smithi114.stdout:(57/157): libpmemobj-1.12.1-1.el9.x86_64.rpm 3.8 MB/s | 160 kB 00:00 2024-09-21T03:26:36.534 INFO:teuthology.orchestra.run.smithi114.stdout:(58/157): libnbd-1.20.2-2.el9.x86_64.rpm 1.8 MB/s | 171 kB 00:00 2024-09-21T03:26:36.584 INFO:teuthology.orchestra.run.smithi114.stdout:(59/157): librabbitmq-0.11.0-7.el9.x86_64.rpm 603 kB/s | 45 kB 00:00 2024-09-21T03:26:36.693 INFO:teuthology.orchestra.run.smithi114.stdout:(60/157): libtool-2.4.6-46.el9.x86_64.rpm 5.2 MB/s | 581 kB 00:00 2024-09-21T03:26:36.752 INFO:teuthology.orchestra.run.smithi114.stdout:(61/157): librdkafka-1.6.1-102.el9.x86_64.rpm 3.0 MB/s | 662 kB 00:00 2024-09-21T03:26:36.777 INFO:teuthology.orchestra.run.smithi114.stdout:(62/157): libuuid-devel-2.37.4-20.el9.x86_64.rp 292 kB/s | 24 kB 00:00 2024-09-21T03:26:36.802 INFO:teuthology.orchestra.run.smithi114.stdout:(63/157): libzstd-devel-1.5.1-2.el9.x86_64.rpm 948 kB/s | 47 kB 00:00 2024-09-21T03:26:36.878 INFO:teuthology.orchestra.run.smithi114.stdout:(64/157): lua-5.4.4-4.el9.x86_64.rpm 2.5 MB/s | 188 kB 00:00 2024-09-21T03:26:36.903 INFO:teuthology.orchestra.run.smithi114.stdout:(65/157): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.3 MB/s | 292 kB 00:00 2024-09-21T03:26:36.928 INFO:teuthology.orchestra.run.smithi114.stdout:(66/157): openblas-0.3.26-2.el9.x86_64.rpm 769 kB/s | 38 kB 00:00 2024-09-21T03:26:37.253 INFO:teuthology.orchestra.run.smithi114.stdout:(67/157): flexiblas-netlib-3.0.4-8.el9.x86_64.r 3.4 MB/s | 3.0 MB 00:00 2024-09-21T03:26:37.657 INFO:teuthology.orchestra.run.smithi110.stdout:Running transaction check 2024-09-21T03:26:37.687 INFO:teuthology.orchestra.run.smithi114.stdout:(68/157): postgresql-13.16-1.el9.x86_64.rpm 3.7 MB/s | 1.6 MB 00:00 2024-09-21T03:26:37.807 INFO:teuthology.orchestra.run.smithi110.stdout:Transaction check succeeded. 2024-09-21T03:26:37.807 INFO:teuthology.orchestra.run.smithi110.stdout:Running transaction test 2024-09-21T03:26:37.954 INFO:teuthology.orchestra.run.smithi114.stdout:(69/157): postgresql-contrib-13.16-1.el9.x86_64 3.3 MB/s | 910 kB 00:00 2024-09-21T03:26:38.021 INFO:teuthology.orchestra.run.smithi114.stdout:(70/157): postgresql-private-libs-13.16-1.el9.x 2.0 MB/s | 137 kB 00:00 2024-09-21T03:26:38.026 INFO:teuthology.orchestra.run.smithi089.stdout:(82/157): postgresql-server-13.16-1.el9.x86_64. 2.0 MB/s | 5.8 MB 00:02 2024-09-21T03:26:38.088 INFO:teuthology.orchestra.run.smithi114.stdout:(71/157): openssl-devel-3.2.2-6.el9.x86_64.rpm 3.8 MB/s | 4.4 MB 00:01 2024-09-21T03:26:38.177 INFO:teuthology.orchestra.run.smithi089.stdout:(83/157): python3-numpy-f2py-1.20.1-5.el9.x86_6 1.6 MB/s | 250 kB 00:00 2024-09-21T03:26:38.319 INFO:teuthology.orchestra.run.smithi089.stdout:(84/157): python3-oauthlib-3.1.1-5.el9.noarch.r 1.5 MB/s | 222 kB 00:00 2024-09-21T03:26:38.338 INFO:teuthology.orchestra.run.smithi114.stdout:(72/157): openblas-openmp-0.3.26-2.el9.x86_64.r 3.4 MB/s | 4.9 MB 00:01 2024-09-21T03:26:38.380 INFO:teuthology.orchestra.run.smithi114.stdout:(73/157): protobuf-3.14.0-13.el9.x86_64.rpm 3.5 MB/s | 1.0 MB 00:00 2024-09-21T03:26:38.402 INFO:teuthology.orchestra.run.smithi089.stdout:(85/157): python3-packaging-20.9-5.el9.noarch.r 926 kB/s | 77 kB 00:00 2024-09-21T03:26:38.413 INFO:teuthology.orchestra.run.smithi114.stdout:(74/157): python3-attrs-20.3.0-7.el9.noarch.rpm 1.1 MB/s | 87 kB 00:00 2024-09-21T03:26:38.447 INFO:teuthology.orchestra.run.smithi114.stdout:(75/157): python3-colorama-0.4.6-3.el9.noarch.r 1.8 MB/s | 61 kB 00:00 2024-09-21T03:26:38.503 INFO:teuthology.orchestra.run.smithi089.stdout:(86/157): python3-babel-2.9.1-2.el9.noarch.rpm 2.0 MB/s | 6.0 MB 00:03 2024-09-21T03:26:38.528 INFO:teuthology.orchestra.run.smithi089.stdout:(87/157): python3-prettytable-0.7.2-27.el9.noar 337 kB/s | 42 kB 00:00 2024-09-21T03:26:38.539 INFO:teuthology.orchestra.run.smithi114.stdout:(76/157): python3-devel-3.9.19-8.el9.x86_64.rpm 2.6 MB/s | 245 kB 00:00 2024-09-21T03:26:38.631 INFO:teuthology.orchestra.run.smithi114.stdout:(77/157): python3-jinja2-2.11.3-6.el9.noarch.rp 2.7 MB/s | 248 kB 00:00 2024-09-21T03:26:38.637 INFO:teuthology.orchestra.run.smithi089.stdout:(88/157): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.4 MB/s | 159 kB 00:00 2024-09-21T03:26:38.673 INFO:teuthology.orchestra.run.smithi114.stdout:(78/157): python3-jsonpatch-1.21-16.el9.noarch. 617 kB/s | 26 kB 00:00 2024-09-21T03:26:38.695 INFO:teuthology.orchestra.run.smithi089.stdout:(89/157): python3-protobuf-3.14.0-13.el9.noarch 1.4 MB/s | 269 kB 00:00 2024-09-21T03:26:38.715 INFO:teuthology.orchestra.run.smithi114.stdout:(79/157): python3-jsonpointer-2.0-4.el9.noarch. 460 kB/s | 19 kB 00:00 2024-09-21T03:26:38.745 INFO:teuthology.orchestra.run.smithi089.stdout:(90/157): python3-pytz-2021.1-5.el9.noarch.rpm 1.0 MB/s | 51 kB 00:00 2024-09-21T03:26:38.790 INFO:teuthology.orchestra.run.smithi114.stdout:(80/157): python3-mako-1.1.4-6.el9.noarch.rpm 2.2 MB/s | 172 kB 00:00 2024-09-21T03:26:38.796 INFO:teuthology.orchestra.run.smithi089.stdout:(91/157): python3-requests-oauthlib-1.3.0-12.el 1.0 MB/s | 54 kB 00:00 2024-09-21T03:26:38.832 INFO:teuthology.orchestra.run.smithi114.stdout:(81/157): python3-markupsafe-1.1.1-12.el9.x86_6 834 kB/s | 35 kB 00:00 2024-09-21T03:26:38.846 INFO:teuthology.orchestra.run.smithi089.stdout:(92/157): python3-pyasn1-modules-0.4.8-6.el9.no 1.3 MB/s | 279 kB 00:00 2024-09-21T03:26:38.897 INFO:teuthology.orchestra.run.smithi089.stdout:(93/157): python3-toml-0.10.2-6.el9.noarch.rpm 836 kB/s | 42 kB 00:00 2024-09-21T03:26:38.972 INFO:teuthology.orchestra.run.smithi089.stdout:(94/157): python3-numpy-1.20.1-5.el9.x86_64.rpm 1.9 MB/s | 5.1 MB 00:02 2024-09-21T03:26:38.989 INFO:teuthology.orchestra.run.smithi089.stdout:(95/157): python3-wcwidth-0.2.5-8.el9.noarch.rp 480 kB/s | 44 kB 00:00 2024-09-21T03:26:39.031 INFO:teuthology.orchestra.run.smithi089.stdout:(96/157): qatlib-service-24.02.0-1.el9.x86_64.r 861 kB/s | 36 kB 00:00 2024-09-21T03:26:39.049 INFO:teuthology.orchestra.run.smithi114.stdout:(82/157): python3-babel-2.9.1-2.el9.noarch.rpm 8.9 MB/s | 6.0 MB 00:00 2024-09-21T03:26:39.106 INFO:teuthology.orchestra.run.smithi089.stdout:(97/157): qatzip-libs-1.2.0-1.el9.x86_64.rpm 629 kB/s | 47 kB 00:00 2024-09-21T03:26:39.107 INFO:teuthology.orchestra.run.smithi114.stdout:(83/157): python3-numpy-f2py-1.20.1-5.el9.x86_6 4.2 MB/s | 250 kB 00:00 2024-09-21T03:26:39.157 INFO:teuthology.orchestra.run.smithi089.stdout:(98/157): qatlib-24.02.0-1.el9.x86_64.rpm 1.2 MB/s | 221 kB 00:00 2024-09-21T03:26:39.174 INFO:teuthology.orchestra.run.smithi114.stdout:(84/157): postgresql-server-13.16-1.el9.x86_64. 5.0 MB/s | 5.8 MB 00:01 2024-09-21T03:26:39.200 INFO:teuthology.orchestra.run.smithi114.stdout:(85/157): python3-oauthlib-3.1.1-5.el9.noarch.r 2.4 MB/s | 222 kB 00:00 2024-09-21T03:26:39.207 INFO:teuthology.orchestra.run.smithi089.stdout:(99/157): uuid-1.6.2-55.el9.x86_64.rpm 1.1 MB/s | 58 kB 00:00 2024-09-21T03:26:39.225 INFO:teuthology.orchestra.run.smithi114.stdout:(86/157): python3-packaging-20.9-5.el9.noarch.r 1.5 MB/s | 77 kB 00:00 2024-09-21T03:26:39.242 INFO:teuthology.orchestra.run.smithi114.stdout:(87/157): python3-prettytable-0.7.2-27.el9.noar 1.0 MB/s | 42 kB 00:00 2024-09-21T03:26:39.291 INFO:teuthology.orchestra.run.smithi089.stdout:(100/157): xfsprogs-devel-6.4.0-4.el9.x86_64.rp 1.1 MB/s | 96 kB 00:00 2024-09-21T03:26:39.309 INFO:teuthology.orchestra.run.smithi114.stdout:(88/157): python3-numpy-1.20.1-5.el9.x86_64.rpm 11 MB/s | 5.1 MB 00:00 2024-09-21T03:26:39.316 INFO:teuthology.orchestra.run.smithi089.stdout:(101/157): socat-1.7.4.1-6.el9.x86_64.rpm 1.4 MB/s | 304 kB 00:00 2024-09-21T03:26:39.334 INFO:teuthology.orchestra.run.smithi114.stdout:(89/157): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.7 MB/s | 159 kB 00:00 2024-09-21T03:26:39.359 INFO:teuthology.orchestra.run.smithi114.stdout:(90/157): python3-protobuf-3.14.0-13.el9.noarch 2.0 MB/s | 269 kB 00:00 2024-09-21T03:26:39.366 INFO:teuthology.orchestra.run.smithi089.stdout:(102/157): zlib-devel-1.2.11-41.el9.x86_64.rpm 898 kB/s | 45 kB 00:00 2024-09-21T03:26:39.387 INFO:teuthology.orchestra.run.smithi114.stdout:(91/157): python3-pytz-2021.1-5.el9.noarch.rpm 976 kB/s | 51 kB 00:00 2024-09-21T03:26:39.391 INFO:teuthology.orchestra.run.smithi089.stdout:(103/157): xmlstarlet-1.6.1-20.el9.x86_64.rpm 633 kB/s | 64 kB 00:00 2024-09-21T03:26:39.412 INFO:teuthology.orchestra.run.smithi114.stdout:(92/157): python3-requests-oauthlib-1.3.0-12.el 1.0 MB/s | 54 kB 00:00 2024-09-21T03:26:39.437 INFO:teuthology.orchestra.run.smithi114.stdout:(93/157): python3-pyasn1-modules-0.4.8-6.el9.no 2.1 MB/s | 279 kB 00:00 2024-09-21T03:26:39.463 INFO:teuthology.orchestra.run.smithi114.stdout:(94/157): python3-toml-0.10.2-6.el9.noarch.rpm 828 kB/s | 42 kB 00:00 2024-09-21T03:26:39.525 INFO:teuthology.orchestra.run.smithi089.stdout:(104/157): lua-devel-5.4.4-4.el9.x86_64.rpm 141 kB/s | 22 kB 00:00 2024-09-21T03:26:39.543 INFO:teuthology.orchestra.run.smithi114.stdout:(95/157): python3-wcwidth-0.2.5-8.el9.noarch.rp 420 kB/s | 44 kB 00:00 2024-09-21T03:26:39.580 INFO:teuthology.orchestra.run.smithi114.stdout:(96/157): qatlib-24.02.0-1.el9.x86_64.rpm 1.8 MB/s | 221 kB 00:00 2024-09-21T03:26:39.584 INFO:teuthology.orchestra.run.smithi089.stdout:(105/157): abseil-cpp-20211102.0-3.el9.x86_64.r 9.2 MB/s | 548 kB 00:00 2024-09-21T03:26:39.605 INFO:teuthology.orchestra.run.smithi114.stdout:(97/157): qatlib-service-24.02.0-1.el9.x86_64.r 580 kB/s | 36 kB 00:00 2024-09-21T03:26:39.609 INFO:teuthology.orchestra.run.smithi089.stdout:(106/157): gperftools-libs-2.9.1-3.el9.x86_64.r 12 MB/s | 308 kB 00:00 2024-09-21T03:26:39.631 INFO:teuthology.orchestra.run.smithi114.stdout:(98/157): qatzip-libs-1.2.0-1.el9.x86_64.rpm 943 kB/s | 47 kB 00:00 2024-09-21T03:26:39.634 INFO:teuthology.orchestra.run.smithi089.stdout:(107/157): grpc-data-1.46.7-10.el9.noarch.rpm 783 kB/s | 19 kB 00:00 2024-09-21T03:26:39.664 INFO:teuthology.orchestra.run.smithi114.stdout:(99/157): socat-1.7.4.1-6.el9.x86_64.rpm 5.1 MB/s | 304 kB 00:00 2024-09-21T03:26:39.689 INFO:teuthology.orchestra.run.smithi114.stdout:(100/157): uuid-1.6.2-55.el9.x86_64.rpm 991 kB/s | 58 kB 00:00 2024-09-21T03:26:39.715 INFO:teuthology.orchestra.run.smithi114.stdout:(101/157): xfsprogs-devel-6.4.0-4.el9.x86_64.rp 1.9 MB/s | 96 kB 00:00 2024-09-21T03:26:39.740 INFO:teuthology.orchestra.run.smithi114.stdout:(102/157): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.2 MB/s | 64 kB 00:00 2024-09-21T03:26:39.751 INFO:teuthology.orchestra.run.smithi089.stdout:(108/157): libarrow-9.0.0-11.el9.x86_64.rpm 38 MB/s | 4.4 MB 00:00 2024-09-21T03:26:39.765 INFO:teuthology.orchestra.run.smithi114.stdout:(103/157): zlib-devel-1.2.11-41.el9.x86_64.rpm 894 kB/s | 45 kB 00:00 2024-09-21T03:26:39.776 INFO:teuthology.orchestra.run.smithi089.stdout:(109/157): protobuf-compiler-3.14.0-13.el9.x86_ 2.2 MB/s | 863 kB 00:00 2024-09-21T03:26:39.802 INFO:teuthology.orchestra.run.smithi089.stdout:(110/157): libarrow-doc-9.0.0-11.el9.noarch.rpm 516 kB/s | 26 kB 00:00 2024-09-21T03:26:39.835 INFO:teuthology.orchestra.run.smithi089.stdout:(111/157): libunwind-1.6.2-1.el9.x86_64.rpm 2.5 MB/s | 67 kB 00:00 2024-09-21T03:26:39.857 INFO:teuthology.orchestra.run.smithi114.stdout:(104/157): lua-devel-5.4.4-4.el9.x86_64.rpm 191 kB/s | 22 kB 00:00 2024-09-21T03:26:39.861 INFO:teuthology.orchestra.run.smithi089.stdout:(112/157): luarocks-3.9.2-5.el9.noarch.rpm 6.4 MB/s | 151 kB 00:00 2024-09-21T03:26:39.886 INFO:teuthology.orchestra.run.smithi089.stdout:(113/157): liboath-2.6.7-2.el9.x86_64.rpm 445 kB/s | 49 kB 00:00 2024-09-21T03:26:39.928 INFO:teuthology.orchestra.run.smithi089.stdout:(114/157): parquet-libs-9.0.0-11.el9.x86_64.rpm 12 MB/s | 839 kB 00:00 2024-09-21T03:26:39.945 INFO:teuthology.orchestra.run.smithi089.stdout:(115/157): python3-bcrypt-3.2.2-1.el9.x86_64.rp 2.5 MB/s | 43 kB 00:00 2024-09-21T03:26:39.963 INFO:teuthology.orchestra.run.smithi089.stdout:(116/157): python3-cachetools-4.2.4-1.el9.noarc 1.9 MB/s | 32 kB 00:00 2024-09-21T03:26:39.979 INFO:teuthology.orchestra.run.smithi089.stdout:(117/157): python3-certifi-2023.05.07-4.el9.noa 847 kB/s | 14 kB 00:00 2024-09-21T03:26:40.013 INFO:teuthology.orchestra.run.smithi089.stdout:(118/157): python3-asyncssh-2.13.2-5.el9.noarch 4.2 MB/s | 548 kB 00:00 2024-09-21T03:26:40.038 INFO:teuthology.orchestra.run.smithi089.stdout:(119/157): python3-cheroot-8.6.0-4.el9.noarch.r 2.9 MB/s | 172 kB 00:00 2024-09-21T03:26:40.041 INFO:teuthology.orchestra.run.smithi114.stdout:(105/157): python3-scipy-1.6.2-8.el9.x86_64.rpm 25 MB/s | 16 MB 00:00 2024-09-21T03:26:40.042 INFO:teuthology.orchestra.run.smithi110.stdout:Transaction test succeeded. 2024-09-21T03:26:40.042 INFO:teuthology.orchestra.run.smithi110.stdout:Running transaction 2024-09-21T03:26:40.063 INFO:teuthology.orchestra.run.smithi089.stdout:(120/157): python3-cherrypy-18.6.1-2.el9.noarch 7.0 MB/s | 358 kB 00:00 2024-09-21T03:26:40.089 INFO:teuthology.orchestra.run.smithi089.stdout:(121/157): python3-cmd2-2.4.2-2.el9.noarch.rpm 5.9 MB/s | 303 kB 00:00 2024-09-21T03:26:40.091 INFO:teuthology.orchestra.run.smithi114.stdout:(106/157): abseil-cpp-20211102.0-3.el9.x86_64.r 2.3 MB/s | 548 kB 00:00 2024-09-21T03:26:40.114 INFO:teuthology.orchestra.run.smithi089.stdout:(122/157): python3-google-auth-2.34.0-1.el9.noa 4.3 MB/s | 220 kB 00:00 2024-09-21T03:26:40.116 INFO:teuthology.orchestra.run.smithi114.stdout:(107/157): grpc-data-1.46.7-10.el9.noarch.rpm 855 kB/s | 19 kB 00:00 2024-09-21T03:26:40.147 INFO:teuthology.orchestra.run.smithi089.stdout:(123/157): python3-grpcio-tools-1.46.7-10.el9.x 4.2 MB/s | 144 kB 00:00 2024-09-21T03:26:40.150 INFO:teuthology.orchestra.run.smithi114.stdout:(108/157): protobuf-compiler-3.14.0-13.el9.x86_ 2.2 MB/s | 863 kB 00:00 2024-09-21T03:26:40.173 INFO:teuthology.orchestra.run.smithi089.stdout:(124/157): python3-jaraco-8.2.1-3.el9.noarch.rp 423 kB/s | 11 kB 00:00 2024-09-21T03:26:40.183 INFO:teuthology.orchestra.run.smithi114.stdout:(109/157): gperftools-libs-2.9.1-3.el9.x86_64.r 2.1 MB/s | 308 kB 00:00 2024-09-21T03:26:40.198 INFO:teuthology.orchestra.run.smithi089.stdout:(125/157): python3-jaraco-classes-3.2.1-5.el9.n 710 kB/s | 18 kB 00:00 2024-09-21T03:26:40.209 INFO:teuthology.orchestra.run.smithi114.stdout:(110/157): liboath-2.6.7-2.el9.x86_64.rpm 2.0 MB/s | 49 kB 00:00 2024-09-21T03:26:40.224 INFO:teuthology.orchestra.run.smithi089.stdout:(126/157): python3-jaraco-collections-3.0.0-8.e 925 kB/s | 23 kB 00:00 2024-09-21T03:26:40.234 INFO:teuthology.orchestra.run.smithi114.stdout:(111/157): libarrow-doc-9.0.0-11.el9.noarch.rpm 311 kB/s | 26 kB 00:00 2024-09-21T03:26:40.249 INFO:teuthology.orchestra.run.smithi089.stdout:(127/157): python3-jaraco-functools-3.5.0-2.el9 780 kB/s | 19 kB 00:00 2024-09-21T03:26:40.251 INFO:teuthology.orchestra.run.smithi114.stdout:(112/157): libunwind-1.6.2-1.el9.x86_64.rpm 1.6 MB/s | 67 kB 00:00 2024-09-21T03:26:40.275 INFO:teuthology.orchestra.run.smithi089.stdout:(128/157): python3-jaraco-text-3.2.0-6.el9.noar 792 kB/s | 20 kB 00:00 2024-09-21T03:26:40.276 INFO:teuthology.orchestra.run.smithi114.stdout:(113/157): luarocks-3.9.2-5.el9.noarch.rpm 3.5 MB/s | 151 kB 00:00 2024-09-21T03:26:40.325 INFO:teuthology.orchestra.run.smithi089.stdout:(129/157): python3-grpcio-1.46.7-10.el9.x86_64. 8.6 MB/s | 2.0 MB 00:00 2024-09-21T03:26:40.326 INFO:teuthology.orchestra.run.smithi114.stdout:(114/157): python3-asyncssh-2.13.2-5.el9.noarch 11 MB/s | 548 kB 00:00 2024-09-21T03:26:40.350 INFO:teuthology.orchestra.run.smithi089.stdout:(130/157): python3-logutils-0.3.5-21.el9.noarch 1.8 MB/s | 46 kB 00:00 2024-09-21T03:26:40.360 INFO:teuthology.orchestra.run.smithi114.stdout:(115/157): parquet-libs-9.0.0-11.el9.x86_64.rpm 7.5 MB/s | 839 kB 00:00 2024-09-21T03:26:40.367 INFO:teuthology.orchestra.run.smithi089.stdout:(131/157): python3-more-itertools-8.12.0-2.el9. 4.6 MB/s | 79 kB 00:00 2024-09-21T03:26:40.385 INFO:teuthology.orchestra.run.smithi114.stdout:(116/157): python3-bcrypt-3.2.2-1.el9.x86_64.rp 742 kB/s | 43 kB 00:00 2024-09-21T03:26:40.392 INFO:teuthology.orchestra.run.smithi089.stdout:(132/157): python3-natsort-7.1.1-5.el9.noarch.r 2.3 MB/s | 58 kB 00:00 2024-09-21T03:26:40.410 INFO:teuthology.orchestra.run.smithi114.stdout:(117/157): python3-cachetools-4.2.4-1.el9.noarc 641 kB/s | 32 kB 00:00 2024-09-21T03:26:40.426 INFO:teuthology.orchestra.run.smithi089.stdout:(133/157): python3-kubernetes-26.1.0-2.el9.noar 6.8 MB/s | 1.0 MB 00:00 2024-09-21T03:26:40.436 INFO:teuthology.orchestra.run.smithi114.stdout:(118/157): python3-certifi-2023.05.07-4.el9.noa 281 kB/s | 14 kB 00:00 2024-09-21T03:26:40.451 INFO:teuthology.orchestra.run.smithi089.stdout:(134/157): python3-portend-3.1.0-2.el9.noarch.r 655 kB/s | 16 kB 00:00 2024-09-21T03:26:40.477 INFO:teuthology.orchestra.run.smithi089.stdout:(135/157): python3-pecan-1.4.2-3.el9.noarch.rpm 3.2 MB/s | 272 kB 00:00 2024-09-21T03:26:40.478 INFO:teuthology.orchestra.run.smithi114.stdout:(119/157): python3-cheroot-8.6.0-4.el9.noarch.r 2.5 MB/s | 172 kB 00:00 2024-09-21T03:26:40.502 INFO:teuthology.orchestra.run.smithi089.stdout:(136/157): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.8 MB/s | 90 kB 00:00 2024-09-21T03:26:40.511 INFO:teuthology.orchestra.run.smithi114.stdout:(120/157): python3-cherrypy-18.6.1-2.el9.noarch 4.7 MB/s | 358 kB 00:00 2024-09-21T03:26:40.527 INFO:teuthology.orchestra.run.smithi089.stdout:(137/157): python3-pyperclip-1.8.2-2.el9.noarch 563 kB/s | 28 kB 00:00 2024-09-21T03:26:40.537 INFO:teuthology.orchestra.run.smithi114.stdout:(121/157): python3-cmd2-2.4.2-2.el9.noarch.rpm 5.1 MB/s | 303 kB 00:00 2024-09-21T03:26:40.552 INFO:teuthology.orchestra.run.smithi089.stdout:(138/157): python3-repoze-lru-0.7-16.el9.noarch 611 kB/s | 31 kB 00:00 2024-09-21T03:26:40.562 INFO:teuthology.orchestra.run.smithi114.stdout:(122/157): python3-google-auth-2.34.0-1.el9.noa 4.3 MB/s | 220 kB 00:00 2024-09-21T03:26:40.578 INFO:teuthology.orchestra.run.smithi089.stdout:(139/157): python3-rsa-4.9-2.el9.noarch.rpm 2.3 MB/s | 59 kB 00:00 2024-09-21T03:26:40.596 INFO:teuthology.orchestra.run.smithi114.stdout:(123/157): python3-grpcio-tools-1.46.7-10.el9.x 4.2 MB/s | 144 kB 00:00 2024-09-21T03:26:40.603 INFO:teuthology.orchestra.run.smithi089.stdout:(140/157): python3-routes-2.5.1-5.el9.noarch.rp 2.4 MB/s | 188 kB 00:00 2024-09-21T03:26:40.621 INFO:teuthology.orchestra.run.smithi114.stdout:(124/157): python3-jaraco-8.2.1-3.el9.noarch.rp 430 kB/s | 11 kB 00:00 2024-09-21T03:26:40.629 INFO:teuthology.orchestra.run.smithi089.stdout:(141/157): python3-tempora-5.0.0-2.el9.noarch.r 712 kB/s | 36 kB 00:00 2024-09-21T03:26:40.646 INFO:teuthology.orchestra.run.smithi114.stdout:(125/157): python3-jaraco-classes-3.2.1-5.el9.n 712 kB/s | 18 kB 00:00 2024-09-21T03:26:40.654 INFO:teuthology.orchestra.run.smithi089.stdout:(142/157): python3-typing-extensions-4.12.2-2.e 1.4 MB/s | 74 kB 00:00 2024-09-21T03:26:40.672 INFO:teuthology.orchestra.run.smithi114.stdout:(126/157): python3-jaraco-collections-3.0.0-8.e 926 kB/s | 23 kB 00:00 2024-09-21T03:26:40.679 INFO:teuthology.orchestra.run.smithi089.stdout:(143/157): python3-websocket-client-1.2.3-2.el9 3.5 MB/s | 90 kB 00:00 2024-09-21T03:26:40.697 INFO:teuthology.orchestra.run.smithi114.stdout:(127/157): python3-jaraco-functools-3.5.0-2.el9 788 kB/s | 19 kB 00:00 2024-09-21T03:26:40.704 INFO:teuthology.orchestra.run.smithi089.stdout:(144/157): python3-webob-1.8.8-2.el9.noarch.rpm 3.0 MB/s | 230 kB 00:00 2024-09-21T03:26:40.714 INFO:teuthology.orchestra.run.smithi114.stdout:(128/157): python3-jaraco-text-3.2.0-6.el9.noar 1.2 MB/s | 20 kB 00:00 2024-09-21T03:26:40.730 INFO:teuthology.orchestra.run.smithi089.stdout:(145/157): python3-zc-lockfile-2.0-10.el9.noarc 798 kB/s | 20 kB 00:00 2024-09-21T03:26:40.755 INFO:teuthology.orchestra.run.smithi089.stdout:(146/157): python3-werkzeug-2.0.3-3.el9.1.noarc 5.5 MB/s | 427 kB 00:00 2024-09-21T03:26:40.780 INFO:teuthology.orchestra.run.smithi089.stdout:(147/157): re2-20211101-20.el9.x86_64.rpm 3.7 MB/s | 191 kB 00:00 2024-09-21T03:26:40.797 INFO:teuthology.orchestra.run.smithi114.stdout:(129/157): libarrow-9.0.0-11.el9.x86_64.rpm 6.5 MB/s | 4.4 MB 00:00 2024-09-21T03:26:40.831 INFO:teuthology.orchestra.run.smithi089.stdout:(148/157): thrift-0.15.0-2.el9.x86_64.rpm 21 MB/s | 1.6 MB 00:00 2024-09-21T03:26:40.839 INFO:teuthology.orchestra.run.smithi114.stdout:(130/157): python3-grpcio-1.46.7-10.el9.x86_64. 6.7 MB/s | 2.0 MB 00:00 2024-09-21T03:26:40.856 INFO:teuthology.orchestra.run.smithi114.stdout:(131/157): python3-logutils-0.3.5-21.el9.noarch 790 kB/s | 46 kB 00:00 2024-09-21T03:26:40.882 INFO:teuthology.orchestra.run.smithi114.stdout:(132/157): python3-more-itertools-8.12.0-2.el9. 1.8 MB/s | 79 kB 00:00 2024-09-21T03:26:40.907 INFO:teuthology.orchestra.run.smithi114.stdout:(133/157): python3-natsort-7.1.1-5.el9.noarch.r 1.1 MB/s | 58 kB 00:00 2024-09-21T03:26:40.941 INFO:teuthology.orchestra.run.smithi114.stdout:(134/157): python3-kubernetes-26.1.0-2.el9.noar 4.5 MB/s | 1.0 MB 00:00 2024-09-21T03:26:40.982 INFO:teuthology.orchestra.run.smithi114.stdout:(135/157): python3-portend-3.1.0-2.el9.noarch.r 219 kB/s | 16 kB 00:00 2024-09-21T03:26:41.008 INFO:teuthology.orchestra.run.smithi114.stdout:(136/157): python3-pyperclip-1.8.2-2.el9.noarch 1.1 MB/s | 28 kB 00:00 2024-09-21T03:26:41.033 INFO:teuthology.orchestra.run.smithi114.stdout:(137/157): python3-pyOpenSSL-21.0.0-1.el9.noarc 978 kB/s | 90 kB 00:00 2024-09-21T03:26:41.058 INFO:teuthology.orchestra.run.smithi114.stdout:(138/157): python3-repoze-lru-0.7-16.el9.noarch 612 kB/s | 31 kB 00:00 2024-09-21T03:26:41.073 INFO:teuthology.orchestra.run.smithi089.stdout:(149/157): librbd1-19.3.0-5120.g18432218.el9.x8 13 MB/s | 3.2 MB 00:00 2024-09-21T03:26:41.083 INFO:teuthology.orchestra.run.smithi114.stdout:(139/157): python3-pecan-1.4.2-3.el9.noarch.rpm 1.3 MB/s | 272 kB 00:00 2024-09-21T03:26:41.109 INFO:teuthology.orchestra.run.smithi114.stdout:(140/157): python3-rsa-4.9-2.el9.noarch.rpm 1.2 MB/s | 59 kB 00:00 2024-09-21T03:26:41.134 INFO:teuthology.orchestra.run.smithi114.stdout:(141/157): python3-tempora-5.0.0-2.el9.noarch.r 712 kB/s | 36 kB 00:00 2024-09-21T03:26:41.168 INFO:teuthology.orchestra.run.smithi114.stdout:(142/157): python3-routes-2.5.1-5.el9.noarch.rp 1.4 MB/s | 188 kB 00:00 2024-09-21T03:26:41.185 INFO:teuthology.orchestra.run.smithi114.stdout:(143/157): python3-typing-extensions-4.12.2-2.e 990 kB/s | 74 kB 00:00 2024-09-21T03:26:41.206 INFO:teuthology.orchestra.run.smithi089.stdout:(150/157): librados2-19.3.0-5120.g18432218.el9. 8.0 MB/s | 3.4 MB 00:00 2024-09-21T03:26:41.210 INFO:teuthology.orchestra.run.smithi114.stdout:(144/157): python3-websocket-client-1.2.3-2.el9 2.1 MB/s | 90 kB 00:00 2024-09-21T03:26:41.235 INFO:teuthology.orchestra.run.smithi114.stdout:(145/157): python3-zc-lockfile-2.0-10.el9.noarc 800 kB/s | 20 kB 00:00 2024-09-21T03:26:41.260 INFO:teuthology.orchestra.run.smithi114.stdout:(146/157): python3-webob-1.8.8-2.el9.noarch.rpm 1.8 MB/s | 230 kB 00:00 2024-09-21T03:26:41.286 INFO:teuthology.orchestra.run.smithi114.stdout:(147/157): re2-20211101-20.el9.x86_64.rpm 3.7 MB/s | 191 kB 00:00 2024-09-21T03:26:41.311 INFO:teuthology.orchestra.run.smithi114.stdout:(148/157): python3-werkzeug-2.0.3-3.el9.1.noarc 3.3 MB/s | 427 kB 00:00 2024-09-21T03:26:41.361 INFO:teuthology.orchestra.run.smithi114.stdout:(149/157): thrift-0.15.0-2.el9.x86_64.rpm 16 MB/s | 1.6 MB 00:00 2024-09-21T03:26:41.503 INFO:teuthology.orchestra.run.smithi114.stdout:(150/157): NetworkManager-1.51.0-1.el9.x86_64.r 17 MB/s | 2.3 MB 00:00 2024-09-21T03:26:41.570 INFO:teuthology.orchestra.run.smithi114.stdout:(151/157): librados2-19.3.0-5120.g18432218.el9. 12 MB/s | 3.4 MB 00:00 2024-09-21T03:26:41.629 INFO:teuthology.orchestra.run.smithi114.stdout:(152/157): NetworkManager-libnm-1.51.0-1.el9.x8 14 MB/s | 1.8 MB 00:00 2024-09-21T03:26:41.654 INFO:teuthology.orchestra.run.smithi114.stdout:(153/157): NetworkManager-team-1.51.0-1.el9.x86 489 kB/s | 41 kB 00:00 2024-09-21T03:26:41.696 INFO:teuthology.orchestra.run.smithi114.stdout:(154/157): NetworkManager-tui-1.51.0-1.el9.x86_ 3.6 MB/s | 249 kB 00:00 2024-09-21T03:26:41.746 INFO:teuthology.orchestra.run.smithi114.stdout:(155/157): openssl-3.2.2-6.el9.x86_64.rpm 15 MB/s | 1.4 MB 00:00 2024-09-21T03:26:41.796 INFO:teuthology.orchestra.run.smithi114.stdout:(156/157): librbd1-19.3.0-5120.g18432218.el9.x8 6.5 MB/s | 3.2 MB 00:00 2024-09-21T03:26:41.847 INFO:teuthology.orchestra.run.smithi114.stdout:(157/157): openssl-libs-3.2.2-6.el9.x86_64.rpm 16 MB/s | 2.4 MB 00:00 2024-09-21T03:26:41.853 INFO:teuthology.orchestra.run.smithi114.stdout:-------------------------------------------------------------------------------- 2024-09-21T03:26:41.853 INFO:teuthology.orchestra.run.smithi114.stdout:Total 23 MB/s | 228 MB 00:10 2024-09-21T03:26:42.148 INFO:teuthology.orchestra.run.smithi089.stdout:(151/157): NetworkManager-1.51.0-1.el9.x86_64.r 2.2 MB/s | 2.3 MB 00:01 2024-09-21T03:26:42.257 INFO:teuthology.orchestra.run.smithi089.stdout:(152/157): NetworkManager-libnm-1.51.0-1.el9.x8 1.7 MB/s | 1.8 MB 00:01 2024-09-21T03:26:42.291 INFO:teuthology.orchestra.run.smithi089.stdout:(153/157): NetworkManager-team-1.51.0-1.el9.x86 288 kB/s | 41 kB 00:00 2024-09-21T03:26:42.382 INFO:teuthology.orchestra.run.smithi089.stdout:(154/157): NetworkManager-tui-1.51.0-1.el9.x86_ 1.9 MB/s | 249 kB 00:00 2024-09-21T03:26:42.516 INFO:teuthology.orchestra.run.smithi089.stdout:(155/157): openssl-3.2.2-6.el9.x86_64.rpm 6.0 MB/s | 1.4 MB 00:00 2024-09-21T03:26:42.617 INFO:teuthology.orchestra.run.smithi089.stdout:(156/157): openssl-libs-3.2.2-6.el9.x86_64.rpm 10 MB/s | 2.4 MB 00:00 2024-09-21T03:26:43.095 INFO:teuthology.orchestra.run.smithi114.stdout:Running transaction check 2024-09-21T03:26:43.250 INFO:teuthology.orchestra.run.smithi114.stdout:Transaction check succeeded. 2024-09-21T03:26:43.250 INFO:teuthology.orchestra.run.smithi114.stdout:Running transaction test 2024-09-21T03:26:45.462 INFO:teuthology.orchestra.run.smithi114.stdout:Transaction test succeeded. 2024-09-21T03:26:45.462 INFO:teuthology.orchestra.run.smithi114.stdout:Running transaction 2024-09-21T03:26:46.384 INFO:teuthology.orchestra.run.smithi110.stdout: Preparing : 1/1 2024-09-21T03:26:46.508 INFO:teuthology.orchestra.run.smithi110.stdout: Upgrading : openssl-libs-1:3.2.2-6.el9.x86_64 1/165 2024-09-21T03:26:46.618 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/165 2024-09-21T03:26:46.759 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/165 2024-09-21T03:26:46.851 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/165 2024-09-21T03:26:46.876 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/165 2024-09-21T03:26:46.927 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/165 2024-09-21T03:26:47.267 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/165 2024-09-21T03:26:47.402 INFO:teuthology.orchestra.run.smithi110.stdout: Upgrading : librados2-2:19.3.0-5120.g18432218.el9.x86_64 8/165 2024-09-21T03:26:47.529 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: librados2-2:19.3.0-5120.g18432218.el9.x86_64 8/165 2024-09-21T03:26:47.553 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : libcephfs2-2:19.3.0-5120.g18432218.el9.x86_64 9/165 2024-09-21T03:26:47.645 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: libcephfs2-2:19.3.0-5120.g18432218.el9.x86_64 9/165 2024-09-21T03:26:47.675 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-rados-2:19.3.0-5120.g18432218.el9.x86_64 10/165 2024-09-21T03:26:47.691 INFO:teuthology.orchestra.run.smithi089.stdout:(157/157): python3-scipy-1.6.2-8.el9.x86_64.rpm 1.8 MB/s | 16 MB 00:08 2024-09-21T03:26:47.696 INFO:teuthology.orchestra.run.smithi089.stdout:-------------------------------------------------------------------------------- 2024-09-21T03:26:47.696 INFO:teuthology.orchestra.run.smithi089.stdout:Total 10 MB/s | 228 MB 00:21 2024-09-21T03:26:47.704 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : postgresql-private-libs-13.16-1.el9.x86_64 11/165 2024-09-21T03:26:47.871 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : libnbd-1.20.2-2.el9.x86_64 12/165 2024-09-21T03:26:47.931 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : postgresql-13.16-1.el9.x86_64 13/165 2024-09-21T03:26:47.959 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : libcephsqlite-2:19.3.0-5120.g18432218.el9.x86_64 14/165 2024-09-21T03:26:48.073 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: libcephsqlite-2:19.3.0-5120.g18432218.el9.x86_64 14/165 2024-09-21T03:26:48.101 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : libradosstriper1-2:19.3.0-5120.g18432218.el9.x86 15/165 2024-09-21T03:26:48.164 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: libradosstriper1-2:19.3.0-5120.g18432218.el9.x86 15/165 2024-09-21T03:26:48.186 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 16/165 2024-09-21T03:26:48.209 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 16/165 2024-09-21T03:26:48.396 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 17/165 2024-09-21T03:26:48.571 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 17/165 2024-09-21T03:26:48.701 INFO:teuthology.orchestra.run.smithi110.stdout: Upgrading : NetworkManager-libnm-1:1.51.0-1.el9.x86_64 18/165 2024-09-21T03:26:48.809 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: NetworkManager-1:1.51.0-1.el9.x86_64 19/165 2024-09-21T03:26:48.886 INFO:teuthology.orchestra.run.smithi110.stdout: Upgrading : NetworkManager-1:1.51.0-1.el9.x86_64 19/165 2024-09-21T03:26:48.896 INFO:teuthology.orchestra.run.smithi089.stdout:Running transaction check 2024-09-21T03:26:49.051 INFO:teuthology.orchestra.run.smithi089.stdout:Transaction check succeeded. 2024-09-21T03:26:49.051 INFO:teuthology.orchestra.run.smithi089.stdout:Running transaction test 2024-09-21T03:26:49.110 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: NetworkManager-1:1.51.0-1.el9.x86_64 19/165 2024-09-21T03:26:49.214 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : re2-1:20211101-20.el9.x86_64 20/165 2024-09-21T03:26:49.240 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 21/165 2024-09-21T03:26:49.299 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : liboath-2.6.7-2.el9.x86_64 22/165 2024-09-21T03:26:49.372 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 23/165 2024-09-21T03:26:49.417 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 24/165 2024-09-21T03:26:49.450 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-packaging-20.9-5.el9.noarch 25/165 2024-09-21T03:26:49.492 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 26/165 2024-09-21T03:26:49.526 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 27/165 2024-09-21T03:26:49.562 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : lua-5.4.4-4.el9.x86_64 28/165 2024-09-21T03:26:49.630 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 29/165 2024-09-21T03:26:49.683 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 30/165 2024-09-21T03:26:49.715 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 31/165 2024-09-21T03:26:49.759 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 32/165 2024-09-21T03:26:49.841 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 33/165 2024-09-21T03:26:49.878 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-ceph-common-2:19.3.0-5120.g18432218.el9. 34/165 2024-09-21T03:26:49.922 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-ceph-argparse-2:19.3.0-5120.g18432218.el 35/165 2024-09-21T03:26:49.951 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-cephfs-2:19.3.0-5120.g18432218.el9.x86_6 36/165 2024-09-21T03:26:50.020 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 37/165 2024-09-21T03:26:50.065 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : luarocks-3.9.2-5.el9.noarch 38/165 2024-09-21T03:26:50.139 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 39/165 2024-09-21T03:26:50.275 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 40/165 2024-09-21T03:26:50.325 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 41/165 2024-09-21T03:26:50.846 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-rsa-4.9-2.el9.noarch 42/165 2024-09-21T03:26:50.929 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 43/165 2024-09-21T03:26:50.959 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 44/165 2024-09-21T03:26:51.002 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 45/165 2024-09-21T03:26:51.040 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : librados-devel-2:19.3.0-5120.g18432218.el9.x86_6 46/165 2024-09-21T03:26:51.082 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 47/165 2024-09-21T03:26:51.243 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 48/165 2024-09-21T03:26:51.302 INFO:teuthology.orchestra.run.smithi110.stdout: Upgrading : openssl-1:3.2.2-6.el9.x86_64 49/165 2024-09-21T03:26:51.357 INFO:teuthology.orchestra.run.smithi089.stdout:Transaction test succeeded. 2024-09-21T03:26:51.358 INFO:teuthology.orchestra.run.smithi089.stdout:Running transaction 2024-09-21T03:26:51.391 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 50/165 2024-09-21T03:26:51.456 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 51/165 2024-09-21T03:26:51.522 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 52/165 2024-09-21T03:26:51.553 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 53/165 2024-09-21T03:26:51.590 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 54/165 2024-09-21T03:26:51.636 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 55/165 2024-09-21T03:26:51.672 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 56/165 2024-09-21T03:26:51.715 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 57/165 2024-09-21T03:26:51.720 INFO:teuthology.orchestra.run.smithi114.stdout: Preparing : 1/1 2024-09-21T03:26:51.757 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 58/165 2024-09-21T03:26:51.818 INFO:teuthology.orchestra.run.smithi114.stdout: Upgrading : openssl-libs-1:3.2.2-6.el9.x86_64 1/165 2024-09-21T03:26:51.878 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/165 2024-09-21T03:26:51.892 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 59/165 2024-09-21T03:26:52.027 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/165 2024-09-21T03:26:52.035 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 60/165 2024-09-21T03:26:52.076 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 61/165 2024-09-21T03:26:52.084 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/165 2024-09-21T03:26:52.110 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/165 2024-09-21T03:26:52.177 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 62/165 2024-09-21T03:26:52.183 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/165 2024-09-21T03:26:52.210 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 63/165 2024-09-21T03:26:52.252 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 64/165 2024-09-21T03:26:52.284 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : gperftools-libs-2.9.1-3.el9.x86_64 65/165 2024-09-21T03:26:52.308 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 66/165 2024-09-21T03:26:52.404 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 67/165 2024-09-21T03:26:52.533 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 68/165 2024-09-21T03:26:52.547 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/165 2024-09-21T03:26:52.570 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 69/165 2024-09-21T03:26:52.598 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 70/165 2024-09-21T03:26:52.632 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 71/165 2024-09-21T03:26:52.653 INFO:teuthology.orchestra.run.smithi114.stdout: Upgrading : librados2-2:19.3.0-5120.g18432218.el9.x86_64 8/165 2024-09-21T03:26:52.671 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : uuid-1.6.2-55.el9.x86_64 72/165 2024-09-21T03:26:52.703 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 73/165 2024-09-21T03:26:52.733 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 74/165 2024-09-21T03:26:52.767 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 75/165 2024-09-21T03:26:52.779 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: librados2-2:19.3.0-5120.g18432218.el9.x86_64 8/165 2024-09-21T03:26:52.801 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 76/165 2024-09-21T03:26:52.804 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : libcephfs2-2:19.3.0-5120.g18432218.el9.x86_64 9/165 2024-09-21T03:26:52.842 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 77/165 2024-09-21T03:26:52.876 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 78/165 2024-09-21T03:26:52.879 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: libcephfs2-2:19.3.0-5120.g18432218.el9.x86_64 9/165 2024-09-21T03:26:52.909 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-rados-2:19.3.0-5120.g18432218.el9.x86_64 10/165 2024-09-21T03:26:52.939 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : postgresql-private-libs-13.16-1.el9.x86_64 11/165 2024-09-21T03:26:52.962 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 79/165 2024-09-21T03:26:53.007 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 80/165 2024-09-21T03:26:53.047 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 81/165 2024-09-21T03:26:53.107 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : libnbd-1.20.2-2.el9.x86_64 12/165 2024-09-21T03:26:53.148 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 82/165 2024-09-21T03:26:53.164 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : postgresql-13.16-1.el9.x86_64 13/165 2024-09-21T03:26:53.192 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : libcephsqlite-2:19.3.0-5120.g18432218.el9.x86_64 14/165 2024-09-21T03:26:53.214 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 83/165 2024-09-21T03:26:53.301 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: libcephsqlite-2:19.3.0-5120.g18432218.el9.x86_64 14/165 2024-09-21T03:26:53.327 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : libradosstriper1-2:19.3.0-5120.g18432218.el9.x86 15/165 2024-09-21T03:26:53.373 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: libradosstriper1-2:19.3.0-5120.g18432218.el9.x86 15/165 2024-09-21T03:26:53.394 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 16/165 2024-09-21T03:26:53.420 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 16/165 2024-09-21T03:26:53.627 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 17/165 2024-09-21T03:26:53.829 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 17/165 2024-09-21T03:26:53.920 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 84/165 2024-09-21T03:26:53.960 INFO:teuthology.orchestra.run.smithi114.stdout: Upgrading : NetworkManager-libnm-1:1.51.0-1.el9.x86_64 18/165 2024-09-21T03:26:53.970 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 85/165 2024-09-21T03:26:54.004 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 86/165 2024-09-21T03:26:54.061 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: NetworkManager-1:1.51.0-1.el9.x86_64 19/165 2024-09-21T03:26:54.101 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 87/165 2024-09-21T03:26:54.128 INFO:teuthology.orchestra.run.smithi114.stdout: Upgrading : NetworkManager-1:1.51.0-1.el9.x86_64 19/165 2024-09-21T03:26:54.159 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 88/165 2024-09-21T03:26:54.203 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 89/165 2024-09-21T03:26:54.281 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 90/165 2024-09-21T03:26:54.315 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 91/165 2024-09-21T03:26:54.349 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: NetworkManager-1:1.51.0-1.el9.x86_64 19/165 2024-09-21T03:26:54.464 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : re2-1:20211101-20.el9.x86_64 20/165 2024-09-21T03:26:54.498 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : openblas-0.3.26-2.el9.x86_64 92/165 2024-09-21T03:26:54.499 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 21/165 2024-09-21T03:26:54.521 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 93/165 2024-09-21T03:26:54.533 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : liboath-2.6.7-2.el9.x86_64 22/165 2024-09-21T03:26:54.598 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 23/165 2024-09-21T03:26:54.603 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 94/165 2024-09-21T03:26:54.643 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 24/165 2024-09-21T03:26:54.675 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-packaging-20.9-5.el9.noarch 25/165 2024-09-21T03:26:54.718 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 26/165 2024-09-21T03:26:54.751 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 27/165 2024-09-21T03:26:54.778 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : lua-5.4.4-4.el9.x86_64 28/165 2024-09-21T03:26:54.847 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 29/165 2024-09-21T03:26:54.900 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 30/165 2024-09-21T03:26:54.931 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 31/165 2024-09-21T03:26:54.975 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 32/165 2024-09-21T03:26:55.056 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 33/165 2024-09-21T03:26:55.085 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-ceph-common-2:19.3.0-5120.g18432218.el9. 34/165 2024-09-21T03:26:55.121 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-ceph-argparse-2:19.3.0-5120.g18432218.el 35/165 2024-09-21T03:26:55.149 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-cephfs-2:19.3.0-5120.g18432218.el9.x86_6 36/165 2024-09-21T03:26:55.217 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 37/165 2024-09-21T03:26:55.237 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 95/165 2024-09-21T03:26:55.264 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : luarocks-3.9.2-5.el9.noarch 38/165 2024-09-21T03:26:55.338 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 39/165 2024-09-21T03:26:55.357 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 96/165 2024-09-21T03:26:55.472 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 40/165 2024-09-21T03:26:55.524 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 41/165 2024-09-21T03:26:56.050 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-rsa-4.9-2.el9.noarch 42/165 2024-09-21T03:26:56.145 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 43/165 2024-09-21T03:26:56.174 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 44/165 2024-09-21T03:26:56.217 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 45/165 2024-09-21T03:26:56.255 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : librados-devel-2:19.3.0-5120.g18432218.el9.x86_6 46/165 2024-09-21T03:26:56.298 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 47/165 2024-09-21T03:26:56.460 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 48/165 2024-09-21T03:26:56.541 INFO:teuthology.orchestra.run.smithi114.stdout: Upgrading : openssl-1:3.2.2-6.el9.x86_64 49/165 2024-09-21T03:26:56.601 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 97/165 2024-09-21T03:26:56.632 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 50/165 2024-09-21T03:26:56.697 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 51/165 2024-09-21T03:26:56.698 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 98/165 2024-09-21T03:26:56.726 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 99/165 2024-09-21T03:26:56.760 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : libuuid-devel-2.37.4-20.el9.x86_64 100/165 2024-09-21T03:26:56.804 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 52/165 2024-09-21T03:26:56.804 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 53/165 2024-09-21T03:26:56.837 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 54/165 2024-09-21T03:26:56.887 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 55/165 2024-09-21T03:26:56.921 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 56/165 2024-09-21T03:26:56.964 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 57/165 2024-09-21T03:26:57.005 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 58/165 2024-09-21T03:26:57.085 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 101/165 2024-09-21T03:26:57.123 INFO:teuthology.orchestra.run.smithi110.stdout: Upgrading : librbd1-2:19.3.0-5120.g18432218.el9.x86_64 102/165 2024-09-21T03:26:57.138 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 59/165 2024-09-21T03:26:57.174 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 60/165 2024-09-21T03:26:57.214 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: librbd1-2:19.3.0-5120.g18432218.el9.x86_64 102/165 2024-09-21T03:26:57.216 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 61/165 2024-09-21T03:26:57.245 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-rbd-2:19.3.0-5120.g18432218.el9.x86_64 103/165 2024-09-21T03:26:57.280 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 104/165 2024-09-21T03:26:57.316 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 62/165 2024-09-21T03:26:57.349 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 63/165 2024-09-21T03:26:57.389 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 64/165 2024-09-21T03:26:57.470 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : gperftools-libs-2.9.1-3.el9.x86_64 65/165 2024-09-21T03:26:57.656 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 66/165 2024-09-21T03:26:57.778 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 105/165 2024-09-21T03:26:57.794 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 67/165 2024-09-21T03:26:57.801 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : librgw2-2:19.3.0-5120.g18432218.el9.x86_64 106/165 2024-09-21T03:26:57.863 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: librgw2-2:19.3.0-5120.g18432218.el9.x86_64 106/165 2024-09-21T03:26:57.886 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-rgw-2:19.3.0-5120.g18432218.el9.x86_64 107/165 2024-09-21T03:26:57.922 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 68/165 2024-09-21T03:26:57.963 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 69/165 2024-09-21T03:26:57.991 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 70/165 2024-09-21T03:26:58.025 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 71/165 2024-09-21T03:26:58.064 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : uuid-1.6.2-55.el9.x86_64 72/165 2024-09-21T03:26:58.094 INFO:teuthology.orchestra.run.smithi089.stdout: Preparing : 1/1 2024-09-21T03:26:58.095 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 73/165 2024-09-21T03:26:58.129 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 74/165 2024-09-21T03:26:58.164 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 75/165 2024-09-21T03:26:58.197 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 76/165 2024-09-21T03:26:58.213 INFO:teuthology.orchestra.run.smithi089.stdout: Upgrading : openssl-libs-1:3.2.2-6.el9.x86_64 1/165 2024-09-21T03:26:58.238 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 77/165 2024-09-21T03:26:58.251 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/165 2024-09-21T03:26:58.273 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 78/165 2024-09-21T03:26:58.304 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/165 2024-09-21T03:26:58.346 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/165 2024-09-21T03:26:58.359 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 79/165 2024-09-21T03:26:58.371 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/165 2024-09-21T03:26:58.404 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 80/165 2024-09-21T03:26:58.425 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/165 2024-09-21T03:26:58.434 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 81/165 2024-09-21T03:26:58.537 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 82/165 2024-09-21T03:26:58.585 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 83/165 2024-09-21T03:26:58.779 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/165 2024-09-21T03:26:58.896 INFO:teuthology.orchestra.run.smithi089.stdout: Upgrading : librados2-2:19.3.0-5120.g18432218.el9.x86_64 8/165 2024-09-21T03:26:59.023 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: librados2-2:19.3.0-5120.g18432218.el9.x86_64 8/165 2024-09-21T03:26:59.047 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : libcephfs2-2:19.3.0-5120.g18432218.el9.x86_64 9/165 2024-09-21T03:26:59.138 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: libcephfs2-2:19.3.0-5120.g18432218.el9.x86_64 9/165 2024-09-21T03:26:59.169 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-rados-2:19.3.0-5120.g18432218.el9.x86_64 10/165 2024-09-21T03:26:59.199 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : postgresql-private-libs-13.16-1.el9.x86_64 11/165 2024-09-21T03:26:59.307 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 84/165 2024-09-21T03:26:59.358 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 85/165 2024-09-21T03:26:59.378 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : libnbd-1.20.2-2.el9.x86_64 12/165 2024-09-21T03:26:59.391 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 86/165 2024-09-21T03:26:59.433 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : postgresql-13.16-1.el9.x86_64 13/165 2024-09-21T03:26:59.461 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : libcephsqlite-2:19.3.0-5120.g18432218.el9.x86_64 14/165 2024-09-21T03:26:59.489 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 87/165 2024-09-21T03:26:59.547 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 88/165 2024-09-21T03:26:59.553 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: libcephsqlite-2:19.3.0-5120.g18432218.el9.x86_64 14/165 2024-09-21T03:26:59.579 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : libradosstriper1-2:19.3.0-5120.g18432218.el9.x86 15/165 2024-09-21T03:26:59.591 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 89/165 2024-09-21T03:26:59.635 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: libradosstriper1-2:19.3.0-5120.g18432218.el9.x86 15/165 2024-09-21T03:26:59.672 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 16/165 2024-09-21T03:26:59.678 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 90/165 2024-09-21T03:26:59.694 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 16/165 2024-09-21T03:26:59.719 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 91/165 2024-09-21T03:26:59.889 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 17/165 2024-09-21T03:26:59.902 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : openblas-0.3.26-2.el9.x86_64 92/165 2024-09-21T03:26:59.933 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 93/165 2024-09-21T03:27:00.013 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 94/165 2024-09-21T03:27:00.064 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 17/165 2024-09-21T03:27:00.188 INFO:teuthology.orchestra.run.smithi089.stdout: Upgrading : NetworkManager-libnm-1:1.51.0-1.el9.x86_64 18/165 2024-09-21T03:27:00.313 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: NetworkManager-1:1.51.0-1.el9.x86_64 19/165 2024-09-21T03:27:00.381 INFO:teuthology.orchestra.run.smithi089.stdout: Upgrading : NetworkManager-1:1.51.0-1.el9.x86_64 19/165 2024-09-21T03:27:00.430 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: ceph-common-2:19.3.0-5120.g18432218.el9.x86_64 108/165 2024-09-21T03:27:00.466 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : ceph-common-2:19.3.0-5120.g18432218.el9.x86_64 108/165 2024-09-21T03:27:00.610 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: NetworkManager-1:1.51.0-1.el9.x86_64 19/165 2024-09-21T03:27:00.676 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 95/165 2024-09-21T03:27:00.722 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : re2-1:20211101-20.el9.x86_64 20/165 2024-09-21T03:27:00.758 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 21/165 2024-09-21T03:27:00.790 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 96/165 2024-09-21T03:27:00.795 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : liboath-2.6.7-2.el9.x86_64 22/165 2024-09-21T03:27:00.859 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 23/165 2024-09-21T03:27:00.913 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 24/165 2024-09-21T03:27:00.945 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-packaging-20.9-5.el9.noarch 25/165 2024-09-21T03:27:00.989 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 26/165 2024-09-21T03:27:01.021 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 27/165 2024-09-21T03:27:01.045 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: ceph-common-2:19.3.0-5120.g18432218.el9.x86_64 108/165 2024-09-21T03:27:01.049 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : lua-5.4.4-4.el9.x86_64 28/165 2024-09-21T03:27:01.078 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : ceph-base-2:19.3.0-5120.g18432218.el9.x86_64 109/165 2024-09-21T03:27:01.139 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 29/165 2024-09-21T03:27:01.172 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: ceph-base-2:19.3.0-5120.g18432218.el9.x86_64 109/165 2024-09-21T03:27:01.172 INFO:teuthology.orchestra.run.smithi110.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-21T03:27:01.172 INFO:teuthology.orchestra.run.smithi110.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-21T03:27:01.172 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-21T03:27:01.204 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : ceph-selinux-2:19.3.0-5120.g18432218.el9.x86_64 110/165 2024-09-21T03:27:01.204 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 30/165 2024-09-21T03:27:01.236 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 31/165 2024-09-21T03:27:01.279 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 32/165 2024-09-21T03:27:01.359 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 33/165 2024-09-21T03:27:01.390 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-ceph-common-2:19.3.0-5120.g18432218.el9. 34/165 2024-09-21T03:27:01.434 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-ceph-argparse-2:19.3.0-5120.g18432218.el 35/165 2024-09-21T03:27:01.462 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-cephfs-2:19.3.0-5120.g18432218.el9.x86_6 36/165 2024-09-21T03:27:01.530 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 37/165 2024-09-21T03:27:01.577 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : luarocks-3.9.2-5.el9.noarch 38/165 2024-09-21T03:27:01.650 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 39/165 2024-09-21T03:27:01.782 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 40/165 2024-09-21T03:27:01.829 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 41/165 2024-09-21T03:27:02.056 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 97/165 2024-09-21T03:27:02.138 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 98/165 2024-09-21T03:27:02.165 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 99/165 2024-09-21T03:27:02.199 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : libuuid-devel-2.37.4-20.el9.x86_64 100/165 2024-09-21T03:27:02.351 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-rsa-4.9-2.el9.noarch 42/165 2024-09-21T03:27:02.441 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 43/165 2024-09-21T03:27:02.471 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 44/165 2024-09-21T03:27:02.506 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 45/165 2024-09-21T03:27:02.527 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 101/165 2024-09-21T03:27:02.535 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : librados-devel-2:19.3.0-5120.g18432218.el9.x86_6 46/165 2024-09-21T03:27:02.570 INFO:teuthology.orchestra.run.smithi114.stdout: Upgrading : librbd1-2:19.3.0-5120.g18432218.el9.x86_64 102/165 2024-09-21T03:27:02.587 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 47/165 2024-09-21T03:27:02.660 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: librbd1-2:19.3.0-5120.g18432218.el9.x86_64 102/165 2024-09-21T03:27:02.691 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-rbd-2:19.3.0-5120.g18432218.el9.x86_64 103/165 2024-09-21T03:27:02.727 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 104/165 2024-09-21T03:27:02.749 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 48/165 2024-09-21T03:27:02.805 INFO:teuthology.orchestra.run.smithi089.stdout: Upgrading : openssl-1:3.2.2-6.el9.x86_64 49/165 2024-09-21T03:27:02.896 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 50/165 2024-09-21T03:27:02.961 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 51/165 2024-09-21T03:27:03.035 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 52/165 2024-09-21T03:27:03.066 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 53/165 2024-09-21T03:27:03.102 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 54/165 2024-09-21T03:27:03.149 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 55/165 2024-09-21T03:27:03.185 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 56/165 2024-09-21T03:27:03.227 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 105/165 2024-09-21T03:27:03.228 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 57/165 2024-09-21T03:27:03.256 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : librgw2-2:19.3.0-5120.g18432218.el9.x86_64 106/165 2024-09-21T03:27:03.270 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 58/165 2024-09-21T03:27:03.310 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: librgw2-2:19.3.0-5120.g18432218.el9.x86_64 106/165 2024-09-21T03:27:03.341 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-rgw-2:19.3.0-5120.g18432218.el9.x86_64 107/165 2024-09-21T03:27:03.406 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 59/165 2024-09-21T03:27:03.439 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 60/165 2024-09-21T03:27:03.480 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 61/165 2024-09-21T03:27:03.582 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 62/165 2024-09-21T03:27:03.623 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 63/165 2024-09-21T03:27:03.774 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 64/165 2024-09-21T03:27:03.980 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : gperftools-libs-2.9.1-3.el9.x86_64 65/165 2024-09-21T03:27:04.005 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 66/165 2024-09-21T03:27:04.113 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 67/165 2024-09-21T03:27:04.239 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 68/165 2024-09-21T03:27:04.271 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 69/165 2024-09-21T03:27:04.299 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 70/165 2024-09-21T03:27:04.333 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 71/165 2024-09-21T03:27:04.372 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : uuid-1.6.2-55.el9.x86_64 72/165 2024-09-21T03:27:04.404 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 73/165 2024-09-21T03:27:04.433 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 74/165 2024-09-21T03:27:04.468 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 75/165 2024-09-21T03:27:04.502 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 76/165 2024-09-21T03:27:04.543 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 77/165 2024-09-21T03:27:04.578 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 78/165 2024-09-21T03:27:04.664 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 79/165 2024-09-21T03:27:04.711 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 80/165 2024-09-21T03:27:04.743 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 81/165 2024-09-21T03:27:04.846 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 82/165 2024-09-21T03:27:04.893 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 83/165 2024-09-21T03:27:05.604 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 84/165 2024-09-21T03:27:05.651 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 85/165 2024-09-21T03:27:05.700 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 86/165 2024-09-21T03:27:05.798 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 87/165 2024-09-21T03:27:05.864 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 88/165 2024-09-21T03:27:05.909 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 89/165 2024-09-21T03:27:05.950 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: ceph-common-2:19.3.0-5120.g18432218.el9.x86_64 108/165 2024-09-21T03:27:05.987 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : ceph-common-2:19.3.0-5120.g18432218.el9.x86_64 108/165 2024-09-21T03:27:05.995 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 90/165 2024-09-21T03:27:06.038 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 91/165 2024-09-21T03:27:06.223 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : openblas-0.3.26-2.el9.x86_64 92/165 2024-09-21T03:27:06.252 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 93/165 2024-09-21T03:27:06.330 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 94/165 2024-09-21T03:27:06.566 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: ceph-common-2:19.3.0-5120.g18432218.el9.x86_64 108/165 2024-09-21T03:27:06.599 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : ceph-base-2:19.3.0-5120.g18432218.el9.x86_64 109/165 2024-09-21T03:27:06.690 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: ceph-base-2:19.3.0-5120.g18432218.el9.x86_64 109/165 2024-09-21T03:27:06.690 INFO:teuthology.orchestra.run.smithi114.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-21T03:27:06.690 INFO:teuthology.orchestra.run.smithi114.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-21T03:27:06.690 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-21T03:27:06.734 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : ceph-selinux-2:19.3.0-5120.g18432218.el9.x86_64 110/165 2024-09-21T03:27:06.979 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 95/165 2024-09-21T03:27:07.091 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 96/165 2024-09-21T03:27:08.327 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 97/165 2024-09-21T03:27:08.415 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 98/165 2024-09-21T03:27:08.434 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 99/165 2024-09-21T03:27:08.459 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : libuuid-devel-2.37.4-20.el9.x86_64 100/165 2024-09-21T03:27:08.784 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 101/165 2024-09-21T03:27:08.839 INFO:teuthology.orchestra.run.smithi089.stdout: Upgrading : librbd1-2:19.3.0-5120.g18432218.el9.x86_64 102/165 2024-09-21T03:27:08.929 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: librbd1-2:19.3.0-5120.g18432218.el9.x86_64 102/165 2024-09-21T03:27:08.960 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-rbd-2:19.3.0-5120.g18432218.el9.x86_64 103/165 2024-09-21T03:27:08.996 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 104/165 2024-09-21T03:27:09.498 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 105/165 2024-09-21T03:27:09.525 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : librgw2-2:19.3.0-5120.g18432218.el9.x86_64 106/165 2024-09-21T03:27:09.590 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: librgw2-2:19.3.0-5120.g18432218.el9.x86_64 106/165 2024-09-21T03:27:09.621 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-rgw-2:19.3.0-5120.g18432218.el9.x86_64 107/165 2024-09-21T03:27:11.866 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: ceph-common-2:19.3.0-5120.g18432218.el9.x86_64 108/165 2024-09-21T03:27:11.909 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : ceph-common-2:19.3.0-5120.g18432218.el9.x86_64 108/165 2024-09-21T03:27:12.491 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: ceph-common-2:19.3.0-5120.g18432218.el9.x86_64 108/165 2024-09-21T03:27:12.546 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : ceph-base-2:19.3.0-5120.g18432218.el9.x86_64 109/165 2024-09-21T03:27:12.638 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: ceph-base-2:19.3.0-5120.g18432218.el9.x86_64 109/165 2024-09-21T03:27:12.638 INFO:teuthology.orchestra.run.smithi089.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-21T03:27:12.638 INFO:teuthology.orchestra.run.smithi089.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-21T03:27:12.639 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-09-21T03:27:12.681 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : ceph-selinux-2:19.3.0-5120.g18432218.el9.x86_64 110/165 2024-09-21T03:27:12.987 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: ceph-selinux-2:19.3.0-5120.g18432218.el9.x86_64 110/165 2024-09-21T03:27:12.987 INFO:teuthology.orchestra.run.smithi110.stdout:skipping the directory /sys 2024-09-21T03:27:12.987 INFO:teuthology.orchestra.run.smithi110.stdout:skipping the directory /proc 2024-09-21T03:27:12.988 INFO:teuthology.orchestra.run.smithi110.stdout:skipping the directory /mnt 2024-09-21T03:27:12.988 INFO:teuthology.orchestra.run.smithi110.stdout:skipping the directory /var/tmp 2024-09-21T03:27:12.988 INFO:teuthology.orchestra.run.smithi110.stdout:skipping the directory /home 2024-09-21T03:27:12.988 INFO:teuthology.orchestra.run.smithi110.stdout:skipping the directory /root 2024-09-21T03:27:12.988 INFO:teuthology.orchestra.run.smithi110.stdout:skipping the directory /tmp 2024-09-21T03:27:12.988 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-21T03:27:13.152 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : ceph-volume-2:19.3.0-5120.g18432218.el9.noarch 111/165 2024-09-21T03:27:13.168 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: ceph-volume-2:19.3.0-5120.g18432218.el9.noarch 111/165 2024-09-21T03:27:13.168 INFO:teuthology.orchestra.run.smithi110.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T03:27:13.168 INFO:teuthology.orchestra.run.smithi110.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-21T03:27:13.168 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-21T03:27:14.738 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : ceph-osd-2:19.3.0-5120.g18432218.el9.x86_64 112/165 2024-09-21T03:27:14.765 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: ceph-osd-2:19.3.0-5120.g18432218.el9.x86_64 112/165 2024-09-21T03:27:14.765 INFO:teuthology.orchestra.run.smithi110.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T03:27:14.765 INFO:teuthology.orchestra.run.smithi110.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-21T03:27:14.765 INFO:teuthology.orchestra.run.smithi110.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-21T03:27:14.765 INFO:teuthology.orchestra.run.smithi110.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-21T03:27:14.765 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-21T03:27:14.989 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : ceph-mds-2:19.3.0-5120.g18432218.el9.x86_64 113/165 2024-09-21T03:27:15.012 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: ceph-mds-2:19.3.0-5120.g18432218.el9.x86_64 113/165 2024-09-21T03:27:15.012 INFO:teuthology.orchestra.run.smithi110.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T03:27:15.012 INFO:teuthology.orchestra.run.smithi110.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-21T03:27:15.012 INFO:teuthology.orchestra.run.smithi110.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-21T03:27:15.012 INFO:teuthology.orchestra.run.smithi110.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-21T03:27:15.012 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-21T03:27:15.474 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : ceph-mon-2:19.3.0-5120.g18432218.el9.x86_64 114/165 2024-09-21T03:27:15.496 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: ceph-mon-2:19.3.0-5120.g18432218.el9.x86_64 114/165 2024-09-21T03:27:15.496 INFO:teuthology.orchestra.run.smithi110.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T03:27:15.496 INFO:teuthology.orchestra.run.smithi110.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-21T03:27:15.496 INFO:teuthology.orchestra.run.smithi110.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-21T03:27:15.496 INFO:teuthology.orchestra.run.smithi110.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-21T03:27:15.496 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-21T03:27:15.603 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : automake-1.16.2-8.el9.noarch 115/165 2024-09-21T03:27:15.660 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-ply-3.11-14.el9.noarch 116/165 2024-09-21T03:27:15.724 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 117/165 2024-09-21T03:27:15.908 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 118/165 2024-09-21T03:27:15.957 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 119/165 2024-09-21T03:27:16.035 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 120/165 2024-09-21T03:27:16.132 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 121/165 2024-09-21T03:27:16.278 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 122/165 2024-09-21T03:27:16.316 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 123/165 2024-09-21T03:27:16.348 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 124/165 2024-09-21T03:27:16.376 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : fuse-2.9.9-16.el9.x86_64 125/165 2024-09-21T03:27:16.818 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: cephadm-2:19.3.0-5120.g18432218.el9.noarch 126/165 2024-09-21T03:27:16.850 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : cephadm-2:19.3.0-5120.g18432218.el9.noarch 126/165 2024-09-21T03:27:16.882 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : ceph-prometheus-alerts-2:19.3.0-5120.g18432218.e 127/165 2024-09-21T03:27:16.947 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : ceph-grafana-dashboards-2:19.3.0-5120.g18432218. 128/165 2024-09-21T03:27:16.973 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : ceph-mgr-cephadm-2:19.3.0-5120.g18432218.el9.noa 129/165 2024-09-21T03:27:18.550 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: ceph-selinux-2:19.3.0-5120.g18432218.el9.x86_64 110/165 2024-09-21T03:27:18.550 INFO:teuthology.orchestra.run.smithi114.stdout:skipping the directory /sys 2024-09-21T03:27:18.550 INFO:teuthology.orchestra.run.smithi114.stdout:skipping the directory /proc 2024-09-21T03:27:18.550 INFO:teuthology.orchestra.run.smithi114.stdout:skipping the directory /mnt 2024-09-21T03:27:18.550 INFO:teuthology.orchestra.run.smithi114.stdout:skipping the directory /var/tmp 2024-09-21T03:27:18.550 INFO:teuthology.orchestra.run.smithi114.stdout:skipping the directory /home 2024-09-21T03:27:18.550 INFO:teuthology.orchestra.run.smithi114.stdout:skipping the directory /root 2024-09-21T03:27:18.550 INFO:teuthology.orchestra.run.smithi114.stdout:skipping the directory /tmp 2024-09-21T03:27:18.550 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-21T03:27:18.720 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : ceph-volume-2:19.3.0-5120.g18432218.el9.noarch 111/165 2024-09-21T03:27:18.737 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: ceph-volume-2:19.3.0-5120.g18432218.el9.noarch 111/165 2024-09-21T03:27:18.737 INFO:teuthology.orchestra.run.smithi114.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T03:27:18.737 INFO:teuthology.orchestra.run.smithi114.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-21T03:27:18.737 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-21T03:27:20.129 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.3.0-5120.g18432218.el9.noa 129/165 2024-09-21T03:27:20.187 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : ceph-mgr-dashboard-2:19.3.0-5120.g18432218.el9.n 130/165 2024-09-21T03:27:20.340 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : ceph-osd-2:19.3.0-5120.g18432218.el9.x86_64 112/165 2024-09-21T03:27:20.366 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: ceph-osd-2:19.3.0-5120.g18432218.el9.x86_64 112/165 2024-09-21T03:27:20.367 INFO:teuthology.orchestra.run.smithi114.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T03:27:20.367 INFO:teuthology.orchestra.run.smithi114.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-21T03:27:20.367 INFO:teuthology.orchestra.run.smithi114.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-21T03:27:20.367 INFO:teuthology.orchestra.run.smithi114.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-21T03:27:20.367 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-21T03:27:20.591 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : ceph-mds-2:19.3.0-5120.g18432218.el9.x86_64 113/165 2024-09-21T03:27:20.614 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: ceph-mds-2:19.3.0-5120.g18432218.el9.x86_64 113/165 2024-09-21T03:27:20.614 INFO:teuthology.orchestra.run.smithi114.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T03:27:20.614 INFO:teuthology.orchestra.run.smithi114.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-21T03:27:20.614 INFO:teuthology.orchestra.run.smithi114.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-21T03:27:20.614 INFO:teuthology.orchestra.run.smithi114.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-21T03:27:20.614 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-21T03:27:21.091 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : ceph-mon-2:19.3.0-5120.g18432218.el9.x86_64 114/165 2024-09-21T03:27:21.115 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: ceph-mon-2:19.3.0-5120.g18432218.el9.x86_64 114/165 2024-09-21T03:27:21.115 INFO:teuthology.orchestra.run.smithi114.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T03:27:21.115 INFO:teuthology.orchestra.run.smithi114.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-21T03:27:21.115 INFO:teuthology.orchestra.run.smithi114.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-21T03:27:21.115 INFO:teuthology.orchestra.run.smithi114.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-21T03:27:21.115 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-21T03:27:21.143 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.3.0-5120.g18432218.el9.n 130/165 2024-09-21T03:27:21.179 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : ceph-mgr-diskprediction-local-2:19.3.0-5120.g184 131/165 2024-09-21T03:27:21.221 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : automake-1.16.2-8.el9.noarch 115/165 2024-09-21T03:27:21.269 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-ply-3.11-14.el9.noarch 116/165 2024-09-21T03:27:21.293 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.3.0-5120.g184 131/165 2024-09-21T03:27:21.326 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 117/165 2024-09-21T03:27:21.463 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : ceph-mgr-modules-core-2:19.3.0-5120.g18432218.el 132/165 2024-09-21T03:27:21.490 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : ceph-mgr-2:19.3.0-5120.g18432218.el9.x86_64 133/165 2024-09-21T03:27:21.507 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 118/165 2024-09-21T03:27:21.513 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: ceph-mgr-2:19.3.0-5120.g18432218.el9.x86_64 133/165 2024-09-21T03:27:21.513 INFO:teuthology.orchestra.run.smithi110.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T03:27:21.513 INFO:teuthology.orchestra.run.smithi110.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-21T03:27:21.513 INFO:teuthology.orchestra.run.smithi110.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-21T03:27:21.513 INFO:teuthology.orchestra.run.smithi110.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-21T03:27:21.513 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-21T03:27:21.550 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 119/165 2024-09-21T03:27:21.557 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : ceph-mgr-rook-2:19.3.0-5120.g18432218.el9.noarch 134/165 2024-09-21T03:27:21.568 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: ceph-mgr-rook-2:19.3.0-5120.g18432218.el9.noarch 134/165 2024-09-21T03:27:21.627 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 120/165 2024-09-21T03:27:21.667 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : ceph-2:19.3.0-5120.g18432218.el9.x86_64 135/165 2024-09-21T03:27:21.717 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 121/165 2024-09-21T03:27:21.725 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : ceph-fuse-2:19.3.0-5120.g18432218.el9.x86_64 136/165 2024-09-21T03:27:21.872 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 122/165 2024-09-21T03:27:21.918 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 123/165 2024-09-21T03:27:21.950 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 124/165 2024-09-21T03:27:21.978 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : fuse-2.9.9-16.el9.x86_64 125/165 2024-09-21T03:27:22.404 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: cephadm-2:19.3.0-5120.g18432218.el9.noarch 126/165 2024-09-21T03:27:22.435 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : cephadm-2:19.3.0-5120.g18432218.el9.noarch 126/165 2024-09-21T03:27:22.467 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : ceph-prometheus-alerts-2:19.3.0-5120.g18432218.e 127/165 2024-09-21T03:27:22.532 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : ceph-grafana-dashboards-2:19.3.0-5120.g18432218. 128/165 2024-09-21T03:27:22.558 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : ceph-mgr-cephadm-2:19.3.0-5120.g18432218.el9.noa 129/165 2024-09-21T03:27:22.684 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : libtool-2.4.6-46.el9.x86_64 137/165 2024-09-21T03:27:22.718 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : ceph-radosgw-2:19.3.0-5120.g18432218.el9.x86_64 138/165 2024-09-21T03:27:22.741 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: ceph-radosgw-2:19.3.0-5120.g18432218.el9.x86_64 138/165 2024-09-21T03:27:22.741 INFO:teuthology.orchestra.run.smithi110.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T03:27:22.741 INFO:teuthology.orchestra.run.smithi110.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-21T03:27:22.741 INFO:teuthology.orchestra.run.smithi110.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-21T03:27:22.741 INFO:teuthology.orchestra.run.smithi110.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-21T03:27:22.741 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-21T03:27:22.786 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : ceph-immutable-object-cache-2:19.3.0-5120.g18432 139/165 2024-09-21T03:27:22.808 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.3.0-5120.g18432 139/165 2024-09-21T03:27:22.808 INFO:teuthology.orchestra.run.smithi110.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T03:27:22.808 INFO:teuthology.orchestra.run.smithi110.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-21T03:27:22.808 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-21T03:27:22.854 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : cephfs-mirror-2:19.3.0-5120.g18432218.el9.x86_64 140/165 2024-09-21T03:27:22.875 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: cephfs-mirror-2:19.3.0-5120.g18432218.el9.x86_64 140/165 2024-09-21T03:27:22.875 INFO:teuthology.orchestra.run.smithi110.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T03:27:22.875 INFO:teuthology.orchestra.run.smithi110.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-09-21T03:27:22.875 INFO:teuthology.orchestra.run.smithi110.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-21T03:27:22.875 INFO:teuthology.orchestra.run.smithi110.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-21T03:27:22.876 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-21T03:27:23.203 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : rbd-mirror-2:19.3.0-5120.g18432218.el9.x86_64 141/165 2024-09-21T03:27:23.226 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: rbd-mirror-2:19.3.0-5120.g18432218.el9.x86_64 141/165 2024-09-21T03:27:23.227 INFO:teuthology.orchestra.run.smithi110.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T03:27:23.227 INFO:teuthology.orchestra.run.smithi110.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-21T03:27:23.227 INFO:teuthology.orchestra.run.smithi110.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-21T03:27:23.227 INFO:teuthology.orchestra.run.smithi110.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-21T03:27:23.227 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-21T03:27:24.458 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: ceph-selinux-2:19.3.0-5120.g18432218.el9.x86_64 110/165 2024-09-21T03:27:24.458 INFO:teuthology.orchestra.run.smithi089.stdout:skipping the directory /sys 2024-09-21T03:27:24.458 INFO:teuthology.orchestra.run.smithi089.stdout:skipping the directory /proc 2024-09-21T03:27:24.458 INFO:teuthology.orchestra.run.smithi089.stdout:skipping the directory /mnt 2024-09-21T03:27:24.458 INFO:teuthology.orchestra.run.smithi089.stdout:skipping the directory /var/tmp 2024-09-21T03:27:24.458 INFO:teuthology.orchestra.run.smithi089.stdout:skipping the directory /home 2024-09-21T03:27:24.459 INFO:teuthology.orchestra.run.smithi089.stdout:skipping the directory /root 2024-09-21T03:27:24.459 INFO:teuthology.orchestra.run.smithi089.stdout:skipping the directory /tmp 2024-09-21T03:27:24.459 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-09-21T03:27:24.631 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : ceph-volume-2:19.3.0-5120.g18432218.el9.noarch 111/165 2024-09-21T03:27:24.647 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: ceph-volume-2:19.3.0-5120.g18432218.el9.noarch 111/165 2024-09-21T03:27:24.647 INFO:teuthology.orchestra.run.smithi089.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T03:27:24.647 INFO:teuthology.orchestra.run.smithi089.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-21T03:27:24.647 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-09-21T03:27:25.714 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.3.0-5120.g18432218.el9.noa 129/165 2024-09-21T03:27:25.773 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : ceph-mgr-dashboard-2:19.3.0-5120.g18432218.el9.n 130/165 2024-09-21T03:27:26.251 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : ceph-osd-2:19.3.0-5120.g18432218.el9.x86_64 112/165 2024-09-21T03:27:26.277 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: ceph-osd-2:19.3.0-5120.g18432218.el9.x86_64 112/165 2024-09-21T03:27:26.277 INFO:teuthology.orchestra.run.smithi089.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T03:27:26.277 INFO:teuthology.orchestra.run.smithi089.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-21T03:27:26.277 INFO:teuthology.orchestra.run.smithi089.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-21T03:27:26.278 INFO:teuthology.orchestra.run.smithi089.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-21T03:27:26.278 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-09-21T03:27:26.510 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : ceph-mds-2:19.3.0-5120.g18432218.el9.x86_64 113/165 2024-09-21T03:27:26.533 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: ceph-mds-2:19.3.0-5120.g18432218.el9.x86_64 113/165 2024-09-21T03:27:26.533 INFO:teuthology.orchestra.run.smithi089.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T03:27:26.533 INFO:teuthology.orchestra.run.smithi089.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-21T03:27:26.533 INFO:teuthology.orchestra.run.smithi089.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-21T03:27:26.534 INFO:teuthology.orchestra.run.smithi089.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-21T03:27:26.534 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-09-21T03:27:26.733 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.3.0-5120.g18432218.el9.n 130/165 2024-09-21T03:27:26.764 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : ceph-mgr-diskprediction-local-2:19.3.0-5120.g184 131/165 2024-09-21T03:27:26.878 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.3.0-5120.g184 131/165 2024-09-21T03:27:27.011 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : ceph-mon-2:19.3.0-5120.g18432218.el9.x86_64 114/165 2024-09-21T03:27:27.034 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: ceph-mon-2:19.3.0-5120.g18432218.el9.x86_64 114/165 2024-09-21T03:27:27.034 INFO:teuthology.orchestra.run.smithi089.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T03:27:27.034 INFO:teuthology.orchestra.run.smithi089.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-21T03:27:27.034 INFO:teuthology.orchestra.run.smithi089.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-21T03:27:27.035 INFO:teuthology.orchestra.run.smithi089.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-21T03:27:27.035 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-09-21T03:27:27.057 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : ceph-mgr-modules-core-2:19.3.0-5120.g18432218.el 132/165 2024-09-21T03:27:27.083 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : ceph-mgr-2:19.3.0-5120.g18432218.el9.x86_64 133/165 2024-09-21T03:27:27.107 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: ceph-mgr-2:19.3.0-5120.g18432218.el9.x86_64 133/165 2024-09-21T03:27:27.107 INFO:teuthology.orchestra.run.smithi114.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T03:27:27.108 INFO:teuthology.orchestra.run.smithi114.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-21T03:27:27.108 INFO:teuthology.orchestra.run.smithi114.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-21T03:27:27.108 INFO:teuthology.orchestra.run.smithi114.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-21T03:27:27.108 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-21T03:27:27.133 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : automake-1.16.2-8.el9.noarch 115/165 2024-09-21T03:27:27.159 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : ceph-mgr-rook-2:19.3.0-5120.g18432218.el9.noarch 134/165 2024-09-21T03:27:27.170 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: ceph-mgr-rook-2:19.3.0-5120.g18432218.el9.noarch 134/165 2024-09-21T03:27:27.189 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-ply-3.11-14.el9.noarch 116/165 2024-09-21T03:27:27.209 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : ceph-test-2:19.3.0-5120.g18432218.el9.x86_64 142/165 2024-09-21T03:27:27.249 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : rbd-fuse-2:19.3.0-5120.g18432218.el9.x86_64 143/165 2024-09-21T03:27:27.253 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 117/165 2024-09-21T03:27:27.270 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : ceph-2:19.3.0-5120.g18432218.el9.x86_64 135/165 2024-09-21T03:27:27.290 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : rbd-nbd-2:19.3.0-5120.g18432218.el9.x86_64 144/165 2024-09-21T03:27:27.322 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : xfsprogs-devel-6.4.0-4.el9.x86_64 145/165 2024-09-21T03:27:27.324 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : ceph-fuse-2:19.3.0-5120.g18432218.el9.x86_64 136/165 2024-09-21T03:27:27.349 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 146/165 2024-09-21T03:27:27.437 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 118/165 2024-09-21T03:27:27.486 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 119/165 2024-09-21T03:27:27.547 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : cephfs-shell-2:19.3.0-5120.g18432218.el9.x86_64 147/165 2024-09-21T03:27:27.556 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 120/165 2024-09-21T03:27:27.597 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : postgresql-contrib-13.16-1.el9.x86_64 148/165 2024-09-21T03:27:27.627 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : libcephfs-devel-2:19.3.0-5120.g18432218.el9.x86_ 149/165 2024-09-21T03:27:27.653 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 121/165 2024-09-21T03:27:27.656 INFO:teuthology.orchestra.run.smithi110.stdout: Upgrading : NetworkManager-team-1:1.51.0-1.el9.x86_64 150/165 2024-09-21T03:27:27.694 INFO:teuthology.orchestra.run.smithi110.stdout: Upgrading : NetworkManager-tui-1:1.51.0-1.el9.x86_64 151/165 2024-09-21T03:27:27.792 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 122/165 2024-09-21T03:27:27.837 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 123/165 2024-09-21T03:27:27.869 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 124/165 2024-09-21T03:27:27.897 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : fuse-2.9.9-16.el9.x86_64 125/165 2024-09-21T03:27:28.289 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : libtool-2.4.6-46.el9.x86_64 137/165 2024-09-21T03:27:28.306 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 152/165 2024-09-21T03:27:28.306 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: cephadm-2:19.3.0-5120.g18432218.el9.noarch 126/165 2024-09-21T03:27:28.319 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : ceph-radosgw-2:19.3.0-5120.g18432218.el9.x86_64 138/165 2024-09-21T03:27:28.338 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : cephadm-2:19.3.0-5120.g18432218.el9.noarch 126/165 2024-09-21T03:27:28.343 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: ceph-radosgw-2:19.3.0-5120.g18432218.el9.x86_64 138/165 2024-09-21T03:27:28.343 INFO:teuthology.orchestra.run.smithi114.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T03:27:28.343 INFO:teuthology.orchestra.run.smithi114.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-21T03:27:28.343 INFO:teuthology.orchestra.run.smithi114.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-21T03:27:28.343 INFO:teuthology.orchestra.run.smithi114.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-21T03:27:28.343 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-21T03:27:28.343 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : postgresql-server-13.16-1.el9.x86_64 152/165 2024-09-21T03:27:28.363 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 152/165 2024-09-21T03:27:28.370 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : ceph-prometheus-alerts-2:19.3.0-5120.g18432218.e 127/165 2024-09-21T03:27:28.388 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : ceph-immutable-object-cache-2:19.3.0-5120.g18432 139/165 2024-09-21T03:27:28.410 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.3.0-5120.g18432 139/165 2024-09-21T03:27:28.410 INFO:teuthology.orchestra.run.smithi114.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T03:27:28.410 INFO:teuthology.orchestra.run.smithi114.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-21T03:27:28.410 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-21T03:27:28.437 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : ceph-grafana-dashboards-2:19.3.0-5120.g18432218. 128/165 2024-09-21T03:27:28.454 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : cephfs-mirror-2:19.3.0-5120.g18432218.el9.x86_64 140/165 2024-09-21T03:27:28.469 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : ceph-mgr-cephadm-2:19.3.0-5120.g18432218.el9.noa 129/165 2024-09-21T03:27:28.477 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: cephfs-mirror-2:19.3.0-5120.g18432218.el9.x86_64 140/165 2024-09-21T03:27:28.477 INFO:teuthology.orchestra.run.smithi114.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T03:27:28.478 INFO:teuthology.orchestra.run.smithi114.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-09-21T03:27:28.478 INFO:teuthology.orchestra.run.smithi114.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-21T03:27:28.478 INFO:teuthology.orchestra.run.smithi114.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-21T03:27:28.478 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-21T03:27:28.805 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : rbd-mirror-2:19.3.0-5120.g18432218.el9.x86_64 141/165 2024-09-21T03:27:28.828 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: rbd-mirror-2:19.3.0-5120.g18432218.el9.x86_64 141/165 2024-09-21T03:27:28.829 INFO:teuthology.orchestra.run.smithi114.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T03:27:28.829 INFO:teuthology.orchestra.run.smithi114.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-21T03:27:28.829 INFO:teuthology.orchestra.run.smithi114.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-21T03:27:28.829 INFO:teuthology.orchestra.run.smithi114.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-21T03:27:28.829 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-21T03:27:29.182 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : cephfs-top-2:19.3.0-5120.g18432218.el9.noarch 153/165 2024-09-21T03:27:29.265 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : openssl-devel-1:3.2.2-6.el9.x86_64 154/165 2024-09-21T03:27:29.313 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 155/165 2024-09-21T03:27:29.401 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : flex-2.6.4-9.el9.x86_64 156/165 2024-09-21T03:27:29.435 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : bison-3.7.4-5.el9.x86_64 157/165 2024-09-21T03:27:29.460 INFO:teuthology.orchestra.run.smithi110.stdout: Cleanup : NetworkManager-tui-1:1.48.8-1.el9.x86_64 158/165 2024-09-21T03:27:29.486 INFO:teuthology.orchestra.run.smithi110.stdout: Cleanup : openssl-1:3.2.2-4.el9.x86_64 159/165 2024-09-21T03:27:29.486 INFO:teuthology.orchestra.run.smithi110.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 160/165 2024-09-21T03:27:29.570 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 160/165 2024-09-21T03:27:29.570 INFO:teuthology.orchestra.run.smithi110.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 161/165 2024-09-21T03:27:29.654 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 161/165 2024-09-21T03:27:29.679 INFO:teuthology.orchestra.run.smithi110.stdout: Cleanup : NetworkManager-team-1:1.48.8-1.el9.x86_64 162/165 2024-09-21T03:27:29.684 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 163/165 2024-09-21T03:27:29.684 INFO:teuthology.orchestra.run.smithi110.stdout: Cleanup : NetworkManager-1:1.48.8-1.el9.x86_64 163/165 2024-09-21T03:27:29.873 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 163/165 2024-09-21T03:27:29.873 INFO:teuthology.orchestra.run.smithi110.stdout: Cleanup : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 164/165 2024-09-21T03:27:29.957 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: NetworkManager-libnm-1:1.48.8-1.el9.x86_64 164/165 2024-09-21T03:27:30.021 INFO:teuthology.orchestra.run.smithi110.stdout: Cleanup : openssl-libs-1:3.2.2-4.el9.x86_64 165/165 2024-09-21T03:27:31.646 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.3.0-5120.g18432218.el9.noa 129/165 2024-09-21T03:27:31.710 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : ceph-mgr-dashboard-2:19.3.0-5120.g18432218.el9.n 130/165 2024-09-21T03:27:32.672 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.3.0-5120.g18432218.el9.n 130/165 2024-09-21T03:27:32.702 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : ceph-mgr-diskprediction-local-2:19.3.0-5120.g184 131/165 2024-09-21T03:27:32.816 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.3.0-5120.g184 131/165 2024-09-21T03:27:32.845 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : ceph-test-2:19.3.0-5120.g18432218.el9.x86_64 142/165 2024-09-21T03:27:32.893 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : rbd-fuse-2:19.3.0-5120.g18432218.el9.x86_64 143/165 2024-09-21T03:27:32.932 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : rbd-nbd-2:19.3.0-5120.g18432218.el9.x86_64 144/165 2024-09-21T03:27:32.965 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : xfsprogs-devel-6.4.0-4.el9.x86_64 145/165 2024-09-21T03:27:32.979 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : ceph-mgr-modules-core-2:19.3.0-5120.g18432218.el 132/165 2024-09-21T03:27:32.993 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 146/165 2024-09-21T03:27:33.013 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : ceph-mgr-2:19.3.0-5120.g18432218.el9.x86_64 133/165 2024-09-21T03:27:33.037 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: ceph-mgr-2:19.3.0-5120.g18432218.el9.x86_64 133/165 2024-09-21T03:27:33.038 INFO:teuthology.orchestra.run.smithi089.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T03:27:33.038 INFO:teuthology.orchestra.run.smithi089.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-21T03:27:33.038 INFO:teuthology.orchestra.run.smithi089.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-21T03:27:33.038 INFO:teuthology.orchestra.run.smithi089.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-21T03:27:33.038 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-09-21T03:27:33.088 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : ceph-mgr-rook-2:19.3.0-5120.g18432218.el9.noarch 134/165 2024-09-21T03:27:33.100 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: ceph-mgr-rook-2:19.3.0-5120.g18432218.el9.noarch 134/165 2024-09-21T03:27:33.192 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : cephfs-shell-2:19.3.0-5120.g18432218.el9.x86_64 147/165 2024-09-21T03:27:33.200 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : ceph-2:19.3.0-5120.g18432218.el9.x86_64 135/165 2024-09-21T03:27:33.247 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : postgresql-contrib-13.16-1.el9.x86_64 148/165 2024-09-21T03:27:33.251 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : ceph-fuse-2:19.3.0-5120.g18432218.el9.x86_64 136/165 2024-09-21T03:27:33.277 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : libcephfs-devel-2:19.3.0-5120.g18432218.el9.x86_ 149/165 2024-09-21T03:27:33.307 INFO:teuthology.orchestra.run.smithi114.stdout: Upgrading : NetworkManager-team-1:1.51.0-1.el9.x86_64 150/165 2024-09-21T03:27:33.353 INFO:teuthology.orchestra.run.smithi114.stdout: Upgrading : NetworkManager-tui-1:1.51.0-1.el9.x86_64 151/165 2024-09-21T03:27:33.939 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 152/165 2024-09-21T03:27:33.993 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : postgresql-server-13.16-1.el9.x86_64 152/165 2024-09-21T03:27:34.013 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 152/165 2024-09-21T03:27:34.208 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : libtool-2.4.6-46.el9.x86_64 137/165 2024-09-21T03:27:34.241 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : ceph-radosgw-2:19.3.0-5120.g18432218.el9.x86_64 138/165 2024-09-21T03:27:34.264 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: ceph-radosgw-2:19.3.0-5120.g18432218.el9.x86_64 138/165 2024-09-21T03:27:34.265 INFO:teuthology.orchestra.run.smithi089.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T03:27:34.265 INFO:teuthology.orchestra.run.smithi089.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-21T03:27:34.265 INFO:teuthology.orchestra.run.smithi089.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-21T03:27:34.265 INFO:teuthology.orchestra.run.smithi089.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-21T03:27:34.265 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-09-21T03:27:34.309 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : ceph-immutable-object-cache-2:19.3.0-5120.g18432 139/165 2024-09-21T03:27:34.331 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.3.0-5120.g18432 139/165 2024-09-21T03:27:34.331 INFO:teuthology.orchestra.run.smithi089.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T03:27:34.331 INFO:teuthology.orchestra.run.smithi089.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-21T03:27:34.331 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-09-21T03:27:34.385 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : cephfs-mirror-2:19.3.0-5120.g18432218.el9.x86_64 140/165 2024-09-21T03:27:34.407 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: cephfs-mirror-2:19.3.0-5120.g18432218.el9.x86_64 140/165 2024-09-21T03:27:34.407 INFO:teuthology.orchestra.run.smithi089.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T03:27:34.408 INFO:teuthology.orchestra.run.smithi089.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-09-21T03:27:34.408 INFO:teuthology.orchestra.run.smithi089.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-21T03:27:34.408 INFO:teuthology.orchestra.run.smithi089.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-21T03:27:34.408 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-09-21T03:27:34.727 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : rbd-mirror-2:19.3.0-5120.g18432218.el9.x86_64 141/165 2024-09-21T03:27:34.750 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: rbd-mirror-2:19.3.0-5120.g18432218.el9.x86_64 141/165 2024-09-21T03:27:34.750 INFO:teuthology.orchestra.run.smithi089.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T03:27:34.750 INFO:teuthology.orchestra.run.smithi089.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-21T03:27:34.750 INFO:teuthology.orchestra.run.smithi089.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-21T03:27:34.750 INFO:teuthology.orchestra.run.smithi089.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-21T03:27:34.750 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-09-21T03:27:34.840 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : cephfs-top-2:19.3.0-5120.g18432218.el9.noarch 153/165 2024-09-21T03:27:34.930 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : openssl-devel-1:3.2.2-6.el9.x86_64 154/165 2024-09-21T03:27:34.982 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 155/165 2024-09-21T03:27:35.076 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : flex-2.6.4-9.el9.x86_64 156/165 2024-09-21T03:27:35.108 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : bison-3.7.4-5.el9.x86_64 157/165 2024-09-21T03:27:35.133 INFO:teuthology.orchestra.run.smithi114.stdout: Cleanup : NetworkManager-tui-1:1.48.8-1.el9.x86_64 158/165 2024-09-21T03:27:35.159 INFO:teuthology.orchestra.run.smithi114.stdout: Cleanup : openssl-1:3.2.2-4.el9.x86_64 159/165 2024-09-21T03:27:35.159 INFO:teuthology.orchestra.run.smithi114.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 160/165 2024-09-21T03:27:35.277 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 160/165 2024-09-21T03:27:35.277 INFO:teuthology.orchestra.run.smithi114.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 161/165 2024-09-21T03:27:35.361 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 161/165 2024-09-21T03:27:35.386 INFO:teuthology.orchestra.run.smithi114.stdout: Cleanup : NetworkManager-team-1:1.48.8-1.el9.x86_64 162/165 2024-09-21T03:27:35.391 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 163/165 2024-09-21T03:27:35.391 INFO:teuthology.orchestra.run.smithi114.stdout: Cleanup : NetworkManager-1:1.48.8-1.el9.x86_64 163/165 2024-09-21T03:27:35.571 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 163/165 2024-09-21T03:27:35.571 INFO:teuthology.orchestra.run.smithi114.stdout: Cleanup : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 164/165 2024-09-21T03:27:35.663 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: NetworkManager-libnm-1:1.48.8-1.el9.x86_64 164/165 2024-09-21T03:27:35.728 INFO:teuthology.orchestra.run.smithi114.stdout: Cleanup : openssl-libs-1:3.2.2-4.el9.x86_64 165/165 2024-09-21T03:27:36.504 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: openssl-libs-1:3.2.2-4.el9.x86_64 165/165 2024-09-21T03:27:36.504 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : ceph-2:19.3.0-5120.g18432218.el9.x86_64 1/165 2024-09-21T03:27:36.504 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : ceph-base-2:19.3.0-5120.g18432218.el9.x86_64 2/165 2024-09-21T03:27:36.505 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : ceph-common-2:19.3.0-5120.g18432218.el9.x86_64 3/165 2024-09-21T03:27:36.505 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : ceph-fuse-2:19.3.0-5120.g18432218.el9.x86_64 4/165 2024-09-21T03:27:36.505 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : ceph-immutable-object-cache-2:19.3.0-5120.g18432 5/165 2024-09-21T03:27:36.505 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : ceph-mds-2:19.3.0-5120.g18432218.el9.x86_64 6/165 2024-09-21T03:27:36.505 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : ceph-mgr-2:19.3.0-5120.g18432218.el9.x86_64 7/165 2024-09-21T03:27:36.505 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : ceph-mon-2:19.3.0-5120.g18432218.el9.x86_64 8/165 2024-09-21T03:27:36.505 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : ceph-osd-2:19.3.0-5120.g18432218.el9.x86_64 9/165 2024-09-21T03:27:36.505 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : ceph-radosgw-2:19.3.0-5120.g18432218.el9.x86_64 10/165 2024-09-21T03:27:36.505 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : ceph-selinux-2:19.3.0-5120.g18432218.el9.x86_64 11/165 2024-09-21T03:27:36.505 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : ceph-test-2:19.3.0-5120.g18432218.el9.x86_64 12/165 2024-09-21T03:27:36.505 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : cephfs-mirror-2:19.3.0-5120.g18432218.el9.x86_64 13/165 2024-09-21T03:27:36.505 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : cephfs-shell-2:19.3.0-5120.g18432218.el9.x86_64 14/165 2024-09-21T03:27:36.505 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : libcephfs-devel-2:19.3.0-5120.g18432218.el9.x86_ 15/165 2024-09-21T03:27:36.505 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : libcephfs2-2:19.3.0-5120.g18432218.el9.x86_64 16/165 2024-09-21T03:27:36.506 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : libcephsqlite-2:19.3.0-5120.g18432218.el9.x86_64 17/165 2024-09-21T03:27:36.506 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : librados-devel-2:19.3.0-5120.g18432218.el9.x86_6 18/165 2024-09-21T03:27:36.506 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : libradosstriper1-2:19.3.0-5120.g18432218.el9.x86 19/165 2024-09-21T03:27:36.506 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : librgw2-2:19.3.0-5120.g18432218.el9.x86_64 20/165 2024-09-21T03:27:36.506 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-ceph-argparse-2:19.3.0-5120.g18432218.el 21/165 2024-09-21T03:27:36.506 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-ceph-common-2:19.3.0-5120.g18432218.el9. 22/165 2024-09-21T03:27:36.506 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-cephfs-2:19.3.0-5120.g18432218.el9.x86_6 23/165 2024-09-21T03:27:36.506 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-rados-2:19.3.0-5120.g18432218.el9.x86_64 24/165 2024-09-21T03:27:36.506 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-rbd-2:19.3.0-5120.g18432218.el9.x86_64 25/165 2024-09-21T03:27:36.506 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-rgw-2:19.3.0-5120.g18432218.el9.x86_64 26/165 2024-09-21T03:27:36.506 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : rbd-fuse-2:19.3.0-5120.g18432218.el9.x86_64 27/165 2024-09-21T03:27:36.506 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : rbd-mirror-2:19.3.0-5120.g18432218.el9.x86_64 28/165 2024-09-21T03:27:36.506 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : rbd-nbd-2:19.3.0-5120.g18432218.el9.x86_64 29/165 2024-09-21T03:27:36.506 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : ceph-grafana-dashboards-2:19.3.0-5120.g18432218. 30/165 2024-09-21T03:27:36.518 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : ceph-mgr-cephadm-2:19.3.0-5120.g18432218.el9.noa 31/165 2024-09-21T03:27:36.518 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : ceph-mgr-dashboard-2:19.3.0-5120.g18432218.el9.n 32/165 2024-09-21T03:27:36.518 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.3.0-5120.g184 33/165 2024-09-21T03:27:36.518 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : ceph-mgr-modules-core-2:19.3.0-5120.g18432218.el 34/165 2024-09-21T03:27:36.518 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : ceph-mgr-rook-2:19.3.0-5120.g18432218.el9.noarch 35/165 2024-09-21T03:27:36.518 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : ceph-prometheus-alerts-2:19.3.0-5120.g18432218.e 36/165 2024-09-21T03:27:36.518 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : ceph-volume-2:19.3.0-5120.g18432218.el9.noarch 37/165 2024-09-21T03:27:36.518 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : cephadm-2:19.3.0-5120.g18432218.el9.noarch 38/165 2024-09-21T03:27:36.519 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : cephfs-top-2:19.3.0-5120.g18432218.el9.noarch 39/165 2024-09-21T03:27:36.519 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 40/165 2024-09-21T03:27:36.519 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 41/165 2024-09-21T03:27:36.519 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 42/165 2024-09-21T03:27:36.519 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/165 2024-09-21T03:27:36.519 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/165 2024-09-21T03:27:36.519 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/165 2024-09-21T03:27:36.519 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/165 2024-09-21T03:27:36.519 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/165 2024-09-21T03:27:36.519 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 48/165 2024-09-21T03:27:36.519 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/165 2024-09-21T03:27:36.519 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/165 2024-09-21T03:27:36.519 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/165 2024-09-21T03:27:36.519 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/165 2024-09-21T03:27:36.520 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/165 2024-09-21T03:27:36.520 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/165 2024-09-21T03:27:36.520 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/165 2024-09-21T03:27:36.520 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/165 2024-09-21T03:27:36.520 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/165 2024-09-21T03:27:36.520 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : libnbd-1.20.2-2.el9.x86_64 58/165 2024-09-21T03:27:36.520 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 59/165 2024-09-21T03:27:36.520 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 60/165 2024-09-21T03:27:36.520 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 61/165 2024-09-21T03:27:36.520 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : libtool-2.4.6-46.el9.x86_64 62/165 2024-09-21T03:27:36.520 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : libuuid-devel-2.37.4-20.el9.x86_64 63/165 2024-09-21T03:27:36.520 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 64/165 2024-09-21T03:27:36.520 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 65/165 2024-09-21T03:27:36.520 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : lua-5.4.4-4.el9.x86_64 66/165 2024-09-21T03:27:36.520 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 67/165 2024-09-21T03:27:36.521 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 68/165 2024-09-21T03:27:36.521 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : openssl-devel-1:3.2.2-6.el9.x86_64 69/165 2024-09-21T03:27:36.521 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : postgresql-13.16-1.el9.x86_64 70/165 2024-09-21T03:27:36.521 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : postgresql-contrib-13.16-1.el9.x86_64 71/165 2024-09-21T03:27:36.521 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : postgresql-private-libs-13.16-1.el9.x86_64 72/165 2024-09-21T03:27:36.521 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : postgresql-server-13.16-1.el9.x86_64 73/165 2024-09-21T03:27:36.521 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 74/165 2024-09-21T03:27:36.521 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 75/165 2024-09-21T03:27:36.521 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 76/165 2024-09-21T03:27:36.521 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 77/165 2024-09-21T03:27:36.521 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 78/165 2024-09-21T03:27:36.521 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 79/165 2024-09-21T03:27:36.521 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 80/165 2024-09-21T03:27:36.521 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 81/165 2024-09-21T03:27:36.522 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 82/165 2024-09-21T03:27:36.522 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 83/165 2024-09-21T03:27:36.522 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 84/165 2024-09-21T03:27:36.522 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 85/165 2024-09-21T03:27:36.522 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 86/165 2024-09-21T03:27:36.522 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 87/165 2024-09-21T03:27:36.522 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 88/165 2024-09-21T03:27:36.522 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 89/165 2024-09-21T03:27:36.522 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 90/165 2024-09-21T03:27:36.522 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 91/165 2024-09-21T03:27:36.522 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 92/165 2024-09-21T03:27:36.522 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 93/165 2024-09-21T03:27:36.522 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 94/165 2024-09-21T03:27:36.523 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 95/165 2024-09-21T03:27:36.523 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 96/165 2024-09-21T03:27:36.523 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 97/165 2024-09-21T03:27:36.523 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 98/165 2024-09-21T03:27:36.523 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 99/165 2024-09-21T03:27:36.523 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 100/165 2024-09-21T03:27:36.523 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 101/165 2024-09-21T03:27:36.523 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : xfsprogs-devel-6.4.0-4.el9.x86_64 102/165 2024-09-21T03:27:36.523 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 103/165 2024-09-21T03:27:36.523 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 104/165 2024-09-21T03:27:36.523 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 105/165 2024-09-21T03:27:36.523 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 106/165 2024-09-21T03:27:36.523 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 107/165 2024-09-21T03:27:36.523 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : gperftools-libs-2.9.1-3.el9.x86_64 108/165 2024-09-21T03:27:36.523 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 109/165 2024-09-21T03:27:36.524 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 110/165 2024-09-21T03:27:36.524 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 111/165 2024-09-21T03:27:36.524 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 112/165 2024-09-21T03:27:36.524 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 113/165 2024-09-21T03:27:36.524 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 114/165 2024-09-21T03:27:36.524 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 115/165 2024-09-21T03:27:36.524 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 116/165 2024-09-21T03:27:36.524 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 117/165 2024-09-21T03:27:36.524 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 118/165 2024-09-21T03:27:36.524 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 119/165 2024-09-21T03:27:36.524 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 120/165 2024-09-21T03:27:36.524 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 121/165 2024-09-21T03:27:36.524 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 122/165 2024-09-21T03:27:36.524 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 123/165 2024-09-21T03:27:36.524 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 124/165 2024-09-21T03:27:36.525 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 125/165 2024-09-21T03:27:36.525 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 126/165 2024-09-21T03:27:36.525 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 127/165 2024-09-21T03:27:36.525 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 128/165 2024-09-21T03:27:36.525 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 129/165 2024-09-21T03:27:36.525 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 130/165 2024-09-21T03:27:36.525 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 131/165 2024-09-21T03:27:36.525 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 132/165 2024-09-21T03:27:36.525 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 133/165 2024-09-21T03:27:36.525 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 134/165 2024-09-21T03:27:36.525 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 135/165 2024-09-21T03:27:36.525 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 136/165 2024-09-21T03:27:36.525 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 137/165 2024-09-21T03:27:36.525 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 138/165 2024-09-21T03:27:36.525 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 139/165 2024-09-21T03:27:36.526 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 140/165 2024-09-21T03:27:36.526 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 141/165 2024-09-21T03:27:36.526 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 142/165 2024-09-21T03:27:36.526 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 143/165 2024-09-21T03:27:36.526 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 144/165 2024-09-21T03:27:36.526 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 145/165 2024-09-21T03:27:36.526 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 146/165 2024-09-21T03:27:36.526 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 147/165 2024-09-21T03:27:36.526 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : re2-1:20211101-20.el9.x86_64 148/165 2024-09-21T03:27:36.526 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 149/165 2024-09-21T03:27:36.526 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : librados2-2:19.3.0-5120.g18432218.el9.x86_64 150/165 2024-09-21T03:27:36.526 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 151/165 2024-09-21T03:27:36.526 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : librbd1-2:19.3.0-5120.g18432218.el9.x86_64 152/165 2024-09-21T03:27:36.526 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 153/165 2024-09-21T03:27:36.526 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : NetworkManager-1:1.51.0-1.el9.x86_64 154/165 2024-09-21T03:27:36.527 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : NetworkManager-1:1.48.8-1.el9.x86_64 155/165 2024-09-21T03:27:36.527 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : NetworkManager-libnm-1:1.51.0-1.el9.x86_64 156/165 2024-09-21T03:27:36.527 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 157/165 2024-09-21T03:27:36.527 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : NetworkManager-team-1:1.51.0-1.el9.x86_64 158/165 2024-09-21T03:27:36.527 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : NetworkManager-team-1:1.48.8-1.el9.x86_64 159/165 2024-09-21T03:27:36.527 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : NetworkManager-tui-1:1.51.0-1.el9.x86_64 160/165 2024-09-21T03:27:36.527 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : NetworkManager-tui-1:1.48.8-1.el9.x86_64 161/165 2024-09-21T03:27:36.527 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : openssl-1:3.2.2-6.el9.x86_64 162/165 2024-09-21T03:27:36.527 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : openssl-1:3.2.2-4.el9.x86_64 163/165 2024-09-21T03:27:36.527 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : openssl-libs-1:3.2.2-6.el9.x86_64 164/165 2024-09-21T03:27:38.777 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : ceph-test-2:19.3.0-5120.g18432218.el9.x86_64 142/165 2024-09-21T03:27:38.815 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : rbd-fuse-2:19.3.0-5120.g18432218.el9.x86_64 143/165 2024-09-21T03:27:38.855 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : rbd-nbd-2:19.3.0-5120.g18432218.el9.x86_64 144/165 2024-09-21T03:27:38.896 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : xfsprogs-devel-6.4.0-4.el9.x86_64 145/165 2024-09-21T03:27:38.923 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 146/165 2024-09-21T03:27:38.969 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : openssl-libs-1:3.2.2-4.el9.x86_64 165/165 2024-09-21T03:27:38.969 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-21T03:27:38.969 INFO:teuthology.orchestra.run.smithi110.stdout:Upgraded: 2024-09-21T03:27:38.969 INFO:teuthology.orchestra.run.smithi110.stdout: NetworkManager-1:1.51.0-1.el9.x86_64 2024-09-21T03:27:38.969 INFO:teuthology.orchestra.run.smithi110.stdout: NetworkManager-libnm-1:1.51.0-1.el9.x86_64 2024-09-21T03:27:38.969 INFO:teuthology.orchestra.run.smithi110.stdout: NetworkManager-team-1:1.51.0-1.el9.x86_64 2024-09-21T03:27:38.969 INFO:teuthology.orchestra.run.smithi110.stdout: NetworkManager-tui-1:1.51.0-1.el9.x86_64 2024-09-21T03:27:38.969 INFO:teuthology.orchestra.run.smithi110.stdout: librados2-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:27:38.969 INFO:teuthology.orchestra.run.smithi110.stdout: librbd1-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:27:38.970 INFO:teuthology.orchestra.run.smithi110.stdout: openssl-1:3.2.2-6.el9.x86_64 2024-09-21T03:27:38.970 INFO:teuthology.orchestra.run.smithi110.stdout: openssl-libs-1:3.2.2-6.el9.x86_64 2024-09-21T03:27:38.970 INFO:teuthology.orchestra.run.smithi110.stdout:Installed: 2024-09-21T03:27:38.970 INFO:teuthology.orchestra.run.smithi110.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-09-21T03:27:38.970 INFO:teuthology.orchestra.run.smithi110.stdout: automake-1.16.2-8.el9.noarch 2024-09-21T03:27:38.970 INFO:teuthology.orchestra.run.smithi110.stdout: bison-3.7.4-5.el9.x86_64 2024-09-21T03:27:38.970 INFO:teuthology.orchestra.run.smithi110.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-21T03:27:38.970 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:27:38.970 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-base-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:27:38.970 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-common-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:27:38.970 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-fuse-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:27:38.970 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-grafana-dashboards-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T03:27:38.971 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-immutable-object-cache-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:27:38.971 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-mds-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:27:38.971 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-mgr-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:27:38.971 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-mgr-cephadm-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T03:27:38.971 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-mgr-dashboard-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T03:27:38.971 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-mgr-diskprediction-local-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T03:27:38.971 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-mgr-modules-core-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T03:27:38.971 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-mgr-rook-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T03:27:38.971 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-mon-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:27:38.971 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-osd-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:27:38.971 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-prometheus-alerts-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T03:27:38.971 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-radosgw-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:27:38.972 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-selinux-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:27:38.972 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-test-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:27:38.972 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-volume-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T03:27:38.972 INFO:teuthology.orchestra.run.smithi110.stdout: cephadm-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T03:27:38.972 INFO:teuthology.orchestra.run.smithi110.stdout: cephfs-mirror-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:27:38.972 INFO:teuthology.orchestra.run.smithi110.stdout: cephfs-shell-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:27:38.972 INFO:teuthology.orchestra.run.smithi110.stdout: cephfs-top-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T03:27:38.972 INFO:teuthology.orchestra.run.smithi110.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-09-21T03:27:38.972 INFO:teuthology.orchestra.run.smithi110.stdout: flex-2.6.4-9.el9.x86_64 2024-09-21T03:27:38.972 INFO:teuthology.orchestra.run.smithi110.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-21T03:27:38.972 INFO:teuthology.orchestra.run.smithi110.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-21T03:27:38.973 INFO:teuthology.orchestra.run.smithi110.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-21T03:27:38.973 INFO:teuthology.orchestra.run.smithi110.stdout: fuse-2.9.9-16.el9.x86_64 2024-09-21T03:27:38.973 INFO:teuthology.orchestra.run.smithi110.stdout: gperftools-libs-2.9.1-3.el9.x86_64 2024-09-21T03:27:38.973 INFO:teuthology.orchestra.run.smithi110.stdout: grpc-data-1.46.7-10.el9.noarch 2024-09-21T03:27:38.973 INFO:teuthology.orchestra.run.smithi110.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-09-21T03:27:38.973 INFO:teuthology.orchestra.run.smithi110.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-21T03:27:38.973 INFO:teuthology.orchestra.run.smithi110.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-21T03:27:38.973 INFO:teuthology.orchestra.run.smithi110.stdout: libcephfs-devel-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:27:38.973 INFO:teuthology.orchestra.run.smithi110.stdout: libcephfs2-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:27:38.973 INFO:teuthology.orchestra.run.smithi110.stdout: libcephsqlite-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:27:38.973 INFO:teuthology.orchestra.run.smithi110.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-21T03:27:38.973 INFO:teuthology.orchestra.run.smithi110.stdout: libnbd-1.20.2-2.el9.x86_64 2024-09-21T03:27:38.973 INFO:teuthology.orchestra.run.smithi110.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-21T03:27:38.973 INFO:teuthology.orchestra.run.smithi110.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-21T03:27:38.974 INFO:teuthology.orchestra.run.smithi110.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-21T03:27:38.974 INFO:teuthology.orchestra.run.smithi110.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-21T03:27:38.974 INFO:teuthology.orchestra.run.smithi110.stdout: librados-devel-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:27:38.974 INFO:teuthology.orchestra.run.smithi110.stdout: libradosstriper1-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:27:38.974 INFO:teuthology.orchestra.run.smithi110.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-21T03:27:38.974 INFO:teuthology.orchestra.run.smithi110.stdout: librgw2-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:27:38.974 INFO:teuthology.orchestra.run.smithi110.stdout: libtool-2.4.6-46.el9.x86_64 2024-09-21T03:27:38.974 INFO:teuthology.orchestra.run.smithi110.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-21T03:27:38.974 INFO:teuthology.orchestra.run.smithi110.stdout: libuuid-devel-2.37.4-20.el9.x86_64 2024-09-21T03:27:38.974 INFO:teuthology.orchestra.run.smithi110.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-09-21T03:27:38.974 INFO:teuthology.orchestra.run.smithi110.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-21T03:27:38.974 INFO:teuthology.orchestra.run.smithi110.stdout: lua-5.4.4-4.el9.x86_64 2024-09-21T03:27:38.974 INFO:teuthology.orchestra.run.smithi110.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-09-21T03:27:38.974 INFO:teuthology.orchestra.run.smithi110.stdout: luarocks-3.9.2-5.el9.noarch 2024-09-21T03:27:38.975 INFO:teuthology.orchestra.run.smithi110.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-21T03:27:38.975 INFO:teuthology.orchestra.run.smithi110.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-21T03:27:38.975 INFO:teuthology.orchestra.run.smithi110.stdout: openssl-devel-1:3.2.2-6.el9.x86_64 2024-09-21T03:27:38.975 INFO:teuthology.orchestra.run.smithi110.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-21T03:27:38.975 INFO:teuthology.orchestra.run.smithi110.stdout: postgresql-13.16-1.el9.x86_64 2024-09-21T03:27:38.975 INFO:teuthology.orchestra.run.smithi110.stdout: postgresql-contrib-13.16-1.el9.x86_64 2024-09-21T03:27:38.975 INFO:teuthology.orchestra.run.smithi110.stdout: postgresql-private-libs-13.16-1.el9.x86_64 2024-09-21T03:27:38.975 INFO:teuthology.orchestra.run.smithi110.stdout: postgresql-server-13.16-1.el9.x86_64 2024-09-21T03:27:38.975 INFO:teuthology.orchestra.run.smithi110.stdout: protobuf-3.14.0-13.el9.x86_64 2024-09-21T03:27:38.975 INFO:teuthology.orchestra.run.smithi110.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-09-21T03:27:38.975 INFO:teuthology.orchestra.run.smithi110.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-21T03:27:38.975 INFO:teuthology.orchestra.run.smithi110.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-09-21T03:27:38.975 INFO:teuthology.orchestra.run.smithi110.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-21T03:27:38.975 INFO:teuthology.orchestra.run.smithi110.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-21T03:27:38.976 INFO:teuthology.orchestra.run.smithi110.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-21T03:27:38.976 INFO:teuthology.orchestra.run.smithi110.stdout: python3-ceph-argparse-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:27:38.976 INFO:teuthology.orchestra.run.smithi110.stdout: python3-ceph-common-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:27:38.976 INFO:teuthology.orchestra.run.smithi110.stdout: python3-cephfs-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:27:38.976 INFO:teuthology.orchestra.run.smithi110.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-21T03:27:38.976 INFO:teuthology.orchestra.run.smithi110.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-21T03:27:38.976 INFO:teuthology.orchestra.run.smithi110.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-21T03:27:38.976 INFO:teuthology.orchestra.run.smithi110.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-21T03:27:38.976 INFO:teuthology.orchestra.run.smithi110.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-09-21T03:27:38.976 INFO:teuthology.orchestra.run.smithi110.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-09-21T03:27:38.976 INFO:teuthology.orchestra.run.smithi110.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-21T03:27:38.976 INFO:teuthology.orchestra.run.smithi110.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-21T03:27:38.977 INFO:teuthology.orchestra.run.smithi110.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-21T03:27:38.977 INFO:teuthology.orchestra.run.smithi110.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-09-21T03:27:38.977 INFO:teuthology.orchestra.run.smithi110.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-09-21T03:27:38.977 INFO:teuthology.orchestra.run.smithi110.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-21T03:27:38.977 INFO:teuthology.orchestra.run.smithi110.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-21T03:27:38.977 INFO:teuthology.orchestra.run.smithi110.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-21T03:27:38.977 INFO:teuthology.orchestra.run.smithi110.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-21T03:27:38.977 INFO:teuthology.orchestra.run.smithi110.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-21T03:27:38.977 INFO:teuthology.orchestra.run.smithi110.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-21T03:27:38.977 INFO:teuthology.orchestra.run.smithi110.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-21T03:27:38.977 INFO:teuthology.orchestra.run.smithi110.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-21T03:27:38.977 INFO:teuthology.orchestra.run.smithi110.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-21T03:27:38.978 INFO:teuthology.orchestra.run.smithi110.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-21T03:27:38.978 INFO:teuthology.orchestra.run.smithi110.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-21T03:27:38.978 INFO:teuthology.orchestra.run.smithi110.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-21T03:27:38.978 INFO:teuthology.orchestra.run.smithi110.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-21T03:27:38.978 INFO:teuthology.orchestra.run.smithi110.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-21T03:27:38.978 INFO:teuthology.orchestra.run.smithi110.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-21T03:27:38.978 INFO:teuthology.orchestra.run.smithi110.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-21T03:27:38.978 INFO:teuthology.orchestra.run.smithi110.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-21T03:27:38.978 INFO:teuthology.orchestra.run.smithi110.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-21T03:27:38.978 INFO:teuthology.orchestra.run.smithi110.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-21T03:27:38.978 INFO:teuthology.orchestra.run.smithi110.stdout: python3-ply-3.11-14.el9.noarch 2024-09-21T03:27:38.978 INFO:teuthology.orchestra.run.smithi110.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-21T03:27:38.978 INFO:teuthology.orchestra.run.smithi110.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-21T03:27:38.979 INFO:teuthology.orchestra.run.smithi110.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-09-21T03:27:38.979 INFO:teuthology.orchestra.run.smithi110.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-21T03:27:38.979 INFO:teuthology.orchestra.run.smithi110.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-21T03:27:38.979 INFO:teuthology.orchestra.run.smithi110.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-21T03:27:38.979 INFO:teuthology.orchestra.run.smithi110.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-21T03:27:38.979 INFO:teuthology.orchestra.run.smithi110.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-09-21T03:27:38.979 INFO:teuthology.orchestra.run.smithi110.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-21T03:27:38.979 INFO:teuthology.orchestra.run.smithi110.stdout: python3-rados-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:27:38.979 INFO:teuthology.orchestra.run.smithi110.stdout: python3-rbd-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:27:38.980 INFO:teuthology.orchestra.run.smithi110.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-21T03:27:38.980 INFO:teuthology.orchestra.run.smithi110.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-21T03:27:38.980 INFO:teuthology.orchestra.run.smithi110.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-21T03:27:38.980 INFO:teuthology.orchestra.run.smithi110.stdout: python3-rgw-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:27:38.980 INFO:teuthology.orchestra.run.smithi110.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-21T03:27:38.980 INFO:teuthology.orchestra.run.smithi110.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-21T03:27:38.980 INFO:teuthology.orchestra.run.smithi110.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-21T03:27:38.980 INFO:teuthology.orchestra.run.smithi110.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-21T03:27:38.980 INFO:teuthology.orchestra.run.smithi110.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-21T03:27:38.980 INFO:teuthology.orchestra.run.smithi110.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-21T03:27:38.980 INFO:teuthology.orchestra.run.smithi110.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-21T03:27:38.980 INFO:teuthology.orchestra.run.smithi110.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-09-21T03:27:38.980 INFO:teuthology.orchestra.run.smithi110.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-21T03:27:38.980 INFO:teuthology.orchestra.run.smithi110.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-21T03:27:38.981 INFO:teuthology.orchestra.run.smithi110.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-21T03:27:38.981 INFO:teuthology.orchestra.run.smithi110.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-21T03:27:38.981 INFO:teuthology.orchestra.run.smithi110.stdout: qatlib-24.02.0-1.el9.x86_64 2024-09-21T03:27:38.981 INFO:teuthology.orchestra.run.smithi110.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-09-21T03:27:38.981 INFO:teuthology.orchestra.run.smithi110.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-09-21T03:27:38.981 INFO:teuthology.orchestra.run.smithi110.stdout: rbd-fuse-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:27:38.981 INFO:teuthology.orchestra.run.smithi110.stdout: rbd-mirror-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:27:38.981 INFO:teuthology.orchestra.run.smithi110.stdout: rbd-nbd-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:27:38.981 INFO:teuthology.orchestra.run.smithi110.stdout: re2-1:20211101-20.el9.x86_64 2024-09-21T03:27:38.981 INFO:teuthology.orchestra.run.smithi110.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-21T03:27:38.981 INFO:teuthology.orchestra.run.smithi110.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-21T03:27:38.981 INFO:teuthology.orchestra.run.smithi110.stdout: uuid-1.6.2-55.el9.x86_64 2024-09-21T03:27:38.981 INFO:teuthology.orchestra.run.smithi110.stdout: xfsprogs-devel-6.4.0-4.el9.x86_64 2024-09-21T03:27:38.982 INFO:teuthology.orchestra.run.smithi110.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-21T03:27:38.982 INFO:teuthology.orchestra.run.smithi110.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-09-21T03:27:38.982 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-21T03:27:38.982 INFO:teuthology.orchestra.run.smithi110.stdout:Complete! 2024-09-21T03:27:39.140 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : cephfs-shell-2:19.3.0-5120.g18432218.el9.x86_64 147/165 2024-09-21T03:27:39.187 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : postgresql-contrib-13.16-1.el9.x86_64 148/165 2024-09-21T03:27:39.218 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : libcephfs-devel-2:19.3.0-5120.g18432218.el9.x86_ 149/165 2024-09-21T03:27:39.247 INFO:teuthology.orchestra.run.smithi089.stdout: Upgrading : NetworkManager-team-1:1.51.0-1.el9.x86_64 150/165 2024-09-21T03:27:39.294 INFO:teuthology.orchestra.run.smithi089.stdout: Upgrading : NetworkManager-tui-1:1.51.0-1.el9.x86_64 151/165 2024-09-21T03:27:39.396 DEBUG:teuthology.parallel:result is None 2024-09-21T03:27:39.913 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 152/165 2024-09-21T03:27:39.951 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : postgresql-server-13.16-1.el9.x86_64 152/165 2024-09-21T03:27:39.971 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 152/165 2024-09-21T03:27:40.823 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : cephfs-top-2:19.3.0-5120.g18432218.el9.noarch 153/165 2024-09-21T03:27:40.916 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : openssl-devel-1:3.2.2-6.el9.x86_64 154/165 2024-09-21T03:27:40.962 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 155/165 2024-09-21T03:27:41.079 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : flex-2.6.4-9.el9.x86_64 156/165 2024-09-21T03:27:41.379 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : bison-3.7.4-5.el9.x86_64 157/165 2024-09-21T03:27:41.461 INFO:teuthology.orchestra.run.smithi089.stdout: Cleanup : NetworkManager-tui-1:1.48.8-1.el9.x86_64 158/165 2024-09-21T03:27:41.487 INFO:teuthology.orchestra.run.smithi089.stdout: Cleanup : openssl-1:3.2.2-4.el9.x86_64 159/165 2024-09-21T03:27:41.487 INFO:teuthology.orchestra.run.smithi089.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 160/165 2024-09-21T03:27:41.571 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 160/165 2024-09-21T03:27:41.572 INFO:teuthology.orchestra.run.smithi089.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 161/165 2024-09-21T03:27:41.664 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 161/165 2024-09-21T03:27:41.689 INFO:teuthology.orchestra.run.smithi089.stdout: Cleanup : NetworkManager-team-1:1.48.8-1.el9.x86_64 162/165 2024-09-21T03:27:41.694 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 163/165 2024-09-21T03:27:41.694 INFO:teuthology.orchestra.run.smithi089.stdout: Cleanup : NetworkManager-1:1.48.8-1.el9.x86_64 163/165 2024-09-21T03:27:41.874 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 163/165 2024-09-21T03:27:41.874 INFO:teuthology.orchestra.run.smithi089.stdout: Cleanup : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 164/165 2024-09-21T03:27:41.958 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: NetworkManager-libnm-1:1.48.8-1.el9.x86_64 164/165 2024-09-21T03:27:42.023 INFO:teuthology.orchestra.run.smithi089.stdout: Cleanup : openssl-libs-1:3.2.2-4.el9.x86_64 165/165 2024-09-21T03:27:42.228 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: openssl-libs-1:3.2.2-4.el9.x86_64 165/165 2024-09-21T03:27:42.228 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : ceph-2:19.3.0-5120.g18432218.el9.x86_64 1/165 2024-09-21T03:27:42.228 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : ceph-base-2:19.3.0-5120.g18432218.el9.x86_64 2/165 2024-09-21T03:27:42.228 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : ceph-common-2:19.3.0-5120.g18432218.el9.x86_64 3/165 2024-09-21T03:27:42.228 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : ceph-fuse-2:19.3.0-5120.g18432218.el9.x86_64 4/165 2024-09-21T03:27:42.228 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : ceph-immutable-object-cache-2:19.3.0-5120.g18432 5/165 2024-09-21T03:27:42.228 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : ceph-mds-2:19.3.0-5120.g18432218.el9.x86_64 6/165 2024-09-21T03:27:42.228 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : ceph-mgr-2:19.3.0-5120.g18432218.el9.x86_64 7/165 2024-09-21T03:27:42.228 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : ceph-mon-2:19.3.0-5120.g18432218.el9.x86_64 8/165 2024-09-21T03:27:42.228 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : ceph-osd-2:19.3.0-5120.g18432218.el9.x86_64 9/165 2024-09-21T03:27:42.228 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : ceph-radosgw-2:19.3.0-5120.g18432218.el9.x86_64 10/165 2024-09-21T03:27:42.228 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : ceph-selinux-2:19.3.0-5120.g18432218.el9.x86_64 11/165 2024-09-21T03:27:42.228 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : ceph-test-2:19.3.0-5120.g18432218.el9.x86_64 12/165 2024-09-21T03:27:42.229 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : cephfs-mirror-2:19.3.0-5120.g18432218.el9.x86_64 13/165 2024-09-21T03:27:42.229 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : cephfs-shell-2:19.3.0-5120.g18432218.el9.x86_64 14/165 2024-09-21T03:27:42.229 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : libcephfs-devel-2:19.3.0-5120.g18432218.el9.x86_ 15/165 2024-09-21T03:27:42.229 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : libcephfs2-2:19.3.0-5120.g18432218.el9.x86_64 16/165 2024-09-21T03:27:42.229 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : libcephsqlite-2:19.3.0-5120.g18432218.el9.x86_64 17/165 2024-09-21T03:27:42.229 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : librados-devel-2:19.3.0-5120.g18432218.el9.x86_6 18/165 2024-09-21T03:27:42.229 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : libradosstriper1-2:19.3.0-5120.g18432218.el9.x86 19/165 2024-09-21T03:27:42.229 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : librgw2-2:19.3.0-5120.g18432218.el9.x86_64 20/165 2024-09-21T03:27:42.229 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-ceph-argparse-2:19.3.0-5120.g18432218.el 21/165 2024-09-21T03:27:42.229 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-ceph-common-2:19.3.0-5120.g18432218.el9. 22/165 2024-09-21T03:27:42.229 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-cephfs-2:19.3.0-5120.g18432218.el9.x86_6 23/165 2024-09-21T03:27:42.229 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-rados-2:19.3.0-5120.g18432218.el9.x86_64 24/165 2024-09-21T03:27:42.229 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-rbd-2:19.3.0-5120.g18432218.el9.x86_64 25/165 2024-09-21T03:27:42.229 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-rgw-2:19.3.0-5120.g18432218.el9.x86_64 26/165 2024-09-21T03:27:42.230 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : rbd-fuse-2:19.3.0-5120.g18432218.el9.x86_64 27/165 2024-09-21T03:27:42.230 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : rbd-mirror-2:19.3.0-5120.g18432218.el9.x86_64 28/165 2024-09-21T03:27:42.230 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : rbd-nbd-2:19.3.0-5120.g18432218.el9.x86_64 29/165 2024-09-21T03:27:42.230 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : ceph-grafana-dashboards-2:19.3.0-5120.g18432218. 30/165 2024-09-21T03:27:42.241 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : ceph-mgr-cephadm-2:19.3.0-5120.g18432218.el9.noa 31/165 2024-09-21T03:27:42.241 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : ceph-mgr-dashboard-2:19.3.0-5120.g18432218.el9.n 32/165 2024-09-21T03:27:42.241 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.3.0-5120.g184 33/165 2024-09-21T03:27:42.241 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : ceph-mgr-modules-core-2:19.3.0-5120.g18432218.el 34/165 2024-09-21T03:27:42.241 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : ceph-mgr-rook-2:19.3.0-5120.g18432218.el9.noarch 35/165 2024-09-21T03:27:42.241 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : ceph-prometheus-alerts-2:19.3.0-5120.g18432218.e 36/165 2024-09-21T03:27:42.242 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : ceph-volume-2:19.3.0-5120.g18432218.el9.noarch 37/165 2024-09-21T03:27:42.242 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : cephadm-2:19.3.0-5120.g18432218.el9.noarch 38/165 2024-09-21T03:27:42.242 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : cephfs-top-2:19.3.0-5120.g18432218.el9.noarch 39/165 2024-09-21T03:27:42.242 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 40/165 2024-09-21T03:27:42.242 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 41/165 2024-09-21T03:27:42.242 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 42/165 2024-09-21T03:27:42.242 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/165 2024-09-21T03:27:42.242 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/165 2024-09-21T03:27:42.242 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/165 2024-09-21T03:27:42.242 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/165 2024-09-21T03:27:42.242 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/165 2024-09-21T03:27:42.242 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 48/165 2024-09-21T03:27:42.242 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/165 2024-09-21T03:27:42.243 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/165 2024-09-21T03:27:42.243 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/165 2024-09-21T03:27:42.243 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/165 2024-09-21T03:27:42.243 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/165 2024-09-21T03:27:42.243 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/165 2024-09-21T03:27:42.243 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/165 2024-09-21T03:27:42.243 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/165 2024-09-21T03:27:42.243 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/165 2024-09-21T03:27:42.243 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : libnbd-1.20.2-2.el9.x86_64 58/165 2024-09-21T03:27:42.243 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 59/165 2024-09-21T03:27:42.243 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 60/165 2024-09-21T03:27:42.243 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 61/165 2024-09-21T03:27:42.243 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : libtool-2.4.6-46.el9.x86_64 62/165 2024-09-21T03:27:42.243 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : libuuid-devel-2.37.4-20.el9.x86_64 63/165 2024-09-21T03:27:42.244 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 64/165 2024-09-21T03:27:42.244 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 65/165 2024-09-21T03:27:42.244 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : lua-5.4.4-4.el9.x86_64 66/165 2024-09-21T03:27:42.244 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 67/165 2024-09-21T03:27:42.244 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 68/165 2024-09-21T03:27:42.244 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : openssl-devel-1:3.2.2-6.el9.x86_64 69/165 2024-09-21T03:27:42.244 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : postgresql-13.16-1.el9.x86_64 70/165 2024-09-21T03:27:42.244 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : postgresql-contrib-13.16-1.el9.x86_64 71/165 2024-09-21T03:27:42.244 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : postgresql-private-libs-13.16-1.el9.x86_64 72/165 2024-09-21T03:27:42.244 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : postgresql-server-13.16-1.el9.x86_64 73/165 2024-09-21T03:27:42.244 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 74/165 2024-09-21T03:27:42.244 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 75/165 2024-09-21T03:27:42.244 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 76/165 2024-09-21T03:27:42.244 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 77/165 2024-09-21T03:27:42.244 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 78/165 2024-09-21T03:27:42.245 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 79/165 2024-09-21T03:27:42.245 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 80/165 2024-09-21T03:27:42.245 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 81/165 2024-09-21T03:27:42.245 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 82/165 2024-09-21T03:27:42.245 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 83/165 2024-09-21T03:27:42.245 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 84/165 2024-09-21T03:27:42.245 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 85/165 2024-09-21T03:27:42.245 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 86/165 2024-09-21T03:27:42.245 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 87/165 2024-09-21T03:27:42.245 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 88/165 2024-09-21T03:27:42.245 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 89/165 2024-09-21T03:27:42.245 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 90/165 2024-09-21T03:27:42.245 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 91/165 2024-09-21T03:27:42.245 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 92/165 2024-09-21T03:27:42.246 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 93/165 2024-09-21T03:27:42.246 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 94/165 2024-09-21T03:27:42.246 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 95/165 2024-09-21T03:27:42.246 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 96/165 2024-09-21T03:27:42.246 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 97/165 2024-09-21T03:27:42.246 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 98/165 2024-09-21T03:27:42.246 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 99/165 2024-09-21T03:27:42.246 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 100/165 2024-09-21T03:27:42.246 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 101/165 2024-09-21T03:27:42.246 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : xfsprogs-devel-6.4.0-4.el9.x86_64 102/165 2024-09-21T03:27:42.246 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 103/165 2024-09-21T03:27:42.246 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 104/165 2024-09-21T03:27:42.246 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 105/165 2024-09-21T03:27:42.246 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 106/165 2024-09-21T03:27:42.247 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 107/165 2024-09-21T03:27:42.247 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : gperftools-libs-2.9.1-3.el9.x86_64 108/165 2024-09-21T03:27:42.247 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 109/165 2024-09-21T03:27:42.247 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 110/165 2024-09-21T03:27:42.247 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 111/165 2024-09-21T03:27:42.247 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 112/165 2024-09-21T03:27:42.247 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 113/165 2024-09-21T03:27:42.247 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 114/165 2024-09-21T03:27:42.247 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 115/165 2024-09-21T03:27:42.247 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 116/165 2024-09-21T03:27:42.247 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 117/165 2024-09-21T03:27:42.247 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 118/165 2024-09-21T03:27:42.247 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 119/165 2024-09-21T03:27:42.247 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 120/165 2024-09-21T03:27:42.247 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 121/165 2024-09-21T03:27:42.248 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 122/165 2024-09-21T03:27:42.248 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 123/165 2024-09-21T03:27:42.248 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 124/165 2024-09-21T03:27:42.248 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 125/165 2024-09-21T03:27:42.248 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 126/165 2024-09-21T03:27:42.248 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 127/165 2024-09-21T03:27:42.248 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 128/165 2024-09-21T03:27:42.248 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 129/165 2024-09-21T03:27:42.248 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 130/165 2024-09-21T03:27:42.248 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 131/165 2024-09-21T03:27:42.248 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 132/165 2024-09-21T03:27:42.249 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 133/165 2024-09-21T03:27:42.249 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 134/165 2024-09-21T03:27:42.249 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 135/165 2024-09-21T03:27:42.249 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 136/165 2024-09-21T03:27:42.249 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 137/165 2024-09-21T03:27:42.249 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 138/165 2024-09-21T03:27:42.249 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 139/165 2024-09-21T03:27:42.249 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 140/165 2024-09-21T03:27:42.249 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 141/165 2024-09-21T03:27:42.249 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 142/165 2024-09-21T03:27:42.249 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 143/165 2024-09-21T03:27:42.249 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 144/165 2024-09-21T03:27:42.249 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 145/165 2024-09-21T03:27:42.249 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 146/165 2024-09-21T03:27:42.250 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 147/165 2024-09-21T03:27:42.250 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : re2-1:20211101-20.el9.x86_64 148/165 2024-09-21T03:27:42.250 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 149/165 2024-09-21T03:27:42.250 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : librados2-2:19.3.0-5120.g18432218.el9.x86_64 150/165 2024-09-21T03:27:42.250 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 151/165 2024-09-21T03:27:42.250 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : librbd1-2:19.3.0-5120.g18432218.el9.x86_64 152/165 2024-09-21T03:27:42.250 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 153/165 2024-09-21T03:27:42.250 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : NetworkManager-1:1.51.0-1.el9.x86_64 154/165 2024-09-21T03:27:42.250 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : NetworkManager-1:1.48.8-1.el9.x86_64 155/165 2024-09-21T03:27:42.250 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : NetworkManager-libnm-1:1.51.0-1.el9.x86_64 156/165 2024-09-21T03:27:42.250 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 157/165 2024-09-21T03:27:42.250 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : NetworkManager-team-1:1.51.0-1.el9.x86_64 158/165 2024-09-21T03:27:42.250 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : NetworkManager-team-1:1.48.8-1.el9.x86_64 159/165 2024-09-21T03:27:42.250 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : NetworkManager-tui-1:1.51.0-1.el9.x86_64 160/165 2024-09-21T03:27:42.251 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : NetworkManager-tui-1:1.48.8-1.el9.x86_64 161/165 2024-09-21T03:27:42.251 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : openssl-1:3.2.2-6.el9.x86_64 162/165 2024-09-21T03:27:42.251 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : openssl-1:3.2.2-4.el9.x86_64 163/165 2024-09-21T03:27:42.251 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : openssl-libs-1:3.2.2-6.el9.x86_64 164/165 2024-09-21T03:27:44.751 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : openssl-libs-1:3.2.2-4.el9.x86_64 165/165 2024-09-21T03:27:44.752 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-21T03:27:44.752 INFO:teuthology.orchestra.run.smithi114.stdout:Upgraded: 2024-09-21T03:27:44.752 INFO:teuthology.orchestra.run.smithi114.stdout: NetworkManager-1:1.51.0-1.el9.x86_64 2024-09-21T03:27:44.752 INFO:teuthology.orchestra.run.smithi114.stdout: NetworkManager-libnm-1:1.51.0-1.el9.x86_64 2024-09-21T03:27:44.752 INFO:teuthology.orchestra.run.smithi114.stdout: NetworkManager-team-1:1.51.0-1.el9.x86_64 2024-09-21T03:27:44.752 INFO:teuthology.orchestra.run.smithi114.stdout: NetworkManager-tui-1:1.51.0-1.el9.x86_64 2024-09-21T03:27:44.752 INFO:teuthology.orchestra.run.smithi114.stdout: librados2-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:27:44.752 INFO:teuthology.orchestra.run.smithi114.stdout: librbd1-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:27:44.752 INFO:teuthology.orchestra.run.smithi114.stdout: openssl-1:3.2.2-6.el9.x86_64 2024-09-21T03:27:44.752 INFO:teuthology.orchestra.run.smithi114.stdout: openssl-libs-1:3.2.2-6.el9.x86_64 2024-09-21T03:27:44.752 INFO:teuthology.orchestra.run.smithi114.stdout:Installed: 2024-09-21T03:27:44.752 INFO:teuthology.orchestra.run.smithi114.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-09-21T03:27:44.753 INFO:teuthology.orchestra.run.smithi114.stdout: automake-1.16.2-8.el9.noarch 2024-09-21T03:27:44.753 INFO:teuthology.orchestra.run.smithi114.stdout: bison-3.7.4-5.el9.x86_64 2024-09-21T03:27:44.753 INFO:teuthology.orchestra.run.smithi114.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-21T03:27:44.753 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:27:44.753 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-base-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:27:44.753 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-common-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:27:44.753 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-fuse-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:27:44.753 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-grafana-dashboards-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T03:27:44.753 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-immutable-object-cache-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:27:44.753 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-mds-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:27:44.753 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-mgr-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:27:44.754 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-mgr-cephadm-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T03:27:44.754 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-mgr-dashboard-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T03:27:44.754 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-mgr-diskprediction-local-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T03:27:44.754 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-mgr-modules-core-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T03:27:44.754 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-mgr-rook-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T03:27:44.754 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-mon-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:27:44.754 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-osd-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:27:44.754 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-prometheus-alerts-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T03:27:44.754 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-radosgw-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:27:44.754 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-selinux-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:27:44.754 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-test-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:27:44.754 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-volume-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T03:27:44.755 INFO:teuthology.orchestra.run.smithi114.stdout: cephadm-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T03:27:44.755 INFO:teuthology.orchestra.run.smithi114.stdout: cephfs-mirror-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:27:44.755 INFO:teuthology.orchestra.run.smithi114.stdout: cephfs-shell-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:27:44.755 INFO:teuthology.orchestra.run.smithi114.stdout: cephfs-top-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T03:27:44.755 INFO:teuthology.orchestra.run.smithi114.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-09-21T03:27:44.755 INFO:teuthology.orchestra.run.smithi114.stdout: flex-2.6.4-9.el9.x86_64 2024-09-21T03:27:44.755 INFO:teuthology.orchestra.run.smithi114.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-21T03:27:44.755 INFO:teuthology.orchestra.run.smithi114.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-21T03:27:44.755 INFO:teuthology.orchestra.run.smithi114.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-21T03:27:44.755 INFO:teuthology.orchestra.run.smithi114.stdout: fuse-2.9.9-16.el9.x86_64 2024-09-21T03:27:44.755 INFO:teuthology.orchestra.run.smithi114.stdout: gperftools-libs-2.9.1-3.el9.x86_64 2024-09-21T03:27:44.755 INFO:teuthology.orchestra.run.smithi114.stdout: grpc-data-1.46.7-10.el9.noarch 2024-09-21T03:27:44.755 INFO:teuthology.orchestra.run.smithi114.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-09-21T03:27:44.755 INFO:teuthology.orchestra.run.smithi114.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-21T03:27:44.756 INFO:teuthology.orchestra.run.smithi114.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-21T03:27:44.756 INFO:teuthology.orchestra.run.smithi114.stdout: libcephfs-devel-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:27:44.756 INFO:teuthology.orchestra.run.smithi114.stdout: libcephfs2-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:27:44.756 INFO:teuthology.orchestra.run.smithi114.stdout: libcephsqlite-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:27:44.756 INFO:teuthology.orchestra.run.smithi114.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-21T03:27:44.756 INFO:teuthology.orchestra.run.smithi114.stdout: libnbd-1.20.2-2.el9.x86_64 2024-09-21T03:27:44.756 INFO:teuthology.orchestra.run.smithi114.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-21T03:27:44.756 INFO:teuthology.orchestra.run.smithi114.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-21T03:27:44.756 INFO:teuthology.orchestra.run.smithi114.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-21T03:27:44.756 INFO:teuthology.orchestra.run.smithi114.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-21T03:27:44.756 INFO:teuthology.orchestra.run.smithi114.stdout: librados-devel-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:27:44.756 INFO:teuthology.orchestra.run.smithi114.stdout: libradosstriper1-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:27:44.756 INFO:teuthology.orchestra.run.smithi114.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-21T03:27:44.757 INFO:teuthology.orchestra.run.smithi114.stdout: librgw2-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:27:44.757 INFO:teuthology.orchestra.run.smithi114.stdout: libtool-2.4.6-46.el9.x86_64 2024-09-21T03:27:44.757 INFO:teuthology.orchestra.run.smithi114.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-21T03:27:44.757 INFO:teuthology.orchestra.run.smithi114.stdout: libuuid-devel-2.37.4-20.el9.x86_64 2024-09-21T03:27:44.757 INFO:teuthology.orchestra.run.smithi114.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-09-21T03:27:44.757 INFO:teuthology.orchestra.run.smithi114.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-21T03:27:44.757 INFO:teuthology.orchestra.run.smithi114.stdout: lua-5.4.4-4.el9.x86_64 2024-09-21T03:27:44.757 INFO:teuthology.orchestra.run.smithi114.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-09-21T03:27:44.757 INFO:teuthology.orchestra.run.smithi114.stdout: luarocks-3.9.2-5.el9.noarch 2024-09-21T03:27:44.757 INFO:teuthology.orchestra.run.smithi114.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-21T03:27:44.757 INFO:teuthology.orchestra.run.smithi114.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-21T03:27:44.757 INFO:teuthology.orchestra.run.smithi114.stdout: openssl-devel-1:3.2.2-6.el9.x86_64 2024-09-21T03:27:44.757 INFO:teuthology.orchestra.run.smithi114.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-21T03:27:44.758 INFO:teuthology.orchestra.run.smithi114.stdout: postgresql-13.16-1.el9.x86_64 2024-09-21T03:27:44.758 INFO:teuthology.orchestra.run.smithi114.stdout: postgresql-contrib-13.16-1.el9.x86_64 2024-09-21T03:27:44.758 INFO:teuthology.orchestra.run.smithi114.stdout: postgresql-private-libs-13.16-1.el9.x86_64 2024-09-21T03:27:44.758 INFO:teuthology.orchestra.run.smithi114.stdout: postgresql-server-13.16-1.el9.x86_64 2024-09-21T03:27:44.758 INFO:teuthology.orchestra.run.smithi114.stdout: protobuf-3.14.0-13.el9.x86_64 2024-09-21T03:27:44.758 INFO:teuthology.orchestra.run.smithi114.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-09-21T03:27:44.758 INFO:teuthology.orchestra.run.smithi114.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-21T03:27:44.758 INFO:teuthology.orchestra.run.smithi114.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-09-21T03:27:44.758 INFO:teuthology.orchestra.run.smithi114.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-21T03:27:44.758 INFO:teuthology.orchestra.run.smithi114.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-21T03:27:44.758 INFO:teuthology.orchestra.run.smithi114.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-21T03:27:44.758 INFO:teuthology.orchestra.run.smithi114.stdout: python3-ceph-argparse-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:27:44.758 INFO:teuthology.orchestra.run.smithi114.stdout: python3-ceph-common-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:27:44.759 INFO:teuthology.orchestra.run.smithi114.stdout: python3-cephfs-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:27:44.759 INFO:teuthology.orchestra.run.smithi114.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-21T03:27:44.759 INFO:teuthology.orchestra.run.smithi114.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-21T03:27:44.759 INFO:teuthology.orchestra.run.smithi114.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-21T03:27:44.759 INFO:teuthology.orchestra.run.smithi114.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-21T03:27:44.759 INFO:teuthology.orchestra.run.smithi114.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-09-21T03:27:44.759 INFO:teuthology.orchestra.run.smithi114.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-09-21T03:27:44.759 INFO:teuthology.orchestra.run.smithi114.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-21T03:27:44.759 INFO:teuthology.orchestra.run.smithi114.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-21T03:27:44.759 INFO:teuthology.orchestra.run.smithi114.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-21T03:27:44.759 INFO:teuthology.orchestra.run.smithi114.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-09-21T03:27:44.759 INFO:teuthology.orchestra.run.smithi114.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-09-21T03:27:44.759 INFO:teuthology.orchestra.run.smithi114.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-21T03:27:44.759 INFO:teuthology.orchestra.run.smithi114.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-21T03:27:44.759 INFO:teuthology.orchestra.run.smithi114.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-21T03:27:44.760 INFO:teuthology.orchestra.run.smithi114.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-21T03:27:44.760 INFO:teuthology.orchestra.run.smithi114.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-21T03:27:44.760 INFO:teuthology.orchestra.run.smithi114.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-21T03:27:44.760 INFO:teuthology.orchestra.run.smithi114.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-21T03:27:44.760 INFO:teuthology.orchestra.run.smithi114.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-21T03:27:44.760 INFO:teuthology.orchestra.run.smithi114.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-21T03:27:44.760 INFO:teuthology.orchestra.run.smithi114.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-21T03:27:44.760 INFO:teuthology.orchestra.run.smithi114.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-21T03:27:44.760 INFO:teuthology.orchestra.run.smithi114.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-21T03:27:44.760 INFO:teuthology.orchestra.run.smithi114.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-21T03:27:44.760 INFO:teuthology.orchestra.run.smithi114.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-21T03:27:44.760 INFO:teuthology.orchestra.run.smithi114.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-21T03:27:44.760 INFO:teuthology.orchestra.run.smithi114.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-21T03:27:44.761 INFO:teuthology.orchestra.run.smithi114.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-21T03:27:44.761 INFO:teuthology.orchestra.run.smithi114.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-21T03:27:44.761 INFO:teuthology.orchestra.run.smithi114.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-21T03:27:44.761 INFO:teuthology.orchestra.run.smithi114.stdout: python3-ply-3.11-14.el9.noarch 2024-09-21T03:27:44.761 INFO:teuthology.orchestra.run.smithi114.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-21T03:27:44.761 INFO:teuthology.orchestra.run.smithi114.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-21T03:27:44.761 INFO:teuthology.orchestra.run.smithi114.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-09-21T03:27:44.761 INFO:teuthology.orchestra.run.smithi114.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-21T03:27:44.761 INFO:teuthology.orchestra.run.smithi114.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-21T03:27:44.761 INFO:teuthology.orchestra.run.smithi114.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-21T03:27:44.761 INFO:teuthology.orchestra.run.smithi114.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-21T03:27:44.761 INFO:teuthology.orchestra.run.smithi114.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-09-21T03:27:44.761 INFO:teuthology.orchestra.run.smithi114.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-21T03:27:44.761 INFO:teuthology.orchestra.run.smithi114.stdout: python3-rados-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:27:44.762 INFO:teuthology.orchestra.run.smithi114.stdout: python3-rbd-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:27:44.762 INFO:teuthology.orchestra.run.smithi114.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-21T03:27:44.762 INFO:teuthology.orchestra.run.smithi114.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-21T03:27:44.762 INFO:teuthology.orchestra.run.smithi114.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-21T03:27:44.762 INFO:teuthology.orchestra.run.smithi114.stdout: python3-rgw-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:27:44.762 INFO:teuthology.orchestra.run.smithi114.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-21T03:27:44.762 INFO:teuthology.orchestra.run.smithi114.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-21T03:27:44.762 INFO:teuthology.orchestra.run.smithi114.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-21T03:27:44.762 INFO:teuthology.orchestra.run.smithi114.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-21T03:27:44.762 INFO:teuthology.orchestra.run.smithi114.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-21T03:27:44.762 INFO:teuthology.orchestra.run.smithi114.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-21T03:27:44.762 INFO:teuthology.orchestra.run.smithi114.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-21T03:27:44.762 INFO:teuthology.orchestra.run.smithi114.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-09-21T03:27:44.763 INFO:teuthology.orchestra.run.smithi114.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-21T03:27:44.763 INFO:teuthology.orchestra.run.smithi114.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-21T03:27:44.763 INFO:teuthology.orchestra.run.smithi114.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-21T03:27:44.763 INFO:teuthology.orchestra.run.smithi114.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-21T03:27:44.763 INFO:teuthology.orchestra.run.smithi114.stdout: qatlib-24.02.0-1.el9.x86_64 2024-09-21T03:27:44.763 INFO:teuthology.orchestra.run.smithi114.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-09-21T03:27:44.763 INFO:teuthology.orchestra.run.smithi114.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-09-21T03:27:44.763 INFO:teuthology.orchestra.run.smithi114.stdout: rbd-fuse-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:27:44.763 INFO:teuthology.orchestra.run.smithi114.stdout: rbd-mirror-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:27:44.763 INFO:teuthology.orchestra.run.smithi114.stdout: rbd-nbd-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:27:44.763 INFO:teuthology.orchestra.run.smithi114.stdout: re2-1:20211101-20.el9.x86_64 2024-09-21T03:27:44.763 INFO:teuthology.orchestra.run.smithi114.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-21T03:27:44.763 INFO:teuthology.orchestra.run.smithi114.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-21T03:27:44.763 INFO:teuthology.orchestra.run.smithi114.stdout: uuid-1.6.2-55.el9.x86_64 2024-09-21T03:27:44.763 INFO:teuthology.orchestra.run.smithi114.stdout: xfsprogs-devel-6.4.0-4.el9.x86_64 2024-09-21T03:27:44.763 INFO:teuthology.orchestra.run.smithi114.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-21T03:27:44.764 INFO:teuthology.orchestra.run.smithi114.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-09-21T03:27:44.764 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-21T03:27:44.764 INFO:teuthology.orchestra.run.smithi114.stdout:Complete! 2024-09-21T03:27:45.204 DEBUG:teuthology.parallel:result is None 2024-09-21T03:27:48.150 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: openssl-libs-1:3.2.2-4.el9.x86_64 165/165 2024-09-21T03:27:48.151 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : ceph-2:19.3.0-5120.g18432218.el9.x86_64 1/165 2024-09-21T03:27:48.151 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : ceph-base-2:19.3.0-5120.g18432218.el9.x86_64 2/165 2024-09-21T03:27:48.151 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : ceph-common-2:19.3.0-5120.g18432218.el9.x86_64 3/165 2024-09-21T03:27:48.151 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : ceph-fuse-2:19.3.0-5120.g18432218.el9.x86_64 4/165 2024-09-21T03:27:48.151 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : ceph-immutable-object-cache-2:19.3.0-5120.g18432 5/165 2024-09-21T03:27:48.151 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : ceph-mds-2:19.3.0-5120.g18432218.el9.x86_64 6/165 2024-09-21T03:27:48.151 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : ceph-mgr-2:19.3.0-5120.g18432218.el9.x86_64 7/165 2024-09-21T03:27:48.151 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : ceph-mon-2:19.3.0-5120.g18432218.el9.x86_64 8/165 2024-09-21T03:27:48.151 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : ceph-osd-2:19.3.0-5120.g18432218.el9.x86_64 9/165 2024-09-21T03:27:48.151 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : ceph-radosgw-2:19.3.0-5120.g18432218.el9.x86_64 10/165 2024-09-21T03:27:48.151 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : ceph-selinux-2:19.3.0-5120.g18432218.el9.x86_64 11/165 2024-09-21T03:27:48.152 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : ceph-test-2:19.3.0-5120.g18432218.el9.x86_64 12/165 2024-09-21T03:27:48.152 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : cephfs-mirror-2:19.3.0-5120.g18432218.el9.x86_64 13/165 2024-09-21T03:27:48.152 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : cephfs-shell-2:19.3.0-5120.g18432218.el9.x86_64 14/165 2024-09-21T03:27:48.152 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : libcephfs-devel-2:19.3.0-5120.g18432218.el9.x86_ 15/165 2024-09-21T03:27:48.152 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : libcephfs2-2:19.3.0-5120.g18432218.el9.x86_64 16/165 2024-09-21T03:27:48.152 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : libcephsqlite-2:19.3.0-5120.g18432218.el9.x86_64 17/165 2024-09-21T03:27:48.152 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : librados-devel-2:19.3.0-5120.g18432218.el9.x86_6 18/165 2024-09-21T03:27:48.152 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : libradosstriper1-2:19.3.0-5120.g18432218.el9.x86 19/165 2024-09-21T03:27:48.152 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : librgw2-2:19.3.0-5120.g18432218.el9.x86_64 20/165 2024-09-21T03:27:48.152 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-ceph-argparse-2:19.3.0-5120.g18432218.el 21/165 2024-09-21T03:27:48.152 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-ceph-common-2:19.3.0-5120.g18432218.el9. 22/165 2024-09-21T03:27:48.152 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-cephfs-2:19.3.0-5120.g18432218.el9.x86_6 23/165 2024-09-21T03:27:48.152 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-rados-2:19.3.0-5120.g18432218.el9.x86_64 24/165 2024-09-21T03:27:48.152 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-rbd-2:19.3.0-5120.g18432218.el9.x86_64 25/165 2024-09-21T03:27:48.152 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-rgw-2:19.3.0-5120.g18432218.el9.x86_64 26/165 2024-09-21T03:27:48.153 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : rbd-fuse-2:19.3.0-5120.g18432218.el9.x86_64 27/165 2024-09-21T03:27:48.153 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : rbd-mirror-2:19.3.0-5120.g18432218.el9.x86_64 28/165 2024-09-21T03:27:48.153 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : rbd-nbd-2:19.3.0-5120.g18432218.el9.x86_64 29/165 2024-09-21T03:27:48.164 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : ceph-grafana-dashboards-2:19.3.0-5120.g18432218. 30/165 2024-09-21T03:27:48.164 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : ceph-mgr-cephadm-2:19.3.0-5120.g18432218.el9.noa 31/165 2024-09-21T03:27:48.164 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : ceph-mgr-dashboard-2:19.3.0-5120.g18432218.el9.n 32/165 2024-09-21T03:27:48.164 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.3.0-5120.g184 33/165 2024-09-21T03:27:48.164 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : ceph-mgr-modules-core-2:19.3.0-5120.g18432218.el 34/165 2024-09-21T03:27:48.164 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : ceph-mgr-rook-2:19.3.0-5120.g18432218.el9.noarch 35/165 2024-09-21T03:27:48.164 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : ceph-prometheus-alerts-2:19.3.0-5120.g18432218.e 36/165 2024-09-21T03:27:48.165 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : ceph-volume-2:19.3.0-5120.g18432218.el9.noarch 37/165 2024-09-21T03:27:48.165 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : cephadm-2:19.3.0-5120.g18432218.el9.noarch 38/165 2024-09-21T03:27:48.165 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : cephfs-top-2:19.3.0-5120.g18432218.el9.noarch 39/165 2024-09-21T03:27:48.165 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 40/165 2024-09-21T03:27:48.165 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 41/165 2024-09-21T03:27:48.165 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 42/165 2024-09-21T03:27:48.165 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/165 2024-09-21T03:27:48.165 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/165 2024-09-21T03:27:48.165 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/165 2024-09-21T03:27:48.165 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/165 2024-09-21T03:27:48.165 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/165 2024-09-21T03:27:48.165 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 48/165 2024-09-21T03:27:48.165 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/165 2024-09-21T03:27:48.165 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/165 2024-09-21T03:27:48.165 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/165 2024-09-21T03:27:48.166 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/165 2024-09-21T03:27:48.166 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/165 2024-09-21T03:27:48.166 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/165 2024-09-21T03:27:48.166 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/165 2024-09-21T03:27:48.166 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/165 2024-09-21T03:27:48.166 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/165 2024-09-21T03:27:48.166 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : libnbd-1.20.2-2.el9.x86_64 58/165 2024-09-21T03:27:48.166 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 59/165 2024-09-21T03:27:48.166 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 60/165 2024-09-21T03:27:48.166 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 61/165 2024-09-21T03:27:48.166 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : libtool-2.4.6-46.el9.x86_64 62/165 2024-09-21T03:27:48.166 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : libuuid-devel-2.37.4-20.el9.x86_64 63/165 2024-09-21T03:27:48.166 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 64/165 2024-09-21T03:27:48.167 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 65/165 2024-09-21T03:27:48.167 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : lua-5.4.4-4.el9.x86_64 66/165 2024-09-21T03:27:48.167 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 67/165 2024-09-21T03:27:48.167 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 68/165 2024-09-21T03:27:48.167 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : openssl-devel-1:3.2.2-6.el9.x86_64 69/165 2024-09-21T03:27:48.167 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : postgresql-13.16-1.el9.x86_64 70/165 2024-09-21T03:27:48.167 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : postgresql-contrib-13.16-1.el9.x86_64 71/165 2024-09-21T03:27:48.167 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : postgresql-private-libs-13.16-1.el9.x86_64 72/165 2024-09-21T03:27:48.167 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : postgresql-server-13.16-1.el9.x86_64 73/165 2024-09-21T03:27:48.167 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 74/165 2024-09-21T03:27:48.167 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 75/165 2024-09-21T03:27:48.167 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 76/165 2024-09-21T03:27:48.167 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 77/165 2024-09-21T03:27:48.168 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 78/165 2024-09-21T03:27:48.168 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 79/165 2024-09-21T03:27:48.168 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 80/165 2024-09-21T03:27:48.168 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 81/165 2024-09-21T03:27:48.168 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 82/165 2024-09-21T03:27:48.168 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 83/165 2024-09-21T03:27:48.168 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 84/165 2024-09-21T03:27:48.168 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 85/165 2024-09-21T03:27:48.168 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 86/165 2024-09-21T03:27:48.168 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 87/165 2024-09-21T03:27:48.168 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 88/165 2024-09-21T03:27:48.168 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 89/165 2024-09-21T03:27:48.168 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 90/165 2024-09-21T03:27:48.168 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 91/165 2024-09-21T03:27:48.168 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 92/165 2024-09-21T03:27:48.169 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 93/165 2024-09-21T03:27:48.169 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 94/165 2024-09-21T03:27:48.169 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 95/165 2024-09-21T03:27:48.169 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 96/165 2024-09-21T03:27:48.169 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 97/165 2024-09-21T03:27:48.169 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 98/165 2024-09-21T03:27:48.169 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 99/165 2024-09-21T03:27:48.169 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 100/165 2024-09-21T03:27:48.169 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 101/165 2024-09-21T03:27:48.169 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : xfsprogs-devel-6.4.0-4.el9.x86_64 102/165 2024-09-21T03:27:48.169 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 103/165 2024-09-21T03:27:48.169 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 104/165 2024-09-21T03:27:48.169 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 105/165 2024-09-21T03:27:48.169 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 106/165 2024-09-21T03:27:48.169 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 107/165 2024-09-21T03:27:48.170 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : gperftools-libs-2.9.1-3.el9.x86_64 108/165 2024-09-21T03:27:48.170 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 109/165 2024-09-21T03:27:48.170 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 110/165 2024-09-21T03:27:48.170 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 111/165 2024-09-21T03:27:48.170 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 112/165 2024-09-21T03:27:48.170 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 113/165 2024-09-21T03:27:48.170 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 114/165 2024-09-21T03:27:48.170 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 115/165 2024-09-21T03:27:48.170 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 116/165 2024-09-21T03:27:48.170 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 117/165 2024-09-21T03:27:48.170 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 118/165 2024-09-21T03:27:48.170 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 119/165 2024-09-21T03:27:48.170 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 120/165 2024-09-21T03:27:48.171 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 121/165 2024-09-21T03:27:48.171 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 122/165 2024-09-21T03:27:48.171 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 123/165 2024-09-21T03:27:48.171 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 124/165 2024-09-21T03:27:48.171 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 125/165 2024-09-21T03:27:48.171 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 126/165 2024-09-21T03:27:48.171 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 127/165 2024-09-21T03:27:48.171 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 128/165 2024-09-21T03:27:48.171 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 129/165 2024-09-21T03:27:48.171 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 130/165 2024-09-21T03:27:48.171 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 131/165 2024-09-21T03:27:48.171 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 132/165 2024-09-21T03:27:48.171 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 133/165 2024-09-21T03:27:48.171 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 134/165 2024-09-21T03:27:48.172 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 135/165 2024-09-21T03:27:48.172 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 136/165 2024-09-21T03:27:48.172 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 137/165 2024-09-21T03:27:48.172 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 138/165 2024-09-21T03:27:48.172 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 139/165 2024-09-21T03:27:48.172 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 140/165 2024-09-21T03:27:48.172 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 141/165 2024-09-21T03:27:48.172 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 142/165 2024-09-21T03:27:48.172 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 143/165 2024-09-21T03:27:48.172 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 144/165 2024-09-21T03:27:48.172 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 145/165 2024-09-21T03:27:48.172 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 146/165 2024-09-21T03:27:48.172 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 147/165 2024-09-21T03:27:48.172 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : re2-1:20211101-20.el9.x86_64 148/165 2024-09-21T03:27:48.173 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 149/165 2024-09-21T03:27:48.173 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : librados2-2:19.3.0-5120.g18432218.el9.x86_64 150/165 2024-09-21T03:27:48.173 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 151/165 2024-09-21T03:27:48.173 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : librbd1-2:19.3.0-5120.g18432218.el9.x86_64 152/165 2024-09-21T03:27:48.173 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 153/165 2024-09-21T03:27:48.173 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : NetworkManager-1:1.51.0-1.el9.x86_64 154/165 2024-09-21T03:27:48.173 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : NetworkManager-1:1.48.8-1.el9.x86_64 155/165 2024-09-21T03:27:48.173 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : NetworkManager-libnm-1:1.51.0-1.el9.x86_64 156/165 2024-09-21T03:27:48.173 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 157/165 2024-09-21T03:27:48.173 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : NetworkManager-team-1:1.51.0-1.el9.x86_64 158/165 2024-09-21T03:27:48.173 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : NetworkManager-team-1:1.48.8-1.el9.x86_64 159/165 2024-09-21T03:27:48.173 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : NetworkManager-tui-1:1.51.0-1.el9.x86_64 160/165 2024-09-21T03:27:48.173 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : NetworkManager-tui-1:1.48.8-1.el9.x86_64 161/165 2024-09-21T03:27:48.173 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : openssl-1:3.2.2-6.el9.x86_64 162/165 2024-09-21T03:27:48.173 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : openssl-1:3.2.2-4.el9.x86_64 163/165 2024-09-21T03:27:48.174 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : openssl-libs-1:3.2.2-6.el9.x86_64 164/165 2024-09-21T03:27:50.802 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : openssl-libs-1:3.2.2-4.el9.x86_64 165/165 2024-09-21T03:27:50.803 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-09-21T03:27:50.803 INFO:teuthology.orchestra.run.smithi089.stdout:Upgraded: 2024-09-21T03:27:50.803 INFO:teuthology.orchestra.run.smithi089.stdout: NetworkManager-1:1.51.0-1.el9.x86_64 2024-09-21T03:27:50.803 INFO:teuthology.orchestra.run.smithi089.stdout: NetworkManager-libnm-1:1.51.0-1.el9.x86_64 2024-09-21T03:27:50.803 INFO:teuthology.orchestra.run.smithi089.stdout: NetworkManager-team-1:1.51.0-1.el9.x86_64 2024-09-21T03:27:50.803 INFO:teuthology.orchestra.run.smithi089.stdout: NetworkManager-tui-1:1.51.0-1.el9.x86_64 2024-09-21T03:27:50.803 INFO:teuthology.orchestra.run.smithi089.stdout: librados2-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:27:50.803 INFO:teuthology.orchestra.run.smithi089.stdout: librbd1-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:27:50.803 INFO:teuthology.orchestra.run.smithi089.stdout: openssl-1:3.2.2-6.el9.x86_64 2024-09-21T03:27:50.803 INFO:teuthology.orchestra.run.smithi089.stdout: openssl-libs-1:3.2.2-6.el9.x86_64 2024-09-21T03:27:50.803 INFO:teuthology.orchestra.run.smithi089.stdout:Installed: 2024-09-21T03:27:50.804 INFO:teuthology.orchestra.run.smithi089.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-09-21T03:27:50.804 INFO:teuthology.orchestra.run.smithi089.stdout: automake-1.16.2-8.el9.noarch 2024-09-21T03:27:50.804 INFO:teuthology.orchestra.run.smithi089.stdout: bison-3.7.4-5.el9.x86_64 2024-09-21T03:27:50.804 INFO:teuthology.orchestra.run.smithi089.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-21T03:27:50.804 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:27:50.804 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-base-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:27:50.804 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-common-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:27:50.804 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-fuse-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:27:50.804 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-grafana-dashboards-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T03:27:50.804 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-immutable-object-cache-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:27:50.804 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-mds-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:27:50.804 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-mgr-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:27:50.804 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-mgr-cephadm-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T03:27:50.804 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-mgr-dashboard-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T03:27:50.805 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-mgr-diskprediction-local-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T03:27:50.805 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-mgr-modules-core-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T03:27:50.805 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-mgr-rook-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T03:27:50.805 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-mon-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:27:50.805 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-osd-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:27:50.805 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-prometheus-alerts-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T03:27:50.805 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-radosgw-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:27:50.805 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-selinux-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:27:50.805 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-test-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:27:50.805 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-volume-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T03:27:50.805 INFO:teuthology.orchestra.run.smithi089.stdout: cephadm-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T03:27:50.805 INFO:teuthology.orchestra.run.smithi089.stdout: cephfs-mirror-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:27:50.805 INFO:teuthology.orchestra.run.smithi089.stdout: cephfs-shell-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:27:50.805 INFO:teuthology.orchestra.run.smithi089.stdout: cephfs-top-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T03:27:50.806 INFO:teuthology.orchestra.run.smithi089.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-09-21T03:27:50.806 INFO:teuthology.orchestra.run.smithi089.stdout: flex-2.6.4-9.el9.x86_64 2024-09-21T03:27:50.806 INFO:teuthology.orchestra.run.smithi089.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-21T03:27:50.806 INFO:teuthology.orchestra.run.smithi089.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-21T03:27:50.806 INFO:teuthology.orchestra.run.smithi089.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-21T03:27:50.806 INFO:teuthology.orchestra.run.smithi089.stdout: fuse-2.9.9-16.el9.x86_64 2024-09-21T03:27:50.806 INFO:teuthology.orchestra.run.smithi089.stdout: gperftools-libs-2.9.1-3.el9.x86_64 2024-09-21T03:27:50.806 INFO:teuthology.orchestra.run.smithi089.stdout: grpc-data-1.46.7-10.el9.noarch 2024-09-21T03:27:50.806 INFO:teuthology.orchestra.run.smithi089.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-09-21T03:27:50.806 INFO:teuthology.orchestra.run.smithi089.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-21T03:27:50.806 INFO:teuthology.orchestra.run.smithi089.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-21T03:27:50.806 INFO:teuthology.orchestra.run.smithi089.stdout: libcephfs-devel-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:27:50.806 INFO:teuthology.orchestra.run.smithi089.stdout: libcephfs2-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:27:50.807 INFO:teuthology.orchestra.run.smithi089.stdout: libcephsqlite-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:27:50.807 INFO:teuthology.orchestra.run.smithi089.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-21T03:27:50.807 INFO:teuthology.orchestra.run.smithi089.stdout: libnbd-1.20.2-2.el9.x86_64 2024-09-21T03:27:50.807 INFO:teuthology.orchestra.run.smithi089.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-21T03:27:50.807 INFO:teuthology.orchestra.run.smithi089.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-21T03:27:50.807 INFO:teuthology.orchestra.run.smithi089.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-21T03:27:50.807 INFO:teuthology.orchestra.run.smithi089.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-21T03:27:50.807 INFO:teuthology.orchestra.run.smithi089.stdout: librados-devel-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:27:50.807 INFO:teuthology.orchestra.run.smithi089.stdout: libradosstriper1-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:27:50.807 INFO:teuthology.orchestra.run.smithi089.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-21T03:27:50.807 INFO:teuthology.orchestra.run.smithi089.stdout: librgw2-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:27:50.807 INFO:teuthology.orchestra.run.smithi089.stdout: libtool-2.4.6-46.el9.x86_64 2024-09-21T03:27:50.807 INFO:teuthology.orchestra.run.smithi089.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-21T03:27:50.807 INFO:teuthology.orchestra.run.smithi089.stdout: libuuid-devel-2.37.4-20.el9.x86_64 2024-09-21T03:27:50.808 INFO:teuthology.orchestra.run.smithi089.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-09-21T03:27:50.808 INFO:teuthology.orchestra.run.smithi089.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-21T03:27:50.808 INFO:teuthology.orchestra.run.smithi089.stdout: lua-5.4.4-4.el9.x86_64 2024-09-21T03:27:50.808 INFO:teuthology.orchestra.run.smithi089.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-09-21T03:27:50.808 INFO:teuthology.orchestra.run.smithi089.stdout: luarocks-3.9.2-5.el9.noarch 2024-09-21T03:27:50.808 INFO:teuthology.orchestra.run.smithi089.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-21T03:27:50.808 INFO:teuthology.orchestra.run.smithi089.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-21T03:27:50.808 INFO:teuthology.orchestra.run.smithi089.stdout: openssl-devel-1:3.2.2-6.el9.x86_64 2024-09-21T03:27:50.808 INFO:teuthology.orchestra.run.smithi089.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-21T03:27:50.808 INFO:teuthology.orchestra.run.smithi089.stdout: postgresql-13.16-1.el9.x86_64 2024-09-21T03:27:50.808 INFO:teuthology.orchestra.run.smithi089.stdout: postgresql-contrib-13.16-1.el9.x86_64 2024-09-21T03:27:50.808 INFO:teuthology.orchestra.run.smithi089.stdout: postgresql-private-libs-13.16-1.el9.x86_64 2024-09-21T03:27:50.808 INFO:teuthology.orchestra.run.smithi089.stdout: postgresql-server-13.16-1.el9.x86_64 2024-09-21T03:27:50.808 INFO:teuthology.orchestra.run.smithi089.stdout: protobuf-3.14.0-13.el9.x86_64 2024-09-21T03:27:50.809 INFO:teuthology.orchestra.run.smithi089.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-09-21T03:27:50.809 INFO:teuthology.orchestra.run.smithi089.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-21T03:27:50.809 INFO:teuthology.orchestra.run.smithi089.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-09-21T03:27:50.809 INFO:teuthology.orchestra.run.smithi089.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-21T03:27:50.809 INFO:teuthology.orchestra.run.smithi089.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-21T03:27:50.809 INFO:teuthology.orchestra.run.smithi089.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-21T03:27:50.809 INFO:teuthology.orchestra.run.smithi089.stdout: python3-ceph-argparse-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:27:50.809 INFO:teuthology.orchestra.run.smithi089.stdout: python3-ceph-common-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:27:50.809 INFO:teuthology.orchestra.run.smithi089.stdout: python3-cephfs-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:27:50.809 INFO:teuthology.orchestra.run.smithi089.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-21T03:27:50.809 INFO:teuthology.orchestra.run.smithi089.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-21T03:27:50.810 INFO:teuthology.orchestra.run.smithi089.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-21T03:27:50.810 INFO:teuthology.orchestra.run.smithi089.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-21T03:27:50.810 INFO:teuthology.orchestra.run.smithi089.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-09-21T03:27:50.810 INFO:teuthology.orchestra.run.smithi089.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-09-21T03:27:50.810 INFO:teuthology.orchestra.run.smithi089.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-21T03:27:50.810 INFO:teuthology.orchestra.run.smithi089.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-21T03:27:50.810 INFO:teuthology.orchestra.run.smithi089.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-21T03:27:50.810 INFO:teuthology.orchestra.run.smithi089.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-09-21T03:27:50.810 INFO:teuthology.orchestra.run.smithi089.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-09-21T03:27:50.810 INFO:teuthology.orchestra.run.smithi089.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-21T03:27:50.810 INFO:teuthology.orchestra.run.smithi089.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-21T03:27:50.810 INFO:teuthology.orchestra.run.smithi089.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-21T03:27:50.810 INFO:teuthology.orchestra.run.smithi089.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-21T03:27:50.811 INFO:teuthology.orchestra.run.smithi089.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-21T03:27:50.811 INFO:teuthology.orchestra.run.smithi089.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-21T03:27:50.811 INFO:teuthology.orchestra.run.smithi089.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-21T03:27:50.811 INFO:teuthology.orchestra.run.smithi089.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-21T03:27:50.811 INFO:teuthology.orchestra.run.smithi089.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-21T03:27:50.811 INFO:teuthology.orchestra.run.smithi089.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-21T03:27:50.811 INFO:teuthology.orchestra.run.smithi089.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-21T03:27:50.811 INFO:teuthology.orchestra.run.smithi089.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-21T03:27:50.811 INFO:teuthology.orchestra.run.smithi089.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-21T03:27:50.812 INFO:teuthology.orchestra.run.smithi089.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-21T03:27:50.812 INFO:teuthology.orchestra.run.smithi089.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-21T03:27:50.812 INFO:teuthology.orchestra.run.smithi089.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-21T03:27:50.812 INFO:teuthology.orchestra.run.smithi089.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-21T03:27:50.812 INFO:teuthology.orchestra.run.smithi089.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-21T03:27:50.812 INFO:teuthology.orchestra.run.smithi089.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-21T03:27:50.812 INFO:teuthology.orchestra.run.smithi089.stdout: python3-ply-3.11-14.el9.noarch 2024-09-21T03:27:50.812 INFO:teuthology.orchestra.run.smithi089.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-21T03:27:50.812 INFO:teuthology.orchestra.run.smithi089.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-21T03:27:50.812 INFO:teuthology.orchestra.run.smithi089.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-09-21T03:27:50.812 INFO:teuthology.orchestra.run.smithi089.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-21T03:27:50.812 INFO:teuthology.orchestra.run.smithi089.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-21T03:27:50.812 INFO:teuthology.orchestra.run.smithi089.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-21T03:27:50.813 INFO:teuthology.orchestra.run.smithi089.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-21T03:27:50.813 INFO:teuthology.orchestra.run.smithi089.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-09-21T03:27:50.813 INFO:teuthology.orchestra.run.smithi089.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-21T03:27:50.813 INFO:teuthology.orchestra.run.smithi089.stdout: python3-rados-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:27:50.813 INFO:teuthology.orchestra.run.smithi089.stdout: python3-rbd-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:27:50.813 INFO:teuthology.orchestra.run.smithi089.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-21T03:27:50.813 INFO:teuthology.orchestra.run.smithi089.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-21T03:27:50.813 INFO:teuthology.orchestra.run.smithi089.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-21T03:27:50.813 INFO:teuthology.orchestra.run.smithi089.stdout: python3-rgw-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:27:50.813 INFO:teuthology.orchestra.run.smithi089.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-21T03:27:50.813 INFO:teuthology.orchestra.run.smithi089.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-21T03:27:50.813 INFO:teuthology.orchestra.run.smithi089.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-21T03:27:50.813 INFO:teuthology.orchestra.run.smithi089.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-21T03:27:50.813 INFO:teuthology.orchestra.run.smithi089.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-21T03:27:50.814 INFO:teuthology.orchestra.run.smithi089.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-21T03:27:50.814 INFO:teuthology.orchestra.run.smithi089.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-21T03:27:50.814 INFO:teuthology.orchestra.run.smithi089.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-09-21T03:27:50.814 INFO:teuthology.orchestra.run.smithi089.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-21T03:27:50.814 INFO:teuthology.orchestra.run.smithi089.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-21T03:27:50.814 INFO:teuthology.orchestra.run.smithi089.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-21T03:27:50.814 INFO:teuthology.orchestra.run.smithi089.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-21T03:27:50.814 INFO:teuthology.orchestra.run.smithi089.stdout: qatlib-24.02.0-1.el9.x86_64 2024-09-21T03:27:50.814 INFO:teuthology.orchestra.run.smithi089.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-09-21T03:27:50.814 INFO:teuthology.orchestra.run.smithi089.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-09-21T03:27:50.814 INFO:teuthology.orchestra.run.smithi089.stdout: rbd-fuse-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:27:50.814 INFO:teuthology.orchestra.run.smithi089.stdout: rbd-mirror-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:27:50.815 INFO:teuthology.orchestra.run.smithi089.stdout: rbd-nbd-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:27:50.815 INFO:teuthology.orchestra.run.smithi089.stdout: re2-1:20211101-20.el9.x86_64 2024-09-21T03:27:50.815 INFO:teuthology.orchestra.run.smithi089.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-21T03:27:50.815 INFO:teuthology.orchestra.run.smithi089.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-21T03:27:50.815 INFO:teuthology.orchestra.run.smithi089.stdout: uuid-1.6.2-55.el9.x86_64 2024-09-21T03:27:50.815 INFO:teuthology.orchestra.run.smithi089.stdout: xfsprogs-devel-6.4.0-4.el9.x86_64 2024-09-21T03:27:50.815 INFO:teuthology.orchestra.run.smithi089.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-21T03:27:50.815 INFO:teuthology.orchestra.run.smithi089.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-09-21T03:27:50.815 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-09-21T03:27:50.816 INFO:teuthology.orchestra.run.smithi089.stdout:Complete! 2024-09-21T03:27:51.245 DEBUG:teuthology.parallel:result is None 2024-09-21T03:27:51.245 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=1843221895adccfb3b692060251dd6a0183bc410 2024-09-21T03:27:51.392 DEBUG:teuthology.orchestra.run.smithi089:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-21T03:27:51.413 INFO:teuthology.orchestra.run.smithi089.stdout:19.3.0-5120.g18432218.el9 2024-09-21T03:27:51.414 INFO:teuthology.packaging:The installed version of ceph is 19.3.0-5120.g18432218.el9 2024-09-21T03:27:51.414 INFO:teuthology.task.install:The correct ceph version 19.3.0-5120.g18432218 is installed. 2024-09-21T03:27:51.416 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=1843221895adccfb3b692060251dd6a0183bc410 2024-09-21T03:27:51.558 DEBUG:teuthology.orchestra.run.smithi110:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-21T03:27:51.585 INFO:teuthology.orchestra.run.smithi110.stdout:19.3.0-5120.g18432218.el9 2024-09-21T03:27:51.585 INFO:teuthology.packaging:The installed version of ceph is 19.3.0-5120.g18432218.el9 2024-09-21T03:27:51.585 INFO:teuthology.task.install:The correct ceph version 19.3.0-5120.g18432218 is installed. 2024-09-21T03:27:51.587 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=1843221895adccfb3b692060251dd6a0183bc410 2024-09-21T03:27:51.726 DEBUG:teuthology.orchestra.run.smithi114:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-21T03:27:51.752 INFO:teuthology.orchestra.run.smithi114.stdout:19.3.0-5120.g18432218.el9 2024-09-21T03:27:51.753 INFO:teuthology.packaging:The installed version of ceph is 19.3.0-5120.g18432218.el9 2024-09-21T03:27:51.753 INFO:teuthology.task.install:The correct ceph version 19.3.0-5120.g18432218 is installed. 2024-09-21T03:27:51.755 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-09-21T03:27:51.755 DEBUG:teuthology.orchestra.run.smithi089:> set -ex 2024-09-21T03:27:51.755 DEBUG:teuthology.orchestra.run.smithi089:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-21T03:27:51.782 DEBUG:teuthology.orchestra.run.smithi110:> set -ex 2024-09-21T03:27:51.782 DEBUG:teuthology.orchestra.run.smithi110:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-21T03:27:51.816 DEBUG:teuthology.orchestra.run.smithi114:> set -ex 2024-09-21T03:27:51.816 DEBUG:teuthology.orchestra.run.smithi114:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-21T03:27:51.848 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-09-21T03:27:51.848 DEBUG:teuthology.orchestra.run.smithi089:> set -ex 2024-09-21T03:27:51.848 DEBUG:teuthology.orchestra.run.smithi089:> sudo dd of=/usr/bin/daemon-helper 2024-09-21T03:27:51.873 DEBUG:teuthology.orchestra.run.smithi089:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-21T03:27:51.937 DEBUG:teuthology.orchestra.run.smithi110:> set -ex 2024-09-21T03:27:51.937 DEBUG:teuthology.orchestra.run.smithi110:> sudo dd of=/usr/bin/daemon-helper 2024-09-21T03:27:51.967 DEBUG:teuthology.orchestra.run.smithi110:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-21T03:27:52.035 DEBUG:teuthology.orchestra.run.smithi114:> set -ex 2024-09-21T03:27:52.035 DEBUG:teuthology.orchestra.run.smithi114:> sudo dd of=/usr/bin/daemon-helper 2024-09-21T03:27:52.064 DEBUG:teuthology.orchestra.run.smithi114:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-21T03:27:52.132 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-09-21T03:27:52.132 DEBUG:teuthology.orchestra.run.smithi089:> set -ex 2024-09-21T03:27:52.132 DEBUG:teuthology.orchestra.run.smithi089:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-21T03:27:52.156 DEBUG:teuthology.orchestra.run.smithi089:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-21T03:27:52.221 DEBUG:teuthology.orchestra.run.smithi110:> set -ex 2024-09-21T03:27:52.221 DEBUG:teuthology.orchestra.run.smithi110:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-21T03:27:52.250 DEBUG:teuthology.orchestra.run.smithi110:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-21T03:27:52.318 DEBUG:teuthology.orchestra.run.smithi114:> set -ex 2024-09-21T03:27:52.318 DEBUG:teuthology.orchestra.run.smithi114:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-21T03:27:52.346 DEBUG:teuthology.orchestra.run.smithi114:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-21T03:27:52.415 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-09-21T03:27:52.415 DEBUG:teuthology.orchestra.run.smithi089:> set -ex 2024-09-21T03:27:52.415 DEBUG:teuthology.orchestra.run.smithi089:> sudo dd of=/usr/bin/stdin-killer 2024-09-21T03:27:52.440 DEBUG:teuthology.orchestra.run.smithi089:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-21T03:27:52.505 DEBUG:teuthology.orchestra.run.smithi110:> set -ex 2024-09-21T03:27:52.505 DEBUG:teuthology.orchestra.run.smithi110:> sudo dd of=/usr/bin/stdin-killer 2024-09-21T03:27:52.530 DEBUG:teuthology.orchestra.run.smithi110:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-21T03:27:52.595 DEBUG:teuthology.orchestra.run.smithi114:> set -ex 2024-09-21T03:27:52.595 DEBUG:teuthology.orchestra.run.smithi114:> sudo dd of=/usr/bin/stdin-killer 2024-09-21T03:27:52.625 DEBUG:teuthology.orchestra.run.smithi114:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-21T03:27:52.693 INFO:teuthology.run_tasks:Running task cephadm... 2024-09-21T03:27:52.789 INFO:tasks.cephadm:Config: {'roleless': False, 'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd'], 'max_mds': 3, 'session_timeout': 300, 'standby_replay': True}, 'cluster-conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 2, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'client mount timeout': 30, 'debug client': 20, 'debug mgr': 20, 'debug ms': 1, 'mon warn on pool no app': False}, 'mon': {'mon op complaint time': 120}, 'osd': {'osd op complaint time': 180}}, 'conf': {'mds': {'mds_export_ephemeral_random_max': 0.1}, 'mgr': {'cephsqlite lock renewal timeout': 900000, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd shutdown pgref assert': True, 'osd_max_omap_entries_per_request': 10000}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'fs.*is degraded', 'filesystem is degraded', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'filesystem is offline', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'insufficient standby MDS daemons available', 'MDS_UP_LESS_THAN_MAX', 'online, but wants', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'Degraded data redundancy'], 'sha1': '1843221895adccfb3b692060251dd6a0183bc410', 'subvols': {'create': 2}} 2024-09-21T03:27:52.789 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 2024-09-21T03:27:52.790 INFO:tasks.cephadm:Cluster fsid is 7c374098-77c9-11ef-baf5-efdc52797490 2024-09-21T03:27:52.790 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-09-21T03:27:52.790 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.89', 'mon.b': '172.21.15.110', 'mon.c': '172.21.15.114'} 2024-09-21T03:27:52.790 INFO:tasks.cephadm:First mon is mon.a on smithi089 2024-09-21T03:27:52.790 INFO:tasks.cephadm:First mgr is x 2024-09-21T03:27:52.790 INFO:tasks.cephadm:Normalizing hostnames... 2024-09-21T03:27:52.791 DEBUG:teuthology.orchestra.run.smithi089:> sudo hostname $(hostname -s) 2024-09-21T03:27:52.816 DEBUG:teuthology.orchestra.run.smithi110:> sudo hostname $(hostname -s) 2024-09-21T03:27:52.841 DEBUG:teuthology.orchestra.run.smithi114:> sudo hostname $(hostname -s) 2024-09-21T03:27:52.870 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra 2024-09-21T03:27:52.871 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=1843221895adccfb3b692060251dd6a0183bc410 2024-09-21T03:27:53.008 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': '1843221895adccfb3b692060251dd6a0183bc410', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-new-build/ARCH=arm64,AVAILABLE_ARCH=arm64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/83190/', 'root_build_cause': 'SCMTRIGGER', 'version': '19.3.0-5120-g18432218', 'node_name': '172.21.4.75+confusa13', 'job_name': 'ceph-dev-new-build/ARCH=arm64,AVAILABLE_ARCH=arm64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '19.3.0-5120.g18432218'}, 'url': 'https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-09-21 00:04:17.484514', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'wip-pdonnell-testing-20240920.212106-debug', 'chacra_url': 'https://4.chacra.ceph.com/repos/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/flavors/default/', 'archs': ['x86_64', 'source', 'arm64'], 'distro': 'centos'}, {'status': 'ready', 'sha1': '1843221895adccfb3b692060251dd6a0183bc410', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/83190/', 'root_build_cause': 'SCMTRIGGER', 'version': '19.3.0-5120-g18432218', 'node_name': '172.21.2.22+braggi22', 'job_name': 'ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '19.3.0-5120.g18432218'}, 'url': 'https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-09-20 22:54:16.499090', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'wip-pdonnell-testing-20240920.212106-debug', 'chacra_url': 'https://3.chacra.ceph.com/repos/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/flavors/default/', 'archs': ['source', 'x86_64'], 'distro': 'centos'}] 2024-09-21T03:27:53.155 INFO:tasks.util.chacra:got chacra host 4.chacra.ceph.com, ref wip-pdonnell-testing-20240920.212106-debug, sha1 1843221895adccfb3b692060251dd6a0183bc410 from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F9%2Fx86_64&flavor=default&sha1=1843221895adccfb3b692060251dd6a0183bc410 2024-09-21T03:27:53.156 INFO:tasks.cephadm:Discovered cachra url: https://4.chacra.ceph.com/binaries/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/x86_64/flavors/default/cephadm 2024-09-21T03:27:53.156 INFO:tasks.cephadm:Downloading cephadm from url: https://4.chacra.ceph.com/binaries/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/x86_64/flavors/default/cephadm 2024-09-21T03:27:53.156 DEBUG:teuthology.orchestra.run.smithi089:> curl --silent -L https://4.chacra.ceph.com/binaries/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-09-21T03:27:53.683 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r--. 1 ubuntu ubuntu 804283 Sep 21 03:27 /home/ubuntu/cephtest/cephadm 2024-09-21T03:27:53.683 DEBUG:teuthology.orchestra.run.smithi110:> curl --silent -L https://4.chacra.ceph.com/binaries/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-09-21T03:27:54.212 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r--. 1 ubuntu ubuntu 804283 Sep 21 03:27 /home/ubuntu/cephtest/cephadm 2024-09-21T03:27:54.212 DEBUG:teuthology.orchestra.run.smithi114:> curl --silent -L https://4.chacra.ceph.com/binaries/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-09-21T03:27:54.746 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-r--r--. 1 ubuntu ubuntu 804283 Sep 21 03:27 /home/ubuntu/cephtest/cephadm 2024-09-21T03:27:54.746 DEBUG:teuthology.orchestra.run.smithi089:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-09-21T03:27:54.766 DEBUG:teuthology.orchestra.run.smithi110:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-09-21T03:27:54.784 DEBUG:teuthology.orchestra.run.smithi114:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-09-21T03:27:54.820 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 on all hosts... 2024-09-21T03:27:54.820 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 pull 2024-09-21T03:27:54.823 DEBUG:teuthology.orchestra.run.smithi110:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 pull 2024-09-21T03:27:54.826 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 pull 2024-09-21T03:27:56.417 INFO:teuthology.orchestra.run.smithi114.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410... 2024-09-21T03:27:56.482 INFO:teuthology.orchestra.run.smithi089.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410... 2024-09-21T03:27:56.501 INFO:teuthology.orchestra.run.smithi110.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410... 2024-09-21T03:28:33.686 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-21T03:28:33.686 INFO:teuthology.orchestra.run.smithi110.stdout: "ceph_version": "ceph version 19.3.0-5120-g18432218 (1843221895adccfb3b692060251dd6a0183bc410) squid (dev)", 2024-09-21T03:28:33.686 INFO:teuthology.orchestra.run.smithi110.stdout: "image_id": "ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977", 2024-09-21T03:28:33.686 INFO:teuthology.orchestra.run.smithi110.stdout: "repo_digests": [ 2024-09-21T03:28:33.686 INFO:teuthology.orchestra.run.smithi110.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262" 2024-09-21T03:28:33.686 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-21T03:28:33.686 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-21T03:28:47.967 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-09-21T03:28:47.967 INFO:teuthology.orchestra.run.smithi089.stdout: "ceph_version": "ceph version 19.3.0-5120-g18432218 (1843221895adccfb3b692060251dd6a0183bc410) squid (dev)", 2024-09-21T03:28:47.967 INFO:teuthology.orchestra.run.smithi089.stdout: "image_id": "ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977", 2024-09-21T03:28:47.967 INFO:teuthology.orchestra.run.smithi089.stdout: "repo_digests": [ 2024-09-21T03:28:47.967 INFO:teuthology.orchestra.run.smithi089.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262" 2024-09-21T03:28:47.967 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-21T03:28:47.967 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-09-21T03:28:48.016 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-21T03:28:48.016 INFO:teuthology.orchestra.run.smithi114.stdout: "ceph_version": "ceph version 19.3.0-5120-g18432218 (1843221895adccfb3b692060251dd6a0183bc410) squid (dev)", 2024-09-21T03:28:48.016 INFO:teuthology.orchestra.run.smithi114.stdout: "image_id": "ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977", 2024-09-21T03:28:48.016 INFO:teuthology.orchestra.run.smithi114.stdout: "repo_digests": [ 2024-09-21T03:28:48.016 INFO:teuthology.orchestra.run.smithi114.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262" 2024-09-21T03:28:48.016 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-21T03:28:48.016 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-21T03:28:48.042 DEBUG:teuthology.orchestra.run.smithi089:> sudo mkdir -p /etc/ceph 2024-09-21T03:28:48.069 DEBUG:teuthology.orchestra.run.smithi110:> sudo mkdir -p /etc/ceph 2024-09-21T03:28:48.102 DEBUG:teuthology.orchestra.run.smithi114:> sudo mkdir -p /etc/ceph 2024-09-21T03:28:48.129 DEBUG:teuthology.orchestra.run.smithi089:> sudo chmod 777 /etc/ceph 2024-09-21T03:28:48.153 DEBUG:teuthology.orchestra.run.smithi110:> sudo chmod 777 /etc/ceph 2024-09-21T03:28:48.182 DEBUG:teuthology.orchestra.run.smithi114:> sudo chmod 777 /etc/ceph 2024-09-21T03:28:48.207 INFO:tasks.cephadm:Writing seed config... 2024-09-21T03:28:48.208 INFO:tasks.cephadm: override: [mds] mds_export_ephemeral_random_max = 0.1 2024-09-21T03:28:48.208 INFO:tasks.cephadm: override: [mgr] cephsqlite lock renewal timeout = 900000 2024-09-21T03:28:48.208 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-09-21T03:28:48.208 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-09-21T03:28:48.208 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-09-21T03:28:48.208 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-09-21T03:28:48.208 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-09-21T03:28:48.208 INFO:tasks.cephadm: override: [mon] mon down mkfs grace = 300 2024-09-21T03:28:48.208 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2024-09-21T03:28:48.208 INFO:tasks.cephadm: override: [osd] bluestore compression mode = aggressive 2024-09-21T03:28:48.208 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2024-09-21T03:28:48.209 INFO:tasks.cephadm: override: [osd] debug bluefs = 20 2024-09-21T03:28:48.209 INFO:tasks.cephadm: override: [osd] debug bluestore = 20 2024-09-21T03:28:48.209 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-09-21T03:28:48.209 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-09-21T03:28:48.209 INFO:tasks.cephadm: override: [osd] debug rocksdb = 10 2024-09-21T03:28:48.209 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2024-09-21T03:28:48.209 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2024-09-21T03:28:48.209 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2024-09-21T03:28:48.209 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2024-09-21T03:28:48.209 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2024-09-21T03:28:48.209 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2024-09-21T03:28:48.209 INFO:tasks.cephadm: override: [osd] osd_max_omap_entries_per_request = 10000 2024-09-21T03:28:48.210 DEBUG:teuthology.orchestra.run.smithi089:> set -ex 2024-09-21T03:28:48.210 DEBUG:teuthology.orchestra.run.smithi089:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-09-21T03:28:48.225 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log to file = true mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = 7c374098-77c9-11ef-baf5-efdc52797490 [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true bluestore block size = 96636764160 bluestore compression mode = aggressive bluestore fsck on mount = True debug bluefs = 20 debug bluestore = 20 debug ms = 1 debug osd = 20 debug rocksdb = 10 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd objectstore = bluestore osd_max_omap_entries_per_request = 10000 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false cephsqlite lock renewal timeout = 900000 debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon down mkfs grace = 300 [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [mds] mds_export_ephemeral_random_max = 0.1 2024-09-21T03:28:48.225 DEBUG:teuthology.orchestra.run.smithi089:mon.a> sudo journalctl -f -n 0 -u ceph-7c374098-77c9-11ef-baf5-efdc52797490@mon.a.service 2024-09-21T03:28:48.267 DEBUG:teuthology.orchestra.run.smithi089:mgr.x> sudo journalctl -f -n 0 -u ceph-7c374098-77c9-11ef-baf5-efdc52797490@mgr.x.service 2024-09-21T03:28:48.310 INFO:tasks.cephadm:Bootstrapping... 2024-09-21T03:28:48.310 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 -v bootstrap --fsid 7c374098-77c9-11ef-baf5-efdc52797490 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id x --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.89 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-09-21T03:28:48.512 INFO:teuthology.orchestra.run.smithi089.stdout:-------------------------------------------------------------------------------- 2024-09-21T03:28:48.512 INFO:teuthology.orchestra.run.smithi089.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', '-v', 'bootstrap', '--fsid', '7c374098-77c9-11ef-baf5-efdc52797490', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'x', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.89', '--skip-admin-label'] 2024-09-21T03:28:48.512 INFO:teuthology.orchestra.run.smithi089.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-09-21T03:28:48.512 INFO:teuthology.orchestra.run.smithi089.stdout:Verifying podman|docker is present... 2024-09-21T03:28:48.536 INFO:teuthology.orchestra.run.smithi089.stdout:/bin/podman: stdout 5.2.2 2024-09-21T03:28:48.536 INFO:teuthology.orchestra.run.smithi089.stdout:Verifying lvm2 is present... 2024-09-21T03:28:48.536 INFO:teuthology.orchestra.run.smithi089.stdout:Verifying time synchronization is in place... 2024-09-21T03:28:48.544 INFO:teuthology.orchestra.run.smithi089.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-09-21T03:28:48.544 INFO:teuthology.orchestra.run.smithi089.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-09-21T03:28:48.550 INFO:teuthology.orchestra.run.smithi089.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-09-21T03:28:48.550 INFO:teuthology.orchestra.run.smithi089.stdout:systemctl: stdout inactive 2024-09-21T03:28:48.557 INFO:teuthology.orchestra.run.smithi089.stdout:systemctl: stdout enabled 2024-09-21T03:28:48.563 INFO:teuthology.orchestra.run.smithi089.stdout:systemctl: stdout active 2024-09-21T03:28:48.563 INFO:teuthology.orchestra.run.smithi089.stdout:Unit chronyd.service is enabled and running 2024-09-21T03:28:48.563 INFO:teuthology.orchestra.run.smithi089.stdout:Repeating the final host check... 2024-09-21T03:28:48.585 INFO:teuthology.orchestra.run.smithi089.stdout:/bin/podman: stdout 5.2.2 2024-09-21T03:28:48.585 INFO:teuthology.orchestra.run.smithi089.stdout:podman (/bin/podman) version 5.2.2 is present 2024-09-21T03:28:48.585 INFO:teuthology.orchestra.run.smithi089.stdout:systemctl is present 2024-09-21T03:28:48.585 INFO:teuthology.orchestra.run.smithi089.stdout:lvcreate is present 2024-09-21T03:28:48.591 INFO:teuthology.orchestra.run.smithi089.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-09-21T03:28:48.591 INFO:teuthology.orchestra.run.smithi089.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-09-21T03:28:48.597 INFO:teuthology.orchestra.run.smithi089.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-09-21T03:28:48.597 INFO:teuthology.orchestra.run.smithi089.stdout:systemctl: stdout inactive 2024-09-21T03:28:48.604 INFO:teuthology.orchestra.run.smithi089.stdout:systemctl: stdout enabled 2024-09-21T03:28:48.610 INFO:teuthology.orchestra.run.smithi089.stdout:systemctl: stdout active 2024-09-21T03:28:48.610 INFO:teuthology.orchestra.run.smithi089.stdout:Unit chronyd.service is enabled and running 2024-09-21T03:28:48.610 INFO:teuthology.orchestra.run.smithi089.stdout:Host looks OK 2024-09-21T03:28:48.611 INFO:teuthology.orchestra.run.smithi089.stdout:Cluster fsid: 7c374098-77c9-11ef-baf5-efdc52797490 2024-09-21T03:28:48.611 INFO:teuthology.orchestra.run.smithi089.stdout:Acquiring lock 139911826746000 on /run/cephadm/7c374098-77c9-11ef-baf5-efdc52797490.lock 2024-09-21T03:28:48.611 INFO:teuthology.orchestra.run.smithi089.stdout:Lock 139911826746000 acquired on /run/cephadm/7c374098-77c9-11ef-baf5-efdc52797490.lock 2024-09-21T03:28:48.611 INFO:teuthology.orchestra.run.smithi089.stdout:Verifying IP 172.21.15.89 port 3300 ... 2024-09-21T03:28:48.611 INFO:teuthology.orchestra.run.smithi089.stdout:Verifying IP 172.21.15.89 port 6789 ... 2024-09-21T03:28:48.611 INFO:teuthology.orchestra.run.smithi089.stdout:Base mon IP(s) is [172.21.15.89:3300, 172.21.15.89:6789], mon addrv is [v2:172.21.15.89:3300,v1:172.21.15.89:6789] 2024-09-21T03:28:48.614 INFO:teuthology.orchestra.run.smithi089.stdout:/sbin/ip: stdout default via 172.21.15.254 dev enp3s0f1 proto dhcp src 172.21.15.89 metric 100 2024-09-21T03:28:48.615 INFO:teuthology.orchestra.run.smithi089.stdout:/sbin/ip: stdout 172.21.0.0/20 dev enp3s0f1 proto kernel scope link src 172.21.15.89 metric 100 2024-09-21T03:28:48.617 INFO:teuthology.orchestra.run.smithi089.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-09-21T03:28:48.617 INFO:teuthology.orchestra.run.smithi089.stdout:/sbin/ip: stdout fe80::/64 dev enp3s0f1 proto kernel metric 1024 pref medium 2024-09-21T03:28:48.617 INFO:teuthology.orchestra.run.smithi089.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev enp3s0f1 proto ra metric 100 pref medium 2024-09-21T03:28:48.619 INFO:teuthology.orchestra.run.smithi089.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-09-21T03:28:48.619 INFO:teuthology.orchestra.run.smithi089.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-09-21T03:28:48.620 INFO:teuthology.orchestra.run.smithi089.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-09-21T03:28:48.620 INFO:teuthology.orchestra.run.smithi089.stdout:/sbin/ip: stdout 5: enp3s0f1: mtu 1500 state UP qlen 1000 2024-09-21T03:28:48.620 INFO:teuthology.orchestra.run.smithi089.stdout:/sbin/ip: stdout inet6 fe80::ec4:7aff:fe8f:d501/64 scope link noprefixroute 2024-09-21T03:28:48.620 INFO:teuthology.orchestra.run.smithi089.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-09-21T03:28:48.620 INFO:teuthology.orchestra.run.smithi089.stdout:Mon IP `172.21.15.89` is in CIDR network `172.21.0.0/20` 2024-09-21T03:28:48.620 INFO:teuthology.orchestra.run.smithi089.stdout:Mon IP `172.21.15.89` is in CIDR network `172.21.0.0/20` 2024-09-21T03:28:48.620 INFO:teuthology.orchestra.run.smithi089.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-09-21T03:28:48.621 INFO:teuthology.orchestra.run.smithi089.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-09-21T03:28:48.621 INFO:teuthology.orchestra.run.smithi089.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410... 2024-09-21T03:28:49.011 INFO:teuthology.orchestra.run.smithi089.stdout:/bin/podman: stdout ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 2024-09-21T03:28:49.011 INFO:teuthology.orchestra.run.smithi089.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410... 2024-09-21T03:28:49.011 INFO:teuthology.orchestra.run.smithi089.stdout:/bin/podman: stderr Getting image source signatures 2024-09-21T03:28:49.011 INFO:teuthology.orchestra.run.smithi089.stdout:/bin/podman: stderr Copying blob sha256:8425bc7408d91b180e7fdc62f23e7ae6603757f4f36e9969a4eaf9fd9416b764 2024-09-21T03:28:49.012 INFO:teuthology.orchestra.run.smithi089.stdout:/bin/podman: stderr Copying blob sha256:e8b54c863393b7a8216a71737771b73b16a8e43ec7acf927c7faa175c255e551 2024-09-21T03:28:49.012 INFO:teuthology.orchestra.run.smithi089.stdout:/bin/podman: stderr Copying config sha256:ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 2024-09-21T03:28:49.012 INFO:teuthology.orchestra.run.smithi089.stdout:/bin/podman: stderr Writing manifest to image destination 2024-09-21T03:28:49.899 INFO:teuthology.orchestra.run.smithi089.stdout:ceph: stdout ceph version 19.3.0-5120-g18432218 (1843221895adccfb3b692060251dd6a0183bc410) squid (dev) 2024-09-21T03:28:49.899 INFO:teuthology.orchestra.run.smithi089.stdout:Ceph version: ceph version 19.3.0-5120-g18432218 (1843221895adccfb3b692060251dd6a0183bc410) squid (dev) 2024-09-21T03:28:49.899 INFO:teuthology.orchestra.run.smithi089.stdout:Extracting ceph user uid/gid from container image... 2024-09-21T03:28:50.746 INFO:teuthology.orchestra.run.smithi089.stdout:stat: stdout 167 167 2024-09-21T03:28:50.747 INFO:teuthology.orchestra.run.smithi089.stdout:Creating initial keys... 2024-09-21T03:28:51.677 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph-authtool: stdout AQBzPe5m0exHAxAA0sFo7BMI++nsHSwBYUwkLw== 2024-09-21T03:28:52.524 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph-authtool: stdout AQBzPe5m3uc8OBAAuKAJjG4o5dC5eZ2i9h86Mg== 2024-09-21T03:28:53.414 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph-authtool: stdout AQB0Pe5m5HkSMxAAU1EYXgw1v607WQNkszfD5g== 2024-09-21T03:28:53.414 INFO:teuthology.orchestra.run.smithi089.stdout:Creating initial monmap... 2024-09-21T03:28:54.278 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-09-21T03:28:54.279 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = quincy 2024-09-21T03:28:54.279 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to 7c374098-77c9-11ef-baf5-efdc52797490 2024-09-21T03:28:54.279 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-09-21T03:28:54.279 INFO:teuthology.orchestra.run.smithi089.stdout:monmaptool for a [v2:172.21.15.89:3300,v1:172.21.15.89:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-09-21T03:28:54.279 INFO:teuthology.orchestra.run.smithi089.stdout:setting min_mon_release = quincy 2024-09-21T03:28:54.279 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/monmaptool: set fsid to 7c374098-77c9-11ef-baf5-efdc52797490 2024-09-21T03:28:54.279 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-09-21T03:28:54.279 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-09-21T03:28:54.279 INFO:teuthology.orchestra.run.smithi089.stdout:Creating mon... 2024-09-21T03:28:55.226 INFO:teuthology.orchestra.run.smithi089.stdout:create mon.a on 2024-09-21T03:28:55.442 INFO:teuthology.orchestra.run.smithi089.stdout:systemctl: stderr Removed "/etc/systemd/system/multi-user.target.wants/ceph.target". 2024-09-21T03:28:55.596 INFO:teuthology.orchestra.run.smithi089.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-09-21T03:28:55.773 INFO:teuthology.orchestra.run.smithi089.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-7c374098-77c9-11ef-baf5-efdc52797490.target → /etc/systemd/system/ceph-7c374098-77c9-11ef-baf5-efdc52797490.target. 2024-09-21T03:28:55.773 INFO:teuthology.orchestra.run.smithi089.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-7c374098-77c9-11ef-baf5-efdc52797490.target → /etc/systemd/system/ceph-7c374098-77c9-11ef-baf5-efdc52797490.target. 2024-09-21T03:28:56.027 INFO:teuthology.orchestra.run.smithi089.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-7c374098-77c9-11ef-baf5-efdc52797490@mon.a 2024-09-21T03:28:56.027 INFO:teuthology.orchestra.run.smithi089.stdout:systemctl: stderr Failed to reset failed state of unit ceph-7c374098-77c9-11ef-baf5-efdc52797490@mon.a.service: Unit ceph-7c374098-77c9-11ef-baf5-efdc52797490@mon.a.service not loaded. 2024-09-21T03:28:56.201 INFO:teuthology.orchestra.run.smithi089.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-7c374098-77c9-11ef-baf5-efdc52797490.target.wants/ceph-7c374098-77c9-11ef-baf5-efdc52797490@mon.a.service → /etc/systemd/system/ceph-7c374098-77c9-11ef-baf5-efdc52797490@.service. 2024-09-21T03:28:56.727 INFO:teuthology.orchestra.run.smithi089.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-21T03:28:56.727 INFO:teuthology.orchestra.run.smithi089.stdout:systemctl: stdout disabled 2024-09-21T03:28:56.734 INFO:teuthology.orchestra.run.smithi089.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-21T03:28:56.734 INFO:teuthology.orchestra.run.smithi089.stdout:systemctl: stdout inactive 2024-09-21T03:28:56.734 INFO:teuthology.orchestra.run.smithi089.stdout:firewalld.service is not enabled 2024-09-21T03:28:56.735 INFO:teuthology.orchestra.run.smithi089.stdout:Not possible to enable service . firewalld.service is not available 2024-09-21T03:28:56.735 INFO:teuthology.orchestra.run.smithi089.stdout:Waiting for mon to start... 2024-09-21T03:28:56.735 INFO:teuthology.orchestra.run.smithi089.stdout:Waiting for mon... 2024-09-21T03:28:56.811 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:28:56 smithi089 systemd[1]: Started Ceph mon.a for 7c374098-77c9-11ef-baf5-efdc52797490. 2024-09-21T03:28:57.798 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout cluster: 2024-09-21T03:28:57.799 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout id: 7c374098-77c9-11ef-baf5-efdc52797490 2024-09-21T03:28:57.799 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-09-21T03:28:57.799 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout 2024-09-21T03:28:57.799 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout services: 2024-09-21T03:28:57.799 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 0.615937s) 2024-09-21T03:28:57.799 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-09-21T03:28:57.799 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-09-21T03:28:57.799 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout 2024-09-21T03:28:57.799 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout data: 2024-09-21T03:28:57.800 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-09-21T03:28:57.800 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-09-21T03:28:57.800 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-09-21T03:28:57.800 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout pgs: 2024-09-21T03:28:57.800 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout 2024-09-21T03:28:57.800 INFO:teuthology.orchestra.run.smithi089.stdout:mon is available 2024-09-21T03:28:57.800 INFO:teuthology.orchestra.run.smithi089.stdout:Assimilating anything we can from ceph.conf... 2024-09-21T03:28:58.005 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:28:57 smithi089 ceph-mon[30563]: mon.a is new leader, mons a in quorum (ranks 0) 2024-09-21T03:28:58.005 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:28:57 smithi089 ceph-mon[30563]: monmap epoch 1 2024-09-21T03:28:58.005 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:28:57 smithi089 ceph-mon[30563]: fsid 7c374098-77c9-11ef-baf5-efdc52797490 2024-09-21T03:28:58.005 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:28:57 smithi089 ceph-mon[30563]: last_changed 2024-09-21T03:28:53.768514+0000 2024-09-21T03:28:58.005 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:28:57 smithi089 ceph-mon[30563]: created 2024-09-21T03:28:53.768514+0000 2024-09-21T03:28:58.006 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:28:57 smithi089 ceph-mon[30563]: min_mon_release 19 (squid) 2024-09-21T03:28:58.006 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:28:57 smithi089 ceph-mon[30563]: election_strategy: 1 2024-09-21T03:28:58.006 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:28:57 smithi089 ceph-mon[30563]: 0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-09-21T03:28:58.006 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:28:57 smithi089 ceph-mon[30563]: fsmap 2024-09-21T03:28:58.006 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:28:57 smithi089 ceph-mon[30563]: osdmap e1: 0 total, 0 up, 0 in 2024-09-21T03:28:58.006 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:28:57 smithi089 ceph-mon[30563]: mgrmap e1: no daemons active 2024-09-21T03:28:58.006 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:28:57 smithi089 ceph-mon[30563]: from='client.? 172.21.15.89:0/875334580' entity='client.admin' cmd={"prefix": "status"} : dispatch 2024-09-21T03:28:58.786 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout 2024-09-21T03:28:58.788 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout [global] 2024-09-21T03:28:58.789 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout fsid = 7c374098-77c9-11ef-baf5-efdc52797490 2024-09-21T03:28:58.789 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout mon_cluster_log_file_level = debug 2024-09-21T03:28:58.789 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.89:3300,v1:172.21.15.89:6789] 2024-09-21T03:28:58.789 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-09-21T03:28:58.789 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-09-21T03:28:58.789 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-09-21T03:28:58.789 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-09-21T03:28:58.789 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout 2024-09-21T03:28:58.789 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout [mgr] 2024-09-21T03:28:58.789 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-09-21T03:28:58.789 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout 2024-09-21T03:28:58.789 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout [osd] 2024-09-21T03:28:58.789 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-09-21T03:28:58.789 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-09-21T03:28:58.790 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-09-21T03:28:58.790 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-09-21T03:28:58.790 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-09-21T03:28:58.790 INFO:teuthology.orchestra.run.smithi089.stdout:Generating new minimal ceph.conf... 2024-09-21T03:28:58.995 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:28:58 smithi089 ceph-mon[30563]: from='client.? 172.21.15.89:0/3210698375' entity='client.admin' cmd={"prefix": "config assimilate-conf"} : dispatch 2024-09-21T03:28:58.995 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:28:58 smithi089 ceph-mon[30563]: from='client.? 172.21.15.89:0/3210698375' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-09-21T03:28:59.727 INFO:teuthology.orchestra.run.smithi089.stdout:Restarting the monitor... 2024-09-21T03:29:00.012 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:28:59 smithi089 ceph-mon[30563]: from='client.? 172.21.15.89:0/2730275374' entity='client.admin' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:29:00.012 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:28:59 smithi089 systemd[1]: Stopping Ceph mon.a for 7c374098-77c9-11ef-baf5-efdc52797490... 2024-09-21T03:29:00.311 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:29:00 smithi089 ceph-7c374098-77c9-11ef-baf5-efdc52797490-mon-a[30559]: 2024-09-21T03:29:00.010+0000 7f95ec5c6640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T03:29:00.311 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:29:00 smithi089 ceph-7c374098-77c9-11ef-baf5-efdc52797490-mon-a[30559]: 2024-09-21T03:29:00.010+0000 7f95ec5c6640 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2024-09-21T03:29:00.311 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:29:00 smithi089 podman[30805]: 2024-09-21 03:29:00.224877239 +0000 UTC m=+0.315439888 container died bcb4132f09f58f54221aa83a2cba4808110d950ad7eba64a757d05224237367e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410, name=ceph-7c374098-77c9-11ef-baf5-efdc52797490-mon-a, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-21T03:29:00.811 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:29:00 smithi089 podman[30805]: 2024-09-21 03:29:00.576797132 +0000 UTC m=+0.667359780 container cleanup bcb4132f09f58f54221aa83a2cba4808110d950ad7eba64a757d05224237367e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410, name=ceph-7c374098-77c9-11ef-baf5-efdc52797490-mon-a, GIT_BRANCH=HEAD, ceph=True, GIT_CLEAN=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-21T03:29:00.811 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:29:00 smithi089 bash[30805]: ceph-7c374098-77c9-11ef-baf5-efdc52797490-mon-a 2024-09-21T03:29:01.219 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:29:00 smithi089 podman[30817]: 2024-09-21 03:29:00.844482582 +0000 UTC m=+0.615542907 container remove bcb4132f09f58f54221aa83a2cba4808110d950ad7eba64a757d05224237367e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410, name=ceph-7c374098-77c9-11ef-baf5-efdc52797490-mon-a, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.schema-version=1.0) 2024-09-21T03:29:01.219 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:29:00 smithi089 systemd[1]: ceph-7c374098-77c9-11ef-baf5-efdc52797490@mon.a.service: Deactivated successfully. 2024-09-21T03:29:01.219 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:29:00 smithi089 systemd[1]: Stopped Ceph mon.a for 7c374098-77c9-11ef-baf5-efdc52797490. 2024-09-21T03:29:01.219 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:29:00 smithi089 systemd[1]: Starting Ceph mon.a for 7c374098-77c9-11ef-baf5-efdc52797490... 2024-09-21T03:29:01.482 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:29:01 smithi089 podman[30910]: 2024-09-21 03:29:01.120789596 +0000 UTC m=+0.015571436 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 2024-09-21T03:29:01.482 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:29:01 smithi089 podman[30910]: 2024-09-21 03:29:01.2201938 +0000 UTC m=+0.114975625 container create 768910a1020e482627b41daee6c7c4475f062377a34e6dbfd751444fd47ba7a1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410, name=ceph-7c374098-77c9-11ef-baf5-efdc52797490-mon-a, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0) 2024-09-21T03:29:01.482 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:29:01 smithi089 podman[30910]: 2024-09-21 03:29:01.481253153 +0000 UTC m=+0.376034992 container init 768910a1020e482627b41daee6c7c4475f062377a34e6dbfd751444fd47ba7a1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410, name=ceph-7c374098-77c9-11ef-baf5-efdc52797490-mon-a, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, ceph=True, maintainer=Guillaume Abrioux ) 2024-09-21T03:29:01.570 INFO:teuthology.orchestra.run.smithi089.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-09-21T03:29:01.776 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:29:01 smithi089 podman[30910]: 2024-09-21 03:29:01.484394208 +0000 UTC m=+0.379176028 container start 768910a1020e482627b41daee6c7c4475f062377a34e6dbfd751444fd47ba7a1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410, name=ceph-7c374098-77c9-11ef-baf5-efdc52797490-mon-a, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221) 2024-09-21T03:29:01.776 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:29:01 smithi089 ceph-mon[30928]: set uid:gid to 167:167 (ceph:ceph) 2024-09-21T03:29:01.776 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:29:01 smithi089 ceph-mon[30928]: ceph version 19.3.0-5120-g18432218 (1843221895adccfb3b692060251dd6a0183bc410) squid (dev), process ceph-mon, pid 2 2024-09-21T03:29:01.776 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:29:01 smithi089 ceph-mon[30928]: pidfile_write: ignore empty --pid-file 2024-09-21T03:29:01.776 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:29:01 smithi089 ceph-mon[30928]: load: jerasure load: lrc 2024-09-21T03:29:01.776 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:29:01 smithi089 ceph-mon[30928]: rocksdb: RocksDB version: 7.9.2 2024-09-21T03:29:01.776 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:29:01 smithi089 ceph-mon[30928]: rocksdb: Git sha 0 2024-09-21T03:29:01.777 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:29:01 smithi089 ceph-mon[30928]: rocksdb: Compile date 2024-09-20 22:29:49 2024-09-21T03:29:01.777 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:29:01 smithi089 ceph-mon[30928]: rocksdb: DB SUMMARY 2024-09-21T03:29:01.777 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:29:01 smithi089 ceph-mon[30928]: rocksdb: DB Session ID: 7GV2HAAN7IB2326DG1HR 2024-09-21T03:29:01.777 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:29:01 smithi089 ceph-mon[30928]: rocksdb: CURRENT file: CURRENT 2024-09-21T03:29:01.777 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:29:01 smithi089 ceph-mon[30928]: rocksdb: IDENTITY file: IDENTITY 2024-09-21T03:29:01.777 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:29:01 smithi089 ceph-mon[30928]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2024-09-21T03:29:01.777 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:29:01 smithi089 ceph-mon[30928]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2024-09-21T03:29:01.777 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:29:01 smithi089 ceph-mon[30928]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000009.log size: 94320 ; 2024-09-21T03:29:01.777 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:29:01 smithi089 ceph-mon[30928]: rocksdb: Options.error_if_exists: 0 2024-09-21T03:29:01.777 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:29:01 smithi089 ceph-mon[30928]: rocksdb: Options.create_if_missing: 0 2024-09-21T03:29:01.777 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:29:01 smithi089 ceph-mon[30928]: rocksdb: Options.paranoid_checks: 1 2024-09-21T03:29:01.777 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:29:01 smithi089 ceph-mon[30928]: rocksdb: Options.flush_verify_memtable_count: 1 2024-09-21T03:29:01.777 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:29:01 smithi089 ceph-mon[30928]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-09-21T03:29:01.777 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:29:01 smithi089 ceph-mon[30928]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-09-21T03:29:01.777 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:29:01 smithi089 ceph-mon[30928]: rocksdb: Options.env: 0x55cafdb2dd60 2024-09-21T03:29:01.778 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:29:01 smithi089 ceph-mon[30928]: rocksdb: Options.fs: PosixFileSystem 2024-09-21T03:29:01.778 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:29:01 smithi089 ceph-mon[30928]: rocksdb: Options.info_log: 0x55cb02481360 2024-09-21T03:29:01.778 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:29:01 smithi089 ceph-mon[30928]: rocksdb: Options.max_file_opening_threads: 16 2024-09-21T03:29:01.778 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:29:01 smithi089 ceph-mon[30928]: rocksdb: Options.statistics: (nil) 2024-09-21T03:29:01.778 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:29:01 smithi089 ceph-mon[30928]: rocksdb: Options.use_fsync: 0 2024-09-21T03:29:01.778 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:29:01 smithi089 ceph-mon[30928]: rocksdb: Options.max_log_file_size: 0 2024-09-21T03:29:01.778 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:29:01 smithi089 ceph-mon[30928]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-09-21T03:29:01.778 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:29:01 smithi089 ceph-mon[30928]: rocksdb: Options.log_file_time_to_roll: 0 2024-09-21T03:29:01.778 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:29:01 smithi089 ceph-mon[30928]: rocksdb: Options.keep_log_file_num: 1000 2024-09-21T03:29:01.778 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:29:01 smithi089 ceph-mon[30928]: rocksdb: Options.recycle_log_file_num: 0 2024-09-21T03:29:01.778 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:29:01 smithi089 ceph-mon[30928]: rocksdb: Options.allow_fallocate: 1 2024-09-21T03:29:01.779 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:29:01 smithi089 ceph-mon[30928]: rocksdb: Options.allow_mmap_reads: 0 2024-09-21T03:29:01.780 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:29:01 smithi089 ceph-mon[30928]: rocksdb: Options.allow_mmap_writes: 0 2024-09-21T03:29:01.780 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:29:01 smithi089 ceph-mon[30928]: rocksdb: Options.use_direct_reads: 0 2024-09-21T03:29:01.780 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:29:01 smithi089 ceph-mon[30928]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-09-21T03:29:01.780 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:29:01 smithi089 ceph-mon[30928]: rocksdb: Options.create_missing_column_families: 0 2024-09-21T03:29:01.780 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:29:01 smithi089 ceph-mon[30928]: rocksdb: Options.db_log_dir: 2024-09-21T03:29:01.780 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:29:01 smithi089 ceph-mon[30928]: rocksdb: Options.wal_dir: 2024-09-21T03:29:01.780 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:29:01 smithi089 ceph-mon[30928]: rocksdb: Options.table_cache_numshardbits: 6 2024-09-21T03:29:01.780 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:29:01 smithi089 ceph-mon[30928]: rocksdb: Options.WAL_ttl_seconds: 0 2024-09-21T03:29:01.780 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:29:01 smithi089 ceph-mon[30928]: rocksdb: Options.WAL_size_limit_MB: 0 2024-09-21T03:29:01.780 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:29:01 smithi089 ceph-mon[30928]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-09-21T03:29:01.780 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:29:01 smithi089 ceph-mon[30928]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-09-21T03:29:01.780 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:29:01 smithi089 ceph-mon[30928]: rocksdb: Options.is_fd_close_on_exec: 1 2024-09-21T03:29:01.780 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:29:01 smithi089 ceph-mon[30928]: rocksdb: Options.advise_random_on_open: 1 2024-09-21T03:29:01.781 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:29:01 smithi089 ceph-mon[30928]: rocksdb: Options.db_write_buffer_size: 0 2024-09-21T03:29:01.781 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:29:01 smithi089 ceph-mon[30928]: rocksdb: Options.write_buffer_manager: 0x55cb02485d60 2024-09-21T03:29:01.781 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:29:01 smithi089 ceph-mon[30928]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-09-21T03:29:01.781 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:29:01 smithi089 ceph-mon[30928]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-09-21T03:29:01.781 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:29:01 smithi089 ceph-mon[30928]: rocksdb: Options.use_adaptive_mutex: 0 2024-09-21T03:29:01.781 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:29:01 smithi089 ceph-mon[30928]: rocksdb: Options.rate_limiter: (nil) 2024-09-21T03:29:01.781 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:29:01 smithi089 ceph-mon[30928]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-09-21T03:29:01.781 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:29:01 smithi089 ceph-mon[30928]: rocksdb: Options.wal_recovery_mode: 2 2024-09-21T03:29:01.781 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:29:01 smithi089 ceph-mon[30928]: rocksdb: Options.enable_thread_tracking: 0 2024-09-21T03:29:01.781 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:29:01 smithi089 ceph-mon[30928]: rocksdb: Options.enable_pipelined_write: 0 2024-09-21T03:29:01.781 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:29:01 smithi089 ceph-mon[30928]: rocksdb: Options.unordered_write: 0 2024-09-21T03:29:01.781 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:29:01 smithi089 ceph-mon[30928]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-09-21T03:29:01.781 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:29:01 smithi089 ceph-mon[30928]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-09-21T03:29:01.781 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:29:01 smithi089 ceph-mon[30928]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-09-21T03:29:01.782 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:29:01 smithi089 ceph-mon[30928]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-09-21T03:29:01.782 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:29:01 smithi089 ceph-mon[30928]: rocksdb: Options.row_cache: None 2024-09-21T03:29:01.782 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:29:01 smithi089 ceph-mon[30928]: rocksdb: Options.wal_filter: None 2024-09-21T03:29:01.782 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:29:01 smithi089 ceph-mon[30928]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-09-21T03:29:01.782 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:29:01 smithi089 ceph-mon[30928]: rocksdb: Options.allow_ingest_behind: 0 2024-09-21T03:29:01.782 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:29:01 smithi089 ceph-mon[30928]: rocksdb: Options.two_write_queues: 0 2024-09-21T03:29:01.782 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:29:01 smithi089 ceph-mon[30928]: rocksdb: Options.manual_wal_flush: 0 2024-09-21T03:29:01.782 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:29:01 smithi089 ceph-mon[30928]: rocksdb: Options.wal_compression: 0 2024-09-21T03:29:01.782 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:29:01 smithi089 ceph-mon[30928]: rocksdb: Options.atomic_flush: 0 2024-09-21T03:29:01.782 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:29:01 smithi089 ceph-mon[30928]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-09-21T03:29:01.782 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:29:01 smithi089 ceph-mon[30928]: rocksdb: Options.persist_stats_to_disk: 0 2024-09-21T03:29:01.782 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:29:01 smithi089 ceph-mon[30928]: rocksdb: Options.write_dbid_to_manifest: 0 2024-09-21T03:29:01.782 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:29:01 smithi089 ceph-mon[30928]: rocksdb: Options.log_readahead_size: 0 2024-09-21T03:29:01.782 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:29:01 smithi089 ceph-mon[30928]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-09-21T03:29:01.783 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:29:01 smithi089 ceph-mon[30928]: rocksdb: Options.best_efforts_recovery: 0 2024-09-21T03:29:01.783 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:29:01 smithi089 ceph-mon[30928]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-09-21T03:29:01.783 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:29:01 smithi089 ceph-mon[30928]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-09-21T03:29:01.783 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:29:01 smithi089 ceph-mon[30928]: rocksdb: Options.allow_data_in_errors: 0 2024-09-21T03:29:01.783 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:29:01 smithi089 ceph-mon[30928]: rocksdb: Options.db_host_id: __hostname__ 2024-09-21T03:29:01.783 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:29:01 smithi089 ceph-mon[30928]: rocksdb: Options.enforce_single_del_contracts: true 2024-09-21T03:29:01.783 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:29:01 smithi089 ceph-mon[30928]: rocksdb: Options.max_background_jobs: 2 2024-09-21T03:29:01.783 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:29:01 smithi089 ceph-mon[30928]: rocksdb: Options.max_background_compactions: -1 2024-09-21T03:29:01.783 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:29:01 smithi089 ceph-mon[30928]: rocksdb: Options.max_subcompactions: 1 2024-09-21T03:29:01.783 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:29:01 smithi089 ceph-mon[30928]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-09-21T03:29:01.783 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:29:01 smithi089 ceph-mon[30928]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-09-21T03:29:01.783 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:29:01 smithi089 ceph-mon[30928]: rocksdb: Options.delayed_write_rate : 16777216 2024-09-21T03:29:01.783 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:29:01 smithi089 ceph-mon[30928]: rocksdb: Options.max_total_wal_size: 0 2024-09-21T03:29:01.783 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:29:01 smithi089 ceph-mon[30928]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-09-21T03:29:01.783 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:29:01 smithi089 ceph-mon[30928]: rocksdb: Options.stats_dump_period_sec: 600 2024-09-21T03:29:01.784 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:29:01 smithi089 ceph-mon[30928]: rocksdb: Options.stats_persist_period_sec: 600 2024-09-21T03:29:01.784 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:29:01 smithi089 ceph-mon[30928]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-09-21T03:29:01.784 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:29:01 smithi089 ceph-mon[30928]: rocksdb: Options.max_open_files: -1 2024-09-21T03:29:01.784 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:29:01 smithi089 ceph-mon[30928]: rocksdb: Options.bytes_per_sync: 0 2024-09-21T03:29:01.784 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:29:01 smithi089 ceph-mon[30928]: rocksdb: Options.wal_bytes_per_sync: 0 2024-09-21T03:29:01.784 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:29:01 smithi089 ceph-mon[30928]: rocksdb: Options.strict_bytes_per_sync: 0 2024-09-21T03:29:01.784 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:29:01 smithi089 ceph-mon[30928]: rocksdb: Options.compaction_readahead_size: 0 2024-09-21T03:29:01.784 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:29:01 smithi089 ceph-mon[30928]: rocksdb: Options.max_background_flushes: -1 2024-09-21T03:29:01.784 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:29:01 smithi089 ceph-mon[30928]: rocksdb: Compression algorithms supported: 2024-09-21T03:29:01.784 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:29:01 smithi089 ceph-mon[30928]: rocksdb: kZSTD supported: 0 2024-09-21T03:29:01.784 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:29:01 smithi089 ceph-mon[30928]: rocksdb: kXpressCompression supported: 0 2024-09-21T03:29:01.784 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:29:01 smithi089 ceph-mon[30928]: rocksdb: kBZip2Compression supported: 0 2024-09-21T03:29:01.784 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:29:01 smithi089 ceph-mon[30928]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-09-21T03:29:01.784 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:29:01 smithi089 ceph-mon[30928]: rocksdb: kLZ4Compression supported: 1 2024-09-21T03:29:01.785 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:29:01 smithi089 ceph-mon[30928]: rocksdb: kZlibCompression supported: 1 2024-09-21T03:29:01.785 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:29:01 smithi089 ceph-mon[30928]: rocksdb: kLZ4HCCompression supported: 1 2024-09-21T03:29:01.785 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:29:01 smithi089 ceph-mon[30928]: rocksdb: kSnappyCompression supported: 1 2024-09-21T03:29:01.785 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:29:01 smithi089 ceph-mon[30928]: rocksdb: Fast CRC32 supported: Supported on x86 2024-09-21T03:29:01.785 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:29:01 smithi089 ceph-mon[30928]: rocksdb: DMutex implementation: pthread_mutex_t 2024-09-21T03:29:01.785 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:29:01 smithi089 ceph-mon[30928]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2024-09-21T03:29:01.785 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:29:01 smithi089 ceph-mon[30928]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-09-21T03:29:01.785 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:29:01 smithi089 ceph-mon[30928]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-09-21T03:29:01.785 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:29:01 smithi089 ceph-mon[30928]: rocksdb: Options.merge_operator: 2024-09-21T03:29:01.785 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:29:01 smithi089 ceph-mon[30928]: rocksdb: Options.compaction_filter: None 2024-09-21T03:29:01.785 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:29:01 smithi089 ceph-mon[30928]: rocksdb: Options.compaction_filter_factory: None 2024-09-21T03:29:01.785 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:29:01 smithi089 ceph-mon[30928]: rocksdb: Options.sst_partitioner_factory: None 2024-09-21T03:29:01.785 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:29:01 smithi089 ceph-mon[30928]: rocksdb: Options.memtable_factory: SkipListFactory 2024-09-21T03:29:01.785 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:29:01 smithi089 ceph-mon[30928]: rocksdb: Options.table_factory: BlockBasedTable 2024-09-21T03:29:01.785 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:29:01 smithi089 ceph-mon[30928]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55cb02480fa0) 2024-09-21T03:29:01.786 INFO:journalctl@ceph.mon.a.smithi089.stdout: cache_index_and_filter_blocks: 1 2024-09-21T03:29:01.786 INFO:journalctl@ceph.mon.a.smithi089.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-09-21T03:29:01.786 INFO:journalctl@ceph.mon.a.smithi089.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-09-21T03:29:01.786 INFO:journalctl@ceph.mon.a.smithi089.stdout: pin_top_level_index_and_filter: 1 2024-09-21T03:29:01.786 INFO:journalctl@ceph.mon.a.smithi089.stdout: index_type: 0 2024-09-21T03:29:01.786 INFO:journalctl@ceph.mon.a.smithi089.stdout: data_block_index_type: 0 2024-09-21T03:29:01.786 INFO:journalctl@ceph.mon.a.smithi089.stdout: index_shortening: 1 2024-09-21T03:29:01.786 INFO:journalctl@ceph.mon.a.smithi089.stdout: data_block_hash_table_util_ratio: 0.750000 2024-09-21T03:29:01.786 INFO:journalctl@ceph.mon.a.smithi089.stdout: checksum: 4 2024-09-21T03:29:01.786 INFO:journalctl@ceph.mon.a.smithi089.stdout: no_block_cache: 0 2024-09-21T03:29:01.786 INFO:journalctl@ceph.mon.a.smithi089.stdout: block_cache: 0x55cafe445610 2024-09-21T03:29:01.786 INFO:journalctl@ceph.mon.a.smithi089.stdout: block_cache_name: BinnedLRUCache 2024-09-21T03:29:01.786 INFO:journalctl@ceph.mon.a.smithi089.stdout: block_cache_options: 2024-09-21T03:29:01.786 INFO:journalctl@ceph.mon.a.smithi089.stdout: capacity : 536870912 2024-09-21T03:29:01.787 INFO:journalctl@ceph.mon.a.smithi089.stdout: num_shard_bits : 4 2024-09-21T03:29:01.787 INFO:journalctl@ceph.mon.a.smithi089.stdout: strict_capacity_limit : 0 2024-09-21T03:29:01.787 INFO:journalctl@ceph.mon.a.smithi089.stdout: high_pri_pool_ratio: 0.000 2024-09-21T03:29:01.787 INFO:journalctl@ceph.mon.a.smithi089.stdout: block_cache_compressed: (nil) 2024-09-21T03:29:01.787 INFO:journalctl@ceph.mon.a.smithi089.stdout: persistent_cache: (nil) 2024-09-21T03:29:01.787 INFO:journalctl@ceph.mon.a.smithi089.stdout: block_size: 4096 2024-09-21T03:29:01.787 INFO:journalctl@ceph.mon.a.smithi089.stdout: block_size_deviation: 10 2024-09-21T03:29:01.787 INFO:journalctl@ceph.mon.a.smithi089.stdout: block_restart_interval: 16 2024-09-21T03:29:01.787 INFO:journalctl@ceph.mon.a.smithi089.stdout: index_block_restart_interval: 1 2024-09-21T03:29:01.787 INFO:journalctl@ceph.mon.a.smithi089.stdout: metadata_block_size: 4096 2024-09-21T03:29:01.787 INFO:journalctl@ceph.mon.a.smithi089.stdout: partition_filters: 0 2024-09-21T03:29:01.787 INFO:journalctl@ceph.mon.a.smithi089.stdout: use_delta_encoding: 1 2024-09-21T03:29:01.787 INFO:journalctl@ceph.mon.a.smithi089.stdout: filter_policy: bloomfilter 2024-09-21T03:29:01.787 INFO:journalctl@ceph.mon.a.smithi089.stdout: whole_key_filtering: 1 2024-09-21T03:29:01.787 INFO:journalctl@ceph.mon.a.smithi089.stdout: verify_compression: 0 2024-09-21T03:29:01.788 INFO:journalctl@ceph.mon.a.smithi089.stdout: read_amp_bytes_per_bit: 0 2024-09-21T03:29:01.788 INFO:journalctl@ceph.mon.a.smithi089.stdout: format_version: 5 2024-09-21T03:29:01.788 INFO:journalctl@ceph.mon.a.smithi089.stdout: enable_index_compression: 1 2024-09-21T03:29:01.788 INFO:journalctl@ceph.mon.a.smithi089.stdout: block_align: 0 2024-09-21T03:29:01.788 INFO:journalctl@ceph.mon.a.smithi089.stdout: max_auto_readahead_size: 262144 2024-09-21T03:29:01.788 INFO:journalctl@ceph.mon.a.smithi089.stdout: prepopulate_block_cache: 0 2024-09-21T03:29:01.788 INFO:journalctl@ceph.mon.a.smithi089.stdout: initial_auto_readahead_size: 8192 2024-09-21T03:29:01.788 INFO:journalctl@ceph.mon.a.smithi089.stdout: num_file_reads_for_auto_readahead: 2 2024-09-21T03:29:01.788 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:29:01 smithi089 ceph-mon[30928]: rocksdb: Options.write_buffer_size: 33554432 2024-09-21T03:29:01.788 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:29:01 smithi089 ceph-mon[30928]: rocksdb: Options.max_write_buffer_number: 2 2024-09-21T03:29:01.788 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:29:01 smithi089 ceph-mon[30928]: rocksdb: Options.compression: NoCompression 2024-09-21T03:29:01.788 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:29:01 smithi089 ceph-mon[30928]: rocksdb: Options.bottommost_compression: Disabled 2024-09-21T03:29:01.788 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:29:01 smithi089 ceph-mon[30928]: rocksdb: Options.prefix_extractor: nullptr 2024-09-21T03:29:01.788 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:29:01 smithi089 ceph-mon[30928]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-09-21T03:29:01.789 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:29:01 smithi089 ceph-mon[30928]: rocksdb: Options.num_levels: 7 2024-09-21T03:29:01.789 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:29:01 smithi089 ceph-mon[30928]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-09-21T03:29:01.789 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:29:01 smithi089 ceph-mon[30928]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-09-21T03:29:01.789 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:29:01 smithi089 ceph-mon[30928]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-09-21T03:29:01.789 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:29:01 smithi089 ceph-mon[30928]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-09-21T03:29:01.789 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:29:01 smithi089 ceph-mon[30928]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-09-21T03:29:01.789 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:29:01 smithi089 ceph-mon[30928]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-09-21T03:29:01.789 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:29:01 smithi089 ceph-mon[30928]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-09-21T03:29:01.789 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:29:01 smithi089 ceph-mon[30928]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-09-21T03:29:01.789 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:29:01 smithi089 ceph-mon[30928]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-09-21T03:29:01.789 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:29:01 smithi089 ceph-mon[30928]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-09-21T03:29:01.789 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:29:01 smithi089 ceph-mon[30928]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-09-21T03:29:01.789 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:29:01 smithi089 ceph-mon[30928]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-09-21T03:29:01.789 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:29:01 smithi089 ceph-mon[30928]: rocksdb: Options.compression_opts.window_bits: -14 2024-09-21T03:29:01.789 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:29:01 smithi089 ceph-mon[30928]: rocksdb: Options.compression_opts.level: 32767 2024-09-21T03:29:01.790 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:29:01 smithi089 ceph-mon[30928]: rocksdb: Options.compression_opts.strategy: 0 2024-09-21T03:29:01.790 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:29:01 smithi089 ceph-mon[30928]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-09-21T03:29:01.790 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:29:01 smithi089 ceph-mon[30928]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-09-21T03:29:01.790 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:29:01 smithi089 ceph-mon[30928]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-09-21T03:29:01.790 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:29:01 smithi089 ceph-mon[30928]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-09-21T03:29:01.790 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:29:01 smithi089 ceph-mon[30928]: rocksdb: Options.compression_opts.enabled: false 2024-09-21T03:29:01.790 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:29:01 smithi089 ceph-mon[30928]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-09-21T03:29:01.790 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:29:01 smithi089 ceph-mon[30928]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-09-21T03:29:01.790 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:29:01 smithi089 ceph-mon[30928]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-09-21T03:29:01.790 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:29:01 smithi089 ceph-mon[30928]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-09-21T03:29:01.790 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:29:01 smithi089 ceph-mon[30928]: rocksdb: Options.target_file_size_base: 67108864 2024-09-21T03:29:01.790 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:29:01 smithi089 ceph-mon[30928]: rocksdb: Options.target_file_size_multiplier: 1 2024-09-21T03:29:01.790 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:29:01 smithi089 ceph-mon[30928]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-09-21T03:29:01.790 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:29:01 smithi089 ceph-mon[30928]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-09-21T03:29:01.790 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:29:01 smithi089 ceph-mon[30928]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-09-21T03:29:01.791 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:29:01 smithi089 ceph-mon[30928]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-09-21T03:29:01.791 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:29:01 smithi089 ceph-mon[30928]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-09-21T03:29:01.791 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:29:01 smithi089 ceph-mon[30928]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-09-21T03:29:01.791 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:29:01 smithi089 ceph-mon[30928]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-09-21T03:29:01.791 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:29:01 smithi089 ceph-mon[30928]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-09-21T03:29:01.791 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:29:01 smithi089 ceph-mon[30928]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-09-21T03:29:01.791 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:29:01 smithi089 ceph-mon[30928]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-09-21T03:29:01.791 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:29:01 smithi089 ceph-mon[30928]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-09-21T03:29:01.791 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:29:01 smithi089 ceph-mon[30928]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-09-21T03:29:01.791 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:29:01 smithi089 ceph-mon[30928]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-09-21T03:29:01.791 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:29:01 smithi089 ceph-mon[30928]: rocksdb: Options.arena_block_size: 1048576 2024-09-21T03:29:01.791 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:29:01 smithi089 ceph-mon[30928]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-09-21T03:29:01.791 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:29:01 smithi089 ceph-mon[30928]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-09-21T03:29:01.791 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:29:01 smithi089 ceph-mon[30928]: rocksdb: Options.disable_auto_compactions: 0 2024-09-21T03:29:01.791 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:29:01 smithi089 ceph-mon[30928]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-09-21T03:29:01.792 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:29:01 smithi089 ceph-mon[30928]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-09-21T03:29:01.792 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:29:01 smithi089 ceph-mon[30928]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-09-21T03:29:01.792 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:29:01 smithi089 ceph-mon[30928]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-09-21T03:29:01.792 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:29:01 smithi089 ceph-mon[30928]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-09-21T03:29:01.792 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:29:01 smithi089 ceph-mon[30928]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-09-21T03:29:01.792 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:29:01 smithi089 ceph-mon[30928]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-09-21T03:29:01.792 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:29:01 smithi089 ceph-mon[30928]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-09-21T03:29:01.792 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:29:01 smithi089 ceph-mon[30928]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-09-21T03:29:01.792 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:29:01 smithi089 ceph-mon[30928]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-09-21T03:29:01.792 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:29:01 smithi089 ceph-mon[30928]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-09-21T03:29:01.792 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:29:01 smithi089 ceph-mon[30928]: rocksdb: Options.inplace_update_support: 0 2024-09-21T03:29:01.792 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:29:01 smithi089 ceph-mon[30928]: rocksdb: Options.inplace_update_num_locks: 10000 2024-09-21T03:29:01.792 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:29:01 smithi089 ceph-mon[30928]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-09-21T03:29:01.792 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:29:01 smithi089 ceph-mon[30928]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-09-21T03:29:01.792 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:29:01 smithi089 ceph-mon[30928]: rocksdb: Options.memtable_huge_page_size: 0 2024-09-21T03:29:01.793 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:29:01 smithi089 ceph-mon[30928]: rocksdb: Options.bloom_locality: 0 2024-09-21T03:29:01.793 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:29:01 smithi089 ceph-mon[30928]: rocksdb: Options.max_successive_merges: 0 2024-09-21T03:29:01.793 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:29:01 smithi089 ceph-mon[30928]: rocksdb: Options.optimize_filters_for_hits: 0 2024-09-21T03:29:01.793 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:29:01 smithi089 ceph-mon[30928]: rocksdb: Options.paranoid_file_checks: 0 2024-09-21T03:29:01.793 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:29:01 smithi089 ceph-mon[30928]: rocksdb: Options.force_consistency_checks: 1 2024-09-21T03:29:01.793 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:29:01 smithi089 ceph-mon[30928]: rocksdb: Options.report_bg_io_stats: 0 2024-09-21T03:29:01.793 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:29:01 smithi089 ceph-mon[30928]: rocksdb: Options.ttl: 2592000 2024-09-21T03:29:01.793 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:29:01 smithi089 ceph-mon[30928]: rocksdb: Options.periodic_compaction_seconds: 0 2024-09-21T03:29:01.793 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:29:01 smithi089 ceph-mon[30928]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-09-21T03:29:01.793 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:29:01 smithi089 ceph-mon[30928]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-09-21T03:29:01.793 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:29:01 smithi089 ceph-mon[30928]: rocksdb: Options.enable_blob_files: false 2024-09-21T03:29:01.793 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:29:01 smithi089 ceph-mon[30928]: rocksdb: Options.min_blob_size: 0 2024-09-21T03:29:01.793 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:29:01 smithi089 ceph-mon[30928]: rocksdb: Options.blob_file_size: 268435456 2024-09-21T03:29:01.793 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:29:01 smithi089 ceph-mon[30928]: rocksdb: Options.blob_compression_type: NoCompression 2024-09-21T03:29:01.793 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:29:01 smithi089 ceph-mon[30928]: rocksdb: Options.enable_blob_garbage_collection: false 2024-09-21T03:29:01.794 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:29:01 smithi089 ceph-mon[30928]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-09-21T03:29:01.794 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:29:01 smithi089 ceph-mon[30928]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-09-21T03:29:01.794 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:29:01 smithi089 ceph-mon[30928]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-09-21T03:29:01.794 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:29:01 smithi089 ceph-mon[30928]: rocksdb: Options.blob_file_starting_level: 0 2024-09-21T03:29:01.794 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:29:01 smithi089 ceph-mon[30928]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-09-21T03:29:01.794 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:29:01 smithi089 ceph-mon[30928]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2024-09-21T03:29:01.794 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:29:01 smithi089 ceph-mon[30928]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2024-09-21T03:29:01.794 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:29:01 smithi089 ceph-mon[30928]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 3db78647-7565-4e7c-ad74-9f1bf23ec625 2024-09-21T03:29:01.794 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:29:01 smithi089 ceph-mon[30928]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726889341521465, "job": 1, "event": "recovery_started", "wal_files": [9]} 2024-09-21T03:29:01.794 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:29:01 smithi089 ceph-mon[30928]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2024-09-21T03:29:01.794 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:29:01 smithi089 ceph-mon[30928]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726889341522546, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 90624, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 281, "table_properties": {"data_size": 88665, "index_size": 279, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 645, "raw_key_size": 11470, "raw_average_key_size": 47, "raw_value_size": 82137, "raw_average_value_size": 342, "num_data_blocks": 12, "num_entries": 240, "num_filter_entries": 240, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1726889341, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "3db78647-7565-4e7c-ad74-9f1bf23ec625", "db_session_id": "7GV2HAAN7IB2326DG1HR", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2024-09-21T03:29:01.794 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:29:01 smithi089 ceph-mon[30928]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726889341522641, "job": 1, "event": "recovery_finished"} 2024-09-21T03:29:01.794 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:29:01 smithi089 ceph-mon[30928]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2024-09-21T03:29:01.795 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:29:01 smithi089 ceph-mon[30928]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-09-21T03:29:01.795 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:29:01 smithi089 ceph-mon[30928]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55cb024a4a80 2024-09-21T03:29:01.795 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:29:01 smithi089 ceph-mon[30928]: rocksdb: DB pointer 0x55cb0324a000 2024-09-21T03:29:01.795 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:29:01 smithi089 ceph-mon[30928]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-09-21T03:29:01.795 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:29:01 smithi089 ceph-mon[30928]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-09-21T03:29:01.795 INFO:journalctl@ceph.mon.a.smithi089.stdout: ** DB Stats ** 2024-09-21T03:29:01.795 INFO:journalctl@ceph.mon.a.smithi089.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-21T03:29:01.795 INFO:journalctl@ceph.mon.a.smithi089.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-09-21T03:29:01.795 INFO:journalctl@ceph.mon.a.smithi089.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-21T03:29:01.795 INFO:journalctl@ceph.mon.a.smithi089.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-21T03:29:01.795 INFO:journalctl@ceph.mon.a.smithi089.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-09-21T03:29:01.795 INFO:journalctl@ceph.mon.a.smithi089.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-21T03:29:01.795 INFO:journalctl@ceph.mon.a.smithi089.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-21T03:29:01.795 INFO:journalctl@ceph.mon.a.smithi089.stdout: 2024-09-21T03:29:01.795 INFO:journalctl@ceph.mon.a.smithi089.stdout: ** Compaction Stats [default] ** 2024-09-21T03:29:01.796 INFO:journalctl@ceph.mon.a.smithi089.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-21T03:29:01.796 INFO:journalctl@ceph.mon.a.smithi089.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-09-21T03:29:01.796 INFO:journalctl@ceph.mon.a.smithi089.stdout: L0 2/0 90.36 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 129.0 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-21T03:29:01.796 INFO:journalctl@ceph.mon.a.smithi089.stdout: Sum 2/0 90.36 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 129.0 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-21T03:29:01.796 INFO:journalctl@ceph.mon.a.smithi089.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 129.0 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-21T03:29:01.796 INFO:journalctl@ceph.mon.a.smithi089.stdout: 2024-09-21T03:29:01.796 INFO:journalctl@ceph.mon.a.smithi089.stdout: ** Compaction Stats [default] ** 2024-09-21T03:29:01.796 INFO:journalctl@ceph.mon.a.smithi089.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-21T03:29:01.796 INFO:journalctl@ceph.mon.a.smithi089.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-21T03:29:01.796 INFO:journalctl@ceph.mon.a.smithi089.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 129.0 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-21T03:29:01.796 INFO:journalctl@ceph.mon.a.smithi089.stdout: 2024-09-21T03:29:01.796 INFO:journalctl@ceph.mon.a.smithi089.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-09-21T03:29:01.796 INFO:journalctl@ceph.mon.a.smithi089.stdout: 2024-09-21T03:29:01.796 INFO:journalctl@ceph.mon.a.smithi089.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-21T03:29:01.797 INFO:journalctl@ceph.mon.a.smithi089.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-09-21T03:29:01.797 INFO:journalctl@ceph.mon.a.smithi089.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-09-21T03:29:01.797 INFO:journalctl@ceph.mon.a.smithi089.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-09-21T03:29:01.797 INFO:journalctl@ceph.mon.a.smithi089.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-09-21T03:29:01.797 INFO:journalctl@ceph.mon.a.smithi089.stdout: AddFile(Keys): cumulative 0, interval 0 2024-09-21T03:29:01.797 INFO:journalctl@ceph.mon.a.smithi089.stdout: Cumulative compaction: 0.00 GB write, 26.27 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-21T03:29:01.797 INFO:journalctl@ceph.mon.a.smithi089.stdout: Interval compaction: 0.00 GB write, 26.27 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-21T03:29:01.797 INFO:journalctl@ceph.mon.a.smithi089.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-09-21T03:29:01.797 INFO:journalctl@ceph.mon.a.smithi089.stdout: Block cache BinnedLRUCache@0x55cafe445610#2 capacity: 512.00 MB usage: 1.31 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 8e-06 secs_since: 0 2024-09-21T03:29:01.797 INFO:journalctl@ceph.mon.a.smithi089.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,0.84 KB,0.000160933%) IndexBlock(2,0.47 KB,8.9407e-05%) Misc(1,0.00 KB,0%) 2024-09-21T03:29:01.797 INFO:journalctl@ceph.mon.a.smithi089.stdout: 2024-09-21T03:29:01.797 INFO:journalctl@ceph.mon.a.smithi089.stdout: ** File Read Latency Histogram By Level [default] ** 2024-09-21T03:29:01.797 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:29:01 smithi089 ceph-mon[30928]: starting mon.a rank 0 at public addrs [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] at bind addrs [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid 7c374098-77c9-11ef-baf5-efdc52797490 2024-09-21T03:29:01.797 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:29:01 smithi089 ceph-mon[30928]: mon.a@-1(???) e1 preinit fsid 7c374098-77c9-11ef-baf5-efdc52797490 2024-09-21T03:29:01.797 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:29:01 smithi089 ceph-mon[30928]: mon.a@-1(???).mds e1 new map 2024-09-21T03:29:01.798 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:29:01 smithi089 ceph-mon[30928]: mon.a@-1(???).mds e1 print_map 2024-09-21T03:29:01.798 INFO:journalctl@ceph.mon.a.smithi089.stdout: e1 2024-09-21T03:29:01.798 INFO:journalctl@ceph.mon.a.smithi089.stdout: btime 2024-09-21T03:28:56:681239+0000 2024-09-21T03:29:01.798 INFO:journalctl@ceph.mon.a.smithi089.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-09-21T03:29:01.798 INFO:journalctl@ceph.mon.a.smithi089.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-21T03:29:01.798 INFO:journalctl@ceph.mon.a.smithi089.stdout: legacy client fscid: -1 2024-09-21T03:29:01.798 INFO:journalctl@ceph.mon.a.smithi089.stdout: 2024-09-21T03:29:01.798 INFO:journalctl@ceph.mon.a.smithi089.stdout: No filesystems configured 2024-09-21T03:29:01.798 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:29:01 smithi089 ceph-mon[30928]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-09-21T03:29:01.798 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:29:01 smithi089 ceph-mon[30928]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-09-21T03:29:01.798 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:29:01 smithi089 ceph-mon[30928]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-09-21T03:29:01.798 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:29:01 smithi089 ceph-mon[30928]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-09-21T03:29:01.799 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:29:01 smithi089 ceph-mon[30928]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-09-21T03:29:01.799 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:29:01 smithi089 bash[30910]: 768910a1020e482627b41daee6c7c4475f062377a34e6dbfd751444fd47ba7a1 2024-09-21T03:29:01.799 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:29:01 smithi089 systemd[1]: Started Ceph mon.a for 7c374098-77c9-11ef-baf5-efdc52797490. 2024-09-21T03:29:01.799 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:29:01 smithi089 ceph-mon[30928]: mon.a is new leader, mons a in quorum (ranks 0) 2024-09-21T03:29:01.799 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:29:01 smithi089 ceph-mon[30928]: monmap epoch 1 2024-09-21T03:29:01.799 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:29:01 smithi089 ceph-mon[30928]: fsid 7c374098-77c9-11ef-baf5-efdc52797490 2024-09-21T03:29:01.799 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:29:01 smithi089 ceph-mon[30928]: last_changed 2024-09-21T03:28:53.768514+0000 2024-09-21T03:29:01.799 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:29:01 smithi089 ceph-mon[30928]: created 2024-09-21T03:28:53.768514+0000 2024-09-21T03:29:01.799 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:29:01 smithi089 ceph-mon[30928]: min_mon_release 19 (squid) 2024-09-21T03:29:01.799 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:29:01 smithi089 ceph-mon[30928]: election_strategy: 1 2024-09-21T03:29:01.799 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:29:01 smithi089 ceph-mon[30928]: 0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-09-21T03:29:01.799 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:29:01 smithi089 ceph-mon[30928]: fsmap 2024-09-21T03:29:01.799 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:29:01 smithi089 ceph-mon[30928]: osdmap e1: 0 total, 0 up, 0 in 2024-09-21T03:29:01.799 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:29:01 smithi089 ceph-mon[30928]: mgrmap e1: no daemons active 2024-09-21T03:29:02.608 INFO:teuthology.orchestra.run.smithi089.stdout:Wrote config to /etc/ceph/ceph.conf 2024-09-21T03:29:02.630 INFO:teuthology.orchestra.run.smithi089.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-09-21T03:29:02.630 INFO:teuthology.orchestra.run.smithi089.stdout:Creating mgr... 2024-09-21T03:29:02.630 INFO:teuthology.orchestra.run.smithi089.stdout:Verifying port 0.0.0.0:9283 ... 2024-09-21T03:29:02.631 INFO:teuthology.orchestra.run.smithi089.stdout:Verifying port 0.0.0.0:8765 ... 2024-09-21T03:29:02.886 INFO:teuthology.orchestra.run.smithi089.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-7c374098-77c9-11ef-baf5-efdc52797490@mgr.x 2024-09-21T03:29:02.886 INFO:teuthology.orchestra.run.smithi089.stdout:systemctl: stderr Failed to reset failed state of unit ceph-7c374098-77c9-11ef-baf5-efdc52797490@mgr.x.service: Unit ceph-7c374098-77c9-11ef-baf5-efdc52797490@mgr.x.service not loaded. 2024-09-21T03:29:03.032 INFO:teuthology.orchestra.run.smithi089.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-7c374098-77c9-11ef-baf5-efdc52797490.target.wants/ceph-7c374098-77c9-11ef-baf5-efdc52797490@mgr.x.service → /etc/systemd/system/ceph-7c374098-77c9-11ef-baf5-efdc52797490@.service. 2024-09-21T03:29:03.258 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:29:03 smithi089 ceph-mon[30928]: from='client.? 172.21.15.89:0/2744133190' entity='client.admin' 2024-09-21T03:29:03.535 INFO:journalctl@ceph.mgr.x.smithi089.stdout:Sep 21 03:29:03 smithi089 podman[31139]: 2024-09-21 03:29:03.453455147 +0000 UTC m=+0.305872988 container start 5ac528282a8ca47b9972574f752ae5a3dd4a05fe49fe281c9cece957a5e6a224 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410, name=ceph-7c374098-77c9-11ef-baf5-efdc52797490-mgr-x, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, ceph=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, io.buildah.version=1.37.2) 2024-09-21T03:29:03.535 INFO:journalctl@ceph.mgr.x.smithi089.stdout:Sep 21 03:29:03 smithi089 bash[31139]: 5ac528282a8ca47b9972574f752ae5a3dd4a05fe49fe281c9cece957a5e6a224 2024-09-21T03:29:03.552 INFO:teuthology.orchestra.run.smithi089.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-21T03:29:03.552 INFO:teuthology.orchestra.run.smithi089.stdout:systemctl: stdout disabled 2024-09-21T03:29:03.560 INFO:teuthology.orchestra.run.smithi089.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-21T03:29:03.561 INFO:teuthology.orchestra.run.smithi089.stdout:systemctl: stdout inactive 2024-09-21T03:29:03.561 INFO:teuthology.orchestra.run.smithi089.stdout:firewalld.service is not enabled 2024-09-21T03:29:03.561 INFO:teuthology.orchestra.run.smithi089.stdout:Not possible to enable service . firewalld.service is not available 2024-09-21T03:29:03.571 INFO:teuthology.orchestra.run.smithi089.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-21T03:29:03.571 INFO:teuthology.orchestra.run.smithi089.stdout:systemctl: stdout disabled 2024-09-21T03:29:03.580 INFO:teuthology.orchestra.run.smithi089.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-21T03:29:03.580 INFO:teuthology.orchestra.run.smithi089.stdout:systemctl: stdout inactive 2024-09-21T03:29:03.580 INFO:teuthology.orchestra.run.smithi089.stdout:firewalld.service is not enabled 2024-09-21T03:29:03.580 INFO:teuthology.orchestra.run.smithi089.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2024-09-21T03:29:03.580 INFO:teuthology.orchestra.run.smithi089.stdout:Waiting for mgr to start... 2024-09-21T03:29:03.580 INFO:teuthology.orchestra.run.smithi089.stdout:Waiting for mgr... 2024-09-21T03:29:03.789 INFO:journalctl@ceph.mgr.x.smithi089.stdout:Sep 21 03:29:03 smithi089 systemd[1]: Started Ceph mgr.x for 7c374098-77c9-11ef-baf5-efdc52797490. 2024-09-21T03:29:04.341 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:29:04 smithi089 ceph-mon[30928]: from='client.? 172.21.15.89:0/1300376437' entity='client.admin' cmd={"prefix": "status", "format": "json-pretty"} : dispatch 2024-09-21T03:29:04.619 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout 2024-09-21T03:29:04.627 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout { 2024-09-21T03:29:04.628 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "fsid": "7c374098-77c9-11ef-baf5-efdc52797490", 2024-09-21T03:29:04.628 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "health": { 2024-09-21T03:29:04.628 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-09-21T03:29:04.628 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-09-21T03:29:04.628 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-09-21T03:29:04.628 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout }, 2024-09-21T03:29:04.628 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-09-21T03:29:04.628 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-09-21T03:29:04.628 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout 0 2024-09-21T03:29:04.629 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout ], 2024-09-21T03:29:04.629 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-09-21T03:29:04.629 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "a" 2024-09-21T03:29:04.629 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout ], 2024-09-21T03:29:04.629 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "quorum_age": 2, 2024-09-21T03:29:04.629 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "monmap": { 2024-09-21T03:29:04.629 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-21T03:29:04.629 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-09-21T03:29:04.629 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-09-21T03:29:04.629 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout }, 2024-09-21T03:29:04.629 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-09-21T03:29:04.630 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-21T03:29:04.630 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-09-21T03:29:04.630 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-09-21T03:29:04.630 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-09-21T03:29:04.630 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-09-21T03:29:04.630 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-09-21T03:29:04.630 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-09-21T03:29:04.630 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout }, 2024-09-21T03:29:04.630 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-09-21T03:29:04.630 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-09-21T03:29:04.630 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-09-21T03:29:04.630 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-09-21T03:29:04.630 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-09-21T03:29:04.630 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-09-21T03:29:04.631 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-09-21T03:29:04.631 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-09-21T03:29:04.631 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-09-21T03:29:04.631 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout }, 2024-09-21T03:29:04.631 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-09-21T03:29:04.631 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-21T03:29:04.631 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "btime": "2024-09-21T03:28:56:681239+0000", 2024-09-21T03:29:04.631 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-09-21T03:29:04.631 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-09-21T03:29:04.631 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout }, 2024-09-21T03:29:04.632 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-09-21T03:29:04.632 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "available": false, 2024-09-21T03:29:04.632 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-09-21T03:29:04.632 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "modules": [ 2024-09-21T03:29:04.632 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "iostat", 2024-09-21T03:29:04.632 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "nfs", 2024-09-21T03:29:04.632 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "restful" 2024-09-21T03:29:04.632 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout ], 2024-09-21T03:29:04.632 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-21T03:29:04.632 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout }, 2024-09-21T03:29:04.632 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-09-21T03:29:04.632 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-21T03:29:04.632 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "modified": "2024-09-21T03:28:56.682056+0000", 2024-09-21T03:29:04.632 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-21T03:29:04.632 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout }, 2024-09-21T03:29:04.633 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-09-21T03:29:04.633 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout } 2024-09-21T03:29:04.633 INFO:teuthology.orchestra.run.smithi089.stdout:mgr not available, waiting (1/15)... 2024-09-21T03:29:07.560 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:29:07 smithi089 ceph-mon[30928]: from='client.? 172.21.15.89:0/1210219625' entity='client.admin' cmd={"prefix": "status", "format": "json-pretty"} : dispatch 2024-09-21T03:29:07.611 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout 2024-09-21T03:29:07.612 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout { 2024-09-21T03:29:07.612 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "fsid": "7c374098-77c9-11ef-baf5-efdc52797490", 2024-09-21T03:29:07.612 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "health": { 2024-09-21T03:29:07.612 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-09-21T03:29:07.612 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-09-21T03:29:07.612 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-09-21T03:29:07.612 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout }, 2024-09-21T03:29:07.612 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-09-21T03:29:07.612 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-09-21T03:29:07.612 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout 0 2024-09-21T03:29:07.612 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout ], 2024-09-21T03:29:07.612 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-09-21T03:29:07.612 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "a" 2024-09-21T03:29:07.612 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout ], 2024-09-21T03:29:07.612 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "quorum_age": 5, 2024-09-21T03:29:07.613 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "monmap": { 2024-09-21T03:29:07.613 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-21T03:29:07.613 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-09-21T03:29:07.613 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-09-21T03:29:07.613 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout }, 2024-09-21T03:29:07.613 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-09-21T03:29:07.613 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-21T03:29:07.613 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-09-21T03:29:07.613 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-09-21T03:29:07.613 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-09-21T03:29:07.617 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-09-21T03:29:07.617 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-09-21T03:29:07.617 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-09-21T03:29:07.617 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout }, 2024-09-21T03:29:07.617 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-09-21T03:29:07.617 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-09-21T03:29:07.617 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-09-21T03:29:07.617 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-09-21T03:29:07.617 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-09-21T03:29:07.618 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-09-21T03:29:07.618 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-09-21T03:29:07.618 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-09-21T03:29:07.618 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-09-21T03:29:07.618 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout }, 2024-09-21T03:29:07.618 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-09-21T03:29:07.618 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-21T03:29:07.618 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "btime": "2024-09-21T03:28:56:681239+0000", 2024-09-21T03:29:07.618 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-09-21T03:29:07.618 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-09-21T03:29:07.618 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout }, 2024-09-21T03:29:07.618 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-09-21T03:29:07.618 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "available": false, 2024-09-21T03:29:07.618 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-09-21T03:29:07.619 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "modules": [ 2024-09-21T03:29:07.619 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "iostat", 2024-09-21T03:29:07.619 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "nfs", 2024-09-21T03:29:07.619 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "restful" 2024-09-21T03:29:07.619 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout ], 2024-09-21T03:29:07.619 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-21T03:29:07.619 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout }, 2024-09-21T03:29:07.619 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-09-21T03:29:07.619 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-21T03:29:07.619 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "modified": "2024-09-21T03:28:56.682056+0000", 2024-09-21T03:29:07.619 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-21T03:29:07.619 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout }, 2024-09-21T03:29:07.619 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-09-21T03:29:07.619 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout } 2024-09-21T03:29:07.619 INFO:teuthology.orchestra.run.smithi089.stdout:mgr not available, waiting (2/15)... 2024-09-21T03:29:08.811 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:29:08 smithi089 ceph-mon[30928]: Activating manager daemon x 2024-09-21T03:29:08.811 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:29:08 smithi089 ceph-mon[30928]: mgrmap e2: x(active, starting, since 0.00400367s) 2024-09-21T03:29:08.811 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:29:08 smithi089 ceph-mon[30928]: from='mgr.14100 172.21.15.89:0/363629460' entity='mgr.x' cmd={"prefix": "mds metadata"} : dispatch 2024-09-21T03:29:08.811 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:29:08 smithi089 ceph-mon[30928]: from='mgr.14100 172.21.15.89:0/363629460' entity='mgr.x' cmd={"prefix": "osd metadata"} : dispatch 2024-09-21T03:29:08.811 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:29:08 smithi089 ceph-mon[30928]: from='mgr.14100 172.21.15.89:0/363629460' entity='mgr.x' cmd={"prefix": "mon metadata"} : dispatch 2024-09-21T03:29:08.811 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:29:08 smithi089 ceph-mon[30928]: from='mgr.14100 172.21.15.89:0/363629460' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-21T03:29:08.811 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:29:08 smithi089 ceph-mon[30928]: from='mgr.14100 172.21.15.89:0/363629460' entity='mgr.x' cmd={"prefix": "mgr metadata", "who": "x", "id": "x"} : dispatch 2024-09-21T03:29:08.811 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:29:08 smithi089 ceph-mon[30928]: Manager daemon x is now available 2024-09-21T03:29:08.812 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:29:08 smithi089 ceph-mon[30928]: from='mgr.14100 172.21.15.89:0/363629460' entity='mgr.x' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"} : dispatch 2024-09-21T03:29:08.812 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:29:08 smithi089 ceph-mon[30928]: from='mgr.14100 172.21.15.89:0/363629460' entity='mgr.x' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"} : dispatch 2024-09-21T03:29:08.812 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:29:08 smithi089 ceph-mon[30928]: from='mgr.14100 172.21.15.89:0/363629460' entity='mgr.x' 2024-09-21T03:29:08.812 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:29:08 smithi089 ceph-mon[30928]: from='mgr.14100 172.21.15.89:0/363629460' entity='mgr.x' 2024-09-21T03:29:08.812 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:29:08 smithi089 ceph-mon[30928]: from='mgr.14100 172.21.15.89:0/363629460' entity='mgr.x' 2024-09-21T03:29:10.561 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:29:10 smithi089 ceph-mon[30928]: mgrmap e3: x(active, since 1.00728s) 2024-09-21T03:29:10.561 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:29:10 smithi089 ceph-mon[30928]: from='client.? 172.21.15.89:0/1982300490' entity='client.admin' cmd={"prefix": "status", "format": "json-pretty"} : dispatch 2024-09-21T03:29:10.817 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout 2024-09-21T03:29:10.817 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout { 2024-09-21T03:29:10.818 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "fsid": "7c374098-77c9-11ef-baf5-efdc52797490", 2024-09-21T03:29:10.818 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "health": { 2024-09-21T03:29:10.818 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-09-21T03:29:10.818 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-09-21T03:29:10.818 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-09-21T03:29:10.818 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout }, 2024-09-21T03:29:10.818 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-09-21T03:29:10.818 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-09-21T03:29:10.818 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout 0 2024-09-21T03:29:10.818 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout ], 2024-09-21T03:29:10.818 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-09-21T03:29:10.818 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "a" 2024-09-21T03:29:10.818 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout ], 2024-09-21T03:29:10.818 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "quorum_age": 8, 2024-09-21T03:29:10.819 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "monmap": { 2024-09-21T03:29:10.819 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-21T03:29:10.819 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-09-21T03:29:10.819 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-09-21T03:29:10.819 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout }, 2024-09-21T03:29:10.819 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-09-21T03:29:10.819 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-21T03:29:10.819 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-09-21T03:29:10.819 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-09-21T03:29:10.819 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-09-21T03:29:10.822 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-09-21T03:29:10.823 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-09-21T03:29:10.823 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-09-21T03:29:10.823 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout }, 2024-09-21T03:29:10.823 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-09-21T03:29:10.823 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-09-21T03:29:10.823 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-09-21T03:29:10.823 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-09-21T03:29:10.823 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-09-21T03:29:10.823 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-09-21T03:29:10.823 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-09-21T03:29:10.823 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-09-21T03:29:10.823 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-09-21T03:29:10.823 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout }, 2024-09-21T03:29:10.824 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-09-21T03:29:10.824 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-21T03:29:10.824 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "btime": "2024-09-21T03:28:56:681239+0000", 2024-09-21T03:29:10.824 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-09-21T03:29:10.824 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-09-21T03:29:10.824 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout }, 2024-09-21T03:29:10.824 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-09-21T03:29:10.824 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "available": true, 2024-09-21T03:29:10.824 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-09-21T03:29:10.824 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "modules": [ 2024-09-21T03:29:10.824 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "iostat", 2024-09-21T03:29:10.824 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "nfs", 2024-09-21T03:29:10.824 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "restful" 2024-09-21T03:29:10.824 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout ], 2024-09-21T03:29:10.825 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-21T03:29:10.825 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout }, 2024-09-21T03:29:10.825 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-09-21T03:29:10.825 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-21T03:29:10.825 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "modified": "2024-09-21T03:28:56.682056+0000", 2024-09-21T03:29:10.825 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-21T03:29:10.825 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout }, 2024-09-21T03:29:10.825 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-09-21T03:29:10.825 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout } 2024-09-21T03:29:10.825 INFO:teuthology.orchestra.run.smithi089.stdout:mgr is available 2024-09-21T03:29:11.811 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:29:11 smithi089 ceph-mon[30928]: mgrmap e4: x(active, since 2s) 2024-09-21T03:29:11.811 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:29:11 smithi089 ceph-mon[30928]: from='client.? 172.21.15.89:0/2092401346' entity='client.admin' cmd={"prefix": "config assimilate-conf"} : dispatch 2024-09-21T03:29:17.607 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout 2024-09-21T03:29:17.608 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout [global] 2024-09-21T03:29:17.608 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout fsid = 7c374098-77c9-11ef-baf5-efdc52797490 2024-09-21T03:29:17.608 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout mon_cluster_log_file_level = debug 2024-09-21T03:29:17.608 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.89:3300,v1:172.21.15.89:6789] 2024-09-21T03:29:17.608 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-09-21T03:29:17.608 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-09-21T03:29:17.608 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-09-21T03:29:17.608 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-09-21T03:29:17.608 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout 2024-09-21T03:29:17.608 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout [mgr] 2024-09-21T03:29:17.608 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-09-21T03:29:17.608 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout 2024-09-21T03:29:17.608 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout [osd] 2024-09-21T03:29:17.608 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-09-21T03:29:17.608 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-09-21T03:29:17.609 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-09-21T03:29:17.609 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-09-21T03:29:17.609 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-09-21T03:29:17.609 INFO:teuthology.orchestra.run.smithi089.stdout:Enabling cephadm module... 2024-09-21T03:29:21.311 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:29:20 smithi089 ceph-mon[30928]: from='client.? 172.21.15.89:0/4263268320' entity='client.admin' cmd={"prefix": "mgr module enable", "module": "cephadm"} : dispatch 2024-09-21T03:29:21.312 INFO:journalctl@ceph.mgr.x.smithi089.stdout:Sep 21 03:29:21 smithi089 ceph-7c374098-77c9-11ef-baf5-efdc52797490-mgr-x[31154]: ignoring --setuser ceph since I am not root 2024-09-21T03:29:21.312 INFO:journalctl@ceph.mgr.x.smithi089.stdout:Sep 21 03:29:21 smithi089 ceph-7c374098-77c9-11ef-baf5-efdc52797490-mgr-x[31154]: ignoring --setgroup ceph since I am not root 2024-09-21T03:29:22.311 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:29:21 smithi089 ceph-mon[30928]: from='client.? 172.21.15.89:0/4263268320' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-09-21T03:29:22.311 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:29:21 smithi089 ceph-mon[30928]: mgrmap e5: x(active, since 12s) 2024-09-21T03:29:26.061 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:29:25 smithi089 ceph-mon[30928]: from='client.? 172.21.15.89:0/1017490508' entity='client.admin' cmd={"prefix": "mgr stat"} : dispatch 2024-09-21T03:29:26.344 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout { 2024-09-21T03:29:26.344 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-09-21T03:29:26.344 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "available": true, 2024-09-21T03:29:26.344 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-09-21T03:29:26.344 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-09-21T03:29:26.344 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout } 2024-09-21T03:29:26.344 INFO:teuthology.orchestra.run.smithi089.stdout:Waiting for the mgr to restart... 2024-09-21T03:29:26.345 INFO:teuthology.orchestra.run.smithi089.stdout:Waiting for mgr epoch 5... 2024-09-21T03:29:27.311 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:29:26 smithi089 ceph-mon[30928]: Active manager daemon x restarted 2024-09-21T03:29:27.311 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:29:26 smithi089 ceph-mon[30928]: Activating manager daemon x 2024-09-21T03:29:27.311 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:29:26 smithi089 ceph-mon[30928]: osdmap e2: 0 total, 0 up, 0 in 2024-09-21T03:29:27.311 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:29:26 smithi089 ceph-mon[30928]: mgrmap e6: x(active, starting, since 0.00406263s) 2024-09-21T03:29:27.311 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:29:26 smithi089 ceph-mon[30928]: from='mgr.14118 172.21.15.89:0/1900267975' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-21T03:29:27.311 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:29:26 smithi089 ceph-mon[30928]: from='mgr.14118 172.21.15.89:0/1900267975' entity='mgr.x' cmd={"prefix": "mgr metadata", "who": "x", "id": "x"} : dispatch 2024-09-21T03:29:27.312 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:29:26 smithi089 ceph-mon[30928]: from='mgr.14118 172.21.15.89:0/1900267975' entity='mgr.x' cmd={"prefix": "mds metadata"} : dispatch 2024-09-21T03:29:27.312 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:29:26 smithi089 ceph-mon[30928]: from='mgr.14118 172.21.15.89:0/1900267975' entity='mgr.x' cmd={"prefix": "osd metadata"} : dispatch 2024-09-21T03:29:27.312 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:29:26 smithi089 ceph-mon[30928]: from='mgr.14118 172.21.15.89:0/1900267975' entity='mgr.x' cmd={"prefix": "mon metadata"} : dispatch 2024-09-21T03:29:27.312 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:29:26 smithi089 ceph-mon[30928]: Manager daemon x is now available 2024-09-21T03:29:27.312 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:29:26 smithi089 ceph-mon[30928]: from='mgr.14118 172.21.15.89:0/1900267975' entity='mgr.x' 2024-09-21T03:29:27.312 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:29:26 smithi089 ceph-mon[30928]: from='mgr.14118 172.21.15.89:0/1900267975' entity='mgr.x' 2024-09-21T03:29:27.312 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:29:26 smithi089 ceph-mon[30928]: from='mgr.14118 172.21.15.89:0/1900267975' entity='mgr.x' 2024-09-21T03:29:27.312 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:29:26 smithi089 ceph-mon[30928]: from='mgr.14118 172.21.15.89:0/1900267975' entity='mgr.x' 2024-09-21T03:29:27.312 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:29:26 smithi089 ceph-mon[30928]: from='mgr.14118 172.21.15.89:0/1900267975' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:29:27.312 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:29:26 smithi089 ceph-mon[30928]: from='mgr.14118 172.21.15.89:0/1900267975' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:29:27.312 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:29:26 smithi089 ceph-mon[30928]: from='mgr.14118 172.21.15.89:0/1900267975' entity='mgr.x' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"} : dispatch 2024-09-21T03:29:27.312 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:29:26 smithi089 ceph-mon[30928]: from='mgr.14118 172.21.15.89:0/1900267975' entity='mgr.x' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"} : dispatch 2024-09-21T03:29:27.577 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout { 2024-09-21T03:29:27.577 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-09-21T03:29:27.578 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "initialized": true 2024-09-21T03:29:27.578 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout } 2024-09-21T03:29:27.578 INFO:teuthology.orchestra.run.smithi089.stdout:mgr epoch 5 is available 2024-09-21T03:29:27.578 INFO:teuthology.orchestra.run.smithi089.stdout:Verifying orchestrator module is enabled... 2024-09-21T03:29:27.902 INFO:journalctl@ceph.mgr.x.smithi089.stdout:Sep 21 03:29:27 smithi089 ceph-7c374098-77c9-11ef-baf5-efdc52797490-mgr-x[31154]: 2024-09-21T03:29:27.807+0000 7f1c55c2b640 -1 log_channel(cephadm) log [ERR] : [21/Sep/2024:03:29:27] ENGINE Error in HTTPServer.serve 2024-09-21T03:29:27.902 INFO:journalctl@ceph.mgr.x.smithi089.stdout:Sep 21 03:29:27 smithi089 ceph-7c374098-77c9-11ef-baf5-efdc52797490-mgr-x[31154]: Traceback (most recent call last): 2024-09-21T03:29:27.903 INFO:journalctl@ceph.mgr.x.smithi089.stdout:Sep 21 03:29:27 smithi089 ceph-7c374098-77c9-11ef-baf5-efdc52797490-mgr-x[31154]: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-21T03:29:27.903 INFO:journalctl@ceph.mgr.x.smithi089.stdout:Sep 21 03:29:27 smithi089 ceph-7c374098-77c9-11ef-baf5-efdc52797490-mgr-x[31154]: self._connections.run(self.expiration_interval) 2024-09-21T03:29:27.903 INFO:journalctl@ceph.mgr.x.smithi089.stdout:Sep 21 03:29:27 smithi089 ceph-7c374098-77c9-11ef-baf5-efdc52797490-mgr-x[31154]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-21T03:29:27.903 INFO:journalctl@ceph.mgr.x.smithi089.stdout:Sep 21 03:29:27 smithi089 ceph-7c374098-77c9-11ef-baf5-efdc52797490-mgr-x[31154]: self._run(expiration_interval) 2024-09-21T03:29:27.903 INFO:journalctl@ceph.mgr.x.smithi089.stdout:Sep 21 03:29:27 smithi089 ceph-7c374098-77c9-11ef-baf5-efdc52797490-mgr-x[31154]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-21T03:29:27.903 INFO:journalctl@ceph.mgr.x.smithi089.stdout:Sep 21 03:29:27 smithi089 ceph-7c374098-77c9-11ef-baf5-efdc52797490-mgr-x[31154]: new_conn = self._from_server_socket(self.server.socket) 2024-09-21T03:29:27.903 INFO:journalctl@ceph.mgr.x.smithi089.stdout:Sep 21 03:29:27 smithi089 ceph-7c374098-77c9-11ef-baf5-efdc52797490-mgr-x[31154]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-21T03:29:27.903 INFO:journalctl@ceph.mgr.x.smithi089.stdout:Sep 21 03:29:27 smithi089 ceph-7c374098-77c9-11ef-baf5-efdc52797490-mgr-x[31154]: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-21T03:29:27.903 INFO:journalctl@ceph.mgr.x.smithi089.stdout:Sep 21 03:29:27 smithi089 ceph-7c374098-77c9-11ef-baf5-efdc52797490-mgr-x[31154]: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-21T03:29:27.903 INFO:journalctl@ceph.mgr.x.smithi089.stdout:Sep 21 03:29:27 smithi089 ceph-7c374098-77c9-11ef-baf5-efdc52797490-mgr-x[31154]: s = self.context.wrap_socket( 2024-09-21T03:29:27.903 INFO:journalctl@ceph.mgr.x.smithi089.stdout:Sep 21 03:29:27 smithi089 ceph-7c374098-77c9-11ef-baf5-efdc52797490-mgr-x[31154]: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-21T03:29:27.903 INFO:journalctl@ceph.mgr.x.smithi089.stdout:Sep 21 03:29:27 smithi089 ceph-7c374098-77c9-11ef-baf5-efdc52797490-mgr-x[31154]: return self.sslsocket_class._create( 2024-09-21T03:29:27.903 INFO:journalctl@ceph.mgr.x.smithi089.stdout:Sep 21 03:29:27 smithi089 ceph-7c374098-77c9-11ef-baf5-efdc52797490-mgr-x[31154]: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-21T03:29:27.904 INFO:journalctl@ceph.mgr.x.smithi089.stdout:Sep 21 03:29:27 smithi089 ceph-7c374098-77c9-11ef-baf5-efdc52797490-mgr-x[31154]: self.do_handshake() 2024-09-21T03:29:27.904 INFO:journalctl@ceph.mgr.x.smithi089.stdout:Sep 21 03:29:27 smithi089 ceph-7c374098-77c9-11ef-baf5-efdc52797490-mgr-x[31154]: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-21T03:29:27.904 INFO:journalctl@ceph.mgr.x.smithi089.stdout:Sep 21 03:29:27 smithi089 ceph-7c374098-77c9-11ef-baf5-efdc52797490-mgr-x[31154]: self._sslobj.do_handshake() 2024-09-21T03:29:27.904 INFO:journalctl@ceph.mgr.x.smithi089.stdout:Sep 21 03:29:27 smithi089 ceph-7c374098-77c9-11ef-baf5-efdc52797490-mgr-x[31154]: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-21T03:29:27.904 INFO:journalctl@ceph.mgr.x.smithi089.stdout:Sep 21 03:29:27 smithi089 ceph-7c374098-77c9-11ef-baf5-efdc52797490-mgr-x[31154]: 2024-09-21T03:29:28.311 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:29:28 smithi089 ceph-mon[30928]: Found migration_current of "None". Setting to last migration. 2024-09-21T03:29:28.311 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:29:28 smithi089 ceph-mon[30928]: from='client.14128 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-09-21T03:29:28.311 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:29:28 smithi089 ceph-mon[30928]: mgrmap e7: x(active, since 1.00659s) 2024-09-21T03:29:28.311 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:29:28 smithi089 ceph-mon[30928]: from='client.14128 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-09-21T03:29:28.311 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:29:28 smithi089 ceph-mon[30928]: from='mgr.14118 172.21.15.89:0/1900267975' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:29:29.311 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:29:29 smithi089 ceph-mon[30928]: [21/Sep/2024:03:29:27] ENGINE Bus STARTING 2024-09-21T03:29:29.311 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:29:29 smithi089 ceph-mon[30928]: [21/Sep/2024:03:29:27] ENGINE Serving on https://172.21.15.89:7150 2024-09-21T03:29:29.311 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:29:29 smithi089 ceph-mon[30928]: [21/Sep/2024:03:29:27] ENGINE Error in HTTPServer.serve 2024-09-21T03:29:29.311 INFO:journalctl@ceph.mon.a.smithi089.stdout: Traceback (most recent call last): 2024-09-21T03:29:29.311 INFO:journalctl@ceph.mon.a.smithi089.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-21T03:29:29.311 INFO:journalctl@ceph.mon.a.smithi089.stdout: self._connections.run(self.expiration_interval) 2024-09-21T03:29:29.311 INFO:journalctl@ceph.mon.a.smithi089.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-21T03:29:29.311 INFO:journalctl@ceph.mon.a.smithi089.stdout: self._run(expiration_interval) 2024-09-21T03:29:29.312 INFO:journalctl@ceph.mon.a.smithi089.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-21T03:29:29.312 INFO:journalctl@ceph.mon.a.smithi089.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-21T03:29:29.312 INFO:journalctl@ceph.mon.a.smithi089.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-21T03:29:29.312 INFO:journalctl@ceph.mon.a.smithi089.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-21T03:29:29.312 INFO:journalctl@ceph.mon.a.smithi089.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-21T03:29:29.312 INFO:journalctl@ceph.mon.a.smithi089.stdout: s = self.context.wrap_socket( 2024-09-21T03:29:29.312 INFO:journalctl@ceph.mon.a.smithi089.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-21T03:29:29.312 INFO:journalctl@ceph.mon.a.smithi089.stdout: return self.sslsocket_class._create( 2024-09-21T03:29:29.312 INFO:journalctl@ceph.mon.a.smithi089.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-21T03:29:29.312 INFO:journalctl@ceph.mon.a.smithi089.stdout: self.do_handshake() 2024-09-21T03:29:29.312 INFO:journalctl@ceph.mon.a.smithi089.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-21T03:29:29.312 INFO:journalctl@ceph.mon.a.smithi089.stdout: self._sslobj.do_handshake() 2024-09-21T03:29:29.312 INFO:journalctl@ceph.mon.a.smithi089.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-21T03:29:29.312 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:29:29 smithi089 ceph-mon[30928]: [21/Sep/2024:03:29:27] ENGINE Serving on http://172.21.15.89:8765 2024-09-21T03:29:29.312 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:29:29 smithi089 ceph-mon[30928]: [21/Sep/2024:03:29:27] ENGINE Bus STARTED 2024-09-21T03:29:29.313 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:29:29 smithi089 ceph-mon[30928]: mgrmap e8: x(active, since 2s) 2024-09-21T03:29:29.313 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:29:29 smithi089 ceph-mon[30928]: from='client.? 172.21.15.89:0/310806474' entity='client.admin' cmd={"prefix": "mgr module enable", "module": "orchestrator"} : dispatch 2024-09-21T03:29:29.542 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr module 'orchestrator' is already enabled (always-on) 2024-09-21T03:29:29.542 INFO:teuthology.orchestra.run.smithi089.stdout:Setting orchestrator backend to cephadm... 2024-09-21T03:29:30.177 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:29:30 smithi089 ceph-mon[30928]: from='client.? 172.21.15.89:0/310806474' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "orchestrator"}]': finished 2024-09-21T03:29:30.177 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:29:30 smithi089 ceph-mon[30928]: mgrmap e9: x(active, since 3s) 2024-09-21T03:29:31.301 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:29:31 smithi089 ceph-mon[30928]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:29:31.301 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:29:31 smithi089 ceph-mon[30928]: from='mgr.14118 172.21.15.89:0/1900267975' entity='mgr.x' 2024-09-21T03:29:31.301 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:29:31 smithi089 ceph-mon[30928]: from='mgr.14118 172.21.15.89:0/1900267975' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:29:31.813 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout value unchanged 2024-09-21T03:29:31.813 INFO:teuthology.orchestra.run.smithi089.stdout:Generating ssh key... 2024-09-21T03:29:32.188 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:29:32 smithi089 ceph-mon[30928]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:29:32.900 INFO:journalctl@ceph.mgr.x.smithi089.stdout:Sep 21 03:29:32 smithi089 ceph-7c374098-77c9-11ef-baf5-efdc52797490-mgr-x[31154]: Generating public/private rsa key pair. 2024-09-21T03:29:32.900 INFO:journalctl@ceph.mgr.x.smithi089.stdout:Sep 21 03:29:32 smithi089 ceph-7c374098-77c9-11ef-baf5-efdc52797490-mgr-x[31154]: Your identification has been saved in /tmp/tmppjta9g7d/key 2024-09-21T03:29:32.900 INFO:journalctl@ceph.mgr.x.smithi089.stdout:Sep 21 03:29:32 smithi089 ceph-7c374098-77c9-11ef-baf5-efdc52797490-mgr-x[31154]: Your public key has been saved in /tmp/tmppjta9g7d/key.pub 2024-09-21T03:29:32.900 INFO:journalctl@ceph.mgr.x.smithi089.stdout:Sep 21 03:29:32 smithi089 ceph-7c374098-77c9-11ef-baf5-efdc52797490-mgr-x[31154]: The key fingerprint is: 2024-09-21T03:29:32.901 INFO:journalctl@ceph.mgr.x.smithi089.stdout:Sep 21 03:29:32 smithi089 ceph-7c374098-77c9-11ef-baf5-efdc52797490-mgr-x[31154]: SHA256:EIE8kdL+zQDJXFdm0jqRNK/gWe22JbUUAWH3fLQK60g ceph-7c374098-77c9-11ef-baf5-efdc52797490 2024-09-21T03:29:32.901 INFO:journalctl@ceph.mgr.x.smithi089.stdout:Sep 21 03:29:32 smithi089 ceph-7c374098-77c9-11ef-baf5-efdc52797490-mgr-x[31154]: The key's randomart image is: 2024-09-21T03:29:32.901 INFO:journalctl@ceph.mgr.x.smithi089.stdout:Sep 21 03:29:32 smithi089 ceph-7c374098-77c9-11ef-baf5-efdc52797490-mgr-x[31154]: +---[RSA 3072]----+ 2024-09-21T03:29:32.901 INFO:journalctl@ceph.mgr.x.smithi089.stdout:Sep 21 03:29:32 smithi089 ceph-7c374098-77c9-11ef-baf5-efdc52797490-mgr-x[31154]: | =.*+o=+Bo+. .| 2024-09-21T03:29:32.901 INFO:journalctl@ceph.mgr.x.smithi089.stdout:Sep 21 03:29:32 smithi089 ceph-7c374098-77c9-11ef-baf5-efdc52797490-mgr-x[31154]: | . @ ooO.. + ..| 2024-09-21T03:29:32.901 INFO:journalctl@ceph.mgr.x.smithi089.stdout:Sep 21 03:29:32 smithi089 ceph-7c374098-77c9-11ef-baf5-efdc52797490-mgr-x[31154]: | o oo .oo.o o..| 2024-09-21T03:29:32.901 INFO:journalctl@ceph.mgr.x.smithi089.stdout:Sep 21 03:29:32 smithi089 ceph-7c374098-77c9-11ef-baf5-efdc52797490-mgr-x[31154]: | ...=oo oo... | 2024-09-21T03:29:32.901 INFO:journalctl@ceph.mgr.x.smithi089.stdout:Sep 21 03:29:32 smithi089 ceph-7c374098-77c9-11ef-baf5-efdc52797490-mgr-x[31154]: | .o+SE+.o. | 2024-09-21T03:29:32.901 INFO:journalctl@ceph.mgr.x.smithi089.stdout:Sep 21 03:29:32 smithi089 ceph-7c374098-77c9-11ef-baf5-efdc52797490-mgr-x[31154]: | . +.o+ | 2024-09-21T03:29:32.901 INFO:journalctl@ceph.mgr.x.smithi089.stdout:Sep 21 03:29:32 smithi089 ceph-7c374098-77c9-11ef-baf5-efdc52797490-mgr-x[31154]: | ... | 2024-09-21T03:29:32.901 INFO:journalctl@ceph.mgr.x.smithi089.stdout:Sep 21 03:29:32 smithi089 ceph-7c374098-77c9-11ef-baf5-efdc52797490-mgr-x[31154]: | | 2024-09-21T03:29:32.901 INFO:journalctl@ceph.mgr.x.smithi089.stdout:Sep 21 03:29:32 smithi089 ceph-7c374098-77c9-11ef-baf5-efdc52797490-mgr-x[31154]: | | 2024-09-21T03:29:32.901 INFO:journalctl@ceph.mgr.x.smithi089.stdout:Sep 21 03:29:32 smithi089 ceph-7c374098-77c9-11ef-baf5-efdc52797490-mgr-x[31154]: +----[SHA256]-----+ 2024-09-21T03:29:33.737 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:29:33 smithi089 ceph-mon[30928]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:29:33.737 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:29:33 smithi089 ceph-mon[30928]: Generating ssh key... 2024-09-21T03:29:33.737 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:29:33 smithi089 ceph-mon[30928]: from='mgr.14118 172.21.15.89:0/1900267975' entity='mgr.x' 2024-09-21T03:29:33.737 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:29:33 smithi089 ceph-mon[30928]: from='mgr.14118 172.21.15.89:0/1900267975' entity='mgr.x' 2024-09-21T03:29:34.326 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDLqsiRutqhAyrI9d3w8MSil+haY3hjFAxlCWm3cqoc5TqprQ8dj+0hPiaD52CJNa3tYxfl+jxR7wCU6JekcZ1HRi0i2pMCCAkvOe0X9xRDIkJjn2FaP9595rlcO2iOCbyEwq7Yhg2Wa52hbCIpElu1BrOofBks8sAJgKC/IfEiunVLZDmYxsUt222J7aJBfPzAUxTJgX1DGvZYczngJiPTHRafgRKdKaGxl0iGlPdfOG2umPQrmNIPq5cU3HplFKOKqFgctYZ+3wknCxKvxkQtdbbqS5d6RLbSsgvAp2wCl9RNw9ZeAwpqQVHVrOUb52gD9ntdq3VibRCYjhnHobCedUMFJQpJRz9+f+LfrQN+g8cruh+r1vpZ0jGTyx0lJN0I86JgzpTMH7YYDVuDIif8ne7wgvuK1tuZGRyK5U746cm7liFiaPkMTslp8Rb0Vk4c4LlB757zZIjYYRKRI4MUj4ExJTpPot/+11OWIfhedRsZdeZiSrbsWOVBKIurOqk= ceph-7c374098-77c9-11ef-baf5-efdc52797490 2024-09-21T03:29:34.327 INFO:teuthology.orchestra.run.smithi089.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-09-21T03:29:34.327 INFO:teuthology.orchestra.run.smithi089.stdout:Adding key to root@localhost authorized_keys... 2024-09-21T03:29:34.334 INFO:teuthology.orchestra.run.smithi089.stdout:Adding host smithi089... 2024-09-21T03:29:35.040 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:29:34 smithi089 ceph-mon[30928]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:29:35.685 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:29:35 smithi089 ceph-mon[30928]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi089", "addr": "172.21.15.89", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:29:37.061 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:29:36 smithi089 ceph-mon[30928]: Deploying cephadm binary to smithi089 2024-09-21T03:29:38.686 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout Added host 'smithi089' with addr '172.21.15.89' 2024-09-21T03:29:38.686 INFO:teuthology.orchestra.run.smithi089.stdout:Deploying unmanaged mon service... 2024-09-21T03:29:39.290 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:29:38 smithi089 ceph-mon[30928]: from='mgr.14118 172.21.15.89:0/1900267975' entity='mgr.x' 2024-09-21T03:29:39.290 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:29:38 smithi089 ceph-mon[30928]: Added host smithi089 2024-09-21T03:29:39.290 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:29:38 smithi089 ceph-mon[30928]: from='mgr.14118 172.21.15.89:0/1900267975' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:29:40.281 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-09-21T03:29:40.281 INFO:teuthology.orchestra.run.smithi089.stdout:Deploying unmanaged mgr service... 2024-09-21T03:29:40.561 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:29:40 smithi089 ceph-mon[30928]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:29:40.561 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:29:40 smithi089 ceph-mon[30928]: Saving service mon spec with placement count:5 2024-09-21T03:29:40.561 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:29:40 smithi089 ceph-mon[30928]: from='mgr.14118 172.21.15.89:0/1900267975' entity='mgr.x' 2024-09-21T03:29:41.811 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:29:41 smithi089 ceph-mon[30928]: from='mgr.14118 172.21.15.89:0/1900267975' entity='mgr.x' 2024-09-21T03:29:41.811 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:29:41 smithi089 ceph-mon[30928]: from='mgr.14118 172.21.15.89:0/1900267975' entity='mgr.x' 2024-09-21T03:29:41.811 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:29:41 smithi089 ceph-mon[30928]: from='mgr.14118 172.21.15.89:0/1900267975' entity='mgr.x' 2024-09-21T03:29:42.576 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-09-21T03:29:42.910 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:29:42 smithi089 ceph-mon[30928]: from='client.14144 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:29:42.910 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:29:42 smithi089 ceph-mon[30928]: Saving service mgr spec with placement count:2 2024-09-21T03:29:42.910 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:29:42 smithi089 ceph-mon[30928]: from='mgr.14118 172.21.15.89:0/1900267975' entity='mgr.x' 2024-09-21T03:29:44.166 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:29:44 smithi089 ceph-mon[30928]: from='mgr.14118 172.21.15.89:0/1900267975' entity='mgr.x' 2024-09-21T03:29:44.166 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:29:44 smithi089 ceph-mon[30928]: from='client.? 172.21.15.89:0/2037437775' entity='client.admin' 2024-09-21T03:29:44.925 INFO:teuthology.orchestra.run.smithi089.stdout:Enabling the dashboard module... 2024-09-21T03:29:45.500 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:29:45 smithi089 ceph-mon[30928]: from='client.? 172.21.15.89:0/1106817356' entity='client.admin' 2024-09-21T03:29:46.733 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:29:46 smithi089 ceph-mon[30928]: from='client.? 172.21.15.89:0/1920345284' entity='client.admin' cmd={"prefix": "mgr module enable", "module": "dashboard"} : dispatch 2024-09-21T03:29:46.733 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:29:46 smithi089 ceph-mon[30928]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:29:46.733 INFO:journalctl@ceph.mgr.x.smithi089.stdout:Sep 21 03:29:46 smithi089 ceph-7c374098-77c9-11ef-baf5-efdc52797490-mgr-x[31154]: ignoring --setuser ceph since I am not root 2024-09-21T03:29:46.733 INFO:journalctl@ceph.mgr.x.smithi089.stdout:Sep 21 03:29:46 smithi089 ceph-7c374098-77c9-11ef-baf5-efdc52797490-mgr-x[31154]: ignoring --setgroup ceph since I am not root 2024-09-21T03:29:47.654 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:29:47 smithi089 ceph-mon[30928]: from='client.? 172.21.15.89:0/1920345284' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-09-21T03:29:47.654 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:29:47 smithi089 ceph-mon[30928]: mgrmap e10: x(active, since 20s) 2024-09-21T03:29:48.245 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout { 2024-09-21T03:29:48.246 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "epoch": 10, 2024-09-21T03:29:48.246 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "available": true, 2024-09-21T03:29:48.246 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-09-21T03:29:48.246 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-09-21T03:29:48.246 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout } 2024-09-21T03:29:48.246 INFO:teuthology.orchestra.run.smithi089.stdout:Waiting for the mgr to restart... 2024-09-21T03:29:48.246 INFO:teuthology.orchestra.run.smithi089.stdout:Waiting for mgr epoch 10... 2024-09-21T03:29:48.559 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:29:48 smithi089 ceph-mon[30928]: from='client.? 172.21.15.89:0/2344741714' entity='client.admin' cmd={"prefix": "mgr stat"} : dispatch 2024-09-21T03:29:51.899 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:29:51 smithi089 ceph-mon[30928]: Active manager daemon x restarted 2024-09-21T03:29:51.899 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:29:51 smithi089 ceph-mon[30928]: Activating manager daemon x 2024-09-21T03:29:51.899 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:29:51 smithi089 ceph-mon[30928]: osdmap e3: 0 total, 0 up, 0 in 2024-09-21T03:29:51.899 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:29:51 smithi089 ceph-mon[30928]: mgrmap e11: x(active, starting, since 0.0041339s) 2024-09-21T03:29:51.899 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:29:51 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-21T03:29:51.899 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:29:51 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "mgr metadata", "who": "x", "id": "x"} : dispatch 2024-09-21T03:29:51.899 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:29:51 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "mds metadata"} : dispatch 2024-09-21T03:29:51.899 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:29:51 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "osd metadata"} : dispatch 2024-09-21T03:29:51.899 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:29:51 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "mon metadata"} : dispatch 2024-09-21T03:29:51.900 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:29:51 smithi089 ceph-mon[30928]: Manager daemon x is now available 2024-09-21T03:29:52.811 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:29:52 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:29:52.811 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:29:52 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"} : dispatch 2024-09-21T03:29:52.811 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:29:52 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"} : dispatch 2024-09-21T03:29:52.811 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:29:52 smithi089 ceph-mon[30928]: mgrmap e12: x(active, since 1.00661s) 2024-09-21T03:29:53.092 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout { 2024-09-21T03:29:53.093 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 12, 2024-09-21T03:29:53.093 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "initialized": true 2024-09-21T03:29:53.093 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout } 2024-09-21T03:29:53.093 INFO:teuthology.orchestra.run.smithi089.stdout:mgr epoch 10 is available 2024-09-21T03:29:53.093 INFO:teuthology.orchestra.run.smithi089.stdout:Using certmgr to generate dashboard self-signed certificate... 2024-09-21T03:29:53.753 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:29:53 smithi089 ceph-mon[30928]: [21/Sep/2024:03:29:53] ENGINE Bus STARTING 2024-09-21T03:29:53.753 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:29:53 smithi089 ceph-mon[30928]: [21/Sep/2024:03:29:53] ENGINE Serving on http://172.21.15.89:8765 2024-09-21T03:29:53.753 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:29:53 smithi089 ceph-mon[30928]: [21/Sep/2024:03:29:53] ENGINE Serving on https://172.21.15.89:7150 2024-09-21T03:29:53.753 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:29:53 smithi089 ceph-mon[30928]: [21/Sep/2024:03:29:53] ENGINE Bus STARTED 2024-09-21T03:29:53.753 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:29:53 smithi089 ceph-mon[30928]: [21/Sep/2024:03:29:53] ENGINE Error in HTTPServer.serve 2024-09-21T03:29:53.753 INFO:journalctl@ceph.mon.a.smithi089.stdout: Traceback (most recent call last): 2024-09-21T03:29:53.754 INFO:journalctl@ceph.mon.a.smithi089.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-21T03:29:53.754 INFO:journalctl@ceph.mon.a.smithi089.stdout: self._connections.run(self.expiration_interval) 2024-09-21T03:29:53.754 INFO:journalctl@ceph.mon.a.smithi089.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-21T03:29:53.754 INFO:journalctl@ceph.mon.a.smithi089.stdout: self._run(expiration_interval) 2024-09-21T03:29:53.754 INFO:journalctl@ceph.mon.a.smithi089.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-21T03:29:53.754 INFO:journalctl@ceph.mon.a.smithi089.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-21T03:29:53.754 INFO:journalctl@ceph.mon.a.smithi089.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-21T03:29:53.754 INFO:journalctl@ceph.mon.a.smithi089.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-21T03:29:53.754 INFO:journalctl@ceph.mon.a.smithi089.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-21T03:29:53.754 INFO:journalctl@ceph.mon.a.smithi089.stdout: s = self.context.wrap_socket( 2024-09-21T03:29:53.754 INFO:journalctl@ceph.mon.a.smithi089.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-21T03:29:53.754 INFO:journalctl@ceph.mon.a.smithi089.stdout: return self.sslsocket_class._create( 2024-09-21T03:29:53.754 INFO:journalctl@ceph.mon.a.smithi089.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-21T03:29:53.754 INFO:journalctl@ceph.mon.a.smithi089.stdout: self.do_handshake() 2024-09-21T03:29:53.755 INFO:journalctl@ceph.mon.a.smithi089.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-21T03:29:53.755 INFO:journalctl@ceph.mon.a.smithi089.stdout: self._sslobj.do_handshake() 2024-09-21T03:29:53.755 INFO:journalctl@ceph.mon.a.smithi089.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-21T03:29:53.755 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:29:53 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:29:53.755 INFO:journalctl@ceph.mgr.x.smithi089.stdout:Sep 21 03:29:53 smithi089 ceph-7c374098-77c9-11ef-baf5-efdc52797490-mgr-x[31154]: 2024-09-21T03:29:53.441+0000 7f54a73af640 -1 log_channel(cephadm) log [ERR] : [21/Sep/2024:03:29:53] ENGINE Error in HTTPServer.serve 2024-09-21T03:29:53.755 INFO:journalctl@ceph.mgr.x.smithi089.stdout:Sep 21 03:29:53 smithi089 ceph-7c374098-77c9-11ef-baf5-efdc52797490-mgr-x[31154]: Traceback (most recent call last): 2024-09-21T03:29:53.755 INFO:journalctl@ceph.mgr.x.smithi089.stdout:Sep 21 03:29:53 smithi089 ceph-7c374098-77c9-11ef-baf5-efdc52797490-mgr-x[31154]: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-21T03:29:53.755 INFO:journalctl@ceph.mgr.x.smithi089.stdout:Sep 21 03:29:53 smithi089 ceph-7c374098-77c9-11ef-baf5-efdc52797490-mgr-x[31154]: self._connections.run(self.expiration_interval) 2024-09-21T03:29:53.755 INFO:journalctl@ceph.mgr.x.smithi089.stdout:Sep 21 03:29:53 smithi089 ceph-7c374098-77c9-11ef-baf5-efdc52797490-mgr-x[31154]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-21T03:29:53.755 INFO:journalctl@ceph.mgr.x.smithi089.stdout:Sep 21 03:29:53 smithi089 ceph-7c374098-77c9-11ef-baf5-efdc52797490-mgr-x[31154]: self._run(expiration_interval) 2024-09-21T03:29:53.756 INFO:journalctl@ceph.mgr.x.smithi089.stdout:Sep 21 03:29:53 smithi089 ceph-7c374098-77c9-11ef-baf5-efdc52797490-mgr-x[31154]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-21T03:29:53.756 INFO:journalctl@ceph.mgr.x.smithi089.stdout:Sep 21 03:29:53 smithi089 ceph-7c374098-77c9-11ef-baf5-efdc52797490-mgr-x[31154]: new_conn = self._from_server_socket(self.server.socket) 2024-09-21T03:29:53.756 INFO:journalctl@ceph.mgr.x.smithi089.stdout:Sep 21 03:29:53 smithi089 ceph-7c374098-77c9-11ef-baf5-efdc52797490-mgr-x[31154]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-21T03:29:53.756 INFO:journalctl@ceph.mgr.x.smithi089.stdout:Sep 21 03:29:53 smithi089 ceph-7c374098-77c9-11ef-baf5-efdc52797490-mgr-x[31154]: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-21T03:29:53.756 INFO:journalctl@ceph.mgr.x.smithi089.stdout:Sep 21 03:29:53 smithi089 ceph-7c374098-77c9-11ef-baf5-efdc52797490-mgr-x[31154]: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-21T03:29:53.756 INFO:journalctl@ceph.mgr.x.smithi089.stdout:Sep 21 03:29:53 smithi089 ceph-7c374098-77c9-11ef-baf5-efdc52797490-mgr-x[31154]: s = self.context.wrap_socket( 2024-09-21T03:29:53.756 INFO:journalctl@ceph.mgr.x.smithi089.stdout:Sep 21 03:29:53 smithi089 ceph-7c374098-77c9-11ef-baf5-efdc52797490-mgr-x[31154]: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-21T03:29:53.756 INFO:journalctl@ceph.mgr.x.smithi089.stdout:Sep 21 03:29:53 smithi089 ceph-7c374098-77c9-11ef-baf5-efdc52797490-mgr-x[31154]: return self.sslsocket_class._create( 2024-09-21T03:29:53.756 INFO:journalctl@ceph.mgr.x.smithi089.stdout:Sep 21 03:29:53 smithi089 ceph-7c374098-77c9-11ef-baf5-efdc52797490-mgr-x[31154]: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-21T03:29:53.756 INFO:journalctl@ceph.mgr.x.smithi089.stdout:Sep 21 03:29:53 smithi089 ceph-7c374098-77c9-11ef-baf5-efdc52797490-mgr-x[31154]: self.do_handshake() 2024-09-21T03:29:53.756 INFO:journalctl@ceph.mgr.x.smithi089.stdout:Sep 21 03:29:53 smithi089 ceph-7c374098-77c9-11ef-baf5-efdc52797490-mgr-x[31154]: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-21T03:29:53.756 INFO:journalctl@ceph.mgr.x.smithi089.stdout:Sep 21 03:29:53 smithi089 ceph-7c374098-77c9-11ef-baf5-efdc52797490-mgr-x[31154]: self._sslobj.do_handshake() 2024-09-21T03:29:53.756 INFO:journalctl@ceph.mgr.x.smithi089.stdout:Sep 21 03:29:53 smithi089 ceph-7c374098-77c9-11ef-baf5-efdc52797490-mgr-x[31154]: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-21T03:29:53.756 INFO:journalctl@ceph.mgr.x.smithi089.stdout:Sep 21 03:29:53 smithi089 ceph-7c374098-77c9-11ef-baf5-efdc52797490-mgr-x[31154]: 2024-09-21T03:29:54.811 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:29:54 smithi089 ceph-mon[30928]: mgrmap e13: x(active, since 2s) 2024-09-21T03:29:54.811 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:29:54 smithi089 ceph-mon[30928]: from='client.14164 -' entity='client.admin' cmd=[{"prefix": "orch certmgr generate-certificates", "module_name": "dashboard", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:29:54.811 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:29:54 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:29:56.074 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout {"cert": "-----BEGIN CERTIFICATE-----\nMIIFEDCCAvigAwIBAgIUbnGHSnOIynYrPF0oXDDihTzEa0MwDQYJKoZIhvcNAQEL\nBQAwFzEVMBMGA1UEAwwMY2VwaGFkbS1yb290MB4XDTI0MDkyMTAzMjk1NFoXDTM0\nMDkyMjAzMjk1NFowFzEVMBMGA1UEAwwMMTcyLjIxLjE1Ljg5MIICIjANBgkqhkiG\n9w0BAQEFAAOCAg8AMIICCgKCAgEAtbuR6qnMhzBilA6/9CA0CSHZuJk2FkltXBuK\n4itugAD9i+wgNMfer50Ma/WQhn7r/PGPZ8m8xN9i+19o0ACzO73YBGhtwcW5WDkw\nUSLJDQmdScemPUsekANGaoqSue4QdIF/lHVtFX/sdSuhhABKClS5cwiaWMQNsf8n\nrUtdmgE31a8S/D9wUJop1X1Eezl1WiJlp2dEXgRGAAD2EbVjcqcBT6jj8U/NDga5\nlnpP8De+aZjBa/fTRcZ1thytfSX0fGMNpBGA9uivuc+Pk63JjPdU8+qZc2+7ldLN\nNR4w8bBX3eqJ1XOueX8CXRgAju13rwO9pVxw6znwycDzw0lHRGrD0CjTQ+/7UCND\nP92Nzsiv7/ni7O+jYtKGbZgB4caLmgoZBC5744Z+nAUhGVvq8U1a9XwYKlTLgrbK\nXhKA01841buLjYEzMC9FLwvuivl893SD4hWPbYNBTY3x/KQJqEE3YknCsxygE77h\nXYpgf/dHci+WyzBPdrdqWLPhNZ41rq1Tfdpws8o51/Zrrwjt6zzpVAEKKWDe2s4g\nCH3EmOCCKLubL5bD9nzlzyGp1hI2mgVbdlZ7LOijrATwY3vfg1//4YpztNdngS3Q\nbNUnH9Cp2LjstgyOpVx3/n9vZdNvsox20BeuzzJ6Z4A1Lt/xuCj/ZTS2lu6sRoXd\nZFTOYYsCAwEAAaNUMFIwQgYDVR0RBDswOYIec21pdGhpMDg5LmZyb250LnNlcGlh\nLmNlcGguY29tghFkYXNoYm9hcmRfc2VydmVyc4cErBUPWTAMBgNVHRMBAf8EAjAA\nMA0GCSqGSIb3DQEBCwUAA4ICAQAaRs6n9AAtTBvxWZKXl7ulNsCGLGbOQ6quucLv\nYkPjJdDwFMxX6S2tiugDE27PkD9b603M44PnmjtMZa1+fXrhbw/aQ3VwHl5+hFqw\nqUw1C+bhTNdgHpoBRs2GJsuVFA74ljqmzjeMSA3/ZhoWD72HQvcvezut+0AZ33Y0\npnuwyvPkjEJDLIsob6dH36NHOo37U/RM+G+D0t/esiAZUSNMOfDIiooy6Wnk0Bmd\ntLWdUav0iJLH+vJ7WKZ3HvVMfUyMstu+EneP4lqA9a69ZEp3SLpJ3c5P3ITVkp1U\nx53npZrGAI9DQD9gtHeA3wIjrkBeaMJW+EyAEz8XqL9fLYXh1IR+vJ3FyAN29Qy8\nUBAAm234s34V4xSItf7NVmaef3b+1msatYm0Jmd9336Y5CXxOwWCdrPD8yY066IG\nyhFKso1lNmkbrBsg6/4S2iww2oAhCpCOnv+0X9T7TfIvQAzIu8duaterSPCMwHB2\n8+iy6FRLYvlFSuxfM2L/GbSroeA47FMBpwEvjD9cIAIGBxdRjyctE2KDSo5NOQX/\nuwBoiuDTBasQM4KVjmdDW3j6KHKIyC6U6pSPvGIsMjJbWukJ/64F3VLXHTV8r2EX\niJURSlGIuug9waP+6hLHvY1XDpjltFnZ1zDnbOcyuGg+r6tcxzLujkXN49rXnxQ8\n0hjtyQ==\n-----END CERTIFICATE-----\n", "key": "-----BEGIN RSA PRIVATE KEY-----\nMIIJJwIBAAKCAgEAtbuR6qnMhzBilA6/9CA0CSHZuJk2FkltXBuK4itugAD9i+wg\nNMfer50Ma/WQhn7r/PGPZ8m8xN9i+19o0ACzO73YBGhtwcW5WDkwUSLJDQmdScem\nPUsekANGaoqSue4QdIF/lHVtFX/sdSuhhABKClS5cwiaWMQNsf8nrUtdmgE31a8S\n/D9wUJop1X1Eezl1WiJlp2dEXgRGAAD2EbVjcqcBT6jj8U/NDga5lnpP8De+aZjB\na/fTRcZ1thytfSX0fGMNpBGA9uivuc+Pk63JjPdU8+qZc2+7ldLNNR4w8bBX3eqJ\n1XOueX8CXRgAju13rwO9pVxw6znwycDzw0lHRGrD0CjTQ+/7UCNDP92Nzsiv7/ni\n7O+jYtKGbZgB4caLmgoZBC5744Z+nAUhGVvq8U1a9XwYKlTLgrbKXhKA01841buL\njYEzMC9FLwvuivl893SD4hWPbYNBTY3x/KQJqEE3YknCsxygE77hXYpgf/dHci+W\nyzBPdrdqWLPhNZ41rq1Tfdpws8o51/Zrrwjt6zzpVAEKKWDe2s4gCH3EmOCCKLub\nL5bD9nzlzyGp1hI2mgVbdlZ7LOijrATwY3vfg1//4YpztNdngS3QbNUnH9Cp2Ljs\ntgyOpVx3/n9vZdNvsox20BeuzzJ6Z4A1Lt/xuCj/ZTS2lu6sRoXdZFTOYYsCAwEA\nAQKCAgAw/dlm5RKNqVYR6u4Ffn74rUb39Fm35WCVcZldwm6eFGZFx8ig4fbe6dwU\n6JTnJSUyQd6ykJiSjFQvRvfHB0FKTDIFci+QiuCd74Dehp2jpaFg5/EhWfRpx3A+\nvY+zjwGhnt2JZKSdJKjWp4X5w2gyY8fAwqN1ZZj5kLseBRC9OTHXqcRY+zMoj37S\nnlwNSh6gR2CXIgg0vjsuuhOcD4W5PVtlobadcbFqZBa5Ac02CZkL95HBLQSoLUnD\nZ/S/JbHErKFbRxq4tV7WHSXsHnAU0AJ3ZRmsHIV57OPEO5jfNvZhDV7OC8sP1tUI\nyrUv0sWuiTghcYvBZhPLF/i+/yy2SPyRP96kk+Z2hY91yAc9Wt8uriaXeA5rycBG\npOXKZPbXhTNWI0YxX9f90DGzrzV76ALrnTJc5IJE1vH8f9vkCMb5HpBrkWBIF+AJ\nqy5wJU4ATrrZTAOUfTVIf4GHli/0lJm8DNsVkZnGGCEb6SjWA7y90uFnqfIMgoFF\nNtY3CRgng8gNqPjlwOefzTu2wSv32T7NNUBWJfSdLH3SoF6y4DQbAcdWFPO+EriG\nS8s+RVKv3mSeixQqmW2XtVQIdH6pYR1O/VJ6pD23Qp9pIe6JYvqK3ANimVyuMiYq\nsxUb77NYjmWRKWxEqldKSPYiFXrmNBfSDIOoj/DBIBnTVogLZQKCAQEA5U/vE99Z\nrotNy4zvXV5Gu5eWByoeZZNI8M4t4DNDRliLZjT2HxmkgfGHm61Y/Jfto/2hiAnJ\nsCIlpTBP6EAFbgWtnNCQIXUGNvGs4wPKtTKpJi2+qR552BG30DcST6rh7R4GhIzA\n+gPYLaoe1k6sQXDRbsu/Yh1OXEQ+Nf7DZHxLTFku4kldkmf+oVvkJKN80s7i3dUq\nkF6/7PZmf4EYXBWTTYFra0EjPAjdh2YAIkxgIfoL9NkWoIURj6G72mJYxyMeuZGF\naC71rgJrfFjWaN51ANQ7ppwoC/G+jYDNmVch86hPXpcdvulUczO5qElFT8zo6g1V\nvXsJO2F7JBTurwKCAQEAyuIQaflG7qnamLbLf44YF+z/M05+J4/wultmNTXzExR4\nIhexJGHPLzCEHnWk5p0mbhxWyM+6rY5XlMuyzv9ZFC9/4KIqqkJc4px4uyICvimb\nmy5p+hEuT/QhRGuh8xATh6C0zzMVzn3WW1jCa3bpw0nddsStWvZJPXR7u1ZQ/uZV\nkEsiRmI6RvTH9CczfRcwArzPGoi/k41ap71BBi19AxwGL03D56wFAxzuS1aIsM4M\nOrujQi/jwulJLapv1k2IYP5vGUheU+jWB8Do8r5uITyuV1myCSoZ0KOBziFaOGHO\ntiWZxNhIvvcnYF/aAAnw7qgIWm4tpNTdjvMOCkjR5QKCAQBiQ+K7rVaNCX0I4Rgh\n9qhw8wCDiXUFKhxztKbMEKuflYoG/mMJ6L8qQApQl/QuKhzXzXRPC4K2S6u+Ifnc\nGLM69ovzSxy4eaF0IhRoI9PV2iPKDIhLiJfGE7GlNDRQl3XBWsCIRJjdipA04VXd\nmFEQqfvVctJMhSpzN3tBQC8S/IB38y+4IKBMgOwI0Yvud3CiFa36ScK3oyKztNMv\niG6b51L8P518lKk4rMPhpcAg8oGvjatZ/T1JBD0GQ9RwPPy2OUKNWuLERntZWggd\nfSRERC/3bvsw5nHyzGFP2gZA06nk9Bcu/artiKd7xoqkpuFg1PpqcdRwIi1OmPNf\n/uhZAoIBAH7oiEHyB3fVUgMDHSpuFQ4qVryeVfWjaX75G2cflyyROlCBJ3LNqA8p\ntwtipcMH/HkWSwcUQf68snZGMVSgj/U7fS9j4XJdTsMGA9NzA+o2FrssfmhJhQHI\ngWLOPSDRJpZJZbDSkMp5FrRPQlZ8rN9W/iyD/LiQpOHHUnquFL5OEKnRBeCnIzMt\nfUJxnxiZf5PGzXoYnlXLLzEMBne5zqA6UHgYmfn6Naxr1XfU6m5GhYnbw9TsJfnP\nedIR8iFIoI2jupVDGpY8aF2zQ0RM8JDj516GogJvjnRfQxxQ6qGoNOjlflmOvXqP\nGdeEPdaWYVq4S3ldwDmCWhFIYrj2Mi0CggEAS0A7Yr5xo6daFtbuylHxytfEiq0r\nYd+P1b6kkpgjAGl+B/3wzQhzL5LJpV7SLApwv+QXCgfLDHMd/X2pCCn1XWGf6KJI\n1o67SK3Pvb8h4HK0B5QMqpgMw8aWIcn/CO6vM6ObAxky4TfBHvHpr7fff2UFeJmq\np+4LdGAPoQNugXmmuiAmnF2Y7v8dA6qXpEj+yTLUAOZljQEmIrDtgaFPhXf1wRuB\nM58Ct3TqXBvjsdFxneodOOub8ZA/GmhaurDkxHUukpueBU+SIMu9KI7rd0dvyC5K\neL5THFTLrWwEtRkqV7U+5GpdO/9COiyXFNvztmiZSOMOYlEGt5jXAcA/jQ==\n-----END RSA PRIVATE KEY-----\n"} 2024-09-21T03:29:57.250 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout SSL certificate updated 2024-09-21T03:29:58.061 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:29:57 smithi089 ceph-mon[30928]: from='client.14166 -' entity='client.admin' cmd=[{"prefix": "dashboard set-ssl-certificate", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:29:58.061 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:29:57 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:29:58.412 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout SSL certificate key updated 2024-09-21T03:29:58.412 INFO:teuthology.orchestra.run.smithi089.stdout:Creating initial admin user... 2024-09-21T03:29:59.061 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:29:58 smithi089 ceph-mon[30928]: from='client.14168 -' entity='client.admin' cmd=[{"prefix": "dashboard set-ssl-certificate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:29:59.061 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:29:58 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:29:59.793 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$u6O7jal4VFKDGwWtuqR3M.PMGoDPVnkz8fvu7xwU9lGN3YK9Jxfg6", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1726889399, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-09-21T03:29:59.793 INFO:teuthology.orchestra.run.smithi089.stdout:Fetching dashboard port number... 2024-09-21T03:30:00.357 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:30:00 smithi089 ceph-mon[30928]: from='client.14170 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:30:00.357 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:30:00 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:30:00.927 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout 8443 2024-09-21T03:30:00.938 INFO:teuthology.orchestra.run.smithi089.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-21T03:30:00.938 INFO:teuthology.orchestra.run.smithi089.stdout:systemctl: stdout disabled 2024-09-21T03:30:00.946 INFO:teuthology.orchestra.run.smithi089.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-21T03:30:00.946 INFO:teuthology.orchestra.run.smithi089.stdout:systemctl: stdout inactive 2024-09-21T03:30:00.946 INFO:teuthology.orchestra.run.smithi089.stdout:firewalld.service is not enabled 2024-09-21T03:30:00.946 INFO:teuthology.orchestra.run.smithi089.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-09-21T03:30:00.948 INFO:teuthology.orchestra.run.smithi089.stdout:Ceph Dashboard is now available at: 2024-09-21T03:30:00.948 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-09-21T03:30:00.948 INFO:teuthology.orchestra.run.smithi089.stdout: URL: https://smithi089.front.sepia.ceph.com:8443/ 2024-09-21T03:30:00.948 INFO:teuthology.orchestra.run.smithi089.stdout: User: admin 2024-09-21T03:30:00.948 INFO:teuthology.orchestra.run.smithi089.stdout: Password: zc6zauhkkk 2024-09-21T03:30:00.948 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-09-21T03:30:00.949 INFO:teuthology.orchestra.run.smithi089.stdout:Saving cluster configuration to /var/lib/ceph/7c374098-77c9-11ef-baf5-efdc52797490/config directory 2024-09-21T03:30:01.561 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:30:01 smithi089 ceph-mon[30928]: from='client.? 172.21.15.89:0/829443869' entity='client.admin' cmd={"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} : dispatch 2024-09-21T03:30:02.274 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-09-21T03:30:02.274 INFO:teuthology.orchestra.run.smithi089.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-09-21T03:30:02.274 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-09-21T03:30:02.275 INFO:teuthology.orchestra.run.smithi089.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid 7c374098-77c9-11ef-baf5-efdc52797490 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-09-21T03:30:02.275 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-09-21T03:30:02.275 INFO:teuthology.orchestra.run.smithi089.stdout:Or, if you are only running a single cluster on this host: 2024-09-21T03:30:02.275 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-09-21T03:30:02.275 INFO:teuthology.orchestra.run.smithi089.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-09-21T03:30:02.275 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-09-21T03:30:02.275 INFO:teuthology.orchestra.run.smithi089.stdout:Please consider enabling telemetry to help improve Ceph: 2024-09-21T03:30:02.275 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-09-21T03:30:02.275 INFO:teuthology.orchestra.run.smithi089.stdout: ceph telemetry on 2024-09-21T03:30:02.275 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-09-21T03:30:02.275 INFO:teuthology.orchestra.run.smithi089.stdout:For more information see: 2024-09-21T03:30:02.275 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-09-21T03:30:02.275 INFO:teuthology.orchestra.run.smithi089.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2024-09-21T03:30:02.276 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-09-21T03:30:02.276 INFO:teuthology.orchestra.run.smithi089.stdout:Bootstrap complete. 2024-09-21T03:30:02.314 INFO:tasks.cephadm:Fetching config... 2024-09-21T03:30:02.314 DEBUG:teuthology.orchestra.run.smithi089:> set -ex 2024-09-21T03:30:02.314 DEBUG:teuthology.orchestra.run.smithi089:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-09-21T03:30:02.329 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-09-21T03:30:02.329 DEBUG:teuthology.orchestra.run.smithi089:> set -ex 2024-09-21T03:30:02.330 DEBUG:teuthology.orchestra.run.smithi089:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-09-21T03:30:02.385 INFO:tasks.cephadm:Fetching mon keyring... 2024-09-21T03:30:02.385 DEBUG:teuthology.orchestra.run.smithi089:> set -ex 2024-09-21T03:30:02.386 DEBUG:teuthology.orchestra.run.smithi089:> sudo dd if=/var/lib/ceph/7c374098-77c9-11ef-baf5-efdc52797490/mon.a/keyring of=/dev/stdout 2024-09-21T03:30:02.451 INFO:tasks.cephadm:Fetching pub ssh key... 2024-09-21T03:30:02.451 DEBUG:teuthology.orchestra.run.smithi089:> set -ex 2024-09-21T03:30:02.452 DEBUG:teuthology.orchestra.run.smithi089:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-09-21T03:30:02.508 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-09-21T03:30:02.509 DEBUG:teuthology.orchestra.run.smithi089:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDLqsiRutqhAyrI9d3w8MSil+haY3hjFAxlCWm3cqoc5TqprQ8dj+0hPiaD52CJNa3tYxfl+jxR7wCU6JekcZ1HRi0i2pMCCAkvOe0X9xRDIkJjn2FaP9595rlcO2iOCbyEwq7Yhg2Wa52hbCIpElu1BrOofBks8sAJgKC/IfEiunVLZDmYxsUt222J7aJBfPzAUxTJgX1DGvZYczngJiPTHRafgRKdKaGxl0iGlPdfOG2umPQrmNIPq5cU3HplFKOKqFgctYZ+3wknCxKvxkQtdbbqS5d6RLbSsgvAp2wCl9RNw9ZeAwpqQVHVrOUb52gD9ntdq3VibRCYjhnHobCedUMFJQpJRz9+f+LfrQN+g8cruh+r1vpZ0jGTyx0lJN0I86JgzpTMH7YYDVuDIif8ne7wgvuK1tuZGRyK5U746cm7liFiaPkMTslp8Rb0Vk4c4LlB757zZIjYYRKRI4MUj4ExJTpPot/+11OWIfhedRsZdeZiSrbsWOVBKIurOqk= ceph-7c374098-77c9-11ef-baf5-efdc52797490' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-09-21T03:30:02.582 INFO:teuthology.orchestra.run.smithi089.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDLqsiRutqhAyrI9d3w8MSil+haY3hjFAxlCWm3cqoc5TqprQ8dj+0hPiaD52CJNa3tYxfl+jxR7wCU6JekcZ1HRi0i2pMCCAkvOe0X9xRDIkJjn2FaP9595rlcO2iOCbyEwq7Yhg2Wa52hbCIpElu1BrOofBks8sAJgKC/IfEiunVLZDmYxsUt222J7aJBfPzAUxTJgX1DGvZYczngJiPTHRafgRKdKaGxl0iGlPdfOG2umPQrmNIPq5cU3HplFKOKqFgctYZ+3wknCxKvxkQtdbbqS5d6RLbSsgvAp2wCl9RNw9ZeAwpqQVHVrOUb52gD9ntdq3VibRCYjhnHobCedUMFJQpJRz9+f+LfrQN+g8cruh+r1vpZ0jGTyx0lJN0I86JgzpTMH7YYDVuDIif8ne7wgvuK1tuZGRyK5U746cm7liFiaPkMTslp8Rb0Vk4c4LlB757zZIjYYRKRI4MUj4ExJTpPot/+11OWIfhedRsZdeZiSrbsWOVBKIurOqk= ceph-7c374098-77c9-11ef-baf5-efdc52797490 2024-09-21T03:30:02.595 DEBUG:teuthology.orchestra.run.smithi110:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDLqsiRutqhAyrI9d3w8MSil+haY3hjFAxlCWm3cqoc5TqprQ8dj+0hPiaD52CJNa3tYxfl+jxR7wCU6JekcZ1HRi0i2pMCCAkvOe0X9xRDIkJjn2FaP9595rlcO2iOCbyEwq7Yhg2Wa52hbCIpElu1BrOofBks8sAJgKC/IfEiunVLZDmYxsUt222J7aJBfPzAUxTJgX1DGvZYczngJiPTHRafgRKdKaGxl0iGlPdfOG2umPQrmNIPq5cU3HplFKOKqFgctYZ+3wknCxKvxkQtdbbqS5d6RLbSsgvAp2wCl9RNw9ZeAwpqQVHVrOUb52gD9ntdq3VibRCYjhnHobCedUMFJQpJRz9+f+LfrQN+g8cruh+r1vpZ0jGTyx0lJN0I86JgzpTMH7YYDVuDIif8ne7wgvuK1tuZGRyK5U746cm7liFiaPkMTslp8Rb0Vk4c4LlB757zZIjYYRKRI4MUj4ExJTpPot/+11OWIfhedRsZdeZiSrbsWOVBKIurOqk= ceph-7c374098-77c9-11ef-baf5-efdc52797490' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-09-21T03:30:02.659 INFO:teuthology.orchestra.run.smithi110.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDLqsiRutqhAyrI9d3w8MSil+haY3hjFAxlCWm3cqoc5TqprQ8dj+0hPiaD52CJNa3tYxfl+jxR7wCU6JekcZ1HRi0i2pMCCAkvOe0X9xRDIkJjn2FaP9595rlcO2iOCbyEwq7Yhg2Wa52hbCIpElu1BrOofBks8sAJgKC/IfEiunVLZDmYxsUt222J7aJBfPzAUxTJgX1DGvZYczngJiPTHRafgRKdKaGxl0iGlPdfOG2umPQrmNIPq5cU3HplFKOKqFgctYZ+3wknCxKvxkQtdbbqS5d6RLbSsgvAp2wCl9RNw9ZeAwpqQVHVrOUb52gD9ntdq3VibRCYjhnHobCedUMFJQpJRz9+f+LfrQN+g8cruh+r1vpZ0jGTyx0lJN0I86JgzpTMH7YYDVuDIif8ne7wgvuK1tuZGRyK5U746cm7liFiaPkMTslp8Rb0Vk4c4LlB757zZIjYYRKRI4MUj4ExJTpPot/+11OWIfhedRsZdeZiSrbsWOVBKIurOqk= ceph-7c374098-77c9-11ef-baf5-efdc52797490 2024-09-21T03:30:02.673 DEBUG:teuthology.orchestra.run.smithi114:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDLqsiRutqhAyrI9d3w8MSil+haY3hjFAxlCWm3cqoc5TqprQ8dj+0hPiaD52CJNa3tYxfl+jxR7wCU6JekcZ1HRi0i2pMCCAkvOe0X9xRDIkJjn2FaP9595rlcO2iOCbyEwq7Yhg2Wa52hbCIpElu1BrOofBks8sAJgKC/IfEiunVLZDmYxsUt222J7aJBfPzAUxTJgX1DGvZYczngJiPTHRafgRKdKaGxl0iGlPdfOG2umPQrmNIPq5cU3HplFKOKqFgctYZ+3wknCxKvxkQtdbbqS5d6RLbSsgvAp2wCl9RNw9ZeAwpqQVHVrOUb52gD9ntdq3VibRCYjhnHobCedUMFJQpJRz9+f+LfrQN+g8cruh+r1vpZ0jGTyx0lJN0I86JgzpTMH7YYDVuDIif8ne7wgvuK1tuZGRyK5U746cm7liFiaPkMTslp8Rb0Vk4c4LlB757zZIjYYRKRI4MUj4ExJTpPot/+11OWIfhedRsZdeZiSrbsWOVBKIurOqk= ceph-7c374098-77c9-11ef-baf5-efdc52797490' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-09-21T03:30:02.806 INFO:teuthology.orchestra.run.smithi114.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDLqsiRutqhAyrI9d3w8MSil+haY3hjFAxlCWm3cqoc5TqprQ8dj+0hPiaD52CJNa3tYxfl+jxR7wCU6JekcZ1HRi0i2pMCCAkvOe0X9xRDIkJjn2FaP9595rlcO2iOCbyEwq7Yhg2Wa52hbCIpElu1BrOofBks8sAJgKC/IfEiunVLZDmYxsUt222J7aJBfPzAUxTJgX1DGvZYczngJiPTHRafgRKdKaGxl0iGlPdfOG2umPQrmNIPq5cU3HplFKOKqFgctYZ+3wknCxKvxkQtdbbqS5d6RLbSsgvAp2wCl9RNw9ZeAwpqQVHVrOUb52gD9ntdq3VibRCYjhnHobCedUMFJQpJRz9+f+LfrQN+g8cruh+r1vpZ0jGTyx0lJN0I86JgzpTMH7YYDVuDIif8ne7wgvuK1tuZGRyK5U746cm7liFiaPkMTslp8Rb0Vk4c4LlB757zZIjYYRKRI4MUj4ExJTpPot/+11OWIfhedRsZdeZiSrbsWOVBKIurOqk= ceph-7c374098-77c9-11ef-baf5-efdc52797490 2024-09-21T03:30:02.820 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7c374098-77c9-11ef-baf5-efdc52797490 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-09-21T03:30:02.825 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:30:02 smithi089 ceph-mon[30928]: from='client.? 172.21.15.89:0/1444363627' entity='client.admin' 2024-09-21T03:30:03.069 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/7c374098-77c9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:30:04.549 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-09-21T03:30:04.549 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7c374098-77c9-11ef-baf5-efdc52797490 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-09-21T03:30:04.811 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:30:04 smithi089 ceph-mon[30928]: mgrmap e14: x(active, since 12s) 2024-09-21T03:30:04.811 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:30:04 smithi089 ceph-mon[30928]: from='client.? 172.21.15.89:0/225339948' entity='client.admin' 2024-09-21T03:30:04.811 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:30:04 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:30:04.811 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:30:04 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:30:04.811 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:30:04 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd/host:smithi089", "name": "osd_memory_target"} : dispatch 2024-09-21T03:30:04.811 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:30:04 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:30:04.811 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:30:04 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:30:04.812 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:30:04 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:30:05.088 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/7c374098-77c9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:30:06.896 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:30:06 smithi089 ceph-mon[30928]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:30:06.896 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:30:06 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:30:06.896 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:30:06 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:30:06.896 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:30:06 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:30:06.897 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:30:06 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:30:06.897 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:30:06 smithi089 ceph-mon[30928]: Updating smithi089:/etc/ceph/ceph.conf 2024-09-21T03:30:06.897 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:30:06 smithi089 ceph-mon[30928]: Updating smithi089:/var/lib/ceph/7c374098-77c9-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-21T03:30:06.897 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:30:06 smithi089 ceph-mon[30928]: Updating smithi089:/etc/ceph/ceph.client.admin.keyring 2024-09-21T03:30:06.897 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:30:06 smithi089 ceph-mon[30928]: Updating smithi089:/var/lib/ceph/7c374098-77c9-11ef-baf5-efdc52797490/config/ceph.client.admin.keyring 2024-09-21T03:30:06.897 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:30:06 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:30:06.897 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:30:06 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:30:06.897 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:30:06 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:30:06.925 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi110 2024-09-21T03:30:06.925 DEBUG:teuthology.orchestra.run.smithi110:> set -ex 2024-09-21T03:30:06.925 DEBUG:teuthology.orchestra.run.smithi110:> dd of=/etc/ceph/ceph.conf 2024-09-21T03:30:06.945 DEBUG:teuthology.orchestra.run.smithi110:> set -ex 2024-09-21T03:30:06.946 DEBUG:teuthology.orchestra.run.smithi110:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-21T03:30:07.004 INFO:tasks.cephadm:Adding host smithi110 to orchestrator... 2024-09-21T03:30:07.004 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7c374098-77c9-11ef-baf5-efdc52797490 -- ceph orch host add smithi110 2024-09-21T03:30:07.250 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/7c374098-77c9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:30:08.811 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:30:08 smithi089 ceph-mon[30928]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi110", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:30:09.811 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:30:09 smithi089 ceph-mon[30928]: Deploying cephadm binary to smithi110 2024-09-21T03:30:10.932 INFO:teuthology.orchestra.run.smithi089.stdout:Added host 'smithi110' with addr '172.21.15.110' 2024-09-21T03:30:11.517 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7c374098-77c9-11ef-baf5-efdc52797490 -- ceph orch host ls --format=json 2024-09-21T03:30:11.759 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/7c374098-77c9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:30:12.180 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:30:11 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:30:12.180 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:30:11 smithi089 ceph-mon[30928]: Added host smithi110 2024-09-21T03:30:12.181 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:30:11 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:30:12.181 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:30:11 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:30:12.181 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:30:11 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:30:12.463 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-09-21T03:30:12.469 INFO:teuthology.orchestra.run.smithi089.stdout:[{"addr": "172.21.15.89", "hostname": "smithi089", "labels": [], "status": ""}, {"addr": "172.21.15.110", "hostname": "smithi110", "labels": [], "status": ""}] 2024-09-21T03:30:13.077 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi114 2024-09-21T03:30:13.078 DEBUG:teuthology.orchestra.run.smithi114:> set -ex 2024-09-21T03:30:13.078 DEBUG:teuthology.orchestra.run.smithi114:> dd of=/etc/ceph/ceph.conf 2024-09-21T03:30:13.098 DEBUG:teuthology.orchestra.run.smithi114:> set -ex 2024-09-21T03:30:13.098 DEBUG:teuthology.orchestra.run.smithi114:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-21T03:30:13.156 INFO:tasks.cephadm:Adding host smithi114 to orchestrator... 2024-09-21T03:30:13.156 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7c374098-77c9-11ef-baf5-efdc52797490 -- ceph orch host add smithi114 2024-09-21T03:30:13.311 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:30:13 smithi089 ceph-mon[30928]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:30:13.311 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:30:13 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:30:13.311 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:30:13 smithi089 ceph-mon[30928]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T03:30:13.397 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/7c374098-77c9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:30:15.561 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:30:15 smithi089 ceph-mon[30928]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:30:15.561 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:30:15 smithi089 ceph-mon[30928]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi114", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:30:15.561 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:30:15 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:30:15.561 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:30:15 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:30:15.561 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:30:15 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:30:15.561 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:30:15 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:30:15.561 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:30:15 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd/host:smithi110", "name": "osd_memory_target"} : dispatch 2024-09-21T03:30:15.561 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:30:15 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:30:15.561 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:30:15 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:30:16.811 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:30:16 smithi089 ceph-mon[30928]: Deploying cephadm binary to smithi114 2024-09-21T03:30:16.811 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:30:16 smithi089 ceph-mon[30928]: Updating smithi110:/etc/ceph/ceph.conf 2024-09-21T03:30:16.811 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:30:16 smithi089 ceph-mon[30928]: Updating smithi110:/var/lib/ceph/7c374098-77c9-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-21T03:30:16.811 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:30:16 smithi089 ceph-mon[30928]: Updating smithi110:/etc/ceph/ceph.client.admin.keyring 2024-09-21T03:30:16.811 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:30:16 smithi089 ceph-mon[30928]: Updating smithi110:/var/lib/ceph/7c374098-77c9-11ef-baf5-efdc52797490/config/ceph.client.admin.keyring 2024-09-21T03:30:16.811 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:30:16 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:30:16.811 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:30:16 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:30:16.811 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:30:16 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:30:17.103 INFO:teuthology.orchestra.run.smithi089.stdout:Added host 'smithi114' with addr '172.21.15.114' 2024-09-21T03:30:17.667 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:30:17 smithi089 ceph-mon[30928]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:30:17.667 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:30:17 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:30:17.667 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:30:17 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:30:17.667 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:30:17 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:30:17.694 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7c374098-77c9-11ef-baf5-efdc52797490 -- ceph orch host ls --format=json 2024-09-21T03:30:17.933 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/7c374098-77c9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:30:18.583 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-09-21T03:30:18.584 INFO:teuthology.orchestra.run.smithi089.stdout:[{"addr": "172.21.15.89", "hostname": "smithi089", "labels": [], "status": ""}, {"addr": "172.21.15.110", "hostname": "smithi110", "labels": [], "status": ""}, {"addr": "172.21.15.114", "hostname": "smithi114", "labels": [], "status": ""}] 2024-09-21T03:30:18.811 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:30:18 smithi089 ceph-mon[30928]: Added host smithi114 2024-09-21T03:30:18.811 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:30:18 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:30:18.811 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:30:18 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:30:19.186 INFO:tasks.cephadm:Setting crush tunables to default 2024-09-21T03:30:19.186 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7c374098-77c9-11ef-baf5-efdc52797490 -- ceph osd crush tunables default 2024-09-21T03:30:19.432 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/7c374098-77c9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:30:19.802 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:30:19 smithi089 ceph-mon[30928]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:30:20.484 INFO:teuthology.orchestra.run.smithi089.stderr:adjusted tunables profile to default 2024-09-21T03:30:20.780 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:30:20 smithi089 ceph-mon[30928]: from='client.14186 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T03:30:20.780 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:30:20 smithi089 ceph-mon[30928]: from='client.? 172.21.15.89:0/1311650935' entity='client.admin' cmd={"prefix": "osd crush tunables", "profile": "default"} : dispatch 2024-09-21T03:30:21.069 INFO:tasks.cephadm:Adding mon.a on smithi089 2024-09-21T03:30:21.070 INFO:tasks.cephadm:Adding mon.b on smithi110 2024-09-21T03:30:21.070 INFO:tasks.cephadm:Adding mon.c on smithi114 2024-09-21T03:30:21.070 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7c374098-77c9-11ef-baf5-efdc52797490 -- ceph orch apply mon '3;smithi089:172.21.15.89=a;smithi110:172.21.15.110=b;smithi114:172.21.15.114=c' 2024-09-21T03:30:21.374 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/7c374098-77c9-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-21T03:30:21.811 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:30:21 smithi089 ceph-mon[30928]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:30:21.811 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:30:21 smithi089 ceph-mon[30928]: from='client.? 172.21.15.89:0/1311650935' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-09-21T03:30:21.811 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:30:21 smithi089 ceph-mon[30928]: osdmap e4: 0 total, 0 up, 0 in 2024-09-21T03:30:21.811 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:30:21 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:30:21.811 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:30:21 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:30:21.811 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:30:21 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:30:21.811 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:30:21 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:30:21.811 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:30:21 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd/host:smithi114", "name": "osd_memory_target"} : dispatch 2024-09-21T03:30:21.812 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:30:21 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:30:21.812 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:30:21 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:30:22.075 INFO:teuthology.orchestra.run.smithi114.stdout:Scheduled mon update... 2024-09-21T03:30:22.674 DEBUG:teuthology.orchestra.run.smithi110:mon.b> sudo journalctl -f -n 0 -u ceph-7c374098-77c9-11ef-baf5-efdc52797490@mon.b.service 2024-09-21T03:30:22.677 DEBUG:teuthology.orchestra.run.smithi114:mon.c> sudo journalctl -f -n 0 -u ceph-7c374098-77c9-11ef-baf5-efdc52797490@mon.c.service 2024-09-21T03:30:22.680 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-21T03:30:22.680 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7c374098-77c9-11ef-baf5-efdc52797490 -- ceph mon dump -f json 2024-09-21T03:30:22.811 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:30:22 smithi089 ceph-mon[30928]: Updating smithi114:/etc/ceph/ceph.conf 2024-09-21T03:30:22.811 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:30:22 smithi089 ceph-mon[30928]: Updating smithi114:/var/lib/ceph/7c374098-77c9-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-21T03:30:22.811 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:30:22 smithi089 ceph-mon[30928]: Updating smithi114:/etc/ceph/ceph.client.admin.keyring 2024-09-21T03:30:22.811 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:30:22 smithi089 ceph-mon[30928]: Updating smithi114:/var/lib/ceph/7c374098-77c9-11ef-baf5-efdc52797490/config/ceph.client.admin.keyring 2024-09-21T03:30:22.811 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:30:22 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:30:22.812 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:30:22 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:30:22.812 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:30:22 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:30:22.812 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:30:22 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:30:22.812 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:30:22 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:30:22.812 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:30:22 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:30:22.812 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:30:22 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:30:22.812 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:30:22 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:30:22.812 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:30:22 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-21T03:30:22.812 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:30:22 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:30:23.811 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:30:23 smithi089 ceph-mon[30928]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:30:23.811 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:30:23 smithi089 ceph-mon[30928]: from='client.14190 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi089:172.21.15.89=a;smithi110:172.21.15.110=b;smithi114:172.21.15.114=c", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:30:23.811 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:30:23 smithi089 ceph-mon[30928]: Saving service mon spec with placement smithi089:172.21.15.89=a;smithi110:172.21.15.110=b;smithi114:172.21.15.114=c;count:3 2024-09-21T03:30:23.811 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:30:23 smithi089 ceph-mon[30928]: Deploying daemon mon.c on smithi114 2024-09-21T03:30:24.621 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/7c374098-77c9-11ef-baf5-efdc52797490/mon.c/config 2024-09-21T03:30:25.454 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-21T03:30:25.454 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":1,"fsid":"7c374098-77c9-11ef-baf5-efdc52797490","modified":"2024-09-21T03:28:53.768514Z","created":"2024-09-21T03:28:53.768514Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:3300","nonce":0},{"type":"v1","addr":"172.21.15.89:6789","nonce":0}]},"addr":"172.21.15.89:6789/0","public_addr":"172.21.15.89:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-09-21T03:30:25.454 INFO:teuthology.orchestra.run.smithi114.stderr:dumped monmap epoch 1 2024-09-21T03:30:25.711 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:25 smithi114 systemd[1]: Starting Ceph mon.c for 7c374098-77c9-11ef-baf5-efdc52797490... 2024-09-21T03:30:25.811 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:30:25 smithi089 ceph-mon[30928]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:30:25.811 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:30:25 smithi089 ceph-mon[30928]: from='client.? 172.21.15.114:0/3837685406' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2024-09-21T03:30:26.332 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:26 smithi114 podman[34121]: 2024-09-21 03:30:26.22361032 +0000 UTC m=+0.105765040 container create 1205e7833f22c32e2ce7f554d863b6bc7df66e2260cb028062b7b1edd85e0abb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-7c374098-77c9-11ef-baf5-efdc52797490-mon-c, ceph=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-09-21T03:30:26.332 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:26 smithi114 podman[34121]: 2024-09-21 03:30:26.133669181 +0000 UTC m=+0.015823916 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T03:30:26.712 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:26 smithi114 podman[34121]: 2024-09-21 03:30:26.409325998 +0000 UTC m=+0.291480723 container init 1205e7833f22c32e2ce7f554d863b6bc7df66e2260cb028062b7b1edd85e0abb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-7c374098-77c9-11ef-baf5-efdc52797490-mon-c, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, io.buildah.version=1.37.2) 2024-09-21T03:30:26.712 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:26 smithi114 podman[34121]: 2024-09-21 03:30:26.412478803 +0000 UTC m=+0.294633523 container start 1205e7833f22c32e2ce7f554d863b6bc7df66e2260cb028062b7b1edd85e0abb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-7c374098-77c9-11ef-baf5-efdc52797490-mon-c, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_BRANCH=HEAD) 2024-09-21T03:30:26.712 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:26 smithi114 ceph-mon[34140]: set uid:gid to 167:167 (ceph:ceph) 2024-09-21T03:30:26.712 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:26 smithi114 ceph-mon[34140]: ceph version 19.3.0-5120-g18432218 (1843221895adccfb3b692060251dd6a0183bc410) squid (dev), process ceph-mon, pid 2 2024-09-21T03:30:26.712 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:26 smithi114 ceph-mon[34140]: pidfile_write: ignore empty --pid-file 2024-09-21T03:30:26.712 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:26 smithi114 ceph-mon[34140]: load: jerasure load: lrc 2024-09-21T03:30:26.712 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:26 smithi114 ceph-mon[34140]: rocksdb: RocksDB version: 7.9.2 2024-09-21T03:30:26.712 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:26 smithi114 ceph-mon[34140]: rocksdb: Git sha 0 2024-09-21T03:30:26.712 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:26 smithi114 ceph-mon[34140]: rocksdb: Compile date 2024-09-20 22:29:49 2024-09-21T03:30:26.712 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:26 smithi114 ceph-mon[34140]: rocksdb: DB SUMMARY 2024-09-21T03:30:26.712 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:26 smithi114 ceph-mon[34140]: rocksdb: DB Session ID: 37BCF4T6GBT3B39JJZ75 2024-09-21T03:30:26.712 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:26 smithi114 ceph-mon[34140]: rocksdb: CURRENT file: CURRENT 2024-09-21T03:30:26.713 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:26 smithi114 ceph-mon[34140]: rocksdb: IDENTITY file: IDENTITY 2024-09-21T03:30:26.713 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:26 smithi114 ceph-mon[34140]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2024-09-21T03:30:26.713 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:26 smithi114 ceph-mon[34140]: rocksdb: SST files in /var/lib/ceph/mon/ceph-c/store.db dir, Total Num: 0, files: 2024-09-21T03:30:26.713 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:26 smithi114 ceph-mon[34140]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-c/store.db: 000004.log size: 511 ; 2024-09-21T03:30:26.713 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:26 smithi114 ceph-mon[34140]: rocksdb: Options.error_if_exists: 0 2024-09-21T03:30:26.713 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:26 smithi114 ceph-mon[34140]: rocksdb: Options.create_if_missing: 0 2024-09-21T03:30:26.713 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:26 smithi114 ceph-mon[34140]: rocksdb: Options.paranoid_checks: 1 2024-09-21T03:30:26.713 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:26 smithi114 ceph-mon[34140]: rocksdb: Options.flush_verify_memtable_count: 1 2024-09-21T03:30:26.713 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:26 smithi114 ceph-mon[34140]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-09-21T03:30:26.713 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:26 smithi114 ceph-mon[34140]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-09-21T03:30:26.713 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:26 smithi114 ceph-mon[34140]: rocksdb: Options.env: 0x5644bd9fcd60 2024-09-21T03:30:26.713 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:26 smithi114 ceph-mon[34140]: rocksdb: Options.fs: PosixFileSystem 2024-09-21T03:30:26.713 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:26 smithi114 ceph-mon[34140]: rocksdb: Options.info_log: 0x5644c2db7360 2024-09-21T03:30:26.713 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:26 smithi114 ceph-mon[34140]: rocksdb: Options.max_file_opening_threads: 16 2024-09-21T03:30:26.714 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:26 smithi114 ceph-mon[34140]: rocksdb: Options.statistics: (nil) 2024-09-21T03:30:26.714 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:26 smithi114 ceph-mon[34140]: rocksdb: Options.use_fsync: 0 2024-09-21T03:30:26.714 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:26 smithi114 ceph-mon[34140]: rocksdb: Options.max_log_file_size: 0 2024-09-21T03:30:26.715 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:26 smithi114 ceph-mon[34140]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-09-21T03:30:26.715 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:26 smithi114 ceph-mon[34140]: rocksdb: Options.log_file_time_to_roll: 0 2024-09-21T03:30:26.715 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:26 smithi114 ceph-mon[34140]: rocksdb: Options.keep_log_file_num: 1000 2024-09-21T03:30:26.715 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:26 smithi114 ceph-mon[34140]: rocksdb: Options.recycle_log_file_num: 0 2024-09-21T03:30:26.715 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:26 smithi114 ceph-mon[34140]: rocksdb: Options.allow_fallocate: 1 2024-09-21T03:30:26.715 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:26 smithi114 ceph-mon[34140]: rocksdb: Options.allow_mmap_reads: 0 2024-09-21T03:30:26.715 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:26 smithi114 ceph-mon[34140]: rocksdb: Options.allow_mmap_writes: 0 2024-09-21T03:30:26.715 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:26 smithi114 ceph-mon[34140]: rocksdb: Options.use_direct_reads: 0 2024-09-21T03:30:26.715 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:26 smithi114 ceph-mon[34140]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-09-21T03:30:26.715 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:26 smithi114 ceph-mon[34140]: rocksdb: Options.create_missing_column_families: 0 2024-09-21T03:30:26.715 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:26 smithi114 ceph-mon[34140]: rocksdb: Options.db_log_dir: 2024-09-21T03:30:26.715 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:26 smithi114 ceph-mon[34140]: rocksdb: Options.wal_dir: 2024-09-21T03:30:26.716 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:26 smithi114 ceph-mon[34140]: rocksdb: Options.table_cache_numshardbits: 6 2024-09-21T03:30:26.716 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:26 smithi114 ceph-mon[34140]: rocksdb: Options.WAL_ttl_seconds: 0 2024-09-21T03:30:26.716 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:26 smithi114 ceph-mon[34140]: rocksdb: Options.WAL_size_limit_MB: 0 2024-09-21T03:30:26.716 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:26 smithi114 ceph-mon[34140]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-09-21T03:30:26.716 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:26 smithi114 ceph-mon[34140]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-09-21T03:30:26.716 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:26 smithi114 ceph-mon[34140]: rocksdb: Options.is_fd_close_on_exec: 1 2024-09-21T03:30:26.716 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:26 smithi114 ceph-mon[34140]: rocksdb: Options.advise_random_on_open: 1 2024-09-21T03:30:26.716 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:26 smithi114 ceph-mon[34140]: rocksdb: Options.db_write_buffer_size: 0 2024-09-21T03:30:26.716 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:26 smithi114 ceph-mon[34140]: rocksdb: Options.write_buffer_manager: 0x5644c2dbbd60 2024-09-21T03:30:26.716 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:26 smithi114 ceph-mon[34140]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-09-21T03:30:26.716 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:26 smithi114 ceph-mon[34140]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-09-21T03:30:26.716 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:26 smithi114 ceph-mon[34140]: rocksdb: Options.use_adaptive_mutex: 0 2024-09-21T03:30:26.716 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:26 smithi114 ceph-mon[34140]: rocksdb: Options.rate_limiter: (nil) 2024-09-21T03:30:26.716 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:26 smithi114 ceph-mon[34140]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-09-21T03:30:26.717 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:26 smithi114 ceph-mon[34140]: rocksdb: Options.wal_recovery_mode: 2 2024-09-21T03:30:26.717 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:26 smithi114 ceph-mon[34140]: rocksdb: Options.enable_thread_tracking: 0 2024-09-21T03:30:26.717 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:26 smithi114 ceph-mon[34140]: rocksdb: Options.enable_pipelined_write: 0 2024-09-21T03:30:26.717 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:26 smithi114 ceph-mon[34140]: rocksdb: Options.unordered_write: 0 2024-09-21T03:30:26.717 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:26 smithi114 ceph-mon[34140]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-09-21T03:30:26.717 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:26 smithi114 ceph-mon[34140]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-09-21T03:30:26.717 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:26 smithi114 ceph-mon[34140]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-09-21T03:30:26.717 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:26 smithi114 ceph-mon[34140]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-09-21T03:30:26.717 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:26 smithi114 ceph-mon[34140]: rocksdb: Options.row_cache: None 2024-09-21T03:30:26.717 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:26 smithi114 ceph-mon[34140]: rocksdb: Options.wal_filter: None 2024-09-21T03:30:26.717 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:26 smithi114 ceph-mon[34140]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-09-21T03:30:26.717 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:26 smithi114 ceph-mon[34140]: rocksdb: Options.allow_ingest_behind: 0 2024-09-21T03:30:26.717 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:26 smithi114 ceph-mon[34140]: rocksdb: Options.two_write_queues: 0 2024-09-21T03:30:26.717 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:26 smithi114 ceph-mon[34140]: rocksdb: Options.manual_wal_flush: 0 2024-09-21T03:30:26.718 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:26 smithi114 ceph-mon[34140]: rocksdb: Options.wal_compression: 0 2024-09-21T03:30:26.718 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:26 smithi114 ceph-mon[34140]: rocksdb: Options.atomic_flush: 0 2024-09-21T03:30:26.718 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:26 smithi114 ceph-mon[34140]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-09-21T03:30:26.718 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:26 smithi114 ceph-mon[34140]: rocksdb: Options.persist_stats_to_disk: 0 2024-09-21T03:30:26.718 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:26 smithi114 ceph-mon[34140]: rocksdb: Options.write_dbid_to_manifest: 0 2024-09-21T03:30:26.718 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:26 smithi114 ceph-mon[34140]: rocksdb: Options.log_readahead_size: 0 2024-09-21T03:30:26.718 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:26 smithi114 ceph-mon[34140]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-09-21T03:30:26.718 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:26 smithi114 ceph-mon[34140]: rocksdb: Options.best_efforts_recovery: 0 2024-09-21T03:30:26.718 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:26 smithi114 ceph-mon[34140]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-09-21T03:30:26.718 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:26 smithi114 ceph-mon[34140]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-09-21T03:30:26.718 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:26 smithi114 ceph-mon[34140]: rocksdb: Options.allow_data_in_errors: 0 2024-09-21T03:30:26.718 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:26 smithi114 ceph-mon[34140]: rocksdb: Options.db_host_id: __hostname__ 2024-09-21T03:30:26.718 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:26 smithi114 ceph-mon[34140]: rocksdb: Options.enforce_single_del_contracts: true 2024-09-21T03:30:26.718 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:26 smithi114 ceph-mon[34140]: rocksdb: Options.max_background_jobs: 2 2024-09-21T03:30:26.719 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:26 smithi114 ceph-mon[34140]: rocksdb: Options.max_background_compactions: -1 2024-09-21T03:30:26.719 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:26 smithi114 ceph-mon[34140]: rocksdb: Options.max_subcompactions: 1 2024-09-21T03:30:26.719 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:26 smithi114 ceph-mon[34140]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-09-21T03:30:26.719 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:26 smithi114 ceph-mon[34140]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-09-21T03:30:26.719 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:26 smithi114 ceph-mon[34140]: rocksdb: Options.delayed_write_rate : 16777216 2024-09-21T03:30:26.719 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:26 smithi114 ceph-mon[34140]: rocksdb: Options.max_total_wal_size: 0 2024-09-21T03:30:26.719 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:26 smithi114 ceph-mon[34140]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-09-21T03:30:26.719 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:26 smithi114 ceph-mon[34140]: rocksdb: Options.stats_dump_period_sec: 600 2024-09-21T03:30:26.719 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:26 smithi114 ceph-mon[34140]: rocksdb: Options.stats_persist_period_sec: 600 2024-09-21T03:30:26.719 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:26 smithi114 ceph-mon[34140]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-09-21T03:30:26.719 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:26 smithi114 ceph-mon[34140]: rocksdb: Options.max_open_files: -1 2024-09-21T03:30:26.719 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:26 smithi114 ceph-mon[34140]: rocksdb: Options.bytes_per_sync: 0 2024-09-21T03:30:26.719 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:26 smithi114 ceph-mon[34140]: rocksdb: Options.wal_bytes_per_sync: 0 2024-09-21T03:30:26.719 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:26 smithi114 ceph-mon[34140]: rocksdb: Options.strict_bytes_per_sync: 0 2024-09-21T03:30:26.719 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:26 smithi114 ceph-mon[34140]: rocksdb: Options.compaction_readahead_size: 0 2024-09-21T03:30:26.719 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:26 smithi114 ceph-mon[34140]: rocksdb: Options.max_background_flushes: -1 2024-09-21T03:30:26.720 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:26 smithi114 ceph-mon[34140]: rocksdb: Compression algorithms supported: 2024-09-21T03:30:26.720 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:26 smithi114 ceph-mon[34140]: rocksdb: kZSTD supported: 0 2024-09-21T03:30:26.720 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:26 smithi114 ceph-mon[34140]: rocksdb: kXpressCompression supported: 0 2024-09-21T03:30:26.720 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:26 smithi114 ceph-mon[34140]: rocksdb: kBZip2Compression supported: 0 2024-09-21T03:30:26.720 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:26 smithi114 ceph-mon[34140]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-09-21T03:30:26.720 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:26 smithi114 ceph-mon[34140]: rocksdb: kLZ4Compression supported: 1 2024-09-21T03:30:26.720 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:26 smithi114 ceph-mon[34140]: rocksdb: kZlibCompression supported: 1 2024-09-21T03:30:26.720 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:26 smithi114 ceph-mon[34140]: rocksdb: kLZ4HCCompression supported: 1 2024-09-21T03:30:26.720 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:26 smithi114 ceph-mon[34140]: rocksdb: kSnappyCompression supported: 1 2024-09-21T03:30:26.720 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:26 smithi114 ceph-mon[34140]: rocksdb: Fast CRC32 supported: Supported on x86 2024-09-21T03:30:26.720 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:26 smithi114 ceph-mon[34140]: rocksdb: DMutex implementation: pthread_mutex_t 2024-09-21T03:30:26.720 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:26 smithi114 ceph-mon[34140]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 2024-09-21T03:30:26.720 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:26 smithi114 ceph-mon[34140]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-09-21T03:30:26.720 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:26 smithi114 ceph-mon[34140]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-09-21T03:30:26.721 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:26 smithi114 ceph-mon[34140]: rocksdb: Options.merge_operator: 2024-09-21T03:30:26.721 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:26 smithi114 ceph-mon[34140]: rocksdb: Options.compaction_filter: None 2024-09-21T03:30:26.721 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:26 smithi114 ceph-mon[34140]: rocksdb: Options.compaction_filter_factory: None 2024-09-21T03:30:26.721 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:26 smithi114 ceph-mon[34140]: rocksdb: Options.sst_partitioner_factory: None 2024-09-21T03:30:26.721 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:26 smithi114 ceph-mon[34140]: rocksdb: Options.memtable_factory: SkipListFactory 2024-09-21T03:30:26.721 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:26 smithi114 ceph-mon[34140]: rocksdb: Options.table_factory: BlockBasedTable 2024-09-21T03:30:26.721 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:26 smithi114 ceph-mon[34140]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x5644c2db6fa0) 2024-09-21T03:30:26.721 INFO:journalctl@ceph.mon.c.smithi114.stdout: cache_index_and_filter_blocks: 1 2024-09-21T03:30:26.721 INFO:journalctl@ceph.mon.c.smithi114.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-09-21T03:30:26.721 INFO:journalctl@ceph.mon.c.smithi114.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-09-21T03:30:26.721 INFO:journalctl@ceph.mon.c.smithi114.stdout: pin_top_level_index_and_filter: 1 2024-09-21T03:30:26.721 INFO:journalctl@ceph.mon.c.smithi114.stdout: index_type: 0 2024-09-21T03:30:26.721 INFO:journalctl@ceph.mon.c.smithi114.stdout: data_block_index_type: 0 2024-09-21T03:30:26.721 INFO:journalctl@ceph.mon.c.smithi114.stdout: index_shortening: 1 2024-09-21T03:30:26.721 INFO:journalctl@ceph.mon.c.smithi114.stdout: data_block_hash_table_util_ratio: 0.750000 2024-09-21T03:30:26.722 INFO:journalctl@ceph.mon.c.smithi114.stdout: checksum: 4 2024-09-21T03:30:26.722 INFO:journalctl@ceph.mon.c.smithi114.stdout: no_block_cache: 0 2024-09-21T03:30:26.722 INFO:journalctl@ceph.mon.c.smithi114.stdout: block_cache: 0x5644bed7b610 2024-09-21T03:30:26.722 INFO:journalctl@ceph.mon.c.smithi114.stdout: block_cache_name: BinnedLRUCache 2024-09-21T03:30:26.722 INFO:journalctl@ceph.mon.c.smithi114.stdout: block_cache_options: 2024-09-21T03:30:26.722 INFO:journalctl@ceph.mon.c.smithi114.stdout: capacity : 536870912 2024-09-21T03:30:26.722 INFO:journalctl@ceph.mon.c.smithi114.stdout: num_shard_bits : 4 2024-09-21T03:30:26.722 INFO:journalctl@ceph.mon.c.smithi114.stdout: strict_capacity_limit : 0 2024-09-21T03:30:26.722 INFO:journalctl@ceph.mon.c.smithi114.stdout: high_pri_pool_ratio: 0.000 2024-09-21T03:30:26.722 INFO:journalctl@ceph.mon.c.smithi114.stdout: block_cache_compressed: (nil) 2024-09-21T03:30:26.722 INFO:journalctl@ceph.mon.c.smithi114.stdout: persistent_cache: (nil) 2024-09-21T03:30:26.722 INFO:journalctl@ceph.mon.c.smithi114.stdout: block_size: 4096 2024-09-21T03:30:26.722 INFO:journalctl@ceph.mon.c.smithi114.stdout: block_size_deviation: 10 2024-09-21T03:30:26.722 INFO:journalctl@ceph.mon.c.smithi114.stdout: block_restart_interval: 16 2024-09-21T03:30:26.723 INFO:journalctl@ceph.mon.c.smithi114.stdout: index_block_restart_interval: 1 2024-09-21T03:30:26.723 INFO:journalctl@ceph.mon.c.smithi114.stdout: metadata_block_size: 4096 2024-09-21T03:30:26.723 INFO:journalctl@ceph.mon.c.smithi114.stdout: partition_filters: 0 2024-09-21T03:30:26.723 INFO:journalctl@ceph.mon.c.smithi114.stdout: use_delta_encoding: 1 2024-09-21T03:30:26.723 INFO:journalctl@ceph.mon.c.smithi114.stdout: filter_policy: bloomfilter 2024-09-21T03:30:26.723 INFO:journalctl@ceph.mon.c.smithi114.stdout: whole_key_filtering: 1 2024-09-21T03:30:26.723 INFO:journalctl@ceph.mon.c.smithi114.stdout: verify_compression: 0 2024-09-21T03:30:26.723 INFO:journalctl@ceph.mon.c.smithi114.stdout: read_amp_bytes_per_bit: 0 2024-09-21T03:30:26.723 INFO:journalctl@ceph.mon.c.smithi114.stdout: format_version: 5 2024-09-21T03:30:26.723 INFO:journalctl@ceph.mon.c.smithi114.stdout: enable_index_compression: 1 2024-09-21T03:30:26.723 INFO:journalctl@ceph.mon.c.smithi114.stdout: block_align: 0 2024-09-21T03:30:26.723 INFO:journalctl@ceph.mon.c.smithi114.stdout: max_auto_readahead_size: 262144 2024-09-21T03:30:26.723 INFO:journalctl@ceph.mon.c.smithi114.stdout: prepopulate_block_cache: 0 2024-09-21T03:30:26.723 INFO:journalctl@ceph.mon.c.smithi114.stdout: initial_auto_readahead_size: 8192 2024-09-21T03:30:26.723 INFO:journalctl@ceph.mon.c.smithi114.stdout: num_file_reads_for_auto_readahead: 2 2024-09-21T03:30:26.723 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:26 smithi114 ceph-mon[34140]: rocksdb: Options.write_buffer_size: 33554432 2024-09-21T03:30:26.724 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:26 smithi114 ceph-mon[34140]: rocksdb: Options.max_write_buffer_number: 2 2024-09-21T03:30:26.724 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:26 smithi114 ceph-mon[34140]: rocksdb: Options.compression: NoCompression 2024-09-21T03:30:26.724 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:26 smithi114 ceph-mon[34140]: rocksdb: Options.bottommost_compression: Disabled 2024-09-21T03:30:26.724 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:26 smithi114 ceph-mon[34140]: rocksdb: Options.prefix_extractor: nullptr 2024-09-21T03:30:26.724 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:26 smithi114 ceph-mon[34140]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-09-21T03:30:26.724 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:26 smithi114 ceph-mon[34140]: rocksdb: Options.num_levels: 7 2024-09-21T03:30:26.724 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:26 smithi114 ceph-mon[34140]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-09-21T03:30:26.724 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:26 smithi114 ceph-mon[34140]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-09-21T03:30:26.724 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:26 smithi114 ceph-mon[34140]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-09-21T03:30:26.724 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:26 smithi114 ceph-mon[34140]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-09-21T03:30:26.724 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:26 smithi114 ceph-mon[34140]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-09-21T03:30:26.724 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:26 smithi114 ceph-mon[34140]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-09-21T03:30:26.724 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:26 smithi114 ceph-mon[34140]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-09-21T03:30:26.724 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:26 smithi114 ceph-mon[34140]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-09-21T03:30:26.724 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:26 smithi114 ceph-mon[34140]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-09-21T03:30:26.725 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:26 smithi114 ceph-mon[34140]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-09-21T03:30:26.725 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:26 smithi114 ceph-mon[34140]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-09-21T03:30:26.725 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:26 smithi114 ceph-mon[34140]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-09-21T03:30:26.725 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:26 smithi114 ceph-mon[34140]: rocksdb: Options.compression_opts.window_bits: -14 2024-09-21T03:30:26.725 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:26 smithi114 ceph-mon[34140]: rocksdb: Options.compression_opts.level: 32767 2024-09-21T03:30:26.725 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:26 smithi114 ceph-mon[34140]: rocksdb: Options.compression_opts.strategy: 0 2024-09-21T03:30:26.725 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:26 smithi114 ceph-mon[34140]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-09-21T03:30:26.725 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:26 smithi114 ceph-mon[34140]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-09-21T03:30:26.725 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:26 smithi114 ceph-mon[34140]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-09-21T03:30:26.725 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:26 smithi114 ceph-mon[34140]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-09-21T03:30:26.725 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:26 smithi114 ceph-mon[34140]: rocksdb: Options.compression_opts.enabled: false 2024-09-21T03:30:26.725 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:26 smithi114 ceph-mon[34140]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-09-21T03:30:26.725 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:26 smithi114 ceph-mon[34140]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-09-21T03:30:26.725 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:26 smithi114 ceph-mon[34140]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-09-21T03:30:26.725 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:26 smithi114 ceph-mon[34140]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-09-21T03:30:26.726 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:26 smithi114 ceph-mon[34140]: rocksdb: Options.target_file_size_base: 67108864 2024-09-21T03:30:26.726 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:26 smithi114 ceph-mon[34140]: rocksdb: Options.target_file_size_multiplier: 1 2024-09-21T03:30:26.726 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:26 smithi114 ceph-mon[34140]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-09-21T03:30:26.726 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:26 smithi114 ceph-mon[34140]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-09-21T03:30:26.726 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:26 smithi114 ceph-mon[34140]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-09-21T03:30:26.726 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:26 smithi114 ceph-mon[34140]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-09-21T03:30:26.727 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:26 smithi114 ceph-mon[34140]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-09-21T03:30:26.727 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:26 smithi114 ceph-mon[34140]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-09-21T03:30:26.727 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:26 smithi114 ceph-mon[34140]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-09-21T03:30:26.727 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:26 smithi114 ceph-mon[34140]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-09-21T03:30:26.727 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:26 smithi114 ceph-mon[34140]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-09-21T03:30:26.727 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:26 smithi114 ceph-mon[34140]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-09-21T03:30:26.727 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:26 smithi114 ceph-mon[34140]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-09-21T03:30:26.727 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:26 smithi114 ceph-mon[34140]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-09-21T03:30:26.728 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:26 smithi114 ceph-mon[34140]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-09-21T03:30:26.728 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:26 smithi114 ceph-mon[34140]: rocksdb: Options.arena_block_size: 1048576 2024-09-21T03:30:26.728 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:26 smithi114 ceph-mon[34140]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-09-21T03:30:26.728 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:26 smithi114 ceph-mon[34140]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-09-21T03:30:26.728 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:26 smithi114 ceph-mon[34140]: rocksdb: Options.disable_auto_compactions: 0 2024-09-21T03:30:26.728 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:26 smithi114 ceph-mon[34140]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-09-21T03:30:26.728 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:26 smithi114 ceph-mon[34140]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-09-21T03:30:26.728 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:26 smithi114 ceph-mon[34140]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-09-21T03:30:26.728 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:26 smithi114 ceph-mon[34140]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-09-21T03:30:26.728 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:26 smithi114 ceph-mon[34140]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-09-21T03:30:26.728 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:26 smithi114 ceph-mon[34140]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-09-21T03:30:26.728 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:26 smithi114 ceph-mon[34140]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-09-21T03:30:26.728 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:26 smithi114 ceph-mon[34140]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-09-21T03:30:26.728 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:26 smithi114 ceph-mon[34140]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-09-21T03:30:26.729 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:26 smithi114 ceph-mon[34140]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-09-21T03:30:26.729 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:26 smithi114 ceph-mon[34140]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-09-21T03:30:26.729 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:26 smithi114 ceph-mon[34140]: rocksdb: Options.inplace_update_support: 0 2024-09-21T03:30:26.729 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:26 smithi114 ceph-mon[34140]: rocksdb: Options.inplace_update_num_locks: 10000 2024-09-21T03:30:26.729 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:26 smithi114 ceph-mon[34140]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-09-21T03:30:26.729 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:26 smithi114 ceph-mon[34140]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-09-21T03:30:26.729 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:26 smithi114 ceph-mon[34140]: rocksdb: Options.memtable_huge_page_size: 0 2024-09-21T03:30:26.729 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:26 smithi114 ceph-mon[34140]: rocksdb: Options.bloom_locality: 0 2024-09-21T03:30:26.729 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:26 smithi114 ceph-mon[34140]: rocksdb: Options.max_successive_merges: 0 2024-09-21T03:30:26.729 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:26 smithi114 ceph-mon[34140]: rocksdb: Options.optimize_filters_for_hits: 0 2024-09-21T03:30:26.729 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:26 smithi114 ceph-mon[34140]: rocksdb: Options.paranoid_file_checks: 0 2024-09-21T03:30:26.729 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:26 smithi114 ceph-mon[34140]: rocksdb: Options.force_consistency_checks: 1 2024-09-21T03:30:26.729 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:26 smithi114 ceph-mon[34140]: rocksdb: Options.report_bg_io_stats: 0 2024-09-21T03:30:26.729 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:26 smithi114 ceph-mon[34140]: rocksdb: Options.ttl: 2592000 2024-09-21T03:30:26.729 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:26 smithi114 ceph-mon[34140]: rocksdb: Options.periodic_compaction_seconds: 0 2024-09-21T03:30:26.730 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:26 smithi114 ceph-mon[34140]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-09-21T03:30:26.730 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:26 smithi114 ceph-mon[34140]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-09-21T03:30:26.730 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:26 smithi114 ceph-mon[34140]: rocksdb: Options.enable_blob_files: false 2024-09-21T03:30:26.730 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:26 smithi114 ceph-mon[34140]: rocksdb: Options.min_blob_size: 0 2024-09-21T03:30:26.730 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:26 smithi114 ceph-mon[34140]: rocksdb: Options.blob_file_size: 268435456 2024-09-21T03:30:26.730 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:26 smithi114 ceph-mon[34140]: rocksdb: Options.blob_compression_type: NoCompression 2024-09-21T03:30:26.730 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:26 smithi114 ceph-mon[34140]: rocksdb: Options.enable_blob_garbage_collection: false 2024-09-21T03:30:26.730 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:26 smithi114 ceph-mon[34140]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-09-21T03:30:26.730 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:26 smithi114 ceph-mon[34140]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-09-21T03:30:26.730 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:26 smithi114 ceph-mon[34140]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-09-21T03:30:26.730 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:26 smithi114 ceph-mon[34140]: rocksdb: Options.blob_file_starting_level: 0 2024-09-21T03:30:26.730 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:26 smithi114 ceph-mon[34140]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-09-21T03:30:26.730 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:26 smithi114 ceph-mon[34140]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-09-21T03:30:26.730 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:26 smithi114 ceph-mon[34140]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2024-09-21T03:30:26.730 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:26 smithi114 ceph-mon[34140]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 1d617996-c252-4451-90f2-cb8140fd47d2 2024-09-21T03:30:26.731 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:26 smithi114 ceph-mon[34140]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726889426449215, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-09-21T03:30:26.731 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:26 smithi114 ceph-mon[34140]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2024-09-21T03:30:26.731 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:26 smithi114 ceph-mon[34140]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726889426449835, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1648, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 523, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 401, "raw_average_value_size": 80, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1726889426, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "1d617996-c252-4451-90f2-cb8140fd47d2", "db_session_id": "37BCF4T6GBT3B39JJZ75", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2024-09-21T03:30:26.731 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:26 smithi114 ceph-mon[34140]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726889426449962, "job": 1, "event": "recovery_finished"} 2024-09-21T03:30:26.731 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:26 smithi114 ceph-mon[34140]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2024-09-21T03:30:26.731 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:26 smithi114 ceph-mon[34140]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-c/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-09-21T03:30:26.731 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:26 smithi114 ceph-mon[34140]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x5644c2ddaa80 2024-09-21T03:30:26.731 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:26 smithi114 ceph-mon[34140]: rocksdb: DB pointer 0x5644c3b80000 2024-09-21T03:30:26.731 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:26 smithi114 ceph-mon[34140]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-09-21T03:30:26.731 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:26 smithi114 ceph-mon[34140]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-09-21T03:30:26.731 INFO:journalctl@ceph.mon.c.smithi114.stdout: ** DB Stats ** 2024-09-21T03:30:26.731 INFO:journalctl@ceph.mon.c.smithi114.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-21T03:30:26.731 INFO:journalctl@ceph.mon.c.smithi114.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-09-21T03:30:26.732 INFO:journalctl@ceph.mon.c.smithi114.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-21T03:30:26.732 INFO:journalctl@ceph.mon.c.smithi114.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-21T03:30:26.732 INFO:journalctl@ceph.mon.c.smithi114.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-09-21T03:30:26.732 INFO:journalctl@ceph.mon.c.smithi114.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-21T03:30:26.732 INFO:journalctl@ceph.mon.c.smithi114.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-21T03:30:26.732 INFO:journalctl@ceph.mon.c.smithi114.stdout: 2024-09-21T03:30:26.732 INFO:journalctl@ceph.mon.c.smithi114.stdout: ** Compaction Stats [default] ** 2024-09-21T03:30:26.732 INFO:journalctl@ceph.mon.c.smithi114.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-21T03:30:26.732 INFO:journalctl@ceph.mon.c.smithi114.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-09-21T03:30:26.732 INFO:journalctl@ceph.mon.c.smithi114.stdout: L0 1/0 1.61 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 2.6 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-21T03:30:26.732 INFO:journalctl@ceph.mon.c.smithi114.stdout: Sum 1/0 1.61 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 2.6 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-21T03:30:26.732 INFO:journalctl@ceph.mon.c.smithi114.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 2.6 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-21T03:30:26.732 INFO:journalctl@ceph.mon.c.smithi114.stdout: 2024-09-21T03:30:26.732 INFO:journalctl@ceph.mon.c.smithi114.stdout: ** Compaction Stats [default] ** 2024-09-21T03:30:26.732 INFO:journalctl@ceph.mon.c.smithi114.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-21T03:30:26.733 INFO:journalctl@ceph.mon.c.smithi114.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-21T03:30:26.733 INFO:journalctl@ceph.mon.c.smithi114.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 2.6 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-21T03:30:26.733 INFO:journalctl@ceph.mon.c.smithi114.stdout: 2024-09-21T03:30:26.733 INFO:journalctl@ceph.mon.c.smithi114.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-09-21T03:30:26.733 INFO:journalctl@ceph.mon.c.smithi114.stdout: 2024-09-21T03:30:26.733 INFO:journalctl@ceph.mon.c.smithi114.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-21T03:30:26.733 INFO:journalctl@ceph.mon.c.smithi114.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-09-21T03:30:26.733 INFO:journalctl@ceph.mon.c.smithi114.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-09-21T03:30:26.733 INFO:journalctl@ceph.mon.c.smithi114.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-09-21T03:30:26.733 INFO:journalctl@ceph.mon.c.smithi114.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-09-21T03:30:26.733 INFO:journalctl@ceph.mon.c.smithi114.stdout: AddFile(Keys): cumulative 0, interval 0 2024-09-21T03:30:26.733 INFO:journalctl@ceph.mon.c.smithi114.stdout: Cumulative compaction: 0.00 GB write, 0.41 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-21T03:30:26.733 INFO:journalctl@ceph.mon.c.smithi114.stdout: Interval compaction: 0.00 GB write, 0.41 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-21T03:30:26.733 INFO:journalctl@ceph.mon.c.smithi114.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-09-21T03:30:26.733 INFO:journalctl@ceph.mon.c.smithi114.stdout: Block cache BinnedLRUCache@0x5644bed7b610#2 capacity: 512.00 MB usage: 0.23 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1e-05 secs_since: 0 2024-09-21T03:30:26.733 INFO:journalctl@ceph.mon.c.smithi114.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.12 KB,2.23517e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2024-09-21T03:30:26.734 INFO:journalctl@ceph.mon.c.smithi114.stdout: 2024-09-21T03:30:26.734 INFO:journalctl@ceph.mon.c.smithi114.stdout: ** File Read Latency Histogram By Level [default] ** 2024-09-21T03:30:26.734 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:26 smithi114 ceph-mon[34140]: mon.c does not exist in monmap, will attempt to join an existing cluster 2024-09-21T03:30:26.734 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:26 smithi114 ceph-mon[34140]: using public_addr v2:172.21.15.114:0/0 -> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] 2024-09-21T03:30:26.734 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:26 smithi114 ceph-mon[34140]: starting mon.c rank -1 at public addrs [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] at bind addrs [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] mon_data /var/lib/ceph/mon/ceph-c fsid 7c374098-77c9-11ef-baf5-efdc52797490 2024-09-21T03:30:26.734 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:26 smithi114 ceph-mon[34140]: mon.c@-1(???) e0 preinit fsid 7c374098-77c9-11ef-baf5-efdc52797490 2024-09-21T03:30:26.734 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:26 smithi114 ceph-mon[34140]: mon.c@-1(synchronizing).mds e1 new map 2024-09-21T03:30:26.734 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:26 smithi114 ceph-mon[34140]: mon.c@-1(synchronizing).mds e1 print_map 2024-09-21T03:30:26.734 INFO:journalctl@ceph.mon.c.smithi114.stdout: e1 2024-09-21T03:30:26.734 INFO:journalctl@ceph.mon.c.smithi114.stdout: btime 2024-09-21T03:28:56:681239+0000 2024-09-21T03:30:26.734 INFO:journalctl@ceph.mon.c.smithi114.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-09-21T03:30:26.734 INFO:journalctl@ceph.mon.c.smithi114.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-21T03:30:26.734 INFO:journalctl@ceph.mon.c.smithi114.stdout: legacy client fscid: -1 2024-09-21T03:30:26.734 INFO:journalctl@ceph.mon.c.smithi114.stdout: 2024-09-21T03:30:26.735 INFO:journalctl@ceph.mon.c.smithi114.stdout: No filesystems configured 2024-09-21T03:30:26.735 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:26 smithi114 ceph-mon[34140]: mon.c@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-09-21T03:30:26.735 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:26 smithi114 ceph-mon[34140]: mon.c@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-09-21T03:30:26.735 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:26 smithi114 ceph-mon[34140]: mon.c@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-09-21T03:30:26.735 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:26 smithi114 ceph-mon[34140]: mon.c@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-09-21T03:30:26.735 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:26 smithi114 ceph-mon[34140]: mon.c@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-09-21T03:30:26.735 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:26 smithi114 ceph-mon[34140]: mon.c@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-09-21T03:30:26.735 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:26 smithi114 ceph-mon[34140]: mon.c@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-09-21T03:30:26.735 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:26 smithi114 ceph-mon[34140]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-21T03:30:26.735 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:26 smithi114 ceph-mon[34140]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-21T03:30:26.735 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:26 smithi114 ceph-mon[34140]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-21T03:30:26.735 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:26 smithi114 ceph-mon[34140]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:30:26.735 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:26 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:30:26.735 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:26 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:30:26.735 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:26 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:30:26.736 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:26 smithi114 ceph-mon[34140]: Added host smithi114 2024-09-21T03:30:26.736 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:26 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:30:26.736 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:26 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:30:26.736 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:26 smithi114 ceph-mon[34140]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:30:26.736 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:26 smithi114 ceph-mon[34140]: from='client.14186 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T03:30:26.736 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:26 smithi114 ceph-mon[34140]: from='client.? 172.21.15.89:0/1311650935' entity='client.admin' cmd={"prefix": "osd crush tunables", "profile": "default"} : dispatch 2024-09-21T03:30:26.736 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:26 smithi114 ceph-mon[34140]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:30:26.736 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:26 smithi114 ceph-mon[34140]: from='client.? 172.21.15.89:0/1311650935' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-09-21T03:30:26.736 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:26 smithi114 ceph-mon[34140]: osdmap e4: 0 total, 0 up, 0 in 2024-09-21T03:30:26.736 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:26 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:30:26.736 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:26 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:30:26.736 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:26 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:30:26.736 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:26 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:30:26.736 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:26 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd/host:smithi114", "name": "osd_memory_target"} : dispatch 2024-09-21T03:30:26.737 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:26 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:30:26.737 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:26 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:30:26.737 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:26 smithi114 ceph-mon[34140]: Updating smithi114:/etc/ceph/ceph.conf 2024-09-21T03:30:26.737 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:26 smithi114 ceph-mon[34140]: Updating smithi114:/var/lib/ceph/7c374098-77c9-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-21T03:30:26.737 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:26 smithi114 ceph-mon[34140]: Updating smithi114:/etc/ceph/ceph.client.admin.keyring 2024-09-21T03:30:26.737 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:26 smithi114 ceph-mon[34140]: Updating smithi114:/var/lib/ceph/7c374098-77c9-11ef-baf5-efdc52797490/config/ceph.client.admin.keyring 2024-09-21T03:30:26.737 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:26 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:30:26.737 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:26 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:30:26.737 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:26 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:30:26.737 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:26 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:30:26.737 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:26 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:30:26.737 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:26 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:30:26.737 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:26 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:30:26.737 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:26 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:30:26.737 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:26 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-21T03:30:26.738 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:26 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:30:26.738 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:26 smithi114 ceph-mon[34140]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:30:26.738 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:26 smithi114 ceph-mon[34140]: from='client.14190 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi089:172.21.15.89=a;smithi110:172.21.15.110=b;smithi114:172.21.15.114=c", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:30:26.738 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:26 smithi114 ceph-mon[34140]: Saving service mon spec with placement smithi089:172.21.15.89=a;smithi110:172.21.15.110=b;smithi114:172.21.15.114=c;count:3 2024-09-21T03:30:26.738 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:26 smithi114 ceph-mon[34140]: Deploying daemon mon.c on smithi114 2024-09-21T03:30:26.738 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:26 smithi114 ceph-mon[34140]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:30:26.738 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:26 smithi114 ceph-mon[34140]: from='client.? 172.21.15.114:0/3837685406' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2024-09-21T03:30:26.738 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:26 smithi114 ceph-mon[34140]: mon.c@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-09-21T03:30:26.738 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:26 smithi114 bash[34121]: 1205e7833f22c32e2ce7f554d863b6bc7df66e2260cb028062b7b1edd85e0abb 2024-09-21T03:30:26.738 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:26 smithi114 systemd[1]: Started Ceph mon.c for 7c374098-77c9-11ef-baf5-efdc52797490. 2024-09-21T03:30:27.060 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-21T03:30:27.061 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7c374098-77c9-11ef-baf5-efdc52797490 -- ceph mon dump -f json 2024-09-21T03:30:27.313 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/7c374098-77c9-11ef-baf5-efdc52797490/mon.c/config 2024-09-21T03:30:31.811 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:30:31 smithi089 ceph-mon[30928]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:30:31.811 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:30:31 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-21T03:30:31.811 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:30:31 smithi089 ceph-mon[30928]: mon.a calling monitor election 2024-09-21T03:30:31.812 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:30:31 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T03:30:31.812 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:30:31 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T03:30:31.812 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:30:31 smithi089 ceph-mon[30928]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:30:31.812 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:30:31 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T03:30:31.812 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:30:31 smithi089 ceph-mon[30928]: mon.c calling monitor election 2024-09-21T03:30:31.812 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:30:31 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T03:30:31.812 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:30:31 smithi089 ceph-mon[30928]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:30:31.812 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:30:31 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T03:30:31.812 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:30:31 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T03:30:31.812 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:30:31 smithi089 ceph-mon[30928]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-21T03:30:31.812 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:30:31 smithi089 ceph-mon[30928]: monmap epoch 2 2024-09-21T03:30:31.812 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:30:31 smithi089 ceph-mon[30928]: fsid 7c374098-77c9-11ef-baf5-efdc52797490 2024-09-21T03:30:31.812 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:30:31 smithi089 ceph-mon[30928]: last_changed 2024-09-21T03:30:26.479324+0000 2024-09-21T03:30:31.812 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:30:31 smithi089 ceph-mon[30928]: created 2024-09-21T03:28:53.768514+0000 2024-09-21T03:30:31.813 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:30:31 smithi089 ceph-mon[30928]: min_mon_release 19 (squid) 2024-09-21T03:30:31.813 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:30:31 smithi089 ceph-mon[30928]: election_strategy: 1 2024-09-21T03:30:31.813 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:30:31 smithi089 ceph-mon[30928]: 0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-09-21T03:30:31.813 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:30:31 smithi089 ceph-mon[30928]: 1: [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] mon.c 2024-09-21T03:30:31.813 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:30:31 smithi089 ceph-mon[30928]: fsmap 2024-09-21T03:30:31.813 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:30:31 smithi089 ceph-mon[30928]: osdmap e4: 0 total, 0 up, 0 in 2024-09-21T03:30:31.813 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:30:31 smithi089 ceph-mon[30928]: mgrmap e14: x(active, since 39s) 2024-09-21T03:30:31.813 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:30:31 smithi089 ceph-mon[30928]: overall HEALTH_OK 2024-09-21T03:30:31.813 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:30:31 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:30:31.813 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:30:31 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:30:31.813 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:30:31 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:30:31.813 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:30:31 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-21T03:30:31.813 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:30:31 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:30:31.961 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:31 smithi114 ceph-mon[34140]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:30:31.961 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:31 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-21T03:30:31.961 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:31 smithi114 ceph-mon[34140]: mon.a calling monitor election 2024-09-21T03:30:31.961 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:31 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T03:30:31.961 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:31 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T03:30:31.961 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:31 smithi114 ceph-mon[34140]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:30:31.962 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:31 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T03:30:31.962 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:31 smithi114 ceph-mon[34140]: mon.c calling monitor election 2024-09-21T03:30:31.962 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:31 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T03:30:31.962 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:31 smithi114 ceph-mon[34140]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:30:31.962 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:31 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T03:30:31.962 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:31 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T03:30:31.962 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:31 smithi114 ceph-mon[34140]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-21T03:30:31.962 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:31 smithi114 ceph-mon[34140]: monmap epoch 2 2024-09-21T03:30:31.962 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:31 smithi114 ceph-mon[34140]: fsid 7c374098-77c9-11ef-baf5-efdc52797490 2024-09-21T03:30:31.962 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:31 smithi114 ceph-mon[34140]: last_changed 2024-09-21T03:30:26.479324+0000 2024-09-21T03:30:31.962 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:31 smithi114 ceph-mon[34140]: created 2024-09-21T03:28:53.768514+0000 2024-09-21T03:30:31.962 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:31 smithi114 ceph-mon[34140]: min_mon_release 19 (squid) 2024-09-21T03:30:31.962 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:31 smithi114 ceph-mon[34140]: election_strategy: 1 2024-09-21T03:30:31.962 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:31 smithi114 ceph-mon[34140]: 0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-09-21T03:30:31.963 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:31 smithi114 ceph-mon[34140]: 1: [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] mon.c 2024-09-21T03:30:31.963 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:31 smithi114 ceph-mon[34140]: fsmap 2024-09-21T03:30:31.963 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:31 smithi114 ceph-mon[34140]: osdmap e4: 0 total, 0 up, 0 in 2024-09-21T03:30:31.963 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:31 smithi114 ceph-mon[34140]: mgrmap e14: x(active, since 39s) 2024-09-21T03:30:31.963 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:31 smithi114 ceph-mon[34140]: overall HEALTH_OK 2024-09-21T03:30:31.963 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:31 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:30:31.963 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:31 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:30:31.963 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:31 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:30:31.963 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:31 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-21T03:30:31.963 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:31 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:30:32.430 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-21T03:30:32.430 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":2,"fsid":"7c374098-77c9-11ef-baf5-efdc52797490","modified":"2024-09-21T03:30:26.479324Z","created":"2024-09-21T03:28:53.768514Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:3300","nonce":0},{"type":"v1","addr":"172.21.15.89:6789","nonce":0}]},"addr":"172.21.15.89:6789/0","public_addr":"172.21.15.89:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:3300","nonce":0},{"type":"v1","addr":"172.21.15.114:6789","nonce":0}]},"addr":"172.21.15.114:6789/0","public_addr":"172.21.15.114:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-09-21T03:30:32.430 INFO:teuthology.orchestra.run.smithi114.stderr:dumped monmap epoch 2 2024-09-21T03:30:32.710 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:32 smithi114 ceph-mon[34140]: Deploying daemon mon.b on smithi110 2024-09-21T03:30:32.711 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:32 smithi114 ceph-mon[34140]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:30:32.711 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:32 smithi114 ceph-mon[34140]: from='client.? 172.21.15.114:0/595339574' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2024-09-21T03:30:32.711 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:32 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T03:30:32.811 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:30:32 smithi089 ceph-mon[30928]: Deploying daemon mon.b on smithi110 2024-09-21T03:30:32.811 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:30:32 smithi089 ceph-mon[30928]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:30:32.811 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:30:32 smithi089 ceph-mon[30928]: from='client.? 172.21.15.114:0/595339574' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2024-09-21T03:30:32.811 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:30:32 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T03:30:34.013 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-21T03:30:34.013 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7c374098-77c9-11ef-baf5-efdc52797490 -- ceph mon dump -f json 2024-09-21T03:30:34.264 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/7c374098-77c9-11ef-baf5-efdc52797490/mon.c/config 2024-09-21T03:30:34.811 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:30:34 smithi089 ceph-mon[30928]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:30:34.961 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:34 smithi114 ceph-mon[34140]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:30:34.962 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-21T03:30:34.962 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":2,"fsid":"7c374098-77c9-11ef-baf5-efdc52797490","modified":"2024-09-21T03:30:26.479324Z","created":"2024-09-21T03:28:53.768514Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:3300","nonce":0},{"type":"v1","addr":"172.21.15.89:6789","nonce":0}]},"addr":"172.21.15.89:6789/0","public_addr":"172.21.15.89:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:3300","nonce":0},{"type":"v1","addr":"172.21.15.114:6789","nonce":0}]},"addr":"172.21.15.114:6789/0","public_addr":"172.21.15.114:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-09-21T03:30:34.962 INFO:teuthology.orchestra.run.smithi114.stderr:dumped monmap epoch 2 2024-09-21T03:30:35.000 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:34 smithi110 podman[33980]: 2024-09-21 03:30:34.958818049 +0000 UTC m=+0.123854874 container create 7e32224c6f9aa4dc9fd73c8be4c596132725591097fee7942c7cfd7aeafd46a7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-7c374098-77c9-11ef-baf5-efdc52797490-mon-b, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-09-21T03:30:35.500 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:35 smithi110 podman[33980]: 2024-09-21 03:30:35.161051712 +0000 UTC m=+0.326088547 container init 7e32224c6f9aa4dc9fd73c8be4c596132725591097fee7942c7cfd7aeafd46a7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-7c374098-77c9-11ef-baf5-efdc52797490-mon-b, GIT_CLEAN=True, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-21T03:30:35.501 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:35 smithi110 podman[33980]: 2024-09-21 03:30:35.164241038 +0000 UTC m=+0.329277866 container start 7e32224c6f9aa4dc9fd73c8be4c596132725591097fee7942c7cfd7aeafd46a7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-7c374098-77c9-11ef-baf5-efdc52797490-mon-b, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , ceph=True) 2024-09-21T03:30:35.501 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:35 smithi110 ceph-mon[34000]: set uid:gid to 167:167 (ceph:ceph) 2024-09-21T03:30:35.501 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:35 smithi110 ceph-mon[34000]: ceph version 19.3.0-5120-g18432218 (1843221895adccfb3b692060251dd6a0183bc410) squid (dev), process ceph-mon, pid 2 2024-09-21T03:30:35.501 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:35 smithi110 ceph-mon[34000]: pidfile_write: ignore empty --pid-file 2024-09-21T03:30:35.501 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:35 smithi110 ceph-mon[34000]: load: jerasure load: lrc 2024-09-21T03:30:35.501 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:35 smithi110 ceph-mon[34000]: rocksdb: RocksDB version: 7.9.2 2024-09-21T03:30:35.501 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:35 smithi110 ceph-mon[34000]: rocksdb: Git sha 0 2024-09-21T03:30:35.501 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:35 smithi110 ceph-mon[34000]: rocksdb: Compile date 2024-09-20 22:29:49 2024-09-21T03:30:35.501 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:35 smithi110 ceph-mon[34000]: rocksdb: DB SUMMARY 2024-09-21T03:30:35.501 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:35 smithi110 ceph-mon[34000]: rocksdb: DB Session ID: D1KD5TQ7LL9EV26RIDNK 2024-09-21T03:30:35.501 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:35 smithi110 ceph-mon[34000]: rocksdb: CURRENT file: CURRENT 2024-09-21T03:30:35.502 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:35 smithi110 ceph-mon[34000]: rocksdb: IDENTITY file: IDENTITY 2024-09-21T03:30:35.502 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:35 smithi110 ceph-mon[34000]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2024-09-21T03:30:35.502 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:35 smithi110 ceph-mon[34000]: rocksdb: SST files in /var/lib/ceph/mon/ceph-b/store.db dir, Total Num: 0, files: 2024-09-21T03:30:35.502 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:35 smithi110 ceph-mon[34000]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-b/store.db: 000004.log size: 636 ; 2024-09-21T03:30:35.502 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:35 smithi110 ceph-mon[34000]: rocksdb: Options.error_if_exists: 0 2024-09-21T03:30:35.502 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:35 smithi110 ceph-mon[34000]: rocksdb: Options.create_if_missing: 0 2024-09-21T03:30:35.502 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:35 smithi110 ceph-mon[34000]: rocksdb: Options.paranoid_checks: 1 2024-09-21T03:30:35.502 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:35 smithi110 ceph-mon[34000]: rocksdb: Options.flush_verify_memtable_count: 1 2024-09-21T03:30:35.502 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:35 smithi110 ceph-mon[34000]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-09-21T03:30:35.502 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:35 smithi110 ceph-mon[34000]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-09-21T03:30:35.502 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:35 smithi110 ceph-mon[34000]: rocksdb: Options.env: 0x56295bd9fd60 2024-09-21T03:30:35.502 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:35 smithi110 ceph-mon[34000]: rocksdb: Options.fs: PosixFileSystem 2024-09-21T03:30:35.503 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:35 smithi110 ceph-mon[34000]: rocksdb: Options.info_log: 0x562961df7360 2024-09-21T03:30:35.503 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:35 smithi110 ceph-mon[34000]: rocksdb: Options.max_file_opening_threads: 16 2024-09-21T03:30:35.503 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:35 smithi110 ceph-mon[34000]: rocksdb: Options.statistics: (nil) 2024-09-21T03:30:35.503 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:35 smithi110 ceph-mon[34000]: rocksdb: Options.use_fsync: 0 2024-09-21T03:30:35.503 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:35 smithi110 ceph-mon[34000]: rocksdb: Options.max_log_file_size: 0 2024-09-21T03:30:35.503 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:35 smithi110 ceph-mon[34000]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-09-21T03:30:35.503 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:35 smithi110 ceph-mon[34000]: rocksdb: Options.log_file_time_to_roll: 0 2024-09-21T03:30:35.503 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:35 smithi110 ceph-mon[34000]: rocksdb: Options.keep_log_file_num: 1000 2024-09-21T03:30:35.503 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:35 smithi110 ceph-mon[34000]: rocksdb: Options.recycle_log_file_num: 0 2024-09-21T03:30:35.503 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:35 smithi110 ceph-mon[34000]: rocksdb: Options.allow_fallocate: 1 2024-09-21T03:30:35.503 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:35 smithi110 ceph-mon[34000]: rocksdb: Options.allow_mmap_reads: 0 2024-09-21T03:30:35.503 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:35 smithi110 ceph-mon[34000]: rocksdb: Options.allow_mmap_writes: 0 2024-09-21T03:30:35.503 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:35 smithi110 ceph-mon[34000]: rocksdb: Options.use_direct_reads: 0 2024-09-21T03:30:35.503 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:35 smithi110 ceph-mon[34000]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-09-21T03:30:35.504 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:35 smithi110 ceph-mon[34000]: rocksdb: Options.create_missing_column_families: 0 2024-09-21T03:30:35.504 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:35 smithi110 ceph-mon[34000]: rocksdb: Options.db_log_dir: 2024-09-21T03:30:35.504 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:35 smithi110 ceph-mon[34000]: rocksdb: Options.wal_dir: 2024-09-21T03:30:35.504 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:35 smithi110 ceph-mon[34000]: rocksdb: Options.table_cache_numshardbits: 6 2024-09-21T03:30:35.504 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:35 smithi110 ceph-mon[34000]: rocksdb: Options.WAL_ttl_seconds: 0 2024-09-21T03:30:35.504 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:35 smithi110 ceph-mon[34000]: rocksdb: Options.WAL_size_limit_MB: 0 2024-09-21T03:30:35.504 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:35 smithi110 ceph-mon[34000]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-09-21T03:30:35.504 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:35 smithi110 ceph-mon[34000]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-09-21T03:30:35.504 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:35 smithi110 ceph-mon[34000]: rocksdb: Options.is_fd_close_on_exec: 1 2024-09-21T03:30:35.504 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:35 smithi110 ceph-mon[34000]: rocksdb: Options.advise_random_on_open: 1 2024-09-21T03:30:35.504 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:35 smithi110 ceph-mon[34000]: rocksdb: Options.db_write_buffer_size: 0 2024-09-21T03:30:35.504 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:35 smithi110 ceph-mon[34000]: rocksdb: Options.write_buffer_manager: 0x562961dfbd60 2024-09-21T03:30:35.505 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:35 smithi110 ceph-mon[34000]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-09-21T03:30:35.505 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:35 smithi110 ceph-mon[34000]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-09-21T03:30:35.505 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:35 smithi110 ceph-mon[34000]: rocksdb: Options.use_adaptive_mutex: 0 2024-09-21T03:30:35.505 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:35 smithi110 ceph-mon[34000]: rocksdb: Options.rate_limiter: (nil) 2024-09-21T03:30:35.505 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:35 smithi110 ceph-mon[34000]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-09-21T03:30:35.505 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:35 smithi110 ceph-mon[34000]: rocksdb: Options.wal_recovery_mode: 2 2024-09-21T03:30:35.505 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:35 smithi110 ceph-mon[34000]: rocksdb: Options.enable_thread_tracking: 0 2024-09-21T03:30:35.505 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:35 smithi110 ceph-mon[34000]: rocksdb: Options.enable_pipelined_write: 0 2024-09-21T03:30:35.505 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:35 smithi110 ceph-mon[34000]: rocksdb: Options.unordered_write: 0 2024-09-21T03:30:35.505 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:35 smithi110 ceph-mon[34000]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-09-21T03:30:35.505 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:35 smithi110 ceph-mon[34000]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-09-21T03:30:35.505 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:35 smithi110 ceph-mon[34000]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-09-21T03:30:35.505 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:35 smithi110 ceph-mon[34000]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-09-21T03:30:35.505 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:35 smithi110 ceph-mon[34000]: rocksdb: Options.row_cache: None 2024-09-21T03:30:35.506 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:35 smithi110 ceph-mon[34000]: rocksdb: Options.wal_filter: None 2024-09-21T03:30:35.507 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:35 smithi110 ceph-mon[34000]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-09-21T03:30:35.507 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:35 smithi110 ceph-mon[34000]: rocksdb: Options.allow_ingest_behind: 0 2024-09-21T03:30:35.507 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:35 smithi110 ceph-mon[34000]: rocksdb: Options.two_write_queues: 0 2024-09-21T03:30:35.507 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:35 smithi110 ceph-mon[34000]: rocksdb: Options.manual_wal_flush: 0 2024-09-21T03:30:35.507 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:35 smithi110 ceph-mon[34000]: rocksdb: Options.wal_compression: 0 2024-09-21T03:30:35.507 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:35 smithi110 ceph-mon[34000]: rocksdb: Options.atomic_flush: 0 2024-09-21T03:30:35.507 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:35 smithi110 ceph-mon[34000]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-09-21T03:30:35.508 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:35 smithi110 ceph-mon[34000]: rocksdb: Options.persist_stats_to_disk: 0 2024-09-21T03:30:35.508 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:35 smithi110 ceph-mon[34000]: rocksdb: Options.write_dbid_to_manifest: 0 2024-09-21T03:30:35.508 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:35 smithi110 ceph-mon[34000]: rocksdb: Options.log_readahead_size: 0 2024-09-21T03:30:35.508 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:35 smithi110 ceph-mon[34000]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-09-21T03:30:35.508 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:35 smithi110 ceph-mon[34000]: rocksdb: Options.best_efforts_recovery: 0 2024-09-21T03:30:35.508 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:35 smithi110 ceph-mon[34000]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-09-21T03:30:35.508 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:35 smithi110 ceph-mon[34000]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-09-21T03:30:35.508 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:35 smithi110 ceph-mon[34000]: rocksdb: Options.allow_data_in_errors: 0 2024-09-21T03:30:35.508 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:35 smithi110 ceph-mon[34000]: rocksdb: Options.db_host_id: __hostname__ 2024-09-21T03:30:35.508 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:35 smithi110 ceph-mon[34000]: rocksdb: Options.enforce_single_del_contracts: true 2024-09-21T03:30:35.508 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:35 smithi110 ceph-mon[34000]: rocksdb: Options.max_background_jobs: 2 2024-09-21T03:30:35.508 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:35 smithi110 ceph-mon[34000]: rocksdb: Options.max_background_compactions: -1 2024-09-21T03:30:35.508 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:35 smithi110 ceph-mon[34000]: rocksdb: Options.max_subcompactions: 1 2024-09-21T03:30:35.508 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:35 smithi110 ceph-mon[34000]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-09-21T03:30:35.509 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:35 smithi110 ceph-mon[34000]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-09-21T03:30:35.509 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:35 smithi110 ceph-mon[34000]: rocksdb: Options.delayed_write_rate : 16777216 2024-09-21T03:30:35.509 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:35 smithi110 ceph-mon[34000]: rocksdb: Options.max_total_wal_size: 0 2024-09-21T03:30:35.509 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:35 smithi110 ceph-mon[34000]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-09-21T03:30:35.509 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:35 smithi110 ceph-mon[34000]: rocksdb: Options.stats_dump_period_sec: 600 2024-09-21T03:30:35.509 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:35 smithi110 ceph-mon[34000]: rocksdb: Options.stats_persist_period_sec: 600 2024-09-21T03:30:35.509 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:35 smithi110 ceph-mon[34000]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-09-21T03:30:35.509 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:35 smithi110 ceph-mon[34000]: rocksdb: Options.max_open_files: -1 2024-09-21T03:30:35.509 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:35 smithi110 ceph-mon[34000]: rocksdb: Options.bytes_per_sync: 0 2024-09-21T03:30:35.509 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:35 smithi110 ceph-mon[34000]: rocksdb: Options.wal_bytes_per_sync: 0 2024-09-21T03:30:35.509 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:35 smithi110 ceph-mon[34000]: rocksdb: Options.strict_bytes_per_sync: 0 2024-09-21T03:30:35.509 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:35 smithi110 ceph-mon[34000]: rocksdb: Options.compaction_readahead_size: 0 2024-09-21T03:30:35.509 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:35 smithi110 ceph-mon[34000]: rocksdb: Options.max_background_flushes: -1 2024-09-21T03:30:35.509 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:35 smithi110 ceph-mon[34000]: rocksdb: Compression algorithms supported: 2024-09-21T03:30:35.509 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:35 smithi110 ceph-mon[34000]: rocksdb: kZSTD supported: 0 2024-09-21T03:30:35.510 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:35 smithi110 ceph-mon[34000]: rocksdb: kXpressCompression supported: 0 2024-09-21T03:30:35.510 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:35 smithi110 ceph-mon[34000]: rocksdb: kBZip2Compression supported: 0 2024-09-21T03:30:35.510 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:35 smithi110 ceph-mon[34000]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-09-21T03:30:35.510 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:35 smithi110 ceph-mon[34000]: rocksdb: kLZ4Compression supported: 1 2024-09-21T03:30:35.510 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:35 smithi110 ceph-mon[34000]: rocksdb: kZlibCompression supported: 1 2024-09-21T03:30:35.510 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:35 smithi110 ceph-mon[34000]: rocksdb: kLZ4HCCompression supported: 1 2024-09-21T03:30:35.510 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:35 smithi110 ceph-mon[34000]: rocksdb: kSnappyCompression supported: 1 2024-09-21T03:30:35.510 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:35 smithi110 ceph-mon[34000]: rocksdb: Fast CRC32 supported: Supported on x86 2024-09-21T03:30:35.510 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:35 smithi110 ceph-mon[34000]: rocksdb: DMutex implementation: pthread_mutex_t 2024-09-21T03:30:35.510 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:35 smithi110 ceph-mon[34000]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 2024-09-21T03:30:35.510 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:35 smithi110 ceph-mon[34000]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-09-21T03:30:35.510 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:35 smithi110 ceph-mon[34000]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-09-21T03:30:35.510 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:35 smithi110 ceph-mon[34000]: rocksdb: Options.merge_operator: 2024-09-21T03:30:35.510 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:35 smithi110 ceph-mon[34000]: rocksdb: Options.compaction_filter: None 2024-09-21T03:30:35.511 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:35 smithi110 ceph-mon[34000]: rocksdb: Options.compaction_filter_factory: None 2024-09-21T03:30:35.511 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:35 smithi110 ceph-mon[34000]: rocksdb: Options.sst_partitioner_factory: None 2024-09-21T03:30:35.511 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:35 smithi110 ceph-mon[34000]: rocksdb: Options.memtable_factory: SkipListFactory 2024-09-21T03:30:35.511 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:35 smithi110 ceph-mon[34000]: rocksdb: Options.table_factory: BlockBasedTable 2024-09-21T03:30:35.511 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:35 smithi110 ceph-mon[34000]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x562961df6fa0) 2024-09-21T03:30:35.511 INFO:journalctl@ceph.mon.b.smithi110.stdout: cache_index_and_filter_blocks: 1 2024-09-21T03:30:35.511 INFO:journalctl@ceph.mon.b.smithi110.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-09-21T03:30:35.511 INFO:journalctl@ceph.mon.b.smithi110.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-09-21T03:30:35.511 INFO:journalctl@ceph.mon.b.smithi110.stdout: pin_top_level_index_and_filter: 1 2024-09-21T03:30:35.511 INFO:journalctl@ceph.mon.b.smithi110.stdout: index_type: 0 2024-09-21T03:30:35.511 INFO:journalctl@ceph.mon.b.smithi110.stdout: data_block_index_type: 0 2024-09-21T03:30:35.511 INFO:journalctl@ceph.mon.b.smithi110.stdout: index_shortening: 1 2024-09-21T03:30:35.511 INFO:journalctl@ceph.mon.b.smithi110.stdout: data_block_hash_table_util_ratio: 0.750000 2024-09-21T03:30:35.511 INFO:journalctl@ceph.mon.b.smithi110.stdout: checksum: 4 2024-09-21T03:30:35.512 INFO:journalctl@ceph.mon.b.smithi110.stdout: no_block_cache: 0 2024-09-21T03:30:35.512 INFO:journalctl@ceph.mon.b.smithi110.stdout: block_cache: 0x56295ddbb610 2024-09-21T03:30:35.512 INFO:journalctl@ceph.mon.b.smithi110.stdout: block_cache_name: BinnedLRUCache 2024-09-21T03:30:35.512 INFO:journalctl@ceph.mon.b.smithi110.stdout: block_cache_options: 2024-09-21T03:30:35.512 INFO:journalctl@ceph.mon.b.smithi110.stdout: capacity : 536870912 2024-09-21T03:30:35.512 INFO:journalctl@ceph.mon.b.smithi110.stdout: num_shard_bits : 4 2024-09-21T03:30:35.512 INFO:journalctl@ceph.mon.b.smithi110.stdout: strict_capacity_limit : 0 2024-09-21T03:30:35.512 INFO:journalctl@ceph.mon.b.smithi110.stdout: high_pri_pool_ratio: 0.000 2024-09-21T03:30:35.512 INFO:journalctl@ceph.mon.b.smithi110.stdout: block_cache_compressed: (nil) 2024-09-21T03:30:35.512 INFO:journalctl@ceph.mon.b.smithi110.stdout: persistent_cache: (nil) 2024-09-21T03:30:35.512 INFO:journalctl@ceph.mon.b.smithi110.stdout: block_size: 4096 2024-09-21T03:30:35.512 INFO:journalctl@ceph.mon.b.smithi110.stdout: block_size_deviation: 10 2024-09-21T03:30:35.512 INFO:journalctl@ceph.mon.b.smithi110.stdout: block_restart_interval: 16 2024-09-21T03:30:35.512 INFO:journalctl@ceph.mon.b.smithi110.stdout: index_block_restart_interval: 1 2024-09-21T03:30:35.513 INFO:journalctl@ceph.mon.b.smithi110.stdout: metadata_block_size: 4096 2024-09-21T03:30:35.513 INFO:journalctl@ceph.mon.b.smithi110.stdout: partition_filters: 0 2024-09-21T03:30:35.513 INFO:journalctl@ceph.mon.b.smithi110.stdout: use_delta_encoding: 1 2024-09-21T03:30:35.513 INFO:journalctl@ceph.mon.b.smithi110.stdout: filter_policy: bloomfilter 2024-09-21T03:30:35.513 INFO:journalctl@ceph.mon.b.smithi110.stdout: whole_key_filtering: 1 2024-09-21T03:30:35.513 INFO:journalctl@ceph.mon.b.smithi110.stdout: verify_compression: 0 2024-09-21T03:30:35.513 INFO:journalctl@ceph.mon.b.smithi110.stdout: read_amp_bytes_per_bit: 0 2024-09-21T03:30:35.513 INFO:journalctl@ceph.mon.b.smithi110.stdout: format_version: 5 2024-09-21T03:30:35.513 INFO:journalctl@ceph.mon.b.smithi110.stdout: enable_index_compression: 1 2024-09-21T03:30:35.513 INFO:journalctl@ceph.mon.b.smithi110.stdout: block_align: 0 2024-09-21T03:30:35.513 INFO:journalctl@ceph.mon.b.smithi110.stdout: max_auto_readahead_size: 262144 2024-09-21T03:30:35.513 INFO:journalctl@ceph.mon.b.smithi110.stdout: prepopulate_block_cache: 0 2024-09-21T03:30:35.513 INFO:journalctl@ceph.mon.b.smithi110.stdout: initial_auto_readahead_size: 8192 2024-09-21T03:30:35.513 INFO:journalctl@ceph.mon.b.smithi110.stdout: num_file_reads_for_auto_readahead: 2 2024-09-21T03:30:35.513 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:35 smithi110 ceph-mon[34000]: rocksdb: Options.write_buffer_size: 33554432 2024-09-21T03:30:35.514 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:35 smithi110 ceph-mon[34000]: rocksdb: Options.max_write_buffer_number: 2 2024-09-21T03:30:35.514 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:35 smithi110 ceph-mon[34000]: rocksdb: Options.compression: NoCompression 2024-09-21T03:30:35.514 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:35 smithi110 ceph-mon[34000]: rocksdb: Options.bottommost_compression: Disabled 2024-09-21T03:30:35.514 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:35 smithi110 ceph-mon[34000]: rocksdb: Options.prefix_extractor: nullptr 2024-09-21T03:30:35.514 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:35 smithi110 ceph-mon[34000]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-09-21T03:30:35.514 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:35 smithi110 ceph-mon[34000]: rocksdb: Options.num_levels: 7 2024-09-21T03:30:35.514 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:35 smithi110 ceph-mon[34000]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-09-21T03:30:35.514 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:35 smithi110 ceph-mon[34000]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-09-21T03:30:35.514 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:35 smithi110 ceph-mon[34000]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-09-21T03:30:35.514 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:35 smithi110 ceph-mon[34000]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-09-21T03:30:35.514 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:35 smithi110 ceph-mon[34000]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-09-21T03:30:35.514 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:35 smithi110 ceph-mon[34000]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-09-21T03:30:35.514 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:35 smithi110 ceph-mon[34000]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-09-21T03:30:35.514 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:35 smithi110 ceph-mon[34000]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-09-21T03:30:35.514 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:35 smithi110 ceph-mon[34000]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-09-21T03:30:35.515 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:35 smithi110 ceph-mon[34000]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-09-21T03:30:35.515 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:35 smithi110 ceph-mon[34000]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-09-21T03:30:35.515 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:35 smithi110 ceph-mon[34000]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-09-21T03:30:35.515 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:35 smithi110 ceph-mon[34000]: rocksdb: Options.compression_opts.window_bits: -14 2024-09-21T03:30:35.515 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:35 smithi110 ceph-mon[34000]: rocksdb: Options.compression_opts.level: 32767 2024-09-21T03:30:35.515 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:35 smithi110 ceph-mon[34000]: rocksdb: Options.compression_opts.strategy: 0 2024-09-21T03:30:35.515 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:35 smithi110 ceph-mon[34000]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-09-21T03:30:35.515 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:35 smithi110 ceph-mon[34000]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-09-21T03:30:35.515 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:35 smithi110 ceph-mon[34000]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-09-21T03:30:35.515 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:35 smithi110 ceph-mon[34000]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-09-21T03:30:35.515 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:35 smithi110 ceph-mon[34000]: rocksdb: Options.compression_opts.enabled: false 2024-09-21T03:30:35.515 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:35 smithi110 ceph-mon[34000]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-09-21T03:30:35.515 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:35 smithi110 ceph-mon[34000]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-09-21T03:30:35.515 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:35 smithi110 ceph-mon[34000]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-09-21T03:30:35.516 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:35 smithi110 ceph-mon[34000]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-09-21T03:30:35.516 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:35 smithi110 ceph-mon[34000]: rocksdb: Options.target_file_size_base: 67108864 2024-09-21T03:30:35.516 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:35 smithi110 ceph-mon[34000]: rocksdb: Options.target_file_size_multiplier: 1 2024-09-21T03:30:35.516 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:35 smithi110 ceph-mon[34000]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-09-21T03:30:35.516 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:35 smithi110 ceph-mon[34000]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-09-21T03:30:35.516 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:35 smithi110 ceph-mon[34000]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-09-21T03:30:35.516 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:35 smithi110 ceph-mon[34000]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-09-21T03:30:35.516 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:35 smithi110 ceph-mon[34000]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-09-21T03:30:35.516 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:35 smithi110 ceph-mon[34000]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-09-21T03:30:35.516 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:35 smithi110 ceph-mon[34000]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-09-21T03:30:35.516 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:35 smithi110 ceph-mon[34000]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-09-21T03:30:35.516 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:35 smithi110 ceph-mon[34000]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-09-21T03:30:35.516 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:35 smithi110 ceph-mon[34000]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-09-21T03:30:35.516 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:35 smithi110 ceph-mon[34000]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-09-21T03:30:35.516 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:35 smithi110 ceph-mon[34000]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-09-21T03:30:35.517 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:35 smithi110 ceph-mon[34000]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-09-21T03:30:35.517 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:35 smithi110 ceph-mon[34000]: rocksdb: Options.arena_block_size: 1048576 2024-09-21T03:30:35.517 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:35 smithi110 ceph-mon[34000]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-09-21T03:30:35.517 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:35 smithi110 ceph-mon[34000]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-09-21T03:30:35.517 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:35 smithi110 ceph-mon[34000]: rocksdb: Options.disable_auto_compactions: 0 2024-09-21T03:30:35.517 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:35 smithi110 ceph-mon[34000]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-09-21T03:30:35.517 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:35 smithi110 ceph-mon[34000]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-09-21T03:30:35.517 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:35 smithi110 ceph-mon[34000]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-09-21T03:30:35.517 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:35 smithi110 ceph-mon[34000]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-09-21T03:30:35.517 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:35 smithi110 ceph-mon[34000]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-09-21T03:30:35.517 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:35 smithi110 ceph-mon[34000]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-09-21T03:30:35.517 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:35 smithi110 ceph-mon[34000]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-09-21T03:30:35.517 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:35 smithi110 ceph-mon[34000]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-09-21T03:30:35.517 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:35 smithi110 ceph-mon[34000]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-09-21T03:30:35.517 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:35 smithi110 ceph-mon[34000]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-09-21T03:30:35.518 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:35 smithi110 ceph-mon[34000]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-09-21T03:30:35.518 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:35 smithi110 ceph-mon[34000]: rocksdb: Options.inplace_update_support: 0 2024-09-21T03:30:35.518 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:35 smithi110 ceph-mon[34000]: rocksdb: Options.inplace_update_num_locks: 10000 2024-09-21T03:30:35.518 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:35 smithi110 ceph-mon[34000]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-09-21T03:30:35.518 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:35 smithi110 ceph-mon[34000]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-09-21T03:30:35.518 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:35 smithi110 ceph-mon[34000]: rocksdb: Options.memtable_huge_page_size: 0 2024-09-21T03:30:35.518 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:35 smithi110 ceph-mon[34000]: rocksdb: Options.bloom_locality: 0 2024-09-21T03:30:35.518 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:35 smithi110 ceph-mon[34000]: rocksdb: Options.max_successive_merges: 0 2024-09-21T03:30:35.518 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:35 smithi110 ceph-mon[34000]: rocksdb: Options.optimize_filters_for_hits: 0 2024-09-21T03:30:35.518 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:35 smithi110 ceph-mon[34000]: rocksdb: Options.paranoid_file_checks: 0 2024-09-21T03:30:35.518 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:35 smithi110 ceph-mon[34000]: rocksdb: Options.force_consistency_checks: 1 2024-09-21T03:30:35.518 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:35 smithi110 ceph-mon[34000]: rocksdb: Options.report_bg_io_stats: 0 2024-09-21T03:30:35.518 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:35 smithi110 ceph-mon[34000]: rocksdb: Options.ttl: 2592000 2024-09-21T03:30:35.518 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:35 smithi110 ceph-mon[34000]: rocksdb: Options.periodic_compaction_seconds: 0 2024-09-21T03:30:35.519 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:35 smithi110 ceph-mon[34000]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-09-21T03:30:35.519 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:35 smithi110 ceph-mon[34000]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-09-21T03:30:35.519 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:35 smithi110 ceph-mon[34000]: rocksdb: Options.enable_blob_files: false 2024-09-21T03:30:35.519 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:35 smithi110 ceph-mon[34000]: rocksdb: Options.min_blob_size: 0 2024-09-21T03:30:35.519 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:35 smithi110 ceph-mon[34000]: rocksdb: Options.blob_file_size: 268435456 2024-09-21T03:30:35.519 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:35 smithi110 ceph-mon[34000]: rocksdb: Options.blob_compression_type: NoCompression 2024-09-21T03:30:35.519 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:35 smithi110 ceph-mon[34000]: rocksdb: Options.enable_blob_garbage_collection: false 2024-09-21T03:30:35.519 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:35 smithi110 ceph-mon[34000]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-09-21T03:30:35.519 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:35 smithi110 ceph-mon[34000]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-09-21T03:30:35.519 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:35 smithi110 ceph-mon[34000]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-09-21T03:30:35.519 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:35 smithi110 ceph-mon[34000]: rocksdb: Options.blob_file_starting_level: 0 2024-09-21T03:30:35.519 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:35 smithi110 ceph-mon[34000]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-09-21T03:30:35.519 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:35 smithi110 ceph-mon[34000]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-09-21T03:30:35.519 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:35 smithi110 ceph-mon[34000]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2024-09-21T03:30:35.519 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:35 smithi110 ceph-mon[34000]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: e38873f3-e902-4902-9adb-f8e420a3d0a3 2024-09-21T03:30:35.520 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:35 smithi110 ceph-mon[34000]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726889435200673, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-09-21T03:30:35.520 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:35 smithi110 ceph-mon[34000]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2024-09-21T03:30:35.520 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:35 smithi110 ceph-mon[34000]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726889435201123, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1773, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 648, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 526, "raw_average_value_size": 105, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1726889435, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "e38873f3-e902-4902-9adb-f8e420a3d0a3", "db_session_id": "D1KD5TQ7LL9EV26RIDNK", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2024-09-21T03:30:35.520 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:35 smithi110 ceph-mon[34000]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726889435201220, "job": 1, "event": "recovery_finished"} 2024-09-21T03:30:35.520 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:35 smithi110 ceph-mon[34000]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2024-09-21T03:30:35.520 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:35 smithi110 ceph-mon[34000]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-b/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-09-21T03:30:35.520 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:35 smithi110 ceph-mon[34000]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x562961e1aa80 2024-09-21T03:30:35.520 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:35 smithi110 ceph-mon[34000]: rocksdb: DB pointer 0x562962bc0000 2024-09-21T03:30:35.520 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:35 smithi110 ceph-mon[34000]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-09-21T03:30:35.520 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:35 smithi110 ceph-mon[34000]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-09-21T03:30:35.520 INFO:journalctl@ceph.mon.b.smithi110.stdout: ** DB Stats ** 2024-09-21T03:30:35.520 INFO:journalctl@ceph.mon.b.smithi110.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-21T03:30:35.520 INFO:journalctl@ceph.mon.b.smithi110.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-09-21T03:30:35.520 INFO:journalctl@ceph.mon.b.smithi110.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-21T03:30:35.521 INFO:journalctl@ceph.mon.b.smithi110.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-21T03:30:35.521 INFO:journalctl@ceph.mon.b.smithi110.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-09-21T03:30:35.521 INFO:journalctl@ceph.mon.b.smithi110.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-21T03:30:35.521 INFO:journalctl@ceph.mon.b.smithi110.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-21T03:30:35.521 INFO:journalctl@ceph.mon.b.smithi110.stdout: 2024-09-21T03:30:35.521 INFO:journalctl@ceph.mon.b.smithi110.stdout: ** Compaction Stats [default] ** 2024-09-21T03:30:35.521 INFO:journalctl@ceph.mon.b.smithi110.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-21T03:30:35.521 INFO:journalctl@ceph.mon.b.smithi110.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-09-21T03:30:35.521 INFO:journalctl@ceph.mon.b.smithi110.stdout: L0 1/0 1.73 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.9 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-09-21T03:30:35.521 INFO:journalctl@ceph.mon.b.smithi110.stdout: Sum 1/0 1.73 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.9 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-09-21T03:30:35.521 INFO:journalctl@ceph.mon.b.smithi110.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.9 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-09-21T03:30:35.521 INFO:journalctl@ceph.mon.b.smithi110.stdout: 2024-09-21T03:30:35.521 INFO:journalctl@ceph.mon.b.smithi110.stdout: ** Compaction Stats [default] ** 2024-09-21T03:30:35.521 INFO:journalctl@ceph.mon.b.smithi110.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-21T03:30:35.522 INFO:journalctl@ceph.mon.b.smithi110.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-21T03:30:35.522 INFO:journalctl@ceph.mon.b.smithi110.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 3.9 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-09-21T03:30:35.522 INFO:journalctl@ceph.mon.b.smithi110.stdout: 2024-09-21T03:30:35.522 INFO:journalctl@ceph.mon.b.smithi110.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-09-21T03:30:35.522 INFO:journalctl@ceph.mon.b.smithi110.stdout: 2024-09-21T03:30:35.522 INFO:journalctl@ceph.mon.b.smithi110.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-21T03:30:35.522 INFO:journalctl@ceph.mon.b.smithi110.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-09-21T03:30:35.522 INFO:journalctl@ceph.mon.b.smithi110.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-09-21T03:30:35.522 INFO:journalctl@ceph.mon.b.smithi110.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-09-21T03:30:35.522 INFO:journalctl@ceph.mon.b.smithi110.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-09-21T03:30:35.522 INFO:journalctl@ceph.mon.b.smithi110.stdout: AddFile(Keys): cumulative 0, interval 0 2024-09-21T03:30:35.522 INFO:journalctl@ceph.mon.b.smithi110.stdout: Cumulative compaction: 0.00 GB write, 0.64 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-21T03:30:35.522 INFO:journalctl@ceph.mon.b.smithi110.stdout: Interval compaction: 0.00 GB write, 0.64 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-21T03:30:35.522 INFO:journalctl@ceph.mon.b.smithi110.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-09-21T03:30:35.522 INFO:journalctl@ceph.mon.b.smithi110.stdout: Block cache BinnedLRUCache@0x56295ddbb610#2 capacity: 512.00 MB usage: 0.23 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 9e-06 secs_since: 0 2024-09-21T03:30:35.523 INFO:journalctl@ceph.mon.b.smithi110.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.12 KB,2.23517e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2024-09-21T03:30:35.523 INFO:journalctl@ceph.mon.b.smithi110.stdout: 2024-09-21T03:30:35.523 INFO:journalctl@ceph.mon.b.smithi110.stdout: ** File Read Latency Histogram By Level [default] ** 2024-09-21T03:30:35.523 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:35 smithi110 ceph-mon[34000]: mon.b does not exist in monmap, will attempt to join an existing cluster 2024-09-21T03:30:35.523 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:35 smithi110 ceph-mon[34000]: using public_addr v2:172.21.15.110:0/0 -> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] 2024-09-21T03:30:35.523 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:35 smithi110 ceph-mon[34000]: starting mon.b rank -1 at public addrs [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] at bind addrs [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] mon_data /var/lib/ceph/mon/ceph-b fsid 7c374098-77c9-11ef-baf5-efdc52797490 2024-09-21T03:30:35.523 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:35 smithi110 ceph-mon[34000]: mon.b@-1(???) e0 preinit fsid 7c374098-77c9-11ef-baf5-efdc52797490 2024-09-21T03:30:35.523 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:35 smithi110 ceph-mon[34000]: mon.b@-1(synchronizing).mds e1 new map 2024-09-21T03:30:35.523 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:35 smithi110 ceph-mon[34000]: mon.b@-1(synchronizing).mds e1 print_map 2024-09-21T03:30:35.523 INFO:journalctl@ceph.mon.b.smithi110.stdout: e1 2024-09-21T03:30:35.523 INFO:journalctl@ceph.mon.b.smithi110.stdout: btime 2024-09-21T03:28:56:681239+0000 2024-09-21T03:30:35.523 INFO:journalctl@ceph.mon.b.smithi110.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-09-21T03:30:35.523 INFO:journalctl@ceph.mon.b.smithi110.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-21T03:30:35.523 INFO:journalctl@ceph.mon.b.smithi110.stdout: legacy client fscid: -1 2024-09-21T03:30:35.524 INFO:journalctl@ceph.mon.b.smithi110.stdout: 2024-09-21T03:30:35.524 INFO:journalctl@ceph.mon.b.smithi110.stdout: No filesystems configured 2024-09-21T03:30:35.524 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:35 smithi110 ceph-mon[34000]: mon.b@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-09-21T03:30:35.524 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:35 smithi110 ceph-mon[34000]: mon.b@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-09-21T03:30:35.524 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:35 smithi110 ceph-mon[34000]: mon.b@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-09-21T03:30:35.524 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:35 smithi110 ceph-mon[34000]: mon.b@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-09-21T03:30:35.524 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:35 smithi110 ceph-mon[34000]: mon.b@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-09-21T03:30:35.524 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:35 smithi110 ceph-mon[34000]: mon.b@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-09-21T03:30:35.524 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:35 smithi110 ceph-mon[34000]: mon.b@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-09-21T03:30:35.524 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:35 smithi110 ceph-mon[34000]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-21T03:30:35.524 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:35 smithi110 ceph-mon[34000]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-21T03:30:35.524 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:35 smithi110 ceph-mon[34000]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-21T03:30:35.524 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:35 smithi110 ceph-mon[34000]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:30:35.524 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:35 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:30:35.524 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:35 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:30:35.525 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:35 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:30:35.525 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:35 smithi110 ceph-mon[34000]: Added host smithi114 2024-09-21T03:30:35.525 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:35 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:30:35.525 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:35 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:30:35.525 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:35 smithi110 ceph-mon[34000]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:30:35.525 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:35 smithi110 ceph-mon[34000]: from='client.14186 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T03:30:35.525 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:35 smithi110 ceph-mon[34000]: from='client.? 172.21.15.89:0/1311650935' entity='client.admin' cmd={"prefix": "osd crush tunables", "profile": "default"} : dispatch 2024-09-21T03:30:35.525 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:35 smithi110 ceph-mon[34000]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:30:35.525 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:35 smithi110 ceph-mon[34000]: from='client.? 172.21.15.89:0/1311650935' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-09-21T03:30:35.525 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:35 smithi110 ceph-mon[34000]: osdmap e4: 0 total, 0 up, 0 in 2024-09-21T03:30:35.525 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:35 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:30:35.525 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:35 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:30:35.525 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:35 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:30:35.525 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:35 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:30:35.525 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:35 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd/host:smithi114", "name": "osd_memory_target"} : dispatch 2024-09-21T03:30:35.526 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:35 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:30:35.526 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:35 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:30:35.526 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:35 smithi110 ceph-mon[34000]: Updating smithi114:/etc/ceph/ceph.conf 2024-09-21T03:30:35.526 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:35 smithi110 ceph-mon[34000]: Updating smithi114:/var/lib/ceph/7c374098-77c9-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-21T03:30:35.526 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:35 smithi110 ceph-mon[34000]: Updating smithi114:/etc/ceph/ceph.client.admin.keyring 2024-09-21T03:30:35.526 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:35 smithi110 ceph-mon[34000]: Updating smithi114:/var/lib/ceph/7c374098-77c9-11ef-baf5-efdc52797490/config/ceph.client.admin.keyring 2024-09-21T03:30:35.526 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:35 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:30:35.526 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:35 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:30:35.526 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:35 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:30:35.526 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:35 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:30:35.526 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:35 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:30:35.526 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:35 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:30:35.526 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:35 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:30:35.526 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:35 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:30:35.526 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:35 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-21T03:30:35.527 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:35 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:30:35.527 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:35 smithi110 ceph-mon[34000]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:30:35.527 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:35 smithi110 ceph-mon[34000]: from='client.14190 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi089:172.21.15.89=a;smithi110:172.21.15.110=b;smithi114:172.21.15.114=c", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:30:35.527 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:35 smithi110 ceph-mon[34000]: Saving service mon spec with placement smithi089:172.21.15.89=a;smithi110:172.21.15.110=b;smithi114:172.21.15.114=c;count:3 2024-09-21T03:30:35.527 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:35 smithi110 ceph-mon[34000]: Deploying daemon mon.c on smithi114 2024-09-21T03:30:35.527 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:35 smithi110 ceph-mon[34000]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:30:35.527 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:35 smithi110 ceph-mon[34000]: from='client.? 172.21.15.114:0/3837685406' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2024-09-21T03:30:35.527 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:35 smithi110 ceph-mon[34000]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:30:35.527 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:35 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-21T03:30:35.527 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:35 smithi110 ceph-mon[34000]: mon.a calling monitor election 2024-09-21T03:30:35.527 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:35 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T03:30:35.527 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:35 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T03:30:35.527 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:35 smithi110 ceph-mon[34000]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:30:35.527 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:35 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T03:30:35.528 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:35 smithi110 ceph-mon[34000]: mon.c calling monitor election 2024-09-21T03:30:35.528 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:35 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T03:30:35.528 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:35 smithi110 ceph-mon[34000]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:30:35.528 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:35 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T03:30:35.528 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:35 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T03:30:35.528 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:35 smithi110 ceph-mon[34000]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-21T03:30:35.528 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:35 smithi110 ceph-mon[34000]: monmap epoch 2 2024-09-21T03:30:35.528 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:35 smithi110 ceph-mon[34000]: fsid 7c374098-77c9-11ef-baf5-efdc52797490 2024-09-21T03:30:35.528 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:35 smithi110 ceph-mon[34000]: last_changed 2024-09-21T03:30:26.479324+0000 2024-09-21T03:30:35.528 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:35 smithi110 ceph-mon[34000]: created 2024-09-21T03:28:53.768514+0000 2024-09-21T03:30:35.528 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:35 smithi110 ceph-mon[34000]: min_mon_release 19 (squid) 2024-09-21T03:30:35.528 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:35 smithi110 ceph-mon[34000]: election_strategy: 1 2024-09-21T03:30:35.528 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:35 smithi110 ceph-mon[34000]: 0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-09-21T03:30:35.528 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:35 smithi110 ceph-mon[34000]: 1: [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] mon.c 2024-09-21T03:30:35.529 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:35 smithi110 ceph-mon[34000]: fsmap 2024-09-21T03:30:35.529 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:35 smithi110 ceph-mon[34000]: osdmap e4: 0 total, 0 up, 0 in 2024-09-21T03:30:35.529 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:35 smithi110 ceph-mon[34000]: mgrmap e14: x(active, since 39s) 2024-09-21T03:30:35.529 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:35 smithi110 ceph-mon[34000]: overall HEALTH_OK 2024-09-21T03:30:35.529 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:35 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:30:35.529 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:35 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:30:35.529 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:35 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:30:35.529 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:35 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-21T03:30:35.529 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:35 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:30:35.529 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:35 smithi110 ceph-mon[34000]: Deploying daemon mon.b on smithi110 2024-09-21T03:30:35.529 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:35 smithi110 ceph-mon[34000]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:30:35.529 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:35 smithi110 ceph-mon[34000]: from='client.? 172.21.15.114:0/595339574' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2024-09-21T03:30:35.529 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:35 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T03:30:35.529 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:35 smithi110 ceph-mon[34000]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:30:35.529 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:35 smithi110 ceph-mon[34000]: mon.b@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-09-21T03:30:35.529 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:35 smithi110 bash[33980]: 7e32224c6f9aa4dc9fd73c8be4c596132725591097fee7942c7cfd7aeafd46a7 2024-09-21T03:30:35.530 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:35 smithi110 systemd[1]: Started Ceph mon.b for 7c374098-77c9-11ef-baf5-efdc52797490. 2024-09-21T03:30:36.531 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-21T03:30:36.531 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7c374098-77c9-11ef-baf5-efdc52797490 -- ceph mon dump -f json 2024-09-21T03:30:36.784 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/7c374098-77c9-11ef-baf5-efdc52797490/mon.c/config 2024-09-21T03:30:40.562 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:30:40 smithi089 ceph-mon[30928]: mon.c calling monitor election 2024-09-21T03:30:40.562 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:30:40 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-21T03:30:40.562 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:30:40 smithi089 ceph-mon[30928]: mon.a calling monitor election 2024-09-21T03:30:40.562 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:30:40 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T03:30:40.562 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:30:40 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T03:30:40.562 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:30:40 smithi089 ceph-mon[30928]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:30:40.562 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:30:40 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T03:30:40.562 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:30:40 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T03:30:40.562 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:30:40 smithi089 ceph-mon[30928]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:30:40.562 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:30:40 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T03:30:40.562 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:30:40 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T03:30:40.563 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:30:40 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T03:30:40.563 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:30:40 smithi089 ceph-mon[30928]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-21T03:30:40.563 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:30:40 smithi089 ceph-mon[30928]: monmap epoch 3 2024-09-21T03:30:40.563 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:30:40 smithi089 ceph-mon[30928]: fsid 7c374098-77c9-11ef-baf5-efdc52797490 2024-09-21T03:30:40.563 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:30:40 smithi089 ceph-mon[30928]: last_changed 2024-09-21T03:30:35.227618+0000 2024-09-21T03:30:40.563 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:30:40 smithi089 ceph-mon[30928]: created 2024-09-21T03:28:53.768514+0000 2024-09-21T03:30:40.563 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:30:40 smithi089 ceph-mon[30928]: min_mon_release 19 (squid) 2024-09-21T03:30:40.563 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:30:40 smithi089 ceph-mon[30928]: election_strategy: 1 2024-09-21T03:30:40.563 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:30:40 smithi089 ceph-mon[30928]: 0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-09-21T03:30:40.563 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:30:40 smithi089 ceph-mon[30928]: 1: [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] mon.c 2024-09-21T03:30:40.563 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:30:40 smithi089 ceph-mon[30928]: 2: [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] mon.b 2024-09-21T03:30:40.563 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:30:40 smithi089 ceph-mon[30928]: fsmap 2024-09-21T03:30:40.563 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:30:40 smithi089 ceph-mon[30928]: osdmap e4: 0 total, 0 up, 0 in 2024-09-21T03:30:40.563 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:30:40 smithi089 ceph-mon[30928]: mgrmap e14: x(active, since 48s) 2024-09-21T03:30:40.563 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:30:40 smithi089 ceph-mon[30928]: overall HEALTH_OK 2024-09-21T03:30:40.564 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:30:40 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:30:40.564 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:30:40 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:30:40.564 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:30:40 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:30:40.564 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:30:40 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:30:40.711 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:40 smithi114 ceph-mon[34140]: mon.c calling monitor election 2024-09-21T03:30:40.711 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:40 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-21T03:30:40.711 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:40 smithi114 ceph-mon[34140]: mon.a calling monitor election 2024-09-21T03:30:40.711 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:40 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T03:30:40.711 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:40 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T03:30:40.711 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:40 smithi114 ceph-mon[34140]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:30:40.711 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:40 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T03:30:40.712 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:40 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T03:30:40.712 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:40 smithi114 ceph-mon[34140]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:30:40.712 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:40 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T03:30:40.712 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:40 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T03:30:40.712 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:40 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T03:30:40.712 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:40 smithi114 ceph-mon[34140]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-21T03:30:40.712 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:40 smithi114 ceph-mon[34140]: monmap epoch 3 2024-09-21T03:30:40.712 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:40 smithi114 ceph-mon[34140]: fsid 7c374098-77c9-11ef-baf5-efdc52797490 2024-09-21T03:30:40.712 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:40 smithi114 ceph-mon[34140]: last_changed 2024-09-21T03:30:35.227618+0000 2024-09-21T03:30:40.712 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:40 smithi114 ceph-mon[34140]: created 2024-09-21T03:28:53.768514+0000 2024-09-21T03:30:40.712 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:40 smithi114 ceph-mon[34140]: min_mon_release 19 (squid) 2024-09-21T03:30:40.712 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:40 smithi114 ceph-mon[34140]: election_strategy: 1 2024-09-21T03:30:40.712 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:40 smithi114 ceph-mon[34140]: 0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-09-21T03:30:40.712 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:40 smithi114 ceph-mon[34140]: 1: [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] mon.c 2024-09-21T03:30:40.712 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:40 smithi114 ceph-mon[34140]: 2: [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] mon.b 2024-09-21T03:30:40.713 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:40 smithi114 ceph-mon[34140]: fsmap 2024-09-21T03:30:40.713 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:40 smithi114 ceph-mon[34140]: osdmap e4: 0 total, 0 up, 0 in 2024-09-21T03:30:40.713 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:40 smithi114 ceph-mon[34140]: mgrmap e14: x(active, since 48s) 2024-09-21T03:30:40.713 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:40 smithi114 ceph-mon[34140]: overall HEALTH_OK 2024-09-21T03:30:40.713 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:40 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:30:40.713 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:40 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:30:40.713 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:40 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:30:40.713 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:40 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:30:40.730 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-21T03:30:40.730 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":3,"fsid":"7c374098-77c9-11ef-baf5-efdc52797490","modified":"2024-09-21T03:30:35.227618Z","created":"2024-09-21T03:28:53.768514Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:3300","nonce":0},{"type":"v1","addr":"172.21.15.89:6789","nonce":0}]},"addr":"172.21.15.89:6789/0","public_addr":"172.21.15.89:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:3300","nonce":0},{"type":"v1","addr":"172.21.15.114:6789","nonce":0}]},"addr":"172.21.15.114:6789/0","public_addr":"172.21.15.114:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:3300","nonce":0},{"type":"v1","addr":"172.21.15.110:6789","nonce":0}]},"addr":"172.21.15.110:6789/0","public_addr":"172.21.15.110:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-09-21T03:30:40.731 INFO:teuthology.orchestra.run.smithi114.stderr:dumped monmap epoch 3 2024-09-21T03:30:41.561 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:30:41 smithi089 ceph-mon[30928]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:30:41.561 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:30:41 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:30:41.561 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:30:41 smithi089 ceph-mon[30928]: from='client.? 172.21.15.114:0/3495195695' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2024-09-21T03:30:41.561 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:30:41 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:30:41.561 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:30:41 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:30:41.561 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:30:41 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T03:30:41.561 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:30:41 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:30:41.561 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:30:41 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:30:41.561 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:30:41 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:30:41.561 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:30:41 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:30:41.711 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:41 smithi114 ceph-mon[34140]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:30:41.711 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:41 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:30:41.711 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:41 smithi114 ceph-mon[34140]: from='client.? 172.21.15.114:0/3495195695' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2024-09-21T03:30:41.711 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:41 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:30:41.711 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:41 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:30:41.711 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:41 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T03:30:41.711 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:41 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:30:41.711 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:41 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:30:41.711 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:41 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:30:41.711 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:41 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:30:41.837 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-09-21T03:30:41.837 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7c374098-77c9-11ef-baf5-efdc52797490 -- ceph config generate-minimal-conf 2024-09-21T03:30:42.500 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:42 smithi110 ceph-mon[34000]: mon.c calling monitor election 2024-09-21T03:30:42.500 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:42 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-21T03:30:42.501 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:42 smithi110 ceph-mon[34000]: mon.a calling monitor election 2024-09-21T03:30:42.501 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:42 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T03:30:42.501 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:42 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T03:30:42.501 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:42 smithi110 ceph-mon[34000]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:30:42.501 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:42 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T03:30:42.501 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:42 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T03:30:42.501 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:42 smithi110 ceph-mon[34000]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:30:42.501 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:42 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T03:30:42.501 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:42 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T03:30:42.501 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:42 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T03:30:42.501 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:42 smithi110 ceph-mon[34000]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-21T03:30:42.501 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:42 smithi110 ceph-mon[34000]: monmap epoch 3 2024-09-21T03:30:42.501 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:42 smithi110 ceph-mon[34000]: fsid 7c374098-77c9-11ef-baf5-efdc52797490 2024-09-21T03:30:42.502 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:42 smithi110 ceph-mon[34000]: last_changed 2024-09-21T03:30:35.227618+0000 2024-09-21T03:30:42.502 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:42 smithi110 ceph-mon[34000]: created 2024-09-21T03:28:53.768514+0000 2024-09-21T03:30:42.502 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:42 smithi110 ceph-mon[34000]: min_mon_release 19 (squid) 2024-09-21T03:30:42.502 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:42 smithi110 ceph-mon[34000]: election_strategy: 1 2024-09-21T03:30:42.502 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:42 smithi110 ceph-mon[34000]: 0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-09-21T03:30:42.502 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:42 smithi110 ceph-mon[34000]: 1: [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] mon.c 2024-09-21T03:30:42.502 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:42 smithi110 ceph-mon[34000]: 2: [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] mon.b 2024-09-21T03:30:42.502 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:42 smithi110 ceph-mon[34000]: fsmap 2024-09-21T03:30:42.502 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:42 smithi110 ceph-mon[34000]: osdmap e4: 0 total, 0 up, 0 in 2024-09-21T03:30:42.502 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:42 smithi110 ceph-mon[34000]: mgrmap e14: x(active, since 48s) 2024-09-21T03:30:42.502 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:42 smithi110 ceph-mon[34000]: overall HEALTH_OK 2024-09-21T03:30:42.502 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:42 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:30:42.502 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:42 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:30:42.502 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:42 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:30:42.503 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:42 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:30:42.503 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:42 smithi110 ceph-mon[34000]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:30:42.503 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:42 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:30:42.503 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:42 smithi110 ceph-mon[34000]: from='client.? 172.21.15.114:0/3495195695' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2024-09-21T03:30:42.503 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:42 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:30:42.503 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:42 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:30:42.503 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:42 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T03:30:42.503 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:42 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:30:42.503 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:42 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:30:42.503 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:42 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:30:42.503 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:42 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:30:42.902 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/7c374098-77c9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:30:43.045 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:30:42 smithi089 ceph-mon[30928]: mon.b calling monitor election 2024-09-21T03:30:43.045 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:30:42 smithi089 ceph-mon[30928]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:30:43.045 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:30:42 smithi089 ceph-mon[30928]: Reconfiguring mon.a (unknown last config time)... 2024-09-21T03:30:43.045 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:30:42 smithi089 ceph-mon[30928]: Reconfiguring daemon mon.a on smithi089 2024-09-21T03:30:43.046 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:30:42 smithi089 ceph-mon[30928]: mon.b calling monitor election 2024-09-21T03:30:43.046 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:30:42 smithi089 ceph-mon[30928]: mon.c calling monitor election 2024-09-21T03:30:43.046 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:30:42 smithi089 ceph-mon[30928]: mon.a calling monitor election 2024-09-21T03:30:43.046 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:30:42 smithi089 ceph-mon[30928]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-09-21T03:30:43.046 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:30:42 smithi089 ceph-mon[30928]: monmap epoch 3 2024-09-21T03:30:43.046 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:30:42 smithi089 ceph-mon[30928]: fsid 7c374098-77c9-11ef-baf5-efdc52797490 2024-09-21T03:30:43.046 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:30:42 smithi089 ceph-mon[30928]: last_changed 2024-09-21T03:30:35.227618+0000 2024-09-21T03:30:43.046 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:30:42 smithi089 ceph-mon[30928]: created 2024-09-21T03:28:53.768514+0000 2024-09-21T03:30:43.046 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:30:42 smithi089 ceph-mon[30928]: min_mon_release 19 (squid) 2024-09-21T03:30:43.046 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:30:42 smithi089 ceph-mon[30928]: election_strategy: 1 2024-09-21T03:30:43.046 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:30:42 smithi089 ceph-mon[30928]: 0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-09-21T03:30:43.046 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:30:42 smithi089 ceph-mon[30928]: 1: [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] mon.c 2024-09-21T03:30:43.046 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:30:42 smithi089 ceph-mon[30928]: 2: [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] mon.b 2024-09-21T03:30:43.046 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:30:42 smithi089 ceph-mon[30928]: fsmap 2024-09-21T03:30:43.047 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:30:42 smithi089 ceph-mon[30928]: osdmap e4: 0 total, 0 up, 0 in 2024-09-21T03:30:43.047 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:30:42 smithi089 ceph-mon[30928]: mgrmap e14: x(active, since 50s) 2024-09-21T03:30:43.047 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:30:42 smithi089 ceph-mon[30928]: overall HEALTH_OK 2024-09-21T03:30:43.211 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:42 smithi114 ceph-mon[34140]: mon.b calling monitor election 2024-09-21T03:30:43.211 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:42 smithi114 ceph-mon[34140]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:30:43.211 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:42 smithi114 ceph-mon[34140]: Reconfiguring mon.a (unknown last config time)... 2024-09-21T03:30:43.211 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:42 smithi114 ceph-mon[34140]: Reconfiguring daemon mon.a on smithi089 2024-09-21T03:30:43.211 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:42 smithi114 ceph-mon[34140]: mon.b calling monitor election 2024-09-21T03:30:43.211 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:42 smithi114 ceph-mon[34140]: mon.c calling monitor election 2024-09-21T03:30:43.211 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:42 smithi114 ceph-mon[34140]: mon.a calling monitor election 2024-09-21T03:30:43.211 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:42 smithi114 ceph-mon[34140]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-09-21T03:30:43.211 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:42 smithi114 ceph-mon[34140]: monmap epoch 3 2024-09-21T03:30:43.212 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:42 smithi114 ceph-mon[34140]: fsid 7c374098-77c9-11ef-baf5-efdc52797490 2024-09-21T03:30:43.212 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:42 smithi114 ceph-mon[34140]: last_changed 2024-09-21T03:30:35.227618+0000 2024-09-21T03:30:43.212 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:42 smithi114 ceph-mon[34140]: created 2024-09-21T03:28:53.768514+0000 2024-09-21T03:30:43.212 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:42 smithi114 ceph-mon[34140]: min_mon_release 19 (squid) 2024-09-21T03:30:43.212 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:42 smithi114 ceph-mon[34140]: election_strategy: 1 2024-09-21T03:30:43.212 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:42 smithi114 ceph-mon[34140]: 0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-09-21T03:30:43.212 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:42 smithi114 ceph-mon[34140]: 1: [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] mon.c 2024-09-21T03:30:43.212 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:42 smithi114 ceph-mon[34140]: 2: [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] mon.b 2024-09-21T03:30:43.212 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:42 smithi114 ceph-mon[34140]: fsmap 2024-09-21T03:30:43.212 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:42 smithi114 ceph-mon[34140]: osdmap e4: 0 total, 0 up, 0 in 2024-09-21T03:30:43.212 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:42 smithi114 ceph-mon[34140]: mgrmap e14: x(active, since 50s) 2024-09-21T03:30:43.212 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:42 smithi114 ceph-mon[34140]: overall HEALTH_OK 2024-09-21T03:30:43.250 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:42 smithi110 ceph-mon[34000]: mon.b calling monitor election 2024-09-21T03:30:43.250 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:42 smithi110 ceph-mon[34000]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:30:43.250 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:42 smithi110 ceph-mon[34000]: Reconfiguring mon.a (unknown last config time)... 2024-09-21T03:30:43.250 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:42 smithi110 ceph-mon[34000]: Reconfiguring daemon mon.a on smithi089 2024-09-21T03:30:43.250 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:42 smithi110 ceph-mon[34000]: mon.b calling monitor election 2024-09-21T03:30:43.250 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:42 smithi110 ceph-mon[34000]: mon.c calling monitor election 2024-09-21T03:30:43.251 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:42 smithi110 ceph-mon[34000]: mon.a calling monitor election 2024-09-21T03:30:43.251 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:42 smithi110 ceph-mon[34000]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-09-21T03:30:43.251 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:42 smithi110 ceph-mon[34000]: monmap epoch 3 2024-09-21T03:30:43.251 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:42 smithi110 ceph-mon[34000]: fsid 7c374098-77c9-11ef-baf5-efdc52797490 2024-09-21T03:30:43.251 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:42 smithi110 ceph-mon[34000]: last_changed 2024-09-21T03:30:35.227618+0000 2024-09-21T03:30:43.251 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:42 smithi110 ceph-mon[34000]: created 2024-09-21T03:28:53.768514+0000 2024-09-21T03:30:43.251 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:42 smithi110 ceph-mon[34000]: min_mon_release 19 (squid) 2024-09-21T03:30:43.251 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:42 smithi110 ceph-mon[34000]: election_strategy: 1 2024-09-21T03:30:43.251 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:42 smithi110 ceph-mon[34000]: 0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-09-21T03:30:43.251 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:42 smithi110 ceph-mon[34000]: 1: [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] mon.c 2024-09-21T03:30:43.251 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:42 smithi110 ceph-mon[34000]: 2: [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] mon.b 2024-09-21T03:30:43.251 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:42 smithi110 ceph-mon[34000]: fsmap 2024-09-21T03:30:43.251 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:42 smithi110 ceph-mon[34000]: osdmap e4: 0 total, 0 up, 0 in 2024-09-21T03:30:43.251 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:42 smithi110 ceph-mon[34000]: mgrmap e14: x(active, since 50s) 2024-09-21T03:30:43.251 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:42 smithi110 ceph-mon[34000]: overall HEALTH_OK 2024-09-21T03:30:43.568 INFO:teuthology.orchestra.run.smithi089.stdout:# minimal ceph.conf for 7c374098-77c9-11ef-baf5-efdc52797490 2024-09-21T03:30:43.569 INFO:teuthology.orchestra.run.smithi089.stdout:[global] 2024-09-21T03:30:43.569 INFO:teuthology.orchestra.run.smithi089.stdout: fsid = 7c374098-77c9-11ef-baf5-efdc52797490 2024-09-21T03:30:43.569 INFO:teuthology.orchestra.run.smithi089.stdout: mon_host = [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] 2024-09-21T03:30:44.204 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:30:43 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:30:44.204 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:30:43 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:30:44.204 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:30:43 smithi089 ceph-mon[30928]: Reconfiguring mon.b (monmap changed)... 2024-09-21T03:30:44.204 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:30:43 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-21T03:30:44.204 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:30:43 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-09-21T03:30:44.204 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:30:43 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:30:44.204 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:30:43 smithi089 ceph-mon[30928]: Reconfiguring daemon mon.b on smithi110 2024-09-21T03:30:44.204 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:30:43 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T03:30:44.204 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:30:43 smithi089 ceph-mon[30928]: from='client.? 172.21.15.89:0/400740134' entity='client.admin' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:30:44.205 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-09-21T03:30:44.206 DEBUG:teuthology.orchestra.run.smithi089:> set -ex 2024-09-21T03:30:44.206 DEBUG:teuthology.orchestra.run.smithi089:> sudo dd of=/etc/ceph/ceph.conf 2024-09-21T03:30:44.211 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:43 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:30:44.211 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:43 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:30:44.211 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:43 smithi114 ceph-mon[34140]: Reconfiguring mon.b (monmap changed)... 2024-09-21T03:30:44.211 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:43 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-21T03:30:44.211 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:43 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-09-21T03:30:44.211 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:43 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:30:44.211 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:43 smithi114 ceph-mon[34140]: Reconfiguring daemon mon.b on smithi110 2024-09-21T03:30:44.211 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:43 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T03:30:44.211 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:43 smithi114 ceph-mon[34140]: from='client.? 172.21.15.89:0/400740134' entity='client.admin' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:30:44.235 DEBUG:teuthology.orchestra.run.smithi089:> set -ex 2024-09-21T03:30:44.235 DEBUG:teuthology.orchestra.run.smithi089:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-21T03:30:44.250 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:43 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:30:44.250 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:43 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:30:44.250 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:43 smithi110 ceph-mon[34000]: Reconfiguring mon.b (monmap changed)... 2024-09-21T03:30:44.250 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:43 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-21T03:30:44.250 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:43 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-09-21T03:30:44.250 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:43 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:30:44.250 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:43 smithi110 ceph-mon[34000]: Reconfiguring daemon mon.b on smithi110 2024-09-21T03:30:44.250 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:43 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T03:30:44.250 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:43 smithi110 ceph-mon[34000]: from='client.? 172.21.15.89:0/400740134' entity='client.admin' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:30:44.301 DEBUG:teuthology.orchestra.run.smithi110:> set -ex 2024-09-21T03:30:44.301 DEBUG:teuthology.orchestra.run.smithi110:> sudo dd of=/etc/ceph/ceph.conf 2024-09-21T03:30:44.329 DEBUG:teuthology.orchestra.run.smithi110:> set -ex 2024-09-21T03:30:44.329 DEBUG:teuthology.orchestra.run.smithi110:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-21T03:30:44.395 DEBUG:teuthology.orchestra.run.smithi114:> set -ex 2024-09-21T03:30:44.395 DEBUG:teuthology.orchestra.run.smithi114:> sudo dd of=/etc/ceph/ceph.conf 2024-09-21T03:30:44.424 DEBUG:teuthology.orchestra.run.smithi114:> set -ex 2024-09-21T03:30:44.424 DEBUG:teuthology.orchestra.run.smithi114:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-21T03:30:44.490 INFO:tasks.cephadm:Adding mgr.x on smithi089 2024-09-21T03:30:44.490 INFO:tasks.cephadm:Adding mgr.y on smithi110 2024-09-21T03:30:44.490 INFO:tasks.cephadm:Adding mgr.z on smithi114 2024-09-21T03:30:44.490 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7c374098-77c9-11ef-baf5-efdc52797490 -- ceph orch apply mgr '3;smithi089=x;smithi110=y;smithi114=z' 2024-09-21T03:30:45.210 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/7c374098-77c9-11ef-baf5-efdc52797490/mon.c/config 2024-09-21T03:30:45.584 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:45 smithi114 ceph-mon[34140]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:30:45.584 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:45 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:30:45.584 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:45 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:30:45.585 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:45 smithi114 ceph-mon[34140]: Reconfiguring mon.c (monmap changed)... 2024-09-21T03:30:45.585 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:45 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-21T03:30:45.585 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:45 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-09-21T03:30:45.585 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:45 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:30:45.585 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:45 smithi114 ceph-mon[34140]: Reconfiguring daemon mon.c on smithi114 2024-09-21T03:30:45.750 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:45 smithi110 ceph-mon[34000]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:30:45.750 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:45 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:30:45.750 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:45 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:30:45.750 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:45 smithi110 ceph-mon[34000]: Reconfiguring mon.c (monmap changed)... 2024-09-21T03:30:45.750 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:45 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-21T03:30:45.750 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:45 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-09-21T03:30:45.750 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:45 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:30:45.750 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:45 smithi110 ceph-mon[34000]: Reconfiguring daemon mon.c on smithi114 2024-09-21T03:30:45.811 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:30:45 smithi089 ceph-mon[30928]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:30:45.811 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:30:45 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:30:45.811 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:30:45 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:30:45.811 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:30:45 smithi089 ceph-mon[30928]: Reconfiguring mon.c (monmap changed)... 2024-09-21T03:30:45.811 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:30:45 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-21T03:30:45.811 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:30:45 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-09-21T03:30:45.811 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:30:45 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:30:45.811 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:30:45 smithi089 ceph-mon[30928]: Reconfiguring daemon mon.c on smithi114 2024-09-21T03:30:45.891 INFO:teuthology.orchestra.run.smithi114.stdout:Scheduled mgr update... 2024-09-21T03:30:46.734 DEBUG:teuthology.orchestra.run.smithi110:mgr.y> sudo journalctl -f -n 0 -u ceph-7c374098-77c9-11ef-baf5-efdc52797490@mgr.y.service 2024-09-21T03:30:46.737 DEBUG:teuthology.orchestra.run.smithi114:mgr.z> sudo journalctl -f -n 0 -u ceph-7c374098-77c9-11ef-baf5-efdc52797490@mgr.z.service 2024-09-21T03:30:46.739 INFO:tasks.cephadm:setting config [client] client_mount_timeout = 600 2024-09-21T03:30:46.739 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7c374098-77c9-11ef-baf5-efdc52797490 -- ceph config set client client_mount_timeout 600 2024-09-21T03:30:46.742 INFO:tasks.cephadm:setting config [client] debug_client = 20 2024-09-21T03:30:46.742 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7c374098-77c9-11ef-baf5-efdc52797490 -- ceph config set client debug_client 20 2024-09-21T03:30:46.786 INFO:tasks.cephadm:setting config [client] debug_ms = 1 2024-09-21T03:30:46.786 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7c374098-77c9-11ef-baf5-efdc52797490 -- ceph config set client debug_ms 1 2024-09-21T03:30:46.829 INFO:tasks.cephadm:setting config [client] rados_mon_op_timeout = 900 2024-09-21T03:30:46.829 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7c374098-77c9-11ef-baf5-efdc52797490 -- ceph config set client rados_mon_op_timeout 900 2024-09-21T03:30:46.872 INFO:tasks.cephadm:setting config [client] rados_osd_op_timeout = 900 2024-09-21T03:30:46.872 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7c374098-77c9-11ef-baf5-efdc52797490 -- ceph config set client rados_osd_op_timeout 900 2024-09-21T03:30:46.916 INFO:tasks.cephadm:setting config [mds] debug_mds = 20 2024-09-21T03:30:46.916 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7c374098-77c9-11ef-baf5-efdc52797490 -- ceph config set mds debug_mds 20 2024-09-21T03:30:46.960 INFO:tasks.cephadm:setting config [mds] debug_mds_balancer = 20 2024-09-21T03:30:46.960 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7c374098-77c9-11ef-baf5-efdc52797490 -- ceph config set mds debug_mds_balancer 20 2024-09-21T03:30:46.961 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:46 smithi114 ceph-mon[34140]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:30:46.961 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:46 smithi114 ceph-mon[34140]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi089=x;smithi110=y;smithi114=z", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:30:46.961 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:46 smithi114 ceph-mon[34140]: Saving service mgr spec with placement smithi089=x;smithi110=y;smithi114=z;count:3 2024-09-21T03:30:46.961 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:46 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:30:46.961 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:46 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:30:46.961 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:46 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:30:46.961 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:46 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:30:46.961 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:46 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:30:46.961 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:46 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:30:46.961 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:46 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:30:46.961 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:46 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]} : dispatch 2024-09-21T03:30:46.962 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:46 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-21T03:30:46.962 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:46 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "mgr services"} : dispatch 2024-09-21T03:30:46.962 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:46 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:30:46.962 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:46 smithi114 ceph-mon[34140]: Deploying daemon mgr.z on smithi114 2024-09-21T03:30:47.003 INFO:tasks.cephadm:setting config [mds] debug_ms = 2 2024-09-21T03:30:47.003 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7c374098-77c9-11ef-baf5-efdc52797490 -- ceph config set mds debug_ms 2 2024-09-21T03:30:47.035 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:30:46 smithi089 ceph-mon[30928]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:30:47.035 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:30:46 smithi089 ceph-mon[30928]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi089=x;smithi110=y;smithi114=z", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:30:47.035 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:30:46 smithi089 ceph-mon[30928]: Saving service mgr spec with placement smithi089=x;smithi110=y;smithi114=z;count:3 2024-09-21T03:30:47.035 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:30:46 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:30:47.035 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:30:46 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:30:47.035 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:30:46 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:30:47.035 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:30:46 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:30:47.036 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:30:46 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:30:47.036 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:30:46 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:30:47.036 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:30:46 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:30:47.036 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:30:46 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]} : dispatch 2024-09-21T03:30:47.036 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:30:46 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-21T03:30:47.036 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:30:46 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "mgr services"} : dispatch 2024-09-21T03:30:47.036 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:30:46 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:30:47.036 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:30:46 smithi089 ceph-mon[30928]: Deploying daemon mgr.z on smithi114 2024-09-21T03:30:47.047 INFO:tasks.cephadm:setting config [mds] mds_bal_fragment_size_max = 10000 2024-09-21T03:30:47.047 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7c374098-77c9-11ef-baf5-efdc52797490 -- ceph config set mds mds_bal_fragment_size_max 10000 2024-09-21T03:30:47.052 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/7c374098-77c9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:30:47.054 INFO:tasks.cephadm:setting config [mds] mds_bal_merge_size = 5 2024-09-21T03:30:47.054 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7c374098-77c9-11ef-baf5-efdc52797490 -- ceph config set mds mds_bal_merge_size 5 2024-09-21T03:30:47.107 INFO:tasks.cephadm:setting config [mds] mds_bal_split_bits = 3 2024-09-21T03:30:47.108 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7c374098-77c9-11ef-baf5-efdc52797490 -- ceph config set mds mds_bal_split_bits 3 2024-09-21T03:30:47.151 INFO:tasks.cephadm:setting config [mds] mds_bal_split_size = 100 2024-09-21T03:30:47.151 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7c374098-77c9-11ef-baf5-efdc52797490 -- ceph config set mds mds_bal_split_size 100 2024-09-21T03:30:47.194 INFO:tasks.cephadm:setting config [mds] mds_debug_frag = True 2024-09-21T03:30:47.194 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7c374098-77c9-11ef-baf5-efdc52797490 -- ceph config set mds mds_debug_frag True 2024-09-21T03:30:47.238 INFO:tasks.cephadm:setting config [mds] mds_debug_scatterstat = True 2024-09-21T03:30:47.239 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7c374098-77c9-11ef-baf5-efdc52797490 -- ceph config set mds mds_debug_scatterstat True 2024-09-21T03:30:47.250 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:46 smithi110 ceph-mon[34000]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:30:47.250 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:46 smithi110 ceph-mon[34000]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi089=x;smithi110=y;smithi114=z", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:30:47.250 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:46 smithi110 ceph-mon[34000]: Saving service mgr spec with placement smithi089=x;smithi110=y;smithi114=z;count:3 2024-09-21T03:30:47.250 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:46 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:30:47.250 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:46 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:30:47.250 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:46 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:30:47.250 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:46 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:30:47.250 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:46 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:30:47.250 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:46 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:30:47.250 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:46 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:30:47.251 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:46 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]} : dispatch 2024-09-21T03:30:47.251 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:46 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-21T03:30:47.251 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:46 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "mgr services"} : dispatch 2024-09-21T03:30:47.251 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:46 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:30:47.251 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:46 smithi110 ceph-mon[34000]: Deploying daemon mgr.z on smithi114 2024-09-21T03:30:47.287 INFO:tasks.cephadm:setting config [mds] mds_op_complaint_time = 180 2024-09-21T03:30:47.287 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7c374098-77c9-11ef-baf5-efdc52797490 -- ceph config set mds mds_op_complaint_time 180 2024-09-21T03:30:47.334 INFO:tasks.cephadm:setting config [mds] mds_verify_scatter = True 2024-09-21T03:30:47.334 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7c374098-77c9-11ef-baf5-efdc52797490 -- ceph config set mds mds_verify_scatter True 2024-09-21T03:30:47.390 INFO:tasks.cephadm:setting config [mds] osd_op_complaint_time = 180 2024-09-21T03:30:47.390 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7c374098-77c9-11ef-baf5-efdc52797490 -- ceph config set mds osd_op_complaint_time 180 2024-09-21T03:30:47.435 INFO:tasks.cephadm:setting config [mds] rados_mon_op_timeout = 900 2024-09-21T03:30:47.435 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7c374098-77c9-11ef-baf5-efdc52797490 -- ceph config set mds rados_mon_op_timeout 900 2024-09-21T03:30:47.487 INFO:tasks.cephadm:setting config [mds] rados_osd_op_timeout = 900 2024-09-21T03:30:47.487 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7c374098-77c9-11ef-baf5-efdc52797490 -- ceph config set mds rados_osd_op_timeout 900 2024-09-21T03:30:47.531 INFO:tasks.cephadm:setting config [mgr] client_mount_timeout = 30 2024-09-21T03:30:47.531 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7c374098-77c9-11ef-baf5-efdc52797490 -- ceph config set mgr client_mount_timeout 30 2024-09-21T03:30:47.548 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/7c374098-77c9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:30:47.565 INFO:tasks.cephadm:setting config [mgr] debug_client = 20 2024-09-21T03:30:47.565 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7c374098-77c9-11ef-baf5-efdc52797490 -- ceph config set mgr debug_client 20 2024-09-21T03:30:47.618 INFO:tasks.cephadm:setting config [mgr] debug_mgr = 20 2024-09-21T03:30:47.618 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7c374098-77c9-11ef-baf5-efdc52797490 -- ceph config set mgr debug_mgr 20 2024-09-21T03:30:47.662 INFO:tasks.cephadm:setting config [mgr] debug_ms = 1 2024-09-21T03:30:47.662 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7c374098-77c9-11ef-baf5-efdc52797490 -- ceph config set mgr debug_ms 1 2024-09-21T03:30:47.712 INFO:tasks.cephadm:setting config [mgr] mon_warn_on_pool_no_app = False 2024-09-21T03:30:47.713 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7c374098-77c9-11ef-baf5-efdc52797490 -- ceph config set mgr mon_warn_on_pool_no_app False 2024-09-21T03:30:47.775 INFO:tasks.cephadm:setting config [mon] mon_op_complaint_time = 120 2024-09-21T03:30:47.775 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7c374098-77c9-11ef-baf5-efdc52797490 -- ceph config set mon mon_op_complaint_time 120 2024-09-21T03:30:47.831 INFO:tasks.cephadm:setting config [osd] osd_op_complaint_time = 180 2024-09-21T03:30:47.831 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7c374098-77c9-11ef-baf5-efdc52797490 -- ceph config set osd osd_op_complaint_time 180 2024-09-21T03:30:47.889 DEBUG:tasks.cephadm:set 26 configs 2024-09-21T03:30:47.889 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '7c374098-77c9-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'client', 'client_mount_timeout', '600'], check_status=True, hostname='smithi089') 2024-09-21T03:30:48.457 INFO:journalctl@ceph.mgr.z.smithi114.stdout:Sep 21 03:30:48 smithi114 systemd[1]: Starting Ceph mgr.z for 7c374098-77c9-11ef-baf5-efdc52797490... 2024-09-21T03:30:48.711 INFO:journalctl@ceph.mgr.z.smithi114.stdout:Sep 21 03:30:48 smithi114 podman[35392]: 2024-09-21 03:30:48.358501651 +0000 UTC m=+0.015903653 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T03:30:48.711 INFO:journalctl@ceph.mgr.z.smithi114.stdout:Sep 21 03:30:48 smithi114 podman[35392]: 2024-09-21 03:30:48.456463323 +0000 UTC m=+0.113865317 container create 78bce9af8cd41306077d6b35dde07ede99d5cec0b61ff16efe9837b909d0391b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-7c374098-77c9-11ef-baf5-efdc52797490-mgr-z, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_CLEAN=True, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221) 2024-09-21T03:30:48.711 INFO:journalctl@ceph.mgr.z.smithi114.stdout:Sep 21 03:30:48 smithi114 podman[35392]: 2024-09-21 03:30:48.692304988 +0000 UTC m=+0.349706988 container init 78bce9af8cd41306077d6b35dde07ede99d5cec0b61ff16efe9837b909d0391b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-7c374098-77c9-11ef-baf5-efdc52797490-mgr-z, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, ceph=True) 2024-09-21T03:30:48.711 INFO:journalctl@ceph.mgr.z.smithi114.stdout:Sep 21 03:30:48 smithi114 podman[35392]: 2024-09-21 03:30:48.694964911 +0000 UTC m=+0.352366905 container start 78bce9af8cd41306077d6b35dde07ede99d5cec0b61ff16efe9837b909d0391b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-7c374098-77c9-11ef-baf5-efdc52797490-mgr-z, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221) 2024-09-21T03:30:48.862 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '7c374098-77c9-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'client', 'debug_client', '20'], check_status=True, hostname='smithi089') 2024-09-21T03:30:49.142 INFO:journalctl@ceph.mgr.z.smithi114.stdout:Sep 21 03:30:48 smithi114 ceph-mgr[35411]: -- 172.21.15.114:0/1511053297 <== mon.1 v2:172.21.15.114:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55ae1e2690e0 con 0x55ae2300c480 2024-09-21T03:30:49.142 INFO:journalctl@ceph.mgr.z.smithi114.stdout:Sep 21 03:30:48 smithi114 bash[35392]: 78bce9af8cd41306077d6b35dde07ede99d5cec0b61ff16efe9837b909d0391b 2024-09-21T03:30:49.142 INFO:journalctl@ceph.mgr.z.smithi114.stdout:Sep 21 03:30:48 smithi114 systemd[1]: Started Ceph mgr.z for 7c374098-77c9-11ef-baf5-efdc52797490. 2024-09-21T03:30:49.183 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/7c374098-77c9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:30:49.186 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/7c374098-77c9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:30:49.187 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/7c374098-77c9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:30:49.187 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/7c374098-77c9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:30:49.191 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/7c374098-77c9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:30:49.197 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/7c374098-77c9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:30:49.202 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/7c374098-77c9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:30:49.202 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/7c374098-77c9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:30:49.208 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/7c374098-77c9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:30:49.213 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/7c374098-77c9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:30:49.213 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/7c374098-77c9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:30:49.216 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/7c374098-77c9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:30:49.364 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/7c374098-77c9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:30:49.410 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:30:49 smithi089 ceph-mon[30928]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:30:49.410 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:30:49 smithi089 ceph-mon[30928]: from='client.? ' entity='client.admin' 2024-09-21T03:30:49.410 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:30:49 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:30:49.410 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:30:49 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:30:49.410 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:30:49 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:30:49.410 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:30:49 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]} : dispatch 2024-09-21T03:30:49.410 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:30:49 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-21T03:30:49.410 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:30:49 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "mgr services"} : dispatch 2024-09-21T03:30:49.410 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:30:49 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:30:49.424 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:49 smithi110 ceph-mon[34000]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:30:49.424 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:49 smithi110 ceph-mon[34000]: from='client.? ' entity='client.admin' 2024-09-21T03:30:49.424 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:49 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:30:49.424 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:49 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:30:49.424 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:49 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:30:49.424 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:49 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]} : dispatch 2024-09-21T03:30:49.424 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:49 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-21T03:30:49.424 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:49 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "mgr services"} : dispatch 2024-09-21T03:30:49.424 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:49 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:30:49.460 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:49 smithi114 ceph-mon[34140]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:30:49.461 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:49 smithi114 ceph-mon[34140]: from='client.? ' entity='client.admin' 2024-09-21T03:30:49.461 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:49 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:30:49.461 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:49 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:30:49.461 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:49 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:30:49.461 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:49 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]} : dispatch 2024-09-21T03:30:49.461 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:49 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-21T03:30:49.461 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:49 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "mgr services"} : dispatch 2024-09-21T03:30:49.461 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:49 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:30:50.500 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:50 smithi110 ceph-mon[34000]: Deploying daemon mgr.y on smithi110 2024-09-21T03:30:50.500 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:50 smithi110 ceph-mon[34000]: from='client.? 172.21.15.89:0/878072353' entity='client.admin' 2024-09-21T03:30:50.500 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:50 smithi110 ceph-mon[34000]: from='client.? 172.21.15.89:0/1927923507' entity='client.admin' 2024-09-21T03:30:50.560 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:30:50 smithi089 ceph-mon[30928]: Deploying daemon mgr.y on smithi110 2024-09-21T03:30:50.560 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:30:50 smithi089 ceph-mon[30928]: from='client.? 172.21.15.89:0/878072353' entity='client.admin' 2024-09-21T03:30:50.560 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:30:50 smithi089 ceph-mon[30928]: from='client.? 172.21.15.89:0/1927923507' entity='client.admin' 2024-09-21T03:30:50.711 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:50 smithi114 ceph-mon[34140]: Deploying daemon mgr.y on smithi110 2024-09-21T03:30:50.711 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:50 smithi114 ceph-mon[34140]: from='client.? 172.21.15.89:0/878072353' entity='client.admin' 2024-09-21T03:30:50.711 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:50 smithi114 ceph-mon[34140]: from='client.? 172.21.15.89:0/1927923507' entity='client.admin' 2024-09-21T03:30:50.886 INFO:journalctl@ceph.mgr.y.smithi110.stdout:Sep 21 03:30:50 smithi110 systemd[1]: Starting Ceph mgr.y for 7c374098-77c9-11ef-baf5-efdc52797490... 2024-09-21T03:30:51.166 INFO:journalctl@ceph.mgr.y.smithi110.stdout:Sep 21 03:30:50 smithi110 podman[34804]: 2024-09-21 03:30:50.787792798 +0000 UTC m=+0.015521165 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T03:30:51.166 INFO:journalctl@ceph.mgr.y.smithi110.stdout:Sep 21 03:30:50 smithi110 podman[34804]: 2024-09-21 03:30:50.887496933 +0000 UTC m=+0.115225346 container create a8ac88b4b1dda616f9894d41e6bf8828602dff61d040e5b9fe38f3b9e38dbfb6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-7c374098-77c9-11ef-baf5-efdc52797490-mgr-y, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, ceph=True, GIT_CLEAN=True, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS) 2024-09-21T03:30:51.166 INFO:journalctl@ceph.mgr.y.smithi110.stdout:Sep 21 03:30:51 smithi110 podman[34804]: 2024-09-21 03:30:51.089976888 +0000 UTC m=+0.317705259 container init a8ac88b4b1dda616f9894d41e6bf8828602dff61d040e5b9fe38f3b9e38dbfb6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-7c374098-77c9-11ef-baf5-efdc52797490-mgr-y, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221) 2024-09-21T03:30:51.166 INFO:journalctl@ceph.mgr.y.smithi110.stdout:Sep 21 03:30:51 smithi110 podman[34804]: 2024-09-21 03:30:51.092762714 +0000 UTC m=+0.320491084 container start a8ac88b4b1dda616f9894d41e6bf8828602dff61d040e5b9fe38f3b9e38dbfb6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-7c374098-77c9-11ef-baf5-efdc52797490-mgr-y, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-21T03:30:51.166 INFO:journalctl@ceph.mgr.y.smithi110.stdout:Sep 21 03:30:51 smithi110 ceph-mgr[34823]: -- 172.21.15.110:0/3589640197 <== mon.2 v2:172.21.15.110:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x555c3fad70e0 con 0x555c3faf5a80 2024-09-21T03:30:51.166 INFO:journalctl@ceph.mgr.y.smithi110.stdout:Sep 21 03:30:51 smithi110 bash[34804]: a8ac88b4b1dda616f9894d41e6bf8828602dff61d040e5b9fe38f3b9e38dbfb6 2024-09-21T03:30:51.422 INFO:journalctl@ceph.mgr.y.smithi110.stdout:Sep 21 03:30:51 smithi110 systemd[1]: Started Ceph mgr.y for 7c374098-77c9-11ef-baf5-efdc52797490. 2024-09-21T03:30:51.711 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:51 smithi114 ceph-mon[34140]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:30:51.711 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:51 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:30:51.711 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:51 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:30:51.711 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:51 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:30:51.711 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:51 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:30:51.711 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:51 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:30:51.729 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:30:51 smithi089 ceph-mon[30928]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:30:51.729 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:30:51 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:30:51.729 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:30:51 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:30:51.729 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:30:51 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:30:51.729 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:30:51 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:30:51.729 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:30:51 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:30:51.750 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:51 smithi110 ceph-mon[34000]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:30:51.750 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:51 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:30:51.750 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:51 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:30:51.750 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:51 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:30:51.750 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:51 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:30:51.750 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:51 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:30:52.678 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:30:52 smithi089 ceph-mon[30928]: from='client.? 172.21.15.89:0/501388508' entity='client.admin' 2024-09-21T03:30:52.678 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:30:52 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:30:52.678 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:30:52 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:30:52.679 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:30:52 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:30:52.679 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:30:52 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:30:52.679 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:30:52 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:30:52.679 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:30:52 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:30:52.679 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:30:52 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:30:52.679 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:30:52 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:30:52.679 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:30:52 smithi089 ceph-mon[30928]: from='client.? 172.21.15.89:0/610159894' entity='client.admin' 2024-09-21T03:30:52.679 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:30:52 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]} : dispatch 2024-09-21T03:30:52.679 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:30:52 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "mgr services"} : dispatch 2024-09-21T03:30:52.679 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:30:52 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:30:52.750 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:52 smithi110 ceph-mon[34000]: from='client.? 172.21.15.89:0/501388508' entity='client.admin' 2024-09-21T03:30:52.750 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:52 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:30:52.750 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:52 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:30:52.750 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:52 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:30:52.750 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:52 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:30:52.750 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:52 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:30:52.750 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:52 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:30:52.750 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:52 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:30:52.751 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:52 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:30:52.751 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:52 smithi110 ceph-mon[34000]: from='client.? 172.21.15.89:0/610159894' entity='client.admin' 2024-09-21T03:30:52.751 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:52 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]} : dispatch 2024-09-21T03:30:52.751 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:52 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "mgr services"} : dispatch 2024-09-21T03:30:52.751 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:52 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:30:52.961 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:52 smithi114 ceph-mon[34140]: from='client.? 172.21.15.89:0/501388508' entity='client.admin' 2024-09-21T03:30:52.961 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:52 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:30:52.961 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:52 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:30:52.961 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:52 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:30:52.961 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:52 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:30:52.961 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:52 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:30:52.961 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:52 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:30:52.961 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:52 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:30:52.961 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:52 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:30:52.961 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:52 smithi114 ceph-mon[34140]: from='client.? 172.21.15.89:0/610159894' entity='client.admin' 2024-09-21T03:30:52.961 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:52 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]} : dispatch 2024-09-21T03:30:52.961 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:52 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "mgr services"} : dispatch 2024-09-21T03:30:52.961 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:52 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:30:53.561 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:30:53 smithi089 ceph-mon[30928]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:30:53.561 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:30:53 smithi089 ceph-mon[30928]: Reconfiguring mgr.x (unknown last config time)... 2024-09-21T03:30:53.561 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:30:53 smithi089 ceph-mon[30928]: Reconfiguring daemon mgr.x on smithi089 2024-09-21T03:30:53.561 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:30:53 smithi089 ceph-mon[30928]: from='client.? 172.21.15.89:0/1013266261' entity='client.admin' 2024-09-21T03:30:53.561 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:30:53 smithi089 ceph-mon[30928]: from='client.? 172.21.15.89:0/4019991123' entity='client.admin' 2024-09-21T03:30:53.750 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:53 smithi110 ceph-mon[34000]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:30:53.750 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:53 smithi110 ceph-mon[34000]: Reconfiguring mgr.x (unknown last config time)... 2024-09-21T03:30:53.750 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:53 smithi110 ceph-mon[34000]: Reconfiguring daemon mgr.x on smithi089 2024-09-21T03:30:53.750 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:53 smithi110 ceph-mon[34000]: from='client.? 172.21.15.89:0/1013266261' entity='client.admin' 2024-09-21T03:30:53.750 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:53 smithi110 ceph-mon[34000]: from='client.? 172.21.15.89:0/4019991123' entity='client.admin' 2024-09-21T03:30:53.960 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:53 smithi114 ceph-mon[34140]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:30:53.961 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:53 smithi114 ceph-mon[34140]: Reconfiguring mgr.x (unknown last config time)... 2024-09-21T03:30:53.961 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:53 smithi114 ceph-mon[34140]: Reconfiguring daemon mgr.x on smithi089 2024-09-21T03:30:53.961 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:53 smithi114 ceph-mon[34140]: from='client.? 172.21.15.89:0/1013266261' entity='client.admin' 2024-09-21T03:30:53.961 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:53 smithi114 ceph-mon[34140]: from='client.? 172.21.15.89:0/4019991123' entity='client.admin' 2024-09-21T03:30:54.852 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:30:54 smithi089 ceph-mon[30928]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:30:54.852 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:30:54 smithi089 ceph-mon[30928]: from='client.? 172.21.15.89:0/2745101280' entity='client.admin' 2024-09-21T03:30:54.852 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:30:54 smithi089 ceph-mon[30928]: Standby manager daemon z started 2024-09-21T03:30:54.852 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:30:54 smithi089 ceph-mon[30928]: from='mgr.? 172.21.15.114:0/2242649347' entity='mgr.z' cmd={"prefix": "config-key get", "key": "mgr/dashboard/z/crt"} : dispatch 2024-09-21T03:30:54.852 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:30:54 smithi089 ceph-mon[30928]: from='mgr.? 172.21.15.114:0/2242649347' entity='mgr.z' cmd={"prefix": "config-key get", "key": "mgr/dashboard/crt"} : dispatch 2024-09-21T03:30:54.852 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:30:54 smithi089 ceph-mon[30928]: from='mgr.? 172.21.15.114:0/2242649347' entity='mgr.z' cmd={"prefix": "config-key get", "key": "mgr/dashboard/z/key"} : dispatch 2024-09-21T03:30:54.852 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:30:54 smithi089 ceph-mon[30928]: from='mgr.? 172.21.15.114:0/2242649347' entity='mgr.z' cmd={"prefix": "config-key get", "key": "mgr/dashboard/key"} : dispatch 2024-09-21T03:30:54.853 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:30:54 smithi089 ceph-mon[30928]: from='client.? 172.21.15.89:0/420715937' entity='client.admin' 2024-09-21T03:30:54.961 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:54 smithi114 ceph-mon[34140]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:30:54.961 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:54 smithi114 ceph-mon[34140]: from='client.? 172.21.15.89:0/2745101280' entity='client.admin' 2024-09-21T03:30:54.961 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:54 smithi114 ceph-mon[34140]: Standby manager daemon z started 2024-09-21T03:30:54.961 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:54 smithi114 ceph-mon[34140]: from='mgr.? 172.21.15.114:0/2242649347' entity='mgr.z' cmd={"prefix": "config-key get", "key": "mgr/dashboard/z/crt"} : dispatch 2024-09-21T03:30:54.961 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:54 smithi114 ceph-mon[34140]: from='mgr.? 172.21.15.114:0/2242649347' entity='mgr.z' cmd={"prefix": "config-key get", "key": "mgr/dashboard/crt"} : dispatch 2024-09-21T03:30:54.961 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:54 smithi114 ceph-mon[34140]: from='mgr.? 172.21.15.114:0/2242649347' entity='mgr.z' cmd={"prefix": "config-key get", "key": "mgr/dashboard/z/key"} : dispatch 2024-09-21T03:30:54.961 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:54 smithi114 ceph-mon[34140]: from='mgr.? 172.21.15.114:0/2242649347' entity='mgr.z' cmd={"prefix": "config-key get", "key": "mgr/dashboard/key"} : dispatch 2024-09-21T03:30:54.961 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:54 smithi114 ceph-mon[34140]: from='client.? 172.21.15.89:0/420715937' entity='client.admin' 2024-09-21T03:30:55.000 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:54 smithi110 ceph-mon[34000]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:30:55.000 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:54 smithi110 ceph-mon[34000]: from='client.? 172.21.15.89:0/2745101280' entity='client.admin' 2024-09-21T03:30:55.000 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:54 smithi110 ceph-mon[34000]: Standby manager daemon z started 2024-09-21T03:30:55.000 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:54 smithi110 ceph-mon[34000]: from='mgr.? 172.21.15.114:0/2242649347' entity='mgr.z' cmd={"prefix": "config-key get", "key": "mgr/dashboard/z/crt"} : dispatch 2024-09-21T03:30:55.000 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:54 smithi110 ceph-mon[34000]: from='mgr.? 172.21.15.114:0/2242649347' entity='mgr.z' cmd={"prefix": "config-key get", "key": "mgr/dashboard/crt"} : dispatch 2024-09-21T03:30:55.000 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:54 smithi110 ceph-mon[34000]: from='mgr.? 172.21.15.114:0/2242649347' entity='mgr.z' cmd={"prefix": "config-key get", "key": "mgr/dashboard/z/key"} : dispatch 2024-09-21T03:30:55.000 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:54 smithi110 ceph-mon[34000]: from='mgr.? 172.21.15.114:0/2242649347' entity='mgr.z' cmd={"prefix": "config-key get", "key": "mgr/dashboard/key"} : dispatch 2024-09-21T03:30:55.000 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:54 smithi110 ceph-mon[34000]: from='client.? 172.21.15.89:0/420715937' entity='client.admin' 2024-09-21T03:30:55.073 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:30:55.070+0000 7fba2204a640 1 -- 172.21.15.89:0/2710657020 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7fba0407afd0 msgr2=0x7fba0407d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:30:55.073 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:30:55.070+0000 7fba2204a640 1 --2- 172.21.15.89:0/2710657020 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7fba0407afd0 0x7fba0407d4e0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fba0c009bd0 tx=0x7fba0c005f90 comp rx=0 tx=0).stop 2024-09-21T03:30:55.073 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:30:55.070+0000 7fba2204a640 1 -- 172.21.15.89:0/2710657020 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fba1c109470 msgr2=0x7fba1c19ad90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:30:55.073 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:30:55.070+0000 7fba2204a640 1 --2- 172.21.15.89:0/2710657020 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fba1c109470 0x7fba1c19ad90 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fba14031840 tx=0x7fba14003c20 comp rx=0 tx=0).stop 2024-09-21T03:30:55.073 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:30:55.070+0000 7fba2204a640 1 -- 172.21.15.89:0/2710657020 shutdown_connections 2024-09-21T03:30:55.073 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:30:55.070+0000 7fba2204a640 1 --2- 172.21.15.89:0/2710657020 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7fba0407afd0 0x7fba0407d4e0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:55.073 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:30:55.070+0000 7fba2204a640 1 --2- 172.21.15.89:0/2710657020 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fba1c109470 0x7fba1c19ad90 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:55.073 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:30:55.070+0000 7fba2204a640 1 --2- 172.21.15.89:0/2710657020 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fba1c102530 0x7fba1c19a1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:55.073 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:30:55.070+0000 7fba2204a640 1 --2- 172.21.15.89:0/2710657020 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fba1c101b00 0x7fba1c199b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:55.073 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:30:55.070+0000 7fba2204a640 1 -- 172.21.15.89:0/2710657020 >> 172.21.15.89:0/2710657020 conn(0x7fba1c0fd770 msgr2=0x7fba1c1077f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:30:55.073 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:30:55.071+0000 7fba2204a640 1 -- 172.21.15.89:0/2710657020 shutdown_connections 2024-09-21T03:30:55.074 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:30:55.071+0000 7fba2204a640 1 -- 172.21.15.89:0/2710657020 wait complete. 2024-09-21T03:30:55.524 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:30:55.521+0000 7f9264a6d640 1 -- 172.21.15.89:0/2705697551 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9260101b00 msgr2=0x7f9260101f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:30:55.524 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:30:55.521+0000 7f9264a6d640 1 --2- 172.21.15.89:0/2705697551 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9260101b00 0x7f9260101f10 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f9250009900 tx=0x7f925002a410 comp rx=0 tx=0).stop 2024-09-21T03:30:55.524 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:30:55.521+0000 7f9264a6d640 1 -- 172.21.15.89:0/2705697551 shutdown_connections 2024-09-21T03:30:55.524 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:30:55.521+0000 7f9264a6d640 1 --2- 172.21.15.89:0/2705697551 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9260109500 0x7f926010b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:55.524 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:30:55.521+0000 7f9264a6d640 1 --2- 172.21.15.89:0/2705697551 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9260106ba0 0x7f9260108fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:55.524 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:30:55.521+0000 7f9264a6d640 1 --2- 172.21.15.89:0/2705697551 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9260101b00 0x7f9260101f10 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:55.524 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:30:55.521+0000 7f9264a6d640 1 -- 172.21.15.89:0/2705697551 >> 172.21.15.89:0/2705697551 conn(0x7f92600fd770 msgr2=0x7f92600ffbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:30:55.524 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:30:55.521+0000 7f9264a6d640 1 -- 172.21.15.89:0/2705697551 shutdown_connections 2024-09-21T03:30:55.524 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:30:55.521+0000 7f9264a6d640 1 -- 172.21.15.89:0/2705697551 wait complete. 2024-09-21T03:30:55.526 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:30:55.522+0000 7f9264a6d640 1 Processor -- start 2024-09-21T03:30:55.526 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:30:55.522+0000 7f9264a6d640 1 -- start start 2024-09-21T03:30:55.526 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:30:55.522+0000 7f9264a6d640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9260101b00 0x7f9260199db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:55.526 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:30:55.522+0000 7f9264a6d640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9260106ba0 0x7f926019a420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:55.526 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:30:55.522+0000 7f9264a6d640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9260109500 0x7f926019afc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:55.526 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:30:55.522+0000 7f9264a6d640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f926010f120 con 0x7f9260106ba0 2024-09-21T03:30:55.526 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:30:55.522+0000 7f9264a6d640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f926010efa0 con 0x7f9260109500 2024-09-21T03:30:55.526 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:30:55.522+0000 7f9264a6d640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f926010f2a0 con 0x7f9260101b00 2024-09-21T03:30:55.526 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:30:55.523+0000 7f925dd9b640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9260106ba0 0x7f926019a420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:55.527 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:30:55.523+0000 7f925dd9b640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9260106ba0 0x7f926019a420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:49658/0 (socket says 172.21.15.89:49658) 2024-09-21T03:30:55.527 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:30:55.523+0000 7f925dd9b640 1 -- 172.21.15.89:0/1066075519 learned_addr learned my addr 172.21.15.89:0/1066075519 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-21T03:30:55.527 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:30:55.523+0000 7f925e59c640 1 --2- 172.21.15.89:0/1066075519 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9260101b00 0x7f9260199db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:55.527 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:30:55.523+0000 7f925ed9d640 1 --2- 172.21.15.89:0/1066075519 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9260109500 0x7f926019afc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:55.527 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:30:55.523+0000 7f925dd9b640 1 -- 172.21.15.89:0/1066075519 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9260101b00 msgr2=0x7f9260199db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:30:55.527 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:30:55.523+0000 7f925dd9b640 1 --2- 172.21.15.89:0/1066075519 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9260101b00 0x7f9260199db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:55.527 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:30:55.523+0000 7f925dd9b640 1 -- 172.21.15.89:0/1066075519 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9260109500 msgr2=0x7f926019afc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:30:55.527 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:30:55.523+0000 7f925dd9b640 1 --2- 172.21.15.89:0/1066075519 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9260109500 0x7f926019afc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:55.527 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:30:55.523+0000 7f925dd9b640 1 -- 172.21.15.89:0/1066075519 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f926019b7e0 con 0x7f9260106ba0 2024-09-21T03:30:55.527 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:30:55.523+0000 7f925dd9b640 1 --2- 172.21.15.89:0/1066075519 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9260106ba0 0x7f926019a420 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f925400c890 tx=0x7f925400cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:30:55.527 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:30:55.523+0000 7f924f7fe640 1 -- 172.21.15.89:0/1066075519 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9254004220 con 0x7f9260106ba0 2024-09-21T03:30:55.527 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:30:55.523+0000 7f9264a6d640 1 -- 172.21.15.89:0/1066075519 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f926019bad0 con 0x7f9260106ba0 2024-09-21T03:30:55.527 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:30:55.523+0000 7f924f7fe640 1 -- 172.21.15.89:0/1066075519 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(24 keys) ==== 1032+0+0 (secure 0 0 0) 0x7f9254007e20 con 0x7f9260106ba0 2024-09-21T03:30:55.527 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:30:55.523+0000 7f924f7fe640 1 -- 172.21.15.89:0/1066075519 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92540047a0 con 0x7f9260106ba0 2024-09-21T03:30:55.527 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:30:55.523+0000 7f9264a6d640 1 -- 172.21.15.89:0/1066075519 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f92601a21d0 con 0x7f9260106ba0 2024-09-21T03:30:55.528 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:30:55.525+0000 7f9264a6d640 1 -- 172.21.15.89:0/1066075519 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f92600ff290 con 0x7f9260106ba0 2024-09-21T03:30:55.528 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:30:55.525+0000 7f924f7fe640 1 -- 172.21.15.89:0/1066075519 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 15) ==== 103177+0+0 (secure 0 0 0) 0x7f9254004940 con 0x7f9260106ba0 2024-09-21T03:30:55.528 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:30:55.525+0000 7f924f7fe640 1 --2- 172.21.15.89:0/1066075519 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f924807afd0 0x7f924807d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:55.528 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:30:55.526+0000 7f925e59c640 1 --2- 172.21.15.89:0/1066075519 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f924807afd0 0x7f924807d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:55.528 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:30:55.526+0000 7f924f7fe640 1 -- 172.21.15.89:0/1066075519 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f925409ca00 con 0x7f9260106ba0 2024-09-21T03:30:55.529 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:30:55.526+0000 7f925e59c640 1 --2- 172.21.15.89:0/1066075519 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f924807afd0 0x7f924807d4e0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f9250005ab0 tx=0x7f9250005a20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:30:55.532 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:30:55.530+0000 7f924f7fe640 1 -- 172.21.15.89:0/1066075519 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f925409ee20 con 0x7f9260106ba0 2024-09-21T03:30:55.717 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:30:55.715+0000 7f9264a6d640 1 -- 172.21.15.89:0/1066075519 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command([{prefix=config set, name=mds_bal_fragment_size_max}] v 0) -- 0x7f9260063fc0 con 0x7f9260106ba0 2024-09-21T03:30:55.718 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:30:55 smithi089 ceph-mon[30928]: mgrmap e15: x(active, since 63s), standbys: z 2024-09-21T03:30:55.718 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:30:55 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "mgr metadata", "who": "z", "id": "z"} : dispatch 2024-09-21T03:30:55.718 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:30:55 smithi089 ceph-mon[30928]: from='client.? 172.21.15.89:0/980071785' entity='client.admin' 2024-09-21T03:30:55.718 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:30:55 smithi089 ceph-mon[30928]: from='client.? 172.21.15.89:0/2710657020' entity='client.admin' 2024-09-21T03:30:55.721 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:30:55.719+0000 7f924f7fe640 1 -- 172.21.15.89:0/1066075519 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_bal_fragment_size_max}]=0 v21)=0 v21) ==== 127+0+0 (secure 0 0 0) 0x7f9254014030 con 0x7f9260106ba0 2024-09-21T03:30:55.724 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:30:55.721+0000 7f9264a6d640 1 -- 172.21.15.89:0/1066075519 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f924807afd0 msgr2=0x7f924807d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:30:55.724 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:30:55.721+0000 7f9264a6d640 1 --2- 172.21.15.89:0/1066075519 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f924807afd0 0x7f924807d4e0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f9250005ab0 tx=0x7f9250005a20 comp rx=0 tx=0).stop 2024-09-21T03:30:55.724 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:30:55.721+0000 7f9264a6d640 1 -- 172.21.15.89:0/1066075519 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9260106ba0 msgr2=0x7f926019a420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:30:55.724 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:30:55.721+0000 7f9264a6d640 1 --2- 172.21.15.89:0/1066075519 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9260106ba0 0x7f926019a420 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f925400c890 tx=0x7f925400cd50 comp rx=0 tx=0).stop 2024-09-21T03:30:55.724 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:30:55.721+0000 7f9264a6d640 1 -- 172.21.15.89:0/1066075519 shutdown_connections 2024-09-21T03:30:55.724 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:30:55.721+0000 7f9264a6d640 1 --2- 172.21.15.89:0/1066075519 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f924807afd0 0x7f924807d4e0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:55.724 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:30:55.721+0000 7f9264a6d640 1 --2- 172.21.15.89:0/1066075519 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9260109500 0x7f926019afc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:55.724 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:30:55.721+0000 7f9264a6d640 1 --2- 172.21.15.89:0/1066075519 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9260106ba0 0x7f926019a420 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:55.724 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:30:55.721+0000 7f9264a6d640 1 --2- 172.21.15.89:0/1066075519 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9260101b00 0x7f9260199db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:55.724 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:30:55.721+0000 7f9264a6d640 1 -- 172.21.15.89:0/1066075519 >> 172.21.15.89:0/1066075519 conn(0x7f92600fd770 msgr2=0x7f9260107a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:30:55.724 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:30:55.721+0000 7f9264a6d640 1 -- 172.21.15.89:0/1066075519 shutdown_connections 2024-09-21T03:30:55.725 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:30:55.722+0000 7f9264a6d640 1 -- 172.21.15.89:0/1066075519 wait complete. 2024-09-21T03:30:55.961 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:55 smithi114 ceph-mon[34140]: mgrmap e15: x(active, since 63s), standbys: z 2024-09-21T03:30:55.961 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:55 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "mgr metadata", "who": "z", "id": "z"} : dispatch 2024-09-21T03:30:55.961 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:55 smithi114 ceph-mon[34140]: from='client.? 172.21.15.89:0/980071785' entity='client.admin' 2024-09-21T03:30:55.961 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:55 smithi114 ceph-mon[34140]: from='client.? 172.21.15.89:0/2710657020' entity='client.admin' 2024-09-21T03:30:55.993 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:30:55.991+0000 7ff66eae0640 1 -- 172.21.15.89:0/2562547704 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff668102670 msgr2=0x7ff668102ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:30:55.993 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:30:55.991+0000 7ff66eae0640 1 --2- 172.21.15.89:0/2562547704 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff668102670 0x7ff668102ae0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7ff65c009900 tx=0x7ff65c02a480 comp rx=0 tx=0).stop 2024-09-21T03:30:55.994 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:30:55.991+0000 7ff66eae0640 1 -- 172.21.15.89:0/2562547704 shutdown_connections 2024-09-21T03:30:55.994 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:30:55.991+0000 7ff66eae0640 1 --2- 172.21.15.89:0/2562547704 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff668103700 0x7ff668103b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:55.994 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:30:55.991+0000 7ff66eae0640 1 --2- 172.21.15.89:0/2562547704 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff6681040a0 0x7ff66810aa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:55.994 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:30:55.991+0000 7ff66eae0640 1 --2- 172.21.15.89:0/2562547704 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff668102670 0x7ff668102ae0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:55.994 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:30:55.991+0000 7ff66eae0640 1 -- 172.21.15.89:0/2562547704 >> 172.21.15.89:0/2562547704 conn(0x7ff6680fe000 msgr2=0x7ff668100490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:30:55.994 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:30:55.991+0000 7ff66eae0640 1 -- 172.21.15.89:0/2562547704 shutdown_connections 2024-09-21T03:30:55.994 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:30:55.991+0000 7ff66eae0640 1 -- 172.21.15.89:0/2562547704 wait complete. 2024-09-21T03:30:55.996 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:30:55.991+0000 7ff66eae0640 1 Processor -- start 2024-09-21T03:30:55.996 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:30:55.992+0000 7ff66eae0640 1 -- start start 2024-09-21T03:30:55.996 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:30:55.992+0000 7ff66eae0640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff668102670 0x7ff66806c320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:55.996 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:30:55.992+0000 7ff66eae0640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff668103700 0x7ff66819eb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:55.996 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:30:55.992+0000 7ff66eae0640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff6681040a0 0x7ff6681a10c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:55.996 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:30:55.992+0000 7ff66eae0640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7ff66810e000 con 0x7ff668102670 2024-09-21T03:30:55.996 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:30:55.992+0000 7ff66eae0640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7ff66810de80 con 0x7ff668103700 2024-09-21T03:30:55.996 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:30:55.992+0000 7ff66eae0640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7ff66810e180 con 0x7ff6681040a0 2024-09-21T03:30:55.996 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:30:55.992+0000 7ff66c87c640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff668102670 0x7ff66806c320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:55.997 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:30:55.992+0000 7ff66c87c640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff668102670 0x7ff66806c320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:49674/0 (socket says 172.21.15.89:49674) 2024-09-21T03:30:55.997 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:30:55.992+0000 7ff66c87c640 1 -- 172.21.15.89:0/3218307798 learned_addr learned my addr 172.21.15.89:0/3218307798 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-21T03:30:55.997 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:30:55.992+0000 7ff667fff640 1 --2- 172.21.15.89:0/3218307798 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff668103700 0x7ff66819eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:55.997 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:30:55.992+0000 7ff66d07d640 1 --2- 172.21.15.89:0/3218307798 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff6681040a0 0x7ff6681a10c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:55.997 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:30:55.992+0000 7ff66c87c640 1 -- 172.21.15.89:0/3218307798 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff6681040a0 msgr2=0x7ff6681a10c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:30:55.997 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:30:55.992+0000 7ff66c87c640 1 --2- 172.21.15.89:0/3218307798 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff6681040a0 0x7ff6681a10c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:55.997 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:30:55.992+0000 7ff66c87c640 1 -- 172.21.15.89:0/3218307798 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff668103700 msgr2=0x7ff66819eb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:30:55.997 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:30:55.992+0000 7ff66c87c640 1 --2- 172.21.15.89:0/3218307798 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff668103700 0x7ff66819eb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:55.997 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:30:55.992+0000 7ff66c87c640 1 -- 172.21.15.89:0/3218307798 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6681a18c0 con 0x7ff668102670 2024-09-21T03:30:55.997 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:30:55.992+0000 7ff66c87c640 1 --2- 172.21.15.89:0/3218307798 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff668102670 0x7ff66806c320 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7ff65c006010 tx=0x7ff65c003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:30:55.997 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:30:55.993+0000 7ff665ffb640 1 -- 172.21.15.89:0/3218307798 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff65c03f070 con 0x7ff668102670 2024-09-21T03:30:55.997 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:30:55.993+0000 7ff665ffb640 1 -- 172.21.15.89:0/3218307798 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(24 keys) ==== 1032+0+0 (secure 0 0 0) 0x7ff65c031cf0 con 0x7ff668102670 2024-09-21T03:30:55.997 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:30:55.993+0000 7ff66eae0640 1 -- 172.21.15.89:0/3218307798 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff6681a1b50 con 0x7ff668102670 2024-09-21T03:30:55.998 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:30:55.993+0000 7ff665ffb640 1 -- 172.21.15.89:0/3218307798 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff65c035030 con 0x7ff668102670 2024-09-21T03:30:55.998 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:30:55.993+0000 7ff66eae0640 1 -- 172.21.15.89:0/3218307798 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff6681a2080 con 0x7ff668102670 2024-09-21T03:30:55.998 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:30:55.994+0000 7ff66eae0640 1 -- 172.21.15.89:0/3218307798 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff668109850 con 0x7ff668102670 2024-09-21T03:30:55.998 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:30:55.995+0000 7ff665ffb640 1 -- 172.21.15.89:0/3218307798 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 15) ==== 103177+0+0 (secure 0 0 0) 0x7ff65c042070 con 0x7ff668102670 2024-09-21T03:30:55.998 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:30:55.995+0000 7ff665ffb640 1 --2- 172.21.15.89:0/3218307798 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7ff65807afd0 0x7ff65807d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:55.998 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:30:55.995+0000 7ff667fff640 1 --2- 172.21.15.89:0/3218307798 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7ff65807afd0 0x7ff65807d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:55.999 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:30:55.995+0000 7ff665ffb640 1 -- 172.21.15.89:0/3218307798 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7ff65c0bd610 con 0x7ff668102670 2024-09-21T03:30:55.999 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:30:55.996+0000 7ff667fff640 1 --2- 172.21.15.89:0/3218307798 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7ff65807afd0 0x7ff65807d4e0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7ff654009bd0 tx=0x7ff654005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:30:55.999 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:55 smithi110 ceph-mon[34000]: mgrmap e15: x(active, since 63s), standbys: z 2024-09-21T03:30:56.000 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:55 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "mgr metadata", "who": "z", "id": "z"} : dispatch 2024-09-21T03:30:56.000 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:55 smithi110 ceph-mon[34000]: from='client.? 172.21.15.89:0/980071785' entity='client.admin' 2024-09-21T03:30:56.000 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:55 smithi110 ceph-mon[34000]: from='client.? 172.21.15.89:0/2710657020' entity='client.admin' 2024-09-21T03:30:56.001 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:30:56.000+0000 7ff665ffb640 1 -- 172.21.15.89:0/3218307798 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7ff65c08a0f0 con 0x7ff668102670 2024-09-21T03:30:56.188 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:30:56.186+0000 7ff66eae0640 1 -- 172.21.15.89:0/3218307798 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command([{prefix=config set, name=rados_osd_op_timeout}] v 0) -- 0x7ff668063fc0 con 0x7ff668102670 2024-09-21T03:30:56.191 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:30:56.189+0000 7ff665ffb640 1 -- 172.21.15.89:0/3218307798 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{prefix=config set, name=rados_osd_op_timeout}]=0 v22)=0 v22) ==== 123+0+0 (secure 0 0 0) 0x7ff65c08efa0 con 0x7ff668102670 2024-09-21T03:30:56.192 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:30:56.190+0000 7ff665ffb640 1 -- 172.21.15.89:0/3218307798 <== mon.0 v2:172.21.15.89:3300/0 8 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff65c0851d0 con 0x7ff668102670 2024-09-21T03:30:56.194 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:30:56.192+0000 7ff66eae0640 1 -- 172.21.15.89:0/3218307798 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7ff65807afd0 msgr2=0x7ff65807d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:30:56.194 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:30:56.192+0000 7ff66eae0640 1 --2- 172.21.15.89:0/3218307798 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7ff65807afd0 0x7ff65807d4e0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7ff654009bd0 tx=0x7ff654005f90 comp rx=0 tx=0).stop 2024-09-21T03:30:56.195 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:30:56.192+0000 7ff66eae0640 1 -- 172.21.15.89:0/3218307798 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff668102670 msgr2=0x7ff66806c320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:30:56.195 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:30:56.192+0000 7ff66eae0640 1 --2- 172.21.15.89:0/3218307798 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff668102670 0x7ff66806c320 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7ff65c006010 tx=0x7ff65c003c10 comp rx=0 tx=0).stop 2024-09-21T03:30:56.195 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:30:56.192+0000 7ff66eae0640 1 -- 172.21.15.89:0/3218307798 shutdown_connections 2024-09-21T03:30:56.195 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:30:56.192+0000 7ff66eae0640 1 --2- 172.21.15.89:0/3218307798 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7ff65807afd0 0x7ff65807d4e0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:56.195 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:30:56.192+0000 7ff66eae0640 1 --2- 172.21.15.89:0/3218307798 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff668103700 0x7ff66819eb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:56.195 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:30:56.192+0000 7ff66eae0640 1 --2- 172.21.15.89:0/3218307798 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff6681040a0 0x7ff6681a10c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:56.195 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:30:56.192+0000 7ff66eae0640 1 --2- 172.21.15.89:0/3218307798 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff668102670 0x7ff66806c320 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:56.195 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:30:56.192+0000 7ff66eae0640 1 -- 172.21.15.89:0/3218307798 >> 172.21.15.89:0/3218307798 conn(0x7ff6680fe000 msgr2=0x7ff668106a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:30:56.195 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:30:56.192+0000 7ff66eae0640 1 -- 172.21.15.89:0/3218307798 shutdown_connections 2024-09-21T03:30:56.195 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:30:56.192+0000 7ff66eae0640 1 -- 172.21.15.89:0/3218307798 wait complete. 2024-09-21T03:30:56.798 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:30:56 smithi089 ceph-mon[30928]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:30:56.798 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:30:56 smithi089 ceph-mon[30928]: from='client.? 172.21.15.89:0/1066075519' entity='client.admin' 2024-09-21T03:30:56.798 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:30:56 smithi089 ceph-mon[30928]: Standby manager daemon y started 2024-09-21T03:30:56.798 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:30:56 smithi089 ceph-mon[30928]: from='mgr.? 172.21.15.110:0/1279913693' entity='mgr.y' cmd={"prefix": "config-key get", "key": "mgr/dashboard/y/crt"} : dispatch 2024-09-21T03:30:56.798 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:30:56 smithi089 ceph-mon[30928]: from='mgr.? 172.21.15.110:0/1279913693' entity='mgr.y' cmd={"prefix": "config-key get", "key": "mgr/dashboard/crt"} : dispatch 2024-09-21T03:30:56.799 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:30:56 smithi089 ceph-mon[30928]: from='mgr.? 172.21.15.110:0/1279913693' entity='mgr.y' cmd={"prefix": "config-key get", "key": "mgr/dashboard/y/key"} : dispatch 2024-09-21T03:30:56.799 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:30:56 smithi089 ceph-mon[30928]: from='mgr.? 172.21.15.110:0/1279913693' entity='mgr.y' cmd={"prefix": "config-key get", "key": "mgr/dashboard/key"} : dispatch 2024-09-21T03:30:56.799 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:30:56 smithi089 ceph-mon[30928]: from='client.? 172.21.15.89:0/3218307798' entity='client.admin' 2024-09-21T03:30:57.000 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:56 smithi110 ceph-mon[34000]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:30:57.000 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:56 smithi110 ceph-mon[34000]: from='client.? 172.21.15.89:0/1066075519' entity='client.admin' 2024-09-21T03:30:57.000 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:56 smithi110 ceph-mon[34000]: Standby manager daemon y started 2024-09-21T03:30:57.000 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:56 smithi110 ceph-mon[34000]: from='mgr.? 172.21.15.110:0/1279913693' entity='mgr.y' cmd={"prefix": "config-key get", "key": "mgr/dashboard/y/crt"} : dispatch 2024-09-21T03:30:57.000 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:56 smithi110 ceph-mon[34000]: from='mgr.? 172.21.15.110:0/1279913693' entity='mgr.y' cmd={"prefix": "config-key get", "key": "mgr/dashboard/crt"} : dispatch 2024-09-21T03:30:57.000 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:56 smithi110 ceph-mon[34000]: from='mgr.? 172.21.15.110:0/1279913693' entity='mgr.y' cmd={"prefix": "config-key get", "key": "mgr/dashboard/y/key"} : dispatch 2024-09-21T03:30:57.000 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:56 smithi110 ceph-mon[34000]: from='mgr.? 172.21.15.110:0/1279913693' entity='mgr.y' cmd={"prefix": "config-key get", "key": "mgr/dashboard/key"} : dispatch 2024-09-21T03:30:57.001 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:56 smithi110 ceph-mon[34000]: from='client.? 172.21.15.89:0/3218307798' entity='client.admin' 2024-09-21T03:30:57.211 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:56 smithi114 ceph-mon[34140]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:30:57.211 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:56 smithi114 ceph-mon[34140]: from='client.? 172.21.15.89:0/1066075519' entity='client.admin' 2024-09-21T03:30:57.211 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:56 smithi114 ceph-mon[34140]: Standby manager daemon y started 2024-09-21T03:30:57.211 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:56 smithi114 ceph-mon[34140]: from='mgr.? 172.21.15.110:0/1279913693' entity='mgr.y' cmd={"prefix": "config-key get", "key": "mgr/dashboard/y/crt"} : dispatch 2024-09-21T03:30:57.211 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:56 smithi114 ceph-mon[34140]: from='mgr.? 172.21.15.110:0/1279913693' entity='mgr.y' cmd={"prefix": "config-key get", "key": "mgr/dashboard/crt"} : dispatch 2024-09-21T03:30:57.211 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:56 smithi114 ceph-mon[34140]: from='mgr.? 172.21.15.110:0/1279913693' entity='mgr.y' cmd={"prefix": "config-key get", "key": "mgr/dashboard/y/key"} : dispatch 2024-09-21T03:30:57.211 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:56 smithi114 ceph-mon[34140]: from='mgr.? 172.21.15.110:0/1279913693' entity='mgr.y' cmd={"prefix": "config-key get", "key": "mgr/dashboard/key"} : dispatch 2024-09-21T03:30:57.211 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:56 smithi114 ceph-mon[34140]: from='client.? 172.21.15.89:0/3218307798' entity='client.admin' 2024-09-21T03:30:57.665 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '7c374098-77c9-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'client', 'debug_ms', '1'], check_status=True, hostname='smithi089') 2024-09-21T03:30:58.109 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:30:58.106+0000 7f2c5083a640 1 -- 172.21.15.89:0/1157658297 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2c4c103770 msgr2=0x7f2c4c103be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:30:58.109 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:30:58.106+0000 7f2c5083a640 1 --2- 172.21.15.89:0/1157658297 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2c4c103770 0x7f2c4c103be0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f2c3c009900 tx=0x7f2c3c02a480 comp rx=0 tx=0).stop 2024-09-21T03:30:58.109 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:30:58.106+0000 7f2c5083a640 1 -- 172.21.15.89:0/1157658297 shutdown_connections 2024-09-21T03:30:58.109 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:30:58.106+0000 7f2c5083a640 1 --2- 172.21.15.89:0/1157658297 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2c4c1051a0 0x7f2c4c10bb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:58.109 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:30:58.106+0000 7f2c5083a640 1 --2- 172.21.15.89:0/1157658297 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2c4c104800 0x7f2c4c104c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:58.109 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:30:58.106+0000 7f2c5083a640 1 --2- 172.21.15.89:0/1157658297 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2c4c103770 0x7f2c4c103be0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:58.110 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:30:58.106+0000 7f2c5083a640 1 -- 172.21.15.89:0/1157658297 >> 172.21.15.89:0/1157658297 conn(0x7f2c4c0ff0c0 msgr2=0x7f2c4c101590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:30:58.110 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:30:58.106+0000 7f2c5083a640 1 -- 172.21.15.89:0/1157658297 shutdown_connections 2024-09-21T03:30:58.110 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:30:58.106+0000 7f2c5083a640 1 -- 172.21.15.89:0/1157658297 wait complete. 2024-09-21T03:30:58.110 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:30:58.107+0000 7f2c5083a640 1 Processor -- start 2024-09-21T03:30:58.110 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:30:58.107+0000 7f2c5083a640 1 -- start start 2024-09-21T03:30:58.112 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:30:58.107+0000 7f2c5083a640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2c4c103770 0x7f2c4c199f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:58.112 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:30:58.107+0000 7f2c5083a640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2c4c104800 0x7f2c4c19a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:58.112 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:30:58.107+0000 7f2c5083a640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2c4c1051a0 0x7f2c4c19b140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:58.112 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:30:58.107+0000 7f2c5083a640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f2c4c10f2c0 con 0x7f2c4c1051a0 2024-09-21T03:30:58.112 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:30:58.107+0000 7f2c5083a640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f2c4c10f140 con 0x7f2c4c103770 2024-09-21T03:30:58.112 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:30:58.107+0000 7f2c5083a640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f2c4c10f440 con 0x7f2c4c104800 2024-09-21T03:30:58.112 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:30:58.107+0000 7f2c4ad9d640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2c4c1051a0 0x7f2c4c19b140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:58.112 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:30:58.107+0000 7f2c4a59c640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2c4c103770 0x7f2c4c199f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:58.112 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:30:58.107+0000 7f2c4ad9d640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2c4c1051a0 0x7f2c4c19b140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:49686/0 (socket says 172.21.15.89:49686) 2024-09-21T03:30:58.112 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:30:58.107+0000 7f2c4a59c640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2c4c103770 0x7f2c4c199f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.89:54722/0 (socket says 172.21.15.89:54722) 2024-09-21T03:30:58.112 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:30:58.107+0000 7f2c4ad9d640 1 -- 172.21.15.89:0/2640594229 learned_addr learned my addr 172.21.15.89:0/2640594229 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-21T03:30:58.113 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:30:58.108+0000 7f2c49d9b640 1 --2- 172.21.15.89:0/2640594229 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2c4c104800 0x7f2c4c19a5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:58.113 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:30:58.108+0000 7f2c4ad9d640 1 -- 172.21.15.89:0/2640594229 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2c4c104800 msgr2=0x7f2c4c19a5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:30:58.113 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:30:58.108+0000 7f2c4ad9d640 1 --2- 172.21.15.89:0/2640594229 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2c4c104800 0x7f2c4c19a5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:58.113 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:30:58.108+0000 7f2c4ad9d640 1 -- 172.21.15.89:0/2640594229 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2c4c103770 msgr2=0x7f2c4c199f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:30:58.113 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:30:58.108+0000 7f2c4ad9d640 1 --2- 172.21.15.89:0/2640594229 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2c4c103770 0x7f2c4c199f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:58.113 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:30:58.108+0000 7f2c4ad9d640 1 -- 172.21.15.89:0/2640594229 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c4c19b8d0 con 0x7f2c4c1051a0 2024-09-21T03:30:58.113 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:30:58.108+0000 7f2c4ad9d640 1 --2- 172.21.15.89:0/2640594229 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2c4c1051a0 0x7f2c4c19b140 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f2c4400b970 tx=0x7f2c4400bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:30:58.113 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:30:58.108+0000 7f2c437fe640 1 -- 172.21.15.89:0/2640594229 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c44019070 con 0x7f2c4c1051a0 2024-09-21T03:30:58.113 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:30:58.108+0000 7f2c437fe640 1 -- 172.21.15.89:0/2640594229 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2c44014440 con 0x7f2c4c1051a0 2024-09-21T03:30:58.113 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:30:58.108+0000 7f2c5083a640 1 -- 172.21.15.89:0/2640594229 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2c4c19bbc0 con 0x7f2c4c1051a0 2024-09-21T03:30:58.113 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:30:58.108+0000 7f2c437fe640 1 -- 172.21.15.89:0/2640594229 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c440075d0 con 0x7f2c4c1051a0 2024-09-21T03:30:58.113 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:30:58.108+0000 7f2c5083a640 1 -- 172.21.15.89:0/2640594229 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2c4c1a2200 con 0x7f2c4c1051a0 2024-09-21T03:30:58.114 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:30:58.109+0000 7f2c5083a640 1 -- 172.21.15.89:0/2640594229 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2c4c10a950 con 0x7f2c4c1051a0 2024-09-21T03:30:58.114 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:30:58.111+0000 7f2c437fe640 1 -- 172.21.15.89:0/2640594229 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 16) ==== 154456+0+0 (secure 0 0 0) 0x7f2c4401c070 con 0x7f2c4c1051a0 2024-09-21T03:30:58.114 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:30:58.111+0000 7f2c437fe640 1 --2- 172.21.15.89:0/2640594229 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f2c380b6480 0x7f2c380b8990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:58.115 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:30:58.111+0000 7f2c4a59c640 1 --2- 172.21.15.89:0/2640594229 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f2c380b6480 0x7f2c380b8990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:58.115 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:30:58.111+0000 7f2c437fe640 1 -- 172.21.15.89:0/2640594229 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f2c440e49b0 con 0x7f2c4c1051a0 2024-09-21T03:30:58.115 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:30:58.112+0000 7f2c4a59c640 1 --2- 172.21.15.89:0/2640594229 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f2c380b6480 0x7f2c380b8990 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f2c3c0057c0 tx=0x7f2c3c000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:30:58.120 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:30:58.118+0000 7f2c437fe640 1 -- 172.21.15.89:0/2640594229 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f2c440b12f0 con 0x7f2c4c1051a0 2024-09-21T03:30:58.216 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:30:58 smithi089 ceph-mon[30928]: mgrmap e16: x(active, since 65s), standbys: z, y 2024-09-21T03:30:58.216 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:30:58 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "mgr metadata", "who": "y", "id": "y"} : dispatch 2024-09-21T03:30:58.309 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:30:58.306+0000 7f2c5083a640 1 -- 172.21.15.89:0/2640594229 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command([{prefix=config set, name=mon_warn_on_pool_no_app}] v 0) -- 0x7f2c4c02e640 con 0x7f2c4c1051a0 2024-09-21T03:30:58.314 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:30:58.312+0000 7f2c437fe640 1 -- 172.21.15.89:0/2640594229 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_warn_on_pool_no_app}]=0 v23)=0 v23) ==== 125+0+0 (secure 0 0 0) 0x7f2c440b6230 con 0x7f2c4c1051a0 2024-09-21T03:30:58.317 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:30:58.314+0000 7f2c5083a640 1 -- 172.21.15.89:0/2640594229 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f2c380b6480 msgr2=0x7f2c380b8990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:30:58.317 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:30:58.314+0000 7f2c5083a640 1 --2- 172.21.15.89:0/2640594229 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f2c380b6480 0x7f2c380b8990 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f2c3c0057c0 tx=0x7f2c3c000bc0 comp rx=0 tx=0).stop 2024-09-21T03:30:58.317 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:30:58.314+0000 7f2c5083a640 1 -- 172.21.15.89:0/2640594229 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2c4c1051a0 msgr2=0x7f2c4c19b140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:30:58.317 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:30:58.314+0000 7f2c5083a640 1 --2- 172.21.15.89:0/2640594229 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2c4c1051a0 0x7f2c4c19b140 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f2c4400b970 tx=0x7f2c4400bd70 comp rx=0 tx=0).stop 2024-09-21T03:30:58.317 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:30:58.314+0000 7f2c5083a640 1 -- 172.21.15.89:0/2640594229 shutdown_connections 2024-09-21T03:30:58.317 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:30:58.314+0000 7f2c5083a640 1 --2- 172.21.15.89:0/2640594229 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f2c380b6480 0x7f2c380b8990 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:58.317 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:30:58.314+0000 7f2c5083a640 1 --2- 172.21.15.89:0/2640594229 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2c4c1051a0 0x7f2c4c19b140 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:58.317 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:30:58.314+0000 7f2c5083a640 1 --2- 172.21.15.89:0/2640594229 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2c4c104800 0x7f2c4c19a5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:58.317 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:30:58.314+0000 7f2c5083a640 1 --2- 172.21.15.89:0/2640594229 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2c4c103770 0x7f2c4c199f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:58.317 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:30:58.315+0000 7f2c5083a640 1 -- 172.21.15.89:0/2640594229 >> 172.21.15.89:0/2640594229 conn(0x7f2c4c0ff0c0 msgr2=0x7f2c4c107b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:30:58.318 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:30:58.315+0000 7f2c5083a640 1 -- 172.21.15.89:0/2640594229 shutdown_connections 2024-09-21T03:30:58.318 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:30:58.315+0000 7f2c5083a640 1 -- 172.21.15.89:0/2640594229 wait complete. 2024-09-21T03:30:58.461 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:58 smithi114 ceph-mon[34140]: mgrmap e16: x(active, since 65s), standbys: z, y 2024-09-21T03:30:58.461 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:58 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "mgr metadata", "who": "y", "id": "y"} : dispatch 2024-09-21T03:30:58.500 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:58 smithi110 ceph-mon[34000]: mgrmap e16: x(active, since 65s), standbys: z, y 2024-09-21T03:30:58.500 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:58 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "mgr metadata", "who": "y", "id": "y"} : dispatch 2024-09-21T03:30:59.561 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:30:59 smithi089 ceph-mon[30928]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:30:59.561 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:30:59 smithi089 ceph-mon[30928]: from='client.? 172.21.15.89:0/2640594229' entity='client.admin' 2024-09-21T03:30:59.710 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:59 smithi114 ceph-mon[34140]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:30:59.711 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:30:59 smithi114 ceph-mon[34140]: from='client.? 172.21.15.89:0/2640594229' entity='client.admin' 2024-09-21T03:30:59.750 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:59 smithi110 ceph-mon[34000]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:30:59.750 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:30:59 smithi110 ceph-mon[34000]: from='client.? 172.21.15.89:0/2640594229' entity='client.admin' 2024-09-21T03:30:59.977 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/7c374098-77c9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:30:59.978 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/7c374098-77c9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:30:59.981 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/7c374098-77c9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:30:59.982 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/7c374098-77c9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:30:59.984 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/7c374098-77c9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:30:59.985 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/7c374098-77c9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:31:01.189 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:01.187+0000 7efeaef93640 1 -- 172.21.15.89:0/3452224545 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efea8102640 msgr2=0x7efea8102ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:31:01.189 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:01.187+0000 7efeaef93640 1 --2- 172.21.15.89:0/3452224545 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efea8102640 0x7efea8102ab0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7efe9c009900 tx=0x7efe9c02a480 comp rx=0 tx=0).stop 2024-09-21T03:31:01.189 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:01.187+0000 7efeaef93640 1 -- 172.21.15.89:0/3452224545 shutdown_connections 2024-09-21T03:31:01.189 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:01.187+0000 7efeaef93640 1 --2- 172.21.15.89:0/3452224545 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7efea8104070 0x7efea810aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:01.190 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:01.187+0000 7efeaef93640 1 --2- 172.21.15.89:0/3452224545 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7efea81036d0 0x7efea8103ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:01.190 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:01.187+0000 7efeaef93640 1 --2- 172.21.15.89:0/3452224545 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efea8102640 0x7efea8102ab0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:01.190 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:01.187+0000 7efeaef93640 1 -- 172.21.15.89:0/3452224545 >> 172.21.15.89:0/3452224545 conn(0x7efea80fdfb0 msgr2=0x7efea8100460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:31:01.190 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:01.187+0000 7efeaef93640 1 -- 172.21.15.89:0/3452224545 shutdown_connections 2024-09-21T03:31:01.190 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:01.187+0000 7efeaef93640 1 -- 172.21.15.89:0/3452224545 wait complete. 2024-09-21T03:31:01.192 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:01.187+0000 7efeaef93640 1 Processor -- start 2024-09-21T03:31:01.192 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:01.188+0000 7efeaef93640 1 -- start start 2024-09-21T03:31:01.192 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:01.188+0000 7efeaef93640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efea8102640 0x7efea806b600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:01.192 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:01.188+0000 7efeaef93640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7efea81036d0 0x7efea81a1040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:01.192 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:01.188+0000 7efeaef93640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7efea8104070 0x7efea81a3590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:01.192 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:01.188+0000 7efeaef93640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7efea810e160 con 0x7efea8102640 2024-09-21T03:31:01.192 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:01.188+0000 7efeaef93640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7efea810dfe0 con 0x7efea81036d0 2024-09-21T03:31:01.192 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:01.188+0000 7efeaef93640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7efea810e2e0 con 0x7efea8104070 2024-09-21T03:31:01.192 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:01.188+0000 7efeacd2f640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efea8102640 0x7efea806b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:01.193 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:01.188+0000 7efeacd2f640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efea8102640 0x7efea806b600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:55070/0 (socket says 172.21.15.89:55070) 2024-09-21T03:31:01.193 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:01.188+0000 7efeacd2f640 1 -- 172.21.15.89:0/2320774956 learned_addr learned my addr 172.21.15.89:0/2320774956 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-21T03:31:01.193 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:01.188+0000 7efead530640 1 --2- 172.21.15.89:0/2320774956 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7efea8104070 0x7efea81a3590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:01.193 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:01.188+0000 7efea7fff640 1 --2- 172.21.15.89:0/2320774956 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7efea81036d0 0x7efea81a1040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:01.193 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:01.188+0000 7efeacd2f640 1 -- 172.21.15.89:0/2320774956 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7efea8104070 msgr2=0x7efea81a3590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:31:01.193 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:01.188+0000 7efeacd2f640 1 --2- 172.21.15.89:0/2320774956 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7efea8104070 0x7efea81a3590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:01.193 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:01.188+0000 7efeacd2f640 1 -- 172.21.15.89:0/2320774956 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7efea81036d0 msgr2=0x7efea81a1040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:31:01.193 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:01.188+0000 7efeacd2f640 1 --2- 172.21.15.89:0/2320774956 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7efea81036d0 0x7efea81a1040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:01.193 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:01.188+0000 7efeacd2f640 1 -- 172.21.15.89:0/2320774956 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efea81a3ba0 con 0x7efea8102640 2024-09-21T03:31:01.193 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:01.188+0000 7efeacd2f640 1 --2- 172.21.15.89:0/2320774956 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efea8102640 0x7efea806b600 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7efe9c0057c0 tx=0x7efe9c003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:31:01.193 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:01.189+0000 7efea5ffb640 1 -- 172.21.15.89:0/2320774956 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe9c03f070 con 0x7efea8102640 2024-09-21T03:31:01.193 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:01.189+0000 7efea5ffb640 1 -- 172.21.15.89:0/2320774956 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7efe9c031b90 con 0x7efea8102640 2024-09-21T03:31:01.193 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:01.189+0000 7efea5ffb640 1 -- 172.21.15.89:0/2320774956 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe9c035030 con 0x7efea8102640 2024-09-21T03:31:01.193 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:01.189+0000 7efeaef93640 1 -- 172.21.15.89:0/2320774956 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efea81a3e30 con 0x7efea8102640 2024-09-21T03:31:01.193 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:01.189+0000 7efeaef93640 1 -- 172.21.15.89:0/2320774956 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efea81a4280 con 0x7efea8102640 2024-09-21T03:31:01.194 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:01.190+0000 7efeaef93640 1 -- 172.21.15.89:0/2320774956 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efea8109820 con 0x7efea8102640 2024-09-21T03:31:01.194 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:01.192+0000 7efea5ffb640 1 -- 172.21.15.89:0/2320774956 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 16) ==== 154456+0+0 (secure 0 0 0) 0x7efe9c042070 con 0x7efea8102640 2024-09-21T03:31:01.194 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:01.192+0000 7efea5ffb640 1 --2- 172.21.15.89:0/2320774956 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7efe980b6500 0x7efe980b8a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:01.195 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:01.192+0000 7efea7fff640 1 --2- 172.21.15.89:0/2320774956 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7efe980b6500 0x7efe980b8a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:01.195 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:01.192+0000 7efea5ffb640 1 -- 172.21.15.89:0/2320774956 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7efe9c1061c0 con 0x7efea8102640 2024-09-21T03:31:01.195 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:01.193+0000 7efea7fff640 1 --2- 172.21.15.89:0/2320774956 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7efe980b6500 0x7efe980b8a10 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7efe94009b60 tx=0x7efe94005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:31:01.197 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:01.195+0000 7efea5ffb640 1 -- 172.21.15.89:0/2320774956 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7efe9c0ce720 con 0x7efea8102640 2024-09-21T03:31:01.382 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:31:01 smithi089 ceph-mon[30928]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:31:01.382 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:01.379+0000 7efeaef93640 1 -- 172.21.15.89:0/2320774956 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command([{prefix=config set, name=mds_bal_split_bits}] v 0) -- 0x7efea802e640 con 0x7efea8102640 2024-09-21T03:31:01.385 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:01.384+0000 7efea5ffb640 1 -- 172.21.15.89:0/2320774956 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_bal_split_bits}]=0 v24)=0 v24) ==== 116+0+0 (secure 0 0 0) 0x7efe9c0d2b00 con 0x7efea8102640 2024-09-21T03:31:01.388 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:01.386+0000 7efeaef93640 1 -- 172.21.15.89:0/2320774956 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7efe980b6500 msgr2=0x7efe980b8a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:31:01.388 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:01.386+0000 7efeaef93640 1 --2- 172.21.15.89:0/2320774956 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7efe980b6500 0x7efe980b8a10 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7efe94009b60 tx=0x7efe94005f90 comp rx=0 tx=0).stop 2024-09-21T03:31:01.388 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:01.386+0000 7efeaef93640 1 -- 172.21.15.89:0/2320774956 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efea8102640 msgr2=0x7efea806b600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:31:01.389 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:01.386+0000 7efeaef93640 1 --2- 172.21.15.89:0/2320774956 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efea8102640 0x7efea806b600 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7efe9c0057c0 tx=0x7efe9c003c10 comp rx=0 tx=0).stop 2024-09-21T03:31:01.389 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:01.386+0000 7efeaef93640 1 -- 172.21.15.89:0/2320774956 shutdown_connections 2024-09-21T03:31:01.389 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:01.386+0000 7efeaef93640 1 --2- 172.21.15.89:0/2320774956 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7efe980b6500 0x7efe980b8a10 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:01.389 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:01.386+0000 7efeaef93640 1 --2- 172.21.15.89:0/2320774956 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7efea81036d0 0x7efea81a1040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:01.389 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:01.386+0000 7efeaef93640 1 --2- 172.21.15.89:0/2320774956 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7efea8104070 0x7efea81a3590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:01.389 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:01.386+0000 7efeaef93640 1 --2- 172.21.15.89:0/2320774956 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efea8102640 0x7efea806b600 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:01.389 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:01.386+0000 7efeaef93640 1 -- 172.21.15.89:0/2320774956 >> 172.21.15.89:0/2320774956 conn(0x7efea80fdfb0 msgr2=0x7efea81069f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:31:01.389 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:01.386+0000 7efeaef93640 1 -- 172.21.15.89:0/2320774956 shutdown_connections 2024-09-21T03:31:01.389 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:01.386+0000 7efeaef93640 1 -- 172.21.15.89:0/2320774956 wait complete. 2024-09-21T03:31:01.711 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:31:01 smithi114 ceph-mon[34140]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:31:01.750 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:31:01 smithi110 ceph-mon[34000]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:31:02.669 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:31:02 smithi110 ceph-mon[34000]: from='client.? 172.21.15.89:0/2320774956' entity='client.admin' 2024-09-21T03:31:02.710 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:31:02 smithi114 ceph-mon[34140]: from='client.? 172.21.15.89:0/2320774956' entity='client.admin' 2024-09-21T03:31:02.720 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:31:02 smithi089 ceph-mon[30928]: from='client.? 172.21.15.89:0/2320774956' entity='client.admin' 2024-09-21T03:31:03.601 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:31:03 smithi089 ceph-mon[30928]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:31:03.710 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:31:03 smithi114 ceph-mon[34140]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:31:03.750 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:31:03 smithi110 ceph-mon[34000]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:31:03.797 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/7c374098-77c9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:31:03.976 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/7c374098-77c9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:31:04.799 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:04.794+0000 7f3d75307640 1 -- 172.21.15.89:0/3402917106 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3d70102640 msgr2=0x7f3d70102ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:31:04.799 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:04.794+0000 7f3d75307640 1 --2- 172.21.15.89:0/3402917106 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3d70102640 0x7f3d70102ab0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f3d60009900 tx=0x7f3d6002a480 comp rx=0 tx=0).stop 2024-09-21T03:31:04.799 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:04.795+0000 7f3d75307640 1 -- 172.21.15.89:0/3402917106 shutdown_connections 2024-09-21T03:31:04.799 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:04.795+0000 7f3d75307640 1 --2- 172.21.15.89:0/3402917106 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3d70104070 0x7f3d7010aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:04.799 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:04.795+0000 7f3d75307640 1 --2- 172.21.15.89:0/3402917106 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3d701036d0 0x7f3d70103ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:04.799 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:04.795+0000 7f3d75307640 1 --2- 172.21.15.89:0/3402917106 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3d70102640 0x7f3d70102ab0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:04.799 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:04.795+0000 7f3d75307640 1 -- 172.21.15.89:0/3402917106 >> 172.21.15.89:0/3402917106 conn(0x7f3d700fdfb0 msgr2=0x7f3d70100460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:31:04.799 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:04.795+0000 7f3d75307640 1 -- 172.21.15.89:0/3402917106 shutdown_connections 2024-09-21T03:31:04.800 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:04.795+0000 7f3d75307640 1 -- 172.21.15.89:0/3402917106 wait complete. 2024-09-21T03:31:04.800 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:04.795+0000 7f3d75307640 1 Processor -- start 2024-09-21T03:31:04.800 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:04.795+0000 7f3d75307640 1 -- start start 2024-09-21T03:31:04.800 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:04.796+0000 7f3d75307640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3d70102640 0x7f3d7006b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:04.800 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:04.796+0000 7f3d75307640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3d701036d0 0x7f3d7019ec40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:04.800 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:04.796+0000 7f3d75307640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3d70104070 0x7f3d701a1190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:04.800 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:04.796+0000 7f3d75307640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f3d7010e110 con 0x7f3d70104070 2024-09-21T03:31:04.800 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:04.796+0000 7f3d75307640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f3d7010df90 con 0x7f3d70102640 2024-09-21T03:31:04.800 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:04.796+0000 7f3d75307640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f3d7010e290 con 0x7f3d701036d0 2024-09-21T03:31:04.800 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:04.796+0000 7f3d6f7fe640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3d70104070 0x7f3d701a1190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:04.800 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:04.796+0000 7f3d6f7fe640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3d70104070 0x7f3d701a1190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:55108/0 (socket says 172.21.15.89:55108) 2024-09-21T03:31:04.800 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:04.796+0000 7f3d6f7fe640 1 -- 172.21.15.89:0/4279683196 learned_addr learned my addr 172.21.15.89:0/4279683196 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-21T03:31:04.800 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:04.796+0000 7f3d67fff640 1 --2- 172.21.15.89:0/4279683196 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3d701036d0 0x7f3d7019ec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:04.801 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:04.796+0000 7f3d6effd640 1 --2- 172.21.15.89:0/4279683196 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3d70102640 0x7f3d7006b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:04.801 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:04.796+0000 7f3d6f7fe640 1 -- 172.21.15.89:0/4279683196 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3d701036d0 msgr2=0x7f3d7019ec40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:31:04.801 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:04.796+0000 7f3d6f7fe640 1 --2- 172.21.15.89:0/4279683196 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3d701036d0 0x7f3d7019ec40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:04.801 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:04.796+0000 7f3d6f7fe640 1 -- 172.21.15.89:0/4279683196 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3d70102640 msgr2=0x7f3d7006b6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:31:04.801 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:04.796+0000 7f3d6f7fe640 1 --2- 172.21.15.89:0/4279683196 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3d70102640 0x7f3d7006b6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:04.801 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:04.796+0000 7f3d6f7fe640 1 -- 172.21.15.89:0/4279683196 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d701a1990 con 0x7f3d70104070 2024-09-21T03:31:04.801 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:04.796+0000 7f3d6f7fe640 1 --2- 172.21.15.89:0/4279683196 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3d70104070 0x7f3d701a1190 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f3d6800eaa0 tx=0x7f3d6800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:31:04.801 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:04.796+0000 7f3d6cff9640 1 -- 172.21.15.89:0/4279683196 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d6800cdb0 con 0x7f3d70104070 2024-09-21T03:31:04.801 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:04.796+0000 7f3d6cff9640 1 -- 172.21.15.89:0/4279683196 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3d68005020 con 0x7f3d70104070 2024-09-21T03:31:04.801 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:04.796+0000 7f3d75307640 1 -- 172.21.15.89:0/4279683196 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3d701a1c80 con 0x7f3d70104070 2024-09-21T03:31:04.801 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:04.796+0000 7f3d6cff9640 1 -- 172.21.15.89:0/4279683196 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d680116d0 con 0x7f3d70104070 2024-09-21T03:31:04.801 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:04.797+0000 7f3d75307640 1 -- 172.21.15.89:0/4279683196 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3d701a21c0 con 0x7f3d70104070 2024-09-21T03:31:04.802 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:04.798+0000 7f3d75307640 1 -- 172.21.15.89:0/4279683196 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3d70109820 con 0x7f3d70104070 2024-09-21T03:31:04.802 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:04.800+0000 7f3d6cff9640 1 -- 172.21.15.89:0/4279683196 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 16) ==== 154456+0+0 (secure 0 0 0) 0x7f3d680051d0 con 0x7f3d70104070 2024-09-21T03:31:04.803 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:04.801+0000 7f3d6cff9640 1 --2- 172.21.15.89:0/4279683196 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f3d5c0b6500 0x7f3d5c0b8a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:04.803 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:04.801+0000 7f3d6effd640 1 --2- 172.21.15.89:0/4279683196 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f3d5c0b6500 0x7f3d5c0b8a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:04.803 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:04.801+0000 7f3d6cff9640 1 -- 172.21.15.89:0/4279683196 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f3d680e5ad0 con 0x7f3d70104070 2024-09-21T03:31:04.803 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:04.801+0000 7f3d6effd640 1 --2- 172.21.15.89:0/4279683196 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f3d5c0b6500 0x7f3d5c0b8a10 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f3d60000c00 tx=0x7f3d60005720 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:31:04.806 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:04.804+0000 7f3d6cff9640 1 -- 172.21.15.89:0/4279683196 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f3d68016030 con 0x7f3d70104070 2024-09-21T03:31:04.895 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:31:04 smithi089 ceph-mon[30928]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:31:04.895 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:31:04 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:31:04.896 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:31:04 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:31:04.896 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:31:04 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:31:04.896 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:31:04 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:31:04.896 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:31:04 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:31:04.896 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:31:04 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:31:04.960 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:31:04 smithi114 ceph-mon[34140]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:31:04.960 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:31:04 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:31:04.961 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:31:04 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:31:04.961 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:31:04 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:31:04.961 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:31:04 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:31:04.961 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:31:04 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:31:04.961 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:31:04 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:31:04.988 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:04.986+0000 7f8339e41640 1 -- 172.21.15.89:0/4177960505 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8334103780 msgr2=0x7f8334103bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:31:04.988 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:04.986+0000 7f8339e41640 1 --2- 172.21.15.89:0/4177960505 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8334103780 0x7f8334103bf0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f8324009900 tx=0x7f832402a480 comp rx=0 tx=0).stop 2024-09-21T03:31:04.988 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:04.986+0000 7f8339e41640 1 -- 172.21.15.89:0/4177960505 shutdown_connections 2024-09-21T03:31:04.989 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:04.986+0000 7f8339e41640 1 --2- 172.21.15.89:0/4177960505 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8334104810 0x7f8334104c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:04.989 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:04.986+0000 7f8339e41640 1 --2- 172.21.15.89:0/4177960505 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f83341051b0 0x7f833410bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:04.989 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:04.986+0000 7f8339e41640 1 --2- 172.21.15.89:0/4177960505 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8334103780 0x7f8334103bf0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:04.989 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:04.986+0000 7f8339e41640 1 -- 172.21.15.89:0/4177960505 >> 172.21.15.89:0/4177960505 conn(0x7f83340ff0f0 msgr2=0x7f83341015a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:31:04.989 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:04.986+0000 7f8339e41640 1 -- 172.21.15.89:0/4177960505 shutdown_connections 2024-09-21T03:31:04.989 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:04.986+0000 7f8339e41640 1 -- 172.21.15.89:0/4177960505 wait complete. 2024-09-21T03:31:04.990 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:04.986+0000 7f8339e41640 1 Processor -- start 2024-09-21T03:31:04.990 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:04.987+0000 7f8339e41640 1 -- start start 2024-09-21T03:31:04.990 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:04.987+0000 7f8339e41640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8334103780 0x7f8334199ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:04.991 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:04.987+0000 7f8339e41640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8334104810 0x7f833419a540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:04.991 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:04.987+0000 7f8339e41640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f83341051b0 0x7f833419b0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:04.991 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:04.987+0000 7f8339e41640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f833410f270 con 0x7f8334103780 2024-09-21T03:31:04.991 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:04.987+0000 7f8339e41640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f833410f0f0 con 0x7f83341051b0 2024-09-21T03:31:04.991 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:04.987+0000 7f8339e41640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f833410f3f0 con 0x7f8334104810 2024-09-21T03:31:04.991 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:04.987+0000 7f83337fe640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8334103780 0x7f8334199ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:04.991 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:04.987+0000 7f83337fe640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8334103780 0x7f8334199ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:55134/0 (socket says 172.21.15.89:55134) 2024-09-21T03:31:04.991 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:04.987+0000 7f83337fe640 1 -- 172.21.15.89:0/878242595 learned_addr learned my addr 172.21.15.89:0/878242595 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-21T03:31:04.991 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:04.987+0000 7f8332ffd640 1 --2- 172.21.15.89:0/878242595 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8334104810 0x7f833419a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:04.991 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:04.987+0000 7f8333fff640 1 --2- 172.21.15.89:0/878242595 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f83341051b0 0x7f833419b0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:04.991 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:04.987+0000 7f83337fe640 1 -- 172.21.15.89:0/878242595 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8334104810 msgr2=0x7f833419a540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:31:04.991 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:04.987+0000 7f83337fe640 1 --2- 172.21.15.89:0/878242595 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8334104810 0x7f833419a540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:04.992 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:04.987+0000 7f83337fe640 1 -- 172.21.15.89:0/878242595 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f83341051b0 msgr2=0x7f833419b0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:31:04.992 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:04.987+0000 7f83337fe640 1 --2- 172.21.15.89:0/878242595 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f83341051b0 0x7f833419b0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:04.992 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:04.987+0000 7f83337fe640 1 -- 172.21.15.89:0/878242595 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f833419b900 con 0x7f8334103780 2024-09-21T03:31:04.992 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:04.987+0000 7f3d75307640 1 -- 172.21.15.89:0/4279683196 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command([{prefix=config set, name=rados_osd_op_timeout}] v 0) -- 0x7f3d7002e640 con 0x7f3d70104070 2024-09-21T03:31:04.993 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:04.988+0000 7f83337fe640 1 --2- 172.21.15.89:0/878242595 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8334103780 0x7f8334199ed0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f8324009230 tx=0x7f83240042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:31:04.993 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:04.988+0000 7f8330ff9640 1 -- 172.21.15.89:0/878242595 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f832403f030 con 0x7f8334103780 2024-09-21T03:31:04.993 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:04.988+0000 7f8330ff9640 1 -- 172.21.15.89:0/878242595 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8324003c90 con 0x7f8334103780 2024-09-21T03:31:04.993 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:04.988+0000 7f8339e41640 1 -- 172.21.15.89:0/878242595 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f833419bb90 con 0x7f8334103780 2024-09-21T03:31:04.993 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:04.988+0000 7f8330ff9640 1 -- 172.21.15.89:0/878242595 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8324035070 con 0x7f8334103780 2024-09-21T03:31:04.993 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:04.988+0000 7f8339e41640 1 -- 172.21.15.89:0/878242595 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83341a22f0 con 0x7f8334103780 2024-09-21T03:31:04.993 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:04.989+0000 7f8339e41640 1 -- 172.21.15.89:0/878242595 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8334103bf0 con 0x7f8334103780 2024-09-21T03:31:04.993 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:04.991+0000 7f8330ff9640 1 -- 172.21.15.89:0/878242595 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 16) ==== 154456+0+0 (secure 0 0 0) 0x7f8324042030 con 0x7f8334103780 2024-09-21T03:31:04.994 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:04.992+0000 7f8330ff9640 1 --2- 172.21.15.89:0/878242595 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f83200b6500 0x7f83200b8a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:04.994 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:04.992+0000 7f8332ffd640 1 --2- 172.21.15.89:0/878242595 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f83200b6500 0x7f83200b8a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:04.994 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:04.992+0000 7f8330ff9640 1 -- 172.21.15.89:0/878242595 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f83241060c0 con 0x7f8334103780 2024-09-21T03:31:04.994 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:04.992+0000 7f3d6cff9640 1 -- 172.21.15.89:0/4279683196 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{prefix=config set, name=rados_osd_op_timeout}]=0 v25)=0 v25) ==== 120+0+0 (secure 0 0 0) 0x7f3d680ae0b0 con 0x7f3d70104070 2024-09-21T03:31:04.995 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:04.992+0000 7f8332ffd640 1 --2- 172.21.15.89:0/878242595 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f83200b6500 0x7f83200b8a10 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f831c00bbe0 tx=0x7f831c00b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:31:04.997 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:04.994+0000 7f3d75307640 1 -- 172.21.15.89:0/4279683196 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f3d5c0b6500 msgr2=0x7f3d5c0b8a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:31:04.997 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:04.994+0000 7f3d75307640 1 --2- 172.21.15.89:0/4279683196 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f3d5c0b6500 0x7f3d5c0b8a10 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f3d60000c00 tx=0x7f3d60005720 comp rx=0 tx=0).stop 2024-09-21T03:31:04.997 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:04.994+0000 7f3d75307640 1 -- 172.21.15.89:0/4279683196 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3d70104070 msgr2=0x7f3d701a1190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:31:04.997 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:04.994+0000 7f3d75307640 1 --2- 172.21.15.89:0/4279683196 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3d70104070 0x7f3d701a1190 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f3d6800eaa0 tx=0x7f3d6800ef60 comp rx=0 tx=0).stop 2024-09-21T03:31:04.997 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:04.995+0000 7f3d75307640 1 -- 172.21.15.89:0/4279683196 shutdown_connections 2024-09-21T03:31:04.997 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:04.995+0000 7f3d75307640 1 --2- 172.21.15.89:0/4279683196 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f3d5c0b6500 0x7f3d5c0b8a10 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:04.997 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:04.995+0000 7f3d75307640 1 --2- 172.21.15.89:0/4279683196 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3d70104070 0x7f3d701a1190 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:04.997 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:04.995+0000 7f3d75307640 1 --2- 172.21.15.89:0/4279683196 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3d701036d0 0x7f3d7019ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:04.997 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:04.995+0000 7f3d75307640 1 --2- 172.21.15.89:0/4279683196 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3d70102640 0x7f3d7006b6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:04.997 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:04.995+0000 7f3d75307640 1 -- 172.21.15.89:0/4279683196 >> 172.21.15.89:0/4279683196 conn(0x7f3d700fdfb0 msgr2=0x7f3d701069f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:31:04.998 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:04.995+0000 7f3d75307640 1 -- 172.21.15.89:0/4279683196 shutdown_connections 2024-09-21T03:31:04.998 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:04.995+0000 7f3d75307640 1 -- 172.21.15.89:0/4279683196 wait complete. 2024-09-21T03:31:05.000 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:31:04 smithi110 ceph-mon[34000]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:31:05.000 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:31:04 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:31:05.000 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:31:04 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:31:05.000 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:31:04 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:31:05.000 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:31:04 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:31:05.000 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:31:04 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:31:05.000 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:31:04 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:31:05.001 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:04.999+0000 7f8330ff9640 1 -- 172.21.15.89:0/878242595 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f83240ce580 con 0x7f8334103780 2024-09-21T03:31:05.184 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:05.182+0000 7f8339e41640 1 -- 172.21.15.89:0/878242595 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command([{prefix=config set, name=mds_op_complaint_time}] v 0) -- 0x7f833410a960 con 0x7f8334103780 2024-09-21T03:31:05.188 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:05.186+0000 7f8330ff9640 1 -- 172.21.15.89:0/878242595 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_op_complaint_time}]=0 v26)=0 v26) ==== 121+0+0 (secure 0 0 0) 0x7f83240d2960 con 0x7f8334103780 2024-09-21T03:31:05.191 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:05.188+0000 7f8339e41640 1 -- 172.21.15.89:0/878242595 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f83200b6500 msgr2=0x7f83200b8a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:31:05.191 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:05.188+0000 7f8339e41640 1 --2- 172.21.15.89:0/878242595 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f83200b6500 0x7f83200b8a10 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f831c00bbe0 tx=0x7f831c00b390 comp rx=0 tx=0).stop 2024-09-21T03:31:05.191 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:05.188+0000 7f8339e41640 1 -- 172.21.15.89:0/878242595 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8334103780 msgr2=0x7f8334199ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:31:05.191 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:05.188+0000 7f8339e41640 1 --2- 172.21.15.89:0/878242595 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8334103780 0x7f8334199ed0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f8324009230 tx=0x7f83240042f0 comp rx=0 tx=0).stop 2024-09-21T03:31:05.191 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:05.188+0000 7f8339e41640 1 -- 172.21.15.89:0/878242595 shutdown_connections 2024-09-21T03:31:05.191 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:05.188+0000 7f8339e41640 1 --2- 172.21.15.89:0/878242595 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f83200b6500 0x7f83200b8a10 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:05.191 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:05.188+0000 7f8339e41640 1 --2- 172.21.15.89:0/878242595 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f83341051b0 0x7f833419b0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:05.191 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:05.188+0000 7f8339e41640 1 --2- 172.21.15.89:0/878242595 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8334104810 0x7f833419a540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:05.191 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:05.188+0000 7f8339e41640 1 --2- 172.21.15.89:0/878242595 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8334103780 0x7f8334199ed0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:05.192 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:05.188+0000 7f8339e41640 1 -- 172.21.15.89:0/878242595 >> 172.21.15.89:0/878242595 conn(0x7f83340ff0f0 msgr2=0x7f8334107b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:31:05.192 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:05.188+0000 7f8339e41640 1 -- 172.21.15.89:0/878242595 shutdown_connections 2024-09-21T03:31:05.192 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:05.189+0000 7f8339e41640 1 -- 172.21.15.89:0/878242595 wait complete. 2024-09-21T03:31:05.918 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:02.989+0000 7fde70006640 1 -- 172.21.15.89:0/3536147439 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fde68102640 msgr2=0x7fde68102ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:31:05.918 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:02.989+0000 7fde70006640 1 --2- 172.21.15.89:0/3536147439 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fde68102640 0x7fde68102ab0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fde5c009900 tx=0x7fde5c02a480 comp rx=0 tx=0).stop 2024-09-21T03:31:05.918 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:02.989+0000 7fde70006640 1 -- 172.21.15.89:0/3536147439 shutdown_connections 2024-09-21T03:31:05.918 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:02.989+0000 7fde70006640 1 --2- 172.21.15.89:0/3536147439 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fde68104070 0x7fde6810aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:05.918 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:02.989+0000 7fde70006640 1 --2- 172.21.15.89:0/3536147439 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fde681036d0 0x7fde68103ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:05.918 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:02.989+0000 7fde70006640 1 --2- 172.21.15.89:0/3536147439 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fde68102640 0x7fde68102ab0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:05.918 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:02.989+0000 7fde70006640 1 -- 172.21.15.89:0/3536147439 >> 172.21.15.89:0/3536147439 conn(0x7fde680fdfb0 msgr2=0x7fde68100460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:31:05.918 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:02.989+0000 7fde70006640 1 -- 172.21.15.89:0/3536147439 shutdown_connections 2024-09-21T03:31:05.918 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:02.989+0000 7fde70006640 1 -- 172.21.15.89:0/3536147439 wait complete. 2024-09-21T03:31:05.918 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:02.990+0000 7fde70006640 1 Processor -- start 2024-09-21T03:31:05.919 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:02.990+0000 7fde70006640 1 -- start start 2024-09-21T03:31:05.919 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:02.990+0000 7fde70006640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fde68102640 0x7fde6806b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:05.919 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:02.990+0000 7fde70006640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fde681036d0 0x7fde6819ec50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:05.919 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:02.990+0000 7fde70006640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fde68104070 0x7fde681a11a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:05.919 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:02.990+0000 7fde70006640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fde6810e130 con 0x7fde68104070 2024-09-21T03:31:05.919 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:02.990+0000 7fde70006640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fde6810dfb0 con 0x7fde68102640 2024-09-21T03:31:05.919 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:02.990+0000 7fde70006640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fde6810e2b0 con 0x7fde681036d0 2024-09-21T03:31:05.919 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:02.990+0000 7fde6e5a3640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fde68104070 0x7fde681a11a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:05.919 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:02.990+0000 7fde6dda2640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fde68102640 0x7fde6806b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:05.919 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:02.990+0000 7fde6e5a3640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fde68104070 0x7fde681a11a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:55098/0 (socket says 172.21.15.89:55098) 2024-09-21T03:31:05.919 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:02.990+0000 7fde6d5a1640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fde681036d0 0x7fde6819ec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:05.919 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:02.990+0000 7fde6e5a3640 1 -- 172.21.15.89:0/4048117178 learned_addr learned my addr 172.21.15.89:0/4048117178 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-21T03:31:05.919 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:02.990+0000 7fde6e5a3640 1 -- 172.21.15.89:0/4048117178 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fde681036d0 msgr2=0x7fde6819ec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:31:05.919 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:02.990+0000 7fde6e5a3640 1 --2- 172.21.15.89:0/4048117178 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fde681036d0 0x7fde6819ec50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:05.920 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:02.990+0000 7fde6e5a3640 1 -- 172.21.15.89:0/4048117178 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fde68102640 msgr2=0x7fde6806b6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:31:05.920 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:02.990+0000 7fde6e5a3640 1 --2- 172.21.15.89:0/4048117178 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fde68102640 0x7fde6806b6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:05.920 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:02.990+0000 7fde6e5a3640 1 -- 172.21.15.89:0/4048117178 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde681a19c0 con 0x7fde68104070 2024-09-21T03:31:05.920 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:02.990+0000 7fde6d5a1640 1 --2- 172.21.15.89:0/4048117178 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fde681036d0 0x7fde6819ec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:31:05.920 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:02.991+0000 7fde6e5a3640 1 --2- 172.21.15.89:0/4048117178 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fde68104070 0x7fde681a11a0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fde6000eb10 tx=0x7fde6000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:31:05.920 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:02.991+0000 7fde5affd640 1 -- 172.21.15.89:0/4048117178 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde60004b20 con 0x7fde68104070 2024-09-21T03:31:05.920 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:02.991+0000 7fde70006640 1 -- 172.21.15.89:0/4048117178 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fde681a1cb0 con 0x7fde68104070 2024-09-21T03:31:05.920 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:02.991+0000 7fde5affd640 1 -- 172.21.15.89:0/4048117178 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fde60004cc0 con 0x7fde68104070 2024-09-21T03:31:05.920 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:02.991+0000 7fde5affd640 1 -- 172.21.15.89:0/4048117178 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde60010690 con 0x7fde68104070 2024-09-21T03:31:05.920 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:02.991+0000 7fde70006640 1 -- 172.21.15.89:0/4048117178 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fde681a21a0 con 0x7fde68104070 2024-09-21T03:31:05.920 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:02.992+0000 7fde70006640 1 -- 172.21.15.89:0/4048117178 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fde68109820 con 0x7fde68104070 2024-09-21T03:31:05.920 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:02.994+0000 7fde5affd640 1 -- 172.21.15.89:0/4048117178 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 16) ==== 154456+0+0 (secure 0 0 0) 0x7fde60005000 con 0x7fde68104070 2024-09-21T03:31:05.920 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:02.994+0000 7fde5affd640 1 --2- 172.21.15.89:0/4048117178 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7fde540b6480 0x7fde540b8990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:05.920 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:02.994+0000 7fde6dda2640 1 --2- 172.21.15.89:0/4048117178 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7fde540b6480 0x7fde540b8990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:05.921 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:02.995+0000 7fde5affd640 1 -- 172.21.15.89:0/4048117178 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fde600e5de0 con 0x7fde68104070 2024-09-21T03:31:05.921 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:02.995+0000 7fde6dda2640 1 --2- 172.21.15.89:0/4048117178 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7fde540b6480 0x7fde540b8990 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fde5c0057c0 tx=0x7fde5c000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:31:05.921 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:02.999+0000 7fde5affd640 1 -- 172.21.15.89:0/4048117178 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fde60056710 con 0x7fde68104070 2024-09-21T03:31:05.921 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:03.181+0000 7fde70006640 1 -- 172.21.15.89:0/4048117178 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command([{prefix=config set, name=debug_ms}] v 0) -- 0x7fde6802e640 con 0x7fde68104070 2024-09-21T03:31:05.921 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:03.181+0000 7fde5affd640 1 -- 172.21.15.89:0/4048117178 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{prefix=config set, name=debug_ms}]=0 v24)=0 v24) ==== 106+0+0 (secure 0 0 0) 0x7fde600ae340 con 0x7fde68104070 2024-09-21T03:31:05.921 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:03.184+0000 7fde70006640 1 -- 172.21.15.89:0/4048117178 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7fde540b6480 msgr2=0x7fde540b8990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:31:05.921 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:03.184+0000 7fde70006640 1 --2- 172.21.15.89:0/4048117178 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7fde540b6480 0x7fde540b8990 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fde5c0057c0 tx=0x7fde5c000bc0 comp rx=0 tx=0).stop 2024-09-21T03:31:05.921 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:03.184+0000 7fde70006640 1 -- 172.21.15.89:0/4048117178 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fde68104070 msgr2=0x7fde681a11a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:31:05.921 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:03.184+0000 7fde70006640 1 --2- 172.21.15.89:0/4048117178 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fde68104070 0x7fde681a11a0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fde6000eb10 tx=0x7fde6000c2d0 comp rx=0 tx=0).stop 2024-09-21T03:31:05.921 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:03.184+0000 7fde70006640 1 -- 172.21.15.89:0/4048117178 shutdown_connections 2024-09-21T03:31:05.921 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:03.184+0000 7fde70006640 1 --2- 172.21.15.89:0/4048117178 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7fde540b6480 0x7fde540b8990 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:05.921 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:03.184+0000 7fde70006640 1 --2- 172.21.15.89:0/4048117178 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fde68104070 0x7fde681a11a0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:05.921 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:03.184+0000 7fde70006640 1 --2- 172.21.15.89:0/4048117178 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fde681036d0 0x7fde6819ec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:05.921 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:03.184+0000 7fde70006640 1 --2- 172.21.15.89:0/4048117178 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fde68102640 0x7fde6806b6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:05.921 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:03.184+0000 7fde70006640 1 -- 172.21.15.89:0/4048117178 >> 172.21.15.89:0/4048117178 conn(0x7fde680fdfb0 msgr2=0x7fde681069f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:31:05.922 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:03.184+0000 7fde70006640 1 -- 172.21.15.89:0/4048117178 shutdown_connections 2024-09-21T03:31:05.922 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:03.184+0000 7fde70006640 1 -- 172.21.15.89:0/4048117178 wait complete. 2024-09-21T03:31:06.061 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:31:05 smithi089 ceph-mon[30928]: from='client.? 172.21.15.89:0/4279683196' entity='client.admin' 2024-09-21T03:31:06.061 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:31:05 smithi089 ceph-mon[30928]: from='client.? 172.21.15.89:0/878242595' entity='client.admin' 2024-09-21T03:31:06.250 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:31:05 smithi110 ceph-mon[34000]: from='client.? 172.21.15.89:0/4279683196' entity='client.admin' 2024-09-21T03:31:06.250 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:31:05 smithi110 ceph-mon[34000]: from='client.? 172.21.15.89:0/878242595' entity='client.admin' 2024-09-21T03:31:06.460 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:31:05 smithi114 ceph-mon[34140]: from='client.? 172.21.15.89:0/4279683196' entity='client.admin' 2024-09-21T03:31:06.460 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:31:05 smithi114 ceph-mon[34140]: from='client.? 172.21.15.89:0/878242595' entity='client.admin' 2024-09-21T03:31:06.761 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '7c374098-77c9-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'client', 'rados_mon_op_timeout', '900'], check_status=True, hostname='smithi089') 2024-09-21T03:31:06.761 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '7c374098-77c9-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'client', 'rados_osd_op_timeout', '900'], check_status=True, hostname='smithi089') 2024-09-21T03:31:06.761 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '7c374098-77c9-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mds', 'debug_mds', '20'], check_status=True, hostname='smithi089') 2024-09-21T03:31:06.762 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '7c374098-77c9-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mds', 'debug_mds_balancer', '20'], check_status=True, hostname='smithi089') 2024-09-21T03:31:06.762 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '7c374098-77c9-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mds', 'debug_ms', '2'], check_status=True, hostname='smithi089') 2024-09-21T03:31:06.762 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '7c374098-77c9-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mds', 'mds_bal_fragment_size_max', '10000'], check_status=True, hostname='smithi089') 2024-09-21T03:31:07.279 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:31:07 smithi089 ceph-mon[30928]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:31:07.283 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/7c374098-77c9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:31:07.283 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/7c374098-77c9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:31:07.283 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/7c374098-77c9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:31:07.319 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '7c374098-77c9-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mds', 'mds_bal_merge_size', '5'], check_status=True, hostname='smithi089') 2024-09-21T03:31:07.319 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '7c374098-77c9-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mds', 'mds_bal_split_bits', '3'], check_status=True, hostname='smithi089') 2024-09-21T03:31:07.319 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '7c374098-77c9-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mds', 'mds_bal_split_size', '100'], check_status=True, hostname='smithi089') 2024-09-21T03:31:07.319 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '7c374098-77c9-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mds', 'mds_debug_frag', 'True'], check_status=True, hostname='smithi089') 2024-09-21T03:31:07.319 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '7c374098-77c9-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mds', 'mds_debug_scatterstat', 'True'], check_status=True, hostname='smithi089') 2024-09-21T03:31:07.319 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '7c374098-77c9-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mds', 'mds_op_complaint_time', '180'], check_status=True, hostname='smithi089') 2024-09-21T03:31:07.319 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '7c374098-77c9-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mds', 'mds_verify_scatter', 'True'], check_status=True, hostname='smithi089') 2024-09-21T03:31:07.460 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:31:07 smithi114 ceph-mon[34140]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:31:07.479 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:07.478+0000 7f39cd6e0640 1 -- 172.21.15.89:0/2120869935 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f39c8102640 msgr2=0x7f39c8102ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:31:07.480 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:07.478+0000 7f39cd6e0640 1 --2- 172.21.15.89:0/2120869935 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f39c8102640 0x7f39c8102ab0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f39b8009900 tx=0x7f39b802a480 comp rx=0 tx=0).stop 2024-09-21T03:31:07.480 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:07.478+0000 7f39cd6e0640 1 -- 172.21.15.89:0/2120869935 shutdown_connections 2024-09-21T03:31:07.480 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:07.478+0000 7f39cd6e0640 1 --2- 172.21.15.89:0/2120869935 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f39c81036d0 0x7f39c8103ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:07.480 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:07.478+0000 7f39cd6e0640 1 --2- 172.21.15.89:0/2120869935 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f39c8104070 0x7f39c810aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:07.480 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:07.478+0000 7f39cd6e0640 1 --2- 172.21.15.89:0/2120869935 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f39c8102640 0x7f39c8102ab0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:07.480 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:07.478+0000 7f39cd6e0640 1 -- 172.21.15.89:0/2120869935 >> 172.21.15.89:0/2120869935 conn(0x7f39c80fdfb0 msgr2=0x7f39c8100460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:31:07.480 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:07.478+0000 7f39cd6e0640 1 -- 172.21.15.89:0/2120869935 shutdown_connections 2024-09-21T03:31:07.480 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:07.478+0000 7f39cd6e0640 1 -- 172.21.15.89:0/2120869935 wait complete. 2024-09-21T03:31:07.480 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:07.479+0000 7f39cd6e0640 1 Processor -- start 2024-09-21T03:31:07.482 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:07.479+0000 7f39cd6e0640 1 -- start start 2024-09-21T03:31:07.482 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:07.479+0000 7f39cd6e0640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f39c8102640 0x7f39c806b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:07.483 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:07.479+0000 7f39cd6e0640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f39c81036d0 0x7f39c819ec20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:07.483 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:07.479+0000 7f39cd6e0640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f39c8104070 0x7f39c81a1170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:07.483 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:07.479+0000 7f39cd6e0640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f39c810e110 con 0x7f39c8102640 2024-09-21T03:31:07.483 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:07.479+0000 7f39cd6e0640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f39c810df90 con 0x7f39c81036d0 2024-09-21T03:31:07.483 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:07.479+0000 7f39cd6e0640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f39c810e290 con 0x7f39c8104070 2024-09-21T03:31:07.483 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:07.479+0000 7f39c6ffd640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f39c8102640 0x7f39c806b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:07.483 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:07.479+0000 7f39c6ffd640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f39c8102640 0x7f39c806b6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:55158/0 (socket says 172.21.15.89:55158) 2024-09-21T03:31:07.483 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:07.479+0000 7f39c6ffd640 1 -- 172.21.15.89:0/2176219034 learned_addr learned my addr 172.21.15.89:0/2176219034 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-21T03:31:07.483 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:07.479+0000 7f39c67fc640 1 --2- 172.21.15.89:0/2176219034 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f39c81036d0 0x7f39c819ec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:07.483 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:07.479+0000 7f39c77fe640 1 --2- 172.21.15.89:0/2176219034 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f39c8104070 0x7f39c81a1170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:07.483 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:07.479+0000 7f39c6ffd640 1 -- 172.21.15.89:0/2176219034 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f39c8104070 msgr2=0x7f39c81a1170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:31:07.483 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:07.479+0000 7f39c6ffd640 1 --2- 172.21.15.89:0/2176219034 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f39c8104070 0x7f39c81a1170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:07.483 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:07.479+0000 7f39c6ffd640 1 -- 172.21.15.89:0/2176219034 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f39c81036d0 msgr2=0x7f39c819ec20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:31:07.483 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:07.479+0000 7f39c6ffd640 1 --2- 172.21.15.89:0/2176219034 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f39c81036d0 0x7f39c819ec20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:07.484 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:07.480+0000 7f39c6ffd640 1 -- 172.21.15.89:0/2176219034 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39c81a1970 con 0x7f39c8102640 2024-09-21T03:31:07.484 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:07.480+0000 7f39c77fe640 1 --2- 172.21.15.89:0/2176219034 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f39c8104070 0x7f39c81a1170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:31:07.484 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:07.480+0000 7f39c6ffd640 1 --2- 172.21.15.89:0/2176219034 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f39c8102640 0x7f39c806b6a0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f39b8009230 tx=0x7f39b80042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:31:07.484 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:07.480+0000 7f39affff640 1 -- 172.21.15.89:0/2176219034 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39b803f030 con 0x7f39c8102640 2024-09-21T03:31:07.484 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:07.480+0000 7f39affff640 1 -- 172.21.15.89:0/2176219034 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f39b8003c90 con 0x7f39c8102640 2024-09-21T03:31:07.484 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:07.480+0000 7f39cd6e0640 1 -- 172.21.15.89:0/2176219034 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f39c81a1c00 con 0x7f39c8102640 2024-09-21T03:31:07.484 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:07.480+0000 7f39affff640 1 -- 172.21.15.89:0/2176219034 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39b8035070 con 0x7f39c8102640 2024-09-21T03:31:07.484 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:07.480+0000 7f39cd6e0640 1 -- 172.21.15.89:0/2176219034 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39c81a2130 con 0x7f39c8102640 2024-09-21T03:31:07.484 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:07.481+0000 7f39cd6e0640 1 -- 172.21.15.89:0/2176219034 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f39c8109820 con 0x7f39c8102640 2024-09-21T03:31:07.485 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:07.482+0000 7f39affff640 1 -- 172.21.15.89:0/2176219034 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 16) ==== 154456+0+0 (secure 0 0 0) 0x7f39b8042030 con 0x7f39c8102640 2024-09-21T03:31:07.485 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:07.483+0000 7f39affff640 1 --2- 172.21.15.89:0/2176219034 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f39b00b6500 0x7f39b00b8a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:07.485 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:07.483+0000 7f39c67fc640 1 --2- 172.21.15.89:0/2176219034 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f39b00b6500 0x7f39b00b8a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:07.485 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:07.483+0000 7f39affff640 1 -- 172.21.15.89:0/2176219034 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f39b8106020 con 0x7f39c8102640 2024-09-21T03:31:07.485 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:07.484+0000 7f39c67fc640 1 --2- 172.21.15.89:0/2176219034 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f39b00b6500 0x7f39b00b8a10 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f39bc00bbe0 tx=0x7f39bc00b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:31:07.487 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:07.486+0000 7f39affff640 1 -- 172.21.15.89:0/2176219034 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f39b80ce580 con 0x7f39c8102640 2024-09-21T03:31:07.500 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:31:07 smithi110 ceph-mon[34000]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:31:07.675 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:07.674+0000 7f39cd6e0640 1 -- 172.21.15.89:0/2176219034 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command([{prefix=config set, name=debug_mgr}] v 0) -- 0x7f39c802e640 con 0x7f39c8102640 2024-09-21T03:31:07.676 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:07.675+0000 7f39affff640 1 -- 172.21.15.89:0/2176219034 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{prefix=config set, name=debug_mgr}]=0 v26)=0 v26) ==== 108+0+0 (secure 0 0 0) 0x7f39b80d2960 con 0x7f39c8102640 2024-09-21T03:31:07.678 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:07.677+0000 7f39cd6e0640 1 -- 172.21.15.89:0/2176219034 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f39b00b6500 msgr2=0x7f39b00b8a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:31:07.679 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:07.677+0000 7f39cd6e0640 1 --2- 172.21.15.89:0/2176219034 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f39b00b6500 0x7f39b00b8a10 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f39bc00bbe0 tx=0x7f39bc00b390 comp rx=0 tx=0).stop 2024-09-21T03:31:07.679 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:07.677+0000 7f39cd6e0640 1 -- 172.21.15.89:0/2176219034 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f39c8102640 msgr2=0x7f39c806b6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:31:07.679 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:07.677+0000 7f39cd6e0640 1 --2- 172.21.15.89:0/2176219034 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f39c8102640 0x7f39c806b6a0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f39b8009230 tx=0x7f39b80042f0 comp rx=0 tx=0).stop 2024-09-21T03:31:07.679 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:07.677+0000 7f39cd6e0640 1 -- 172.21.15.89:0/2176219034 shutdown_connections 2024-09-21T03:31:07.679 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:07.677+0000 7f39cd6e0640 1 --2- 172.21.15.89:0/2176219034 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f39b00b6500 0x7f39b00b8a10 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:07.679 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:07.677+0000 7f39cd6e0640 1 --2- 172.21.15.89:0/2176219034 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f39c81036d0 0x7f39c819ec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:07.679 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:07.677+0000 7f39cd6e0640 1 --2- 172.21.15.89:0/2176219034 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f39c8104070 0x7f39c81a1170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:07.679 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:07.677+0000 7f39cd6e0640 1 --2- 172.21.15.89:0/2176219034 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f39c8102640 0x7f39c806b6a0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:07.679 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:07.677+0000 7f39cd6e0640 1 -- 172.21.15.89:0/2176219034 >> 172.21.15.89:0/2176219034 conn(0x7f39c80fdfb0 msgr2=0x7f39c81069f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:31:07.679 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:07.677+0000 7f39cd6e0640 1 -- 172.21.15.89:0/2176219034 shutdown_connections 2024-09-21T03:31:07.679 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:07.677+0000 7f39cd6e0640 1 -- 172.21.15.89:0/2176219034 wait complete. 2024-09-21T03:31:07.683 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:07.682+0000 7f5b33a0c640 1 -- 172.21.15.89:0/744316935 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5b2c102530 msgr2=0x7f5b2c108d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:31:07.683 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:07.682+0000 7f5b33a0c640 1 --2- 172.21.15.89:0/744316935 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5b2c102530 0x7f5b2c108d30 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f5b1c009860 tx=0x7f5b1c02a340 comp rx=0 tx=0).stop 2024-09-21T03:31:07.683 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:07.682+0000 7f5b33a0c640 1 -- 172.21.15.89:0/744316935 shutdown_connections 2024-09-21T03:31:07.683 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:07.682+0000 7f5b33a0c640 1 --2- 172.21.15.89:0/744316935 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5b2c102530 0x7f5b2c108d30 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:07.683 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:07.682+0000 7f5b33a0c640 1 --2- 172.21.15.89:0/744316935 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5b2c109470 0x7f5b2c10b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:07.684 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:07.682+0000 7f5b33a0c640 1 --2- 172.21.15.89:0/744316935 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5b2c101b00 0x7f5b2c101f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:07.684 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:07.682+0000 7f5b33a0c640 1 -- 172.21.15.89:0/744316935 >> 172.21.15.89:0/744316935 conn(0x7f5b2c0fd770 msgr2=0x7f5b2c0ffbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:31:07.684 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:07.682+0000 7f5b33a0c640 1 -- 172.21.15.89:0/744316935 shutdown_connections 2024-09-21T03:31:07.684 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:07.682+0000 7f5b33a0c640 1 -- 172.21.15.89:0/744316935 wait complete. 2024-09-21T03:31:07.685 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:07.683+0000 7f5b33a0c640 1 Processor -- start 2024-09-21T03:31:07.685 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:07.683+0000 7f5b33a0c640 1 -- start start 2024-09-21T03:31:07.686 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:07.683+0000 7f5b33a0c640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5b2c101b00 0x7f5b2c199e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:07.686 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:07.683+0000 7f5b33a0c640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5b2c102530 0x7f5b2c19a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:07.686 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:07.683+0000 7f5b33a0c640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5b2c109470 0x7f5b2c19b080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:07.686 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:07.683+0000 7f5b33a0c640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f5b2c10f210 con 0x7f5b2c102530 2024-09-21T03:31:07.686 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:07.683+0000 7f5b33a0c640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f5b2c10f090 con 0x7f5b2c101b00 2024-09-21T03:31:07.686 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:07.683+0000 7f5b33a0c640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f5b2c10f390 con 0x7f5b2c109470 2024-09-21T03:31:07.686 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:07.683+0000 7f5b30fa7640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5b2c102530 0x7f5b2c19a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:07.686 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:07.683+0000 7f5b30fa7640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5b2c102530 0x7f5b2c19a4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:55178/0 (socket says 172.21.15.89:55178) 2024-09-21T03:31:07.686 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:07.683+0000 7f5b30fa7640 1 -- 172.21.15.89:0/4144302819 learned_addr learned my addr 172.21.15.89:0/4144302819 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-21T03:31:07.686 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:07.683+0000 7f5b317a8640 1 --2- 172.21.15.89:0/4144302819 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5b2c101b00 0x7f5b2c199e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:07.686 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:07.683+0000 7f5b31fa9640 1 --2- 172.21.15.89:0/4144302819 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5b2c109470 0x7f5b2c19b080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:07.686 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:07.683+0000 7f5b30fa7640 1 -- 172.21.15.89:0/4144302819 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5b2c109470 msgr2=0x7f5b2c19b080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:31:07.686 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:07.683+0000 7f5b30fa7640 1 --2- 172.21.15.89:0/4144302819 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5b2c109470 0x7f5b2c19b080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:07.686 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:07.683+0000 7f5b30fa7640 1 -- 172.21.15.89:0/4144302819 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5b2c101b00 msgr2=0x7f5b2c199e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:31:07.687 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:07.683+0000 7f5b30fa7640 1 --2- 172.21.15.89:0/4144302819 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5b2c101b00 0x7f5b2c199e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:07.687 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:07.683+0000 7f5b30fa7640 1 -- 172.21.15.89:0/4144302819 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b2c19b8a0 con 0x7f5b2c102530 2024-09-21T03:31:07.687 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:07.684+0000 7f5b30fa7640 1 --2- 172.21.15.89:0/4144302819 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5b2c102530 0x7f5b2c19a4e0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f5b1c005cd0 tx=0x7f5b1c0306e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:31:07.687 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:07.684+0000 7f5b227fc640 1 -- 172.21.15.89:0/4144302819 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b1c042070 con 0x7f5b2c102530 2024-09-21T03:31:07.687 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:07.684+0000 7f5b227fc640 1 -- 172.21.15.89:0/4144302819 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5b1c032df0 con 0x7f5b2c102530 2024-09-21T03:31:07.687 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:07.684+0000 7f5b33a0c640 1 -- 172.21.15.89:0/4144302819 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5b2c19bb30 con 0x7f5b2c102530 2024-09-21T03:31:07.687 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:07.684+0000 7f5b227fc640 1 -- 172.21.15.89:0/4144302819 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b1c037040 con 0x7f5b2c102530 2024-09-21T03:31:07.687 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:07.684+0000 7f5b33a0c640 1 -- 172.21.15.89:0/4144302819 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b2c1a20c0 con 0x7f5b2c102530 2024-09-21T03:31:07.687 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:07.685+0000 7f5b33a0c640 1 -- 172.21.15.89:0/4144302819 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5b2c0ff290 con 0x7f5b2c102530 2024-09-21T03:31:07.688 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:07.687+0000 7f5b227fc640 1 -- 172.21.15.89:0/4144302819 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 16) ==== 154456+0+0 (secure 0 0 0) 0x7f5b1c045070 con 0x7f5b2c102530 2024-09-21T03:31:07.689 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:07.687+0000 7f5b227fc640 1 --2- 172.21.15.89:0/4144302819 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f5b180b6480 0x7f5b180b8990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:07.689 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:07.688+0000 7f5b317a8640 1 --2- 172.21.15.89:0/4144302819 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f5b180b6480 0x7f5b180b8990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:07.689 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:07.688+0000 7f5b227fc640 1 -- 172.21.15.89:0/4144302819 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f5b1c105390 con 0x7f5b2c102530 2024-09-21T03:31:07.689 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:07.688+0000 7f5b317a8640 1 --2- 172.21.15.89:0/4144302819 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f5b180b6480 0x7f5b180b8990 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f5b280045f0 tx=0x7f5b2800b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:31:07.692 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:07.691+0000 7f5b227fc640 1 -- 172.21.15.89:0/4144302819 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f5b1c04c020 con 0x7f5b2c102530 2024-09-21T03:31:07.873 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:07.871+0000 7f5b33a0c640 1 -- 172.21.15.89:0/4144302819 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command([{prefix=config set, name=rados_mon_op_timeout}] v 0) -- 0x7f5b2c02e640 con 0x7f5b2c102530 2024-09-21T03:31:07.877 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:07.876+0000 7f5b227fc640 1 -- 172.21.15.89:0/4144302819 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{prefix=config set, name=rados_mon_op_timeout}]=0 v27)=0 v27) ==== 120+0+0 (secure 0 0 0) 0x7f5b1c0cd8f0 con 0x7f5b2c102530 2024-09-21T03:31:07.879 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:07.878+0000 7f5b33a0c640 1 -- 172.21.15.89:0/4144302819 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f5b180b6480 msgr2=0x7f5b180b8990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:31:07.879 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:07.878+0000 7f5b33a0c640 1 --2- 172.21.15.89:0/4144302819 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f5b180b6480 0x7f5b180b8990 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f5b280045f0 tx=0x7f5b2800b390 comp rx=0 tx=0).stop 2024-09-21T03:31:07.879 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:07.878+0000 7f5b33a0c640 1 -- 172.21.15.89:0/4144302819 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5b2c102530 msgr2=0x7f5b2c19a4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:31:07.880 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:07.878+0000 7f5b33a0c640 1 --2- 172.21.15.89:0/4144302819 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5b2c102530 0x7f5b2c19a4e0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f5b1c005cd0 tx=0x7f5b1c0306e0 comp rx=0 tx=0).stop 2024-09-21T03:31:07.880 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:07.878+0000 7f5b33a0c640 1 -- 172.21.15.89:0/4144302819 shutdown_connections 2024-09-21T03:31:07.880 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:07.878+0000 7f5b33a0c640 1 --2- 172.21.15.89:0/4144302819 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f5b180b6480 0x7f5b180b8990 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:07.880 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:07.878+0000 7f5b33a0c640 1 --2- 172.21.15.89:0/4144302819 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5b2c109470 0x7f5b2c19b080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:07.880 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:07.878+0000 7f5b33a0c640 1 --2- 172.21.15.89:0/4144302819 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5b2c102530 0x7f5b2c19a4e0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:07.880 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:07.878+0000 7f5b33a0c640 1 --2- 172.21.15.89:0/4144302819 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5b2c101b00 0x7f5b2c199e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:07.880 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:07.878+0000 7f5b33a0c640 1 -- 172.21.15.89:0/4144302819 >> 172.21.15.89:0/4144302819 conn(0x7f5b2c0fd770 msgr2=0x7f5b2c1077e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:31:07.880 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:07.878+0000 7f5b33a0c640 1 -- 172.21.15.89:0/4144302819 shutdown_connections 2024-09-21T03:31:07.880 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:07.878+0000 7f5b33a0c640 1 -- 172.21.15.89:0/4144302819 wait complete. 2024-09-21T03:31:08.411 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:08.410+0000 7f1179687640 1 -- 172.21.15.89:0/1404839607 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f117406b980 msgr2=0x7f1174106390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:31:08.411 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:08.410+0000 7f1179687640 1 --2- 172.21.15.89:0/1404839607 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f117406b980 0x7f1174106390 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f1164009ca0 tx=0x7f116402a8c0 comp rx=0 tx=0).stop 2024-09-21T03:31:08.412 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:08.410+0000 7f1179687640 1 -- 172.21.15.89:0/1404839607 shutdown_connections 2024-09-21T03:31:08.412 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:08.410+0000 7f1179687640 1 --2- 172.21.15.89:0/1404839607 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f11741068d0 0x7f1174108d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:08.412 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:08.410+0000 7f1179687640 1 --2- 172.21.15.89:0/1404839607 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1174109470 0x7f117410b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:08.412 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:08.410+0000 7f1179687640 1 --2- 172.21.15.89:0/1404839607 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f117406b980 0x7f1174106390 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:08.412 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:08.410+0000 7f1179687640 1 -- 172.21.15.89:0/1404839607 >> 172.21.15.89:0/1404839607 conn(0x7f11740fdfb0 msgr2=0x7f1174100460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:31:08.412 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:08.410+0000 7f1179687640 1 -- 172.21.15.89:0/1404839607 shutdown_connections 2024-09-21T03:31:08.412 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:08.410+0000 7f1179687640 1 -- 172.21.15.89:0/1404839607 wait complete. 2024-09-21T03:31:08.412 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:08.410+0000 7f1179687640 1 Processor -- start 2024-09-21T03:31:08.414 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:08.411+0000 7f1179687640 1 -- start start 2024-09-21T03:31:08.414 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:08.411+0000 7f1179687640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f117406b980 0x7f1174199f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:08.414 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:08.411+0000 7f1179687640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f11741068d0 0x7f117419a580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:08.414 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:08.411+0000 7f1179687640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1174109470 0x7f117419b120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:08.414 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:08.411+0000 7f1179687640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f117410f2e0 con 0x7f11741068d0 2024-09-21T03:31:08.414 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:08.411+0000 7f1179687640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f117410f160 con 0x7f117406b980 2024-09-21T03:31:08.414 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:08.411+0000 7f1179687640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f117410f460 con 0x7f1174109470 2024-09-21T03:31:08.414 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:08.411+0000 7f11727fc640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f11741068d0 0x7f117419a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:08.414 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:08.411+0000 7f11727fc640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f11741068d0 0x7f117419a580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:46912/0 (socket says 172.21.15.89:46912) 2024-09-21T03:31:08.415 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:08.411+0000 7f11727fc640 1 -- 172.21.15.89:0/266231142 learned_addr learned my addr 172.21.15.89:0/266231142 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-21T03:31:08.415 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:08.411+0000 7f1172ffd640 1 --2- 172.21.15.89:0/266231142 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f117406b980 0x7f1174199f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:08.415 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:08.411+0000 7f11737fe640 1 --2- 172.21.15.89:0/266231142 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1174109470 0x7f117419b120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:08.415 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:08.411+0000 7f11727fc640 1 -- 172.21.15.89:0/266231142 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1174109470 msgr2=0x7f117419b120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:31:08.415 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:08.411+0000 7f11727fc640 1 --2- 172.21.15.89:0/266231142 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1174109470 0x7f117419b120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:08.415 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:08.411+0000 7f11727fc640 1 -- 172.21.15.89:0/266231142 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f117406b980 msgr2=0x7f1174199f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:31:08.415 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:08.411+0000 7f11727fc640 1 --2- 172.21.15.89:0/266231142 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f117406b980 0x7f1174199f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:08.415 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:08.411+0000 7f11727fc640 1 -- 172.21.15.89:0/266231142 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f117419b940 con 0x7f11741068d0 2024-09-21T03:31:08.415 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:08.412+0000 7f11727fc640 1 --2- 172.21.15.89:0/266231142 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f11741068d0 0x7f117419a580 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f115c00c890 tx=0x7f115c00cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:31:08.415 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:08.412+0000 7f116bfff640 1 -- 172.21.15.89:0/266231142 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f115c004220 con 0x7f11741068d0 2024-09-21T03:31:08.415 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:08.412+0000 7f116bfff640 1 -- 172.21.15.89:0/266231142 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f115c007e40 con 0x7f11741068d0 2024-09-21T03:31:08.415 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:08.412+0000 7f1179687640 1 -- 172.21.15.89:0/266231142 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f117419bc30 con 0x7f11741068d0 2024-09-21T03:31:08.415 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:08.412+0000 7f116bfff640 1 -- 172.21.15.89:0/266231142 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f115c004a00 con 0x7f11741068d0 2024-09-21T03:31:08.415 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:08.412+0000 7f1179687640 1 -- 172.21.15.89:0/266231142 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1174102340 con 0x7f11741068d0 2024-09-21T03:31:08.416 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:08.413+0000 7f1179687640 1 -- 172.21.15.89:0/266231142 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1174106d20 con 0x7f11741068d0 2024-09-21T03:31:08.416 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:08.414+0000 7f116bfff640 1 -- 172.21.15.89:0/266231142 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 16) ==== 154456+0+0 (secure 0 0 0) 0x7f115c0079b0 con 0x7f11741068d0 2024-09-21T03:31:08.416 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:08.415+0000 7f116bfff640 1 --2- 172.21.15.89:0/266231142 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f11600b6480 0x7f11600b8990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:08.416 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:08.415+0000 7f1172ffd640 1 --2- 172.21.15.89:0/266231142 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f11600b6480 0x7f11600b8990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:08.416 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:08.415+0000 7f116bfff640 1 -- 172.21.15.89:0/266231142 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f115c0e5690 con 0x7f11741068d0 2024-09-21T03:31:08.417 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:08.416+0000 7f1172ffd640 1 --2- 172.21.15.89:0/266231142 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f11600b6480 0x7f11600b8990 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f11640094e0 tx=0x7f11640318e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:31:08.419 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:08.418+0000 7f116bfff640 1 -- 172.21.15.89:0/266231142 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f115c014030 con 0x7f11741068d0 2024-09-21T03:31:08.605 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:08.604+0000 7f1179687640 1 -- 172.21.15.89:0/266231142 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command([{prefix=config set, name=osd_op_complaint_time}] v 0) -- 0x7f117402e9b0 con 0x7f11741068d0 2024-09-21T03:31:08.609 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:08.608+0000 7f116bfff640 1 -- 172.21.15.89:0/266231142 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_op_complaint_time}]=0 v28)=0 v28) ==== 121+0+0 (secure 0 0 0) 0x7f115c0b1fd0 con 0x7f11741068d0 2024-09-21T03:31:08.611 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:08.610+0000 7f1179687640 1 -- 172.21.15.89:0/266231142 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f11600b6480 msgr2=0x7f11600b8990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:31:08.612 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:08.610+0000 7f1179687640 1 --2- 172.21.15.89:0/266231142 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f11600b6480 0x7f11600b8990 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f11640094e0 tx=0x7f11640318e0 comp rx=0 tx=0).stop 2024-09-21T03:31:08.612 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:08.610+0000 7f1179687640 1 -- 172.21.15.89:0/266231142 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f11741068d0 msgr2=0x7f117419a580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:31:08.612 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:08.610+0000 7f4368f14640 1 -- 172.21.15.89:0/868154066 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4364102640 msgr2=0x7f4364102ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:31:08.612 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:08.610+0000 7f4368f14640 1 --2- 172.21.15.89:0/868154066 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4364102640 0x7f4364102ab0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f4354009900 tx=0x7f435402a480 comp rx=0 tx=0).stop 2024-09-21T03:31:08.612 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:08.611+0000 7f4368f14640 1 -- 172.21.15.89:0/868154066 shutdown_connections 2024-09-21T03:31:08.612 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:08.611+0000 7f4368f14640 1 --2- 172.21.15.89:0/868154066 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f43641036d0 0x7f4364103ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:08.612 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:08.611+0000 7f4368f14640 1 --2- 172.21.15.89:0/868154066 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4364104070 0x7f436410aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:08.612 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:08.611+0000 7f4368f14640 1 --2- 172.21.15.89:0/868154066 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4364102640 0x7f4364102ab0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:08.613 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:08.611+0000 7f4368f14640 1 -- 172.21.15.89:0/868154066 >> 172.21.15.89:0/868154066 conn(0x7f43640fdfb0 msgr2=0x7f4364100460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:31:08.613 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:08.610+0000 7f1179687640 1 --2- 172.21.15.89:0/266231142 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f11741068d0 0x7f117419a580 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f115c00c890 tx=0x7f115c00cd50 comp rx=0 tx=0).stop 2024-09-21T03:31:08.613 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:08.611+0000 7f1179687640 1 -- 172.21.15.89:0/266231142 shutdown_connections 2024-09-21T03:31:08.613 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:08.611+0000 7f1179687640 1 --2- 172.21.15.89:0/266231142 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f11600b6480 0x7f11600b8990 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:08.613 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:08.611+0000 7f1179687640 1 --2- 172.21.15.89:0/266231142 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1174109470 0x7f117419b120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:08.613 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:08.611+0000 7f1179687640 1 --2- 172.21.15.89:0/266231142 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f11741068d0 0x7f117419a580 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:08.613 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:08.611+0000 7f1179687640 1 --2- 172.21.15.89:0/266231142 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f117406b980 0x7f1174199f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:08.613 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:08.611+0000 7f1179687640 1 -- 172.21.15.89:0/266231142 >> 172.21.15.89:0/266231142 conn(0x7f11740fdfb0 msgr2=0x7f1174104d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:31:08.614 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:08.611+0000 7f4368f14640 1 -- 172.21.15.89:0/868154066 shutdown_connections 2024-09-21T03:31:08.614 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:08.611+0000 7f4368f14640 1 -- 172.21.15.89:0/868154066 wait complete. 2024-09-21T03:31:08.614 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:08.611+0000 7f1179687640 1 -- 172.21.15.89:0/266231142 shutdown_connections 2024-09-21T03:31:08.614 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:08.611+0000 7f1179687640 1 -- 172.21.15.89:0/266231142 wait complete. 2024-09-21T03:31:08.616 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:08.612+0000 7f4368f14640 1 Processor -- start 2024-09-21T03:31:08.616 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:08.612+0000 7f4368f14640 1 -- start start 2024-09-21T03:31:08.616 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:08.612+0000 7f4368f14640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4364102640 0x7f4364199d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:08.616 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:08.612+0000 7f4368f14640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f43641036d0 0x7f436419a370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:08.616 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:08.612+0000 7f4368f14640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4364104070 0x7f436419af10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:08.616 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:08.612+0000 7f4368f14640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f436410e130 con 0x7f4364102640 2024-09-21T03:31:08.616 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:08.612+0000 7f4368f14640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f436410dfb0 con 0x7f43641036d0 2024-09-21T03:31:08.616 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:08.612+0000 7f4368f14640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f436410e2b0 con 0x7f4364104070 2024-09-21T03:31:08.616 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:08.612+0000 7f436259c640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4364102640 0x7f4364199d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:08.616 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:08.612+0000 7f436259c640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4364102640 0x7f4364199d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:46936/0 (socket says 172.21.15.89:46936) 2024-09-21T03:31:08.617 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:08.612+0000 7f436259c640 1 -- 172.21.15.89:0/3146037656 learned_addr learned my addr 172.21.15.89:0/3146037656 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-21T03:31:08.617 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:08.612+0000 7f4359d9b640 1 --2- 172.21.15.89:0/3146037656 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f43641036d0 0x7f436419a370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:08.617 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:08.612+0000 7f4362d9d640 1 --2- 172.21.15.89:0/3146037656 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4364104070 0x7f436419af10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:08.617 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:08.613+0000 7f436259c640 1 -- 172.21.15.89:0/3146037656 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4364104070 msgr2=0x7f436419af10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:31:08.617 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:08.613+0000 7f436259c640 1 --2- 172.21.15.89:0/3146037656 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4364104070 0x7f436419af10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:08.617 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:08.613+0000 7f436259c640 1 -- 172.21.15.89:0/3146037656 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f43641036d0 msgr2=0x7f436419a370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:31:08.617 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:08.613+0000 7f436259c640 1 --2- 172.21.15.89:0/3146037656 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f43641036d0 0x7f436419a370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:08.617 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:08.613+0000 7f436259c640 1 -- 172.21.15.89:0/3146037656 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f436419b730 con 0x7f4364102640 2024-09-21T03:31:08.617 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:08.613+0000 7f436259c640 1 --2- 172.21.15.89:0/3146037656 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4364102640 0x7f4364199d00 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f4354009850 tx=0x7f43540042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:31:08.617 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:08.613+0000 7f435bfff640 1 -- 172.21.15.89:0/3146037656 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f435403f030 con 0x7f4364102640 2024-09-21T03:31:08.617 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:08.613+0000 7f435bfff640 1 -- 172.21.15.89:0/3146037656 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4354003c90 con 0x7f4364102640 2024-09-21T03:31:08.617 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:08.613+0000 7f435bfff640 1 -- 172.21.15.89:0/3146037656 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4354035070 con 0x7f4364102640 2024-09-21T03:31:08.617 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:08.613+0000 7f4368f14640 1 -- 172.21.15.89:0/3146037656 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f436419b9c0 con 0x7f4364102640 2024-09-21T03:31:08.617 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:08.613+0000 7f4368f14640 1 -- 172.21.15.89:0/3146037656 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43641049e0 con 0x7f4364102640 2024-09-21T03:31:08.618 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:08.615+0000 7f4368f14640 1 -- 172.21.15.89:0/3146037656 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4364104ca0 con 0x7f4364102640 2024-09-21T03:31:08.618 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:08.616+0000 7f435bfff640 1 -- 172.21.15.89:0/3146037656 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 16) ==== 154456+0+0 (secure 0 0 0) 0x7f4354042030 con 0x7f4364102640 2024-09-21T03:31:08.618 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:08.617+0000 7f435bfff640 1 --2- 172.21.15.89:0/3146037656 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f43500b6480 0x7f43500b8990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:08.619 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:08.617+0000 7f4359d9b640 1 --2- 172.21.15.89:0/3146037656 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f43500b6480 0x7f43500b8990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:08.619 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:08.617+0000 7f435bfff640 1 -- 172.21.15.89:0/3146037656 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f4354105df0 con 0x7f4364102640 2024-09-21T03:31:08.619 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:08.617+0000 7f4359d9b640 1 --2- 172.21.15.89:0/3146037656 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f43500b6480 0x7f43500b8990 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f434c009bd0 tx=0x7f434c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:31:08.622 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:08.621+0000 7f435bfff640 1 -- 172.21.15.89:0/3146037656 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f4354049020 con 0x7f4364102640 2024-09-21T03:31:08.814 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:08.813+0000 7f4368f14640 1 -- 172.21.15.89:0/3146037656 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command([{prefix=config set, name=debug_client}] v 0) -- 0x7f4364105420 con 0x7f4364102640 2024-09-21T03:31:08.818 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:08.817+0000 7f435bfff640 1 -- 172.21.15.89:0/3146037656 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{prefix=config set, name=debug_client}]=0 v29)=0 v29) ==== 111+0+0 (secure 0 0 0) 0x7f43540ce330 con 0x7f4364102640 2024-09-21T03:31:08.822 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:08.820+0000 7f4368f14640 1 -- 172.21.15.89:0/3146037656 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f43500b6480 msgr2=0x7f43500b8990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:31:08.822 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:08.820+0000 7f4368f14640 1 --2- 172.21.15.89:0/3146037656 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f43500b6480 0x7f43500b8990 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f434c009bd0 tx=0x7f434c005f90 comp rx=0 tx=0).stop 2024-09-21T03:31:08.822 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:08.820+0000 7f4368f14640 1 -- 172.21.15.89:0/3146037656 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4364102640 msgr2=0x7f4364199d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:31:08.822 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:08.820+0000 7f4368f14640 1 --2- 172.21.15.89:0/3146037656 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4364102640 0x7f4364199d00 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f4354009850 tx=0x7f43540042f0 comp rx=0 tx=0).stop 2024-09-21T03:31:08.822 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:08.820+0000 7f4368f14640 1 -- 172.21.15.89:0/3146037656 shutdown_connections 2024-09-21T03:31:08.822 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:08.820+0000 7f4368f14640 1 --2- 172.21.15.89:0/3146037656 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f43500b6480 0x7f43500b8990 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:08.822 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:08.820+0000 7f4368f14640 1 --2- 172.21.15.89:0/3146037656 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f43641036d0 0x7f436419a370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:08.822 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:08.820+0000 7f4368f14640 1 --2- 172.21.15.89:0/3146037656 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4364104070 0x7f436419af10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:08.823 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:08.820+0000 7f4368f14640 1 --2- 172.21.15.89:0/3146037656 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4364102640 0x7f4364199d00 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:08.823 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:08.820+0000 7f4368f14640 1 -- 172.21.15.89:0/3146037656 >> 172.21.15.89:0/3146037656 conn(0x7f43640fdfb0 msgr2=0x7f43640ff720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:31:08.823 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:08.821+0000 7f4368f14640 1 -- 172.21.15.89:0/3146037656 shutdown_connections 2024-09-21T03:31:08.823 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:08.821+0000 7f4368f14640 1 -- 172.21.15.89:0/3146037656 wait complete. 2024-09-21T03:31:09.061 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:31:08 smithi089 ceph-mon[30928]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:31:09.061 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:31:08 smithi089 ceph-mon[30928]: from='client.? 172.21.15.89:0/4144302819' entity='client.admin' 2024-09-21T03:31:09.061 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:31:08 smithi089 ceph-mon[30928]: from='client.? 172.21.15.89:0/266231142' entity='client.admin' 2024-09-21T03:31:09.061 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:31:08 smithi089 ceph-mon[30928]: from='client.? 172.21.15.89:0/3146037656' entity='client.admin' 2024-09-21T03:31:09.061 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:31:08 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:31:09.061 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:31:08 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:31:09.061 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:31:08 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:31:09.062 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:31:08 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:31:09.211 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:31:08 smithi114 ceph-mon[34140]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:31:09.211 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:31:08 smithi114 ceph-mon[34140]: from='client.? 172.21.15.89:0/4144302819' entity='client.admin' 2024-09-21T03:31:09.211 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:31:08 smithi114 ceph-mon[34140]: from='client.? 172.21.15.89:0/266231142' entity='client.admin' 2024-09-21T03:31:09.211 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:31:08 smithi114 ceph-mon[34140]: from='client.? 172.21.15.89:0/3146037656' entity='client.admin' 2024-09-21T03:31:09.211 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:31:08 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:31:09.211 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:31:08 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:31:09.211 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:31:08 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:31:09.211 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:31:08 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:31:09.250 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:31:08 smithi110 ceph-mon[34000]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:31:09.250 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:31:08 smithi110 ceph-mon[34000]: from='client.? 172.21.15.89:0/4144302819' entity='client.admin' 2024-09-21T03:31:09.250 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:31:08 smithi110 ceph-mon[34000]: from='client.? 172.21.15.89:0/266231142' entity='client.admin' 2024-09-21T03:31:09.250 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:31:08 smithi110 ceph-mon[34000]: from='client.? 172.21.15.89:0/3146037656' entity='client.admin' 2024-09-21T03:31:09.250 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:31:08 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:31:09.250 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:31:08 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:31:09.250 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:31:08 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:31:09.250 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:31:08 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:31:09.251 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:09.249+0000 7fb26d976640 1 -- 172.21.15.89:0/2583169306 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb268109470 msgr2=0x7fb26810b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:31:09.251 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:09.249+0000 7fb26d976640 1 --2- 172.21.15.89:0/2583169306 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb268109470 0x7fb26810b980 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fb260009ca0 tx=0x7fb26002a8c0 comp rx=0 tx=0).stop 2024-09-21T03:31:09.251 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:09.249+0000 7fb26d976640 1 -- 172.21.15.89:0/2583169306 shutdown_connections 2024-09-21T03:31:09.251 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:09.249+0000 7fb26d976640 1 --2- 172.21.15.89:0/2583169306 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb2681068d0 0x7fb268108d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:09.251 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:09.249+0000 7fb26d976640 1 --2- 172.21.15.89:0/2583169306 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb268109470 0x7fb26810b980 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:09.251 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:09.249+0000 7fb26d976640 1 --2- 172.21.15.89:0/2583169306 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb26806b980 0x7fb268106390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:09.252 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:09.249+0000 7fb26d976640 1 -- 172.21.15.89:0/2583169306 >> 172.21.15.89:0/2583169306 conn(0x7fb2680fdfb0 msgr2=0x7fb268100460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:31:09.252 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:09.249+0000 7fb26d976640 1 -- 172.21.15.89:0/2583169306 shutdown_connections 2024-09-21T03:31:09.252 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:09.250+0000 7fb26d976640 1 -- 172.21.15.89:0/2583169306 wait complete. 2024-09-21T03:31:09.252 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:09.250+0000 7fb26d976640 1 Processor -- start 2024-09-21T03:31:09.253 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:09.250+0000 7fb26d976640 1 -- start start 2024-09-21T03:31:09.253 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:09.251+0000 7fb26d976640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb26806b980 0x7fb268199e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:09.253 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:09.251+0000 7fb26d976640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb2681068d0 0x7fb26819a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:09.254 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:09.251+0000 7fb26d976640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb268109470 0x7fb26819b090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:09.254 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:09.251+0000 7fb26d976640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fb26810f290 con 0x7fb26806b980 2024-09-21T03:31:09.254 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:09.251+0000 7fb26d976640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fb26810f110 con 0x7fb268109470 2024-09-21T03:31:09.254 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:09.251+0000 7fb26d976640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fb26810f410 con 0x7fb2681068d0 2024-09-21T03:31:09.254 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:09.251+0000 7fb266ffd640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb26806b980 0x7fb268199e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:09.254 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:09.251+0000 7fb266ffd640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb26806b980 0x7fb268199e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:46950/0 (socket says 172.21.15.89:46950) 2024-09-21T03:31:09.254 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:09.251+0000 7fb266ffd640 1 -- 172.21.15.89:0/3125046394 learned_addr learned my addr 172.21.15.89:0/3125046394 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-21T03:31:09.254 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:09.251+0000 7fb2677fe640 1 --2- 172.21.15.89:0/3125046394 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb268109470 0x7fb26819b090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:09.254 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:09.251+0000 7fb2667fc640 1 --2- 172.21.15.89:0/3125046394 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb2681068d0 0x7fb26819a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:09.254 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:09.251+0000 7fb266ffd640 1 -- 172.21.15.89:0/3125046394 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb2681068d0 msgr2=0x7fb26819a4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:31:09.254 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:09.251+0000 7fb266ffd640 1 --2- 172.21.15.89:0/3125046394 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb2681068d0 0x7fb26819a4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:09.254 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:09.251+0000 7fb266ffd640 1 -- 172.21.15.89:0/3125046394 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb268109470 msgr2=0x7fb26819b090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:31:09.254 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:09.251+0000 7fb266ffd640 1 --2- 172.21.15.89:0/3125046394 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb268109470 0x7fb26819b090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:09.255 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:09.251+0000 7fb266ffd640 1 -- 172.21.15.89:0/3125046394 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb26819b8b0 con 0x7fb26806b980 2024-09-21T03:31:09.255 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:09.251+0000 7fb266ffd640 1 --2- 172.21.15.89:0/3125046394 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb26806b980 0x7fb268199e80 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fb25800d770 tx=0x7fb25800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:31:09.255 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:09.251+0000 7fb26c974640 1 -- 172.21.15.89:0/3125046394 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb258017070 con 0x7fb26806b980 2024-09-21T03:31:09.255 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:09.251+0000 7fb26c974640 1 -- 172.21.15.89:0/3125046394 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb2580054b0 con 0x7fb26806b980 2024-09-21T03:31:09.255 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:09.251+0000 7fb26d976640 1 -- 172.21.15.89:0/3125046394 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb26819bba0 con 0x7fb26806b980 2024-09-21T03:31:09.255 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:09.251+0000 7fb26c974640 1 -- 172.21.15.89:0/3125046394 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb258004c60 con 0x7fb26806b980 2024-09-21T03:31:09.255 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:09.252+0000 7fb26d976640 1 -- 172.21.15.89:0/3125046394 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2681a22a0 con 0x7fb26806b980 2024-09-21T03:31:09.255 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:09.253+0000 7fb26d976640 1 -- 172.21.15.89:0/3125046394 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb268106d20 con 0x7fb26806b980 2024-09-21T03:31:09.256 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:09.254+0000 7fb26c974640 1 -- 172.21.15.89:0/3125046394 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 16) ==== 154456+0+0 (secure 0 0 0) 0x7fb258004e00 con 0x7fb26806b980 2024-09-21T03:31:09.256 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:09.255+0000 7fb26c974640 1 --2- 172.21.15.89:0/3125046394 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7fb2500b6480 0x7fb2500b8990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:09.256 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:09.255+0000 7fb2667fc640 1 --2- 172.21.15.89:0/3125046394 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7fb2500b6480 0x7fb2500b8990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:09.256 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:09.255+0000 7fb26c974640 1 -- 172.21.15.89:0/3125046394 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fb2580e5490 con 0x7fb26806b980 2024-09-21T03:31:09.257 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:09.256+0000 7fb2667fc640 1 --2- 172.21.15.89:0/3125046394 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7fb2500b6480 0x7fb2500b8990 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fb25c005fd0 tx=0x7fb25c00b590 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:31:09.259 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:09.258+0000 7fb26c974640 1 -- 172.21.15.89:0/3125046394 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fb2580ad9f0 con 0x7fb26806b980 2024-09-21T03:31:09.442 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:09.441+0000 7fb26d976640 1 -- 172.21.15.89:0/3125046394 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command([{prefix=config set, name=osd_op_complaint_time}] v 0) -- 0x7fb2680ffad0 con 0x7fb26806b980 2024-09-21T03:31:09.446 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:09.445+0000 7fb26c974640 1 -- 172.21.15.89:0/3125046394 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_op_complaint_time}]=0 v30)=0 v30) ==== 121+0+0 (secure 0 0 0) 0x7fb2580b1dd0 con 0x7fb26806b980 2024-09-21T03:31:09.449 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:09.447+0000 7fb26d976640 1 -- 172.21.15.89:0/3125046394 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7fb2500b6480 msgr2=0x7fb2500b8990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:31:09.449 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:09.447+0000 7fb26d976640 1 --2- 172.21.15.89:0/3125046394 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7fb2500b6480 0x7fb2500b8990 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fb25c005fd0 tx=0x7fb25c00b590 comp rx=0 tx=0).stop 2024-09-21T03:31:09.449 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:09.447+0000 7fb26d976640 1 -- 172.21.15.89:0/3125046394 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb26806b980 msgr2=0x7fb268199e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:31:09.449 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:09.447+0000 7fb26d976640 1 --2- 172.21.15.89:0/3125046394 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb26806b980 0x7fb268199e80 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fb25800d770 tx=0x7fb25800dc30 comp rx=0 tx=0).stop 2024-09-21T03:31:09.449 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:09.447+0000 7fb26d976640 1 -- 172.21.15.89:0/3125046394 shutdown_connections 2024-09-21T03:31:09.449 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:09.447+0000 7fb26d976640 1 --2- 172.21.15.89:0/3125046394 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7fb2500b6480 0x7fb2500b8990 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:09.449 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:09.447+0000 7fb26d976640 1 --2- 172.21.15.89:0/3125046394 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb268109470 0x7fb26819b090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:09.449 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:09.447+0000 7fb26d976640 1 --2- 172.21.15.89:0/3125046394 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb2681068d0 0x7fb26819a4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:09.449 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:09.447+0000 7fb26d976640 1 --2- 172.21.15.89:0/3125046394 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb26806b980 0x7fb268199e80 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:09.449 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:09.447+0000 7fb26d976640 1 -- 172.21.15.89:0/3125046394 >> 172.21.15.89:0/3125046394 conn(0x7fb2680fdfb0 msgr2=0x7fb268104de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:31:09.449 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:09.448+0000 7fb26d976640 1 -- 172.21.15.89:0/3125046394 shutdown_connections 2024-09-21T03:31:09.450 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:09.448+0000 7fb26d976640 1 -- 172.21.15.89:0/3125046394 wait complete. 2024-09-21T03:31:10.710 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:31:10 smithi114 ceph-mon[34140]: from='client.? 172.21.15.89:0/3125046394' entity='client.admin' 2024-09-21T03:31:10.719 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:31:10 smithi089 ceph-mon[30928]: from='client.? 172.21.15.89:0/3125046394' entity='client.admin' 2024-09-21T03:31:10.750 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:31:10 smithi110 ceph-mon[34000]: from='client.? 172.21.15.89:0/3125046394' entity='client.admin' 2024-09-21T03:31:11.602 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:31:11 smithi089 ceph-mon[30928]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:31:11.701 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:11.700+0000 7f106b3ba640 1 -- 172.21.15.89:0/3217322658 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f106406b980 msgr2=0x7f1064106390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:31:11.701 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:11.700+0000 7f106b3ba640 1 --2- 172.21.15.89:0/3217322658 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f106406b980 0x7f1064106390 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f1054009ca0 tx=0x7f105402a8c0 comp rx=0 tx=0).stop 2024-09-21T03:31:11.701 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:11.700+0000 7f106b3ba640 1 -- 172.21.15.89:0/3217322658 shutdown_connections 2024-09-21T03:31:11.701 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:11.700+0000 7f106b3ba640 1 --2- 172.21.15.89:0/3217322658 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f10641068d0 0x7f1064108d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:11.701 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:11.700+0000 7f106b3ba640 1 --2- 172.21.15.89:0/3217322658 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1064109470 0x7f106410b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:11.701 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:11.700+0000 7f106b3ba640 1 --2- 172.21.15.89:0/3217322658 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f106406b980 0x7f1064106390 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:11.702 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:11.700+0000 7f106b3ba640 1 -- 172.21.15.89:0/3217322658 >> 172.21.15.89:0/3217322658 conn(0x7f10640fdfb0 msgr2=0x7f1064100460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:31:11.702 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:11.700+0000 7f106b3ba640 1 -- 172.21.15.89:0/3217322658 shutdown_connections 2024-09-21T03:31:11.702 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:11.700+0000 7f106b3ba640 1 -- 172.21.15.89:0/3217322658 wait complete. 2024-09-21T03:31:11.704 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:11.700+0000 7f106b3ba640 1 Processor -- start 2024-09-21T03:31:11.704 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:11.701+0000 7f106b3ba640 1 -- start start 2024-09-21T03:31:11.704 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:11.701+0000 7f106b3ba640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f106406b980 0x7f1064199ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:11.704 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:11.701+0000 7f106b3ba640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f10641068d0 0x7f106419a540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:11.704 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:11.701+0000 7f106b3ba640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1064109470 0x7f106419b0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:11.704 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:11.701+0000 7f106b3ba640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f106410f280 con 0x7f1064109470 2024-09-21T03:31:11.704 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:11.701+0000 7f106b3ba640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f106410f100 con 0x7f10641068d0 2024-09-21T03:31:11.704 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:11.701+0000 7f106b3ba640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f106410f400 con 0x7f106406b980 2024-09-21T03:31:11.704 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:11.701+0000 7f1069957640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1064109470 0x7f106419b0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:11.704 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:11.701+0000 7f1069957640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1064109470 0x7f106419b0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:46972/0 (socket says 172.21.15.89:46972) 2024-09-21T03:31:11.704 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:11.701+0000 7f1069957640 1 -- 172.21.15.89:0/578512476 learned_addr learned my addr 172.21.15.89:0/578512476 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-21T03:31:11.705 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:11.701+0000 7f1069156640 1 --2- 172.21.15.89:0/578512476 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f106406b980 0x7f1064199ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:11.705 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:11.701+0000 7f1068955640 1 --2- 172.21.15.89:0/578512476 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f10641068d0 0x7f106419a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:11.705 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:11.701+0000 7f1069957640 1 -- 172.21.15.89:0/578512476 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f106406b980 msgr2=0x7f1064199ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:31:11.705 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:11.701+0000 7f1069957640 1 --2- 172.21.15.89:0/578512476 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f106406b980 0x7f1064199ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:11.705 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:11.701+0000 7f1069957640 1 -- 172.21.15.89:0/578512476 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f10641068d0 msgr2=0x7f106419a540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:31:11.705 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:11.701+0000 7f1069957640 1 --2- 172.21.15.89:0/578512476 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f10641068d0 0x7f106419a540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:11.705 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:11.701+0000 7f1069957640 1 -- 172.21.15.89:0/578512476 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f106419b900 con 0x7f1064109470 2024-09-21T03:31:11.705 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:11.701+0000 7f1069957640 1 --2- 172.21.15.89:0/578512476 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1064109470 0x7f106419b0e0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f105c00d750 tx=0x7f105c00dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:31:11.705 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:11.702+0000 7f10627fc640 1 -- 172.21.15.89:0/578512476 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f105c017070 con 0x7f1064109470 2024-09-21T03:31:11.705 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:11.702+0000 7f10627fc640 1 -- 172.21.15.89:0/578512476 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f105c0054b0 con 0x7f1064109470 2024-09-21T03:31:11.705 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:11.702+0000 7f10627fc640 1 -- 172.21.15.89:0/578512476 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f105c004b90 con 0x7f1064109470 2024-09-21T03:31:11.705 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:11.702+0000 7f106b3ba640 1 -- 172.21.15.89:0/578512476 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f106419bbf0 con 0x7f1064109470 2024-09-21T03:31:11.705 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:11.702+0000 7f106b3ba640 1 -- 172.21.15.89:0/578512476 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10641a22f0 con 0x7f1064109470 2024-09-21T03:31:11.705 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:11.703+0000 7f106b3ba640 1 -- 172.21.15.89:0/578512476 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f10640ffad0 con 0x7f1064109470 2024-09-21T03:31:11.706 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:11.704+0000 7f10627fc640 1 -- 172.21.15.89:0/578512476 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 16) ==== 154456+0+0 (secure 0 0 0) 0x7f105c004d30 con 0x7f1064109470 2024-09-21T03:31:11.706 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:11.705+0000 7f10627fc640 1 --2- 172.21.15.89:0/578512476 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f104c0b6500 0x7f104c0b8a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:11.706 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:11.705+0000 7f1069156640 1 --2- 172.21.15.89:0/578512476 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f104c0b6500 0x7f104c0b8a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:11.706 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:11.705+0000 7f10627fc640 1 -- 172.21.15.89:0/578512476 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f105c0e5410 con 0x7f1064109470 2024-09-21T03:31:11.707 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:11.706+0000 7f1069156640 1 --2- 172.21.15.89:0/578512476 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f104c0b6500 0x7f104c0b8a10 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f10540094e0 tx=0x7f10540318e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:31:11.709 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:11.708+0000 7f10627fc640 1 -- 172.21.15.89:0/578512476 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f105c0ad9f0 con 0x7f1064109470 2024-09-21T03:31:11.710 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:31:11 smithi114 ceph-mon[34140]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:31:11.750 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:31:11 smithi110 ceph-mon[34000]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:31:11.891 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:11.890+0000 7f106b3ba640 1 -- 172.21.15.89:0/578512476 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command([{prefix=config set, name=mon_op_complaint_time}] v 0) -- 0x7f106402e640 con 0x7f1064109470 2024-09-21T03:31:11.895 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:11.894+0000 7f10627fc640 1 -- 172.21.15.89:0/578512476 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_op_complaint_time}]=0 v31)=0 v31) ==== 121+0+0 (secure 0 0 0) 0x7f105c0b1dd0 con 0x7f1064109470 2024-09-21T03:31:11.897 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:11.896+0000 7f106b3ba640 1 -- 172.21.15.89:0/578512476 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f104c0b6500 msgr2=0x7f104c0b8a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:31:11.897 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:11.896+0000 7f106b3ba640 1 --2- 172.21.15.89:0/578512476 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f104c0b6500 0x7f104c0b8a10 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f10540094e0 tx=0x7f10540318e0 comp rx=0 tx=0).stop 2024-09-21T03:31:11.898 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:11.896+0000 7f106b3ba640 1 -- 172.21.15.89:0/578512476 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1064109470 msgr2=0x7f106419b0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:31:11.898 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:11.896+0000 7f106b3ba640 1 --2- 172.21.15.89:0/578512476 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1064109470 0x7f106419b0e0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f105c00d750 tx=0x7f105c00dc10 comp rx=0 tx=0).stop 2024-09-21T03:31:11.898 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:11.896+0000 7f106b3ba640 1 -- 172.21.15.89:0/578512476 shutdown_connections 2024-09-21T03:31:11.898 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:11.896+0000 7f106b3ba640 1 --2- 172.21.15.89:0/578512476 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f104c0b6500 0x7f104c0b8a10 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:11.898 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:11.896+0000 7f106b3ba640 1 --2- 172.21.15.89:0/578512476 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f10641068d0 0x7f106419a540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:11.898 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:11.896+0000 7f106b3ba640 1 --2- 172.21.15.89:0/578512476 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1064109470 0x7f106419b0e0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:11.898 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:11.896+0000 7f106b3ba640 1 --2- 172.21.15.89:0/578512476 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f106406b980 0x7f1064199ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:11.898 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:11.896+0000 7f106b3ba640 1 -- 172.21.15.89:0/578512476 >> 172.21.15.89:0/578512476 conn(0x7f10640fdfb0 msgr2=0x7f1064104ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:31:11.898 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:11.896+0000 7f106b3ba640 1 -- 172.21.15.89:0/578512476 shutdown_connections 2024-09-21T03:31:11.898 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:11.896+0000 7f106b3ba640 1 -- 172.21.15.89:0/578512476 wait complete. 2024-09-21T03:31:13.061 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:31:12 smithi089 ceph-mon[30928]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:31:13.061 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:31:12 smithi089 ceph-mon[30928]: from='client.? 172.21.15.89:0/578512476' entity='client.admin' 2024-09-21T03:31:13.061 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:31:12 smithi089 ceph-mon[30928]: from='client.? 172.21.15.89:0/511579686' entity='client.admin' 2024-09-21T03:31:13.061 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:31:12 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:31:13.137 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:12.403+0000 7f92adf7d640 1 -- 172.21.15.89:0/672398105 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f92a810b6f0 msgr2=0x7f92a810db90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:31:13.137 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:12.403+0000 7f92adf7d640 1 --2- 172.21.15.89:0/672398105 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f92a810b6f0 0x7f92a810db90 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f92a0009ca0 tx=0x7f92a002a8c0 comp rx=0 tx=0).stop 2024-09-21T03:31:13.137 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:12.403+0000 7f92adf7d640 1 -- 172.21.15.89:0/672398105 shutdown_connections 2024-09-21T03:31:13.137 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:12.403+0000 7f92adf7d640 1 --2- 172.21.15.89:0/672398105 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f92a8075cf0 0x7f92a8076100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:13.137 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:12.403+0000 7f92adf7d640 1 --2- 172.21.15.89:0/672398105 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f92a810b6f0 0x7f92a810db90 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:13.137 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:12.403+0000 7f92adf7d640 1 --2- 172.21.15.89:0/672398105 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f92a80770e0 0x7f92a80757b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:13.137 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:12.403+0000 7f92adf7d640 1 -- 172.21.15.89:0/672398105 >> 172.21.15.89:0/672398105 conn(0x7f92a80ff0c0 msgr2=0x7f92a8101570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:31:13.137 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:12.403+0000 7f92adf7d640 1 -- 172.21.15.89:0/672398105 shutdown_connections 2024-09-21T03:31:13.137 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:12.403+0000 7f92adf7d640 1 -- 172.21.15.89:0/672398105 wait complete. 2024-09-21T03:31:13.137 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:12.403+0000 7f92adf7d640 1 Processor -- start 2024-09-21T03:31:13.137 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:12.403+0000 7f92adf7d640 1 -- start start 2024-09-21T03:31:13.137 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:12.404+0000 7f92adf7d640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f92a8075cf0 0x7f92a819c0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:13.137 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:12.404+0000 7f92adf7d640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f92a80770e0 0x7f92a819c720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:13.137 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:12.404+0000 7f92adf7d640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f92a810b6f0 0x7f92a819d2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:13.138 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:12.404+0000 7f92adf7d640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f92a8111490 con 0x7f92a8075cf0 2024-09-21T03:31:13.138 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:12.404+0000 7f92adf7d640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f92a8111310 con 0x7f92a810b6f0 2024-09-21T03:31:13.138 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:12.404+0000 7f92adf7d640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f92a8111610 con 0x7f92a80770e0 2024-09-21T03:31:13.138 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:12.404+0000 7f92a77fe640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f92a8075cf0 0x7f92a819c0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:13.138 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:12.404+0000 7f92a77fe640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f92a8075cf0 0x7f92a819c0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:46992/0 (socket says 172.21.15.89:46992) 2024-09-21T03:31:13.138 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:12.404+0000 7f92a77fe640 1 -- 172.21.15.89:0/511579686 learned_addr learned my addr 172.21.15.89:0/511579686 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-21T03:31:13.138 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:12.404+0000 7f92a6ffd640 1 --2- 172.21.15.89:0/511579686 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f92a80770e0 0x7f92a819c720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:13.138 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:12.404+0000 7f92a7fff640 1 --2- 172.21.15.89:0/511579686 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f92a810b6f0 0x7f92a819d2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:13.138 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:12.404+0000 7f92a77fe640 1 -- 172.21.15.89:0/511579686 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f92a80770e0 msgr2=0x7f92a819c720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:31:13.138 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:12.404+0000 7f92a77fe640 1 --2- 172.21.15.89:0/511579686 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f92a80770e0 0x7f92a819c720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:13.138 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:12.404+0000 7f92a77fe640 1 -- 172.21.15.89:0/511579686 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f92a810b6f0 msgr2=0x7f92a819d2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:31:13.138 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:12.404+0000 7f92a77fe640 1 --2- 172.21.15.89:0/511579686 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f92a810b6f0 0x7f92a819d2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:13.138 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:12.404+0000 7f92a77fe640 1 -- 172.21.15.89:0/511579686 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92a819dae0 con 0x7f92a8075cf0 2024-09-21T03:31:13.139 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:12.404+0000 7f92a7fff640 1 --2- 172.21.15.89:0/511579686 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f92a810b6f0 0x7f92a819d2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:31:13.139 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:12.404+0000 7f92a77fe640 1 --2- 172.21.15.89:0/511579686 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f92a8075cf0 0x7f92a819c0b0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f929800e9f0 tx=0x7f929800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:31:13.139 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:12.405+0000 7f92a4ff9640 1 -- 172.21.15.89:0/511579686 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f929800cd80 con 0x7f92a8075cf0 2024-09-21T03:31:13.139 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:12.405+0000 7f92a4ff9640 1 -- 172.21.15.89:0/511579686 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f92980054b0 con 0x7f92a8075cf0 2024-09-21T03:31:13.139 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:12.405+0000 7f92adf7d640 1 -- 172.21.15.89:0/511579686 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f92a819ddd0 con 0x7f92a8075cf0 2024-09-21T03:31:13.139 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:12.405+0000 7f92a4ff9640 1 -- 172.21.15.89:0/511579686 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9298010430 con 0x7f92a8075cf0 2024-09-21T03:31:13.139 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:12.405+0000 7f92adf7d640 1 -- 172.21.15.89:0/511579686 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f92a81a44d0 con 0x7f92a8075cf0 2024-09-21T03:31:13.139 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:12.406+0000 7f92adf7d640 1 -- 172.21.15.89:0/511579686 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f92a810a500 con 0x7f92a8075cf0 2024-09-21T03:31:13.139 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:12.407+0000 7f92a4ff9640 1 -- 172.21.15.89:0/511579686 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 16) ==== 154456+0+0 (secure 0 0 0) 0x7f92980046c0 con 0x7f92a8075cf0 2024-09-21T03:31:13.139 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:12.408+0000 7f92a4ff9640 1 --2- 172.21.15.89:0/511579686 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f92900b6480 0x7f92900b8990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:13.139 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:12.408+0000 7f92a6ffd640 1 --2- 172.21.15.89:0/511579686 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f92900b6480 0x7f92900b8990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:13.139 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:12.408+0000 7f92a4ff9640 1 -- 172.21.15.89:0/511579686 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f92980e5c80 con 0x7f92a8075cf0 2024-09-21T03:31:13.139 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:12.408+0000 7f92a6ffd640 1 --2- 172.21.15.89:0/511579686 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f92900b6480 0x7f92900b8990 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f929c009bd0 tx=0x7f929c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:31:13.139 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:12.411+0000 7f92a4ff9640 1 -- 172.21.15.89:0/511579686 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f9298056710 con 0x7f92a8075cf0 2024-09-21T03:31:13.140 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:12.599+0000 7f92adf7d640 1 -- 172.21.15.89:0/511579686 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command([{prefix=config set, name=client_mount_timeout}] v 0) -- 0x7f92a802e640 con 0x7f92a8075cf0 2024-09-21T03:31:13.140 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:12.603+0000 7f92a4ff9640 1 -- 172.21.15.89:0/511579686 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{prefix=config set, name=client_mount_timeout}]=0 v32)=0 v32) ==== 119+0+0 (secure 0 0 0) 0x7f92980ae1e0 con 0x7f92a8075cf0 2024-09-21T03:31:13.140 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:12.605+0000 7f92adf7d640 1 -- 172.21.15.89:0/511579686 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f92900b6480 msgr2=0x7f92900b8990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:31:13.140 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:12.605+0000 7f92adf7d640 1 --2- 172.21.15.89:0/511579686 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f92900b6480 0x7f92900b8990 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f929c009bd0 tx=0x7f929c005f90 comp rx=0 tx=0).stop 2024-09-21T03:31:13.140 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:12.605+0000 7f92adf7d640 1 -- 172.21.15.89:0/511579686 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f92a8075cf0 msgr2=0x7f92a819c0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:31:13.140 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:12.605+0000 7f92adf7d640 1 --2- 172.21.15.89:0/511579686 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f92a8075cf0 0x7f92a819c0b0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f929800e9f0 tx=0x7f929800edf0 comp rx=0 tx=0).stop 2024-09-21T03:31:13.140 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:12.605+0000 7f92adf7d640 1 -- 172.21.15.89:0/511579686 shutdown_connections 2024-09-21T03:31:13.140 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:12.605+0000 7f92adf7d640 1 --2- 172.21.15.89:0/511579686 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f92900b6480 0x7f92900b8990 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:13.140 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:12.606+0000 7f92adf7d640 1 --2- 172.21.15.89:0/511579686 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f92a810b6f0 0x7f92a819d2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:13.140 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:12.606+0000 7f92adf7d640 1 --2- 172.21.15.89:0/511579686 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f92a80770e0 0x7f92a819c720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:13.140 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:12.606+0000 7f92adf7d640 1 --2- 172.21.15.89:0/511579686 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f92a8075cf0 0x7f92a819c0b0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:13.140 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:12.606+0000 7f92adf7d640 1 -- 172.21.15.89:0/511579686 >> 172.21.15.89:0/511579686 conn(0x7f92a80ff0c0 msgr2=0x7f92a81076d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:31:13.140 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:12.606+0000 7f92adf7d640 1 -- 172.21.15.89:0/511579686 shutdown_connections 2024-09-21T03:31:13.140 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:12.606+0000 7f92adf7d640 1 -- 172.21.15.89:0/511579686 wait complete. 2024-09-21T03:31:13.211 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:31:12 smithi114 ceph-mon[34140]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:31:13.211 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:31:12 smithi114 ceph-mon[34140]: from='client.? 172.21.15.89:0/578512476' entity='client.admin' 2024-09-21T03:31:13.211 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:31:12 smithi114 ceph-mon[34140]: from='client.? 172.21.15.89:0/511579686' entity='client.admin' 2024-09-21T03:31:13.211 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:31:12 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:31:13.250 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:31:12 smithi110 ceph-mon[34000]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:31:13.250 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:31:12 smithi110 ceph-mon[34000]: from='client.? 172.21.15.89:0/578512476' entity='client.admin' 2024-09-21T03:31:13.250 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:31:12 smithi110 ceph-mon[34000]: from='client.? 172.21.15.89:0/511579686' entity='client.admin' 2024-09-21T03:31:13.250 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:31:12 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:31:13.474 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:13.471+0000 7f3053aa9640 1 -- 172.21.15.89:0/502093231 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f304c105180 msgr2=0x7f304c10bb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:31:13.474 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:13.471+0000 7f3053aa9640 1 --2- 172.21.15.89:0/502093231 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f304c105180 0x7f304c10bb20 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f3044009ca0 tx=0x7f304402a950 comp rx=0 tx=0).stop 2024-09-21T03:31:13.474 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:13.471+0000 7f3053aa9640 1 -- 172.21.15.89:0/502093231 shutdown_connections 2024-09-21T03:31:13.474 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:13.471+0000 7f3053aa9640 1 --2- 172.21.15.89:0/502093231 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f304c105180 0x7f304c10bb20 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:13.475 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:13.471+0000 7f3053aa9640 1 --2- 172.21.15.89:0/502093231 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f304c1047e0 0x7f304c104bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:13.475 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:13.471+0000 7f3053aa9640 1 --2- 172.21.15.89:0/502093231 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f304c103750 0x7f304c103bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:13.475 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:13.471+0000 7f3053aa9640 1 -- 172.21.15.89:0/502093231 >> 172.21.15.89:0/502093231 conn(0x7f304c0ff0c0 msgr2=0x7f304c101570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:31:13.475 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:13.472+0000 7f3053aa9640 1 -- 172.21.15.89:0/502093231 shutdown_connections 2024-09-21T03:31:13.475 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:13.472+0000 7f3053aa9640 1 -- 172.21.15.89:0/502093231 wait complete. 2024-09-21T03:31:13.475 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:13.472+0000 7f3053aa9640 1 Processor -- start 2024-09-21T03:31:13.475 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:13.472+0000 7f3053aa9640 1 -- start start 2024-09-21T03:31:13.475 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:13.472+0000 7f3053aa9640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f304c103750 0x7f304c199e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:13.475 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:13.472+0000 7f3053aa9640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f304c1047e0 0x7f304c19a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:13.475 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:13.472+0000 7f3053aa9640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f304c105180 0x7f304c19b070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:13.475 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:13.472+0000 7f3053aa9640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f304c10f230 con 0x7f304c1047e0 2024-09-21T03:31:13.476 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:13.472+0000 7f3053aa9640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f304c10f0b0 con 0x7f304c103750 2024-09-21T03:31:13.476 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:13.472+0000 7f3053aa9640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f304c10f3b0 con 0x7f304c105180 2024-09-21T03:31:13.476 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:13.473+0000 7f3051044640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f304c1047e0 0x7f304c19a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:13.476 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:13.473+0000 7f3051044640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f304c1047e0 0x7f304c19a4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:47014/0 (socket says 172.21.15.89:47014) 2024-09-21T03:31:13.476 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:13.473+0000 7f3051044640 1 -- 172.21.15.89:0/1629577295 learned_addr learned my addr 172.21.15.89:0/1629577295 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-21T03:31:13.476 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:13.473+0000 7f3051845640 1 --2- 172.21.15.89:0/1629577295 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f304c103750 0x7f304c199e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:13.476 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:13.473+0000 7f3052046640 1 --2- 172.21.15.89:0/1629577295 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f304c105180 0x7f304c19b070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:13.476 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:13.473+0000 7f3051044640 1 -- 172.21.15.89:0/1629577295 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f304c105180 msgr2=0x7f304c19b070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:31:13.476 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:13.473+0000 7f3051044640 1 --2- 172.21.15.89:0/1629577295 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f304c105180 0x7f304c19b070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:13.476 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:13.473+0000 7f3051044640 1 -- 172.21.15.89:0/1629577295 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f304c103750 msgr2=0x7f304c199e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:31:13.476 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:13.473+0000 7f3051044640 1 --2- 172.21.15.89:0/1629577295 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f304c103750 0x7f304c199e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:13.476 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:13.473+0000 7f3051044640 1 -- 172.21.15.89:0/1629577295 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f304c19b890 con 0x7f304c1047e0 2024-09-21T03:31:13.477 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:13.473+0000 7f3051044640 1 --2- 172.21.15.89:0/1629577295 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f304c1047e0 0x7f304c19a4d0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f303c00d950 tx=0x7f303c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:31:13.477 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:13.473+0000 7f3042ffd640 1 -- 172.21.15.89:0/1629577295 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f303c017030 con 0x7f304c1047e0 2024-09-21T03:31:13.477 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:13.473+0000 7f3053aa9640 1 -- 172.21.15.89:0/1629577295 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f304c19bb80 con 0x7f304c1047e0 2024-09-21T03:31:13.477 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:13.473+0000 7f3042ffd640 1 -- 172.21.15.89:0/1629577295 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f303c004020 con 0x7f304c1047e0 2024-09-21T03:31:13.478 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:13.473+0000 7f3042ffd640 1 -- 172.21.15.89:0/1629577295 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f303c0055d0 con 0x7f304c1047e0 2024-09-21T03:31:13.478 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:13.473+0000 7f3053aa9640 1 -- 172.21.15.89:0/1629577295 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f304c1a2280 con 0x7f304c1047e0 2024-09-21T03:31:13.478 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:13.474+0000 7f3053aa9640 1 -- 172.21.15.89:0/1629577295 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f304c103bc0 con 0x7f304c1047e0 2024-09-21T03:31:13.478 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:13.476+0000 7f3042ffd640 1 -- 172.21.15.89:0/1629577295 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 16) ==== 154456+0+0 (secure 0 0 0) 0x7f303c009a30 con 0x7f304c1047e0 2024-09-21T03:31:13.479 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:13.477+0000 7f3042ffd640 1 --2- 172.21.15.89:0/1629577295 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f30380b6480 0x7f30380b8990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:13.479 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:13.477+0000 7f3051845640 1 --2- 172.21.15.89:0/1629577295 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f30380b6480 0x7f30380b8990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:13.479 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:13.477+0000 7f3042ffd640 1 -- 172.21.15.89:0/1629577295 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f303c0e55f0 con 0x7f304c1047e0 2024-09-21T03:31:13.479 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:13.477+0000 7f3051845640 1 --2- 172.21.15.89:0/1629577295 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f30380b6480 0x7f30380b8990 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f304800bbe0 tx=0x7f304800b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:31:13.481 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:13.480+0000 7f3042ffd640 1 -- 172.21.15.89:0/1629577295 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f303c0adb50 con 0x7f304c1047e0 2024-09-21T03:31:13.668 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:13.667+0000 7f3053aa9640 1 -- 172.21.15.89:0/1629577295 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command([{prefix=config set, name=mds_verify_scatter}] v 0) -- 0x7f304c02e640 con 0x7f304c1047e0 2024-09-21T03:31:13.672 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:13.671+0000 7f3042ffd640 1 -- 172.21.15.89:0/1629577295 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_verify_scatter}]=0 v33)=0 v33) ==== 119+0+0 (secure 0 0 0) 0x7f303c0b1f30 con 0x7f304c1047e0 2024-09-21T03:31:13.675 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:13.673+0000 7f3053aa9640 1 -- 172.21.15.89:0/1629577295 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f30380b6480 msgr2=0x7f30380b8990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:31:13.675 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:13.673+0000 7f3053aa9640 1 --2- 172.21.15.89:0/1629577295 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f30380b6480 0x7f30380b8990 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f304800bbe0 tx=0x7f304800b390 comp rx=0 tx=0).stop 2024-09-21T03:31:13.675 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:13.673+0000 7f3053aa9640 1 -- 172.21.15.89:0/1629577295 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f304c1047e0 msgr2=0x7f304c19a4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:31:13.675 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:13.673+0000 7f3053aa9640 1 --2- 172.21.15.89:0/1629577295 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f304c1047e0 0x7f304c19a4d0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f303c00d950 tx=0x7f303c00de10 comp rx=0 tx=0).stop 2024-09-21T03:31:13.675 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:13.673+0000 7f3053aa9640 1 -- 172.21.15.89:0/1629577295 shutdown_connections 2024-09-21T03:31:13.675 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:13.673+0000 7f3053aa9640 1 --2- 172.21.15.89:0/1629577295 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f30380b6480 0x7f30380b8990 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:13.675 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:13.673+0000 7f3053aa9640 1 --2- 172.21.15.89:0/1629577295 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f304c105180 0x7f304c19b070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:13.675 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:13.673+0000 7f3053aa9640 1 --2- 172.21.15.89:0/1629577295 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f304c1047e0 0x7f304c19a4d0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:13.675 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:13.673+0000 7f3053aa9640 1 --2- 172.21.15.89:0/1629577295 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f304c103750 0x7f304c199e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:13.675 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:13.673+0000 7f3053aa9640 1 -- 172.21.15.89:0/1629577295 >> 172.21.15.89:0/1629577295 conn(0x7f304c0ff0c0 msgr2=0x7f304c107ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:31:13.675 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:13.673+0000 7f3053aa9640 1 -- 172.21.15.89:0/1629577295 shutdown_connections 2024-09-21T03:31:13.676 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:13.673+0000 7f3053aa9640 1 -- 172.21.15.89:0/1629577295 wait complete. 2024-09-21T03:31:14.061 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:31:13 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:31:14.061 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:31:13 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:31:14.061 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:31:13 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:31:14.061 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:31:13 smithi089 ceph-mon[30928]: from='client.? 172.21.15.89:0/1629577295' entity='client.admin' 2024-09-21T03:31:14.210 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:31:13 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:31:14.211 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:31:13 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:31:14.211 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:31:13 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:31:14.211 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:31:13 smithi114 ceph-mon[34140]: from='client.? 172.21.15.89:0/1629577295' entity='client.admin' 2024-09-21T03:31:14.250 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:31:13 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:31:14.250 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:31:13 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:31:14.250 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:31:13 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:31:14.250 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:31:13 smithi110 ceph-mon[34000]: from='client.? 172.21.15.89:0/1629577295' entity='client.admin' 2024-09-21T03:31:14.356 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '7c374098-77c9-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mds', 'osd_op_complaint_time', '180'], check_status=True, hostname='smithi089') 2024-09-21T03:31:14.749 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '7c374098-77c9-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mds', 'rados_mon_op_timeout', '900'], check_status=True, hostname='smithi089') 2024-09-21T03:31:14.749 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '7c374098-77c9-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mds', 'rados_osd_op_timeout', '900'], check_status=True, hostname='smithi089') 2024-09-21T03:31:14.749 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '7c374098-77c9-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mgr', 'client_mount_timeout', '30'], check_status=True, hostname='smithi089') 2024-09-21T03:31:14.749 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '7c374098-77c9-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mgr', 'debug_client', '20'], check_status=True, hostname='smithi089') 2024-09-21T03:31:14.750 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '7c374098-77c9-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mgr', 'debug_mgr', '20'], check_status=True, hostname='smithi089') 2024-09-21T03:31:14.750 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '7c374098-77c9-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mgr', 'debug_ms', '1'], check_status=True, hostname='smithi089') 2024-09-21T03:31:14.750 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '7c374098-77c9-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mgr', 'mon_warn_on_pool_no_app', 'False'], check_status=True, hostname='smithi089') 2024-09-21T03:31:14.750 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '7c374098-77c9-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mon', 'mon_op_complaint_time', '120'], check_status=True, hostname='smithi089') 2024-09-21T03:31:14.750 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '7c374098-77c9-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'osd', 'osd_op_complaint_time', '180'], check_status=True, hostname='smithi089') 2024-09-21T03:31:14.750 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7c374098-77c9-11ef-baf5-efdc52797490 -- ceph config dump 2024-09-21T03:31:14.990 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/7c374098-77c9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:31:15.061 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:31:14 smithi089 ceph-mon[30928]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:31:15.211 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:31:14 smithi114 ceph-mon[34140]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:31:15.250 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:31:14 smithi110 ceph-mon[34000]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:31:15.441 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:15.437+0000 7f56ce274640 1 -- 172.21.15.89:0/4189591864 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f56c81037e0 msgr2=0x7f56c8103c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:31:15.441 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:15.437+0000 7f56ce274640 1 --2- 172.21.15.89:0/4189591864 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f56c81037e0 0x7f56c8103c50 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f56b8009900 tx=0x7f56b802a480 comp rx=0 tx=0).stop 2024-09-21T03:31:15.442 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:15.437+0000 7f56ce274640 1 -- 172.21.15.89:0/4189591864 shutdown_connections 2024-09-21T03:31:15.442 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:15.437+0000 7f56ce274640 1 --2- 172.21.15.89:0/4189591864 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f56c8105210 0x7f56c810bbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:15.442 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:15.437+0000 7f56ce274640 1 --2- 172.21.15.89:0/4189591864 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f56c8104870 0x7f56c8104c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:15.442 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:15.437+0000 7f56ce274640 1 --2- 172.21.15.89:0/4189591864 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f56c81037e0 0x7f56c8103c50 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:15.442 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:15.437+0000 7f56ce274640 1 -- 172.21.15.89:0/4189591864 >> 172.21.15.89:0/4189591864 conn(0x7f56c80ff150 msgr2=0x7f56c8101600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:31:15.442 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:15.437+0000 7f56ce274640 1 -- 172.21.15.89:0/4189591864 shutdown_connections 2024-09-21T03:31:15.442 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:15.437+0000 7f56ce274640 1 -- 172.21.15.89:0/4189591864 wait complete. 2024-09-21T03:31:15.442 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:15.437+0000 7f56ce274640 1 Processor -- start 2024-09-21T03:31:15.442 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:15.437+0000 7f56ce274640 1 -- start start 2024-09-21T03:31:15.442 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:15.438+0000 7f56ce274640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f56c81037e0 0x7f56c8199f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:15.442 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:15.438+0000 7f56ce274640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f56c8104870 0x7f56c819a580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:15.442 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:15.438+0000 7f56ce274640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f56c8105210 0x7f56c819b120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:15.442 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:15.438+0000 7f56ce274640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f56c810f2b0 con 0x7f56c8104870 2024-09-21T03:31:15.443 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:15.438+0000 7f56ce274640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f56c810f130 con 0x7f56c81037e0 2024-09-21T03:31:15.443 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:15.438+0000 7f56ce274640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f56c810f430 con 0x7f56c8105210 2024-09-21T03:31:15.443 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:15.438+0000 7f56c77fe640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f56c8104870 0x7f56c819a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:15.443 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:15.438+0000 7f56c77fe640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f56c8104870 0x7f56c819a580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:47028/0 (socket says 172.21.15.89:47028) 2024-09-21T03:31:15.443 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:15.438+0000 7f56c77fe640 1 -- 172.21.15.89:0/12117581 learned_addr learned my addr 172.21.15.89:0/12117581 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-21T03:31:15.443 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:15.438+0000 7f56c7fff640 1 --2- 172.21.15.89:0/12117581 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f56c81037e0 0x7f56c8199f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:15.443 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:15.438+0000 7f56cc811640 1 --2- 172.21.15.89:0/12117581 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f56c8105210 0x7f56c819b120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:15.443 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:15.438+0000 7f56c77fe640 1 -- 172.21.15.89:0/12117581 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f56c8105210 msgr2=0x7f56c819b120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:31:15.443 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:15.438+0000 7f56c77fe640 1 --2- 172.21.15.89:0/12117581 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f56c8105210 0x7f56c819b120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:15.443 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:15.438+0000 7f56c77fe640 1 -- 172.21.15.89:0/12117581 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f56c81037e0 msgr2=0x7f56c8199f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:31:15.443 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:15.438+0000 7f56c77fe640 1 --2- 172.21.15.89:0/12117581 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f56c81037e0 0x7f56c8199f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:15.443 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:15.438+0000 7f56c77fe640 1 -- 172.21.15.89:0/12117581 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56c819b940 con 0x7f56c8104870 2024-09-21T03:31:15.443 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:15.439+0000 7f56c77fe640 1 --2- 172.21.15.89:0/12117581 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f56c8104870 0x7f56c819a580 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f56bc00e9f0 tx=0x7f56bc00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:31:15.443 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:15.439+0000 7f56c57fa640 1 -- 172.21.15.89:0/12117581 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56bc00cd80 con 0x7f56c8104870 2024-09-21T03:31:15.444 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:15.439+0000 7f56c57fa640 1 -- 172.21.15.89:0/12117581 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f56bc0054b0 con 0x7f56c8104870 2024-09-21T03:31:15.444 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:15.439+0000 7f56ce274640 1 -- 172.21.15.89:0/12117581 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f56c819bc30 con 0x7f56c8104870 2024-09-21T03:31:15.444 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:15.439+0000 7f56c57fa640 1 -- 172.21.15.89:0/12117581 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56bc010430 con 0x7f56c8104870 2024-09-21T03:31:15.444 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:15.439+0000 7f56ce274640 1 -- 172.21.15.89:0/12117581 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56c80757c0 con 0x7f56c8104870 2024-09-21T03:31:15.444 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:15.440+0000 7f56ce274640 1 -- 172.21.15.89:0/12117581 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f56c8103c50 con 0x7f56c8104870 2024-09-21T03:31:15.445 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:15.442+0000 7f56c57fa640 1 -- 172.21.15.89:0/12117581 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 16) ==== 154456+0+0 (secure 0 0 0) 0x7f56bc0046c0 con 0x7f56c8104870 2024-09-21T03:31:15.445 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:15.442+0000 7f56c57fa640 1 --2- 172.21.15.89:0/12117581 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f56b00b6500 0x7f56b00b8a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:15.445 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:15.442+0000 7f56c7fff640 1 --2- 172.21.15.89:0/12117581 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f56b00b6500 0x7f56b00b8a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:15.445 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:15.442+0000 7f56c57fa640 1 -- 172.21.15.89:0/12117581 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f56bc0e5c50 con 0x7f56c8104870 2024-09-21T03:31:15.445 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:15.443+0000 7f56c7fff640 1 --2- 172.21.15.89:0/12117581 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f56b00b6500 0x7f56b00b8a10 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f56b8006010 tx=0x7f56b8005720 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:31:15.447 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:15.445+0000 7f56c57fa640 1 -- 172.21.15.89:0/12117581 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f56bc056710 con 0x7f56c8104870 2024-09-21T03:31:15.637 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:15.633+0000 7f56ce274640 1 -- 172.21.15.89:0/12117581 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "config dump"} v 0) -- 0x7f56c802e9b0 con 0x7f56c8104870 2024-09-21T03:31:15.637 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:15.634+0000 7f56c57fa640 1 -- 172.21.15.89:0/12117581 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "config dump"}]=0 v33) ==== 59+0+21112 (secure 0 0 0) 0x7f56bc0a9030 con 0x7f56c8104870 2024-09-21T03:31:15.638 INFO:teuthology.orchestra.run.smithi089.stdout:WHO MASK LEVEL OPTION VALUE RO 2024-09-21T03:31:15.638 INFO:teuthology.orchestra.run.smithi089.stdout:global dev auth_debug true 2024-09-21T03:31:15.638 INFO:teuthology.orchestra.run.smithi089.stdout:global basic container_image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 * 2024-09-21T03:31:15.638 INFO:teuthology.orchestra.run.smithi089.stdout:global dev debug_asserts_on_shutdown true 2024-09-21T03:31:15.639 INFO:teuthology.orchestra.run.smithi089.stdout:global basic log_to_file true 2024-09-21T03:31:15.639 INFO:teuthology.orchestra.run.smithi089.stdout:global basic log_to_journald false 2024-09-21T03:31:15.639 INFO:teuthology.orchestra.run.smithi089.stdout:global basic log_to_stderr false 2024-09-21T03:31:15.639 INFO:teuthology.orchestra.run.smithi089.stdout:global advanced mon_allow_pool_delete true 2024-09-21T03:31:15.639 INFO:teuthology.orchestra.run.smithi089.stdout:global advanced mon_clock_drift_allowed 1.000000 2024-09-21T03:31:15.639 INFO:teuthology.orchestra.run.smithi089.stdout:global advanced mon_cluster_log_to_file true 2024-09-21T03:31:15.640 INFO:teuthology.orchestra.run.smithi089.stdout:global advanced mon_max_pg_per_osd 10000 2024-09-21T03:31:15.640 INFO:teuthology.orchestra.run.smithi089.stdout:global advanced mon_pg_warn_max_object_skew 0.000000 2024-09-21T03:31:15.640 INFO:teuthology.orchestra.run.smithi089.stdout:global advanced mon_warn_on_crush_straw_calc_version_zero false 2024-09-21T03:31:15.640 INFO:teuthology.orchestra.run.smithi089.stdout:global advanced mon_warn_on_legacy_crush_tunables false 2024-09-21T03:31:15.640 INFO:teuthology.orchestra.run.smithi089.stdout:global advanced mon_warn_on_osd_down_out_interval_zero false 2024-09-21T03:31:15.640 INFO:teuthology.orchestra.run.smithi089.stdout:global dev mon_warn_on_pool_pg_num_not_power_of_two false 2024-09-21T03:31:15.640 INFO:teuthology.orchestra.run.smithi089.stdout:global advanced mon_warn_on_too_few_osds false 2024-09-21T03:31:15.641 INFO:teuthology.orchestra.run.smithi089.stdout:global dev ms_die_on_bug true 2024-09-21T03:31:15.641 INFO:teuthology.orchestra.run.smithi089.stdout:global dev ms_die_on_old_message true 2024-09-21T03:31:15.641 INFO:teuthology.orchestra.run.smithi089.stdout:global advanced osd_pool_default_erasure_code_profile plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-09-21T03:31:15.641 INFO:teuthology.orchestra.run.smithi089.stdout:global advanced osd_pool_default_pg_autoscale_mode off 2024-09-21T03:31:15.641 INFO:teuthology.orchestra.run.smithi089.stdout:mon advanced auth_allow_insecure_global_id_reclaim false 2024-09-21T03:31:15.641 INFO:teuthology.orchestra.run.smithi089.stdout:mon advanced auth_mon_ticket_ttl 660.000000 2024-09-21T03:31:15.641 INFO:teuthology.orchestra.run.smithi089.stdout:mon advanced auth_service_ticket_ttl 240.000000 2024-09-21T03:31:15.642 INFO:teuthology.orchestra.run.smithi089.stdout:mon advanced debug_mon 20/20 2024-09-21T03:31:15.642 INFO:teuthology.orchestra.run.smithi089.stdout:mon advanced debug_ms 1/1 2024-09-21T03:31:15.642 INFO:teuthology.orchestra.run.smithi089.stdout:mon advanced debug_paxos 20/20 2024-09-21T03:31:15.642 INFO:teuthology.orchestra.run.smithi089.stdout:mon advanced mon_data_avail_warn 5 2024-09-21T03:31:15.643 INFO:teuthology.orchestra.run.smithi089.stdout:mon advanced mon_down_mkfs_grace 300 2024-09-21T03:31:15.643 INFO:teuthology.orchestra.run.smithi089.stdout:mon advanced mon_mgr_mkfs_grace 240 2024-09-21T03:31:15.643 INFO:teuthology.orchestra.run.smithi089.stdout:mon advanced mon_op_complaint_time 120 2024-09-21T03:31:15.643 INFO:teuthology.orchestra.run.smithi089.stdout:mon dev mon_osd_prime_pg_temp true 2024-09-21T03:31:15.643 INFO:teuthology.orchestra.run.smithi089.stdout:mon advanced mon_osd_reporter_subtree_level osd 2024-09-21T03:31:15.643 INFO:teuthology.orchestra.run.smithi089.stdout:mon advanced mon_reweight_min_bytes_per_osd 10 2024-09-21T03:31:15.643 INFO:teuthology.orchestra.run.smithi089.stdout:mon advanced mon_reweight_min_pgs_per_osd 4 2024-09-21T03:31:15.643 INFO:teuthology.orchestra.run.smithi089.stdout:mon advanced mon_warn_on_insecure_global_id_reclaim false 2024-09-21T03:31:15.643 INFO:teuthology.orchestra.run.smithi089.stdout:mon advanced mon_warn_on_insecure_global_id_reclaim_allowed false 2024-09-21T03:31:15.643 INFO:teuthology.orchestra.run.smithi089.stdout:mon advanced public_network 172.21.0.0/20 * 2024-09-21T03:31:15.643 INFO:teuthology.orchestra.run.smithi089.stdout:mgr advanced cephsqlite_lock_renewal_timeout 900000 2024-09-21T03:31:15.643 INFO:teuthology.orchestra.run.smithi089.stdout:mgr advanced client_mount_timeout 30 2024-09-21T03:31:15.643 INFO:teuthology.orchestra.run.smithi089.stdout:mgr advanced debug_client 20/20 2024-09-21T03:31:15.643 INFO:teuthology.orchestra.run.smithi089.stdout:mgr advanced debug_mgr 20/20 2024-09-21T03:31:15.644 INFO:teuthology.orchestra.run.smithi089.stdout:mgr advanced debug_ms 1/1 2024-09-21T03:31:15.644 INFO:teuthology.orchestra.run.smithi089.stdout:mgr advanced mgr/cephadm/allow_ptrace true * 2024-09-21T03:31:15.644 INFO:teuthology.orchestra.run.smithi089.stdout:mgr advanced mgr/cephadm/container_init True * 2024-09-21T03:31:15.644 INFO:teuthology.orchestra.run.smithi089.stdout:mgr advanced mgr/cephadm/migration_current 7 * 2024-09-21T03:31:15.644 INFO:teuthology.orchestra.run.smithi089.stdout:mgr advanced mgr/dashboard/ssl_server_port 8443 * 2024-09-21T03:31:15.644 INFO:teuthology.orchestra.run.smithi089.stdout:mgr advanced mgr/orchestrator/orchestrator cephadm 2024-09-21T03:31:15.644 INFO:teuthology.orchestra.run.smithi089.stdout:mgr advanced mon_reweight_min_bytes_per_osd 10 2024-09-21T03:31:15.644 INFO:teuthology.orchestra.run.smithi089.stdout:mgr advanced mon_reweight_min_pgs_per_osd 4 2024-09-21T03:31:15.644 INFO:teuthology.orchestra.run.smithi089.stdout:mgr dev mon_warn_on_pool_no_app false 2024-09-21T03:31:15.644 INFO:teuthology.orchestra.run.smithi089.stdout:osd dev bdev_debug_aio true 2024-09-21T03:31:15.644 INFO:teuthology.orchestra.run.smithi089.stdout:osd dev bluestore_block_size 96636764160 * 2024-09-21T03:31:15.644 INFO:teuthology.orchestra.run.smithi089.stdout:osd advanced bluestore_compression_mode aggressive 2024-09-21T03:31:15.644 INFO:teuthology.orchestra.run.smithi089.stdout:osd dev bluestore_fsck_on_mount true 2024-09-21T03:31:15.644 INFO:teuthology.orchestra.run.smithi089.stdout:osd advanced debug_bluefs 20/20 2024-09-21T03:31:15.644 INFO:teuthology.orchestra.run.smithi089.stdout:osd advanced debug_bluestore 20/20 2024-09-21T03:31:15.645 INFO:teuthology.orchestra.run.smithi089.stdout:osd advanced debug_ms 1/1 2024-09-21T03:31:15.645 INFO:teuthology.orchestra.run.smithi089.stdout:osd advanced debug_osd 20/20 2024-09-21T03:31:15.645 INFO:teuthology.orchestra.run.smithi089.stdout:osd advanced debug_rocksdb 10/10 2024-09-21T03:31:15.645 INFO:teuthology.orchestra.run.smithi089.stdout:osd dev osd_debug_misdirected_ops true 2024-09-21T03:31:15.645 INFO:teuthology.orchestra.run.smithi089.stdout:osd dev osd_debug_op_order true 2024-09-21T03:31:15.645 INFO:teuthology.orchestra.run.smithi089.stdout:osd dev osd_debug_pg_log_writeout true 2024-09-21T03:31:15.645 INFO:teuthology.orchestra.run.smithi089.stdout:osd dev osd_debug_shutdown true 2024-09-21T03:31:15.645 INFO:teuthology.orchestra.run.smithi089.stdout:osd dev osd_debug_verify_cached_snaps true 2024-09-21T03:31:15.645 INFO:teuthology.orchestra.run.smithi089.stdout:osd dev osd_debug_verify_missing_on_start true 2024-09-21T03:31:15.645 INFO:teuthology.orchestra.run.smithi089.stdout:osd dev osd_debug_verify_stray_on_activate true 2024-09-21T03:31:15.645 INFO:teuthology.orchestra.run.smithi089.stdout:osd advanced osd_deep_scrub_update_digest_min_age 30 2024-09-21T03:31:15.645 INFO:teuthology.orchestra.run.smithi089.stdout:osd advanced osd_failsafe_full_ratio 0.950000 2024-09-21T03:31:15.645 INFO:teuthology.orchestra.run.smithi089.stdout:osd advanced osd_max_omap_entries_per_request 10000 2024-09-21T03:31:15.645 INFO:teuthology.orchestra.run.smithi089.stdout:osd advanced osd_mclock_profile high_recovery_ops 2024-09-21T03:31:15.646 INFO:teuthology.orchestra.run.smithi089.stdout:osd advanced osd_memory_target_autotune true 2024-09-21T03:31:15.646 INFO:teuthology.orchestra.run.smithi089.stdout:osd advanced osd_objectstore bluestore * 2024-09-21T03:31:15.646 INFO:teuthology.orchestra.run.smithi089.stdout:osd advanced osd_op_complaint_time 180.000000 2024-09-21T03:31:15.646 INFO:teuthology.orchestra.run.smithi089.stdout:osd advanced osd_op_queue debug_random * 2024-09-21T03:31:15.646 INFO:teuthology.orchestra.run.smithi089.stdout:osd advanced osd_op_queue_cut_off debug_random * 2024-09-21T03:31:15.646 INFO:teuthology.orchestra.run.smithi089.stdout:osd advanced osd_recover_clone_overlap true 2024-09-21T03:31:15.646 INFO:teuthology.orchestra.run.smithi089.stdout:osd advanced osd_recovery_max_chunk 1048576 2024-09-21T03:31:15.646 INFO:teuthology.orchestra.run.smithi089.stdout:osd advanced osd_scrub_load_threshold 5.000000 2024-09-21T03:31:15.646 INFO:teuthology.orchestra.run.smithi089.stdout:osd advanced osd_scrub_max_interval 600.000000 2024-09-21T03:31:15.646 INFO:teuthology.orchestra.run.smithi089.stdout:osd advanced osd_shutdown_pgref_assert true 2024-09-21T03:31:15.646 INFO:teuthology.orchestra.run.smithi089.stdout:mds advanced debug_mds 20/20 2024-09-21T03:31:15.646 INFO:teuthology.orchestra.run.smithi089.stdout:mds advanced debug_mds_balancer 20/20 2024-09-21T03:31:15.646 INFO:teuthology.orchestra.run.smithi089.stdout:mds advanced debug_ms 2/2 2024-09-21T03:31:15.646 INFO:teuthology.orchestra.run.smithi089.stdout:mds advanced mds_bal_fragment_size_max 10000 2024-09-21T03:31:15.647 INFO:teuthology.orchestra.run.smithi089.stdout:mds advanced mds_bal_merge_size 5 2024-09-21T03:31:15.647 INFO:teuthology.orchestra.run.smithi089.stdout:mds advanced mds_bal_split_bits 3 2024-09-21T03:31:15.647 INFO:teuthology.orchestra.run.smithi089.stdout:mds advanced mds_bal_split_size 100 2024-09-21T03:31:15.647 INFO:teuthology.orchestra.run.smithi089.stdout:mds dev mds_debug_frag true 2024-09-21T03:31:15.647 INFO:teuthology.orchestra.run.smithi089.stdout:mds dev mds_debug_scatterstat true 2024-09-21T03:31:15.647 INFO:teuthology.orchestra.run.smithi089.stdout:mds advanced mds_export_ephemeral_random_max 0.100000 2024-09-21T03:31:15.647 INFO:teuthology.orchestra.run.smithi089.stdout:mds advanced mds_op_complaint_time 180.000000 2024-09-21T03:31:15.647 INFO:teuthology.orchestra.run.smithi089.stdout:mds dev mds_verify_scatter true 2024-09-21T03:31:15.647 INFO:teuthology.orchestra.run.smithi089.stdout:mds advanced osd_op_complaint_time 180.000000 2024-09-21T03:31:15.647 INFO:teuthology.orchestra.run.smithi089.stdout:mds advanced rados_mon_op_timeout 900 2024-09-21T03:31:15.647 INFO:teuthology.orchestra.run.smithi089.stdout:mds advanced rados_osd_op_timeout 900 2024-09-21T03:31:15.647 INFO:teuthology.orchestra.run.smithi089.stdout:client advanced client_mount_timeout 600 2024-09-21T03:31:15.647 INFO:teuthology.orchestra.run.smithi089.stdout:client advanced debug_client 20/20 2024-09-21T03:31:15.648 INFO:teuthology.orchestra.run.smithi089.stdout:client advanced debug_ms 1/1 2024-09-21T03:31:15.648 INFO:teuthology.orchestra.run.smithi089.stdout:client advanced rados_mon_op_timeout 900 2024-09-21T03:31:15.648 INFO:teuthology.orchestra.run.smithi089.stdout:client advanced rados_osd_op_timeout 900 2024-09-21T03:31:15.648 INFO:teuthology.orchestra.run.smithi089.stdout:client.rgw advanced rgw_cache_enabled true 2024-09-21T03:31:15.648 INFO:teuthology.orchestra.run.smithi089.stdout:client.rgw advanced rgw_enable_ops_log true 2024-09-21T03:31:15.648 INFO:teuthology.orchestra.run.smithi089.stdout:client.rgw advanced rgw_enable_usage_log true 2024-09-21T03:31:15.648 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:15.637+0000 7f56ce274640 1 -- 172.21.15.89:0/12117581 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f56b00b6500 msgr2=0x7f56b00b8a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:31:15.649 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:15.637+0000 7f56ce274640 1 --2- 172.21.15.89:0/12117581 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f56b00b6500 0x7f56b00b8a10 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f56b8006010 tx=0x7f56b8005720 comp rx=0 tx=0).stop 2024-09-21T03:31:15.649 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:15.637+0000 7f56ce274640 1 -- 172.21.15.89:0/12117581 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f56c8104870 msgr2=0x7f56c819a580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:31:15.649 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:15.637+0000 7f56ce274640 1 --2- 172.21.15.89:0/12117581 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f56c8104870 0x7f56c819a580 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f56bc00e9f0 tx=0x7f56bc00edf0 comp rx=0 tx=0).stop 2024-09-21T03:31:15.649 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:15.637+0000 7f56ce274640 1 -- 172.21.15.89:0/12117581 shutdown_connections 2024-09-21T03:31:15.649 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:15.637+0000 7f56ce274640 1 --2- 172.21.15.89:0/12117581 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f56b00b6500 0x7f56b00b8a10 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:15.649 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:15.637+0000 7f56ce274640 1 --2- 172.21.15.89:0/12117581 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f56c8105210 0x7f56c819b120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:15.649 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:15.637+0000 7f56ce274640 1 --2- 172.21.15.89:0/12117581 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f56c8104870 0x7f56c819a580 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:15.649 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:15.637+0000 7f56ce274640 1 --2- 172.21.15.89:0/12117581 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f56c81037e0 0x7f56c8199f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:15.649 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:15.637+0000 7f56ce274640 1 -- 172.21.15.89:0/12117581 >> 172.21.15.89:0/12117581 conn(0x7f56c80ff150 msgr2=0x7f56c8107b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:31:15.649 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:15.637+0000 7f56ce274640 1 -- 172.21.15.89:0/12117581 shutdown_connections 2024-09-21T03:31:15.649 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:15.637+0000 7f56ce274640 1 -- 172.21.15.89:0/12117581 wait complete. 2024-09-21T03:31:16.210 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:31:15 smithi114 ceph-mon[34140]: from='client.? 172.21.15.89:0/12117581' entity='client.admin' cmd={"prefix": "config dump"} : dispatch 2024-09-21T03:31:16.246 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:31:15 smithi089 ceph-mon[30928]: from='client.? 172.21.15.89:0/12117581' entity='client.admin' cmd={"prefix": "config dump"} : dispatch 2024-09-21T03:31:16.249 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:31:15 smithi110 ceph-mon[34000]: from='client.? 172.21.15.89:0/12117581' entity='client.admin' cmd={"prefix": "config dump"} : dispatch 2024-09-21T03:31:16.276 INFO:tasks.cephadm:Deploying OSDs... 2024-09-21T03:31:16.276 DEBUG:teuthology.orchestra.run.smithi089:> set -ex 2024-09-21T03:31:16.276 DEBUG:teuthology.orchestra.run.smithi089:> dd if=/scratch_devs of=/dev/stdout 2024-09-21T03:31:16.292 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-21T03:31:16.292 DEBUG:teuthology.orchestra.run.smithi089:> stat /dev/vg_nvme/lv_1 2024-09-21T03:31:16.348 INFO:teuthology.orchestra.run.smithi089.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-21T03:31:16.348 INFO:teuthology.orchestra.run.smithi089.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-21T03:31:16.348 INFO:teuthology.orchestra.run.smithi089.stdout:Device: 5h/5d Inode: 666 Links: 1 2024-09-21T03:31:16.348 INFO:teuthology.orchestra.run.smithi089.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-21T03:31:16.348 INFO:teuthology.orchestra.run.smithi089.stdout:Context: system_u:object_r:device_t:s0 2024-09-21T03:31:16.348 INFO:teuthology.orchestra.run.smithi089.stdout:Access: 2024-09-21 03:30:05.370607367 +0000 2024-09-21T03:31:16.348 INFO:teuthology.orchestra.run.smithi089.stdout:Modify: 2024-09-21 03:25:38.008244890 +0000 2024-09-21T03:31:16.348 INFO:teuthology.orchestra.run.smithi089.stdout:Change: 2024-09-21 03:25:38.008244890 +0000 2024-09-21T03:31:16.348 INFO:teuthology.orchestra.run.smithi089.stdout: Birth: 2024-09-21 03:25:38.008244890 +0000 2024-09-21T03:31:16.349 DEBUG:teuthology.orchestra.run.smithi089:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-21T03:31:16.412 INFO:teuthology.orchestra.run.smithi089.stderr:1+0 records in 2024-09-21T03:31:16.412 INFO:teuthology.orchestra.run.smithi089.stderr:1+0 records out 2024-09-21T03:31:16.413 INFO:teuthology.orchestra.run.smithi089.stderr:512 bytes copied, 0.000214606 s, 2.4 MB/s 2024-09-21T03:31:16.414 DEBUG:teuthology.orchestra.run.smithi089:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-21T03:31:16.471 DEBUG:teuthology.orchestra.run.smithi089:> stat /dev/vg_nvme/lv_2 2024-09-21T03:31:16.526 INFO:teuthology.orchestra.run.smithi089.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-21T03:31:16.526 INFO:teuthology.orchestra.run.smithi089.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-21T03:31:16.526 INFO:teuthology.orchestra.run.smithi089.stdout:Device: 5h/5d Inode: 684 Links: 1 2024-09-21T03:31:16.526 INFO:teuthology.orchestra.run.smithi089.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-21T03:31:16.526 INFO:teuthology.orchestra.run.smithi089.stdout:Context: system_u:object_r:device_t:s0 2024-09-21T03:31:16.526 INFO:teuthology.orchestra.run.smithi089.stdout:Access: 2024-09-21 03:30:05.395606789 +0000 2024-09-21T03:31:16.526 INFO:teuthology.orchestra.run.smithi089.stdout:Modify: 2024-09-21 03:25:38.344237084 +0000 2024-09-21T03:31:16.526 INFO:teuthology.orchestra.run.smithi089.stdout:Change: 2024-09-21 03:25:38.344237084 +0000 2024-09-21T03:31:16.526 INFO:teuthology.orchestra.run.smithi089.stdout: Birth: 2024-09-21 03:25:38.344237084 +0000 2024-09-21T03:31:16.527 DEBUG:teuthology.orchestra.run.smithi089:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-21T03:31:16.589 INFO:teuthology.orchestra.run.smithi089.stderr:1+0 records in 2024-09-21T03:31:16.589 INFO:teuthology.orchestra.run.smithi089.stderr:1+0 records out 2024-09-21T03:31:16.589 INFO:teuthology.orchestra.run.smithi089.stderr:512 bytes copied, 0.000314686 s, 1.6 MB/s 2024-09-21T03:31:16.590 DEBUG:teuthology.orchestra.run.smithi089:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-21T03:31:16.647 DEBUG:teuthology.orchestra.run.smithi089:> stat /dev/vg_nvme/lv_3 2024-09-21T03:31:16.702 INFO:teuthology.orchestra.run.smithi089.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-21T03:31:16.702 INFO:teuthology.orchestra.run.smithi089.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-21T03:31:16.703 INFO:teuthology.orchestra.run.smithi089.stdout:Device: 5h/5d Inode: 703 Links: 1 2024-09-21T03:31:16.703 INFO:teuthology.orchestra.run.smithi089.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-21T03:31:16.703 INFO:teuthology.orchestra.run.smithi089.stdout:Context: system_u:object_r:device_t:s0 2024-09-21T03:31:16.703 INFO:teuthology.orchestra.run.smithi089.stdout:Access: 2024-09-21 03:30:05.421606189 +0000 2024-09-21T03:31:16.703 INFO:teuthology.orchestra.run.smithi089.stdout:Modify: 2024-09-21 03:25:38.663229674 +0000 2024-09-21T03:31:16.703 INFO:teuthology.orchestra.run.smithi089.stdout:Change: 2024-09-21 03:25:38.663229674 +0000 2024-09-21T03:31:16.703 INFO:teuthology.orchestra.run.smithi089.stdout: Birth: 2024-09-21 03:25:38.663229674 +0000 2024-09-21T03:31:16.703 DEBUG:teuthology.orchestra.run.smithi089:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-21T03:31:16.766 INFO:teuthology.orchestra.run.smithi089.stderr:1+0 records in 2024-09-21T03:31:16.766 INFO:teuthology.orchestra.run.smithi089.stderr:1+0 records out 2024-09-21T03:31:16.766 INFO:teuthology.orchestra.run.smithi089.stderr:512 bytes copied, 0.000220147 s, 2.3 MB/s 2024-09-21T03:31:16.767 DEBUG:teuthology.orchestra.run.smithi089:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-21T03:31:16.824 DEBUG:teuthology.orchestra.run.smithi089:> stat /dev/vg_nvme/lv_4 2024-09-21T03:31:16.879 INFO:teuthology.orchestra.run.smithi089.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-21T03:31:16.879 INFO:teuthology.orchestra.run.smithi089.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-21T03:31:16.879 INFO:teuthology.orchestra.run.smithi089.stdout:Device: 5h/5d Inode: 722 Links: 1 2024-09-21T03:31:16.879 INFO:teuthology.orchestra.run.smithi089.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-21T03:31:16.879 INFO:teuthology.orchestra.run.smithi089.stdout:Context: system_u:object_r:device_t:s0 2024-09-21T03:31:16.879 INFO:teuthology.orchestra.run.smithi089.stdout:Access: 2024-09-21 03:30:05.446605611 +0000 2024-09-21T03:31:16.879 INFO:teuthology.orchestra.run.smithi089.stdout:Modify: 2024-09-21 03:25:38.989222101 +0000 2024-09-21T03:31:16.879 INFO:teuthology.orchestra.run.smithi089.stdout:Change: 2024-09-21 03:25:38.989222101 +0000 2024-09-21T03:31:16.880 INFO:teuthology.orchestra.run.smithi089.stdout: Birth: 2024-09-21 03:25:38.989222101 +0000 2024-09-21T03:31:16.880 DEBUG:teuthology.orchestra.run.smithi089:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-21T03:31:16.943 INFO:teuthology.orchestra.run.smithi089.stderr:1+0 records in 2024-09-21T03:31:16.944 INFO:teuthology.orchestra.run.smithi089.stderr:1+0 records out 2024-09-21T03:31:16.944 INFO:teuthology.orchestra.run.smithi089.stderr:512 bytes copied, 0.000190015 s, 2.7 MB/s 2024-09-21T03:31:16.944 DEBUG:teuthology.orchestra.run.smithi089:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-21T03:31:17.000 DEBUG:teuthology.orchestra.run.smithi110:> set -ex 2024-09-21T03:31:17.000 DEBUG:teuthology.orchestra.run.smithi110:> dd if=/scratch_devs of=/dev/stdout 2024-09-21T03:31:17.020 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-21T03:31:17.020 DEBUG:teuthology.orchestra.run.smithi110:> stat /dev/vg_nvme/lv_1 2024-09-21T03:31:17.078 INFO:teuthology.orchestra.run.smithi110.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-21T03:31:17.079 INFO:teuthology.orchestra.run.smithi110.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-21T03:31:17.079 INFO:teuthology.orchestra.run.smithi110.stdout:Device: 5h/5d Inode: 666 Links: 1 2024-09-21T03:31:17.079 INFO:teuthology.orchestra.run.smithi110.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-21T03:31:17.079 INFO:teuthology.orchestra.run.smithi110.stdout:Context: system_u:object_r:device_t:s0 2024-09-21T03:31:17.079 INFO:teuthology.orchestra.run.smithi110.stdout:Access: 2024-09-21 03:30:50.608105974 +0000 2024-09-21T03:31:17.079 INFO:teuthology.orchestra.run.smithi110.stdout:Modify: 2024-09-21 03:25:47.741678175 +0000 2024-09-21T03:31:17.079 INFO:teuthology.orchestra.run.smithi110.stdout:Change: 2024-09-21 03:25:47.741678175 +0000 2024-09-21T03:31:17.079 INFO:teuthology.orchestra.run.smithi110.stdout: Birth: 2024-09-21 03:25:47.741678175 +0000 2024-09-21T03:31:17.079 DEBUG:teuthology.orchestra.run.smithi110:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-21T03:31:17.147 INFO:teuthology.orchestra.run.smithi110.stderr:1+0 records in 2024-09-21T03:31:17.147 INFO:teuthology.orchestra.run.smithi110.stderr:1+0 records out 2024-09-21T03:31:17.147 INFO:teuthology.orchestra.run.smithi110.stderr:512 bytes copied, 0.000127877 s, 4.0 MB/s 2024-09-21T03:31:17.148 DEBUG:teuthology.orchestra.run.smithi110:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-21T03:31:17.206 DEBUG:teuthology.orchestra.run.smithi110:> stat /dev/vg_nvme/lv_2 2024-09-21T03:31:17.210 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:31:16 smithi114 ceph-mon[34140]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:31:17.249 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:31:16 smithi110 ceph-mon[34000]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:31:17.263 INFO:teuthology.orchestra.run.smithi110.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-21T03:31:17.263 INFO:teuthology.orchestra.run.smithi110.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-21T03:31:17.263 INFO:teuthology.orchestra.run.smithi110.stdout:Device: 5h/5d Inode: 687 Links: 1 2024-09-21T03:31:17.263 INFO:teuthology.orchestra.run.smithi110.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-21T03:31:17.263 INFO:teuthology.orchestra.run.smithi110.stdout:Context: system_u:object_r:device_t:s0 2024-09-21T03:31:17.263 INFO:teuthology.orchestra.run.smithi110.stdout:Access: 2024-09-21 03:30:50.609105957 +0000 2024-09-21T03:31:17.264 INFO:teuthology.orchestra.run.smithi110.stdout:Modify: 2024-09-21 03:25:48.061672702 +0000 2024-09-21T03:31:17.264 INFO:teuthology.orchestra.run.smithi110.stdout:Change: 2024-09-21 03:25:48.061672702 +0000 2024-09-21T03:31:17.264 INFO:teuthology.orchestra.run.smithi110.stdout: Birth: 2024-09-21 03:25:48.061672702 +0000 2024-09-21T03:31:17.264 DEBUG:teuthology.orchestra.run.smithi110:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-21T03:31:17.311 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:31:16 smithi089 ceph-mon[30928]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:31:17.329 INFO:teuthology.orchestra.run.smithi110.stderr:1+0 records in 2024-09-21T03:31:17.329 INFO:teuthology.orchestra.run.smithi110.stderr:1+0 records out 2024-09-21T03:31:17.329 INFO:teuthology.orchestra.run.smithi110.stderr:512 bytes copied, 0.000139169 s, 3.7 MB/s 2024-09-21T03:31:17.330 DEBUG:teuthology.orchestra.run.smithi110:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-21T03:31:17.387 DEBUG:teuthology.orchestra.run.smithi110:> stat /dev/vg_nvme/lv_3 2024-09-21T03:31:17.443 INFO:teuthology.orchestra.run.smithi110.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-21T03:31:17.444 INFO:teuthology.orchestra.run.smithi110.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-21T03:31:17.444 INFO:teuthology.orchestra.run.smithi110.stdout:Device: 5h/5d Inode: 706 Links: 1 2024-09-21T03:31:17.444 INFO:teuthology.orchestra.run.smithi110.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-21T03:31:17.444 INFO:teuthology.orchestra.run.smithi110.stdout:Context: system_u:object_r:device_t:s0 2024-09-21T03:31:17.444 INFO:teuthology.orchestra.run.smithi110.stdout:Access: 2024-09-21 03:30:50.609105957 +0000 2024-09-21T03:31:17.444 INFO:teuthology.orchestra.run.smithi110.stdout:Modify: 2024-09-21 03:25:48.372667382 +0000 2024-09-21T03:31:17.444 INFO:teuthology.orchestra.run.smithi110.stdout:Change: 2024-09-21 03:25:48.372667382 +0000 2024-09-21T03:31:17.444 INFO:teuthology.orchestra.run.smithi110.stdout: Birth: 2024-09-21 03:25:48.372667382 +0000 2024-09-21T03:31:17.444 DEBUG:teuthology.orchestra.run.smithi110:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-21T03:31:17.507 INFO:teuthology.orchestra.run.smithi110.stderr:1+0 records in 2024-09-21T03:31:17.507 INFO:teuthology.orchestra.run.smithi110.stderr:1+0 records out 2024-09-21T03:31:17.507 INFO:teuthology.orchestra.run.smithi110.stderr:512 bytes copied, 0.000113832 s, 4.5 MB/s 2024-09-21T03:31:17.508 DEBUG:teuthology.orchestra.run.smithi110:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-21T03:31:17.566 DEBUG:teuthology.orchestra.run.smithi110:> stat /dev/vg_nvme/lv_4 2024-09-21T03:31:17.621 INFO:teuthology.orchestra.run.smithi110.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-21T03:31:17.621 INFO:teuthology.orchestra.run.smithi110.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-21T03:31:17.621 INFO:teuthology.orchestra.run.smithi110.stdout:Device: 5h/5d Inode: 725 Links: 1 2024-09-21T03:31:17.621 INFO:teuthology.orchestra.run.smithi110.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-21T03:31:17.621 INFO:teuthology.orchestra.run.smithi110.stdout:Context: system_u:object_r:device_t:s0 2024-09-21T03:31:17.621 INFO:teuthology.orchestra.run.smithi110.stdout:Access: 2024-09-21 03:30:50.609105957 +0000 2024-09-21T03:31:17.621 INFO:teuthology.orchestra.run.smithi110.stdout:Modify: 2024-09-21 03:25:48.694661874 +0000 2024-09-21T03:31:17.621 INFO:teuthology.orchestra.run.smithi110.stdout:Change: 2024-09-21 03:25:48.694661874 +0000 2024-09-21T03:31:17.621 INFO:teuthology.orchestra.run.smithi110.stdout: Birth: 2024-09-21 03:25:48.694661874 +0000 2024-09-21T03:31:17.621 DEBUG:teuthology.orchestra.run.smithi110:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-21T03:31:17.684 INFO:teuthology.orchestra.run.smithi110.stderr:1+0 records in 2024-09-21T03:31:17.684 INFO:teuthology.orchestra.run.smithi110.stderr:1+0 records out 2024-09-21T03:31:17.684 INFO:teuthology.orchestra.run.smithi110.stderr:512 bytes copied, 8.4808e-05 s, 6.0 MB/s 2024-09-21T03:31:17.685 DEBUG:teuthology.orchestra.run.smithi110:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-21T03:31:17.741 DEBUG:teuthology.orchestra.run.smithi114:> set -ex 2024-09-21T03:31:17.742 DEBUG:teuthology.orchestra.run.smithi114:> dd if=/scratch_devs of=/dev/stdout 2024-09-21T03:31:17.762 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-21T03:31:17.762 DEBUG:teuthology.orchestra.run.smithi114:> stat /dev/vg_nvme/lv_1 2024-09-21T03:31:17.819 INFO:teuthology.orchestra.run.smithi114.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-21T03:31:17.820 INFO:teuthology.orchestra.run.smithi114.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-21T03:31:17.820 INFO:teuthology.orchestra.run.smithi114.stdout:Device: 5h/5d Inode: 665 Links: 1 2024-09-21T03:31:17.820 INFO:teuthology.orchestra.run.smithi114.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-21T03:31:17.820 INFO:teuthology.orchestra.run.smithi114.stdout:Context: system_u:object_r:device_t:s0 2024-09-21T03:31:17.820 INFO:teuthology.orchestra.run.smithi114.stdout:Access: 2024-09-21 03:30:48.179453031 +0000 2024-09-21T03:31:17.820 INFO:teuthology.orchestra.run.smithi114.stdout:Modify: 2024-09-21 03:25:38.024782604 +0000 2024-09-21T03:31:17.820 INFO:teuthology.orchestra.run.smithi114.stdout:Change: 2024-09-21 03:25:38.024782604 +0000 2024-09-21T03:31:17.820 INFO:teuthology.orchestra.run.smithi114.stdout: Birth: 2024-09-21 03:25:38.024782604 +0000 2024-09-21T03:31:17.820 DEBUG:teuthology.orchestra.run.smithi114:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-21T03:31:17.889 INFO:teuthology.orchestra.run.smithi114.stderr:1+0 records in 2024-09-21T03:31:17.889 INFO:teuthology.orchestra.run.smithi114.stderr:1+0 records out 2024-09-21T03:31:17.889 INFO:teuthology.orchestra.run.smithi114.stderr:512 bytes copied, 0.000235973 s, 2.2 MB/s 2024-09-21T03:31:17.890 DEBUG:teuthology.orchestra.run.smithi114:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-21T03:31:17.948 DEBUG:teuthology.orchestra.run.smithi114:> stat /dev/vg_nvme/lv_2 2024-09-21T03:31:18.006 INFO:teuthology.orchestra.run.smithi114.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-21T03:31:18.006 INFO:teuthology.orchestra.run.smithi114.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-21T03:31:18.006 INFO:teuthology.orchestra.run.smithi114.stdout:Device: 5h/5d Inode: 686 Links: 1 2024-09-21T03:31:18.006 INFO:teuthology.orchestra.run.smithi114.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-21T03:31:18.006 INFO:teuthology.orchestra.run.smithi114.stdout:Context: system_u:object_r:device_t:s0 2024-09-21T03:31:18.006 INFO:teuthology.orchestra.run.smithi114.stdout:Access: 2024-09-21 03:30:48.179453031 +0000 2024-09-21T03:31:18.006 INFO:teuthology.orchestra.run.smithi114.stdout:Modify: 2024-09-21 03:25:38.346781161 +0000 2024-09-21T03:31:18.006 INFO:teuthology.orchestra.run.smithi114.stdout:Change: 2024-09-21 03:25:38.346781161 +0000 2024-09-21T03:31:18.006 INFO:teuthology.orchestra.run.smithi114.stdout: Birth: 2024-09-21 03:25:38.346781161 +0000 2024-09-21T03:31:18.007 DEBUG:teuthology.orchestra.run.smithi114:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-21T03:31:18.070 INFO:teuthology.orchestra.run.smithi114.stderr:1+0 records in 2024-09-21T03:31:18.071 INFO:teuthology.orchestra.run.smithi114.stderr:1+0 records out 2024-09-21T03:31:18.071 INFO:teuthology.orchestra.run.smithi114.stderr:512 bytes copied, 0.000207422 s, 2.5 MB/s 2024-09-21T03:31:18.072 DEBUG:teuthology.orchestra.run.smithi114:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-21T03:31:18.129 DEBUG:teuthology.orchestra.run.smithi114:> stat /dev/vg_nvme/lv_3 2024-09-21T03:31:18.186 INFO:teuthology.orchestra.run.smithi114.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-21T03:31:18.186 INFO:teuthology.orchestra.run.smithi114.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-21T03:31:18.186 INFO:teuthology.orchestra.run.smithi114.stdout:Device: 5h/5d Inode: 704 Links: 1 2024-09-21T03:31:18.186 INFO:teuthology.orchestra.run.smithi114.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-21T03:31:18.186 INFO:teuthology.orchestra.run.smithi114.stdout:Context: system_u:object_r:device_t:s0 2024-09-21T03:31:18.186 INFO:teuthology.orchestra.run.smithi114.stdout:Access: 2024-09-21 03:30:48.179453031 +0000 2024-09-21T03:31:18.186 INFO:teuthology.orchestra.run.smithi114.stdout:Modify: 2024-09-21 03:25:38.649779803 +0000 2024-09-21T03:31:18.186 INFO:teuthology.orchestra.run.smithi114.stdout:Change: 2024-09-21 03:25:38.649779803 +0000 2024-09-21T03:31:18.186 INFO:teuthology.orchestra.run.smithi114.stdout: Birth: 2024-09-21 03:25:38.649779803 +0000 2024-09-21T03:31:18.187 DEBUG:teuthology.orchestra.run.smithi114:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-21T03:31:18.249 INFO:teuthology.orchestra.run.smithi114.stderr:1+0 records in 2024-09-21T03:31:18.250 INFO:teuthology.orchestra.run.smithi114.stderr:1+0 records out 2024-09-21T03:31:18.250 INFO:teuthology.orchestra.run.smithi114.stderr:512 bytes copied, 0.000204665 s, 2.5 MB/s 2024-09-21T03:31:18.251 DEBUG:teuthology.orchestra.run.smithi114:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-21T03:31:18.308 DEBUG:teuthology.orchestra.run.smithi114:> stat /dev/vg_nvme/lv_4 2024-09-21T03:31:18.364 INFO:teuthology.orchestra.run.smithi114.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-21T03:31:18.364 INFO:teuthology.orchestra.run.smithi114.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-21T03:31:18.364 INFO:teuthology.orchestra.run.smithi114.stdout:Device: 5h/5d Inode: 722 Links: 1 2024-09-21T03:31:18.364 INFO:teuthology.orchestra.run.smithi114.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-21T03:31:18.364 INFO:teuthology.orchestra.run.smithi114.stdout:Context: system_u:object_r:device_t:s0 2024-09-21T03:31:18.364 INFO:teuthology.orchestra.run.smithi114.stdout:Access: 2024-09-21 03:30:48.180453027 +0000 2024-09-21T03:31:18.364 INFO:teuthology.orchestra.run.smithi114.stdout:Modify: 2024-09-21 03:25:38.959778413 +0000 2024-09-21T03:31:18.364 INFO:teuthology.orchestra.run.smithi114.stdout:Change: 2024-09-21 03:25:38.959778413 +0000 2024-09-21T03:31:18.364 INFO:teuthology.orchestra.run.smithi114.stdout: Birth: 2024-09-21 03:25:38.959778413 +0000 2024-09-21T03:31:18.364 DEBUG:teuthology.orchestra.run.smithi114:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-21T03:31:18.428 INFO:teuthology.orchestra.run.smithi114.stderr:1+0 records in 2024-09-21T03:31:18.429 INFO:teuthology.orchestra.run.smithi114.stderr:1+0 records out 2024-09-21T03:31:18.429 INFO:teuthology.orchestra.run.smithi114.stderr:512 bytes copied, 0.000201164 s, 2.5 MB/s 2024-09-21T03:31:18.430 DEBUG:teuthology.orchestra.run.smithi114:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-21T03:31:18.487 INFO:tasks.cephadm:Deploying osd.0 on smithi089 with /dev/vg_nvme/lv_4... 2024-09-21T03:31:18.487 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7c374098-77c9-11ef-baf5-efdc52797490 -- lvm zap /dev/vg_nvme/lv_4 2024-09-21T03:31:18.741 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/7c374098-77c9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:31:19.141 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:31:18 smithi089 ceph-mon[30928]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:31:19.210 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:31:18 smithi114 ceph-mon[34140]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:31:19.250 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:31:18 smithi110 ceph-mon[34000]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:31:20.988 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-09-21T03:31:21.015 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7c374098-77c9-11ef-baf5-efdc52797490 -- ceph orch daemon add osd smithi089:vg_nvme/lv_4 2024-09-21T03:31:21.211 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:31:20 smithi114 ceph-mon[34140]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:31:21.250 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:31:20 smithi110 ceph-mon[34000]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:31:21.255 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/7c374098-77c9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:31:21.311 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:31:20 smithi089 ceph-mon[30928]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:31:21.733 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:21.728+0000 7f91e3afa640 1 -- 172.21.15.89:0/3022229547 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f91dc102660 msgr2=0x7f91dc102ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:31:21.733 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:21.728+0000 7f91e3afa640 1 --2- 172.21.15.89:0/3022229547 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f91dc102660 0x7f91dc102ad0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f91d0009900 tx=0x7f91d002a480 comp rx=0 tx=0).stop 2024-09-21T03:31:21.733 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:21.728+0000 7f91e3afa640 1 -- 172.21.15.89:0/3022229547 shutdown_connections 2024-09-21T03:31:21.733 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:21.728+0000 7f91e3afa640 1 --2- 172.21.15.89:0/3022229547 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f91dc104090 0x7f91dc10aa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:21.733 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:21.728+0000 7f91e3afa640 1 --2- 172.21.15.89:0/3022229547 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f91dc1036f0 0x7f91dc103b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:21.733 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:21.728+0000 7f91e3afa640 1 --2- 172.21.15.89:0/3022229547 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f91dc102660 0x7f91dc102ad0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:21.733 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:21.728+0000 7f91e3afa640 1 -- 172.21.15.89:0/3022229547 >> 172.21.15.89:0/3022229547 conn(0x7f91dc0fdfd0 msgr2=0x7f91dc100480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:31:21.733 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:21.728+0000 7f91e3afa640 1 -- 172.21.15.89:0/3022229547 shutdown_connections 2024-09-21T03:31:21.733 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:21.728+0000 7f91e3afa640 1 -- 172.21.15.89:0/3022229547 wait complete. 2024-09-21T03:31:21.733 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:21.729+0000 7f91e3afa640 1 Processor -- start 2024-09-21T03:31:21.734 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:21.729+0000 7f91e3afa640 1 -- start start 2024-09-21T03:31:21.734 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:21.729+0000 7f91e3afa640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f91dc102660 0x7f91dc06bfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:21.734 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:21.729+0000 7f91e3afa640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f91dc1036f0 0x7f91dc1a1140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:21.734 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:21.729+0000 7f91e3afa640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f91dc104090 0x7f91dc1a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:21.734 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:21.729+0000 7f91e3afa640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f91dc10e150 con 0x7f91dc102660 2024-09-21T03:31:21.734 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:21.729+0000 7f91e3afa640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f91dc10dfd0 con 0x7f91dc104090 2024-09-21T03:31:21.734 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:21.729+0000 7f91e3afa640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f91dc10e2d0 con 0x7f91dc1036f0 2024-09-21T03:31:21.734 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:21.729+0000 7f91e1896640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f91dc102660 0x7f91dc06bfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:21.734 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:21.729+0000 7f91e1896640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f91dc102660 0x7f91dc06bfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:38668/0 (socket says 172.21.15.89:38668) 2024-09-21T03:31:21.734 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:21.729+0000 7f91e1896640 1 -- 172.21.15.89:0/2211499726 learned_addr learned my addr 172.21.15.89:0/2211499726 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-21T03:31:21.734 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:21.730+0000 7f91e2097640 1 --2- 172.21.15.89:0/2211499726 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f91dc104090 0x7f91dc1a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:21.734 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:21.730+0000 7f91e1095640 1 --2- 172.21.15.89:0/2211499726 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f91dc1036f0 0x7f91dc1a1140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:21.735 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:21.730+0000 7f91e1896640 1 -- 172.21.15.89:0/2211499726 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f91dc1036f0 msgr2=0x7f91dc1a1140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:31:21.735 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:21.730+0000 7f91e1896640 1 --2- 172.21.15.89:0/2211499726 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f91dc1036f0 0x7f91dc1a1140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:21.735 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:21.730+0000 7f91e1896640 1 -- 172.21.15.89:0/2211499726 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f91dc104090 msgr2=0x7f91dc1a3690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:31:21.735 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:21.730+0000 7f91e1896640 1 --2- 172.21.15.89:0/2211499726 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f91dc104090 0x7f91dc1a3690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:21.735 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:21.730+0000 7f91e1896640 1 -- 172.21.15.89:0/2211499726 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91dc1a3ca0 con 0x7f91dc102660 2024-09-21T03:31:21.735 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:21.730+0000 7f91e1095640 1 --2- 172.21.15.89:0/2211499726 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f91dc1036f0 0x7f91dc1a1140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:31:21.735 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:21.730+0000 7f91e2097640 1 --2- 172.21.15.89:0/2211499726 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f91dc104090 0x7f91dc1a3690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:31:21.735 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:21.730+0000 7f91e1896640 1 --2- 172.21.15.89:0/2211499726 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f91dc102660 0x7f91dc06bfc0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f91d0009a20 tx=0x7f91d00042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:31:21.735 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:21.730+0000 7f91ceffd640 1 -- 172.21.15.89:0/2211499726 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91d003f030 con 0x7f91dc102660 2024-09-21T03:31:21.735 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:21.730+0000 7f91ceffd640 1 -- 172.21.15.89:0/2211499726 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f91d0003c90 con 0x7f91dc102660 2024-09-21T03:31:21.735 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:21.730+0000 7f91e3afa640 1 -- 172.21.15.89:0/2211499726 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f91dc1a3f30 con 0x7f91dc102660 2024-09-21T03:31:21.735 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:21.730+0000 7f91ceffd640 1 -- 172.21.15.89:0/2211499726 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91d0035070 con 0x7f91dc102660 2024-09-21T03:31:21.736 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:21.730+0000 7f91e3afa640 1 -- 172.21.15.89:0/2211499726 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f91dc1a4380 con 0x7f91dc102660 2024-09-21T03:31:21.736 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:21.731+0000 7f91e3afa640 1 -- 172.21.15.89:0/2211499726 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f91dc109840 con 0x7f91dc102660 2024-09-21T03:31:21.736 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:21.733+0000 7f91ceffd640 1 -- 172.21.15.89:0/2211499726 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 16) ==== 154456+0+0 (secure 0 0 0) 0x7f91d0042030 con 0x7f91dc102660 2024-09-21T03:31:21.736 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:21.733+0000 7f91ceffd640 1 --2- 172.21.15.89:0/2211499726 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f91c80b6500 0x7f91c80b8a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:21.736 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:21.734+0000 7f91e1095640 1 --2- 172.21.15.89:0/2211499726 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f91c80b6500 0x7f91c80b8a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:21.737 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:21.734+0000 7f91ceffd640 1 -- 172.21.15.89:0/2211499726 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f91d0106100 con 0x7f91dc102660 2024-09-21T03:31:21.737 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:21.734+0000 7f91e1095640 1 --2- 172.21.15.89:0/2211499726 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f91c80b6500 0x7f91c80b8a10 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f91d80058c0 tx=0x7f91d8005d30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:31:21.739 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:21.737+0000 7f91ceffd640 1 -- 172.21.15.89:0/2211499726 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f91d00ce660 con 0x7f91dc102660 2024-09-21T03:31:21.927 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:21.925+0000 7f91e3afa640 1 -- 172.21.15.89:0/2211499726 --> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi089:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f91dc063800 con 0x7f91c80b6500 2024-09-21T03:31:22.211 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:31:21 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T03:31:22.211 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:31:21 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T03:31:22.211 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:31:21 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:31:22.250 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:31:21 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T03:31:22.250 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:31:21 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T03:31:22.250 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:31:21 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:31:22.297 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:31:21 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T03:31:22.297 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:31:21 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T03:31:22.297 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:31:21 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:31:23.228 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:31:22 smithi089 ceph-mon[30928]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:31:23.228 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:31:22 smithi089 ceph-mon[30928]: from='client.14391 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi089:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:31:23.250 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:31:22 smithi110 ceph-mon[34000]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:31:23.250 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:31:22 smithi110 ceph-mon[34000]: from='client.14391 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi089:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:31:23.460 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:31:22 smithi114 ceph-mon[34140]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:31:23.461 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:31:22 smithi114 ceph-mon[34140]: from='client.14391 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi089:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:31:25.061 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:31:24 smithi089 ceph-mon[30928]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:31:25.061 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:31:24 smithi089 ceph-mon[30928]: from='client.? 172.21.15.89:0/394681935' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "27ca5ed8-17d1-4f8e-a7ae-084a914721a0"} : dispatch 2024-09-21T03:31:25.061 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:31:24 smithi089 ceph-mon[30928]: from='client.? 172.21.15.89:0/394681935' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "27ca5ed8-17d1-4f8e-a7ae-084a914721a0"}]': finished 2024-09-21T03:31:25.061 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:31:24 smithi089 ceph-mon[30928]: osdmap e5: 1 total, 0 up, 1 in 2024-09-21T03:31:25.062 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:31:24 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-21T03:31:25.062 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:31:24 smithi089 ceph-mon[30928]: from='client.? 172.21.15.89:0/3759348579' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T03:31:25.250 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:31:24 smithi110 ceph-mon[34000]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:31:25.250 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:31:24 smithi110 ceph-mon[34000]: from='client.? 172.21.15.89:0/394681935' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "27ca5ed8-17d1-4f8e-a7ae-084a914721a0"} : dispatch 2024-09-21T03:31:25.250 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:31:24 smithi110 ceph-mon[34000]: from='client.? 172.21.15.89:0/394681935' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "27ca5ed8-17d1-4f8e-a7ae-084a914721a0"}]': finished 2024-09-21T03:31:25.250 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:31:24 smithi110 ceph-mon[34000]: osdmap e5: 1 total, 0 up, 1 in 2024-09-21T03:31:25.250 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:31:24 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-21T03:31:25.250 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:31:24 smithi110 ceph-mon[34000]: from='client.? 172.21.15.89:0/3759348579' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T03:31:25.461 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:31:24 smithi114 ceph-mon[34140]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:31:25.461 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:31:24 smithi114 ceph-mon[34140]: from='client.? 172.21.15.89:0/394681935' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "27ca5ed8-17d1-4f8e-a7ae-084a914721a0"} : dispatch 2024-09-21T03:31:25.461 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:31:24 smithi114 ceph-mon[34140]: from='client.? 172.21.15.89:0/394681935' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "27ca5ed8-17d1-4f8e-a7ae-084a914721a0"}]': finished 2024-09-21T03:31:25.461 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:31:24 smithi114 ceph-mon[34140]: osdmap e5: 1 total, 0 up, 1 in 2024-09-21T03:31:25.461 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:31:24 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-21T03:31:25.461 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:31:24 smithi114 ceph-mon[34140]: from='client.? 172.21.15.89:0/3759348579' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T03:31:27.061 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:31:26 smithi089 ceph-mon[30928]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:31:27.250 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:31:26 smithi110 ceph-mon[34000]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:31:27.461 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:31:26 smithi114 ceph-mon[34140]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:31:29.250 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:31:28 smithi110 ceph-mon[34000]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:31:29.311 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:31:28 smithi089 ceph-mon[30928]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:31:29.461 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:31:28 smithi114 ceph-mon[34140]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:31:31.061 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:31:30 smithi089 ceph-mon[30928]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:31:31.250 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:31:30 smithi110 ceph-mon[34000]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:31:31.460 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:31:30 smithi114 ceph-mon[34140]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:31:33.243 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:31:32 smithi089 ceph-mon[30928]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:31:33.243 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:31:32 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.0"} : dispatch 2024-09-21T03:31:33.243 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:31:32 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:31:33.249 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:31:32 smithi110 ceph-mon[34000]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:31:33.250 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:31:32 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.0"} : dispatch 2024-09-21T03:31:33.250 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:31:32 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:31:33.263 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:31:32 smithi114 ceph-mon[34140]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:31:33.263 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:31:32 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.0"} : dispatch 2024-09-21T03:31:33.263 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:31:32 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:31:34.237 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:31:33 smithi089 ceph-mon[30928]: Deploying daemon osd.0 on smithi089 2024-09-21T03:31:34.250 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:31:33 smithi110 ceph-mon[34000]: Deploying daemon osd.0 on smithi089 2024-09-21T03:31:34.460 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:31:33 smithi114 ceph-mon[34140]: Deploying daemon osd.0 on smithi089 2024-09-21T03:31:35.250 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:31:34 smithi110 ceph-mon[34000]: pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:31:35.311 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:31:34 smithi089 ceph-mon[30928]: pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:31:35.460 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:31:34 smithi114 ceph-mon[34140]: pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:31:37.250 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:31:36 smithi110 ceph-mon[34000]: pgmap v47: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:31:37.311 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:31:36 smithi089 ceph-mon[30928]: pgmap v47: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:31:37.460 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:31:36 smithi114 ceph-mon[34140]: pgmap v47: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:31:39.250 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:31:38 smithi110 ceph-mon[34000]: pgmap v48: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:31:39.311 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:31:38 smithi089 ceph-mon[30928]: pgmap v48: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:31:39.460 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:31:38 smithi114 ceph-mon[34140]: pgmap v48: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:31:40.061 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:31:39 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:31:40.061 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:31:39 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:31:40.061 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:31:39 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:31:40.250 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:31:39 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:31:40.250 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:31:39 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:31:40.250 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:31:39 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:31:40.460 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:31:39 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:31:40.461 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:31:39 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:31:40.461 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:31:39 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:31:41.247 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:31:41 smithi089 ceph-mon[30928]: pgmap v49: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:31:41.460 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:31:41 smithi114 ceph-mon[34140]: pgmap v49: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:31:41.500 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:31:41 smithi110 ceph-mon[34000]: pgmap v49: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:31:42.711 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:31:42 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:31:42.711 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:31:42 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:31:42.711 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:31:42 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:31:42.711 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:31:42 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:31:42.711 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:31:42 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:31:42.724 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:31:42 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:31:42.724 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:31:42 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:31:42.724 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:31:42 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:31:42.724 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:31:42 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:31:42.724 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:31:42 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:31:42.750 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:31:42 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:31:42.750 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:31:42 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:31:42.750 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:31:42 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:31:42.750 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:31:42 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:31:42.750 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:31:42 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:31:42.979 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:42.977+0000 7f91ceffd640 1 -- 172.21.15.89:0/2211499726 <== mgr.14152 v2:172.21.15.89:6800/1591532358 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f91dc063800 con 0x7f91c80b6500 2024-09-21T03:31:42.980 INFO:teuthology.orchestra.run.smithi089.stdout:Created osd(s) 0 on host 'smithi089' 2024-09-21T03:31:42.982 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:42.979+0000 7f91e3afa640 1 -- 172.21.15.89:0/2211499726 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f91c80b6500 msgr2=0x7f91c80b8a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:31:42.982 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:42.979+0000 7f91e3afa640 1 --2- 172.21.15.89:0/2211499726 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f91c80b6500 0x7f91c80b8a10 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f91d80058c0 tx=0x7f91d8005d30 comp rx=0 tx=0).stop 2024-09-21T03:31:42.982 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:42.979+0000 7f91e3afa640 1 -- 172.21.15.89:0/2211499726 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f91dc102660 msgr2=0x7f91dc06bfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:31:42.982 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:42.979+0000 7f91e3afa640 1 --2- 172.21.15.89:0/2211499726 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f91dc102660 0x7f91dc06bfc0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f91d0009a20 tx=0x7f91d00042f0 comp rx=0 tx=0).stop 2024-09-21T03:31:42.982 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:42.979+0000 7f91e3afa640 1 -- 172.21.15.89:0/2211499726 shutdown_connections 2024-09-21T03:31:42.982 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:42.979+0000 7f91e3afa640 1 --2- 172.21.15.89:0/2211499726 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f91c80b6500 0x7f91c80b8a10 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:42.982 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:42.979+0000 7f91e3afa640 1 --2- 172.21.15.89:0/2211499726 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f91dc104090 0x7f91dc1a3690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:42.983 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:42.979+0000 7f91e3afa640 1 --2- 172.21.15.89:0/2211499726 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f91dc1036f0 0x7f91dc1a1140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:42.983 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:42.979+0000 7f91e3afa640 1 --2- 172.21.15.89:0/2211499726 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f91dc102660 0x7f91dc06bfc0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:42.983 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:42.980+0000 7f91e3afa640 1 -- 172.21.15.89:0/2211499726 >> 172.21.15.89:0/2211499726 conn(0x7f91dc0fdfd0 msgr2=0x7f91dc106a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:31:42.983 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:42.980+0000 7f91e3afa640 1 -- 172.21.15.89:0/2211499726 shutdown_connections 2024-09-21T03:31:42.983 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:31:42.980+0000 7f91e3afa640 1 -- 172.21.15.89:0/2211499726 wait complete. 2024-09-21T03:31:43.537 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:31:43 smithi089 ceph-mon[30928]: pgmap v50: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:31:43.537 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:31:43 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:31:43.538 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:31:43 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:31:43.538 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:31:43 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:31:43.565 DEBUG:teuthology.orchestra.run.smithi089:osd.0> sudo journalctl -f -n 0 -u ceph-7c374098-77c9-11ef-baf5-efdc52797490@osd.0.service 2024-09-21T03:31:43.567 INFO:tasks.cephadm:Deploying osd.1 on smithi110 with /dev/vg_nvme/lv_4... 2024-09-21T03:31:43.568 DEBUG:teuthology.orchestra.run.smithi110:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7c374098-77c9-11ef-baf5-efdc52797490 -- lvm zap /dev/vg_nvme/lv_4 2024-09-21T03:31:43.710 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:31:43 smithi114 ceph-mon[34140]: pgmap v50: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:31:43.711 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:31:43 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:31:43.711 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:31:43 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:31:43.711 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:31:43 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:31:43.750 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:31:43 smithi110 ceph-mon[34000]: pgmap v50: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:31:43.750 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:31:43 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:31:43.750 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:31:43 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:31:43.750 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:31:43 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:31:43.820 INFO:teuthology.orchestra.run.smithi110.stderr:Inferring config /var/lib/ceph/7c374098-77c9-11ef-baf5-efdc52797490/mon.b/config 2024-09-21T03:31:45.311 INFO:journalctl@ceph.osd.0.smithi089.stdout:Sep 21 03:31:45 smithi089 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-0[46605]: 2024-09-21T03:31:45.188+0000 7f543b692640 -1 osd.0 0 log_to_monitors true 2024-09-21T03:31:45.710 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:31:45 smithi114 ceph-mon[34140]: pgmap v51: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:31:45.711 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:31:45 smithi114 ceph-mon[34140]: from='osd.0 [v2:172.21.15.89:6802/990539042,v1:172.21.15.89:6803/990539042]' entity='osd.0' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]} : dispatch 2024-09-21T03:31:45.719 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:31:45 smithi089 ceph-mon[30928]: pgmap v51: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:31:45.719 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:31:45 smithi089 ceph-mon[30928]: from='osd.0 [v2:172.21.15.89:6802/990539042,v1:172.21.15.89:6803/990539042]' entity='osd.0' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]} : dispatch 2024-09-21T03:31:45.750 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:31:45 smithi110 ceph-mon[34000]: pgmap v51: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:31:45.750 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:31:45 smithi110 ceph-mon[34000]: from='osd.0 [v2:172.21.15.89:6802/990539042,v1:172.21.15.89:6803/990539042]' entity='osd.0' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]} : dispatch 2024-09-21T03:31:46.065 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-21T03:31:46.090 DEBUG:teuthology.orchestra.run.smithi110:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7c374098-77c9-11ef-baf5-efdc52797490 -- ceph orch daemon add osd smithi110:vg_nvme/lv_4 2024-09-21T03:31:46.341 INFO:teuthology.orchestra.run.smithi110.stderr:Inferring config /var/lib/ceph/7c374098-77c9-11ef-baf5-efdc52797490/mon.b/config 2024-09-21T03:31:46.658 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:31:46 smithi089 ceph-mon[30928]: from='osd.0 [v2:172.21.15.89:6802/990539042,v1:172.21.15.89:6803/990539042]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-09-21T03:31:46.658 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:31:46 smithi089 ceph-mon[30928]: osdmap e6: 1 total, 0 up, 1 in 2024-09-21T03:31:46.658 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:31:46 smithi089 ceph-mon[30928]: from='osd.0 [v2:172.21.15.89:6802/990539042,v1:172.21.15.89:6803/990539042]' entity='osd.0' cmd={"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi089", "root=default"]} : dispatch 2024-09-21T03:31:46.659 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:31:46 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-21T03:31:46.659 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:31:46 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:31:46.659 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:31:46 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:31:46.659 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:31:46 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-21T03:31:46.659 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:31:46 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:31:46.659 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:31:46 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:31:46.659 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:31:46 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:31:46.659 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:31:46 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:31:46.691 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:31:46 smithi110 ceph-mon[34000]: from='osd.0 [v2:172.21.15.89:6802/990539042,v1:172.21.15.89:6803/990539042]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-09-21T03:31:46.692 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:31:46 smithi110 ceph-mon[34000]: osdmap e6: 1 total, 0 up, 1 in 2024-09-21T03:31:46.692 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:31:46 smithi110 ceph-mon[34000]: from='osd.0 [v2:172.21.15.89:6802/990539042,v1:172.21.15.89:6803/990539042]' entity='osd.0' cmd={"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi089", "root=default"]} : dispatch 2024-09-21T03:31:46.692 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:31:46 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-21T03:31:46.692 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:31:46 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:31:46.692 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:31:46 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:31:46.692 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:31:46 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-21T03:31:46.692 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:31:46 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:31:46.692 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:31:46 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:31:46.692 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:31:46 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:31:46.692 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:31:46 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:31:46.787 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T03:31:46.784+0000 7fe551903640 1 -- 172.21.15.110:0/1964709351 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe54c10ae50 msgr2=0x7fe54c10b320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:31:46.787 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T03:31:46.784+0000 7fe551903640 1 --2- 172.21.15.110:0/1964709351 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe54c10ae50 0x7fe54c10b320 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fe534009900 tx=0x7fe53402a410 comp rx=0 tx=0).stop 2024-09-21T03:31:46.787 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T03:31:46.785+0000 7fe551903640 1 -- 172.21.15.110:0/1964709351 shutdown_connections 2024-09-21T03:31:46.787 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T03:31:46.785+0000 7fe551903640 1 --2- 172.21.15.110:0/1964709351 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe54c10ae50 0x7fe54c10b320 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:46.787 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T03:31:46.785+0000 7fe551903640 1 --2- 172.21.15.110:0/1964709351 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe54c111f90 0x7fe54c1143b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:46.787 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T03:31:46.785+0000 7fe551903640 1 --2- 172.21.15.110:0/1964709351 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe54c10a3c0 0x7fe54c10a810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:46.787 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T03:31:46.785+0000 7fe551903640 1 -- 172.21.15.110:0/1964709351 >> 172.21.15.110:0/1964709351 conn(0x7fe54c070950 msgr2=0x7fe54c072d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:31:46.788 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T03:31:46.785+0000 7fe551903640 1 -- 172.21.15.110:0/1964709351 shutdown_connections 2024-09-21T03:31:46.788 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T03:31:46.785+0000 7fe551903640 1 -- 172.21.15.110:0/1964709351 wait complete. 2024-09-21T03:31:46.788 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T03:31:46.785+0000 7fe551903640 1 Processor -- start 2024-09-21T03:31:46.788 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T03:31:46.785+0000 7fe551903640 1 -- start start 2024-09-21T03:31:46.789 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T03:31:46.786+0000 7fe551903640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe54c10a3c0 0x7fe54c10d5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:46.789 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T03:31:46.786+0000 7fe551903640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe54c10ae50 0x7fe54c10be80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:46.789 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T03:31:46.786+0000 7fe551903640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe54c111f90 0x7fe54c10c3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:46.789 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T03:31:46.786+0000 7fe551903640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fe54c117a60 con 0x7fe54c10ae50 2024-09-21T03:31:46.789 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T03:31:46.786+0000 7fe551903640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fe54c1178e0 con 0x7fe54c10a3c0 2024-09-21T03:31:46.789 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T03:31:46.786+0000 7fe551903640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fe54c117be0 con 0x7fe54c111f90 2024-09-21T03:31:46.790 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T03:31:46.786+0000 7fe54affd640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe54c10a3c0 0x7fe54c10d5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:46.790 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T03:31:46.786+0000 7fe54affd640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe54c10a3c0 0x7fe54c10d5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:38702/0 (socket says 172.21.15.110:38702) 2024-09-21T03:31:46.790 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T03:31:46.786+0000 7fe54affd640 1 -- 172.21.15.110:0/1295952947 learned_addr learned my addr 172.21.15.110:0/1295952947 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-21T03:31:46.790 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T03:31:46.786+0000 7fe54a7fc640 1 --2- 172.21.15.110:0/1295952947 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe54c10ae50 0x7fe54c10be80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:46.790 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T03:31:46.786+0000 7fe54b7fe640 1 --2- 172.21.15.110:0/1295952947 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe54c111f90 0x7fe54c10c3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:46.790 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T03:31:46.786+0000 7fe54affd640 1 -- 172.21.15.110:0/1295952947 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe54c111f90 msgr2=0x7fe54c10c3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:31:46.790 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T03:31:46.786+0000 7fe54affd640 1 --2- 172.21.15.110:0/1295952947 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe54c111f90 0x7fe54c10c3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:46.790 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T03:31:46.786+0000 7fe54affd640 1 -- 172.21.15.110:0/1295952947 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe54c10ae50 msgr2=0x7fe54c10be80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:31:46.790 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T03:31:46.786+0000 7fe54affd640 1 --2- 172.21.15.110:0/1295952947 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe54c10ae50 0x7fe54c10be80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:46.790 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T03:31:46.786+0000 7fe54affd640 1 -- 172.21.15.110:0/1295952947 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe54c10cac0 con 0x7fe54c10a3c0 2024-09-21T03:31:46.790 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T03:31:46.786+0000 7fe54affd640 1 --2- 172.21.15.110:0/1295952947 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe54c10a3c0 0x7fe54c10d5b0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fe54400ea80 tx=0x7fe54400ef40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:31:46.790 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T03:31:46.786+0000 7fe550901640 1 -- 172.21.15.110:0/1295952947 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe54400ce40 con 0x7fe54c10a3c0 2024-09-21T03:31:46.790 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T03:31:46.787+0000 7fe551903640 1 -- 172.21.15.110:0/1295952947 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe54c1ac790 con 0x7fe54c10a3c0 2024-09-21T03:31:46.790 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T03:31:46.787+0000 7fe550901640 1 -- 172.21.15.110:0/1295952947 <== mon.2 v2:172.21.15.110:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe544011900 con 0x7fe54c10a3c0 2024-09-21T03:31:46.791 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T03:31:46.787+0000 7fe551903640 1 -- 172.21.15.110:0/1295952947 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe54c1acc40 con 0x7fe54c10a3c0 2024-09-21T03:31:46.791 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T03:31:46.787+0000 7fe550901640 1 -- 172.21.15.110:0/1295952947 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe54401a9c0 con 0x7fe54c10a3c0 2024-09-21T03:31:46.791 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T03:31:46.788+0000 7fe551903640 1 -- 172.21.15.110:0/1295952947 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe54c110da0 con 0x7fe54c10a3c0 2024-09-21T03:31:46.791 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T03:31:46.789+0000 7fe550901640 1 -- 172.21.15.110:0/1295952947 <== mon.2 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 16) ==== 154456+0+0 (secure 0 0 0) 0x7fe544011470 con 0x7fe54c10a3c0 2024-09-21T03:31:46.791 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T03:31:46.790+0000 7fe550901640 1 --2- 172.21.15.110:0/1295952947 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7fe5380b6480 0x7fe5380b8990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:46.792 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T03:31:46.790+0000 7fe550901640 1 -- 172.21.15.110:0/1295952947 <== mon.2 v2:172.21.15.110:3300/0 5 ==== osd_map(7..7 src has 1..7) ==== 1487+0+0 (secure 0 0 0) 0x7fe5440e5c20 con 0x7fe54c10a3c0 2024-09-21T03:31:46.792 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T03:31:46.790+0000 7fe54a7fc640 1 --2- 172.21.15.110:0/1295952947 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7fe5380b6480 0x7fe5380b8990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:46.792 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T03:31:46.791+0000 7fe54a7fc640 1 --2- 172.21.15.110:0/1295952947 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7fe5380b6480 0x7fe5380b8990 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fe5340057c0 tx=0x7fe534000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:31:46.796 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T03:31:46.794+0000 7fe550901640 1 -- 172.21.15.110:0/1295952947 <== mon.2 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fe544016030 con 0x7fe54c10a3c0 2024-09-21T03:31:46.961 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:31:46 smithi114 ceph-mon[34140]: from='osd.0 [v2:172.21.15.89:6802/990539042,v1:172.21.15.89:6803/990539042]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-09-21T03:31:46.961 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:31:46 smithi114 ceph-mon[34140]: osdmap e6: 1 total, 0 up, 1 in 2024-09-21T03:31:46.961 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:31:46 smithi114 ceph-mon[34140]: from='osd.0 [v2:172.21.15.89:6802/990539042,v1:172.21.15.89:6803/990539042]' entity='osd.0' cmd={"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi089", "root=default"]} : dispatch 2024-09-21T03:31:46.961 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:31:46 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-21T03:31:46.961 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:31:46 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:31:46.961 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:31:46 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:31:46.961 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:31:46 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-21T03:31:46.961 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:31:46 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:31:46.961 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:31:46 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:31:46.961 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:31:46 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:31:46.961 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:31:46 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:31:46.984 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T03:31:46.982+0000 7fe551903640 1 -- 172.21.15.110:0/1295952947 --> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi110:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7fe54c063800 con 0x7fe5380b6480 2024-09-21T03:31:47.311 INFO:journalctl@ceph.osd.0.smithi089.stdout:Sep 21 03:31:46 smithi089 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-0[46605]: 2024-09-21T03:31:46.908+0000 7f5437614640 -1 osd.0 0 waiting for initial osdmap 2024-09-21T03:31:47.311 INFO:journalctl@ceph.osd.0.smithi089.stdout:Sep 21 03:31:46 smithi089 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-0[46605]: 2024-09-21T03:31:46.914+0000 7f542f40a640 -1 osd.0 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-21T03:31:47.638 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:31:47 smithi089 ceph-mon[30928]: pgmap v53: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:31:47.638 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:31:47 smithi089 ceph-mon[30928]: Detected new or changed devices on smithi089 2024-09-21T03:31:47.638 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:31:47 smithi089 ceph-mon[30928]: Adjusting osd_memory_target on smithi089 to 17087M 2024-09-21T03:31:47.638 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:31:47 smithi089 ceph-mon[30928]: from='osd.0 [v2:172.21.15.89:6802/990539042,v1:172.21.15.89:6803/990539042]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi089", "root=default"]}]': finished 2024-09-21T03:31:47.638 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:31:47 smithi089 ceph-mon[30928]: osdmap e7: 1 total, 0 up, 1 in 2024-09-21T03:31:47.638 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:31:47 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-21T03:31:47.638 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:31:47 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-21T03:31:47.638 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:31:47 smithi089 ceph-mon[30928]: from='osd.0 [v2:172.21.15.89:6802/990539042,v1:172.21.15.89:6803/990539042]' entity='osd.0' 2024-09-21T03:31:47.638 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:31:47 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T03:31:47.638 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:31:47 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T03:31:47.638 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:31:47 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:31:47.645 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:31:47 smithi110 ceph-mon[34000]: pgmap v53: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:31:47.645 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:31:47 smithi110 ceph-mon[34000]: Detected new or changed devices on smithi089 2024-09-21T03:31:47.645 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:31:47 smithi110 ceph-mon[34000]: Adjusting osd_memory_target on smithi089 to 17087M 2024-09-21T03:31:47.645 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:31:47 smithi110 ceph-mon[34000]: from='osd.0 [v2:172.21.15.89:6802/990539042,v1:172.21.15.89:6803/990539042]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi089", "root=default"]}]': finished 2024-09-21T03:31:47.645 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:31:47 smithi110 ceph-mon[34000]: osdmap e7: 1 total, 0 up, 1 in 2024-09-21T03:31:47.645 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:31:47 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-21T03:31:47.645 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:31:47 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-21T03:31:47.645 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:31:47 smithi110 ceph-mon[34000]: from='osd.0 [v2:172.21.15.89:6802/990539042,v1:172.21.15.89:6803/990539042]' entity='osd.0' 2024-09-21T03:31:47.645 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:31:47 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T03:31:47.645 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:31:47 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T03:31:47.645 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:31:47 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:31:47.961 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:31:47 smithi114 ceph-mon[34140]: pgmap v53: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:31:47.961 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:31:47 smithi114 ceph-mon[34140]: Detected new or changed devices on smithi089 2024-09-21T03:31:47.961 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:31:47 smithi114 ceph-mon[34140]: Adjusting osd_memory_target on smithi089 to 17087M 2024-09-21T03:31:47.961 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:31:47 smithi114 ceph-mon[34140]: from='osd.0 [v2:172.21.15.89:6802/990539042,v1:172.21.15.89:6803/990539042]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi089", "root=default"]}]': finished 2024-09-21T03:31:47.961 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:31:47 smithi114 ceph-mon[34140]: osdmap e7: 1 total, 0 up, 1 in 2024-09-21T03:31:47.961 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:31:47 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-21T03:31:47.961 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:31:47 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-21T03:31:47.961 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:31:47 smithi114 ceph-mon[34140]: from='osd.0 [v2:172.21.15.89:6802/990539042,v1:172.21.15.89:6803/990539042]' entity='osd.0' 2024-09-21T03:31:47.961 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:31:47 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T03:31:47.962 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:31:47 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T03:31:47.962 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:31:47 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:31:48.635 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:31:48 smithi110 ceph-mon[34000]: purged_snaps scrub starts 2024-09-21T03:31:48.635 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:31:48 smithi110 ceph-mon[34000]: purged_snaps scrub ok 2024-09-21T03:31:48.635 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:31:48 smithi110 ceph-mon[34000]: from='client.24296 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi110:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:31:48.635 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:31:48 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-21T03:31:48.635 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:31:48 smithi110 ceph-mon[34000]: osd.0 [v2:172.21.15.89:6802/990539042,v1:172.21.15.89:6803/990539042] boot 2024-09-21T03:31:48.635 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:31:48 smithi110 ceph-mon[34000]: osdmap e8: 1 total, 1 up, 1 in 2024-09-21T03:31:48.636 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:31:48 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-21T03:31:48.811 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:31:48 smithi089 ceph-mon[30928]: purged_snaps scrub starts 2024-09-21T03:31:48.811 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:31:48 smithi089 ceph-mon[30928]: purged_snaps scrub ok 2024-09-21T03:31:48.811 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:31:48 smithi089 ceph-mon[30928]: from='client.24296 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi110:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:31:48.811 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:31:48 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-21T03:31:48.811 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:31:48 smithi089 ceph-mon[30928]: osd.0 [v2:172.21.15.89:6802/990539042,v1:172.21.15.89:6803/990539042] boot 2024-09-21T03:31:48.811 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:31:48 smithi089 ceph-mon[30928]: osdmap e8: 1 total, 1 up, 1 in 2024-09-21T03:31:48.811 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:31:48 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-21T03:31:48.961 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:31:48 smithi114 ceph-mon[34140]: purged_snaps scrub starts 2024-09-21T03:31:48.961 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:31:48 smithi114 ceph-mon[34140]: purged_snaps scrub ok 2024-09-21T03:31:48.961 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:31:48 smithi114 ceph-mon[34140]: from='client.24296 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi110:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:31:48.961 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:31:48 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-21T03:31:48.961 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:31:48 smithi114 ceph-mon[34140]: osd.0 [v2:172.21.15.89:6802/990539042,v1:172.21.15.89:6803/990539042] boot 2024-09-21T03:31:48.961 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:31:48 smithi114 ceph-mon[34140]: osdmap e8: 1 total, 1 up, 1 in 2024-09-21T03:31:48.961 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:31:48 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-21T03:31:49.742 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:31:49 smithi110 ceph-mon[34000]: pgmap v55: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:31:49.742 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:31:49 smithi110 ceph-mon[34000]: osdmap e9: 1 total, 1 up, 1 in 2024-09-21T03:31:49.743 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:31:49 smithi110 ceph-mon[34000]: from='client.? 172.21.15.110:0/614269431' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "5176a910-5cf9-49d7-aedb-a20cf36ea075"} : dispatch 2024-09-21T03:31:49.743 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:31:49 smithi110 ceph-mon[34000]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "5176a910-5cf9-49d7-aedb-a20cf36ea075"} : dispatch 2024-09-21T03:31:49.743 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:31:49 smithi110 ceph-mon[34000]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5176a910-5cf9-49d7-aedb-a20cf36ea075"}]': finished 2024-09-21T03:31:49.743 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:31:49 smithi110 ceph-mon[34000]: osdmap e10: 2 total, 1 up, 2 in 2024-09-21T03:31:49.743 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:31:49 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-21T03:31:49.811 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:31:49 smithi089 ceph-mon[30928]: pgmap v55: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:31:49.811 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:31:49 smithi089 ceph-mon[30928]: osdmap e9: 1 total, 1 up, 1 in 2024-09-21T03:31:49.811 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:31:49 smithi089 ceph-mon[30928]: from='client.? 172.21.15.110:0/614269431' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "5176a910-5cf9-49d7-aedb-a20cf36ea075"} : dispatch 2024-09-21T03:31:49.811 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:31:49 smithi089 ceph-mon[30928]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "5176a910-5cf9-49d7-aedb-a20cf36ea075"} : dispatch 2024-09-21T03:31:49.811 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:31:49 smithi089 ceph-mon[30928]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5176a910-5cf9-49d7-aedb-a20cf36ea075"}]': finished 2024-09-21T03:31:49.811 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:31:49 smithi089 ceph-mon[30928]: osdmap e10: 2 total, 1 up, 2 in 2024-09-21T03:31:49.811 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:31:49 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-21T03:31:49.961 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:31:49 smithi114 ceph-mon[34140]: pgmap v55: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:31:49.961 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:31:49 smithi114 ceph-mon[34140]: osdmap e9: 1 total, 1 up, 1 in 2024-09-21T03:31:49.961 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:31:49 smithi114 ceph-mon[34140]: from='client.? 172.21.15.110:0/614269431' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "5176a910-5cf9-49d7-aedb-a20cf36ea075"} : dispatch 2024-09-21T03:31:49.961 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:31:49 smithi114 ceph-mon[34140]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "5176a910-5cf9-49d7-aedb-a20cf36ea075"} : dispatch 2024-09-21T03:31:49.961 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:31:49 smithi114 ceph-mon[34140]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5176a910-5cf9-49d7-aedb-a20cf36ea075"}]': finished 2024-09-21T03:31:49.961 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:31:49 smithi114 ceph-mon[34140]: osdmap e10: 2 total, 1 up, 2 in 2024-09-21T03:31:49.961 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:31:49 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-21T03:31:50.750 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:31:50 smithi110 ceph-mon[34000]: from='client.? 172.21.15.110:0/4247401994' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T03:31:50.811 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:31:50 smithi089 ceph-mon[30928]: from='client.? 172.21.15.110:0/4247401994' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T03:31:50.960 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:31:50 smithi114 ceph-mon[34140]: from='client.? 172.21.15.110:0/4247401994' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T03:31:51.750 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:31:51 smithi110 ceph-mon[34000]: pgmap v59: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-21T03:31:51.811 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:31:51 smithi089 ceph-mon[30928]: pgmap v59: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-21T03:31:51.960 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:31:51 smithi114 ceph-mon[34140]: pgmap v59: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-21T03:31:53.750 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:31:53 smithi110 ceph-mon[34000]: pgmap v60: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-21T03:31:53.811 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:31:53 smithi089 ceph-mon[30928]: pgmap v60: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-21T03:31:53.960 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:31:53 smithi114 ceph-mon[34140]: pgmap v60: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-21T03:31:55.559 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:31:55 smithi110 ceph-mon[34000]: pgmap v61: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-21T03:31:55.811 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:31:55 smithi089 ceph-mon[30928]: pgmap v61: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-21T03:31:55.960 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:31:55 smithi114 ceph-mon[34140]: pgmap v61: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-21T03:31:57.750 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:31:57 smithi110 ceph-mon[34000]: pgmap v62: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-21T03:31:57.750 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:31:57 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.1"} : dispatch 2024-09-21T03:31:57.750 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:31:57 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:31:57.811 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:31:57 smithi089 ceph-mon[30928]: pgmap v62: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-21T03:31:57.811 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:31:57 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.1"} : dispatch 2024-09-21T03:31:57.811 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:31:57 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:31:57.960 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:31:57 smithi114 ceph-mon[34140]: pgmap v62: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-21T03:31:57.961 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:31:57 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.1"} : dispatch 2024-09-21T03:31:57.961 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:31:57 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:31:58.549 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:31:58 smithi110 ceph-mon[34000]: Deploying daemon osd.1 on smithi110 2024-09-21T03:31:58.811 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:31:58 smithi089 ceph-mon[30928]: Deploying daemon osd.1 on smithi110 2024-09-21T03:31:58.960 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:31:58 smithi114 ceph-mon[34140]: Deploying daemon osd.1 on smithi110 2024-09-21T03:31:59.692 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:31:59 smithi110 ceph-mon[34000]: pgmap v63: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-21T03:31:59.811 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:31:59 smithi089 ceph-mon[30928]: pgmap v63: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-21T03:31:59.960 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:31:59 smithi114 ceph-mon[34140]: pgmap v63: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-21T03:32:01.811 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:32:01 smithi089 ceph-mon[30928]: pgmap v64: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-21T03:32:01.821 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:32:01 smithi110 ceph-mon[34000]: pgmap v64: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-21T03:32:01.960 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:32:01 smithi114 ceph-mon[34140]: pgmap v64: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-21T03:32:03.781 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:32:03 smithi110 ceph-mon[34000]: pgmap v65: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-21T03:32:03.781 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:32:03 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:32:03.781 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:32:03 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:32:03.781 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:32:03 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:32:03.811 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:32:03 smithi089 ceph-mon[30928]: pgmap v65: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-21T03:32:03.811 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:32:03 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:32:03.811 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:32:03 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:32:03.811 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:32:03 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:32:03.961 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:32:03 smithi114 ceph-mon[34140]: pgmap v65: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-21T03:32:03.961 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:32:03 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:32:03.961 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:32:03 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:32:03.961 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:32:03 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:32:05.750 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:32:05 smithi110 ceph-mon[34000]: pgmap v66: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-21T03:32:05.811 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:32:05 smithi089 ceph-mon[30928]: pgmap v66: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-21T03:32:05.960 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:32:05 smithi114 ceph-mon[34140]: pgmap v66: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-21T03:32:06.919 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:32:06 smithi110 ceph-mon[34000]: pgmap v67: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-21T03:32:06.919 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:32:06 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:32:06.919 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:32:06 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:32:06.919 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:32:06 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:32:06.919 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:32:06 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:32:06.919 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:32:06 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:32:06.961 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:32:06 smithi114 ceph-mon[34140]: pgmap v67: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-21T03:32:06.961 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:32:06 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:32:06.961 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:32:06 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:32:06.961 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:32:06 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:32:06.961 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:32:06 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:32:06.961 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:32:06 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:32:07.061 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:32:06 smithi089 ceph-mon[30928]: pgmap v67: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-21T03:32:07.061 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:32:06 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:32:07.061 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:32:06 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:32:07.061 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:32:06 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:32:07.061 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:32:06 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:32:07.061 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:32:06 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:32:07.246 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T03:32:07.244+0000 7fe550901640 1 -- 172.21.15.110:0/1295952947 <== mgr.14152 v2:172.21.15.89:6800/1591532358 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fe54c063800 con 0x7fe5380b6480 2024-09-21T03:32:07.247 INFO:teuthology.orchestra.run.smithi110.stdout:Created osd(s) 1 on host 'smithi110' 2024-09-21T03:32:07.249 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T03:32:07.247+0000 7fe551903640 1 -- 172.21.15.110:0/1295952947 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7fe5380b6480 msgr2=0x7fe5380b8990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:32:07.250 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T03:32:07.247+0000 7fe551903640 1 --2- 172.21.15.110:0/1295952947 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7fe5380b6480 0x7fe5380b8990 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fe5340057c0 tx=0x7fe534000bc0 comp rx=0 tx=0).stop 2024-09-21T03:32:07.250 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T03:32:07.247+0000 7fe551903640 1 -- 172.21.15.110:0/1295952947 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe54c10a3c0 msgr2=0x7fe54c10d5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:32:07.250 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T03:32:07.247+0000 7fe551903640 1 --2- 172.21.15.110:0/1295952947 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe54c10a3c0 0x7fe54c10d5b0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fe54400ea80 tx=0x7fe54400ef40 comp rx=0 tx=0).stop 2024-09-21T03:32:07.250 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T03:32:07.247+0000 7fe551903640 1 -- 172.21.15.110:0/1295952947 shutdown_connections 2024-09-21T03:32:07.250 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T03:32:07.247+0000 7fe551903640 1 --2- 172.21.15.110:0/1295952947 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7fe5380b6480 0x7fe5380b8990 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:07.250 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T03:32:07.247+0000 7fe551903640 1 --2- 172.21.15.110:0/1295952947 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe54c111f90 0x7fe54c10c3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:07.251 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T03:32:07.247+0000 7fe551903640 1 --2- 172.21.15.110:0/1295952947 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe54c10ae50 0x7fe54c10be80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:07.251 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T03:32:07.247+0000 7fe551903640 1 --2- 172.21.15.110:0/1295952947 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe54c10a3c0 0x7fe54c10d5b0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:07.251 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T03:32:07.247+0000 7fe551903640 1 -- 172.21.15.110:0/1295952947 >> 172.21.15.110:0/1295952947 conn(0x7fe54c070950 msgr2=0x7fe54c10df70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:32:07.251 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T03:32:07.247+0000 7fe551903640 1 -- 172.21.15.110:0/1295952947 shutdown_connections 2024-09-21T03:32:07.251 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T03:32:07.247+0000 7fe551903640 1 -- 172.21.15.110:0/1295952947 wait complete. 2024-09-21T03:32:07.775 DEBUG:teuthology.orchestra.run.smithi110:osd.1> sudo journalctl -f -n 0 -u ceph-7c374098-77c9-11ef-baf5-efdc52797490@osd.1.service 2024-09-21T03:32:07.777 INFO:tasks.cephadm:Deploying osd.2 on smithi114 with /dev/vg_nvme/lv_4... 2024-09-21T03:32:07.777 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7c374098-77c9-11ef-baf5-efdc52797490 -- lvm zap /dev/vg_nvme/lv_4 2024-09-21T03:32:07.797 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:32:07 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:32:07.797 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:32:07 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:32:07.797 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:32:07 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:32:07.960 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:32:07 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:32:07.960 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:32:07 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:32:07.960 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:32:07 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:32:08.028 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/7c374098-77c9-11ef-baf5-efdc52797490/mon.c/config 2024-09-21T03:32:08.061 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:32:07 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:32:08.061 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:32:07 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:32:08.061 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:32:07 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:32:09.500 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:32:09 smithi110 ceph-mon[34000]: pgmap v68: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-21T03:32:09.500 INFO:journalctl@ceph.osd.1.smithi110.stdout:Sep 21 03:32:09 smithi110 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-1[37931]: 2024-09-21T03:32:09.314+0000 7f1f6d696640 -1 osd.1 0 log_to_monitors true 2024-09-21T03:32:09.561 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:32:09 smithi089 ceph-mon[30928]: pgmap v68: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-21T03:32:09.583 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:32:09 smithi114 ceph-mon[34140]: pgmap v68: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-21T03:32:10.334 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-21T03:32:10.359 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7c374098-77c9-11ef-baf5-efdc52797490 -- ceph orch daemon add osd smithi114:vg_nvme/lv_4 2024-09-21T03:32:10.446 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:32:10 smithi110 ceph-mon[34000]: from='osd.1 [v2:172.21.15.110:6800/3235852628,v1:172.21.15.110:6801/3235852628]' entity='osd.1' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]} : dispatch 2024-09-21T03:32:10.446 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:32:10 smithi110 ceph-mon[34000]: from='osd.1 ' entity='osd.1' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]} : dispatch 2024-09-21T03:32:10.460 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:32:10 smithi114 ceph-mon[34140]: from='osd.1 [v2:172.21.15.110:6800/3235852628,v1:172.21.15.110:6801/3235852628]' entity='osd.1' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]} : dispatch 2024-09-21T03:32:10.461 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:32:10 smithi114 ceph-mon[34140]: from='osd.1 ' entity='osd.1' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]} : dispatch 2024-09-21T03:32:10.561 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:32:10 smithi089 ceph-mon[30928]: from='osd.1 [v2:172.21.15.110:6800/3235852628,v1:172.21.15.110:6801/3235852628]' entity='osd.1' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]} : dispatch 2024-09-21T03:32:10.561 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:32:10 smithi089 ceph-mon[30928]: from='osd.1 ' entity='osd.1' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]} : dispatch 2024-09-21T03:32:10.611 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/7c374098-77c9-11ef-baf5-efdc52797490/mon.c/config 2024-09-21T03:32:11.092 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T03:32:11.088+0000 7febdac7b640 1 -- 172.21.15.114:0/2526307251 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7febd4102670 msgr2=0x7febd4102ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:32:11.092 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T03:32:11.088+0000 7febdac7b640 1 --2- 172.21.15.114:0/2526307251 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7febd4102670 0x7febd4102ae0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7febd0009ca0 tx=0x7febd002a950 comp rx=0 tx=0).stop 2024-09-21T03:32:11.092 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T03:32:11.088+0000 7febdac7b640 1 -- 172.21.15.114:0/2526307251 shutdown_connections 2024-09-21T03:32:11.092 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T03:32:11.088+0000 7febdac7b640 1 --2- 172.21.15.114:0/2526307251 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7febd4103700 0x7febd4103b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:11.092 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T03:32:11.088+0000 7febdac7b640 1 --2- 172.21.15.114:0/2526307251 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7febd41040a0 0x7febd410aa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:11.092 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T03:32:11.088+0000 7febdac7b640 1 --2- 172.21.15.114:0/2526307251 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7febd4102670 0x7febd4102ae0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:11.092 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T03:32:11.088+0000 7febdac7b640 1 -- 172.21.15.114:0/2526307251 >> 172.21.15.114:0/2526307251 conn(0x7febd40fdfe0 msgr2=0x7febd4100490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:32:11.092 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T03:32:11.088+0000 7febdac7b640 1 -- 172.21.15.114:0/2526307251 shutdown_connections 2024-09-21T03:32:11.092 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T03:32:11.088+0000 7febdac7b640 1 -- 172.21.15.114:0/2526307251 wait complete. 2024-09-21T03:32:11.092 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T03:32:11.089+0000 7febdac7b640 1 Processor -- start 2024-09-21T03:32:11.093 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T03:32:11.089+0000 7febdac7b640 1 -- start start 2024-09-21T03:32:11.093 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T03:32:11.089+0000 7febdac7b640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7febd4102670 0x7febd4198e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:11.093 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T03:32:11.089+0000 7febdac7b640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7febd4103700 0x7febd41994b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:11.093 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T03:32:11.089+0000 7febdac7b640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7febd41040a0 0x7febd419a050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:11.093 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T03:32:11.089+0000 7febdac7b640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7febd410e1b0 con 0x7febd4103700 2024-09-21T03:32:11.093 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T03:32:11.089+0000 7febdac7b640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7febd410e030 con 0x7febd4102670 2024-09-21T03:32:11.093 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T03:32:11.089+0000 7febdac7b640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7febd410e330 con 0x7febd41040a0 2024-09-21T03:32:11.093 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T03:32:11.089+0000 7febd9218640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7febd41040a0 0x7febd419a050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:11.093 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T03:32:11.089+0000 7febd9218640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7febd41040a0 0x7febd419a050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:48302/0 (socket says 172.21.15.114:48302) 2024-09-21T03:32:11.093 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T03:32:11.089+0000 7febd9218640 1 -- 172.21.15.114:0/2289789717 learned_addr learned my addr 172.21.15.114:0/2289789717 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T03:32:11.093 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T03:32:11.089+0000 7febd8a17640 1 --2- 172.21.15.114:0/2289789717 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7febd4102670 0x7febd4198e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:11.093 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T03:32:11.089+0000 7febcbfff640 1 --2- 172.21.15.114:0/2289789717 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7febd4103700 0x7febd41994b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:11.093 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T03:32:11.089+0000 7febd9218640 1 -- 172.21.15.114:0/2289789717 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7febd4102670 msgr2=0x7febd4198e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:32:11.093 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T03:32:11.089+0000 7febd9218640 1 --2- 172.21.15.114:0/2289789717 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7febd4102670 0x7febd4198e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:11.093 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T03:32:11.089+0000 7febd9218640 1 -- 172.21.15.114:0/2289789717 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7febd4103700 msgr2=0x7febd41994b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:32:11.094 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T03:32:11.089+0000 7febd9218640 1 --2- 172.21.15.114:0/2289789717 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7febd4103700 0x7febd41994b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:11.094 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T03:32:11.089+0000 7febd9218640 1 -- 172.21.15.114:0/2289789717 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7febd419a870 con 0x7febd41040a0 2024-09-21T03:32:11.094 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T03:32:11.090+0000 7febd9218640 1 --2- 172.21.15.114:0/2289789717 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7febd41040a0 0x7febd419a050 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7febcc007850 tx=0x7febcc007d10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:32:11.094 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T03:32:11.090+0000 7febc9ffb640 1 -- 172.21.15.114:0/2289789717 <== mon.1 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febcc019020 con 0x7febd41040a0 2024-09-21T03:32:11.094 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T03:32:11.090+0000 7febc9ffb640 1 -- 172.21.15.114:0/2289789717 <== mon.1 v2:172.21.15.114:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7febcc0054b0 con 0x7febd41040a0 2024-09-21T03:32:11.094 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T03:32:11.090+0000 7febc9ffb640 1 -- 172.21.15.114:0/2289789717 <== mon.1 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febcc00e630 con 0x7febd41040a0 2024-09-21T03:32:11.094 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T03:32:11.090+0000 7febdac7b640 1 -- 172.21.15.114:0/2289789717 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7febd419ab60 con 0x7febd41040a0 2024-09-21T03:32:11.094 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T03:32:11.090+0000 7febdac7b640 1 -- 172.21.15.114:0/2289789717 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7febd406bd30 con 0x7febd41040a0 2024-09-21T03:32:11.094 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T03:32:11.091+0000 7febdac7b640 1 -- 172.21.15.114:0/2289789717 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7febd419adc0 con 0x7febd41040a0 2024-09-21T03:32:11.094 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T03:32:11.092+0000 7febc9ffb640 1 -- 172.21.15.114:0/2289789717 <== mon.1 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 16) ==== 154456+0+0 (secure 0 0 0) 0x7febcc0046c0 con 0x7febd41040a0 2024-09-21T03:32:11.095 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T03:32:11.093+0000 7febc9ffb640 1 --2- 172.21.15.114:0/2289789717 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7febbc0b6500 0x7febbc0b8a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:11.095 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T03:32:11.094+0000 7febc9ffb640 1 -- 172.21.15.114:0/2289789717 <== mon.1 v2:172.21.15.114:3300/0 5 ==== osd_map(11..11 src has 1..11) ==== 1903+0+0 (secure 0 0 0) 0x7febcc0e7340 con 0x7febd41040a0 2024-09-21T03:32:11.095 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T03:32:11.094+0000 7febd8a17640 1 --2- 172.21.15.114:0/2289789717 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7febbc0b6500 0x7febbc0b8a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:11.096 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T03:32:11.094+0000 7febd8a17640 1 --2- 172.21.15.114:0/2289789717 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7febbc0b6500 0x7febbc0b8a10 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7febd002af40 tx=0x7febd0000f40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:32:11.099 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T03:32:11.097+0000 7febc9ffb640 1 -- 172.21.15.114:0/2289789717 <== mon.1 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7febcc023020 con 0x7febd41040a0 2024-09-21T03:32:11.294 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T03:32:11.293+0000 7febdac7b640 1 -- 172.21.15.114:0/2289789717 --> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi114:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7febd4109360 con 0x7febbc0b6500 2024-09-21T03:32:11.561 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:32:11 smithi089 ceph-mon[30928]: pgmap v69: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-21T03:32:11.561 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:32:11 smithi089 ceph-mon[30928]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-09-21T03:32:11.561 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:32:11 smithi089 ceph-mon[30928]: from='osd.1 [v2:172.21.15.110:6800/3235852628,v1:172.21.15.110:6801/3235852628]' entity='osd.1' cmd={"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi110", "root=default"]} : dispatch 2024-09-21T03:32:11.561 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:32:11 smithi089 ceph-mon[30928]: osdmap e11: 2 total, 1 up, 2 in 2024-09-21T03:32:11.561 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:32:11 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-21T03:32:11.561 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:32:11 smithi089 ceph-mon[30928]: from='osd.1 ' entity='osd.1' cmd={"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi110", "root=default"]} : dispatch 2024-09-21T03:32:11.562 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:32:11 smithi089 ceph-mon[30928]: Detected new or changed devices on smithi110 2024-09-21T03:32:11.562 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:32:11 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:32:11.562 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:32:11 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:32:11.562 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:32:11 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-21T03:32:11.562 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:32:11 smithi089 ceph-mon[30928]: Adjusting osd_memory_target on smithi110 to 17088M 2024-09-21T03:32:11.562 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:32:11 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:32:11.562 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:32:11 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:32:11.562 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:32:11 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:32:11.562 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:32:11 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:32:11.623 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:32:11 smithi114 ceph-mon[34140]: pgmap v69: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-21T03:32:11.623 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:32:11 smithi114 ceph-mon[34140]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-09-21T03:32:11.623 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:32:11 smithi114 ceph-mon[34140]: from='osd.1 [v2:172.21.15.110:6800/3235852628,v1:172.21.15.110:6801/3235852628]' entity='osd.1' cmd={"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi110", "root=default"]} : dispatch 2024-09-21T03:32:11.623 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:32:11 smithi114 ceph-mon[34140]: osdmap e11: 2 total, 1 up, 2 in 2024-09-21T03:32:11.623 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:32:11 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-21T03:32:11.623 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:32:11 smithi114 ceph-mon[34140]: from='osd.1 ' entity='osd.1' cmd={"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi110", "root=default"]} : dispatch 2024-09-21T03:32:11.623 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:32:11 smithi114 ceph-mon[34140]: Detected new or changed devices on smithi110 2024-09-21T03:32:11.624 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:32:11 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:32:11.624 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:32:11 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:32:11.624 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:32:11 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-21T03:32:11.624 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:32:11 smithi114 ceph-mon[34140]: Adjusting osd_memory_target on smithi110 to 17088M 2024-09-21T03:32:11.624 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:32:11 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:32:11.624 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:32:11 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:32:11.624 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:32:11 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:32:11.624 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:32:11 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:32:11.678 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:32:11 smithi110 ceph-mon[34000]: pgmap v69: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-21T03:32:11.678 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:32:11 smithi110 ceph-mon[34000]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-09-21T03:32:11.678 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:32:11 smithi110 ceph-mon[34000]: from='osd.1 [v2:172.21.15.110:6800/3235852628,v1:172.21.15.110:6801/3235852628]' entity='osd.1' cmd={"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi110", "root=default"]} : dispatch 2024-09-21T03:32:11.678 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:32:11 smithi110 ceph-mon[34000]: osdmap e11: 2 total, 1 up, 2 in 2024-09-21T03:32:11.678 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:32:11 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-21T03:32:11.678 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:32:11 smithi110 ceph-mon[34000]: from='osd.1 ' entity='osd.1' cmd={"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi110", "root=default"]} : dispatch 2024-09-21T03:32:11.678 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:32:11 smithi110 ceph-mon[34000]: Detected new or changed devices on smithi110 2024-09-21T03:32:11.678 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:32:11 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:32:11.678 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:32:11 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:32:11.678 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:32:11 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-21T03:32:11.678 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:32:11 smithi110 ceph-mon[34000]: Adjusting osd_memory_target on smithi110 to 17088M 2024-09-21T03:32:11.678 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:32:11 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:32:11.679 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:32:11 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:32:11.679 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:32:11 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:32:11.679 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:32:11 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:32:12.000 INFO:journalctl@ceph.osd.1.smithi110.stdout:Sep 21 03:32:11 smithi110 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-1[37931]: 2024-09-21T03:32:11.675+0000 7f1f69618640 -1 osd.1 0 waiting for initial osdmap 2024-09-21T03:32:12.000 INFO:journalctl@ceph.osd.1.smithi110.stdout:Sep 21 03:32:11 smithi110 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-1[37931]: 2024-09-21T03:32:11.678+0000 7f1f6140e640 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-21T03:32:12.561 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:32:12 smithi089 ceph-mon[30928]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi110", "root=default"]}]': finished 2024-09-21T03:32:12.561 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:32:12 smithi089 ceph-mon[30928]: osdmap e12: 2 total, 1 up, 2 in 2024-09-21T03:32:12.561 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:32:12 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-21T03:32:12.561 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:32:12 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-21T03:32:12.561 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:32:12 smithi089 ceph-mon[30928]: from='client.24295 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi114:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:32:12.561 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:32:12 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T03:32:12.561 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:32:12 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T03:32:12.562 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:32:12 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:32:12.562 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:32:12 smithi089 ceph-mon[30928]: from='osd.1 ' entity='osd.1' 2024-09-21T03:32:12.618 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:32:12 smithi114 ceph-mon[34140]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi110", "root=default"]}]': finished 2024-09-21T03:32:12.618 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:32:12 smithi114 ceph-mon[34140]: osdmap e12: 2 total, 1 up, 2 in 2024-09-21T03:32:12.618 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:32:12 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-21T03:32:12.618 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:32:12 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-21T03:32:12.618 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:32:12 smithi114 ceph-mon[34140]: from='client.24295 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi114:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:32:12.618 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:32:12 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T03:32:12.619 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:32:12 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T03:32:12.619 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:32:12 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:32:12.619 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:32:12 smithi114 ceph-mon[34140]: from='osd.1 ' entity='osd.1' 2024-09-21T03:32:12.750 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:32:12 smithi110 ceph-mon[34000]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi110", "root=default"]}]': finished 2024-09-21T03:32:12.750 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:32:12 smithi110 ceph-mon[34000]: osdmap e12: 2 total, 1 up, 2 in 2024-09-21T03:32:12.750 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:32:12 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-21T03:32:12.750 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:32:12 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-21T03:32:12.750 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:32:12 smithi110 ceph-mon[34000]: from='client.24295 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi114:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:32:12.750 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:32:12 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T03:32:12.750 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:32:12 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T03:32:12.751 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:32:12 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:32:12.751 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:32:12 smithi110 ceph-mon[34000]: from='osd.1 ' entity='osd.1' 2024-09-21T03:32:13.461 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:32:13 smithi114 ceph-mon[34140]: purged_snaps scrub starts 2024-09-21T03:32:13.461 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:32:13 smithi114 ceph-mon[34140]: purged_snaps scrub ok 2024-09-21T03:32:13.461 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:32:13 smithi114 ceph-mon[34140]: pgmap v72: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-21T03:32:13.461 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:32:13 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-21T03:32:13.461 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:32:13 smithi114 ceph-mon[34140]: osd.1 [v2:172.21.15.110:6800/3235852628,v1:172.21.15.110:6801/3235852628] boot 2024-09-21T03:32:13.461 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:32:13 smithi114 ceph-mon[34140]: osdmap e13: 2 total, 2 up, 2 in 2024-09-21T03:32:13.461 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:32:13 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-21T03:32:13.561 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:32:13 smithi089 ceph-mon[30928]: purged_snaps scrub starts 2024-09-21T03:32:13.561 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:32:13 smithi089 ceph-mon[30928]: purged_snaps scrub ok 2024-09-21T03:32:13.561 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:32:13 smithi089 ceph-mon[30928]: pgmap v72: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-21T03:32:13.561 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:32:13 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-21T03:32:13.561 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:32:13 smithi089 ceph-mon[30928]: osd.1 [v2:172.21.15.110:6800/3235852628,v1:172.21.15.110:6801/3235852628] boot 2024-09-21T03:32:13.561 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:32:13 smithi089 ceph-mon[30928]: osdmap e13: 2 total, 2 up, 2 in 2024-09-21T03:32:13.561 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:32:13 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-21T03:32:13.750 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:32:13 smithi110 ceph-mon[34000]: purged_snaps scrub starts 2024-09-21T03:32:13.750 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:32:13 smithi110 ceph-mon[34000]: purged_snaps scrub ok 2024-09-21T03:32:13.750 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:32:13 smithi110 ceph-mon[34000]: pgmap v72: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-21T03:32:13.750 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:32:13 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-21T03:32:13.750 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:32:13 smithi110 ceph-mon[34000]: osd.1 [v2:172.21.15.110:6800/3235852628,v1:172.21.15.110:6801/3235852628] boot 2024-09-21T03:32:13.750 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:32:13 smithi110 ceph-mon[34000]: osdmap e13: 2 total, 2 up, 2 in 2024-09-21T03:32:13.750 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:32:13 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-21T03:32:14.460 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:32:14 smithi114 ceph-mon[34140]: from='client.? 172.21.15.114:0/3210309064' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "d4977df1-b118-4d7e-b9a9-626a8accc812"} : dispatch 2024-09-21T03:32:14.461 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:32:14 smithi114 ceph-mon[34140]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "d4977df1-b118-4d7e-b9a9-626a8accc812"} : dispatch 2024-09-21T03:32:14.461 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:32:14 smithi114 ceph-mon[34140]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d4977df1-b118-4d7e-b9a9-626a8accc812"}]': finished 2024-09-21T03:32:14.461 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:32:14 smithi114 ceph-mon[34140]: osdmap e14: 3 total, 2 up, 3 in 2024-09-21T03:32:14.461 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:32:14 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-21T03:32:14.461 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:32:14 smithi114 ceph-mon[34140]: from='client.? 172.21.15.114:0/2546111902' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T03:32:14.561 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:32:14 smithi089 ceph-mon[30928]: from='client.? 172.21.15.114:0/3210309064' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "d4977df1-b118-4d7e-b9a9-626a8accc812"} : dispatch 2024-09-21T03:32:14.561 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:32:14 smithi089 ceph-mon[30928]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "d4977df1-b118-4d7e-b9a9-626a8accc812"} : dispatch 2024-09-21T03:32:14.561 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:32:14 smithi089 ceph-mon[30928]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d4977df1-b118-4d7e-b9a9-626a8accc812"}]': finished 2024-09-21T03:32:14.561 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:32:14 smithi089 ceph-mon[30928]: osdmap e14: 3 total, 2 up, 3 in 2024-09-21T03:32:14.561 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:32:14 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-21T03:32:14.561 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:32:14 smithi089 ceph-mon[30928]: from='client.? 172.21.15.114:0/2546111902' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T03:32:14.750 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:32:14 smithi110 ceph-mon[34000]: from='client.? 172.21.15.114:0/3210309064' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "d4977df1-b118-4d7e-b9a9-626a8accc812"} : dispatch 2024-09-21T03:32:14.750 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:32:14 smithi110 ceph-mon[34000]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "d4977df1-b118-4d7e-b9a9-626a8accc812"} : dispatch 2024-09-21T03:32:14.750 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:32:14 smithi110 ceph-mon[34000]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d4977df1-b118-4d7e-b9a9-626a8accc812"}]': finished 2024-09-21T03:32:14.750 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:32:14 smithi110 ceph-mon[34000]: osdmap e14: 3 total, 2 up, 3 in 2024-09-21T03:32:14.750 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:32:14 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-21T03:32:14.751 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:32:14 smithi110 ceph-mon[34000]: from='client.? 172.21.15.114:0/2546111902' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T03:32:15.750 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:32:15 smithi110 ceph-mon[34000]: pgmap v75: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-09-21T03:32:15.811 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:32:15 smithi089 ceph-mon[30928]: pgmap v75: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-09-21T03:32:15.960 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:32:15 smithi114 ceph-mon[34140]: pgmap v75: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-09-21T03:32:17.750 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:32:17 smithi110 ceph-mon[34000]: pgmap v76: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-09-21T03:32:17.811 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:32:17 smithi089 ceph-mon[30928]: pgmap v76: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-09-21T03:32:17.960 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:32:17 smithi114 ceph-mon[34140]: pgmap v76: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-09-21T03:32:19.723 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:32:19 smithi114 ceph-mon[34140]: pgmap v77: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-09-21T03:32:19.750 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:32:19 smithi110 ceph-mon[34000]: pgmap v77: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-09-21T03:32:19.810 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:32:19 smithi089 ceph-mon[30928]: pgmap v77: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-09-21T03:32:21.572 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:32:21 smithi114 ceph-mon[34140]: pgmap v78: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-09-21T03:32:21.750 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:32:21 smithi110 ceph-mon[34000]: pgmap v78: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-09-21T03:32:21.811 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:32:21 smithi089 ceph-mon[30928]: pgmap v78: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-09-21T03:32:22.710 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:32:22 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.2"} : dispatch 2024-09-21T03:32:22.711 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:32:22 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:32:22.750 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:32:22 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.2"} : dispatch 2024-09-21T03:32:22.750 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:32:22 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:32:22.811 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:32:22 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.2"} : dispatch 2024-09-21T03:32:22.811 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:32:22 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:32:23.711 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:32:23 smithi114 ceph-mon[34140]: pgmap v79: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-09-21T03:32:23.711 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:32:23 smithi114 ceph-mon[34140]: Deploying daemon osd.2 on smithi114 2024-09-21T03:32:23.750 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:32:23 smithi110 ceph-mon[34000]: pgmap v79: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-09-21T03:32:23.750 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:32:23 smithi110 ceph-mon[34000]: Deploying daemon osd.2 on smithi114 2024-09-21T03:32:23.811 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:32:23 smithi089 ceph-mon[30928]: pgmap v79: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-09-21T03:32:23.811 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:32:23 smithi089 ceph-mon[30928]: Deploying daemon osd.2 on smithi114 2024-09-21T03:32:25.750 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:32:25 smithi110 ceph-mon[34000]: pgmap v80: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-09-21T03:32:25.811 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:32:25 smithi089 ceph-mon[30928]: pgmap v80: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-09-21T03:32:25.961 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:32:25 smithi114 ceph-mon[34140]: pgmap v80: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-09-21T03:32:27.638 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:32:27 smithi114 ceph-mon[34140]: pgmap v81: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-09-21T03:32:27.750 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:32:27 smithi110 ceph-mon[34000]: pgmap v81: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-09-21T03:32:27.811 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:32:27 smithi089 ceph-mon[30928]: pgmap v81: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-09-21T03:32:28.695 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:32:28 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:32:28.695 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:32:28 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:32:28.695 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:32:28 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:32:28.811 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:32:28 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:32:28.811 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:32:28 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:32:28.811 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:32:28 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:32:29.000 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:32:28 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:32:29.000 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:32:28 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:32:29.000 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:32:28 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:32:29.591 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:32:29 smithi114 ceph-mon[34140]: pgmap v82: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-09-21T03:32:29.811 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:32:29 smithi089 ceph-mon[30928]: pgmap v82: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-09-21T03:32:30.000 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:32:29 smithi110 ceph-mon[34000]: pgmap v82: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-09-21T03:32:31.311 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:32:31 smithi089 ceph-mon[30928]: pgmap v83: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-09-21T03:32:31.311 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:32:31 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:32:31.311 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:32:31 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:32:31.311 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:32:31 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:32:31.311 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:32:31 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:32:31.311 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:32:31 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:32:31.371 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:32:31 smithi114 ceph-mon[34140]: pgmap v83: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-09-21T03:32:31.371 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:32:31 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:32:31.371 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:32:31 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:32:31.371 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:32:31 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:32:31.371 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:32:31 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:32:31.372 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:32:31 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:32:31.500 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:32:31 smithi110 ceph-mon[34000]: pgmap v83: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-09-21T03:32:31.500 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:32:31 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:32:31.500 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:32:31 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:32:31.500 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:32:31 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:32:31.500 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:32:31 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:32:31.500 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:32:31 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:32:31.703 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T03:32:31.700+0000 7febc9ffb640 1 -- 172.21.15.114:0/2289789717 <== mgr.14152 v2:172.21.15.89:6800/1591532358 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7febd4109360 con 0x7febbc0b6500 2024-09-21T03:32:31.704 INFO:teuthology.orchestra.run.smithi114.stdout:Created osd(s) 2 on host 'smithi114' 2024-09-21T03:32:31.706 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T03:32:31.703+0000 7febdac7b640 1 -- 172.21.15.114:0/2289789717 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7febbc0b6500 msgr2=0x7febbc0b8a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:32:31.706 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T03:32:31.703+0000 7febdac7b640 1 --2- 172.21.15.114:0/2289789717 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7febbc0b6500 0x7febbc0b8a10 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7febd002af40 tx=0x7febd0000f40 comp rx=0 tx=0).stop 2024-09-21T03:32:31.706 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T03:32:31.703+0000 7febdac7b640 1 -- 172.21.15.114:0/2289789717 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7febd41040a0 msgr2=0x7febd419a050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:32:31.706 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T03:32:31.703+0000 7febdac7b640 1 --2- 172.21.15.114:0/2289789717 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7febd41040a0 0x7febd419a050 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7febcc007850 tx=0x7febcc007d10 comp rx=0 tx=0).stop 2024-09-21T03:32:31.706 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T03:32:31.703+0000 7febdac7b640 1 -- 172.21.15.114:0/2289789717 shutdown_connections 2024-09-21T03:32:31.706 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T03:32:31.703+0000 7febdac7b640 1 --2- 172.21.15.114:0/2289789717 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7febbc0b6500 0x7febbc0b8a10 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:31.706 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T03:32:31.703+0000 7febdac7b640 1 --2- 172.21.15.114:0/2289789717 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7febd41040a0 0x7febd419a050 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:31.706 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T03:32:31.703+0000 7febdac7b640 1 --2- 172.21.15.114:0/2289789717 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7febd4103700 0x7febd41994b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:31.706 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T03:32:31.703+0000 7febdac7b640 1 --2- 172.21.15.114:0/2289789717 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7febd4102670 0x7febd4198e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:31.706 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T03:32:31.703+0000 7febdac7b640 1 -- 172.21.15.114:0/2289789717 >> 172.21.15.114:0/2289789717 conn(0x7febd40fdfe0 msgr2=0x7febd4106a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:32:31.706 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T03:32:31.703+0000 7febdac7b640 1 -- 172.21.15.114:0/2289789717 shutdown_connections 2024-09-21T03:32:31.706 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T03:32:31.703+0000 7febdac7b640 1 -- 172.21.15.114:0/2289789717 wait complete. 2024-09-21T03:32:32.300 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:32:32 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:32:32.300 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:32:32 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:32:32.301 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:32:32 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:32:32.311 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:32:32 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:32:32.311 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:32:32 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:32:32.311 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:32:32 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:32:32.326 DEBUG:teuthology.orchestra.run.smithi114:osd.2> sudo journalctl -f -n 0 -u ceph-7c374098-77c9-11ef-baf5-efdc52797490@osd.2.service 2024-09-21T03:32:32.328 INFO:tasks.cephadm:Deploying osd.3 on smithi089 with /dev/vg_nvme/lv_3... 2024-09-21T03:32:32.328 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7c374098-77c9-11ef-baf5-efdc52797490 -- lvm zap /dev/vg_nvme/lv_3 2024-09-21T03:32:32.500 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:32:32 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:32:32.500 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:32:32 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:32:32.500 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:32:32 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:32:32.580 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/7c374098-77c9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:32:33.311 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:32:33 smithi089 ceph-mon[30928]: pgmap v84: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-09-21T03:32:33.401 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:32:33 smithi110 ceph-mon[34000]: pgmap v84: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-09-21T03:32:33.440 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:32:33 smithi114 ceph-mon[34140]: pgmap v84: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-09-21T03:32:34.035 INFO:journalctl@ceph.osd.2.smithi114.stdout:Sep 21 03:32:33 smithi114 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-2[38568]: 2024-09-21T03:32:33.813+0000 7fd1881c0640 -1 osd.2 0 log_to_monitors true 2024-09-21T03:32:34.311 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:32:34 smithi089 ceph-mon[30928]: from='osd.2 [v2:172.21.15.114:6800/169452122,v1:172.21.15.114:6801/169452122]' entity='osd.2' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]} : dispatch 2024-09-21T03:32:34.311 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:32:34 smithi089 ceph-mon[30928]: from='osd.2 ' entity='osd.2' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]} : dispatch 2024-09-21T03:32:34.460 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:32:34 smithi114 ceph-mon[34140]: from='osd.2 [v2:172.21.15.114:6800/169452122,v1:172.21.15.114:6801/169452122]' entity='osd.2' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]} : dispatch 2024-09-21T03:32:34.461 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:32:34 smithi114 ceph-mon[34140]: from='osd.2 ' entity='osd.2' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]} : dispatch 2024-09-21T03:32:34.499 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:32:34 smithi110 ceph-mon[34000]: from='osd.2 [v2:172.21.15.114:6800/169452122,v1:172.21.15.114:6801/169452122]' entity='osd.2' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]} : dispatch 2024-09-21T03:32:34.500 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:32:34 smithi110 ceph-mon[34000]: from='osd.2 ' entity='osd.2' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]} : dispatch 2024-09-21T03:32:35.142 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-09-21T03:32:35.169 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7c374098-77c9-11ef-baf5-efdc52797490 -- ceph orch daemon add osd smithi089:vg_nvme/lv_3 2024-09-21T03:32:35.311 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:32:35 smithi089 ceph-mon[30928]: pgmap v85: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-09-21T03:32:35.311 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:32:35 smithi089 ceph-mon[30928]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-09-21T03:32:35.311 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:32:35 smithi089 ceph-mon[30928]: from='osd.2 [v2:172.21.15.114:6800/169452122,v1:172.21.15.114:6801/169452122]' entity='osd.2' cmd={"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi114", "root=default"]} : dispatch 2024-09-21T03:32:35.311 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:32:35 smithi089 ceph-mon[30928]: osdmap e15: 3 total, 2 up, 3 in 2024-09-21T03:32:35.311 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:32:35 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-21T03:32:35.311 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:32:35 smithi089 ceph-mon[30928]: from='osd.2 ' entity='osd.2' cmd={"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi114", "root=default"]} : dispatch 2024-09-21T03:32:35.426 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/7c374098-77c9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:32:35.460 INFO:journalctl@ceph.osd.2.smithi114.stdout:Sep 21 03:32:35 smithi114 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-2[38568]: 2024-09-21T03:32:35.049+0000 7fd184142640 -1 osd.2 0 waiting for initial osdmap 2024-09-21T03:32:35.461 INFO:journalctl@ceph.osd.2.smithi114.stdout:Sep 21 03:32:35 smithi114 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-2[38568]: 2024-09-21T03:32:35.053+0000 7fd17ff40640 -1 osd.2 16 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-21T03:32:35.461 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:32:35 smithi114 ceph-mon[34140]: pgmap v85: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-09-21T03:32:35.461 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:32:35 smithi114 ceph-mon[34140]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-09-21T03:32:35.461 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:32:35 smithi114 ceph-mon[34140]: from='osd.2 [v2:172.21.15.114:6800/169452122,v1:172.21.15.114:6801/169452122]' entity='osd.2' cmd={"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi114", "root=default"]} : dispatch 2024-09-21T03:32:35.461 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:32:35 smithi114 ceph-mon[34140]: osdmap e15: 3 total, 2 up, 3 in 2024-09-21T03:32:35.461 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:32:35 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-21T03:32:35.461 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:32:35 smithi114 ceph-mon[34140]: from='osd.2 ' entity='osd.2' cmd={"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi114", "root=default"]} : dispatch 2024-09-21T03:32:35.500 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:32:35 smithi110 ceph-mon[34000]: pgmap v85: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-09-21T03:32:35.500 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:32:35 smithi110 ceph-mon[34000]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-09-21T03:32:35.500 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:32:35 smithi110 ceph-mon[34000]: from='osd.2 [v2:172.21.15.114:6800/169452122,v1:172.21.15.114:6801/169452122]' entity='osd.2' cmd={"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi114", "root=default"]} : dispatch 2024-09-21T03:32:35.500 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:32:35 smithi110 ceph-mon[34000]: osdmap e15: 3 total, 2 up, 3 in 2024-09-21T03:32:35.500 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:32:35 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-21T03:32:35.500 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:32:35 smithi110 ceph-mon[34000]: from='osd.2 ' entity='osd.2' cmd={"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi114", "root=default"]} : dispatch 2024-09-21T03:32:35.938 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:32:35.935+0000 7f1a3a4ef640 1 -- 172.21.15.89:0/1685782607 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1a34102570 msgr2=0x7f1a34108d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:32:35.938 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:32:35.935+0000 7f1a3a4ef640 1 --2- 172.21.15.89:0/1685782607 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1a34102570 0x7f1a34108d70 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f1a1c009900 tx=0x7f1a1c02a410 comp rx=0 tx=0).stop 2024-09-21T03:32:35.938 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:32:35.935+0000 7f1a3a4ef640 1 -- 172.21.15.89:0/1685782607 shutdown_connections 2024-09-21T03:32:35.939 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:32:35.935+0000 7f1a3a4ef640 1 --2- 172.21.15.89:0/1685782607 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1a341094b0 0x7f1a3410b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:35.939 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:32:35.935+0000 7f1a3a4ef640 1 --2- 172.21.15.89:0/1685782607 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1a34102570 0x7f1a34108d70 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:35.939 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:32:35.935+0000 7f1a3a4ef640 1 --2- 172.21.15.89:0/1685782607 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1a34101b40 0x7f1a34101f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:35.939 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:32:35.935+0000 7f1a3a4ef640 1 -- 172.21.15.89:0/1685782607 >> 172.21.15.89:0/1685782607 conn(0x7f1a340fd770 msgr2=0x7f1a340ffc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:32:35.939 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:32:35.935+0000 7f1a3a4ef640 1 -- 172.21.15.89:0/1685782607 shutdown_connections 2024-09-21T03:32:35.939 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:32:35.935+0000 7f1a3a4ef640 1 -- 172.21.15.89:0/1685782607 wait complete. 2024-09-21T03:32:35.939 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:32:35.935+0000 7f1a3a4ef640 1 Processor -- start 2024-09-21T03:32:35.939 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:32:35.935+0000 7f1a3a4ef640 1 -- start start 2024-09-21T03:32:35.939 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:32:35.936+0000 7f1a3a4ef640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1a34101b40 0x7f1a34199ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:35.939 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:32:35.936+0000 7f1a3a4ef640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1a34102570 0x7f1a3419a540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:35.939 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:32:35.936+0000 7f1a3a4ef640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1a341094b0 0x7f1a3419b0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:35.940 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:32:35.936+0000 7f1a3a4ef640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f1a3410f250 con 0x7f1a341094b0 2024-09-21T03:32:35.940 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:32:35.936+0000 7f1a3a4ef640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f1a3410f0d0 con 0x7f1a34101b40 2024-09-21T03:32:35.940 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:32:35.936+0000 7f1a3a4ef640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f1a3410f3d0 con 0x7f1a34102570 2024-09-21T03:32:35.940 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:32:35.936+0000 7f1a38a8c640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1a341094b0 0x7f1a3419b0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:35.940 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:32:35.936+0000 7f1a38a8c640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1a341094b0 0x7f1a3419b0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:54922/0 (socket says 172.21.15.89:54922) 2024-09-21T03:32:35.940 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:32:35.936+0000 7f1a38a8c640 1 -- 172.21.15.89:0/2279986941 learned_addr learned my addr 172.21.15.89:0/2279986941 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-21T03:32:35.940 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:32:35.936+0000 7f1a33fff640 1 --2- 172.21.15.89:0/2279986941 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1a34101b40 0x7f1a34199ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:35.940 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:32:35.936+0000 7f1a2bfff640 1 --2- 172.21.15.89:0/2279986941 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1a34102570 0x7f1a3419a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:35.940 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:32:35.936+0000 7f1a38a8c640 1 -- 172.21.15.89:0/2279986941 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1a34102570 msgr2=0x7f1a3419a540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:32:35.940 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:32:35.936+0000 7f1a38a8c640 1 --2- 172.21.15.89:0/2279986941 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1a34102570 0x7f1a3419a540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:35.940 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:32:35.936+0000 7f1a38a8c640 1 -- 172.21.15.89:0/2279986941 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1a34101b40 msgr2=0x7f1a34199ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:32:35.940 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:32:35.936+0000 7f1a38a8c640 1 --2- 172.21.15.89:0/2279986941 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1a34101b40 0x7f1a34199ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:35.940 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:32:35.936+0000 7f1a38a8c640 1 -- 172.21.15.89:0/2279986941 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a3419b870 con 0x7f1a341094b0 2024-09-21T03:32:35.940 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:32:35.936+0000 7f1a33fff640 1 --2- 172.21.15.89:0/2279986941 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1a34101b40 0x7f1a34199ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:32:35.941 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:32:35.936+0000 7f1a38a8c640 1 --2- 172.21.15.89:0/2279986941 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1a341094b0 0x7f1a3419b0e0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f1a2c00cad0 tx=0x7f1a2c00cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:32:35.941 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:32:35.936+0000 7f1a31ffb640 1 -- 172.21.15.89:0/2279986941 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a2c00ae20 con 0x7f1a341094b0 2024-09-21T03:32:35.941 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:32:35.936+0000 7f1a3a4ef640 1 -- 172.21.15.89:0/2279986941 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1a3419bb60 con 0x7f1a341094b0 2024-09-21T03:32:35.941 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:32:35.936+0000 7f1a31ffb640 1 -- 172.21.15.89:0/2279986941 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1a2c005020 con 0x7f1a341094b0 2024-09-21T03:32:35.941 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:32:35.936+0000 7f1a31ffb640 1 -- 172.21.15.89:0/2279986941 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a2c00f6b0 con 0x7f1a341094b0 2024-09-21T03:32:35.941 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:32:35.937+0000 7f1a3a4ef640 1 -- 172.21.15.89:0/2279986941 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1a341a2120 con 0x7f1a341094b0 2024-09-21T03:32:35.942 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:32:35.938+0000 7f1a3a4ef640 1 -- 172.21.15.89:0/2279986941 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1a340ff290 con 0x7f1a341094b0 2024-09-21T03:32:35.942 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:32:35.939+0000 7f1a31ffb640 1 -- 172.21.15.89:0/2279986941 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 16) ==== 154456+0+0 (secure 0 0 0) 0x7f1a2c0051d0 con 0x7f1a341094b0 2024-09-21T03:32:35.942 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:32:35.940+0000 7f1a31ffb640 1 --2- 172.21.15.89:0/2279986941 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f1a200b6480 0x7f1a200b8990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:35.942 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:32:35.940+0000 7f1a33fff640 1 --2- 172.21.15.89:0/2279986941 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f1a200b6480 0x7f1a200b8990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:35.942 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:32:35.940+0000 7f1a31ffb640 1 -- 172.21.15.89:0/2279986941 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 2571+0+0 (secure 0 0 0) 0x7f1a2c0e6030 con 0x7f1a341094b0 2024-09-21T03:32:35.942 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:32:35.940+0000 7f1a33fff640 1 --2- 172.21.15.89:0/2279986941 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f1a200b6480 0x7f1a200b8990 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f1a2400abe0 tx=0x7f1a2400a5c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:32:35.945 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:32:35.944+0000 7f1a31ffb640 1 -- 172.21.15.89:0/2279986941 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f1a2c014030 con 0x7f1a341094b0 2024-09-21T03:32:36.134 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:32:36.133+0000 7f1a3a4ef640 1 -- 172.21.15.89:0/2279986941 --> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi089:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f1a34063800 con 0x7f1a200b6480 2024-09-21T03:32:36.135 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:32:36 smithi089 ceph-mon[30928]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi114", "root=default"]}]': finished 2024-09-21T03:32:36.135 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:32:36 smithi089 ceph-mon[30928]: osdmap e16: 3 total, 2 up, 3 in 2024-09-21T03:32:36.135 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:32:36 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-21T03:32:36.135 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:32:36 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-21T03:32:36.135 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:32:36 smithi089 ceph-mon[30928]: Detected new or changed devices on smithi114 2024-09-21T03:32:36.135 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:32:36 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:32:36.135 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:32:36 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:32:36.135 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:32:36 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-21T03:32:36.135 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:32:36 smithi089 ceph-mon[30928]: Adjusting osd_memory_target on smithi114 to 17088M 2024-09-21T03:32:36.135 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:32:36 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:32:36.135 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:32:36 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:32:36.135 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:32:36 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:32:36.136 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:32:36 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:32:36.461 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:32:36 smithi114 ceph-mon[34140]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi114", "root=default"]}]': finished 2024-09-21T03:32:36.461 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:32:36 smithi114 ceph-mon[34140]: osdmap e16: 3 total, 2 up, 3 in 2024-09-21T03:32:36.461 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:32:36 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-21T03:32:36.461 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:32:36 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-21T03:32:36.461 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:32:36 smithi114 ceph-mon[34140]: Detected new or changed devices on smithi114 2024-09-21T03:32:36.461 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:32:36 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:32:36.462 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:32:36 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:32:36.462 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:32:36 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-21T03:32:36.462 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:32:36 smithi114 ceph-mon[34140]: Adjusting osd_memory_target on smithi114 to 17088M 2024-09-21T03:32:36.462 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:32:36 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:32:36.462 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:32:36 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:32:36.462 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:32:36 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:32:36.462 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:32:36 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:32:36.500 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:32:36 smithi110 ceph-mon[34000]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi114", "root=default"]}]': finished 2024-09-21T03:32:36.500 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:32:36 smithi110 ceph-mon[34000]: osdmap e16: 3 total, 2 up, 3 in 2024-09-21T03:32:36.500 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:32:36 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-21T03:32:36.500 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:32:36 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-21T03:32:36.500 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:32:36 smithi110 ceph-mon[34000]: Detected new or changed devices on smithi114 2024-09-21T03:32:36.500 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:32:36 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:32:36.500 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:32:36 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:32:36.500 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:32:36 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-21T03:32:36.500 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:32:36 smithi110 ceph-mon[34000]: Adjusting osd_memory_target on smithi114 to 17088M 2024-09-21T03:32:36.500 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:32:36 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:32:36.501 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:32:36 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:32:36.501 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:32:36 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:32:36.501 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:32:36 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:32:37.311 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:32:37 smithi089 ceph-mon[30928]: purged_snaps scrub starts 2024-09-21T03:32:37.311 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:32:37 smithi089 ceph-mon[30928]: purged_snaps scrub ok 2024-09-21T03:32:37.311 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:32:37 smithi089 ceph-mon[30928]: pgmap v88: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-09-21T03:32:37.311 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:32:37 smithi089 ceph-mon[30928]: osd.2 [v2:172.21.15.114:6800/169452122,v1:172.21.15.114:6801/169452122] boot 2024-09-21T03:32:37.311 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:32:37 smithi089 ceph-mon[30928]: osdmap e17: 3 total, 3 up, 3 in 2024-09-21T03:32:37.311 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:32:37 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-21T03:32:37.311 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:32:37 smithi089 ceph-mon[30928]: from='client.14457 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi089:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:32:37.311 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:32:37 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T03:32:37.311 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:32:37 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T03:32:37.311 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:32:37 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:32:37.461 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:32:37 smithi114 ceph-mon[34140]: purged_snaps scrub starts 2024-09-21T03:32:37.461 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:32:37 smithi114 ceph-mon[34140]: purged_snaps scrub ok 2024-09-21T03:32:37.461 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:32:37 smithi114 ceph-mon[34140]: pgmap v88: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-09-21T03:32:37.461 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:32:37 smithi114 ceph-mon[34140]: osd.2 [v2:172.21.15.114:6800/169452122,v1:172.21.15.114:6801/169452122] boot 2024-09-21T03:32:37.461 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:32:37 smithi114 ceph-mon[34140]: osdmap e17: 3 total, 3 up, 3 in 2024-09-21T03:32:37.461 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:32:37 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-21T03:32:37.461 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:32:37 smithi114 ceph-mon[34140]: from='client.14457 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi089:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:32:37.461 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:32:37 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T03:32:37.461 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:32:37 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T03:32:37.461 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:32:37 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:32:37.500 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:32:37 smithi110 ceph-mon[34000]: purged_snaps scrub starts 2024-09-21T03:32:37.500 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:32:37 smithi110 ceph-mon[34000]: purged_snaps scrub ok 2024-09-21T03:32:37.500 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:32:37 smithi110 ceph-mon[34000]: pgmap v88: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-09-21T03:32:37.500 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:32:37 smithi110 ceph-mon[34000]: osd.2 [v2:172.21.15.114:6800/169452122,v1:172.21.15.114:6801/169452122] boot 2024-09-21T03:32:37.500 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:32:37 smithi110 ceph-mon[34000]: osdmap e17: 3 total, 3 up, 3 in 2024-09-21T03:32:37.500 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:32:37 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-21T03:32:37.500 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:32:37 smithi110 ceph-mon[34000]: from='client.14457 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi089:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:32:37.500 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:32:37 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T03:32:37.500 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:32:37 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T03:32:37.500 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:32:37 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:32:38.500 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:32:38 smithi110 ceph-mon[34000]: osdmap e18: 3 total, 3 up, 3 in 2024-09-21T03:32:38.500 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:32:38 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true} : dispatch 2024-09-21T03:32:38.561 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:32:38 smithi089 ceph-mon[30928]: osdmap e18: 3 total, 3 up, 3 in 2024-09-21T03:32:38.561 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:32:38 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true} : dispatch 2024-09-21T03:32:38.711 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:32:38 smithi114 ceph-mon[34140]: osdmap e18: 3 total, 3 up, 3 in 2024-09-21T03:32:38.711 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:32:38 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true} : dispatch 2024-09-21T03:32:38.877 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:32:38 smithi089 sudo[50790]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-21T03:32:38.877 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:32:38 smithi089 sudo[50790]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-21T03:32:38.877 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:32:38 smithi089 sudo[50790]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-21T03:32:38.877 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:32:38 smithi089 sudo[50790]: pam_unix(sudo:session): session closed for user root 2024-09-21T03:32:38.877 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:32:38 smithi089 sudo[50793]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-21T03:32:38.877 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:32:38 smithi089 sudo[50793]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-21T03:32:38.877 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:32:38 smithi089 sudo[50793]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-21T03:32:38.878 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:32:38 smithi089 sudo[50793]: pam_unix(sudo:session): session closed for user root 2024-09-21T03:32:38.878 INFO:journalctl@ceph.osd.0.smithi089.stdout:Sep 21 03:32:38 smithi089 sudo[50778]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-21T03:32:38.878 INFO:journalctl@ceph.osd.0.smithi089.stdout:Sep 21 03:32:38 smithi089 sudo[50778]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-21T03:32:38.878 INFO:journalctl@ceph.osd.0.smithi089.stdout:Sep 21 03:32:38 smithi089 sudo[50778]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-21T03:32:38.878 INFO:journalctl@ceph.osd.0.smithi089.stdout:Sep 21 03:32:38 smithi089 sudo[50778]: pam_unix(sudo:session): session closed for user root 2024-09-21T03:32:38.878 INFO:journalctl@ceph.osd.0.smithi089.stdout:Sep 21 03:32:38 smithi089 sudo[50781]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-21T03:32:38.878 INFO:journalctl@ceph.osd.0.smithi089.stdout:Sep 21 03:32:38 smithi089 sudo[50781]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-21T03:32:38.878 INFO:journalctl@ceph.osd.0.smithi089.stdout:Sep 21 03:32:38 smithi089 sudo[50781]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-21T03:32:38.878 INFO:journalctl@ceph.osd.0.smithi089.stdout:Sep 21 03:32:38 smithi089 sudo[50781]: pam_unix(sudo:session): session closed for user root 2024-09-21T03:32:39.000 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:32:38 smithi110 sudo[41511]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-21T03:32:39.000 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:32:38 smithi110 sudo[41511]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-21T03:32:39.000 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:32:38 smithi110 sudo[41511]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-21T03:32:39.000 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:32:38 smithi110 sudo[41511]: pam_unix(sudo:session): session closed for user root 2024-09-21T03:32:39.000 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:32:38 smithi110 sudo[41514]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-21T03:32:39.000 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:32:38 smithi110 sudo[41514]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-21T03:32:39.000 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:32:38 smithi110 sudo[41514]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-21T03:32:39.001 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:32:38 smithi110 sudo[41514]: pam_unix(sudo:session): session closed for user root 2024-09-21T03:32:39.001 INFO:journalctl@ceph.osd.1.smithi110.stdout:Sep 21 03:32:38 smithi110 sudo[41505]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-21T03:32:39.001 INFO:journalctl@ceph.osd.1.smithi110.stdout:Sep 21 03:32:38 smithi110 sudo[41505]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-21T03:32:39.001 INFO:journalctl@ceph.osd.1.smithi110.stdout:Sep 21 03:32:38 smithi110 sudo[41505]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-21T03:32:39.001 INFO:journalctl@ceph.osd.1.smithi110.stdout:Sep 21 03:32:38 smithi110 sudo[41505]: pam_unix(sudo:session): session closed for user root 2024-09-21T03:32:39.001 INFO:journalctl@ceph.osd.1.smithi110.stdout:Sep 21 03:32:38 smithi110 sudo[41508]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-21T03:32:39.001 INFO:journalctl@ceph.osd.1.smithi110.stdout:Sep 21 03:32:38 smithi110 sudo[41508]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-21T03:32:39.001 INFO:journalctl@ceph.osd.1.smithi110.stdout:Sep 21 03:32:38 smithi110 sudo[41508]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-21T03:32:39.001 INFO:journalctl@ceph.osd.1.smithi110.stdout:Sep 21 03:32:38 smithi110 sudo[41508]: pam_unix(sudo:session): session closed for user root 2024-09-21T03:32:39.211 INFO:journalctl@ceph.osd.2.smithi114.stdout:Sep 21 03:32:38 smithi114 sudo[42488]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-21T03:32:39.211 INFO:journalctl@ceph.osd.2.smithi114.stdout:Sep 21 03:32:38 smithi114 sudo[42488]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-21T03:32:39.211 INFO:journalctl@ceph.osd.2.smithi114.stdout:Sep 21 03:32:38 smithi114 sudo[42488]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-21T03:32:39.211 INFO:journalctl@ceph.osd.2.smithi114.stdout:Sep 21 03:32:38 smithi114 sudo[42488]: pam_unix(sudo:session): session closed for user root 2024-09-21T03:32:39.211 INFO:journalctl@ceph.osd.2.smithi114.stdout:Sep 21 03:32:38 smithi114 sudo[42491]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-21T03:32:39.211 INFO:journalctl@ceph.osd.2.smithi114.stdout:Sep 21 03:32:38 smithi114 sudo[42491]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-21T03:32:39.211 INFO:journalctl@ceph.osd.2.smithi114.stdout:Sep 21 03:32:38 smithi114 sudo[42491]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-21T03:32:39.211 INFO:journalctl@ceph.osd.2.smithi114.stdout:Sep 21 03:32:38 smithi114 sudo[42491]: pam_unix(sudo:session): session closed for user root 2024-09-21T03:32:39.211 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:32:38 smithi114 sudo[42494]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-21T03:32:39.212 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:32:38 smithi114 sudo[42494]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-21T03:32:39.212 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:32:38 smithi114 sudo[42494]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-21T03:32:39.212 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:32:38 smithi114 sudo[42494]: pam_unix(sudo:session): session closed for user root 2024-09-21T03:32:39.212 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:32:38 smithi114 sudo[42497]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-21T03:32:39.212 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:32:38 smithi114 sudo[42497]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-21T03:32:39.212 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:32:38 smithi114 sudo[42497]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-21T03:32:39.212 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:32:38 smithi114 sudo[42497]: pam_unix(sudo:session): session closed for user root 2024-09-21T03:32:39.500 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:32:39 smithi110 ceph-mon[34000]: pgmap v91: 0 pgs: ; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-09-21T03:32:39.500 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:32:39 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-09-21T03:32:39.500 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:32:39 smithi110 ceph-mon[34000]: osdmap e19: 3 total, 3 up, 3 in 2024-09-21T03:32:39.500 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:32:39 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true} : dispatch 2024-09-21T03:32:39.500 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:32:39 smithi110 ceph-mon[34000]: from='client.? 172.21.15.89:0/85516683' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "2c7ae1b4-0bf2-4954-81ed-55d03eab82c6"} : dispatch 2024-09-21T03:32:39.500 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:32:39 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-09-21T03:32:39.500 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:32:39 smithi110 ceph-mon[34000]: from='client.? 172.21.15.89:0/85516683' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2c7ae1b4-0bf2-4954-81ed-55d03eab82c6"}]': finished 2024-09-21T03:32:39.501 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:32:39 smithi110 ceph-mon[34000]: osdmap e20: 4 total, 3 up, 4 in 2024-09-21T03:32:39.501 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:32:39 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-21T03:32:39.501 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:32:39 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:32:39.501 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:32:39 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:32:39.501 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:32:39 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:32:39.501 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:32:39 smithi110 ceph-mon[34000]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-21T03:32:39.501 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:32:39 smithi110 ceph-mon[34000]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-21T03:32:39.501 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:32:39 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-21T03:32:39.501 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:32:39 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T03:32:39.501 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:32:39 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T03:32:39.501 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:32:39 smithi110 ceph-mon[34000]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-21T03:32:39.502 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:32:39 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-21T03:32:39.502 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:32:39 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T03:32:39.502 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:32:39 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T03:32:39.502 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:32:39 smithi110 ceph-mon[34000]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-21T03:32:39.502 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:32:39 smithi110 ceph-mon[34000]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-21T03:32:39.502 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:32:39 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-21T03:32:39.502 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:32:39 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T03:32:39.502 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:32:39 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T03:32:39.502 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:32:39 smithi110 ceph-mon[34000]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-21T03:32:39.502 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:32:39 smithi110 ceph-mon[34000]: from='client.? 172.21.15.89:0/1772916929' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T03:32:39.561 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:32:39 smithi089 ceph-mon[30928]: pgmap v91: 0 pgs: ; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-09-21T03:32:39.561 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:32:39 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-09-21T03:32:39.561 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:32:39 smithi089 ceph-mon[30928]: osdmap e19: 3 total, 3 up, 3 in 2024-09-21T03:32:39.561 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:32:39 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true} : dispatch 2024-09-21T03:32:39.561 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:32:39 smithi089 ceph-mon[30928]: from='client.? 172.21.15.89:0/85516683' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "2c7ae1b4-0bf2-4954-81ed-55d03eab82c6"} : dispatch 2024-09-21T03:32:39.561 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:32:39 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-09-21T03:32:39.561 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:32:39 smithi089 ceph-mon[30928]: from='client.? 172.21.15.89:0/85516683' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2c7ae1b4-0bf2-4954-81ed-55d03eab82c6"}]': finished 2024-09-21T03:32:39.561 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:32:39 smithi089 ceph-mon[30928]: osdmap e20: 4 total, 3 up, 4 in 2024-09-21T03:32:39.562 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:32:39 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-21T03:32:39.562 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:32:39 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:32:39.562 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:32:39 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:32:39.562 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:32:39 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:32:39.562 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:32:39 smithi089 ceph-mon[30928]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-21T03:32:39.562 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:32:39 smithi089 ceph-mon[30928]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-21T03:32:39.562 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:32:39 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-21T03:32:39.562 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:32:39 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T03:32:39.562 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:32:39 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T03:32:39.562 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:32:39 smithi089 ceph-mon[30928]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-21T03:32:39.562 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:32:39 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-21T03:32:39.562 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:32:39 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T03:32:39.562 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:32:39 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T03:32:39.562 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:32:39 smithi089 ceph-mon[30928]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-21T03:32:39.563 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:32:39 smithi089 ceph-mon[30928]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-21T03:32:39.563 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:32:39 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-21T03:32:39.563 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:32:39 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T03:32:39.563 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:32:39 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T03:32:39.563 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:32:39 smithi089 ceph-mon[30928]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-21T03:32:39.563 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:32:39 smithi089 ceph-mon[30928]: from='client.? 172.21.15.89:0/1772916929' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T03:32:39.711 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:32:39 smithi114 ceph-mon[34140]: pgmap v91: 0 pgs: ; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-09-21T03:32:39.711 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:32:39 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-09-21T03:32:39.711 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:32:39 smithi114 ceph-mon[34140]: osdmap e19: 3 total, 3 up, 3 in 2024-09-21T03:32:39.711 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:32:39 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true} : dispatch 2024-09-21T03:32:39.711 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:32:39 smithi114 ceph-mon[34140]: from='client.? 172.21.15.89:0/85516683' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "2c7ae1b4-0bf2-4954-81ed-55d03eab82c6"} : dispatch 2024-09-21T03:32:39.712 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:32:39 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-09-21T03:32:39.712 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:32:39 smithi114 ceph-mon[34140]: from='client.? 172.21.15.89:0/85516683' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2c7ae1b4-0bf2-4954-81ed-55d03eab82c6"}]': finished 2024-09-21T03:32:39.712 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:32:39 smithi114 ceph-mon[34140]: osdmap e20: 4 total, 3 up, 4 in 2024-09-21T03:32:39.712 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:32:39 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-21T03:32:39.712 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:32:39 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:32:39.712 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:32:39 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:32:39.712 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:32:39 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:32:39.712 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:32:39 smithi114 ceph-mon[34140]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-21T03:32:39.712 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:32:39 smithi114 ceph-mon[34140]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-21T03:32:39.712 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:32:39 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-21T03:32:39.712 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:32:39 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T03:32:39.712 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:32:39 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T03:32:39.712 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:32:39 smithi114 ceph-mon[34140]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-21T03:32:39.712 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:32:39 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-21T03:32:39.713 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:32:39 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T03:32:39.713 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:32:39 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T03:32:39.713 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:32:39 smithi114 ceph-mon[34140]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-21T03:32:39.713 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:32:39 smithi114 ceph-mon[34140]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-21T03:32:39.713 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:32:39 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-21T03:32:39.713 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:32:39 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T03:32:39.713 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:32:39 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T03:32:39.713 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:32:39 smithi114 ceph-mon[34140]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-21T03:32:39.713 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:32:39 smithi114 ceph-mon[34140]: from='client.? 172.21.15.89:0/1772916929' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T03:32:40.558 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:32:40.556+0000 7f1a31ffb640 1 -- 172.21.15.89:0/2279986941 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mgrmap(e 17) ==== 154512+0+0 (secure 0 0 0) 0x7f1a2c0a9b80 con 0x7f1a341094b0 2024-09-21T03:32:40.811 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:32:40 smithi089 ceph-mon[30928]: osdmap e21: 4 total, 3 up, 4 in 2024-09-21T03:32:40.811 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:32:40 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-21T03:32:40.811 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:32:40 smithi089 ceph-mon[30928]: pgmap v95: 1 pgs: 1 creating+peering; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-09-21T03:32:40.961 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:32:40 smithi114 ceph-mon[34140]: osdmap e21: 4 total, 3 up, 4 in 2024-09-21T03:32:40.961 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:32:40 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-21T03:32:40.961 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:32:40 smithi114 ceph-mon[34140]: pgmap v95: 1 pgs: 1 creating+peering; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-09-21T03:32:41.000 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:32:40 smithi110 ceph-mon[34000]: osdmap e21: 4 total, 3 up, 4 in 2024-09-21T03:32:41.000 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:32:40 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-21T03:32:41.000 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:32:40 smithi110 ceph-mon[34000]: pgmap v95: 1 pgs: 1 creating+peering; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-09-21T03:32:41.811 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:32:41 smithi089 ceph-mon[30928]: mgrmap e17: x(active, since 2m), standbys: z, y 2024-09-21T03:32:41.960 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:32:41 smithi114 ceph-mon[34140]: mgrmap e17: x(active, since 2m), standbys: z, y 2024-09-21T03:32:42.000 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:32:41 smithi110 ceph-mon[34000]: mgrmap e17: x(active, since 2m), standbys: z, y 2024-09-21T03:32:42.811 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:32:42 smithi089 ceph-mon[30928]: pgmap v96: 1 pgs: 1 creating+peering; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-09-21T03:32:42.960 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:32:42 smithi114 ceph-mon[34140]: pgmap v96: 1 pgs: 1 creating+peering; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-09-21T03:32:42.999 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:32:42 smithi110 ceph-mon[34000]: pgmap v96: 1 pgs: 1 creating+peering; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-09-21T03:32:44.785 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:32:44 smithi089 ceph-mon[30928]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-09-21T03:32:44.960 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:32:44 smithi114 ceph-mon[34140]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-09-21T03:32:45.000 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:32:44 smithi110 ceph-mon[34000]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-09-21T03:32:46.732 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:32:46 smithi089 ceph-mon[30928]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-09-21T03:32:46.960 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:32:46 smithi114 ceph-mon[34140]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-09-21T03:32:47.000 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:32:46 smithi110 ceph-mon[34000]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-09-21T03:32:47.961 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:32:47 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.3"} : dispatch 2024-09-21T03:32:47.961 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:32:47 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:32:47.961 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:32:47 smithi114 ceph-mon[34140]: Deploying daemon osd.3 on smithi089 2024-09-21T03:32:48.000 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:32:47 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.3"} : dispatch 2024-09-21T03:32:48.000 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:32:47 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:32:48.000 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:32:47 smithi110 ceph-mon[34000]: Deploying daemon osd.3 on smithi089 2024-09-21T03:32:48.061 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:32:47 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.3"} : dispatch 2024-09-21T03:32:48.061 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:32:47 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:32:48.061 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:32:47 smithi089 ceph-mon[30928]: Deploying daemon osd.3 on smithi089 2024-09-21T03:32:48.694 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:32:48 smithi089 ceph-mon[30928]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-09-21T03:32:48.960 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:32:48 smithi114 ceph-mon[34140]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-09-21T03:32:49.000 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:32:48 smithi110 ceph-mon[34000]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-09-21T03:32:50.789 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:32:50 smithi089 ceph-mon[30928]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-09-21T03:32:50.960 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:32:50 smithi114 ceph-mon[34140]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-09-21T03:32:51.000 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:32:50 smithi110 ceph-mon[34000]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-09-21T03:32:52.887 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:32:52 smithi089 ceph-mon[30928]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-09-21T03:32:52.960 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:32:52 smithi114 ceph-mon[34140]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-09-21T03:32:53.000 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:32:52 smithi110 ceph-mon[34000]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-09-21T03:32:53.892 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:32:53 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:32:53.893 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:32:53 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:32:53.893 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:32:53 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:32:53.960 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:32:53 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:32:53.961 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:32:53 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:32:53.961 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:32:53 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:32:54.000 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:32:53 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:32:54.000 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:32:53 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:32:54.000 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:32:53 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:32:55.750 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:32:55 smithi110 ceph-mon[34000]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-09-21T03:32:55.811 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:32:55 smithi089 ceph-mon[30928]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-09-21T03:32:55.960 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:32:55 smithi114 ceph-mon[34140]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-09-21T03:32:57.461 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:32:57 smithi114 ceph-mon[34140]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-09-21T03:32:57.461 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:32:57 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:32:57.461 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:32:57 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:32:57.461 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:32:57 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:32:57.461 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:32:57 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:32:57.461 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:32:57 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:32:57.467 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:32:57 smithi089 ceph-mon[30928]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-09-21T03:32:57.467 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:32:57 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:32:57.467 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:32:57 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:32:57.468 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:32:57 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:32:57.468 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:32:57 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:32:57.468 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:32:57 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:32:57.500 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:32:57 smithi110 ceph-mon[34000]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-09-21T03:32:57.500 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:32:57 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:32:57.500 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:32:57 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:32:57.500 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:32:57 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:32:57.500 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:32:57 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:32:57.500 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:32:57 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:32:57.509 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:32:57.506+0000 7f1a31ffb640 1 -- 172.21.15.89:0/2279986941 <== mgr.14152 v2:172.21.15.89:6800/1591532358 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f1a34063800 con 0x7f1a200b6480 2024-09-21T03:32:57.510 INFO:teuthology.orchestra.run.smithi089.stdout:Created osd(s) 3 on host 'smithi089' 2024-09-21T03:32:57.511 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:32:57.508+0000 7f1a3a4ef640 1 -- 172.21.15.89:0/2279986941 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f1a200b6480 msgr2=0x7f1a200b8990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:32:57.511 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:32:57.508+0000 7f1a3a4ef640 1 --2- 172.21.15.89:0/2279986941 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f1a200b6480 0x7f1a200b8990 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f1a2400abe0 tx=0x7f1a2400a5c0 comp rx=0 tx=0).stop 2024-09-21T03:32:57.511 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:32:57.508+0000 7f1a3a4ef640 1 -- 172.21.15.89:0/2279986941 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1a341094b0 msgr2=0x7f1a3419b0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:32:57.511 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:32:57.508+0000 7f1a3a4ef640 1 --2- 172.21.15.89:0/2279986941 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1a341094b0 0x7f1a3419b0e0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f1a2c00cad0 tx=0x7f1a2c00cf90 comp rx=0 tx=0).stop 2024-09-21T03:32:57.511 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:32:57.508+0000 7f1a3a4ef640 1 -- 172.21.15.89:0/2279986941 shutdown_connections 2024-09-21T03:32:57.511 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:32:57.508+0000 7f1a3a4ef640 1 --2- 172.21.15.89:0/2279986941 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f1a200b6480 0x7f1a200b8990 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:57.511 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:32:57.508+0000 7f1a3a4ef640 1 --2- 172.21.15.89:0/2279986941 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1a341094b0 0x7f1a3419b0e0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:57.511 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:32:57.508+0000 7f1a3a4ef640 1 --2- 172.21.15.89:0/2279986941 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1a34102570 0x7f1a3419a540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:57.511 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:32:57.508+0000 7f1a3a4ef640 1 --2- 172.21.15.89:0/2279986941 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1a34101b40 0x7f1a34199ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:57.511 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:32:57.508+0000 7f1a3a4ef640 1 -- 172.21.15.89:0/2279986941 >> 172.21.15.89:0/2279986941 conn(0x7f1a340fd770 msgr2=0x7f1a3410a3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:32:57.511 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:32:57.509+0000 7f1a3a4ef640 1 -- 172.21.15.89:0/2279986941 shutdown_connections 2024-09-21T03:32:57.511 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:32:57.509+0000 7f1a3a4ef640 1 -- 172.21.15.89:0/2279986941 wait complete. 2024-09-21T03:32:58.171 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:32:58 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:32:58.171 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:32:58 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:32:58.171 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:32:58 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:32:58.200 DEBUG:teuthology.orchestra.run.smithi089:osd.3> sudo journalctl -f -n 0 -u ceph-7c374098-77c9-11ef-baf5-efdc52797490@osd.3.service 2024-09-21T03:32:58.202 INFO:tasks.cephadm:Deploying osd.4 on smithi110 with /dev/vg_nvme/lv_3... 2024-09-21T03:32:58.202 DEBUG:teuthology.orchestra.run.smithi110:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7c374098-77c9-11ef-baf5-efdc52797490 -- lvm zap /dev/vg_nvme/lv_3 2024-09-21T03:32:58.455 INFO:teuthology.orchestra.run.smithi110.stderr:Inferring config /var/lib/ceph/7c374098-77c9-11ef-baf5-efdc52797490/mon.b/config 2024-09-21T03:32:58.461 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:32:58 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:32:58.461 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:32:58 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:32:58.461 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:32:58 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:32:58.499 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:32:58 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:32:58.500 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:32:58 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:32:58.500 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:32:58 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:32:59.623 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:32:59 smithi089 ceph-mon[30928]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-09-21T03:32:59.807 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:32:59 smithi110 ceph-mon[34000]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-09-21T03:32:59.960 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:32:59 smithi114 ceph-mon[34140]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-09-21T03:33:00.047 INFO:journalctl@ceph.osd.3.smithi089.stdout:Sep 21 03:32:59 smithi089 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-3[52765]: 2024-09-21T03:32:59.828+0000 7fef27f9f640 -1 osd.3 0 log_to_monitors true 2024-09-21T03:33:00.667 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:33:00 smithi110 ceph-mon[34000]: from='osd.3 [v2:172.21.15.89:6810/3016980877,v1:172.21.15.89:6811/3016980877]' entity='osd.3' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]} : dispatch 2024-09-21T03:33:00.668 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:33:00 smithi089 ceph-mon[30928]: from='osd.3 [v2:172.21.15.89:6810/3016980877,v1:172.21.15.89:6811/3016980877]' entity='osd.3' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]} : dispatch 2024-09-21T03:33:00.946 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-21T03:33:00.960 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:33:00 smithi114 ceph-mon[34140]: from='osd.3 [v2:172.21.15.89:6810/3016980877,v1:172.21.15.89:6811/3016980877]' entity='osd.3' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]} : dispatch 2024-09-21T03:33:00.972 DEBUG:teuthology.orchestra.run.smithi110:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7c374098-77c9-11ef-baf5-efdc52797490 -- ceph orch daemon add osd smithi110:vg_nvme/lv_3 2024-09-21T03:33:01.228 INFO:teuthology.orchestra.run.smithi110.stderr:Inferring config /var/lib/ceph/7c374098-77c9-11ef-baf5-efdc52797490/mon.b/config 2024-09-21T03:33:01.596 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:33:01 smithi110 ceph-mon[34000]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-09-21T03:33:01.596 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:33:01 smithi110 ceph-mon[34000]: from='osd.3 [v2:172.21.15.89:6810/3016980877,v1:172.21.15.89:6811/3016980877]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-09-21T03:33:01.597 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:33:01 smithi110 ceph-mon[34000]: osdmap e22: 4 total, 3 up, 4 in 2024-09-21T03:33:01.597 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:33:01 smithi110 ceph-mon[34000]: from='osd.3 [v2:172.21.15.89:6810/3016980877,v1:172.21.15.89:6811/3016980877]' entity='osd.3' cmd={"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi089", "root=default"]} : dispatch 2024-09-21T03:33:01.597 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:33:01 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-21T03:33:01.597 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:33:01 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:33:01.597 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:33:01 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:33:01.597 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:33:01 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-21T03:33:01.597 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:33:01 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-21T03:33:01.597 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:33:01 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:33:01.597 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:33:01 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:33:01.597 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:33:01 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:33:01.597 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:33:01 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:33:01.705 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T03:33:01.687+0000 7fc58e74a640 1 -- 172.21.15.110:0/3318950909 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc5881036d0 msgr2=0x7fc588103ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:01.705 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T03:33:01.687+0000 7fc58e74a640 1 --2- 172.21.15.110:0/3318950909 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc5881036d0 0x7fc588103ae0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fc57c009900 tx=0x7fc57c02a480 comp rx=0 tx=0).stop 2024-09-21T03:33:01.705 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T03:33:01.687+0000 7fc58e74a640 1 -- 172.21.15.110:0/3318950909 shutdown_connections 2024-09-21T03:33:01.705 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T03:33:01.687+0000 7fc58e74a640 1 --2- 172.21.15.110:0/3318950909 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc5881036d0 0x7fc588103ae0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:01.705 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T03:33:01.687+0000 7fc58e74a640 1 --2- 172.21.15.110:0/3318950909 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc588104070 0x7fc58810aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:01.705 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T03:33:01.687+0000 7fc58e74a640 1 --2- 172.21.15.110:0/3318950909 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc588102640 0x7fc588102ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:01.705 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T03:33:01.687+0000 7fc58e74a640 1 -- 172.21.15.110:0/3318950909 >> 172.21.15.110:0/3318950909 conn(0x7fc5880fdfb0 msgr2=0x7fc588100460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:33:01.705 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T03:33:01.687+0000 7fc58e74a640 1 -- 172.21.15.110:0/3318950909 shutdown_connections 2024-09-21T03:33:01.705 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T03:33:01.687+0000 7fc58e74a640 1 -- 172.21.15.110:0/3318950909 wait complete. 2024-09-21T03:33:01.705 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T03:33:01.688+0000 7fc58e74a640 1 Processor -- start 2024-09-21T03:33:01.705 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T03:33:01.688+0000 7fc58e74a640 1 -- start start 2024-09-21T03:33:01.705 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T03:33:01.688+0000 7fc58e74a640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc588102640 0x7fc588198db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:01.705 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T03:33:01.688+0000 7fc58e74a640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc5881036d0 0x7fc588199420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:01.706 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T03:33:01.688+0000 7fc58e74a640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc588104070 0x7fc588199fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:01.706 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T03:33:01.688+0000 7fc58e74a640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fc58810e120 con 0x7fc5881036d0 2024-09-21T03:33:01.706 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T03:33:01.688+0000 7fc58e74a640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fc58810dfa0 con 0x7fc588102640 2024-09-21T03:33:01.706 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T03:33:01.688+0000 7fc58e74a640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fc58810e2a0 con 0x7fc588104070 2024-09-21T03:33:01.706 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T03:33:01.688+0000 7fc587fff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc588102640 0x7fc588198db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:01.706 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T03:33:01.688+0000 7fc587fff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc588102640 0x7fc588198db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:52156/0 (socket says 172.21.15.110:52156) 2024-09-21T03:33:01.706 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T03:33:01.688+0000 7fc587fff640 1 -- 172.21.15.110:0/73225394 learned_addr learned my addr 172.21.15.110:0/73225394 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-21T03:33:01.706 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T03:33:01.688+0000 7fc5877fe640 1 --2- 172.21.15.110:0/73225394 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc5881036d0 0x7fc588199420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:01.706 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T03:33:01.688+0000 7fc58cce7640 1 --2- 172.21.15.110:0/73225394 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc588104070 0x7fc588199fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:01.706 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T03:33:01.689+0000 7fc587fff640 1 -- 172.21.15.110:0/73225394 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc588104070 msgr2=0x7fc588199fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:01.706 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T03:33:01.689+0000 7fc587fff640 1 --2- 172.21.15.110:0/73225394 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc588104070 0x7fc588199fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:01.706 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T03:33:01.689+0000 7fc587fff640 1 -- 172.21.15.110:0/73225394 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc5881036d0 msgr2=0x7fc588199420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:01.707 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T03:33:01.689+0000 7fc587fff640 1 --2- 172.21.15.110:0/73225394 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc5881036d0 0x7fc588199420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:01.707 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T03:33:01.689+0000 7fc587fff640 1 -- 172.21.15.110:0/73225394 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc58819a750 con 0x7fc588102640 2024-09-21T03:33:01.707 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T03:33:01.689+0000 7fc587fff640 1 --2- 172.21.15.110:0/73225394 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc588102640 0x7fc588198db0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fc57800c920 tx=0x7fc57800cde0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:33:01.707 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T03:33:01.689+0000 7fc5857fa640 1 -- 172.21.15.110:0/73225394 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc578007c70 con 0x7fc588102640 2024-09-21T03:33:01.707 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T03:33:01.689+0000 7fc58e74a640 1 -- 172.21.15.110:0/73225394 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc58819aa40 con 0x7fc588102640 2024-09-21T03:33:01.707 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T03:33:01.689+0000 7fc5857fa640 1 -- 172.21.15.110:0/73225394 <== mon.2 v2:172.21.15.110:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc578007e10 con 0x7fc588102640 2024-09-21T03:33:01.707 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T03:33:01.689+0000 7fc5857fa640 1 -- 172.21.15.110:0/73225394 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5780186f0 con 0x7fc588102640 2024-09-21T03:33:01.707 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T03:33:01.689+0000 7fc58e74a640 1 -- 172.21.15.110:0/73225394 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5881a11d0 con 0x7fc588102640 2024-09-21T03:33:01.707 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T03:33:01.690+0000 7fc58e74a640 1 -- 172.21.15.110:0/73225394 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc588109820 con 0x7fc588102640 2024-09-21T03:33:01.707 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T03:33:01.692+0000 7fc5857fa640 1 -- 172.21.15.110:0/73225394 <== mon.2 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 17) ==== 154512+0+0 (secure 0 0 0) 0x7fc57800f450 con 0x7fc588102640 2024-09-21T03:33:01.707 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T03:33:01.692+0000 7fc5857fa640 1 --2- 172.21.15.110:0/73225394 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7fc5700b6510 0x7fc5700b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:01.707 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T03:33:01.693+0000 7fc5857fa640 1 -- 172.21.15.110:0/73225394 <== mon.2 v2:172.21.15.110:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 3702+0+0 (secure 0 0 0) 0x7fc5780e6560 con 0x7fc588102640 2024-09-21T03:33:01.707 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T03:33:01.693+0000 7fc5877fe640 1 --2- 172.21.15.110:0/73225394 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7fc5700b6510 0x7fc5700b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:01.707 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T03:33:01.693+0000 7fc5877fe640 1 --2- 172.21.15.110:0/73225394 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7fc5700b6510 0x7fc5700b8a20 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fc57c0057c0 tx=0x7fc57c000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:33:01.708 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T03:33:01.696+0000 7fc5857fa640 1 -- 172.21.15.110:0/73225394 <== mon.2 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fc578014030 con 0x7fc588102640 2024-09-21T03:33:01.811 INFO:journalctl@ceph.osd.3.smithi089.stdout:Sep 21 03:33:01 smithi089 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-3[52765]: 2024-09-21T03:33:01.520+0000 7fef23f21640 -1 osd.3 0 waiting for initial osdmap 2024-09-21T03:33:01.811 INFO:journalctl@ceph.osd.3.smithi089.stdout:Sep 21 03:33:01 smithi089 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-3[52765]: 2024-09-21T03:33:01.526+0000 7fef1fd1f640 -1 osd.3 23 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-21T03:33:01.811 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:33:01 smithi089 ceph-mon[30928]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-09-21T03:33:01.811 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:33:01 smithi089 ceph-mon[30928]: from='osd.3 [v2:172.21.15.89:6810/3016980877,v1:172.21.15.89:6811/3016980877]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-09-21T03:33:01.812 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:33:01 smithi089 ceph-mon[30928]: osdmap e22: 4 total, 3 up, 4 in 2024-09-21T03:33:01.812 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:33:01 smithi089 ceph-mon[30928]: from='osd.3 [v2:172.21.15.89:6810/3016980877,v1:172.21.15.89:6811/3016980877]' entity='osd.3' cmd={"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi089", "root=default"]} : dispatch 2024-09-21T03:33:01.812 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:33:01 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-21T03:33:01.812 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:33:01 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:33:01.812 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:33:01 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:33:01.812 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:33:01 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-21T03:33:01.812 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:33:01 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-21T03:33:01.812 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:33:01 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:33:01.812 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:33:01 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:33:01.812 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:33:01 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:33:01.812 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:33:01 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:33:01.883 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T03:33:01.881+0000 7fc58e74a640 1 -- 172.21.15.110:0/73225394 --> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi110:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7fc588063800 con 0x7fc5700b6510 2024-09-21T03:33:01.961 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:33:01 smithi114 ceph-mon[34140]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-09-21T03:33:01.961 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:33:01 smithi114 ceph-mon[34140]: from='osd.3 [v2:172.21.15.89:6810/3016980877,v1:172.21.15.89:6811/3016980877]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-09-21T03:33:01.961 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:33:01 smithi114 ceph-mon[34140]: osdmap e22: 4 total, 3 up, 4 in 2024-09-21T03:33:01.961 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:33:01 smithi114 ceph-mon[34140]: from='osd.3 [v2:172.21.15.89:6810/3016980877,v1:172.21.15.89:6811/3016980877]' entity='osd.3' cmd={"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi089", "root=default"]} : dispatch 2024-09-21T03:33:01.961 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:33:01 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-21T03:33:01.961 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:33:01 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:33:01.961 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:33:01 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:33:01.961 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:33:01 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-21T03:33:01.961 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:33:01 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-21T03:33:01.961 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:33:01 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:33:01.961 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:33:01 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:33:01.962 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:33:01 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:33:01.962 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:33:01 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:33:02.811 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:33:02 smithi089 ceph-mon[30928]: Detected new or changed devices on smithi089 2024-09-21T03:33:02.811 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:33:02 smithi089 ceph-mon[30928]: Adjusting osd_memory_target on smithi089 to 8543M 2024-09-21T03:33:02.811 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:33:02 smithi089 ceph-mon[30928]: from='osd.3 [v2:172.21.15.89:6810/3016980877,v1:172.21.15.89:6811/3016980877]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi089", "root=default"]}]': finished 2024-09-21T03:33:02.811 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:33:02 smithi089 ceph-mon[30928]: osdmap e23: 4 total, 3 up, 4 in 2024-09-21T03:33:02.811 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:33:02 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-21T03:33:02.811 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:33:02 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-21T03:33:02.812 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:33:02 smithi089 ceph-mon[30928]: osd.3 [v2:172.21.15.89:6810/3016980877,v1:172.21.15.89:6811/3016980877] boot 2024-09-21T03:33:02.812 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:33:02 smithi089 ceph-mon[30928]: osdmap e24: 4 total, 4 up, 4 in 2024-09-21T03:33:02.812 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:33:02 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-21T03:33:02.812 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:33:02 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T03:33:02.812 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:33:02 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T03:33:02.812 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:33:02 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:33:02.894 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:33:02 smithi110 ceph-mon[34000]: Detected new or changed devices on smithi089 2024-09-21T03:33:02.894 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:33:02 smithi110 ceph-mon[34000]: Adjusting osd_memory_target on smithi089 to 8543M 2024-09-21T03:33:02.894 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:33:02 smithi110 ceph-mon[34000]: from='osd.3 [v2:172.21.15.89:6810/3016980877,v1:172.21.15.89:6811/3016980877]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi089", "root=default"]}]': finished 2024-09-21T03:33:02.894 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:33:02 smithi110 ceph-mon[34000]: osdmap e23: 4 total, 3 up, 4 in 2024-09-21T03:33:02.895 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:33:02 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-21T03:33:02.895 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:33:02 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-21T03:33:02.895 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:33:02 smithi110 ceph-mon[34000]: osd.3 [v2:172.21.15.89:6810/3016980877,v1:172.21.15.89:6811/3016980877] boot 2024-09-21T03:33:02.895 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:33:02 smithi110 ceph-mon[34000]: osdmap e24: 4 total, 4 up, 4 in 2024-09-21T03:33:02.895 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:33:02 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-21T03:33:02.895 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:33:02 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T03:33:02.895 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:33:02 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T03:33:02.895 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:33:02 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:33:02.961 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:33:02 smithi114 ceph-mon[34140]: Detected new or changed devices on smithi089 2024-09-21T03:33:02.961 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:33:02 smithi114 ceph-mon[34140]: Adjusting osd_memory_target on smithi089 to 8543M 2024-09-21T03:33:02.961 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:33:02 smithi114 ceph-mon[34140]: from='osd.3 [v2:172.21.15.89:6810/3016980877,v1:172.21.15.89:6811/3016980877]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi089", "root=default"]}]': finished 2024-09-21T03:33:02.961 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:33:02 smithi114 ceph-mon[34140]: osdmap e23: 4 total, 3 up, 4 in 2024-09-21T03:33:02.961 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:33:02 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-21T03:33:02.961 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:33:02 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-21T03:33:02.961 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:33:02 smithi114 ceph-mon[34140]: osd.3 [v2:172.21.15.89:6810/3016980877,v1:172.21.15.89:6811/3016980877] boot 2024-09-21T03:33:02.961 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:33:02 smithi114 ceph-mon[34140]: osdmap e24: 4 total, 4 up, 4 in 2024-09-21T03:33:02.962 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:33:02 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-21T03:33:02.962 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:33:02 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T03:33:02.962 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:33:02 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T03:33:02.962 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:33:02 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:33:03.750 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:33:03 smithi110 ceph-mon[34000]: purged_snaps scrub starts 2024-09-21T03:33:03.750 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:33:03 smithi110 ceph-mon[34000]: purged_snaps scrub ok 2024-09-21T03:33:03.750 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:33:03 smithi110 ceph-mon[34000]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-09-21T03:33:03.750 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:33:03 smithi110 ceph-mon[34000]: from='client.24359 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi110:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:33:03.751 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:33:03 smithi110 ceph-mon[34000]: osdmap e25: 4 total, 4 up, 4 in 2024-09-21T03:33:03.811 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:33:03 smithi089 ceph-mon[30928]: purged_snaps scrub starts 2024-09-21T03:33:03.811 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:33:03 smithi089 ceph-mon[30928]: purged_snaps scrub ok 2024-09-21T03:33:03.811 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:33:03 smithi089 ceph-mon[30928]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-09-21T03:33:03.811 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:33:03 smithi089 ceph-mon[30928]: from='client.24359 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi110:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:33:03.811 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:33:03 smithi089 ceph-mon[30928]: osdmap e25: 4 total, 4 up, 4 in 2024-09-21T03:33:03.862 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:33:03 smithi114 ceph-mon[34140]: purged_snaps scrub starts 2024-09-21T03:33:03.862 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:33:03 smithi114 ceph-mon[34140]: purged_snaps scrub ok 2024-09-21T03:33:03.862 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:33:03 smithi114 ceph-mon[34140]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-09-21T03:33:03.862 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:33:03 smithi114 ceph-mon[34140]: from='client.24359 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi110:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:33:03.862 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:33:03 smithi114 ceph-mon[34140]: osdmap e25: 4 total, 4 up, 4 in 2024-09-21T03:33:04.750 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:33:04 smithi110 ceph-mon[34000]: pgmap v111: 1 pgs: 1 peering; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-09-21T03:33:04.750 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:33:04 smithi110 ceph-mon[34000]: osdmap e26: 4 total, 4 up, 4 in 2024-09-21T03:33:04.750 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:33:04 smithi110 ceph-mon[34000]: from='client.? 172.21.15.110:0/2582063912' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "8e65399d-6e55-4da4-98ba-f805f0c655a7"} : dispatch 2024-09-21T03:33:04.750 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:33:04 smithi110 ceph-mon[34000]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "8e65399d-6e55-4da4-98ba-f805f0c655a7"} : dispatch 2024-09-21T03:33:04.750 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:33:04 smithi110 ceph-mon[34000]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8e65399d-6e55-4da4-98ba-f805f0c655a7"}]': finished 2024-09-21T03:33:04.750 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:33:04 smithi110 ceph-mon[34000]: osdmap e27: 5 total, 4 up, 5 in 2024-09-21T03:33:04.750 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:33:04 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-21T03:33:04.879 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:33:04 smithi114 ceph-mon[34140]: pgmap v111: 1 pgs: 1 peering; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-09-21T03:33:04.879 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:33:04 smithi114 ceph-mon[34140]: osdmap e26: 4 total, 4 up, 4 in 2024-09-21T03:33:04.879 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:33:04 smithi114 ceph-mon[34140]: from='client.? 172.21.15.110:0/2582063912' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "8e65399d-6e55-4da4-98ba-f805f0c655a7"} : dispatch 2024-09-21T03:33:04.879 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:33:04 smithi114 ceph-mon[34140]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "8e65399d-6e55-4da4-98ba-f805f0c655a7"} : dispatch 2024-09-21T03:33:04.879 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:33:04 smithi114 ceph-mon[34140]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8e65399d-6e55-4da4-98ba-f805f0c655a7"}]': finished 2024-09-21T03:33:04.879 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:33:04 smithi114 ceph-mon[34140]: osdmap e27: 5 total, 4 up, 5 in 2024-09-21T03:33:04.879 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:33:04 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-21T03:33:05.061 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:33:04 smithi089 ceph-mon[30928]: pgmap v111: 1 pgs: 1 peering; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-09-21T03:33:05.061 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:33:04 smithi089 ceph-mon[30928]: osdmap e26: 4 total, 4 up, 4 in 2024-09-21T03:33:05.061 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:33:04 smithi089 ceph-mon[30928]: from='client.? 172.21.15.110:0/2582063912' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "8e65399d-6e55-4da4-98ba-f805f0c655a7"} : dispatch 2024-09-21T03:33:05.061 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:33:04 smithi089 ceph-mon[30928]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "8e65399d-6e55-4da4-98ba-f805f0c655a7"} : dispatch 2024-09-21T03:33:05.061 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:33:04 smithi089 ceph-mon[30928]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8e65399d-6e55-4da4-98ba-f805f0c655a7"}]': finished 2024-09-21T03:33:05.061 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:33:04 smithi089 ceph-mon[30928]: osdmap e27: 5 total, 4 up, 5 in 2024-09-21T03:33:05.061 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:33:04 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-21T03:33:05.960 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:33:05 smithi114 ceph-mon[34140]: from='client.? 172.21.15.110:0/2113691474' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T03:33:06.000 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:33:05 smithi110 ceph-mon[34000]: from='client.? 172.21.15.110:0/2113691474' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T03:33:06.061 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:33:05 smithi089 ceph-mon[30928]: from='client.? 172.21.15.110:0/2113691474' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T03:33:06.960 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:33:06 smithi114 ceph-mon[34140]: pgmap v114: 1 pgs: 1 peering; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-09-21T03:33:07.000 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:33:06 smithi110 ceph-mon[34000]: pgmap v114: 1 pgs: 1 peering; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-09-21T03:33:07.061 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:33:06 smithi089 ceph-mon[30928]: pgmap v114: 1 pgs: 1 peering; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-09-21T03:33:08.905 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:33:08 smithi110 ceph-mon[34000]: pgmap v115: 1 pgs: 1 peering; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-09-21T03:33:08.960 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:33:08 smithi114 ceph-mon[34140]: pgmap v115: 1 pgs: 1 peering; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-09-21T03:33:09.061 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:33:08 smithi089 ceph-mon[30928]: pgmap v115: 1 pgs: 1 peering; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-09-21T03:33:10.738 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:33:10 smithi110 ceph-mon[34000]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 83 KiB/s, 0 objects/s recovering 2024-09-21T03:33:10.961 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:33:10 smithi114 ceph-mon[34140]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 83 KiB/s, 0 objects/s recovering 2024-09-21T03:33:11.061 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:33:10 smithi089 ceph-mon[30928]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 83 KiB/s, 0 objects/s recovering 2024-09-21T03:33:12.913 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:33:12 smithi110 ceph-mon[34000]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-21T03:33:12.913 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:33:12 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.4"} : dispatch 2024-09-21T03:33:12.913 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:33:12 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:33:12.913 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:33:12 smithi110 ceph-mon[34000]: Deploying daemon osd.4 on smithi110 2024-09-21T03:33:12.961 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:33:12 smithi114 ceph-mon[34140]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-21T03:33:12.961 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:33:12 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.4"} : dispatch 2024-09-21T03:33:12.961 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:33:12 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:33:12.961 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:33:12 smithi114 ceph-mon[34140]: Deploying daemon osd.4 on smithi110 2024-09-21T03:33:13.061 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:33:12 smithi089 ceph-mon[30928]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-21T03:33:13.061 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:33:12 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.4"} : dispatch 2024-09-21T03:33:13.061 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:33:12 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:33:13.061 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:33:12 smithi089 ceph-mon[30928]: Deploying daemon osd.4 on smithi110 2024-09-21T03:33:14.674 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:33:14 smithi110 ceph-mon[34000]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-21T03:33:14.961 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:33:14 smithi114 ceph-mon[34140]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-21T03:33:15.061 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:33:14 smithi089 ceph-mon[30928]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-21T03:33:16.931 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:33:16 smithi110 ceph-mon[34000]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 51 KiB/s, 0 objects/s recovering 2024-09-21T03:33:16.962 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:33:16 smithi114 ceph-mon[34140]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 51 KiB/s, 0 objects/s recovering 2024-09-21T03:33:17.061 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:33:16 smithi089 ceph-mon[30928]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 51 KiB/s, 0 objects/s recovering 2024-09-21T03:33:18.863 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:33:18 smithi089 ceph-mon[30928]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-21T03:33:18.863 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:33:18 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:33:18.863 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:33:18 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:33:18.863 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:33:18 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:33:18.902 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:33:18 smithi110 ceph-mon[34000]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-21T03:33:18.902 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:33:18 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:33:18.902 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:33:18 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:33:18.902 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:33:18 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:33:18.961 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:33:18 smithi114 ceph-mon[34140]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-21T03:33:18.961 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:33:18 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:33:18.961 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:33:18 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:33:18.961 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:33:18 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:33:20.750 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:33:20 smithi110 ceph-mon[34000]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-21T03:33:20.960 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:33:20 smithi114 ceph-mon[34140]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-21T03:33:21.061 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:33:20 smithi089 ceph-mon[30928]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-21T03:33:21.921 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:33:21 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:33:21.922 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:33:21 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:33:21.922 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:33:21 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:33:21.922 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:33:21 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:33:21.922 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:33:21 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:33:21.963 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T03:33:21.961+0000 7fc5857fa640 1 -- 172.21.15.110:0/73225394 <== mgr.14152 v2:172.21.15.89:6800/1591532358 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fc588063800 con 0x7fc5700b6510 2024-09-21T03:33:21.963 INFO:teuthology.orchestra.run.smithi110.stdout:Created osd(s) 4 on host 'smithi110' 2024-09-21T03:33:21.964 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T03:33:21.963+0000 7fc58e74a640 1 -- 172.21.15.110:0/73225394 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7fc5700b6510 msgr2=0x7fc5700b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:21.965 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T03:33:21.963+0000 7fc58e74a640 1 --2- 172.21.15.110:0/73225394 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7fc5700b6510 0x7fc5700b8a20 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fc57c0057c0 tx=0x7fc57c000bc0 comp rx=0 tx=0).stop 2024-09-21T03:33:21.965 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T03:33:21.963+0000 7fc58e74a640 1 -- 172.21.15.110:0/73225394 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc588102640 msgr2=0x7fc588198db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:21.965 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T03:33:21.963+0000 7fc58e74a640 1 --2- 172.21.15.110:0/73225394 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc588102640 0x7fc588198db0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fc57800c920 tx=0x7fc57800cde0 comp rx=0 tx=0).stop 2024-09-21T03:33:21.965 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T03:33:21.963+0000 7fc58e74a640 1 -- 172.21.15.110:0/73225394 shutdown_connections 2024-09-21T03:33:21.966 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T03:33:21.963+0000 7fc58e74a640 1 --2- 172.21.15.110:0/73225394 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7fc5700b6510 0x7fc5700b8a20 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:21.966 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T03:33:21.963+0000 7fc58e74a640 1 --2- 172.21.15.110:0/73225394 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc588104070 0x7fc588199fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:21.966 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T03:33:21.963+0000 7fc58e74a640 1 --2- 172.21.15.110:0/73225394 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc5881036d0 0x7fc588199420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:21.966 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T03:33:21.963+0000 7fc58e74a640 1 --2- 172.21.15.110:0/73225394 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc588102640 0x7fc588198db0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:21.966 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T03:33:21.963+0000 7fc58e74a640 1 -- 172.21.15.110:0/73225394 >> 172.21.15.110:0/73225394 conn(0x7fc5880fdfb0 msgr2=0x7fc5881069f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:33:21.966 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T03:33:21.963+0000 7fc58e74a640 1 -- 172.21.15.110:0/73225394 shutdown_connections 2024-09-21T03:33:21.966 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T03:33:21.963+0000 7fc58e74a640 1 -- 172.21.15.110:0/73225394 wait complete. 2024-09-21T03:33:22.211 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:33:21 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:33:22.211 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:33:21 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:33:22.211 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:33:21 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:33:22.211 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:33:21 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:33:22.211 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:33:21 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:33:22.311 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:33:21 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:33:22.311 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:33:21 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:33:22.311 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:33:21 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:33:22.311 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:33:21 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:33:22.311 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:33:21 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:33:22.578 DEBUG:teuthology.orchestra.run.smithi110:osd.4> sudo journalctl -f -n 0 -u ceph-7c374098-77c9-11ef-baf5-efdc52797490@osd.4.service 2024-09-21T03:33:22.581 INFO:tasks.cephadm:Deploying osd.5 on smithi114 with /dev/vg_nvme/lv_3... 2024-09-21T03:33:22.581 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7c374098-77c9-11ef-baf5-efdc52797490 -- lvm zap /dev/vg_nvme/lv_3 2024-09-21T03:33:22.836 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/7c374098-77c9-11ef-baf5-efdc52797490/mon.c/config 2024-09-21T03:33:23.164 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:33:22 smithi114 ceph-mon[34140]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-09-21T03:33:23.164 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:33:22 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:33:23.164 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:33:22 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:33:23.164 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:33:22 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:33:23.250 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:33:22 smithi110 ceph-mon[34000]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-09-21T03:33:23.250 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:33:22 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:33:23.250 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:33:22 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:33:23.250 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:33:22 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:33:23.311 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:33:22 smithi089 ceph-mon[30928]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-09-21T03:33:23.311 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:33:22 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:33:23.311 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:33:22 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:33:23.311 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:33:22 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:33:24.500 INFO:journalctl@ceph.osd.4.smithi110.stdout:Sep 21 03:33:24 smithi110 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-4[44035]: 2024-09-21T03:33:24.211+0000 7fe44038d640 -1 osd.4 0 log_to_monitors true 2024-09-21T03:33:25.149 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:33:24 smithi110 ceph-mon[34000]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-09-21T03:33:25.149 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:33:24 smithi110 ceph-mon[34000]: from='osd.4 [v2:172.21.15.110:6808/3217880476,v1:172.21.15.110:6809/3217880476]' entity='osd.4' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]} : dispatch 2024-09-21T03:33:25.149 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:33:24 smithi110 ceph-mon[34000]: from='osd.4 ' entity='osd.4' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]} : dispatch 2024-09-21T03:33:25.211 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:33:24 smithi114 ceph-mon[34140]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-09-21T03:33:25.211 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:33:24 smithi114 ceph-mon[34140]: from='osd.4 [v2:172.21.15.110:6808/3217880476,v1:172.21.15.110:6809/3217880476]' entity='osd.4' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]} : dispatch 2024-09-21T03:33:25.211 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:33:24 smithi114 ceph-mon[34140]: from='osd.4 ' entity='osd.4' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]} : dispatch 2024-09-21T03:33:25.311 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:33:24 smithi089 ceph-mon[30928]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-09-21T03:33:25.311 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:33:24 smithi089 ceph-mon[30928]: from='osd.4 [v2:172.21.15.110:6808/3217880476,v1:172.21.15.110:6809/3217880476]' entity='osd.4' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]} : dispatch 2024-09-21T03:33:25.311 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:33:24 smithi089 ceph-mon[30928]: from='osd.4 ' entity='osd.4' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]} : dispatch 2024-09-21T03:33:25.324 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-21T03:33:25.350 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7c374098-77c9-11ef-baf5-efdc52797490 -- ceph orch daemon add osd smithi114:vg_nvme/lv_3 2024-09-21T03:33:25.593 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/7c374098-77c9-11ef-baf5-efdc52797490/mon.c/config 2024-09-21T03:33:26.050 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:33:25 smithi114 ceph-mon[34140]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-09-21T03:33:26.051 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:33:25 smithi114 ceph-mon[34140]: from='osd.4 [v2:172.21.15.110:6808/3217880476,v1:172.21.15.110:6809/3217880476]' entity='osd.4' cmd={"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi110", "root=default"]} : dispatch 2024-09-21T03:33:26.051 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:33:25 smithi114 ceph-mon[34140]: osdmap e28: 5 total, 4 up, 5 in 2024-09-21T03:33:26.051 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:33:25 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-21T03:33:26.051 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:33:25 smithi114 ceph-mon[34140]: from='osd.4 ' entity='osd.4' cmd={"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi110", "root=default"]} : dispatch 2024-09-21T03:33:26.051 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:33:25 smithi114 ceph-mon[34140]: Detected new or changed devices on smithi110 2024-09-21T03:33:26.051 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:33:25 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:33:26.051 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:33:25 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:33:26.051 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:33:25 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-21T03:33:26.052 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:33:25 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-21T03:33:26.052 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:33:25 smithi114 ceph-mon[34140]: Adjusting osd_memory_target on smithi110 to 8544M 2024-09-21T03:33:26.052 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:33:25 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:33:26.052 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:33:25 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:33:26.052 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:33:25 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:33:26.052 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:33:25 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:33:26.055 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T03:33:26.048+0000 7fc95e2f5640 1 -- 172.21.15.114:0/2323304833 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc958106900 msgr2=0x7fc958108d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:26.055 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T03:33:26.048+0000 7fc95e2f5640 1 --2- 172.21.15.114:0/2323304833 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc958106900 0x7fc958108d60 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fc94c009900 tx=0x7fc94c02a410 comp rx=0 tx=0).stop 2024-09-21T03:33:26.055 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T03:33:26.048+0000 7fc95e2f5640 1 -- 172.21.15.114:0/2323304833 shutdown_connections 2024-09-21T03:33:26.055 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T03:33:26.048+0000 7fc95e2f5640 1 --2- 172.21.15.114:0/2323304833 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc9581094a0 0x7fc95810b9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:26.055 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T03:33:26.048+0000 7fc95e2f5640 1 --2- 172.21.15.114:0/2323304833 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc958106900 0x7fc958108d60 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:26.055 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T03:33:26.048+0000 7fc95e2f5640 1 --2- 172.21.15.114:0/2323304833 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc95806c680 0x7fc9581063c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:26.055 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T03:33:26.048+0000 7fc95e2f5640 1 -- 172.21.15.114:0/2323304833 >> 172.21.15.114:0/2323304833 conn(0x7fc9580fdfe0 msgr2=0x7fc958100490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:33:26.055 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T03:33:26.049+0000 7fc95e2f5640 1 -- 172.21.15.114:0/2323304833 shutdown_connections 2024-09-21T03:33:26.055 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T03:33:26.049+0000 7fc95e2f5640 1 -- 172.21.15.114:0/2323304833 wait complete. 2024-09-21T03:33:26.056 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T03:33:26.049+0000 7fc95e2f5640 1 Processor -- start 2024-09-21T03:33:26.056 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T03:33:26.049+0000 7fc95e2f5640 1 -- start start 2024-09-21T03:33:26.056 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T03:33:26.050+0000 7fc95e2f5640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc95806c680 0x7fc958199f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:26.056 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T03:33:26.050+0000 7fc95e2f5640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc958106900 0x7fc95819a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:26.056 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T03:33:26.050+0000 7fc95e2f5640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc9581094a0 0x7fc95819b140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:26.056 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T03:33:26.050+0000 7fc95e2f5640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fc95810f240 con 0x7fc95806c680 2024-09-21T03:33:26.056 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T03:33:26.050+0000 7fc95e2f5640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fc95810f0c0 con 0x7fc958106900 2024-09-21T03:33:26.056 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T03:33:26.050+0000 7fc95e2f5640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fc95810f3c0 con 0x7fc9581094a0 2024-09-21T03:33:26.056 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T03:33:26.050+0000 7fc95c892640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc9581094a0 0x7fc95819b140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:26.056 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T03:33:26.050+0000 7fc95c892640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc9581094a0 0x7fc95819b140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:39932/0 (socket says 172.21.15.114:39932) 2024-09-21T03:33:26.056 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T03:33:26.050+0000 7fc95c892640 1 -- 172.21.15.114:0/778701048 learned_addr learned my addr 172.21.15.114:0/778701048 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T03:33:26.056 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T03:33:26.050+0000 7fc957fff640 1 --2- 172.21.15.114:0/778701048 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc95806c680 0x7fc958199f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:26.057 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T03:33:26.050+0000 7fc9577fe640 1 --2- 172.21.15.114:0/778701048 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc958106900 0x7fc95819a5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:26.057 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T03:33:26.050+0000 7fc95c892640 1 -- 172.21.15.114:0/778701048 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc958106900 msgr2=0x7fc95819a5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:26.057 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T03:33:26.050+0000 7fc95c892640 1 --2- 172.21.15.114:0/778701048 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc958106900 0x7fc95819a5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:26.057 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T03:33:26.051+0000 7fc95c892640 1 -- 172.21.15.114:0/778701048 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc95806c680 msgr2=0x7fc958199f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:26.057 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T03:33:26.051+0000 7fc95c892640 1 --2- 172.21.15.114:0/778701048 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc95806c680 0x7fc958199f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:26.057 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T03:33:26.051+0000 7fc95c892640 1 -- 172.21.15.114:0/778701048 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc95819b8d0 con 0x7fc9581094a0 2024-09-21T03:33:26.057 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T03:33:26.051+0000 7fc957fff640 1 --2- 172.21.15.114:0/778701048 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc95806c680 0x7fc958199f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:33:26.057 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T03:33:26.051+0000 7fc95c892640 1 --2- 172.21.15.114:0/778701048 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc9581094a0 0x7fc95819b140 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fc95000eb10 tx=0x7fc95000ef10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:33:26.057 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T03:33:26.051+0000 7fc9557fa640 1 -- 172.21.15.114:0/778701048 <== mon.1 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc95000ce20 con 0x7fc9581094a0 2024-09-21T03:33:26.057 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T03:33:26.051+0000 7fc9557fa640 1 -- 172.21.15.114:0/778701048 <== mon.1 v2:172.21.15.114:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc950005020 con 0x7fc9581094a0 2024-09-21T03:33:26.057 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T03:33:26.051+0000 7fc95e2f5640 1 -- 172.21.15.114:0/778701048 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc95819bbc0 con 0x7fc9581094a0 2024-09-21T03:33:26.058 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T03:33:26.051+0000 7fc9557fa640 1 -- 172.21.15.114:0/778701048 <== mon.1 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc950010690 con 0x7fc9581094a0 2024-09-21T03:33:26.058 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T03:33:26.051+0000 7fc95e2f5640 1 -- 172.21.15.114:0/778701048 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9581021a0 con 0x7fc9581094a0 2024-09-21T03:33:26.058 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T03:33:26.052+0000 7fc95e2f5640 1 -- 172.21.15.114:0/778701048 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc9580ffb00 con 0x7fc9581094a0 2024-09-21T03:33:26.059 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T03:33:26.054+0000 7fc9557fa640 1 -- 172.21.15.114:0/778701048 <== mon.1 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 17) ==== 154512+0+0 (secure 0 0 0) 0x7fc950004a40 con 0x7fc9581094a0 2024-09-21T03:33:26.059 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T03:33:26.054+0000 7fc9557fa640 1 --2- 172.21.15.114:0/778701048 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7fc9400b6510 0x7fc9400b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:26.059 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T03:33:26.054+0000 7fc9557fa640 1 -- 172.21.15.114:0/778701048 <== mon.1 v2:172.21.15.114:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 3821+0+0 (secure 0 0 0) 0x7fc9500e6920 con 0x7fc9581094a0 2024-09-21T03:33:26.059 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T03:33:26.055+0000 7fc957fff640 1 --2- 172.21.15.114:0/778701048 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7fc9400b6510 0x7fc9400b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:26.059 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T03:33:26.055+0000 7fc957fff640 1 --2- 172.21.15.114:0/778701048 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7fc9400b6510 0x7fc9400b8a20 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fc94800afb0 tx=0x7fc94800a2d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:33:26.060 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T03:33:26.059+0000 7fc9557fa640 1 -- 172.21.15.114:0/778701048 <== mon.1 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fc9500ae2a0 con 0x7fc9581094a0 2024-09-21T03:33:26.247 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T03:33:26.245+0000 7fc95e2f5640 1 -- 172.21.15.114:0/778701048 --> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi114:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7fc958063800 con 0x7fc9400b6510 2024-09-21T03:33:26.250 INFO:journalctl@ceph.osd.4.smithi110.stdout:Sep 21 03:33:25 smithi110 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-4[44035]: 2024-09-21T03:33:25.929+0000 7fe43c30f640 -1 osd.4 0 waiting for initial osdmap 2024-09-21T03:33:26.250 INFO:journalctl@ceph.osd.4.smithi110.stdout:Sep 21 03:33:25 smithi110 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-4[44035]: 2024-09-21T03:33:25.935+0000 7fe43810d640 -1 osd.4 29 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-21T03:33:26.250 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:33:25 smithi110 ceph-mon[34000]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-09-21T03:33:26.250 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:33:25 smithi110 ceph-mon[34000]: from='osd.4 [v2:172.21.15.110:6808/3217880476,v1:172.21.15.110:6809/3217880476]' entity='osd.4' cmd={"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi110", "root=default"]} : dispatch 2024-09-21T03:33:26.250 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:33:25 smithi110 ceph-mon[34000]: osdmap e28: 5 total, 4 up, 5 in 2024-09-21T03:33:26.251 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:33:25 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-21T03:33:26.251 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:33:25 smithi110 ceph-mon[34000]: from='osd.4 ' entity='osd.4' cmd={"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi110", "root=default"]} : dispatch 2024-09-21T03:33:26.251 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:33:25 smithi110 ceph-mon[34000]: Detected new or changed devices on smithi110 2024-09-21T03:33:26.251 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:33:25 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:33:26.251 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:33:25 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:33:26.251 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:33:25 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-21T03:33:26.251 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:33:25 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-21T03:33:26.251 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:33:25 smithi110 ceph-mon[34000]: Adjusting osd_memory_target on smithi110 to 8544M 2024-09-21T03:33:26.251 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:33:25 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:33:26.251 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:33:25 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:33:26.251 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:33:25 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:33:26.251 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:33:25 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:33:26.311 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:33:25 smithi089 ceph-mon[30928]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-09-21T03:33:26.311 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:33:25 smithi089 ceph-mon[30928]: from='osd.4 [v2:172.21.15.110:6808/3217880476,v1:172.21.15.110:6809/3217880476]' entity='osd.4' cmd={"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi110", "root=default"]} : dispatch 2024-09-21T03:33:26.311 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:33:25 smithi089 ceph-mon[30928]: osdmap e28: 5 total, 4 up, 5 in 2024-09-21T03:33:26.311 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:33:25 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-21T03:33:26.311 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:33:25 smithi089 ceph-mon[30928]: from='osd.4 ' entity='osd.4' cmd={"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi110", "root=default"]} : dispatch 2024-09-21T03:33:26.312 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:33:25 smithi089 ceph-mon[30928]: Detected new or changed devices on smithi110 2024-09-21T03:33:26.312 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:33:25 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:33:26.312 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:33:25 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:33:26.312 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:33:25 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-21T03:33:26.312 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:33:25 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-21T03:33:26.312 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:33:25 smithi089 ceph-mon[30928]: Adjusting osd_memory_target on smithi110 to 8544M 2024-09-21T03:33:26.312 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:33:25 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:33:26.312 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:33:25 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:33:26.312 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:33:25 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:33:26.312 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:33:25 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:33:27.211 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:33:26 smithi114 ceph-mon[34140]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-09-21T03:33:27.211 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:33:26 smithi114 ceph-mon[34140]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi110", "root=default"]}]': finished 2024-09-21T03:33:27.211 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:33:26 smithi114 ceph-mon[34140]: osdmap e29: 5 total, 4 up, 5 in 2024-09-21T03:33:27.211 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:33:26 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-21T03:33:27.211 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:33:26 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-21T03:33:27.211 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:33:26 smithi114 ceph-mon[34140]: from='client.24361 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi114:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:33:27.211 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:33:26 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T03:33:27.211 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:33:26 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T03:33:27.212 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:33:26 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:33:27.250 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:33:26 smithi110 ceph-mon[34000]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-09-21T03:33:27.250 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:33:26 smithi110 ceph-mon[34000]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi110", "root=default"]}]': finished 2024-09-21T03:33:27.250 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:33:26 smithi110 ceph-mon[34000]: osdmap e29: 5 total, 4 up, 5 in 2024-09-21T03:33:27.250 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:33:26 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-21T03:33:27.250 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:33:26 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-21T03:33:27.250 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:33:26 smithi110 ceph-mon[34000]: from='client.24361 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi114:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:33:27.250 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:33:26 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T03:33:27.250 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:33:26 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T03:33:27.250 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:33:26 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:33:27.311 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:33:26 smithi089 ceph-mon[30928]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-09-21T03:33:27.311 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:33:26 smithi089 ceph-mon[30928]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi110", "root=default"]}]': finished 2024-09-21T03:33:27.311 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:33:26 smithi089 ceph-mon[30928]: osdmap e29: 5 total, 4 up, 5 in 2024-09-21T03:33:27.311 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:33:26 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-21T03:33:27.311 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:33:26 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-21T03:33:27.311 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:33:26 smithi089 ceph-mon[30928]: from='client.24361 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi114:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:33:27.311 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:33:26 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T03:33:27.312 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:33:26 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T03:33:27.312 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:33:26 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:33:28.211 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:33:27 smithi114 ceph-mon[34140]: purged_snaps scrub starts 2024-09-21T03:33:28.211 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:33:27 smithi114 ceph-mon[34140]: purged_snaps scrub ok 2024-09-21T03:33:28.211 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:33:27 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-21T03:33:28.211 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:33:27 smithi114 ceph-mon[34140]: osd.4 [v2:172.21.15.110:6808/3217880476,v1:172.21.15.110:6809/3217880476] boot 2024-09-21T03:33:28.212 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:33:27 smithi114 ceph-mon[34140]: osdmap e30: 5 total, 5 up, 5 in 2024-09-21T03:33:28.212 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:33:27 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-21T03:33:28.250 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:33:27 smithi110 ceph-mon[34000]: purged_snaps scrub starts 2024-09-21T03:33:28.250 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:33:27 smithi110 ceph-mon[34000]: purged_snaps scrub ok 2024-09-21T03:33:28.250 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:33:27 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-21T03:33:28.250 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:33:27 smithi110 ceph-mon[34000]: osd.4 [v2:172.21.15.110:6808/3217880476,v1:172.21.15.110:6809/3217880476] boot 2024-09-21T03:33:28.250 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:33:27 smithi110 ceph-mon[34000]: osdmap e30: 5 total, 5 up, 5 in 2024-09-21T03:33:28.251 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:33:27 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-21T03:33:28.311 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:33:27 smithi089 ceph-mon[30928]: purged_snaps scrub starts 2024-09-21T03:33:28.311 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:33:27 smithi089 ceph-mon[30928]: purged_snaps scrub ok 2024-09-21T03:33:28.311 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:33:27 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-21T03:33:28.311 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:33:27 smithi089 ceph-mon[30928]: osd.4 [v2:172.21.15.110:6808/3217880476,v1:172.21.15.110:6809/3217880476] boot 2024-09-21T03:33:28.311 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:33:27 smithi089 ceph-mon[30928]: osdmap e30: 5 total, 5 up, 5 in 2024-09-21T03:33:28.311 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:33:27 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-21T03:33:29.186 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:33:28 smithi114 ceph-mon[34140]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-09-21T03:33:29.186 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:33:28 smithi114 ceph-mon[34140]: osdmap e31: 5 total, 5 up, 5 in 2024-09-21T03:33:29.186 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:33:28 smithi114 ceph-mon[34140]: from='client.? 172.21.15.114:0/2124989608' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "b9ec2889-46bd-40e5-8446-73ab28271c67"} : dispatch 2024-09-21T03:33:29.186 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:33:28 smithi114 ceph-mon[34140]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "b9ec2889-46bd-40e5-8446-73ab28271c67"} : dispatch 2024-09-21T03:33:29.186 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:33:28 smithi114 ceph-mon[34140]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b9ec2889-46bd-40e5-8446-73ab28271c67"}]': finished 2024-09-21T03:33:29.186 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:33:28 smithi114 ceph-mon[34140]: osdmap e32: 6 total, 5 up, 6 in 2024-09-21T03:33:29.186 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:33:28 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-21T03:33:29.250 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:33:28 smithi110 ceph-mon[34000]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-09-21T03:33:29.250 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:33:28 smithi110 ceph-mon[34000]: osdmap e31: 5 total, 5 up, 5 in 2024-09-21T03:33:29.250 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:33:28 smithi110 ceph-mon[34000]: from='client.? 172.21.15.114:0/2124989608' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "b9ec2889-46bd-40e5-8446-73ab28271c67"} : dispatch 2024-09-21T03:33:29.250 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:33:28 smithi110 ceph-mon[34000]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "b9ec2889-46bd-40e5-8446-73ab28271c67"} : dispatch 2024-09-21T03:33:29.250 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:33:28 smithi110 ceph-mon[34000]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b9ec2889-46bd-40e5-8446-73ab28271c67"}]': finished 2024-09-21T03:33:29.250 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:33:28 smithi110 ceph-mon[34000]: osdmap e32: 6 total, 5 up, 6 in 2024-09-21T03:33:29.250 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:33:28 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-21T03:33:29.311 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:33:28 smithi089 ceph-mon[30928]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-09-21T03:33:29.311 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:33:28 smithi089 ceph-mon[30928]: osdmap e31: 5 total, 5 up, 5 in 2024-09-21T03:33:29.311 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:33:28 smithi089 ceph-mon[30928]: from='client.? 172.21.15.114:0/2124989608' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "b9ec2889-46bd-40e5-8446-73ab28271c67"} : dispatch 2024-09-21T03:33:29.311 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:33:28 smithi089 ceph-mon[30928]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "b9ec2889-46bd-40e5-8446-73ab28271c67"} : dispatch 2024-09-21T03:33:29.311 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:33:28 smithi089 ceph-mon[30928]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b9ec2889-46bd-40e5-8446-73ab28271c67"}]': finished 2024-09-21T03:33:29.311 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:33:28 smithi089 ceph-mon[30928]: osdmap e32: 6 total, 5 up, 6 in 2024-09-21T03:33:29.311 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:33:28 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-21T03:33:30.210 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:33:29 smithi114 ceph-mon[34140]: from='client.? 172.21.15.114:0/908931652' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T03:33:30.250 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:33:29 smithi110 ceph-mon[34000]: from='client.? 172.21.15.114:0/908931652' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T03:33:30.311 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:33:29 smithi089 ceph-mon[30928]: from='client.? 172.21.15.114:0/908931652' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T03:33:31.211 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:33:30 smithi114 ceph-mon[34140]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-21T03:33:31.250 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:33:30 smithi110 ceph-mon[34000]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-21T03:33:31.311 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:33:30 smithi089 ceph-mon[30928]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-21T03:33:33.210 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:33:32 smithi114 ceph-mon[34140]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-21T03:33:33.250 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:33:32 smithi110 ceph-mon[34000]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-21T03:33:33.311 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:33:32 smithi089 ceph-mon[30928]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-21T03:33:35.152 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:33:34 smithi114 ceph-mon[34140]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-21T03:33:35.250 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:33:34 smithi110 ceph-mon[34000]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-21T03:33:35.311 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:33:34 smithi089 ceph-mon[30928]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-21T03:33:37.079 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:33:36 smithi114 ceph-mon[34140]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-21T03:33:37.250 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:33:36 smithi110 ceph-mon[34000]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-21T03:33:37.311 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:33:36 smithi089 ceph-mon[30928]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-21T03:33:38.211 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:33:37 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.5"} : dispatch 2024-09-21T03:33:38.211 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:33:37 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:33:38.211 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:33:37 smithi114 ceph-mon[34140]: Deploying daemon osd.5 on smithi114 2024-09-21T03:33:38.250 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:33:37 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.5"} : dispatch 2024-09-21T03:33:38.250 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:33:37 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:33:38.250 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:33:37 smithi110 ceph-mon[34000]: Deploying daemon osd.5 on smithi114 2024-09-21T03:33:38.311 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:33:37 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.5"} : dispatch 2024-09-21T03:33:38.311 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:33:37 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:33:38.311 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:33:37 smithi089 ceph-mon[30928]: Deploying daemon osd.5 on smithi114 2024-09-21T03:33:39.014 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:33:38 smithi114 ceph-mon[34140]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-21T03:33:39.250 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:33:38 smithi110 ceph-mon[34000]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-21T03:33:39.311 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:33:38 smithi089 ceph-mon[30928]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-21T03:33:41.250 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:33:40 smithi110 ceph-mon[34000]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-21T03:33:41.311 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:33:40 smithi089 ceph-mon[30928]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-21T03:33:41.461 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:33:40 smithi114 ceph-mon[34140]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-21T03:33:43.239 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:33:42 smithi114 ceph-mon[34140]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-21T03:33:43.250 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:33:42 smithi110 ceph-mon[34000]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-21T03:33:43.311 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:33:42 smithi089 ceph-mon[30928]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-21T03:33:43.984 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:33:43 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:33:43.986 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:33:43 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:33:43.986 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:33:43 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:33:44.250 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:33:43 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:33:44.250 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:33:43 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:33:44.250 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:33:43 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:33:44.311 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:33:43 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:33:44.311 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:33:43 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:33:44.311 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:33:43 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:33:45.461 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:33:45 smithi114 ceph-mon[34140]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-21T03:33:45.750 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:33:45 smithi110 ceph-mon[34000]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-21T03:33:45.811 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:33:45 smithi089 ceph-mon[30928]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-21T03:33:46.765 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T03:33:46.763+0000 7fc9557fa640 1 -- 172.21.15.114:0/778701048 <== mgr.14152 v2:172.21.15.89:6800/1591532358 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fc958063800 con 0x7fc9400b6510 2024-09-21T03:33:46.765 INFO:teuthology.orchestra.run.smithi114.stdout:Created osd(s) 5 on host 'smithi114' 2024-09-21T03:33:46.767 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T03:33:46.765+0000 7fc95e2f5640 1 -- 172.21.15.114:0/778701048 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7fc9400b6510 msgr2=0x7fc9400b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:46.768 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T03:33:46.765+0000 7fc95e2f5640 1 --2- 172.21.15.114:0/778701048 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7fc9400b6510 0x7fc9400b8a20 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fc94800afb0 tx=0x7fc94800a2d0 comp rx=0 tx=0).stop 2024-09-21T03:33:46.768 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T03:33:46.765+0000 7fc95e2f5640 1 -- 172.21.15.114:0/778701048 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc9581094a0 msgr2=0x7fc95819b140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:46.768 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T03:33:46.765+0000 7fc95e2f5640 1 --2- 172.21.15.114:0/778701048 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc9581094a0 0x7fc95819b140 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fc95000eb10 tx=0x7fc95000ef10 comp rx=0 tx=0).stop 2024-09-21T03:33:46.768 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T03:33:46.766+0000 7fc95e2f5640 1 -- 172.21.15.114:0/778701048 shutdown_connections 2024-09-21T03:33:46.768 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T03:33:46.766+0000 7fc95e2f5640 1 --2- 172.21.15.114:0/778701048 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7fc9400b6510 0x7fc9400b8a20 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:46.768 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T03:33:46.766+0000 7fc95e2f5640 1 --2- 172.21.15.114:0/778701048 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc958106900 0x7fc95819a5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:46.768 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T03:33:46.766+0000 7fc95e2f5640 1 --2- 172.21.15.114:0/778701048 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc9581094a0 0x7fc95819b140 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:46.768 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T03:33:46.766+0000 7fc95e2f5640 1 --2- 172.21.15.114:0/778701048 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc95806c680 0x7fc958199f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:46.768 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T03:33:46.766+0000 7fc95e2f5640 1 -- 172.21.15.114:0/778701048 >> 172.21.15.114:0/778701048 conn(0x7fc9580fdfe0 msgr2=0x7fc958104f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:33:46.768 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T03:33:46.766+0000 7fc95e2f5640 1 -- 172.21.15.114:0/778701048 shutdown_connections 2024-09-21T03:33:46.768 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T03:33:46.766+0000 7fc95e2f5640 1 -- 172.21.15.114:0/778701048 wait complete. 2024-09-21T03:33:47.093 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:33:46 smithi114 ceph-mon[34140]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-21T03:33:47.093 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:33:46 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:33:47.093 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:33:46 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:33:47.093 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:33:46 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:33:47.093 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:33:46 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:33:47.094 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:33:46 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:33:47.094 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:33:46 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:33:47.094 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:33:46 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:33:47.094 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:33:46 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:33:47.250 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:33:46 smithi110 ceph-mon[34000]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-21T03:33:47.250 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:33:46 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:33:47.250 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:33:46 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:33:47.250 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:33:46 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:33:47.250 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:33:46 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:33:47.250 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:33:46 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:33:47.250 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:33:46 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:33:47.250 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:33:46 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:33:47.250 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:33:46 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:33:47.311 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:33:46 smithi089 ceph-mon[30928]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-21T03:33:47.311 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:33:46 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:33:47.311 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:33:46 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:33:47.311 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:33:46 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:33:47.311 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:33:46 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:33:47.311 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:33:46 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:33:47.311 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:33:46 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:33:47.311 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:33:46 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:33:47.311 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:33:46 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:33:47.544 DEBUG:teuthology.orchestra.run.smithi114:osd.5> sudo journalctl -f -n 0 -u ceph-7c374098-77c9-11ef-baf5-efdc52797490@osd.5.service 2024-09-21T03:33:47.546 INFO:tasks.cephadm:Deploying osd.6 on smithi089 with /dev/vg_nvme/lv_2... 2024-09-21T03:33:47.547 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7c374098-77c9-11ef-baf5-efdc52797490 -- lvm zap /dev/vg_nvme/lv_2 2024-09-21T03:33:47.805 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/7c374098-77c9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:33:49.184 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:33:48 smithi089 ceph-mon[30928]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-21T03:33:49.208 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:33:48 smithi114 ceph-mon[34140]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-21T03:33:49.250 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:33:48 smithi110 ceph-mon[34000]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-21T03:33:49.461 INFO:journalctl@ceph.osd.5.smithi114.stdout:Sep 21 03:33:49 smithi114 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-5[45121]: 2024-09-21T03:33:49.206+0000 7fa958c5f640 -1 osd.5 0 log_to_monitors true 2024-09-21T03:33:50.061 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:33:49 smithi089 ceph-mon[30928]: from='osd.5 [v2:172.21.15.114:6808/1144140707,v1:172.21.15.114:6809/1144140707]' entity='osd.5' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]} : dispatch 2024-09-21T03:33:50.061 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:33:49 smithi089 ceph-mon[30928]: from='osd.5 ' entity='osd.5' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]} : dispatch 2024-09-21T03:33:50.210 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:33:49 smithi114 ceph-mon[34140]: from='osd.5 [v2:172.21.15.114:6808/1144140707,v1:172.21.15.114:6809/1144140707]' entity='osd.5' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]} : dispatch 2024-09-21T03:33:50.211 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:33:49 smithi114 ceph-mon[34140]: from='osd.5 ' entity='osd.5' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]} : dispatch 2024-09-21T03:33:50.250 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:33:49 smithi110 ceph-mon[34000]: from='osd.5 [v2:172.21.15.114:6808/1144140707,v1:172.21.15.114:6809/1144140707]' entity='osd.5' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]} : dispatch 2024-09-21T03:33:50.250 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:33:49 smithi110 ceph-mon[34000]: from='osd.5 ' entity='osd.5' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]} : dispatch 2024-09-21T03:33:50.562 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-09-21T03:33:50.588 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7c374098-77c9-11ef-baf5-efdc52797490 -- ceph orch daemon add osd smithi089:vg_nvme/lv_2 2024-09-21T03:33:50.845 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/7c374098-77c9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:33:51.094 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:33:50 smithi089 ceph-mon[30928]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-21T03:33:51.094 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:33:50 smithi089 ceph-mon[30928]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-09-21T03:33:51.094 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:33:50 smithi089 ceph-mon[30928]: from='osd.5 [v2:172.21.15.114:6808/1144140707,v1:172.21.15.114:6809/1144140707]' entity='osd.5' cmd={"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi114", "root=default"]} : dispatch 2024-09-21T03:33:51.094 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:33:50 smithi089 ceph-mon[30928]: osdmap e33: 6 total, 5 up, 6 in 2024-09-21T03:33:51.094 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:33:50 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-21T03:33:51.094 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:33:50 smithi089 ceph-mon[30928]: from='osd.5 ' entity='osd.5' cmd={"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi114", "root=default"]} : dispatch 2024-09-21T03:33:51.094 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:33:50 smithi089 ceph-mon[30928]: Detected new or changed devices on smithi114 2024-09-21T03:33:51.094 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:33:50 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:33:51.095 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:33:50 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:33:51.095 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:33:50 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-21T03:33:51.095 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:33:50 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-21T03:33:51.095 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:33:50 smithi089 ceph-mon[30928]: Adjusting osd_memory_target on smithi114 to 8544M 2024-09-21T03:33:51.095 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:33:50 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:33:51.095 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:33:50 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:33:51.095 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:33:50 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:33:51.095 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:33:50 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:33:51.250 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:33:50 smithi110 ceph-mon[34000]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-21T03:33:51.250 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:33:50 smithi110 ceph-mon[34000]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-09-21T03:33:51.250 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:33:50 smithi110 ceph-mon[34000]: from='osd.5 [v2:172.21.15.114:6808/1144140707,v1:172.21.15.114:6809/1144140707]' entity='osd.5' cmd={"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi114", "root=default"]} : dispatch 2024-09-21T03:33:51.250 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:33:50 smithi110 ceph-mon[34000]: osdmap e33: 6 total, 5 up, 6 in 2024-09-21T03:33:51.250 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:33:50 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-21T03:33:51.250 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:33:50 smithi110 ceph-mon[34000]: from='osd.5 ' entity='osd.5' cmd={"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi114", "root=default"]} : dispatch 2024-09-21T03:33:51.250 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:33:50 smithi110 ceph-mon[34000]: Detected new or changed devices on smithi114 2024-09-21T03:33:51.250 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:33:50 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:33:51.251 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:33:50 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:33:51.251 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:33:50 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-21T03:33:51.251 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:33:50 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-21T03:33:51.251 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:33:50 smithi110 ceph-mon[34000]: Adjusting osd_memory_target on smithi114 to 8544M 2024-09-21T03:33:51.251 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:33:50 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:33:51.251 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:33:50 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:33:51.251 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:33:50 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:33:51.251 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:33:50 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:33:51.324 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:33:51.322+0000 7fa3cbd66640 1 -- 172.21.15.89:0/3524862568 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa3c4103780 msgr2=0x7fa3c4103bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:51.325 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:33:51.322+0000 7fa3cbd66640 1 --2- 172.21.15.89:0/3524862568 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa3c4103780 0x7fa3c4103bf0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fa3c0009900 tx=0x7fa3c002a480 comp rx=0 tx=0).stop 2024-09-21T03:33:51.325 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:33:51.322+0000 7fa3cbd66640 1 -- 172.21.15.89:0/3524862568 shutdown_connections 2024-09-21T03:33:51.325 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:33:51.322+0000 7fa3cbd66640 1 --2- 172.21.15.89:0/3524862568 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa3c41051b0 0x7fa3c410bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:51.325 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:33:51.322+0000 7fa3cbd66640 1 --2- 172.21.15.89:0/3524862568 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa3c4104810 0x7fa3c4104c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:51.325 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:33:51.322+0000 7fa3cbd66640 1 --2- 172.21.15.89:0/3524862568 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa3c4103780 0x7fa3c4103bf0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:51.325 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:33:51.322+0000 7fa3cbd66640 1 -- 172.21.15.89:0/3524862568 >> 172.21.15.89:0/3524862568 conn(0x7fa3c40ff0f0 msgr2=0x7fa3c41015a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:33:51.325 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:33:51.322+0000 7fa3cbd66640 1 -- 172.21.15.89:0/3524862568 shutdown_connections 2024-09-21T03:33:51.325 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:33:51.322+0000 7fa3cbd66640 1 -- 172.21.15.89:0/3524862568 wait complete. 2024-09-21T03:33:51.327 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:33:51.323+0000 7fa3cbd66640 1 Processor -- start 2024-09-21T03:33:51.327 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:33:51.323+0000 7fa3cbd66640 1 -- start start 2024-09-21T03:33:51.327 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:33:51.323+0000 7fa3cbd66640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa3c4103780 0x7fa3c419c0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:51.327 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:33:51.323+0000 7fa3cbd66640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa3c4104810 0x7fa3c419c760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:51.327 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:33:51.323+0000 7fa3cbd66640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa3c41051b0 0x7fa3c419d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:51.327 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:33:51.323+0000 7fa3cbd66640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fa3c410f250 con 0x7fa3c4103780 2024-09-21T03:33:51.327 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:33:51.323+0000 7fa3cbd66640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fa3c410f0d0 con 0x7fa3c4104810 2024-09-21T03:33:51.327 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:33:51.323+0000 7fa3cbd66640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fa3c410f3d0 con 0x7fa3c41051b0 2024-09-21T03:33:51.327 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:33:51.323+0000 7fa3c9b02640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa3c4103780 0x7fa3c419c0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:51.327 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:33:51.323+0000 7fa3c9b02640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa3c4103780 0x7fa3c419c0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:36502/0 (socket says 172.21.15.89:36502) 2024-09-21T03:33:51.327 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:33:51.323+0000 7fa3c9b02640 1 -- 172.21.15.89:0/3979760263 learned_addr learned my addr 172.21.15.89:0/3979760263 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-21T03:33:51.328 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:33:51.323+0000 7fa3c9301640 1 --2- 172.21.15.89:0/3979760263 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa3c4104810 0x7fa3c419c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:51.328 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:33:51.323+0000 7fa3ca303640 1 --2- 172.21.15.89:0/3979760263 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa3c41051b0 0x7fa3c419d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:51.328 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:33:51.324+0000 7fa3c9b02640 1 -- 172.21.15.89:0/3979760263 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa3c41051b0 msgr2=0x7fa3c419d300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:51.328 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:33:51.324+0000 7fa3c9b02640 1 --2- 172.21.15.89:0/3979760263 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa3c41051b0 0x7fa3c419d300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:51.328 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:33:51.324+0000 7fa3c9b02640 1 -- 172.21.15.89:0/3979760263 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa3c4104810 msgr2=0x7fa3c419c760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:51.328 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:33:51.324+0000 7fa3c9b02640 1 --2- 172.21.15.89:0/3979760263 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa3c4104810 0x7fa3c419c760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:51.328 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:33:51.324+0000 7fa3c9b02640 1 -- 172.21.15.89:0/3979760263 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3c419db20 con 0x7fa3c4103780 2024-09-21T03:33:51.328 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:33:51.324+0000 7fa3ca303640 1 --2- 172.21.15.89:0/3979760263 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa3c41051b0 0x7fa3c419d300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:33:51.328 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:33:51.324+0000 7fa3c9b02640 1 --2- 172.21.15.89:0/3979760263 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa3c4103780 0x7fa3c419c0f0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fa3c0009230 tx=0x7fa3c00042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:33:51.328 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:33:51.324+0000 7fa3b6ffd640 1 -- 172.21.15.89:0/3979760263 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3c003f030 con 0x7fa3c4103780 2024-09-21T03:33:51.328 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:33:51.324+0000 7fa3b6ffd640 1 -- 172.21.15.89:0/3979760263 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa3c0003c90 con 0x7fa3c4103780 2024-09-21T03:33:51.328 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:33:51.324+0000 7fa3cbd66640 1 -- 172.21.15.89:0/3979760263 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa3c419ddb0 con 0x7fa3c4103780 2024-09-21T03:33:51.329 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:33:51.324+0000 7fa3b6ffd640 1 -- 172.21.15.89:0/3979760263 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3c0035070 con 0x7fa3c4103780 2024-09-21T03:33:51.329 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:33:51.324+0000 7fa3cbd66640 1 -- 172.21.15.89:0/3979760263 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3c41a4510 con 0x7fa3c4103780 2024-09-21T03:33:51.329 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:33:51.325+0000 7fa3cbd66640 1 -- 172.21.15.89:0/3979760263 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa3c41a46e0 con 0x7fa3c4103780 2024-09-21T03:33:51.330 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:33:51.328+0000 7fa3b6ffd640 1 -- 172.21.15.89:0/3979760263 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 17) ==== 154512+0+0 (secure 0 0 0) 0x7fa3c0042030 con 0x7fa3c4103780 2024-09-21T03:33:51.330 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:33:51.328+0000 7fa3b6ffd640 1 --2- 172.21.15.89:0/3979760263 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7fa3b00b6590 0x7fa3b00b8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:51.330 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:33:51.328+0000 7fa3c9301640 1 --2- 172.21.15.89:0/3979760263 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7fa3b00b6590 0x7fa3b00b8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:51.331 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:33:51.329+0000 7fa3b6ffd640 1 -- 172.21.15.89:0/3979760263 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 4253+0+0 (secure 0 0 0) 0x7fa3c0106d80 con 0x7fa3c4103780 2024-09-21T03:33:51.331 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:33:51.329+0000 7fa3c9301640 1 --2- 172.21.15.89:0/3979760263 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7fa3b00b6590 0x7fa3b00b8aa0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fa3b800bbe0 tx=0x7fa3b800b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:33:51.333 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:33:51.331+0000 7fa3b6ffd640 1 -- 172.21.15.89:0/3979760263 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fa3c00ce550 con 0x7fa3c4103780 2024-09-21T03:33:51.430 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:33:50 smithi114 ceph-mon[34140]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-21T03:33:51.430 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:33:50 smithi114 ceph-mon[34140]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-09-21T03:33:51.430 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:33:50 smithi114 ceph-mon[34140]: from='osd.5 [v2:172.21.15.114:6808/1144140707,v1:172.21.15.114:6809/1144140707]' entity='osd.5' cmd={"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi114", "root=default"]} : dispatch 2024-09-21T03:33:51.430 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:33:50 smithi114 ceph-mon[34140]: osdmap e33: 6 total, 5 up, 6 in 2024-09-21T03:33:51.430 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:33:50 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-21T03:33:51.430 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:33:50 smithi114 ceph-mon[34140]: from='osd.5 ' entity='osd.5' cmd={"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi114", "root=default"]} : dispatch 2024-09-21T03:33:51.430 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:33:50 smithi114 ceph-mon[34140]: Detected new or changed devices on smithi114 2024-09-21T03:33:51.430 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:33:50 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:33:51.430 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:33:50 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:33:51.430 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:33:50 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-21T03:33:51.431 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:33:50 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-21T03:33:51.431 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:33:50 smithi114 ceph-mon[34140]: Adjusting osd_memory_target on smithi114 to 8544M 2024-09-21T03:33:51.431 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:33:50 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:33:51.431 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:33:50 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:33:51.431 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:33:50 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:33:51.431 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:33:50 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:33:51.531 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:33:51.529+0000 7fa3cbd66640 1 -- 172.21.15.89:0/3979760263 --> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi089:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7fa3c4076d10 con 0x7fa3b00b6590 2024-09-21T03:33:51.711 INFO:journalctl@ceph.osd.5.smithi114.stdout:Sep 21 03:33:51 smithi114 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-5[45121]: 2024-09-21T03:33:51.427+0000 7fa9553f6640 -1 osd.5 0 waiting for initial osdmap 2024-09-21T03:33:51.711 INFO:journalctl@ceph.osd.5.smithi114.stdout:Sep 21 03:33:51 smithi114 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-5[45121]: 2024-09-21T03:33:51.434+0000 7fa94c9d7640 -1 osd.5 34 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-21T03:33:52.165 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:33:51 smithi089 ceph-mon[30928]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi114", "root=default"]}]': finished 2024-09-21T03:33:52.166 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:33:51 smithi089 ceph-mon[30928]: osdmap e34: 6 total, 5 up, 6 in 2024-09-21T03:33:52.166 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:33:51 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-21T03:33:52.166 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:33:51 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-21T03:33:52.166 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:33:51 smithi089 ceph-mon[30928]: from='osd.5 ' entity='osd.5' 2024-09-21T03:33:52.166 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:33:51 smithi089 ceph-mon[30928]: from='client.14532 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi089:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:33:52.166 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:33:51 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T03:33:52.166 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:33:51 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T03:33:52.166 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:33:51 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:33:52.166 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:33:51 smithi089 ceph-mon[30928]: osd.5 [v2:172.21.15.114:6808/1144140707,v1:172.21.15.114:6809/1144140707] boot 2024-09-21T03:33:52.166 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:33:51 smithi089 ceph-mon[30928]: osdmap e35: 6 total, 6 up, 6 in 2024-09-21T03:33:52.166 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:33:51 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-21T03:33:52.250 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:33:51 smithi110 ceph-mon[34000]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi114", "root=default"]}]': finished 2024-09-21T03:33:52.250 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:33:51 smithi110 ceph-mon[34000]: osdmap e34: 6 total, 5 up, 6 in 2024-09-21T03:33:52.250 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:33:51 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-21T03:33:52.250 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:33:51 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-21T03:33:52.250 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:33:51 smithi110 ceph-mon[34000]: from='osd.5 ' entity='osd.5' 2024-09-21T03:33:52.250 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:33:51 smithi110 ceph-mon[34000]: from='client.14532 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi089:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:33:52.250 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:33:51 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T03:33:52.250 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:33:51 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T03:33:52.250 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:33:51 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:33:52.251 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:33:51 smithi110 ceph-mon[34000]: osd.5 [v2:172.21.15.114:6808/1144140707,v1:172.21.15.114:6809/1144140707] boot 2024-09-21T03:33:52.251 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:33:51 smithi110 ceph-mon[34000]: osdmap e35: 6 total, 6 up, 6 in 2024-09-21T03:33:52.251 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:33:51 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-21T03:33:52.461 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:33:51 smithi114 ceph-mon[34140]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi114", "root=default"]}]': finished 2024-09-21T03:33:52.461 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:33:51 smithi114 ceph-mon[34140]: osdmap e34: 6 total, 5 up, 6 in 2024-09-21T03:33:52.461 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:33:51 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-21T03:33:52.461 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:33:51 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-21T03:33:52.461 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:33:51 smithi114 ceph-mon[34140]: from='osd.5 ' entity='osd.5' 2024-09-21T03:33:52.461 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:33:51 smithi114 ceph-mon[34140]: from='client.14532 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi089:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:33:52.461 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:33:51 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T03:33:52.461 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:33:51 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T03:33:52.461 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:33:51 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:33:52.461 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:33:51 smithi114 ceph-mon[34140]: osd.5 [v2:172.21.15.114:6808/1144140707,v1:172.21.15.114:6809/1144140707] boot 2024-09-21T03:33:52.462 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:33:51 smithi114 ceph-mon[34140]: osdmap e35: 6 total, 6 up, 6 in 2024-09-21T03:33:52.462 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:33:51 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-21T03:33:53.250 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:33:52 smithi110 ceph-mon[34000]: purged_snaps scrub starts 2024-09-21T03:33:53.250 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:33:52 smithi110 ceph-mon[34000]: purged_snaps scrub ok 2024-09-21T03:33:53.250 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:33:52 smithi110 ceph-mon[34000]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-21T03:33:53.250 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:33:52 smithi110 ceph-mon[34000]: osdmap e36: 6 total, 6 up, 6 in 2024-09-21T03:33:53.311 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:33:52 smithi089 ceph-mon[30928]: purged_snaps scrub starts 2024-09-21T03:33:53.311 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:33:52 smithi089 ceph-mon[30928]: purged_snaps scrub ok 2024-09-21T03:33:53.311 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:33:52 smithi089 ceph-mon[30928]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-21T03:33:53.311 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:33:52 smithi089 ceph-mon[30928]: osdmap e36: 6 total, 6 up, 6 in 2024-09-21T03:33:53.461 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:33:52 smithi114 ceph-mon[34140]: purged_snaps scrub starts 2024-09-21T03:33:53.461 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:33:52 smithi114 ceph-mon[34140]: purged_snaps scrub ok 2024-09-21T03:33:53.461 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:33:52 smithi114 ceph-mon[34140]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-21T03:33:53.461 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:33:52 smithi114 ceph-mon[34140]: osdmap e36: 6 total, 6 up, 6 in 2024-09-21T03:33:54.961 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:33:54 smithi114 ceph-mon[34140]: pgmap v147: 1 pgs: 1 peering; 577 KiB data, 209 MiB used, 536 GiB / 536 GiB avail 2024-09-21T03:33:54.961 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:33:54 smithi114 ceph-mon[34140]: osdmap e37: 6 total, 6 up, 6 in 2024-09-21T03:33:54.961 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:33:54 smithi114 ceph-mon[34140]: from='client.? 172.21.15.89:0/3356185048' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "9b79989e-adb0-42b2-b2f6-a2dce56b66e7"} : dispatch 2024-09-21T03:33:54.961 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:33:54 smithi114 ceph-mon[34140]: from='client.? 172.21.15.89:0/3356185048' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9b79989e-adb0-42b2-b2f6-a2dce56b66e7"}]': finished 2024-09-21T03:33:54.961 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:33:54 smithi114 ceph-mon[34140]: osdmap e38: 7 total, 6 up, 7 in 2024-09-21T03:33:54.961 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:33:54 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-21T03:33:55.000 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:33:54 smithi110 ceph-mon[34000]: pgmap v147: 1 pgs: 1 peering; 577 KiB data, 209 MiB used, 536 GiB / 536 GiB avail 2024-09-21T03:33:55.000 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:33:54 smithi110 ceph-mon[34000]: osdmap e37: 6 total, 6 up, 6 in 2024-09-21T03:33:55.000 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:33:54 smithi110 ceph-mon[34000]: from='client.? 172.21.15.89:0/3356185048' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "9b79989e-adb0-42b2-b2f6-a2dce56b66e7"} : dispatch 2024-09-21T03:33:55.000 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:33:54 smithi110 ceph-mon[34000]: from='client.? 172.21.15.89:0/3356185048' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9b79989e-adb0-42b2-b2f6-a2dce56b66e7"}]': finished 2024-09-21T03:33:55.000 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:33:54 smithi110 ceph-mon[34000]: osdmap e38: 7 total, 6 up, 7 in 2024-09-21T03:33:55.000 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:33:54 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-21T03:33:55.061 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:33:54 smithi089 ceph-mon[30928]: pgmap v147: 1 pgs: 1 peering; 577 KiB data, 209 MiB used, 536 GiB / 536 GiB avail 2024-09-21T03:33:55.061 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:33:54 smithi089 ceph-mon[30928]: osdmap e37: 6 total, 6 up, 6 in 2024-09-21T03:33:55.061 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:33:54 smithi089 ceph-mon[30928]: from='client.? 172.21.15.89:0/3356185048' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "9b79989e-adb0-42b2-b2f6-a2dce56b66e7"} : dispatch 2024-09-21T03:33:55.061 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:33:54 smithi089 ceph-mon[30928]: from='client.? 172.21.15.89:0/3356185048' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9b79989e-adb0-42b2-b2f6-a2dce56b66e7"}]': finished 2024-09-21T03:33:55.061 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:33:54 smithi089 ceph-mon[30928]: osdmap e38: 7 total, 6 up, 7 in 2024-09-21T03:33:55.061 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:33:54 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-21T03:33:55.961 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:33:55 smithi114 ceph-mon[34140]: from='client.? 172.21.15.89:0/3438754476' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T03:33:56.000 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:33:55 smithi110 ceph-mon[34000]: from='client.? 172.21.15.89:0/3438754476' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T03:33:56.061 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:33:55 smithi089 ceph-mon[30928]: from='client.? 172.21.15.89:0/3438754476' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T03:33:56.961 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:33:56 smithi114 ceph-mon[34140]: pgmap v150: 1 pgs: 1 peering; 577 KiB data, 209 MiB used, 536 GiB / 536 GiB avail 2024-09-21T03:33:57.000 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:33:56 smithi110 ceph-mon[34000]: pgmap v150: 1 pgs: 1 peering; 577 KiB data, 209 MiB used, 536 GiB / 536 GiB avail 2024-09-21T03:33:57.061 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:33:56 smithi089 ceph-mon[30928]: pgmap v150: 1 pgs: 1 peering; 577 KiB data, 209 MiB used, 536 GiB / 536 GiB avail 2024-09-21T03:33:58.922 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:33:58 smithi089 ceph-mon[30928]: pgmap v151: 1 pgs: 1 peering; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-09-21T03:33:58.960 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:33:58 smithi114 ceph-mon[34140]: pgmap v151: 1 pgs: 1 peering; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-09-21T03:33:59.000 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:33:58 smithi110 ceph-mon[34000]: pgmap v151: 1 pgs: 1 peering; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-09-21T03:34:00.825 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:34:00 smithi089 ceph-mon[30928]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 83 KiB/s, 0 objects/s recovering 2024-09-21T03:34:00.961 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:34:00 smithi114 ceph-mon[34140]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 83 KiB/s, 0 objects/s recovering 2024-09-21T03:34:01.000 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:34:00 smithi110 ceph-mon[34000]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 83 KiB/s, 0 objects/s recovering 2024-09-21T03:34:02.892 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:34:02 smithi089 ceph-mon[30928]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-21T03:34:02.961 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:34:02 smithi114 ceph-mon[34140]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-21T03:34:03.000 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:34:02 smithi110 ceph-mon[34000]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-21T03:34:03.733 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:34:03 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.6"} : dispatch 2024-09-21T03:34:03.733 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:34:03 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:34:03.734 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:34:03 smithi089 ceph-mon[30928]: Deploying daemon osd.6 on smithi089 2024-09-21T03:34:03.862 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:34:03 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.6"} : dispatch 2024-09-21T03:34:03.862 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:34:03 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:34:03.862 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:34:03 smithi114 ceph-mon[34140]: Deploying daemon osd.6 on smithi089 2024-09-21T03:34:03.901 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:34:03 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.6"} : dispatch 2024-09-21T03:34:03.901 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:34:03 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:34:03.901 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:34:03 smithi110 ceph-mon[34000]: Deploying daemon osd.6 on smithi089 2024-09-21T03:34:04.912 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:34:04 smithi089 ceph-mon[30928]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-21T03:34:04.960 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:34:04 smithi114 ceph-mon[34140]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-21T03:34:05.000 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:34:04 smithi110 ceph-mon[34000]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-21T03:34:06.809 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:34:06 smithi089 ceph-mon[30928]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 50 KiB/s, 0 objects/s recovering 2024-09-21T03:34:06.960 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:34:06 smithi114 ceph-mon[34140]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 50 KiB/s, 0 objects/s recovering 2024-09-21T03:34:07.000 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:34:06 smithi110 ceph-mon[34000]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 50 KiB/s, 0 objects/s recovering 2024-09-21T03:34:08.811 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:34:08 smithi089 ceph-mon[30928]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-21T03:34:08.961 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:34:08 smithi114 ceph-mon[34140]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-21T03:34:09.000 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:34:08 smithi110 ceph-mon[34000]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-21T03:34:09.870 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:34:09 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:34:09.871 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:34:09 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:34:09.871 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:34:09 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:34:09.960 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:34:09 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:34:09.960 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:34:09 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:34:09.961 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:34:09 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:34:10.000 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:34:09 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:34:10.000 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:34:09 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:34:10.000 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:34:09 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:34:11.745 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:34:11 smithi089 ceph-mon[30928]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-21T03:34:11.750 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:34:11 smithi110 ceph-mon[34000]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-21T03:34:11.961 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:34:11 smithi114 ceph-mon[34140]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-21T03:34:13.311 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:34:13 smithi089 ceph-mon[30928]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-09-21T03:34:13.311 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:34:13 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:34:13.311 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:34:13 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:34:13.311 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:34:13 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:34:13.311 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:34:13 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:34:13.311 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:34:13 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:34:13.396 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:34:13.393+0000 7fa3b6ffd640 1 -- 172.21.15.89:0/3979760263 <== mgr.14152 v2:172.21.15.89:6800/1591532358 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fa3c4076d10 con 0x7fa3b00b6590 2024-09-21T03:34:13.396 INFO:teuthology.orchestra.run.smithi089.stdout:Created osd(s) 6 on host 'smithi089' 2024-09-21T03:34:13.399 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:34:13.397+0000 7fa3cbd66640 1 -- 172.21.15.89:0/3979760263 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7fa3b00b6590 msgr2=0x7fa3b00b8aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:13.399 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:34:13.397+0000 7fa3cbd66640 1 --2- 172.21.15.89:0/3979760263 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7fa3b00b6590 0x7fa3b00b8aa0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fa3b800bbe0 tx=0x7fa3b800b390 comp rx=0 tx=0).stop 2024-09-21T03:34:13.399 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:34:13.397+0000 7fa3cbd66640 1 -- 172.21.15.89:0/3979760263 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa3c4103780 msgr2=0x7fa3c419c0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:13.399 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:34:13.397+0000 7fa3cbd66640 1 --2- 172.21.15.89:0/3979760263 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa3c4103780 0x7fa3c419c0f0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fa3c0009230 tx=0x7fa3c00042f0 comp rx=0 tx=0).stop 2024-09-21T03:34:13.399 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:34:13.397+0000 7fa3cbd66640 1 -- 172.21.15.89:0/3979760263 shutdown_connections 2024-09-21T03:34:13.399 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:34:13.397+0000 7fa3cbd66640 1 --2- 172.21.15.89:0/3979760263 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7fa3b00b6590 0x7fa3b00b8aa0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:13.400 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:34:13.397+0000 7fa3cbd66640 1 --2- 172.21.15.89:0/3979760263 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa3c4104810 0x7fa3c419c760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:13.400 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:34:13.397+0000 7fa3cbd66640 1 --2- 172.21.15.89:0/3979760263 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa3c41051b0 0x7fa3c419d300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:13.400 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:34:13.397+0000 7fa3cbd66640 1 --2- 172.21.15.89:0/3979760263 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa3c4103780 0x7fa3c419c0f0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:13.400 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:34:13.397+0000 7fa3cbd66640 1 -- 172.21.15.89:0/3979760263 >> 172.21.15.89:0/3979760263 conn(0x7fa3c40ff0f0 msgr2=0x7fa3c40755f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:34:13.400 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:34:13.397+0000 7fa3cbd66640 1 -- 172.21.15.89:0/3979760263 shutdown_connections 2024-09-21T03:34:13.400 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:34:13.397+0000 7fa3cbd66640 1 -- 172.21.15.89:0/3979760263 wait complete. 2024-09-21T03:34:13.461 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:34:13 smithi114 ceph-mon[34140]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-09-21T03:34:13.461 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:34:13 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:34:13.461 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:34:13 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:34:13.461 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:34:13 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:34:13.461 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:34:13 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:34:13.461 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:34:13 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:34:13.500 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:34:13 smithi110 ceph-mon[34000]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-09-21T03:34:13.500 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:34:13 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:34:13.500 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:34:13 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:34:13.500 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:34:13 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:34:13.500 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:34:13 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:34:13.500 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:34:13 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:34:14.103 DEBUG:teuthology.orchestra.run.smithi089:osd.6> sudo journalctl -f -n 0 -u ceph-7c374098-77c9-11ef-baf5-efdc52797490@osd.6.service 2024-09-21T03:34:14.105 INFO:tasks.cephadm:Deploying osd.7 on smithi110 with /dev/vg_nvme/lv_2... 2024-09-21T03:34:14.106 DEBUG:teuthology.orchestra.run.smithi110:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7c374098-77c9-11ef-baf5-efdc52797490 -- lvm zap /dev/vg_nvme/lv_2 2024-09-21T03:34:14.362 INFO:teuthology.orchestra.run.smithi110.stderr:Inferring config /var/lib/ceph/7c374098-77c9-11ef-baf5-efdc52797490/mon.b/config 2024-09-21T03:34:14.450 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:34:14 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:34:14.450 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:34:14 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:34:14.450 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:34:14 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:34:14.460 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:34:14 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:34:14.461 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:34:14 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:34:14.461 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:34:14 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:34:14.482 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:34:14 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:34:14.482 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:34:14 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:34:14.482 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:34:14 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:34:15.460 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:34:15 smithi114 ceph-mon[34140]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-09-21T03:34:15.551 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:34:15 smithi089 ceph-mon[30928]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-09-21T03:34:15.665 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:34:15 smithi110 ceph-mon[34000]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-09-21T03:34:15.811 INFO:journalctl@ceph.osd.6.smithi089.stdout:Sep 21 03:34:15 smithi089 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-6[59191]: 2024-09-21T03:34:15.548+0000 7f845c0af640 -1 osd.6 0 log_to_monitors true 2024-09-21T03:34:16.500 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:34:16 smithi110 ceph-mon[34000]: from='osd.6 [v2:172.21.15.89:6818/1522586255,v1:172.21.15.89:6819/1522586255]' entity='osd.6' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]} : dispatch 2024-09-21T03:34:16.710 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:34:16 smithi114 ceph-mon[34140]: from='osd.6 [v2:172.21.15.89:6818/1522586255,v1:172.21.15.89:6819/1522586255]' entity='osd.6' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]} : dispatch 2024-09-21T03:34:16.811 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:34:16 smithi089 ceph-mon[30928]: from='osd.6 [v2:172.21.15.89:6818/1522586255,v1:172.21.15.89:6819/1522586255]' entity='osd.6' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]} : dispatch 2024-09-21T03:34:17.000 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-21T03:34:17.026 DEBUG:teuthology.orchestra.run.smithi110:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7c374098-77c9-11ef-baf5-efdc52797490 -- ceph orch daemon add osd smithi110:vg_nvme/lv_2 2024-09-21T03:34:17.275 INFO:teuthology.orchestra.run.smithi110.stderr:Inferring config /var/lib/ceph/7c374098-77c9-11ef-baf5-efdc52797490/mon.b/config 2024-09-21T03:34:17.684 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:34:17 smithi110 ceph-mon[34000]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-09-21T03:34:17.684 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:34:17 smithi110 ceph-mon[34000]: from='osd.6 [v2:172.21.15.89:6818/1522586255,v1:172.21.15.89:6819/1522586255]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-09-21T03:34:17.685 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:34:17 smithi110 ceph-mon[34000]: osdmap e39: 7 total, 6 up, 7 in 2024-09-21T03:34:17.685 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:34:17 smithi110 ceph-mon[34000]: from='osd.6 [v2:172.21.15.89:6818/1522586255,v1:172.21.15.89:6819/1522586255]' entity='osd.6' cmd={"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi089", "root=default"]} : dispatch 2024-09-21T03:34:17.685 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:34:17 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-21T03:34:17.685 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:34:17 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:34:17.685 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:34:17 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:34:17.685 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:34:17 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-21T03:34:17.685 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:34:17 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-21T03:34:17.685 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:34:17 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-09-21T03:34:17.685 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:34:17 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:34:17.686 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:34:17 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:34:17.686 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:34:17 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:34:17.686 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:34:17 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:34:17.696 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T03:34:17.692+0000 7ff5f7937640 1 -- 172.21.15.110:0/3396260222 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff5f0103710 msgr2=0x7ff5f0103b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:17.696 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T03:34:17.692+0000 7ff5f7937640 1 --2- 172.21.15.110:0/3396260222 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff5f0103710 0x7ff5f0103b20 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7ff5e0009900 tx=0x7ff5e002a480 comp rx=0 tx=0).stop 2024-09-21T03:34:17.696 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T03:34:17.692+0000 7ff5f7937640 1 -- 172.21.15.110:0/3396260222 shutdown_connections 2024-09-21T03:34:17.696 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T03:34:17.692+0000 7ff5f7937640 1 --2- 172.21.15.110:0/3396260222 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff5f0103710 0x7ff5f0103b20 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:17.696 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T03:34:17.692+0000 7ff5f7937640 1 --2- 172.21.15.110:0/3396260222 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff5f01040b0 0x7ff5f010aa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:17.696 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T03:34:17.692+0000 7ff5f7937640 1 --2- 172.21.15.110:0/3396260222 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff5f0102680 0x7ff5f0102af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:17.696 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T03:34:17.692+0000 7ff5f7937640 1 -- 172.21.15.110:0/3396260222 >> 172.21.15.110:0/3396260222 conn(0x7ff5f00fdff0 msgr2=0x7ff5f01004a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:34:17.696 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T03:34:17.692+0000 7ff5f7937640 1 -- 172.21.15.110:0/3396260222 shutdown_connections 2024-09-21T03:34:17.696 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T03:34:17.692+0000 7ff5f7937640 1 -- 172.21.15.110:0/3396260222 wait complete. 2024-09-21T03:34:17.697 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T03:34:17.692+0000 7ff5f7937640 1 Processor -- start 2024-09-21T03:34:17.697 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T03:34:17.693+0000 7ff5f7937640 1 -- start start 2024-09-21T03:34:17.697 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T03:34:17.693+0000 7ff5f7937640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff5f0102680 0x7ff5f0198e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:17.697 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T03:34:17.693+0000 7ff5f7937640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff5f0103710 0x7ff5f0199480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:17.697 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T03:34:17.693+0000 7ff5f7937640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff5f01040b0 0x7ff5f019a020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:17.697 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T03:34:17.693+0000 7ff5f7937640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7ff5f010e160 con 0x7ff5f0102680 2024-09-21T03:34:17.697 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T03:34:17.693+0000 7ff5f7937640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7ff5f010dfe0 con 0x7ff5f0103710 2024-09-21T03:34:17.697 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T03:34:17.693+0000 7ff5f7937640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7ff5f010e2e0 con 0x7ff5f01040b0 2024-09-21T03:34:17.697 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T03:34:17.693+0000 7ff5f4ed2640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff5f0103710 0x7ff5f0199480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:17.697 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T03:34:17.693+0000 7ff5f4ed2640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff5f0103710 0x7ff5f0199480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:41414/0 (socket says 172.21.15.110:41414) 2024-09-21T03:34:17.697 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T03:34:17.693+0000 7ff5f4ed2640 1 -- 172.21.15.110:0/601432403 learned_addr learned my addr 172.21.15.110:0/601432403 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-21T03:34:17.697 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T03:34:17.693+0000 7ff5f56d3640 1 --2- 172.21.15.110:0/601432403 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff5f0102680 0x7ff5f0198e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:17.698 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T03:34:17.693+0000 7ff5f5ed4640 1 --2- 172.21.15.110:0/601432403 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff5f01040b0 0x7ff5f019a020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:17.698 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T03:34:17.693+0000 7ff5f4ed2640 1 -- 172.21.15.110:0/601432403 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff5f01040b0 msgr2=0x7ff5f019a020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:17.698 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T03:34:17.693+0000 7ff5f4ed2640 1 --2- 172.21.15.110:0/601432403 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff5f01040b0 0x7ff5f019a020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:17.698 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T03:34:17.693+0000 7ff5f4ed2640 1 -- 172.21.15.110:0/601432403 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff5f0102680 msgr2=0x7ff5f0198e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:17.698 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T03:34:17.693+0000 7ff5f4ed2640 1 --2- 172.21.15.110:0/601432403 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff5f0102680 0x7ff5f0198e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:17.698 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T03:34:17.693+0000 7ff5f4ed2640 1 -- 172.21.15.110:0/601432403 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5f019a840 con 0x7ff5f0103710 2024-09-21T03:34:17.698 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T03:34:17.693+0000 7ff5f5ed4640 1 --2- 172.21.15.110:0/601432403 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff5f01040b0 0x7ff5f019a020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:34:17.698 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T03:34:17.693+0000 7ff5f4ed2640 1 --2- 172.21.15.110:0/601432403 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff5f0103710 0x7ff5f0199480 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7ff5e0009230 tx=0x7ff5e0004270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:34:17.698 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T03:34:17.694+0000 7ff5e67fc640 1 -- 172.21.15.110:0/601432403 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5e0046070 con 0x7ff5f0103710 2024-09-21T03:34:17.698 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T03:34:17.694+0000 7ff5e67fc640 1 -- 172.21.15.110:0/601432403 <== mon.2 v2:172.21.15.110:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff5e0003a80 con 0x7ff5f0103710 2024-09-21T03:34:17.698 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T03:34:17.694+0000 7ff5f7937640 1 -- 172.21.15.110:0/601432403 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff5f019aad0 con 0x7ff5f0103710 2024-09-21T03:34:17.699 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T03:34:17.694+0000 7ff5e67fc640 1 -- 172.21.15.110:0/601432403 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5e0035050 con 0x7ff5f0103710 2024-09-21T03:34:17.699 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T03:34:17.694+0000 7ff5f7937640 1 -- 172.21.15.110:0/601432403 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5f006bb80 con 0x7ff5f0103710 2024-09-21T03:34:17.700 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T03:34:17.695+0000 7ff5f7937640 1 -- 172.21.15.110:0/601432403 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff5f006c5a0 con 0x7ff5f0103710 2024-09-21T03:34:17.700 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T03:34:17.696+0000 7ff5e67fc640 1 -- 172.21.15.110:0/601432403 <== mon.2 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 17) ==== 154512+0+0 (secure 0 0 0) 0x7ff5e0049070 con 0x7ff5f0103710 2024-09-21T03:34:17.700 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T03:34:17.697+0000 7ff5e67fc640 1 --2- 172.21.15.110:0/601432403 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7ff5dc0b6510 0x7ff5dc0b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:17.700 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T03:34:17.697+0000 7ff5e67fc640 1 -- 172.21.15.110:0/601432403 <== mon.2 v2:172.21.15.110:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 4685+0+0 (secure 0 0 0) 0x7ff5e0108720 con 0x7ff5f0103710 2024-09-21T03:34:17.700 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T03:34:17.697+0000 7ff5f56d3640 1 --2- 172.21.15.110:0/601432403 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7ff5dc0b6510 0x7ff5dc0b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:17.700 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T03:34:17.698+0000 7ff5f56d3640 1 --2- 172.21.15.110:0/601432403 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7ff5dc0b6510 0x7ff5dc0b8a20 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7ff5e8005c40 tx=0x7ff5e800a590 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:34:17.702 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T03:34:17.700+0000 7ff5e67fc640 1 -- 172.21.15.110:0/601432403 <== mon.2 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7ff5e0050020 con 0x7ff5f0103710 2024-09-21T03:34:17.711 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:34:17 smithi114 ceph-mon[34140]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-09-21T03:34:17.711 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:34:17 smithi114 ceph-mon[34140]: from='osd.6 [v2:172.21.15.89:6818/1522586255,v1:172.21.15.89:6819/1522586255]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-09-21T03:34:17.711 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:34:17 smithi114 ceph-mon[34140]: osdmap e39: 7 total, 6 up, 7 in 2024-09-21T03:34:17.711 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:34:17 smithi114 ceph-mon[34140]: from='osd.6 [v2:172.21.15.89:6818/1522586255,v1:172.21.15.89:6819/1522586255]' entity='osd.6' cmd={"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi089", "root=default"]} : dispatch 2024-09-21T03:34:17.711 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:34:17 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-21T03:34:17.711 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:34:17 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:34:17.711 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:34:17 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:34:17.711 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:34:17 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-21T03:34:17.711 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:34:17 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-21T03:34:17.711 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:34:17 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-09-21T03:34:17.712 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:34:17 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:34:17.712 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:34:17 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:34:17.712 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:34:17 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:34:17.712 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:34:17 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:34:17.811 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:34:17 smithi089 ceph-mon[30928]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-09-21T03:34:17.811 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:34:17 smithi089 ceph-mon[30928]: from='osd.6 [v2:172.21.15.89:6818/1522586255,v1:172.21.15.89:6819/1522586255]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-09-21T03:34:17.811 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:34:17 smithi089 ceph-mon[30928]: osdmap e39: 7 total, 6 up, 7 in 2024-09-21T03:34:17.811 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:34:17 smithi089 ceph-mon[30928]: from='osd.6 [v2:172.21.15.89:6818/1522586255,v1:172.21.15.89:6819/1522586255]' entity='osd.6' cmd={"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi089", "root=default"]} : dispatch 2024-09-21T03:34:17.811 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:34:17 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-21T03:34:17.811 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:34:17 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:34:17.812 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:34:17 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:34:17.812 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:34:17 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-21T03:34:17.812 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:34:17 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-21T03:34:17.812 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:34:17 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-09-21T03:34:17.812 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:34:17 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:34:17.812 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:34:17 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:34:17.812 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:34:17 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:34:17.812 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:34:17 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:34:17.887 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T03:34:17.885+0000 7ff5f7937640 1 -- 172.21.15.110:0/601432403 --> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi110:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7ff5f0109350 con 0x7ff5dc0b6510 2024-09-21T03:34:18.316 INFO:journalctl@ceph.osd.6.smithi089.stdout:Sep 21 03:34:17 smithi089 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-6[59191]: 2024-09-21T03:34:17.811+0000 7f8458031640 -1 osd.6 0 waiting for initial osdmap 2024-09-21T03:34:18.316 INFO:journalctl@ceph.osd.6.smithi089.stdout:Sep 21 03:34:17 smithi089 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-6[59191]: 2024-09-21T03:34:17.819+0000 7f844fe27640 -1 osd.6 40 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-21T03:34:18.553 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:34:18 smithi110 ceph-mon[34000]: Detected new or changed devices on smithi089 2024-09-21T03:34:18.555 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:34:18 smithi110 ceph-mon[34000]: Adjusting osd_memory_target on smithi089 to 5695M 2024-09-21T03:34:18.555 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:34:18 smithi110 ceph-mon[34000]: from='osd.6 [v2:172.21.15.89:6818/1522586255,v1:172.21.15.89:6819/1522586255]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi089", "root=default"]}]': finished 2024-09-21T03:34:18.556 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:34:18 smithi110 ceph-mon[34000]: osdmap e40: 7 total, 6 up, 7 in 2024-09-21T03:34:18.556 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:34:18 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-21T03:34:18.556 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:34:18 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-21T03:34:18.556 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:34:18 smithi110 ceph-mon[34000]: from='osd.6 [v2:172.21.15.89:6818/1522586255,v1:172.21.15.89:6819/1522586255]' entity='osd.6' 2024-09-21T03:34:18.556 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:34:18 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T03:34:18.556 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:34:18 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T03:34:18.556 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:34:18 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:34:18.579 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:34:18 smithi089 ceph-mon[30928]: Detected new or changed devices on smithi089 2024-09-21T03:34:18.579 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:34:18 smithi089 ceph-mon[30928]: Adjusting osd_memory_target on smithi089 to 5695M 2024-09-21T03:34:18.579 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:34:18 smithi089 ceph-mon[30928]: from='osd.6 [v2:172.21.15.89:6818/1522586255,v1:172.21.15.89:6819/1522586255]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi089", "root=default"]}]': finished 2024-09-21T03:34:18.579 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:34:18 smithi089 ceph-mon[30928]: osdmap e40: 7 total, 6 up, 7 in 2024-09-21T03:34:18.579 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:34:18 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-21T03:34:18.579 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:34:18 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-21T03:34:18.579 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:34:18 smithi089 ceph-mon[30928]: from='osd.6 [v2:172.21.15.89:6818/1522586255,v1:172.21.15.89:6819/1522586255]' entity='osd.6' 2024-09-21T03:34:18.580 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:34:18 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T03:34:18.580 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:34:18 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T03:34:18.580 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:34:18 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:34:18.711 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:34:18 smithi114 ceph-mon[34140]: Detected new or changed devices on smithi089 2024-09-21T03:34:18.711 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:34:18 smithi114 ceph-mon[34140]: Adjusting osd_memory_target on smithi089 to 5695M 2024-09-21T03:34:18.711 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:34:18 smithi114 ceph-mon[34140]: from='osd.6 [v2:172.21.15.89:6818/1522586255,v1:172.21.15.89:6819/1522586255]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi089", "root=default"]}]': finished 2024-09-21T03:34:18.712 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:34:18 smithi114 ceph-mon[34140]: osdmap e40: 7 total, 6 up, 7 in 2024-09-21T03:34:18.712 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:34:18 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-21T03:34:18.712 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:34:18 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-21T03:34:18.712 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:34:18 smithi114 ceph-mon[34140]: from='osd.6 [v2:172.21.15.89:6818/1522586255,v1:172.21.15.89:6819/1522586255]' entity='osd.6' 2024-09-21T03:34:18.712 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:34:18 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T03:34:18.712 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:34:18 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T03:34:18.713 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:34:18 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:34:19.711 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:34:19 smithi114 ceph-mon[34140]: purged_snaps scrub starts 2024-09-21T03:34:19.711 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:34:19 smithi114 ceph-mon[34140]: purged_snaps scrub ok 2024-09-21T03:34:19.711 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:34:19 smithi114 ceph-mon[34140]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-09-21T03:34:19.711 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:34:19 smithi114 ceph-mon[34140]: from='client.24428 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi110:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:34:19.711 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:34:19 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-21T03:34:19.711 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:34:19 smithi114 ceph-mon[34140]: osd.6 [v2:172.21.15.89:6818/1522586255,v1:172.21.15.89:6819/1522586255] boot 2024-09-21T03:34:19.711 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:34:19 smithi114 ceph-mon[34140]: osdmap e41: 7 total, 7 up, 7 in 2024-09-21T03:34:19.711 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:34:19 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-21T03:34:19.749 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:34:19 smithi110 ceph-mon[34000]: purged_snaps scrub starts 2024-09-21T03:34:19.749 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:34:19 smithi110 ceph-mon[34000]: purged_snaps scrub ok 2024-09-21T03:34:19.749 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:34:19 smithi110 ceph-mon[34000]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-09-21T03:34:19.749 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:34:19 smithi110 ceph-mon[34000]: from='client.24428 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi110:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:34:19.749 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:34:19 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-21T03:34:19.749 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:34:19 smithi110 ceph-mon[34000]: osd.6 [v2:172.21.15.89:6818/1522586255,v1:172.21.15.89:6819/1522586255] boot 2024-09-21T03:34:19.749 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:34:19 smithi110 ceph-mon[34000]: osdmap e41: 7 total, 7 up, 7 in 2024-09-21T03:34:19.749 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:34:19 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-21T03:34:19.811 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:34:19 smithi089 ceph-mon[30928]: purged_snaps scrub starts 2024-09-21T03:34:19.811 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:34:19 smithi089 ceph-mon[30928]: purged_snaps scrub ok 2024-09-21T03:34:19.811 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:34:19 smithi089 ceph-mon[30928]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-09-21T03:34:19.811 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:34:19 smithi089 ceph-mon[30928]: from='client.24428 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi110:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:34:19.811 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:34:19 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-21T03:34:19.811 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:34:19 smithi089 ceph-mon[30928]: osd.6 [v2:172.21.15.89:6818/1522586255,v1:172.21.15.89:6819/1522586255] boot 2024-09-21T03:34:19.811 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:34:19 smithi089 ceph-mon[30928]: osdmap e41: 7 total, 7 up, 7 in 2024-09-21T03:34:19.811 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:34:19 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-21T03:34:20.711 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:34:20 smithi114 ceph-mon[34140]: from='client.? 172.21.15.110:0/1428248550' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "b381821f-141c-47a1-be65-f57ca730391b"} : dispatch 2024-09-21T03:34:20.711 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:34:20 smithi114 ceph-mon[34140]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "b381821f-141c-47a1-be65-f57ca730391b"} : dispatch 2024-09-21T03:34:20.711 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:34:20 smithi114 ceph-mon[34140]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b381821f-141c-47a1-be65-f57ca730391b"}]': finished 2024-09-21T03:34:20.711 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:34:20 smithi114 ceph-mon[34140]: osdmap e42: 8 total, 7 up, 8 in 2024-09-21T03:34:20.711 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:34:20 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-21T03:34:20.732 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:34:20 smithi110 ceph-mon[34000]: from='client.? 172.21.15.110:0/1428248550' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "b381821f-141c-47a1-be65-f57ca730391b"} : dispatch 2024-09-21T03:34:20.732 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:34:20 smithi110 ceph-mon[34000]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "b381821f-141c-47a1-be65-f57ca730391b"} : dispatch 2024-09-21T03:34:20.732 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:34:20 smithi110 ceph-mon[34000]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b381821f-141c-47a1-be65-f57ca730391b"}]': finished 2024-09-21T03:34:20.732 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:34:20 smithi110 ceph-mon[34000]: osdmap e42: 8 total, 7 up, 8 in 2024-09-21T03:34:20.732 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:34:20 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-21T03:34:20.811 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:34:20 smithi089 ceph-mon[30928]: from='client.? 172.21.15.110:0/1428248550' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "b381821f-141c-47a1-be65-f57ca730391b"} : dispatch 2024-09-21T03:34:20.811 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:34:20 smithi089 ceph-mon[30928]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "b381821f-141c-47a1-be65-f57ca730391b"} : dispatch 2024-09-21T03:34:20.811 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:34:20 smithi089 ceph-mon[30928]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b381821f-141c-47a1-be65-f57ca730391b"}]': finished 2024-09-21T03:34:20.811 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:34:20 smithi089 ceph-mon[30928]: osdmap e42: 8 total, 7 up, 8 in 2024-09-21T03:34:20.811 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:34:20 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-21T03:34:21.710 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:34:21 smithi114 ceph-mon[34140]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-09-21T03:34:21.711 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:34:21 smithi114 ceph-mon[34140]: from='client.? 172.21.15.110:0/818688698' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T03:34:21.750 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:34:21 smithi110 ceph-mon[34000]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-09-21T03:34:21.750 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:34:21 smithi110 ceph-mon[34000]: from='client.? 172.21.15.110:0/818688698' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T03:34:21.811 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:34:21 smithi089 ceph-mon[30928]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-09-21T03:34:21.811 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:34:21 smithi089 ceph-mon[30928]: from='client.? 172.21.15.110:0/818688698' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T03:34:23.711 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:34:23 smithi114 ceph-mon[34140]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-09-21T03:34:23.750 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:34:23 smithi110 ceph-mon[34000]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-09-21T03:34:23.811 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:34:23 smithi089 ceph-mon[30928]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-09-21T03:34:25.667 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:34:25 smithi110 ceph-mon[34000]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-09-21T03:34:25.710 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:34:25 smithi114 ceph-mon[34140]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-09-21T03:34:25.811 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:34:25 smithi089 ceph-mon[30928]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-09-21T03:34:27.623 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:34:27 smithi110 ceph-mon[34000]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-09-21T03:34:27.711 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:34:27 smithi114 ceph-mon[34140]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-09-21T03:34:27.811 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:34:27 smithi089 ceph-mon[30928]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-09-21T03:34:28.750 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:34:28 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.7"} : dispatch 2024-09-21T03:34:28.750 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:34:28 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:34:28.811 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:34:28 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.7"} : dispatch 2024-09-21T03:34:28.811 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:34:28 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:34:28.961 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:34:28 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.7"} : dispatch 2024-09-21T03:34:28.961 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:34:28 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:34:29.750 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:34:29 smithi110 ceph-mon[34000]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-09-21T03:34:29.750 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:34:29 smithi110 ceph-mon[34000]: Deploying daemon osd.7 on smithi110 2024-09-21T03:34:29.811 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:34:29 smithi089 ceph-mon[30928]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-09-21T03:34:29.811 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:34:29 smithi089 ceph-mon[30928]: Deploying daemon osd.7 on smithi110 2024-09-21T03:34:29.960 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:34:29 smithi114 ceph-mon[34140]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-09-21T03:34:29.961 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:34:29 smithi114 ceph-mon[34140]: Deploying daemon osd.7 on smithi110 2024-09-21T03:34:31.750 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:34:31 smithi110 ceph-mon[34000]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-09-21T03:34:31.811 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:34:31 smithi089 ceph-mon[30928]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-09-21T03:34:31.961 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:34:31 smithi114 ceph-mon[34140]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-09-21T03:34:33.750 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:34:33 smithi110 ceph-mon[34000]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-09-21T03:34:33.763 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:34:33 smithi114 ceph-mon[34140]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-09-21T03:34:33.811 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:34:33 smithi089 ceph-mon[30928]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-09-21T03:34:34.538 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:34:34 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:34:34.538 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:34:34 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:34:34.538 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:34:34 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:34:34.811 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:34:34 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:34:34.811 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:34:34 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:34:34.811 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:34:34 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:34:34.960 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:34:34 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:34:34.960 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:34:34 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:34:34.961 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:34:34 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:34:35.750 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:34:35 smithi110 ceph-mon[34000]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-09-21T03:34:35.811 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:34:35 smithi089 ceph-mon[30928]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-09-21T03:34:35.960 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:34:35 smithi114 ceph-mon[34140]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-09-21T03:34:37.811 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:34:37 smithi089 ceph-mon[30928]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-09-21T03:34:37.811 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:34:37 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:34:37.811 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:34:37 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:34:37.811 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:34:37 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:34:37.811 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:34:37 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:34:37.811 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:34:37 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:34:37.848 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:34:37 smithi110 ceph-mon[34000]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-09-21T03:34:37.848 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:34:37 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:34:37.848 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:34:37 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:34:37.848 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:34:37 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:34:37.848 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:34:37 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:34:37.848 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:34:37 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:34:37.960 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:34:37 smithi114 ceph-mon[34140]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-09-21T03:34:37.961 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:34:37 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:34:37.961 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:34:37 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:34:37.961 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:34:37 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:34:37.961 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:34:37 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:34:37.961 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:34:37 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:34:38.158 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T03:34:38.156+0000 7ff5e67fc640 1 -- 172.21.15.110:0/601432403 <== mgr.14152 v2:172.21.15.89:6800/1591532358 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7ff5f0109350 con 0x7ff5dc0b6510 2024-09-21T03:34:38.159 INFO:teuthology.orchestra.run.smithi110.stdout:Created osd(s) 7 on host 'smithi110' 2024-09-21T03:34:38.161 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T03:34:38.159+0000 7ff5f7937640 1 -- 172.21.15.110:0/601432403 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7ff5dc0b6510 msgr2=0x7ff5dc0b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:38.161 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T03:34:38.159+0000 7ff5f7937640 1 --2- 172.21.15.110:0/601432403 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7ff5dc0b6510 0x7ff5dc0b8a20 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7ff5e8005c40 tx=0x7ff5e800a590 comp rx=0 tx=0).stop 2024-09-21T03:34:38.161 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T03:34:38.159+0000 7ff5f7937640 1 -- 172.21.15.110:0/601432403 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff5f0103710 msgr2=0x7ff5f0199480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:38.161 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T03:34:38.159+0000 7ff5f7937640 1 --2- 172.21.15.110:0/601432403 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff5f0103710 0x7ff5f0199480 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7ff5e0009230 tx=0x7ff5e0004270 comp rx=0 tx=0).stop 2024-09-21T03:34:38.161 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T03:34:38.159+0000 7ff5f7937640 1 -- 172.21.15.110:0/601432403 shutdown_connections 2024-09-21T03:34:38.162 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T03:34:38.159+0000 7ff5f7937640 1 --2- 172.21.15.110:0/601432403 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7ff5dc0b6510 0x7ff5dc0b8a20 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:38.162 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T03:34:38.159+0000 7ff5f7937640 1 --2- 172.21.15.110:0/601432403 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff5f0103710 0x7ff5f0199480 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:38.162 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T03:34:38.159+0000 7ff5f7937640 1 --2- 172.21.15.110:0/601432403 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff5f01040b0 0x7ff5f019a020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:38.162 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T03:34:38.159+0000 7ff5f7937640 1 --2- 172.21.15.110:0/601432403 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff5f0102680 0x7ff5f0198e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:38.162 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T03:34:38.159+0000 7ff5f7937640 1 -- 172.21.15.110:0/601432403 >> 172.21.15.110:0/601432403 conn(0x7ff5f00fdff0 msgr2=0x7ff5f0106a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:34:38.162 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T03:34:38.159+0000 7ff5f7937640 1 -- 172.21.15.110:0/601432403 shutdown_connections 2024-09-21T03:34:38.162 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T03:34:38.159+0000 7ff5f7937640 1 -- 172.21.15.110:0/601432403 wait complete. 2024-09-21T03:34:38.710 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:34:38 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:34:38.710 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:34:38 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:34:38.710 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:34:38 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:34:38.711 DEBUG:teuthology.orchestra.run.smithi110:osd.7> sudo journalctl -f -n 0 -u ceph-7c374098-77c9-11ef-baf5-efdc52797490@osd.7.service 2024-09-21T03:34:38.713 INFO:tasks.cephadm:Deploying osd.8 on smithi114 with /dev/vg_nvme/lv_2... 2024-09-21T03:34:38.714 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7c374098-77c9-11ef-baf5-efdc52797490 -- lvm zap /dev/vg_nvme/lv_2 2024-09-21T03:34:38.811 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:34:38 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:34:38.811 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:34:38 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:34:38.811 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:34:38 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:34:38.960 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:34:38 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:34:38.961 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:34:38 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:34:38.961 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:34:38 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:34:38.968 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/7c374098-77c9-11ef-baf5-efdc52797490/mon.c/config 2024-09-21T03:34:39.711 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:34:39 smithi114 ceph-mon[34140]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-09-21T03:34:39.750 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:34:39 smithi110 ceph-mon[34000]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-09-21T03:34:39.811 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:34:39 smithi089 ceph-mon[30928]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-09-21T03:34:40.711 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:34:40 smithi114 ceph-mon[34140]: from='osd.7 [v2:172.21.15.110:6816/1331818323,v1:172.21.15.110:6817/1331818323]' entity='osd.7' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]} : dispatch 2024-09-21T03:34:40.711 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:34:40 smithi114 ceph-mon[34140]: from='osd.7 ' entity='osd.7' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]} : dispatch 2024-09-21T03:34:40.789 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:34:40 smithi110 ceph-mon[34000]: from='osd.7 [v2:172.21.15.110:6816/1331818323,v1:172.21.15.110:6817/1331818323]' entity='osd.7' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]} : dispatch 2024-09-21T03:34:40.789 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:34:40 smithi110 ceph-mon[34000]: from='osd.7 ' entity='osd.7' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]} : dispatch 2024-09-21T03:34:40.789 INFO:journalctl@ceph.osd.7.smithi110.stdout:Sep 21 03:34:40 smithi110 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-7[50481]: 2024-09-21T03:34:40.498+0000 7fb9fcdca640 -1 osd.7 0 log_to_monitors true 2024-09-21T03:34:40.811 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:34:40 smithi089 ceph-mon[30928]: from='osd.7 [v2:172.21.15.110:6816/1331818323,v1:172.21.15.110:6817/1331818323]' entity='osd.7' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]} : dispatch 2024-09-21T03:34:40.811 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:34:40 smithi089 ceph-mon[30928]: from='osd.7 ' entity='osd.7' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]} : dispatch 2024-09-21T03:34:41.485 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-21T03:34:41.521 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7c374098-77c9-11ef-baf5-efdc52797490 -- ceph orch daemon add osd smithi114:vg_nvme/lv_2 2024-09-21T03:34:41.750 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:34:41 smithi110 ceph-mon[34000]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-09-21T03:34:41.750 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:34:41 smithi110 ceph-mon[34000]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-09-21T03:34:41.750 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:34:41 smithi110 ceph-mon[34000]: from='osd.7 [v2:172.21.15.110:6816/1331818323,v1:172.21.15.110:6817/1331818323]' entity='osd.7' cmd={"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi110", "root=default"]} : dispatch 2024-09-21T03:34:41.750 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:34:41 smithi110 ceph-mon[34000]: osdmap e43: 8 total, 7 up, 8 in 2024-09-21T03:34:41.750 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:34:41 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-21T03:34:41.750 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:34:41 smithi110 ceph-mon[34000]: from='osd.7 ' entity='osd.7' cmd={"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi110", "root=default"]} : dispatch 2024-09-21T03:34:41.750 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:34:41 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:34:41.750 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:34:41 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:34:41.751 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:34:41 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-21T03:34:41.751 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:34:41 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-21T03:34:41.751 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:34:41 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-09-21T03:34:41.751 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:34:41 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:34:41.751 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:34:41 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:34:41.751 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:34:41 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:34:41.751 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:34:41 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:34:41.773 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/7c374098-77c9-11ef-baf5-efdc52797490/mon.c/config 2024-09-21T03:34:41.811 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:34:41 smithi089 ceph-mon[30928]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-09-21T03:34:41.811 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:34:41 smithi089 ceph-mon[30928]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-09-21T03:34:41.811 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:34:41 smithi089 ceph-mon[30928]: from='osd.7 [v2:172.21.15.110:6816/1331818323,v1:172.21.15.110:6817/1331818323]' entity='osd.7' cmd={"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi110", "root=default"]} : dispatch 2024-09-21T03:34:41.811 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:34:41 smithi089 ceph-mon[30928]: osdmap e43: 8 total, 7 up, 8 in 2024-09-21T03:34:41.811 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:34:41 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-21T03:34:41.811 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:34:41 smithi089 ceph-mon[30928]: from='osd.7 ' entity='osd.7' cmd={"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi110", "root=default"]} : dispatch 2024-09-21T03:34:41.812 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:34:41 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:34:41.812 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:34:41 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:34:41.812 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:34:41 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-21T03:34:41.812 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:34:41 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-21T03:34:41.812 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:34:41 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-09-21T03:34:41.812 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:34:41 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:34:41.812 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:34:41 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:34:41.812 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:34:41 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:34:41.812 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:34:41 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:34:41.894 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:34:41 smithi114 ceph-mon[34140]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-09-21T03:34:41.895 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:34:41 smithi114 ceph-mon[34140]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-09-21T03:34:41.895 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:34:41 smithi114 ceph-mon[34140]: from='osd.7 [v2:172.21.15.110:6816/1331818323,v1:172.21.15.110:6817/1331818323]' entity='osd.7' cmd={"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi110", "root=default"]} : dispatch 2024-09-21T03:34:41.895 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:34:41 smithi114 ceph-mon[34140]: osdmap e43: 8 total, 7 up, 8 in 2024-09-21T03:34:41.895 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:34:41 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-21T03:34:41.895 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:34:41 smithi114 ceph-mon[34140]: from='osd.7 ' entity='osd.7' cmd={"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi110", "root=default"]} : dispatch 2024-09-21T03:34:41.895 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:34:41 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:34:41.895 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:34:41 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:34:41.895 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:34:41 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-21T03:34:41.895 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:34:41 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-21T03:34:41.895 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:34:41 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-09-21T03:34:41.895 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:34:41 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:34:41.895 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:34:41 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:34:41.896 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:34:41 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:34:41.896 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:34:41 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:34:42.250 INFO:journalctl@ceph.osd.7.smithi110.stdout:Sep 21 03:34:41 smithi110 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-7[50481]: 2024-09-21T03:34:41.978+0000 7fb9f8d4c640 -1 osd.7 0 waiting for initial osdmap 2024-09-21T03:34:42.250 INFO:journalctl@ceph.osd.7.smithi110.stdout:Sep 21 03:34:41 smithi110 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-7[50481]: 2024-09-21T03:34:41.989+0000 7fb9f0b42640 -1 osd.7 45 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-21T03:34:42.260 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T03:34:42.258+0000 7f932e5e8640 1 -- 172.21.15.114:0/1838186914 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9328105180 msgr2=0x7f932810bb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:42.260 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T03:34:42.258+0000 7f932e5e8640 1 --2- 172.21.15.114:0/1838186914 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9328105180 0x7f932810bb20 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f9320009ca0 tx=0x7f932002a950 comp rx=0 tx=0).stop 2024-09-21T03:34:42.260 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T03:34:42.258+0000 7f932e5e8640 1 -- 172.21.15.114:0/1838186914 shutdown_connections 2024-09-21T03:34:42.260 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T03:34:42.258+0000 7f932e5e8640 1 --2- 172.21.15.114:0/1838186914 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f93281047e0 0x7f9328104bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:42.261 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T03:34:42.258+0000 7f932e5e8640 1 --2- 172.21.15.114:0/1838186914 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9328105180 0x7f932810bb20 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:42.261 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T03:34:42.258+0000 7f932e5e8640 1 --2- 172.21.15.114:0/1838186914 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9328103750 0x7f9328103bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:42.261 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T03:34:42.258+0000 7f932e5e8640 1 -- 172.21.15.114:0/1838186914 >> 172.21.15.114:0/1838186914 conn(0x7f93280ff0c0 msgr2=0x7f9328101570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:34:42.261 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T03:34:42.258+0000 7f932e5e8640 1 -- 172.21.15.114:0/1838186914 shutdown_connections 2024-09-21T03:34:42.261 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T03:34:42.258+0000 7f932e5e8640 1 -- 172.21.15.114:0/1838186914 wait complete. 2024-09-21T03:34:42.261 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T03:34:42.259+0000 7f932e5e8640 1 Processor -- start 2024-09-21T03:34:42.261 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T03:34:42.259+0000 7f932e5e8640 1 -- start start 2024-09-21T03:34:42.266 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T03:34:42.259+0000 7f932e5e8640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9328103750 0x7f9328199ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:42.266 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T03:34:42.259+0000 7f932e5e8640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f93281047e0 0x7f932819a540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:42.267 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T03:34:42.259+0000 7f932e5e8640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9328105180 0x7f932819b0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:42.267 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T03:34:42.259+0000 7f932e5e8640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f932810f2c0 con 0x7f93281047e0 2024-09-21T03:34:42.267 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T03:34:42.259+0000 7f932e5e8640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f932810f140 con 0x7f9328103750 2024-09-21T03:34:42.267 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T03:34:42.259+0000 7f932e5e8640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f932810f440 con 0x7f9328105180 2024-09-21T03:34:42.268 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T03:34:42.259+0000 7f932cb85640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9328105180 0x7f932819b0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:42.268 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T03:34:42.259+0000 7f932cb85640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9328105180 0x7f932819b0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:60196/0 (socket says 172.21.15.114:60196) 2024-09-21T03:34:42.268 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T03:34:42.259+0000 7f9327fff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9328103750 0x7f9328199ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:42.268 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T03:34:42.259+0000 7f932cb85640 1 -- 172.21.15.114:0/2353653452 learned_addr learned my addr 172.21.15.114:0/2353653452 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T03:34:42.269 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T03:34:42.259+0000 7f93277fe640 1 --2- 172.21.15.114:0/2353653452 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f93281047e0 0x7f932819a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:42.269 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T03:34:42.260+0000 7f932cb85640 1 -- 172.21.15.114:0/2353653452 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9328103750 msgr2=0x7f9328199ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:42.269 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T03:34:42.260+0000 7f932cb85640 1 --2- 172.21.15.114:0/2353653452 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9328103750 0x7f9328199ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:42.269 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T03:34:42.260+0000 7f932cb85640 1 -- 172.21.15.114:0/2353653452 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f93281047e0 msgr2=0x7f932819a540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:42.269 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T03:34:42.260+0000 7f932cb85640 1 --2- 172.21.15.114:0/2353653452 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f93281047e0 0x7f932819a540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:42.270 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T03:34:42.260+0000 7f932cb85640 1 -- 172.21.15.114:0/2353653452 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f932819b900 con 0x7f9328105180 2024-09-21T03:34:42.270 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T03:34:42.260+0000 7f932cb85640 1 --2- 172.21.15.114:0/2353653452 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9328105180 0x7f932819b0e0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f932002f800 tx=0x7f9320003f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:34:42.270 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T03:34:42.260+0000 7f93257fa640 1 -- 172.21.15.114:0/2353653452 <== mon.1 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9320034040 con 0x7f9328105180 2024-09-21T03:34:42.270 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T03:34:42.260+0000 7f93257fa640 1 -- 172.21.15.114:0/2353653452 <== mon.1 v2:172.21.15.114:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9320030850 con 0x7f9328105180 2024-09-21T03:34:42.270 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T03:34:42.260+0000 7f93257fa640 1 -- 172.21.15.114:0/2353653452 <== mon.1 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9320004790 con 0x7f9328105180 2024-09-21T03:34:42.271 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T03:34:42.260+0000 7f932e5e8640 1 -- 172.21.15.114:0/2353653452 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f932819bb90 con 0x7f9328105180 2024-09-21T03:34:42.271 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T03:34:42.260+0000 7f932e5e8640 1 -- 172.21.15.114:0/2353653452 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f93281a21a0 con 0x7f9328105180 2024-09-21T03:34:42.271 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T03:34:42.261+0000 7f932e5e8640 1 -- 172.21.15.114:0/2353653452 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f932810a930 con 0x7f9328105180 2024-09-21T03:34:42.271 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T03:34:42.263+0000 7f93257fa640 1 -- 172.21.15.114:0/2353653452 <== mon.1 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 17) ==== 154512+0+0 (secure 0 0 0) 0x7f9320037020 con 0x7f9328105180 2024-09-21T03:34:42.271 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T03:34:42.263+0000 7f93257fa640 1 --2- 172.21.15.114:0/2353653452 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f93100b6510 0x7f93100b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:42.271 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T03:34:42.263+0000 7f93257fa640 1 -- 172.21.15.114:0/2353653452 <== mon.1 v2:172.21.15.114:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5150+0+0 (secure 0 0 0) 0x7f9320107770 con 0x7f9328105180 2024-09-21T03:34:42.272 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T03:34:42.263+0000 7f9327fff640 1 --2- 172.21.15.114:0/2353653452 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f93100b6510 0x7f93100b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:42.272 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T03:34:42.264+0000 7f9327fff640 1 --2- 172.21.15.114:0/2353653452 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f93100b6510 0x7f93100b8a20 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f9318009990 tx=0x7f9318009110 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:34:42.272 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T03:34:42.267+0000 7f93257fa640 1 -- 172.21.15.114:0/2353653452 <== mon.1 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f93200cecf0 con 0x7f9328105180 2024-09-21T03:34:42.456 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T03:34:42.455+0000 7f932e5e8640 1 -- 172.21.15.114:0/2353653452 --> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi114:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f9328063800 con 0x7f93100b6510 2024-09-21T03:34:42.711 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:34:42 smithi114 ceph-mon[34140]: Detected new or changed devices on smithi110 2024-09-21T03:34:42.711 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:34:42 smithi114 ceph-mon[34140]: Adjusting osd_memory_target on smithi110 to 5696M 2024-09-21T03:34:42.711 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:34:42 smithi114 ceph-mon[34140]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi110", "root=default"]}]': finished 2024-09-21T03:34:42.711 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:34:42 smithi114 ceph-mon[34140]: osdmap e44: 8 total, 7 up, 8 in 2024-09-21T03:34:42.711 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:34:42 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-21T03:34:42.711 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:34:42 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-21T03:34:42.711 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:34:42 smithi114 ceph-mon[34140]: osdmap e45: 8 total, 7 up, 8 in 2024-09-21T03:34:42.711 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:34:42 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-21T03:34:42.711 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:34:42 smithi114 ceph-mon[34140]: from='osd.7 ' entity='osd.7' 2024-09-21T03:34:42.711 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:34:42 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T03:34:42.712 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:34:42 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T03:34:42.712 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:34:42 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:34:42.811 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:34:42 smithi089 ceph-mon[30928]: Detected new or changed devices on smithi110 2024-09-21T03:34:42.811 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:34:42 smithi089 ceph-mon[30928]: Adjusting osd_memory_target on smithi110 to 5696M 2024-09-21T03:34:42.811 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:34:42 smithi089 ceph-mon[30928]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi110", "root=default"]}]': finished 2024-09-21T03:34:42.811 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:34:42 smithi089 ceph-mon[30928]: osdmap e44: 8 total, 7 up, 8 in 2024-09-21T03:34:42.812 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:34:42 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-21T03:34:42.812 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:34:42 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-21T03:34:42.812 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:34:42 smithi089 ceph-mon[30928]: osdmap e45: 8 total, 7 up, 8 in 2024-09-21T03:34:42.812 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:34:42 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-21T03:34:42.812 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:34:42 smithi089 ceph-mon[30928]: from='osd.7 ' entity='osd.7' 2024-09-21T03:34:42.812 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:34:42 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T03:34:42.812 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:34:42 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T03:34:42.812 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:34:42 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:34:43.000 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:34:42 smithi110 ceph-mon[34000]: Detected new or changed devices on smithi110 2024-09-21T03:34:43.000 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:34:42 smithi110 ceph-mon[34000]: Adjusting osd_memory_target on smithi110 to 5696M 2024-09-21T03:34:43.000 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:34:42 smithi110 ceph-mon[34000]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi110", "root=default"]}]': finished 2024-09-21T03:34:43.001 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:34:42 smithi110 ceph-mon[34000]: osdmap e44: 8 total, 7 up, 8 in 2024-09-21T03:34:43.001 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:34:42 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-21T03:34:43.001 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:34:42 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-21T03:34:43.001 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:34:42 smithi110 ceph-mon[34000]: osdmap e45: 8 total, 7 up, 8 in 2024-09-21T03:34:43.001 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:34:42 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-21T03:34:43.001 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:34:42 smithi110 ceph-mon[34000]: from='osd.7 ' entity='osd.7' 2024-09-21T03:34:43.001 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:34:42 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T03:34:43.001 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:34:42 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T03:34:43.001 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:34:42 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:34:43.792 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:34:43 smithi114 ceph-mon[34140]: purged_snaps scrub starts 2024-09-21T03:34:43.792 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:34:43 smithi114 ceph-mon[34140]: purged_snaps scrub ok 2024-09-21T03:34:43.792 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:34:43 smithi114 ceph-mon[34140]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-09-21T03:34:43.792 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:34:43 smithi114 ceph-mon[34140]: from='client.24436 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi114:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:34:43.792 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:34:43 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-21T03:34:43.792 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:34:43 smithi114 ceph-mon[34140]: osd.7 [v2:172.21.15.110:6816/1331818323,v1:172.21.15.110:6817/1331818323] boot 2024-09-21T03:34:43.792 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:34:43 smithi114 ceph-mon[34140]: osdmap e46: 8 total, 8 up, 8 in 2024-09-21T03:34:43.793 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:34:43 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-21T03:34:43.811 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:34:43 smithi089 ceph-mon[30928]: purged_snaps scrub starts 2024-09-21T03:34:43.811 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:34:43 smithi089 ceph-mon[30928]: purged_snaps scrub ok 2024-09-21T03:34:43.811 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:34:43 smithi089 ceph-mon[30928]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-09-21T03:34:43.811 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:34:43 smithi089 ceph-mon[30928]: from='client.24436 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi114:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:34:43.811 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:34:43 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-21T03:34:43.811 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:34:43 smithi089 ceph-mon[30928]: osd.7 [v2:172.21.15.110:6816/1331818323,v1:172.21.15.110:6817/1331818323] boot 2024-09-21T03:34:43.811 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:34:43 smithi089 ceph-mon[30928]: osdmap e46: 8 total, 8 up, 8 in 2024-09-21T03:34:43.811 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:34:43 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-21T03:34:44.000 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:34:43 smithi110 ceph-mon[34000]: purged_snaps scrub starts 2024-09-21T03:34:44.000 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:34:43 smithi110 ceph-mon[34000]: purged_snaps scrub ok 2024-09-21T03:34:44.000 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:34:43 smithi110 ceph-mon[34000]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-09-21T03:34:44.000 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:34:43 smithi110 ceph-mon[34000]: from='client.24436 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi114:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:34:44.000 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:34:43 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-21T03:34:44.000 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:34:43 smithi110 ceph-mon[34000]: osd.7 [v2:172.21.15.110:6816/1331818323,v1:172.21.15.110:6817/1331818323] boot 2024-09-21T03:34:44.000 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:34:43 smithi110 ceph-mon[34000]: osdmap e46: 8 total, 8 up, 8 in 2024-09-21T03:34:44.000 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:34:43 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-21T03:34:44.710 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:34:44 smithi114 ceph-mon[34140]: pgmap v182: 1 pgs: 1 remapped+peering; 577 KiB data, 243 MiB used, 715 GiB / 715 GiB avail 2024-09-21T03:34:44.711 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:34:44 smithi114 ceph-mon[34140]: osdmap e47: 8 total, 8 up, 8 in 2024-09-21T03:34:45.000 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:34:44 smithi110 ceph-mon[34000]: pgmap v182: 1 pgs: 1 remapped+peering; 577 KiB data, 243 MiB used, 715 GiB / 715 GiB avail 2024-09-21T03:34:45.000 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:34:44 smithi110 ceph-mon[34000]: osdmap e47: 8 total, 8 up, 8 in 2024-09-21T03:34:45.061 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:34:44 smithi089 ceph-mon[30928]: pgmap v182: 1 pgs: 1 remapped+peering; 577 KiB data, 243 MiB used, 715 GiB / 715 GiB avail 2024-09-21T03:34:45.061 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:34:44 smithi089 ceph-mon[30928]: osdmap e47: 8 total, 8 up, 8 in 2024-09-21T03:34:45.711 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:34:45 smithi114 ceph-mon[34140]: osdmap e48: 8 total, 8 up, 8 in 2024-09-21T03:34:45.711 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:34:45 smithi114 ceph-mon[34140]: from='client.? 172.21.15.114:0/2700215125' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "1bc2a5b4-15f3-456f-9d67-e51307a27bcc"} : dispatch 2024-09-21T03:34:45.711 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:34:45 smithi114 ceph-mon[34140]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "1bc2a5b4-15f3-456f-9d67-e51307a27bcc"} : dispatch 2024-09-21T03:34:45.711 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:34:45 smithi114 ceph-mon[34140]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1bc2a5b4-15f3-456f-9d67-e51307a27bcc"}]': finished 2024-09-21T03:34:45.711 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:34:45 smithi114 ceph-mon[34140]: osdmap e49: 9 total, 8 up, 9 in 2024-09-21T03:34:45.711 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:34:45 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-21T03:34:45.711 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:34:45 smithi114 ceph-mon[34140]: from='client.? 172.21.15.114:0/3174523843' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T03:34:46.000 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:34:45 smithi110 ceph-mon[34000]: osdmap e48: 8 total, 8 up, 8 in 2024-09-21T03:34:46.000 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:34:45 smithi110 ceph-mon[34000]: from='client.? 172.21.15.114:0/2700215125' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "1bc2a5b4-15f3-456f-9d67-e51307a27bcc"} : dispatch 2024-09-21T03:34:46.000 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:34:45 smithi110 ceph-mon[34000]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "1bc2a5b4-15f3-456f-9d67-e51307a27bcc"} : dispatch 2024-09-21T03:34:46.000 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:34:45 smithi110 ceph-mon[34000]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1bc2a5b4-15f3-456f-9d67-e51307a27bcc"}]': finished 2024-09-21T03:34:46.000 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:34:45 smithi110 ceph-mon[34000]: osdmap e49: 9 total, 8 up, 9 in 2024-09-21T03:34:46.000 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:34:45 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-21T03:34:46.000 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:34:45 smithi110 ceph-mon[34000]: from='client.? 172.21.15.114:0/3174523843' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T03:34:46.061 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:34:45 smithi089 ceph-mon[30928]: osdmap e48: 8 total, 8 up, 8 in 2024-09-21T03:34:46.061 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:34:45 smithi089 ceph-mon[30928]: from='client.? 172.21.15.114:0/2700215125' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "1bc2a5b4-15f3-456f-9d67-e51307a27bcc"} : dispatch 2024-09-21T03:34:46.061 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:34:45 smithi089 ceph-mon[30928]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "1bc2a5b4-15f3-456f-9d67-e51307a27bcc"} : dispatch 2024-09-21T03:34:46.061 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:34:45 smithi089 ceph-mon[30928]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1bc2a5b4-15f3-456f-9d67-e51307a27bcc"}]': finished 2024-09-21T03:34:46.061 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:34:45 smithi089 ceph-mon[30928]: osdmap e49: 9 total, 8 up, 9 in 2024-09-21T03:34:46.061 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:34:45 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-21T03:34:46.061 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:34:45 smithi089 ceph-mon[30928]: from='client.? 172.21.15.114:0/3174523843' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T03:34:47.061 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:34:46 smithi089 ceph-mon[30928]: pgmap v186: 1 pgs: 1 remapped+peering; 577 KiB data, 243 MiB used, 715 GiB / 715 GiB avail 2024-09-21T03:34:47.210 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:34:46 smithi114 ceph-mon[34140]: pgmap v186: 1 pgs: 1 remapped+peering; 577 KiB data, 243 MiB used, 715 GiB / 715 GiB avail 2024-09-21T03:34:47.250 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:34:46 smithi110 ceph-mon[34000]: pgmap v186: 1 pgs: 1 remapped+peering; 577 KiB data, 243 MiB used, 715 GiB / 715 GiB avail 2024-09-21T03:34:49.061 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:34:48 smithi089 ceph-mon[30928]: pgmap v187: 1 pgs: 1 remapped+peering; 577 KiB data, 243 MiB used, 715 GiB / 715 GiB avail 2024-09-21T03:34:49.210 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:34:48 smithi114 ceph-mon[34140]: pgmap v187: 1 pgs: 1 remapped+peering; 577 KiB data, 243 MiB used, 715 GiB / 715 GiB avail 2024-09-21T03:34:49.358 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:34:48 smithi110 ceph-mon[34000]: pgmap v187: 1 pgs: 1 remapped+peering; 577 KiB data, 243 MiB used, 715 GiB / 715 GiB avail 2024-09-21T03:34:51.061 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:34:50 smithi089 ceph-mon[30928]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 192 KiB/s, 0 objects/s recovering 2024-09-21T03:34:51.105 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:34:50 smithi114 ceph-mon[34140]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 192 KiB/s, 0 objects/s recovering 2024-09-21T03:34:51.250 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:34:50 smithi110 ceph-mon[34000]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 192 KiB/s, 0 objects/s recovering 2024-09-21T03:34:52.916 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:34:52 smithi114 ceph-mon[34140]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 145 KiB/s, 0 objects/s recovering 2024-09-21T03:34:53.061 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:34:52 smithi089 ceph-mon[30928]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 145 KiB/s, 0 objects/s recovering 2024-09-21T03:34:53.250 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:34:52 smithi110 ceph-mon[34000]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 145 KiB/s, 0 objects/s recovering 2024-09-21T03:34:53.961 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:34:53 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.8"} : dispatch 2024-09-21T03:34:53.961 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:34:53 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:34:53.961 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:34:53 smithi114 ceph-mon[34140]: Deploying daemon osd.8 on smithi114 2024-09-21T03:34:54.250 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:34:53 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.8"} : dispatch 2024-09-21T03:34:54.250 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:34:53 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:34:54.250 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:34:53 smithi110 ceph-mon[34000]: Deploying daemon osd.8 on smithi114 2024-09-21T03:34:54.311 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:34:53 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.8"} : dispatch 2024-09-21T03:34:54.311 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:34:53 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:34:54.311 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:34:53 smithi089 ceph-mon[30928]: Deploying daemon osd.8 on smithi114 2024-09-21T03:34:54.909 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:34:54 smithi114 ceph-mon[34140]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 128 KiB/s, 0 objects/s recovering 2024-09-21T03:34:55.250 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:34:54 smithi110 ceph-mon[34000]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 128 KiB/s, 0 objects/s recovering 2024-09-21T03:34:55.311 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:34:54 smithi089 ceph-mon[30928]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 128 KiB/s, 0 objects/s recovering 2024-09-21T03:34:56.961 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:34:56 smithi114 ceph-mon[34140]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 107 KiB/s, 0 objects/s recovering 2024-09-21T03:34:57.250 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:34:56 smithi110 ceph-mon[34000]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 107 KiB/s, 0 objects/s recovering 2024-09-21T03:34:57.311 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:34:56 smithi089 ceph-mon[30928]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 107 KiB/s, 0 objects/s recovering 2024-09-21T03:34:59.210 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:34:58 smithi114 ceph-mon[34140]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-21T03:34:59.250 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:34:58 smithi110 ceph-mon[34000]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-21T03:34:59.311 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:34:58 smithi089 ceph-mon[30928]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-21T03:35:00.061 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:34:59 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:35:00.062 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:34:59 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:35:00.062 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:34:59 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:35:00.211 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:34:59 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:35:00.211 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:34:59 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:35:00.211 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:34:59 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:35:00.250 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:34:59 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:35:00.250 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:34:59 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:35:00.250 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:34:59 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:35:01.311 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:35:01 smithi089 ceph-mon[30928]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-21T03:35:01.350 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:35:01 smithi114 ceph-mon[34140]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-21T03:35:01.500 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:35:01 smithi110 ceph-mon[34000]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-21T03:35:02.279 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T03:35:02.277+0000 7f93257fa640 1 -- 172.21.15.114:0/2353653452 <== mgr.14152 v2:172.21.15.89:6800/1591532358 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f9328063800 con 0x7f93100b6510 2024-09-21T03:35:02.279 INFO:teuthology.orchestra.run.smithi114.stdout:Created osd(s) 8 on host 'smithi114' 2024-09-21T03:35:02.281 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T03:35:02.280+0000 7f932e5e8640 1 -- 172.21.15.114:0/2353653452 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f93100b6510 msgr2=0x7f93100b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:35:02.282 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T03:35:02.280+0000 7f932e5e8640 1 --2- 172.21.15.114:0/2353653452 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f93100b6510 0x7f93100b8a20 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f9318009990 tx=0x7f9318009110 comp rx=0 tx=0).stop 2024-09-21T03:35:02.282 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T03:35:02.280+0000 7f932e5e8640 1 -- 172.21.15.114:0/2353653452 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9328105180 msgr2=0x7f932819b0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:35:02.282 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T03:35:02.280+0000 7f932e5e8640 1 --2- 172.21.15.114:0/2353653452 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9328105180 0x7f932819b0e0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f932002f800 tx=0x7f9320003f90 comp rx=0 tx=0).stop 2024-09-21T03:35:02.282 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T03:35:02.280+0000 7f932e5e8640 1 -- 172.21.15.114:0/2353653452 shutdown_connections 2024-09-21T03:35:02.282 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T03:35:02.280+0000 7f932e5e8640 1 --2- 172.21.15.114:0/2353653452 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f93100b6510 0x7f93100b8a20 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:02.282 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T03:35:02.280+0000 7f932e5e8640 1 --2- 172.21.15.114:0/2353653452 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9328105180 0x7f932819b0e0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:02.282 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T03:35:02.280+0000 7f932e5e8640 1 --2- 172.21.15.114:0/2353653452 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f93281047e0 0x7f932819a540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:02.282 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T03:35:02.280+0000 7f932e5e8640 1 --2- 172.21.15.114:0/2353653452 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9328103750 0x7f9328199ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:02.282 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T03:35:02.280+0000 7f932e5e8640 1 -- 172.21.15.114:0/2353653452 >> 172.21.15.114:0/2353653452 conn(0x7f93280ff0c0 msgr2=0x7f9328107b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:35:02.282 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T03:35:02.280+0000 7f932e5e8640 1 -- 172.21.15.114:0/2353653452 shutdown_connections 2024-09-21T03:35:02.282 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T03:35:02.280+0000 7f932e5e8640 1 -- 172.21.15.114:0/2353653452 wait complete. 2024-09-21T03:35:02.893 DEBUG:teuthology.orchestra.run.smithi114:osd.8> sudo journalctl -f -n 0 -u ceph-7c374098-77c9-11ef-baf5-efdc52797490@osd.8.service 2024-09-21T03:35:02.896 INFO:tasks.cephadm:Deploying osd.9 on smithi089 with /dev/vg_nvme/lv_1... 2024-09-21T03:35:02.896 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7c374098-77c9-11ef-baf5-efdc52797490 -- lvm zap /dev/vg_nvme/lv_1 2024-09-21T03:35:03.156 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/7c374098-77c9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:35:03.518 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:35:03 smithi114 ceph-mon[34140]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 715 GiB / 715 GiB avail 2024-09-21T03:35:03.519 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:35:03 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:35:03.519 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:35:03 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:35:03.519 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:35:03 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:35:03.519 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:35:03 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:35:03.561 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:35:03 smithi089 ceph-mon[30928]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 715 GiB / 715 GiB avail 2024-09-21T03:35:03.561 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:35:03 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:35:03.561 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:35:03 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:35:03.561 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:35:03 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:35:03.562 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:35:03 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:35:03.566 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:35:03 smithi110 ceph-mon[34000]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 715 GiB / 715 GiB avail 2024-09-21T03:35:03.566 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:35:03 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:35:03.566 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:35:03 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:35:03.566 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:35:03 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:35:03.566 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:35:03 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:35:04.915 INFO:journalctl@ceph.osd.8.smithi114.stdout:Sep 21 03:35:04 smithi114 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-8[51702]: 2024-09-21T03:35:04.839+0000 7f3e6d9a2640 -1 osd.8 0 log_to_monitors true 2024-09-21T03:35:05.446 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:35:05 smithi089 ceph-mon[30928]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 715 GiB / 715 GiB avail 2024-09-21T03:35:05.446 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:35:05 smithi089 ceph-mon[30928]: from='osd.8 [v2:172.21.15.114:6816/2138556699,v1:172.21.15.114:6817/2138556699]' entity='osd.8' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]} : dispatch 2024-09-21T03:35:05.711 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:35:05 smithi114 ceph-mon[34140]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 715 GiB / 715 GiB avail 2024-09-21T03:35:05.711 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:35:05 smithi114 ceph-mon[34140]: from='osd.8 [v2:172.21.15.114:6816/2138556699,v1:172.21.15.114:6817/2138556699]' entity='osd.8' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]} : dispatch 2024-09-21T03:35:05.750 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:35:05 smithi110 ceph-mon[34000]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 715 GiB / 715 GiB avail 2024-09-21T03:35:05.750 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:35:05 smithi110 ceph-mon[34000]: from='osd.8 [v2:172.21.15.114:6816/2138556699,v1:172.21.15.114:6817/2138556699]' entity='osd.8' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]} : dispatch 2024-09-21T03:35:05.950 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-09-21T03:35:05.977 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7c374098-77c9-11ef-baf5-efdc52797490 -- ceph orch daemon add osd smithi089:vg_nvme/lv_1 2024-09-21T03:35:06.243 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/7c374098-77c9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:35:06.461 INFO:journalctl@ceph.osd.8.smithi114.stdout:Sep 21 03:35:06 smithi114 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-8[51702]: 2024-09-21T03:35:06.288+0000 7f3e69924640 -1 osd.8 0 waiting for initial osdmap 2024-09-21T03:35:06.461 INFO:journalctl@ceph.osd.8.smithi114.stdout:Sep 21 03:35:06 smithi114 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-8[51702]: 2024-09-21T03:35:06.300+0000 7f3e65722640 -1 osd.8 51 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-21T03:35:06.461 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:35:06 smithi114 ceph-mon[34140]: from='osd.8 [v2:172.21.15.114:6816/2138556699,v1:172.21.15.114:6817/2138556699]' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-09-21T03:35:06.461 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:35:06 smithi114 ceph-mon[34140]: osdmap e50: 9 total, 8 up, 9 in 2024-09-21T03:35:06.461 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:35:06 smithi114 ceph-mon[34140]: from='osd.8 [v2:172.21.15.114:6816/2138556699,v1:172.21.15.114:6817/2138556699]' entity='osd.8' cmd={"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi114", "root=default"]} : dispatch 2024-09-21T03:35:06.461 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:35:06 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-21T03:35:06.461 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:35:06 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:35:06.461 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:35:06 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:35:06.461 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:35:06 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-21T03:35:06.462 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:35:06 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-21T03:35:06.462 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:35:06 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"} : dispatch 2024-09-21T03:35:06.462 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:35:06 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:35:06.462 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:35:06 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:35:06.462 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:35:06 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:35:06.462 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:35:06 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:35:06.462 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:35:06 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:35:06.462 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:35:06 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:35:06.462 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:35:06 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:35:06.462 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:35:06 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:35:06.547 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:35:06 smithi089 ceph-mon[30928]: from='osd.8 [v2:172.21.15.114:6816/2138556699,v1:172.21.15.114:6817/2138556699]' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-09-21T03:35:06.547 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:35:06 smithi089 ceph-mon[30928]: osdmap e50: 9 total, 8 up, 9 in 2024-09-21T03:35:06.547 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:35:06 smithi089 ceph-mon[30928]: from='osd.8 [v2:172.21.15.114:6816/2138556699,v1:172.21.15.114:6817/2138556699]' entity='osd.8' cmd={"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi114", "root=default"]} : dispatch 2024-09-21T03:35:06.547 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:35:06 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-21T03:35:06.547 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:35:06 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:35:06.548 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:35:06 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:35:06.548 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:35:06 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-21T03:35:06.548 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:35:06 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-21T03:35:06.548 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:35:06 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"} : dispatch 2024-09-21T03:35:06.548 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:35:06 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:35:06.548 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:35:06 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:35:06.548 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:35:06 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:35:06.548 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:35:06 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:35:06.548 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:35:06 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:35:06.548 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:35:06 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:35:06.548 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:35:06 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:35:06.548 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:35:06 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:35:06.750 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:35:06 smithi110 ceph-mon[34000]: from='osd.8 [v2:172.21.15.114:6816/2138556699,v1:172.21.15.114:6817/2138556699]' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-09-21T03:35:06.750 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:35:06 smithi110 ceph-mon[34000]: osdmap e50: 9 total, 8 up, 9 in 2024-09-21T03:35:06.750 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:35:06 smithi110 ceph-mon[34000]: from='osd.8 [v2:172.21.15.114:6816/2138556699,v1:172.21.15.114:6817/2138556699]' entity='osd.8' cmd={"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi114", "root=default"]} : dispatch 2024-09-21T03:35:06.750 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:35:06 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-21T03:35:06.751 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:35:06 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:35:06.751 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:35:06 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:35:06.751 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:35:06 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-21T03:35:06.751 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:35:06 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-21T03:35:06.751 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:35:06 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"} : dispatch 2024-09-21T03:35:06.751 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:35:06 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:35:06.751 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:35:06 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:35:06.751 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:35:06 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:35:06.751 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:35:06 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:35:06.751 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:35:06 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:35:06.751 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:35:06 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:35:06.751 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:35:06 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:35:06.751 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:35:06 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:35:06.769 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:35:06.759+0000 7fb8b6b62640 1 -- 172.21.15.89:0/679755120 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb8b0105180 msgr2=0x7fb8b010bb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:35:06.769 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:35:06.759+0000 7fb8b6b62640 1 --2- 172.21.15.89:0/679755120 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb8b0105180 0x7fb8b010bb20 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fb8a8009ca0 tx=0x7fb8a802a950 comp rx=0 tx=0).stop 2024-09-21T03:35:06.769 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:35:06.759+0000 7fb8b6b62640 1 -- 172.21.15.89:0/679755120 shutdown_connections 2024-09-21T03:35:06.769 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:35:06.759+0000 7fb8b6b62640 1 --2- 172.21.15.89:0/679755120 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb8b0105180 0x7fb8b010bb20 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:06.769 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:35:06.759+0000 7fb8b6b62640 1 --2- 172.21.15.89:0/679755120 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb8b01047e0 0x7fb8b0104bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:06.769 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:35:06.759+0000 7fb8b6b62640 1 --2- 172.21.15.89:0/679755120 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb8b0103750 0x7fb8b0103bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:06.769 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:35:06.759+0000 7fb8b6b62640 1 -- 172.21.15.89:0/679755120 >> 172.21.15.89:0/679755120 conn(0x7fb8b00ff0c0 msgr2=0x7fb8b0101570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:35:06.769 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:35:06.760+0000 7fb8b6b62640 1 -- 172.21.15.89:0/679755120 shutdown_connections 2024-09-21T03:35:06.769 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:35:06.760+0000 7fb8b6b62640 1 -- 172.21.15.89:0/679755120 wait complete. 2024-09-21T03:35:06.769 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:35:06.760+0000 7fb8b6b62640 1 Processor -- start 2024-09-21T03:35:06.769 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:35:06.760+0000 7fb8b6b62640 1 -- start start 2024-09-21T03:35:06.769 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:35:06.761+0000 7fb8b6b62640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb8b0103750 0x7fb8b0076e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:06.770 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:35:06.761+0000 7fb8b6b62640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb8b01047e0 0x7fb8b0075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:06.770 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:35:06.761+0000 7fb8b6b62640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb8b0105180 0x7fb8b0075c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:06.770 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:35:06.761+0000 7fb8b6b62640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fb8b010f250 con 0x7fb8b01047e0 2024-09-21T03:35:06.770 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:35:06.761+0000 7fb8b6b62640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fb8b010f0d0 con 0x7fb8b0105180 2024-09-21T03:35:06.770 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:35:06.761+0000 7fb8b6b62640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fb8b010f3d0 con 0x7fb8b0103750 2024-09-21T03:35:06.770 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:35:06.761+0000 7fb8a7fff640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb8b01047e0 0x7fb8b0075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:06.770 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:35:06.761+0000 7fb8a7fff640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb8b01047e0 0x7fb8b0075710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:60996/0 (socket says 172.21.15.89:60996) 2024-09-21T03:35:06.770 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:35:06.761+0000 7fb8a7fff640 1 -- 172.21.15.89:0/273253646 learned_addr learned my addr 172.21.15.89:0/273253646 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-21T03:35:06.770 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:35:06.761+0000 7fb8b48fe640 1 --2- 172.21.15.89:0/273253646 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb8b0103750 0x7fb8b0076e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:06.770 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:35:06.761+0000 7fb8a7fff640 1 -- 172.21.15.89:0/273253646 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb8b0103750 msgr2=0x7fb8b0076e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:35:06.770 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:35:06.761+0000 7fb8a7fff640 1 --2- 172.21.15.89:0/273253646 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb8b0103750 0x7fb8b0076e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:06.770 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:35:06.761+0000 7fb8b50ff640 1 --2- 172.21.15.89:0/273253646 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb8b0105180 0x7fb8b0075c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:06.770 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:35:06.761+0000 7fb8a7fff640 1 -- 172.21.15.89:0/273253646 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb8b0105180 msgr2=0x7fb8b0075c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:35:06.771 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:35:06.761+0000 7fb8a7fff640 1 --2- 172.21.15.89:0/273253646 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb8b0105180 0x7fb8b0075c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:06.771 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:35:06.761+0000 7fb8a7fff640 1 -- 172.21.15.89:0/273253646 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8b0076350 con 0x7fb8b01047e0 2024-09-21T03:35:06.771 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:35:06.761+0000 7fb8a7fff640 1 --2- 172.21.15.89:0/273253646 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb8b01047e0 0x7fb8b0075710 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fb8a000d770 tx=0x7fb8a000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:35:06.771 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:35:06.761+0000 7fb8a5ffb640 1 -- 172.21.15.89:0/273253646 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8a0017070 con 0x7fb8b01047e0 2024-09-21T03:35:06.771 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:35:06.761+0000 7fb8a5ffb640 1 -- 172.21.15.89:0/273253646 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb8a00054b0 con 0x7fb8b01047e0 2024-09-21T03:35:06.771 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:35:06.761+0000 7fb8b6b62640 1 -- 172.21.15.89:0/273253646 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb8b01a3f10 con 0x7fb8b01047e0 2024-09-21T03:35:06.771 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:35:06.761+0000 7fb8a5ffb640 1 -- 172.21.15.89:0/273253646 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8a0004c60 con 0x7fb8b01047e0 2024-09-21T03:35:06.771 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:35:06.762+0000 7fb8b6b62640 1 -- 172.21.15.89:0/273253646 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8b01a42b0 con 0x7fb8b01047e0 2024-09-21T03:35:06.771 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:35:06.763+0000 7fb8b6b62640 1 -- 172.21.15.89:0/273253646 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb8b0103bc0 con 0x7fb8b01047e0 2024-09-21T03:35:06.771 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:35:06.764+0000 7fb8a5ffb640 1 -- 172.21.15.89:0/273253646 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 17) ==== 154512+0+0 (secure 0 0 0) 0x7fb8a0004e00 con 0x7fb8b01047e0 2024-09-21T03:35:06.771 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:35:06.764+0000 7fb8a5ffb640 1 --2- 172.21.15.89:0/273253646 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7fb8980b6590 0x7fb8980b8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:06.771 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:35:06.765+0000 7fb8a5ffb640 1 -- 172.21.15.89:0/273253646 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(52..52 src has 1..52) ==== 5862+0+0 (secure 0 0 0) 0x7fb8a00e6450 con 0x7fb8b01047e0 2024-09-21T03:35:06.771 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:35:06.765+0000 7fb8b48fe640 1 --2- 172.21.15.89:0/273253646 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7fb8980b6590 0x7fb8980b8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:06.772 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:35:06.765+0000 7fb8b48fe640 1 --2- 172.21.15.89:0/273253646 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7fb8980b6590 0x7fb8980b8aa0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fb8ac005fd0 tx=0x7fb8ac005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:35:06.772 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:35:06.769+0000 7fb8a5ffb640 1 -- 172.21.15.89:0/273253646 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fb8a00ad670 con 0x7fb8b01047e0 2024-09-21T03:35:06.964 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:35:06.961+0000 7fb8b6b62640 1 -- 172.21.15.89:0/273253646 --> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi089:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7fb8b0063800 con 0x7fb8980b6590 2024-09-21T03:35:07.561 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:35:07 smithi089 ceph-mon[30928]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 715 GiB / 715 GiB avail 2024-09-21T03:35:07.561 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:35:07 smithi089 ceph-mon[30928]: Detected new or changed devices on smithi114 2024-09-21T03:35:07.561 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:35:07 smithi089 ceph-mon[30928]: Adjusting osd_memory_target on smithi114 to 5696M 2024-09-21T03:35:07.561 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:35:07 smithi089 ceph-mon[30928]: from='osd.8 [v2:172.21.15.114:6816/2138556699,v1:172.21.15.114:6817/2138556699]' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi114", "root=default"]}]': finished 2024-09-21T03:35:07.561 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:35:07 smithi089 ceph-mon[30928]: osdmap e51: 9 total, 8 up, 9 in 2024-09-21T03:35:07.561 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:35:07 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-21T03:35:07.561 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:35:07 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-21T03:35:07.561 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:35:07 smithi089 ceph-mon[30928]: osd.8 [v2:172.21.15.114:6816/2138556699,v1:172.21.15.114:6817/2138556699] boot 2024-09-21T03:35:07.561 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:35:07 smithi089 ceph-mon[30928]: osdmap e52: 9 total, 9 up, 9 in 2024-09-21T03:35:07.562 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:35:07 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-21T03:35:07.562 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:35:07 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T03:35:07.562 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:35:07 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T03:35:07.562 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:35:07 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:35:07.711 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:35:07 smithi114 ceph-mon[34140]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 715 GiB / 715 GiB avail 2024-09-21T03:35:07.711 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:35:07 smithi114 ceph-mon[34140]: Detected new or changed devices on smithi114 2024-09-21T03:35:07.711 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:35:07 smithi114 ceph-mon[34140]: Adjusting osd_memory_target on smithi114 to 5696M 2024-09-21T03:35:07.711 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:35:07 smithi114 ceph-mon[34140]: from='osd.8 [v2:172.21.15.114:6816/2138556699,v1:172.21.15.114:6817/2138556699]' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi114", "root=default"]}]': finished 2024-09-21T03:35:07.711 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:35:07 smithi114 ceph-mon[34140]: osdmap e51: 9 total, 8 up, 9 in 2024-09-21T03:35:07.711 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:35:07 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-21T03:35:07.711 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:35:07 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-21T03:35:07.711 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:35:07 smithi114 ceph-mon[34140]: osd.8 [v2:172.21.15.114:6816/2138556699,v1:172.21.15.114:6817/2138556699] boot 2024-09-21T03:35:07.711 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:35:07 smithi114 ceph-mon[34140]: osdmap e52: 9 total, 9 up, 9 in 2024-09-21T03:35:07.711 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:35:07 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-21T03:35:07.712 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:35:07 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T03:35:07.712 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:35:07 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T03:35:07.712 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:35:07 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:35:07.750 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:35:07 smithi110 ceph-mon[34000]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 715 GiB / 715 GiB avail 2024-09-21T03:35:07.750 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:35:07 smithi110 ceph-mon[34000]: Detected new or changed devices on smithi114 2024-09-21T03:35:07.750 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:35:07 smithi110 ceph-mon[34000]: Adjusting osd_memory_target on smithi114 to 5696M 2024-09-21T03:35:07.750 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:35:07 smithi110 ceph-mon[34000]: from='osd.8 [v2:172.21.15.114:6816/2138556699,v1:172.21.15.114:6817/2138556699]' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi114", "root=default"]}]': finished 2024-09-21T03:35:07.750 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:35:07 smithi110 ceph-mon[34000]: osdmap e51: 9 total, 8 up, 9 in 2024-09-21T03:35:07.750 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:35:07 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-21T03:35:07.751 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:35:07 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-21T03:35:07.751 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:35:07 smithi110 ceph-mon[34000]: osd.8 [v2:172.21.15.114:6816/2138556699,v1:172.21.15.114:6817/2138556699] boot 2024-09-21T03:35:07.751 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:35:07 smithi110 ceph-mon[34000]: osdmap e52: 9 total, 9 up, 9 in 2024-09-21T03:35:07.751 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:35:07 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-21T03:35:07.751 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:35:07 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T03:35:07.751 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:35:07 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T03:35:07.751 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:35:07 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:35:08.470 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:35:08 smithi089 ceph-mon[30928]: purged_snaps scrub starts 2024-09-21T03:35:08.470 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:35:08 smithi089 ceph-mon[30928]: purged_snaps scrub ok 2024-09-21T03:35:08.470 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:35:08 smithi089 ceph-mon[30928]: from='client.14589 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi089:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:35:08.471 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:35:08 smithi089 ceph-mon[30928]: osdmap e53: 9 total, 9 up, 9 in 2024-09-21T03:35:08.711 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:35:08 smithi114 ceph-mon[34140]: purged_snaps scrub starts 2024-09-21T03:35:08.711 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:35:08 smithi114 ceph-mon[34140]: purged_snaps scrub ok 2024-09-21T03:35:08.711 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:35:08 smithi114 ceph-mon[34140]: from='client.14589 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi089:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:35:08.711 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:35:08 smithi114 ceph-mon[34140]: osdmap e53: 9 total, 9 up, 9 in 2024-09-21T03:35:08.750 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:35:08 smithi110 ceph-mon[34000]: purged_snaps scrub starts 2024-09-21T03:35:08.750 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:35:08 smithi110 ceph-mon[34000]: purged_snaps scrub ok 2024-09-21T03:35:08.750 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:35:08 smithi110 ceph-mon[34000]: from='client.14589 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi089:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:35:08.750 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:35:08 smithi110 ceph-mon[34000]: osdmap e53: 9 total, 9 up, 9 in 2024-09-21T03:35:09.561 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:35:09 smithi089 ceph-mon[30928]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 804 GiB / 805 GiB avail 2024-09-21T03:35:09.561 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:35:09 smithi089 ceph-mon[30928]: osdmap e54: 9 total, 9 up, 9 in 2024-09-21T03:35:09.711 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:35:09 smithi114 ceph-mon[34140]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 804 GiB / 805 GiB avail 2024-09-21T03:35:09.711 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:35:09 smithi114 ceph-mon[34140]: osdmap e54: 9 total, 9 up, 9 in 2024-09-21T03:35:09.750 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:35:09 smithi110 ceph-mon[34000]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 804 GiB / 805 GiB avail 2024-09-21T03:35:09.750 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:35:09 smithi110 ceph-mon[34000]: osdmap e54: 9 total, 9 up, 9 in 2024-09-21T03:35:10.561 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:35:10 smithi089 ceph-mon[30928]: from='client.? 172.21.15.89:0/2948755701' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "91f5fd29-d4a7-4800-ac7a-e83b13add89e"} : dispatch 2024-09-21T03:35:10.561 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:35:10 smithi089 ceph-mon[30928]: from='client.? 172.21.15.89:0/2948755701' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "91f5fd29-d4a7-4800-ac7a-e83b13add89e"}]': finished 2024-09-21T03:35:10.561 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:35:10 smithi089 ceph-mon[30928]: osdmap e55: 10 total, 9 up, 10 in 2024-09-21T03:35:10.561 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:35:10 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-21T03:35:10.561 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:35:10 smithi089 ceph-mon[30928]: from='client.? 172.21.15.89:0/1651533467' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T03:35:10.711 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:35:10 smithi114 ceph-mon[34140]: from='client.? 172.21.15.89:0/2948755701' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "91f5fd29-d4a7-4800-ac7a-e83b13add89e"} : dispatch 2024-09-21T03:35:10.711 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:35:10 smithi114 ceph-mon[34140]: from='client.? 172.21.15.89:0/2948755701' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "91f5fd29-d4a7-4800-ac7a-e83b13add89e"}]': finished 2024-09-21T03:35:10.711 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:35:10 smithi114 ceph-mon[34140]: osdmap e55: 10 total, 9 up, 10 in 2024-09-21T03:35:10.711 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:35:10 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-21T03:35:10.711 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:35:10 smithi114 ceph-mon[34140]: from='client.? 172.21.15.89:0/1651533467' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T03:35:10.750 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:35:10 smithi110 ceph-mon[34000]: from='client.? 172.21.15.89:0/2948755701' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "91f5fd29-d4a7-4800-ac7a-e83b13add89e"} : dispatch 2024-09-21T03:35:10.750 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:35:10 smithi110 ceph-mon[34000]: from='client.? 172.21.15.89:0/2948755701' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "91f5fd29-d4a7-4800-ac7a-e83b13add89e"}]': finished 2024-09-21T03:35:10.750 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:35:10 smithi110 ceph-mon[34000]: osdmap e55: 10 total, 9 up, 10 in 2024-09-21T03:35:10.750 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:35:10 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-21T03:35:10.750 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:35:10 smithi110 ceph-mon[34000]: from='client.? 172.21.15.89:0/1651533467' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T03:35:11.561 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:35:11 smithi089 ceph-mon[30928]: pgmap v204: 1 pgs: 1 peering; 577 KiB data, 271 MiB used, 804 GiB / 805 GiB avail 2024-09-21T03:35:11.711 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:35:11 smithi114 ceph-mon[34140]: pgmap v204: 1 pgs: 1 peering; 577 KiB data, 271 MiB used, 804 GiB / 805 GiB avail 2024-09-21T03:35:11.750 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:35:11 smithi110 ceph-mon[34000]: pgmap v204: 1 pgs: 1 peering; 577 KiB data, 271 MiB used, 804 GiB / 805 GiB avail 2024-09-21T03:35:13.561 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:35:13 smithi089 ceph-mon[30928]: pgmap v205: 1 pgs: 1 peering; 577 KiB data, 271 MiB used, 804 GiB / 805 GiB avail 2024-09-21T03:35:13.710 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:35:13 smithi114 ceph-mon[34140]: pgmap v205: 1 pgs: 1 peering; 577 KiB data, 271 MiB used, 804 GiB / 805 GiB avail 2024-09-21T03:35:13.750 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:35:13 smithi110 ceph-mon[34000]: pgmap v205: 1 pgs: 1 peering; 577 KiB data, 271 MiB used, 804 GiB / 805 GiB avail 2024-09-21T03:35:15.561 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:35:15 smithi089 ceph-mon[30928]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-21T03:35:15.711 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:35:15 smithi114 ceph-mon[34140]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-21T03:35:15.750 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:35:15 smithi110 ceph-mon[34000]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-21T03:35:17.561 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:35:17 smithi089 ceph-mon[30928]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-21T03:35:17.711 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:35:17 smithi114 ceph-mon[34140]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-21T03:35:17.749 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:35:17 smithi110 ceph-mon[34000]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-21T03:35:18.672 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:35:18 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.9"} : dispatch 2024-09-21T03:35:18.672 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:35:18 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:35:18.710 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:35:18 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.9"} : dispatch 2024-09-21T03:35:18.711 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:35:18 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:35:18.750 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:35:18 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.9"} : dispatch 2024-09-21T03:35:18.750 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:35:18 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:35:19.536 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:35:19 smithi089 ceph-mon[30928]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 804 GiB / 805 GiB avail; 64 KiB/s, 0 objects/s recovering 2024-09-21T03:35:19.536 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:35:19 smithi089 ceph-mon[30928]: Deploying daemon osd.9 on smithi089 2024-09-21T03:35:19.711 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:35:19 smithi114 ceph-mon[34140]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 804 GiB / 805 GiB avail; 64 KiB/s, 0 objects/s recovering 2024-09-21T03:35:19.711 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:35:19 smithi114 ceph-mon[34140]: Deploying daemon osd.9 on smithi089 2024-09-21T03:35:19.750 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:35:19 smithi110 ceph-mon[34000]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 804 GiB / 805 GiB avail; 64 KiB/s, 0 objects/s recovering 2024-09-21T03:35:19.750 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:35:19 smithi110 ceph-mon[34000]: Deploying daemon osd.9 on smithi089 2024-09-21T03:35:21.561 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:35:21 smithi089 ceph-mon[30928]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-21T03:35:21.710 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:35:21 smithi114 ceph-mon[34140]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-21T03:35:21.750 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:35:21 smithi110 ceph-mon[34000]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-21T03:35:23.550 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:35:23 smithi089 ceph-mon[30928]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 804 GiB / 805 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-21T03:35:23.711 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:35:23 smithi114 ceph-mon[34140]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 804 GiB / 805 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-21T03:35:23.750 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:35:23 smithi110 ceph-mon[34000]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 804 GiB / 805 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-21T03:35:24.673 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:35:24 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:35:24.673 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:35:24 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:35:24.673 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:35:24 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:35:24.710 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:35:24 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:35:24.711 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:35:24 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:35:24.711 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:35:24 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:35:24.750 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:35:24 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:35:24.750 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:35:24 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:35:24.750 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:35:24 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:35:25.711 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:35:25 smithi114 ceph-mon[34140]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 804 GiB / 805 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-21T03:35:25.750 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:35:25 smithi110 ceph-mon[34000]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 804 GiB / 805 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-21T03:35:25.799 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:35:25 smithi089 ceph-mon[30928]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 804 GiB / 805 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-21T03:35:27.711 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:35:27 smithi114 ceph-mon[34140]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 804 GiB / 805 GiB avail 2024-09-21T03:35:27.711 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:35:27 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:35:27.711 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:35:27 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:35:27.711 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:35:27 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:35:27.711 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:35:27 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:35:27.711 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:35:27 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:35:27.720 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:35:27 smithi089 ceph-mon[30928]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 804 GiB / 805 GiB avail 2024-09-21T03:35:27.720 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:35:27 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:35:27.720 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:35:27 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:35:27.720 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:35:27 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:35:27.720 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:35:27 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:35:27.720 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:35:27 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:35:27.750 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:35:27 smithi110 ceph-mon[34000]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 804 GiB / 805 GiB avail 2024-09-21T03:35:27.750 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:35:27 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:35:27.750 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:35:27 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:35:27.750 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:35:27 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:35:27.750 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:35:27 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:35:27.750 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:35:27 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:35:27.760 INFO:teuthology.orchestra.run.smithi089.stdout:Created osd(s) 9 on host 'smithi089' 2024-09-21T03:35:27.761 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:35:27.759+0000 7fb8a5ffb640 1 -- 172.21.15.89:0/273253646 <== mgr.14152 v2:172.21.15.89:6800/1591532358 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fb8b0063800 con 0x7fb8980b6590 2024-09-21T03:35:27.763 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:35:27.761+0000 7fb8b6b62640 1 -- 172.21.15.89:0/273253646 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7fb8980b6590 msgr2=0x7fb8980b8aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:35:27.763 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:35:27.761+0000 7fb8b6b62640 1 --2- 172.21.15.89:0/273253646 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7fb8980b6590 0x7fb8980b8aa0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fb8ac005fd0 tx=0x7fb8ac005850 comp rx=0 tx=0).stop 2024-09-21T03:35:27.763 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:35:27.761+0000 7fb8b6b62640 1 -- 172.21.15.89:0/273253646 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb8b01047e0 msgr2=0x7fb8b0075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:35:27.763 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:35:27.761+0000 7fb8b6b62640 1 --2- 172.21.15.89:0/273253646 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb8b01047e0 0x7fb8b0075710 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fb8a000d770 tx=0x7fb8a000dc30 comp rx=0 tx=0).stop 2024-09-21T03:35:27.763 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:35:27.761+0000 7fb8b6b62640 1 -- 172.21.15.89:0/273253646 shutdown_connections 2024-09-21T03:35:27.763 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:35:27.761+0000 7fb8b6b62640 1 --2- 172.21.15.89:0/273253646 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7fb8980b6590 0x7fb8980b8aa0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:27.763 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:35:27.761+0000 7fb8b6b62640 1 --2- 172.21.15.89:0/273253646 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb8b0105180 0x7fb8b0075c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:27.763 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:35:27.761+0000 7fb8b6b62640 1 --2- 172.21.15.89:0/273253646 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb8b01047e0 0x7fb8b0075710 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:27.763 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:35:27.761+0000 7fb8b6b62640 1 --2- 172.21.15.89:0/273253646 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb8b0103750 0x7fb8b0076e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:27.763 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:35:27.761+0000 7fb8b6b62640 1 -- 172.21.15.89:0/273253646 >> 172.21.15.89:0/273253646 conn(0x7fb8b00ff0c0 msgr2=0x7fb8b0107ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:35:27.763 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:35:27.762+0000 7fb8b6b62640 1 -- 172.21.15.89:0/273253646 shutdown_connections 2024-09-21T03:35:27.763 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:35:27.762+0000 7fb8b6b62640 1 -- 172.21.15.89:0/273253646 wait complete. 2024-09-21T03:35:28.459 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:35:28 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:35:28.459 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:35:28 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:35:28.459 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:35:28 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:35:28.491 DEBUG:teuthology.orchestra.run.smithi089:osd.9> sudo journalctl -f -n 0 -u ceph-7c374098-77c9-11ef-baf5-efdc52797490@osd.9.service 2024-09-21T03:35:28.494 INFO:tasks.cephadm:Deploying osd.10 on smithi110 with /dev/vg_nvme/lv_1... 2024-09-21T03:35:28.494 DEBUG:teuthology.orchestra.run.smithi110:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7c374098-77c9-11ef-baf5-efdc52797490 -- lvm zap /dev/vg_nvme/lv_1 2024-09-21T03:35:28.711 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:35:28 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:35:28.711 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:35:28 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:35:28.711 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:35:28 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:35:28.750 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:35:28 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:35:28.750 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:35:28 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:35:28.750 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:35:28 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:35:28.750 INFO:teuthology.orchestra.run.smithi110.stderr:Inferring config /var/lib/ceph/7c374098-77c9-11ef-baf5-efdc52797490/mon.b/config 2024-09-21T03:35:29.500 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:35:29 smithi110 ceph-mon[34000]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 804 GiB / 805 GiB avail 2024-09-21T03:35:29.543 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:35:29 smithi089 ceph-mon[30928]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 804 GiB / 805 GiB avail 2024-09-21T03:35:29.711 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:35:29 smithi114 ceph-mon[34140]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 804 GiB / 805 GiB avail 2024-09-21T03:35:30.514 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:35:30 smithi089 ceph-mon[30928]: from='osd.9 [v2:172.21.15.89:6826/1730253068,v1:172.21.15.89:6827/1730253068]' entity='osd.9' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]} : dispatch 2024-09-21T03:35:30.514 INFO:journalctl@ceph.osd.9.smithi089.stdout:Sep 21 03:35:30 smithi089 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-9[65404]: 2024-09-21T03:35:30.228+0000 7f6c55242640 -1 osd.9 0 log_to_monitors true 2024-09-21T03:35:30.663 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:35:30 smithi110 ceph-mon[34000]: from='osd.9 [v2:172.21.15.89:6826/1730253068,v1:172.21.15.89:6827/1730253068]' entity='osd.9' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]} : dispatch 2024-09-21T03:35:30.711 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:35:30 smithi114 ceph-mon[34140]: from='osd.9 [v2:172.21.15.89:6826/1730253068,v1:172.21.15.89:6827/1730253068]' entity='osd.9' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]} : dispatch 2024-09-21T03:35:31.500 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:35:31 smithi110 ceph-mon[34000]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 804 GiB / 805 GiB avail 2024-09-21T03:35:31.500 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:35:31 smithi110 ceph-mon[34000]: from='osd.9 [v2:172.21.15.89:6826/1730253068,v1:172.21.15.89:6827/1730253068]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-09-21T03:35:31.500 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:35:31 smithi110 ceph-mon[34000]: osdmap e56: 10 total, 9 up, 10 in 2024-09-21T03:35:31.500 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:35:31 smithi110 ceph-mon[34000]: from='osd.9 [v2:172.21.15.89:6826/1730253068,v1:172.21.15.89:6827/1730253068]' entity='osd.9' cmd={"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi089", "root=default"]} : dispatch 2024-09-21T03:35:31.500 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:35:31 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-21T03:35:31.500 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:35:31 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:35:31.501 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:35:31 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:35:31.501 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:35:31 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-21T03:35:31.501 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:35:31 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-21T03:35:31.501 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:35:31 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-09-21T03:35:31.501 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:35:31 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"} : dispatch 2024-09-21T03:35:31.501 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:35:31 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:35:31.501 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:35:31 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:35:31.501 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:35:31 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:35:31.501 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:35:31 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:35:31.531 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-21T03:35:31.557 DEBUG:teuthology.orchestra.run.smithi110:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7c374098-77c9-11ef-baf5-efdc52797490 -- ceph orch daemon add osd smithi110:vg_nvme/lv_1 2024-09-21T03:35:31.561 INFO:journalctl@ceph.osd.9.smithi089.stdout:Sep 21 03:35:31 smithi089 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-9[65404]: 2024-09-21T03:35:31.354+0000 7f6c511c4640 -1 osd.9 0 waiting for initial osdmap 2024-09-21T03:35:31.561 INFO:journalctl@ceph.osd.9.smithi089.stdout:Sep 21 03:35:31 smithi089 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-9[65404]: 2024-09-21T03:35:31.365+0000 7f6c4cfc2640 -1 osd.9 57 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-21T03:35:31.562 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:35:31 smithi089 ceph-mon[30928]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 804 GiB / 805 GiB avail 2024-09-21T03:35:31.562 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:35:31 smithi089 ceph-mon[30928]: from='osd.9 [v2:172.21.15.89:6826/1730253068,v1:172.21.15.89:6827/1730253068]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-09-21T03:35:31.562 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:35:31 smithi089 ceph-mon[30928]: osdmap e56: 10 total, 9 up, 10 in 2024-09-21T03:35:31.562 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:35:31 smithi089 ceph-mon[30928]: from='osd.9 [v2:172.21.15.89:6826/1730253068,v1:172.21.15.89:6827/1730253068]' entity='osd.9' cmd={"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi089", "root=default"]} : dispatch 2024-09-21T03:35:31.562 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:35:31 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-21T03:35:31.562 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:35:31 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:35:31.562 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:35:31 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:35:31.562 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:35:31 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-21T03:35:31.562 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:35:31 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-21T03:35:31.562 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:35:31 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-09-21T03:35:31.562 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:35:31 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"} : dispatch 2024-09-21T03:35:31.563 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:35:31 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:35:31.563 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:35:31 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:35:31.563 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:35:31 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:35:31.563 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:35:31 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:35:31.711 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:35:31 smithi114 ceph-mon[34140]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 804 GiB / 805 GiB avail 2024-09-21T03:35:31.711 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:35:31 smithi114 ceph-mon[34140]: from='osd.9 [v2:172.21.15.89:6826/1730253068,v1:172.21.15.89:6827/1730253068]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-09-21T03:35:31.711 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:35:31 smithi114 ceph-mon[34140]: osdmap e56: 10 total, 9 up, 10 in 2024-09-21T03:35:31.712 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:35:31 smithi114 ceph-mon[34140]: from='osd.9 [v2:172.21.15.89:6826/1730253068,v1:172.21.15.89:6827/1730253068]' entity='osd.9' cmd={"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi089", "root=default"]} : dispatch 2024-09-21T03:35:31.712 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:35:31 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-21T03:35:31.712 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:35:31 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:35:31.712 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:35:31 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:35:31.712 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:35:31 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-21T03:35:31.712 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:35:31 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-21T03:35:31.712 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:35:31 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-09-21T03:35:31.713 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:35:31 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"} : dispatch 2024-09-21T03:35:31.713 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:35:31 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:35:31.713 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:35:31 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:35:31.713 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:35:31 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:35:31.713 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:35:31 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:35:31.820 INFO:teuthology.orchestra.run.smithi110.stderr:Inferring config /var/lib/ceph/7c374098-77c9-11ef-baf5-efdc52797490/mon.b/config 2024-09-21T03:35:32.372 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T03:35:32.367+0000 7f8f255b1640 1 -- 172.21.15.110:0/2738338886 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8f20103780 msgr2=0x7f8f20103bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:35:32.372 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T03:35:32.367+0000 7f8f255b1640 1 --2- 172.21.15.110:0/2738338886 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8f20103780 0x7f8f20103bf0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f8f10009900 tx=0x7f8f1002a480 comp rx=0 tx=0).stop 2024-09-21T03:35:32.373 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T03:35:32.367+0000 7f8f255b1640 1 -- 172.21.15.110:0/2738338886 shutdown_connections 2024-09-21T03:35:32.373 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T03:35:32.367+0000 7f8f255b1640 1 --2- 172.21.15.110:0/2738338886 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8f201051b0 0x7f8f2010bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:32.373 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T03:35:32.367+0000 7f8f255b1640 1 --2- 172.21.15.110:0/2738338886 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8f20104810 0x7f8f20104c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:32.373 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T03:35:32.367+0000 7f8f255b1640 1 --2- 172.21.15.110:0/2738338886 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8f20103780 0x7f8f20103bf0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:32.373 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T03:35:32.367+0000 7f8f255b1640 1 -- 172.21.15.110:0/2738338886 >> 172.21.15.110:0/2738338886 conn(0x7f8f200ff0f0 msgr2=0x7f8f201015a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:35:32.373 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T03:35:32.367+0000 7f8f255b1640 1 -- 172.21.15.110:0/2738338886 shutdown_connections 2024-09-21T03:35:32.373 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T03:35:32.367+0000 7f8f255b1640 1 -- 172.21.15.110:0/2738338886 wait complete. 2024-09-21T03:35:32.373 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T03:35:32.368+0000 7f8f255b1640 1 Processor -- start 2024-09-21T03:35:32.373 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T03:35:32.368+0000 7f8f255b1640 1 -- start start 2024-09-21T03:35:32.373 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T03:35:32.369+0000 7f8f255b1640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8f20103780 0x7f8f20199f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:32.373 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T03:35:32.369+0000 7f8f255b1640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8f20104810 0x7f8f2019a590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:32.373 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T03:35:32.369+0000 7f8f255b1640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8f201051b0 0x7f8f2019b130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:32.373 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T03:35:32.369+0000 7f8f255b1640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f8f2010f270 con 0x7f8f20103780 2024-09-21T03:35:32.374 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T03:35:32.369+0000 7f8f255b1640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f8f2010f0f0 con 0x7f8f20104810 2024-09-21T03:35:32.374 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T03:35:32.369+0000 7f8f255b1640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f8f2010f3f0 con 0x7f8f201051b0 2024-09-21T03:35:32.374 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T03:35:32.369+0000 7f8f167fc640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8f20104810 0x7f8f2019a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:32.374 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T03:35:32.369+0000 7f8f167fc640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8f20104810 0x7f8f2019a590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:60242/0 (socket says 172.21.15.110:60242) 2024-09-21T03:35:32.374 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T03:35:32.369+0000 7f8f167fc640 1 -- 172.21.15.110:0/4130813265 learned_addr learned my addr 172.21.15.110:0/4130813265 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-21T03:35:32.374 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T03:35:32.369+0000 7f8f1effd640 1 --2- 172.21.15.110:0/4130813265 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8f20103780 0x7f8f20199f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:32.374 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T03:35:32.369+0000 7f8f1f7fe640 1 --2- 172.21.15.110:0/4130813265 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8f201051b0 0x7f8f2019b130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:32.374 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T03:35:32.369+0000 7f8f167fc640 1 -- 172.21.15.110:0/4130813265 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8f201051b0 msgr2=0x7f8f2019b130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:35:32.374 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T03:35:32.369+0000 7f8f167fc640 1 --2- 172.21.15.110:0/4130813265 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8f201051b0 0x7f8f2019b130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:32.374 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T03:35:32.369+0000 7f8f167fc640 1 -- 172.21.15.110:0/4130813265 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8f20103780 msgr2=0x7f8f20199f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:35:32.374 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T03:35:32.369+0000 7f8f167fc640 1 --2- 172.21.15.110:0/4130813265 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8f20103780 0x7f8f20199f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:32.374 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T03:35:32.369+0000 7f8f167fc640 1 -- 172.21.15.110:0/4130813265 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f2019b950 con 0x7f8f20104810 2024-09-21T03:35:32.374 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T03:35:32.370+0000 7f8f167fc640 1 --2- 172.21.15.110:0/4130813265 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8f20104810 0x7f8f2019a590 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f8f0800d770 tx=0x7f8f0800dc30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:35:32.375 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T03:35:32.370+0000 7f8f1cff9640 1 -- 172.21.15.110:0/4130813265 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f08017070 con 0x7f8f20104810 2024-09-21T03:35:32.375 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T03:35:32.370+0000 7f8f255b1640 1 -- 172.21.15.110:0/4130813265 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f2019bc40 con 0x7f8f20104810 2024-09-21T03:35:32.375 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T03:35:32.370+0000 7f8f1cff9640 1 -- 172.21.15.110:0/4130813265 <== mon.2 v2:172.21.15.110:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8f080054b0 con 0x7f8f20104810 2024-09-21T03:35:32.375 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T03:35:32.370+0000 7f8f1cff9640 1 -- 172.21.15.110:0/4130813265 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f08004c60 con 0x7f8f20104810 2024-09-21T03:35:32.375 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T03:35:32.370+0000 7f8f255b1640 1 -- 172.21.15.110:0/4130813265 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f201a2340 con 0x7f8f20104810 2024-09-21T03:35:32.375 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T03:35:32.371+0000 7f8f255b1640 1 -- 172.21.15.110:0/4130813265 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8f2010a960 con 0x7f8f20104810 2024-09-21T03:35:32.375 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T03:35:32.374+0000 7f8f1cff9640 1 -- 172.21.15.110:0/4130813265 <== mon.2 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 17) ==== 154512+0+0 (secure 0 0 0) 0x7f8f08004e00 con 0x7f8f20104810 2024-09-21T03:35:32.376 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T03:35:32.374+0000 7f8f1cff9640 1 --2- 172.21.15.110:0/4130813265 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f8f0c0b6510 0x7f8f0c0b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:32.376 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T03:35:32.375+0000 7f8f1cff9640 1 -- 172.21.15.110:0/4130813265 <== mon.2 v2:172.21.15.110:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 6261+0+0 (secure 0 0 0) 0x7f8f080e68b0 con 0x7f8f20104810 2024-09-21T03:35:32.376 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T03:35:32.375+0000 7f8f1effd640 1 --2- 172.21.15.110:0/4130813265 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f8f0c0b6510 0x7f8f0c0b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:32.377 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T03:35:32.375+0000 7f8f1effd640 1 --2- 172.21.15.110:0/4130813265 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f8f0c0b6510 0x7f8f0c0b8a20 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f8f100057c0 tx=0x7f8f10000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:35:32.379 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T03:35:32.378+0000 7f8f1cff9640 1 -- 172.21.15.110:0/4130813265 <== mon.2 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f8f080b1d20 con 0x7f8f20104810 2024-09-21T03:35:32.530 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:35:32 smithi110 ceph-mon[34000]: Detected new or changed devices on smithi089 2024-09-21T03:35:32.530 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:35:32 smithi110 ceph-mon[34000]: Adjusting osd_memory_target on smithi089 to 4271M 2024-09-21T03:35:32.530 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:35:32 smithi110 ceph-mon[34000]: from='osd.9 [v2:172.21.15.89:6826/1730253068,v1:172.21.15.89:6827/1730253068]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi089", "root=default"]}]': finished 2024-09-21T03:35:32.530 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:35:32 smithi110 ceph-mon[34000]: osdmap e57: 10 total, 9 up, 10 in 2024-09-21T03:35:32.531 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:35:32 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-21T03:35:32.531 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:35:32 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-21T03:35:32.573 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T03:35:32.571+0000 7f8f255b1640 1 -- 172.21.15.110:0/4130813265 --> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi110:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f8f20063800 con 0x7f8f0c0b6510 2024-09-21T03:35:32.711 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:35:32 smithi114 ceph-mon[34140]: Detected new or changed devices on smithi089 2024-09-21T03:35:32.711 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:35:32 smithi114 ceph-mon[34140]: Adjusting osd_memory_target on smithi089 to 4271M 2024-09-21T03:35:32.711 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:35:32 smithi114 ceph-mon[34140]: from='osd.9 [v2:172.21.15.89:6826/1730253068,v1:172.21.15.89:6827/1730253068]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi089", "root=default"]}]': finished 2024-09-21T03:35:32.711 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:35:32 smithi114 ceph-mon[34140]: osdmap e57: 10 total, 9 up, 10 in 2024-09-21T03:35:32.711 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:35:32 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-21T03:35:32.711 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:35:32 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-21T03:35:32.811 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:35:32 smithi089 ceph-mon[30928]: Detected new or changed devices on smithi089 2024-09-21T03:35:32.811 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:35:32 smithi089 ceph-mon[30928]: Adjusting osd_memory_target on smithi089 to 4271M 2024-09-21T03:35:32.811 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:35:32 smithi089 ceph-mon[30928]: from='osd.9 [v2:172.21.15.89:6826/1730253068,v1:172.21.15.89:6827/1730253068]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi089", "root=default"]}]': finished 2024-09-21T03:35:32.811 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:35:32 smithi089 ceph-mon[30928]: osdmap e57: 10 total, 9 up, 10 in 2024-09-21T03:35:32.811 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:35:32 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-21T03:35:32.811 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:35:32 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-21T03:35:33.612 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:35:33 smithi114 ceph-mon[34140]: purged_snaps scrub starts 2024-09-21T03:35:33.612 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:35:33 smithi114 ceph-mon[34140]: purged_snaps scrub ok 2024-09-21T03:35:33.612 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:35:33 smithi114 ceph-mon[34140]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 804 GiB / 805 GiB avail 2024-09-21T03:35:33.612 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:35:33 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-21T03:35:33.612 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:35:33 smithi114 ceph-mon[34140]: osd.9 [v2:172.21.15.89:6826/1730253068,v1:172.21.15.89:6827/1730253068] boot 2024-09-21T03:35:33.612 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:35:33 smithi114 ceph-mon[34140]: osdmap e58: 10 total, 10 up, 10 in 2024-09-21T03:35:33.613 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:35:33 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-21T03:35:33.613 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:35:33 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T03:35:33.613 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:35:33 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T03:35:33.613 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:35:33 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:35:33.655 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:35:33 smithi110 ceph-mon[34000]: purged_snaps scrub starts 2024-09-21T03:35:33.655 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:35:33 smithi110 ceph-mon[34000]: purged_snaps scrub ok 2024-09-21T03:35:33.655 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:35:33 smithi110 ceph-mon[34000]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 804 GiB / 805 GiB avail 2024-09-21T03:35:33.655 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:35:33 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-21T03:35:33.655 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:35:33 smithi110 ceph-mon[34000]: osd.9 [v2:172.21.15.89:6826/1730253068,v1:172.21.15.89:6827/1730253068] boot 2024-09-21T03:35:33.655 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:35:33 smithi110 ceph-mon[34000]: osdmap e58: 10 total, 10 up, 10 in 2024-09-21T03:35:33.655 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:35:33 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-21T03:35:33.655 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:35:33 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T03:35:33.655 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:35:33 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T03:35:33.655 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:35:33 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:35:33.811 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:35:33 smithi089 ceph-mon[30928]: purged_snaps scrub starts 2024-09-21T03:35:33.811 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:35:33 smithi089 ceph-mon[30928]: purged_snaps scrub ok 2024-09-21T03:35:33.811 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:35:33 smithi089 ceph-mon[30928]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 804 GiB / 805 GiB avail 2024-09-21T03:35:33.811 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:35:33 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-21T03:35:33.811 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:35:33 smithi089 ceph-mon[30928]: osd.9 [v2:172.21.15.89:6826/1730253068,v1:172.21.15.89:6827/1730253068] boot 2024-09-21T03:35:33.811 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:35:33 smithi089 ceph-mon[30928]: osdmap e58: 10 total, 10 up, 10 in 2024-09-21T03:35:33.811 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:35:33 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-21T03:35:33.812 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:35:33 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T03:35:33.812 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:35:33 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T03:35:33.812 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:35:33 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:35:34.681 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:35:34 smithi110 ceph-mon[34000]: from='client.24500 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi110:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:35:34.681 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:35:34 smithi110 ceph-mon[34000]: osdmap e59: 10 total, 10 up, 10 in 2024-09-21T03:35:34.711 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:35:34 smithi114 ceph-mon[34140]: from='client.24500 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi110:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:35:34.711 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:35:34 smithi114 ceph-mon[34140]: osdmap e59: 10 total, 10 up, 10 in 2024-09-21T03:35:34.811 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:35:34 smithi089 ceph-mon[30928]: from='client.24500 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi110:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:35:34.811 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:35:34 smithi089 ceph-mon[30928]: osdmap e59: 10 total, 10 up, 10 in 2024-09-21T03:35:35.644 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:35:35 smithi110 ceph-mon[34000]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 894 GiB / 894 GiB avail 2024-09-21T03:35:35.644 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:35:35 smithi110 ceph-mon[34000]: from='client.? 172.21.15.110:0/2730092607' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "01390499-2a4d-4188-b57a-fcb10c668e2c"} : dispatch 2024-09-21T03:35:35.644 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:35:35 smithi110 ceph-mon[34000]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "01390499-2a4d-4188-b57a-fcb10c668e2c"} : dispatch 2024-09-21T03:35:35.644 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:35:35 smithi110 ceph-mon[34000]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "01390499-2a4d-4188-b57a-fcb10c668e2c"}]': finished 2024-09-21T03:35:35.644 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:35:35 smithi110 ceph-mon[34000]: osdmap e60: 11 total, 10 up, 11 in 2024-09-21T03:35:35.644 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:35:35 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-21T03:35:35.711 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:35:35 smithi114 ceph-mon[34140]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 894 GiB / 894 GiB avail 2024-09-21T03:35:35.711 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:35:35 smithi114 ceph-mon[34140]: from='client.? 172.21.15.110:0/2730092607' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "01390499-2a4d-4188-b57a-fcb10c668e2c"} : dispatch 2024-09-21T03:35:35.711 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:35:35 smithi114 ceph-mon[34140]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "01390499-2a4d-4188-b57a-fcb10c668e2c"} : dispatch 2024-09-21T03:35:35.711 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:35:35 smithi114 ceph-mon[34140]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "01390499-2a4d-4188-b57a-fcb10c668e2c"}]': finished 2024-09-21T03:35:35.711 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:35:35 smithi114 ceph-mon[34140]: osdmap e60: 11 total, 10 up, 11 in 2024-09-21T03:35:35.711 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:35:35 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-21T03:35:35.811 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:35:35 smithi089 ceph-mon[30928]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 894 GiB / 894 GiB avail 2024-09-21T03:35:35.811 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:35:35 smithi089 ceph-mon[30928]: from='client.? 172.21.15.110:0/2730092607' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "01390499-2a4d-4188-b57a-fcb10c668e2c"} : dispatch 2024-09-21T03:35:35.811 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:35:35 smithi089 ceph-mon[30928]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "01390499-2a4d-4188-b57a-fcb10c668e2c"} : dispatch 2024-09-21T03:35:35.811 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:35:35 smithi089 ceph-mon[30928]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "01390499-2a4d-4188-b57a-fcb10c668e2c"}]': finished 2024-09-21T03:35:35.811 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:35:35 smithi089 ceph-mon[30928]: osdmap e60: 11 total, 10 up, 11 in 2024-09-21T03:35:35.812 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:35:35 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-21T03:35:36.710 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:35:36 smithi114 ceph-mon[34140]: from='client.? 172.21.15.110:0/1710142037' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T03:35:36.750 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:35:36 smithi110 ceph-mon[34000]: from='client.? 172.21.15.110:0/1710142037' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T03:35:36.811 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:35:36 smithi089 ceph-mon[30928]: from='client.? 172.21.15.110:0/1710142037' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T03:35:37.710 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:35:37 smithi114 ceph-mon[34140]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 894 GiB / 894 GiB avail 2024-09-21T03:35:37.750 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:35:37 smithi110 ceph-mon[34000]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 894 GiB / 894 GiB avail 2024-09-21T03:35:37.811 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:35:37 smithi089 ceph-mon[30928]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 894 GiB / 894 GiB avail 2024-09-21T03:35:39.710 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:35:39 smithi114 ceph-mon[34140]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 894 GiB / 894 GiB avail 2024-09-21T03:35:39.750 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:35:39 smithi110 ceph-mon[34000]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 894 GiB / 894 GiB avail 2024-09-21T03:35:39.811 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:35:39 smithi089 ceph-mon[30928]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 894 GiB / 894 GiB avail 2024-09-21T03:35:41.520 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:35:41 smithi110 ceph-mon[34000]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 894 GiB / 894 GiB avail 2024-09-21T03:35:41.711 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:35:41 smithi114 ceph-mon[34140]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 894 GiB / 894 GiB avail 2024-09-21T03:35:41.811 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:35:41 smithi089 ceph-mon[30928]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 894 GiB / 894 GiB avail 2024-09-21T03:35:43.500 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:35:43 smithi110 ceph-mon[34000]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 894 GiB / 894 GiB avail 2024-09-21T03:35:43.710 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:35:43 smithi114 ceph-mon[34140]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 894 GiB / 894 GiB avail 2024-09-21T03:35:43.811 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:35:43 smithi089 ceph-mon[30928]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 894 GiB / 894 GiB avail 2024-09-21T03:35:44.500 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:35:44 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.10"} : dispatch 2024-09-21T03:35:44.500 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:35:44 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:35:44.711 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:35:44 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.10"} : dispatch 2024-09-21T03:35:44.711 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:35:44 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:35:44.811 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:35:44 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.10"} : dispatch 2024-09-21T03:35:44.811 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:35:44 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:35:45.570 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:35:45 smithi110 ceph-mon[34000]: Deploying daemon osd.10 on smithi110 2024-09-21T03:35:45.571 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:35:45 smithi110 ceph-mon[34000]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 894 GiB / 894 GiB avail 2024-09-21T03:35:45.711 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:35:45 smithi114 ceph-mon[34140]: Deploying daemon osd.10 on smithi110 2024-09-21T03:35:45.711 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:35:45 smithi114 ceph-mon[34140]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 894 GiB / 894 GiB avail 2024-09-21T03:35:45.811 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:35:45 smithi089 ceph-mon[30928]: Deploying daemon osd.10 on smithi110 2024-09-21T03:35:45.811 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:35:45 smithi089 ceph-mon[30928]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 894 GiB / 894 GiB avail 2024-09-21T03:35:47.711 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:35:47 smithi114 ceph-mon[34140]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 894 GiB / 894 GiB avail 2024-09-21T03:35:47.750 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:35:47 smithi110 ceph-mon[34000]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 894 GiB / 894 GiB avail 2024-09-21T03:35:47.811 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:35:47 smithi089 ceph-mon[30928]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 894 GiB / 894 GiB avail 2024-09-21T03:35:49.710 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:35:49 smithi114 ceph-mon[34140]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 894 GiB / 894 GiB avail 2024-09-21T03:35:49.771 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:35:49 smithi110 ceph-mon[34000]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 894 GiB / 894 GiB avail 2024-09-21T03:35:49.777 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:35:49 smithi089 ceph-mon[30928]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 894 GiB / 894 GiB avail 2024-09-21T03:35:50.561 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:35:50 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:35:50.561 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:35:50 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:35:50.561 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:35:50 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:35:50.710 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:35:50 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:35:50.711 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:35:50 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:35:50.711 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:35:50 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:35:50.750 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:35:50 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:35:50.750 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:35:50 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:35:50.750 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:35:50 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:35:51.711 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:35:51 smithi114 ceph-mon[34140]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 894 GiB / 894 GiB avail 2024-09-21T03:35:51.720 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:35:51 smithi110 ceph-mon[34000]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 894 GiB / 894 GiB avail 2024-09-21T03:35:51.811 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:35:51 smithi089 ceph-mon[30928]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 894 GiB / 894 GiB avail 2024-09-21T03:35:53.028 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T03:35:53.025+0000 7f8f1cff9640 1 -- 172.21.15.110:0/4130813265 <== mgr.14152 v2:172.21.15.89:6800/1591532358 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7f8f20063800 con 0x7f8f0c0b6510 2024-09-21T03:35:53.029 INFO:teuthology.orchestra.run.smithi110.stdout:Created osd(s) 10 on host 'smithi110' 2024-09-21T03:35:53.031 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T03:35:53.027+0000 7f8f255b1640 1 -- 172.21.15.110:0/4130813265 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f8f0c0b6510 msgr2=0x7f8f0c0b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:35:53.031 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T03:35:53.027+0000 7f8f255b1640 1 --2- 172.21.15.110:0/4130813265 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f8f0c0b6510 0x7f8f0c0b8a20 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f8f100057c0 tx=0x7f8f10000bc0 comp rx=0 tx=0).stop 2024-09-21T03:35:53.031 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T03:35:53.027+0000 7f8f255b1640 1 -- 172.21.15.110:0/4130813265 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8f20104810 msgr2=0x7f8f2019a590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:35:53.031 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T03:35:53.027+0000 7f8f255b1640 1 --2- 172.21.15.110:0/4130813265 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8f20104810 0x7f8f2019a590 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f8f0800d770 tx=0x7f8f0800dc30 comp rx=0 tx=0).stop 2024-09-21T03:35:53.031 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T03:35:53.027+0000 7f8f255b1640 1 -- 172.21.15.110:0/4130813265 shutdown_connections 2024-09-21T03:35:53.031 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T03:35:53.027+0000 7f8f255b1640 1 --2- 172.21.15.110:0/4130813265 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f8f0c0b6510 0x7f8f0c0b8a20 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:53.031 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T03:35:53.027+0000 7f8f255b1640 1 --2- 172.21.15.110:0/4130813265 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8f20104810 0x7f8f2019a590 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:53.031 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T03:35:53.027+0000 7f8f255b1640 1 --2- 172.21.15.110:0/4130813265 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8f201051b0 0x7f8f2019b130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:53.031 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T03:35:53.027+0000 7f8f255b1640 1 --2- 172.21.15.110:0/4130813265 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8f20103780 0x7f8f20199f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:53.031 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T03:35:53.027+0000 7f8f255b1640 1 -- 172.21.15.110:0/4130813265 >> 172.21.15.110:0/4130813265 conn(0x7f8f200ff0f0 msgr2=0x7f8f20107b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:35:53.032 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T03:35:53.027+0000 7f8f255b1640 1 -- 172.21.15.110:0/4130813265 shutdown_connections 2024-09-21T03:35:53.032 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T03:35:53.027+0000 7f8f255b1640 1 -- 172.21.15.110:0/4130813265 wait complete. 2024-09-21T03:35:53.561 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:35:53 smithi089 ceph-mon[30928]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 894 GiB / 894 GiB avail 2024-09-21T03:35:53.561 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:35:53 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:35:53.561 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:35:53 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:35:53.561 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:35:53 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:35:53.561 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:35:53 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:35:53.561 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:35:53 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:35:53.561 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:35:53 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:35:53.561 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:35:53 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:35:53.562 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:35:53 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:35:53.641 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:35:53 smithi110 ceph-mon[34000]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 894 GiB / 894 GiB avail 2024-09-21T03:35:53.641 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:35:53 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:35:53.641 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:35:53 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:35:53.641 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:35:53 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:35:53.641 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:35:53 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:35:53.641 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:35:53 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:35:53.641 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:35:53 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:35:53.642 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:35:53 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:35:53.642 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:35:53 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:35:53.666 DEBUG:teuthology.orchestra.run.smithi110:osd.10> sudo journalctl -f -n 0 -u ceph-7c374098-77c9-11ef-baf5-efdc52797490@osd.10.service 2024-09-21T03:35:53.673 INFO:tasks.cephadm:Deploying osd.11 on smithi114 with /dev/vg_nvme/lv_1... 2024-09-21T03:35:53.674 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7c374098-77c9-11ef-baf5-efdc52797490 -- lvm zap /dev/vg_nvme/lv_1 2024-09-21T03:35:53.707 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:35:53 smithi114 ceph-mon[34140]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 894 GiB / 894 GiB avail 2024-09-21T03:35:53.707 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:35:53 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:35:53.708 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:35:53 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:35:53.708 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:35:53 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:35:53.708 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:35:53 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:35:53.708 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:35:53 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:35:53.708 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:35:53 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:35:53.708 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:35:53 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:35:53.708 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:35:53 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:35:53.939 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/7c374098-77c9-11ef-baf5-efdc52797490/mon.c/config 2024-09-21T03:35:55.461 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:35:55 smithi114 ceph-mon[34140]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 894 GiB / 894 GiB avail 2024-09-21T03:35:55.490 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:35:55 smithi110 ceph-mon[34000]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 894 GiB / 894 GiB avail 2024-09-21T03:35:55.561 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:35:55 smithi089 ceph-mon[30928]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 894 GiB / 894 GiB avail 2024-09-21T03:35:55.750 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 21 03:35:55 smithi110 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-10[57089]: 2024-09-21T03:35:55.487+0000 7f9d52461640 -1 osd.10 0 log_to_monitors true 2024-09-21T03:35:56.441 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:35:56 smithi110 ceph-mon[34000]: from='osd.10 [v2:172.21.15.110:6824/2904016117,v1:172.21.15.110:6825/2904016117]' entity='osd.10' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]} : dispatch 2024-09-21T03:35:56.441 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:35:56 smithi110 ceph-mon[34000]: from='osd.10 ' entity='osd.10' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]} : dispatch 2024-09-21T03:35:56.561 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:35:56 smithi089 ceph-mon[30928]: from='osd.10 [v2:172.21.15.110:6824/2904016117,v1:172.21.15.110:6825/2904016117]' entity='osd.10' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]} : dispatch 2024-09-21T03:35:56.561 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:35:56 smithi089 ceph-mon[30928]: from='osd.10 ' entity='osd.10' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]} : dispatch 2024-09-21T03:35:56.646 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:35:56 smithi114 ceph-mon[34140]: from='osd.10 [v2:172.21.15.110:6824/2904016117,v1:172.21.15.110:6825/2904016117]' entity='osd.10' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]} : dispatch 2024-09-21T03:35:56.646 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:35:56 smithi114 ceph-mon[34140]: from='osd.10 ' entity='osd.10' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]} : dispatch 2024-09-21T03:35:56.649 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-21T03:35:56.676 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7c374098-77c9-11ef-baf5-efdc52797490 -- ceph orch daemon add osd smithi114:vg_nvme/lv_1 2024-09-21T03:35:56.968 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/7c374098-77c9-11ef-baf5-efdc52797490/mon.c/config 2024-09-21T03:35:57.385 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:35:57 smithi114 ceph-mon[34140]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 894 GiB / 894 GiB avail 2024-09-21T03:35:57.385 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:35:57 smithi114 ceph-mon[34140]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-09-21T03:35:57.386 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:35:57 smithi114 ceph-mon[34140]: from='osd.10 [v2:172.21.15.110:6824/2904016117,v1:172.21.15.110:6825/2904016117]' entity='osd.10' cmd={"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi110", "root=default"]} : dispatch 2024-09-21T03:35:57.386 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:35:57 smithi114 ceph-mon[34140]: osdmap e61: 11 total, 10 up, 11 in 2024-09-21T03:35:57.386 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:35:57 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-21T03:35:57.386 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:35:57 smithi114 ceph-mon[34140]: from='osd.10 ' entity='osd.10' cmd={"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi110", "root=default"]} : dispatch 2024-09-21T03:35:57.386 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:35:57 smithi114 ceph-mon[34140]: Detected new or changed devices on smithi110 2024-09-21T03:35:57.386 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:35:57 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:35:57.386 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:35:57 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:35:57.386 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:35:57 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-21T03:35:57.386 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:35:57 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-21T03:35:57.387 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:35:57 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-09-21T03:35:57.387 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:35:57 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"} : dispatch 2024-09-21T03:35:57.387 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:35:57 smithi114 ceph-mon[34140]: Adjusting osd_memory_target on smithi110 to 4272M 2024-09-21T03:35:57.387 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:35:57 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:35:57.387 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:35:57 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:35:57.387 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:35:57 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:35:57.387 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:35:57 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:35:57.486 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T03:35:57.478+0000 7f1e7a727640 1 -- 172.21.15.114:0/3587033295 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1e741047e0 msgr2=0x7f1e74104bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:35:57.486 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T03:35:57.478+0000 7f1e7a727640 1 --2- 172.21.15.114:0/3587033295 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1e741047e0 0x7f1e74104bf0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f1e68009900 tx=0x7f1e6802a480 comp rx=0 tx=0).stop 2024-09-21T03:35:57.486 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T03:35:57.478+0000 7f1e7a727640 1 -- 172.21.15.114:0/3587033295 shutdown_connections 2024-09-21T03:35:57.487 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T03:35:57.478+0000 7f1e7a727640 1 --2- 172.21.15.114:0/3587033295 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1e74105180 0x7f1e7410bb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:57.487 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T03:35:57.478+0000 7f1e7a727640 1 --2- 172.21.15.114:0/3587033295 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1e741047e0 0x7f1e74104bf0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:57.487 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T03:35:57.478+0000 7f1e7a727640 1 --2- 172.21.15.114:0/3587033295 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1e74103750 0x7f1e74103bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:57.487 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T03:35:57.478+0000 7f1e7a727640 1 -- 172.21.15.114:0/3587033295 >> 172.21.15.114:0/3587033295 conn(0x7f1e740ff0c0 msgr2=0x7f1e74101570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:35:57.487 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T03:35:57.478+0000 7f1e7a727640 1 -- 172.21.15.114:0/3587033295 shutdown_connections 2024-09-21T03:35:57.487 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T03:35:57.478+0000 7f1e7a727640 1 -- 172.21.15.114:0/3587033295 wait complete. 2024-09-21T03:35:57.487 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T03:35:57.478+0000 7f1e7a727640 1 Processor -- start 2024-09-21T03:35:57.487 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T03:35:57.478+0000 7f1e7a727640 1 -- start start 2024-09-21T03:35:57.487 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T03:35:57.479+0000 7f1e7a727640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1e74103750 0x7f1e74199f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:57.487 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T03:35:57.479+0000 7f1e7a727640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1e741047e0 0x7f1e7419a570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:57.487 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T03:35:57.479+0000 7f1e7a727640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1e74105180 0x7f1e7419b110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:57.487 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T03:35:57.479+0000 7f1e7a727640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f1e7410f250 con 0x7f1e74103750 2024-09-21T03:35:57.488 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T03:35:57.479+0000 7f1e7a727640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f1e7410f0d0 con 0x7f1e741047e0 2024-09-21T03:35:57.488 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T03:35:57.479+0000 7f1e7a727640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f1e7410f3d0 con 0x7f1e74105180 2024-09-21T03:35:57.488 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T03:35:57.479+0000 7f1e78cc4640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1e74105180 0x7f1e7419b110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:57.488 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T03:35:57.479+0000 7f1e78cc4640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1e74105180 0x7f1e7419b110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:57560/0 (socket says 172.21.15.114:57560) 2024-09-21T03:35:57.488 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T03:35:57.479+0000 7f1e78cc4640 1 -- 172.21.15.114:0/2847500404 learned_addr learned my addr 172.21.15.114:0/2847500404 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T03:35:57.488 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T03:35:57.479+0000 7f1e737fe640 1 --2- 172.21.15.114:0/2847500404 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1e741047e0 0x7f1e7419a570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:57.488 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T03:35:57.479+0000 7f1e73fff640 1 --2- 172.21.15.114:0/2847500404 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1e74103750 0x7f1e74199f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:57.488 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T03:35:57.479+0000 7f1e78cc4640 1 -- 172.21.15.114:0/2847500404 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1e741047e0 msgr2=0x7f1e7419a570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:35:57.488 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T03:35:57.479+0000 7f1e78cc4640 1 --2- 172.21.15.114:0/2847500404 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1e741047e0 0x7f1e7419a570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:57.488 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T03:35:57.479+0000 7f1e78cc4640 1 -- 172.21.15.114:0/2847500404 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1e74103750 msgr2=0x7f1e74199f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:35:57.488 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T03:35:57.479+0000 7f1e78cc4640 1 --2- 172.21.15.114:0/2847500404 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1e74103750 0x7f1e74199f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:57.488 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T03:35:57.479+0000 7f1e78cc4640 1 -- 172.21.15.114:0/2847500404 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e7419b930 con 0x7f1e74105180 2024-09-21T03:35:57.488 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T03:35:57.479+0000 7f1e78cc4640 1 --2- 172.21.15.114:0/2847500404 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1e74105180 0x7f1e7419b110 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f1e6c00eb10 tx=0x7f1e6c00c2d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:35:57.488 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T03:35:57.479+0000 7f1e717fa640 1 -- 172.21.15.114:0/2847500404 <== mon.1 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e6c004b20 con 0x7f1e74105180 2024-09-21T03:35:57.488 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T03:35:57.480+0000 7f1e7a727640 1 -- 172.21.15.114:0/2847500404 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1e7419bc20 con 0x7f1e74105180 2024-09-21T03:35:57.489 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T03:35:57.480+0000 7f1e717fa640 1 -- 172.21.15.114:0/2847500404 <== mon.1 v2:172.21.15.114:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1e6c004cc0 con 0x7f1e74105180 2024-09-21T03:35:57.489 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T03:35:57.480+0000 7f1e717fa640 1 -- 172.21.15.114:0/2847500404 <== mon.1 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e6c010690 con 0x7f1e74105180 2024-09-21T03:35:57.489 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T03:35:57.480+0000 7f1e7a727640 1 -- 172.21.15.114:0/2847500404 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1e741a2320 con 0x7f1e74105180 2024-09-21T03:35:57.489 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T03:35:57.481+0000 7f1e7a727640 1 -- 172.21.15.114:0/2847500404 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1e7410a930 con 0x7f1e74105180 2024-09-21T03:35:57.489 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T03:35:57.482+0000 7f1e717fa640 1 -- 172.21.15.114:0/2847500404 <== mon.1 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 17) ==== 154512+0+0 (secure 0 0 0) 0x7f1e6c005000 con 0x7f1e74105180 2024-09-21T03:35:57.489 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T03:35:57.483+0000 7f1e717fa640 1 --2- 172.21.15.114:0/2847500404 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f1e5c0b6510 0x7f1e5c0b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:57.489 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T03:35:57.484+0000 7f1e717fa640 1 -- 172.21.15.114:0/2847500404 <== mon.1 v2:172.21.15.114:3300/0 5 ==== osd_map(62..62 src has 1..62) ==== 6414+0+0 (secure 0 0 0) 0x7f1e6c0e73c0 con 0x7f1e74105180 2024-09-21T03:35:57.489 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T03:35:57.484+0000 7f1e73fff640 1 --2- 172.21.15.114:0/2847500404 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f1e5c0b6510 0x7f1e5c0b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:57.489 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T03:35:57.485+0000 7f1e73fff640 1 --2- 172.21.15.114:0/2847500404 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f1e5c0b6510 0x7f1e5c0b8a20 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f1e64009b60 tx=0x7f1e64005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:35:57.489 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T03:35:57.487+0000 7f1e717fa640 1 -- 172.21.15.114:0/2847500404 <== mon.1 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f1e6c056710 con 0x7f1e74105180 2024-09-21T03:35:57.561 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:35:57 smithi089 ceph-mon[30928]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 894 GiB / 894 GiB avail 2024-09-21T03:35:57.561 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:35:57 smithi089 ceph-mon[30928]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-09-21T03:35:57.561 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:35:57 smithi089 ceph-mon[30928]: from='osd.10 [v2:172.21.15.110:6824/2904016117,v1:172.21.15.110:6825/2904016117]' entity='osd.10' cmd={"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi110", "root=default"]} : dispatch 2024-09-21T03:35:57.561 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:35:57 smithi089 ceph-mon[30928]: osdmap e61: 11 total, 10 up, 11 in 2024-09-21T03:35:57.561 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:35:57 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-21T03:35:57.561 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:35:57 smithi089 ceph-mon[30928]: from='osd.10 ' entity='osd.10' cmd={"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi110", "root=default"]} : dispatch 2024-09-21T03:35:57.561 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:35:57 smithi089 ceph-mon[30928]: Detected new or changed devices on smithi110 2024-09-21T03:35:57.561 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:35:57 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:35:57.561 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:35:57 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:35:57.562 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:35:57 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-21T03:35:57.562 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:35:57 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-21T03:35:57.562 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:35:57 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-09-21T03:35:57.562 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:35:57 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"} : dispatch 2024-09-21T03:35:57.562 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:35:57 smithi089 ceph-mon[30928]: Adjusting osd_memory_target on smithi110 to 4272M 2024-09-21T03:35:57.562 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:35:57 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:35:57.562 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:35:57 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:35:57.562 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:35:57 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:35:57.562 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:35:57 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:35:57.678 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T03:35:57.676+0000 7f1e7a727640 1 -- 172.21.15.114:0/2847500404 --> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi114:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f1e74063800 con 0x7f1e5c0b6510 2024-09-21T03:35:57.745 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:35:57 smithi110 ceph-mon[34000]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 894 GiB / 894 GiB avail 2024-09-21T03:35:57.746 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:35:57 smithi110 ceph-mon[34000]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-09-21T03:35:57.746 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:35:57 smithi110 ceph-mon[34000]: from='osd.10 [v2:172.21.15.110:6824/2904016117,v1:172.21.15.110:6825/2904016117]' entity='osd.10' cmd={"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi110", "root=default"]} : dispatch 2024-09-21T03:35:57.746 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:35:57 smithi110 ceph-mon[34000]: osdmap e61: 11 total, 10 up, 11 in 2024-09-21T03:35:57.746 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:35:57 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-21T03:35:57.746 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:35:57 smithi110 ceph-mon[34000]: from='osd.10 ' entity='osd.10' cmd={"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi110", "root=default"]} : dispatch 2024-09-21T03:35:57.746 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:35:57 smithi110 ceph-mon[34000]: Detected new or changed devices on smithi110 2024-09-21T03:35:57.746 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:35:57 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:35:57.746 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:35:57 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:35:57.746 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:35:57 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-21T03:35:57.746 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:35:57 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-21T03:35:57.746 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:35:57 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-09-21T03:35:57.746 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:35:57 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"} : dispatch 2024-09-21T03:35:57.747 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:35:57 smithi110 ceph-mon[34000]: Adjusting osd_memory_target on smithi110 to 4272M 2024-09-21T03:35:57.747 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:35:57 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:35:57.747 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:35:57 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:35:57.747 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:35:57 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:35:57.747 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:35:57 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:35:58.000 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 21 03:35:57 smithi110 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-10[57089]: 2024-09-21T03:35:57.743+0000 7f9d4e3e3640 -1 osd.10 0 waiting for initial osdmap 2024-09-21T03:35:58.000 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 21 03:35:57 smithi110 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-10[57089]: 2024-09-21T03:35:57.759+0000 7f9d461d9640 -1 osd.10 62 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-21T03:35:58.711 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:35:58 smithi114 ceph-mon[34140]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi110", "root=default"]}]': finished 2024-09-21T03:35:58.711 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:35:58 smithi114 ceph-mon[34140]: osdmap e62: 11 total, 10 up, 11 in 2024-09-21T03:35:58.711 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:35:58 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-21T03:35:58.711 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:35:58 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-21T03:35:58.711 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:35:58 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T03:35:58.711 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:35:58 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T03:35:58.711 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:35:58 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:35:58.711 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:35:58 smithi114 ceph-mon[34140]: from='osd.10 ' entity='osd.10' 2024-09-21T03:35:58.750 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:35:58 smithi110 ceph-mon[34000]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi110", "root=default"]}]': finished 2024-09-21T03:35:58.750 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:35:58 smithi110 ceph-mon[34000]: osdmap e62: 11 total, 10 up, 11 in 2024-09-21T03:35:58.750 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:35:58 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-21T03:35:58.750 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:35:58 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-21T03:35:58.750 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:35:58 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T03:35:58.750 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:35:58 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T03:35:58.750 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:35:58 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:35:58.750 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:35:58 smithi110 ceph-mon[34000]: from='osd.10 ' entity='osd.10' 2024-09-21T03:35:58.811 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:35:58 smithi089 ceph-mon[30928]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi110", "root=default"]}]': finished 2024-09-21T03:35:58.811 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:35:58 smithi089 ceph-mon[30928]: osdmap e62: 11 total, 10 up, 11 in 2024-09-21T03:35:58.811 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:35:58 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-21T03:35:58.811 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:35:58 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-21T03:35:58.811 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:35:58 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T03:35:58.811 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:35:58 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T03:35:58.811 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:35:58 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:35:58.811 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:35:58 smithi089 ceph-mon[30928]: from='osd.10 ' entity='osd.10' 2024-09-21T03:35:59.597 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:35:59 smithi114 ceph-mon[34140]: purged_snaps scrub starts 2024-09-21T03:35:59.597 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:35:59 smithi114 ceph-mon[34140]: purged_snaps scrub ok 2024-09-21T03:35:59.597 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:35:59 smithi114 ceph-mon[34140]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 894 GiB / 894 GiB avail 2024-09-21T03:35:59.597 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:35:59 smithi114 ceph-mon[34140]: from='client.24508 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi114:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:35:59.597 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:35:59 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-21T03:35:59.597 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:35:59 smithi114 ceph-mon[34140]: osd.10 [v2:172.21.15.110:6824/2904016117,v1:172.21.15.110:6825/2904016117] boot 2024-09-21T03:35:59.597 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:35:59 smithi114 ceph-mon[34140]: osdmap e63: 11 total, 11 up, 11 in 2024-09-21T03:35:59.598 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:35:59 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-21T03:35:59.750 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:35:59 smithi110 ceph-mon[34000]: purged_snaps scrub starts 2024-09-21T03:35:59.750 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:35:59 smithi110 ceph-mon[34000]: purged_snaps scrub ok 2024-09-21T03:35:59.750 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:35:59 smithi110 ceph-mon[34000]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 894 GiB / 894 GiB avail 2024-09-21T03:35:59.750 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:35:59 smithi110 ceph-mon[34000]: from='client.24508 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi114:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:35:59.750 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:35:59 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-21T03:35:59.750 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:35:59 smithi110 ceph-mon[34000]: osd.10 [v2:172.21.15.110:6824/2904016117,v1:172.21.15.110:6825/2904016117] boot 2024-09-21T03:35:59.750 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:35:59 smithi110 ceph-mon[34000]: osdmap e63: 11 total, 11 up, 11 in 2024-09-21T03:35:59.750 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:35:59 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-21T03:35:59.811 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:35:59 smithi089 ceph-mon[30928]: purged_snaps scrub starts 2024-09-21T03:35:59.811 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:35:59 smithi089 ceph-mon[30928]: purged_snaps scrub ok 2024-09-21T03:35:59.811 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:35:59 smithi089 ceph-mon[30928]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 894 GiB / 894 GiB avail 2024-09-21T03:35:59.811 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:35:59 smithi089 ceph-mon[30928]: from='client.24508 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi114:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:35:59.811 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:35:59 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-21T03:35:59.811 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:35:59 smithi089 ceph-mon[30928]: osd.10 [v2:172.21.15.110:6824/2904016117,v1:172.21.15.110:6825/2904016117] boot 2024-09-21T03:35:59.812 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:35:59 smithi089 ceph-mon[30928]: osdmap e63: 11 total, 11 up, 11 in 2024-09-21T03:35:59.812 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:35:59 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-21T03:36:00.711 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:36:00 smithi114 ceph-mon[34140]: osdmap e64: 11 total, 11 up, 11 in 2024-09-21T03:36:00.711 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:36:00 smithi114 ceph-mon[34140]: from='client.? 172.21.15.114:0/4042019499' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "c3c0f38a-c3d3-415d-a9f8-875d5b756e35"} : dispatch 2024-09-21T03:36:00.711 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:36:00 smithi114 ceph-mon[34140]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "c3c0f38a-c3d3-415d-a9f8-875d5b756e35"} : dispatch 2024-09-21T03:36:00.711 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:36:00 smithi114 ceph-mon[34140]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c3c0f38a-c3d3-415d-a9f8-875d5b756e35"}]': finished 2024-09-21T03:36:00.711 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:36:00 smithi114 ceph-mon[34140]: osdmap e65: 12 total, 11 up, 12 in 2024-09-21T03:36:00.711 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:36:00 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-21T03:36:00.750 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:36:00 smithi110 ceph-mon[34000]: osdmap e64: 11 total, 11 up, 11 in 2024-09-21T03:36:00.750 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:36:00 smithi110 ceph-mon[34000]: from='client.? 172.21.15.114:0/4042019499' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "c3c0f38a-c3d3-415d-a9f8-875d5b756e35"} : dispatch 2024-09-21T03:36:00.750 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:36:00 smithi110 ceph-mon[34000]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "c3c0f38a-c3d3-415d-a9f8-875d5b756e35"} : dispatch 2024-09-21T03:36:00.750 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:36:00 smithi110 ceph-mon[34000]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c3c0f38a-c3d3-415d-a9f8-875d5b756e35"}]': finished 2024-09-21T03:36:00.750 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:36:00 smithi110 ceph-mon[34000]: osdmap e65: 12 total, 11 up, 12 in 2024-09-21T03:36:00.750 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:36:00 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-21T03:36:00.811 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:36:00 smithi089 ceph-mon[30928]: osdmap e64: 11 total, 11 up, 11 in 2024-09-21T03:36:00.811 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:36:00 smithi089 ceph-mon[30928]: from='client.? 172.21.15.114:0/4042019499' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "c3c0f38a-c3d3-415d-a9f8-875d5b756e35"} : dispatch 2024-09-21T03:36:00.811 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:36:00 smithi089 ceph-mon[30928]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "c3c0f38a-c3d3-415d-a9f8-875d5b756e35"} : dispatch 2024-09-21T03:36:00.811 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:36:00 smithi089 ceph-mon[30928]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c3c0f38a-c3d3-415d-a9f8-875d5b756e35"}]': finished 2024-09-21T03:36:00.811 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:36:00 smithi089 ceph-mon[30928]: osdmap e65: 12 total, 11 up, 12 in 2024-09-21T03:36:00.811 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:36:00 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-21T03:36:01.711 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:36:01 smithi114 ceph-mon[34140]: pgmap v238: 1 pgs: 1 remapped+peering; 577 KiB data, 334 MiB used, 983 GiB / 983 GiB avail 2024-09-21T03:36:01.711 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:36:01 smithi114 ceph-mon[34140]: from='client.? 172.21.15.114:0/3885292025' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T03:36:01.750 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:36:01 smithi110 ceph-mon[34000]: pgmap v238: 1 pgs: 1 remapped+peering; 577 KiB data, 334 MiB used, 983 GiB / 983 GiB avail 2024-09-21T03:36:01.750 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:36:01 smithi110 ceph-mon[34000]: from='client.? 172.21.15.114:0/3885292025' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T03:36:01.811 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:36:01 smithi089 ceph-mon[30928]: pgmap v238: 1 pgs: 1 remapped+peering; 577 KiB data, 334 MiB used, 983 GiB / 983 GiB avail 2024-09-21T03:36:01.811 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:36:01 smithi089 ceph-mon[30928]: from='client.? 172.21.15.114:0/3885292025' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T03:36:03.710 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:36:03 smithi114 ceph-mon[34140]: pgmap v240: 1 pgs: 1 remapped+peering; 577 KiB data, 334 MiB used, 983 GiB / 983 GiB avail 2024-09-21T03:36:03.750 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:36:03 smithi110 ceph-mon[34000]: pgmap v240: 1 pgs: 1 remapped+peering; 577 KiB data, 334 MiB used, 983 GiB / 983 GiB avail 2024-09-21T03:36:03.811 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:36:03 smithi089 ceph-mon[30928]: pgmap v240: 1 pgs: 1 remapped+peering; 577 KiB data, 334 MiB used, 983 GiB / 983 GiB avail 2024-09-21T03:36:05.461 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:36:05 smithi114 ceph-mon[34140]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 334 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-21T03:36:05.750 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:36:05 smithi110 ceph-mon[34000]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 334 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-21T03:36:05.811 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:36:05 smithi089 ceph-mon[30928]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 334 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-21T03:36:07.552 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:36:07 smithi114 ceph-mon[34140]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 334 MiB used, 983 GiB / 983 GiB avail; 79 KiB/s, 0 objects/s recovering 2024-09-21T03:36:07.750 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:36:07 smithi110 ceph-mon[34000]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 334 MiB used, 983 GiB / 983 GiB avail; 79 KiB/s, 0 objects/s recovering 2024-09-21T03:36:07.811 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:36:07 smithi089 ceph-mon[30928]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 334 MiB used, 983 GiB / 983 GiB avail; 79 KiB/s, 0 objects/s recovering 2024-09-21T03:36:09.710 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:36:09 smithi114 ceph-mon[34140]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 334 MiB used, 983 GiB / 983 GiB avail; 70 KiB/s, 0 objects/s recovering 2024-09-21T03:36:09.711 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:36:09 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.11"} : dispatch 2024-09-21T03:36:09.711 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:36:09 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:36:09.750 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:36:09 smithi110 ceph-mon[34000]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 334 MiB used, 983 GiB / 983 GiB avail; 70 KiB/s, 0 objects/s recovering 2024-09-21T03:36:09.750 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:36:09 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.11"} : dispatch 2024-09-21T03:36:09.750 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:36:09 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:36:09.811 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:36:09 smithi089 ceph-mon[30928]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 334 MiB used, 983 GiB / 983 GiB avail; 70 KiB/s, 0 objects/s recovering 2024-09-21T03:36:09.811 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:36:09 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.11"} : dispatch 2024-09-21T03:36:09.811 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:36:09 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:36:10.647 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:36:10 smithi114 ceph-mon[34140]: Deploying daemon osd.11 on smithi114 2024-09-21T03:36:10.749 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:36:10 smithi110 ceph-mon[34000]: Deploying daemon osd.11 on smithi114 2024-09-21T03:36:10.811 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:36:10 smithi089 ceph-mon[30928]: Deploying daemon osd.11 on smithi114 2024-09-21T03:36:11.651 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:36:11 smithi114 ceph-mon[34140]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 334 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-21T03:36:11.750 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:36:11 smithi110 ceph-mon[34000]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 334 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-21T03:36:11.811 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:36:11 smithi089 ceph-mon[30928]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 334 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-21T03:36:13.588 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:36:13 smithi114 ceph-mon[34140]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 334 MiB used, 983 GiB / 983 GiB avail; 50 KiB/s, 0 objects/s recovering 2024-09-21T03:36:13.750 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:36:13 smithi110 ceph-mon[34000]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 334 MiB used, 983 GiB / 983 GiB avail; 50 KiB/s, 0 objects/s recovering 2024-09-21T03:36:13.811 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:36:13 smithi089 ceph-mon[30928]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 334 MiB used, 983 GiB / 983 GiB avail; 50 KiB/s, 0 objects/s recovering 2024-09-21T03:36:15.710 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:36:15 smithi114 ceph-mon[34140]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 334 MiB used, 983 GiB / 983 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-21T03:36:15.711 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:36:15 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:36:15.711 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:36:15 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:36:15.711 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:36:15 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:36:15.750 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:36:15 smithi110 ceph-mon[34000]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 334 MiB used, 983 GiB / 983 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-21T03:36:15.750 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:36:15 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:36:15.750 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:36:15 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:36:15.750 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:36:15 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:36:15.811 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:36:15 smithi089 ceph-mon[30928]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 334 MiB used, 983 GiB / 983 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-21T03:36:15.811 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:36:15 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:36:15.811 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:36:15 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:36:15.811 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:36:15 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:36:17.711 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:36:17 smithi114 ceph-mon[34140]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 334 MiB used, 983 GiB / 983 GiB avail 2024-09-21T03:36:17.750 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:36:17 smithi110 ceph-mon[34000]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 334 MiB used, 983 GiB / 983 GiB avail 2024-09-21T03:36:17.811 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:36:17 smithi089 ceph-mon[30928]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 334 MiB used, 983 GiB / 983 GiB avail 2024-09-21T03:36:17.902 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T03:36:17.899+0000 7f1e717fa640 1 -- 172.21.15.114:0/2847500404 <== mgr.14152 v2:172.21.15.89:6800/1591532358 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7f1e74063800 con 0x7f1e5c0b6510 2024-09-21T03:36:17.902 INFO:teuthology.orchestra.run.smithi114.stdout:Created osd(s) 11 on host 'smithi114' 2024-09-21T03:36:17.904 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T03:36:17.901+0000 7f1e7a727640 1 -- 172.21.15.114:0/2847500404 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f1e5c0b6510 msgr2=0x7f1e5c0b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:17.904 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T03:36:17.901+0000 7f1e7a727640 1 --2- 172.21.15.114:0/2847500404 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f1e5c0b6510 0x7f1e5c0b8a20 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f1e64009b60 tx=0x7f1e64005f90 comp rx=0 tx=0).stop 2024-09-21T03:36:17.904 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T03:36:17.901+0000 7f1e7a727640 1 -- 172.21.15.114:0/2847500404 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1e74105180 msgr2=0x7f1e7419b110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:17.904 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T03:36:17.901+0000 7f1e7a727640 1 --2- 172.21.15.114:0/2847500404 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1e74105180 0x7f1e7419b110 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f1e6c00eb10 tx=0x7f1e6c00c2d0 comp rx=0 tx=0).stop 2024-09-21T03:36:17.905 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T03:36:17.901+0000 7f1e7a727640 1 -- 172.21.15.114:0/2847500404 shutdown_connections 2024-09-21T03:36:17.905 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T03:36:17.901+0000 7f1e7a727640 1 --2- 172.21.15.114:0/2847500404 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f1e5c0b6510 0x7f1e5c0b8a20 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:17.905 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T03:36:17.901+0000 7f1e7a727640 1 --2- 172.21.15.114:0/2847500404 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1e741047e0 0x7f1e7419a570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:17.905 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T03:36:17.901+0000 7f1e7a727640 1 --2- 172.21.15.114:0/2847500404 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1e74105180 0x7f1e7419b110 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:17.905 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T03:36:17.901+0000 7f1e7a727640 1 --2- 172.21.15.114:0/2847500404 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1e74103750 0x7f1e74199f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:17.905 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T03:36:17.901+0000 7f1e7a727640 1 -- 172.21.15.114:0/2847500404 >> 172.21.15.114:0/2847500404 conn(0x7f1e740ff0c0 msgr2=0x7f1e74107b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:36:17.905 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T03:36:17.901+0000 7f1e7a727640 1 -- 172.21.15.114:0/2847500404 shutdown_connections 2024-09-21T03:36:17.905 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T03:36:17.902+0000 7f1e7a727640 1 -- 172.21.15.114:0/2847500404 wait complete. 2024-09-21T03:36:18.572 DEBUG:teuthology.orchestra.run.smithi114:osd.11> sudo journalctl -f -n 0 -u ceph-7c374098-77c9-11ef-baf5-efdc52797490@osd.11.service 2024-09-21T03:36:18.574 INFO:tasks.cephadm:Waiting for 12 OSDs to come up... 2024-09-21T03:36:18.574 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7c374098-77c9-11ef-baf5-efdc52797490 -- ceph osd stat -f json 2024-09-21T03:36:18.835 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/7c374098-77c9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:36:19.163 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:36:18 smithi089 ceph-mon[30928]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 334 MiB used, 983 GiB / 983 GiB avail 2024-09-21T03:36:19.163 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:36:18 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:36:19.163 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:36:18 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:36:19.163 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:36:18 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:36:19.163 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:36:18 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:36:19.207 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:36:18 smithi114 ceph-mon[34140]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 334 MiB used, 983 GiB / 983 GiB avail 2024-09-21T03:36:19.207 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:36:18 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:36:19.207 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:36:18 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:36:19.207 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:36:18 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:36:19.207 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:36:18 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:36:19.250 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:36:18 smithi110 ceph-mon[34000]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 334 MiB used, 983 GiB / 983 GiB avail 2024-09-21T03:36:19.250 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:36:18 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:36:19.250 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:36:18 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:36:19.250 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:36:18 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:36:19.250 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:36:18 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:36:19.382 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:19.379+0000 7f950d646640 1 -- 172.21.15.89:0/3749046759 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9508103780 msgr2=0x7f9508103b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:19.382 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:19.379+0000 7f950d646640 1 --2- 172.21.15.89:0/3749046759 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9508103780 0x7f9508103b90 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f94fc009900 tx=0x7f94fc02a480 comp rx=0 tx=0).stop 2024-09-21T03:36:19.382 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:19.379+0000 7f950d646640 1 -- 172.21.15.89:0/3749046759 shutdown_connections 2024-09-21T03:36:19.382 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:19.379+0000 7f950d646640 1 --2- 172.21.15.89:0/3749046759 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9508104120 0x7f950810aac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:19.382 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:19.379+0000 7f950d646640 1 --2- 172.21.15.89:0/3749046759 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9508103780 0x7f9508103b90 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:19.382 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:19.379+0000 7f950d646640 1 --2- 172.21.15.89:0/3749046759 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f95081026f0 0x7f9508102b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:19.383 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:19.379+0000 7f950d646640 1 -- 172.21.15.89:0/3749046759 >> 172.21.15.89:0/3749046759 conn(0x7f95080fe080 msgr2=0x7f9508100510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:36:19.383 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:19.379+0000 7f950d646640 1 -- 172.21.15.89:0/3749046759 shutdown_connections 2024-09-21T03:36:19.383 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:19.379+0000 7f950d646640 1 -- 172.21.15.89:0/3749046759 wait complete. 2024-09-21T03:36:19.383 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:19.380+0000 7f950d646640 1 Processor -- start 2024-09-21T03:36:19.383 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:19.380+0000 7f950d646640 1 -- start start 2024-09-21T03:36:19.385 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:19.380+0000 7f950d646640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f95081026f0 0x7f9508198e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:19.385 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:19.380+0000 7f950d646640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9508103780 0x7f95081994b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:19.385 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:19.380+0000 7f950d646640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9508104120 0x7f950819a050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:19.385 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:19.380+0000 7f950d646640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f950810e180 con 0x7f9508104120 2024-09-21T03:36:19.385 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:19.380+0000 7f950d646640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f950810e000 con 0x7f9508103780 2024-09-21T03:36:19.385 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:19.380+0000 7f950d646640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f950810e300 con 0x7f95081026f0 2024-09-21T03:36:19.385 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:19.381+0000 7f95077fe640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9508104120 0x7f950819a050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:19.385 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:19.381+0000 7f95077fe640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9508104120 0x7f950819a050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:34306/0 (socket says 172.21.15.89:34306) 2024-09-21T03:36:19.385 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:19.381+0000 7f95077fe640 1 -- 172.21.15.89:0/1567594351 learned_addr learned my addr 172.21.15.89:0/1567594351 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-21T03:36:19.386 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:19.381+0000 7f9506ffd640 1 --2- 172.21.15.89:0/1567594351 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f95081026f0 0x7f9508198e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:19.386 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:19.381+0000 7f95067fc640 1 --2- 172.21.15.89:0/1567594351 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9508103780 0x7f95081994b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:19.386 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:19.381+0000 7f95077fe640 1 -- 172.21.15.89:0/1567594351 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f95081026f0 msgr2=0x7f9508198e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:19.386 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:19.381+0000 7f95077fe640 1 --2- 172.21.15.89:0/1567594351 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f95081026f0 0x7f9508198e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:19.386 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:19.381+0000 7f95077fe640 1 -- 172.21.15.89:0/1567594351 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9508103780 msgr2=0x7f95081994b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:19.386 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:19.381+0000 7f95077fe640 1 --2- 172.21.15.89:0/1567594351 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9508103780 0x7f95081994b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:19.386 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:19.381+0000 7f95077fe640 1 -- 172.21.15.89:0/1567594351 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f950819a870 con 0x7f9508104120 2024-09-21T03:36:19.386 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:19.381+0000 7f95067fc640 1 --2- 172.21.15.89:0/1567594351 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9508103780 0x7f95081994b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:36:19.386 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:19.381+0000 7f95077fe640 1 --2- 172.21.15.89:0/1567594351 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9508104120 0x7f950819a050 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f9500007c30 tx=0x7f950000e420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:36:19.386 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:19.381+0000 7f94effff640 1 -- 172.21.15.89:0/1567594351 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9500019020 con 0x7f9508104120 2024-09-21T03:36:19.386 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:19.381+0000 7f950d646640 1 -- 172.21.15.89:0/1567594351 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f950819ab60 con 0x7f9508104120 2024-09-21T03:36:19.386 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:19.381+0000 7f94effff640 1 -- 172.21.15.89:0/1567594351 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9500007e40 con 0x7f9508104120 2024-09-21T03:36:19.386 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:19.381+0000 7f94effff640 1 -- 172.21.15.89:0/1567594351 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9500005490 con 0x7f9508104120 2024-09-21T03:36:19.386 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:19.381+0000 7f950d646640 1 -- 172.21.15.89:0/1567594351 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f950806b9f0 con 0x7f9508104120 2024-09-21T03:36:19.387 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:19.383+0000 7f950d646640 1 -- 172.21.15.89:0/1567594351 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9508102b60 con 0x7f9508104120 2024-09-21T03:36:19.387 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:19.384+0000 7f94effff640 1 -- 172.21.15.89:0/1567594351 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 17) ==== 154512+0+0 (secure 0 0 0) 0x7f9500005630 con 0x7f9508104120 2024-09-21T03:36:19.387 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:19.385+0000 7f94effff640 1 --2- 172.21.15.89:0/1567594351 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f94f00b6510 0x7f94f00b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:19.387 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:19.385+0000 7f9506ffd640 1 --2- 172.21.15.89:0/1567594351 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f94f00b6510 0x7f94f00b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:19.388 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:19.385+0000 7f94effff640 1 -- 172.21.15.89:0/1567594351 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(65..65 src has 1..65) ==== 6809+0+0 (secure 0 0 0) 0x7f95000e8b70 con 0x7f9508104120 2024-09-21T03:36:19.388 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:19.385+0000 7f9506ffd640 1 --2- 172.21.15.89:0/1567594351 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f94f00b6510 0x7f94f00b8a20 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f94f8005fd0 tx=0x7f94f800b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:36:19.390 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:19.388+0000 7f94effff640 1 -- 172.21.15.89:0/1567594351 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f9500023020 con 0x7f9508104120 2024-09-21T03:36:19.579 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:19.576+0000 7f950d646640 1 -- 172.21.15.89:0/1567594351 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7f9508109370 con 0x7f9508104120 2024-09-21T03:36:19.579 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:19.577+0000 7f94effff640 1 -- 172.21.15.89:0/1567594351 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v65) ==== 74+0+133 (secure 0 0 0) 0x7f95000afa70 con 0x7f9508104120 2024-09-21T03:36:19.580 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-09-21T03:36:19.583 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:19.580+0000 7f950d646640 1 -- 172.21.15.89:0/1567594351 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f94f00b6510 msgr2=0x7f94f00b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:19.583 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:19.580+0000 7f950d646640 1 --2- 172.21.15.89:0/1567594351 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f94f00b6510 0x7f94f00b8a20 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f94f8005fd0 tx=0x7f94f800b390 comp rx=0 tx=0).stop 2024-09-21T03:36:19.583 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:19.580+0000 7f950d646640 1 -- 172.21.15.89:0/1567594351 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9508104120 msgr2=0x7f950819a050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:19.583 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:19.580+0000 7f950d646640 1 --2- 172.21.15.89:0/1567594351 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9508104120 0x7f950819a050 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f9500007c30 tx=0x7f950000e420 comp rx=0 tx=0).stop 2024-09-21T03:36:19.583 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:19.580+0000 7f950d646640 1 -- 172.21.15.89:0/1567594351 shutdown_connections 2024-09-21T03:36:19.583 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:19.580+0000 7f950d646640 1 --2- 172.21.15.89:0/1567594351 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f94f00b6510 0x7f94f00b8a20 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:19.583 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:19.580+0000 7f950d646640 1 --2- 172.21.15.89:0/1567594351 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9508103780 0x7f95081994b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:19.583 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:19.580+0000 7f950d646640 1 --2- 172.21.15.89:0/1567594351 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9508104120 0x7f950819a050 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:19.583 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:19.580+0000 7f950d646640 1 --2- 172.21.15.89:0/1567594351 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f95081026f0 0x7f9508198e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:19.583 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:19.580+0000 7f950d646640 1 -- 172.21.15.89:0/1567594351 >> 172.21.15.89:0/1567594351 conn(0x7f95080fe080 msgr2=0x7f9508106aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:36:19.583 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:19.580+0000 7f950d646640 1 -- 172.21.15.89:0/1567594351 shutdown_connections 2024-09-21T03:36:19.583 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:19.580+0000 7f950d646640 1 -- 172.21.15.89:0/1567594351 wait complete. 2024-09-21T03:36:20.210 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:36:19 smithi114 ceph-mon[34140]: from='client.? 172.21.15.89:0/1567594351' entity='client.admin' cmd={"prefix": "osd stat", "format": "json"} : dispatch 2024-09-21T03:36:20.250 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:36:19 smithi110 ceph-mon[34000]: from='client.? 172.21.15.89:0/1567594351' entity='client.admin' cmd={"prefix": "osd stat", "format": "json"} : dispatch 2024-09-21T03:36:20.311 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:36:19 smithi089 ceph-mon[30928]: from='client.? 172.21.15.89:0/1567594351' entity='client.admin' cmd={"prefix": "osd stat", "format": "json"} : dispatch 2024-09-21T03:36:20.382 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":65,"num_osds":12,"num_up_osds":11,"osd_up_since":1726889758,"num_in_osds":12,"osd_in_since":1726889760,"num_remapped_pgs":0} 2024-09-21T03:36:20.907 INFO:journalctl@ceph.osd.11.smithi114.stdout:Sep 21 03:36:20 smithi114 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-11[58313]: 2024-09-21T03:36:20.741+0000 7f766276e640 -1 osd.11 0 log_to_monitors true 2024-09-21T03:36:21.210 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:36:20 smithi114 ceph-mon[34140]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 334 MiB used, 983 GiB / 983 GiB avail 2024-09-21T03:36:21.211 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:36:20 smithi114 ceph-mon[34140]: from='osd.11 [v2:172.21.15.114:6824/3749854486,v1:172.21.15.114:6825/3749854486]' entity='osd.11' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]} : dispatch 2024-09-21T03:36:21.211 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:36:20 smithi114 ceph-mon[34140]: from='osd.11 ' entity='osd.11' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]} : dispatch 2024-09-21T03:36:21.250 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:36:20 smithi110 ceph-mon[34000]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 334 MiB used, 983 GiB / 983 GiB avail 2024-09-21T03:36:21.250 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:36:20 smithi110 ceph-mon[34000]: from='osd.11 [v2:172.21.15.114:6824/3749854486,v1:172.21.15.114:6825/3749854486]' entity='osd.11' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]} : dispatch 2024-09-21T03:36:21.250 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:36:20 smithi110 ceph-mon[34000]: from='osd.11 ' entity='osd.11' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]} : dispatch 2024-09-21T03:36:21.311 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:36:20 smithi089 ceph-mon[30928]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 334 MiB used, 983 GiB / 983 GiB avail 2024-09-21T03:36:21.311 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:36:20 smithi089 ceph-mon[30928]: from='osd.11 [v2:172.21.15.114:6824/3749854486,v1:172.21.15.114:6825/3749854486]' entity='osd.11' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]} : dispatch 2024-09-21T03:36:21.311 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:36:20 smithi089 ceph-mon[30928]: from='osd.11 ' entity='osd.11' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]} : dispatch 2024-09-21T03:36:21.383 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7c374098-77c9-11ef-baf5-efdc52797490 -- ceph osd stat -f json 2024-09-21T03:36:21.635 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/7c374098-77c9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:36:22.050 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:36:21 smithi089 ceph-mon[30928]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-09-21T03:36:22.050 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:36:21 smithi089 ceph-mon[30928]: osdmap e66: 12 total, 11 up, 12 in 2024-09-21T03:36:22.050 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:36:21 smithi089 ceph-mon[30928]: from='osd.11 [v2:172.21.15.114:6824/3749854486,v1:172.21.15.114:6825/3749854486]' entity='osd.11' cmd={"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi114", "root=default"]} : dispatch 2024-09-21T03:36:22.050 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:36:21 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-21T03:36:22.050 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:36:21 smithi089 ceph-mon[30928]: from='osd.11 ' entity='osd.11' cmd={"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi114", "root=default"]} : dispatch 2024-09-21T03:36:22.050 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:36:21 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:36:22.050 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:36:21 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:36:22.050 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:36:21 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-21T03:36:22.051 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:36:21 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-21T03:36:22.051 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:36:21 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"} : dispatch 2024-09-21T03:36:22.051 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:36:21 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"} : dispatch 2024-09-21T03:36:22.051 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:36:21 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:36:22.051 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:36:21 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:36:22.051 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:36:21 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:36:22.051 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:36:21 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:36:22.051 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:36:21 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:36:22.051 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:36:21 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:36:22.051 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:36:21 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:36:22.051 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:36:21 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:36:22.051 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:36:21 smithi089 ceph-mon[30928]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi114", "root=default"]}]': finished 2024-09-21T03:36:22.150 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:22.137+0000 7f5383259640 1 -- 172.21.15.89:0/149046200 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f537c103750 msgr2=0x7f537c103bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:22.150 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:22.137+0000 7f5383259640 1 --2- 172.21.15.89:0/149046200 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f537c103750 0x7f537c103bc0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f5364009900 tx=0x7f536402a480 comp rx=0 tx=0).stop 2024-09-21T03:36:22.150 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:22.137+0000 7f5383259640 1 -- 172.21.15.89:0/149046200 shutdown_connections 2024-09-21T03:36:22.151 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:22.137+0000 7f5383259640 1 --2- 172.21.15.89:0/149046200 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f537c1047e0 0x7f537c104bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:22.151 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:22.137+0000 7f5383259640 1 --2- 172.21.15.89:0/149046200 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f537c105180 0x7f537c10bb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:22.151 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:22.137+0000 7f5383259640 1 --2- 172.21.15.89:0/149046200 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f537c103750 0x7f537c103bc0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:22.151 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:22.137+0000 7f5383259640 1 -- 172.21.15.89:0/149046200 >> 172.21.15.89:0/149046200 conn(0x7f537c0ff0c0 msgr2=0x7f537c101570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:36:22.151 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:22.137+0000 7f5383259640 1 -- 172.21.15.89:0/149046200 shutdown_connections 2024-09-21T03:36:22.151 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:22.137+0000 7f5383259640 1 -- 172.21.15.89:0/149046200 wait complete. 2024-09-21T03:36:22.151 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:22.138+0000 7f5383259640 1 Processor -- start 2024-09-21T03:36:22.151 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:22.138+0000 7f5383259640 1 -- start start 2024-09-21T03:36:22.151 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:22.138+0000 7f5383259640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f537c103750 0x7f537c199ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:22.151 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:22.138+0000 7f5383259640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f537c1047e0 0x7f537c19a530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:22.151 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:22.138+0000 7f5383259640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f537c105180 0x7f537c19b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:22.151 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:22.138+0000 7f5383259640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f537c10f240 con 0x7f537c1047e0 2024-09-21T03:36:22.151 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:22.138+0000 7f5383259640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f537c10f0c0 con 0x7f537c105180 2024-09-21T03:36:22.151 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:22.138+0000 7f5383259640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f537c10f3c0 con 0x7f537c103750 2024-09-21T03:36:22.152 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:22.138+0000 7f537bfff640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f537c1047e0 0x7f537c19a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:22.152 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:22.138+0000 7f5380ff5640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f537c103750 0x7f537c199ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:22.152 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:22.138+0000 7f537bfff640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f537c1047e0 0x7f537c19a530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:34324/0 (socket says 172.21.15.89:34324) 2024-09-21T03:36:22.152 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:22.138+0000 7f537bfff640 1 -- 172.21.15.89:0/633094442 learned_addr learned my addr 172.21.15.89:0/633094442 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-21T03:36:22.152 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:22.138+0000 7f53817f6640 1 --2- 172.21.15.89:0/633094442 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f537c105180 0x7f537c19b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:22.152 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:22.138+0000 7f537bfff640 1 -- 172.21.15.89:0/633094442 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f537c103750 msgr2=0x7f537c199ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:22.152 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:22.138+0000 7f537bfff640 1 --2- 172.21.15.89:0/633094442 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f537c103750 0x7f537c199ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:22.152 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:22.139+0000 7f537bfff640 1 -- 172.21.15.89:0/633094442 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f537c105180 msgr2=0x7f537c19b0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:22.152 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:22.139+0000 7f537bfff640 1 --2- 172.21.15.89:0/633094442 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f537c105180 0x7f537c19b0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:22.152 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:22.139+0000 7f537bfff640 1 -- 172.21.15.89:0/633094442 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f537c19b8f0 con 0x7f537c1047e0 2024-09-21T03:36:22.152 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:22.139+0000 7f537bfff640 1 --2- 172.21.15.89:0/633094442 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f537c1047e0 0x7f537c19a530 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f536c00cce0 tx=0x7f536c00a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:36:22.152 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:22.139+0000 7f5379ffb640 1 -- 172.21.15.89:0/633094442 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f536c004b20 con 0x7f537c1047e0 2024-09-21T03:36:22.152 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:22.139+0000 7f5379ffb640 1 -- 172.21.15.89:0/633094442 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f536c004cc0 con 0x7f537c1047e0 2024-09-21T03:36:22.153 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:22.139+0000 7f5383259640 1 -- 172.21.15.89:0/633094442 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f537c19bbe0 con 0x7f537c1047e0 2024-09-21T03:36:22.153 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:22.139+0000 7f5379ffb640 1 -- 172.21.15.89:0/633094442 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f536c00f6b0 con 0x7f537c1047e0 2024-09-21T03:36:22.153 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:22.139+0000 7f5383259640 1 -- 172.21.15.89:0/633094442 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f537c1a22e0 con 0x7f537c1047e0 2024-09-21T03:36:22.153 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:22.140+0000 7f5383259640 1 -- 172.21.15.89:0/633094442 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f537c10a930 con 0x7f537c1047e0 2024-09-21T03:36:22.153 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:22.142+0000 7f5379ffb640 1 -- 172.21.15.89:0/633094442 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 17) ==== 154512+0+0 (secure 0 0 0) 0x7f536c00f850 con 0x7f537c1047e0 2024-09-21T03:36:22.153 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:22.143+0000 7f5379ffb640 1 --2- 172.21.15.89:0/633094442 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f53680b6590 0x7f53680b8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:22.153 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:22.143+0000 7f5380ff5640 1 --2- 172.21.15.89:0/633094442 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f53680b6590 0x7f53680b8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:22.153 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:22.143+0000 7f5379ffb640 1 -- 172.21.15.89:0/633094442 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(67..67 src has 1..67) ==== 6847+0+0 (secure 0 0 0) 0x7f536c0e6d10 con 0x7f537c1047e0 2024-09-21T03:36:22.153 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:22.143+0000 7f5380ff5640 1 --2- 172.21.15.89:0/633094442 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f53680b6590 0x7f53680b8aa0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f53640057c0 tx=0x7f5364000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:36:22.153 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:22.147+0000 7f5379ffb640 1 -- 172.21.15.89:0/633094442 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f536c014030 con 0x7f537c1047e0 2024-09-21T03:36:22.211 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:36:21 smithi114 ceph-mon[34140]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-09-21T03:36:22.211 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:36:21 smithi114 ceph-mon[34140]: osdmap e66: 12 total, 11 up, 12 in 2024-09-21T03:36:22.211 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:36:21 smithi114 ceph-mon[34140]: from='osd.11 [v2:172.21.15.114:6824/3749854486,v1:172.21.15.114:6825/3749854486]' entity='osd.11' cmd={"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi114", "root=default"]} : dispatch 2024-09-21T03:36:22.211 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:36:21 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-21T03:36:22.211 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:36:21 smithi114 ceph-mon[34140]: from='osd.11 ' entity='osd.11' cmd={"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi114", "root=default"]} : dispatch 2024-09-21T03:36:22.211 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:36:21 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:36:22.211 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:36:21 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:36:22.211 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:36:21 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-21T03:36:22.211 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:36:21 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-21T03:36:22.212 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:36:21 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"} : dispatch 2024-09-21T03:36:22.212 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:36:21 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"} : dispatch 2024-09-21T03:36:22.212 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:36:21 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:36:22.212 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:36:21 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:36:22.212 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:36:21 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:36:22.212 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:36:21 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:36:22.212 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:36:21 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:36:22.212 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:36:21 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:36:22.212 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:36:21 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:36:22.212 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:36:21 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:36:22.212 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:36:21 smithi114 ceph-mon[34140]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi114", "root=default"]}]': finished 2024-09-21T03:36:22.250 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:36:21 smithi110 ceph-mon[34000]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-09-21T03:36:22.250 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:36:21 smithi110 ceph-mon[34000]: osdmap e66: 12 total, 11 up, 12 in 2024-09-21T03:36:22.250 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:36:21 smithi110 ceph-mon[34000]: from='osd.11 [v2:172.21.15.114:6824/3749854486,v1:172.21.15.114:6825/3749854486]' entity='osd.11' cmd={"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi114", "root=default"]} : dispatch 2024-09-21T03:36:22.250 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:36:21 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-21T03:36:22.251 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:36:21 smithi110 ceph-mon[34000]: from='osd.11 ' entity='osd.11' cmd={"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi114", "root=default"]} : dispatch 2024-09-21T03:36:22.251 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:36:21 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:36:22.251 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:36:21 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:36:22.251 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:36:21 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-21T03:36:22.251 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:36:21 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-21T03:36:22.251 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:36:21 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"} : dispatch 2024-09-21T03:36:22.251 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:36:21 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"} : dispatch 2024-09-21T03:36:22.251 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:36:21 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:36:22.251 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:36:21 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:36:22.251 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:36:21 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:36:22.251 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:36:21 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:36:22.251 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:36:21 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:36:22.251 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:36:21 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:36:22.251 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:36:21 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:36:22.251 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:36:21 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:36:22.252 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:36:21 smithi110 ceph-mon[34000]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi114", "root=default"]}]': finished 2024-09-21T03:36:22.330 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:22.328+0000 7f5383259640 1 -- 172.21.15.89:0/633094442 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7f537c063fc0 con 0x7f537c1047e0 2024-09-21T03:36:22.330 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:22.328+0000 7f5379ffb640 1 -- 172.21.15.89:0/633094442 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v67) ==== 74+0+133 (secure 0 0 0) 0x7f536c0adb60 con 0x7f537c1047e0 2024-09-21T03:36:22.331 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-09-21T03:36:22.333 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:22.331+0000 7f5383259640 1 -- 172.21.15.89:0/633094442 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f53680b6590 msgr2=0x7f53680b8aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:22.333 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:22.331+0000 7f5383259640 1 --2- 172.21.15.89:0/633094442 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f53680b6590 0x7f53680b8aa0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f53640057c0 tx=0x7f5364000bc0 comp rx=0 tx=0).stop 2024-09-21T03:36:22.334 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:22.331+0000 7f5383259640 1 -- 172.21.15.89:0/633094442 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f537c1047e0 msgr2=0x7f537c19a530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:22.334 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:22.331+0000 7f5383259640 1 --2- 172.21.15.89:0/633094442 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f537c1047e0 0x7f537c19a530 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f536c00cce0 tx=0x7f536c00a620 comp rx=0 tx=0).stop 2024-09-21T03:36:22.334 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:22.331+0000 7f5383259640 1 -- 172.21.15.89:0/633094442 shutdown_connections 2024-09-21T03:36:22.334 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:22.331+0000 7f5383259640 1 --2- 172.21.15.89:0/633094442 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f53680b6590 0x7f53680b8aa0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:22.334 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:22.331+0000 7f5383259640 1 --2- 172.21.15.89:0/633094442 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f537c105180 0x7f537c19b0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:22.334 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:22.331+0000 7f5383259640 1 --2- 172.21.15.89:0/633094442 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f537c1047e0 0x7f537c19a530 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:22.334 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:22.331+0000 7f5383259640 1 --2- 172.21.15.89:0/633094442 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f537c103750 0x7f537c199ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:22.334 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:22.331+0000 7f5383259640 1 -- 172.21.15.89:0/633094442 >> 172.21.15.89:0/633094442 conn(0x7f537c0ff0c0 msgr2=0x7f537c107b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:36:22.334 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:22.331+0000 7f5383259640 1 -- 172.21.15.89:0/633094442 shutdown_connections 2024-09-21T03:36:22.334 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:22.331+0000 7f5383259640 1 -- 172.21.15.89:0/633094442 wait complete. 2024-09-21T03:36:22.711 INFO:journalctl@ceph.osd.11.smithi114.stdout:Sep 21 03:36:22 smithi114 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-11[58313]: 2024-09-21T03:36:22.364+0000 7f765e6f0640 -1 osd.11 0 waiting for initial osdmap 2024-09-21T03:36:22.711 INFO:journalctl@ceph.osd.11.smithi114.stdout:Sep 21 03:36:22 smithi114 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-11[58313]: 2024-09-21T03:36:22.376+0000 7f76564e6640 -1 osd.11 67 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-21T03:36:23.077 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":67,"num_osds":12,"num_up_osds":11,"osd_up_since":1726889758,"num_in_osds":12,"osd_in_since":1726889760,"num_remapped_pgs":0} 2024-09-21T03:36:23.211 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:36:22 smithi114 ceph-mon[34140]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 334 MiB used, 983 GiB / 983 GiB avail 2024-09-21T03:36:23.211 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:36:22 smithi114 ceph-mon[34140]: Detected new or changed devices on smithi114 2024-09-21T03:36:23.211 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:36:22 smithi114 ceph-mon[34140]: Adjusting osd_memory_target on smithi114 to 4272M 2024-09-21T03:36:23.211 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:36:22 smithi114 ceph-mon[34140]: osdmap e67: 12 total, 11 up, 12 in 2024-09-21T03:36:23.211 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:36:22 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-21T03:36:23.211 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:36:22 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-21T03:36:23.211 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:36:22 smithi114 ceph-mon[34140]: from='client.? 172.21.15.89:0/633094442' entity='client.admin' cmd={"prefix": "osd stat", "format": "json"} : dispatch 2024-09-21T03:36:23.211 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:36:22 smithi114 ceph-mon[34140]: from='osd.11 ' entity='osd.11' 2024-09-21T03:36:23.250 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:36:22 smithi110 ceph-mon[34000]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 334 MiB used, 983 GiB / 983 GiB avail 2024-09-21T03:36:23.250 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:36:22 smithi110 ceph-mon[34000]: Detected new or changed devices on smithi114 2024-09-21T03:36:23.250 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:36:22 smithi110 ceph-mon[34000]: Adjusting osd_memory_target on smithi114 to 4272M 2024-09-21T03:36:23.250 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:36:22 smithi110 ceph-mon[34000]: osdmap e67: 12 total, 11 up, 12 in 2024-09-21T03:36:23.250 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:36:22 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-21T03:36:23.250 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:36:22 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-21T03:36:23.250 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:36:22 smithi110 ceph-mon[34000]: from='client.? 172.21.15.89:0/633094442' entity='client.admin' cmd={"prefix": "osd stat", "format": "json"} : dispatch 2024-09-21T03:36:23.250 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:36:22 smithi110 ceph-mon[34000]: from='osd.11 ' entity='osd.11' 2024-09-21T03:36:23.311 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:36:22 smithi089 ceph-mon[30928]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 334 MiB used, 983 GiB / 983 GiB avail 2024-09-21T03:36:23.575 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:36:22 smithi089 ceph-mon[30928]: Detected new or changed devices on smithi114 2024-09-21T03:36:23.575 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:36:22 smithi089 ceph-mon[30928]: Adjusting osd_memory_target on smithi114 to 4272M 2024-09-21T03:36:23.576 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:36:22 smithi089 ceph-mon[30928]: osdmap e67: 12 total, 11 up, 12 in 2024-09-21T03:36:23.576 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:36:22 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-21T03:36:23.576 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:36:22 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-21T03:36:23.576 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:36:22 smithi089 ceph-mon[30928]: from='client.? 172.21.15.89:0/633094442' entity='client.admin' cmd={"prefix": "osd stat", "format": "json"} : dispatch 2024-09-21T03:36:23.576 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:36:22 smithi089 ceph-mon[30928]: from='osd.11 ' entity='osd.11' 2024-09-21T03:36:24.078 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7c374098-77c9-11ef-baf5-efdc52797490 -- ceph osd stat -f json 2024-09-21T03:36:24.211 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:36:23 smithi114 ceph-mon[34140]: purged_snaps scrub starts 2024-09-21T03:36:24.211 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:36:23 smithi114 ceph-mon[34140]: purged_snaps scrub ok 2024-09-21T03:36:24.211 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:36:23 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-21T03:36:24.211 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:36:23 smithi114 ceph-mon[34140]: osd.11 [v2:172.21.15.114:6824/3749854486,v1:172.21.15.114:6825/3749854486] boot 2024-09-21T03:36:24.211 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:36:23 smithi114 ceph-mon[34140]: osdmap e68: 12 total, 12 up, 12 in 2024-09-21T03:36:24.211 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:36:23 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-21T03:36:24.250 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:36:23 smithi110 ceph-mon[34000]: purged_snaps scrub starts 2024-09-21T03:36:24.250 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:36:23 smithi110 ceph-mon[34000]: purged_snaps scrub ok 2024-09-21T03:36:24.250 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:36:23 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-21T03:36:24.250 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:36:23 smithi110 ceph-mon[34000]: osd.11 [v2:172.21.15.114:6824/3749854486,v1:172.21.15.114:6825/3749854486] boot 2024-09-21T03:36:24.250 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:36:23 smithi110 ceph-mon[34000]: osdmap e68: 12 total, 12 up, 12 in 2024-09-21T03:36:24.250 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:36:23 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-21T03:36:24.311 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:36:23 smithi089 ceph-mon[30928]: purged_snaps scrub starts 2024-09-21T03:36:24.311 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:36:23 smithi089 ceph-mon[30928]: purged_snaps scrub ok 2024-09-21T03:36:24.311 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:36:23 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-21T03:36:24.311 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:36:23 smithi089 ceph-mon[30928]: osd.11 [v2:172.21.15.114:6824/3749854486,v1:172.21.15.114:6825/3749854486] boot 2024-09-21T03:36:24.311 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:36:23 smithi089 ceph-mon[30928]: osdmap e68: 12 total, 12 up, 12 in 2024-09-21T03:36:24.311 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:36:23 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-21T03:36:24.339 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/7c374098-77c9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:36:24.819 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:24.813+0000 7f46990c7640 1 -- 172.21.15.89:0/2955443899 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4694105180 msgr2=0x7f469410bb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:24.819 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:24.813+0000 7f46990c7640 1 --2- 172.21.15.89:0/2955443899 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4694105180 0x7f469410bb20 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f468c009ca0 tx=0x7f468c02a950 comp rx=0 tx=0).stop 2024-09-21T03:36:24.819 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:24.813+0000 7f46990c7640 1 -- 172.21.15.89:0/2955443899 shutdown_connections 2024-09-21T03:36:24.819 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:24.813+0000 7f46990c7640 1 --2- 172.21.15.89:0/2955443899 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f46941047e0 0x7f4694104bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:24.819 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:24.813+0000 7f46990c7640 1 --2- 172.21.15.89:0/2955443899 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4694105180 0x7f469410bb20 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:24.819 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:24.813+0000 7f46990c7640 1 --2- 172.21.15.89:0/2955443899 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4694103750 0x7f4694103bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:24.820 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:24.813+0000 7f46990c7640 1 -- 172.21.15.89:0/2955443899 >> 172.21.15.89:0/2955443899 conn(0x7f46940ff0c0 msgr2=0x7f4694101570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:36:24.820 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:24.813+0000 7f46990c7640 1 -- 172.21.15.89:0/2955443899 shutdown_connections 2024-09-21T03:36:24.820 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:24.813+0000 7f46990c7640 1 -- 172.21.15.89:0/2955443899 wait complete. 2024-09-21T03:36:24.820 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:24.814+0000 7f46990c7640 1 Processor -- start 2024-09-21T03:36:24.820 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:24.814+0000 7f46990c7640 1 -- start start 2024-09-21T03:36:24.820 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:24.815+0000 7f46990c7640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4694103750 0x7f4694199e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:24.820 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:24.815+0000 7f46990c7640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f46941047e0 0x7f469419a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:24.820 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:24.815+0000 7f46990c7640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4694105180 0x7f469419b090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:24.820 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:24.815+0000 7f46990c7640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f469410f230 con 0x7f4694103750 2024-09-21T03:36:24.820 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:24.815+0000 7f46990c7640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f469410f0b0 con 0x7f46941047e0 2024-09-21T03:36:24.820 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:24.815+0000 7f46990c7640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f469410f3b0 con 0x7f4694105180 2024-09-21T03:36:24.820 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:24.815+0000 7f4692d9d640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4694103750 0x7f4694199e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:24.820 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:24.815+0000 7f4692d9d640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4694103750 0x7f4694199e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:34350/0 (socket says 172.21.15.89:34350) 2024-09-21T03:36:24.821 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:24.815+0000 7f4692d9d640 1 -- 172.21.15.89:0/870528699 learned_addr learned my addr 172.21.15.89:0/870528699 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-21T03:36:24.821 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:24.815+0000 7f4692d9d640 1 -- 172.21.15.89:0/870528699 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4694105180 msgr2=0x7f469419b090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:24.821 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:24.815+0000 7f4692d9d640 1 --2- 172.21.15.89:0/870528699 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4694105180 0x7f469419b090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:24.821 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:24.815+0000 7f4692d9d640 1 -- 172.21.15.89:0/870528699 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f46941047e0 msgr2=0x7f469419a4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:24.821 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:24.815+0000 7f4692d9d640 1 --2- 172.21.15.89:0/870528699 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f46941047e0 0x7f469419a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:24.821 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:24.815+0000 7f4692d9d640 1 -- 172.21.15.89:0/870528699 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f469419b8b0 con 0x7f4694103750 2024-09-21T03:36:24.821 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:24.815+0000 7f4692d9d640 1 --2- 172.21.15.89:0/870528699 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4694103750 0x7f4694199e80 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f468400d950 tx=0x7f468400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:36:24.821 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:24.815+0000 7f4690d99640 1 -- 172.21.15.89:0/870528699 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4684017070 con 0x7f4694103750 2024-09-21T03:36:24.821 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:24.815+0000 7f46990c7640 1 -- 172.21.15.89:0/870528699 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f469419bba0 con 0x7f4694103750 2024-09-21T03:36:24.821 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:24.815+0000 7f4690d99640 1 -- 172.21.15.89:0/870528699 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4684009c30 con 0x7f4694103750 2024-09-21T03:36:24.821 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:24.815+0000 7f4690d99640 1 -- 172.21.15.89:0/870528699 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f468400f620 con 0x7f4694103750 2024-09-21T03:36:24.821 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:24.815+0000 7f46990c7640 1 -- 172.21.15.89:0/870528699 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46941a22a0 con 0x7f4694103750 2024-09-21T03:36:24.821 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:24.816+0000 7f46990c7640 1 -- 172.21.15.89:0/870528699 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f469410a910 con 0x7f4694103750 2024-09-21T03:36:24.822 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:24.818+0000 7f4690d99640 1 -- 172.21.15.89:0/870528699 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 17) ==== 154512+0+0 (secure 0 0 0) 0x7f4684010460 con 0x7f4694103750 2024-09-21T03:36:24.822 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:24.819+0000 7f4690d99640 1 --2- 172.21.15.89:0/870528699 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f46800b6510 0x7f46800b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:24.822 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:24.819+0000 7f468bfff640 1 --2- 172.21.15.89:0/870528699 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f46800b6510 0x7f46800b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:24.822 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:24.819+0000 7f4690d99640 1 -- 172.21.15.89:0/870528699 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 7127+0+0 (secure 0 0 0) 0x7f46840e6b30 con 0x7f4694103750 2024-09-21T03:36:24.823 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:24.819+0000 7f468bfff640 1 --2- 172.21.15.89:0/870528699 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f46800b6510 0x7f46800b8a20 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f467c005fd0 tx=0x7f467c005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:36:24.824 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:24.822+0000 7f4690d99640 1 -- 172.21.15.89:0/870528699 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f46840ad860 con 0x7f4694103750 2024-09-21T03:36:25.005 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:25.003+0000 7f46990c7640 1 -- 172.21.15.89:0/870528699 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7f469402e860 con 0x7f4694103750 2024-09-21T03:36:25.006 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:25.004+0000 7f4690d99640 1 -- 172.21.15.89:0/870528699 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v69) ==== 74+0+133 (secure 0 0 0) 0x7f4684009de0 con 0x7f4694103750 2024-09-21T03:36:25.006 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-09-21T03:36:25.008 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:25.006+0000 7f46990c7640 1 -- 172.21.15.89:0/870528699 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f46800b6510 msgr2=0x7f46800b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:25.008 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:25.006+0000 7f46990c7640 1 --2- 172.21.15.89:0/870528699 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f46800b6510 0x7f46800b8a20 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f467c005fd0 tx=0x7f467c005850 comp rx=0 tx=0).stop 2024-09-21T03:36:25.008 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:25.006+0000 7f46990c7640 1 -- 172.21.15.89:0/870528699 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4694103750 msgr2=0x7f4694199e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:25.008 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:25.006+0000 7f46990c7640 1 --2- 172.21.15.89:0/870528699 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4694103750 0x7f4694199e80 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f468400d950 tx=0x7f468400de10 comp rx=0 tx=0).stop 2024-09-21T03:36:25.008 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:25.006+0000 7f46990c7640 1 -- 172.21.15.89:0/870528699 shutdown_connections 2024-09-21T03:36:25.008 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:25.006+0000 7f46990c7640 1 --2- 172.21.15.89:0/870528699 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f46800b6510 0x7f46800b8a20 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:25.008 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:25.006+0000 7f46990c7640 1 --2- 172.21.15.89:0/870528699 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f46941047e0 0x7f469419a4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:25.009 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:25.006+0000 7f46990c7640 1 --2- 172.21.15.89:0/870528699 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4694105180 0x7f469419b090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:25.009 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:25.006+0000 7f46990c7640 1 --2- 172.21.15.89:0/870528699 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4694103750 0x7f4694199e80 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:25.009 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:25.006+0000 7f46990c7640 1 -- 172.21.15.89:0/870528699 >> 172.21.15.89:0/870528699 conn(0x7f46940ff0c0 msgr2=0x7f4694107ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:36:25.009 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:25.006+0000 7f46990c7640 1 -- 172.21.15.89:0/870528699 shutdown_connections 2024-09-21T03:36:25.009 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:25.006+0000 7f46990c7640 1 -- 172.21.15.89:0/870528699 wait complete. 2024-09-21T03:36:25.633 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:36:25 smithi089 ceph-mon[30928]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 334 MiB used, 983 GiB / 983 GiB avail 2024-09-21T03:36:25.633 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:36:25 smithi089 ceph-mon[30928]: osdmap e69: 12 total, 12 up, 12 in 2024-09-21T03:36:25.633 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:36:25 smithi089 ceph-mon[30928]: from='client.? 172.21.15.89:0/870528699' entity='client.admin' cmd={"prefix": "osd stat", "format": "json"} : dispatch 2024-09-21T03:36:25.660 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":69,"num_osds":12,"num_up_osds":12,"osd_up_since":1726889783,"num_in_osds":12,"osd_in_since":1726889760,"num_remapped_pgs":0} 2024-09-21T03:36:25.660 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 7c374098-77c9-11ef-baf5-efdc52797490 -- ceph osd dump --format=json 2024-09-21T03:36:25.711 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:36:25 smithi114 ceph-mon[34140]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 334 MiB used, 983 GiB / 983 GiB avail 2024-09-21T03:36:25.711 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:36:25 smithi114 ceph-mon[34140]: osdmap e69: 12 total, 12 up, 12 in 2024-09-21T03:36:25.711 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:36:25 smithi114 ceph-mon[34140]: from='client.? 172.21.15.89:0/870528699' entity='client.admin' cmd={"prefix": "osd stat", "format": "json"} : dispatch 2024-09-21T03:36:25.750 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:36:25 smithi110 ceph-mon[34000]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 334 MiB used, 983 GiB / 983 GiB avail 2024-09-21T03:36:25.750 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:36:25 smithi110 ceph-mon[34000]: osdmap e69: 12 total, 12 up, 12 in 2024-09-21T03:36:25.750 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:36:25 smithi110 ceph-mon[34000]: from='client.? 172.21.15.89:0/870528699' entity='client.admin' cmd={"prefix": "osd stat", "format": "json"} : dispatch 2024-09-21T03:36:25.943 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/7c374098-77c9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:36:26.429 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:26.400+0000 7f3444d1c640 1 -- 172.21.15.89:0/877946518 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3440109df0 msgr2=0x7f3440110620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:26.429 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:26.400+0000 7f3444d1c640 1 --2- 172.21.15.89:0/877946518 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3440109df0 0x7f3440110620 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f3438009ca0 tx=0x7f343802a8c0 comp rx=0 tx=0).stop 2024-09-21T03:36:26.430 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:26.400+0000 7f3444d1c640 1 -- 172.21.15.89:0/877946518 shutdown_connections 2024-09-21T03:36:26.430 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:26.401+0000 7f3444d1c640 1 --2- 172.21.15.89:0/877946518 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f34401094a0 0x7f34401098b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:26.430 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:26.401+0000 7f3444d1c640 1 --2- 172.21.15.89:0/877946518 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3440109df0 0x7f3440110620 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:26.430 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:26.401+0000 7f3444d1c640 1 --2- 172.21.15.89:0/877946518 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3440103b60 0x7f3440103f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:26.430 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:26.401+0000 7f3444d1c640 1 -- 172.21.15.89:0/877946518 >> 172.21.15.89:0/877946518 conn(0x7f344007b850 msgr2=0x7f344007bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:36:26.430 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:26.401+0000 7f3444d1c640 1 -- 172.21.15.89:0/877946518 shutdown_connections 2024-09-21T03:36:26.430 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:26.401+0000 7f3444d1c640 1 -- 172.21.15.89:0/877946518 wait complete. 2024-09-21T03:36:26.430 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:26.401+0000 7f3444d1c640 1 Processor -- start 2024-09-21T03:36:26.430 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:26.401+0000 7f3444d1c640 1 -- start start 2024-09-21T03:36:26.430 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:26.402+0000 7f3444d1c640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3440103b60 0x7f3440075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:26.430 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:26.402+0000 7f3444d1c640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f34401094a0 0x7f3440075c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:26.430 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:26.402+0000 7f3444d1c640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3440109df0 0x7f3440076190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:26.430 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:26.402+0000 7f3444d1c640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f34401139b0 con 0x7f3440109df0 2024-09-21T03:36:26.430 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:26.402+0000 7f3444d1c640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f3440113830 con 0x7f34401094a0 2024-09-21T03:36:26.431 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:26.402+0000 7f3444d1c640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f3440113b30 con 0x7f3440103b60 2024-09-21T03:36:26.431 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:26.402+0000 7f343ed9d640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3440109df0 0x7f3440076190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:26.431 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:26.402+0000 7f343ed9d640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3440109df0 0x7f3440076190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:34360/0 (socket says 172.21.15.89:34360) 2024-09-21T03:36:26.431 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:26.402+0000 7f343ed9d640 1 -- 172.21.15.89:0/2345453502 learned_addr learned my addr 172.21.15.89:0/2345453502 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-21T03:36:26.431 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:26.402+0000 7f343e59c640 1 --2- 172.21.15.89:0/2345453502 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3440103b60 0x7f3440075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:26.431 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:26.402+0000 7f343dd9b640 1 --2- 172.21.15.89:0/2345453502 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f34401094a0 0x7f3440075c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:26.431 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:26.402+0000 7f343ed9d640 1 -- 172.21.15.89:0/2345453502 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3440103b60 msgr2=0x7f3440075710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:26.431 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:26.402+0000 7f343ed9d640 1 --2- 172.21.15.89:0/2345453502 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3440103b60 0x7f3440075710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:26.431 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:26.402+0000 7f343ed9d640 1 -- 172.21.15.89:0/2345453502 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f34401094a0 msgr2=0x7f3440075c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:26.431 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:26.402+0000 7f343ed9d640 1 --2- 172.21.15.89:0/2345453502 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f34401094a0 0x7f3440075c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:26.431 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:26.402+0000 7f343ed9d640 1 -- 172.21.15.89:0/2345453502 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34401a8780 con 0x7f3440109df0 2024-09-21T03:36:26.431 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:26.402+0000 7f343dd9b640 1 --2- 172.21.15.89:0/2345453502 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f34401094a0 0x7f3440075c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:36:26.431 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:26.402+0000 7f343ed9d640 1 --2- 172.21.15.89:0/2345453502 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3440109df0 0x7f3440076190 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f3438005e30 tx=0x7f34380093d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:36:26.432 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:26.402+0000 7f342f7fe640 1 -- 172.21.15.89:0/2345453502 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3438032d10 con 0x7f3440109df0 2024-09-21T03:36:26.432 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:26.403+0000 7f342f7fe640 1 -- 172.21.15.89:0/2345453502 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3438032eb0 con 0x7f3440109df0 2024-09-21T03:36:26.432 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:26.403+0000 7f3444d1c640 1 -- 172.21.15.89:0/2345453502 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f34401a8a10 con 0x7f3440109df0 2024-09-21T03:36:26.432 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:26.403+0000 7f342f7fe640 1 -- 172.21.15.89:0/2345453502 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3438030b20 con 0x7f3440109df0 2024-09-21T03:36:26.432 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:26.403+0000 7f3444d1c640 1 -- 172.21.15.89:0/2345453502 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34401a8f50 con 0x7f3440109df0 2024-09-21T03:36:26.432 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:26.404+0000 7f3444d1c640 1 -- 172.21.15.89:0/2345453502 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3440103f70 con 0x7f3440109df0 2024-09-21T03:36:26.432 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:26.405+0000 7f342f7fe640 1 -- 172.21.15.89:0/2345453502 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 17) ==== 154512+0+0 (secure 0 0 0) 0x7f3438030cc0 con 0x7f3440109df0 2024-09-21T03:36:26.432 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:26.406+0000 7f342f7fe640 1 --2- 172.21.15.89:0/2345453502 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f34280b6510 0x7f34280b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:26.432 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:26.406+0000 7f343e59c640 1 --2- 172.21.15.89:0/2345453502 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f34280b6510 0x7f34280b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:26.432 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:26.406+0000 7f342f7fe640 1 -- 172.21.15.89:0/2345453502 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 7127+0+0 (secure 0 0 0) 0x7f3438108290 con 0x7f3440109df0 2024-09-21T03:36:26.432 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:26.407+0000 7f343e59c640 1 --2- 172.21.15.89:0/2345453502 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f34280b6510 0x7f34280b8a20 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f3430005fd0 tx=0x7f3430005e20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:36:26.432 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:26.409+0000 7f342f7fe640 1 -- 172.21.15.89:0/2345453502 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f34380cefc0 con 0x7f3440109df0 2024-09-21T03:36:26.589 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:26.587+0000 7f3444d1c640 1 -- 172.21.15.89:0/2345453502 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f3440076a10 con 0x7f3440109df0 2024-09-21T03:36:26.590 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:26.588+0000 7f342f7fe640 1 -- 172.21.15.89:0/2345453502 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v69) ==== 74+0+18492 (secure 0 0 0) 0x7f3440076a10 con 0x7f3440109df0 2024-09-21T03:36:26.591 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-09-21T03:36:26.591 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":69,"fsid":"7c374098-77c9-11ef-baf5-efdc52797490","created":"2024-09-21T03:28:56.681458+0000","modified":"2024-09-21T03:36:24.370417+0000","last_up_change":"2024-09-21T03:36:23.368379+0000","last_in_change":"2024-09-21T03:36:00.131331+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T03:32:37.834339+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"27ca5ed8-17d1-4f8e-a7ae-084a914721a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6802","nonce":990539042},{"type":"v1","addr":"172.21.15.89:6803","nonce":990539042}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6804","nonce":990539042},{"type":"v1","addr":"172.21.15.89:6805","nonce":990539042}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6808","nonce":990539042},{"type":"v1","addr":"172.21.15.89:6809","nonce":990539042}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6806","nonce":990539042},{"type":"v1","addr":"172.21.15.89:6807","nonce":990539042}]},"public_addr":"172.21.15.89:6803/990539042","cluster_addr":"172.21.15.89:6805/990539042","heartbeat_back_addr":"172.21.15.89:6809/990539042","heartbeat_front_addr":"172.21.15.89:6807/990539042","state":["exists","up"]},{"osd":1,"uuid":"5176a910-5cf9-49d7-aedb-a20cf36ea075","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6800","nonce":3235852628},{"type":"v1","addr":"172.21.15.110:6801","nonce":3235852628}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6802","nonce":3235852628},{"type":"v1","addr":"172.21.15.110:6803","nonce":3235852628}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6806","nonce":3235852628},{"type":"v1","addr":"172.21.15.110:6807","nonce":3235852628}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6804","nonce":3235852628},{"type":"v1","addr":"172.21.15.110:6805","nonce":3235852628}]},"public_addr":"172.21.15.110:6801/3235852628","cluster_addr":"172.21.15.110:6803/3235852628","heartbeat_back_addr":"172.21.15.110:6807/3235852628","heartbeat_front_addr":"172.21.15.110:6805/3235852628","state":["exists","up"]},{"osd":2,"uuid":"d4977df1-b118-4d7e-b9a9-626a8accc812","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6800","nonce":169452122},{"type":"v1","addr":"172.21.15.114:6801","nonce":169452122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6802","nonce":169452122},{"type":"v1","addr":"172.21.15.114:6803","nonce":169452122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6806","nonce":169452122},{"type":"v1","addr":"172.21.15.114:6807","nonce":169452122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6804","nonce":169452122},{"type":"v1","addr":"172.21.15.114:6805","nonce":169452122}]},"public_addr":"172.21.15.114:6801/169452122","cluster_addr":"172.21.15.114:6803/169452122","heartbeat_back_addr":"172.21.15.114:6807/169452122","heartbeat_front_addr":"172.21.15.114:6805/169452122","state":["exists","up"]},{"osd":3,"uuid":"2c7ae1b4-0bf2-4954-81ed-55d03eab82c6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6810","nonce":3016980877},{"type":"v1","addr":"172.21.15.89:6811","nonce":3016980877}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6812","nonce":3016980877},{"type":"v1","addr":"172.21.15.89:6813","nonce":3016980877}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6816","nonce":3016980877},{"type":"v1","addr":"172.21.15.89:6817","nonce":3016980877}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6814","nonce":3016980877},{"type":"v1","addr":"172.21.15.89:6815","nonce":3016980877}]},"public_addr":"172.21.15.89:6811/3016980877","cluster_addr":"172.21.15.89:6813/3016980877","heartbeat_back_addr":"172.21.15.89:6817/3016980877","heartbeat_front_addr":"172.21.15.89:6815/3016980877","state":["exists","up"]},{"osd":4,"uuid":"8e65399d-6e55-4da4-98ba-f805f0c655a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":30,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6808","nonce":3217880476},{"type":"v1","addr":"172.21.15.110:6809","nonce":3217880476}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6810","nonce":3217880476},{"type":"v1","addr":"172.21.15.110:6811","nonce":3217880476}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6814","nonce":3217880476},{"type":"v1","addr":"172.21.15.110:6815","nonce":3217880476}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6812","nonce":3217880476},{"type":"v1","addr":"172.21.15.110:6813","nonce":3217880476}]},"public_addr":"172.21.15.110:6809/3217880476","cluster_addr":"172.21.15.110:6811/3217880476","heartbeat_back_addr":"172.21.15.110:6815/3217880476","heartbeat_front_addr":"172.21.15.110:6813/3217880476","state":["exists","up"]},{"osd":5,"uuid":"b9ec2889-46bd-40e5-8446-73ab28271c67","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6808","nonce":1144140707},{"type":"v1","addr":"172.21.15.114:6809","nonce":1144140707}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6810","nonce":1144140707},{"type":"v1","addr":"172.21.15.114:6811","nonce":1144140707}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6814","nonce":1144140707},{"type":"v1","addr":"172.21.15.114:6815","nonce":1144140707}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6812","nonce":1144140707},{"type":"v1","addr":"172.21.15.114:6813","nonce":1144140707}]},"public_addr":"172.21.15.114:6809/1144140707","cluster_addr":"172.21.15.114:6811/1144140707","heartbeat_back_addr":"172.21.15.114:6815/1144140707","heartbeat_front_addr":"172.21.15.114:6813/1144140707","state":["exists","up"]},{"osd":6,"uuid":"9b79989e-adb0-42b2-b2f6-a2dce56b66e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":41,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6818","nonce":1522586255},{"type":"v1","addr":"172.21.15.89:6819","nonce":1522586255}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6820","nonce":1522586255},{"type":"v1","addr":"172.21.15.89:6821","nonce":1522586255}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6824","nonce":1522586255},{"type":"v1","addr":"172.21.15.89:6825","nonce":1522586255}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6822","nonce":1522586255},{"type":"v1","addr":"172.21.15.89:6823","nonce":1522586255}]},"public_addr":"172.21.15.89:6819/1522586255","cluster_addr":"172.21.15.89:6821/1522586255","heartbeat_back_addr":"172.21.15.89:6825/1522586255","heartbeat_front_addr":"172.21.15.89:6823/1522586255","state":["exists","up"]},{"osd":7,"uuid":"b381821f-141c-47a1-be65-f57ca730391b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6816","nonce":1331818323},{"type":"v1","addr":"172.21.15.110:6817","nonce":1331818323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6818","nonce":1331818323},{"type":"v1","addr":"172.21.15.110:6819","nonce":1331818323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6822","nonce":1331818323},{"type":"v1","addr":"172.21.15.110:6823","nonce":1331818323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6820","nonce":1331818323},{"type":"v1","addr":"172.21.15.110:6821","nonce":1331818323}]},"public_addr":"172.21.15.110:6817/1331818323","cluster_addr":"172.21.15.110:6819/1331818323","heartbeat_back_addr":"172.21.15.110:6823/1331818323","heartbeat_front_addr":"172.21.15.110:6821/1331818323","state":["exists","up"]},{"osd":8,"uuid":"1bc2a5b4-15f3-456f-9d67-e51307a27bcc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6816","nonce":2138556699},{"type":"v1","addr":"172.21.15.114:6817","nonce":2138556699}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6818","nonce":2138556699},{"type":"v1","addr":"172.21.15.114:6819","nonce":2138556699}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6822","nonce":2138556699},{"type":"v1","addr":"172.21.15.114:6823","nonce":2138556699}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6820","nonce":2138556699},{"type":"v1","addr":"172.21.15.114:6821","nonce":2138556699}]},"public_addr":"172.21.15.114:6817/2138556699","cluster_addr":"172.21.15.114:6819/2138556699","heartbeat_back_addr":"172.21.15.114:6823/2138556699","heartbeat_front_addr":"172.21.15.114:6821/2138556699","state":["exists","up"]},{"osd":9,"uuid":"91f5fd29-d4a7-4800-ac7a-e83b13add89e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6826","nonce":1730253068},{"type":"v1","addr":"172.21.15.89:6827","nonce":1730253068}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6828","nonce":1730253068},{"type":"v1","addr":"172.21.15.89:6829","nonce":1730253068}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6832","nonce":1730253068},{"type":"v1","addr":"172.21.15.89:6833","nonce":1730253068}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6830","nonce":1730253068},{"type":"v1","addr":"172.21.15.89:6831","nonce":1730253068}]},"public_addr":"172.21.15.89:6827/1730253068","cluster_addr":"172.21.15.89:6829/1730253068","heartbeat_back_addr":"172.21.15.89:6833/1730253068","heartbeat_front_addr":"172.21.15.89:6831/1730253068","state":["exists","up"]},{"osd":10,"uuid":"01390499-2a4d-4188-b57a-fcb10c668e2c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6824","nonce":2904016117},{"type":"v1","addr":"172.21.15.110:6825","nonce":2904016117}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6826","nonce":2904016117},{"type":"v1","addr":"172.21.15.110:6827","nonce":2904016117}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6830","nonce":2904016117},{"type":"v1","addr":"172.21.15.110:6831","nonce":2904016117}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6828","nonce":2904016117},{"type":"v1","addr":"172.21.15.110:6829","nonce":2904016117}]},"public_addr":"172.21.15.110:6825/2904016117","cluster_addr":"172.21.15.110:6827/2904016117","heartbeat_back_addr":"172.21.15.110:6831/2904016117","heartbeat_front_addr":"172.21.15.110:6829/2904016117","state":["exists","up"]},{"osd":11,"uuid":"c3c0f38a-c3d3-415d-a9f8-875d5b756e35","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6824","nonce":3749854486},{"type":"v1","addr":"172.21.15.114:6825","nonce":3749854486}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6826","nonce":3749854486},{"type":"v1","addr":"172.21.15.114:6827","nonce":3749854486}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6830","nonce":3749854486},{"type":"v1","addr":"172.21.15.114:6831","nonce":3749854486}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6828","nonce":3749854486},{"type":"v1","addr":"172.21.15.114:6829","nonce":3749854486}]},"public_addr":"172.21.15.114:6825/3749854486","cluster_addr":"172.21.15.114:6827/3749854486","heartbeat_back_addr":"172.21.15.114:6831/3749854486","heartbeat_front_addr":"172.21.15.114:6829/3749854486","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:31:46.218908+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:32:10.338230+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:32:34.830596+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:33:00.816102+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:33:25.223838+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:33:50.188230+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:34:16.564255+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:34:41.498230+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:35:05.826322+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:35:31.234450+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:35:56.474373+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:36:21.741254+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.89:6801/655110910":"2024-09-22T03:29:51.519106+0000","172.21.15.89:0/1618544983":"2024-09-22T03:29:51.519106+0000","172.21.15.89:0/1265502758":"2024-09-22T03:29:51.519106+0000","172.21.15.89:0/4017684419":"2024-09-22T03:29:51.519106+0000","172.21.15.89:6801/3312712420":"2024-09-22T03:29:26.004686+0000","172.21.15.89:6800/3312712420":"2024-09-22T03:29:26.004686+0000","172.21.15.89:0/2056576483":"2024-09-22T03:29:26.004686+0000","172.21.15.89:6800/655110910":"2024-09-22T03:29:51.519106+0000","172.21.15.89:0/2382866302":"2024-09-22T03:29:26.004686+0000","172.21.15.89:0/1584351801":"2024-09-22T03:29:26.004686+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T03:36:26.593 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:26.591+0000 7f3444d1c640 1 -- 172.21.15.89:0/2345453502 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f34280b6510 msgr2=0x7f34280b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:26.593 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:26.591+0000 7f3444d1c640 1 --2- 172.21.15.89:0/2345453502 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f34280b6510 0x7f34280b8a20 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f3430005fd0 tx=0x7f3430005e20 comp rx=0 tx=0).stop 2024-09-21T03:36:26.593 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:26.591+0000 7f3444d1c640 1 -- 172.21.15.89:0/2345453502 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3440109df0 msgr2=0x7f3440076190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:26.593 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:26.591+0000 7f3444d1c640 1 --2- 172.21.15.89:0/2345453502 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3440109df0 0x7f3440076190 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f3438005e30 tx=0x7f34380093d0 comp rx=0 tx=0).stop 2024-09-21T03:36:26.593 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:26.591+0000 7f3444d1c640 1 -- 172.21.15.89:0/2345453502 shutdown_connections 2024-09-21T03:36:26.594 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:26.591+0000 7f3444d1c640 1 --2- 172.21.15.89:0/2345453502 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f34280b6510 0x7f34280b8a20 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:26.594 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:26.591+0000 7f3444d1c640 1 --2- 172.21.15.89:0/2345453502 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f34401094a0 0x7f3440075c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:26.594 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:26.591+0000 7f3444d1c640 1 --2- 172.21.15.89:0/2345453502 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3440109df0 0x7f3440076190 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:26.594 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:26.591+0000 7f3444d1c640 1 --2- 172.21.15.89:0/2345453502 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3440103b60 0x7f3440075710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:26.594 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:26.591+0000 7f3444d1c640 1 -- 172.21.15.89:0/2345453502 >> 172.21.15.89:0/2345453502 conn(0x7f344007b850 msgr2=0x7f3440105610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:36:26.594 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:26.591+0000 7f3444d1c640 1 -- 172.21.15.89:0/2345453502 shutdown_connections 2024-09-21T03:36:26.594 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:26.591+0000 7f3444d1c640 1 -- 172.21.15.89:0/2345453502 wait complete. 2024-09-21T03:36:27.215 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-21T03:32:37.834339+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-21T03:36:27.215 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 7c374098-77c9-11ef-baf5-efdc52797490 -- ceph osd pool get .mgr pg_num 2024-09-21T03:36:27.458 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/7c374098-77c9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:36:27.706 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:36:27 smithi089 ceph-mon[30928]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:36:27.706 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:36:27 smithi089 ceph-mon[30928]: from='client.? 172.21.15.89:0/2345453502' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T03:36:27.711 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:36:27 smithi114 ceph-mon[34140]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:36:27.711 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:36:27 smithi114 ceph-mon[34140]: from='client.? 172.21.15.89:0/2345453502' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T03:36:27.750 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:36:27 smithi110 ceph-mon[34000]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:36:27.750 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:36:27 smithi110 ceph-mon[34000]: from='client.? 172.21.15.89:0/2345453502' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T03:36:27.915 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:27.913+0000 7fb5f3006640 1 -- 172.21.15.89:0/3677680536 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb5ec10de10 msgr2=0x7fb5ec110300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:27.916 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:27.913+0000 7fb5f3006640 1 --2- 172.21.15.89:0/3677680536 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb5ec10de10 0x7fb5ec110300 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fb5e4009ca0 tx=0x7fb5e402a8c0 comp rx=0 tx=0).stop 2024-09-21T03:36:27.916 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:27.913+0000 7fb5f3006640 1 -- 172.21.15.89:0/3677680536 shutdown_connections 2024-09-21T03:36:27.916 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:27.913+0000 7fb5f3006640 1 --2- 172.21.15.89:0/3677680536 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb5ec10de10 0x7fb5ec110300 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:27.916 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:27.913+0000 7fb5f3006640 1 --2- 172.21.15.89:0/3677680536 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb5ec105300 0x7fb5ec10d850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:27.916 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:27.913+0000 7fb5f3006640 1 --2- 172.21.15.89:0/3677680536 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb5ec1049b0 0x7fb5ec104dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:27.916 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:27.913+0000 7fb5f3006640 1 -- 172.21.15.89:0/3677680536 >> 172.21.15.89:0/3677680536 conn(0x7fb5ec079730 msgr2=0x7fb5ec079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:36:27.916 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:27.913+0000 7fb5f3006640 1 -- 172.21.15.89:0/3677680536 shutdown_connections 2024-09-21T03:36:27.916 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:27.913+0000 7fb5f3006640 1 -- 172.21.15.89:0/3677680536 wait complete. 2024-09-21T03:36:27.918 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:27.914+0000 7fb5f3006640 1 Processor -- start 2024-09-21T03:36:27.918 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:27.914+0000 7fb5f3006640 1 -- start start 2024-09-21T03:36:27.918 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:27.914+0000 7fb5f3006640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb5ec1049b0 0x7fb5ec19a3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:27.918 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:27.914+0000 7fb5f3006640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb5ec105300 0x7fb5ec19a960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:27.918 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:27.914+0000 7fb5f3006640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb5ec10de10 0x7fb5ec19fc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:27.918 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:27.914+0000 7fb5f3006640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fb5ec113970 con 0x7fb5ec10de10 2024-09-21T03:36:27.919 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:27.914+0000 7fb5f3006640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fb5ec1137f0 con 0x7fb5ec1049b0 2024-09-21T03:36:27.919 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:27.914+0000 7fb5f3006640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fb5ec113af0 con 0x7fb5ec105300 2024-09-21T03:36:27.919 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:27.914+0000 7fb5f15a3640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb5ec10de10 0x7fb5ec19fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:27.919 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:27.914+0000 7fb5f15a3640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb5ec10de10 0x7fb5ec19fc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:34386/0 (socket says 172.21.15.89:34386) 2024-09-21T03:36:27.919 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:27.914+0000 7fb5f15a3640 1 -- 172.21.15.89:0/553443455 learned_addr learned my addr 172.21.15.89:0/553443455 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-21T03:36:27.919 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:27.914+0000 7fb5ebfff640 1 --2- 172.21.15.89:0/553443455 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb5ec105300 0x7fb5ec19a960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:27.919 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:27.914+0000 7fb5f0da2640 1 --2- 172.21.15.89:0/553443455 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb5ec1049b0 0x7fb5ec19a3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:27.919 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:27.915+0000 7fb5f15a3640 1 -- 172.21.15.89:0/553443455 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb5ec105300 msgr2=0x7fb5ec19a960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:27.919 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:27.915+0000 7fb5f15a3640 1 --2- 172.21.15.89:0/553443455 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb5ec105300 0x7fb5ec19a960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:27.919 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:27.915+0000 7fb5f15a3640 1 -- 172.21.15.89:0/553443455 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb5ec1049b0 msgr2=0x7fb5ec19a3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:27.919 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:27.915+0000 7fb5f15a3640 1 --2- 172.21.15.89:0/553443455 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb5ec1049b0 0x7fb5ec19a3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:27.919 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:27.915+0000 7fb5f15a3640 1 -- 172.21.15.89:0/553443455 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5ec1a0270 con 0x7fb5ec10de10 2024-09-21T03:36:27.919 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:27.915+0000 7fb5ebfff640 1 --2- 172.21.15.89:0/553443455 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb5ec105300 0x7fb5ec19a960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:36:27.919 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:27.915+0000 7fb5f15a3640 1 --2- 172.21.15.89:0/553443455 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb5ec10de10 0x7fb5ec19fc00 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fb5e4005cc0 tx=0x7fb5e4032c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:36:27.920 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:27.915+0000 7fb5e9ffb640 1 -- 172.21.15.89:0/553443455 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5e4004040 con 0x7fb5ec10de10 2024-09-21T03:36:27.920 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:27.915+0000 7fb5f3006640 1 -- 172.21.15.89:0/553443455 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb5ec1a0500 con 0x7fb5ec10de10 2024-09-21T03:36:27.920 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:27.915+0000 7fb5e9ffb640 1 -- 172.21.15.89:0/553443455 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb5e4004670 con 0x7fb5ec10de10 2024-09-21T03:36:27.920 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:27.915+0000 7fb5e9ffb640 1 -- 172.21.15.89:0/553443455 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5e4030cc0 con 0x7fb5ec10de10 2024-09-21T03:36:27.920 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:27.915+0000 7fb5f3006640 1 -- 172.21.15.89:0/553443455 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5ec1a6d80 con 0x7fb5ec10de10 2024-09-21T03:36:27.920 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:27.916+0000 7fb5f3006640 1 -- 172.21.15.89:0/553443455 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb5ec104dc0 con 0x7fb5ec10de10 2024-09-21T03:36:27.920 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:27.918+0000 7fb5e9ffb640 1 -- 172.21.15.89:0/553443455 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 17) ==== 154512+0+0 (secure 0 0 0) 0x7fb5e40041e0 con 0x7fb5ec10de10 2024-09-21T03:36:27.921 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:27.919+0000 7fb5e9ffb640 1 --2- 172.21.15.89:0/553443455 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7fb5d40b6540 0x7fb5d40b8a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:27.921 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:27.919+0000 7fb5f0da2640 1 --2- 172.21.15.89:0/553443455 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7fb5d40b6540 0x7fb5d40b8a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:27.921 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:27.919+0000 7fb5e9ffb640 1 -- 172.21.15.89:0/553443455 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 7127+0+0 (secure 0 0 0) 0x7fb5e4108410 con 0x7fb5ec10de10 2024-09-21T03:36:27.921 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:27.919+0000 7fb5f0da2640 1 --2- 172.21.15.89:0/553443455 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7fb5d40b6540 0x7fb5d40b8a50 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fb5ec06bc60 tx=0x7fb5dc005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:36:27.924 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:27.922+0000 7fb5e9ffb640 1 -- 172.21.15.89:0/553443455 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fb5e40cf140 con 0x7fb5ec10de10 2024-09-21T03:36:28.101 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:28.098+0000 7fb5f3006640 1 -- 172.21.15.89:0/553443455 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fb5ec108f10 con 0x7fb5ec10de10 2024-09-21T03:36:28.101 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:28.099+0000 7fb5e9ffb640 1 -- 172.21.15.89:0/553443455 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v69) ==== 93+0+10 (secure 0 0 0) 0x7fb5e40d3520 con 0x7fb5ec10de10 2024-09-21T03:36:28.101 INFO:teuthology.orchestra.run.smithi089.stdout:pg_num: 1 2024-09-21T03:36:28.103 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:28.101+0000 7fb5f3006640 1 -- 172.21.15.89:0/553443455 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7fb5d40b6540 msgr2=0x7fb5d40b8a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:28.103 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:28.101+0000 7fb5f3006640 1 --2- 172.21.15.89:0/553443455 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7fb5d40b6540 0x7fb5d40b8a50 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fb5ec06bc60 tx=0x7fb5dc005f90 comp rx=0 tx=0).stop 2024-09-21T03:36:28.104 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:28.101+0000 7fb5f3006640 1 -- 172.21.15.89:0/553443455 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb5ec10de10 msgr2=0x7fb5ec19fc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:28.104 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:28.101+0000 7fb5f3006640 1 --2- 172.21.15.89:0/553443455 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb5ec10de10 0x7fb5ec19fc00 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fb5e4005cc0 tx=0x7fb5e4032c40 comp rx=0 tx=0).stop 2024-09-21T03:36:28.104 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:28.101+0000 7fb5f3006640 1 -- 172.21.15.89:0/553443455 shutdown_connections 2024-09-21T03:36:28.104 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:28.101+0000 7fb5f3006640 1 --2- 172.21.15.89:0/553443455 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7fb5d40b6540 0x7fb5d40b8a50 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:28.104 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:28.101+0000 7fb5f3006640 1 --2- 172.21.15.89:0/553443455 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb5ec10de10 0x7fb5ec19fc00 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:28.104 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:28.101+0000 7fb5f3006640 1 --2- 172.21.15.89:0/553443455 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb5ec105300 0x7fb5ec19a960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:28.104 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:28.101+0000 7fb5f3006640 1 --2- 172.21.15.89:0/553443455 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb5ec1049b0 0x7fb5ec19a3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:28.104 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:28.101+0000 7fb5f3006640 1 -- 172.21.15.89:0/553443455 >> 172.21.15.89:0/553443455 conn(0x7fb5ec079730 msgr2=0x7fb5ec100770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:36:28.104 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:28.101+0000 7fb5f3006640 1 -- 172.21.15.89:0/553443455 shutdown_connections 2024-09-21T03:36:28.104 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:28.101+0000 7fb5f3006640 1 -- 172.21.15.89:0/553443455 wait complete. 2024-09-21T03:36:28.711 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:36:28 smithi114 ceph-mon[34140]: from='client.? 172.21.15.89:0/553443455' entity='client.admin' cmd={"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} : dispatch 2024-09-21T03:36:28.750 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:36:28 smithi110 ceph-mon[34000]: from='client.? 172.21.15.89:0/553443455' entity='client.admin' cmd={"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} : dispatch 2024-09-21T03:36:28.754 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:36:28 smithi089 ceph-mon[30928]: from='client.? 172.21.15.89:0/553443455' entity='client.admin' cmd={"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} : dispatch 2024-09-21T03:36:28.783 INFO:tasks.cephadm:Adding mds.a on smithi089 2024-09-21T03:36:28.784 INFO:tasks.cephadm:Adding mds.d on smithi089 2024-09-21T03:36:28.784 INFO:tasks.cephadm:Adding mds.g on smithi089 2024-09-21T03:36:28.784 INFO:tasks.cephadm:Adding mds.j on smithi089 2024-09-21T03:36:28.784 INFO:tasks.cephadm:Adding mds.b on smithi110 2024-09-21T03:36:28.784 INFO:tasks.cephadm:Adding mds.e on smithi110 2024-09-21T03:36:28.784 INFO:tasks.cephadm:Adding mds.h on smithi110 2024-09-21T03:36:28.784 INFO:tasks.cephadm:Adding mds.k on smithi110 2024-09-21T03:36:28.784 INFO:tasks.cephadm:Adding mds.c on smithi114 2024-09-21T03:36:28.784 INFO:tasks.cephadm:Adding mds.f on smithi114 2024-09-21T03:36:28.784 INFO:tasks.cephadm:Adding mds.i on smithi114 2024-09-21T03:36:28.784 INFO:tasks.cephadm:Adding mds.l on smithi114 2024-09-21T03:36:28.784 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7c374098-77c9-11ef-baf5-efdc52797490 -- ceph orch apply mds all '12;smithi089=a;smithi089=d;smithi089=g;smithi089=j;smithi110=b;smithi110=e;smithi110=h;smithi110=k;smithi114=c;smithi114=f;smithi114=i;smithi114=l' 2024-09-21T03:36:29.046 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/7c374098-77c9-11ef-baf5-efdc52797490/mon.c/config 2024-09-21T03:36:29.434 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:36:29 smithi114 ceph-mon[34140]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:36:29.534 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T03:36:29.525+0000 7f1d67d0d640 1 -- 172.21.15.114:0/2056724846 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1d60104800 msgr2=0x7f1d60104c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:29.534 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T03:36:29.525+0000 7f1d67d0d640 1 --2- 172.21.15.114:0/2056724846 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1d60104800 0x7f1d60104c10 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f1d50009900 tx=0x7f1d5002a480 comp rx=0 tx=0).stop 2024-09-21T03:36:29.534 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T03:36:29.526+0000 7f1d67d0d640 1 -- 172.21.15.114:0/2056724846 shutdown_connections 2024-09-21T03:36:29.534 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T03:36:29.526+0000 7f1d67d0d640 1 --2- 172.21.15.114:0/2056724846 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1d601051a0 0x7f1d6010bb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:29.534 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T03:36:29.526+0000 7f1d67d0d640 1 --2- 172.21.15.114:0/2056724846 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1d60104800 0x7f1d60104c10 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:29.534 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T03:36:29.526+0000 7f1d67d0d640 1 --2- 172.21.15.114:0/2056724846 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1d60103770 0x7f1d60103be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:29.534 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T03:36:29.526+0000 7f1d67d0d640 1 -- 172.21.15.114:0/2056724846 >> 172.21.15.114:0/2056724846 conn(0x7f1d600ff0e0 msgr2=0x7f1d60101590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:36:29.534 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T03:36:29.526+0000 7f1d67d0d640 1 -- 172.21.15.114:0/2056724846 shutdown_connections 2024-09-21T03:36:29.534 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T03:36:29.526+0000 7f1d67d0d640 1 -- 172.21.15.114:0/2056724846 wait complete. 2024-09-21T03:36:29.534 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T03:36:29.527+0000 7f1d67d0d640 1 Processor -- start 2024-09-21T03:36:29.534 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T03:36:29.527+0000 7f1d67d0d640 1 -- start start 2024-09-21T03:36:29.535 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T03:36:29.527+0000 7f1d67d0d640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1d60103770 0x7f1d60199f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:29.535 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T03:36:29.527+0000 7f1d67d0d640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1d60104800 0x7f1d6019a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:29.535 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T03:36:29.527+0000 7f1d67d0d640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1d601051a0 0x7f1d6019b140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:29.535 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T03:36:29.527+0000 7f1d67d0d640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f1d6010f2d0 con 0x7f1d601051a0 2024-09-21T03:36:29.535 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T03:36:29.527+0000 7f1d67d0d640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f1d6010f150 con 0x7f1d60104800 2024-09-21T03:36:29.535 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T03:36:29.527+0000 7f1d67d0d640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f1d6010f450 con 0x7f1d60103770 2024-09-21T03:36:29.535 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T03:36:29.527+0000 7f1d65aa9640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1d60103770 0x7f1d60199f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:29.535 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T03:36:29.527+0000 7f1d65aa9640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1d60103770 0x7f1d60199f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:55562/0 (socket says 172.21.15.114:55562) 2024-09-21T03:36:29.535 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T03:36:29.527+0000 7f1d65aa9640 1 -- 172.21.15.114:0/799522934 learned_addr learned my addr 172.21.15.114:0/799522934 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T03:36:29.535 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T03:36:29.527+0000 7f1d662aa640 1 --2- 172.21.15.114:0/799522934 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1d601051a0 0x7f1d6019b140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:29.535 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T03:36:29.527+0000 7f1d652a8640 1 --2- 172.21.15.114:0/799522934 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1d60104800 0x7f1d6019a5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:29.535 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T03:36:29.527+0000 7f1d65aa9640 1 -- 172.21.15.114:0/799522934 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1d60104800 msgr2=0x7f1d6019a5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:29.535 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T03:36:29.527+0000 7f1d65aa9640 1 --2- 172.21.15.114:0/799522934 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1d60104800 0x7f1d6019a5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:29.535 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T03:36:29.527+0000 7f1d65aa9640 1 -- 172.21.15.114:0/799522934 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1d601051a0 msgr2=0x7f1d6019b140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:29.536 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T03:36:29.527+0000 7f1d65aa9640 1 --2- 172.21.15.114:0/799522934 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1d601051a0 0x7f1d6019b140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:29.536 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T03:36:29.527+0000 7f1d65aa9640 1 -- 172.21.15.114:0/799522934 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d6019b960 con 0x7f1d60103770 2024-09-21T03:36:29.536 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T03:36:29.528+0000 7f1d65aa9640 1 --2- 172.21.15.114:0/799522934 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1d60103770 0x7f1d60199f30 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f1d5c00d950 tx=0x7f1d5c00de10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:36:29.536 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T03:36:29.528+0000 7f1d56ffd640 1 -- 172.21.15.114:0/799522934 <== mon.1 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d5c017070 con 0x7f1d60103770 2024-09-21T03:36:29.536 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T03:36:29.528+0000 7f1d56ffd640 1 -- 172.21.15.114:0/799522934 <== mon.1 v2:172.21.15.114:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1d5c009c30 con 0x7f1d60103770 2024-09-21T03:36:29.536 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T03:36:29.528+0000 7f1d56ffd640 1 -- 172.21.15.114:0/799522934 <== mon.1 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d5c00f620 con 0x7f1d60103770 2024-09-21T03:36:29.536 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T03:36:29.528+0000 7f1d67d0d640 1 -- 172.21.15.114:0/799522934 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1d6019bc50 con 0x7f1d60103770 2024-09-21T03:36:29.536 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T03:36:29.528+0000 7f1d67d0d640 1 -- 172.21.15.114:0/799522934 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1d601a2350 con 0x7f1d60103770 2024-09-21T03:36:29.536 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T03:36:29.529+0000 7f1d67d0d640 1 -- 172.21.15.114:0/799522934 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1d6010a930 con 0x7f1d60103770 2024-09-21T03:36:29.536 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T03:36:29.530+0000 7f1d56ffd640 1 -- 172.21.15.114:0/799522934 <== mon.1 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 17) ==== 154512+0+0 (secure 0 0 0) 0x7f1d5c010460 con 0x7f1d60103770 2024-09-21T03:36:29.536 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T03:36:29.531+0000 7f1d56ffd640 1 --2- 172.21.15.114:0/799522934 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f1d4c0b6590 0x7f1d4c0b8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:29.537 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T03:36:29.532+0000 7f1d56ffd640 1 -- 172.21.15.114:0/799522934 <== mon.1 v2:172.21.15.114:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 7127+0+0 (secure 0 0 0) 0x7f1d5c0e6960 con 0x7f1d60103770 2024-09-21T03:36:29.537 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T03:36:29.532+0000 7f1d652a8640 1 --2- 172.21.15.114:0/799522934 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f1d4c0b6590 0x7f1d4c0b8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:29.537 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T03:36:29.532+0000 7f1d652a8640 1 --2- 172.21.15.114:0/799522934 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f1d4c0b6590 0x7f1d4c0b8aa0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f1d500057c0 tx=0x7f1d50000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:36:29.538 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T03:36:29.536+0000 7f1d56ffd640 1 -- 172.21.15.114:0/799522934 <== mon.1 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f1d5c009de0 con 0x7f1d60103770 2024-09-21T03:36:29.729 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T03:36:29.727+0000 7f1d67d0d640 1 -- 172.21.15.114:0/799522934 --> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] -- mgr_command(tid 0: {"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi089=a;smithi089=d;smithi089=g;smithi089=j;smithi110=b;smithi110=e;smithi110=h;smithi110=k;smithi114=c;smithi114=f;smithi114=i;smithi114=l", "target": ["mon-mgr", ""]}) -- 0x7f1d600639c0 con 0x7f1d4c0b6590 2024-09-21T03:36:29.740 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T03:36:29.738+0000 7f1d56ffd640 1 -- 172.21.15.114:0/799522934 <== mgr.14152 v2:172.21.15.89:6800/1591532358 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+28 (secure 0 0 0) 0x7f1d600639c0 con 0x7f1d4c0b6590 2024-09-21T03:36:29.740 INFO:teuthology.orchestra.run.smithi114.stdout:Scheduled mds.all update... 2024-09-21T03:36:29.743 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T03:36:29.740+0000 7f1d67d0d640 1 -- 172.21.15.114:0/799522934 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f1d4c0b6590 msgr2=0x7f1d4c0b8aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:29.743 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T03:36:29.740+0000 7f1d67d0d640 1 --2- 172.21.15.114:0/799522934 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f1d4c0b6590 0x7f1d4c0b8aa0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f1d500057c0 tx=0x7f1d50000bc0 comp rx=0 tx=0).stop 2024-09-21T03:36:29.743 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T03:36:29.740+0000 7f1d67d0d640 1 -- 172.21.15.114:0/799522934 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1d60103770 msgr2=0x7f1d60199f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:29.743 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T03:36:29.740+0000 7f1d67d0d640 1 --2- 172.21.15.114:0/799522934 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1d60103770 0x7f1d60199f30 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f1d5c00d950 tx=0x7f1d5c00de10 comp rx=0 tx=0).stop 2024-09-21T03:36:29.743 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T03:36:29.740+0000 7f1d67d0d640 1 -- 172.21.15.114:0/799522934 shutdown_connections 2024-09-21T03:36:29.743 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T03:36:29.740+0000 7f1d67d0d640 1 --2- 172.21.15.114:0/799522934 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f1d4c0b6590 0x7f1d4c0b8aa0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:29.743 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T03:36:29.740+0000 7f1d67d0d640 1 --2- 172.21.15.114:0/799522934 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1d60104800 0x7f1d6019a5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:29.744 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T03:36:29.740+0000 7f1d67d0d640 1 --2- 172.21.15.114:0/799522934 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1d601051a0 0x7f1d6019b140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:29.744 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T03:36:29.740+0000 7f1d67d0d640 1 --2- 172.21.15.114:0/799522934 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1d60103770 0x7f1d60199f30 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:29.744 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T03:36:29.740+0000 7f1d67d0d640 1 -- 172.21.15.114:0/799522934 >> 172.21.15.114:0/799522934 conn(0x7f1d600ff0e0 msgr2=0x7f1d60107b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:36:29.744 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T03:36:29.740+0000 7f1d67d0d640 1 -- 172.21.15.114:0/799522934 shutdown_connections 2024-09-21T03:36:29.744 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T03:36:29.740+0000 7f1d67d0d640 1 -- 172.21.15.114:0/799522934 wait complete. 2024-09-21T03:36:29.750 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:36:29 smithi110 ceph-mon[34000]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:36:29.811 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:36:29 smithi089 ceph-mon[30928]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:36:30.442 DEBUG:teuthology.orchestra.run.smithi089:mds.a> sudo journalctl -f -n 0 -u ceph-7c374098-77c9-11ef-baf5-efdc52797490@mds.a.service 2024-09-21T03:36:30.445 DEBUG:teuthology.orchestra.run.smithi089:mds.d> sudo journalctl -f -n 0 -u ceph-7c374098-77c9-11ef-baf5-efdc52797490@mds.d.service 2024-09-21T03:36:30.488 DEBUG:teuthology.orchestra.run.smithi089:mds.g> sudo journalctl -f -n 0 -u ceph-7c374098-77c9-11ef-baf5-efdc52797490@mds.g.service 2024-09-21T03:36:30.531 DEBUG:teuthology.orchestra.run.smithi089:mds.j> sudo journalctl -f -n 0 -u ceph-7c374098-77c9-11ef-baf5-efdc52797490@mds.j.service 2024-09-21T03:36:30.574 DEBUG:teuthology.orchestra.run.smithi110:mds.b> sudo journalctl -f -n 0 -u ceph-7c374098-77c9-11ef-baf5-efdc52797490@mds.b.service 2024-09-21T03:36:30.576 DEBUG:teuthology.orchestra.run.smithi110:mds.e> sudo journalctl -f -n 0 -u ceph-7c374098-77c9-11ef-baf5-efdc52797490@mds.e.service 2024-09-21T03:36:30.620 DEBUG:teuthology.orchestra.run.smithi110:mds.h> sudo journalctl -f -n 0 -u ceph-7c374098-77c9-11ef-baf5-efdc52797490@mds.h.service 2024-09-21T03:36:30.663 DEBUG:teuthology.orchestra.run.smithi110:mds.k> sudo journalctl -f -n 0 -u ceph-7c374098-77c9-11ef-baf5-efdc52797490@mds.k.service 2024-09-21T03:36:30.705 DEBUG:teuthology.orchestra.run.smithi114:mds.c> sudo journalctl -f -n 0 -u ceph-7c374098-77c9-11ef-baf5-efdc52797490@mds.c.service 2024-09-21T03:36:30.708 DEBUG:teuthology.orchestra.run.smithi114:mds.f> sudo journalctl -f -n 0 -u ceph-7c374098-77c9-11ef-baf5-efdc52797490@mds.f.service 2024-09-21T03:36:30.750 DEBUG:teuthology.orchestra.run.smithi114:mds.i> sudo journalctl -f -n 0 -u ceph-7c374098-77c9-11ef-baf5-efdc52797490@mds.i.service 2024-09-21T03:36:30.793 DEBUG:teuthology.orchestra.run.smithi114:mds.l> sudo journalctl -f -n 0 -u ceph-7c374098-77c9-11ef-baf5-efdc52797490@mds.l.service 2024-09-21T03:36:30.836 INFO:tasks.cephadm:Setting up CephFS filesystem(s)... 2024-09-21T03:36:30.837 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 7c374098-77c9-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T03:36:30.961 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:36:30 smithi114 ceph-mon[34140]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:36:30.961 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:36:30 smithi114 ceph-mon[34140]: from='client.24556 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi089=a;smithi089=d;smithi089=g;smithi089=j;smithi110=b;smithi110=e;smithi110=h;smithi110=k;smithi114=c;smithi114=f;smithi114=i;smithi114=l", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:36:30.961 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:36:30 smithi114 ceph-mon[34140]: Saving service mds.all spec with placement smithi089=a;smithi089=d;smithi089=g;smithi089=j;smithi110=b;smithi110=e;smithi110=h;smithi110=k;smithi114=c;smithi114=f;smithi114=i;smithi114=l;count:12 2024-09-21T03:36:30.961 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:36:30 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:36:30.961 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:36:30 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:36:30.961 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:36:30 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:36:30.961 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:36:30 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:36:30.961 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:36:30 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:36:30.961 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:36:30 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T03:36:30.961 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:36:30 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T03:36:30.962 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:36:30 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:36:30.962 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:36:30 smithi114 ceph-mon[34140]: Deploying daemon mds.l on smithi114 2024-09-21T03:36:31.000 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:36:30 smithi110 ceph-mon[34000]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:36:31.000 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:36:30 smithi110 ceph-mon[34000]: from='client.24556 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi089=a;smithi089=d;smithi089=g;smithi089=j;smithi110=b;smithi110=e;smithi110=h;smithi110=k;smithi114=c;smithi114=f;smithi114=i;smithi114=l", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:36:31.000 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:36:30 smithi110 ceph-mon[34000]: Saving service mds.all spec with placement smithi089=a;smithi089=d;smithi089=g;smithi089=j;smithi110=b;smithi110=e;smithi110=h;smithi110=k;smithi114=c;smithi114=f;smithi114=i;smithi114=l;count:12 2024-09-21T03:36:31.000 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:36:30 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:36:31.000 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:36:30 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:36:31.000 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:36:30 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:36:31.001 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:36:30 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:36:31.001 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:36:30 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:36:31.001 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:36:30 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T03:36:31.001 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:36:30 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T03:36:31.001 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:36:30 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:36:31.001 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:36:30 smithi110 ceph-mon[34000]: Deploying daemon mds.l on smithi114 2024-09-21T03:36:31.061 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:36:30 smithi089 ceph-mon[30928]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:36:31.061 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:36:30 smithi089 ceph-mon[30928]: from='client.24556 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi089=a;smithi089=d;smithi089=g;smithi089=j;smithi110=b;smithi110=e;smithi110=h;smithi110=k;smithi114=c;smithi114=f;smithi114=i;smithi114=l", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:36:31.061 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:36:30 smithi089 ceph-mon[30928]: Saving service mds.all spec with placement smithi089=a;smithi089=d;smithi089=g;smithi089=j;smithi110=b;smithi110=e;smithi110=h;smithi110=k;smithi114=c;smithi114=f;smithi114=i;smithi114=l;count:12 2024-09-21T03:36:31.061 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:36:30 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:36:31.061 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:36:30 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:36:31.061 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:36:30 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:36:31.061 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:36:30 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:36:31.061 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:36:30 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:36:31.061 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:36:30 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T03:36:31.062 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:36:30 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T03:36:31.062 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:36:30 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:36:31.062 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:36:30 smithi089 ceph-mon[30928]: Deploying daemon mds.l on smithi114 2024-09-21T03:36:31.080 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/7c374098-77c9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:36:31.559 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:31.538+0000 7f4928c5e640 1 -- 172.21.15.89:0/563224294 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4924102aa0 msgr2=0x7f4924102eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:31.559 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:31.538+0000 7f4928c5e640 1 --2- 172.21.15.89:0/563224294 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4924102aa0 0x7f4924102eb0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f4914009ca0 tx=0x7f491402a8c0 comp rx=0 tx=0).stop 2024-09-21T03:36:31.559 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:31.538+0000 7f4928c5e640 1 -- 172.21.15.89:0/563224294 shutdown_connections 2024-09-21T03:36:31.559 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:31.538+0000 7f4928c5e640 1 --2- 172.21.15.89:0/563224294 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4924108d30 0x7f492410f560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:31.559 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:31.538+0000 7f4928c5e640 1 --2- 172.21.15.89:0/563224294 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f49241083e0 0x7f49241087f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:31.559 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:31.538+0000 7f4928c5e640 1 --2- 172.21.15.89:0/563224294 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4924102aa0 0x7f4924102eb0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:31.559 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:31.538+0000 7f4928c5e640 1 -- 172.21.15.89:0/563224294 >> 172.21.15.89:0/563224294 conn(0x7f49240797e0 msgr2=0x7f4924079bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:36:31.559 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:31.538+0000 7f4928c5e640 1 -- 172.21.15.89:0/563224294 shutdown_connections 2024-09-21T03:36:31.559 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:31.538+0000 7f4928c5e640 1 -- 172.21.15.89:0/563224294 wait complete. 2024-09-21T03:36:31.559 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:31.539+0000 7f4928c5e640 1 Processor -- start 2024-09-21T03:36:31.559 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:31.539+0000 7f4928c5e640 1 -- start start 2024-09-21T03:36:31.560 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:31.539+0000 7f4928c5e640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4924102aa0 0x7f492419b1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:31.560 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:31.539+0000 7f4928c5e640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f49241083e0 0x7f492419b710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:31.560 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:31.539+0000 7f4928c5e640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4924108d30 0x7f492419bc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:31.560 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:31.539+0000 7f4928c5e640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f49241128a0 con 0x7f49241083e0 2024-09-21T03:36:31.560 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:31.539+0000 7f4928c5e640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f4924112720 con 0x7f4924102aa0 2024-09-21T03:36:31.560 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:31.539+0000 7f4928c5e640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f4924112a20 con 0x7f4924108d30 2024-09-21T03:36:31.560 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:31.539+0000 7f4921d9b640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f49241083e0 0x7f492419b710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:31.560 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:31.539+0000 7f4921d9b640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f49241083e0 0x7f492419b710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:52428/0 (socket says 172.21.15.89:52428) 2024-09-21T03:36:31.560 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:31.539+0000 7f4921d9b640 1 -- 172.21.15.89:0/720464059 learned_addr learned my addr 172.21.15.89:0/720464059 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-21T03:36:31.560 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:31.539+0000 7f492259c640 1 --2- 172.21.15.89:0/720464059 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4924102aa0 0x7f492419b1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:31.560 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:31.539+0000 7f4922d9d640 1 --2- 172.21.15.89:0/720464059 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4924108d30 0x7f492419bc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:31.560 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:31.539+0000 7f4921d9b640 1 -- 172.21.15.89:0/720464059 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4924108d30 msgr2=0x7f492419bc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:31.560 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:31.539+0000 7f4921d9b640 1 --2- 172.21.15.89:0/720464059 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4924108d30 0x7f492419bc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:31.561 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:31.539+0000 7f4921d9b640 1 -- 172.21.15.89:0/720464059 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4924102aa0 msgr2=0x7f492419b1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:31.561 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:31.539+0000 7f4921d9b640 1 --2- 172.21.15.89:0/720464059 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4924102aa0 0x7f492419b1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:31.561 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:31.539+0000 7f4921d9b640 1 -- 172.21.15.89:0/720464059 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49241a63d0 con 0x7f49241083e0 2024-09-21T03:36:31.561 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:31.540+0000 7f4922d9d640 1 --2- 172.21.15.89:0/720464059 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4924108d30 0x7f492419bc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:36:31.561 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:31.540+0000 7f4921d9b640 1 --2- 172.21.15.89:0/720464059 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f49241083e0 0x7f492419b710 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f491800c970 tx=0x7f491800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:36:31.561 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:31.540+0000 7f49137fe640 1 -- 172.21.15.89:0/720464059 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4918007bb0 con 0x7f49241083e0 2024-09-21T03:36:31.561 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:31.540+0000 7f49137fe640 1 -- 172.21.15.89:0/720464059 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4918007d50 con 0x7f49241083e0 2024-09-21T03:36:31.561 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:31.540+0000 7f4928c5e640 1 -- 172.21.15.89:0/720464059 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f49241a66c0 con 0x7f49241083e0 2024-09-21T03:36:31.561 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:31.540+0000 7f49137fe640 1 -- 172.21.15.89:0/720464059 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4918018660 con 0x7f49241083e0 2024-09-21T03:36:31.561 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:31.540+0000 7f4928c5e640 1 -- 172.21.15.89:0/720464059 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49241a6bb0 con 0x7f49241083e0 2024-09-21T03:36:31.561 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:31.541+0000 7f4928c5e640 1 -- 172.21.15.89:0/720464059 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4924102eb0 con 0x7f49241083e0 2024-09-21T03:36:31.561 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:31.543+0000 7f49137fe640 1 -- 172.21.15.89:0/720464059 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 17) ==== 154512+0+0 (secure 0 0 0) 0x7f491800f970 con 0x7f49241083e0 2024-09-21T03:36:31.561 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:31.544+0000 7f49137fe640 1 --2- 172.21.15.89:0/720464059 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f490c0b6540 0x7f490c0b8a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:31.561 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:31.544+0000 7f492259c640 1 --2- 172.21.15.89:0/720464059 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f490c0b6540 0x7f490c0b8a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:31.562 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:31.544+0000 7f49137fe640 1 -- 172.21.15.89:0/720464059 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 7127+0+0 (secure 0 0 0) 0x7f49180e7270 con 0x7f49241083e0 2024-09-21T03:36:31.562 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:31.544+0000 7f492259c640 1 --2- 172.21.15.89:0/720464059 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f490c0b6540 0x7f490c0b8a50 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f4914000f80 tx=0x7f4914031000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:36:31.562 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:31.547+0000 7f49137fe640 1 -- 172.21.15.89:0/720464059 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f4918014030 con 0x7f49241083e0 2024-09-21T03:36:31.764 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:31.762+0000 7f4928c5e640 1 -- 172.21.15.89:0/720464059 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f492406bc30 con 0x7f49241083e0 2024-09-21T03:36:31.765 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:31.763+0000 7f49137fe640 1 -- 172.21.15.89:0/720464059 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 1 v1) ==== 93+0+611 (secure 0 0 0) 0x7f492406bc30 con 0x7f49241083e0 2024-09-21T03:36:31.765 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-09-21T03:36:31.765 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":1,"btime":"2024-09-21T03:28:56:681239+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[]} 2024-09-21T03:36:31.766 INFO:teuthology.orchestra.run.smithi089.stderr:dumped fsmap epoch 1 2024-09-21T03:36:31.767 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:31.765+0000 7f4928c5e640 1 -- 172.21.15.89:0/720464059 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f490c0b6540 msgr2=0x7f490c0b8a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:31.767 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:31.765+0000 7f4928c5e640 1 --2- 172.21.15.89:0/720464059 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f490c0b6540 0x7f490c0b8a50 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f4914000f80 tx=0x7f4914031000 comp rx=0 tx=0).stop 2024-09-21T03:36:31.768 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:31.765+0000 7f4928c5e640 1 -- 172.21.15.89:0/720464059 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f49241083e0 msgr2=0x7f492419b710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:31.768 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:31.765+0000 7f4928c5e640 1 --2- 172.21.15.89:0/720464059 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f49241083e0 0x7f492419b710 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f491800c970 tx=0x7f491800ce30 comp rx=0 tx=0).stop 2024-09-21T03:36:31.768 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:31.765+0000 7f4928c5e640 1 -- 172.21.15.89:0/720464059 shutdown_connections 2024-09-21T03:36:31.768 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:31.765+0000 7f4928c5e640 1 --2- 172.21.15.89:0/720464059 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f490c0b6540 0x7f490c0b8a50 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:31.768 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:31.765+0000 7f4928c5e640 1 --2- 172.21.15.89:0/720464059 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4924108d30 0x7f492419bc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:31.768 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:31.765+0000 7f4928c5e640 1 --2- 172.21.15.89:0/720464059 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f49241083e0 0x7f492419b710 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:31.768 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:31.765+0000 7f4928c5e640 1 --2- 172.21.15.89:0/720464059 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4924102aa0 0x7f492419b1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:31.768 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:31.765+0000 7f4928c5e640 1 -- 172.21.15.89:0/720464059 >> 172.21.15.89:0/720464059 conn(0x7f49240797e0 msgr2=0x7f49241014d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:36:31.768 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:31.765+0000 7f4928c5e640 1 -- 172.21.15.89:0/720464059 shutdown_connections 2024-09-21T03:36:31.768 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:31.765+0000 7f4928c5e640 1 -- 172.21.15.89:0/720464059 wait complete. 2024-09-21T03:36:32.365 INFO:journalctl@ceph.mds.l.smithi114.stdout:Sep 21 03:36:32 smithi114 systemd[1]: Starting Ceph mds.l for 7c374098-77c9-11ef-baf5-efdc52797490... 2024-09-21T03:36:32.621 INFO:journalctl@ceph.mds.l.smithi114.stdout:Sep 21 03:36:32 smithi114 podman[62416]: 2024-09-21 03:36:32.266116657 +0000 UTC m=+0.015685357 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T03:36:32.621 INFO:journalctl@ceph.mds.l.smithi114.stdout:Sep 21 03:36:32 smithi114 podman[62416]: 2024-09-21 03:36:32.372558866 +0000 UTC m=+0.122127598 container create 6dbceef3df18ca4ec88a90127f37432ede0905d07c3c2fdade2d7015a094fb78 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-7c374098-77c9-11ef-baf5-efdc52797490-mds-l, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-21T03:36:32.621 INFO:journalctl@ceph.mds.l.smithi114.stdout:Sep 21 03:36:32 smithi114 podman[62416]: 2024-09-21 03:36:32.583562346 +0000 UTC m=+0.333131044 container init 6dbceef3df18ca4ec88a90127f37432ede0905d07c3c2fdade2d7015a094fb78 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-7c374098-77c9-11ef-baf5-efdc52797490-mds-l, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-21T03:36:32.621 INFO:journalctl@ceph.mds.l.smithi114.stdout:Sep 21 03:36:32 smithi114 podman[62416]: 2024-09-21 03:36:32.586530286 +0000 UTC m=+0.336098980 container start 6dbceef3df18ca4ec88a90127f37432ede0905d07c3c2fdade2d7015a094fb78 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-7c374098-77c9-11ef-baf5-efdc52797490-mds-l, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux ) 2024-09-21T03:36:32.621 INFO:journalctl@ceph.mds.l.smithi114.stdout:Sep 21 03:36:32 smithi114 ceph-mds[62435]: -- 172.21.15.114:0/1822353655 <== mon.1 v2:172.21.15.114:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x562686d61860 con 0x56268af09a80 2024-09-21T03:36:32.961 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:36:32 smithi114 ceph-mon[34140]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:36:32.961 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:36:32 smithi114 ceph-mon[34140]: from='client.? 172.21.15.89:0/720464059' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:36:32.961 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:36:32 smithi114 ceph-mon[34140]: mds.? [v2:172.21.15.114:6832/752336912,v1:172.21.15.114:6833/752336912] up:boot 2024-09-21T03:36:32.961 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:36:32 smithi114 ceph-mon[34140]: fsmap 1 up:standby 2024-09-21T03:36:32.961 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:36:32 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "l"} : dispatch 2024-09-21T03:36:32.961 INFO:journalctl@ceph.mds.l.smithi114.stdout:Sep 21 03:36:32 smithi114 ceph-7c374098-77c9-11ef-baf5-efdc52797490-mds-l[62431]: starting mds.l at 2024-09-21T03:36:32.961 INFO:journalctl@ceph.mds.l.smithi114.stdout:Sep 21 03:36:32 smithi114 bash[62416]: 6dbceef3df18ca4ec88a90127f37432ede0905d07c3c2fdade2d7015a094fb78 2024-09-21T03:36:32.961 INFO:journalctl@ceph.mds.l.smithi114.stdout:Sep 21 03:36:32 smithi114 systemd[1]: Started Ceph mds.l for 7c374098-77c9-11ef-baf5-efdc52797490. 2024-09-21T03:36:33.000 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:36:32 smithi110 ceph-mon[34000]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:36:33.000 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:36:32 smithi110 ceph-mon[34000]: from='client.? 172.21.15.89:0/720464059' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:36:33.000 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:36:32 smithi110 ceph-mon[34000]: mds.? [v2:172.21.15.114:6832/752336912,v1:172.21.15.114:6833/752336912] up:boot 2024-09-21T03:36:33.000 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:36:32 smithi110 ceph-mon[34000]: fsmap 1 up:standby 2024-09-21T03:36:33.000 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:36:32 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "l"} : dispatch 2024-09-21T03:36:33.062 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:36:32 smithi089 ceph-mon[30928]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:36:33.062 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:36:32 smithi089 ceph-mon[30928]: from='client.? 172.21.15.89:0/720464059' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:36:33.062 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:36:32 smithi089 ceph-mon[30928]: mds.? [v2:172.21.15.114:6832/752336912,v1:172.21.15.114:6833/752336912] up:boot 2024-09-21T03:36:33.062 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:36:32 smithi089 ceph-mon[30928]: fsmap 1 up:standby 2024-09-21T03:36:33.062 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:36:32 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "l"} : dispatch 2024-09-21T03:36:34.000 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:36:33 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:36:34.001 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:36:33 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:36:34.001 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:36:33 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:36:34.001 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:36:33 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T03:36:34.001 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:36:33 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T03:36:34.001 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:36:33 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:36:34.001 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:36:33 smithi110 ceph-mon[34000]: Deploying daemon mds.k on smithi110 2024-09-21T03:36:34.061 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:36:33 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:36:34.061 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:36:33 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:36:34.062 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:36:33 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:36:34.062 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:36:33 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T03:36:34.062 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:36:33 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T03:36:34.062 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:36:33 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:36:34.062 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:36:33 smithi089 ceph-mon[30928]: Deploying daemon mds.k on smithi110 2024-09-21T03:36:34.211 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:36:33 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:36:34.211 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:36:33 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:36:34.211 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:36:33 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:36:34.211 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:36:33 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T03:36:34.211 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:36:33 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T03:36:34.211 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:36:33 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:36:34.212 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:36:33 smithi114 ceph-mon[34140]: Deploying daemon mds.k on smithi110 2024-09-21T03:36:34.489 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 7c374098-77c9-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T03:36:34.746 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/7c374098-77c9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:36:34.811 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:36:34 smithi089 ceph-mon[30928]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:36:34.998 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:36:34 smithi110 ceph-mon[34000]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:36:34.998 INFO:journalctl@ceph.mds.k.smithi110.stdout:Sep 21 03:36:34 smithi110 systemd[1]: Starting Ceph mds.k for 7c374098-77c9-11ef-baf5-efdc52797490... 2024-09-21T03:36:35.211 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:36:34 smithi114 ceph-mon[34140]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:36:35.216 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:35.214+0000 7f7c55fc0640 1 -- 172.21.15.89:0/251672793 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7c501083d0 msgr2=0x7f7c501087e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:35.216 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:35.214+0000 7f7c55fc0640 1 --2- 172.21.15.89:0/251672793 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7c501083d0 0x7f7c501087e0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f7c44009900 tx=0x7f7c4402a410 comp rx=0 tx=0).stop 2024-09-21T03:36:35.216 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:35.214+0000 7f7c55fc0640 1 -- 172.21.15.89:0/251672793 shutdown_connections 2024-09-21T03:36:35.216 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:35.214+0000 7f7c55fc0640 1 --2- 172.21.15.89:0/251672793 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7c50108d20 0x7f7c5010f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:35.216 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:35.214+0000 7f7c55fc0640 1 --2- 172.21.15.89:0/251672793 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7c501083d0 0x7f7c501087e0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:35.216 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:35.214+0000 7f7c55fc0640 1 --2- 172.21.15.89:0/251672793 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7c50102a90 0x7f7c50102ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:35.217 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:35.214+0000 7f7c55fc0640 1 -- 172.21.15.89:0/251672793 >> 172.21.15.89:0/251672793 conn(0x7f7c50079750 msgr2=0x7f7c50079b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:36:35.217 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:35.214+0000 7f7c55fc0640 1 -- 172.21.15.89:0/251672793 shutdown_connections 2024-09-21T03:36:35.217 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:35.214+0000 7f7c55fc0640 1 -- 172.21.15.89:0/251672793 wait complete. 2024-09-21T03:36:35.218 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:35.215+0000 7f7c55fc0640 1 Processor -- start 2024-09-21T03:36:35.218 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:35.215+0000 7f7c55fc0640 1 -- start start 2024-09-21T03:36:35.218 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:35.215+0000 7f7c55fc0640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7c50102a90 0x7f7c501992b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:35.218 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:35.215+0000 7f7c55fc0640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7c501083d0 0x7f7c50199850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:35.218 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:35.215+0000 7f7c55fc0640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7c50108d20 0x7f7c5019eaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:35.218 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:35.215+0000 7f7c55fc0640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f7c50112840 con 0x7f7c501083d0 2024-09-21T03:36:35.219 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:35.215+0000 7f7c55fc0640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f7c501126c0 con 0x7f7c50108d20 2024-09-21T03:36:35.219 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:35.215+0000 7f7c55fc0640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f7c501129c0 con 0x7f7c50102a90 2024-09-21T03:36:35.219 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:35.215+0000 7f7c4effd640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7c501083d0 0x7f7c50199850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:35.219 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:35.215+0000 7f7c4effd640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7c501083d0 0x7f7c50199850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:52440/0 (socket says 172.21.15.89:52440) 2024-09-21T03:36:35.219 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:35.215+0000 7f7c4effd640 1 -- 172.21.15.89:0/2237766017 learned_addr learned my addr 172.21.15.89:0/2237766017 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-21T03:36:35.219 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:35.215+0000 7f7c4f7fe640 1 --2- 172.21.15.89:0/2237766017 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7c50102a90 0x7f7c501992b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:35.219 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:35.215+0000 7f7c4ffff640 1 --2- 172.21.15.89:0/2237766017 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7c50108d20 0x7f7c5019eaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:35.219 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:35.216+0000 7f7c4effd640 1 -- 172.21.15.89:0/2237766017 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7c50102a90 msgr2=0x7f7c501992b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:35.219 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:35.216+0000 7f7c4effd640 1 --2- 172.21.15.89:0/2237766017 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7c50102a90 0x7f7c501992b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:35.219 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:35.216+0000 7f7c4effd640 1 -- 172.21.15.89:0/2237766017 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7c50108d20 msgr2=0x7f7c5019eaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:35.219 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:35.216+0000 7f7c4effd640 1 --2- 172.21.15.89:0/2237766017 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7c50108d20 0x7f7c5019eaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:35.219 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:35.216+0000 7f7c4effd640 1 -- 172.21.15.89:0/2237766017 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c5019f220 con 0x7f7c501083d0 2024-09-21T03:36:35.219 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:35.216+0000 7f7c4effd640 1 --2- 172.21.15.89:0/2237766017 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7c501083d0 0x7f7c50199850 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f7c44005940 tx=0x7f7c440042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:36:35.219 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:35.216+0000 7f7c4cff9640 1 -- 172.21.15.89:0/2237766017 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c44035030 con 0x7f7c501083d0 2024-09-21T03:36:35.220 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:35.216+0000 7f7c55fc0640 1 -- 172.21.15.89:0/2237766017 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7c5019f4b0 con 0x7f7c501083d0 2024-09-21T03:36:35.220 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:35.216+0000 7f7c4cff9640 1 -- 172.21.15.89:0/2237766017 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7c440403e0 con 0x7f7c501083d0 2024-09-21T03:36:35.220 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:35.216+0000 7f7c4cff9640 1 -- 172.21.15.89:0/2237766017 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c440044e0 con 0x7f7c501083d0 2024-09-21T03:36:35.220 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:35.216+0000 7f7c55fc0640 1 -- 172.21.15.89:0/2237766017 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7c5006bb80 con 0x7f7c501083d0 2024-09-21T03:36:35.220 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:35.217+0000 7f7c55fc0640 1 -- 172.21.15.89:0/2237766017 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7c50103d10 con 0x7f7c501083d0 2024-09-21T03:36:35.221 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:35.219+0000 7f7c4cff9640 1 -- 172.21.15.89:0/2237766017 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 17) ==== 154512+0+0 (secure 0 0 0) 0x7f7c44040890 con 0x7f7c501083d0 2024-09-21T03:36:35.222 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:35.220+0000 7f7c4cff9640 1 --2- 172.21.15.89:0/2237766017 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f7c380b6510 0x7f7c380b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:35.222 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:35.220+0000 7f7c4f7fe640 1 --2- 172.21.15.89:0/2237766017 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f7c380b6510 0x7f7c380b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:35.223 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:35.220+0000 7f7c4cff9640 1 -- 172.21.15.89:0/2237766017 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 7127+0+0 (secure 0 0 0) 0x7f7c441077a0 con 0x7f7c501083d0 2024-09-21T03:36:35.223 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:35.221+0000 7f7c4f7fe640 1 --2- 172.21.15.89:0/2237766017 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f7c380b6510 0x7f7c380b8a20 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f7c40005d00 tx=0x7f7c4000b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:36:35.225 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:35.223+0000 7f7c4cff9640 1 -- 172.21.15.89:0/2237766017 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f7c440ce4d0 con 0x7f7c501083d0 2024-09-21T03:36:35.250 INFO:journalctl@ceph.mds.k.smithi110.stdout:Sep 21 03:36:34 smithi110 podman[61262]: 2024-09-21 03:36:34.899327818 +0000 UTC m=+0.015419059 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T03:36:35.250 INFO:journalctl@ceph.mds.k.smithi110.stdout:Sep 21 03:36:35 smithi110 podman[61262]: 2024-09-21 03:36:35.025995675 +0000 UTC m=+0.142086919 container create c0c7a27c8983c1475a15f7e3d2c47bf505f2b2a4808e0896bd0287331b6094fd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-7c374098-77c9-11ef-baf5-efdc52797490-mds-k, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221) 2024-09-21T03:36:35.443 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:35.440+0000 7f7c55fc0640 1 -- 172.21.15.89:0/2237766017 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7c50102ea0 con 0x7f7c501083d0 2024-09-21T03:36:35.443 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:35.441+0000 7f7c4cff9640 1 -- 172.21.15.89:0/2237766017 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) ==== 93+0+1435 (secure 0 0 0) 0x7f7c50102ea0 con 0x7f7c501083d0 2024-09-21T03:36:35.444 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-09-21T03:36:35.444 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":2,"btime":"2024-09-21T03:36:32:624677+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24565,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6833/752336912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6832","nonce":752336912},{"type":"v1","addr":"172.21.15.114:6833","nonce":752336912}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-09-21T03:36:35.444 INFO:teuthology.orchestra.run.smithi089.stderr:dumped fsmap epoch 2 2024-09-21T03:36:35.446 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:35.444+0000 7f7c55fc0640 1 -- 172.21.15.89:0/2237766017 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f7c380b6510 msgr2=0x7f7c380b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:35.446 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:35.444+0000 7f7c55fc0640 1 --2- 172.21.15.89:0/2237766017 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f7c380b6510 0x7f7c380b8a20 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f7c40005d00 tx=0x7f7c4000b410 comp rx=0 tx=0).stop 2024-09-21T03:36:35.446 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:35.444+0000 7f7c55fc0640 1 -- 172.21.15.89:0/2237766017 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7c501083d0 msgr2=0x7f7c50199850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:35.446 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:35.444+0000 7f7c55fc0640 1 --2- 172.21.15.89:0/2237766017 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7c501083d0 0x7f7c50199850 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f7c44005940 tx=0x7f7c440042f0 comp rx=0 tx=0).stop 2024-09-21T03:36:35.447 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:35.444+0000 7f7c55fc0640 1 -- 172.21.15.89:0/2237766017 shutdown_connections 2024-09-21T03:36:35.447 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:35.444+0000 7f7c55fc0640 1 --2- 172.21.15.89:0/2237766017 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f7c380b6510 0x7f7c380b8a20 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:35.447 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:35.444+0000 7f7c55fc0640 1 --2- 172.21.15.89:0/2237766017 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7c50108d20 0x7f7c5019eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:35.447 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:35.444+0000 7f7c55fc0640 1 --2- 172.21.15.89:0/2237766017 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7c501083d0 0x7f7c50199850 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:35.447 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:35.444+0000 7f7c55fc0640 1 --2- 172.21.15.89:0/2237766017 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7c50102a90 0x7f7c501992b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:35.447 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:35.444+0000 7f7c55fc0640 1 -- 172.21.15.89:0/2237766017 >> 172.21.15.89:0/2237766017 conn(0x7f7c50079750 msgr2=0x7f7c501044c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:36:35.447 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:35.444+0000 7f7c55fc0640 1 -- 172.21.15.89:0/2237766017 shutdown_connections 2024-09-21T03:36:35.447 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:35.444+0000 7f7c55fc0640 1 -- 172.21.15.89:0/2237766017 wait complete. 2024-09-21T03:36:35.750 INFO:journalctl@ceph.mds.k.smithi110.stdout:Sep 21 03:36:35 smithi110 podman[61262]: 2024-09-21 03:36:35.271645248 +0000 UTC m=+0.387736493 container init c0c7a27c8983c1475a15f7e3d2c47bf505f2b2a4808e0896bd0287331b6094fd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-7c374098-77c9-11ef-baf5-efdc52797490-mds-k, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux ) 2024-09-21T03:36:35.750 INFO:journalctl@ceph.mds.k.smithi110.stdout:Sep 21 03:36:35 smithi110 podman[61262]: 2024-09-21 03:36:35.274734539 +0000 UTC m=+0.390825783 container start c0c7a27c8983c1475a15f7e3d2c47bf505f2b2a4808e0896bd0287331b6094fd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-7c374098-77c9-11ef-baf5-efdc52797490-mds-k, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-21T03:36:35.750 INFO:journalctl@ceph.mds.k.smithi110.stdout:Sep 21 03:36:35 smithi110 ceph-mds[61283]: -- 172.21.15.110:0/1687094076 <== mon.2 v2:172.21.15.110:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x56324327f860 con 0x563247427600 2024-09-21T03:36:35.750 INFO:journalctl@ceph.mds.k.smithi110.stdout:Sep 21 03:36:35 smithi110 ceph-7c374098-77c9-11ef-baf5-efdc52797490-mds-k[61279]: starting mds.k at 2024-09-21T03:36:35.750 INFO:journalctl@ceph.mds.k.smithi110.stdout:Sep 21 03:36:35 smithi110 bash[61262]: c0c7a27c8983c1475a15f7e3d2c47bf505f2b2a4808e0896bd0287331b6094fd 2024-09-21T03:36:35.750 INFO:journalctl@ceph.mds.k.smithi110.stdout:Sep 21 03:36:35 smithi110 systemd[1]: Started Ceph mds.k for 7c374098-77c9-11ef-baf5-efdc52797490. 2024-09-21T03:36:36.711 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:36:36 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:36:36.711 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:36:36 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:36:36.711 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:36:36 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:36:36.711 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:36:36 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T03:36:36.711 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:36:36 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T03:36:36.711 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:36:36 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:36:36.711 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:36:36 smithi114 ceph-mon[34140]: Deploying daemon mds.j on smithi089 2024-09-21T03:36:36.711 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:36:36 smithi114 ceph-mon[34140]: from='client.? 172.21.15.89:0/2237766017' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:36:36.712 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:36:36 smithi114 ceph-mon[34140]: mds.? [v2:172.21.15.110:6832/127607331,v1:172.21.15.110:6833/127607331] up:boot 2024-09-21T03:36:36.712 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:36:36 smithi114 ceph-mon[34140]: fsmap 2 up:standby 2024-09-21T03:36:36.712 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:36:36 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "k"} : dispatch 2024-09-21T03:36:36.714 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:36:36 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:36:36.714 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:36:36 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:36:36.714 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:36:36 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:36:36.714 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:36:36 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T03:36:36.715 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:36:36 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T03:36:36.715 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:36:36 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:36:36.715 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:36:36 smithi089 ceph-mon[30928]: Deploying daemon mds.j on smithi089 2024-09-21T03:36:36.715 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:36:36 smithi089 ceph-mon[30928]: from='client.? 172.21.15.89:0/2237766017' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:36:36.715 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:36:36 smithi089 ceph-mon[30928]: mds.? [v2:172.21.15.110:6832/127607331,v1:172.21.15.110:6833/127607331] up:boot 2024-09-21T03:36:36.715 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:36:36 smithi089 ceph-mon[30928]: fsmap 2 up:standby 2024-09-21T03:36:36.715 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:36:36 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "k"} : dispatch 2024-09-21T03:36:36.750 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:36:36 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:36:36.750 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:36:36 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:36:36.750 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:36:36 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:36:36.750 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:36:36 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T03:36:36.750 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:36:36 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T03:36:36.750 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:36:36 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:36:36.750 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:36:36 smithi110 ceph-mon[34000]: Deploying daemon mds.j on smithi089 2024-09-21T03:36:36.751 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:36:36 smithi110 ceph-mon[34000]: from='client.? 172.21.15.89:0/2237766017' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:36:36.751 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:36:36 smithi110 ceph-mon[34000]: mds.? [v2:172.21.15.110:6832/127607331,v1:172.21.15.110:6833/127607331] up:boot 2024-09-21T03:36:36.751 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:36:36 smithi110 ceph-mon[34000]: fsmap 2 up:standby 2024-09-21T03:36:36.751 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:36:36 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "k"} : dispatch 2024-09-21T03:36:37.553 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:36:37 smithi089 ceph-mon[30928]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:36:37.711 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:36:37 smithi114 ceph-mon[34140]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:36:37.750 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:36:37 smithi110 ceph-mon[34000]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:36:38.186 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 7c374098-77c9-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T03:36:38.205 INFO:journalctl@ceph.mds.j.smithi089.stdout:Sep 21 03:36:37 smithi089 systemd[1]: Starting Ceph mds.j for 7c374098-77c9-11ef-baf5-efdc52797490... 2024-09-21T03:36:38.480 INFO:journalctl@ceph.mds.j.smithi089.stdout:Sep 21 03:36:38 smithi089 podman[70389]: 2024-09-21 03:36:38.10661376 +0000 UTC m=+0.015622883 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T03:36:38.480 INFO:journalctl@ceph.mds.j.smithi089.stdout:Sep 21 03:36:38 smithi089 podman[70389]: 2024-09-21 03:36:38.221806528 +0000 UTC m=+0.130815654 container create 67a53e1dba194e1cd9fd98383e31a90d58308856dd632848b46e0a8466e678c6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-7c374098-77c9-11ef-baf5-efdc52797490-mds-j, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-09-21T03:36:38.480 INFO:journalctl@ceph.mds.j.smithi089.stdout:Sep 21 03:36:38 smithi089 podman[70389]: 2024-09-21 03:36:38.441913351 +0000 UTC m=+0.350922473 container init 67a53e1dba194e1cd9fd98383e31a90d58308856dd632848b46e0a8466e678c6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-7c374098-77c9-11ef-baf5-efdc52797490-mds-j, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , ceph=True) 2024-09-21T03:36:38.480 INFO:journalctl@ceph.mds.j.smithi089.stdout:Sep 21 03:36:38 smithi089 podman[70389]: 2024-09-21 03:36:38.445038224 +0000 UTC m=+0.354047342 container start 67a53e1dba194e1cd9fd98383e31a90d58308856dd632848b46e0a8466e678c6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-7c374098-77c9-11ef-baf5-efdc52797490-mds-j, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.license=GPLv2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD) 2024-09-21T03:36:38.480 INFO:journalctl@ceph.mds.j.smithi089.stdout:Sep 21 03:36:38 smithi089 ceph-mds[70437]: -- 172.21.15.89:0/3901997352 <== mon.0 v2:172.21.15.89:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x562e31bf9c20 con 0x562e35da1600 2024-09-21T03:36:38.627 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/7c374098-77c9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:36:38.749 INFO:journalctl@ceph.mds.j.smithi089.stdout:Sep 21 03:36:38 smithi089 ceph-7c374098-77c9-11ef-baf5-efdc52797490-mds-j[70422]: starting mds.j at 2024-09-21T03:36:38.749 INFO:journalctl@ceph.mds.j.smithi089.stdout:Sep 21 03:36:38 smithi089 bash[70389]: 67a53e1dba194e1cd9fd98383e31a90d58308856dd632848b46e0a8466e678c6 2024-09-21T03:36:38.750 INFO:journalctl@ceph.mds.j.smithi089.stdout:Sep 21 03:36:38 smithi089 systemd[1]: Started Ceph mds.j for 7c374098-77c9-11ef-baf5-efdc52797490. 2024-09-21T03:36:39.089 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:39.068+0000 7fc319d03640 1 -- 172.21.15.89:0/1408433407 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc31410de20 msgr2=0x7fc314110330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:39.090 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:39.068+0000 7fc319d03640 1 --2- 172.21.15.89:0/1408433407 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc31410de20 0x7fc314110330 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fc30c009ca0 tx=0x7fc30c02a8c0 comp rx=0 tx=0).stop 2024-09-21T03:36:39.090 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:39.068+0000 7fc319d03640 1 -- 172.21.15.89:0/1408433407 shutdown_connections 2024-09-21T03:36:39.090 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:39.068+0000 7fc319d03640 1 --2- 172.21.15.89:0/1408433407 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc31410de20 0x7fc314110330 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:39.090 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:39.068+0000 7fc319d03640 1 --2- 172.21.15.89:0/1408433407 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc3141052a0 0x7fc31410d8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:39.090 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:39.068+0000 7fc319d03640 1 --2- 172.21.15.89:0/1408433407 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc314104870 0x7fc314104c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:39.090 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:39.068+0000 7fc319d03640 1 -- 172.21.15.89:0/1408433407 >> 172.21.15.89:0/1408433407 conn(0x7fc314079750 msgr2=0x7fc314079b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:36:39.090 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:39.068+0000 7fc319d03640 1 -- 172.21.15.89:0/1408433407 shutdown_connections 2024-09-21T03:36:39.090 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:39.068+0000 7fc319d03640 1 -- 172.21.15.89:0/1408433407 wait complete. 2024-09-21T03:36:39.090 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:39.069+0000 7fc319d03640 1 Processor -- start 2024-09-21T03:36:39.090 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:39.069+0000 7fc319d03640 1 -- start start 2024-09-21T03:36:39.090 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:39.069+0000 7fc319d03640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc314104870 0x7fc31419a410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:39.091 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:39.069+0000 7fc319d03640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc3141052a0 0x7fc31419a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:39.091 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:39.069+0000 7fc319d03640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc31410de20 0x7fc31419fc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:39.091 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:39.069+0000 7fc319d03640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fc314113a40 con 0x7fc314104870 2024-09-21T03:36:39.091 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:39.069+0000 7fc319d03640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fc3141138c0 con 0x7fc3141052a0 2024-09-21T03:36:39.091 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:39.069+0000 7fc319d03640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fc314113bc0 con 0x7fc31410de20 2024-09-21T03:36:39.091 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:39.069+0000 7fc3137fe640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc314104870 0x7fc31419a410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:39.091 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:39.069+0000 7fc3137fe640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc314104870 0x7fc31419a410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:44856/0 (socket says 172.21.15.89:44856) 2024-09-21T03:36:39.091 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:39.069+0000 7fc3137fe640 1 -- 172.21.15.89:0/1520437442 learned_addr learned my addr 172.21.15.89:0/1520437442 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-21T03:36:39.091 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:39.069+0000 7fc312ffd640 1 --2- 172.21.15.89:0/1520437442 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc3141052a0 0x7fc31419a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:39.091 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:39.069+0000 7fc313fff640 1 --2- 172.21.15.89:0/1520437442 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc31410de20 0x7fc31419fc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:39.091 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:39.070+0000 7fc3137fe640 1 -- 172.21.15.89:0/1520437442 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc31410de20 msgr2=0x7fc31419fc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:39.091 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:39.070+0000 7fc3137fe640 1 --2- 172.21.15.89:0/1520437442 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc31410de20 0x7fc31419fc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:39.091 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:39.070+0000 7fc3137fe640 1 -- 172.21.15.89:0/1520437442 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc3141052a0 msgr2=0x7fc31419a9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:39.091 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:39.070+0000 7fc3137fe640 1 --2- 172.21.15.89:0/1520437442 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc3141052a0 0x7fc31419a9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:39.092 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:39.070+0000 7fc3137fe640 1 -- 172.21.15.89:0/1520437442 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3141a0380 con 0x7fc314104870 2024-09-21T03:36:39.092 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:39.070+0000 7fc3137fe640 1 --2- 172.21.15.89:0/1520437442 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc314104870 0x7fc31419a410 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fc30400b450 tx=0x7fc30400b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:36:39.092 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:39.070+0000 7fc310ff9640 1 -- 172.21.15.89:0/1520437442 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc304015020 con 0x7fc314104870 2024-09-21T03:36:39.092 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:39.070+0000 7fc310ff9640 1 -- 172.21.15.89:0/1520437442 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc304011d20 con 0x7fc314104870 2024-09-21T03:36:39.092 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:39.070+0000 7fc319d03640 1 -- 172.21.15.89:0/1520437442 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc3141a0670 con 0x7fc314104870 2024-09-21T03:36:39.092 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:39.070+0000 7fc310ff9640 1 -- 172.21.15.89:0/1520437442 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc304010e10 con 0x7fc314104870 2024-09-21T03:36:39.092 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:39.070+0000 7fc319d03640 1 -- 172.21.15.89:0/1520437442 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3141a6dd0 con 0x7fc314104870 2024-09-21T03:36:39.092 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:39.071+0000 7fc319d03640 1 -- 172.21.15.89:0/1520437442 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc31406c1c0 con 0x7fc314104870 2024-09-21T03:36:39.092 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:39.072+0000 7fc310ff9640 1 -- 172.21.15.89:0/1520437442 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 17) ==== 154512+0+0 (secure 0 0 0) 0x7fc304018020 con 0x7fc314104870 2024-09-21T03:36:39.092 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:39.073+0000 7fc310ff9640 1 --2- 172.21.15.89:0/1520437442 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7fc2fc0b6510 0x7fc2fc0b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:39.092 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:39.073+0000 7fc312ffd640 1 --2- 172.21.15.89:0/1520437442 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7fc2fc0b6510 0x7fc2fc0b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:39.092 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:39.073+0000 7fc310ff9640 1 -- 172.21.15.89:0/1520437442 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 7127+0+0 (secure 0 0 0) 0x7fc3040e6b10 con 0x7fc314104870 2024-09-21T03:36:39.092 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:39.074+0000 7fc312ffd640 1 --2- 172.21.15.89:0/1520437442 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7fc2fc0b6510 0x7fc2fc0b8a20 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fc31410fb90 tx=0x7fc30800a2d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:36:39.092 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:39.076+0000 7fc310ff9640 1 -- 172.21.15.89:0/1520437442 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fc30401f020 con 0x7fc314104870 2024-09-21T03:36:39.318 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:39.316+0000 7fc319d03640 1 -- 172.21.15.89:0/1520437442 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc314104c80 con 0x7fc314104870 2024-09-21T03:36:39.318 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:39.316+0000 7fc310ff9640 1 -- 172.21.15.89:0/1520437442 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) ==== 93+0+2260 (secure 0 0 0) 0x7fc30400d040 con 0x7fc314104870 2024-09-21T03:36:39.319 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-09-21T03:36:39.319 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":3,"btime":"2024-09-21T03:36:35:765297+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24565,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6833/752336912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6832","nonce":752336912},{"type":"v1","addr":"172.21.15.114:6833","nonce":752336912}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24575,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6833/127607331","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6832","nonce":127607331},{"type":"v1","addr":"172.21.15.110:6833","nonce":127607331}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-09-21T03:36:39.319 INFO:teuthology.orchestra.run.smithi089.stderr:dumped fsmap epoch 3 2024-09-21T03:36:39.321 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:39.318+0000 7fc319d03640 1 -- 172.21.15.89:0/1520437442 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7fc2fc0b6510 msgr2=0x7fc2fc0b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:39.321 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:39.318+0000 7fc319d03640 1 --2- 172.21.15.89:0/1520437442 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7fc2fc0b6510 0x7fc2fc0b8a20 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fc31410fb90 tx=0x7fc30800a2d0 comp rx=0 tx=0).stop 2024-09-21T03:36:39.321 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:39.318+0000 7fc319d03640 1 -- 172.21.15.89:0/1520437442 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc314104870 msgr2=0x7fc31419a410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:39.321 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:39.318+0000 7fc319d03640 1 --2- 172.21.15.89:0/1520437442 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc314104870 0x7fc31419a410 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fc30400b450 tx=0x7fc30400b910 comp rx=0 tx=0).stop 2024-09-21T03:36:39.321 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:39.318+0000 7fc319d03640 1 -- 172.21.15.89:0/1520437442 shutdown_connections 2024-09-21T03:36:39.321 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:39.319+0000 7fc319d03640 1 --2- 172.21.15.89:0/1520437442 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7fc2fc0b6510 0x7fc2fc0b8a20 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:39.322 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:39.319+0000 7fc319d03640 1 --2- 172.21.15.89:0/1520437442 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc3141052a0 0x7fc31419a9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:39.322 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:39.319+0000 7fc319d03640 1 --2- 172.21.15.89:0/1520437442 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc31410de20 0x7fc31419fc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:39.322 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:39.319+0000 7fc319d03640 1 --2- 172.21.15.89:0/1520437442 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc314104870 0x7fc31419a410 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:39.322 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:39.319+0000 7fc319d03640 1 -- 172.21.15.89:0/1520437442 >> 172.21.15.89:0/1520437442 conn(0x7fc314079750 msgr2=0x7fc3141079c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:36:39.322 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:39.319+0000 7fc319d03640 1 -- 172.21.15.89:0/1520437442 shutdown_connections 2024-09-21T03:36:39.322 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:39.319+0000 7fc319d03640 1 -- 172.21.15.89:0/1520437442 wait complete. 2024-09-21T03:36:39.711 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:36:39 smithi114 ceph-mon[34140]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:36:39.711 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:36:39 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:36:39.711 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:36:39 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:36:39.711 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:36:39 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:36:39.711 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:36:39 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T03:36:39.712 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:36:39 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T03:36:39.712 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:36:39 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:36:39.712 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:36:39 smithi114 ceph-mon[34140]: from='client.? 172.21.15.89:0/1520437442' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:36:39.750 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:36:39 smithi110 ceph-mon[34000]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:36:39.750 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:36:39 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:36:39.750 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:36:39 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:36:39.750 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:36:39 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:36:39.751 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:36:39 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T03:36:39.751 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:36:39 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T03:36:39.751 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:36:39 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:36:39.751 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:36:39 smithi110 ceph-mon[34000]: from='client.? 172.21.15.89:0/1520437442' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:36:39.811 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:36:39 smithi089 ceph-mon[30928]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:36:39.811 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:36:39 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:36:39.811 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:36:39 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:36:39.811 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:36:39 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:36:39.811 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:36:39 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T03:36:39.812 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:36:39 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T03:36:39.812 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:36:39 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:36:39.812 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:36:39 smithi089 ceph-mon[30928]: from='client.? 172.21.15.89:0/1520437442' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:36:40.705 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:36:40 smithi089 ceph-mon[30928]: Deploying daemon mds.d on smithi089 2024-09-21T03:36:40.705 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:36:40 smithi089 ceph-mon[30928]: mds.? [v2:172.21.15.89:6834/2341538514,v1:172.21.15.89:6835/2341538514] up:boot 2024-09-21T03:36:40.705 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:36:40 smithi089 ceph-mon[30928]: fsmap 3 up:standby 2024-09-21T03:36:40.705 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:36:40 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "j"} : dispatch 2024-09-21T03:36:40.710 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:36:40 smithi114 ceph-mon[34140]: Deploying daemon mds.d on smithi089 2024-09-21T03:36:40.711 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:36:40 smithi114 ceph-mon[34140]: mds.? [v2:172.21.15.89:6834/2341538514,v1:172.21.15.89:6835/2341538514] up:boot 2024-09-21T03:36:40.711 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:36:40 smithi114 ceph-mon[34140]: fsmap 3 up:standby 2024-09-21T03:36:40.711 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:36:40 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "j"} : dispatch 2024-09-21T03:36:40.750 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:36:40 smithi110 ceph-mon[34000]: Deploying daemon mds.d on smithi089 2024-09-21T03:36:40.750 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:36:40 smithi110 ceph-mon[34000]: mds.? [v2:172.21.15.89:6834/2341538514,v1:172.21.15.89:6835/2341538514] up:boot 2024-09-21T03:36:40.750 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:36:40 smithi110 ceph-mon[34000]: fsmap 3 up:standby 2024-09-21T03:36:40.750 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:36:40 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "j"} : dispatch 2024-09-21T03:36:41.711 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:36:41 smithi114 ceph-mon[34140]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:36:41.750 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:36:41 smithi110 ceph-mon[34000]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:36:41.802 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:36:41 smithi089 ceph-mon[30928]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:36:42.145 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 7c374098-77c9-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T03:36:42.395 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/7c374098-77c9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:36:42.901 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:42.889+0000 7ff078421640 1 -- 172.21.15.89:0/1867599222 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff0701094a0 msgr2=0x7ff0701098b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:42.901 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:42.889+0000 7ff078421640 1 --2- 172.21.15.89:0/1867599222 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff0701094a0 0x7ff0701098b0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7ff06c009900 tx=0x7ff06c02a410 comp rx=0 tx=0).stop 2024-09-21T03:36:42.901 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:42.889+0000 7ff078421640 1 -- 172.21.15.89:0/1867599222 shutdown_connections 2024-09-21T03:36:42.901 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:42.889+0000 7ff078421640 1 --2- 172.21.15.89:0/1867599222 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff070109df0 0x7ff070110620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:42.901 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:42.889+0000 7ff078421640 1 --2- 172.21.15.89:0/1867599222 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff0701094a0 0x7ff0701098b0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:42.901 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:42.889+0000 7ff078421640 1 --2- 172.21.15.89:0/1867599222 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff070103b60 0x7ff070103f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:42.902 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:42.889+0000 7ff078421640 1 -- 172.21.15.89:0/1867599222 >> 172.21.15.89:0/1867599222 conn(0x7ff07007b850 msgr2=0x7ff07007bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:36:42.902 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:42.890+0000 7ff078421640 1 -- 172.21.15.89:0/1867599222 shutdown_connections 2024-09-21T03:36:42.902 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:42.890+0000 7ff078421640 1 -- 172.21.15.89:0/1867599222 wait complete. 2024-09-21T03:36:42.902 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:42.890+0000 7ff078421640 1 Processor -- start 2024-09-21T03:36:42.902 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:42.890+0000 7ff078421640 1 -- start start 2024-09-21T03:36:42.902 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:42.890+0000 7ff078421640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff070103b60 0x7ff07019a510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:42.902 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:42.890+0000 7ff078421640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff0701094a0 0x7ff07019aab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:42.902 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:42.890+0000 7ff078421640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff070109df0 0x7ff07019fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:42.902 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:42.890+0000 7ff078421640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7ff0701139a0 con 0x7ff070109df0 2024-09-21T03:36:42.902 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:42.891+0000 7ff078421640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7ff070113820 con 0x7ff070103b60 2024-09-21T03:36:42.902 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:42.891+0000 7ff078421640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7ff070113b20 con 0x7ff0701094a0 2024-09-21T03:36:42.902 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:42.891+0000 7ff0769be640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff070109df0 0x7ff07019fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:42.902 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:42.891+0000 7ff0769be640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff070109df0 0x7ff07019fd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:44894/0 (socket says 172.21.15.89:44894) 2024-09-21T03:36:42.902 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:42.891+0000 7ff0769be640 1 -- 172.21.15.89:0/2870909601 learned_addr learned my addr 172.21.15.89:0/2870909601 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-21T03:36:42.903 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:42.891+0000 7ff0761bd640 1 --2- 172.21.15.89:0/2870909601 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff070103b60 0x7ff07019a510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:42.903 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:42.891+0000 7ff0759bc640 1 --2- 172.21.15.89:0/2870909601 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff0701094a0 0x7ff07019aab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:42.903 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:42.891+0000 7ff0769be640 1 -- 172.21.15.89:0/2870909601 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff0701094a0 msgr2=0x7ff07019aab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:42.903 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:42.891+0000 7ff0769be640 1 --2- 172.21.15.89:0/2870909601 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff0701094a0 0x7ff07019aab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:42.903 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:42.891+0000 7ff0769be640 1 -- 172.21.15.89:0/2870909601 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff070103b60 msgr2=0x7ff07019a510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:42.903 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:42.891+0000 7ff0769be640 1 --2- 172.21.15.89:0/2870909601 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff070103b60 0x7ff07019a510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:42.903 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:42.891+0000 7ff0769be640 1 -- 172.21.15.89:0/2870909601 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0701a0480 con 0x7ff070109df0 2024-09-21T03:36:42.903 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:42.891+0000 7ff0759bc640 1 --2- 172.21.15.89:0/2870909601 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff0701094a0 0x7ff07019aab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:36:42.903 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:42.891+0000 7ff0769be640 1 --2- 172.21.15.89:0/2870909601 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff070109df0 0x7ff07019fd50 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7ff06800b970 tx=0x7ff06800be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:36:42.903 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:42.891+0000 7ff0637fe640 1 -- 172.21.15.89:0/2870909601 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff068019070 con 0x7ff070109df0 2024-09-21T03:36:42.903 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:42.891+0000 7ff078421640 1 -- 172.21.15.89:0/2870909601 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff0701a0770 con 0x7ff070109df0 2024-09-21T03:36:42.903 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:42.891+0000 7ff0637fe640 1 -- 172.21.15.89:0/2870909601 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff068014440 con 0x7ff070109df0 2024-09-21T03:36:42.903 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:42.891+0000 7ff0637fe640 1 -- 172.21.15.89:0/2870909601 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0680075d0 con 0x7ff070109df0 2024-09-21T03:36:42.903 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:42.891+0000 7ff078421640 1 -- 172.21.15.89:0/2870909601 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0701a6ed0 con 0x7ff070109df0 2024-09-21T03:36:42.904 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:42.892+0000 7ff078421640 1 -- 172.21.15.89:0/2870909601 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff070103f70 con 0x7ff070109df0 2024-09-21T03:36:42.904 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:42.894+0000 7ff0637fe640 1 -- 172.21.15.89:0/2870909601 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 17) ==== 154512+0+0 (secure 0 0 0) 0x7ff06801c070 con 0x7ff070109df0 2024-09-21T03:36:42.904 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:42.894+0000 7ff0637fe640 1 --2- 172.21.15.89:0/2870909601 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7ff05c0b6510 0x7ff05c0b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:42.904 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:42.894+0000 7ff0761bd640 1 --2- 172.21.15.89:0/2870909601 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7ff05c0b6510 0x7ff05c0b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:42.904 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:42.895+0000 7ff0637fe640 1 -- 172.21.15.89:0/2870909601 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 7127+0+0 (secure 0 0 0) 0x7ff0680e6790 con 0x7ff070109df0 2024-09-21T03:36:42.904 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:42.895+0000 7ff0761bd640 1 --2- 172.21.15.89:0/2870909601 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7ff05c0b6510 0x7ff05c0b8a20 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7ff0640058c0 tx=0x7ff06400a250 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:36:42.904 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:42.898+0000 7ff0637fe640 1 -- 172.21.15.89:0/2870909601 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7ff0680b18a0 con 0x7ff070109df0 2024-09-21T03:36:43.113 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:43.111+0000 7ff078421640 1 -- 172.21.15.89:0/2870909601 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff07019b640 con 0x7ff070109df0 2024-09-21T03:36:43.114 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:43.111+0000 7ff0637fe640 1 -- 172.21.15.89:0/2870909601 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) ==== 93+0+3907 (secure 0 0 0) 0x7ff0680b67e0 con 0x7ff070109df0 2024-09-21T03:36:43.114 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-09-21T03:36:43.114 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":5,"btime":"2024-09-21T03:36:42:444511+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14721,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6835/2341538514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":2341538514},{"type":"v1","addr":"172.21.15.89:6835","nonce":2341538514}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14733,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6837/146708592","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6836","nonce":146708592},{"type":"v1","addr":"172.21.15.89:6837","nonce":146708592}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24565,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6833/752336912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6832","nonce":752336912},{"type":"v1","addr":"172.21.15.114:6833","nonce":752336912}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24575,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6833/127607331","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6832","nonce":127607331},{"type":"v1","addr":"172.21.15.110:6833","nonce":127607331}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-09-21T03:36:43.114 INFO:teuthology.orchestra.run.smithi089.stderr:dumped fsmap epoch 5 2024-09-21T03:36:43.116 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:43.114+0000 7ff078421640 1 -- 172.21.15.89:0/2870909601 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7ff05c0b6510 msgr2=0x7ff05c0b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:43.116 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:43.114+0000 7ff078421640 1 --2- 172.21.15.89:0/2870909601 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7ff05c0b6510 0x7ff05c0b8a20 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7ff0640058c0 tx=0x7ff06400a250 comp rx=0 tx=0).stop 2024-09-21T03:36:43.116 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:43.114+0000 7ff078421640 1 -- 172.21.15.89:0/2870909601 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff070109df0 msgr2=0x7ff07019fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:43.116 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:43.114+0000 7ff078421640 1 --2- 172.21.15.89:0/2870909601 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff070109df0 0x7ff07019fd50 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7ff06800b970 tx=0x7ff06800be30 comp rx=0 tx=0).stop 2024-09-21T03:36:43.117 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:43.114+0000 7ff078421640 1 -- 172.21.15.89:0/2870909601 shutdown_connections 2024-09-21T03:36:43.117 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:43.114+0000 7ff078421640 1 --2- 172.21.15.89:0/2870909601 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7ff05c0b6510 0x7ff05c0b8a20 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:43.117 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:43.114+0000 7ff078421640 1 --2- 172.21.15.89:0/2870909601 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff070109df0 0x7ff07019fd50 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:43.117 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:43.114+0000 7ff078421640 1 --2- 172.21.15.89:0/2870909601 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff0701094a0 0x7ff07019aab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:43.117 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:43.114+0000 7ff078421640 1 --2- 172.21.15.89:0/2870909601 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff070103b60 0x7ff07019a510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:43.117 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:43.114+0000 7ff078421640 1 -- 172.21.15.89:0/2870909601 >> 172.21.15.89:0/2870909601 conn(0x7ff07007b850 msgr2=0x7ff070105620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:36:43.117 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:43.114+0000 7ff078421640 1 -- 172.21.15.89:0/2870909601 shutdown_connections 2024-09-21T03:36:43.117 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:43.114+0000 7ff078421640 1 -- 172.21.15.89:0/2870909601 wait complete. 2024-09-21T03:36:43.447 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:36:43 smithi110 ceph-mon[34000]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:36:43.447 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:36:43 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:36:43.447 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:36:43 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:36:43.448 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:36:43 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:36:43.448 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:36:43 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T03:36:43.448 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:36:43 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T03:36:43.448 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:36:43 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:36:43.448 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:36:43 smithi110 ceph-mon[34000]: Deploying daemon mds.e on smithi110 2024-09-21T03:36:43.448 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:36:43 smithi110 ceph-mon[34000]: mds.? [v2:172.21.15.89:6836/146708592,v1:172.21.15.89:6837/146708592] up:boot 2024-09-21T03:36:43.448 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:36:43 smithi110 ceph-mon[34000]: fsmap 4 up:standby 2024-09-21T03:36:43.448 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:36:43 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "d"} : dispatch 2024-09-21T03:36:43.461 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:36:43 smithi114 ceph-mon[34140]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:36:43.461 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:36:43 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:36:43.461 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:36:43 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:36:43.461 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:36:43 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:36:43.461 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:36:43 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T03:36:43.461 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:36:43 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T03:36:43.461 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:36:43 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:36:43.461 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:36:43 smithi114 ceph-mon[34140]: Deploying daemon mds.e on smithi110 2024-09-21T03:36:43.461 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:36:43 smithi114 ceph-mon[34140]: mds.? [v2:172.21.15.89:6836/146708592,v1:172.21.15.89:6837/146708592] up:boot 2024-09-21T03:36:43.461 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:36:43 smithi114 ceph-mon[34140]: fsmap 4 up:standby 2024-09-21T03:36:43.461 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:36:43 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "d"} : dispatch 2024-09-21T03:36:43.488 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:36:43 smithi089 ceph-mon[30928]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:36:43.488 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:36:43 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:36:43.488 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:36:43 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:36:43.488 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:36:43 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:36:43.488 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:36:43 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T03:36:43.488 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:36:43 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T03:36:43.488 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:36:43 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:36:43.488 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:36:43 smithi089 ceph-mon[30928]: Deploying daemon mds.e on smithi110 2024-09-21T03:36:43.488 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:36:43 smithi089 ceph-mon[30928]: mds.? [v2:172.21.15.89:6836/146708592,v1:172.21.15.89:6837/146708592] up:boot 2024-09-21T03:36:43.489 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:36:43 smithi089 ceph-mon[30928]: fsmap 4 up:standby 2024-09-21T03:36:43.489 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:36:43 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "d"} : dispatch 2024-09-21T03:36:44.000 INFO:journalctl@ceph.mds.e.smithi110.stdout:Sep 21 03:36:43 smithi110 systemd[1]: Starting Ceph mds.e for 7c374098-77c9-11ef-baf5-efdc52797490... 2024-09-21T03:36:44.311 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:36:44 smithi089 ceph-mon[30928]: from='client.? 172.21.15.89:0/2870909601' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:36:44.394 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:36:44 smithi110 ceph-mon[34000]: from='client.? 172.21.15.89:0/2870909601' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:36:44.394 INFO:journalctl@ceph.mds.e.smithi110.stdout:Sep 21 03:36:44 smithi110 podman[61534]: 2024-09-21 03:36:44.031302936 +0000 UTC m=+0.015733299 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T03:36:44.395 INFO:journalctl@ceph.mds.e.smithi110.stdout:Sep 21 03:36:44 smithi110 podman[61534]: 2024-09-21 03:36:44.14883035 +0000 UTC m=+0.133260705 container create 368b2198fd454390a5266bdd0ef8475d40e70908480e3168995e79d8877c3ea5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-7c374098-77c9-11ef-baf5-efdc52797490-mds-e, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS) 2024-09-21T03:36:44.395 INFO:journalctl@ceph.mds.e.smithi110.stdout:Sep 21 03:36:44 smithi110 podman[61534]: 2024-09-21 03:36:44.393433566 +0000 UTC m=+0.377863924 container init 368b2198fd454390a5266bdd0ef8475d40e70908480e3168995e79d8877c3ea5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-7c374098-77c9-11ef-baf5-efdc52797490-mds-e, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2) 2024-09-21T03:36:44.461 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:36:44 smithi114 ceph-mon[34140]: from='client.? 172.21.15.89:0/2870909601' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:36:44.750 INFO:journalctl@ceph.mds.e.smithi110.stdout:Sep 21 03:36:44 smithi110 podman[61534]: 2024-09-21 03:36:44.396295643 +0000 UTC m=+0.380725997 container start 368b2198fd454390a5266bdd0ef8475d40e70908480e3168995e79d8877c3ea5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-7c374098-77c9-11ef-baf5-efdc52797490-mds-e, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, CEPH_POINT_RELEASE=) 2024-09-21T03:36:44.750 INFO:journalctl@ceph.mds.e.smithi110.stdout:Sep 21 03:36:44 smithi110 ceph-mds[61553]: -- 172.21.15.110:0/3978001327 <== mon.2 v2:172.21.15.110:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55de23403860 con 0x55de275ab180 2024-09-21T03:36:44.750 INFO:journalctl@ceph.mds.e.smithi110.stdout:Sep 21 03:36:44 smithi110 ceph-7c374098-77c9-11ef-baf5-efdc52797490-mds-e[61549]: starting mds.e at 2024-09-21T03:36:44.750 INFO:journalctl@ceph.mds.e.smithi110.stdout:Sep 21 03:36:44 smithi110 bash[61534]: 368b2198fd454390a5266bdd0ef8475d40e70908480e3168995e79d8877c3ea5 2024-09-21T03:36:44.750 INFO:journalctl@ceph.mds.e.smithi110.stdout:Sep 21 03:36:44 smithi110 systemd[1]: Started Ceph mds.e for 7c374098-77c9-11ef-baf5-efdc52797490. 2024-09-21T03:36:45.796 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:36:45 smithi114 ceph-mon[34140]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:36:45.796 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:36:45 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:36:45.797 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:36:45 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:36:45.797 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:36:45 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:36:45.797 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:36:45 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T03:36:45.797 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:36:45 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T03:36:45.797 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:36:45 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:36:45.797 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:36:45 smithi114 ceph-mon[34140]: Deploying daemon mds.f on smithi114 2024-09-21T03:36:45.797 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:36:45 smithi114 ceph-mon[34140]: mds.? [v2:172.21.15.110:6834/671820654,v1:172.21.15.110:6835/671820654] up:boot 2024-09-21T03:36:45.797 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:36:45 smithi114 ceph-mon[34140]: fsmap 5 up:standby 2024-09-21T03:36:45.797 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:36:45 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "e"} : dispatch 2024-09-21T03:36:45.811 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:36:45 smithi089 ceph-mon[30928]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:36:45.811 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:36:45 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:36:45.811 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:36:45 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:36:45.811 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:36:45 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:36:45.811 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:36:45 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T03:36:45.812 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:36:45 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T03:36:45.812 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:36:45 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:36:45.812 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:36:45 smithi089 ceph-mon[30928]: Deploying daemon mds.f on smithi114 2024-09-21T03:36:45.812 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:36:45 smithi089 ceph-mon[30928]: mds.? [v2:172.21.15.110:6834/671820654,v1:172.21.15.110:6835/671820654] up:boot 2024-09-21T03:36:45.812 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:36:45 smithi089 ceph-mon[30928]: fsmap 5 up:standby 2024-09-21T03:36:45.812 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:36:45 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "e"} : dispatch 2024-09-21T03:36:45.955 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 7c374098-77c9-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T03:36:46.000 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:36:45 smithi110 ceph-mon[34000]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:36:46.000 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:36:45 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:36:46.000 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:36:45 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:36:46.000 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:36:45 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:36:46.000 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:36:45 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T03:36:46.000 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:36:45 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T03:36:46.001 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:36:45 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:36:46.001 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:36:45 smithi110 ceph-mon[34000]: Deploying daemon mds.f on smithi114 2024-09-21T03:36:46.001 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:36:45 smithi110 ceph-mon[34000]: mds.? [v2:172.21.15.110:6834/671820654,v1:172.21.15.110:6835/671820654] up:boot 2024-09-21T03:36:46.001 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:36:45 smithi110 ceph-mon[34000]: fsmap 5 up:standby 2024-09-21T03:36:46.001 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:36:45 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "e"} : dispatch 2024-09-21T03:36:46.221 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/7c374098-77c9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:36:46.695 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:46.674+0000 7ff539836640 1 -- 172.21.15.89:0/2733175835 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff5341083a0 msgr2=0x7ff5341087b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:46.696 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:46.674+0000 7ff539836640 1 --2- 172.21.15.89:0/2733175835 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff5341083a0 0x7ff5341087b0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7ff51c009900 tx=0x7ff51c02a410 comp rx=0 tx=0).stop 2024-09-21T03:36:46.696 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:46.674+0000 7ff539836640 1 -- 172.21.15.89:0/2733175835 shutdown_connections 2024-09-21T03:36:46.696 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:46.674+0000 7ff539836640 1 --2- 172.21.15.89:0/2733175835 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff534108cf0 0x7ff53410f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:46.696 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:46.674+0000 7ff539836640 1 --2- 172.21.15.89:0/2733175835 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff5341083a0 0x7ff5341087b0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:46.696 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:46.674+0000 7ff539836640 1 --2- 172.21.15.89:0/2733175835 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff534102a60 0x7ff534102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:46.696 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:46.674+0000 7ff539836640 1 -- 172.21.15.89:0/2733175835 >> 172.21.15.89:0/2733175835 conn(0x7ff534079730 msgr2=0x7ff534079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:36:46.696 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:46.674+0000 7ff539836640 1 -- 172.21.15.89:0/2733175835 shutdown_connections 2024-09-21T03:36:46.696 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:46.674+0000 7ff539836640 1 -- 172.21.15.89:0/2733175835 wait complete. 2024-09-21T03:36:46.696 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:46.675+0000 7ff539836640 1 Processor -- start 2024-09-21T03:36:46.696 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:46.675+0000 7ff539836640 1 -- start start 2024-09-21T03:36:46.696 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:46.676+0000 7ff539836640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff534102a60 0x7ff53419a390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:46.696 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:46.676+0000 7ff539836640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff5341083a0 0x7ff53419a930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:46.696 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:46.676+0000 7ff539836640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff534108cf0 0x7ff53419fbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:46.697 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:46.676+0000 7ff539836640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7ff534112810 con 0x7ff534102a60 2024-09-21T03:36:46.697 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:46.676+0000 7ff539836640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7ff534112690 con 0x7ff5341083a0 2024-09-21T03:36:46.697 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:46.676+0000 7ff539836640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7ff534112990 con 0x7ff534108cf0 2024-09-21T03:36:46.697 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:46.676+0000 7ff532ffd640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff534102a60 0x7ff53419a390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:46.697 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:46.676+0000 7ff532ffd640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff534102a60 0x7ff53419a390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:44904/0 (socket says 172.21.15.89:44904) 2024-09-21T03:36:46.697 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:46.676+0000 7ff532ffd640 1 -- 172.21.15.89:0/1857145478 learned_addr learned my addr 172.21.15.89:0/1857145478 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-21T03:36:46.697 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:46.676+0000 7ff5337fe640 1 --2- 172.21.15.89:0/1857145478 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff534108cf0 0x7ff53419fbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:46.697 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:46.676+0000 7ff52a7fc640 1 --2- 172.21.15.89:0/1857145478 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff5341083a0 0x7ff53419a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:46.697 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:46.676+0000 7ff532ffd640 1 -- 172.21.15.89:0/1857145478 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff534108cf0 msgr2=0x7ff53419fbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:46.697 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:46.676+0000 7ff532ffd640 1 --2- 172.21.15.89:0/1857145478 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff534108cf0 0x7ff53419fbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:46.697 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:46.676+0000 7ff532ffd640 1 -- 172.21.15.89:0/1857145478 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff5341083a0 msgr2=0x7ff53419a930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:46.697 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:46.676+0000 7ff532ffd640 1 --2- 172.21.15.89:0/1857145478 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff5341083a0 0x7ff53419a930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:46.697 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:46.676+0000 7ff532ffd640 1 -- 172.21.15.89:0/1857145478 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5341a0300 con 0x7ff534102a60 2024-09-21T03:36:46.697 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:46.676+0000 7ff52a7fc640 1 --2- 172.21.15.89:0/1857145478 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff5341083a0 0x7ff53419a930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:36:46.698 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:46.676+0000 7ff532ffd640 1 --2- 172.21.15.89:0/1857145478 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff534102a60 0x7ff53419a390 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7ff52400d950 tx=0x7ff52400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:36:46.698 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:46.676+0000 7ff538834640 1 -- 172.21.15.89:0/1857145478 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff524017030 con 0x7ff534102a60 2024-09-21T03:36:46.698 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:46.677+0000 7ff539836640 1 -- 172.21.15.89:0/1857145478 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff5341a05f0 con 0x7ff534102a60 2024-09-21T03:36:46.698 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:46.677+0000 7ff538834640 1 -- 172.21.15.89:0/1857145478 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff524004020 con 0x7ff534102a60 2024-09-21T03:36:46.698 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:46.677+0000 7ff538834640 1 -- 172.21.15.89:0/1857145478 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5240055d0 con 0x7ff534102a60 2024-09-21T03:36:46.698 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:46.677+0000 7ff539836640 1 -- 172.21.15.89:0/1857145478 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5341a6d50 con 0x7ff534102a60 2024-09-21T03:36:46.698 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:46.678+0000 7ff539836640 1 -- 172.21.15.89:0/1857145478 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff534102e70 con 0x7ff534102a60 2024-09-21T03:36:46.698 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:46.679+0000 7ff538834640 1 -- 172.21.15.89:0/1857145478 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 17) ==== 154512+0+0 (secure 0 0 0) 0x7ff524009a30 con 0x7ff534102a60 2024-09-21T03:36:46.698 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:46.680+0000 7ff538834640 1 --2- 172.21.15.89:0/1857145478 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7ff5200b6510 0x7ff5200b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:46.698 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:46.680+0000 7ff52a7fc640 1 --2- 172.21.15.89:0/1857145478 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7ff5200b6510 0x7ff5200b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:46.698 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:46.680+0000 7ff538834640 1 -- 172.21.15.89:0/1857145478 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 7127+0+0 (secure 0 0 0) 0x7ff5240e68f0 con 0x7ff534102a60 2024-09-21T03:36:46.698 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:46.680+0000 7ff52a7fc640 1 --2- 172.21.15.89:0/1857145478 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7ff5200b6510 0x7ff5200b8a20 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7ff51c006010 tx=0x7ff51c000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:36:46.698 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:46.684+0000 7ff538834640 1 -- 172.21.15.89:0/1857145478 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7ff5240ad620 con 0x7ff534102a60 2024-09-21T03:36:46.901 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:46.899+0000 7ff539836640 1 -- 172.21.15.89:0/1857145478 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff53419b4c0 con 0x7ff534102a60 2024-09-21T03:36:46.902 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:46.900+0000 7ff538834640 1 -- 172.21.15.89:0/1857145478 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) ==== 93+0+4732 (secure 0 0 0) 0x7ff5240b1a00 con 0x7ff534102a60 2024-09-21T03:36:46.902 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-09-21T03:36:46.902 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":6,"btime":"2024-09-21T03:36:45:075040+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14721,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6835/2341538514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":2341538514},{"type":"v1","addr":"172.21.15.89:6835","nonce":2341538514}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14733,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6837/146708592","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6836","nonce":146708592},{"type":"v1","addr":"172.21.15.89:6837","nonce":146708592}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24565,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6833/752336912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6832","nonce":752336912},{"type":"v1","addr":"172.21.15.114:6833","nonce":752336912}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24575,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6833/127607331","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6832","nonce":127607331},{"type":"v1","addr":"172.21.15.110:6833","nonce":127607331}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24593,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6835/671820654","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6834","nonce":671820654},{"type":"v1","addr":"172.21.15.110:6835","nonce":671820654}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[]} 2024-09-21T03:36:46.903 INFO:teuthology.orchestra.run.smithi089.stderr:dumped fsmap epoch 6 2024-09-21T03:36:46.905 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:46.902+0000 7ff539836640 1 -- 172.21.15.89:0/1857145478 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7ff5200b6510 msgr2=0x7ff5200b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:46.905 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:46.902+0000 7ff539836640 1 --2- 172.21.15.89:0/1857145478 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7ff5200b6510 0x7ff5200b8a20 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7ff51c006010 tx=0x7ff51c000bc0 comp rx=0 tx=0).stop 2024-09-21T03:36:46.905 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:46.902+0000 7ff539836640 1 -- 172.21.15.89:0/1857145478 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff534102a60 msgr2=0x7ff53419a390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:46.905 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:46.902+0000 7ff539836640 1 --2- 172.21.15.89:0/1857145478 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff534102a60 0x7ff53419a390 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7ff52400d950 tx=0x7ff52400de10 comp rx=0 tx=0).stop 2024-09-21T03:36:46.905 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:46.902+0000 7ff539836640 1 -- 172.21.15.89:0/1857145478 shutdown_connections 2024-09-21T03:36:46.905 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:46.902+0000 7ff539836640 1 --2- 172.21.15.89:0/1857145478 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7ff5200b6510 0x7ff5200b8a20 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:46.905 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:46.902+0000 7ff539836640 1 --2- 172.21.15.89:0/1857145478 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff5341083a0 0x7ff53419a930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:46.905 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:46.902+0000 7ff539836640 1 --2- 172.21.15.89:0/1857145478 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff534108cf0 0x7ff53419fbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:46.905 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:46.902+0000 7ff539836640 1 --2- 172.21.15.89:0/1857145478 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff534102a60 0x7ff53419a390 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:46.905 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:46.902+0000 7ff539836640 1 -- 172.21.15.89:0/1857145478 >> 172.21.15.89:0/1857145478 conn(0x7ff534079730 msgr2=0x7ff53406b250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:36:46.905 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:46.902+0000 7ff539836640 1 -- 172.21.15.89:0/1857145478 shutdown_connections 2024-09-21T03:36:46.906 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:46.902+0000 7ff539836640 1 -- 172.21.15.89:0/1857145478 wait complete. 2024-09-21T03:36:47.211 INFO:journalctl@ceph.mds.f.smithi114.stdout:Sep 21 03:36:47 smithi114 systemd[1]: Started Ceph mds.f for 7c374098-77c9-11ef-baf5-efdc52797490. 2024-09-21T03:36:47.742 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:36:47 smithi114 ceph-mon[34140]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:36:47.742 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:36:47 smithi114 ceph-mon[34140]: from='client.? 172.21.15.89:0/1857145478' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:36:47.742 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:36:47 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:36:47.742 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:36:47 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:36:47.742 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:36:47 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:36:47.742 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:36:47 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T03:36:47.743 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:36:47 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T03:36:47.743 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:36:47 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:36:47.811 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:36:47 smithi089 ceph-mon[30928]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:36:47.811 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:36:47 smithi089 ceph-mon[30928]: from='client.? 172.21.15.89:0/1857145478' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:36:47.811 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:36:47 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:36:47.811 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:36:47 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:36:47.811 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:36:47 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:36:47.811 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:36:47 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T03:36:47.812 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:36:47 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T03:36:47.812 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:36:47 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:36:48.000 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:36:47 smithi110 ceph-mon[34000]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:36:48.000 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:36:47 smithi110 ceph-mon[34000]: from='client.? 172.21.15.89:0/1857145478' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:36:48.000 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:36:47 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:36:48.000 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:36:47 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:36:48.000 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:36:47 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:36:48.001 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:36:47 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T03:36:48.001 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:36:47 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T03:36:48.001 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:36:47 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:36:48.791 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:36:48 smithi114 ceph-mon[34140]: Deploying daemon mds.c on smithi114 2024-09-21T03:36:48.792 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:36:48 smithi114 ceph-mon[34140]: mds.? [v2:172.21.15.114:6834/1049714548,v1:172.21.15.114:6835/1049714548] up:boot 2024-09-21T03:36:48.792 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:36:48 smithi114 ceph-mon[34140]: fsmap 6 up:standby 2024-09-21T03:36:48.792 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:36:48 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "f"} : dispatch 2024-09-21T03:36:48.811 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:36:48 smithi089 ceph-mon[30928]: Deploying daemon mds.c on smithi114 2024-09-21T03:36:48.811 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:36:48 smithi089 ceph-mon[30928]: mds.? [v2:172.21.15.114:6834/1049714548,v1:172.21.15.114:6835/1049714548] up:boot 2024-09-21T03:36:48.811 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:36:48 smithi089 ceph-mon[30928]: fsmap 6 up:standby 2024-09-21T03:36:48.811 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:36:48 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "f"} : dispatch 2024-09-21T03:36:49.000 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:36:48 smithi110 ceph-mon[34000]: Deploying daemon mds.c on smithi114 2024-09-21T03:36:49.000 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:36:48 smithi110 ceph-mon[34000]: mds.? [v2:172.21.15.114:6834/1049714548,v1:172.21.15.114:6835/1049714548] up:boot 2024-09-21T03:36:49.000 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:36:48 smithi110 ceph-mon[34000]: fsmap 6 up:standby 2024-09-21T03:36:49.000 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:36:48 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "f"} : dispatch 2024-09-21T03:36:49.183 INFO:journalctl@ceph.mds.c.smithi114.stdout:Sep 21 03:36:48 smithi114 systemd[1]: Starting Ceph mds.c for 7c374098-77c9-11ef-baf5-efdc52797490... 2024-09-21T03:36:49.461 INFO:journalctl@ceph.mds.c.smithi114.stdout:Sep 21 03:36:49 smithi114 podman[62955]: 2024-09-21 03:36:49.181334616 +0000 UTC m=+0.108997664 container create bf3e0de614f42bd43da8e4e74d065a0be8fe272dd6cc6920e39a48b8abeb5749 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-7c374098-77c9-11ef-baf5-efdc52797490-mds-c, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, ceph=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_CLEAN=True) 2024-09-21T03:36:49.461 INFO:journalctl@ceph.mds.c.smithi114.stdout:Sep 21 03:36:49 smithi114 podman[62955]: 2024-09-21 03:36:49.088847441 +0000 UTC m=+0.016510447 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T03:36:49.461 INFO:journalctl@ceph.mds.c.smithi114.stdout:Sep 21 03:36:49 smithi114 podman[62955]: 2024-09-21 03:36:49.392422126 +0000 UTC m=+0.320085135 container init bf3e0de614f42bd43da8e4e74d065a0be8fe272dd6cc6920e39a48b8abeb5749 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-7c374098-77c9-11ef-baf5-efdc52797490-mds-c, org.label-schema.build-date=20240731, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-21T03:36:49.461 INFO:journalctl@ceph.mds.c.smithi114.stdout:Sep 21 03:36:49 smithi114 podman[62955]: 2024-09-21 03:36:49.395399721 +0000 UTC m=+0.323062724 container start bf3e0de614f42bd43da8e4e74d065a0be8fe272dd6cc6920e39a48b8abeb5749 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-7c374098-77c9-11ef-baf5-efdc52797490-mds-c, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-21T03:36:49.461 INFO:journalctl@ceph.mds.c.smithi114.stdout:Sep 21 03:36:49 smithi114 ceph-mds[62974]: -- 172.21.15.114:0/2099266470 <== mon.1 v2:172.21.15.114:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x564ca86d5860 con 0x564cac87d180 2024-09-21T03:36:49.461 INFO:journalctl@ceph.mds.c.smithi114.stdout:Sep 21 03:36:49 smithi114 ceph-7c374098-77c9-11ef-baf5-efdc52797490-mds-c[62970]: starting mds.c at 2024-09-21T03:36:49.815 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 7c374098-77c9-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T03:36:49.841 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:36:49 smithi089 ceph-mon[30928]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:36:49.841 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:36:49 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:36:49.961 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:36:49 smithi114 ceph-mon[34140]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:36:49.961 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:36:49 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:36:49.961 INFO:journalctl@ceph.mds.c.smithi114.stdout:Sep 21 03:36:49 smithi114 bash[62955]: bf3e0de614f42bd43da8e4e74d065a0be8fe272dd6cc6920e39a48b8abeb5749 2024-09-21T03:36:49.961 INFO:journalctl@ceph.mds.c.smithi114.stdout:Sep 21 03:36:49 smithi114 systemd[1]: Started Ceph mds.c for 7c374098-77c9-11ef-baf5-efdc52797490. 2024-09-21T03:36:49.967 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:36:49 smithi110 ceph-mon[34000]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:36:49.968 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:36:49 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:36:50.075 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/7c374098-77c9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:36:50.615 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:50.590+0000 7fad755fc640 1 -- 172.21.15.89:0/3744944374 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fad70102a60 msgr2=0x7fad70102e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:50.615 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:50.590+0000 7fad755fc640 1 --2- 172.21.15.89:0/3744944374 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fad70102a60 0x7fad70102e70 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fad60009900 tx=0x7fad6002a410 comp rx=0 tx=0).stop 2024-09-21T03:36:50.616 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:50.590+0000 7fad755fc640 1 -- 172.21.15.89:0/3744944374 shutdown_connections 2024-09-21T03:36:50.616 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:50.590+0000 7fad755fc640 1 --2- 172.21.15.89:0/3744944374 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fad70108cf0 0x7fad7010f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:50.616 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:50.590+0000 7fad755fc640 1 --2- 172.21.15.89:0/3744944374 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fad701083a0 0x7fad701087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:50.616 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:50.590+0000 7fad755fc640 1 --2- 172.21.15.89:0/3744944374 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fad70102a60 0x7fad70102e70 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:50.616 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:50.590+0000 7fad755fc640 1 -- 172.21.15.89:0/3744944374 >> 172.21.15.89:0/3744944374 conn(0x7fad70079730 msgr2=0x7fad70079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:36:50.616 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:50.590+0000 7fad755fc640 1 -- 172.21.15.89:0/3744944374 shutdown_connections 2024-09-21T03:36:50.616 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:50.590+0000 7fad755fc640 1 -- 172.21.15.89:0/3744944374 wait complete. 2024-09-21T03:36:50.616 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:50.590+0000 7fad755fc640 1 Processor -- start 2024-09-21T03:36:50.616 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:50.591+0000 7fad755fc640 1 -- start start 2024-09-21T03:36:50.616 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:50.591+0000 7fad755fc640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fad70102a60 0x7fad7019b1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:50.616 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:50.591+0000 7fad755fc640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fad701083a0 0x7fad7019b720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:50.616 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:50.591+0000 7fad755fc640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fad70108cf0 0x7fad7019bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:50.616 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:50.591+0000 7fad755fc640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fad70112810 con 0x7fad70108cf0 2024-09-21T03:36:50.616 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:50.591+0000 7fad755fc640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fad70112690 con 0x7fad701083a0 2024-09-21T03:36:50.617 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:50.591+0000 7fad755fc640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fad70112990 con 0x7fad70102a60 2024-09-21T03:36:50.617 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:50.591+0000 7fad6f7fe640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fad70108cf0 0x7fad7019bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:50.617 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:50.591+0000 7fad6f7fe640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fad70108cf0 0x7fad7019bc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:39808/0 (socket says 172.21.15.89:39808) 2024-09-21T03:36:50.617 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:50.591+0000 7fad6f7fe640 1 -- 172.21.15.89:0/63371736 learned_addr learned my addr 172.21.15.89:0/63371736 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-21T03:36:50.617 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:50.591+0000 7fad6e7fc640 1 --2- 172.21.15.89:0/63371736 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fad701083a0 0x7fad7019b720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:50.617 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:50.591+0000 7fad6effd640 1 --2- 172.21.15.89:0/63371736 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fad70102a60 0x7fad7019b1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:50.617 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:50.592+0000 7fad6f7fe640 1 -- 172.21.15.89:0/63371736 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fad70102a60 msgr2=0x7fad7019b1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:50.617 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:50.592+0000 7fad6f7fe640 1 --2- 172.21.15.89:0/63371736 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fad70102a60 0x7fad7019b1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:50.617 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:50.592+0000 7fad6f7fe640 1 -- 172.21.15.89:0/63371736 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fad701083a0 msgr2=0x7fad7019b720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:50.617 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:50.592+0000 7fad6f7fe640 1 --2- 172.21.15.89:0/63371736 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fad701083a0 0x7fad7019b720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:50.617 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:50.592+0000 7fad6f7fe640 1 -- 172.21.15.89:0/63371736 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad701a6470 con 0x7fad70108cf0 2024-09-21T03:36:50.617 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:50.592+0000 7fad6f7fe640 1 --2- 172.21.15.89:0/63371736 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fad70108cf0 0x7fad7019bc60 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fad6800eaa0 tx=0x7fad6800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:36:50.617 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:50.592+0000 7fad57fff640 1 -- 172.21.15.89:0/63371736 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad6800cdb0 con 0x7fad70108cf0 2024-09-21T03:36:50.617 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:50.592+0000 7fad755fc640 1 -- 172.21.15.89:0/63371736 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fad701a6760 con 0x7fad70108cf0 2024-09-21T03:36:50.618 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:50.592+0000 7fad57fff640 1 -- 172.21.15.89:0/63371736 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fad68005020 con 0x7fad70108cf0 2024-09-21T03:36:50.618 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:50.592+0000 7fad57fff640 1 -- 172.21.15.89:0/63371736 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad680116d0 con 0x7fad70108cf0 2024-09-21T03:36:50.618 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:50.592+0000 7fad755fc640 1 -- 172.21.15.89:0/63371736 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fad701a6ca0 con 0x7fad70108cf0 2024-09-21T03:36:50.618 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:50.593+0000 7fad755fc640 1 -- 172.21.15.89:0/63371736 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fad70102e70 con 0x7fad70108cf0 2024-09-21T03:36:50.618 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:50.596+0000 7fad57fff640 1 -- 172.21.15.89:0/63371736 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 17) ==== 154512+0+0 (secure 0 0 0) 0x7fad680051d0 con 0x7fad70108cf0 2024-09-21T03:36:50.618 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:50.596+0000 7fad57fff640 1 --2- 172.21.15.89:0/63371736 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7fad580b6540 0x7fad580b8a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:50.618 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:50.596+0000 7fad6effd640 1 --2- 172.21.15.89:0/63371736 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7fad580b6540 0x7fad580b8a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:50.618 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:50.596+0000 7fad57fff640 1 -- 172.21.15.89:0/63371736 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 7127+0+0 (secure 0 0 0) 0x7fad680e7300 con 0x7fad70108cf0 2024-09-21T03:36:50.618 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:50.597+0000 7fad6effd640 1 --2- 172.21.15.89:0/63371736 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7fad580b6540 0x7fad580b8a50 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fad7006b9c0 tx=0x7fad600058d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:36:50.618 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:50.599+0000 7fad57fff640 1 -- 172.21.15.89:0/63371736 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fad68016030 con 0x7fad70108cf0 2024-09-21T03:36:50.811 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:36:50 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:36:50.811 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:36:50 smithi089 ceph-mon[30928]: mds.? [v2:172.21.15.114:6836/1941213818,v1:172.21.15.114:6837/1941213818] up:boot 2024-09-21T03:36:50.811 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:36:50 smithi089 ceph-mon[30928]: fsmap 7 up:standby 2024-09-21T03:36:50.811 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:36:50 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "c"} : dispatch 2024-09-21T03:36:50.811 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:36:50 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:36:50.811 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:36:50 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T03:36:50.811 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:36:50 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T03:36:50.811 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:36:50 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:36:50.811 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:36:50 smithi089 ceph-mon[30928]: Deploying daemon mds.h on smithi110 2024-09-21T03:36:50.827 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:50.825+0000 7fad755fc640 1 -- 172.21.15.89:0/63371736 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fad7006b270 con 0x7fad70108cf0 2024-09-21T03:36:50.827 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:50.825+0000 7fad57fff640 1 -- 172.21.15.89:0/63371736 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+6388 (secure 0 0 0) 0x7fad7006b270 con 0x7fad70108cf0 2024-09-21T03:36:50.828 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-09-21T03:36:50.828 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":8,"btime":"2024-09-21T03:36:49:554671+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14721,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6835/2341538514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":2341538514},{"type":"v1","addr":"172.21.15.89:6835","nonce":2341538514}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14733,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6837/146708592","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6836","nonce":146708592},{"type":"v1","addr":"172.21.15.89:6837","nonce":146708592}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24565,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6833/752336912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6832","nonce":752336912},{"type":"v1","addr":"172.21.15.114:6833","nonce":752336912}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24575,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6833/127607331","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6832","nonce":127607331},{"type":"v1","addr":"172.21.15.110:6833","nonce":127607331}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24593,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6835/671820654","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6834","nonce":671820654},{"type":"v1","addr":"172.21.15.110:6835","nonce":671820654}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24604,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6835/1049714548","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6834","nonce":1049714548},{"type":"v1","addr":"172.21.15.114:6835","nonce":1049714548}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24610,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6837/1941213818","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6836","nonce":1941213818},{"type":"v1","addr":"172.21.15.114:6837","nonce":1941213818}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[]} 2024-09-21T03:36:50.828 INFO:teuthology.orchestra.run.smithi089.stderr:dumped fsmap epoch 8 2024-09-21T03:36:50.830 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:50.828+0000 7fad755fc640 1 -- 172.21.15.89:0/63371736 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7fad580b6540 msgr2=0x7fad580b8a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:50.830 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:50.828+0000 7fad755fc640 1 --2- 172.21.15.89:0/63371736 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7fad580b6540 0x7fad580b8a50 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fad7006b9c0 tx=0x7fad600058d0 comp rx=0 tx=0).stop 2024-09-21T03:36:50.830 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:50.828+0000 7fad755fc640 1 -- 172.21.15.89:0/63371736 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fad70108cf0 msgr2=0x7fad7019bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:50.830 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:50.828+0000 7fad755fc640 1 --2- 172.21.15.89:0/63371736 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fad70108cf0 0x7fad7019bc60 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fad6800eaa0 tx=0x7fad6800ef60 comp rx=0 tx=0).stop 2024-09-21T03:36:50.830 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:50.828+0000 7fad755fc640 1 -- 172.21.15.89:0/63371736 shutdown_connections 2024-09-21T03:36:50.830 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:50.828+0000 7fad755fc640 1 --2- 172.21.15.89:0/63371736 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7fad580b6540 0x7fad580b8a50 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:50.830 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:50.828+0000 7fad755fc640 1 --2- 172.21.15.89:0/63371736 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fad701083a0 0x7fad7019b720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:50.830 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:50.828+0000 7fad755fc640 1 --2- 172.21.15.89:0/63371736 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fad70108cf0 0x7fad7019bc60 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:50.831 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:50.828+0000 7fad755fc640 1 --2- 172.21.15.89:0/63371736 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fad70102a60 0x7fad7019b1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:50.831 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:50.828+0000 7fad755fc640 1 -- 172.21.15.89:0/63371736 >> 172.21.15.89:0/63371736 conn(0x7fad70079730 msgr2=0x7fad70101500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:36:50.831 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:50.828+0000 7fad755fc640 1 -- 172.21.15.89:0/63371736 shutdown_connections 2024-09-21T03:36:50.831 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:50.828+0000 7fad755fc640 1 -- 172.21.15.89:0/63371736 wait complete. 2024-09-21T03:36:50.961 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:36:50 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:36:50.961 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:36:50 smithi114 ceph-mon[34140]: mds.? [v2:172.21.15.114:6836/1941213818,v1:172.21.15.114:6837/1941213818] up:boot 2024-09-21T03:36:50.961 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:36:50 smithi114 ceph-mon[34140]: fsmap 7 up:standby 2024-09-21T03:36:50.961 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:36:50 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "c"} : dispatch 2024-09-21T03:36:50.961 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:36:50 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:36:50.961 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:36:50 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T03:36:50.961 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:36:50 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T03:36:50.961 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:36:50 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:36:50.961 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:36:50 smithi114 ceph-mon[34140]: Deploying daemon mds.h on smithi110 2024-09-21T03:36:50.973 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:36:50 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:36:50.973 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:36:50 smithi110 ceph-mon[34000]: mds.? [v2:172.21.15.114:6836/1941213818,v1:172.21.15.114:6837/1941213818] up:boot 2024-09-21T03:36:50.974 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:36:50 smithi110 ceph-mon[34000]: fsmap 7 up:standby 2024-09-21T03:36:50.974 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:36:50 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "c"} : dispatch 2024-09-21T03:36:50.974 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:36:50 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:36:50.974 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:36:50 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T03:36:50.974 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:36:50 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T03:36:50.974 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:36:50 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:36:50.974 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:36:50 smithi110 ceph-mon[34000]: Deploying daemon mds.h on smithi110 2024-09-21T03:36:51.578 INFO:journalctl@ceph.mds.h.smithi110.stdout:Sep 21 03:36:51 smithi110 systemd[1]: Starting Ceph mds.h for 7c374098-77c9-11ef-baf5-efdc52797490... 2024-09-21T03:36:51.811 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:36:51 smithi089 ceph-mon[30928]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:36:51.811 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:36:51 smithi089 ceph-mon[30928]: from='client.? 172.21.15.89:0/63371736' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:36:51.853 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:36:51 smithi110 ceph-mon[34000]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:36:51.854 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:36:51 smithi110 ceph-mon[34000]: from='client.? 172.21.15.89:0/63371736' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:36:51.854 INFO:journalctl@ceph.mds.h.smithi110.stdout:Sep 21 03:36:51 smithi110 podman[61803]: 2024-09-21 03:36:51.630528359 +0000 UTC m=+0.015504362 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T03:36:51.854 INFO:journalctl@ceph.mds.h.smithi110.stdout:Sep 21 03:36:51 smithi110 podman[61803]: 2024-09-21 03:36:51.747401801 +0000 UTC m=+0.132377795 container create 9ea35882772d0e1274196d02245f31d253ca3349e4fa33822481dfc227df39c0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-7c374098-77c9-11ef-baf5-efdc52797490-mds-h, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.build-date=20240731, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux ) 2024-09-21T03:36:51.960 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:36:51 smithi114 ceph-mon[34140]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:36:51.961 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:36:51 smithi114 ceph-mon[34140]: from='client.? 172.21.15.89:0/63371736' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:36:52.250 INFO:journalctl@ceph.mds.h.smithi110.stdout:Sep 21 03:36:51 smithi110 podman[61803]: 2024-09-21 03:36:51.983633282 +0000 UTC m=+0.368609286 container init 9ea35882772d0e1274196d02245f31d253ca3349e4fa33822481dfc227df39c0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-7c374098-77c9-11ef-baf5-efdc52797490-mds-h, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , ceph=True) 2024-09-21T03:36:52.250 INFO:journalctl@ceph.mds.h.smithi110.stdout:Sep 21 03:36:51 smithi110 podman[61803]: 2024-09-21 03:36:51.986474863 +0000 UTC m=+0.371450864 container start 9ea35882772d0e1274196d02245f31d253ca3349e4fa33822481dfc227df39c0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-7c374098-77c9-11ef-baf5-efdc52797490-mds-h, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, ceph=True, io.buildah.version=1.37.2) 2024-09-21T03:36:52.250 INFO:journalctl@ceph.mds.h.smithi110.stdout:Sep 21 03:36:52 smithi110 ceph-mds[61823]: -- 172.21.15.110:0/235294095 <== mon.2 v2:172.21.15.110:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x562d674b9860 con 0x562d6b661180 2024-09-21T03:36:52.250 INFO:journalctl@ceph.mds.h.smithi110.stdout:Sep 21 03:36:52 smithi110 ceph-7c374098-77c9-11ef-baf5-efdc52797490-mds-h[61819]: starting mds.h at 2024-09-21T03:36:52.250 INFO:journalctl@ceph.mds.h.smithi110.stdout:Sep 21 03:36:52 smithi110 bash[61803]: 9ea35882772d0e1274196d02245f31d253ca3349e4fa33822481dfc227df39c0 2024-09-21T03:36:52.250 INFO:journalctl@ceph.mds.h.smithi110.stdout:Sep 21 03:36:52 smithi110 systemd[1]: Started Ceph mds.h for 7c374098-77c9-11ef-baf5-efdc52797490. 2024-09-21T03:36:53.461 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:36:53 smithi114 ceph-mon[34140]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:36:53.461 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:36:53 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:36:53.461 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:36:53 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:36:53.461 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:36:53 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:36:53.461 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:36:53 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T03:36:53.461 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:36:53 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T03:36:53.461 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:36:53 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:36:53.461 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:36:53 smithi114 ceph-mon[34140]: Deploying daemon mds.i on smithi114 2024-09-21T03:36:53.461 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:36:53 smithi114 ceph-mon[34140]: mds.? [v2:172.21.15.110:6836/4047124589,v1:172.21.15.110:6837/4047124589] up:boot 2024-09-21T03:36:53.462 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:36:53 smithi114 ceph-mon[34140]: fsmap 8 up:standby 2024-09-21T03:36:53.462 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:36:53 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "h"} : dispatch 2024-09-21T03:36:53.490 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 7c374098-77c9-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T03:36:53.500 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:36:53 smithi110 ceph-mon[34000]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:36:53.500 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:36:53 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:36:53.500 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:36:53 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:36:53.500 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:36:53 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:36:53.500 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:36:53 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T03:36:53.500 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:36:53 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T03:36:53.500 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:36:53 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:36:53.500 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:36:53 smithi110 ceph-mon[34000]: Deploying daemon mds.i on smithi114 2024-09-21T03:36:53.500 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:36:53 smithi110 ceph-mon[34000]: mds.? [v2:172.21.15.110:6836/4047124589,v1:172.21.15.110:6837/4047124589] up:boot 2024-09-21T03:36:53.500 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:36:53 smithi110 ceph-mon[34000]: fsmap 8 up:standby 2024-09-21T03:36:53.501 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:36:53 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "h"} : dispatch 2024-09-21T03:36:53.520 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:36:53 smithi089 ceph-mon[30928]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:36:53.520 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:36:53 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:36:53.520 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:36:53 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:36:53.520 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:36:53 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:36:53.520 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:36:53 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T03:36:53.521 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:36:53 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T03:36:53.521 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:36:53 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:36:53.521 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:36:53 smithi089 ceph-mon[30928]: Deploying daemon mds.i on smithi114 2024-09-21T03:36:53.521 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:36:53 smithi089 ceph-mon[30928]: mds.? [v2:172.21.15.110:6836/4047124589,v1:172.21.15.110:6837/4047124589] up:boot 2024-09-21T03:36:53.521 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:36:53 smithi089 ceph-mon[30928]: fsmap 8 up:standby 2024-09-21T03:36:53.521 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:36:53 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "h"} : dispatch 2024-09-21T03:36:53.754 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/7c374098-77c9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:36:54.211 INFO:journalctl@ceph.mds.i.smithi114.stdout:Sep 21 03:36:54 smithi114 systemd[1]: Starting Ceph mds.i for 7c374098-77c9-11ef-baf5-efdc52797490... 2024-09-21T03:36:54.267 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:54.256+0000 7ff4a15eb640 1 -- 172.21.15.89:0/2589678483 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff49c108d00 msgr2=0x7ff49c10f530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:54.267 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:54.256+0000 7ff4a15eb640 1 --2- 172.21.15.89:0/2589678483 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff49c108d00 0x7ff49c10f530 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7ff494009ca0 tx=0x7ff49402a8c0 comp rx=0 tx=0).stop 2024-09-21T03:36:54.267 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:54.256+0000 7ff4a15eb640 1 -- 172.21.15.89:0/2589678483 shutdown_connections 2024-09-21T03:36:54.267 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:54.256+0000 7ff4a15eb640 1 --2- 172.21.15.89:0/2589678483 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff49c108d00 0x7ff49c10f530 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:54.267 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:54.256+0000 7ff4a15eb640 1 --2- 172.21.15.89:0/2589678483 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff49c1083b0 0x7ff49c1087c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:54.267 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:54.256+0000 7ff4a15eb640 1 --2- 172.21.15.89:0/2589678483 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff49c102a70 0x7ff49c102e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:54.267 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:54.256+0000 7ff4a15eb640 1 -- 172.21.15.89:0/2589678483 >> 172.21.15.89:0/2589678483 conn(0x7ff49c079710 msgr2=0x7ff49c079b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:36:54.267 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:54.256+0000 7ff4a15eb640 1 -- 172.21.15.89:0/2589678483 shutdown_connections 2024-09-21T03:36:54.267 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:54.256+0000 7ff4a15eb640 1 -- 172.21.15.89:0/2589678483 wait complete. 2024-09-21T03:36:54.267 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:54.256+0000 7ff4a15eb640 1 Processor -- start 2024-09-21T03:36:54.268 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:54.257+0000 7ff4a15eb640 1 -- start start 2024-09-21T03:36:54.268 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:54.257+0000 7ff4a15eb640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff49c102a70 0x7ff49c19a310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:54.268 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:54.257+0000 7ff4a15eb640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff49c1083b0 0x7ff49c19a8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:54.268 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:54.257+0000 7ff4a15eb640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff49c108d00 0x7ff49c19fb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:54.268 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:54.257+0000 7ff4a15eb640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7ff49c1139c0 con 0x7ff49c102a70 2024-09-21T03:36:54.268 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:54.257+0000 7ff4a15eb640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7ff49c113840 con 0x7ff49c108d00 2024-09-21T03:36:54.268 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:54.257+0000 7ff4a15eb640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7ff49c113b40 con 0x7ff49c1083b0 2024-09-21T03:36:54.268 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:54.257+0000 7ff49affd640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff49c102a70 0x7ff49c19a310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:54.268 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:54.257+0000 7ff49affd640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff49c102a70 0x7ff49c19a310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:39824/0 (socket says 172.21.15.89:39824) 2024-09-21T03:36:54.268 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:54.257+0000 7ff49affd640 1 -- 172.21.15.89:0/514370920 learned_addr learned my addr 172.21.15.89:0/514370920 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-21T03:36:54.268 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:54.257+0000 7ff49a7fc640 1 --2- 172.21.15.89:0/514370920 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff49c1083b0 0x7ff49c19a8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:54.268 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:54.257+0000 7ff49b7fe640 1 --2- 172.21.15.89:0/514370920 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff49c108d00 0x7ff49c19fb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:54.268 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:54.257+0000 7ff49affd640 1 -- 172.21.15.89:0/514370920 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff49c1083b0 msgr2=0x7ff49c19a8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:54.268 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:54.257+0000 7ff49affd640 1 --2- 172.21.15.89:0/514370920 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff49c1083b0 0x7ff49c19a8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:54.269 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:54.257+0000 7ff49affd640 1 -- 172.21.15.89:0/514370920 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff49c108d00 msgr2=0x7ff49c19fb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:54.269 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:54.257+0000 7ff49affd640 1 --2- 172.21.15.89:0/514370920 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff49c108d00 0x7ff49c19fb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:54.269 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:54.257+0000 7ff49affd640 1 -- 172.21.15.89:0/514370920 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff49c1a01c0 con 0x7ff49c102a70 2024-09-21T03:36:54.269 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:54.258+0000 7ff49affd640 1 --2- 172.21.15.89:0/514370920 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff49c102a70 0x7ff49c19a310 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7ff48c00c890 tx=0x7ff48c00cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:36:54.269 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:54.258+0000 7ff483fff640 1 -- 172.21.15.89:0/514370920 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff48c004220 con 0x7ff49c102a70 2024-09-21T03:36:54.269 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:54.258+0000 7ff4a15eb640 1 -- 172.21.15.89:0/514370920 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff49c1a04b0 con 0x7ff49c102a70 2024-09-21T03:36:54.269 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:54.258+0000 7ff483fff640 1 -- 172.21.15.89:0/514370920 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff48c007e40 con 0x7ff49c102a70 2024-09-21T03:36:54.269 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:54.258+0000 7ff483fff640 1 -- 172.21.15.89:0/514370920 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff48c0047a0 con 0x7ff49c102a70 2024-09-21T03:36:54.269 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:54.258+0000 7ff4a15eb640 1 -- 172.21.15.89:0/514370920 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff49c1a6cd0 con 0x7ff49c102a70 2024-09-21T03:36:54.269 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:54.259+0000 7ff4a15eb640 1 -- 172.21.15.89:0/514370920 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff49c103cf0 con 0x7ff49c102a70 2024-09-21T03:36:54.269 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:54.260+0000 7ff483fff640 1 -- 172.21.15.89:0/514370920 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 17) ==== 154512+0+0 (secure 0 0 0) 0x7ff48c0079b0 con 0x7ff49c102a70 2024-09-21T03:36:54.269 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:54.261+0000 7ff483fff640 1 --2- 172.21.15.89:0/514370920 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7ff4840b6540 0x7ff4840b8a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:54.269 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:54.261+0000 7ff49a7fc640 1 --2- 172.21.15.89:0/514370920 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7ff4840b6540 0x7ff4840b8a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:54.269 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:54.261+0000 7ff483fff640 1 -- 172.21.15.89:0/514370920 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 7127+0+0 (secure 0 0 0) 0x7ff48c0e6f30 con 0x7ff49c102a70 2024-09-21T03:36:54.270 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:54.261+0000 7ff49a7fc640 1 --2- 172.21.15.89:0/514370920 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7ff4840b6540 0x7ff4840b8a50 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7ff49c106070 tx=0x7ff49000a2d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:36:54.270 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:54.264+0000 7ff483fff640 1 -- 172.21.15.89:0/514370920 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7ff48c014030 con 0x7ff49c102a70 2024-09-21T03:36:54.483 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:54.481+0000 7ff4a15eb640 1 -- 172.21.15.89:0/514370920 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff49c102e80 con 0x7ff49c102a70 2024-09-21T03:36:54.483 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:54.482+0000 7ff483fff640 1 -- 172.21.15.89:0/514370920 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+7216 (secure 0 0 0) 0x7ff48c0adc60 con 0x7ff49c102a70 2024-09-21T03:36:54.484 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-09-21T03:36:54.484 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":9,"btime":"2024-09-21T03:36:52:577810+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14721,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6835/2341538514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":2341538514},{"type":"v1","addr":"172.21.15.89:6835","nonce":2341538514}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14733,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6837/146708592","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6836","nonce":146708592},{"type":"v1","addr":"172.21.15.89:6837","nonce":146708592}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24565,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6833/752336912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6832","nonce":752336912},{"type":"v1","addr":"172.21.15.114:6833","nonce":752336912}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24575,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6833/127607331","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6832","nonce":127607331},{"type":"v1","addr":"172.21.15.110:6833","nonce":127607331}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24593,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6835/671820654","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6834","nonce":671820654},{"type":"v1","addr":"172.21.15.110:6835","nonce":671820654}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24604,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6835/1049714548","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6834","nonce":1049714548},{"type":"v1","addr":"172.21.15.114:6835","nonce":1049714548}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24610,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6837/1941213818","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6836","nonce":1941213818},{"type":"v1","addr":"172.21.15.114:6837","nonce":1941213818}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24614,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6837/4047124589","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6836","nonce":4047124589},{"type":"v1","addr":"172.21.15.110:6837","nonce":4047124589}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[]} 2024-09-21T03:36:54.484 INFO:teuthology.orchestra.run.smithi089.stderr:dumped fsmap epoch 9 2024-09-21T03:36:54.486 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:54.484+0000 7ff4a15eb640 1 -- 172.21.15.89:0/514370920 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7ff4840b6540 msgr2=0x7ff4840b8a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:54.486 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:54.484+0000 7ff4a15eb640 1 --2- 172.21.15.89:0/514370920 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7ff4840b6540 0x7ff4840b8a50 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7ff49c106070 tx=0x7ff49000a2d0 comp rx=0 tx=0).stop 2024-09-21T03:36:54.486 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:54.484+0000 7ff4a15eb640 1 -- 172.21.15.89:0/514370920 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff49c102a70 msgr2=0x7ff49c19a310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:54.486 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:54.484+0000 7ff4a15eb640 1 --2- 172.21.15.89:0/514370920 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff49c102a70 0x7ff49c19a310 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7ff48c00c890 tx=0x7ff48c00cc90 comp rx=0 tx=0).stop 2024-09-21T03:36:54.486 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:54.484+0000 7ff4a15eb640 1 -- 172.21.15.89:0/514370920 shutdown_connections 2024-09-21T03:36:54.486 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:54.484+0000 7ff4a15eb640 1 --2- 172.21.15.89:0/514370920 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7ff4840b6540 0x7ff4840b8a50 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:54.486 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:54.485+0000 7ff4a15eb640 1 --2- 172.21.15.89:0/514370920 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff49c108d00 0x7ff49c19fb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:54.486 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:54.485+0000 7ff4a15eb640 1 --2- 172.21.15.89:0/514370920 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff49c1083b0 0x7ff49c19a8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:54.487 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:54.485+0000 7ff4a15eb640 1 --2- 172.21.15.89:0/514370920 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff49c102a70 0x7ff49c19a310 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:54.487 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:54.485+0000 7ff4a15eb640 1 -- 172.21.15.89:0/514370920 >> 172.21.15.89:0/514370920 conn(0x7ff49c079710 msgr2=0x7ff49c104380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:36:54.487 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:54.485+0000 7ff4a15eb640 1 -- 172.21.15.89:0/514370920 shutdown_connections 2024-09-21T03:36:54.487 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:54.485+0000 7ff4a15eb640 1 -- 172.21.15.89:0/514370920 wait complete. 2024-09-21T03:36:54.607 INFO:journalctl@ceph.mds.i.smithi114.stdout:Sep 21 03:36:54 smithi114 podman[63223]: 2024-09-21 03:36:54.141493588 +0000 UTC m=+0.016321321 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T03:36:54.607 INFO:journalctl@ceph.mds.i.smithi114.stdout:Sep 21 03:36:54 smithi114 podman[63223]: 2024-09-21 03:36:54.244202274 +0000 UTC m=+0.119030009 container create 16eb77ff66450689697786890a8004a04670cc57d9d023916b0728df9c6bb951 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-7c374098-77c9-11ef-baf5-efdc52797490-mds-i, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True) 2024-09-21T03:36:54.607 INFO:journalctl@ceph.mds.i.smithi114.stdout:Sep 21 03:36:54 smithi114 podman[63223]: 2024-09-21 03:36:54.605711062 +0000 UTC m=+0.480538803 container init 16eb77ff66450689697786890a8004a04670cc57d9d023916b0728df9c6bb951 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-7c374098-77c9-11ef-baf5-efdc52797490-mds-i, ceph=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731) 2024-09-21T03:36:54.961 INFO:journalctl@ceph.mds.i.smithi114.stdout:Sep 21 03:36:54 smithi114 podman[63223]: 2024-09-21 03:36:54.608990324 +0000 UTC m=+0.483818068 container start 16eb77ff66450689697786890a8004a04670cc57d9d023916b0728df9c6bb951 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-7c374098-77c9-11ef-baf5-efdc52797490-mds-i, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=) 2024-09-21T03:36:54.993 INFO:journalctl@ceph.mds.i.smithi114.stdout:Sep 21 03:36:54 smithi114 ceph-mds[63242]: -- 172.21.15.114:0/596790163 <== mon.1 v2:172.21.15.114:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x56257f04d860 con 0x5625831f5180 2024-09-21T03:36:54.993 INFO:journalctl@ceph.mds.i.smithi114.stdout:Sep 21 03:36:54 smithi114 ceph-7c374098-77c9-11ef-baf5-efdc52797490-mds-i[63238]: starting mds.i at 2024-09-21T03:36:54.993 INFO:journalctl@ceph.mds.i.smithi114.stdout:Sep 21 03:36:54 smithi114 bash[63223]: 16eb77ff66450689697786890a8004a04670cc57d9d023916b0728df9c6bb951 2024-09-21T03:36:54.993 INFO:journalctl@ceph.mds.i.smithi114.stdout:Sep 21 03:36:54 smithi114 systemd[1]: Started Ceph mds.i for 7c374098-77c9-11ef-baf5-efdc52797490. 2024-09-21T03:36:55.162 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:36:55 smithi089 ceph-mon[30928]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:36:55.162 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:36:55 smithi089 ceph-mon[30928]: from='client.? 172.21.15.89:0/514370920' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:36:55.162 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:36:55 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:36:55.162 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:36:55 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:36:55.162 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:36:55 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:36:55.162 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:36:55 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T03:36:55.163 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:36:55 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T03:36:55.163 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:36:55 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:36:55.461 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:36:55 smithi114 ceph-mon[34140]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:36:55.461 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:36:55 smithi114 ceph-mon[34140]: from='client.? 172.21.15.89:0/514370920' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:36:55.461 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:36:55 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:36:55.461 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:36:55 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:36:55.461 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:36:55 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:36:55.461 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:36:55 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T03:36:55.461 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:36:55 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T03:36:55.461 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:36:55 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:36:55.500 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:36:55 smithi110 ceph-mon[34000]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:36:55.500 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:36:55 smithi110 ceph-mon[34000]: from='client.? 172.21.15.89:0/514370920' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:36:55.500 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:36:55 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:36:55.500 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:36:55 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:36:55.500 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:36:55 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:36:55.500 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:36:55 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T03:36:55.500 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:36:55 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T03:36:55.500 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:36:55 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:36:56.419 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:36:56 smithi089 ceph-mon[30928]: Deploying daemon mds.g on smithi089 2024-09-21T03:36:56.419 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:36:56 smithi089 ceph-mon[30928]: mds.? [v2:172.21.15.114:6838/3427369944,v1:172.21.15.114:6839/3427369944] up:boot 2024-09-21T03:36:56.419 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:36:56 smithi089 ceph-mon[30928]: fsmap 9 up:standby 2024-09-21T03:36:56.419 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:36:56 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "i"} : dispatch 2024-09-21T03:36:56.460 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:36:56 smithi114 ceph-mon[34140]: Deploying daemon mds.g on smithi089 2024-09-21T03:36:56.461 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:36:56 smithi114 ceph-mon[34140]: mds.? [v2:172.21.15.114:6838/3427369944,v1:172.21.15.114:6839/3427369944] up:boot 2024-09-21T03:36:56.461 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:36:56 smithi114 ceph-mon[34140]: fsmap 9 up:standby 2024-09-21T03:36:56.461 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:36:56 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "i"} : dispatch 2024-09-21T03:36:56.500 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:36:56 smithi110 ceph-mon[34000]: Deploying daemon mds.g on smithi089 2024-09-21T03:36:56.500 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:36:56 smithi110 ceph-mon[34000]: mds.? [v2:172.21.15.114:6838/3427369944,v1:172.21.15.114:6839/3427369944] up:boot 2024-09-21T03:36:56.500 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:36:56 smithi110 ceph-mon[34000]: fsmap 9 up:standby 2024-09-21T03:36:56.500 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:36:56 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "i"} : dispatch 2024-09-21T03:36:57.061 INFO:journalctl@ceph.mds.g.smithi089.stdout:Sep 21 03:36:56 smithi089 systemd[1]: Starting Ceph mds.g for 7c374098-77c9-11ef-baf5-efdc52797490... 2024-09-21T03:36:57.189 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 7c374098-77c9-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T03:36:57.352 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:36:57 smithi089 ceph-mon[30928]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:36:57.353 INFO:journalctl@ceph.mds.g.smithi089.stdout:Sep 21 03:36:57 smithi089 podman[71517]: 2024-09-21 03:36:56.996598859 +0000 UTC m=+0.016192458 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T03:36:57.353 INFO:journalctl@ceph.mds.g.smithi089.stdout:Sep 21 03:36:57 smithi089 podman[71517]: 2024-09-21 03:36:57.09743414 +0000 UTC m=+0.117027740 container create bdceadcf48c0f07544364d3e72d42a1271c87f0660dc7bd28a9310e2e2cb1573 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-7c374098-77c9-11ef-baf5-efdc52797490-mds-g, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0) 2024-09-21T03:36:57.353 INFO:journalctl@ceph.mds.g.smithi089.stdout:Sep 21 03:36:57 smithi089 podman[71517]: 2024-09-21 03:36:57.351679806 +0000 UTC m=+0.371273421 container init bdceadcf48c0f07544364d3e72d42a1271c87f0660dc7bd28a9310e2e2cb1573 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-7c374098-77c9-11ef-baf5-efdc52797490-mds-g, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_CLEAN=True) 2024-09-21T03:36:57.461 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:36:57 smithi114 ceph-mon[34140]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:36:57.500 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:36:57 smithi110 ceph-mon[34000]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:36:57.533 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/7c374098-77c9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:36:57.655 INFO:journalctl@ceph.mds.g.smithi089.stdout:Sep 21 03:36:57 smithi089 podman[71517]: 2024-09-21 03:36:57.356056384 +0000 UTC m=+0.375649983 container start bdceadcf48c0f07544364d3e72d42a1271c87f0660dc7bd28a9310e2e2cb1573 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-7c374098-77c9-11ef-baf5-efdc52797490-mds-g, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-21T03:36:57.655 INFO:journalctl@ceph.mds.g.smithi089.stdout:Sep 21 03:36:57 smithi089 ceph-mds[71554]: -- 172.21.15.89:0/3423083182 <== mon.0 v2:172.21.15.89:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x557546d1d860 con 0x55754aec5a80 2024-09-21T03:36:57.656 INFO:journalctl@ceph.mds.g.smithi089.stdout:Sep 21 03:36:57 smithi089 ceph-7c374098-77c9-11ef-baf5-efdc52797490-mds-g[71550]: starting mds.g at 2024-09-21T03:36:57.656 INFO:journalctl@ceph.mds.g.smithi089.stdout:Sep 21 03:36:57 smithi089 bash[71517]: bdceadcf48c0f07544364d3e72d42a1271c87f0660dc7bd28a9310e2e2cb1573 2024-09-21T03:36:57.656 INFO:journalctl@ceph.mds.g.smithi089.stdout:Sep 21 03:36:57 smithi089 systemd[1]: Started Ceph mds.g for 7c374098-77c9-11ef-baf5-efdc52797490. 2024-09-21T03:36:58.058 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:58.054+0000 7fcf884f0640 1 -- 172.21.15.89:0/3665408583 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcf801049b0 msgr2=0x7fcf80104dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:58.058 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:58.054+0000 7fcf884f0640 1 --2- 172.21.15.89:0/3665408583 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcf801049b0 0x7fcf80104dc0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fcf70009ca0 tx=0x7fcf7002a8e0 comp rx=0 tx=0).stop 2024-09-21T03:36:58.058 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:58.054+0000 7fcf884f0640 1 -- 172.21.15.89:0/3665408583 shutdown_connections 2024-09-21T03:36:58.058 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:58.054+0000 7fcf884f0640 1 --2- 172.21.15.89:0/3665408583 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcf80105300 0x7fcf8010d850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:58.058 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:58.054+0000 7fcf884f0640 1 --2- 172.21.15.89:0/3665408583 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fcf8010de10 0x7fcf80110300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:58.058 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:58.054+0000 7fcf884f0640 1 --2- 172.21.15.89:0/3665408583 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcf801049b0 0x7fcf80104dc0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:58.058 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:58.054+0000 7fcf884f0640 1 -- 172.21.15.89:0/3665408583 >> 172.21.15.89:0/3665408583 conn(0x7fcf80079730 msgr2=0x7fcf80079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:36:58.058 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:58.054+0000 7fcf884f0640 1 -- 172.21.15.89:0/3665408583 shutdown_connections 2024-09-21T03:36:58.058 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:58.054+0000 7fcf884f0640 1 -- 172.21.15.89:0/3665408583 wait complete. 2024-09-21T03:36:58.058 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:58.055+0000 7fcf884f0640 1 Processor -- start 2024-09-21T03:36:58.058 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:58.055+0000 7fcf884f0640 1 -- start start 2024-09-21T03:36:58.059 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:58.055+0000 7fcf884f0640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fcf801049b0 0x7fcf8019a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:58.059 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:58.055+0000 7fcf884f0640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcf80105300 0x7fcf8019a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:58.059 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:58.055+0000 7fcf884f0640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcf8010de10 0x7fcf8019fc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:58.059 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:58.055+0000 7fcf884f0640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fcf801139a0 con 0x7fcf80105300 2024-09-21T03:36:58.059 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:58.055+0000 7fcf884f0640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fcf80113820 con 0x7fcf8010de10 2024-09-21T03:36:58.059 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:58.055+0000 7fcf884f0640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fcf80113b20 con 0x7fcf801049b0 2024-09-21T03:36:58.059 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:58.055+0000 7fcf85a8b640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcf80105300 0x7fcf8019a9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:58.059 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:58.055+0000 7fcf85a8b640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcf80105300 0x7fcf8019a9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:39860/0 (socket says 172.21.15.89:39860) 2024-09-21T03:36:58.059 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:58.055+0000 7fcf85a8b640 1 -- 172.21.15.89:0/161840 learned_addr learned my addr 172.21.15.89:0/161840 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-21T03:36:58.059 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:58.055+0000 7fcf86a8d640 1 --2- 172.21.15.89:0/161840 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcf8010de10 0x7fcf8019fc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:58.059 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:58.055+0000 7fcf8628c640 1 --2- 172.21.15.89:0/161840 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fcf801049b0 0x7fcf8019a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:58.059 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:58.056+0000 7fcf85a8b640 1 -- 172.21.15.89:0/161840 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fcf801049b0 msgr2=0x7fcf8019a430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:58.059 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:58.056+0000 7fcf85a8b640 1 --2- 172.21.15.89:0/161840 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fcf801049b0 0x7fcf8019a430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:58.060 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:58.056+0000 7fcf85a8b640 1 -- 172.21.15.89:0/161840 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcf8010de10 msgr2=0x7fcf8019fc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:58.060 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:58.056+0000 7fcf85a8b640 1 --2- 172.21.15.89:0/161840 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcf8010de10 0x7fcf8019fc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:58.060 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:58.056+0000 7fcf85a8b640 1 -- 172.21.15.89:0/161840 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf801a03a0 con 0x7fcf80105300 2024-09-21T03:36:58.060 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:58.056+0000 7fcf85a8b640 1 --2- 172.21.15.89:0/161840 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcf80105300 0x7fcf8019a9d0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fcf74006fd0 tx=0x7fcf7400b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:36:58.060 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:58.056+0000 7fcf7f7fe640 1 -- 172.21.15.89:0/161840 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf74011830 con 0x7fcf80105300 2024-09-21T03:36:58.060 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:58.056+0000 7fcf7f7fe640 1 -- 172.21.15.89:0/161840 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fcf74016070 con 0x7fcf80105300 2024-09-21T03:36:58.060 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:58.056+0000 7fcf884f0640 1 -- 172.21.15.89:0/161840 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcf801a0690 con 0x7fcf80105300 2024-09-21T03:36:58.060 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:58.056+0000 7fcf7f7fe640 1 -- 172.21.15.89:0/161840 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf74010d30 con 0x7fcf80105300 2024-09-21T03:36:58.060 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:58.056+0000 7fcf884f0640 1 -- 172.21.15.89:0/161840 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcf801a6df0 con 0x7fcf80105300 2024-09-21T03:36:58.061 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:58.057+0000 7fcf884f0640 1 -- 172.21.15.89:0/161840 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcf80104dc0 con 0x7fcf80105300 2024-09-21T03:36:58.061 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:58.059+0000 7fcf7f7fe640 1 -- 172.21.15.89:0/161840 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 17) ==== 154512+0+0 (secure 0 0 0) 0x7fcf740119d0 con 0x7fcf80105300 2024-09-21T03:36:58.061 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:58.059+0000 7fcf7f7fe640 1 --2- 172.21.15.89:0/161840 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7fcf680b6510 0x7fcf680b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:58.061 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:58.060+0000 7fcf8628c640 1 --2- 172.21.15.89:0/161840 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7fcf680b6510 0x7fcf680b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:58.061 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:58.060+0000 7fcf7f7fe640 1 -- 172.21.15.89:0/161840 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 7127+0+0 (secure 0 0 0) 0x7fcf740e6960 con 0x7fcf80105300 2024-09-21T03:36:58.061 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:58.060+0000 7fcf8628c640 1 --2- 172.21.15.89:0/161840 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7fcf680b6510 0x7fcf680b8a20 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fcf70000f80 tx=0x7fcf70009360 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:36:58.063 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:58.063+0000 7fcf7f7fe640 1 -- 172.21.15.89:0/161840 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fcf740ad690 con 0x7fcf80105300 2024-09-21T03:36:58.295 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:58.293+0000 7fcf884f0640 1 -- 172.21.15.89:0/161840 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fcf8019b560 con 0x7fcf80105300 2024-09-21T03:36:58.295 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:58.294+0000 7fcf7f7fe640 1 -- 172.21.15.89:0/161840 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) ==== 94+0+8872 (secure 0 0 0) 0x7fcf8019b560 con 0x7fcf80105300 2024-09-21T03:36:58.296 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-09-21T03:36:58.296 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":11,"btime":"2024-09-21T03:36:58:148119+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14721,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6835/2341538514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":2341538514},{"type":"v1","addr":"172.21.15.89:6835","nonce":2341538514}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14733,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6837/146708592","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6836","nonce":146708592},{"type":"v1","addr":"172.21.15.89:6837","nonce":146708592}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14781,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6839/2269965499","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6838","nonce":2269965499},{"type":"v1","addr":"172.21.15.89:6839","nonce":2269965499}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24565,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6833/752336912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6832","nonce":752336912},{"type":"v1","addr":"172.21.15.114:6833","nonce":752336912}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24575,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6833/127607331","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6832","nonce":127607331},{"type":"v1","addr":"172.21.15.110:6833","nonce":127607331}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24593,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6835/671820654","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6834","nonce":671820654},{"type":"v1","addr":"172.21.15.110:6835","nonce":671820654}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24604,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6835/1049714548","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6834","nonce":1049714548},{"type":"v1","addr":"172.21.15.114:6835","nonce":1049714548}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24610,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6837/1941213818","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6836","nonce":1941213818},{"type":"v1","addr":"172.21.15.114:6837","nonce":1941213818}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24614,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6837/4047124589","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6836","nonce":4047124589},{"type":"v1","addr":"172.21.15.110:6837","nonce":4047124589}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24631,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6839/3427369944","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6838","nonce":3427369944},{"type":"v1","addr":"172.21.15.114:6839","nonce":3427369944}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-09-21T03:36:58.296 INFO:teuthology.orchestra.run.smithi089.stderr:dumped fsmap epoch 11 2024-09-21T03:36:58.297 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:58.296+0000 7fcf884f0640 1 -- 172.21.15.89:0/161840 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7fcf680b6510 msgr2=0x7fcf680b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:58.297 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:58.296+0000 7fcf884f0640 1 --2- 172.21.15.89:0/161840 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7fcf680b6510 0x7fcf680b8a20 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fcf70000f80 tx=0x7fcf70009360 comp rx=0 tx=0).stop 2024-09-21T03:36:58.298 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:58.296+0000 7fcf884f0640 1 -- 172.21.15.89:0/161840 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcf80105300 msgr2=0x7fcf8019a9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:58.298 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:58.296+0000 7fcf884f0640 1 --2- 172.21.15.89:0/161840 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcf80105300 0x7fcf8019a9d0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fcf74006fd0 tx=0x7fcf7400b8e0 comp rx=0 tx=0).stop 2024-09-21T03:36:58.298 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:58.296+0000 7fcf884f0640 1 -- 172.21.15.89:0/161840 shutdown_connections 2024-09-21T03:36:58.298 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:58.296+0000 7fcf884f0640 1 --2- 172.21.15.89:0/161840 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7fcf680b6510 0x7fcf680b8a20 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:58.298 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:58.296+0000 7fcf884f0640 1 --2- 172.21.15.89:0/161840 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcf8010de10 0x7fcf8019fc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:58.298 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:58.296+0000 7fcf884f0640 1 --2- 172.21.15.89:0/161840 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcf80105300 0x7fcf8019a9d0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:58.298 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:58.296+0000 7fcf884f0640 1 --2- 172.21.15.89:0/161840 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fcf801049b0 0x7fcf8019a430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:58.298 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:58.296+0000 7fcf884f0640 1 -- 172.21.15.89:0/161840 >> 172.21.15.89:0/161840 conn(0x7fcf80079730 msgr2=0x7fcf800ff730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:36:58.298 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:58.296+0000 7fcf884f0640 1 -- 172.21.15.89:0/161840 shutdown_connections 2024-09-21T03:36:58.298 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:36:58.296+0000 7fcf884f0640 1 -- 172.21.15.89:0/161840 wait complete. 2024-09-21T03:36:58.811 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:36:58 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:36:58.811 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:36:58 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:36:58.811 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:36:58 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:36:58.811 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:36:58 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T03:36:58.812 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:36:58 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T03:36:58.812 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:36:58 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:36:58.812 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:36:58 smithi089 ceph-mon[30928]: Deploying daemon mds.a on smithi089 2024-09-21T03:36:58.812 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:36:58 smithi089 ceph-mon[30928]: mds.? [v2:172.21.15.89:6838/2269965499,v1:172.21.15.89:6839/2269965499] up:boot 2024-09-21T03:36:58.812 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:36:58 smithi089 ceph-mon[30928]: fsmap 10 up:standby 2024-09-21T03:36:58.812 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:36:58 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "g"} : dispatch 2024-09-21T03:36:58.812 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:36:58 smithi089 ceph-mon[30928]: from='client.? 172.21.15.89:0/161840' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:36:58.961 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:36:58 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:36:58.961 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:36:58 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:36:58.961 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:36:58 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:36:58.961 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:36:58 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T03:36:58.961 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:36:58 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T03:36:58.961 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:36:58 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:36:58.961 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:36:58 smithi114 ceph-mon[34140]: Deploying daemon mds.a on smithi089 2024-09-21T03:36:58.961 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:36:58 smithi114 ceph-mon[34140]: mds.? [v2:172.21.15.89:6838/2269965499,v1:172.21.15.89:6839/2269965499] up:boot 2024-09-21T03:36:58.961 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:36:58 smithi114 ceph-mon[34140]: fsmap 10 up:standby 2024-09-21T03:36:58.962 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:36:58 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "g"} : dispatch 2024-09-21T03:36:58.962 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:36:58 smithi114 ceph-mon[34140]: from='client.? 172.21.15.89:0/161840' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:36:59.000 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:36:58 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:36:59.000 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:36:58 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:36:59.000 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:36:58 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:36:59.001 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:36:58 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T03:36:59.001 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:36:58 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T03:36:59.001 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:36:58 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:36:59.001 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:36:58 smithi110 ceph-mon[34000]: Deploying daemon mds.a on smithi089 2024-09-21T03:36:59.001 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:36:58 smithi110 ceph-mon[34000]: mds.? [v2:172.21.15.89:6838/2269965499,v1:172.21.15.89:6839/2269965499] up:boot 2024-09-21T03:36:59.001 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:36:58 smithi110 ceph-mon[34000]: fsmap 10 up:standby 2024-09-21T03:36:59.001 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:36:58 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "g"} : dispatch 2024-09-21T03:36:59.001 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:36:58 smithi110 ceph-mon[34000]: from='client.? 172.21.15.89:0/161840' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:36:59.811 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:36:59 smithi089 ceph-mon[30928]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:36:59.962 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:36:59 smithi114 ceph-mon[34140]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:37:00.000 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:36:59 smithi110 ceph-mon[34000]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:37:00.548 INFO:journalctl@ceph.mds.a.smithi089.stdout:Sep 21 03:37:00 smithi089 systemd[1]: Starting Ceph mds.a for 7c374098-77c9-11ef-baf5-efdc52797490... 2024-09-21T03:37:00.807 INFO:journalctl@ceph.mds.a.smithi089.stdout:Sep 21 03:37:00 smithi089 podman[71899]: 2024-09-21 03:37:00.448713993 +0000 UTC m=+0.015885497 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T03:37:00.807 INFO:journalctl@ceph.mds.a.smithi089.stdout:Sep 21 03:37:00 smithi089 podman[71899]: 2024-09-21 03:37:00.569331668 +0000 UTC m=+0.136503159 container create a9b21a569206b02a008e784f6e1fc614a9001b8fc489f59d33710c72db5018a0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-7c374098-77c9-11ef-baf5-efdc52797490-mds-a, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, ceph=True) 2024-09-21T03:37:00.808 INFO:journalctl@ceph.mds.a.smithi089.stdout:Sep 21 03:37:00 smithi089 podman[71899]: 2024-09-21 03:37:00.806430084 +0000 UTC m=+0.373601573 container init a9b21a569206b02a008e784f6e1fc614a9001b8fc489f59d33710c72db5018a0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-7c374098-77c9-11ef-baf5-efdc52797490-mds-a, GIT_CLEAN=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.schema-version=1.0) 2024-09-21T03:37:01.021 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 7c374098-77c9-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T03:37:01.061 INFO:journalctl@ceph.mds.a.smithi089.stdout:Sep 21 03:37:00 smithi089 podman[71899]: 2024-09-21 03:37:00.809433677 +0000 UTC m=+0.376605168 container start a9b21a569206b02a008e784f6e1fc614a9001b8fc489f59d33710c72db5018a0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-7c374098-77c9-11ef-baf5-efdc52797490-mds-a, org.label-schema.build-date=20240731, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_CLEAN=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-21T03:37:01.061 INFO:journalctl@ceph.mds.a.smithi089.stdout:Sep 21 03:37:00 smithi089 ceph-mds[71919]: -- 172.21.15.89:0/4228575503 <== mon.0 v2:172.21.15.89:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55da93f33860 con 0x55da980db180 2024-09-21T03:37:01.061 INFO:journalctl@ceph.mds.a.smithi089.stdout:Sep 21 03:37:00 smithi089 ceph-7c374098-77c9-11ef-baf5-efdc52797490-mds-a[71915]: starting mds.a at 2024-09-21T03:37:01.061 INFO:journalctl@ceph.mds.a.smithi089.stdout:Sep 21 03:37:00 smithi089 bash[71899]: a9b21a569206b02a008e784f6e1fc614a9001b8fc489f59d33710c72db5018a0 2024-09-21T03:37:01.061 INFO:journalctl@ceph.mds.a.smithi089.stdout:Sep 21 03:37:00 smithi089 systemd[1]: Started Ceph mds.a for 7c374098-77c9-11ef-baf5-efdc52797490. 2024-09-21T03:37:01.279 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/7c374098-77c9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:37:01.663 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:37:01 smithi110 ceph-mon[34000]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:37:01.664 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:37:01 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:37:01.664 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:37:01 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:37:01.664 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:37:01 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:37:01.664 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:37:01 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T03:37:01.664 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:37:01 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T03:37:01.664 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:37:01 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:37:01.673 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:37:01 smithi089 ceph-mon[30928]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:37:01.673 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:37:01 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:37:01.673 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:37:01 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:37:01.673 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:37:01 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:37:01.673 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:37:01 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T03:37:01.673 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:37:01 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T03:37:01.673 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:37:01 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:37:01.788 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:01.762+0000 7f83b94ab640 1 -- 172.21.15.89:0/1940262399 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f83b4102a80 msgr2=0x7f83b4102e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:01.788 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:01.762+0000 7f83b94ab640 1 --2- 172.21.15.89:0/1940262399 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f83b4102a80 0x7f83b4102e90 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f83a4009900 tx=0x7f83a402a410 comp rx=0 tx=0).stop 2024-09-21T03:37:01.788 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:01.762+0000 7f83b94ab640 1 -- 172.21.15.89:0/1940262399 shutdown_connections 2024-09-21T03:37:01.788 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:01.763+0000 7f83b94ab640 1 --2- 172.21.15.89:0/1940262399 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f83b41083c0 0x7f83b41087d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:01.788 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:01.763+0000 7f83b94ab640 1 --2- 172.21.15.89:0/1940262399 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f83b4108d10 0x7f83b410f540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:01.788 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:01.763+0000 7f83b94ab640 1 --2- 172.21.15.89:0/1940262399 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f83b4102a80 0x7f83b4102e90 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:01.789 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:01.763+0000 7f83b94ab640 1 -- 172.21.15.89:0/1940262399 >> 172.21.15.89:0/1940262399 conn(0x7f83b4079740 msgr2=0x7f83b4079b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:37:01.789 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:01.763+0000 7f83b94ab640 1 -- 172.21.15.89:0/1940262399 shutdown_connections 2024-09-21T03:37:01.789 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:01.763+0000 7f83b94ab640 1 -- 172.21.15.89:0/1940262399 wait complete. 2024-09-21T03:37:01.789 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:01.763+0000 7f83b94ab640 1 Processor -- start 2024-09-21T03:37:01.789 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:01.763+0000 7f83b94ab640 1 -- start start 2024-09-21T03:37:01.789 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:01.764+0000 7f83b94ab640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f83b4102a80 0x7f83b419b1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:01.789 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:01.764+0000 7f83b94ab640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f83b41083c0 0x7f83b419b720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:01.789 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:01.764+0000 7f83b94ab640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f83b4108d10 0x7f83b419bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:01.789 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:01.764+0000 7f83b94ab640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f83b4112830 con 0x7f83b4102a80 2024-09-21T03:37:01.789 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:01.764+0000 7f83b94ab640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f83b41126b0 con 0x7f83b4108d10 2024-09-21T03:37:01.789 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:01.764+0000 7f83b94ab640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f83b41129b0 con 0x7f83b41083c0 2024-09-21T03:37:01.789 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:01.764+0000 7f83b2ffd640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f83b4102a80 0x7f83b419b1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:01.789 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:01.764+0000 7f83b2ffd640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f83b4102a80 0x7f83b419b1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:53936/0 (socket says 172.21.15.89:53936) 2024-09-21T03:37:01.789 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:01.764+0000 7f83b2ffd640 1 -- 172.21.15.89:0/1777353196 learned_addr learned my addr 172.21.15.89:0/1777353196 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-21T03:37:01.789 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:01.764+0000 7f83b37fe640 1 --2- 172.21.15.89:0/1777353196 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f83b4108d10 0x7f83b419bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:01.790 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:01.764+0000 7f83aa7fc640 1 --2- 172.21.15.89:0/1777353196 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f83b41083c0 0x7f83b419b720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:01.790 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:01.764+0000 7f83b2ffd640 1 -- 172.21.15.89:0/1777353196 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f83b41083c0 msgr2=0x7f83b419b720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:01.790 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:01.764+0000 7f83b2ffd640 1 --2- 172.21.15.89:0/1777353196 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f83b41083c0 0x7f83b419b720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:01.790 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:01.764+0000 7f83b2ffd640 1 -- 172.21.15.89:0/1777353196 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f83b4108d10 msgr2=0x7f83b419bc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:01.790 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:01.764+0000 7f83b2ffd640 1 --2- 172.21.15.89:0/1777353196 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f83b4108d10 0x7f83b419bc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:01.790 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:01.764+0000 7f83b2ffd640 1 -- 172.21.15.89:0/1777353196 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83b41a6470 con 0x7f83b4102a80 2024-09-21T03:37:01.790 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:01.764+0000 7f83b2ffd640 1 --2- 172.21.15.89:0/1777353196 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f83b4102a80 0x7f83b419b1e0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f83a4009230 tx=0x7f83a4004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:37:01.790 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:01.764+0000 7f83b0ff9640 1 -- 172.21.15.89:0/1777353196 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83a403f030 con 0x7f83b4102a80 2024-09-21T03:37:01.790 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:01.764+0000 7f83b94ab640 1 -- 172.21.15.89:0/1777353196 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f83b41a6700 con 0x7f83b4102a80 2024-09-21T03:37:01.790 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:01.764+0000 7f83b0ff9640 1 -- 172.21.15.89:0/1777353196 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f83a4003a80 con 0x7f83b4102a80 2024-09-21T03:37:01.790 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:01.765+0000 7f83b0ff9640 1 -- 172.21.15.89:0/1777353196 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83a4035070 con 0x7f83b4102a80 2024-09-21T03:37:01.790 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:01.765+0000 7f83b94ab640 1 -- 172.21.15.89:0/1777353196 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83b41a6be0 con 0x7f83b4102a80 2024-09-21T03:37:01.790 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:01.765+0000 7f83b94ab640 1 -- 172.21.15.89:0/1777353196 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f83b4103d00 con 0x7f83b4102a80 2024-09-21T03:37:01.790 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:01.768+0000 7f83b0ff9640 1 -- 172.21.15.89:0/1777353196 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 17) ==== 154512+0+0 (secure 0 0 0) 0x7f83a4042030 con 0x7f83b4102a80 2024-09-21T03:37:01.791 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:01.769+0000 7f83b0ff9640 1 --2- 172.21.15.89:0/1777353196 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f83a00b6540 0x7f83a00b8a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:01.791 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:01.769+0000 7f83aa7fc640 1 --2- 172.21.15.89:0/1777353196 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f83a00b6540 0x7f83a00b8a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:01.791 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:01.769+0000 7f83b0ff9640 1 -- 172.21.15.89:0/1777353196 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 7127+0+0 (secure 0 0 0) 0x7f83a41078d0 con 0x7f83b4102a80 2024-09-21T03:37:01.791 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:01.769+0000 7f83aa7fc640 1 --2- 172.21.15.89:0/1777353196 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f83a00b6540 0x7f83a00b8a50 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f83b4079e40 tx=0x7f839c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:37:01.791 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:01.772+0000 7f83b0ff9640 1 -- 172.21.15.89:0/1777353196 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f83a40d29e0 con 0x7f83b4102a80 2024-09-21T03:37:01.960 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:37:01 smithi114 ceph-mon[34140]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:37:01.961 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:37:01 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:37:01.961 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:37:01 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:37:01.961 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:37:01 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:37:01.961 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:37:01 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T03:37:01.961 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:37:01 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T03:37:01.961 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:37:01 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:37:01.987 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:01.985+0000 7f83b94ab640 1 -- 172.21.15.89:0/1777353196 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f83b4102e90 con 0x7f83b4102a80 2024-09-21T03:37:01.988 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:01.986+0000 7f83b0ff9640 1 -- 172.21.15.89:0/1777353196 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) ==== 94+0+9695 (secure 0 0 0) 0x7f83a40d7920 con 0x7f83b4102a80 2024-09-21T03:37:01.988 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-09-21T03:37:01.989 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":12,"btime":"2024-09-21T03:37:01:538705+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14721,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6835/2341538514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":2341538514},{"type":"v1","addr":"172.21.15.89:6835","nonce":2341538514}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14733,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6837/146708592","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6836","nonce":146708592},{"type":"v1","addr":"172.21.15.89:6837","nonce":146708592}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14781,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6839/2269965499","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6838","nonce":2269965499},{"type":"v1","addr":"172.21.15.89:6839","nonce":2269965499}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14793,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6841/650850383","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6840","nonce":650850383},{"type":"v1","addr":"172.21.15.89:6841","nonce":650850383}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24565,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6833/752336912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6832","nonce":752336912},{"type":"v1","addr":"172.21.15.114:6833","nonce":752336912}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24575,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6833/127607331","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6832","nonce":127607331},{"type":"v1","addr":"172.21.15.110:6833","nonce":127607331}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24593,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6835/671820654","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6834","nonce":671820654},{"type":"v1","addr":"172.21.15.110:6835","nonce":671820654}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24604,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6835/1049714548","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6834","nonce":1049714548},{"type":"v1","addr":"172.21.15.114:6835","nonce":1049714548}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24610,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6837/1941213818","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6836","nonce":1941213818},{"type":"v1","addr":"172.21.15.114:6837","nonce":1941213818}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24614,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6837/4047124589","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6836","nonce":4047124589},{"type":"v1","addr":"172.21.15.110:6837","nonce":4047124589}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24631,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6839/3427369944","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6838","nonce":3427369944},{"type":"v1","addr":"172.21.15.114:6839","nonce":3427369944}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-09-21T03:37:01.989 INFO:teuthology.orchestra.run.smithi089.stderr:dumped fsmap epoch 12 2024-09-21T03:37:01.992 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:01.989+0000 7f83b94ab640 1 -- 172.21.15.89:0/1777353196 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f83a00b6540 msgr2=0x7f83a00b8a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:01.992 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:01.989+0000 7f83b94ab640 1 --2- 172.21.15.89:0/1777353196 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f83a00b6540 0x7f83a00b8a50 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f83b4079e40 tx=0x7f839c005f90 comp rx=0 tx=0).stop 2024-09-21T03:37:01.992 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:01.989+0000 7f83b94ab640 1 -- 172.21.15.89:0/1777353196 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f83b4102a80 msgr2=0x7f83b419b1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:01.992 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:01.989+0000 7f83b94ab640 1 --2- 172.21.15.89:0/1777353196 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f83b4102a80 0x7f83b419b1e0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f83a4009230 tx=0x7f83a4004270 comp rx=0 tx=0).stop 2024-09-21T03:37:01.992 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:01.989+0000 7f83b94ab640 1 -- 172.21.15.89:0/1777353196 shutdown_connections 2024-09-21T03:37:01.992 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:01.989+0000 7f83b94ab640 1 --2- 172.21.15.89:0/1777353196 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f83a00b6540 0x7f83a00b8a50 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:01.992 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:01.989+0000 7f83b94ab640 1 --2- 172.21.15.89:0/1777353196 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f83b4108d10 0x7f83b419bc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:01.992 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:01.989+0000 7f83b94ab640 1 --2- 172.21.15.89:0/1777353196 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f83b41083c0 0x7f83b419b720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:01.992 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:01.989+0000 7f83b94ab640 1 --2- 172.21.15.89:0/1777353196 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f83b4102a80 0x7f83b419b1e0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:01.992 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:01.989+0000 7f83b94ab640 1 -- 172.21.15.89:0/1777353196 >> 172.21.15.89:0/1777353196 conn(0x7f83b4079740 msgr2=0x7f83b4101620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:37:01.992 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:01.989+0000 7f83b94ab640 1 -- 172.21.15.89:0/1777353196 shutdown_connections 2024-09-21T03:37:01.993 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:01.989+0000 7f83b94ab640 1 -- 172.21.15.89:0/1777353196 wait complete. 2024-09-21T03:37:02.620 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:37:02 smithi089 ceph-mon[30928]: Deploying daemon mds.b on smithi110 2024-09-21T03:37:02.620 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:37:02 smithi089 ceph-mon[30928]: mds.? [v2:172.21.15.89:6840/650850383,v1:172.21.15.89:6841/650850383] up:boot 2024-09-21T03:37:02.620 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:37:02 smithi089 ceph-mon[30928]: fsmap 11 up:standby 2024-09-21T03:37:02.620 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:37:02 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "a"} : dispatch 2024-09-21T03:37:02.621 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:37:02 smithi089 ceph-mon[30928]: from='client.? 172.21.15.89:0/1777353196' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:37:02.681 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:37:02 smithi110 ceph-mon[34000]: Deploying daemon mds.b on smithi110 2024-09-21T03:37:02.682 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:37:02 smithi110 ceph-mon[34000]: mds.? [v2:172.21.15.89:6840/650850383,v1:172.21.15.89:6841/650850383] up:boot 2024-09-21T03:37:02.682 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:37:02 smithi110 ceph-mon[34000]: fsmap 11 up:standby 2024-09-21T03:37:02.682 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:37:02 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "a"} : dispatch 2024-09-21T03:37:02.682 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:37:02 smithi110 ceph-mon[34000]: from='client.? 172.21.15.89:0/1777353196' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:37:02.961 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:37:02 smithi114 ceph-mon[34140]: Deploying daemon mds.b on smithi110 2024-09-21T03:37:02.961 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:37:02 smithi114 ceph-mon[34140]: mds.? [v2:172.21.15.89:6840/650850383,v1:172.21.15.89:6841/650850383] up:boot 2024-09-21T03:37:02.961 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:37:02 smithi114 ceph-mon[34140]: fsmap 11 up:standby 2024-09-21T03:37:02.961 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:37:02 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "a"} : dispatch 2024-09-21T03:37:02.961 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:37:02 smithi114 ceph-mon[34140]: from='client.? 172.21.15.89:0/1777353196' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:37:03.000 INFO:journalctl@ceph.mds.b.smithi110.stdout:Sep 21 03:37:02 smithi110 systemd[1]: Starting Ceph mds.b for 7c374098-77c9-11ef-baf5-efdc52797490... 2024-09-21T03:37:03.350 INFO:journalctl@ceph.mds.b.smithi110.stdout:Sep 21 03:37:03 smithi110 podman[62075]: 2024-09-21 03:37:02.990576265 +0000 UTC m=+0.015795716 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T03:37:03.350 INFO:journalctl@ceph.mds.b.smithi110.stdout:Sep 21 03:37:03 smithi110 podman[62075]: 2024-09-21 03:37:03.103595839 +0000 UTC m=+0.128815287 container create 469eb07a1cfbb185c9852eb5cbec18f7b08cd2235178d172647d0ccdd7dc8616 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-7c374098-77c9-11ef-baf5-efdc52797490-mds-b, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.build-date=20240731) 2024-09-21T03:37:03.350 INFO:journalctl@ceph.mds.b.smithi110.stdout:Sep 21 03:37:03 smithi110 podman[62075]: 2024-09-21 03:37:03.348251895 +0000 UTC m=+0.373471340 container init 469eb07a1cfbb185c9852eb5cbec18f7b08cd2235178d172647d0ccdd7dc8616 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-7c374098-77c9-11ef-baf5-efdc52797490-mds-b, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, ceph=True) 2024-09-21T03:37:03.750 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:37:03 smithi110 ceph-mon[34000]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:37:03.750 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:37:03 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:37:03.750 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:37:03 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:37:03.750 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:37:03 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:37:03.750 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:37:03 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:37:03.750 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:37:03 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:37:03.750 INFO:journalctl@ceph.mds.b.smithi110.stdout:Sep 21 03:37:03 smithi110 podman[62075]: 2024-09-21 03:37:03.351257209 +0000 UTC m=+0.376476662 container start 469eb07a1cfbb185c9852eb5cbec18f7b08cd2235178d172647d0ccdd7dc8616 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-7c374098-77c9-11ef-baf5-efdc52797490-mds-b, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.build-date=20240731) 2024-09-21T03:37:03.751 INFO:journalctl@ceph.mds.b.smithi110.stdout:Sep 21 03:37:03 smithi110 ceph-mds[62094]: -- 172.21.15.110:0/678206133 <== mon.2 v2:172.21.15.110:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55f229761860 con 0x55f22d909a80 2024-09-21T03:37:03.751 INFO:journalctl@ceph.mds.b.smithi110.stdout:Sep 21 03:37:03 smithi110 ceph-7c374098-77c9-11ef-baf5-efdc52797490-mds-b[62090]: starting mds.b at 2024-09-21T03:37:03.751 INFO:journalctl@ceph.mds.b.smithi110.stdout:Sep 21 03:37:03 smithi110 bash[62075]: 469eb07a1cfbb185c9852eb5cbec18f7b08cd2235178d172647d0ccdd7dc8616 2024-09-21T03:37:03.751 INFO:journalctl@ceph.mds.b.smithi110.stdout:Sep 21 03:37:03 smithi110 systemd[1]: Started Ceph mds.b for 7c374098-77c9-11ef-baf5-efdc52797490. 2024-09-21T03:37:03.811 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:37:03 smithi089 ceph-mon[30928]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:37:03.811 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:37:03 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:37:03.811 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:37:03 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:37:03.811 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:37:03 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:37:03.811 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:37:03 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:37:03.811 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:37:03 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:37:03.861 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:37:03 smithi114 ceph-mon[34140]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:37:03.861 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:37:03 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:37:03.862 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:37:03 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:37:03.862 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:37:03 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:37:03.862 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:37:03 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:37:03.862 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:37:03 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:37:04.648 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 7c374098-77c9-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T03:37:04.811 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:37:04 smithi089 ceph-mon[30928]: mds.? [v2:172.21.15.110:6838/132366709,v1:172.21.15.110:6839/132366709] up:boot 2024-09-21T03:37:04.811 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:37:04 smithi089 ceph-mon[30928]: fsmap 12 up:standby 2024-09-21T03:37:04.811 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:37:04 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "b"} : dispatch 2024-09-21T03:37:04.812 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:37:04 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:37:04.874 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:37:04 smithi114 ceph-mon[34140]: mds.? [v2:172.21.15.110:6838/132366709,v1:172.21.15.110:6839/132366709] up:boot 2024-09-21T03:37:04.874 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:37:04 smithi114 ceph-mon[34140]: fsmap 12 up:standby 2024-09-21T03:37:04.874 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:37:04 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "b"} : dispatch 2024-09-21T03:37:04.874 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:37:04 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:37:04.907 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/7c374098-77c9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:37:04.915 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:37:04 smithi110 ceph-mon[34000]: mds.? [v2:172.21.15.110:6838/132366709,v1:172.21.15.110:6839/132366709] up:boot 2024-09-21T03:37:04.915 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:37:04 smithi110 ceph-mon[34000]: fsmap 12 up:standby 2024-09-21T03:37:04.915 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:37:04 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "b"} : dispatch 2024-09-21T03:37:04.915 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:37:04 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:37:05.398 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:05.392+0000 7fe708b55640 1 -- 172.21.15.89:0/890123656 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe704102ae0 msgr2=0x7fe704102ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:05.398 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:05.392+0000 7fe708b55640 1 --2- 172.21.15.89:0/890123656 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe704102ae0 0x7fe704102ef0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fe6f4009900 tx=0x7fe6f402a410 comp rx=0 tx=0).stop 2024-09-21T03:37:05.398 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:05.392+0000 7fe708b55640 1 -- 172.21.15.89:0/890123656 shutdown_connections 2024-09-21T03:37:05.398 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:05.392+0000 7fe708b55640 1 --2- 172.21.15.89:0/890123656 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe704108d70 0x7fe70410f5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:05.398 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:05.392+0000 7fe708b55640 1 --2- 172.21.15.89:0/890123656 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe704108420 0x7fe704108830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:05.398 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:05.392+0000 7fe708b55640 1 --2- 172.21.15.89:0/890123656 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe704102ae0 0x7fe704102ef0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:05.399 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:05.392+0000 7fe708b55640 1 -- 172.21.15.89:0/890123656 >> 172.21.15.89:0/890123656 conn(0x7fe704079770 msgr2=0x7fe704079b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:37:05.399 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:05.392+0000 7fe708b55640 1 -- 172.21.15.89:0/890123656 shutdown_connections 2024-09-21T03:37:05.399 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:05.392+0000 7fe708b55640 1 -- 172.21.15.89:0/890123656 wait complete. 2024-09-21T03:37:05.399 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:05.393+0000 7fe708b55640 1 Processor -- start 2024-09-21T03:37:05.399 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:05.393+0000 7fe708b55640 1 -- start start 2024-09-21T03:37:05.399 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:05.393+0000 7fe708b55640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe704102ae0 0x7fe70419a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:05.399 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:05.393+0000 7fe708b55640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe704108420 0x7fe70419aa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:05.399 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:05.393+0000 7fe708b55640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe704108d70 0x7fe70419fcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:05.399 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:05.393+0000 7fe708b55640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fe704113a00 con 0x7fe704102ae0 2024-09-21T03:37:05.399 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:05.393+0000 7fe708b55640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fe704113880 con 0x7fe704108d70 2024-09-21T03:37:05.399 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:05.393+0000 7fe708b55640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fe704113b80 con 0x7fe704108420 2024-09-21T03:37:05.399 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:05.393+0000 7fe70259c640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe704102ae0 0x7fe70419a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:05.399 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:05.393+0000 7fe70259c640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe704102ae0 0x7fe70419a4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:53966/0 (socket says 172.21.15.89:53966) 2024-09-21T03:37:05.399 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:05.393+0000 7fe70259c640 1 -- 172.21.15.89:0/1328186336 learned_addr learned my addr 172.21.15.89:0/1328186336 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-21T03:37:05.400 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:05.393+0000 7fe702d9d640 1 --2- 172.21.15.89:0/1328186336 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe704108d70 0x7fe70419fcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:05.400 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:05.393+0000 7fe701d9b640 1 --2- 172.21.15.89:0/1328186336 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe704108420 0x7fe70419aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:05.400 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:05.393+0000 7fe70259c640 1 -- 172.21.15.89:0/1328186336 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe704108420 msgr2=0x7fe70419aa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:05.400 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:05.393+0000 7fe70259c640 1 --2- 172.21.15.89:0/1328186336 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe704108420 0x7fe70419aa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:05.400 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:05.393+0000 7fe70259c640 1 -- 172.21.15.89:0/1328186336 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe704108d70 msgr2=0x7fe70419fcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:05.400 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:05.393+0000 7fe70259c640 1 --2- 172.21.15.89:0/1328186336 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe704108d70 0x7fe70419fcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:05.400 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:05.393+0000 7fe70259c640 1 -- 172.21.15.89:0/1328186336 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7041a0420 con 0x7fe704102ae0 2024-09-21T03:37:05.400 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:05.394+0000 7fe70259c640 1 --2- 172.21.15.89:0/1328186336 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe704102ae0 0x7fe70419a4b0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fe6f4009230 tx=0x7fe6f4004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:37:05.400 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:05.394+0000 7fe6f37fe640 1 -- 172.21.15.89:0/1328186336 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6f403f030 con 0x7fe704102ae0 2024-09-21T03:37:05.400 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:05.394+0000 7fe6f37fe640 1 -- 172.21.15.89:0/1328186336 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe6f4003a80 con 0x7fe704102ae0 2024-09-21T03:37:05.400 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:05.394+0000 7fe708b55640 1 -- 172.21.15.89:0/1328186336 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe7041a06b0 con 0x7fe704102ae0 2024-09-21T03:37:05.400 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:05.394+0000 7fe6f37fe640 1 -- 172.21.15.89:0/1328186336 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6f4035070 con 0x7fe704102ae0 2024-09-21T03:37:05.400 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:05.394+0000 7fe708b55640 1 -- 172.21.15.89:0/1328186336 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7041a6e70 con 0x7fe704102ae0 2024-09-21T03:37:05.400 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:05.395+0000 7fe708b55640 1 -- 172.21.15.89:0/1328186336 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe704103d60 con 0x7fe704102ae0 2024-09-21T03:37:05.401 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:05.397+0000 7fe6f37fe640 1 -- 172.21.15.89:0/1328186336 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 17) ==== 154512+0+0 (secure 0 0 0) 0x7fe6f4042030 con 0x7fe704102ae0 2024-09-21T03:37:05.401 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:05.397+0000 7fe6f37fe640 1 --2- 172.21.15.89:0/1328186336 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7fe6ec0b6510 0x7fe6ec0b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:05.401 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:05.397+0000 7fe701d9b640 1 --2- 172.21.15.89:0/1328186336 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7fe6ec0b6510 0x7fe6ec0b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:05.401 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:05.397+0000 7fe6f37fe640 1 -- 172.21.15.89:0/1328186336 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 7127+0+0 (secure 0 0 0) 0x7fe6f41078e0 con 0x7fe704102ae0 2024-09-21T03:37:05.401 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:05.398+0000 7fe701d9b640 1 --2- 172.21.15.89:0/1328186336 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7fe6ec0b6510 0x7fe6ec0b8a20 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fe7041053d0 tx=0x7fe6f800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:37:05.403 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:05.401+0000 7fe6f37fe640 1 -- 172.21.15.89:0/1328186336 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fe6f40d29f0 con 0x7fe704102ae0 2024-09-21T03:37:05.658 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:05.656+0000 7fe708b55640 1 -- 172.21.15.89:0/1328186336 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe704102ef0 con 0x7fe704102ae0 2024-09-21T03:37:05.658 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:05.656+0000 7fe6f37fe640 1 -- 172.21.15.89:0/1328186336 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) ==== 94+0+10521 (secure 0 0 0) 0x7fe6f40d7930 con 0x7fe704102ae0 2024-09-21T03:37:05.659 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-09-21T03:37:05.659 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":13,"btime":"2024-09-21T03:37:03:550655+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14721,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6835/2341538514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":2341538514},{"type":"v1","addr":"172.21.15.89:6835","nonce":2341538514}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14733,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6837/146708592","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6836","nonce":146708592},{"type":"v1","addr":"172.21.15.89:6837","nonce":146708592}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14781,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6839/2269965499","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6838","nonce":2269965499},{"type":"v1","addr":"172.21.15.89:6839","nonce":2269965499}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14793,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6841/650850383","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6840","nonce":650850383},{"type":"v1","addr":"172.21.15.89:6841","nonce":650850383}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24565,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6833/752336912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6832","nonce":752336912},{"type":"v1","addr":"172.21.15.114:6833","nonce":752336912}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24575,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6833/127607331","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6832","nonce":127607331},{"type":"v1","addr":"172.21.15.110:6833","nonce":127607331}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24593,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6835/671820654","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6834","nonce":671820654},{"type":"v1","addr":"172.21.15.110:6835","nonce":671820654}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24604,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6835/1049714548","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6834","nonce":1049714548},{"type":"v1","addr":"172.21.15.114:6835","nonce":1049714548}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24610,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6837/1941213818","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6836","nonce":1941213818},{"type":"v1","addr":"172.21.15.114:6837","nonce":1941213818}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24614,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6837/4047124589","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6836","nonce":4047124589},{"type":"v1","addr":"172.21.15.110:6837","nonce":4047124589}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24631,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6839/3427369944","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6838","nonce":3427369944},{"type":"v1","addr":"172.21.15.114:6839","nonce":3427369944}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24638,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6839/132366709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6838","nonce":132366709},{"type":"v1","addr":"172.21.15.110:6839","nonce":132366709}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[]} 2024-09-21T03:37:05.659 INFO:teuthology.orchestra.run.smithi089.stderr:dumped fsmap epoch 13 2024-09-21T03:37:05.661 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:05.658+0000 7fe708b55640 1 -- 172.21.15.89:0/1328186336 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7fe6ec0b6510 msgr2=0x7fe6ec0b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:05.661 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:05.659+0000 7fe708b55640 1 --2- 172.21.15.89:0/1328186336 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7fe6ec0b6510 0x7fe6ec0b8a20 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fe7041053d0 tx=0x7fe6f800b410 comp rx=0 tx=0).stop 2024-09-21T03:37:05.661 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:05.659+0000 7fe708b55640 1 -- 172.21.15.89:0/1328186336 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe704102ae0 msgr2=0x7fe70419a4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:05.661 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:05.659+0000 7fe708b55640 1 --2- 172.21.15.89:0/1328186336 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe704102ae0 0x7fe70419a4b0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fe6f4009230 tx=0x7fe6f4004270 comp rx=0 tx=0).stop 2024-09-21T03:37:05.661 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:05.659+0000 7fe708b55640 1 -- 172.21.15.89:0/1328186336 shutdown_connections 2024-09-21T03:37:05.661 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:05.659+0000 7fe708b55640 1 --2- 172.21.15.89:0/1328186336 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7fe6ec0b6510 0x7fe6ec0b8a20 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:05.661 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:05.659+0000 7fe708b55640 1 --2- 172.21.15.89:0/1328186336 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe704108d70 0x7fe70419fcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:05.661 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:05.659+0000 7fe708b55640 1 --2- 172.21.15.89:0/1328186336 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe704108420 0x7fe70419aa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:05.661 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:05.659+0000 7fe708b55640 1 --2- 172.21.15.89:0/1328186336 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe704102ae0 0x7fe70419a4b0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:05.661 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:05.659+0000 7fe708b55640 1 -- 172.21.15.89:0/1328186336 >> 172.21.15.89:0/1328186336 conn(0x7fe704079770 msgr2=0x7fe704106610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:37:05.661 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:05.659+0000 7fe708b55640 1 -- 172.21.15.89:0/1328186336 shutdown_connections 2024-09-21T03:37:05.662 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:05.659+0000 7fe708b55640 1 -- 172.21.15.89:0/1328186336 wait complete. 2024-09-21T03:37:05.872 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:37:05 smithi110 ceph-mon[34000]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:37:05.872 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:37:05 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:37:05.872 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:37:05 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:37:05.872 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:37:05 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:37:05.873 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:37:05 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:37:05.952 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:37:05 smithi089 ceph-mon[30928]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:37:05.953 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:37:05 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:37:05.953 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:37:05 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:37:05.953 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:37:05 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:37:05.953 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:37:05 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:37:05.960 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:37:05 smithi114 ceph-mon[34140]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:37:05.960 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:37:05 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:37:05.961 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:37:05 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:37:05.961 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:37:05 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:37:05.961 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:37:05 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:37:06.319 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 7c374098-77c9-11ef-baf5-efdc52797490 -- ceph --format=json-pretty osd lspools 2024-09-21T03:37:06.881 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:37:06 smithi114 ceph-mon[34140]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:37:06.881 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:37:06 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:37:06.881 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:37:06 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:37:06.881 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:37:06 smithi114 ceph-mon[34140]: from='client.? 172.21.15.89:0/1328186336' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:37:06.881 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:37:06 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:37:06.881 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:37:06 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:37:06.881 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:37:06 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:37:06.881 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:37:06 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:37:06.884 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:37:06 smithi089 ceph-mon[30928]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:37:06.884 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:37:06 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:37:06.884 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:37:06 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:37:06.884 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:37:06 smithi089 ceph-mon[30928]: from='client.? 172.21.15.89:0/1328186336' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:37:06.884 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:37:06 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:37:06.884 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:37:06 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:37:06.884 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:37:06 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:37:06.884 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:37:06 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:37:07.000 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:37:06 smithi110 ceph-mon[34000]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:37:07.000 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:37:06 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:37:07.000 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:37:06 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:37:07.000 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:37:06 smithi110 ceph-mon[34000]: from='client.? 172.21.15.89:0/1328186336' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:37:07.000 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:37:06 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:37:07.000 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:37:06 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:37:07.000 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:37:06 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:37:07.000 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:37:06 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:37:07.105 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/7c374098-77c9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:37:07.615 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:07.602+0000 7fbdb0139640 1 -- 172.21.15.89:0/183396272 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbda8107c10 msgr2=0x7fbda81080f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:07.615 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:07.602+0000 7fbdb0139640 1 --2- 172.21.15.89:0/183396272 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbda8107c10 0x7fbda81080f0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fbda0009cd0 tx=0x7fbda002a8e0 comp rx=0 tx=0).stop 2024-09-21T03:37:07.616 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:07.603+0000 7fbdb0139640 1 -- 172.21.15.89:0/183396272 shutdown_connections 2024-09-21T03:37:07.616 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:07.603+0000 7fbdb0139640 1 --2- 172.21.15.89:0/183396272 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbda8103120 0x7fbda8107650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:07.616 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:07.603+0000 7fbdb0139640 1 --2- 172.21.15.89:0/183396272 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbda8107c10 0x7fbda81080f0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:07.616 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:07.603+0000 7fbdb0139640 1 --2- 172.21.15.89:0/183396272 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbda81027d0 0x7fbda8102be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:07.616 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:07.603+0000 7fbdb0139640 1 -- 172.21.15.89:0/183396272 >> 172.21.15.89:0/183396272 conn(0x7fbda8079750 msgr2=0x7fbda8079b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:37:07.616 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:07.603+0000 7fbdb0139640 1 -- 172.21.15.89:0/183396272 shutdown_connections 2024-09-21T03:37:07.616 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:07.603+0000 7fbdb0139640 1 -- 172.21.15.89:0/183396272 wait complete. 2024-09-21T03:37:07.616 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:07.603+0000 7fbdb0139640 1 Processor -- start 2024-09-21T03:37:07.616 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:07.603+0000 7fbdb0139640 1 -- start start 2024-09-21T03:37:07.616 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:07.604+0000 7fbdb0139640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbda81027d0 0x7fbda81a09f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:07.616 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:07.604+0000 7fbdb0139640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbda8103120 0x7fbda81a0f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:07.616 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:07.604+0000 7fbdb0139640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbda8107c10 0x7fbda81a6230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:07.616 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:07.604+0000 7fbdb0139640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fbda8119f80 con 0x7fbda8103120 2024-09-21T03:37:07.616 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:07.604+0000 7fbdb0139640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fbda8119e00 con 0x7fbda8107c10 2024-09-21T03:37:07.617 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:07.604+0000 7fbdb0139640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fbda811a100 con 0x7fbda81027d0 2024-09-21T03:37:07.617 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:07.604+0000 7fbdad6d4640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbda8103120 0x7fbda81a0f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:07.617 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:07.604+0000 7fbdad6d4640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbda8103120 0x7fbda81a0f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:53994/0 (socket says 172.21.15.89:53994) 2024-09-21T03:37:07.617 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:07.604+0000 7fbdad6d4640 1 -- 172.21.15.89:0/3040624988 learned_addr learned my addr 172.21.15.89:0/3040624988 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-21T03:37:07.617 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:07.604+0000 7fbdae6d6640 1 --2- 172.21.15.89:0/3040624988 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbda8107c10 0x7fbda81a6230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:07.617 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:07.604+0000 7fbdaded5640 1 --2- 172.21.15.89:0/3040624988 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbda81027d0 0x7fbda81a09f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:07.617 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:07.604+0000 7fbdad6d4640 1 -- 172.21.15.89:0/3040624988 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbda81027d0 msgr2=0x7fbda81a09f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:07.617 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:07.604+0000 7fbdad6d4640 1 --2- 172.21.15.89:0/3040624988 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbda81027d0 0x7fbda81a09f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:07.617 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:07.604+0000 7fbdad6d4640 1 -- 172.21.15.89:0/3040624988 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbda8107c10 msgr2=0x7fbda81a6230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:07.617 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:07.604+0000 7fbdad6d4640 1 --2- 172.21.15.89:0/3040624988 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbda8107c10 0x7fbda81a6230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:07.617 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:07.604+0000 7fbdad6d4640 1 -- 172.21.15.89:0/3040624988 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbda81a6960 con 0x7fbda8103120 2024-09-21T03:37:07.617 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:07.604+0000 7fbdad6d4640 1 --2- 172.21.15.89:0/3040624988 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbda8103120 0x7fbda81a0f90 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fbd9800c970 tx=0x7fbd9800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:37:07.617 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:07.604+0000 7fbd9effd640 1 -- 172.21.15.89:0/3040624988 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd98007bb0 con 0x7fbda8103120 2024-09-21T03:37:07.617 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:07.605+0000 7fbd9effd640 1 -- 172.21.15.89:0/3040624988 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fbd98007d50 con 0x7fbda8103120 2024-09-21T03:37:07.618 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:07.605+0000 7fbdb0139640 1 -- 172.21.15.89:0/3040624988 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbda81a6c50 con 0x7fbda8103120 2024-09-21T03:37:07.618 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:07.605+0000 7fbd9effd640 1 -- 172.21.15.89:0/3040624988 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd98004da0 con 0x7fbda8103120 2024-09-21T03:37:07.618 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:07.605+0000 7fbdb0139640 1 -- 172.21.15.89:0/3040624988 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbda81ad3b0 con 0x7fbda8103120 2024-09-21T03:37:07.618 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:07.606+0000 7fbdb0139640 1 -- 172.21.15.89:0/3040624988 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbda806c280 con 0x7fbda8103120 2024-09-21T03:37:07.618 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:07.607+0000 7fbd9effd640 1 -- 172.21.15.89:0/3040624988 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 17) ==== 154512+0+0 (secure 0 0 0) 0x7fbd98004f80 con 0x7fbda8103120 2024-09-21T03:37:07.618 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:07.608+0000 7fbd9effd640 1 --2- 172.21.15.89:0/3040624988 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7fbd940b6590 0x7fbd940b8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:07.618 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:07.608+0000 7fbdaded5640 1 --2- 172.21.15.89:0/3040624988 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7fbd940b6590 0x7fbd940b8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:07.618 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:07.608+0000 7fbd9effd640 1 -- 172.21.15.89:0/3040624988 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 7127+0+0 (secure 0 0 0) 0x7fbd980e6b70 con 0x7fbda8103120 2024-09-21T03:37:07.618 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:07.608+0000 7fbdaded5640 1 --2- 172.21.15.89:0/3040624988 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7fbd940b6590 0x7fbd940b8aa0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fbda4005fd0 tx=0x7fbda4005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:37:07.618 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:07.613+0000 7fbd9effd640 1 -- 172.21.15.89:0/3040624988 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fbd98014030 con 0x7fbda8103120 2024-09-21T03:37:07.817 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:07.815+0000 7fbdb0139640 1 -- 172.21.15.89:0/3040624988 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7fbda81a1b20 con 0x7fbda8103120 2024-09-21T03:37:07.818 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:07.815+0000 7fbd9effd640 1 -- 172.21.15.89:0/3040624988 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v69) ==== 84+0+65 (secure 0 0 0) 0x7fbd980d0020 con 0x7fbda8103120 2024-09-21T03:37:07.818 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-09-21T03:37:07.818 INFO:teuthology.orchestra.run.smithi089.stdout:[ 2024-09-21T03:37:07.818 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-21T03:37:07.818 INFO:teuthology.orchestra.run.smithi089.stdout: "poolnum": 1, 2024-09-21T03:37:07.818 INFO:teuthology.orchestra.run.smithi089.stdout: "poolname": ".mgr" 2024-09-21T03:37:07.818 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-21T03:37:07.818 INFO:teuthology.orchestra.run.smithi089.stdout:] 2024-09-21T03:37:07.820 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:07.818+0000 7fbdb0139640 1 -- 172.21.15.89:0/3040624988 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7fbd940b6590 msgr2=0x7fbd940b8aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:07.821 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:07.818+0000 7fbdb0139640 1 --2- 172.21.15.89:0/3040624988 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7fbd940b6590 0x7fbd940b8aa0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fbda4005fd0 tx=0x7fbda4005850 comp rx=0 tx=0).stop 2024-09-21T03:37:07.821 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:07.818+0000 7fbdb0139640 1 -- 172.21.15.89:0/3040624988 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbda8103120 msgr2=0x7fbda81a0f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:07.821 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:07.818+0000 7fbdb0139640 1 --2- 172.21.15.89:0/3040624988 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbda8103120 0x7fbda81a0f90 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fbd9800c970 tx=0x7fbd9800ce30 comp rx=0 tx=0).stop 2024-09-21T03:37:07.821 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:07.818+0000 7fbdb0139640 1 -- 172.21.15.89:0/3040624988 shutdown_connections 2024-09-21T03:37:07.821 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:07.818+0000 7fbdb0139640 1 --2- 172.21.15.89:0/3040624988 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7fbd940b6590 0x7fbd940b8aa0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:07.821 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:07.818+0000 7fbdb0139640 1 --2- 172.21.15.89:0/3040624988 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbda8107c10 0x7fbda81a6230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:07.821 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:07.818+0000 7fbdb0139640 1 --2- 172.21.15.89:0/3040624988 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbda8103120 0x7fbda81a0f90 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:07.821 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:07.818+0000 7fbdb0139640 1 --2- 172.21.15.89:0/3040624988 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbda81027d0 0x7fbda81a09f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:07.821 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:07.818+0000 7fbdb0139640 1 -- 172.21.15.89:0/3040624988 >> 172.21.15.89:0/3040624988 conn(0x7fbda8079750 msgr2=0x7fbda8105620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:37:07.821 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:07.818+0000 7fbdb0139640 1 -- 172.21.15.89:0/3040624988 shutdown_connections 2024-09-21T03:37:07.821 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:07.818+0000 7fbdb0139640 1 -- 172.21.15.89:0/3040624988 wait complete. 2024-09-21T03:37:08.062 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:37:07 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:37:08.062 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:37:07 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:37:08.062 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:37:07 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:37:08.062 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:37:07 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:37:08.062 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:37:07 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:37:08.210 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:37:07 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:37:08.211 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:37:07 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:37:08.211 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:37:07 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:37:08.211 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:37:07 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:37:08.211 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:37:07 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:37:08.250 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:37:07 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:37:08.250 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:37:07 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:37:08.250 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:37:07 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:37:08.250 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:37:07 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:37:08.250 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:37:07 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:37:08.500 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-21T03:37:08.500 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 7c374098-77c9-11ef-baf5-efdc52797490 -- ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-21T03:37:08.765 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/7c374098-77c9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:37:09.061 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:37:08 smithi089 ceph-mon[30928]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:37:09.061 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:37:08 smithi089 ceph-mon[30928]: from='client.? 172.21.15.89:0/3040624988' entity='client.admin' cmd={"prefix": "osd lspools", "format": "json-pretty"} : dispatch 2024-09-21T03:37:09.061 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:37:08 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:37:09.061 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:37:08 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:37:09.061 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:37:08 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:37:09.061 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:37:08 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:37:09.061 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:37:08 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:37:09.211 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:37:08 smithi114 ceph-mon[34140]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:37:09.211 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:37:08 smithi114 ceph-mon[34140]: from='client.? 172.21.15.89:0/3040624988' entity='client.admin' cmd={"prefix": "osd lspools", "format": "json-pretty"} : dispatch 2024-09-21T03:37:09.211 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:37:08 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:37:09.211 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:37:08 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:37:09.211 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:37:08 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:37:09.211 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:37:08 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:37:09.211 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:37:08 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:37:09.250 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:37:08 smithi110 ceph-mon[34000]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:37:09.250 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:37:08 smithi110 ceph-mon[34000]: from='client.? 172.21.15.89:0/3040624988' entity='client.admin' cmd={"prefix": "osd lspools", "format": "json-pretty"} : dispatch 2024-09-21T03:37:09.250 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:37:08 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:37:09.250 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:37:08 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:37:09.250 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:37:08 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:37:09.250 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:37:08 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:37:09.250 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:37:08 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:37:09.278 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:09.251+0000 7fd025613640 1 -- 172.21.15.89:0/366736023 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd020109e00 msgr2=0x7fd020110630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:09.278 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:09.251+0000 7fd025613640 1 --2- 172.21.15.89:0/366736023 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd020109e00 0x7fd020110630 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fd018009ca0 tx=0x7fd01802a8c0 comp rx=0 tx=0).stop 2024-09-21T03:37:09.278 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:09.252+0000 7fd025613640 1 -- 172.21.15.89:0/366736023 shutdown_connections 2024-09-21T03:37:09.278 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:09.252+0000 7fd025613640 1 --2- 172.21.15.89:0/366736023 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd020109e00 0x7fd020110630 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:09.278 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:09.252+0000 7fd025613640 1 --2- 172.21.15.89:0/366736023 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd0201094b0 0x7fd0201098c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:09.278 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:09.252+0000 7fd025613640 1 --2- 172.21.15.89:0/366736023 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd020103bc0 0x7fd020103fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:09.278 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:09.252+0000 7fd025613640 1 -- 172.21.15.89:0/366736023 >> 172.21.15.89:0/366736023 conn(0x7fd02007b910 msgr2=0x7fd02007bd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:37:09.278 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:09.252+0000 7fd025613640 1 -- 172.21.15.89:0/366736023 shutdown_connections 2024-09-21T03:37:09.278 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:09.252+0000 7fd025613640 1 -- 172.21.15.89:0/366736023 wait complete. 2024-09-21T03:37:09.278 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:09.252+0000 7fd025613640 1 Processor -- start 2024-09-21T03:37:09.278 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:09.252+0000 7fd025613640 1 -- start start 2024-09-21T03:37:09.279 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:09.253+0000 7fd025613640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd020103bc0 0x7fd02019a350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:09.279 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:09.253+0000 7fd025613640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd0201094b0 0x7fd02019a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:09.279 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:09.253+0000 7fd025613640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd020109e00 0x7fd02019fb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:09.279 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:09.253+0000 7fd025613640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fd0201139c0 con 0x7fd020103bc0 2024-09-21T03:37:09.279 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:09.253+0000 7fd025613640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fd020113840 con 0x7fd0201094b0 2024-09-21T03:37:09.279 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:09.253+0000 7fd025613640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fd020113b40 con 0x7fd020109e00 2024-09-21T03:37:09.279 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:09.253+0000 7fd01effd640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd020103bc0 0x7fd02019a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:09.279 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:09.253+0000 7fd01effd640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd020103bc0 0x7fd02019a350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:37816/0 (socket says 172.21.15.89:37816) 2024-09-21T03:37:09.279 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:09.253+0000 7fd01effd640 1 -- 172.21.15.89:0/2284567826 learned_addr learned my addr 172.21.15.89:0/2284567826 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-21T03:37:09.279 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:09.253+0000 7fd01e7fc640 1 --2- 172.21.15.89:0/2284567826 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd0201094b0 0x7fd02019a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:09.279 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:09.253+0000 7fd01effd640 1 -- 172.21.15.89:0/2284567826 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd020109e00 msgr2=0x7fd02019fb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:09.279 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:09.253+0000 7fd01effd640 1 --2- 172.21.15.89:0/2284567826 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd020109e00 0x7fd02019fb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:09.279 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:09.253+0000 7fd01effd640 1 -- 172.21.15.89:0/2284567826 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd0201094b0 msgr2=0x7fd02019a8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:09.279 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:09.253+0000 7fd01effd640 1 --2- 172.21.15.89:0/2284567826 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd0201094b0 0x7fd02019a8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:09.280 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:09.253+0000 7fd01effd640 1 -- 172.21.15.89:0/2284567826 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0201a02c0 con 0x7fd020103bc0 2024-09-21T03:37:09.280 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:09.253+0000 7fd01effd640 1 --2- 172.21.15.89:0/2284567826 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd020103bc0 0x7fd02019a350 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fd01000d950 tx=0x7fd01000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:37:09.280 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:09.253+0000 7fd007fff640 1 -- 172.21.15.89:0/2284567826 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd010017030 con 0x7fd020103bc0 2024-09-21T03:37:09.280 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:09.253+0000 7fd007fff640 1 -- 172.21.15.89:0/2284567826 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd010004020 con 0x7fd020103bc0 2024-09-21T03:37:09.280 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:09.253+0000 7fd025613640 1 -- 172.21.15.89:0/2284567826 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd0201a05b0 con 0x7fd020103bc0 2024-09-21T03:37:09.280 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:09.253+0000 7fd007fff640 1 -- 172.21.15.89:0/2284567826 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0100055d0 con 0x7fd020103bc0 2024-09-21T03:37:09.280 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:09.253+0000 7fd025613640 1 -- 172.21.15.89:0/2284567826 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0201a6d10 con 0x7fd020103bc0 2024-09-21T03:37:09.280 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:09.255+0000 7fd025613640 1 -- 172.21.15.89:0/2284567826 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd020103fd0 con 0x7fd020103bc0 2024-09-21T03:37:09.280 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:09.256+0000 7fd007fff640 1 -- 172.21.15.89:0/2284567826 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 17) ==== 154512+0+0 (secure 0 0 0) 0x7fd010009a30 con 0x7fd020103bc0 2024-09-21T03:37:09.280 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:09.257+0000 7fd007fff640 1 --2- 172.21.15.89:0/2284567826 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7fd0080b6510 0x7fd0080b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:09.280 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:09.257+0000 7fd01e7fc640 1 --2- 172.21.15.89:0/2284567826 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7fd0080b6510 0x7fd0080b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:09.280 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:09.257+0000 7fd007fff640 1 -- 172.21.15.89:0/2284567826 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 7127+0+0 (secure 0 0 0) 0x7fd0100e6fe0 con 0x7fd020103bc0 2024-09-21T03:37:09.280 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:09.257+0000 7fd01e7fc640 1 --2- 172.21.15.89:0/2284567826 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7fd0080b6510 0x7fd0080b8a20 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fd02007bf80 tx=0x7fd014005be0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:37:09.281 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:09.260+0000 7fd007fff640 1 -- 172.21.15.89:0/2284567826 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fd0100add10 con 0x7fd020103bc0 2024-09-21T03:37:09.440 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:09.438+0000 7fd025613640 1 -- 172.21.15.89:0/2284567826 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7fd02002e640 con 0x7fd020103bc0 2024-09-21T03:37:09.818 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:09.816+0000 7fd007fff640 1 -- 172.21.15.89:0/2284567826 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v70) ==== 138+0+0 (secure 0 0 0) 0x7fd0100b20f0 con 0x7fd020103bc0 2024-09-21T03:37:09.818 INFO:teuthology.orchestra.run.smithi089.stderr:pool 'cephfs_metadata' created 2024-09-21T03:37:09.820 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:09.818+0000 7fd025613640 1 -- 172.21.15.89:0/2284567826 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7fd0080b6510 msgr2=0x7fd0080b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:09.820 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:09.818+0000 7fd025613640 1 --2- 172.21.15.89:0/2284567826 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7fd0080b6510 0x7fd0080b8a20 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fd02007bf80 tx=0x7fd014005be0 comp rx=0 tx=0).stop 2024-09-21T03:37:09.820 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:09.818+0000 7fd025613640 1 -- 172.21.15.89:0/2284567826 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd020103bc0 msgr2=0x7fd02019a350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:09.820 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:09.818+0000 7fd025613640 1 --2- 172.21.15.89:0/2284567826 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd020103bc0 0x7fd02019a350 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fd01000d950 tx=0x7fd01000de10 comp rx=0 tx=0).stop 2024-09-21T03:37:09.821 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:09.819+0000 7fd025613640 1 -- 172.21.15.89:0/2284567826 shutdown_connections 2024-09-21T03:37:09.821 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:09.819+0000 7fd025613640 1 --2- 172.21.15.89:0/2284567826 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7fd0080b6510 0x7fd0080b8a20 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:09.821 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:09.819+0000 7fd025613640 1 --2- 172.21.15.89:0/2284567826 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd0201094b0 0x7fd02019a8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:09.821 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:09.819+0000 7fd025613640 1 --2- 172.21.15.89:0/2284567826 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd020109e00 0x7fd02019fb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:09.821 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:09.819+0000 7fd025613640 1 --2- 172.21.15.89:0/2284567826 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd020103bc0 0x7fd02019a350 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:09.821 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:09.819+0000 7fd025613640 1 -- 172.21.15.89:0/2284567826 >> 172.21.15.89:0/2284567826 conn(0x7fd02007b910 msgr2=0x7fd020107500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:37:09.822 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:09.820+0000 7fd025613640 1 -- 172.21.15.89:0/2284567826 shutdown_connections 2024-09-21T03:37:09.822 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:09.820+0000 7fd025613640 1 -- 172.21.15.89:0/2284567826 wait complete. 2024-09-21T03:37:10.199 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:37:09 smithi089 ceph-mon[30928]: from='client.? 172.21.15.89:0/2284567826' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} : dispatch 2024-09-21T03:37:10.211 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:37:09 smithi114 ceph-mon[34140]: from='client.? 172.21.15.89:0/2284567826' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} : dispatch 2024-09-21T03:37:10.250 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:37:09 smithi110 ceph-mon[34000]: from='client.? 172.21.15.89:0/2284567826' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} : dispatch 2024-09-21T03:37:10.524 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 7c374098-77c9-11ef-baf5-efdc52797490 -- ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.05 2024-09-21T03:37:10.773 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/7c374098-77c9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:37:11.179 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:37:10 smithi089 ceph-mon[30928]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:37:11.180 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:37:10 smithi089 ceph-mon[30928]: from='client.? 172.21.15.89:0/2284567826' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-09-21T03:37:11.180 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:37:10 smithi089 ceph-mon[30928]: osdmap e70: 12 total, 12 up, 12 in 2024-09-21T03:37:11.210 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:37:10 smithi114 ceph-mon[34140]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:37:11.211 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:37:10 smithi114 ceph-mon[34140]: from='client.? 172.21.15.89:0/2284567826' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-09-21T03:37:11.211 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:37:10 smithi114 ceph-mon[34140]: osdmap e70: 12 total, 12 up, 12 in 2024-09-21T03:37:11.250 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:37:10 smithi110 ceph-mon[34000]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:37:11.250 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:37:10 smithi110 ceph-mon[34000]: from='client.? 172.21.15.89:0/2284567826' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-09-21T03:37:11.250 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:37:10 smithi110 ceph-mon[34000]: osdmap e70: 12 total, 12 up, 12 in 2024-09-21T03:37:11.279 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:11.273+0000 7f3a012c6640 1 -- 172.21.15.89:0/496308217 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f39fc108cf0 msgr2=0x7f39fc10f520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:11.280 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:11.273+0000 7f3a012c6640 1 --2- 172.21.15.89:0/496308217 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f39fc108cf0 0x7f39fc10f520 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f39f4009ca0 tx=0x7f39f402a8c0 comp rx=0 tx=0).stop 2024-09-21T03:37:11.280 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:11.273+0000 7f3a012c6640 1 -- 172.21.15.89:0/496308217 shutdown_connections 2024-09-21T03:37:11.280 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:11.273+0000 7f3a012c6640 1 --2- 172.21.15.89:0/496308217 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f39fc1083a0 0x7f39fc1087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:11.280 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:11.273+0000 7f3a012c6640 1 --2- 172.21.15.89:0/496308217 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f39fc108cf0 0x7f39fc10f520 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:11.280 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:11.273+0000 7f3a012c6640 1 --2- 172.21.15.89:0/496308217 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f39fc102a60 0x7f39fc102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:11.280 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:11.273+0000 7f3a012c6640 1 -- 172.21.15.89:0/496308217 >> 172.21.15.89:0/496308217 conn(0x7f39fc079730 msgr2=0x7f39fc079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:37:11.280 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:11.273+0000 7f3a012c6640 1 -- 172.21.15.89:0/496308217 shutdown_connections 2024-09-21T03:37:11.280 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:11.273+0000 7f3a012c6640 1 -- 172.21.15.89:0/496308217 wait complete. 2024-09-21T03:37:11.280 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:11.274+0000 7f3a012c6640 1 Processor -- start 2024-09-21T03:37:11.280 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:11.274+0000 7f3a012c6640 1 -- start start 2024-09-21T03:37:11.280 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:11.274+0000 7f3a012c6640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f39fc102a60 0x7f39fc198f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:11.280 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:11.274+0000 7f3a012c6640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f39fc1083a0 0x7f39fc199460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:11.280 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:11.275+0000 7f3a012c6640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f39fc108cf0 0x7f39fc1999a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:11.280 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:11.275+0000 7f3a012c6640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f39fc112860 con 0x7f39fc102a60 2024-09-21T03:37:11.281 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:11.275+0000 7f3a012c6640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f39fc1126e0 con 0x7f39fc1083a0 2024-09-21T03:37:11.281 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:11.275+0000 7f3a012c6640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f39fc1129e0 con 0x7f39fc108cf0 2024-09-21T03:37:11.281 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:11.275+0000 7f39faffd640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f39fc102a60 0x7f39fc198f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:11.281 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:11.275+0000 7f39faffd640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f39fc102a60 0x7f39fc198f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:37832/0 (socket says 172.21.15.89:37832) 2024-09-21T03:37:11.281 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:11.275+0000 7f39faffd640 1 -- 172.21.15.89:0/1461782538 learned_addr learned my addr 172.21.15.89:0/1461782538 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-21T03:37:11.281 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:11.275+0000 7f39fa7fc640 1 --2- 172.21.15.89:0/1461782538 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f39fc1083a0 0x7f39fc199460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:11.281 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:11.275+0000 7f39fb7fe640 1 --2- 172.21.15.89:0/1461782538 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f39fc108cf0 0x7f39fc1999a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:11.281 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:11.275+0000 7f39faffd640 1 -- 172.21.15.89:0/1461782538 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f39fc108cf0 msgr2=0x7f39fc1999a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:11.281 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:11.275+0000 7f39faffd640 1 --2- 172.21.15.89:0/1461782538 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f39fc108cf0 0x7f39fc1999a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:11.281 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:11.275+0000 7f39faffd640 1 -- 172.21.15.89:0/1461782538 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f39fc1083a0 msgr2=0x7f39fc199460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:11.281 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:11.275+0000 7f39faffd640 1 --2- 172.21.15.89:0/1461782538 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f39fc1083a0 0x7f39fc199460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:11.281 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:11.275+0000 7f39faffd640 1 -- 172.21.15.89:0/1461782538 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39fc1a85d0 con 0x7f39fc102a60 2024-09-21T03:37:11.281 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:11.275+0000 7f39faffd640 1 --2- 172.21.15.89:0/1461782538 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f39fc102a60 0x7f39fc198f20 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f39ec00d950 tx=0x7f39ec00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:37:11.281 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:11.275+0000 7f39e3fff640 1 -- 172.21.15.89:0/1461782538 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39ec017030 con 0x7f39fc102a60 2024-09-21T03:37:11.281 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:11.275+0000 7f39e3fff640 1 -- 172.21.15.89:0/1461782538 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f39ec004020 con 0x7f39fc102a60 2024-09-21T03:37:11.281 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:11.275+0000 7f3a012c6640 1 -- 172.21.15.89:0/1461782538 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f39fc1a88c0 con 0x7f39fc102a60 2024-09-21T03:37:11.282 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:11.275+0000 7f39e3fff640 1 -- 172.21.15.89:0/1461782538 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39ec0055d0 con 0x7f39fc102a60 2024-09-21T03:37:11.282 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:11.275+0000 7f3a012c6640 1 -- 172.21.15.89:0/1461782538 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39fc1a8e00 con 0x7f39fc102a60 2024-09-21T03:37:11.282 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:11.276+0000 7f3a012c6640 1 -- 172.21.15.89:0/1461782538 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f39fc102e70 con 0x7f39fc102a60 2024-09-21T03:37:11.282 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:11.279+0000 7f39e3fff640 1 -- 172.21.15.89:0/1461782538 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 17) ==== 154512+0+0 (secure 0 0 0) 0x7f39ec009a30 con 0x7f39fc102a60 2024-09-21T03:37:11.282 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:11.280+0000 7f39e3fff640 1 --2- 172.21.15.89:0/1461782538 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f39e40b6510 0x7f39e40b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:11.283 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:11.280+0000 7f39fa7fc640 1 --2- 172.21.15.89:0/1461782538 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f39e40b6510 0x7f39e40b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:11.283 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:11.280+0000 7f39e3fff640 1 -- 172.21.15.89:0/1461782538 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 7497+0+0 (secure 0 0 0) 0x7f39ec0e6fc0 con 0x7f39fc102a60 2024-09-21T03:37:11.283 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:11.280+0000 7f39fa7fc640 1 --2- 172.21.15.89:0/1461782538 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f39e40b6510 0x7f39e40b8a20 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f39fc079da0 tx=0x7f39f000b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:37:11.285 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:11.283+0000 7f39e3fff640 1 -- 172.21.15.89:0/1461782538 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f39ec0adb80 con 0x7f39fc102a60 2024-09-21T03:37:11.464 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:11.462+0000 7f3a012c6640 1 -- 172.21.15.89:0/1461782538 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05} v 0) -- 0x7f39fc06b270 con 0x7f39fc102a60 2024-09-21T03:37:11.835 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:11.833+0000 7f39e3fff640 1 -- 172.21.15.89:0/1461782538 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]=0 pool 'cephfs_data' created v72) ==== 171+0+0 (secure 0 0 0) 0x7f39fc06b270 con 0x7f39fc102a60 2024-09-21T03:37:11.835 INFO:teuthology.orchestra.run.smithi089.stderr:pool 'cephfs_data' created 2024-09-21T03:37:11.838 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:11.836+0000 7f3a012c6640 1 -- 172.21.15.89:0/1461782538 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f39e40b6510 msgr2=0x7f39e40b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:11.839 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:11.836+0000 7f3a012c6640 1 --2- 172.21.15.89:0/1461782538 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f39e40b6510 0x7f39e40b8a20 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f39fc079da0 tx=0x7f39f000b390 comp rx=0 tx=0).stop 2024-09-21T03:37:11.839 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:11.836+0000 7f3a012c6640 1 -- 172.21.15.89:0/1461782538 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f39fc102a60 msgr2=0x7f39fc198f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:11.839 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:11.836+0000 7f3a012c6640 1 --2- 172.21.15.89:0/1461782538 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f39fc102a60 0x7f39fc198f20 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f39ec00d950 tx=0x7f39ec00de10 comp rx=0 tx=0).stop 2024-09-21T03:37:11.839 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:11.836+0000 7f3a012c6640 1 -- 172.21.15.89:0/1461782538 shutdown_connections 2024-09-21T03:37:11.839 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:11.836+0000 7f3a012c6640 1 --2- 172.21.15.89:0/1461782538 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f39e40b6510 0x7f39e40b8a20 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:11.839 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:11.836+0000 7f3a012c6640 1 --2- 172.21.15.89:0/1461782538 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f39fc1083a0 0x7f39fc199460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:11.839 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:11.836+0000 7f3a012c6640 1 --2- 172.21.15.89:0/1461782538 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f39fc108cf0 0x7f39fc1999a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:11.839 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:11.836+0000 7f3a012c6640 1 --2- 172.21.15.89:0/1461782538 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f39fc102a60 0x7f39fc198f20 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:11.839 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:11.836+0000 7f3a012c6640 1 -- 172.21.15.89:0/1461782538 >> 172.21.15.89:0/1461782538 conn(0x7f39fc079730 msgr2=0x7f39fc1043d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:37:11.839 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:11.836+0000 7f3a012c6640 1 -- 172.21.15.89:0/1461782538 shutdown_connections 2024-09-21T03:37:11.839 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:11.836+0000 7f3a012c6640 1 -- 172.21.15.89:0/1461782538 wait complete. 2024-09-21T03:37:12.211 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:37:11 smithi114 ceph-mon[34140]: osdmap e71: 12 total, 12 up, 12 in 2024-09-21T03:37:12.211 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:37:11 smithi114 ceph-mon[34140]: from='client.? 172.21.15.89:0/1461782538' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05} : dispatch 2024-09-21T03:37:12.211 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:37:11 smithi089 ceph-mon[30928]: osdmap e71: 12 total, 12 up, 12 in 2024-09-21T03:37:12.211 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:37:11 smithi089 ceph-mon[30928]: from='client.? 172.21.15.89:0/1461782538' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05} : dispatch 2024-09-21T03:37:12.250 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:37:11 smithi110 ceph-mon[34000]: osdmap e71: 12 total, 12 up, 12 in 2024-09-21T03:37:12.250 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:37:11 smithi110 ceph-mon[34000]: from='client.? 172.21.15.89:0/1461782538' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05} : dispatch 2024-09-21T03:37:12.577 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 7c374098-77c9-11ef-baf5-efdc52797490 -- ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-21T03:37:12.825 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/7c374098-77c9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:37:13.119 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:37:12 smithi089 ceph-mon[30928]: pgmap v281: 33 pgs: 32 unknown, 1 active+clean; 577 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:37:13.119 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:37:12 smithi089 ceph-mon[30928]: from='client.? 172.21.15.89:0/1461782538' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2024-09-21T03:37:13.119 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:37:12 smithi089 ceph-mon[30928]: osdmap e72: 12 total, 12 up, 12 in 2024-09-21T03:37:13.211 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:37:12 smithi114 ceph-mon[34140]: pgmap v281: 33 pgs: 32 unknown, 1 active+clean; 577 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:37:13.211 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:37:12 smithi114 ceph-mon[34140]: from='client.? 172.21.15.89:0/1461782538' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2024-09-21T03:37:13.211 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:37:12 smithi114 ceph-mon[34140]: osdmap e72: 12 total, 12 up, 12 in 2024-09-21T03:37:13.250 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:37:12 smithi110 ceph-mon[34000]: pgmap v281: 33 pgs: 32 unknown, 1 active+clean; 577 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:37:13.250 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:37:12 smithi110 ceph-mon[34000]: from='client.? 172.21.15.89:0/1461782538' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2024-09-21T03:37:13.250 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:37:12 smithi110 ceph-mon[34000]: osdmap e72: 12 total, 12 up, 12 in 2024-09-21T03:37:13.346 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:13.344+0000 7f3ec9f9f640 1 -- 172.21.15.89:0/62488036 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3ec4102880 msgr2=0x7f3ec4102c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:13.347 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:13.344+0000 7f3ec9f9f640 1 --2- 172.21.15.89:0/62488036 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3ec4102880 0x7f3ec4102c90 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f3eb8009900 tx=0x7f3eb802a410 comp rx=0 tx=0).stop 2024-09-21T03:37:13.347 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:13.344+0000 7f3ec9f9f640 1 -- 172.21.15.89:0/62488036 shutdown_connections 2024-09-21T03:37:13.347 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:13.344+0000 7f3ec9f9f640 1 --2- 172.21.15.89:0/62488036 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3ec41081c0 0x7f3ec41085d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:13.347 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:13.344+0000 7f3ec9f9f640 1 --2- 172.21.15.89:0/62488036 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3ec4108b10 0x7f3ec410f340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:13.347 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:13.344+0000 7f3ec9f9f640 1 --2- 172.21.15.89:0/62488036 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3ec4102880 0x7f3ec4102c90 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:13.347 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:13.344+0000 7f3ec9f9f640 1 -- 172.21.15.89:0/62488036 >> 172.21.15.89:0/62488036 conn(0x7f3ec40fe2d0 msgr2=0x7f3ec4100720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:37:13.347 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:13.344+0000 7f3ec9f9f640 1 -- 172.21.15.89:0/62488036 shutdown_connections 2024-09-21T03:37:13.347 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:13.344+0000 7f3ec9f9f640 1 -- 172.21.15.89:0/62488036 wait complete. 2024-09-21T03:37:13.349 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:13.345+0000 7f3ec9f9f640 1 Processor -- start 2024-09-21T03:37:13.349 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:13.345+0000 7f3ec9f9f640 1 -- start start 2024-09-21T03:37:13.349 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:13.345+0000 7f3ec9f9f640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3ec4102880 0x7f3ec40fe800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:13.349 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:13.345+0000 7f3ec9f9f640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3ec41081c0 0x7f3ec40fed40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:13.349 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:13.345+0000 7f3ec9f9f640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3ec4108b10 0x7f3ec40ff280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:13.349 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:13.345+0000 7f3ec9f9f640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f3ec4112670 con 0x7f3ec4108b10 2024-09-21T03:37:13.349 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:13.345+0000 7f3ec9f9f640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f3ec41124f0 con 0x7f3ec41081c0 2024-09-21T03:37:13.349 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:13.345+0000 7f3ec9f9f640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f3ec41127f0 con 0x7f3ec4102880 2024-09-21T03:37:13.349 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:13.345+0000 7f3ec979e640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3ec4108b10 0x7f3ec40ff280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:13.349 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:13.345+0000 7f3ec979e640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3ec4108b10 0x7f3ec40ff280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:37852/0 (socket says 172.21.15.89:37852) 2024-09-21T03:37:13.350 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:13.345+0000 7f3ec979e640 1 -- 172.21.15.89:0/1830280673 learned_addr learned my addr 172.21.15.89:0/1830280673 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-21T03:37:13.350 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:13.346+0000 7f3ec3fff640 1 --2- 172.21.15.89:0/1830280673 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3ec41081c0 0x7f3ec40fed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:13.350 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:13.346+0000 7f3ec8f9d640 1 --2- 172.21.15.89:0/1830280673 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3ec4102880 0x7f3ec40fe800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:13.350 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:13.346+0000 7f3ec979e640 1 -- 172.21.15.89:0/1830280673 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3ec4102880 msgr2=0x7f3ec40fe800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:13.350 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:13.346+0000 7f3ec979e640 1 --2- 172.21.15.89:0/1830280673 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3ec4102880 0x7f3ec40fe800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:13.350 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:13.346+0000 7f3ec979e640 1 -- 172.21.15.89:0/1830280673 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3ec41081c0 msgr2=0x7f3ec40fed40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:13.350 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:13.346+0000 7f3ec979e640 1 --2- 172.21.15.89:0/1830280673 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3ec41081c0 0x7f3ec40fed40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:13.350 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:13.346+0000 7f3ec979e640 1 -- 172.21.15.89:0/1830280673 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ec41a8420 con 0x7f3ec4108b10 2024-09-21T03:37:13.350 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:13.346+0000 7f3ec979e640 1 --2- 172.21.15.89:0/1830280673 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3ec4108b10 0x7f3ec40ff280 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f3ebc00cce0 tx=0x7f3ebc00a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:37:13.350 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:13.346+0000 7f3ec1ffb640 1 -- 172.21.15.89:0/1830280673 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ebc00f450 con 0x7f3ec4108b10 2024-09-21T03:37:13.350 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:13.346+0000 7f3ec1ffb640 1 -- 172.21.15.89:0/1830280673 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3ebc00fa80 con 0x7f3ec4108b10 2024-09-21T03:37:13.350 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:13.346+0000 7f3ec9f9f640 1 -- 172.21.15.89:0/1830280673 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ec41a8710 con 0x7f3ec4108b10 2024-09-21T03:37:13.350 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:13.346+0000 7f3ec1ffb640 1 -- 172.21.15.89:0/1830280673 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ebc018ad0 con 0x7f3ec4108b10 2024-09-21T03:37:13.350 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:13.346+0000 7f3ec9f9f640 1 -- 172.21.15.89:0/1830280673 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ec41a8c00 con 0x7f3ec4108b10 2024-09-21T03:37:13.351 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:13.347+0000 7f3ec9f9f640 1 -- 172.21.15.89:0/1830280673 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3ec4102c90 con 0x7f3ec4108b10 2024-09-21T03:37:13.352 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:13.350+0000 7f3ec1ffb640 1 -- 172.21.15.89:0/1830280673 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 17) ==== 154512+0+0 (secure 0 0 0) 0x7f3ebc00f5f0 con 0x7f3ec4108b10 2024-09-21T03:37:13.353 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:13.351+0000 7f3ec1ffb640 1 --2- 172.21.15.89:0/1830280673 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f3eb40b6510 0x7f3eb40b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:13.353 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:13.351+0000 7f3ec8f9d640 1 --2- 172.21.15.89:0/1830280673 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f3eb40b6510 0x7f3eb40b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:13.353 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:13.351+0000 7f3ec1ffb640 1 -- 172.21.15.89:0/1830280673 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 7879+0+0 (secure 0 0 0) 0x7f3ebc0e7a60 con 0x7f3ec4108b10 2024-09-21T03:37:13.353 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:13.351+0000 7f3ec8f9d640 1 --2- 172.21.15.89:0/1830280673 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f3eb40b6510 0x7f3eb40b8a20 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f3eb8006010 tx=0x7f3eb8009680 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:37:13.356 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:13.354+0000 7f3ec1ffb640 1 -- 172.21.15.89:0/1830280673 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f3ebc014030 con 0x7f3ec4108b10 2024-09-21T03:37:13.571 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:13.569+0000 7f3ec9f9f640 1 -- 172.21.15.89:0/1830280673 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f3ec40ffb00 con 0x7f3ec4108b10 2024-09-21T03:37:13.853 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:13.851+0000 7f3ec1ffb640 1 -- 172.21.15.89:0/1830280673 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v14) ==== 172+0+0 (secure 0 0 0) 0x7f3ebc0ae530 con 0x7f3ec4108b10 2024-09-21T03:37:13.853 INFO:teuthology.orchestra.run.smithi089.stderr:new fs with metadata pool 2 and data pool 3 2024-09-21T03:37:13.856 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:13.853+0000 7f3ec9f9f640 1 -- 172.21.15.89:0/1830280673 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f3eb40b6510 msgr2=0x7f3eb40b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:13.856 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:13.853+0000 7f3ec9f9f640 1 --2- 172.21.15.89:0/1830280673 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f3eb40b6510 0x7f3eb40b8a20 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f3eb8006010 tx=0x7f3eb8009680 comp rx=0 tx=0).stop 2024-09-21T03:37:13.856 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:13.853+0000 7f3ec9f9f640 1 -- 172.21.15.89:0/1830280673 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3ec4108b10 msgr2=0x7f3ec40ff280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:13.856 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:13.853+0000 7f3ec9f9f640 1 --2- 172.21.15.89:0/1830280673 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3ec4108b10 0x7f3ec40ff280 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f3ebc00cce0 tx=0x7f3ebc00a620 comp rx=0 tx=0).stop 2024-09-21T03:37:13.856 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:13.854+0000 7f3ec9f9f640 1 -- 172.21.15.89:0/1830280673 shutdown_connections 2024-09-21T03:37:13.856 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:13.854+0000 7f3ec9f9f640 1 --2- 172.21.15.89:0/1830280673 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f3eb40b6510 0x7f3eb40b8a20 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:13.856 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:13.854+0000 7f3ec9f9f640 1 --2- 172.21.15.89:0/1830280673 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3ec41081c0 0x7f3ec40fed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:13.856 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:13.854+0000 7f3ec9f9f640 1 --2- 172.21.15.89:0/1830280673 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3ec4108b10 0x7f3ec40ff280 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:13.856 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:13.854+0000 7f3ec9f9f640 1 --2- 172.21.15.89:0/1830280673 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3ec4102880 0x7f3ec40fe800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:13.857 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:13.854+0000 7f3ec9f9f640 1 -- 172.21.15.89:0/1830280673 >> 172.21.15.89:0/1830280673 conn(0x7f3ec40fe2d0 msgr2=0x7f3ec4101390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:37:13.857 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:13.854+0000 7f3ec9f9f640 1 -- 172.21.15.89:0/1830280673 shutdown_connections 2024-09-21T03:37:13.857 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:13.854+0000 7f3ec9f9f640 1 -- 172.21.15.89:0/1830280673 wait complete. 2024-09-21T03:37:14.208 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:37:13 smithi089 ceph-mon[30928]: osdmap e73: 12 total, 12 up, 12 in 2024-09-21T03:37:14.208 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:37:13 smithi089 ceph-mon[30928]: from='client.? 172.21.15.89:0/1830280673' entity='client.admin' cmd={"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} : dispatch 2024-09-21T03:37:14.208 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:37:13 smithi089 ceph-mon[30928]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-09-21T03:37:14.210 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:37:13 smithi114 ceph-mon[34140]: osdmap e73: 12 total, 12 up, 12 in 2024-09-21T03:37:14.211 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:37:13 smithi114 ceph-mon[34140]: from='client.? 172.21.15.89:0/1830280673' entity='client.admin' cmd={"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} : dispatch 2024-09-21T03:37:14.211 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:37:13 smithi114 ceph-mon[34140]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-09-21T03:37:14.250 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:37:13 smithi110 ceph-mon[34000]: osdmap e73: 12 total, 12 up, 12 in 2024-09-21T03:37:14.250 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:37:13 smithi110 ceph-mon[34000]: from='client.? 172.21.15.89:0/1830280673' entity='client.admin' cmd={"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} : dispatch 2024-09-21T03:37:14.250 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:37:13 smithi110 ceph-mon[34000]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-09-21T03:37:14.523 DEBUG:tasks.cephfs.filesystem:EC profile is ['m=2', 'k=2', 'crush-failure-domain=osd'] 2024-09-21T03:37:14.523 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 7c374098-77c9-11ef-baf5-efdc52797490 -- ceph osd erasure-code-profile set cephfs_data_ec m=2 k=2 crush-failure-domain=osd 2024-09-21T03:37:14.773 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/7c374098-77c9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:37:15.163 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:37:14 smithi089 ceph-mon[30928]: pgmap v284: 97 pgs: 21 creating+peering, 23 unknown, 53 active+clean; 577 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:37:15.163 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:37:14 smithi089 ceph-mon[30928]: osdmap e74: 12 total, 12 up, 12 in 2024-09-21T03:37:15.163 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:37:14 smithi089 ceph-mon[30928]: from='client.? 172.21.15.89:0/1830280673' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-09-21T03:37:15.164 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:37:14 smithi089 ceph-mon[30928]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-09-21T03:37:15.164 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:37:14 smithi089 ceph-mon[30928]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-09-21T03:37:15.211 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:37:14 smithi114 ceph-mon[34140]: pgmap v284: 97 pgs: 21 creating+peering, 23 unknown, 53 active+clean; 577 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:37:15.211 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:37:14 smithi114 ceph-mon[34140]: osdmap e74: 12 total, 12 up, 12 in 2024-09-21T03:37:15.211 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:37:14 smithi114 ceph-mon[34140]: from='client.? 172.21.15.89:0/1830280673' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-09-21T03:37:15.211 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:37:14 smithi114 ceph-mon[34140]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-09-21T03:37:15.211 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:37:14 smithi114 ceph-mon[34140]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-09-21T03:37:15.250 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:37:14 smithi110 ceph-mon[34000]: pgmap v284: 97 pgs: 21 creating+peering, 23 unknown, 53 active+clean; 577 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:37:15.250 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:37:14 smithi110 ceph-mon[34000]: osdmap e74: 12 total, 12 up, 12 in 2024-09-21T03:37:15.250 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:37:14 smithi110 ceph-mon[34000]: from='client.? 172.21.15.89:0/1830280673' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-09-21T03:37:15.250 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:37:14 smithi110 ceph-mon[34000]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-09-21T03:37:15.250 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:37:14 smithi110 ceph-mon[34000]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-09-21T03:37:15.264 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:15.256+0000 7f9e67f8e640 1 -- 172.21.15.89:0/2088180343 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9e60109df0 msgr2=0x7f9e60110620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:15.264 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:15.256+0000 7f9e67f8e640 1 --2- 172.21.15.89:0/2088180343 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9e60109df0 0x7f9e60110620 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f9e58009ca0 tx=0x7f9e5802a8c0 comp rx=0 tx=0).stop 2024-09-21T03:37:15.264 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:15.256+0000 7f9e67f8e640 1 -- 172.21.15.89:0/2088180343 shutdown_connections 2024-09-21T03:37:15.264 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:15.256+0000 7f9e67f8e640 1 --2- 172.21.15.89:0/2088180343 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9e60109df0 0x7f9e60110620 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:15.264 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:15.256+0000 7f9e67f8e640 1 --2- 172.21.15.89:0/2088180343 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9e601094a0 0x7f9e601098b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:15.264 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:15.256+0000 7f9e67f8e640 1 --2- 172.21.15.89:0/2088180343 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9e60103b60 0x7f9e60103f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:15.264 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:15.256+0000 7f9e67f8e640 1 -- 172.21.15.89:0/2088180343 >> 172.21.15.89:0/2088180343 conn(0x7f9e6007b850 msgr2=0x7f9e6007bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:37:15.264 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:15.256+0000 7f9e67f8e640 1 -- 172.21.15.89:0/2088180343 shutdown_connections 2024-09-21T03:37:15.265 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:15.256+0000 7f9e67f8e640 1 -- 172.21.15.89:0/2088180343 wait complete. 2024-09-21T03:37:15.265 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:15.256+0000 7f9e67f8e640 1 Processor -- start 2024-09-21T03:37:15.265 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:15.256+0000 7f9e67f8e640 1 -- start start 2024-09-21T03:37:15.265 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:15.257+0000 7f9e67f8e640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9e60103b60 0x7f9e6019a3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:15.265 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:15.257+0000 7f9e67f8e640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9e601094a0 0x7f9e6019a940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:15.265 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:15.257+0000 7f9e67f8e640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9e60109df0 0x7f9e6019fbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:15.265 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:15.257+0000 7f9e67f8e640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f9e601139b0 con 0x7f9e60103b60 2024-09-21T03:37:15.265 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:15.257+0000 7f9e67f8e640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f9e60113830 con 0x7f9e601094a0 2024-09-21T03:37:15.265 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:15.257+0000 7f9e67f8e640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f9e60113b30 con 0x7f9e60109df0 2024-09-21T03:37:15.265 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:15.257+0000 7f9e65d2a640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9e60103b60 0x7f9e6019a3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:15.265 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:15.257+0000 7f9e65d2a640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9e60103b60 0x7f9e6019a3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:37872/0 (socket says 172.21.15.89:37872) 2024-09-21T03:37:15.265 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:15.257+0000 7f9e65d2a640 1 -- 172.21.15.89:0/4209506568 learned_addr learned my addr 172.21.15.89:0/4209506568 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-21T03:37:15.265 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:15.257+0000 7f9e65529640 1 --2- 172.21.15.89:0/4209506568 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9e601094a0 0x7f9e6019a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:15.266 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:15.257+0000 7f9e65d2a640 1 -- 172.21.15.89:0/4209506568 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9e60109df0 msgr2=0x7f9e6019fbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:15.266 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:15.257+0000 7f9e65d2a640 1 --2- 172.21.15.89:0/4209506568 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9e60109df0 0x7f9e6019fbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:15.266 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:15.257+0000 7f9e65d2a640 1 -- 172.21.15.89:0/4209506568 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9e601094a0 msgr2=0x7f9e6019a940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:15.266 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:15.257+0000 7f9e65d2a640 1 --2- 172.21.15.89:0/4209506568 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9e601094a0 0x7f9e6019a940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:15.266 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:15.257+0000 7f9e65d2a640 1 -- 172.21.15.89:0/4209506568 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e601a0250 con 0x7f9e60103b60 2024-09-21T03:37:15.266 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:15.257+0000 7f9e65d2a640 1 --2- 172.21.15.89:0/4209506568 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9e60103b60 0x7f9e6019a3a0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f9e5c00c890 tx=0x7f9e5c00cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:37:15.266 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:15.257+0000 7f9e52ffd640 1 -- 172.21.15.89:0/4209506568 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e5c004220 con 0x7f9e60103b60 2024-09-21T03:37:15.266 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:15.257+0000 7f9e52ffd640 1 -- 172.21.15.89:0/4209506568 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9e5c007e40 con 0x7f9e60103b60 2024-09-21T03:37:15.266 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:15.257+0000 7f9e52ffd640 1 -- 172.21.15.89:0/4209506568 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e5c004a00 con 0x7f9e60103b60 2024-09-21T03:37:15.266 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:15.257+0000 7f9e67f8e640 1 -- 172.21.15.89:0/4209506568 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9e601a0540 con 0x7f9e60103b60 2024-09-21T03:37:15.266 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:15.257+0000 7f9e67f8e640 1 -- 172.21.15.89:0/4209506568 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9e601a6d60 con 0x7f9e60103b60 2024-09-21T03:37:15.266 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:15.259+0000 7f9e67f8e640 1 -- 172.21.15.89:0/4209506568 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9e60103f70 con 0x7f9e60103b60 2024-09-21T03:37:15.266 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:15.260+0000 7f9e52ffd640 1 -- 172.21.15.89:0/4209506568 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 17) ==== 154512+0+0 (secure 0 0 0) 0x7f9e5c0079b0 con 0x7f9e60103b60 2024-09-21T03:37:15.267 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:15.260+0000 7f9e52ffd640 1 --2- 172.21.15.89:0/4209506568 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f9e4c0b65c0 0x7f9e4c0b8ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:15.267 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:15.261+0000 7f9e65529640 1 --2- 172.21.15.89:0/4209506568 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f9e4c0b65c0 0x7f9e4c0b8ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:15.267 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:15.261+0000 7f9e52ffd640 1 -- 172.21.15.89:0/4209506568 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 7979+0+0 (secure 0 0 0) 0x7f9e5c0e6ea0 con 0x7f9e60103b60 2024-09-21T03:37:15.267 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:15.261+0000 7f9e65529640 1 --2- 172.21.15.89:0/4209506568 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f9e4c0b65c0 0x7f9e4c0b8ad0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f9e60107160 tx=0x7f9e5400c4a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:37:15.267 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:15.265+0000 7f9e52ffd640 1 -- 172.21.15.89:0/4209506568 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f9e5c014030 con 0x7f9e60103b60 2024-09-21T03:37:15.454 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:15.452+0000 7f9e67f8e640 1 -- 172.21.15.89:0/4209506568 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]} v 0) -- 0x7f9e6019b4d0 con 0x7f9e60103b60 2024-09-21T03:37:15.869 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:15.867+0000 7f9e52ffd640 1 -- 172.21.15.89:0/4209506568 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]=0 v75) ==== 157+0+0 (secure 0 0 0) 0x7f9e5c0ad880 con 0x7f9e60103b60 2024-09-21T03:37:15.872 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:15.870+0000 7f9e67f8e640 1 -- 172.21.15.89:0/4209506568 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f9e4c0b65c0 msgr2=0x7f9e4c0b8ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:15.872 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:15.870+0000 7f9e67f8e640 1 --2- 172.21.15.89:0/4209506568 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f9e4c0b65c0 0x7f9e4c0b8ad0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f9e60107160 tx=0x7f9e5400c4a0 comp rx=0 tx=0).stop 2024-09-21T03:37:15.872 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:15.870+0000 7f9e67f8e640 1 -- 172.21.15.89:0/4209506568 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9e60103b60 msgr2=0x7f9e6019a3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:15.873 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:15.870+0000 7f9e67f8e640 1 --2- 172.21.15.89:0/4209506568 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9e60103b60 0x7f9e6019a3a0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f9e5c00c890 tx=0x7f9e5c00cd50 comp rx=0 tx=0).stop 2024-09-21T03:37:15.873 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:15.870+0000 7f9e67f8e640 1 -- 172.21.15.89:0/4209506568 shutdown_connections 2024-09-21T03:37:15.873 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:15.870+0000 7f9e67f8e640 1 --2- 172.21.15.89:0/4209506568 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f9e4c0b65c0 0x7f9e4c0b8ad0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:15.873 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:15.870+0000 7f9e67f8e640 1 --2- 172.21.15.89:0/4209506568 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9e601094a0 0x7f9e6019a940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:15.873 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:15.870+0000 7f9e67f8e640 1 --2- 172.21.15.89:0/4209506568 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9e60109df0 0x7f9e6019fbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:15.873 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:15.870+0000 7f9e67f8e640 1 --2- 172.21.15.89:0/4209506568 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9e60103b60 0x7f9e6019a3a0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:15.873 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:15.870+0000 7f9e67f8e640 1 -- 172.21.15.89:0/4209506568 >> 172.21.15.89:0/4209506568 conn(0x7f9e6007b850 msgr2=0x7f9e601054b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:37:15.873 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:15.870+0000 7f9e67f8e640 1 -- 172.21.15.89:0/4209506568 shutdown_connections 2024-09-21T03:37:15.873 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:15.870+0000 7f9e67f8e640 1 -- 172.21.15.89:0/4209506568 wait complete. 2024-09-21T03:37:16.165 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:37:15 smithi114 ceph-mon[34140]: mds.? [v2:172.21.15.110:6838/132366709,v1:172.21.15.110:6839/132366709] up:active 2024-09-21T03:37:16.165 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:37:15 smithi114 ceph-mon[34140]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-21T03:37:16.165 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:37:15 smithi114 ceph-mon[34140]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-21T03:37:16.165 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:37:15 smithi114 ceph-mon[34140]: from='client.? 172.21.15.89:0/4209506568' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]} : dispatch 2024-09-21T03:37:16.243 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:37:15 smithi089 ceph-mon[30928]: mds.? [v2:172.21.15.110:6838/132366709,v1:172.21.15.110:6839/132366709] up:active 2024-09-21T03:37:16.243 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:37:15 smithi089 ceph-mon[30928]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-21T03:37:16.243 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:37:15 smithi089 ceph-mon[30928]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-21T03:37:16.244 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:37:15 smithi089 ceph-mon[30928]: from='client.? 172.21.15.89:0/4209506568' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]} : dispatch 2024-09-21T03:37:16.250 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:37:15 smithi110 ceph-mon[34000]: mds.? [v2:172.21.15.110:6838/132366709,v1:172.21.15.110:6839/132366709] up:active 2024-09-21T03:37:16.250 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:37:15 smithi110 ceph-mon[34000]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-21T03:37:16.250 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:37:15 smithi110 ceph-mon[34000]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-21T03:37:16.250 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:37:15 smithi110 ceph-mon[34000]: from='client.? 172.21.15.89:0/4209506568' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]} : dispatch 2024-09-21T03:37:16.556 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 7c374098-77c9-11ef-baf5-efdc52797490 -- ceph osd pool create cephfs_data_ec erasure cephfs_data_ec --pg_num_min 64 --target_size_ratio 0.9 2024-09-21T03:37:16.804 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/7c374098-77c9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:37:17.121 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:37:16 smithi089 ceph-mon[30928]: pgmap v286: 97 pgs: 21 creating+peering, 76 active+clean; 578 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 641 B/s wr, 1 op/s 2024-09-21T03:37:17.121 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:37:16 smithi089 ceph-mon[30928]: from='client.? 172.21.15.89:0/4209506568' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2024-09-21T03:37:17.121 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:37:16 smithi089 ceph-mon[30928]: osdmap e75: 12 total, 12 up, 12 in 2024-09-21T03:37:17.211 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:37:16 smithi114 ceph-mon[34140]: pgmap v286: 97 pgs: 21 creating+peering, 76 active+clean; 578 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 641 B/s wr, 1 op/s 2024-09-21T03:37:17.211 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:37:16 smithi114 ceph-mon[34140]: from='client.? 172.21.15.89:0/4209506568' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2024-09-21T03:37:17.211 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:37:16 smithi114 ceph-mon[34140]: osdmap e75: 12 total, 12 up, 12 in 2024-09-21T03:37:17.250 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:37:16 smithi110 ceph-mon[34000]: pgmap v286: 97 pgs: 21 creating+peering, 76 active+clean; 578 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 641 B/s wr, 1 op/s 2024-09-21T03:37:17.250 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:37:16 smithi110 ceph-mon[34000]: from='client.? 172.21.15.89:0/4209506568' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2024-09-21T03:37:17.250 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:37:16 smithi110 ceph-mon[34000]: osdmap e75: 12 total, 12 up, 12 in 2024-09-21T03:37:17.345 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:17.342+0000 7f36942db640 1 -- 172.21.15.89:0/816873170 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f368c109570 msgr2=0x7f368c109980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:17.345 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:17.342+0000 7f36942db640 1 --2- 172.21.15.89:0/816873170 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f368c109570 0x7f368c109980 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f367c009900 tx=0x7f367c02a410 comp rx=0 tx=0).stop 2024-09-21T03:37:17.345 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:17.343+0000 7f36942db640 1 -- 172.21.15.89:0/816873170 shutdown_connections 2024-09-21T03:37:17.345 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:17.343+0000 7f36942db640 1 --2- 172.21.15.89:0/816873170 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f368c109ec0 0x7f368c110620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:17.345 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:17.343+0000 7f36942db640 1 --2- 172.21.15.89:0/816873170 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f368c109570 0x7f368c109980 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:17.345 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:17.343+0000 7f36942db640 1 --2- 172.21.15.89:0/816873170 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f368c101ba0 0x7f368c101fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:17.346 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:17.343+0000 7f36942db640 1 -- 172.21.15.89:0/816873170 >> 172.21.15.89:0/816873170 conn(0x7f368c0776b0 msgr2=0x7f368c077b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:37:17.346 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:17.343+0000 7f36942db640 1 -- 172.21.15.89:0/816873170 shutdown_connections 2024-09-21T03:37:17.346 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:17.343+0000 7f36942db640 1 -- 172.21.15.89:0/816873170 wait complete. 2024-09-21T03:37:17.347 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:17.343+0000 7f36942db640 1 Processor -- start 2024-09-21T03:37:17.347 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:17.344+0000 7f36942db640 1 -- start start 2024-09-21T03:37:17.348 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:17.344+0000 7f36942db640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f368c101ba0 0x7f368c19a410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:17.348 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:17.344+0000 7f36942db640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f368c109570 0x7f368c19a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:17.348 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:17.344+0000 7f36942db640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f368c109ec0 0x7f368c19fc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:17.348 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:17.344+0000 7f36942db640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f368c113960 con 0x7f368c109570 2024-09-21T03:37:17.348 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:17.344+0000 7f36942db640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f368c1137e0 con 0x7f368c101ba0 2024-09-21T03:37:17.348 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:17.344+0000 7f36942db640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f368c113ae0 con 0x7f368c109ec0 2024-09-21T03:37:17.348 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:17.344+0000 7f3691876640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f368c109570 0x7f368c19a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:17.348 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:17.344+0000 7f3691876640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f368c109570 0x7f368c19a9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:37884/0 (socket says 172.21.15.89:37884) 2024-09-21T03:37:17.348 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:17.344+0000 7f3691876640 1 -- 172.21.15.89:0/69367037 learned_addr learned my addr 172.21.15.89:0/69367037 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-21T03:37:17.348 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:17.344+0000 7f3692077640 1 --2- 172.21.15.89:0/69367037 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f368c101ba0 0x7f368c19a410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:17.348 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:17.344+0000 7f3691876640 1 -- 172.21.15.89:0/69367037 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f368c109ec0 msgr2=0x7f368c19fc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:17.348 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:17.344+0000 7f3691876640 1 --2- 172.21.15.89:0/69367037 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f368c109ec0 0x7f368c19fc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:17.348 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:17.344+0000 7f3691876640 1 -- 172.21.15.89:0/69367037 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f368c101ba0 msgr2=0x7f368c19a410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:17.348 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:17.344+0000 7f3691876640 1 --2- 172.21.15.89:0/69367037 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f368c101ba0 0x7f368c19a410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:17.349 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:17.344+0000 7f3691876640 1 -- 172.21.15.89:0/69367037 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f368c1a02c0 con 0x7f368c109570 2024-09-21T03:37:17.349 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:17.344+0000 7f3692077640 1 --2- 172.21.15.89:0/69367037 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f368c101ba0 0x7f368c19a410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:37:17.349 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:17.344+0000 7f3691876640 1 --2- 172.21.15.89:0/69367037 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f368c109570 0x7f368c19a9b0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f367c005940 tx=0x7f367c0040e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:37:17.349 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:17.345+0000 7f36837fe640 1 -- 172.21.15.89:0/69367037 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f367c03f030 con 0x7f368c109570 2024-09-21T03:37:17.349 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:17.345+0000 7f36837fe640 1 -- 172.21.15.89:0/69367037 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f367c003a80 con 0x7f368c109570 2024-09-21T03:37:17.349 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:17.345+0000 7f36837fe640 1 -- 172.21.15.89:0/69367037 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f367c035070 con 0x7f368c109570 2024-09-21T03:37:17.349 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:17.345+0000 7f36942db640 1 -- 172.21.15.89:0/69367037 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f368c1a0550 con 0x7f368c109570 2024-09-21T03:37:17.349 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:17.345+0000 7f36942db640 1 -- 172.21.15.89:0/69367037 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f368c1a0a30 con 0x7f368c109570 2024-09-21T03:37:17.349 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:17.346+0000 7f36942db640 1 -- 172.21.15.89:0/69367037 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f368c101fb0 con 0x7f368c109570 2024-09-21T03:37:17.350 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:17.348+0000 7f36837fe640 1 -- 172.21.15.89:0/69367037 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 17) ==== 154512+0+0 (secure 0 0 0) 0x7f367c042030 con 0x7f368c109570 2024-09-21T03:37:17.351 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:17.349+0000 7f36837fe640 1 --2- 172.21.15.89:0/69367037 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f36780b6540 0x7f36780b8a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:17.351 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:17.349+0000 7f3692077640 1 --2- 172.21.15.89:0/69367037 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f36780b6540 0x7f36780b8a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:17.351 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:17.349+0000 7f36837fe640 1 -- 172.21.15.89:0/69367037 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 8277+0+0 (secure 0 0 0) 0x7f367c107cd0 con 0x7f368c109570 2024-09-21T03:37:17.351 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:17.349+0000 7f3692077640 1 --2- 172.21.15.89:0/69367037 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f36780b6540 0x7f36780b8a50 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f368800abe0 tx=0x7f368800a5c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:37:17.354 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:17.352+0000 7f36837fe640 1 -- 172.21.15.89:0/69367037 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f367c0ce530 con 0x7f368c109570 2024-09-21T03:37:17.532 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:17.530+0000 7f36942db640 1 -- 172.21.15.89:0/69367037 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f368c02e640 con 0x7f368c109570 2024-09-21T03:37:18.211 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:37:17 smithi114 ceph-mon[34140]: from='client.? 172.21.15.89:0/69367037' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9} : dispatch 2024-09-21T03:37:18.250 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:37:17 smithi110 ceph-mon[34000]: from='client.? 172.21.15.89:0/69367037' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9} : dispatch 2024-09-21T03:37:18.311 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:37:17 smithi089 ceph-mon[30928]: from='client.? 172.21.15.89:0/69367037' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9} : dispatch 2024-09-21T03:37:18.879 INFO:journalctl@ceph.mds.b.smithi110.stdout:Sep 21 03:37:18 smithi110 ceph-7c374098-77c9-11ef-baf5-efdc52797490-mds-b[62090]: 2024-09-21T03:37:18.563+0000 7f39912bb640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-21T03:37:18.885 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:18.882+0000 7f36837fe640 1 -- 172.21.15.89:0/69367037 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data_ec' created v77) ==== 228+0+0 (secure 0 0 0) 0x7f367c0d2910 con 0x7f368c109570 2024-09-21T03:37:18.885 INFO:teuthology.orchestra.run.smithi089.stderr:pool 'cephfs_data_ec' created 2024-09-21T03:37:18.889 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:18.886+0000 7f36942db640 1 -- 172.21.15.89:0/69367037 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f36780b6540 msgr2=0x7f36780b8a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:18.889 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:18.886+0000 7f36942db640 1 --2- 172.21.15.89:0/69367037 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f36780b6540 0x7f36780b8a50 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f368800abe0 tx=0x7f368800a5c0 comp rx=0 tx=0).stop 2024-09-21T03:37:18.889 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:18.886+0000 7f36942db640 1 -- 172.21.15.89:0/69367037 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f368c109570 msgr2=0x7f368c19a9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:18.889 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:18.886+0000 7f36942db640 1 --2- 172.21.15.89:0/69367037 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f368c109570 0x7f368c19a9b0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f367c005940 tx=0x7f367c0040e0 comp rx=0 tx=0).stop 2024-09-21T03:37:18.889 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:18.886+0000 7f36942db640 1 -- 172.21.15.89:0/69367037 shutdown_connections 2024-09-21T03:37:18.889 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:18.886+0000 7f36942db640 1 --2- 172.21.15.89:0/69367037 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f36780b6540 0x7f36780b8a50 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:18.889 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:18.886+0000 7f36942db640 1 --2- 172.21.15.89:0/69367037 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f368c109ec0 0x7f368c19fc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:18.889 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:18.886+0000 7f36942db640 1 --2- 172.21.15.89:0/69367037 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f368c109570 0x7f368c19a9b0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:18.889 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:18.886+0000 7f36942db640 1 --2- 172.21.15.89:0/69367037 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f368c101ba0 0x7f368c19a410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:18.889 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:18.886+0000 7f36942db640 1 -- 172.21.15.89:0/69367037 >> 172.21.15.89:0/69367037 conn(0x7f368c0776b0 msgr2=0x7f368c100560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:37:18.890 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:18.886+0000 7f36942db640 1 -- 172.21.15.89:0/69367037 shutdown_connections 2024-09-21T03:37:18.890 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:18.886+0000 7f36942db640 1 -- 172.21.15.89:0/69367037 wait complete. 2024-09-21T03:37:19.211 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:37:18 smithi114 ceph-mon[34140]: pgmap v288: 97 pgs: 97 active+clean; 578 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s wr, 6 op/s 2024-09-21T03:37:19.211 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:37:18 smithi114 ceph-mon[34140]: osdmap e76: 12 total, 12 up, 12 in 2024-09-21T03:37:19.212 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:37:18 smithi089 ceph-mon[30928]: pgmap v288: 97 pgs: 97 active+clean; 578 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s wr, 6 op/s 2024-09-21T03:37:19.212 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:37:18 smithi089 ceph-mon[30928]: osdmap e76: 12 total, 12 up, 12 in 2024-09-21T03:37:19.250 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:37:18 smithi110 ceph-mon[34000]: pgmap v288: 97 pgs: 97 active+clean; 578 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s wr, 6 op/s 2024-09-21T03:37:19.250 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:37:18 smithi110 ceph-mon[34000]: osdmap e76: 12 total, 12 up, 12 in 2024-09-21T03:37:19.631 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 7c374098-77c9-11ef-baf5-efdc52797490 -- ceph osd pool set cephfs_data_ec allow_ec_overwrites true 2024-09-21T03:37:19.882 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/7c374098-77c9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:37:20.150 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:37:19 smithi089 ceph-mon[30928]: from='client.? 172.21.15.89:0/69367037' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-09-21T03:37:20.151 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:37:19 smithi089 ceph-mon[30928]: osdmap e77: 12 total, 12 up, 12 in 2024-09-21T03:37:20.211 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:37:19 smithi114 ceph-mon[34140]: from='client.? 172.21.15.89:0/69367037' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-09-21T03:37:20.211 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:37:19 smithi114 ceph-mon[34140]: osdmap e77: 12 total, 12 up, 12 in 2024-09-21T03:37:20.250 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:37:19 smithi110 ceph-mon[34000]: from='client.? 172.21.15.89:0/69367037' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-09-21T03:37:20.250 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:37:19 smithi110 ceph-mon[34000]: osdmap e77: 12 total, 12 up, 12 in 2024-09-21T03:37:20.378 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:20.375+0000 7fee4e1c8640 1 -- 172.21.15.89:0/3170458057 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fee481083a0 msgr2=0x7fee481087b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:20.378 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:20.375+0000 7fee4e1c8640 1 --2- 172.21.15.89:0/3170458057 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fee481083a0 0x7fee481087b0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fee38009900 tx=0x7fee3802a410 comp rx=0 tx=0).stop 2024-09-21T03:37:20.378 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:20.375+0000 7fee4e1c8640 1 -- 172.21.15.89:0/3170458057 shutdown_connections 2024-09-21T03:37:20.378 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:20.375+0000 7fee4e1c8640 1 --2- 172.21.15.89:0/3170458057 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fee48108cf0 0x7fee4810f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:20.378 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:20.375+0000 7fee4e1c8640 1 --2- 172.21.15.89:0/3170458057 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fee481083a0 0x7fee481087b0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:20.378 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:20.375+0000 7fee4e1c8640 1 --2- 172.21.15.89:0/3170458057 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fee48102a60 0x7fee48102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:20.378 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:20.375+0000 7fee4e1c8640 1 -- 172.21.15.89:0/3170458057 >> 172.21.15.89:0/3170458057 conn(0x7fee48079730 msgr2=0x7fee48079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:37:20.378 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:20.375+0000 7fee4e1c8640 1 -- 172.21.15.89:0/3170458057 shutdown_connections 2024-09-21T03:37:20.378 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:20.375+0000 7fee4e1c8640 1 -- 172.21.15.89:0/3170458057 wait complete. 2024-09-21T03:37:20.378 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:20.376+0000 7fee4e1c8640 1 Processor -- start 2024-09-21T03:37:20.380 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:20.376+0000 7fee4e1c8640 1 -- start start 2024-09-21T03:37:20.380 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:20.376+0000 7fee4e1c8640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fee48102a60 0x7fee48199300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:20.380 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:20.376+0000 7fee4e1c8640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fee481083a0 0x7fee481998a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:20.380 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:20.376+0000 7fee4e1c8640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fee48108cf0 0x7fee4819eb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:20.380 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:20.376+0000 7fee4e1c8640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fee48112850 con 0x7fee481083a0 2024-09-21T03:37:20.380 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:20.376+0000 7fee4e1c8640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fee481126d0 con 0x7fee48108cf0 2024-09-21T03:37:20.380 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:20.376+0000 7fee4e1c8640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fee481129d0 con 0x7fee48102a60 2024-09-21T03:37:20.380 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:20.376+0000 7fee46ffd640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fee481083a0 0x7fee481998a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:20.380 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:20.376+0000 7fee46ffd640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fee481083a0 0x7fee481998a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:37286/0 (socket says 172.21.15.89:37286) 2024-09-21T03:37:20.380 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:20.376+0000 7fee46ffd640 1 -- 172.21.15.89:0/1284324404 learned_addr learned my addr 172.21.15.89:0/1284324404 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-21T03:37:20.381 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:20.376+0000 7fee477fe640 1 --2- 172.21.15.89:0/1284324404 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fee48102a60 0x7fee48199300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:20.381 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:20.376+0000 7fee47fff640 1 --2- 172.21.15.89:0/1284324404 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fee48108cf0 0x7fee4819eb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:20.381 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:20.377+0000 7fee46ffd640 1 -- 172.21.15.89:0/1284324404 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fee48102a60 msgr2=0x7fee48199300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:20.381 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:20.377+0000 7fee46ffd640 1 --2- 172.21.15.89:0/1284324404 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fee48102a60 0x7fee48199300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:20.381 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:20.377+0000 7fee46ffd640 1 -- 172.21.15.89:0/1284324404 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fee48108cf0 msgr2=0x7fee4819eb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:20.381 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:20.377+0000 7fee46ffd640 1 --2- 172.21.15.89:0/1284324404 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fee48108cf0 0x7fee4819eb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:20.381 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:20.377+0000 7fee46ffd640 1 -- 172.21.15.89:0/1284324404 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee4819f270 con 0x7fee481083a0 2024-09-21T03:37:20.381 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:20.377+0000 7fee46ffd640 1 --2- 172.21.15.89:0/1284324404 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fee481083a0 0x7fee481998a0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fee38005940 tx=0x7fee380042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:37:20.381 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:20.377+0000 7fee44ff9640 1 -- 172.21.15.89:0/1284324404 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee3803f030 con 0x7fee481083a0 2024-09-21T03:37:20.381 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:20.377+0000 7fee4e1c8640 1 -- 172.21.15.89:0/1284324404 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fee4819f500 con 0x7fee481083a0 2024-09-21T03:37:20.381 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:20.377+0000 7fee44ff9640 1 -- 172.21.15.89:0/1284324404 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fee38003c90 con 0x7fee481083a0 2024-09-21T03:37:20.381 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:20.377+0000 7fee44ff9640 1 -- 172.21.15.89:0/1284324404 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee38035070 con 0x7fee481083a0 2024-09-21T03:37:20.381 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:20.377+0000 7fee4e1c8640 1 -- 172.21.15.89:0/1284324404 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fee4806ae60 con 0x7fee481083a0 2024-09-21T03:37:20.382 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:20.378+0000 7fee4e1c8640 1 -- 172.21.15.89:0/1284324404 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fee48102e70 con 0x7fee481083a0 2024-09-21T03:37:20.382 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:20.380+0000 7fee44ff9640 1 -- 172.21.15.89:0/1284324404 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 17) ==== 154512+0+0 (secure 0 0 0) 0x7fee38042030 con 0x7fee481083a0 2024-09-21T03:37:20.384 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:20.381+0000 7fee44ff9640 1 --2- 172.21.15.89:0/1284324404 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7fee300b6510 0x7fee300b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:20.384 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:20.382+0000 7fee477fe640 1 --2- 172.21.15.89:0/1284324404 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7fee300b6510 0x7fee300b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:20.384 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:20.382+0000 7fee44ff9640 1 -- 172.21.15.89:0/1284324404 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8770+0+0 (secure 0 0 0) 0x7fee38107f20 con 0x7fee481083a0 2024-09-21T03:37:20.384 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:20.382+0000 7fee477fe640 1 --2- 172.21.15.89:0/1284324404 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7fee300b6510 0x7fee300b8a20 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fee3c0058c0 tx=0x7fee3c00a2d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:37:20.387 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:20.385+0000 7fee44ff9640 1 -- 172.21.15.89:0/1284324404 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fee380d2910 con 0x7fee481083a0 2024-09-21T03:37:20.576 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:20.573+0000 7fee4e1c8640 1 -- 172.21.15.89:0/1284324404 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"} v 0) -- 0x7fee4806b940 con 0x7fee481083a0 2024-09-21T03:37:20.893 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:20.889+0000 7fee44ff9640 1 -- 172.21.15.89:0/1284324404 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]=0 set pool 4 allow_ec_overwrites to true v79) ==== 169+0+0 (secure 0 0 0) 0x7fee380d7850 con 0x7fee481083a0 2024-09-21T03:37:20.893 INFO:teuthology.orchestra.run.smithi089.stderr:set pool 4 allow_ec_overwrites to true 2024-09-21T03:37:20.894 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:20.892+0000 7fee4e1c8640 1 -- 172.21.15.89:0/1284324404 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7fee300b6510 msgr2=0x7fee300b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:20.895 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:20.892+0000 7fee4e1c8640 1 --2- 172.21.15.89:0/1284324404 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7fee300b6510 0x7fee300b8a20 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fee3c0058c0 tx=0x7fee3c00a2d0 comp rx=0 tx=0).stop 2024-09-21T03:37:20.895 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:20.892+0000 7fee4e1c8640 1 -- 172.21.15.89:0/1284324404 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fee481083a0 msgr2=0x7fee481998a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:20.895 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:20.892+0000 7fee4e1c8640 1 --2- 172.21.15.89:0/1284324404 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fee481083a0 0x7fee481998a0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fee38005940 tx=0x7fee380042f0 comp rx=0 tx=0).stop 2024-09-21T03:37:20.895 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:20.892+0000 7fee4e1c8640 1 -- 172.21.15.89:0/1284324404 shutdown_connections 2024-09-21T03:37:20.895 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:20.892+0000 7fee4e1c8640 1 --2- 172.21.15.89:0/1284324404 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7fee300b6510 0x7fee300b8a20 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:20.895 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:20.892+0000 7fee4e1c8640 1 --2- 172.21.15.89:0/1284324404 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fee48108cf0 0x7fee4819eb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:20.895 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:20.892+0000 7fee4e1c8640 1 --2- 172.21.15.89:0/1284324404 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fee481083a0 0x7fee481998a0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:20.895 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:20.892+0000 7fee4e1c8640 1 --2- 172.21.15.89:0/1284324404 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fee48102a60 0x7fee48199300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:20.895 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:20.892+0000 7fee4e1c8640 1 -- 172.21.15.89:0/1284324404 >> 172.21.15.89:0/1284324404 conn(0x7fee48079730 msgr2=0x7fee48105350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:37:20.895 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:20.892+0000 7fee4e1c8640 1 -- 172.21.15.89:0/1284324404 shutdown_connections 2024-09-21T03:37:20.895 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:20.892+0000 7fee4e1c8640 1 -- 172.21.15.89:0/1284324404 wait complete. 2024-09-21T03:37:21.211 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:37:20 smithi114 ceph-mon[34140]: pgmap v291: 129 pgs: 6 creating+peering, 26 unknown, 97 active+clean; 579 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s wr, 7 op/s 2024-09-21T03:37:21.211 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:37:20 smithi114 ceph-mon[34140]: osdmap e78: 12 total, 12 up, 12 in 2024-09-21T03:37:21.211 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:37:20 smithi114 ceph-mon[34140]: from='client.? 172.21.15.89:0/1284324404' entity='client.admin' cmd={"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"} : dispatch 2024-09-21T03:37:21.211 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:37:20 smithi089 ceph-mon[30928]: pgmap v291: 129 pgs: 6 creating+peering, 26 unknown, 97 active+clean; 579 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s wr, 7 op/s 2024-09-21T03:37:21.211 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:37:20 smithi089 ceph-mon[30928]: osdmap e78: 12 total, 12 up, 12 in 2024-09-21T03:37:21.211 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:37:20 smithi089 ceph-mon[30928]: from='client.? 172.21.15.89:0/1284324404' entity='client.admin' cmd={"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"} : dispatch 2024-09-21T03:37:21.250 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:37:20 smithi110 ceph-mon[34000]: pgmap v291: 129 pgs: 6 creating+peering, 26 unknown, 97 active+clean; 579 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s wr, 7 op/s 2024-09-21T03:37:21.250 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:37:20 smithi110 ceph-mon[34000]: osdmap e78: 12 total, 12 up, 12 in 2024-09-21T03:37:21.250 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:37:20 smithi110 ceph-mon[34000]: from='client.? 172.21.15.89:0/1284324404' entity='client.admin' cmd={"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"} : dispatch 2024-09-21T03:37:21.547 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 7c374098-77c9-11ef-baf5-efdc52797490 -- ceph fs add_data_pool cephfs cephfs_data_ec 2024-09-21T03:37:21.800 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/7c374098-77c9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:37:22.199 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:37:21 smithi089 ceph-mon[30928]: from='client.? 172.21.15.89:0/1284324404' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-09-21T03:37:22.199 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:37:21 smithi089 ceph-mon[30928]: osdmap e79: 12 total, 12 up, 12 in 2024-09-21T03:37:22.210 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:37:21 smithi114 ceph-mon[34140]: from='client.? 172.21.15.89:0/1284324404' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-09-21T03:37:22.211 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:37:21 smithi114 ceph-mon[34140]: osdmap e79: 12 total, 12 up, 12 in 2024-09-21T03:37:22.250 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:37:21 smithi110 ceph-mon[34000]: from='client.? 172.21.15.89:0/1284324404' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-09-21T03:37:22.250 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:37:21 smithi110 ceph-mon[34000]: osdmap e79: 12 total, 12 up, 12 in 2024-09-21T03:37:22.299 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:22.288+0000 7f9674f6f640 1 -- 172.21.15.89:0/2221706873 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f96701094a0 msgr2=0x7f96701098b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:22.299 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:22.288+0000 7f9674f6f640 1 --2- 172.21.15.89:0/2221706873 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f96701094a0 0x7f96701098b0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f9658009900 tx=0x7f965802a410 comp rx=0 tx=0).stop 2024-09-21T03:37:22.299 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:22.288+0000 7f9674f6f640 1 -- 172.21.15.89:0/2221706873 shutdown_connections 2024-09-21T03:37:22.299 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:22.288+0000 7f9674f6f640 1 --2- 172.21.15.89:0/2221706873 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9670109df0 0x7f9670110620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:22.299 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:22.288+0000 7f9674f6f640 1 --2- 172.21.15.89:0/2221706873 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f96701094a0 0x7f96701098b0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:22.299 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:22.288+0000 7f9674f6f640 1 --2- 172.21.15.89:0/2221706873 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9670103b60 0x7f9670103f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:22.299 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:22.288+0000 7f9674f6f640 1 -- 172.21.15.89:0/2221706873 >> 172.21.15.89:0/2221706873 conn(0x7f967007b850 msgr2=0x7f967007bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:37:22.299 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:22.288+0000 7f9674f6f640 1 -- 172.21.15.89:0/2221706873 shutdown_connections 2024-09-21T03:37:22.299 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:22.288+0000 7f9674f6f640 1 -- 172.21.15.89:0/2221706873 wait complete. 2024-09-21T03:37:22.300 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:22.289+0000 7f9674f6f640 1 Processor -- start 2024-09-21T03:37:22.300 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:22.289+0000 7f9674f6f640 1 -- start start 2024-09-21T03:37:22.300 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:22.290+0000 7f9674f6f640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9670103b60 0x7f967019a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:22.300 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:22.290+0000 7f9674f6f640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f96701094a0 0x7f967019a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:22.300 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:22.290+0000 7f9674f6f640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9670109df0 0x7f967019fc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:22.300 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:22.290+0000 7f9674f6f640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f9670113960 con 0x7f9670109df0 2024-09-21T03:37:22.300 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:22.290+0000 7f9674f6f640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f96701137e0 con 0x7f9670103b60 2024-09-21T03:37:22.300 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:22.290+0000 7f9674f6f640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f9670113ae0 con 0x7f96701094a0 2024-09-21T03:37:22.300 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:22.290+0000 7f966ed9d640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9670109df0 0x7f967019fc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:22.300 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:22.290+0000 7f966ed9d640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9670109df0 0x7f967019fc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:37294/0 (socket says 172.21.15.89:37294) 2024-09-21T03:37:22.300 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:22.290+0000 7f966ed9d640 1 -- 172.21.15.89:0/2233450738 learned_addr learned my addr 172.21.15.89:0/2233450738 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-21T03:37:22.300 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:22.290+0000 7f9665d9b640 1 --2- 172.21.15.89:0/2233450738 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f96701094a0 0x7f967019a9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:22.300 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:22.290+0000 7f966e59c640 1 --2- 172.21.15.89:0/2233450738 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9670103b60 0x7f967019a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:22.300 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:22.290+0000 7f966ed9d640 1 -- 172.21.15.89:0/2233450738 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f96701094a0 msgr2=0x7f967019a9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:22.300 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:22.290+0000 7f966ed9d640 1 --2- 172.21.15.89:0/2233450738 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f96701094a0 0x7f967019a9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:22.301 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:22.290+0000 7f966ed9d640 1 -- 172.21.15.89:0/2233450738 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9670103b60 msgr2=0x7f967019a430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:22.301 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:22.290+0000 7f966ed9d640 1 --2- 172.21.15.89:0/2233450738 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9670103b60 0x7f967019a430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:22.301 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:22.290+0000 7f966ed9d640 1 -- 172.21.15.89:0/2233450738 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96701a03a0 con 0x7f9670109df0 2024-09-21T03:37:22.301 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:22.290+0000 7f966ed9d640 1 --2- 172.21.15.89:0/2233450738 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9670109df0 0x7f967019fc70 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f966800cad0 tx=0x7f966800cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:37:22.301 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:22.291+0000 7f9667fff640 1 -- 172.21.15.89:0/2233450738 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f966800ae20 con 0x7f9670109df0 2024-09-21T03:37:22.301 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:22.291+0000 7f9667fff640 1 -- 172.21.15.89:0/2233450738 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9668005020 con 0x7f9670109df0 2024-09-21T03:37:22.301 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:22.291+0000 7f9674f6f640 1 -- 172.21.15.89:0/2233450738 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f96701a0690 con 0x7f9670109df0 2024-09-21T03:37:22.301 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:22.291+0000 7f9667fff640 1 -- 172.21.15.89:0/2233450738 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f966800f6b0 con 0x7f9670109df0 2024-09-21T03:37:22.301 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:22.291+0000 7f9674f6f640 1 -- 172.21.15.89:0/2233450738 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96701a6df0 con 0x7f9670109df0 2024-09-21T03:37:22.301 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:22.292+0000 7f9674f6f640 1 -- 172.21.15.89:0/2233450738 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9670103f70 con 0x7f9670109df0 2024-09-21T03:37:22.301 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:22.294+0000 7f9667fff640 1 -- 172.21.15.89:0/2233450738 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 17) ==== 154512+0+0 (secure 0 0 0) 0x7f96680051d0 con 0x7f9670109df0 2024-09-21T03:37:22.301 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:22.295+0000 7f9667fff640 1 --2- 172.21.15.89:0/2233450738 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f965c0b6510 0x7f965c0b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:22.301 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:22.295+0000 7f966e59c640 1 --2- 172.21.15.89:0/2233450738 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f965c0b6510 0x7f965c0b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:22.301 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:22.295+0000 7f9667fff640 1 -- 172.21.15.89:0/2233450738 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8770+0+0 (secure 0 0 0) 0x7f96680e7940 con 0x7f9670109df0 2024-09-21T03:37:22.302 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:22.295+0000 7f966e59c640 1 --2- 172.21.15.89:0/2233450738 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f965c0b6510 0x7f965c0b8a20 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f9660005fd0 tx=0x7f9660005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:37:22.302 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:22.298+0000 7f9667fff640 1 -- 172.21.15.89:0/2233450738 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f9668014030 con 0x7f9670109df0 2024-09-21T03:37:22.518 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:22.516+0000 7f9674f6f640 1 -- 172.21.15.89:0/2233450738 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"} v 0) -- 0x7f967019b560 con 0x7f9670109df0 2024-09-21T03:37:22.922 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:22.919+0000 7f9667fff640 1 -- 172.21.15.89:0/2233450738 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]=0 added data pool 4 to fsmap v17) ==== 137+0+0 (secure 0 0 0) 0x7f967019b560 con 0x7f9670109df0 2024-09-21T03:37:22.922 INFO:teuthology.orchestra.run.smithi089.stderr:added data pool 4 to fsmap 2024-09-21T03:37:22.924 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:22.922+0000 7f9674f6f640 1 -- 172.21.15.89:0/2233450738 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f965c0b6510 msgr2=0x7f965c0b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:22.924 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:22.922+0000 7f9674f6f640 1 --2- 172.21.15.89:0/2233450738 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f965c0b6510 0x7f965c0b8a20 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f9660005fd0 tx=0x7f9660005850 comp rx=0 tx=0).stop 2024-09-21T03:37:22.924 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:22.922+0000 7f9674f6f640 1 -- 172.21.15.89:0/2233450738 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9670109df0 msgr2=0x7f967019fc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:22.924 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:22.922+0000 7f9674f6f640 1 --2- 172.21.15.89:0/2233450738 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9670109df0 0x7f967019fc70 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f966800cad0 tx=0x7f966800cf90 comp rx=0 tx=0).stop 2024-09-21T03:37:22.924 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:22.922+0000 7f9674f6f640 1 -- 172.21.15.89:0/2233450738 shutdown_connections 2024-09-21T03:37:22.924 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:22.922+0000 7f9674f6f640 1 --2- 172.21.15.89:0/2233450738 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f965c0b6510 0x7f965c0b8a20 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:22.924 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:22.922+0000 7f9674f6f640 1 --2- 172.21.15.89:0/2233450738 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9670109df0 0x7f967019fc70 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:22.924 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:22.922+0000 7f9674f6f640 1 --2- 172.21.15.89:0/2233450738 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f96701094a0 0x7f967019a9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:22.925 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:22.922+0000 7f9674f6f640 1 --2- 172.21.15.89:0/2233450738 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9670103b60 0x7f967019a430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:22.925 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:22.922+0000 7f9674f6f640 1 -- 172.21.15.89:0/2233450738 >> 172.21.15.89:0/2233450738 conn(0x7f967007b850 msgr2=0x7f96701055d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:37:22.925 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:22.922+0000 7f9674f6f640 1 -- 172.21.15.89:0/2233450738 shutdown_connections 2024-09-21T03:37:22.925 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:22.922+0000 7f9674f6f640 1 -- 172.21.15.89:0/2233450738 wait complete. 2024-09-21T03:37:23.211 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:37:22 smithi114 ceph-mon[34140]: pgmap v294: 129 pgs: 6 creating+peering, 26 unknown, 97 active+clean; 579 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 1 op/s 2024-09-21T03:37:23.211 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:37:22 smithi114 ceph-mon[34140]: from='client.? 172.21.15.89:0/2233450738' entity='client.admin' cmd={"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"} : dispatch 2024-09-21T03:37:23.250 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:37:22 smithi110 ceph-mon[34000]: pgmap v294: 129 pgs: 6 creating+peering, 26 unknown, 97 active+clean; 579 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 1 op/s 2024-09-21T03:37:23.250 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:37:22 smithi110 ceph-mon[34000]: from='client.? 172.21.15.89:0/2233450738' entity='client.admin' cmd={"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"} : dispatch 2024-09-21T03:37:23.295 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:37:22 smithi089 ceph-mon[30928]: pgmap v294: 129 pgs: 6 creating+peering, 26 unknown, 97 active+clean; 579 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 1 op/s 2024-09-21T03:37:23.295 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:37:22 smithi089 ceph-mon[30928]: from='client.? 172.21.15.89:0/2233450738' entity='client.admin' cmd={"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"} : dispatch 2024-09-21T03:37:23.624 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 7c374098-77c9-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T03:37:23.872 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/7c374098-77c9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:37:24.211 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:37:23 smithi114 ceph-mon[34140]: osdmap e80: 12 total, 12 up, 12 in 2024-09-21T03:37:24.211 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:37:23 smithi114 ceph-mon[34140]: from='client.? 172.21.15.89:0/2233450738' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2024-09-21T03:37:24.211 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:37:23 smithi114 ceph-mon[34140]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-21T03:37:24.250 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:37:23 smithi110 ceph-mon[34000]: osdmap e80: 12 total, 12 up, 12 in 2024-09-21T03:37:24.250 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:37:23 smithi110 ceph-mon[34000]: from='client.? 172.21.15.89:0/2233450738' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2024-09-21T03:37:24.250 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:37:23 smithi110 ceph-mon[34000]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-21T03:37:24.274 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:37:23 smithi089 ceph-mon[30928]: osdmap e80: 12 total, 12 up, 12 in 2024-09-21T03:37:24.275 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:37:23 smithi089 ceph-mon[30928]: from='client.? 172.21.15.89:0/2233450738' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2024-09-21T03:37:24.275 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:37:23 smithi089 ceph-mon[30928]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-21T03:37:24.375 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:24.362+0000 7f73abbf8640 1 -- 172.21.15.89:0/2657496853 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f73a4108cf0 msgr2=0x7f73a410f520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:24.375 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:24.362+0000 7f73abbf8640 1 --2- 172.21.15.89:0/2657496853 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f73a4108cf0 0x7f73a410f520 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f739c009ca0 tx=0x7f739c02a8c0 comp rx=0 tx=0).stop 2024-09-21T03:37:24.375 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:24.362+0000 7f73abbf8640 1 -- 172.21.15.89:0/2657496853 shutdown_connections 2024-09-21T03:37:24.375 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:24.362+0000 7f73abbf8640 1 --2- 172.21.15.89:0/2657496853 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f73a41083a0 0x7f73a41087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:24.376 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:24.362+0000 7f73abbf8640 1 --2- 172.21.15.89:0/2657496853 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f73a4108cf0 0x7f73a410f520 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:24.376 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:24.362+0000 7f73abbf8640 1 --2- 172.21.15.89:0/2657496853 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f73a4102a60 0x7f73a4102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:24.376 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:24.362+0000 7f73abbf8640 1 -- 172.21.15.89:0/2657496853 >> 172.21.15.89:0/2657496853 conn(0x7f73a4079730 msgr2=0x7f73a4079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:37:24.376 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:24.362+0000 7f73abbf8640 1 -- 172.21.15.89:0/2657496853 shutdown_connections 2024-09-21T03:37:24.376 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:24.362+0000 7f73abbf8640 1 -- 172.21.15.89:0/2657496853 wait complete. 2024-09-21T03:37:24.376 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:24.363+0000 7f73abbf8640 1 Processor -- start 2024-09-21T03:37:24.376 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:24.363+0000 7f73abbf8640 1 -- start start 2024-09-21T03:37:24.376 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:24.363+0000 7f73abbf8640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f73a4102a60 0x7f73a419a300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:24.376 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:24.363+0000 7f73abbf8640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f73a41083a0 0x7f73a419a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:24.376 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:24.363+0000 7f73abbf8640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f73a4108cf0 0x7f73a419fb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:24.376 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:24.363+0000 7f73abbf8640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f73a4112860 con 0x7f73a41083a0 2024-09-21T03:37:24.376 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:24.363+0000 7f73abbf8640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f73a41126e0 con 0x7f73a4102a60 2024-09-21T03:37:24.376 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:24.363+0000 7f73abbf8640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f73a41129e0 con 0x7f73a4108cf0 2024-09-21T03:37:24.376 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:24.363+0000 7f73a9193640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f73a41083a0 0x7f73a419a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:24.377 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:24.363+0000 7f73a9193640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f73a41083a0 0x7f73a419a8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:37304/0 (socket says 172.21.15.89:37304) 2024-09-21T03:37:24.377 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:24.363+0000 7f73a9193640 1 -- 172.21.15.89:0/3337435692 learned_addr learned my addr 172.21.15.89:0/3337435692 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-21T03:37:24.377 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:24.363+0000 7f73aa195640 1 --2- 172.21.15.89:0/3337435692 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f73a4108cf0 0x7f73a419fb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:24.377 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:24.363+0000 7f73a9193640 1 -- 172.21.15.89:0/3337435692 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f73a4108cf0 msgr2=0x7f73a419fb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:24.377 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:24.363+0000 7f73a9994640 1 --2- 172.21.15.89:0/3337435692 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f73a4102a60 0x7f73a419a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:24.377 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:24.363+0000 7f73a9193640 1 --2- 172.21.15.89:0/3337435692 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f73a4108cf0 0x7f73a419fb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:24.377 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:24.363+0000 7f73a9193640 1 -- 172.21.15.89:0/3337435692 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f73a4102a60 msgr2=0x7f73a419a300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:24.377 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:24.363+0000 7f73a9193640 1 --2- 172.21.15.89:0/3337435692 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f73a4102a60 0x7f73a419a300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:24.377 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:24.363+0000 7f73a9193640 1 -- 172.21.15.89:0/3337435692 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73a41a0270 con 0x7f73a41083a0 2024-09-21T03:37:24.377 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:24.363+0000 7f73aa195640 1 --2- 172.21.15.89:0/3337435692 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f73a4108cf0 0x7f73a419fb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:37:24.377 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:24.363+0000 7f73a9994640 1 --2- 172.21.15.89:0/3337435692 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f73a4102a60 0x7f73a419a300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:37:24.377 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:24.364+0000 7f73a9193640 1 --2- 172.21.15.89:0/3337435692 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f73a41083a0 0x7f73a419a8a0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f739400d770 tx=0x7f739400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:37:24.377 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:24.364+0000 7f739affd640 1 -- 172.21.15.89:0/3337435692 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7394017070 con 0x7f73a41083a0 2024-09-21T03:37:24.377 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:24.364+0000 7f73abbf8640 1 -- 172.21.15.89:0/3337435692 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f73a41a0560 con 0x7f73a41083a0 2024-09-21T03:37:24.378 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:24.364+0000 7f739affd640 1 -- 172.21.15.89:0/3337435692 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7394009c30 con 0x7f73a41083a0 2024-09-21T03:37:24.378 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:24.364+0000 7f739affd640 1 -- 172.21.15.89:0/3337435692 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f739400f620 con 0x7f73a41083a0 2024-09-21T03:37:24.378 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:24.364+0000 7f73abbf8640 1 -- 172.21.15.89:0/3337435692 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f73a41a6cc0 con 0x7f73a41083a0 2024-09-21T03:37:24.378 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:24.365+0000 7f73abbf8640 1 -- 172.21.15.89:0/3337435692 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f73a4102e70 con 0x7f73a41083a0 2024-09-21T03:37:24.378 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:24.366+0000 7f739affd640 1 -- 172.21.15.89:0/3337435692 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 17) ==== 154512+0+0 (secure 0 0 0) 0x7f739401a070 con 0x7f73a41083a0 2024-09-21T03:37:24.378 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:24.367+0000 7f739affd640 1 --2- 172.21.15.89:0/3337435692 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f73900b6510 0x7f73900b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:24.378 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:24.367+0000 7f73a9994640 1 --2- 172.21.15.89:0/3337435692 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f73900b6510 0x7f73900b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:24.378 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:24.367+0000 7f739affd640 1 -- 172.21.15.89:0/3337435692 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8802+0+0 (secure 0 0 0) 0x7f73940e70f0 con 0x7f73a41083a0 2024-09-21T03:37:24.378 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:24.367+0000 7f73a9994640 1 --2- 172.21.15.89:0/3337435692 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f73900b6510 0x7f73900b8a20 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f73a0009b40 tx=0x7f73a0005e50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:37:24.378 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:24.370+0000 7f739affd640 1 -- 172.21.15.89:0/3337435692 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f7394009de0 con 0x7f73a41083a0 2024-09-21T03:37:24.586 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:24.583+0000 7f73abbf8640 1 -- 172.21.15.89:0/3337435692 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f73a402e640 con 0x7f73a41083a0 2024-09-21T03:37:24.586 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:24.584+0000 7f739affd640 1 -- 172.21.15.89:0/3337435692 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+11839 (secure 0 0 0) 0x7f73940ad740 con 0x7f73a41083a0 2024-09-21T03:37:24.587 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-09-21T03:37:24.587 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":17,"btime":"2024-09-21T03:37:22:908380+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14721,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6835/2341538514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":2341538514},{"type":"v1","addr":"172.21.15.89:6835","nonce":2341538514}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14733,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6837/146708592","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6836","nonce":146708592},{"type":"v1","addr":"172.21.15.89:6837","nonce":146708592}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14781,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6839/2269965499","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6838","nonce":2269965499},{"type":"v1","addr":"172.21.15.89:6839","nonce":2269965499}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14793,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6841/650850383","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6840","nonce":650850383},{"type":"v1","addr":"172.21.15.89:6841","nonce":650850383}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24565,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6833/752336912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6832","nonce":752336912},{"type":"v1","addr":"172.21.15.114:6833","nonce":752336912}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24575,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6833/127607331","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6832","nonce":127607331},{"type":"v1","addr":"172.21.15.110:6833","nonce":127607331}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24593,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6835/671820654","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6834","nonce":671820654},{"type":"v1","addr":"172.21.15.110:6835","nonce":671820654}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24604,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6835/1049714548","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6834","nonce":1049714548},{"type":"v1","addr":"172.21.15.114:6835","nonce":1049714548}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24610,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6837/1941213818","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6836","nonce":1941213818},{"type":"v1","addr":"172.21.15.114:6837","nonce":1941213818}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24614,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6837/4047124589","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6836","nonce":4047124589},{"type":"v1","addr":"172.21.15.110:6837","nonce":4047124589}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24631,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6839/3427369944","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6838","nonce":3427369944},{"type":"v1","addr":"172.21.15.114:6839","nonce":3427369944}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T03:37:13.570694+0000","modified":"2024-09-21T03:37:22.518535+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24638},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24638":{"gid":24638,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.110:6839/132366709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6838","nonce":132366709},{"type":"v1","addr":"172.21.15.110:6839","nonce":132366709}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":24638,"qdb_cluster":[24638]},"id":1}]} 2024-09-21T03:37:24.587 INFO:teuthology.orchestra.run.smithi089.stderr:dumped fsmap epoch 17 2024-09-21T03:37:24.589 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:24.586+0000 7f73abbf8640 1 -- 172.21.15.89:0/3337435692 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f73900b6510 msgr2=0x7f73900b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:24.589 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:24.586+0000 7f73abbf8640 1 --2- 172.21.15.89:0/3337435692 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f73900b6510 0x7f73900b8a20 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f73a0009b40 tx=0x7f73a0005e50 comp rx=0 tx=0).stop 2024-09-21T03:37:24.589 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:24.586+0000 7f73abbf8640 1 -- 172.21.15.89:0/3337435692 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f73a41083a0 msgr2=0x7f73a419a8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:24.589 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:24.586+0000 7f73abbf8640 1 --2- 172.21.15.89:0/3337435692 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f73a41083a0 0x7f73a419a8a0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f739400d770 tx=0x7f739400dc30 comp rx=0 tx=0).stop 2024-09-21T03:37:24.589 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:24.586+0000 7f73abbf8640 1 -- 172.21.15.89:0/3337435692 shutdown_connections 2024-09-21T03:37:24.589 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:24.586+0000 7f73abbf8640 1 --2- 172.21.15.89:0/3337435692 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f73900b6510 0x7f73900b8a20 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:24.589 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:24.586+0000 7f73abbf8640 1 --2- 172.21.15.89:0/3337435692 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f73a4108cf0 0x7f73a419fb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:24.589 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:24.586+0000 7f73abbf8640 1 --2- 172.21.15.89:0/3337435692 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f73a41083a0 0x7f73a419a8a0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:24.589 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:24.586+0000 7f73abbf8640 1 --2- 172.21.15.89:0/3337435692 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f73a4102a60 0x7f73a419a300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:24.589 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:24.586+0000 7f73abbf8640 1 -- 172.21.15.89:0/3337435692 >> 172.21.15.89:0/3337435692 conn(0x7f73a4079730 msgr2=0x7f73a406b1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:37:24.589 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:24.587+0000 7f73abbf8640 1 -- 172.21.15.89:0/3337435692 shutdown_connections 2024-09-21T03:37:24.589 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:24.587+0000 7f73abbf8640 1 -- 172.21.15.89:0/3337435692 wait complete. 2024-09-21T03:37:25.188 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:37:24 smithi089 ceph-mon[30928]: pgmap v296: 129 pgs: 6 creating+peering, 123 active+clean; 579 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:37:25.188 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:37:24 smithi089 ceph-mon[30928]: from='client.? 172.21.15.89:0/3337435692' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:37:25.210 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:37:24 smithi114 ceph-mon[34140]: pgmap v296: 129 pgs: 6 creating+peering, 123 active+clean; 579 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:37:25.211 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:37:24 smithi114 ceph-mon[34140]: from='client.? 172.21.15.89:0/3337435692' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:37:25.217 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 7c374098-77c9-11ef-baf5-efdc52797490 -- ceph osd dump --format=json 2024-09-21T03:37:25.250 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:37:24 smithi110 ceph-mon[34000]: pgmap v296: 129 pgs: 6 creating+peering, 123 active+clean; 579 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:37:25.250 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:37:24 smithi110 ceph-mon[34000]: from='client.? 172.21.15.89:0/3337435692' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:37:25.463 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/7c374098-77c9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:37:25.959 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:25.946+0000 7fda35ece640 1 -- 172.21.15.89:0/4148950881 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fda301083c0 msgr2=0x7fda301087d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:25.959 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:25.946+0000 7fda35ece640 1 --2- 172.21.15.89:0/4148950881 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fda301083c0 0x7fda301087d0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fda18009900 tx=0x7fda1802a410 comp rx=0 tx=0).stop 2024-09-21T03:37:25.959 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:25.946+0000 7fda35ece640 1 -- 172.21.15.89:0/4148950881 shutdown_connections 2024-09-21T03:37:25.959 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:25.946+0000 7fda35ece640 1 --2- 172.21.15.89:0/4148950881 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fda30108d10 0x7fda3010f540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:25.959 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:25.946+0000 7fda35ece640 1 --2- 172.21.15.89:0/4148950881 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fda301083c0 0x7fda301087d0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:25.960 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:25.946+0000 7fda35ece640 1 --2- 172.21.15.89:0/4148950881 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fda30102a80 0x7fda30102e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:25.960 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:25.946+0000 7fda35ece640 1 -- 172.21.15.89:0/4148950881 >> 172.21.15.89:0/4148950881 conn(0x7fda30079740 msgr2=0x7fda30079b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:37:25.960 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:25.946+0000 7fda35ece640 1 -- 172.21.15.89:0/4148950881 shutdown_connections 2024-09-21T03:37:25.960 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:25.946+0000 7fda35ece640 1 -- 172.21.15.89:0/4148950881 wait complete. 2024-09-21T03:37:25.960 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:25.947+0000 7fda35ece640 1 Processor -- start 2024-09-21T03:37:25.960 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:25.947+0000 7fda35ece640 1 -- start start 2024-09-21T03:37:25.960 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:25.947+0000 7fda35ece640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fda30102a80 0x7fda3019b200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:25.960 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:25.947+0000 7fda35ece640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fda301083c0 0x7fda3019b740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:25.960 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:25.947+0000 7fda35ece640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fda30108d10 0x7fda3019bc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:25.960 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:25.947+0000 7fda35ece640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fda30112830 con 0x7fda301083c0 2024-09-21T03:37:25.960 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:25.947+0000 7fda35ece640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fda301126b0 con 0x7fda30102a80 2024-09-21T03:37:25.960 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:25.947+0000 7fda35ece640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fda301129b0 con 0x7fda30108d10 2024-09-21T03:37:25.960 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:25.947+0000 7fda26ffd640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fda301083c0 0x7fda3019b740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:25.960 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:25.947+0000 7fda26ffd640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fda301083c0 0x7fda3019b740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:37324/0 (socket says 172.21.15.89:37324) 2024-09-21T03:37:25.960 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:25.947+0000 7fda26ffd640 1 -- 172.21.15.89:0/353324172 learned_addr learned my addr 172.21.15.89:0/353324172 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-21T03:37:25.960 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:25.947+0000 7fda2f7fe640 1 --2- 172.21.15.89:0/353324172 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fda30102a80 0x7fda3019b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:25.961 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:25.947+0000 7fda2ffff640 1 --2- 172.21.15.89:0/353324172 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fda30108d10 0x7fda3019bc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:25.961 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:25.948+0000 7fda26ffd640 1 -- 172.21.15.89:0/353324172 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fda30108d10 msgr2=0x7fda3019bc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:25.961 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:25.948+0000 7fda26ffd640 1 --2- 172.21.15.89:0/353324172 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fda30108d10 0x7fda3019bc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:25.961 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:25.948+0000 7fda26ffd640 1 -- 172.21.15.89:0/353324172 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fda30102a80 msgr2=0x7fda3019b200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:25.961 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:25.948+0000 7fda26ffd640 1 --2- 172.21.15.89:0/353324172 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fda30102a80 0x7fda3019b200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:25.961 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:25.948+0000 7fda26ffd640 1 -- 172.21.15.89:0/353324172 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda301a6490 con 0x7fda301083c0 2024-09-21T03:37:25.961 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:25.948+0000 7fda26ffd640 1 --2- 172.21.15.89:0/353324172 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fda301083c0 0x7fda3019b740 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fda18005940 tx=0x7fda180042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:37:25.961 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:25.948+0000 7fda2d7fa640 1 -- 172.21.15.89:0/353324172 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda1803f030 con 0x7fda301083c0 2024-09-21T03:37:25.961 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:25.948+0000 7fda2d7fa640 1 -- 172.21.15.89:0/353324172 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fda18003c90 con 0x7fda301083c0 2024-09-21T03:37:25.961 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:25.948+0000 7fda35ece640 1 -- 172.21.15.89:0/353324172 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fda301a6720 con 0x7fda301083c0 2024-09-21T03:37:25.961 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:25.948+0000 7fda2d7fa640 1 -- 172.21.15.89:0/353324172 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda18035070 con 0x7fda301083c0 2024-09-21T03:37:25.961 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:25.948+0000 7fda35ece640 1 -- 172.21.15.89:0/353324172 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fda301a6c00 con 0x7fda301083c0 2024-09-21T03:37:25.961 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:25.949+0000 7fda35ece640 1 -- 172.21.15.89:0/353324172 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fda30103d00 con 0x7fda301083c0 2024-09-21T03:37:25.961 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:25.951+0000 7fda2d7fa640 1 -- 172.21.15.89:0/353324172 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 17) ==== 154512+0+0 (secure 0 0 0) 0x7fda18042030 con 0x7fda301083c0 2024-09-21T03:37:25.961 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:25.951+0000 7fda2d7fa640 1 --2- 172.21.15.89:0/353324172 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7fda1c0b6540 0x7fda1c0b8a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:25.962 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:25.952+0000 7fda2f7fe640 1 --2- 172.21.15.89:0/353324172 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7fda1c0b6540 0x7fda1c0b8a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:25.962 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:25.952+0000 7fda2d7fa640 1 -- 172.21.15.89:0/353324172 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8802+0+0 (secure 0 0 0) 0x7fda18107d60 con 0x7fda301083c0 2024-09-21T03:37:25.962 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:25.952+0000 7fda2f7fe640 1 --2- 172.21.15.89:0/353324172 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7fda1c0b6540 0x7fda1c0b8a50 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fda3006c6a0 tx=0x7fda2000b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:37:25.962 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:25.955+0000 7fda2d7fa640 1 -- 172.21.15.89:0/353324172 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fda180d2790 con 0x7fda301083c0 2024-09-21T03:37:26.134 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:26.132+0000 7fda35ece640 1 -- 172.21.15.89:0/353324172 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fda3006bf50 con 0x7fda301083c0 2024-09-21T03:37:26.136 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:26.134+0000 7fda2d7fa640 1 -- 172.21.15.89:0/353324172 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v80) ==== 74+0+24236 (secure 0 0 0) 0x7fda3006bf50 con 0x7fda301083c0 2024-09-21T03:37:26.136 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-09-21T03:37:26.137 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":80,"fsid":"7c374098-77c9-11ef-baf5-efdc52797490","created":"2024-09-21T03:28:56.681458+0000","modified":"2024-09-21T03:37:22.518390+0000","last_up_change":"2024-09-21T03:36:23.368379+0000","last_in_change":"2024-09-21T03:36:00.131331+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T03:32:37.834339+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-21T03:37:09.443341+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"74","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-21T03:37:11.467821+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"74","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-21T03:37:17.884066+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"27ca5ed8-17d1-4f8e-a7ae-084a914721a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6802","nonce":990539042},{"type":"v1","addr":"172.21.15.89:6803","nonce":990539042}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6804","nonce":990539042},{"type":"v1","addr":"172.21.15.89:6805","nonce":990539042}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6808","nonce":990539042},{"type":"v1","addr":"172.21.15.89:6809","nonce":990539042}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6806","nonce":990539042},{"type":"v1","addr":"172.21.15.89:6807","nonce":990539042}]},"public_addr":"172.21.15.89:6803/990539042","cluster_addr":"172.21.15.89:6805/990539042","heartbeat_back_addr":"172.21.15.89:6809/990539042","heartbeat_front_addr":"172.21.15.89:6807/990539042","state":["exists","up"]},{"osd":1,"uuid":"5176a910-5cf9-49d7-aedb-a20cf36ea075","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6800","nonce":3235852628},{"type":"v1","addr":"172.21.15.110:6801","nonce":3235852628}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6802","nonce":3235852628},{"type":"v1","addr":"172.21.15.110:6803","nonce":3235852628}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6806","nonce":3235852628},{"type":"v1","addr":"172.21.15.110:6807","nonce":3235852628}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6804","nonce":3235852628},{"type":"v1","addr":"172.21.15.110:6805","nonce":3235852628}]},"public_addr":"172.21.15.110:6801/3235852628","cluster_addr":"172.21.15.110:6803/3235852628","heartbeat_back_addr":"172.21.15.110:6807/3235852628","heartbeat_front_addr":"172.21.15.110:6805/3235852628","state":["exists","up"]},{"osd":2,"uuid":"d4977df1-b118-4d7e-b9a9-626a8accc812","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6800","nonce":169452122},{"type":"v1","addr":"172.21.15.114:6801","nonce":169452122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6802","nonce":169452122},{"type":"v1","addr":"172.21.15.114:6803","nonce":169452122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6806","nonce":169452122},{"type":"v1","addr":"172.21.15.114:6807","nonce":169452122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6804","nonce":169452122},{"type":"v1","addr":"172.21.15.114:6805","nonce":169452122}]},"public_addr":"172.21.15.114:6801/169452122","cluster_addr":"172.21.15.114:6803/169452122","heartbeat_back_addr":"172.21.15.114:6807/169452122","heartbeat_front_addr":"172.21.15.114:6805/169452122","state":["exists","up"]},{"osd":3,"uuid":"2c7ae1b4-0bf2-4954-81ed-55d03eab82c6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6810","nonce":3016980877},{"type":"v1","addr":"172.21.15.89:6811","nonce":3016980877}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6812","nonce":3016980877},{"type":"v1","addr":"172.21.15.89:6813","nonce":3016980877}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6816","nonce":3016980877},{"type":"v1","addr":"172.21.15.89:6817","nonce":3016980877}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6814","nonce":3016980877},{"type":"v1","addr":"172.21.15.89:6815","nonce":3016980877}]},"public_addr":"172.21.15.89:6811/3016980877","cluster_addr":"172.21.15.89:6813/3016980877","heartbeat_back_addr":"172.21.15.89:6817/3016980877","heartbeat_front_addr":"172.21.15.89:6815/3016980877","state":["exists","up"]},{"osd":4,"uuid":"8e65399d-6e55-4da4-98ba-f805f0c655a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":30,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6808","nonce":3217880476},{"type":"v1","addr":"172.21.15.110:6809","nonce":3217880476}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6810","nonce":3217880476},{"type":"v1","addr":"172.21.15.110:6811","nonce":3217880476}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6814","nonce":3217880476},{"type":"v1","addr":"172.21.15.110:6815","nonce":3217880476}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6812","nonce":3217880476},{"type":"v1","addr":"172.21.15.110:6813","nonce":3217880476}]},"public_addr":"172.21.15.110:6809/3217880476","cluster_addr":"172.21.15.110:6811/3217880476","heartbeat_back_addr":"172.21.15.110:6815/3217880476","heartbeat_front_addr":"172.21.15.110:6813/3217880476","state":["exists","up"]},{"osd":5,"uuid":"b9ec2889-46bd-40e5-8446-73ab28271c67","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6808","nonce":1144140707},{"type":"v1","addr":"172.21.15.114:6809","nonce":1144140707}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6810","nonce":1144140707},{"type":"v1","addr":"172.21.15.114:6811","nonce":1144140707}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6814","nonce":1144140707},{"type":"v1","addr":"172.21.15.114:6815","nonce":1144140707}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6812","nonce":1144140707},{"type":"v1","addr":"172.21.15.114:6813","nonce":1144140707}]},"public_addr":"172.21.15.114:6809/1144140707","cluster_addr":"172.21.15.114:6811/1144140707","heartbeat_back_addr":"172.21.15.114:6815/1144140707","heartbeat_front_addr":"172.21.15.114:6813/1144140707","state":["exists","up"]},{"osd":6,"uuid":"9b79989e-adb0-42b2-b2f6-a2dce56b66e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":41,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6818","nonce":1522586255},{"type":"v1","addr":"172.21.15.89:6819","nonce":1522586255}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6820","nonce":1522586255},{"type":"v1","addr":"172.21.15.89:6821","nonce":1522586255}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6824","nonce":1522586255},{"type":"v1","addr":"172.21.15.89:6825","nonce":1522586255}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6822","nonce":1522586255},{"type":"v1","addr":"172.21.15.89:6823","nonce":1522586255}]},"public_addr":"172.21.15.89:6819/1522586255","cluster_addr":"172.21.15.89:6821/1522586255","heartbeat_back_addr":"172.21.15.89:6825/1522586255","heartbeat_front_addr":"172.21.15.89:6823/1522586255","state":["exists","up"]},{"osd":7,"uuid":"b381821f-141c-47a1-be65-f57ca730391b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6816","nonce":1331818323},{"type":"v1","addr":"172.21.15.110:6817","nonce":1331818323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6818","nonce":1331818323},{"type":"v1","addr":"172.21.15.110:6819","nonce":1331818323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6822","nonce":1331818323},{"type":"v1","addr":"172.21.15.110:6823","nonce":1331818323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6820","nonce":1331818323},{"type":"v1","addr":"172.21.15.110:6821","nonce":1331818323}]},"public_addr":"172.21.15.110:6817/1331818323","cluster_addr":"172.21.15.110:6819/1331818323","heartbeat_back_addr":"172.21.15.110:6823/1331818323","heartbeat_front_addr":"172.21.15.110:6821/1331818323","state":["exists","up"]},{"osd":8,"uuid":"1bc2a5b4-15f3-456f-9d67-e51307a27bcc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6816","nonce":2138556699},{"type":"v1","addr":"172.21.15.114:6817","nonce":2138556699}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6818","nonce":2138556699},{"type":"v1","addr":"172.21.15.114:6819","nonce":2138556699}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6822","nonce":2138556699},{"type":"v1","addr":"172.21.15.114:6823","nonce":2138556699}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6820","nonce":2138556699},{"type":"v1","addr":"172.21.15.114:6821","nonce":2138556699}]},"public_addr":"172.21.15.114:6817/2138556699","cluster_addr":"172.21.15.114:6819/2138556699","heartbeat_back_addr":"172.21.15.114:6823/2138556699","heartbeat_front_addr":"172.21.15.114:6821/2138556699","state":["exists","up"]},{"osd":9,"uuid":"91f5fd29-d4a7-4800-ac7a-e83b13add89e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6826","nonce":1730253068},{"type":"v1","addr":"172.21.15.89:6827","nonce":1730253068}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6828","nonce":1730253068},{"type":"v1","addr":"172.21.15.89:6829","nonce":1730253068}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6832","nonce":1730253068},{"type":"v1","addr":"172.21.15.89:6833","nonce":1730253068}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6830","nonce":1730253068},{"type":"v1","addr":"172.21.15.89:6831","nonce":1730253068}]},"public_addr":"172.21.15.89:6827/1730253068","cluster_addr":"172.21.15.89:6829/1730253068","heartbeat_back_addr":"172.21.15.89:6833/1730253068","heartbeat_front_addr":"172.21.15.89:6831/1730253068","state":["exists","up"]},{"osd":10,"uuid":"01390499-2a4d-4188-b57a-fcb10c668e2c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6824","nonce":2904016117},{"type":"v1","addr":"172.21.15.110:6825","nonce":2904016117}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6826","nonce":2904016117},{"type":"v1","addr":"172.21.15.110:6827","nonce":2904016117}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6830","nonce":2904016117},{"type":"v1","addr":"172.21.15.110:6831","nonce":2904016117}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6828","nonce":2904016117},{"type":"v1","addr":"172.21.15.110:6829","nonce":2904016117}]},"public_addr":"172.21.15.110:6825/2904016117","cluster_addr":"172.21.15.110:6827/2904016117","heartbeat_back_addr":"172.21.15.110:6831/2904016117","heartbeat_front_addr":"172.21.15.110:6829/2904016117","state":["exists","up"]},{"osd":11,"uuid":"c3c0f38a-c3d3-415d-a9f8-875d5b756e35","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6824","nonce":3749854486},{"type":"v1","addr":"172.21.15.114:6825","nonce":3749854486}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6826","nonce":3749854486},{"type":"v1","addr":"172.21.15.114:6827","nonce":3749854486}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6830","nonce":3749854486},{"type":"v1","addr":"172.21.15.114:6831","nonce":3749854486}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6828","nonce":3749854486},{"type":"v1","addr":"172.21.15.114:6829","nonce":3749854486}]},"public_addr":"172.21.15.114:6825/3749854486","cluster_addr":"172.21.15.114:6827/3749854486","heartbeat_back_addr":"172.21.15.114:6831/3749854486","heartbeat_front_addr":"172.21.15.114:6829/3749854486","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:31:46.218908+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:32:10.338230+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:32:34.830596+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:33:00.816102+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:33:25.223838+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:33:50.188230+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:34:16.564255+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:34:41.498230+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:35:05.826322+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:35:31.234450+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:35:56.474373+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:36:21.741254+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.89:6801/655110910":"2024-09-22T03:29:51.519106+0000","172.21.15.89:0/1618544983":"2024-09-22T03:29:51.519106+0000","172.21.15.89:0/1265502758":"2024-09-22T03:29:51.519106+0000","172.21.15.89:0/4017684419":"2024-09-22T03:29:51.519106+0000","172.21.15.89:6801/3312712420":"2024-09-22T03:29:26.004686+0000","172.21.15.89:6800/3312712420":"2024-09-22T03:29:26.004686+0000","172.21.15.89:0/2056576483":"2024-09-22T03:29:26.004686+0000","172.21.15.89:6800/655110910":"2024-09-22T03:29:51.519106+0000","172.21.15.89:0/2382866302":"2024-09-22T03:29:26.004686+0000","172.21.15.89:0/1584351801":"2024-09-22T03:29:26.004686+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T03:37:26.139 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:26.136+0000 7fda35ece640 1 -- 172.21.15.89:0/353324172 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7fda1c0b6540 msgr2=0x7fda1c0b8a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:26.139 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:26.137+0000 7fda35ece640 1 --2- 172.21.15.89:0/353324172 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7fda1c0b6540 0x7fda1c0b8a50 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fda3006c6a0 tx=0x7fda2000b410 comp rx=0 tx=0).stop 2024-09-21T03:37:26.139 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:26.137+0000 7fda35ece640 1 -- 172.21.15.89:0/353324172 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fda301083c0 msgr2=0x7fda3019b740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:26.139 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:26.137+0000 7fda35ece640 1 --2- 172.21.15.89:0/353324172 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fda301083c0 0x7fda3019b740 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fda18005940 tx=0x7fda180042f0 comp rx=0 tx=0).stop 2024-09-21T03:37:26.139 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:26.137+0000 7fda35ece640 1 -- 172.21.15.89:0/353324172 shutdown_connections 2024-09-21T03:37:26.139 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:26.137+0000 7fda35ece640 1 --2- 172.21.15.89:0/353324172 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7fda1c0b6540 0x7fda1c0b8a50 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:26.139 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:26.137+0000 7fda35ece640 1 --2- 172.21.15.89:0/353324172 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fda30108d10 0x7fda3019bc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:26.139 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:26.137+0000 7fda35ece640 1 --2- 172.21.15.89:0/353324172 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fda301083c0 0x7fda3019b740 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:26.139 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:26.137+0000 7fda35ece640 1 --2- 172.21.15.89:0/353324172 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fda30102a80 0x7fda3019b200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:26.139 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:26.137+0000 7fda35ece640 1 -- 172.21.15.89:0/353324172 >> 172.21.15.89:0/353324172 conn(0x7fda30079740 msgr2=0x7fda301044a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:37:26.140 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:26.137+0000 7fda35ece640 1 -- 172.21.15.89:0/353324172 shutdown_connections 2024-09-21T03:37:26.140 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:26.137+0000 7fda35ece640 1 -- 172.21.15.89:0/353324172 wait complete. 2024-09-21T03:37:26.851 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 7c374098-77c9-11ef-baf5-efdc52797490 -- ceph osd dump --format=json 2024-09-21T03:37:27.100 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/7c374098-77c9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:37:27.211 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:37:26 smithi114 ceph-mon[34140]: pgmap v297: 129 pgs: 129 active+clean; 579 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:37:27.211 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:37:26 smithi114 ceph-mon[34140]: from='client.? 172.21.15.89:0/353324172' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T03:37:27.220 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:37:26 smithi089 ceph-mon[30928]: pgmap v297: 129 pgs: 129 active+clean; 579 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:37:27.220 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:37:26 smithi089 ceph-mon[30928]: from='client.? 172.21.15.89:0/353324172' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T03:37:27.250 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:37:26 smithi110 ceph-mon[34000]: pgmap v297: 129 pgs: 129 active+clean; 579 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:37:27.250 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:37:26 smithi110 ceph-mon[34000]: from='client.? 172.21.15.89:0/353324172' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T03:37:27.594 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:27.580+0000 7fc984816640 1 -- 172.21.15.89:0/500897969 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc9801083a0 msgr2=0x7fc9801087b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:27.594 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:27.580+0000 7fc984816640 1 --2- 172.21.15.89:0/500897969 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc9801083a0 0x7fc9801087b0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fc974009900 tx=0x7fc97402a410 comp rx=0 tx=0).stop 2024-09-21T03:37:27.594 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:27.580+0000 7fc984816640 1 -- 172.21.15.89:0/500897969 shutdown_connections 2024-09-21T03:37:27.594 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:27.580+0000 7fc984816640 1 --2- 172.21.15.89:0/500897969 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc980108cf0 0x7fc98010f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:27.594 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:27.580+0000 7fc984816640 1 --2- 172.21.15.89:0/500897969 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc9801083a0 0x7fc9801087b0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:27.594 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:27.580+0000 7fc984816640 1 --2- 172.21.15.89:0/500897969 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc980102a60 0x7fc980102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:27.594 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:27.580+0000 7fc984816640 1 -- 172.21.15.89:0/500897969 >> 172.21.15.89:0/500897969 conn(0x7fc980079730 msgr2=0x7fc980079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:37:27.594 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:27.580+0000 7fc984816640 1 -- 172.21.15.89:0/500897969 shutdown_connections 2024-09-21T03:37:27.594 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:27.580+0000 7fc984816640 1 -- 172.21.15.89:0/500897969 wait complete. 2024-09-21T03:37:27.595 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:27.581+0000 7fc984816640 1 Processor -- start 2024-09-21T03:37:27.595 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:27.581+0000 7fc984816640 1 -- start start 2024-09-21T03:37:27.595 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:27.581+0000 7fc984816640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc980102a60 0x7fc98019a420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:27.595 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:27.582+0000 7fc984816640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc9801083a0 0x7fc98019a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:27.595 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:27.582+0000 7fc984816640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc980108cf0 0x7fc98019fc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:27.595 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:27.582+0000 7fc984816640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fc9801128a0 con 0x7fc9801083a0 2024-09-21T03:37:27.595 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:27.582+0000 7fc984816640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fc980112720 con 0x7fc980102a60 2024-09-21T03:37:27.595 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:27.582+0000 7fc984816640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fc980112a20 con 0x7fc980108cf0 2024-09-21T03:37:27.595 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:27.582+0000 7fc97dd9b640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc9801083a0 0x7fc98019a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:27.595 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:27.582+0000 7fc97dd9b640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc9801083a0 0x7fc98019a9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:37344/0 (socket says 172.21.15.89:37344) 2024-09-21T03:37:27.595 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:27.582+0000 7fc97dd9b640 1 -- 172.21.15.89:0/1068324675 learned_addr learned my addr 172.21.15.89:0/1068324675 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-21T03:37:27.595 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:27.582+0000 7fc97e59c640 1 --2- 172.21.15.89:0/1068324675 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc980102a60 0x7fc98019a420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:27.595 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:27.582+0000 7fc97ed9d640 1 --2- 172.21.15.89:0/1068324675 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc980108cf0 0x7fc98019fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:27.595 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:27.582+0000 7fc97dd9b640 1 -- 172.21.15.89:0/1068324675 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc980108cf0 msgr2=0x7fc98019fc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:27.595 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:27.582+0000 7fc97dd9b640 1 --2- 172.21.15.89:0/1068324675 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc980108cf0 0x7fc98019fc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:27.596 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:27.582+0000 7fc97dd9b640 1 -- 172.21.15.89:0/1068324675 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc980102a60 msgr2=0x7fc98019a420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:27.596 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:27.582+0000 7fc97dd9b640 1 --2- 172.21.15.89:0/1068324675 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc980102a60 0x7fc98019a420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:27.596 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:27.582+0000 7fc97dd9b640 1 -- 172.21.15.89:0/1068324675 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9801a0390 con 0x7fc9801083a0 2024-09-21T03:37:27.596 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:27.582+0000 7fc97ed9d640 1 --2- 172.21.15.89:0/1068324675 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc980108cf0 0x7fc98019fc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:37:27.596 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:27.582+0000 7fc97dd9b640 1 --2- 172.21.15.89:0/1068324675 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc9801083a0 0x7fc98019a9c0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fc97402a930 tx=0x7fc974003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:37:27.596 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:27.582+0000 7fc96f7fe640 1 -- 172.21.15.89:0/1068324675 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc97403f070 con 0x7fc9801083a0 2024-09-21T03:37:27.596 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:27.582+0000 7fc984816640 1 -- 172.21.15.89:0/1068324675 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc9801a0620 con 0x7fc9801083a0 2024-09-21T03:37:27.596 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:27.582+0000 7fc96f7fe640 1 -- 172.21.15.89:0/1068324675 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc974003e10 con 0x7fc9801083a0 2024-09-21T03:37:27.596 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:27.582+0000 7fc96f7fe640 1 -- 172.21.15.89:0/1068324675 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc974035030 con 0x7fc9801083a0 2024-09-21T03:37:27.596 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:27.582+0000 7fc984816640 1 -- 172.21.15.89:0/1068324675 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9801a6de0 con 0x7fc9801083a0 2024-09-21T03:37:27.596 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:27.584+0000 7fc984816640 1 -- 172.21.15.89:0/1068324675 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc980103ce0 con 0x7fc9801083a0 2024-09-21T03:37:27.596 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:27.585+0000 7fc96f7fe640 1 -- 172.21.15.89:0/1068324675 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 17) ==== 154512+0+0 (secure 0 0 0) 0x7fc974042070 con 0x7fc9801083a0 2024-09-21T03:37:27.596 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:27.586+0000 7fc96f7fe640 1 --2- 172.21.15.89:0/1068324675 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7fc9680b6510 0x7fc9680b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:27.597 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:27.586+0000 7fc97e59c640 1 --2- 172.21.15.89:0/1068324675 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7fc9680b6510 0x7fc9680b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:27.597 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:27.586+0000 7fc96f7fe640 1 -- 172.21.15.89:0/1068324675 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8802+0+0 (secure 0 0 0) 0x7fc974108100 con 0x7fc9801083a0 2024-09-21T03:37:27.597 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:27.587+0000 7fc97e59c640 1 --2- 172.21.15.89:0/1068324675 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7fc9680b6510 0x7fc9680b8a20 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fc97000bbe0 tx=0x7fc97000b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:37:27.597 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:27.589+0000 7fc96f7fe640 1 -- 172.21.15.89:0/1068324675 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fc9740ce750 con 0x7fc9801083a0 2024-09-21T03:37:27.773 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:27.770+0000 7fc984816640 1 -- 172.21.15.89:0/1068324675 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc980102e70 con 0x7fc9801083a0 2024-09-21T03:37:27.774 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:27.772+0000 7fc96f7fe640 1 -- 172.21.15.89:0/1068324675 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v80) ==== 74+0+24236 (secure 0 0 0) 0x7fc9740d2b30 con 0x7fc9801083a0 2024-09-21T03:37:27.775 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-09-21T03:37:27.775 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":80,"fsid":"7c374098-77c9-11ef-baf5-efdc52797490","created":"2024-09-21T03:28:56.681458+0000","modified":"2024-09-21T03:37:22.518390+0000","last_up_change":"2024-09-21T03:36:23.368379+0000","last_in_change":"2024-09-21T03:36:00.131331+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T03:32:37.834339+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-21T03:37:09.443341+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"74","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-21T03:37:11.467821+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"74","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-21T03:37:17.884066+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"27ca5ed8-17d1-4f8e-a7ae-084a914721a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6802","nonce":990539042},{"type":"v1","addr":"172.21.15.89:6803","nonce":990539042}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6804","nonce":990539042},{"type":"v1","addr":"172.21.15.89:6805","nonce":990539042}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6808","nonce":990539042},{"type":"v1","addr":"172.21.15.89:6809","nonce":990539042}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6806","nonce":990539042},{"type":"v1","addr":"172.21.15.89:6807","nonce":990539042}]},"public_addr":"172.21.15.89:6803/990539042","cluster_addr":"172.21.15.89:6805/990539042","heartbeat_back_addr":"172.21.15.89:6809/990539042","heartbeat_front_addr":"172.21.15.89:6807/990539042","state":["exists","up"]},{"osd":1,"uuid":"5176a910-5cf9-49d7-aedb-a20cf36ea075","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6800","nonce":3235852628},{"type":"v1","addr":"172.21.15.110:6801","nonce":3235852628}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6802","nonce":3235852628},{"type":"v1","addr":"172.21.15.110:6803","nonce":3235852628}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6806","nonce":3235852628},{"type":"v1","addr":"172.21.15.110:6807","nonce":3235852628}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6804","nonce":3235852628},{"type":"v1","addr":"172.21.15.110:6805","nonce":3235852628}]},"public_addr":"172.21.15.110:6801/3235852628","cluster_addr":"172.21.15.110:6803/3235852628","heartbeat_back_addr":"172.21.15.110:6807/3235852628","heartbeat_front_addr":"172.21.15.110:6805/3235852628","state":["exists","up"]},{"osd":2,"uuid":"d4977df1-b118-4d7e-b9a9-626a8accc812","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6800","nonce":169452122},{"type":"v1","addr":"172.21.15.114:6801","nonce":169452122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6802","nonce":169452122},{"type":"v1","addr":"172.21.15.114:6803","nonce":169452122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6806","nonce":169452122},{"type":"v1","addr":"172.21.15.114:6807","nonce":169452122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6804","nonce":169452122},{"type":"v1","addr":"172.21.15.114:6805","nonce":169452122}]},"public_addr":"172.21.15.114:6801/169452122","cluster_addr":"172.21.15.114:6803/169452122","heartbeat_back_addr":"172.21.15.114:6807/169452122","heartbeat_front_addr":"172.21.15.114:6805/169452122","state":["exists","up"]},{"osd":3,"uuid":"2c7ae1b4-0bf2-4954-81ed-55d03eab82c6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6810","nonce":3016980877},{"type":"v1","addr":"172.21.15.89:6811","nonce":3016980877}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6812","nonce":3016980877},{"type":"v1","addr":"172.21.15.89:6813","nonce":3016980877}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6816","nonce":3016980877},{"type":"v1","addr":"172.21.15.89:6817","nonce":3016980877}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6814","nonce":3016980877},{"type":"v1","addr":"172.21.15.89:6815","nonce":3016980877}]},"public_addr":"172.21.15.89:6811/3016980877","cluster_addr":"172.21.15.89:6813/3016980877","heartbeat_back_addr":"172.21.15.89:6817/3016980877","heartbeat_front_addr":"172.21.15.89:6815/3016980877","state":["exists","up"]},{"osd":4,"uuid":"8e65399d-6e55-4da4-98ba-f805f0c655a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":30,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6808","nonce":3217880476},{"type":"v1","addr":"172.21.15.110:6809","nonce":3217880476}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6810","nonce":3217880476},{"type":"v1","addr":"172.21.15.110:6811","nonce":3217880476}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6814","nonce":3217880476},{"type":"v1","addr":"172.21.15.110:6815","nonce":3217880476}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6812","nonce":3217880476},{"type":"v1","addr":"172.21.15.110:6813","nonce":3217880476}]},"public_addr":"172.21.15.110:6809/3217880476","cluster_addr":"172.21.15.110:6811/3217880476","heartbeat_back_addr":"172.21.15.110:6815/3217880476","heartbeat_front_addr":"172.21.15.110:6813/3217880476","state":["exists","up"]},{"osd":5,"uuid":"b9ec2889-46bd-40e5-8446-73ab28271c67","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6808","nonce":1144140707},{"type":"v1","addr":"172.21.15.114:6809","nonce":1144140707}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6810","nonce":1144140707},{"type":"v1","addr":"172.21.15.114:6811","nonce":1144140707}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6814","nonce":1144140707},{"type":"v1","addr":"172.21.15.114:6815","nonce":1144140707}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6812","nonce":1144140707},{"type":"v1","addr":"172.21.15.114:6813","nonce":1144140707}]},"public_addr":"172.21.15.114:6809/1144140707","cluster_addr":"172.21.15.114:6811/1144140707","heartbeat_back_addr":"172.21.15.114:6815/1144140707","heartbeat_front_addr":"172.21.15.114:6813/1144140707","state":["exists","up"]},{"osd":6,"uuid":"9b79989e-adb0-42b2-b2f6-a2dce56b66e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":41,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6818","nonce":1522586255},{"type":"v1","addr":"172.21.15.89:6819","nonce":1522586255}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6820","nonce":1522586255},{"type":"v1","addr":"172.21.15.89:6821","nonce":1522586255}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6824","nonce":1522586255},{"type":"v1","addr":"172.21.15.89:6825","nonce":1522586255}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6822","nonce":1522586255},{"type":"v1","addr":"172.21.15.89:6823","nonce":1522586255}]},"public_addr":"172.21.15.89:6819/1522586255","cluster_addr":"172.21.15.89:6821/1522586255","heartbeat_back_addr":"172.21.15.89:6825/1522586255","heartbeat_front_addr":"172.21.15.89:6823/1522586255","state":["exists","up"]},{"osd":7,"uuid":"b381821f-141c-47a1-be65-f57ca730391b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6816","nonce":1331818323},{"type":"v1","addr":"172.21.15.110:6817","nonce":1331818323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6818","nonce":1331818323},{"type":"v1","addr":"172.21.15.110:6819","nonce":1331818323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6822","nonce":1331818323},{"type":"v1","addr":"172.21.15.110:6823","nonce":1331818323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6820","nonce":1331818323},{"type":"v1","addr":"172.21.15.110:6821","nonce":1331818323}]},"public_addr":"172.21.15.110:6817/1331818323","cluster_addr":"172.21.15.110:6819/1331818323","heartbeat_back_addr":"172.21.15.110:6823/1331818323","heartbeat_front_addr":"172.21.15.110:6821/1331818323","state":["exists","up"]},{"osd":8,"uuid":"1bc2a5b4-15f3-456f-9d67-e51307a27bcc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6816","nonce":2138556699},{"type":"v1","addr":"172.21.15.114:6817","nonce":2138556699}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6818","nonce":2138556699},{"type":"v1","addr":"172.21.15.114:6819","nonce":2138556699}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6822","nonce":2138556699},{"type":"v1","addr":"172.21.15.114:6823","nonce":2138556699}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6820","nonce":2138556699},{"type":"v1","addr":"172.21.15.114:6821","nonce":2138556699}]},"public_addr":"172.21.15.114:6817/2138556699","cluster_addr":"172.21.15.114:6819/2138556699","heartbeat_back_addr":"172.21.15.114:6823/2138556699","heartbeat_front_addr":"172.21.15.114:6821/2138556699","state":["exists","up"]},{"osd":9,"uuid":"91f5fd29-d4a7-4800-ac7a-e83b13add89e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6826","nonce":1730253068},{"type":"v1","addr":"172.21.15.89:6827","nonce":1730253068}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6828","nonce":1730253068},{"type":"v1","addr":"172.21.15.89:6829","nonce":1730253068}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6832","nonce":1730253068},{"type":"v1","addr":"172.21.15.89:6833","nonce":1730253068}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6830","nonce":1730253068},{"type":"v1","addr":"172.21.15.89:6831","nonce":1730253068}]},"public_addr":"172.21.15.89:6827/1730253068","cluster_addr":"172.21.15.89:6829/1730253068","heartbeat_back_addr":"172.21.15.89:6833/1730253068","heartbeat_front_addr":"172.21.15.89:6831/1730253068","state":["exists","up"]},{"osd":10,"uuid":"01390499-2a4d-4188-b57a-fcb10c668e2c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6824","nonce":2904016117},{"type":"v1","addr":"172.21.15.110:6825","nonce":2904016117}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6826","nonce":2904016117},{"type":"v1","addr":"172.21.15.110:6827","nonce":2904016117}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6830","nonce":2904016117},{"type":"v1","addr":"172.21.15.110:6831","nonce":2904016117}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6828","nonce":2904016117},{"type":"v1","addr":"172.21.15.110:6829","nonce":2904016117}]},"public_addr":"172.21.15.110:6825/2904016117","cluster_addr":"172.21.15.110:6827/2904016117","heartbeat_back_addr":"172.21.15.110:6831/2904016117","heartbeat_front_addr":"172.21.15.110:6829/2904016117","state":["exists","up"]},{"osd":11,"uuid":"c3c0f38a-c3d3-415d-a9f8-875d5b756e35","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6824","nonce":3749854486},{"type":"v1","addr":"172.21.15.114:6825","nonce":3749854486}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6826","nonce":3749854486},{"type":"v1","addr":"172.21.15.114:6827","nonce":3749854486}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6830","nonce":3749854486},{"type":"v1","addr":"172.21.15.114:6831","nonce":3749854486}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6828","nonce":3749854486},{"type":"v1","addr":"172.21.15.114:6829","nonce":3749854486}]},"public_addr":"172.21.15.114:6825/3749854486","cluster_addr":"172.21.15.114:6827/3749854486","heartbeat_back_addr":"172.21.15.114:6831/3749854486","heartbeat_front_addr":"172.21.15.114:6829/3749854486","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:31:46.218908+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:32:10.338230+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:32:34.830596+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:33:00.816102+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:33:25.223838+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:33:50.188230+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:34:16.564255+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:34:41.498230+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:35:05.826322+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:35:31.234450+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:35:56.474373+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:36:21.741254+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.89:6801/655110910":"2024-09-22T03:29:51.519106+0000","172.21.15.89:0/1618544983":"2024-09-22T03:29:51.519106+0000","172.21.15.89:0/1265502758":"2024-09-22T03:29:51.519106+0000","172.21.15.89:0/4017684419":"2024-09-22T03:29:51.519106+0000","172.21.15.89:6801/3312712420":"2024-09-22T03:29:26.004686+0000","172.21.15.89:6800/3312712420":"2024-09-22T03:29:26.004686+0000","172.21.15.89:0/2056576483":"2024-09-22T03:29:26.004686+0000","172.21.15.89:6800/655110910":"2024-09-22T03:29:51.519106+0000","172.21.15.89:0/2382866302":"2024-09-22T03:29:26.004686+0000","172.21.15.89:0/1584351801":"2024-09-22T03:29:26.004686+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T03:37:27.777 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:27.775+0000 7fc984816640 1 -- 172.21.15.89:0/1068324675 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7fc9680b6510 msgr2=0x7fc9680b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:27.777 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:27.775+0000 7fc984816640 1 --2- 172.21.15.89:0/1068324675 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7fc9680b6510 0x7fc9680b8a20 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fc97000bbe0 tx=0x7fc97000b390 comp rx=0 tx=0).stop 2024-09-21T03:37:27.777 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:27.775+0000 7fc984816640 1 -- 172.21.15.89:0/1068324675 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc9801083a0 msgr2=0x7fc98019a9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:27.777 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:27.775+0000 7fc984816640 1 --2- 172.21.15.89:0/1068324675 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc9801083a0 0x7fc98019a9c0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fc97402a930 tx=0x7fc974003c20 comp rx=0 tx=0).stop 2024-09-21T03:37:27.777 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:27.775+0000 7fc984816640 1 -- 172.21.15.89:0/1068324675 shutdown_connections 2024-09-21T03:37:27.777 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:27.775+0000 7fc984816640 1 --2- 172.21.15.89:0/1068324675 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7fc9680b6510 0x7fc9680b8a20 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:27.777 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:27.775+0000 7fc984816640 1 --2- 172.21.15.89:0/1068324675 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc980108cf0 0x7fc98019fc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:27.777 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:27.775+0000 7fc984816640 1 --2- 172.21.15.89:0/1068324675 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc9801083a0 0x7fc98019a9c0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:27.777 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:27.775+0000 7fc984816640 1 --2- 172.21.15.89:0/1068324675 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc980102a60 0x7fc98019a420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:27.777 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:27.775+0000 7fc984816640 1 -- 172.21.15.89:0/1068324675 >> 172.21.15.89:0/1068324675 conn(0x7fc980079730 msgr2=0x7fc980100c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:37:27.777 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:27.775+0000 7fc984816640 1 -- 172.21.15.89:0/1068324675 shutdown_connections 2024-09-21T03:37:27.777 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:27.775+0000 7fc984816640 1 -- 172.21.15.89:0/1068324675 wait complete. 2024-09-21T03:37:28.061 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:37:27 smithi089 ceph-mon[30928]: from='client.? 172.21.15.89:0/1068324675' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T03:37:28.211 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:37:27 smithi114 ceph-mon[34140]: from='client.? 172.21.15.89:0/1068324675' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T03:37:28.250 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:37:27 smithi110 ceph-mon[34000]: from='client.? 172.21.15.89:0/1068324675' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T03:37:28.429 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 7c374098-77c9-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T03:37:28.678 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/7c374098-77c9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:37:29.061 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:37:28 smithi089 ceph-mon[30928]: pgmap v298: 129 pgs: 129 active+clean; 579 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:37:29.195 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:29.190+0000 7f344875c640 1 -- 172.21.15.89:0/1777378370 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3440103b60 msgr2=0x7f3440103f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:29.195 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:29.190+0000 7f344875c640 1 --2- 172.21.15.89:0/1777378370 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3440103b60 0x7f3440103f70 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f3430009ca0 tx=0x7f343002a8e0 comp rx=0 tx=0).stop 2024-09-21T03:37:29.195 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:29.191+0000 7f344875c640 1 -- 172.21.15.89:0/1777378370 shutdown_connections 2024-09-21T03:37:29.195 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:29.191+0000 7f344875c640 1 --2- 172.21.15.89:0/1777378370 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f34401094a0 0x7f34401098b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:29.195 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:29.191+0000 7f344875c640 1 --2- 172.21.15.89:0/1777378370 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3440109df0 0x7f3440110620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:29.195 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:29.191+0000 7f344875c640 1 --2- 172.21.15.89:0/1777378370 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3440103b60 0x7f3440103f70 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:29.195 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:29.191+0000 7f344875c640 1 -- 172.21.15.89:0/1777378370 >> 172.21.15.89:0/1777378370 conn(0x7f344007b850 msgr2=0x7f344007bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:37:29.195 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:29.191+0000 7f344875c640 1 -- 172.21.15.89:0/1777378370 shutdown_connections 2024-09-21T03:37:29.195 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:29.191+0000 7f344875c640 1 -- 172.21.15.89:0/1777378370 wait complete. 2024-09-21T03:37:29.196 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:29.191+0000 7f344875c640 1 Processor -- start 2024-09-21T03:37:29.196 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:29.191+0000 7f344875c640 1 -- start start 2024-09-21T03:37:29.196 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:29.192+0000 7f344875c640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3440103b60 0x7f344019a470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:29.196 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:29.192+0000 7f344875c640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f34401094a0 0x7f344019aa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:29.196 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:29.192+0000 7f344875c640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3440109df0 0x7f344019fcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:29.196 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:29.192+0000 7f344875c640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f34401139a0 con 0x7f34401094a0 2024-09-21T03:37:29.196 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:29.192+0000 7f344875c640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f3440113820 con 0x7f3440103b60 2024-09-21T03:37:29.196 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:29.192+0000 7f344875c640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f3440113b20 con 0x7f3440109df0 2024-09-21T03:37:29.196 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:29.192+0000 7f3445cf7640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f34401094a0 0x7f344019aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:29.196 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:29.192+0000 7f3445cf7640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f34401094a0 0x7f344019aa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:48526/0 (socket says 172.21.15.89:48526) 2024-09-21T03:37:29.196 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:29.192+0000 7f3445cf7640 1 -- 172.21.15.89:0/1770473410 learned_addr learned my addr 172.21.15.89:0/1770473410 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-21T03:37:29.196 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:29.192+0000 7f3445cf7640 1 -- 172.21.15.89:0/1770473410 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3440109df0 msgr2=0x7f344019fcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:29.196 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:29.192+0000 7f3445cf7640 1 --2- 172.21.15.89:0/1770473410 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3440109df0 0x7f344019fcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:29.196 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:29.192+0000 7f3445cf7640 1 -- 172.21.15.89:0/1770473410 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3440103b60 msgr2=0x7f344019a470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:29.196 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:29.192+0000 7f3445cf7640 1 --2- 172.21.15.89:0/1770473410 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3440103b60 0x7f344019a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:29.197 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:29.192+0000 7f3445cf7640 1 -- 172.21.15.89:0/1770473410 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34401a03e0 con 0x7f34401094a0 2024-09-21T03:37:29.197 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:29.192+0000 7f3445cf7640 1 --2- 172.21.15.89:0/1770473410 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f34401094a0 0x7f344019aa10 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f3434006fd0 tx=0x7f343400b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:37:29.197 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:29.192+0000 7f343f7fe640 1 -- 172.21.15.89:0/1770473410 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3434011830 con 0x7f34401094a0 2024-09-21T03:37:29.197 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:29.192+0000 7f343f7fe640 1 -- 172.21.15.89:0/1770473410 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3434016070 con 0x7f34401094a0 2024-09-21T03:37:29.197 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:29.192+0000 7f344875c640 1 -- 172.21.15.89:0/1770473410 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f34401a06d0 con 0x7f34401094a0 2024-09-21T03:37:29.197 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:29.192+0000 7f343f7fe640 1 -- 172.21.15.89:0/1770473410 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3434010d30 con 0x7f34401094a0 2024-09-21T03:37:29.197 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:29.192+0000 7f344875c640 1 -- 172.21.15.89:0/1770473410 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34401a6e30 con 0x7f34401094a0 2024-09-21T03:37:29.197 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:29.193+0000 7f344875c640 1 -- 172.21.15.89:0/1770473410 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3440103f70 con 0x7f34401094a0 2024-09-21T03:37:29.198 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:29.196+0000 7f343f7fe640 1 -- 172.21.15.89:0/1770473410 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 17) ==== 154512+0+0 (secure 0 0 0) 0x7f34340119d0 con 0x7f34401094a0 2024-09-21T03:37:29.198 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:29.196+0000 7f343f7fe640 1 --2- 172.21.15.89:0/1770473410 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f34280b6510 0x7f34280b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:29.199 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:29.197+0000 7f34464f8640 1 --2- 172.21.15.89:0/1770473410 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f34280b6510 0x7f34280b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:29.199 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:29.197+0000 7f343f7fe640 1 -- 172.21.15.89:0/1770473410 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8802+0+0 (secure 0 0 0) 0x7f34340e71a0 con 0x7f34401094a0 2024-09-21T03:37:29.199 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:29.197+0000 7f34464f8640 1 --2- 172.21.15.89:0/1770473410 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f34280b6510 0x7f34280b8a20 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f3430004af0 tx=0x7f3430000f40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:37:29.202 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:29.200+0000 7f343f7fe640 1 -- 172.21.15.89:0/1770473410 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f34340ad7f0 con 0x7f34401094a0 2024-09-21T03:37:29.210 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:37:28 smithi114 ceph-mon[34140]: pgmap v298: 129 pgs: 129 active+clean; 579 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:37:29.249 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:37:28 smithi110 ceph-mon[34000]: pgmap v298: 129 pgs: 129 active+clean; 579 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:37:29.417 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:29.414+0000 7f344875c640 1 -- 172.21.15.89:0/1770473410 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f344002e640 con 0x7f34401094a0 2024-09-21T03:37:29.417 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:29.415+0000 7f343f7fe640 1 -- 172.21.15.89:0/1770473410 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+11839 (secure 0 0 0) 0x7f34340b1bd0 con 0x7f34401094a0 2024-09-21T03:37:29.418 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-09-21T03:37:29.418 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":17,"btime":"2024-09-21T03:37:22:908380+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14721,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6835/2341538514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":2341538514},{"type":"v1","addr":"172.21.15.89:6835","nonce":2341538514}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14733,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6837/146708592","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6836","nonce":146708592},{"type":"v1","addr":"172.21.15.89:6837","nonce":146708592}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14781,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6839/2269965499","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6838","nonce":2269965499},{"type":"v1","addr":"172.21.15.89:6839","nonce":2269965499}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14793,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6841/650850383","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6840","nonce":650850383},{"type":"v1","addr":"172.21.15.89:6841","nonce":650850383}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24565,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6833/752336912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6832","nonce":752336912},{"type":"v1","addr":"172.21.15.114:6833","nonce":752336912}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24575,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6833/127607331","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6832","nonce":127607331},{"type":"v1","addr":"172.21.15.110:6833","nonce":127607331}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24593,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6835/671820654","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6834","nonce":671820654},{"type":"v1","addr":"172.21.15.110:6835","nonce":671820654}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24604,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6835/1049714548","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6834","nonce":1049714548},{"type":"v1","addr":"172.21.15.114:6835","nonce":1049714548}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24610,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6837/1941213818","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6836","nonce":1941213818},{"type":"v1","addr":"172.21.15.114:6837","nonce":1941213818}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24614,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6837/4047124589","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6836","nonce":4047124589},{"type":"v1","addr":"172.21.15.110:6837","nonce":4047124589}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24631,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6839/3427369944","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6838","nonce":3427369944},{"type":"v1","addr":"172.21.15.114:6839","nonce":3427369944}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T03:37:13.570694+0000","modified":"2024-09-21T03:37:22.518535+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24638},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24638":{"gid":24638,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.110:6839/132366709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6838","nonce":132366709},{"type":"v1","addr":"172.21.15.110:6839","nonce":132366709}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":24638,"qdb_cluster":[24638]},"id":1}]} 2024-09-21T03:37:29.418 INFO:teuthology.orchestra.run.smithi089.stderr:dumped fsmap epoch 17 2024-09-21T03:37:29.420 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:29.417+0000 7f344875c640 1 -- 172.21.15.89:0/1770473410 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f34280b6510 msgr2=0x7f34280b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:29.420 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:29.417+0000 7f344875c640 1 --2- 172.21.15.89:0/1770473410 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f34280b6510 0x7f34280b8a20 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f3430004af0 tx=0x7f3430000f40 comp rx=0 tx=0).stop 2024-09-21T03:37:29.420 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:29.417+0000 7f344875c640 1 -- 172.21.15.89:0/1770473410 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f34401094a0 msgr2=0x7f344019aa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:29.420 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:29.417+0000 7f344875c640 1 --2- 172.21.15.89:0/1770473410 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f34401094a0 0x7f344019aa10 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f3434006fd0 tx=0x7f343400b8e0 comp rx=0 tx=0).stop 2024-09-21T03:37:29.420 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:29.418+0000 7f344875c640 1 -- 172.21.15.89:0/1770473410 shutdown_connections 2024-09-21T03:37:29.420 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:29.418+0000 7f344875c640 1 --2- 172.21.15.89:0/1770473410 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f34280b6510 0x7f34280b8a20 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:29.420 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:29.418+0000 7f344875c640 1 --2- 172.21.15.89:0/1770473410 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3440109df0 0x7f344019fcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:29.420 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:29.418+0000 7f344875c640 1 --2- 172.21.15.89:0/1770473410 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f34401094a0 0x7f344019aa10 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:29.421 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:29.418+0000 7f344875c640 1 --2- 172.21.15.89:0/1770473410 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3440103b60 0x7f344019a470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:29.421 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:29.418+0000 7f344875c640 1 -- 172.21.15.89:0/1770473410 >> 172.21.15.89:0/1770473410 conn(0x7f344007b850 msgr2=0x7f34401075e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:37:29.421 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:29.418+0000 7f344875c640 1 -- 172.21.15.89:0/1770473410 shutdown_connections 2024-09-21T03:37:29.421 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:29.418+0000 7f344875c640 1 -- 172.21.15.89:0/1770473410 wait complete. 2024-09-21T03:37:30.050 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:37:29 smithi089 ceph-mon[30928]: from='client.? 172.21.15.89:0/1770473410' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:37:30.081 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 7c374098-77c9-11ef-baf5-efdc52797490 -- ceph osd dump --format=json 2024-09-21T03:37:30.210 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:37:29 smithi114 ceph-mon[34140]: from='client.? 172.21.15.89:0/1770473410' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:37:30.250 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:37:29 smithi110 ceph-mon[34000]: from='client.? 172.21.15.89:0/1770473410' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:37:30.327 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/7c374098-77c9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:37:30.821 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:30.808+0000 7f55cc528640 1 -- 172.21.15.89:0/536102590 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f55c4107c00 msgr2=0x7f55c41080e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:30.822 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:30.808+0000 7f55cc528640 1 --2- 172.21.15.89:0/536102590 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f55c4107c00 0x7f55c41080e0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f55bc009cd0 tx=0x7f55bc02a8e0 comp rx=0 tx=0).stop 2024-09-21T03:37:30.822 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:30.808+0000 7f55cc528640 1 -- 172.21.15.89:0/536102590 shutdown_connections 2024-09-21T03:37:30.822 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:30.808+0000 7f55cc528640 1 --2- 172.21.15.89:0/536102590 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f55c4103110 0x7f55c4107640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:30.822 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:30.808+0000 7f55cc528640 1 --2- 172.21.15.89:0/536102590 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f55c4107c00 0x7f55c41080e0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:30.822 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:30.808+0000 7f55cc528640 1 --2- 172.21.15.89:0/536102590 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f55c41027c0 0x7f55c4102bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:30.822 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:30.808+0000 7f55cc528640 1 -- 172.21.15.89:0/536102590 >> 172.21.15.89:0/536102590 conn(0x7f55c4079740 msgr2=0x7f55c4079b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:37:30.822 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:30.808+0000 7f55cc528640 1 -- 172.21.15.89:0/536102590 shutdown_connections 2024-09-21T03:37:30.822 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:30.808+0000 7f55cc528640 1 -- 172.21.15.89:0/536102590 wait complete. 2024-09-21T03:37:30.822 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:30.809+0000 7f55cc528640 1 Processor -- start 2024-09-21T03:37:30.822 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:30.809+0000 7f55cc528640 1 -- start start 2024-09-21T03:37:30.822 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:30.809+0000 7f55cc528640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f55c41027c0 0x7f55c41a0a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:30.822 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:30.809+0000 7f55cc528640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f55c4103110 0x7f55c41a0fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:30.822 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:30.809+0000 7f55cc528640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f55c4107c00 0x7f55c41a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:30.822 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:30.809+0000 7f55cc528640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f55c4119ff0 con 0x7f55c4103110 2024-09-21T03:37:30.823 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:30.809+0000 7f55cc528640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f55c4119e70 con 0x7f55c41027c0 2024-09-21T03:37:30.823 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:30.809+0000 7f55cc528640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f55c411a170 con 0x7f55c4107c00 2024-09-21T03:37:30.823 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:30.809+0000 7f55c9ac3640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f55c4103110 0x7f55c41a0fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:30.823 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:30.809+0000 7f55c9ac3640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f55c4103110 0x7f55c41a0fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:48552/0 (socket says 172.21.15.89:48552) 2024-09-21T03:37:30.823 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:30.809+0000 7f55c9ac3640 1 -- 172.21.15.89:0/3057313663 learned_addr learned my addr 172.21.15.89:0/3057313663 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-21T03:37:30.823 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:30.809+0000 7f55c9ac3640 1 -- 172.21.15.89:0/3057313663 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f55c4107c00 msgr2=0x7f55c41a6280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:30.823 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:30.809+0000 7f55c9ac3640 1 --2- 172.21.15.89:0/3057313663 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f55c4107c00 0x7f55c41a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:30.823 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:30.809+0000 7f55c9ac3640 1 -- 172.21.15.89:0/3057313663 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f55c41027c0 msgr2=0x7f55c41a0a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:30.823 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:30.809+0000 7f55c9ac3640 1 --2- 172.21.15.89:0/3057313663 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f55c41027c0 0x7f55c41a0a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:30.823 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:30.809+0000 7f55c9ac3640 1 -- 172.21.15.89:0/3057313663 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55c41a69b0 con 0x7f55c4103110 2024-09-21T03:37:30.823 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:30.809+0000 7f55c9ac3640 1 --2- 172.21.15.89:0/3057313663 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f55c4103110 0x7f55c41a0fe0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f55b400c970 tx=0x7f55b400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:37:30.823 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:30.810+0000 7f55bb7fe640 1 -- 172.21.15.89:0/3057313663 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55b4007bb0 con 0x7f55c4103110 2024-09-21T03:37:30.823 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:30.810+0000 7f55bb7fe640 1 -- 172.21.15.89:0/3057313663 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f55b4007d50 con 0x7f55c4103110 2024-09-21T03:37:30.823 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:30.810+0000 7f55bb7fe640 1 -- 172.21.15.89:0/3057313663 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55b4004da0 con 0x7f55c4103110 2024-09-21T03:37:30.823 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:30.810+0000 7f55cc528640 1 -- 172.21.15.89:0/3057313663 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f55c41a6ca0 con 0x7f55c4103110 2024-09-21T03:37:30.824 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:30.810+0000 7f55cc528640 1 -- 172.21.15.89:0/3057313663 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55c41ad400 con 0x7f55c4103110 2024-09-21T03:37:30.824 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:30.811+0000 7f55cc528640 1 -- 172.21.15.89:0/3057313663 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f55c406c240 con 0x7f55c4103110 2024-09-21T03:37:30.824 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:30.812+0000 7f55bb7fe640 1 -- 172.21.15.89:0/3057313663 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 17) ==== 154512+0+0 (secure 0 0 0) 0x7f55b4004f60 con 0x7f55c4103110 2024-09-21T03:37:30.824 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:30.813+0000 7f55bb7fe640 1 --2- 172.21.15.89:0/3057313663 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f55b00b6510 0x7f55b00b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:30.824 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:30.813+0000 7f55bb7fe640 1 -- 172.21.15.89:0/3057313663 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8802+0+0 (secure 0 0 0) 0x7f55b40e7710 con 0x7f55c4103110 2024-09-21T03:37:30.824 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:30.813+0000 7f55ca2c4640 1 --2- 172.21.15.89:0/3057313663 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f55b00b6510 0x7f55b00b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:30.824 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:30.814+0000 7f55ca2c4640 1 --2- 172.21.15.89:0/3057313663 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f55b00b6510 0x7f55b00b8a20 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f55c406c940 tx=0x7f55c0006c90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:37:30.824 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:30.816+0000 7f55bb7fe640 1 -- 172.21.15.89:0/3057313663 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f55b4014030 con 0x7f55c4103110 2024-09-21T03:37:31.001 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:30.999+0000 7f55cc528640 1 -- 172.21.15.89:0/3057313663 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f55c4107640 con 0x7f55c4103110 2024-09-21T03:37:31.001 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:37:30 smithi089 ceph-mon[30928]: pgmap v299: 129 pgs: 129 active+clean; 579 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:37:31.002 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:31.000+0000 7f55bb7fe640 1 -- 172.21.15.89:0/3057313663 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v80) ==== 74+0+24236 (secure 0 0 0) 0x7f55b40add60 con 0x7f55c4103110 2024-09-21T03:37:31.003 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-09-21T03:37:31.003 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":80,"fsid":"7c374098-77c9-11ef-baf5-efdc52797490","created":"2024-09-21T03:28:56.681458+0000","modified":"2024-09-21T03:37:22.518390+0000","last_up_change":"2024-09-21T03:36:23.368379+0000","last_in_change":"2024-09-21T03:36:00.131331+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T03:32:37.834339+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-21T03:37:09.443341+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"74","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-21T03:37:11.467821+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"74","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-21T03:37:17.884066+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"27ca5ed8-17d1-4f8e-a7ae-084a914721a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6802","nonce":990539042},{"type":"v1","addr":"172.21.15.89:6803","nonce":990539042}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6804","nonce":990539042},{"type":"v1","addr":"172.21.15.89:6805","nonce":990539042}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6808","nonce":990539042},{"type":"v1","addr":"172.21.15.89:6809","nonce":990539042}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6806","nonce":990539042},{"type":"v1","addr":"172.21.15.89:6807","nonce":990539042}]},"public_addr":"172.21.15.89:6803/990539042","cluster_addr":"172.21.15.89:6805/990539042","heartbeat_back_addr":"172.21.15.89:6809/990539042","heartbeat_front_addr":"172.21.15.89:6807/990539042","state":["exists","up"]},{"osd":1,"uuid":"5176a910-5cf9-49d7-aedb-a20cf36ea075","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6800","nonce":3235852628},{"type":"v1","addr":"172.21.15.110:6801","nonce":3235852628}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6802","nonce":3235852628},{"type":"v1","addr":"172.21.15.110:6803","nonce":3235852628}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6806","nonce":3235852628},{"type":"v1","addr":"172.21.15.110:6807","nonce":3235852628}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6804","nonce":3235852628},{"type":"v1","addr":"172.21.15.110:6805","nonce":3235852628}]},"public_addr":"172.21.15.110:6801/3235852628","cluster_addr":"172.21.15.110:6803/3235852628","heartbeat_back_addr":"172.21.15.110:6807/3235852628","heartbeat_front_addr":"172.21.15.110:6805/3235852628","state":["exists","up"]},{"osd":2,"uuid":"d4977df1-b118-4d7e-b9a9-626a8accc812","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6800","nonce":169452122},{"type":"v1","addr":"172.21.15.114:6801","nonce":169452122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6802","nonce":169452122},{"type":"v1","addr":"172.21.15.114:6803","nonce":169452122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6806","nonce":169452122},{"type":"v1","addr":"172.21.15.114:6807","nonce":169452122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6804","nonce":169452122},{"type":"v1","addr":"172.21.15.114:6805","nonce":169452122}]},"public_addr":"172.21.15.114:6801/169452122","cluster_addr":"172.21.15.114:6803/169452122","heartbeat_back_addr":"172.21.15.114:6807/169452122","heartbeat_front_addr":"172.21.15.114:6805/169452122","state":["exists","up"]},{"osd":3,"uuid":"2c7ae1b4-0bf2-4954-81ed-55d03eab82c6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6810","nonce":3016980877},{"type":"v1","addr":"172.21.15.89:6811","nonce":3016980877}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6812","nonce":3016980877},{"type":"v1","addr":"172.21.15.89:6813","nonce":3016980877}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6816","nonce":3016980877},{"type":"v1","addr":"172.21.15.89:6817","nonce":3016980877}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6814","nonce":3016980877},{"type":"v1","addr":"172.21.15.89:6815","nonce":3016980877}]},"public_addr":"172.21.15.89:6811/3016980877","cluster_addr":"172.21.15.89:6813/3016980877","heartbeat_back_addr":"172.21.15.89:6817/3016980877","heartbeat_front_addr":"172.21.15.89:6815/3016980877","state":["exists","up"]},{"osd":4,"uuid":"8e65399d-6e55-4da4-98ba-f805f0c655a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":30,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6808","nonce":3217880476},{"type":"v1","addr":"172.21.15.110:6809","nonce":3217880476}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6810","nonce":3217880476},{"type":"v1","addr":"172.21.15.110:6811","nonce":3217880476}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6814","nonce":3217880476},{"type":"v1","addr":"172.21.15.110:6815","nonce":3217880476}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6812","nonce":3217880476},{"type":"v1","addr":"172.21.15.110:6813","nonce":3217880476}]},"public_addr":"172.21.15.110:6809/3217880476","cluster_addr":"172.21.15.110:6811/3217880476","heartbeat_back_addr":"172.21.15.110:6815/3217880476","heartbeat_front_addr":"172.21.15.110:6813/3217880476","state":["exists","up"]},{"osd":5,"uuid":"b9ec2889-46bd-40e5-8446-73ab28271c67","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6808","nonce":1144140707},{"type":"v1","addr":"172.21.15.114:6809","nonce":1144140707}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6810","nonce":1144140707},{"type":"v1","addr":"172.21.15.114:6811","nonce":1144140707}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6814","nonce":1144140707},{"type":"v1","addr":"172.21.15.114:6815","nonce":1144140707}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6812","nonce":1144140707},{"type":"v1","addr":"172.21.15.114:6813","nonce":1144140707}]},"public_addr":"172.21.15.114:6809/1144140707","cluster_addr":"172.21.15.114:6811/1144140707","heartbeat_back_addr":"172.21.15.114:6815/1144140707","heartbeat_front_addr":"172.21.15.114:6813/1144140707","state":["exists","up"]},{"osd":6,"uuid":"9b79989e-adb0-42b2-b2f6-a2dce56b66e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":41,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6818","nonce":1522586255},{"type":"v1","addr":"172.21.15.89:6819","nonce":1522586255}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6820","nonce":1522586255},{"type":"v1","addr":"172.21.15.89:6821","nonce":1522586255}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6824","nonce":1522586255},{"type":"v1","addr":"172.21.15.89:6825","nonce":1522586255}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6822","nonce":1522586255},{"type":"v1","addr":"172.21.15.89:6823","nonce":1522586255}]},"public_addr":"172.21.15.89:6819/1522586255","cluster_addr":"172.21.15.89:6821/1522586255","heartbeat_back_addr":"172.21.15.89:6825/1522586255","heartbeat_front_addr":"172.21.15.89:6823/1522586255","state":["exists","up"]},{"osd":7,"uuid":"b381821f-141c-47a1-be65-f57ca730391b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6816","nonce":1331818323},{"type":"v1","addr":"172.21.15.110:6817","nonce":1331818323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6818","nonce":1331818323},{"type":"v1","addr":"172.21.15.110:6819","nonce":1331818323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6822","nonce":1331818323},{"type":"v1","addr":"172.21.15.110:6823","nonce":1331818323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6820","nonce":1331818323},{"type":"v1","addr":"172.21.15.110:6821","nonce":1331818323}]},"public_addr":"172.21.15.110:6817/1331818323","cluster_addr":"172.21.15.110:6819/1331818323","heartbeat_back_addr":"172.21.15.110:6823/1331818323","heartbeat_front_addr":"172.21.15.110:6821/1331818323","state":["exists","up"]},{"osd":8,"uuid":"1bc2a5b4-15f3-456f-9d67-e51307a27bcc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6816","nonce":2138556699},{"type":"v1","addr":"172.21.15.114:6817","nonce":2138556699}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6818","nonce":2138556699},{"type":"v1","addr":"172.21.15.114:6819","nonce":2138556699}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6822","nonce":2138556699},{"type":"v1","addr":"172.21.15.114:6823","nonce":2138556699}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6820","nonce":2138556699},{"type":"v1","addr":"172.21.15.114:6821","nonce":2138556699}]},"public_addr":"172.21.15.114:6817/2138556699","cluster_addr":"172.21.15.114:6819/2138556699","heartbeat_back_addr":"172.21.15.114:6823/2138556699","heartbeat_front_addr":"172.21.15.114:6821/2138556699","state":["exists","up"]},{"osd":9,"uuid":"91f5fd29-d4a7-4800-ac7a-e83b13add89e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6826","nonce":1730253068},{"type":"v1","addr":"172.21.15.89:6827","nonce":1730253068}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6828","nonce":1730253068},{"type":"v1","addr":"172.21.15.89:6829","nonce":1730253068}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6832","nonce":1730253068},{"type":"v1","addr":"172.21.15.89:6833","nonce":1730253068}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6830","nonce":1730253068},{"type":"v1","addr":"172.21.15.89:6831","nonce":1730253068}]},"public_addr":"172.21.15.89:6827/1730253068","cluster_addr":"172.21.15.89:6829/1730253068","heartbeat_back_addr":"172.21.15.89:6833/1730253068","heartbeat_front_addr":"172.21.15.89:6831/1730253068","state":["exists","up"]},{"osd":10,"uuid":"01390499-2a4d-4188-b57a-fcb10c668e2c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6824","nonce":2904016117},{"type":"v1","addr":"172.21.15.110:6825","nonce":2904016117}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6826","nonce":2904016117},{"type":"v1","addr":"172.21.15.110:6827","nonce":2904016117}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6830","nonce":2904016117},{"type":"v1","addr":"172.21.15.110:6831","nonce":2904016117}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6828","nonce":2904016117},{"type":"v1","addr":"172.21.15.110:6829","nonce":2904016117}]},"public_addr":"172.21.15.110:6825/2904016117","cluster_addr":"172.21.15.110:6827/2904016117","heartbeat_back_addr":"172.21.15.110:6831/2904016117","heartbeat_front_addr":"172.21.15.110:6829/2904016117","state":["exists","up"]},{"osd":11,"uuid":"c3c0f38a-c3d3-415d-a9f8-875d5b756e35","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6824","nonce":3749854486},{"type":"v1","addr":"172.21.15.114:6825","nonce":3749854486}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6826","nonce":3749854486},{"type":"v1","addr":"172.21.15.114:6827","nonce":3749854486}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6830","nonce":3749854486},{"type":"v1","addr":"172.21.15.114:6831","nonce":3749854486}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6828","nonce":3749854486},{"type":"v1","addr":"172.21.15.114:6829","nonce":3749854486}]},"public_addr":"172.21.15.114:6825/3749854486","cluster_addr":"172.21.15.114:6827/3749854486","heartbeat_back_addr":"172.21.15.114:6831/3749854486","heartbeat_front_addr":"172.21.15.114:6829/3749854486","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:31:46.218908+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:32:10.338230+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:32:34.830596+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:33:00.816102+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:33:25.223838+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:33:50.188230+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:34:16.564255+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:34:41.498230+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:35:05.826322+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:35:31.234450+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:35:56.474373+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:36:21.741254+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.89:6801/655110910":"2024-09-22T03:29:51.519106+0000","172.21.15.89:0/1618544983":"2024-09-22T03:29:51.519106+0000","172.21.15.89:0/1265502758":"2024-09-22T03:29:51.519106+0000","172.21.15.89:0/4017684419":"2024-09-22T03:29:51.519106+0000","172.21.15.89:6801/3312712420":"2024-09-22T03:29:26.004686+0000","172.21.15.89:6800/3312712420":"2024-09-22T03:29:26.004686+0000","172.21.15.89:0/2056576483":"2024-09-22T03:29:26.004686+0000","172.21.15.89:6800/655110910":"2024-09-22T03:29:51.519106+0000","172.21.15.89:0/2382866302":"2024-09-22T03:29:26.004686+0000","172.21.15.89:0/1584351801":"2024-09-22T03:29:26.004686+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T03:37:31.005 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:31.002+0000 7f55cc528640 1 -- 172.21.15.89:0/3057313663 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f55b00b6510 msgr2=0x7f55b00b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:31.005 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:31.002+0000 7f55cc528640 1 --2- 172.21.15.89:0/3057313663 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f55b00b6510 0x7f55b00b8a20 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f55c406c940 tx=0x7f55c0006c90 comp rx=0 tx=0).stop 2024-09-21T03:37:31.005 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:31.002+0000 7f55cc528640 1 -- 172.21.15.89:0/3057313663 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f55c4103110 msgr2=0x7f55c41a0fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:31.005 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:31.002+0000 7f55cc528640 1 --2- 172.21.15.89:0/3057313663 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f55c4103110 0x7f55c41a0fe0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f55b400c970 tx=0x7f55b400ce30 comp rx=0 tx=0).stop 2024-09-21T03:37:31.005 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:31.003+0000 7f55cc528640 1 -- 172.21.15.89:0/3057313663 shutdown_connections 2024-09-21T03:37:31.005 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:31.003+0000 7f55cc528640 1 --2- 172.21.15.89:0/3057313663 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f55b00b6510 0x7f55b00b8a20 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:31.005 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:31.003+0000 7f55cc528640 1 --2- 172.21.15.89:0/3057313663 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f55c4107c00 0x7f55c41a6280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:31.005 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:31.003+0000 7f55cc528640 1 --2- 172.21.15.89:0/3057313663 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f55c4103110 0x7f55c41a0fe0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:31.005 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:31.003+0000 7f55cc528640 1 --2- 172.21.15.89:0/3057313663 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f55c41027c0 0x7f55c41a0a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:31.005 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:31.003+0000 7f55cc528640 1 -- 172.21.15.89:0/3057313663 >> 172.21.15.89:0/3057313663 conn(0x7f55c4079740 msgr2=0x7f55c4105610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:37:31.005 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:31.003+0000 7f55cc528640 1 -- 172.21.15.89:0/3057313663 shutdown_connections 2024-09-21T03:37:31.005 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:31.003+0000 7f55cc528640 1 -- 172.21.15.89:0/3057313663 wait complete. 2024-09-21T03:37:31.211 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:37:30 smithi114 ceph-mon[34140]: pgmap v299: 129 pgs: 129 active+clean; 579 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:37:31.250 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:37:30 smithi110 ceph-mon[34000]: pgmap v299: 129 pgs: 129 active+clean; 579 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:37:31.679 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 17, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-21T03:37:13.570694+0000', 'modified': '2024-09-21T03:37:22.518535+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 24638}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24638': {'gid': 24638, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.110:6839/132366709', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.110:6838', 'nonce': 132366709}, {'type': 'v1', 'addr': '172.21.15.110:6839', 'nonce': 132366709}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 1, 'qdb_leader': 24638, 'qdb_cluster': [24638]} 2024-09-21T03:37:31.680 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-21T03:37:31.680 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 7c374098-77c9-11ef-baf5-efdc52797490 -- ceph tell mds.b status 2024-09-21T03:37:31.926 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/7c374098-77c9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:37:32.206 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:37:31 smithi089 ceph-mon[30928]: from='client.? 172.21.15.89:0/3057313663' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T03:37:32.211 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:37:31 smithi114 ceph-mon[34140]: from='client.? 172.21.15.89:0/3057313663' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T03:37:32.250 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:37:31 smithi110 ceph-mon[34000]: from='client.? 172.21.15.89:0/3057313663' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T03:37:32.422 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:32.417+0000 7f78a43dd640 1 -- 172.21.15.89:0/3916552696 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f789c1083a0 msgr2=0x7f789c1087b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:32.422 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:32.417+0000 7f78a43dd640 1 --2- 172.21.15.89:0/3916552696 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f789c1083a0 0x7f789c1087b0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f7890009900 tx=0x7f789002a410 comp rx=0 tx=0).stop 2024-09-21T03:37:32.422 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:32.417+0000 7f78a43dd640 1 -- 172.21.15.89:0/3916552696 shutdown_connections 2024-09-21T03:37:32.422 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:32.417+0000 7f78a43dd640 1 --2- 172.21.15.89:0/3916552696 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f789c108cf0 0x7f789c10f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:32.422 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:32.417+0000 7f78a43dd640 1 --2- 172.21.15.89:0/3916552696 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f789c1083a0 0x7f789c1087b0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:32.422 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:32.417+0000 7f78a43dd640 1 --2- 172.21.15.89:0/3916552696 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f789c102a60 0x7f789c102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:32.422 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:32.417+0000 7f78a43dd640 1 -- 172.21.15.89:0/3916552696 >> 172.21.15.89:0/3916552696 conn(0x7f789c079730 msgr2=0x7f789c079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:37:32.422 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:32.417+0000 7f78a43dd640 1 -- 172.21.15.89:0/3916552696 shutdown_connections 2024-09-21T03:37:32.422 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:32.417+0000 7f78a43dd640 1 -- 172.21.15.89:0/3916552696 wait complete. 2024-09-21T03:37:32.422 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:32.418+0000 7f78a43dd640 1 Processor -- start 2024-09-21T03:37:32.422 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:32.418+0000 7f78a43dd640 1 -- start start 2024-09-21T03:37:32.422 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:32.418+0000 7f78a43dd640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f789c102a60 0x7f789c199280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:32.423 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:32.418+0000 7f78a43dd640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f789c1083a0 0x7f789c199820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:32.423 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:32.418+0000 7f78a43dd640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f789c108cf0 0x7f789c19eac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:32.423 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:32.418+0000 7f78a43dd640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f789c112810 con 0x7f789c1083a0 2024-09-21T03:37:32.423 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:32.418+0000 7f78a43dd640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f789c112690 con 0x7f789c108cf0 2024-09-21T03:37:32.423 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:32.418+0000 7f78a43dd640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f789c112990 con 0x7f789c102a60 2024-09-21T03:37:32.423 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:32.418+0000 7f78a1978640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f789c1083a0 0x7f789c199820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:32.423 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:32.418+0000 7f78a1978640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f789c1083a0 0x7f789c199820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:48576/0 (socket says 172.21.15.89:48576) 2024-09-21T03:37:32.423 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:32.418+0000 7f78a1978640 1 -- 172.21.15.89:0/3576034598 learned_addr learned my addr 172.21.15.89:0/3576034598 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-21T03:37:32.423 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:32.418+0000 7f78a2179640 1 --2- 172.21.15.89:0/3576034598 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f789c102a60 0x7f789c199280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:32.423 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:32.418+0000 7f78a1978640 1 -- 172.21.15.89:0/3576034598 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f789c102a60 msgr2=0x7f789c199280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:32.423 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:32.418+0000 7f78a1978640 1 --2- 172.21.15.89:0/3576034598 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f789c102a60 0x7f789c199280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:32.423 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:32.418+0000 7f78a297a640 1 --2- 172.21.15.89:0/3576034598 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f789c108cf0 0x7f789c19eac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:32.423 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:32.418+0000 7f78a1978640 1 -- 172.21.15.89:0/3576034598 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f789c108cf0 msgr2=0x7f789c19eac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:32.423 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:32.418+0000 7f78a1978640 1 --2- 172.21.15.89:0/3576034598 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f789c108cf0 0x7f789c19eac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:32.424 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:32.418+0000 7f78a1978640 1 -- 172.21.15.89:0/3576034598 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f789c19f130 con 0x7f789c1083a0 2024-09-21T03:37:32.424 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:32.419+0000 7f78a1978640 1 --2- 172.21.15.89:0/3576034598 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f789c1083a0 0x7f789c199820 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f7890005940 tx=0x7f78900042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:37:32.424 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:32.419+0000 7f788f7fe640 1 -- 172.21.15.89:0/3576034598 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f789003f030 con 0x7f789c1083a0 2024-09-21T03:37:32.424 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:32.419+0000 7f788f7fe640 1 -- 172.21.15.89:0/3576034598 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7890003c90 con 0x7f789c1083a0 2024-09-21T03:37:32.424 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:32.419+0000 7f78a43dd640 1 -- 172.21.15.89:0/3576034598 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f789c19f3c0 con 0x7f789c1083a0 2024-09-21T03:37:32.424 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:32.419+0000 7f788f7fe640 1 -- 172.21.15.89:0/3576034598 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7890035070 con 0x7f789c1083a0 2024-09-21T03:37:32.424 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:32.419+0000 7f78a43dd640 1 -- 172.21.15.89:0/3576034598 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f789c19f8a0 con 0x7f789c1083a0 2024-09-21T03:37:32.429 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:32.422+0000 7f78aa951740 1 Processor -- start 2024-09-21T03:37:32.429 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:32.422+0000 7f78aa951740 1 -- start start 2024-09-21T03:37:32.429 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:32.422+0000 7f78aa951740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562126b85f60 0x562126f2bf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:32.429 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:32.422+0000 7f78aa951740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562126b5d0a0 0x562126f2c440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:32.429 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:32.422+0000 7f78aa951740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562126b65280 0x562126f2c980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:32.429 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:32.422+0000 7f78aa951740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x562126ea3c10 con 0x562126b85f60 2024-09-21T03:37:32.429 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:32.422+0000 7f78aa951740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x562126e93040 con 0x562126b65280 2024-09-21T03:37:32.430 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:32.422+0000 7f78aa951740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x562126e96e50 con 0x562126b5d0a0 2024-09-21T03:37:32.430 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:32.422+0000 7f78a297a640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562126b85f60 0x562126f2bf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:32.430 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:32.422+0000 7f78a297a640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562126b85f60 0x562126f2bf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:48580/0 (socket says 172.21.15.89:48580) 2024-09-21T03:37:32.430 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:32.422+0000 7f78a297a640 1 -- 172.21.15.89:0/3328904845 learned_addr learned my addr 172.21.15.89:0/3328904845 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-21T03:37:32.430 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:32.422+0000 7f78a2179640 1 --2- 172.21.15.89:0/3328904845 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562126b5d0a0 0x562126f2c440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:32.430 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:32.423+0000 7f78a2179640 1 -- 172.21.15.89:0/3328904845 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562126b65280 msgr2=0x562126f2c980 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T03:37:32.430 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:32.423+0000 7f78a2179640 1 --2- 172.21.15.89:0/3328904845 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562126b65280 0x562126f2c980 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:32.430 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:32.423+0000 7f78a2179640 1 -- 172.21.15.89:0/3328904845 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562126b85f60 msgr2=0x562126f2bf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:37:32.430 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:32.423+0000 7f78a2179640 1 --2- 172.21.15.89:0/3328904845 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562126b85f60 0x562126f2bf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:32.430 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:32.423+0000 7f78a2179640 1 -- 172.21.15.89:0/3328904845 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562126bf1540 con 0x562126b5d0a0 2024-09-21T03:37:32.430 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:32.423+0000 7f78a297a640 1 --2- 172.21.15.89:0/3328904845 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562126b85f60 0x562126f2bf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:37:32.430 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:32.423+0000 7f788f7fe640 1 -- 172.21.15.89:0/3576034598 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 17) ==== 154512+0+0 (secure 0 0 0) 0x7f7890042030 con 0x7f789c1083a0 2024-09-21T03:37:32.430 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:32.423+0000 7f78a2179640 1 --2- 172.21.15.89:0/3328904845 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562126b5d0a0 0x562126f2c440 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f789800abf0 tx=0x7f789800f790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=921279722d7ed254 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:37:32.431 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:32.423+0000 7f788cd9a640 1 -- 172.21.15.89:0/3328904845 <== mon.1 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7898005220 con 0x562126b5d0a0 2024-09-21T03:37:32.431 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:32.423+0000 7f788cd9a640 1 -- 172.21.15.89:0/3328904845 <== mon.1 v2:172.21.15.114:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f78980047a0 con 0x562126b5d0a0 2024-09-21T03:37:32.431 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:32.423+0000 7f788cd9a640 1 -- 172.21.15.89:0/3328904845 <== mon.1 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78980186a0 con 0x562126b5d0a0 2024-09-21T03:37:32.431 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:32.423+0000 7f788f7fe640 1 --2- 172.21.15.89:0/3576034598 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f78880b6540 0x7f78880b8a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:32.431 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:32.424+0000 7f78a297a640 1 --2- 172.21.15.89:0/3576034598 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f78880b6540 0x7f78880b8a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:32.431 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:32.424+0000 7f788f7fe640 1 -- 172.21.15.89:0/3576034598 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8802+0+0 (secure 0 0 0) 0x7f7890107b30 con 0x7f789c1083a0 2024-09-21T03:37:32.431 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:32.424+0000 7f78aa951740 1 -- 172.21.15.89:0/3328904845 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562126b5d0a0 msgr2=0x562126f2c440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:32.431 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:32.424+0000 7f78aa951740 1 --2- 172.21.15.89:0/3328904845 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562126b5d0a0 0x562126f2c440 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f789800abf0 tx=0x7f789800f790 comp rx=0 tx=0).stop 2024-09-21T03:37:32.431 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:32.424+0000 7f78a297a640 1 --2- 172.21.15.89:0/3576034598 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f78880b6540 0x7f78880b8a50 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f7894008560 tx=0x7f789400f040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:37:32.431 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:32.424+0000 7f78aa951740 1 -- 172.21.15.89:0/3328904845 shutdown_connections 2024-09-21T03:37:32.431 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:32.424+0000 7f78aa951740 1 --2- 172.21.15.89:0/3328904845 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562126b65280 0x562126f2c980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:32.431 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:32.424+0000 7f78aa951740 1 --2- 172.21.15.89:0/3328904845 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562126b5d0a0 0x562126f2c440 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:32.431 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:32.424+0000 7f78aa951740 1 --2- 172.21.15.89:0/3328904845 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562126b85f60 0x562126f2bf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:32.431 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:32.424+0000 7f78aa951740 1 -- 172.21.15.89:0/3328904845 >> 172.21.15.89:0/3328904845 conn(0x562126b7a350 msgr2=0x562126f29df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:37:32.432 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:32.424+0000 7f78aa951740 1 -- 172.21.15.89:0/3328904845 shutdown_connections 2024-09-21T03:37:32.432 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:32.424+0000 7f78aa951740 1 -- 172.21.15.89:0/3328904845 wait complete. 2024-09-21T03:37:32.432 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:32.425+0000 7f78aa951740 1 -- start start 2024-09-21T03:37:32.432 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:32.425+0000 7f78aa951740 1 Processor -- start 2024-09-21T03:37:32.432 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:32.425+0000 7f78aa951740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562126b5d0a0 0x562126f37a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:32.432 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:32.425+0000 7f78aa951740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562126b65280 0x562126f37fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:32.432 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:32.425+0000 7f78aa951740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562126b85f60 0x562126f43a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:32.432 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:32.425+0000 7f78aa951740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x562126cf45f0 con 0x562126b85f60 2024-09-21T03:37:32.432 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:32.425+0000 7f78aa951740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x562126e2e010 con 0x562126b65280 2024-09-21T03:37:32.432 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:32.425+0000 7f78aa951740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x562126f3a6e0 con 0x562126b5d0a0 2024-09-21T03:37:32.432 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:32.425+0000 7f78aa951740 10 client.0 _ll_register_callbacks cb 0x562126e8d340 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:37:32.432 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:32.425+0000 7f78a297a640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562126b85f60 0x562126f43a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:32.433 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:32.425+0000 7f78a2179640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562126b5d0a0 0x562126f37a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:32.433 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:32.425+0000 7f78a297a640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562126b85f60 0x562126f43a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:48590/0 (socket says 172.21.15.89:48590) 2024-09-21T03:37:32.433 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:32.425+0000 7f78a297a640 1 -- 172.21.15.89:0/3231546530 learned_addr learned my addr 172.21.15.89:0/3231546530 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-21T03:37:32.433 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:32.425+0000 7f78a1978640 1 --2- 172.21.15.89:0/3231546530 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562126b65280 0x562126f37fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:32.433 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:32.425+0000 7f78a297a640 1 -- 172.21.15.89:0/3231546530 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562126b5d0a0 msgr2=0x562126f37a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:37:32.433 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:32.425+0000 7f78a297a640 1 --2- 172.21.15.89:0/3231546530 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562126b5d0a0 0x562126f37a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:32.433 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:32.425+0000 7f78a297a640 1 -- 172.21.15.89:0/3231546530 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562126b65280 msgr2=0x562126f37fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:37:32.433 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:32.425+0000 7f78a297a640 1 --2- 172.21.15.89:0/3231546530 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562126b65280 0x562126f37fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:32.433 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:32.425+0000 7f78a297a640 1 -- 172.21.15.89:0/3231546530 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562126e5ba00 con 0x562126b85f60 2024-09-21T03:37:32.433 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:32.425+0000 7f78a1978640 1 --2- 172.21.15.89:0/3231546530 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562126b65280 0x562126f37fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:37:32.433 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:32.426+0000 7f78a297a640 1 --2- 172.21.15.89:0/3231546530 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562126b85f60 0x562126f43a00 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f789400ca70 tx=0x7f7894004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4769ca274707b41f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:37:32.433 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:32.426+0000 7f7885ffb640 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2024-09-21T03:37:32.433 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:32.426+0000 7f7885ffb640 1 -- 172.21.15.89:0/3231546530 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7894029060 con 0x562126b85f60 2024-09-21T03:37:32.433 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:32.426+0000 7f78aa951740 1 -- 172.21.15.89:0/3231546530 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f78980047a0 con 0x562126b85f60 2024-09-21T03:37:32.433 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:32.426+0000 7f7885ffb640 1 -- 172.21.15.89:0/3231546530 <== mon.0 v2:172.21.15.89:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8802+0+0 (secure 0 0 0) 0x7f789401b910 con 0x562126b85f60 2024-09-21T03:37:32.434 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:32.426+0000 7f7885ffb640 1 -- 172.21.15.89:0/3231546530 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) ==== 24+0+0 (secure 0 0 0) 0x7f7894030050 con 0x562126b85f60 2024-09-21T03:37:32.434 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:32.426+0000 7f78aa951740 10 client.14901 fetch_fsmap learned FSMap version 17 2024-09-21T03:37:32.434 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:32.426+0000 7f78aa951740 1 -- 172.21.15.89:0/3231546530 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=4+}) -- 0x562126f466e0 con 0x562126b85f60 2024-09-21T03:37:32.434 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:32.426+0000 7f7885ffb640 1 -- 172.21.15.89:0/3231546530 <== mon.0 v2:172.21.15.89:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f78980047a0 con 0x562126b85f60 2024-09-21T03:37:32.434 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:32.426+0000 7f7885ffb640 1 -- 172.21.15.89:0/3231546530 <== mon.0 v2:172.21.15.89:3300/0 5 ==== fsmap(e 17) ==== 8350+0+0 (secure 0 0 0) 0x7f78940249c0 con 0x562126b85f60 2024-09-21T03:37:32.434 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:32.426+0000 7f78aa951740 10 client.14901 fetch_fsmap finished waiting for FSMap version 17 2024-09-21T03:37:32.434 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:32.427+0000 7f78aa951740 10 client.14901 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-21T03:37:32.434 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:32.427+0000 7f78aa951740 20 client.14901 populate_metadata read hostname 'smithi089' 2024-09-21T03:37:32.434 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:32.427+0000 7f78aa951740 1 --2- 172.21.15.89:0/3231546530 >> [v2:172.21.15.110:6838/132366709,v1:172.21.15.110:6839/132366709] conn(0x562126b448e0 0x562126f4a1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:32.434 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:32.427+0000 7f78aa951740 4 client.14901 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:37:32.434 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:32.427+0000 7f78aa951740 1 -- 172.21.15.89:0/3231546530 --> [v2:172.21.15.110:6838/132366709,v1:172.21.15.110:6839/132366709] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562126ea5b60 con 0x562126b448e0 2024-09-21T03:37:32.434 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:32.427+0000 7f78a2179640 1 --2- 172.21.15.89:0/3231546530 >> [v2:172.21.15.110:6838/132366709,v1:172.21.15.110:6839/132366709] conn(0x562126b448e0 0x562126f4a1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:32.435 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:32.427+0000 7f7885ffb640 10 client.14901 ms_handle_connect on v2:172.21.15.110:6838/132366709 2024-09-21T03:37:32.436 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:32.427+0000 7f78a2179640 1 --2- 172.21.15.89:0/3231546530 >> [v2:172.21.15.110:6838/132366709,v1:172.21.15.110:6839/132366709] conn(0x562126b448e0 0x562126f4a1d0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ac99cfe219d97b90 server_cookie=6cf70c0e6a969fb5 in_seq=0 out_seq=0 2024-09-21T03:37:32.436 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:32.428+0000 7f7885ffb640 1 -- 172.21.15.89:0/3231546530 <== mds.0 v2:172.21.15.110:6838/132366709 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x562126ea5b60 con 0x562126b448e0 2024-09-21T03:37:32.436 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:32.428+0000 7f7885ffb640 10 client.14901 handle_command_reply: tid=0 2024-09-21T03:37:32.436 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:32.428+0000 7f78aa951740 1 client.14901 shutdown 2024-09-21T03:37:32.436 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:32.428+0000 7f78aa951740 1 -- 172.21.15.89:0/3231546530 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562126b85f60 msgr2=0x562126f43a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:32.436 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:32.428+0000 7f78aa951740 1 --2- 172.21.15.89:0/3231546530 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562126b85f60 0x562126f43a00 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f789400ca70 tx=0x7f7894004ab0 comp rx=0 tx=0).stop 2024-09-21T03:37:32.436 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:32.428+0000 7f78aa951740 1 -- 172.21.15.89:0/3231546530 shutdown_connections 2024-09-21T03:37:32.436 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:32.428+0000 7f78aa951740 1 --2- 172.21.15.89:0/3231546530 >> [v2:172.21.15.110:6838/132366709,v1:172.21.15.110:6839/132366709] conn(0x562126b448e0 0x562126f4a1d0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:32.436 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:32.428+0000 7f78aa951740 1 --2- 172.21.15.89:0/3231546530 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562126b65280 0x562126f37fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:32.436 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:32.428+0000 7f78aa951740 1 --2- 172.21.15.89:0/3231546530 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562126b85f60 0x562126f43a00 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:32.436 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:32.429+0000 7f78aa951740 1 --2- 172.21.15.89:0/3231546530 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562126b5d0a0 0x562126f37a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:32.436 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:32.429+0000 7f78aa951740 1 -- 172.21.15.89:0/3231546530 >> 172.21.15.89:0/3231546530 conn(0x562126b7a350 msgr2=0x562126f2f0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:37:32.436 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:32.429+0000 7f7885ffb640 0 client.14901 ms_handle_reset on v2:172.21.15.110:6838/132366709 2024-09-21T03:37:32.437 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:32.429+0000 7f78aa951740 1 -- 172.21.15.89:0/3231546530 shutdown_connections 2024-09-21T03:37:32.437 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:32.429+0000 7f78aa951740 1 -- 172.21.15.89:0/3231546530 wait complete. 2024-09-21T03:37:32.437 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:32.429+0000 7f78aa951740 20 client.14901 trim_cache size 0 max 16384 2024-09-21T03:37:32.450 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:32.446+0000 7f78aa951740 1 Processor -- start 2024-09-21T03:37:32.450 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:32.447+0000 7f78aa951740 1 -- start start 2024-09-21T03:37:32.450 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:32.447+0000 7f78aa951740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562126b5d0a0 0x562126f43970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:32.450 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:32.447+0000 7f78aa951740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562126b65280 0x562126f43eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:32.450 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:32.447+0000 7f78aa951740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562126b85f60 0x562126f329c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:32.450 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:32.447+0000 7f78aa951740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x562126f48510 con 0x562126b65280 2024-09-21T03:37:32.450 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:32.447+0000 7f78aa951740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x562126f48be0 con 0x562126b5d0a0 2024-09-21T03:37:32.450 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:32.447+0000 7f78aa951740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x562126f492b0 con 0x562126b85f60 2024-09-21T03:37:32.450 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:32.447+0000 7f78a1978640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562126b65280 0x562126f43eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:32.450 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:32.447+0000 7f78a1978640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562126b65280 0x562126f43eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:48594/0 (socket says 172.21.15.89:48594) 2024-09-21T03:37:32.450 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:32.447+0000 7f78a1978640 1 -- 172.21.15.89:0/3076997007 learned_addr learned my addr 172.21.15.89:0/3076997007 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-21T03:37:32.451 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:32.447+0000 7f78a1978640 1 -- 172.21.15.89:0/3076997007 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562126b85f60 msgr2=0x562126f329c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:37:32.451 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:32.447+0000 7f78a2179640 1 --2- 172.21.15.89:0/3076997007 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562126b5d0a0 0x562126f43970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:32.451 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:32.447+0000 7f78a1978640 1 --2- 172.21.15.89:0/3076997007 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562126b85f60 0x562126f329c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:32.451 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:32.447+0000 7f78a1978640 1 -- 172.21.15.89:0/3076997007 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562126b5d0a0 msgr2=0x562126f43970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:37:32.451 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:32.447+0000 7f78a1978640 1 --2- 172.21.15.89:0/3076997007 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562126b5d0a0 0x562126f43970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:32.451 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:32.447+0000 7f78a1978640 1 -- 172.21.15.89:0/3076997007 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562126f35d80 con 0x562126b65280 2024-09-21T03:37:32.451 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:32.447+0000 7f78a1978640 1 --2- 172.21.15.89:0/3076997007 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562126b65280 0x562126f43eb0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f7890105330 tx=0x7f78900cd8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0e60d769b66ee85 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:37:32.451 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:32.448+0000 7f7887fff640 1 -- 172.21.15.89:0/3076997007 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7890049040 con 0x562126b65280 2024-09-21T03:37:32.451 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:32.448+0000 7f7887fff640 1 -- 172.21.15.89:0/3076997007 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f78900f10d0 con 0x562126b65280 2024-09-21T03:37:32.451 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:32.448+0000 7f7887fff640 1 -- 172.21.15.89:0/3076997007 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78900ce120 con 0x562126b65280 2024-09-21T03:37:32.455 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:32.448+0000 7f78aa951740 1 -- 172.21.15.89:0/3076997007 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562126b65280 msgr2=0x562126f43eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:32.455 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:32.448+0000 7f78aa951740 1 --2- 172.21.15.89:0/3076997007 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562126b65280 0x562126f43eb0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f7890105330 tx=0x7f78900cd8b0 comp rx=0 tx=0).stop 2024-09-21T03:37:32.455 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:32.448+0000 7f78aa951740 1 -- 172.21.15.89:0/3076997007 shutdown_connections 2024-09-21T03:37:32.455 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:32.448+0000 7f78aa951740 1 --2- 172.21.15.89:0/3076997007 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562126b85f60 0x562126f329c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:32.455 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:32.448+0000 7f78aa951740 1 --2- 172.21.15.89:0/3076997007 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562126b65280 0x562126f43eb0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:32.455 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:32.448+0000 7f78aa951740 1 --2- 172.21.15.89:0/3076997007 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562126b5d0a0 0x562126f43970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:32.455 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:32.448+0000 7f78aa951740 1 -- 172.21.15.89:0/3076997007 >> 172.21.15.89:0/3076997007 conn(0x562126b7a350 msgr2=0x562126f24ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:37:32.455 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:32.448+0000 7f78aa951740 1 -- 172.21.15.89:0/3076997007 shutdown_connections 2024-09-21T03:37:32.455 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:32.448+0000 7f78aa951740 1 -- 172.21.15.89:0/3076997007 wait complete. 2024-09-21T03:37:32.455 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:32.449+0000 7f78aa951740 1 -- start start 2024-09-21T03:37:32.456 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:32.449+0000 7f78aa951740 1 Processor -- start 2024-09-21T03:37:32.456 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:32.449+0000 7f78aa951740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562126b5d0a0 0x562126f528b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:32.456 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:32.449+0000 7f78aa951740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562126b65280 0x562126f52e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:32.456 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:32.449+0000 7f78aa951740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562126b85f60 0x562126f4bef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:32.456 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:32.449+0000 7f78aa951740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x562126f4a050 con 0x562126b85f60 2024-09-21T03:37:32.456 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:32.449+0000 7f78aa951740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x562126f49980 con 0x562126b65280 2024-09-21T03:37:32.456 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:32.449+0000 7f78aa951740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x562126f4a720 con 0x562126b5d0a0 2024-09-21T03:37:32.456 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:32.449+0000 7f78aa951740 10 client.0 _ll_register_callbacks cb 0x7f789400da90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:37:32.456 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:32.449+0000 7f78a297a640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562126b85f60 0x562126f4bef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:32.456 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:32.449+0000 7f78a297a640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562126b85f60 0x562126f4bef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:48606/0 (socket says 172.21.15.89:48606) 2024-09-21T03:37:32.456 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:32.449+0000 7f78a297a640 1 -- 172.21.15.89:0/3437202083 learned_addr learned my addr 172.21.15.89:0/3437202083 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-21T03:37:32.456 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:32.449+0000 7f78a1978640 1 --2- 172.21.15.89:0/3437202083 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562126b65280 0x562126f52e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:32.456 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:32.449+0000 7f78a2179640 1 --2- 172.21.15.89:0/3437202083 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562126b5d0a0 0x562126f528b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:32.456 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:32.450+0000 7f78a297a640 1 -- 172.21.15.89:0/3437202083 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562126b5d0a0 msgr2=0x562126f528b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:37:32.456 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:32.450+0000 7f78a297a640 1 --2- 172.21.15.89:0/3437202083 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562126b5d0a0 0x562126f528b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:32.456 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:32.450+0000 7f78a297a640 1 -- 172.21.15.89:0/3437202083 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562126b65280 msgr2=0x562126f52e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:37:32.457 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:32.450+0000 7f78a297a640 1 --2- 172.21.15.89:0/3437202083 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562126b65280 0x562126f52e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:32.457 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:32.450+0000 7f78a297a640 1 -- 172.21.15.89:0/3437202083 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562126f4c680 con 0x562126b85f60 2024-09-21T03:37:32.457 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:32.450+0000 7f78a297a640 1 --2- 172.21.15.89:0/3437202083 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562126b85f60 0x562126f4bef0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f789400ca40 tx=0x7f7894005a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a965d14111857abd server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:37:32.457 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:32.450+0000 7f7885ffb640 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2024-09-21T03:37:32.457 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:32.450+0000 7f7885ffb640 1 -- 172.21.15.89:0/3437202083 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f789400d3d0 con 0x562126b85f60 2024-09-21T03:37:32.457 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:32.450+0000 7f7885ffb640 1 -- 172.21.15.89:0/3437202083 <== mon.0 v2:172.21.15.89:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8802+0+0 (secure 0 0 0) 0x7f789401b9f0 con 0x562126b85f60 2024-09-21T03:37:32.457 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:32.450+0000 7f78aa951740 1 -- 172.21.15.89:0/3437202083 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f78900f10d0 con 0x562126b85f60 2024-09-21T03:37:32.457 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:32.450+0000 7f7885ffb640 1 -- 172.21.15.89:0/3437202083 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) ==== 24+0+0 (secure 0 0 0) 0x7f789401bbf0 con 0x562126b85f60 2024-09-21T03:37:32.457 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:32.450+0000 7f78aa951740 10 client.14907 fetch_fsmap learned FSMap version 17 2024-09-21T03:37:32.457 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:32.450+0000 7f78aa951740 1 -- 172.21.15.89:0/3437202083 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=4+}) -- 0x562126f52080 con 0x562126b85f60 2024-09-21T03:37:32.457 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:32.450+0000 7f7885ffb640 1 -- 172.21.15.89:0/3437202083 <== mon.0 v2:172.21.15.89:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f78900f10d0 con 0x562126b85f60 2024-09-21T03:37:32.457 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:32.450+0000 7f7885ffb640 1 -- 172.21.15.89:0/3437202083 <== mon.0 v2:172.21.15.89:3300/0 5 ==== fsmap(e 17) ==== 8350+0+0 (secure 0 0 0) 0x7f789403c030 con 0x562126b85f60 2024-09-21T03:37:32.457 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:32.451+0000 7f78aa951740 10 client.14907 fetch_fsmap finished waiting for FSMap version 17 2024-09-21T03:37:32.457 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:32.451+0000 7f78aa951740 10 client.14907 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-21T03:37:32.457 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:32.451+0000 7f78aa951740 20 client.14907 populate_metadata read hostname 'smithi089' 2024-09-21T03:37:32.457 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:32.451+0000 7f78aa951740 1 --2- 172.21.15.89:0/3437202083 >> [v2:172.21.15.110:6838/132366709,v1:172.21.15.110:6839/132366709] conn(0x562126b19c90 0x562126fb8a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:32.457 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:32.451+0000 7f78aa951740 4 client.14907 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-21T03:37:32.458 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:32.451+0000 7f78aa951740 1 -- 172.21.15.89:0/3437202083 --> [v2:172.21.15.110:6838/132366709,v1:172.21.15.110:6839/132366709] -- command(tid 0: {"prefix": "status"}) -- 0x562126ea1610 con 0x562126b19c90 2024-09-21T03:37:32.458 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:32.451+0000 7f78a2179640 1 --2- 172.21.15.89:0/3437202083 >> [v2:172.21.15.110:6838/132366709,v1:172.21.15.110:6839/132366709] conn(0x562126b19c90 0x562126fb8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:32.458 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:32.451+0000 7f78a2179640 1 --2- 172.21.15.89:0/3437202083 >> [v2:172.21.15.110:6838/132366709,v1:172.21.15.110:6839/132366709] conn(0x562126b19c90 0x562126fb8a70 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=40632b61dbdeb753 server_cookie=2c014bcce22a80b2 in_seq=0 out_seq=0 2024-09-21T03:37:32.458 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:32.451+0000 7f7885ffb640 10 client.14907 ms_handle_connect on v2:172.21.15.110:6838/132366709 2024-09-21T03:37:32.458 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:32.452+0000 7f7885ffb640 1 -- 172.21.15.89:0/3437202083 <== mds.0 v2:172.21.15.110:6838/132366709 1 ==== command_reply(tid 0: 0 ) ==== 8+0+326 (crc 0 0 0) 0x562126ea1610 con 0x562126b19c90 2024-09-21T03:37:32.458 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:32.452+0000 7f7885ffb640 10 client.14907 handle_command_reply: tid=0 2024-09-21T03:37:32.458 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:32.452+0000 7f78aa951740 1 client.14907 shutdown 2024-09-21T03:37:32.458 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:32.452+0000 7f78aa951740 1 -- 172.21.15.89:0/3437202083 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562126b85f60 msgr2=0x562126f4bef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:32.458 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:32.452+0000 7f78aa951740 1 --2- 172.21.15.89:0/3437202083 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562126b85f60 0x562126f4bef0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f789400ca40 tx=0x7f7894005a80 comp rx=0 tx=0).stop 2024-09-21T03:37:32.458 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:32.452+0000 7f78aa951740 1 -- 172.21.15.89:0/3437202083 shutdown_connections 2024-09-21T03:37:32.458 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:32.452+0000 7f78aa951740 1 --2- 172.21.15.89:0/3437202083 >> [v2:172.21.15.110:6838/132366709,v1:172.21.15.110:6839/132366709] conn(0x562126b19c90 0x562126fb8a70 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:32.458 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:32.452+0000 7f78aa951740 1 --2- 172.21.15.89:0/3437202083 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562126b65280 0x562126f52e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:32.458 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:32.452+0000 7f7885ffb640 0 client.14907 ms_handle_reset on v2:172.21.15.110:6838/132366709 2024-09-21T03:37:32.458 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:32.452+0000 7f78aa951740 1 --2- 172.21.15.89:0/3437202083 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562126b85f60 0x562126f4bef0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:32.458 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:32.452+0000 7f78aa951740 1 --2- 172.21.15.89:0/3437202083 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562126b5d0a0 0x562126f528b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:32.459 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:32.452+0000 7f78aa951740 1 -- 172.21.15.89:0/3437202083 >> 172.21.15.89:0/3437202083 conn(0x562126b7a350 msgr2=0x562126f24ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:37:32.459 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:32.452+0000 7f78aa951740 1 -- 172.21.15.89:0/3437202083 shutdown_connections 2024-09-21T03:37:32.459 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:32.452+0000 7f78aa951740 1 -- 172.21.15.89:0/3437202083 wait complete. 2024-09-21T03:37:32.459 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:32.452+0000 7f78aa951740 20 client.14907 trim_cache size 0 max 16384 2024-09-21T03:37:32.459 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-09-21T03:37:32.459 INFO:teuthology.orchestra.run.smithi089.stdout: "cluster_fsid": "7c374098-77c9-11ef-baf5-efdc52797490", 2024-09-21T03:37:32.459 INFO:teuthology.orchestra.run.smithi089.stdout: "whoami": 0, 2024-09-21T03:37:32.459 INFO:teuthology.orchestra.run.smithi089.stdout: "id": 24638, 2024-09-21T03:37:32.459 INFO:teuthology.orchestra.run.smithi089.stdout: "want_state": "up:active", 2024-09-21T03:37:32.459 INFO:teuthology.orchestra.run.smithi089.stdout: "state": "up:active", 2024-09-21T03:37:32.459 INFO:teuthology.orchestra.run.smithi089.stdout: "fs_name": "cephfs", 2024-09-21T03:37:32.459 INFO:teuthology.orchestra.run.smithi089.stdout: "rank_uptime": 18.599067287, 2024-09-21T03:37:32.460 INFO:teuthology.orchestra.run.smithi089.stdout: "mdsmap_epoch": 17, 2024-09-21T03:37:32.460 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap_epoch": 80, 2024-09-21T03:37:32.460 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap_epoch_barrier": 80, 2024-09-21T03:37:32.460 INFO:teuthology.orchestra.run.smithi089.stdout: "uptime": 29.066848232000002 2024-09-21T03:37:32.460 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-09-21T03:37:32.460 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:32.453+0000 7f78a43dd640 1 -- 172.21.15.89:0/3576034598 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f78880b6540 msgr2=0x7f78880b8a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:32.460 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:32.453+0000 7f78a43dd640 1 --2- 172.21.15.89:0/3576034598 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f78880b6540 0x7f78880b8a50 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f7894008560 tx=0x7f789400f040 comp rx=0 tx=0).stop 2024-09-21T03:37:32.460 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:32.453+0000 7f78a43dd640 1 -- 172.21.15.89:0/3576034598 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f789c1083a0 msgr2=0x7f789c199820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:32.461 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:32.453+0000 7f78a43dd640 1 --2- 172.21.15.89:0/3576034598 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f789c1083a0 0x7f789c199820 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f7890005940 tx=0x7f78900042f0 comp rx=0 tx=0).stop 2024-09-21T03:37:32.461 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:32.453+0000 7f78a43dd640 1 -- 172.21.15.89:0/3576034598 shutdown_connections 2024-09-21T03:37:32.461 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:32.453+0000 7f78a43dd640 1 --2- 172.21.15.89:0/3576034598 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f78880b6540 0x7f78880b8a50 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:32.461 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:32.453+0000 7f78a43dd640 1 --2- 172.21.15.89:0/3576034598 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f789c108cf0 0x7f789c19eac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:32.461 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:32.453+0000 7f78a43dd640 1 --2- 172.21.15.89:0/3576034598 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f789c1083a0 0x7f789c199820 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:32.461 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:32.453+0000 7f78a43dd640 1 --2- 172.21.15.89:0/3576034598 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f789c102a60 0x7f789c199280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:32.461 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:32.453+0000 7f78a43dd640 1 -- 172.21.15.89:0/3576034598 >> 172.21.15.89:0/3576034598 conn(0x7f789c079730 msgr2=0x7f789c1044d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:37:32.461 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:32.453+0000 7f78a43dd640 1 -- 172.21.15.89:0/3576034598 shutdown_connections 2024-09-21T03:37:32.461 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:32.453+0000 7f78a43dd640 1 -- 172.21.15.89:0/3576034598 wait complete. 2024-09-21T03:37:33.061 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:37:32 smithi089 ceph-mon[30928]: pgmap v300: 129 pgs: 129 active+clean; 579 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:37:33.184 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-09-21T03:37:33.184 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-09-21T03:37:33.184 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 7c374098-77c9-11ef-baf5-efdc52797490 -- ceph fs ls 2024-09-21T03:37:33.210 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:37:32 smithi114 ceph-mon[34140]: pgmap v300: 129 pgs: 129 active+clean; 579 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:37:33.250 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:37:32 smithi110 ceph-mon[34000]: pgmap v300: 129 pgs: 129 active+clean; 579 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:37:33.432 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/7c374098-77c9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:37:33.928 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:33.926+0000 7f0b6b8aa640 1 -- 172.21.15.89:0/62591956 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0b641094a0 msgr2=0x7f0b641098b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:33.928 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:33.926+0000 7f0b6b8aa640 1 --2- 172.21.15.89:0/62591956 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0b641094a0 0x7f0b641098b0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f0b58009900 tx=0x7f0b5802a410 comp rx=0 tx=0).stop 2024-09-21T03:37:33.929 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:33.926+0000 7f0b6b8aa640 1 -- 172.21.15.89:0/62591956 shutdown_connections 2024-09-21T03:37:33.929 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:33.926+0000 7f0b6b8aa640 1 --2- 172.21.15.89:0/62591956 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0b64109df0 0x7f0b64110620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:33.929 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:33.926+0000 7f0b6b8aa640 1 --2- 172.21.15.89:0/62591956 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0b641094a0 0x7f0b641098b0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:33.929 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:33.926+0000 7f0b6b8aa640 1 --2- 172.21.15.89:0/62591956 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0b64103b60 0x7f0b64103f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:33.929 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:33.926+0000 7f0b6b8aa640 1 -- 172.21.15.89:0/62591956 >> 172.21.15.89:0/62591956 conn(0x7f0b6407b850 msgr2=0x7f0b6407bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:37:33.929 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:33.926+0000 7f0b6b8aa640 1 -- 172.21.15.89:0/62591956 shutdown_connections 2024-09-21T03:37:33.929 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:33.926+0000 7f0b6b8aa640 1 -- 172.21.15.89:0/62591956 wait complete. 2024-09-21T03:37:33.930 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:33.927+0000 7f0b6b8aa640 1 Processor -- start 2024-09-21T03:37:33.931 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:33.927+0000 7f0b6b8aa640 1 -- start start 2024-09-21T03:37:33.931 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:33.927+0000 7f0b6b8aa640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0b64103b60 0x7f0b6419a490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:33.931 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:33.927+0000 7f0b6b8aa640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0b641094a0 0x7f0b6419aa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:33.931 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:33.927+0000 7f0b6b8aa640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0b64109df0 0x7f0b6419fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:33.931 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:33.927+0000 7f0b6b8aa640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f0b64113960 con 0x7f0b64109df0 2024-09-21T03:37:33.931 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:33.927+0000 7f0b6b8aa640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f0b641137e0 con 0x7f0b64103b60 2024-09-21T03:37:33.931 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:33.927+0000 7f0b6b8aa640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f0b64113ae0 con 0x7f0b641094a0 2024-09-21T03:37:33.931 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:33.927+0000 7f0b69e47640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0b64109df0 0x7f0b6419fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:33.931 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:33.927+0000 7f0b69e47640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0b64109df0 0x7f0b6419fcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:48628/0 (socket says 172.21.15.89:48628) 2024-09-21T03:37:33.931 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:33.927+0000 7f0b69e47640 1 -- 172.21.15.89:0/3369373281 learned_addr learned my addr 172.21.15.89:0/3369373281 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-21T03:37:33.931 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:33.927+0000 7f0b69646640 1 --2- 172.21.15.89:0/3369373281 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0b64103b60 0x7f0b6419a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:33.931 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:33.927+0000 7f0b68e45640 1 --2- 172.21.15.89:0/3369373281 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0b641094a0 0x7f0b6419aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:33.931 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:33.927+0000 7f0b69e47640 1 -- 172.21.15.89:0/3369373281 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0b641094a0 msgr2=0x7f0b6419aa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:33.931 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:33.927+0000 7f0b69e47640 1 --2- 172.21.15.89:0/3369373281 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0b641094a0 0x7f0b6419aa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:33.932 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:33.927+0000 7f0b69e47640 1 -- 172.21.15.89:0/3369373281 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0b64103b60 msgr2=0x7f0b6419a490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:33.932 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:33.927+0000 7f0b69e47640 1 --2- 172.21.15.89:0/3369373281 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0b64103b60 0x7f0b6419a490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:33.932 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:33.927+0000 7f0b69e47640 1 -- 172.21.15.89:0/3369373281 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b641a0400 con 0x7f0b64109df0 2024-09-21T03:37:33.932 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:33.927+0000 7f0b69646640 1 --2- 172.21.15.89:0/3369373281 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0b64103b60 0x7f0b6419a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:37:33.932 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:33.928+0000 7f0b69e47640 1 --2- 172.21.15.89:0/3369373281 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0b64109df0 0x7f0b6419fcd0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f0b5c00d750 tx=0x7f0b5c00dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:37:33.932 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:33.928+0000 7f0b627fc640 1 -- 172.21.15.89:0/3369373281 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b5c017070 con 0x7f0b64109df0 2024-09-21T03:37:33.932 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:33.928+0000 7f0b627fc640 1 -- 172.21.15.89:0/3369373281 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0b5c0054b0 con 0x7f0b64109df0 2024-09-21T03:37:33.932 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:33.928+0000 7f0b6b8aa640 1 -- 172.21.15.89:0/3369373281 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0b641a06f0 con 0x7f0b64109df0 2024-09-21T03:37:33.932 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:33.928+0000 7f0b627fc640 1 -- 172.21.15.89:0/3369373281 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b5c004b90 con 0x7f0b64109df0 2024-09-21T03:37:33.932 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:33.928+0000 7f0b6b8aa640 1 -- 172.21.15.89:0/3369373281 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0b641a6e50 con 0x7f0b64109df0 2024-09-21T03:37:33.932 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:33.929+0000 7f0b6b8aa640 1 -- 172.21.15.89:0/3369373281 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0b64103f70 con 0x7f0b64109df0 2024-09-21T03:37:33.933 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:33.930+0000 7f0b627fc640 1 -- 172.21.15.89:0/3369373281 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 17) ==== 154512+0+0 (secure 0 0 0) 0x7f0b5c004d30 con 0x7f0b64109df0 2024-09-21T03:37:33.933 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:33.931+0000 7f0b627fc640 1 --2- 172.21.15.89:0/3369373281 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f0b4c0b6510 0x7f0b4c0b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:33.933 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:33.931+0000 7f0b69646640 1 --2- 172.21.15.89:0/3369373281 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f0b4c0b6510 0x7f0b4c0b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:33.933 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:33.931+0000 7f0b627fc640 1 -- 172.21.15.89:0/3369373281 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8802+0+0 (secure 0 0 0) 0x7f0b5c0e6f10 con 0x7f0b64109df0 2024-09-21T03:37:33.934 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:33.932+0000 7f0b69646640 1 --2- 172.21.15.89:0/3369373281 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f0b4c0b6510 0x7f0b4c0b8a20 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f0b54005fd0 tx=0x7f0b54009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:37:33.937 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:33.935+0000 7f0b627fc640 1 -- 172.21.15.89:0/3369373281 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f0b5c0ad560 con 0x7f0b64109df0 2024-09-21T03:37:34.154 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:34.152+0000 7f0b6b8aa640 1 -- 172.21.15.89:0/3369373281 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f0b6410bbe0 con 0x7f0b64109df0 2024-09-21T03:37:34.155 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:34.153+0000 7f0b627fc640 1 -- 172.21.15.89:0/3369373281 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v17) ==== 53+0+88 (secure 0 0 0) 0x7f0b5c0b1940 con 0x7f0b64109df0 2024-09-21T03:37:34.155 INFO:teuthology.orchestra.run.smithi089.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-09-21T03:37:34.158 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:34.156+0000 7f0b6b8aa640 1 -- 172.21.15.89:0/3369373281 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f0b4c0b6510 msgr2=0x7f0b4c0b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:34.158 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:34.156+0000 7f0b6b8aa640 1 --2- 172.21.15.89:0/3369373281 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f0b4c0b6510 0x7f0b4c0b8a20 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f0b54005fd0 tx=0x7f0b54009500 comp rx=0 tx=0).stop 2024-09-21T03:37:34.158 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:34.156+0000 7f0b6b8aa640 1 -- 172.21.15.89:0/3369373281 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0b64109df0 msgr2=0x7f0b6419fcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:34.158 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:34.156+0000 7f0b6b8aa640 1 --2- 172.21.15.89:0/3369373281 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0b64109df0 0x7f0b6419fcd0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f0b5c00d750 tx=0x7f0b5c00dc10 comp rx=0 tx=0).stop 2024-09-21T03:37:34.159 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:34.156+0000 7f0b6b8aa640 1 -- 172.21.15.89:0/3369373281 shutdown_connections 2024-09-21T03:37:34.159 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:34.156+0000 7f0b6b8aa640 1 --2- 172.21.15.89:0/3369373281 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f0b4c0b6510 0x7f0b4c0b8a20 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:34.159 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:34.156+0000 7f0b6b8aa640 1 --2- 172.21.15.89:0/3369373281 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0b64109df0 0x7f0b6419fcd0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:34.159 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:34.156+0000 7f0b6b8aa640 1 --2- 172.21.15.89:0/3369373281 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0b641094a0 0x7f0b6419aa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:34.159 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:34.156+0000 7f0b6b8aa640 1 --2- 172.21.15.89:0/3369373281 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0b64103b60 0x7f0b6419a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:34.159 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:34.156+0000 7f0b6b8aa640 1 -- 172.21.15.89:0/3369373281 >> 172.21.15.89:0/3369373281 conn(0x7f0b6407b850 msgr2=0x7f0b64105610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:37:34.159 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:34.156+0000 7f0b6b8aa640 1 -- 172.21.15.89:0/3369373281 shutdown_connections 2024-09-21T03:37:34.159 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:34.156+0000 7f0b6b8aa640 1 -- 172.21.15.89:0/3369373281 wait complete. 2024-09-21T03:37:34.946 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-21T03:37:34.946 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-21T03:37:34.946 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi089.front.sepia.ceph.com 2024-09-21T03:37:34.946 INFO:tasks.cephfs.mount:self.client.name = client.admin 2024-09-21T03:37:34.946 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-09-21T03:37:34.946 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-21T03:37:34.946 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-21T03:37:34.946 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-21T03:37:34.946 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.admin' 2024-09-21T03:37:34.947 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:37:34.947 DEBUG:teuthology.orchestra.run.smithi089:> ip addr 2024-09-21T03:37:34.964 INFO:teuthology.orchestra.run.smithi089.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-21T03:37:34.964 INFO:teuthology.orchestra.run.smithi089.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-21T03:37:34.964 INFO:teuthology.orchestra.run.smithi089.stdout: inet 127.0.0.1/8 scope host lo 2024-09-21T03:37:34.964 INFO:teuthology.orchestra.run.smithi089.stdout: valid_lft forever preferred_lft forever 2024-09-21T03:37:34.964 INFO:teuthology.orchestra.run.smithi089.stdout: inet6 ::1/128 scope host 2024-09-21T03:37:34.964 INFO:teuthology.orchestra.run.smithi089.stdout: valid_lft forever preferred_lft forever 2024-09-21T03:37:34.964 INFO:teuthology.orchestra.run.smithi089.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T03:37:34.964 INFO:teuthology.orchestra.run.smithi089.stdout: link/ether 0c:c4:7a:d8:a4:c6 brd ff:ff:ff:ff:ff:ff 2024-09-21T03:37:34.964 INFO:teuthology.orchestra.run.smithi089.stdout: altname enp7s0f0 2024-09-21T03:37:34.964 INFO:teuthology.orchestra.run.smithi089.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T03:37:34.964 INFO:teuthology.orchestra.run.smithi089.stdout: link/ether 0c:c4:7a:d8:a4:c7 brd ff:ff:ff:ff:ff:ff 2024-09-21T03:37:34.964 INFO:teuthology.orchestra.run.smithi089.stdout: altname enp7s0f1 2024-09-21T03:37:34.964 INFO:teuthology.orchestra.run.smithi089.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T03:37:34.965 INFO:teuthology.orchestra.run.smithi089.stdout: link/ether 0c:c4:7a:8f:d5:00 brd ff:ff:ff:ff:ff:ff 2024-09-21T03:37:34.965 INFO:teuthology.orchestra.run.smithi089.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-21T03:37:34.965 INFO:teuthology.orchestra.run.smithi089.stdout: link/ether 0c:c4:7a:8f:d5:01 brd ff:ff:ff:ff:ff:ff 2024-09-21T03:37:34.965 INFO:teuthology.orchestra.run.smithi089.stdout: inet 172.21.15.89/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-09-21T03:37:34.965 INFO:teuthology.orchestra.run.smithi089.stdout: valid_lft 42267sec preferred_lft 42267sec 2024-09-21T03:37:34.965 INFO:teuthology.orchestra.run.smithi089.stdout: inet6 fe80::ec4:7aff:fe8f:d501/64 scope link noprefixroute 2024-09-21T03:37:34.965 INFO:teuthology.orchestra.run.smithi089.stdout: valid_lft forever preferred_lft forever 2024-09-21T03:37:34.965 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-09-21T03:37:34.966 DEBUG:teuthology.orchestra.run.smithi089:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T03:37:34.966 DEBUG:teuthology.orchestra.run.smithi089:> set -e 2024-09-21T03:37:34.966 DEBUG:teuthology.orchestra.run.smithi089:> sudo ip link add name ceph-brx type bridge 2024-09-21T03:37:34.966 DEBUG:teuthology.orchestra.run.smithi089:> sudo ip addr flush dev ceph-brx 2024-09-21T03:37:34.966 DEBUG:teuthology.orchestra.run.smithi089:> sudo ip link set ceph-brx up 2024-09-21T03:37:34.966 DEBUG:teuthology.orchestra.run.smithi089:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-09-21T03:37:34.966 DEBUG:teuthology.orchestra.run.smithi089:> ') 2024-09-21T03:37:35.056 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:35 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T03:37:35.163 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:37:34 smithi089 ceph-mon[30928]: pgmap v301: 129 pgs: 129 active+clean; 579 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:37:35.163 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:37:34 smithi089 ceph-mon[30928]: from='client.? 172.21.15.89:0/3369373281' entity='client.admin' cmd={"prefix": "fs ls"} : dispatch 2024-09-21T03:37:35.164 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:35 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T03:37:35.171 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:37:35.171 DEBUG:teuthology.orchestra.run.smithi089:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-21T03:37:35.195 INFO:teuthology.orchestra.run.smithi089.stdout:1 2024-09-21T03:37:35.197 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:37:35.197 DEBUG:teuthology.orchestra.run.smithi089:> route 2024-09-21T03:37:35.211 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:37:34 smithi114 ceph-mon[34140]: pgmap v301: 129 pgs: 129 active+clean; 579 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:37:35.211 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:37:34 smithi114 ceph-mon[34140]: from='client.? 172.21.15.89:0/3369373281' entity='client.admin' cmd={"prefix": "fs ls"} : dispatch 2024-09-21T03:37:35.250 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:37:34 smithi110 ceph-mon[34000]: pgmap v301: 129 pgs: 129 active+clean; 579 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:37:35.250 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:37:34 smithi110 ceph-mon[34000]: from='client.? 172.21.15.89:0/3369373281' entity='client.admin' cmd={"prefix": "fs ls"} : dispatch 2024-09-21T03:37:35.265 INFO:teuthology.orchestra.run.smithi089.stdout:Kernel IP routing table 2024-09-21T03:37:35.266 INFO:teuthology.orchestra.run.smithi089.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-21T03:37:35.266 INFO:teuthology.orchestra.run.smithi089.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-21T03:37:35.266 INFO:teuthology.orchestra.run.smithi089.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-21T03:37:35.266 INFO:teuthology.orchestra.run.smithi089.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-21T03:37:35.266 DEBUG:teuthology.orchestra.run.smithi089:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T03:37:35.266 DEBUG:teuthology.orchestra.run.smithi089:> set -e 2024-09-21T03:37:35.266 DEBUG:teuthology.orchestra.run.smithi089:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-21T03:37:35.266 DEBUG:teuthology.orchestra.run.smithi089:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-21T03:37:35.266 DEBUG:teuthology.orchestra.run.smithi089:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-21T03:37:35.267 DEBUG:teuthology.orchestra.run.smithi089:> ') 2024-09-21T03:37:35.353 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:35 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T03:37:35.667 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:35 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T03:37:35.674 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:37:35.674 DEBUG:teuthology.orchestra.run.smithi089:> ip netns list 2024-09-21T03:37:35.690 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:37:35.690 DEBUG:teuthology.orchestra.run.smithi089:> ip netns list-id 2024-09-21T03:37:35.745 DEBUG:teuthology.orchestra.run.smithi089:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T03:37:35.745 DEBUG:teuthology.orchestra.run.smithi089:> set -e 2024-09-21T03:37:35.745 DEBUG:teuthology.orchestra.run.smithi089:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-09-21T03:37:35.745 DEBUG:teuthology.orchestra.run.smithi089:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.admin 0 2024-09-21T03:37:35.745 DEBUG:teuthology.orchestra.run.smithi089:> ') 2024-09-21T03:37:35.832 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:35 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T03:37:35.856 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:35 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T03:37:35.862 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.admin' with 192.168.0.1/16 2024-09-21T03:37:35.862 DEBUG:teuthology.orchestra.run.smithi089:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T03:37:35.862 DEBUG:teuthology.orchestra.run.smithi089:> set -e 2024-09-21T03:37:35.863 DEBUG:teuthology.orchestra.run.smithi089:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.admin type veth peer name brx.0 2024-09-21T03:37:35.863 DEBUG:teuthology.orchestra.run.smithi089:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-09-21T03:37:35.863 DEBUG:teuthology.orchestra.run.smithi089:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set veth0 up 2024-09-21T03:37:35.863 DEBUG:teuthology.orchestra.run.smithi089:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set lo up 2024-09-21T03:37:35.863 DEBUG:teuthology.orchestra.run.smithi089:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip route add default via 192.168.255.254 2024-09-21T03:37:35.863 DEBUG:teuthology.orchestra.run.smithi089:> ') 2024-09-21T03:37:35.912 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:35 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T03:37:35.983 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:35 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T03:37:35.989 DEBUG:teuthology.orchestra.run.smithi089:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T03:37:35.989 DEBUG:teuthology.orchestra.run.smithi089:> set -e 2024-09-21T03:37:35.989 DEBUG:teuthology.orchestra.run.smithi089:> sudo ip link set brx.0 up 2024-09-21T03:37:35.989 DEBUG:teuthology.orchestra.run.smithi089:> sudo ip link set dev brx.0 master ceph-brx 2024-09-21T03:37:35.989 DEBUG:teuthology.orchestra.run.smithi089:> ') 2024-09-21T03:37:36.037 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:36 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T03:37:36.059 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:36 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T03:37:36.065 INFO:tasks.cephfs.fuse_mount:Client client.admin config is {} 2024-09-21T03:37:36.065 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-21T03:37:36.065 DEBUG:teuthology.orchestra.run.smithi089:> mkdir -p -v /home/ubuntu/cephtest/mnt.admin 2024-09-21T03:37:36.080 INFO:teuthology.orchestra.run.smithi089.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.admin' 2024-09-21T03:37:36.080 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-21T03:37:36.080 DEBUG:teuthology.orchestra.run.smithi089:> chmod 0000 /home/ubuntu/cephtest/mnt.admin 2024-09-21T03:37:36.135 DEBUG:teuthology.orchestra.run.smithi089:> sudo modprobe fuse 2024-09-21T03:37:36.201 DEBUG:teuthology.orchestra.run.smithi089:> cat /proc/self/mounts | awk '{print $2}' 2024-09-21T03:37:36.257 INFO:teuthology.orchestra.run.smithi089.stdout:/proc 2024-09-21T03:37:36.257 INFO:teuthology.orchestra.run.smithi089.stdout:/sys 2024-09-21T03:37:36.257 INFO:teuthology.orchestra.run.smithi089.stdout:/dev 2024-09-21T03:37:36.257 INFO:teuthology.orchestra.run.smithi089.stdout:/sys/kernel/security 2024-09-21T03:37:36.257 INFO:teuthology.orchestra.run.smithi089.stdout:/dev/shm 2024-09-21T03:37:36.257 INFO:teuthology.orchestra.run.smithi089.stdout:/dev/pts 2024-09-21T03:37:36.257 INFO:teuthology.orchestra.run.smithi089.stdout:/run 2024-09-21T03:37:36.257 INFO:teuthology.orchestra.run.smithi089.stdout:/sys/fs/cgroup 2024-09-21T03:37:36.257 INFO:teuthology.orchestra.run.smithi089.stdout:/sys/fs/pstore 2024-09-21T03:37:36.258 INFO:teuthology.orchestra.run.smithi089.stdout:/sys/fs/bpf 2024-09-21T03:37:36.258 INFO:teuthology.orchestra.run.smithi089.stdout:/ 2024-09-21T03:37:36.258 INFO:teuthology.orchestra.run.smithi089.stdout:/sys/fs/selinux 2024-09-21T03:37:36.258 INFO:teuthology.orchestra.run.smithi089.stdout:/proc/sys/fs/binfmt_misc 2024-09-21T03:37:36.258 INFO:teuthology.orchestra.run.smithi089.stdout:/sys/kernel/debug 2024-09-21T03:37:36.258 INFO:teuthology.orchestra.run.smithi089.stdout:/dev/mqueue 2024-09-21T03:37:36.258 INFO:teuthology.orchestra.run.smithi089.stdout:/sys/kernel/tracing 2024-09-21T03:37:36.258 INFO:teuthology.orchestra.run.smithi089.stdout:/dev/hugepages 2024-09-21T03:37:36.258 INFO:teuthology.orchestra.run.smithi089.stdout:/sys/fs/fuse/connections 2024-09-21T03:37:36.258 INFO:teuthology.orchestra.run.smithi089.stdout:/sys/kernel/config 2024-09-21T03:37:36.258 INFO:teuthology.orchestra.run.smithi089.stdout:/run/credentials/systemd-sysctl.service 2024-09-21T03:37:36.258 INFO:teuthology.orchestra.run.smithi089.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-09-21T03:37:36.258 INFO:teuthology.orchestra.run.smithi089.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-09-21T03:37:36.258 INFO:teuthology.orchestra.run.smithi089.stdout:/var/lib/nfs/rpc_pipefs 2024-09-21T03:37:36.258 INFO:teuthology.orchestra.run.smithi089.stdout:/run/user/1000 2024-09-21T03:37:36.259 INFO:teuthology.orchestra.run.smithi089.stdout:/var/lib/ceph 2024-09-21T03:37:36.259 INFO:teuthology.orchestra.run.smithi089.stdout:/var/lib/containers/storage/overlay 2024-09-21T03:37:36.259 INFO:teuthology.orchestra.run.smithi089.stdout:/var/lib/containers/storage/overlay/0b1266360c1e947b91c951aa5a770f203724c9783f3817320d7b3f833fdd3623/merged 2024-09-21T03:37:36.259 INFO:teuthology.orchestra.run.smithi089.stdout:/var/lib/containers/storage/overlay/74f483b2de23aa3db5ef9fda5e7c9ec9d9869ed0e09e152a27d160b31080f0d7/merged 2024-09-21T03:37:36.259 INFO:teuthology.orchestra.run.smithi089.stdout:/run/user/0 2024-09-21T03:37:36.259 INFO:teuthology.orchestra.run.smithi089.stdout:/proc/sys/fs/binfmt_misc 2024-09-21T03:37:36.259 INFO:teuthology.orchestra.run.smithi089.stdout:/var/lib/containers/storage/overlay/0c33d786533fa7c6c9b4ece0dd8c3bfc33ee457df01a35b7a874175fb0d24091/merged 2024-09-21T03:37:36.259 INFO:teuthology.orchestra.run.smithi089.stdout:/var/lib/containers/storage/overlay/1e263b0a542f58eaf76d73e377dd5959e47135a86a85f6eb5eae463c0822e8c2/merged 2024-09-21T03:37:36.259 INFO:teuthology.orchestra.run.smithi089.stdout:/var/lib/containers/storage/overlay/2f136aa314ce51b9e309ae462848aa731dc41b3b9ee884ab006901dd9af81ad8/merged 2024-09-21T03:37:36.259 INFO:teuthology.orchestra.run.smithi089.stdout:/var/lib/containers/storage/overlay/64f7a59823e11e80a119f6f784180bf56bca738c73a5be4e7e05673bb3400150/merged 2024-09-21T03:37:36.259 INFO:teuthology.orchestra.run.smithi089.stdout:/var/lib/containers/storage/overlay/cc0f955263cae7210080df6836f384f710eb6af21a08203f2d154f023c022d18/merged 2024-09-21T03:37:36.259 INFO:teuthology.orchestra.run.smithi089.stdout:/var/lib/containers/storage/overlay/9a935757ca5b5da6d9a52aaebf493ea1b50d289d444a1fc297330740dbd3169d/merged 2024-09-21T03:37:36.259 INFO:teuthology.orchestra.run.smithi089.stdout:/var/lib/containers/storage/overlay/050ed91c932a05cf2dc3c502aa3bf5b8822fb5b205b8284f46e11cf4638c07a2/merged 2024-09-21T03:37:36.259 INFO:teuthology.orchestra.run.smithi089.stdout:/var/lib/containers/storage/overlay/34ca3c8ad8cdf365681f5fde2e944c6107cead8c5aea4405ffe1ac85eb695e9c/merged 2024-09-21T03:37:36.259 INFO:teuthology.orchestra.run.smithi089.stdout:/run/netns 2024-09-21T03:37:36.259 INFO:teuthology.orchestra.run.smithi089.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-09-21T03:37:36.260 INFO:teuthology.orchestra.run.smithi089.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-09-21T03:37:36.260 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:37:36.260 DEBUG:teuthology.orchestra.run.smithi089:> ls /sys/fs/fuse/connections 2024-09-21T03:37:36.312 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-09-21T03:37:36.312 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.admin --id admin --client_fs=cephfs) 2024-09-21T03:37:36.354 DEBUG:teuthology.orchestra.run.smithi089:> sudo modprobe fuse 2024-09-21T03:37:36.378 DEBUG:teuthology.orchestra.run.smithi089:> cat /proc/self/mounts | awk '{print $2}' 2024-09-21T03:37:36.434 INFO:teuthology.orchestra.run.smithi089.stdout:/proc 2024-09-21T03:37:36.434 INFO:teuthology.orchestra.run.smithi089.stdout:/sys 2024-09-21T03:37:36.434 INFO:teuthology.orchestra.run.smithi089.stdout:/dev 2024-09-21T03:37:36.434 INFO:teuthology.orchestra.run.smithi089.stdout:/sys/kernel/security 2024-09-21T03:37:36.434 INFO:teuthology.orchestra.run.smithi089.stdout:/dev/shm 2024-09-21T03:37:36.434 INFO:teuthology.orchestra.run.smithi089.stdout:/dev/pts 2024-09-21T03:37:36.434 INFO:teuthology.orchestra.run.smithi089.stdout:/run 2024-09-21T03:37:36.434 INFO:teuthology.orchestra.run.smithi089.stdout:/sys/fs/cgroup 2024-09-21T03:37:36.434 INFO:teuthology.orchestra.run.smithi089.stdout:/sys/fs/pstore 2024-09-21T03:37:36.435 INFO:teuthology.orchestra.run.smithi089.stdout:/sys/fs/bpf 2024-09-21T03:37:36.435 INFO:teuthology.orchestra.run.smithi089.stdout:/ 2024-09-21T03:37:36.435 INFO:teuthology.orchestra.run.smithi089.stdout:/sys/fs/selinux 2024-09-21T03:37:36.435 INFO:teuthology.orchestra.run.smithi089.stdout:/proc/sys/fs/binfmt_misc 2024-09-21T03:37:36.435 INFO:teuthology.orchestra.run.smithi089.stdout:/sys/kernel/debug 2024-09-21T03:37:36.435 INFO:teuthology.orchestra.run.smithi089.stdout:/dev/mqueue 2024-09-21T03:37:36.435 INFO:teuthology.orchestra.run.smithi089.stdout:/sys/kernel/tracing 2024-09-21T03:37:36.435 INFO:teuthology.orchestra.run.smithi089.stdout:/dev/hugepages 2024-09-21T03:37:36.435 INFO:teuthology.orchestra.run.smithi089.stdout:/sys/fs/fuse/connections 2024-09-21T03:37:36.435 INFO:teuthology.orchestra.run.smithi089.stdout:/sys/kernel/config 2024-09-21T03:37:36.435 INFO:teuthology.orchestra.run.smithi089.stdout:/run/credentials/systemd-sysctl.service 2024-09-21T03:37:36.435 INFO:teuthology.orchestra.run.smithi089.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-09-21T03:37:36.435 INFO:teuthology.orchestra.run.smithi089.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-09-21T03:37:36.435 INFO:teuthology.orchestra.run.smithi089.stdout:/var/lib/nfs/rpc_pipefs 2024-09-21T03:37:36.436 INFO:teuthology.orchestra.run.smithi089.stdout:/run/user/1000 2024-09-21T03:37:36.436 INFO:teuthology.orchestra.run.smithi089.stdout:/var/lib/ceph 2024-09-21T03:37:36.436 INFO:teuthology.orchestra.run.smithi089.stdout:/var/lib/containers/storage/overlay 2024-09-21T03:37:36.436 INFO:teuthology.orchestra.run.smithi089.stdout:/var/lib/containers/storage/overlay/0b1266360c1e947b91c951aa5a770f203724c9783f3817320d7b3f833fdd3623/merged 2024-09-21T03:37:36.436 INFO:teuthology.orchestra.run.smithi089.stdout:/var/lib/containers/storage/overlay/74f483b2de23aa3db5ef9fda5e7c9ec9d9869ed0e09e152a27d160b31080f0d7/merged 2024-09-21T03:37:36.436 INFO:teuthology.orchestra.run.smithi089.stdout:/run/user/0 2024-09-21T03:37:36.436 INFO:teuthology.orchestra.run.smithi089.stdout:/proc/sys/fs/binfmt_misc 2024-09-21T03:37:36.436 INFO:teuthology.orchestra.run.smithi089.stdout:/var/lib/containers/storage/overlay/0c33d786533fa7c6c9b4ece0dd8c3bfc33ee457df01a35b7a874175fb0d24091/merged 2024-09-21T03:37:36.436 INFO:teuthology.orchestra.run.smithi089.stdout:/var/lib/containers/storage/overlay/1e263b0a542f58eaf76d73e377dd5959e47135a86a85f6eb5eae463c0822e8c2/merged 2024-09-21T03:37:36.436 INFO:teuthology.orchestra.run.smithi089.stdout:/var/lib/containers/storage/overlay/2f136aa314ce51b9e309ae462848aa731dc41b3b9ee884ab006901dd9af81ad8/merged 2024-09-21T03:37:36.436 INFO:teuthology.orchestra.run.smithi089.stdout:/var/lib/containers/storage/overlay/64f7a59823e11e80a119f6f784180bf56bca738c73a5be4e7e05673bb3400150/merged 2024-09-21T03:37:36.436 INFO:teuthology.orchestra.run.smithi089.stdout:/var/lib/containers/storage/overlay/cc0f955263cae7210080df6836f384f710eb6af21a08203f2d154f023c022d18/merged 2024-09-21T03:37:36.436 INFO:teuthology.orchestra.run.smithi089.stdout:/var/lib/containers/storage/overlay/9a935757ca5b5da6d9a52aaebf493ea1b50d289d444a1fc297330740dbd3169d/merged 2024-09-21T03:37:36.436 INFO:teuthology.orchestra.run.smithi089.stdout:/var/lib/containers/storage/overlay/050ed91c932a05cf2dc3c502aa3bf5b8822fb5b205b8284f46e11cf4638c07a2/merged 2024-09-21T03:37:36.436 INFO:teuthology.orchestra.run.smithi089.stdout:/var/lib/containers/storage/overlay/34ca3c8ad8cdf365681f5fde2e944c6107cead8c5aea4405ffe1ac85eb695e9c/merged 2024-09-21T03:37:36.437 INFO:teuthology.orchestra.run.smithi089.stdout:/run/netns 2024-09-21T03:37:36.437 INFO:teuthology.orchestra.run.smithi089.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-09-21T03:37:36.437 INFO:teuthology.orchestra.run.smithi089.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-09-21T03:37:36.437 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:37:36.437 DEBUG:teuthology.orchestra.run.smithi089:> ls /sys/fs/fuse/connections 2024-09-21T03:37:36.467 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi089.stderr:2024-09-21T03:37:36.466+0000 7f13927c9480 -1 init, newargv = 0x7f13800046a0 newargc=15 2024-09-21T03:37:36.467 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi089.stderr:ceph-fuse[75048]: starting ceph client 2024-09-21T03:37:36.474 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi089.stderr:ceph-fuse[75048]: starting fuse 2024-09-21T03:37:36.482 INFO:teuthology.orchestra.run.smithi089.stdout:120 2024-09-21T03:37:36.482 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [120] 2024-09-21T03:37:36.482 DEBUG:teuthology.orchestra.run.smithi089:> sudo stdin-killer -- python3 -c ' 2024-09-21T03:37:36.482 DEBUG:teuthology.orchestra.run.smithi089:> import glob 2024-09-21T03:37:36.483 DEBUG:teuthology.orchestra.run.smithi089:> import re 2024-09-21T03:37:36.483 DEBUG:teuthology.orchestra.run.smithi089:> import os 2024-09-21T03:37:36.483 DEBUG:teuthology.orchestra.run.smithi089:> import subprocess 2024-09-21T03:37:36.483 DEBUG:teuthology.orchestra.run.smithi089:> 2024-09-21T03:37:36.483 DEBUG:teuthology.orchestra.run.smithi089:> def _find_admin_socket(client_name): 2024-09-21T03:37:36.483 DEBUG:teuthology.orchestra.run.smithi089:> asok_path = "/var/run/ceph/ceph-client.admin.*.asok" 2024-09-21T03:37:36.483 DEBUG:teuthology.orchestra.run.smithi089:> files = glob.glob(asok_path) 2024-09-21T03:37:36.483 DEBUG:teuthology.orchestra.run.smithi089:> mountpoint = "/home/ubuntu/cephtest/mnt.admin" 2024-09-21T03:37:36.483 DEBUG:teuthology.orchestra.run.smithi089:> 2024-09-21T03:37:36.483 DEBUG:teuthology.orchestra.run.smithi089:> # Given a non-glob path, it better be there 2024-09-21T03:37:36.483 DEBUG:teuthology.orchestra.run.smithi089:> if "*" not in asok_path: 2024-09-21T03:37:36.483 DEBUG:teuthology.orchestra.run.smithi089:> assert(len(files) == 1) 2024-09-21T03:37:36.483 DEBUG:teuthology.orchestra.run.smithi089:> return files[0] 2024-09-21T03:37:36.483 DEBUG:teuthology.orchestra.run.smithi089:> 2024-09-21T03:37:36.483 DEBUG:teuthology.orchestra.run.smithi089:> for f in files: 2024-09-21T03:37:36.483 DEBUG:teuthology.orchestra.run.smithi089:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-09-21T03:37:36.483 DEBUG:teuthology.orchestra.run.smithi089:> if os.path.exists("/proc/{0}".format(pid)): 2024-09-21T03:37:36.484 DEBUG:teuthology.orchestra.run.smithi089:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-09-21T03:37:36.484 DEBUG:teuthology.orchestra.run.smithi089:> contents = proc_f.read() 2024-09-21T03:37:36.484 DEBUG:teuthology.orchestra.run.smithi089:> if mountpoint in contents: 2024-09-21T03:37:36.484 DEBUG:teuthology.orchestra.run.smithi089:> return f 2024-09-21T03:37:36.484 DEBUG:teuthology.orchestra.run.smithi089:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-09-21T03:37:36.484 DEBUG:teuthology.orchestra.run.smithi089:> 2024-09-21T03:37:36.484 DEBUG:teuthology.orchestra.run.smithi089:> print(_find_admin_socket("client.admin")) 2024-09-21T03:37:36.484 DEBUG:teuthology.orchestra.run.smithi089:> ' 2024-09-21T03:37:36.600 INFO:teuthology.orchestra.run.smithi089.stdout:/var/run/ceph/ceph-client.admin.75048.asok 2024-09-21T03:37:36.603 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:36 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T03:37:36.611 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.admin.75048.asok 2024-09-21T03:37:36.611 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:37:36.611 DEBUG:teuthology.orchestra.run.smithi089:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.admin.75048.asok status 2024-09-21T03:37:36.752 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-09-21T03:37:36.752 INFO:teuthology.orchestra.run.smithi089.stdout: "metadata": { 2024-09-21T03:37:36.752 INFO:teuthology.orchestra.run.smithi089.stdout: "ceph_sha1": "1843221895adccfb3b692060251dd6a0183bc410", 2024-09-21T03:37:36.753 INFO:teuthology.orchestra.run.smithi089.stdout: "ceph_version": "ceph version 19.3.0-5120-g18432218 (1843221895adccfb3b692060251dd6a0183bc410) squid (dev)", 2024-09-21T03:37:36.753 INFO:teuthology.orchestra.run.smithi089.stdout: "entity_id": "admin", 2024-09-21T03:37:36.753 INFO:teuthology.orchestra.run.smithi089.stdout: "hostname": "smithi089", 2024-09-21T03:37:36.753 INFO:teuthology.orchestra.run.smithi089.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.admin", 2024-09-21T03:37:36.753 INFO:teuthology.orchestra.run.smithi089.stdout: "pid": "75048", 2024-09-21T03:37:36.753 INFO:teuthology.orchestra.run.smithi089.stdout: "root": "/" 2024-09-21T03:37:36.753 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:37:36.753 INFO:teuthology.orchestra.run.smithi089.stdout: "dentry_count": 0, 2024-09-21T03:37:36.753 INFO:teuthology.orchestra.run.smithi089.stdout: "dentry_pinned_count": 0, 2024-09-21T03:37:36.753 INFO:teuthology.orchestra.run.smithi089.stdout: "id": 14919, 2024-09-21T03:37:36.753 INFO:teuthology.orchestra.run.smithi089.stdout: "inst": { 2024-09-21T03:37:36.753 INFO:teuthology.orchestra.run.smithi089.stdout: "name": { 2024-09-21T03:37:36.753 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "client", 2024-09-21T03:37:36.753 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 14919 2024-09-21T03:37:36.754 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:37:36.754 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": { 2024-09-21T03:37:36.754 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-09-21T03:37:36.754 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "192.168.0.1:0", 2024-09-21T03:37:36.754 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 4067956457 2024-09-21T03:37:36.754 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-21T03:37:36.754 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:37:36.754 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": { 2024-09-21T03:37:36.754 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-09-21T03:37:36.754 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "192.168.0.1:0", 2024-09-21T03:37:36.754 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 4067956457 2024-09-21T03:37:36.754 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:37:36.754 INFO:teuthology.orchestra.run.smithi089.stdout: "inst_str": "client.14919 192.168.0.1:0/4067956457", 2024-09-21T03:37:36.754 INFO:teuthology.orchestra.run.smithi089.stdout: "addr_str": "192.168.0.1:0/4067956457", 2024-09-21T03:37:36.755 INFO:teuthology.orchestra.run.smithi089.stdout: "inode_count": 1, 2024-09-21T03:37:36.755 INFO:teuthology.orchestra.run.smithi089.stdout: "mds_epoch": 17, 2024-09-21T03:37:36.755 INFO:teuthology.orchestra.run.smithi089.stdout: "osd_epoch": 80, 2024-09-21T03:37:36.755 INFO:teuthology.orchestra.run.smithi089.stdout: "osd_epoch_barrier": 0, 2024-09-21T03:37:36.755 INFO:teuthology.orchestra.run.smithi089.stdout: "blocklisted": false, 2024-09-21T03:37:36.755 INFO:teuthology.orchestra.run.smithi089.stdout: "fs_name": "cephfs" 2024-09-21T03:37:36.755 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-09-21T03:37:36.761 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:37:36.761 DEBUG:teuthology.orchestra.run.smithi089:> stat --file-system '--printf=%T 2024-09-21T03:37:36.761 DEBUG:teuthology.orchestra.run.smithi089:> ' -- /home/ubuntu/cephtest/mnt.admin 2024-09-21T03:37:36.777 INFO:teuthology.orchestra.run.smithi089.stdout:fuseblk 2024-09-21T03:37:36.777 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.admin 2024-09-21T03:37:36.777 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:37:36.777 DEBUG:teuthology.orchestra.run.smithi089:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.admin 2024-09-21T03:37:36.842 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.admin && exec stdin-killer --timeout=900 -- bash -c 'setfattr -n ceph.dir.layout.pool -v cephfs_data_ec . && getfattr -n ceph.dir.layout .') 2024-09-21T03:37:36.932 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:36 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-21T03:37:36.954 INFO:teuthology.orchestra.run.smithi089.stdout:# file: . 2024-09-21T03:37:36.954 INFO:teuthology.orchestra.run.smithi089.stdout:ceph.dir.layout="stripe_unit=4194304 stripe_count=1 object_size=4194304 pool=cephfs_data_ec" 2024-09-21T03:37:36.954 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-09-21T03:37:36.955 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:36 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T03:37:36.961 DEBUG:teuthology.orchestra.run.smithi089:> set -ex 2024-09-21T03:37:36.961 DEBUG:teuthology.orchestra.run.smithi089:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T03:37:36.977 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 7c374098-77c9-11ef-baf5-efdc52797490 -- ceph osd blocklist ls 2024-09-21T03:37:37.210 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:37:36 smithi114 ceph-mon[34140]: pgmap v302: 129 pgs: 129 active+clean; 579 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:37:37.250 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:37:36 smithi110 ceph-mon[34000]: pgmap v302: 129 pgs: 129 active+clean; 579 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:37:37.264 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/7c374098-77c9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:37:37.311 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:37:36 smithi089 ceph-mon[30928]: pgmap v302: 129 pgs: 129 active+clean; 579 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:37:37.804 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:37.793+0000 7fcf94509640 1 -- 172.21.15.89:0/3903205283 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcf8c1083c0 msgr2=0x7fcf8c1087d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:37.804 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:37.793+0000 7fcf94509640 1 --2- 172.21.15.89:0/3903205283 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcf8c1083c0 0x7fcf8c1087d0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fcf80009900 tx=0x7fcf8002a410 comp rx=0 tx=0).stop 2024-09-21T03:37:37.804 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:37.793+0000 7fcf94509640 1 -- 172.21.15.89:0/3903205283 shutdown_connections 2024-09-21T03:37:37.804 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:37.793+0000 7fcf94509640 1 --2- 172.21.15.89:0/3903205283 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fcf8c108d10 0x7fcf8c10f540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:37.804 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:37.793+0000 7fcf94509640 1 --2- 172.21.15.89:0/3903205283 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcf8c1083c0 0x7fcf8c1087d0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:37.804 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:37.793+0000 7fcf94509640 1 --2- 172.21.15.89:0/3903205283 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcf8c102a80 0x7fcf8c102e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:37.804 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:37.793+0000 7fcf94509640 1 -- 172.21.15.89:0/3903205283 >> 172.21.15.89:0/3903205283 conn(0x7fcf8c079740 msgr2=0x7fcf8c079b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:37:37.804 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:37.794+0000 7fcf94509640 1 -- 172.21.15.89:0/3903205283 shutdown_connections 2024-09-21T03:37:37.804 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:37.794+0000 7fcf94509640 1 -- 172.21.15.89:0/3903205283 wait complete. 2024-09-21T03:37:37.804 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:37.794+0000 7fcf94509640 1 Processor -- start 2024-09-21T03:37:37.805 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:37.794+0000 7fcf94509640 1 -- start start 2024-09-21T03:37:37.805 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:37.794+0000 7fcf94509640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fcf8c102a80 0x7fcf8c19a530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:37.805 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:37.794+0000 7fcf94509640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcf8c1083c0 0x7fcf8c19aad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:37.805 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:37.794+0000 7fcf94509640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcf8c108d10 0x7fcf8c19fd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:37.805 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:37.794+0000 7fcf94509640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fcf8c113950 con 0x7fcf8c108d10 2024-09-21T03:37:37.805 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:37.794+0000 7fcf94509640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fcf8c1137d0 con 0x7fcf8c1083c0 2024-09-21T03:37:37.805 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:37.794+0000 7fcf94509640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fcf8c113ad0 con 0x7fcf8c102a80 2024-09-21T03:37:37.805 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:37.795+0000 7fcf92aa6640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcf8c108d10 0x7fcf8c19fd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:37.805 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:37.795+0000 7fcf92aa6640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcf8c108d10 0x7fcf8c19fd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:48642/0 (socket says 172.21.15.89:48642) 2024-09-21T03:37:37.805 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:37.795+0000 7fcf92aa6640 1 -- 172.21.15.89:0/2454385499 learned_addr learned my addr 172.21.15.89:0/2454385499 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-21T03:37:37.805 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:37.795+0000 7fcf91aa4640 1 --2- 172.21.15.89:0/2454385499 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcf8c1083c0 0x7fcf8c19aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:37.805 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:37.795+0000 7fcf922a5640 1 --2- 172.21.15.89:0/2454385499 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fcf8c102a80 0x7fcf8c19a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:37.805 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:37.795+0000 7fcf92aa6640 1 -- 172.21.15.89:0/2454385499 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fcf8c102a80 msgr2=0x7fcf8c19a530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:37.806 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:37.795+0000 7fcf92aa6640 1 --2- 172.21.15.89:0/2454385499 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fcf8c102a80 0x7fcf8c19a530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:37.806 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:37.795+0000 7fcf92aa6640 1 -- 172.21.15.89:0/2454385499 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcf8c1083c0 msgr2=0x7fcf8c19aad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:37.806 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:37.795+0000 7fcf92aa6640 1 --2- 172.21.15.89:0/2454385499 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcf8c1083c0 0x7fcf8c19aad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:37.806 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:37.795+0000 7fcf92aa6640 1 -- 172.21.15.89:0/2454385499 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf8c1a04a0 con 0x7fcf8c108d10 2024-09-21T03:37:37.806 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:37.795+0000 7fcf92aa6640 1 --2- 172.21.15.89:0/2454385499 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcf8c108d10 0x7fcf8c19fd70 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fcf8400d750 tx=0x7fcf8400dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:37:37.806 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:37.795+0000 7fcf8b7fe640 1 -- 172.21.15.89:0/2454385499 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf84017070 con 0x7fcf8c108d10 2024-09-21T03:37:37.806 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:37.795+0000 7fcf8b7fe640 1 -- 172.21.15.89:0/2454385499 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fcf840054b0 con 0x7fcf8c108d10 2024-09-21T03:37:37.806 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:37.795+0000 7fcf94509640 1 -- 172.21.15.89:0/2454385499 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcf8c1a0790 con 0x7fcf8c108d10 2024-09-21T03:37:37.806 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:37.795+0000 7fcf8b7fe640 1 -- 172.21.15.89:0/2454385499 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf84004b90 con 0x7fcf8c108d10 2024-09-21T03:37:37.806 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:37.795+0000 7fcf94509640 1 -- 172.21.15.89:0/2454385499 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcf8c1a6ef0 con 0x7fcf8c108d10 2024-09-21T03:37:37.806 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:37.796+0000 7fcf94509640 1 -- 172.21.15.89:0/2454385499 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcf8c103d00 con 0x7fcf8c108d10 2024-09-21T03:37:37.806 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:37.798+0000 7fcf8b7fe640 1 -- 172.21.15.89:0/2454385499 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 17) ==== 154512+0+0 (secure 0 0 0) 0x7fcf84004d30 con 0x7fcf8c108d10 2024-09-21T03:37:37.806 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:37.799+0000 7fcf8b7fe640 1 --2- 172.21.15.89:0/2454385499 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7fcf740b6590 0x7fcf740b8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:37.806 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:37.799+0000 7fcf922a5640 1 --2- 172.21.15.89:0/2454385499 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7fcf740b6590 0x7fcf740b8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:37.806 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:37.799+0000 7fcf8b7fe640 1 -- 172.21.15.89:0/2454385499 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8802+0+0 (secure 0 0 0) 0x7fcf840e7330 con 0x7fcf8c108d10 2024-09-21T03:37:37.807 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:37.799+0000 7fcf922a5640 1 --2- 172.21.15.89:0/2454385499 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7fcf740b6590 0x7fcf740b8aa0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fcf7c005c40 tx=0x7fcf7c005bd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:37:37.807 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:37.803+0000 7fcf8b7fe640 1 -- 172.21.15.89:0/2454385499 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fcf840ada00 con 0x7fcf8c108d10 2024-09-21T03:37:37.995 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:37.993+0000 7fcf94509640 1 -- 172.21.15.89:0/2454385499 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fcf8c102e90 con 0x7fcf8c108d10 2024-09-21T03:37:37.995 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:37.994+0000 7fcf8b7fe640 1 -- 172.21.15.89:0/2454385499 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v80) ==== 81+0+590 (secure 0 0 0) 0x7fcf840b1de0 con 0x7fcf8c108d10 2024-09-21T03:37:37.995 INFO:teuthology.orchestra.run.smithi089.stdout:172.21.15.89:6801/655110910 2024-09-22T03:29:51.519106+0000 2024-09-21T03:37:37.995 INFO:teuthology.orchestra.run.smithi089.stdout:172.21.15.89:0/1618544983 2024-09-22T03:29:51.519106+0000 2024-09-21T03:37:37.996 INFO:teuthology.orchestra.run.smithi089.stdout:172.21.15.89:0/1265502758 2024-09-22T03:29:51.519106+0000 2024-09-21T03:37:37.996 INFO:teuthology.orchestra.run.smithi089.stdout:172.21.15.89:0/4017684419 2024-09-22T03:29:51.519106+0000 2024-09-21T03:37:37.996 INFO:teuthology.orchestra.run.smithi089.stdout:172.21.15.89:6801/3312712420 2024-09-22T03:29:26.004686+0000 2024-09-21T03:37:37.996 INFO:teuthology.orchestra.run.smithi089.stdout:172.21.15.89:6800/3312712420 2024-09-22T03:29:26.004686+0000 2024-09-21T03:37:37.996 INFO:teuthology.orchestra.run.smithi089.stdout:172.21.15.89:0/2056576483 2024-09-22T03:29:26.004686+0000 2024-09-21T03:37:37.996 INFO:teuthology.orchestra.run.smithi089.stdout:172.21.15.89:6800/655110910 2024-09-22T03:29:51.519106+0000 2024-09-21T03:37:37.996 INFO:teuthology.orchestra.run.smithi089.stdout:172.21.15.89:0/2382866302 2024-09-22T03:29:26.004686+0000 2024-09-21T03:37:37.996 INFO:teuthology.orchestra.run.smithi089.stdout:172.21.15.89:0/1584351801 2024-09-22T03:29:26.004686+0000 2024-09-21T03:37:37.996 INFO:teuthology.orchestra.run.smithi089.stderr:listed 10 entries 2024-09-21T03:37:37.997 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:37.996+0000 7fcf94509640 1 -- 172.21.15.89:0/2454385499 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7fcf740b6590 msgr2=0x7fcf740b8aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:37.997 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:37.996+0000 7fcf94509640 1 --2- 172.21.15.89:0/2454385499 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7fcf740b6590 0x7fcf740b8aa0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fcf7c005c40 tx=0x7fcf7c005bd0 comp rx=0 tx=0).stop 2024-09-21T03:37:37.997 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:37.996+0000 7fcf94509640 1 -- 172.21.15.89:0/2454385499 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcf8c108d10 msgr2=0x7fcf8c19fd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:37.997 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:37.996+0000 7fcf94509640 1 --2- 172.21.15.89:0/2454385499 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcf8c108d10 0x7fcf8c19fd70 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fcf8400d750 tx=0x7fcf8400dc10 comp rx=0 tx=0).stop 2024-09-21T03:37:37.998 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:37.996+0000 7fcf94509640 1 -- 172.21.15.89:0/2454385499 shutdown_connections 2024-09-21T03:37:37.998 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:37.996+0000 7fcf94509640 1 --2- 172.21.15.89:0/2454385499 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7fcf740b6590 0x7fcf740b8aa0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:37.998 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:37.996+0000 7fcf94509640 1 --2- 172.21.15.89:0/2454385499 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcf8c1083c0 0x7fcf8c19aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:37.998 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:37.996+0000 7fcf94509640 1 --2- 172.21.15.89:0/2454385499 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcf8c108d10 0x7fcf8c19fd70 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:37.998 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:37.996+0000 7fcf94509640 1 --2- 172.21.15.89:0/2454385499 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fcf8c102a80 0x7fcf8c19a530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:37.998 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:37.996+0000 7fcf94509640 1 -- 172.21.15.89:0/2454385499 >> 172.21.15.89:0/2454385499 conn(0x7fcf8c079740 msgr2=0x7fcf8c1053d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:37:37.998 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:37.996+0000 7fcf94509640 1 -- 172.21.15.89:0/2454385499 shutdown_connections 2024-09-21T03:37:37.998 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:37.996+0000 7fcf94509640 1 -- 172.21.15.89:0/2454385499 wait complete. 2024-09-21T03:37:38.816 DEBUG:teuthology.orchestra.run.smithi089:> set -ex 2024-09-21T03:37:38.816 DEBUG:teuthology.orchestra.run.smithi089:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T03:37:38.835 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 7c374098-77c9-11ef-baf5-efdc52797490 -- ceph osd blocklist ls 2024-09-21T03:37:39.061 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:37:38 smithi089 ceph-mon[30928]: pgmap v303: 129 pgs: 129 active+clean; 579 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:37:39.061 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:37:38 smithi089 ceph-mon[30928]: from='client.? 172.21.15.89:0/2454385499' entity='client.admin' cmd={"prefix": "osd blocklist ls"} : dispatch 2024-09-21T03:37:39.125 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/7c374098-77c9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:37:39.211 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:37:38 smithi114 ceph-mon[34140]: pgmap v303: 129 pgs: 129 active+clean; 579 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:37:39.211 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:37:38 smithi114 ceph-mon[34140]: from='client.? 172.21.15.89:0/2454385499' entity='client.admin' cmd={"prefix": "osd blocklist ls"} : dispatch 2024-09-21T03:37:39.250 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:37:38 smithi110 ceph-mon[34000]: pgmap v303: 129 pgs: 129 active+clean; 579 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:37:39.250 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:37:38 smithi110 ceph-mon[34000]: from='client.? 172.21.15.89:0/2454385499' entity='client.admin' cmd={"prefix": "osd blocklist ls"} : dispatch 2024-09-21T03:37:39.689 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:39.685+0000 7f42dc81d640 1 -- 172.21.15.89:0/2906581006 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f42d8109e10 msgr2=0x7f42d8110640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:39.689 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:39.685+0000 7f42dc81d640 1 --2- 172.21.15.89:0/2906581006 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f42d8109e10 0x7f42d8110640 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f42d0009ca0 tx=0x7f42d002a8c0 comp rx=0 tx=0).stop 2024-09-21T03:37:39.689 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:39.685+0000 7f42dc81d640 1 -- 172.21.15.89:0/2906581006 shutdown_connections 2024-09-21T03:37:39.689 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:39.685+0000 7f42dc81d640 1 --2- 172.21.15.89:0/2906581006 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f42d8109e10 0x7f42d8110640 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:39.689 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:39.685+0000 7f42dc81d640 1 --2- 172.21.15.89:0/2906581006 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f42d81094c0 0x7f42d81098d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:39.690 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:39.685+0000 7f42dc81d640 1 --2- 172.21.15.89:0/2906581006 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f42d8103b80 0x7f42d8103f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:39.690 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:39.685+0000 7f42dc81d640 1 -- 172.21.15.89:0/2906581006 >> 172.21.15.89:0/2906581006 conn(0x7f42d80ff5d0 msgr2=0x7f42d8101a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:37:39.690 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:39.685+0000 7f42dc81d640 1 -- 172.21.15.89:0/2906581006 shutdown_connections 2024-09-21T03:37:39.690 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:39.685+0000 7f42dc81d640 1 -- 172.21.15.89:0/2906581006 wait complete. 2024-09-21T03:37:39.690 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:39.686+0000 7f42dc81d640 1 Processor -- start 2024-09-21T03:37:39.690 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:39.686+0000 7f42dc81d640 1 -- start start 2024-09-21T03:37:39.690 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:39.686+0000 7f42dc81d640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f42d8103b80 0x7f42d819a2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:39.690 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:39.686+0000 7f42dc81d640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f42d81094c0 0x7f42d819a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:39.690 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:39.686+0000 7f42dc81d640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f42d8109e10 0x7f42d819fae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:39.691 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:39.686+0000 7f42dc81d640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f42d81139d0 con 0x7f42d8103b80 2024-09-21T03:37:39.691 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:39.686+0000 7f42dc81d640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f42d8113850 con 0x7f42d8109e10 2024-09-21T03:37:39.691 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:39.686+0000 7f42dc81d640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f42d8113b50 con 0x7f42d81094c0 2024-09-21T03:37:39.691 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:39.687+0000 7f42d659c640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f42d8103b80 0x7f42d819a2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:39.691 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:39.687+0000 7f42d659c640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f42d8103b80 0x7f42d819a2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:47226/0 (socket says 172.21.15.89:47226) 2024-09-21T03:37:39.691 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:39.687+0000 7f42d659c640 1 -- 172.21.15.89:0/2192288835 learned_addr learned my addr 172.21.15.89:0/2192288835 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-21T03:37:39.691 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:39.687+0000 7f42cdd9b640 1 --2- 172.21.15.89:0/2192288835 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f42d81094c0 0x7f42d819a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:39.691 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:39.687+0000 7f42d6d9d640 1 --2- 172.21.15.89:0/2192288835 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f42d8109e10 0x7f42d819fae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:39.691 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:39.687+0000 7f42d659c640 1 -- 172.21.15.89:0/2192288835 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f42d81094c0 msgr2=0x7f42d819a840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:39.691 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:39.687+0000 7f42d659c640 1 --2- 172.21.15.89:0/2192288835 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f42d81094c0 0x7f42d819a840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:39.692 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:39.687+0000 7f42d659c640 1 -- 172.21.15.89:0/2192288835 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f42d8109e10 msgr2=0x7f42d819fae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:39.692 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:39.687+0000 7f42d659c640 1 --2- 172.21.15.89:0/2192288835 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f42d8109e10 0x7f42d819fae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:39.692 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:39.687+0000 7f42d659c640 1 -- 172.21.15.89:0/2192288835 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42d81a0150 con 0x7f42d8103b80 2024-09-21T03:37:39.692 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:39.687+0000 7f42d659c640 1 --2- 172.21.15.89:0/2192288835 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f42d8103b80 0x7f42d819a2a0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f42c800c920 tx=0x7f42c800cd20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:37:39.692 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:39.687+0000 7f42cffff640 1 -- 172.21.15.89:0/2192288835 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42c8007bb0 con 0x7f42d8103b80 2024-09-21T03:37:39.692 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:39.687+0000 7f42dc81d640 1 -- 172.21.15.89:0/2192288835 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f42d81a0440 con 0x7f42d8103b80 2024-09-21T03:37:39.692 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:39.687+0000 7f42cffff640 1 -- 172.21.15.89:0/2192288835 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f42c8007d50 con 0x7f42d8103b80 2024-09-21T03:37:39.692 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:39.687+0000 7f42cffff640 1 -- 172.21.15.89:0/2192288835 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42c8018660 con 0x7f42d8103b80 2024-09-21T03:37:39.692 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:39.687+0000 7f42dc81d640 1 -- 172.21.15.89:0/2192288835 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f42d81a6c60 con 0x7f42d8103b80 2024-09-21T03:37:39.693 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:39.689+0000 7f42dc81d640 1 -- 172.21.15.89:0/2192288835 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f42d8103f90 con 0x7f42d8103b80 2024-09-21T03:37:39.693 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:39.691+0000 7f42cffff640 1 -- 172.21.15.89:0/2192288835 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 17) ==== 154512+0+0 (secure 0 0 0) 0x7f42c800f450 con 0x7f42d8103b80 2024-09-21T03:37:39.693 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:39.692+0000 7f42cffff640 1 --2- 172.21.15.89:0/2192288835 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f42c40b6540 0x7f42c40b8a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:39.693 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:39.692+0000 7f42cdd9b640 1 --2- 172.21.15.89:0/2192288835 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f42c40b6540 0x7f42c40b8a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:39.693 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:39.692+0000 7f42cffff640 1 -- 172.21.15.89:0/2192288835 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8802+0+0 (secure 0 0 0) 0x7f42c80e7880 con 0x7f42d8103b80 2024-09-21T03:37:39.694 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:39.692+0000 7f42cdd9b640 1 --2- 172.21.15.89:0/2192288835 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f42c40b6540 0x7f42c40b8a50 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f42d8108dd0 tx=0x7f42c000a2d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:37:39.696 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:39.695+0000 7f42cffff640 1 -- 172.21.15.89:0/2192288835 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f42c8014030 con 0x7f42d8103b80 2024-09-21T03:37:39.888 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:39.887+0000 7f42dc81d640 1 -- 172.21.15.89:0/2192288835 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f42d819b3d0 con 0x7f42d8103b80 2024-09-21T03:37:39.888 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:39.887+0000 7f42cffff640 1 -- 172.21.15.89:0/2192288835 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v80) ==== 81+0+590 (secure 0 0 0) 0x7f42d819b3d0 con 0x7f42d8103b80 2024-09-21T03:37:39.889 INFO:teuthology.orchestra.run.smithi089.stdout:172.21.15.89:6801/655110910 2024-09-22T03:29:51.519106+0000 2024-09-21T03:37:39.889 INFO:teuthology.orchestra.run.smithi089.stdout:172.21.15.89:0/1618544983 2024-09-22T03:29:51.519106+0000 2024-09-21T03:37:39.889 INFO:teuthology.orchestra.run.smithi089.stdout:172.21.15.89:0/1265502758 2024-09-22T03:29:51.519106+0000 2024-09-21T03:37:39.889 INFO:teuthology.orchestra.run.smithi089.stdout:172.21.15.89:0/4017684419 2024-09-22T03:29:51.519106+0000 2024-09-21T03:37:39.889 INFO:teuthology.orchestra.run.smithi089.stdout:172.21.15.89:6801/3312712420 2024-09-22T03:29:26.004686+0000 2024-09-21T03:37:39.889 INFO:teuthology.orchestra.run.smithi089.stdout:172.21.15.89:6800/3312712420 2024-09-22T03:29:26.004686+0000 2024-09-21T03:37:39.889 INFO:teuthology.orchestra.run.smithi089.stdout:172.21.15.89:0/2056576483 2024-09-22T03:29:26.004686+0000 2024-09-21T03:37:39.889 INFO:teuthology.orchestra.run.smithi089.stdout:172.21.15.89:6800/655110910 2024-09-22T03:29:51.519106+0000 2024-09-21T03:37:39.889 INFO:teuthology.orchestra.run.smithi089.stdout:172.21.15.89:0/2382866302 2024-09-22T03:29:26.004686+0000 2024-09-21T03:37:39.889 INFO:teuthology.orchestra.run.smithi089.stdout:172.21.15.89:0/1584351801 2024-09-22T03:29:26.004686+0000 2024-09-21T03:37:39.890 INFO:teuthology.orchestra.run.smithi089.stderr:listed 10 entries 2024-09-21T03:37:39.892 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:39.890+0000 7f42dc81d640 1 -- 172.21.15.89:0/2192288835 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f42c40b6540 msgr2=0x7f42c40b8a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:39.892 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:39.890+0000 7f42dc81d640 1 --2- 172.21.15.89:0/2192288835 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f42c40b6540 0x7f42c40b8a50 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f42d8108dd0 tx=0x7f42c000a2d0 comp rx=0 tx=0).stop 2024-09-21T03:37:39.892 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:39.890+0000 7f42dc81d640 1 -- 172.21.15.89:0/2192288835 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f42d8103b80 msgr2=0x7f42d819a2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:39.892 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:39.890+0000 7f42dc81d640 1 --2- 172.21.15.89:0/2192288835 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f42d8103b80 0x7f42d819a2a0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f42c800c920 tx=0x7f42c800cd20 comp rx=0 tx=0).stop 2024-09-21T03:37:39.892 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:39.890+0000 7f42dc81d640 1 -- 172.21.15.89:0/2192288835 shutdown_connections 2024-09-21T03:37:39.892 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:39.890+0000 7f42dc81d640 1 --2- 172.21.15.89:0/2192288835 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f42c40b6540 0x7f42c40b8a50 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:39.892 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:39.890+0000 7f42dc81d640 1 --2- 172.21.15.89:0/2192288835 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f42d8109e10 0x7f42d819fae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:39.892 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:39.891+0000 7f42dc81d640 1 --2- 172.21.15.89:0/2192288835 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f42d81094c0 0x7f42d819a840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:39.892 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:39.891+0000 7f42dc81d640 1 --2- 172.21.15.89:0/2192288835 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f42d8103b80 0x7f42d819a2a0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:39.893 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:39.891+0000 7f42dc81d640 1 -- 172.21.15.89:0/2192288835 >> 172.21.15.89:0/2192288835 conn(0x7f42d80ff5d0 msgr2=0x7f42d81019f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:37:39.893 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:39.891+0000 7f42dc81d640 1 -- 172.21.15.89:0/2192288835 shutdown_connections 2024-09-21T03:37:39.893 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:39.891+0000 7f42dc81d640 1 -- 172.21.15.89:0/2192288835 wait complete. 2024-09-21T03:37:40.250 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:37:39 smithi110 ceph-mon[34000]: from='client.? 172.21.15.89:0/2192288835' entity='client.admin' cmd={"prefix": "osd blocklist ls"} : dispatch 2024-09-21T03:37:40.267 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:37:39 smithi089 ceph-mon[30928]: from='client.? 172.21.15.89:0/2192288835' entity='client.admin' cmd={"prefix": "osd blocklist ls"} : dispatch 2024-09-21T03:37:40.461 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:37:39 smithi114 ceph-mon[34140]: from='client.? 172.21.15.89:0/2192288835' entity='client.admin' cmd={"prefix": "osd blocklist ls"} : dispatch 2024-09-21T03:37:40.547 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi089.front.sepia.ceph.com... 2024-09-21T03:37:40.547 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:37:40.547 DEBUG:teuthology.orchestra.run.smithi089:> sudo fusermount -u /home/ubuntu/cephtest/mnt.admin 2024-09-21T03:37:40.572 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi089.stderr:ceph-fuse[75048]: fuse finished with error 0 and tester_r 0 2024-09-21T03:37:40.574 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T03:37:40.642 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi089.front.sepia.ceph.com 2024-09-21T03:37:40.642 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:37:40.642 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.admin) 2024-09-21T03:37:40.668 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 7c374098-77c9-11ef-baf5-efdc52797490 -- ceph osd dump --format=json 2024-09-21T03:37:40.955 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/7c374098-77c9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:37:41.250 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:37:40 smithi110 ceph-mon[34000]: pgmap v304: 129 pgs: 129 active+clean; 581 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-09-21T03:37:41.311 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:37:40 smithi089 ceph-mon[30928]: pgmap v304: 129 pgs: 129 active+clean; 581 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-09-21T03:37:41.420 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:41.418+0000 7f47a43a9640 1 -- 172.21.15.89:0/174159668 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f479c109e40 msgr2=0x7f479c110670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:41.420 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:41.418+0000 7f47a43a9640 1 --2- 172.21.15.89:0/174159668 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f479c109e40 0x7f479c110670 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f4794009ca0 tx=0x7f479402a8c0 comp rx=0 tx=0).stop 2024-09-21T03:37:41.420 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:41.419+0000 7f47a43a9640 1 -- 172.21.15.89:0/174159668 shutdown_connections 2024-09-21T03:37:41.420 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:41.419+0000 7f47a43a9640 1 --2- 172.21.15.89:0/174159668 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f479c1094f0 0x7f479c109900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:41.421 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:41.419+0000 7f47a43a9640 1 --2- 172.21.15.89:0/174159668 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f479c109e40 0x7f479c110670 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:41.421 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:41.419+0000 7f47a43a9640 1 --2- 172.21.15.89:0/174159668 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f479c103bb0 0x7f479c103fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:41.421 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:41.419+0000 7f47a43a9640 1 -- 172.21.15.89:0/174159668 >> 172.21.15.89:0/174159668 conn(0x7f479c0ff600 msgr2=0x7f479c101a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:37:41.421 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:41.419+0000 7f47a43a9640 1 -- 172.21.15.89:0/174159668 shutdown_connections 2024-09-21T03:37:41.421 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:41.419+0000 7f47a43a9640 1 -- 172.21.15.89:0/174159668 wait complete. 2024-09-21T03:37:41.421 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:41.419+0000 7f47a43a9640 1 Processor -- start 2024-09-21T03:37:41.421 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:41.419+0000 7f47a43a9640 1 -- start start 2024-09-21T03:37:41.423 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:41.419+0000 7f47a43a9640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f479c103bb0 0x7f479c19a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:41.423 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:41.420+0000 7f47a43a9640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f479c1094f0 0x7f479c19a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:41.423 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:41.420+0000 7f47a43a9640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f479c109e40 0x7f479c19fc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:41.423 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:41.420+0000 7f47a43a9640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f479c113a60 con 0x7f479c109e40 2024-09-21T03:37:41.423 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:41.420+0000 7f47a43a9640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f479c1138e0 con 0x7f479c1094f0 2024-09-21T03:37:41.423 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:41.420+0000 7f47a43a9640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f479c113be0 con 0x7f479c103bb0 2024-09-21T03:37:41.423 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:41.420+0000 7f47a2946640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f479c109e40 0x7f479c19fc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:41.423 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:41.420+0000 7f47a2946640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f479c109e40 0x7f479c19fc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:47236/0 (socket says 172.21.15.89:47236) 2024-09-21T03:37:41.423 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:41.420+0000 7f47a2946640 1 -- 172.21.15.89:0/637619456 learned_addr learned my addr 172.21.15.89:0/637619456 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-21T03:37:41.423 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:41.420+0000 7f47a2145640 1 --2- 172.21.15.89:0/637619456 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f479c103bb0 0x7f479c19a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:41.423 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:41.420+0000 7f47a1944640 1 --2- 172.21.15.89:0/637619456 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f479c1094f0 0x7f479c19a9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:41.423 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:41.420+0000 7f47a2946640 1 -- 172.21.15.89:0/637619456 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f479c103bb0 msgr2=0x7f479c19a430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:41.423 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:41.420+0000 7f47a2946640 1 --2- 172.21.15.89:0/637619456 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f479c103bb0 0x7f479c19a430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:41.424 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:41.420+0000 7f47a2946640 1 -- 172.21.15.89:0/637619456 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f479c1094f0 msgr2=0x7f479c19a9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:41.424 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:41.420+0000 7f47a2946640 1 --2- 172.21.15.89:0/637619456 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f479c1094f0 0x7f479c19a9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:41.424 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:41.420+0000 7f47a2946640 1 -- 172.21.15.89:0/637619456 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f479c1a03a0 con 0x7f479c109e40 2024-09-21T03:37:41.424 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:41.420+0000 7f47a2946640 1 --2- 172.21.15.89:0/637619456 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f479c109e40 0x7f479c19fc70 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f4794005cc0 tx=0x7f4794032c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:37:41.424 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:41.420+0000 7f47937fe640 1 -- 172.21.15.89:0/637619456 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4794004040 con 0x7f479c109e40 2024-09-21T03:37:41.424 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:41.420+0000 7f47937fe640 1 -- 172.21.15.89:0/637619456 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4794004670 con 0x7f479c109e40 2024-09-21T03:37:41.424 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:41.420+0000 7f47a43a9640 1 -- 172.21.15.89:0/637619456 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f479c1a0630 con 0x7f479c109e40 2024-09-21T03:37:41.424 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:41.420+0000 7f47937fe640 1 -- 172.21.15.89:0/637619456 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4794030cc0 con 0x7f479c109e40 2024-09-21T03:37:41.424 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:41.421+0000 7f47a43a9640 1 -- 172.21.15.89:0/637619456 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f479c1a6df0 con 0x7f479c109e40 2024-09-21T03:37:41.424 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:41.421+0000 7f47a43a9640 1 -- 172.21.15.89:0/637619456 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f479c103fc0 con 0x7f479c109e40 2024-09-21T03:37:41.425 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:41.423+0000 7f47937fe640 1 -- 172.21.15.89:0/637619456 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 17) ==== 154512+0+0 (secure 0 0 0) 0x7f47940041e0 con 0x7f479c109e40 2024-09-21T03:37:41.425 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:41.424+0000 7f47937fe640 1 --2- 172.21.15.89:0/637619456 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f47880b6510 0x7f47880b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:41.425 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:41.424+0000 7f47a2145640 1 --2- 172.21.15.89:0/637619456 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f47880b6510 0x7f47880b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:41.425 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:41.424+0000 7f47937fe640 1 -- 172.21.15.89:0/637619456 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8802+0+0 (secure 0 0 0) 0x7f4794108b20 con 0x7f479c109e40 2024-09-21T03:37:41.425 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:41.424+0000 7f47a2145640 1 --2- 172.21.15.89:0/637619456 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f47880b6510 0x7f47880b8a20 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f4798009990 tx=0x7f4798009110 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:37:41.428 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:41.427+0000 7f47937fe640 1 -- 172.21.15.89:0/637619456 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f4794108e20 con 0x7f479c109e40 2024-09-21T03:37:41.460 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:37:40 smithi114 ceph-mon[34140]: pgmap v304: 129 pgs: 129 active+clean; 581 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-09-21T03:37:41.608 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:41.606+0000 7f47a43a9640 1 -- 172.21.15.89:0/637619456 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f479c10bc70 con 0x7f479c109e40 2024-09-21T03:37:41.609 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:41.608+0000 7f47937fe640 1 -- 172.21.15.89:0/637619456 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v80) ==== 74+0+24236 (secure 0 0 0) 0x7f47940cf200 con 0x7f479c109e40 2024-09-21T03:37:41.609 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-09-21T03:37:41.609 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":80,"fsid":"7c374098-77c9-11ef-baf5-efdc52797490","created":"2024-09-21T03:28:56.681458+0000","modified":"2024-09-21T03:37:22.518390+0000","last_up_change":"2024-09-21T03:36:23.368379+0000","last_in_change":"2024-09-21T03:36:00.131331+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T03:32:37.834339+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-21T03:37:09.443341+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"74","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-21T03:37:11.467821+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"74","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-21T03:37:17.884066+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"27ca5ed8-17d1-4f8e-a7ae-084a914721a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6802","nonce":990539042},{"type":"v1","addr":"172.21.15.89:6803","nonce":990539042}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6804","nonce":990539042},{"type":"v1","addr":"172.21.15.89:6805","nonce":990539042}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6808","nonce":990539042},{"type":"v1","addr":"172.21.15.89:6809","nonce":990539042}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6806","nonce":990539042},{"type":"v1","addr":"172.21.15.89:6807","nonce":990539042}]},"public_addr":"172.21.15.89:6803/990539042","cluster_addr":"172.21.15.89:6805/990539042","heartbeat_back_addr":"172.21.15.89:6809/990539042","heartbeat_front_addr":"172.21.15.89:6807/990539042","state":["exists","up"]},{"osd":1,"uuid":"5176a910-5cf9-49d7-aedb-a20cf36ea075","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6800","nonce":3235852628},{"type":"v1","addr":"172.21.15.110:6801","nonce":3235852628}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6802","nonce":3235852628},{"type":"v1","addr":"172.21.15.110:6803","nonce":3235852628}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6806","nonce":3235852628},{"type":"v1","addr":"172.21.15.110:6807","nonce":3235852628}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6804","nonce":3235852628},{"type":"v1","addr":"172.21.15.110:6805","nonce":3235852628}]},"public_addr":"172.21.15.110:6801/3235852628","cluster_addr":"172.21.15.110:6803/3235852628","heartbeat_back_addr":"172.21.15.110:6807/3235852628","heartbeat_front_addr":"172.21.15.110:6805/3235852628","state":["exists","up"]},{"osd":2,"uuid":"d4977df1-b118-4d7e-b9a9-626a8accc812","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6800","nonce":169452122},{"type":"v1","addr":"172.21.15.114:6801","nonce":169452122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6802","nonce":169452122},{"type":"v1","addr":"172.21.15.114:6803","nonce":169452122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6806","nonce":169452122},{"type":"v1","addr":"172.21.15.114:6807","nonce":169452122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6804","nonce":169452122},{"type":"v1","addr":"172.21.15.114:6805","nonce":169452122}]},"public_addr":"172.21.15.114:6801/169452122","cluster_addr":"172.21.15.114:6803/169452122","heartbeat_back_addr":"172.21.15.114:6807/169452122","heartbeat_front_addr":"172.21.15.114:6805/169452122","state":["exists","up"]},{"osd":3,"uuid":"2c7ae1b4-0bf2-4954-81ed-55d03eab82c6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6810","nonce":3016980877},{"type":"v1","addr":"172.21.15.89:6811","nonce":3016980877}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6812","nonce":3016980877},{"type":"v1","addr":"172.21.15.89:6813","nonce":3016980877}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6816","nonce":3016980877},{"type":"v1","addr":"172.21.15.89:6817","nonce":3016980877}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6814","nonce":3016980877},{"type":"v1","addr":"172.21.15.89:6815","nonce":3016980877}]},"public_addr":"172.21.15.89:6811/3016980877","cluster_addr":"172.21.15.89:6813/3016980877","heartbeat_back_addr":"172.21.15.89:6817/3016980877","heartbeat_front_addr":"172.21.15.89:6815/3016980877","state":["exists","up"]},{"osd":4,"uuid":"8e65399d-6e55-4da4-98ba-f805f0c655a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":30,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6808","nonce":3217880476},{"type":"v1","addr":"172.21.15.110:6809","nonce":3217880476}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6810","nonce":3217880476},{"type":"v1","addr":"172.21.15.110:6811","nonce":3217880476}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6814","nonce":3217880476},{"type":"v1","addr":"172.21.15.110:6815","nonce":3217880476}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6812","nonce":3217880476},{"type":"v1","addr":"172.21.15.110:6813","nonce":3217880476}]},"public_addr":"172.21.15.110:6809/3217880476","cluster_addr":"172.21.15.110:6811/3217880476","heartbeat_back_addr":"172.21.15.110:6815/3217880476","heartbeat_front_addr":"172.21.15.110:6813/3217880476","state":["exists","up"]},{"osd":5,"uuid":"b9ec2889-46bd-40e5-8446-73ab28271c67","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6808","nonce":1144140707},{"type":"v1","addr":"172.21.15.114:6809","nonce":1144140707}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6810","nonce":1144140707},{"type":"v1","addr":"172.21.15.114:6811","nonce":1144140707}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6814","nonce":1144140707},{"type":"v1","addr":"172.21.15.114:6815","nonce":1144140707}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6812","nonce":1144140707},{"type":"v1","addr":"172.21.15.114:6813","nonce":1144140707}]},"public_addr":"172.21.15.114:6809/1144140707","cluster_addr":"172.21.15.114:6811/1144140707","heartbeat_back_addr":"172.21.15.114:6815/1144140707","heartbeat_front_addr":"172.21.15.114:6813/1144140707","state":["exists","up"]},{"osd":6,"uuid":"9b79989e-adb0-42b2-b2f6-a2dce56b66e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":41,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6818","nonce":1522586255},{"type":"v1","addr":"172.21.15.89:6819","nonce":1522586255}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6820","nonce":1522586255},{"type":"v1","addr":"172.21.15.89:6821","nonce":1522586255}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6824","nonce":1522586255},{"type":"v1","addr":"172.21.15.89:6825","nonce":1522586255}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6822","nonce":1522586255},{"type":"v1","addr":"172.21.15.89:6823","nonce":1522586255}]},"public_addr":"172.21.15.89:6819/1522586255","cluster_addr":"172.21.15.89:6821/1522586255","heartbeat_back_addr":"172.21.15.89:6825/1522586255","heartbeat_front_addr":"172.21.15.89:6823/1522586255","state":["exists","up"]},{"osd":7,"uuid":"b381821f-141c-47a1-be65-f57ca730391b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6816","nonce":1331818323},{"type":"v1","addr":"172.21.15.110:6817","nonce":1331818323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6818","nonce":1331818323},{"type":"v1","addr":"172.21.15.110:6819","nonce":1331818323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6822","nonce":1331818323},{"type":"v1","addr":"172.21.15.110:6823","nonce":1331818323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6820","nonce":1331818323},{"type":"v1","addr":"172.21.15.110:6821","nonce":1331818323}]},"public_addr":"172.21.15.110:6817/1331818323","cluster_addr":"172.21.15.110:6819/1331818323","heartbeat_back_addr":"172.21.15.110:6823/1331818323","heartbeat_front_addr":"172.21.15.110:6821/1331818323","state":["exists","up"]},{"osd":8,"uuid":"1bc2a5b4-15f3-456f-9d67-e51307a27bcc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6816","nonce":2138556699},{"type":"v1","addr":"172.21.15.114:6817","nonce":2138556699}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6818","nonce":2138556699},{"type":"v1","addr":"172.21.15.114:6819","nonce":2138556699}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6822","nonce":2138556699},{"type":"v1","addr":"172.21.15.114:6823","nonce":2138556699}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6820","nonce":2138556699},{"type":"v1","addr":"172.21.15.114:6821","nonce":2138556699}]},"public_addr":"172.21.15.114:6817/2138556699","cluster_addr":"172.21.15.114:6819/2138556699","heartbeat_back_addr":"172.21.15.114:6823/2138556699","heartbeat_front_addr":"172.21.15.114:6821/2138556699","state":["exists","up"]},{"osd":9,"uuid":"91f5fd29-d4a7-4800-ac7a-e83b13add89e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6826","nonce":1730253068},{"type":"v1","addr":"172.21.15.89:6827","nonce":1730253068}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6828","nonce":1730253068},{"type":"v1","addr":"172.21.15.89:6829","nonce":1730253068}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6832","nonce":1730253068},{"type":"v1","addr":"172.21.15.89:6833","nonce":1730253068}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6830","nonce":1730253068},{"type":"v1","addr":"172.21.15.89:6831","nonce":1730253068}]},"public_addr":"172.21.15.89:6827/1730253068","cluster_addr":"172.21.15.89:6829/1730253068","heartbeat_back_addr":"172.21.15.89:6833/1730253068","heartbeat_front_addr":"172.21.15.89:6831/1730253068","state":["exists","up"]},{"osd":10,"uuid":"01390499-2a4d-4188-b57a-fcb10c668e2c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6824","nonce":2904016117},{"type":"v1","addr":"172.21.15.110:6825","nonce":2904016117}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6826","nonce":2904016117},{"type":"v1","addr":"172.21.15.110:6827","nonce":2904016117}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6830","nonce":2904016117},{"type":"v1","addr":"172.21.15.110:6831","nonce":2904016117}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6828","nonce":2904016117},{"type":"v1","addr":"172.21.15.110:6829","nonce":2904016117}]},"public_addr":"172.21.15.110:6825/2904016117","cluster_addr":"172.21.15.110:6827/2904016117","heartbeat_back_addr":"172.21.15.110:6831/2904016117","heartbeat_front_addr":"172.21.15.110:6829/2904016117","state":["exists","up"]},{"osd":11,"uuid":"c3c0f38a-c3d3-415d-a9f8-875d5b756e35","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6824","nonce":3749854486},{"type":"v1","addr":"172.21.15.114:6825","nonce":3749854486}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6826","nonce":3749854486},{"type":"v1","addr":"172.21.15.114:6827","nonce":3749854486}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6830","nonce":3749854486},{"type":"v1","addr":"172.21.15.114:6831","nonce":3749854486}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6828","nonce":3749854486},{"type":"v1","addr":"172.21.15.114:6829","nonce":3749854486}]},"public_addr":"172.21.15.114:6825/3749854486","cluster_addr":"172.21.15.114:6827/3749854486","heartbeat_back_addr":"172.21.15.114:6831/3749854486","heartbeat_front_addr":"172.21.15.114:6829/3749854486","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:31:46.218908+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:32:10.338230+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:32:34.830596+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:33:00.816102+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:33:25.223838+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:33:50.188230+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:34:16.564255+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:34:41.498230+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:35:05.826322+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:35:31.234450+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:35:56.474373+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:36:21.741254+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.89:6801/655110910":"2024-09-22T03:29:51.519106+0000","172.21.15.89:0/1618544983":"2024-09-22T03:29:51.519106+0000","172.21.15.89:0/1265502758":"2024-09-22T03:29:51.519106+0000","172.21.15.89:0/4017684419":"2024-09-22T03:29:51.519106+0000","172.21.15.89:6801/3312712420":"2024-09-22T03:29:26.004686+0000","172.21.15.89:6800/3312712420":"2024-09-22T03:29:26.004686+0000","172.21.15.89:0/2056576483":"2024-09-22T03:29:26.004686+0000","172.21.15.89:6800/655110910":"2024-09-22T03:29:51.519106+0000","172.21.15.89:0/2382866302":"2024-09-22T03:29:26.004686+0000","172.21.15.89:0/1584351801":"2024-09-22T03:29:26.004686+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T03:37:41.612 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:41.610+0000 7f47a43a9640 1 -- 172.21.15.89:0/637619456 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f47880b6510 msgr2=0x7f47880b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:41.612 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:41.610+0000 7f47a43a9640 1 --2- 172.21.15.89:0/637619456 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f47880b6510 0x7f47880b8a20 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f4798009990 tx=0x7f4798009110 comp rx=0 tx=0).stop 2024-09-21T03:37:41.612 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:41.610+0000 7f47a43a9640 1 -- 172.21.15.89:0/637619456 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f479c109e40 msgr2=0x7f479c19fc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:41.612 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:41.610+0000 7f47a43a9640 1 --2- 172.21.15.89:0/637619456 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f479c109e40 0x7f479c19fc70 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f4794005cc0 tx=0x7f4794032c40 comp rx=0 tx=0).stop 2024-09-21T03:37:41.612 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:41.610+0000 7f47a43a9640 1 -- 172.21.15.89:0/637619456 shutdown_connections 2024-09-21T03:37:41.612 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:41.610+0000 7f47a43a9640 1 --2- 172.21.15.89:0/637619456 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f47880b6510 0x7f47880b8a20 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:41.612 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:41.610+0000 7f47a43a9640 1 --2- 172.21.15.89:0/637619456 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f479c1094f0 0x7f479c19a9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:41.612 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:41.610+0000 7f47a43a9640 1 --2- 172.21.15.89:0/637619456 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f479c109e40 0x7f479c19fc70 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:41.612 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:41.610+0000 7f47a43a9640 1 --2- 172.21.15.89:0/637619456 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f479c103bb0 0x7f479c19a430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:41.612 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:41.610+0000 7f47a43a9640 1 -- 172.21.15.89:0/637619456 >> 172.21.15.89:0/637619456 conn(0x7f479c0ff600 msgr2=0x7f479c101a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:37:41.612 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:41.611+0000 7f47a43a9640 1 -- 172.21.15.89:0/637619456 shutdown_connections 2024-09-21T03:37:41.612 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:41.611+0000 7f47a43a9640 1 -- 172.21.15.89:0/637619456 wait complete. 2024-09-21T03:37:42.235 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:37:41 smithi089 ceph-mon[30928]: from='client.? 172.21.15.89:0/637619456' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T03:37:42.250 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:37:41 smithi110 ceph-mon[34000]: from='client.? 172.21.15.89:0/637619456' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T03:37:42.266 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 7c374098-77c9-11ef-baf5-efdc52797490 -- ceph osd dump --format=json 2024-09-21T03:37:42.460 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:37:41 smithi114 ceph-mon[34140]: from='client.? 172.21.15.89:0/637619456' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T03:37:42.514 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/7c374098-77c9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:37:43.017 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:43.005+0000 7f89b7af5640 1 -- 172.21.15.89:0/1452528925 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f89b0103b80 msgr2=0x7f89b0103f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:43.017 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:43.005+0000 7f89b7af5640 1 --2- 172.21.15.89:0/1452528925 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f89b0103b80 0x7f89b0103f90 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f89a8009900 tx=0x7f89a802a410 comp rx=0 tx=0).stop 2024-09-21T03:37:43.017 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:43.005+0000 7f89b7af5640 1 -- 172.21.15.89:0/1452528925 shutdown_connections 2024-09-21T03:37:43.017 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:43.005+0000 7f89b7af5640 1 --2- 172.21.15.89:0/1452528925 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f89b0109e10 0x7f89b0110640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:43.017 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:43.005+0000 7f89b7af5640 1 --2- 172.21.15.89:0/1452528925 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f89b01094c0 0x7f89b01098d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:43.017 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:43.005+0000 7f89b7af5640 1 --2- 172.21.15.89:0/1452528925 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f89b0103b80 0x7f89b0103f90 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:43.017 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:43.005+0000 7f89b7af5640 1 -- 172.21.15.89:0/1452528925 >> 172.21.15.89:0/1452528925 conn(0x7f89b00ff610 msgr2=0x7f89b0101a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:37:43.018 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:43.005+0000 7f89b7af5640 1 -- 172.21.15.89:0/1452528925 shutdown_connections 2024-09-21T03:37:43.018 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:43.005+0000 7f89b7af5640 1 -- 172.21.15.89:0/1452528925 wait complete. 2024-09-21T03:37:43.018 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:43.006+0000 7f89b7af5640 1 Processor -- start 2024-09-21T03:37:43.018 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:43.006+0000 7f89b7af5640 1 -- start start 2024-09-21T03:37:43.018 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:43.006+0000 7f89b7af5640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f89b0103b80 0x7f89b019a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:43.018 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:43.006+0000 7f89b7af5640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f89b01094c0 0x7f89b019a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:43.018 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:43.006+0000 7f89b7af5640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f89b0109e10 0x7f89b019fc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:43.018 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:43.006+0000 7f89b7af5640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f89b0113980 con 0x7f89b0103b80 2024-09-21T03:37:43.018 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:43.006+0000 7f89b7af5640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f89b0113800 con 0x7f89b01094c0 2024-09-21T03:37:43.018 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:43.006+0000 7f89b7af5640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f89b0113b00 con 0x7f89b0109e10 2024-09-21T03:37:43.018 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:43.006+0000 7f89b5891640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f89b0103b80 0x7f89b019a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:43.018 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:43.006+0000 7f89b5891640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f89b0103b80 0x7f89b019a430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:47260/0 (socket says 172.21.15.89:47260) 2024-09-21T03:37:43.018 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:43.006+0000 7f89b5891640 1 -- 172.21.15.89:0/3900210488 learned_addr learned my addr 172.21.15.89:0/3900210488 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-21T03:37:43.019 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:43.006+0000 7f89b6092640 1 --2- 172.21.15.89:0/3900210488 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f89b0109e10 0x7f89b019fc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:43.019 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:43.006+0000 7f89b5090640 1 --2- 172.21.15.89:0/3900210488 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f89b01094c0 0x7f89b019a9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:43.019 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:43.006+0000 7f89b5891640 1 -- 172.21.15.89:0/3900210488 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f89b0109e10 msgr2=0x7f89b019fc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:43.019 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:43.006+0000 7f89b5891640 1 --2- 172.21.15.89:0/3900210488 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f89b0109e10 0x7f89b019fc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:43.019 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:43.006+0000 7f89b5891640 1 -- 172.21.15.89:0/3900210488 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f89b01094c0 msgr2=0x7f89b019a9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:43.019 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:43.006+0000 7f89b5891640 1 --2- 172.21.15.89:0/3900210488 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f89b01094c0 0x7f89b019a9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:43.019 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:43.006+0000 7f89b5891640 1 -- 172.21.15.89:0/3900210488 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89b01a03a0 con 0x7f89b0103b80 2024-09-21T03:37:43.019 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:43.006+0000 7f89b5090640 1 --2- 172.21.15.89:0/3900210488 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f89b01094c0 0x7f89b019a9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:37:43.019 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:43.007+0000 7f89b5891640 1 --2- 172.21.15.89:0/3900210488 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f89b0103b80 0x7f89b019a430 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f89a8009230 tx=0x7f89a8004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:37:43.019 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:43.007+0000 7f89aeffd640 1 -- 172.21.15.89:0/3900210488 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89a803f030 con 0x7f89b0103b80 2024-09-21T03:37:43.019 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:43.007+0000 7f89aeffd640 1 -- 172.21.15.89:0/3900210488 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f89a8003a80 con 0x7f89b0103b80 2024-09-21T03:37:43.019 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:43.007+0000 7f89b7af5640 1 -- 172.21.15.89:0/3900210488 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f89b01a0630 con 0x7f89b0103b80 2024-09-21T03:37:43.019 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:43.007+0000 7f89aeffd640 1 -- 172.21.15.89:0/3900210488 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89a8035070 con 0x7f89b0103b80 2024-09-21T03:37:43.020 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:43.007+0000 7f89b7af5640 1 -- 172.21.15.89:0/3900210488 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f89b01a6df0 con 0x7f89b0103b80 2024-09-21T03:37:43.020 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:43.008+0000 7f89b7af5640 1 -- 172.21.15.89:0/3900210488 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f89b0103f90 con 0x7f89b0103b80 2024-09-21T03:37:43.020 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:43.010+0000 7f89aeffd640 1 -- 172.21.15.89:0/3900210488 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 17) ==== 154512+0+0 (secure 0 0 0) 0x7f89a8042030 con 0x7f89b0103b80 2024-09-21T03:37:43.020 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:43.011+0000 7f89aeffd640 1 --2- 172.21.15.89:0/3900210488 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f89980b6510 0x7f89980b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:43.020 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:43.011+0000 7f89b5090640 1 --2- 172.21.15.89:0/3900210488 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f89980b6510 0x7f89980b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:43.020 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:43.011+0000 7f89aeffd640 1 -- 172.21.15.89:0/3900210488 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8802+0+0 (secure 0 0 0) 0x7f89a81080c0 con 0x7f89b0103b80 2024-09-21T03:37:43.020 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:43.011+0000 7f89b5090640 1 --2- 172.21.15.89:0/3900210488 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f89980b6510 0x7f89980b8a20 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f89a40058c0 tx=0x7f89a4005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:37:43.020 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:43.014+0000 7f89aeffd640 1 -- 172.21.15.89:0/3900210488 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f89a80ce710 con 0x7f89b0103b80 2024-09-21T03:37:43.197 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:43.196+0000 7f89b7af5640 1 -- 172.21.15.89:0/3900210488 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f89b010bbe0 con 0x7f89b0103b80 2024-09-21T03:37:43.197 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:37:42 smithi089 ceph-mon[30928]: pgmap v305: 129 pgs: 129 active+clean; 581 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-09-21T03:37:43.198 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:43.197+0000 7f89aeffd640 1 -- 172.21.15.89:0/3900210488 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v80) ==== 74+0+24236 (secure 0 0 0) 0x7f89a80d2af0 con 0x7f89b0103b80 2024-09-21T03:37:43.199 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-09-21T03:37:43.199 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":80,"fsid":"7c374098-77c9-11ef-baf5-efdc52797490","created":"2024-09-21T03:28:56.681458+0000","modified":"2024-09-21T03:37:22.518390+0000","last_up_change":"2024-09-21T03:36:23.368379+0000","last_in_change":"2024-09-21T03:36:00.131331+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T03:32:37.834339+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-21T03:37:09.443341+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"74","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-21T03:37:11.467821+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"74","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-21T03:37:17.884066+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"27ca5ed8-17d1-4f8e-a7ae-084a914721a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6802","nonce":990539042},{"type":"v1","addr":"172.21.15.89:6803","nonce":990539042}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6804","nonce":990539042},{"type":"v1","addr":"172.21.15.89:6805","nonce":990539042}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6808","nonce":990539042},{"type":"v1","addr":"172.21.15.89:6809","nonce":990539042}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6806","nonce":990539042},{"type":"v1","addr":"172.21.15.89:6807","nonce":990539042}]},"public_addr":"172.21.15.89:6803/990539042","cluster_addr":"172.21.15.89:6805/990539042","heartbeat_back_addr":"172.21.15.89:6809/990539042","heartbeat_front_addr":"172.21.15.89:6807/990539042","state":["exists","up"]},{"osd":1,"uuid":"5176a910-5cf9-49d7-aedb-a20cf36ea075","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6800","nonce":3235852628},{"type":"v1","addr":"172.21.15.110:6801","nonce":3235852628}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6802","nonce":3235852628},{"type":"v1","addr":"172.21.15.110:6803","nonce":3235852628}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6806","nonce":3235852628},{"type":"v1","addr":"172.21.15.110:6807","nonce":3235852628}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6804","nonce":3235852628},{"type":"v1","addr":"172.21.15.110:6805","nonce":3235852628}]},"public_addr":"172.21.15.110:6801/3235852628","cluster_addr":"172.21.15.110:6803/3235852628","heartbeat_back_addr":"172.21.15.110:6807/3235852628","heartbeat_front_addr":"172.21.15.110:6805/3235852628","state":["exists","up"]},{"osd":2,"uuid":"d4977df1-b118-4d7e-b9a9-626a8accc812","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6800","nonce":169452122},{"type":"v1","addr":"172.21.15.114:6801","nonce":169452122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6802","nonce":169452122},{"type":"v1","addr":"172.21.15.114:6803","nonce":169452122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6806","nonce":169452122},{"type":"v1","addr":"172.21.15.114:6807","nonce":169452122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6804","nonce":169452122},{"type":"v1","addr":"172.21.15.114:6805","nonce":169452122}]},"public_addr":"172.21.15.114:6801/169452122","cluster_addr":"172.21.15.114:6803/169452122","heartbeat_back_addr":"172.21.15.114:6807/169452122","heartbeat_front_addr":"172.21.15.114:6805/169452122","state":["exists","up"]},{"osd":3,"uuid":"2c7ae1b4-0bf2-4954-81ed-55d03eab82c6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6810","nonce":3016980877},{"type":"v1","addr":"172.21.15.89:6811","nonce":3016980877}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6812","nonce":3016980877},{"type":"v1","addr":"172.21.15.89:6813","nonce":3016980877}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6816","nonce":3016980877},{"type":"v1","addr":"172.21.15.89:6817","nonce":3016980877}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6814","nonce":3016980877},{"type":"v1","addr":"172.21.15.89:6815","nonce":3016980877}]},"public_addr":"172.21.15.89:6811/3016980877","cluster_addr":"172.21.15.89:6813/3016980877","heartbeat_back_addr":"172.21.15.89:6817/3016980877","heartbeat_front_addr":"172.21.15.89:6815/3016980877","state":["exists","up"]},{"osd":4,"uuid":"8e65399d-6e55-4da4-98ba-f805f0c655a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":30,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6808","nonce":3217880476},{"type":"v1","addr":"172.21.15.110:6809","nonce":3217880476}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6810","nonce":3217880476},{"type":"v1","addr":"172.21.15.110:6811","nonce":3217880476}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6814","nonce":3217880476},{"type":"v1","addr":"172.21.15.110:6815","nonce":3217880476}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6812","nonce":3217880476},{"type":"v1","addr":"172.21.15.110:6813","nonce":3217880476}]},"public_addr":"172.21.15.110:6809/3217880476","cluster_addr":"172.21.15.110:6811/3217880476","heartbeat_back_addr":"172.21.15.110:6815/3217880476","heartbeat_front_addr":"172.21.15.110:6813/3217880476","state":["exists","up"]},{"osd":5,"uuid":"b9ec2889-46bd-40e5-8446-73ab28271c67","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6808","nonce":1144140707},{"type":"v1","addr":"172.21.15.114:6809","nonce":1144140707}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6810","nonce":1144140707},{"type":"v1","addr":"172.21.15.114:6811","nonce":1144140707}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6814","nonce":1144140707},{"type":"v1","addr":"172.21.15.114:6815","nonce":1144140707}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6812","nonce":1144140707},{"type":"v1","addr":"172.21.15.114:6813","nonce":1144140707}]},"public_addr":"172.21.15.114:6809/1144140707","cluster_addr":"172.21.15.114:6811/1144140707","heartbeat_back_addr":"172.21.15.114:6815/1144140707","heartbeat_front_addr":"172.21.15.114:6813/1144140707","state":["exists","up"]},{"osd":6,"uuid":"9b79989e-adb0-42b2-b2f6-a2dce56b66e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":41,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6818","nonce":1522586255},{"type":"v1","addr":"172.21.15.89:6819","nonce":1522586255}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6820","nonce":1522586255},{"type":"v1","addr":"172.21.15.89:6821","nonce":1522586255}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6824","nonce":1522586255},{"type":"v1","addr":"172.21.15.89:6825","nonce":1522586255}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6822","nonce":1522586255},{"type":"v1","addr":"172.21.15.89:6823","nonce":1522586255}]},"public_addr":"172.21.15.89:6819/1522586255","cluster_addr":"172.21.15.89:6821/1522586255","heartbeat_back_addr":"172.21.15.89:6825/1522586255","heartbeat_front_addr":"172.21.15.89:6823/1522586255","state":["exists","up"]},{"osd":7,"uuid":"b381821f-141c-47a1-be65-f57ca730391b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6816","nonce":1331818323},{"type":"v1","addr":"172.21.15.110:6817","nonce":1331818323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6818","nonce":1331818323},{"type":"v1","addr":"172.21.15.110:6819","nonce":1331818323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6822","nonce":1331818323},{"type":"v1","addr":"172.21.15.110:6823","nonce":1331818323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6820","nonce":1331818323},{"type":"v1","addr":"172.21.15.110:6821","nonce":1331818323}]},"public_addr":"172.21.15.110:6817/1331818323","cluster_addr":"172.21.15.110:6819/1331818323","heartbeat_back_addr":"172.21.15.110:6823/1331818323","heartbeat_front_addr":"172.21.15.110:6821/1331818323","state":["exists","up"]},{"osd":8,"uuid":"1bc2a5b4-15f3-456f-9d67-e51307a27bcc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6816","nonce":2138556699},{"type":"v1","addr":"172.21.15.114:6817","nonce":2138556699}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6818","nonce":2138556699},{"type":"v1","addr":"172.21.15.114:6819","nonce":2138556699}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6822","nonce":2138556699},{"type":"v1","addr":"172.21.15.114:6823","nonce":2138556699}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6820","nonce":2138556699},{"type":"v1","addr":"172.21.15.114:6821","nonce":2138556699}]},"public_addr":"172.21.15.114:6817/2138556699","cluster_addr":"172.21.15.114:6819/2138556699","heartbeat_back_addr":"172.21.15.114:6823/2138556699","heartbeat_front_addr":"172.21.15.114:6821/2138556699","state":["exists","up"]},{"osd":9,"uuid":"91f5fd29-d4a7-4800-ac7a-e83b13add89e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6826","nonce":1730253068},{"type":"v1","addr":"172.21.15.89:6827","nonce":1730253068}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6828","nonce":1730253068},{"type":"v1","addr":"172.21.15.89:6829","nonce":1730253068}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6832","nonce":1730253068},{"type":"v1","addr":"172.21.15.89:6833","nonce":1730253068}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6830","nonce":1730253068},{"type":"v1","addr":"172.21.15.89:6831","nonce":1730253068}]},"public_addr":"172.21.15.89:6827/1730253068","cluster_addr":"172.21.15.89:6829/1730253068","heartbeat_back_addr":"172.21.15.89:6833/1730253068","heartbeat_front_addr":"172.21.15.89:6831/1730253068","state":["exists","up"]},{"osd":10,"uuid":"01390499-2a4d-4188-b57a-fcb10c668e2c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6824","nonce":2904016117},{"type":"v1","addr":"172.21.15.110:6825","nonce":2904016117}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6826","nonce":2904016117},{"type":"v1","addr":"172.21.15.110:6827","nonce":2904016117}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6830","nonce":2904016117},{"type":"v1","addr":"172.21.15.110:6831","nonce":2904016117}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6828","nonce":2904016117},{"type":"v1","addr":"172.21.15.110:6829","nonce":2904016117}]},"public_addr":"172.21.15.110:6825/2904016117","cluster_addr":"172.21.15.110:6827/2904016117","heartbeat_back_addr":"172.21.15.110:6831/2904016117","heartbeat_front_addr":"172.21.15.110:6829/2904016117","state":["exists","up"]},{"osd":11,"uuid":"c3c0f38a-c3d3-415d-a9f8-875d5b756e35","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6824","nonce":3749854486},{"type":"v1","addr":"172.21.15.114:6825","nonce":3749854486}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6826","nonce":3749854486},{"type":"v1","addr":"172.21.15.114:6827","nonce":3749854486}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6830","nonce":3749854486},{"type":"v1","addr":"172.21.15.114:6831","nonce":3749854486}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6828","nonce":3749854486},{"type":"v1","addr":"172.21.15.114:6829","nonce":3749854486}]},"public_addr":"172.21.15.114:6825/3749854486","cluster_addr":"172.21.15.114:6827/3749854486","heartbeat_back_addr":"172.21.15.114:6831/3749854486","heartbeat_front_addr":"172.21.15.114:6829/3749854486","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:31:46.218908+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:32:10.338230+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:32:34.830596+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:33:00.816102+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:33:25.223838+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:33:50.188230+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:34:16.564255+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:34:41.498230+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:35:05.826322+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:35:31.234450+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:35:56.474373+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:36:21.741254+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.89:6801/655110910":"2024-09-22T03:29:51.519106+0000","172.21.15.89:0/1618544983":"2024-09-22T03:29:51.519106+0000","172.21.15.89:0/1265502758":"2024-09-22T03:29:51.519106+0000","172.21.15.89:0/4017684419":"2024-09-22T03:29:51.519106+0000","172.21.15.89:6801/3312712420":"2024-09-22T03:29:26.004686+0000","172.21.15.89:6800/3312712420":"2024-09-22T03:29:26.004686+0000","172.21.15.89:0/2056576483":"2024-09-22T03:29:26.004686+0000","172.21.15.89:6800/655110910":"2024-09-22T03:29:51.519106+0000","172.21.15.89:0/2382866302":"2024-09-22T03:29:26.004686+0000","172.21.15.89:0/1584351801":"2024-09-22T03:29:26.004686+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T03:37:43.201 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:43.200+0000 7f89b7af5640 1 -- 172.21.15.89:0/3900210488 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f89980b6510 msgr2=0x7f89980b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:43.201 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:43.200+0000 7f89b7af5640 1 --2- 172.21.15.89:0/3900210488 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f89980b6510 0x7f89980b8a20 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f89a40058c0 tx=0x7f89a4005850 comp rx=0 tx=0).stop 2024-09-21T03:37:43.201 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:43.200+0000 7f89b7af5640 1 -- 172.21.15.89:0/3900210488 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f89b0103b80 msgr2=0x7f89b019a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:43.201 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:43.200+0000 7f89b7af5640 1 --2- 172.21.15.89:0/3900210488 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f89b0103b80 0x7f89b019a430 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f89a8009230 tx=0x7f89a8004270 comp rx=0 tx=0).stop 2024-09-21T03:37:43.201 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:43.200+0000 7f89b7af5640 1 -- 172.21.15.89:0/3900210488 shutdown_connections 2024-09-21T03:37:43.202 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:43.200+0000 7f89b7af5640 1 --2- 172.21.15.89:0/3900210488 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f89980b6510 0x7f89980b8a20 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:43.202 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:43.200+0000 7f89b7af5640 1 --2- 172.21.15.89:0/3900210488 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f89b01094c0 0x7f89b019a9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:43.202 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:43.200+0000 7f89b7af5640 1 --2- 172.21.15.89:0/3900210488 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f89b0109e10 0x7f89b019fc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:43.202 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:43.200+0000 7f89b7af5640 1 --2- 172.21.15.89:0/3900210488 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f89b0103b80 0x7f89b019a430 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:43.202 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:43.200+0000 7f89b7af5640 1 -- 172.21.15.89:0/3900210488 >> 172.21.15.89:0/3900210488 conn(0x7f89b00ff610 msgr2=0x7f89b0101a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:37:43.202 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:43.200+0000 7f89b7af5640 1 -- 172.21.15.89:0/3900210488 shutdown_connections 2024-09-21T03:37:43.202 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:43.200+0000 7f89b7af5640 1 -- 172.21.15.89:0/3900210488 wait complete. 2024-09-21T03:37:43.250 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:37:42 smithi110 ceph-mon[34000]: pgmap v305: 129 pgs: 129 active+clean; 581 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-09-21T03:37:43.461 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:37:42 smithi114 ceph-mon[34140]: pgmap v305: 129 pgs: 129 active+clean; 581 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-09-21T03:37:43.893 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 7c374098-77c9-11ef-baf5-efdc52797490 -- ceph fs set cephfs standby_count_wanted 0 2024-09-21T03:37:44.142 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/7c374098-77c9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:37:44.250 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:37:43 smithi110 ceph-mon[34000]: from='client.? 172.21.15.89:0/3900210488' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T03:37:44.263 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:37:43 smithi089 ceph-mon[30928]: from='client.? 172.21.15.89:0/3900210488' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T03:37:44.461 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:37:43 smithi114 ceph-mon[34140]: from='client.? 172.21.15.89:0/3900210488' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T03:37:44.635 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:44.626+0000 7ff060df7640 1 -- 172.21.15.89:0/3998002746 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff05c107be0 msgr2=0x7ff05c107ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:44.635 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:44.626+0000 7ff060df7640 1 --2- 172.21.15.89:0/3998002746 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff05c107be0 0x7ff05c107ff0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7ff044009900 tx=0x7ff04402a410 comp rx=0 tx=0).stop 2024-09-21T03:37:44.635 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:44.627+0000 7ff060df7640 1 -- 172.21.15.89:0/3998002746 shutdown_connections 2024-09-21T03:37:44.635 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:44.627+0000 7ff060df7640 1 --2- 172.21.15.89:0/3998002746 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff05c108530 0x7ff05c10ed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:44.635 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:44.627+0000 7ff060df7640 1 --2- 172.21.15.89:0/3998002746 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff05c107be0 0x7ff05c107ff0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:44.635 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:44.627+0000 7ff060df7640 1 --2- 172.21.15.89:0/3998002746 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff05c1022a0 0x7ff05c1026b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:44.635 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:44.627+0000 7ff060df7640 1 -- 172.21.15.89:0/3998002746 >> 172.21.15.89:0/3998002746 conn(0x7ff05c079ec0 msgr2=0x7ff05c07a2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:37:44.635 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:44.627+0000 7ff060df7640 1 -- 172.21.15.89:0/3998002746 shutdown_connections 2024-09-21T03:37:44.635 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:44.627+0000 7ff060df7640 1 -- 172.21.15.89:0/3998002746 wait complete. 2024-09-21T03:37:44.635 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:44.627+0000 7ff060df7640 1 Processor -- start 2024-09-21T03:37:44.635 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:44.628+0000 7ff060df7640 1 -- start start 2024-09-21T03:37:44.635 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:44.628+0000 7ff060df7640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff05c1022a0 0x7ff05c19a520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:44.636 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:44.628+0000 7ff060df7640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff05c107be0 0x7ff05c19aac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:44.636 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:44.628+0000 7ff060df7640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff05c108530 0x7ff05c19fd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:44.636 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:44.628+0000 7ff060df7640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7ff05c112050 con 0x7ff05c1022a0 2024-09-21T03:37:44.636 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:44.628+0000 7ff060df7640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7ff05c111ed0 con 0x7ff05c108530 2024-09-21T03:37:44.636 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:44.628+0000 7ff060df7640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7ff05c1121d0 con 0x7ff05c107be0 2024-09-21T03:37:44.636 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:44.628+0000 7ff05a59c640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff05c1022a0 0x7ff05c19a520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:44.636 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:44.628+0000 7ff05a59c640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff05c1022a0 0x7ff05c19a520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:47280/0 (socket says 172.21.15.89:47280) 2024-09-21T03:37:44.636 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:44.628+0000 7ff05a59c640 1 -- 172.21.15.89:0/1490091146 learned_addr learned my addr 172.21.15.89:0/1490091146 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-21T03:37:44.636 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:44.628+0000 7ff05ad9d640 1 --2- 172.21.15.89:0/1490091146 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff05c108530 0x7ff05c19fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:44.636 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:44.628+0000 7ff053fff640 1 --2- 172.21.15.89:0/1490091146 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff05c107be0 0x7ff05c19aac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:44.636 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:44.628+0000 7ff05a59c640 1 -- 172.21.15.89:0/1490091146 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff05c107be0 msgr2=0x7ff05c19aac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:44.636 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:44.628+0000 7ff05a59c640 1 --2- 172.21.15.89:0/1490091146 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff05c107be0 0x7ff05c19aac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:44.636 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:44.628+0000 7ff05a59c640 1 -- 172.21.15.89:0/1490091146 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff05c108530 msgr2=0x7ff05c19fd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:44.637 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:44.628+0000 7ff05a59c640 1 --2- 172.21.15.89:0/1490091146 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff05c108530 0x7ff05c19fd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:44.637 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:44.628+0000 7ff05a59c640 1 -- 172.21.15.89:0/1490091146 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff05c1a03d0 con 0x7ff05c1022a0 2024-09-21T03:37:44.637 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:44.629+0000 7ff05a59c640 1 --2- 172.21.15.89:0/1490091146 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff05c1022a0 0x7ff05c19a520 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7ff04c00c920 tx=0x7ff04c00cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:37:44.637 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:44.629+0000 7ff0537fe640 1 -- 172.21.15.89:0/1490091146 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff04c007c70 con 0x7ff05c1022a0 2024-09-21T03:37:44.637 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:44.629+0000 7ff060df7640 1 -- 172.21.15.89:0/1490091146 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff05c1a06c0 con 0x7ff05c1022a0 2024-09-21T03:37:44.637 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:44.629+0000 7ff0537fe640 1 -- 172.21.15.89:0/1490091146 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff04c007e10 con 0x7ff05c1022a0 2024-09-21T03:37:44.637 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:44.629+0000 7ff0537fe640 1 -- 172.21.15.89:0/1490091146 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff04c018710 con 0x7ff05c1022a0 2024-09-21T03:37:44.637 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:44.629+0000 7ff060df7640 1 -- 172.21.15.89:0/1490091146 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff05c1a6ee0 con 0x7ff05c1022a0 2024-09-21T03:37:44.637 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:44.630+0000 7ff060df7640 1 -- 172.21.15.89:0/1490091146 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff05c1026b0 con 0x7ff05c1022a0 2024-09-21T03:37:44.637 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:44.631+0000 7ff0537fe640 1 -- 172.21.15.89:0/1490091146 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 17) ==== 154512+0+0 (secure 0 0 0) 0x7ff04c00f450 con 0x7ff05c1022a0 2024-09-21T03:37:44.637 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:44.632+0000 7ff0537fe640 1 --2- 172.21.15.89:0/1490091146 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7ff0480b6540 0x7ff0480b8a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:44.637 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:44.632+0000 7ff053fff640 1 --2- 172.21.15.89:0/1490091146 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7ff0480b6540 0x7ff0480b8a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:44.637 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:44.633+0000 7ff0537fe640 1 -- 172.21.15.89:0/1490091146 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8802+0+0 (secure 0 0 0) 0x7ff04c0e78b0 con 0x7ff05c1022a0 2024-09-21T03:37:44.637 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:44.633+0000 7ff053fff640 1 --2- 172.21.15.89:0/1490091146 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7ff0480b6540 0x7ff0480b8a50 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7ff044005940 tx=0x7ff0440058d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:37:44.638 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:44.636+0000 7ff0537fe640 1 -- 172.21.15.89:0/1490091146 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7ff04c014030 con 0x7ff05c1022a0 2024-09-21T03:37:44.852 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:44.851+0000 7ff060df7640 1 -- 172.21.15.89:0/1490091146 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7ff05c10ba40 con 0x7ff05c1022a0 2024-09-21T03:37:44.980 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:44.979+0000 7ff0537fe640 1 -- 172.21.15.89:0/1490091146 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v18) ==== 118+0+0 (secure 0 0 0) 0x7ff04c0adf30 con 0x7ff05c1022a0 2024-09-21T03:37:44.983 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:44.982+0000 7ff060df7640 1 -- 172.21.15.89:0/1490091146 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7ff0480b6540 msgr2=0x7ff0480b8a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:44.983 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:44.982+0000 7ff060df7640 1 --2- 172.21.15.89:0/1490091146 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7ff0480b6540 0x7ff0480b8a50 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7ff044005940 tx=0x7ff0440058d0 comp rx=0 tx=0).stop 2024-09-21T03:37:44.983 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:44.982+0000 7ff060df7640 1 -- 172.21.15.89:0/1490091146 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff05c1022a0 msgr2=0x7ff05c19a520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:44.983 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:44.982+0000 7ff060df7640 1 --2- 172.21.15.89:0/1490091146 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff05c1022a0 0x7ff05c19a520 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7ff04c00c920 tx=0x7ff04c00cde0 comp rx=0 tx=0).stop 2024-09-21T03:37:44.983 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:44.982+0000 7ff060df7640 1 -- 172.21.15.89:0/1490091146 shutdown_connections 2024-09-21T03:37:44.983 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:44.982+0000 7ff060df7640 1 --2- 172.21.15.89:0/1490091146 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7ff0480b6540 0x7ff0480b8a50 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:44.984 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:44.982+0000 7ff060df7640 1 --2- 172.21.15.89:0/1490091146 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff05c108530 0x7ff05c19fd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:44.984 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:44.982+0000 7ff060df7640 1 --2- 172.21.15.89:0/1490091146 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff05c107be0 0x7ff05c19aac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:44.984 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:44.982+0000 7ff060df7640 1 --2- 172.21.15.89:0/1490091146 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff05c1022a0 0x7ff05c19a520 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:44.984 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:44.982+0000 7ff060df7640 1 -- 172.21.15.89:0/1490091146 >> 172.21.15.89:0/1490091146 conn(0x7ff05c079ec0 msgr2=0x7ff05c108c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:37:44.984 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:44.982+0000 7ff060df7640 1 -- 172.21.15.89:0/1490091146 shutdown_connections 2024-09-21T03:37:44.984 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:44.982+0000 7ff060df7640 1 -- 172.21.15.89:0/1490091146 wait complete. 2024-09-21T03:37:45.211 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:37:44 smithi114 ceph-mon[34140]: pgmap v306: 129 pgs: 129 active+clean; 581 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-09-21T03:37:45.211 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:37:44 smithi114 ceph-mon[34140]: from='client.? 172.21.15.89:0/1490091146' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} : dispatch 2024-09-21T03:37:45.217 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:37:44 smithi089 ceph-mon[30928]: pgmap v306: 129 pgs: 129 active+clean; 581 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-09-21T03:37:45.217 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:37:44 smithi089 ceph-mon[30928]: from='client.? 172.21.15.89:0/1490091146' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} : dispatch 2024-09-21T03:37:45.249 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:37:44 smithi110 ceph-mon[34000]: pgmap v306: 129 pgs: 129 active+clean; 581 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-09-21T03:37:45.250 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:37:44 smithi110 ceph-mon[34000]: from='client.? 172.21.15.89:0/1490091146' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} : dispatch 2024-09-21T03:37:45.706 DEBUG:tasks.cephfs.filesystem:fs_config: {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd'], 'max_mds': 3, 'session_timeout': 300, 'standby_replay': True, 'subvols': {'create': 2}} 2024-09-21T03:37:45.706 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 7c374098-77c9-11ef-baf5-efdc52797490 -- ceph fs set cephfs max_mds 3 2024-09-21T03:37:45.956 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/7c374098-77c9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:37:46.250 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:37:45 smithi110 ceph-mon[34000]: from='client.? 172.21.15.89:0/1490091146' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-09-21T03:37:46.250 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:37:45 smithi110 ceph-mon[34000]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-21T03:37:46.311 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:37:45 smithi089 ceph-mon[30928]: from='client.? 172.21.15.89:0/1490091146' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-09-21T03:37:46.311 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:37:45 smithi089 ceph-mon[30928]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-21T03:37:46.445 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:46.443+0000 7efc5343b640 1 -- 172.21.15.89:0/1284925163 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efc4c10de10 msgr2=0x7efc4c110300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:46.446 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:46.443+0000 7efc5343b640 1 --2- 172.21.15.89:0/1284925163 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efc4c10de10 0x7efc4c110300 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7efc44009ca0 tx=0x7efc4402a8c0 comp rx=0 tx=0).stop 2024-09-21T03:37:46.446 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:46.443+0000 7efc5343b640 1 -- 172.21.15.89:0/1284925163 shutdown_connections 2024-09-21T03:37:46.446 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:46.443+0000 7efc5343b640 1 --2- 172.21.15.89:0/1284925163 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7efc4c105300 0x7efc4c10d850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:46.446 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:46.443+0000 7efc5343b640 1 --2- 172.21.15.89:0/1284925163 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efc4c10de10 0x7efc4c110300 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:46.446 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:46.443+0000 7efc5343b640 1 --2- 172.21.15.89:0/1284925163 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7efc4c1049b0 0x7efc4c104dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:46.446 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:46.443+0000 7efc5343b640 1 -- 172.21.15.89:0/1284925163 >> 172.21.15.89:0/1284925163 conn(0x7efc4c079730 msgr2=0x7efc4c079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:37:46.446 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:46.443+0000 7efc5343b640 1 -- 172.21.15.89:0/1284925163 shutdown_connections 2024-09-21T03:37:46.446 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:46.443+0000 7efc5343b640 1 -- 172.21.15.89:0/1284925163 wait complete. 2024-09-21T03:37:46.448 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:46.444+0000 7efc5343b640 1 Processor -- start 2024-09-21T03:37:46.448 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:46.444+0000 7efc5343b640 1 -- start start 2024-09-21T03:37:46.448 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:46.444+0000 7efc5343b640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7efc4c1049b0 0x7efc4c19c5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:46.448 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:46.444+0000 7efc5343b640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7efc4c105300 0x7efc4c19cb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:46.448 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:46.444+0000 7efc5343b640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efc4c10de10 0x7efc4c1a1e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:46.448 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:46.444+0000 7efc5343b640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7efc4c113970 con 0x7efc4c10de10 2024-09-21T03:37:46.448 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:46.444+0000 7efc5343b640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7efc4c1137f0 con 0x7efc4c1049b0 2024-09-21T03:37:46.448 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:46.444+0000 7efc5343b640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7efc4c113af0 con 0x7efc4c105300 2024-09-21T03:37:46.448 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:46.444+0000 7efc519d8640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efc4c10de10 0x7efc4c1a1e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:46.449 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:46.444+0000 7efc519d8640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efc4c10de10 0x7efc4c1a1e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:47288/0 (socket says 172.21.15.89:47288) 2024-09-21T03:37:46.449 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:46.444+0000 7efc519d8640 1 -- 172.21.15.89:0/540067781 learned_addr learned my addr 172.21.15.89:0/540067781 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-21T03:37:46.449 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:46.444+0000 7efc509d6640 1 --2- 172.21.15.89:0/540067781 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7efc4c105300 0x7efc4c19cb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:46.449 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:46.444+0000 7efc511d7640 1 --2- 172.21.15.89:0/540067781 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7efc4c1049b0 0x7efc4c19c5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:46.449 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:46.445+0000 7efc519d8640 1 -- 172.21.15.89:0/540067781 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7efc4c105300 msgr2=0x7efc4c19cb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:46.449 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:46.445+0000 7efc519d8640 1 --2- 172.21.15.89:0/540067781 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7efc4c105300 0x7efc4c19cb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:46.449 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:46.445+0000 7efc519d8640 1 -- 172.21.15.89:0/540067781 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7efc4c1049b0 msgr2=0x7efc4c19c5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:46.449 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:46.445+0000 7efc519d8640 1 --2- 172.21.15.89:0/540067781 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7efc4c1049b0 0x7efc4c19c5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:46.449 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:46.445+0000 7efc519d8640 1 -- 172.21.15.89:0/540067781 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc4c1a2530 con 0x7efc4c10de10 2024-09-21T03:37:46.449 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:46.445+0000 7efc519d8640 1 --2- 172.21.15.89:0/540067781 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efc4c10de10 0x7efc4c1a1e00 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7efc44005cc0 tx=0x7efc44032a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:37:46.449 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:46.445+0000 7efc4a7fc640 1 -- 172.21.15.89:0/540067781 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc44032e00 con 0x7efc4c10de10 2024-09-21T03:37:46.449 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:46.445+0000 7efc5343b640 1 -- 172.21.15.89:0/540067781 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efc4c1a27c0 con 0x7efc4c10de10 2024-09-21T03:37:46.449 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:46.445+0000 7efc4a7fc640 1 -- 172.21.15.89:0/540067781 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7efc440044d0 con 0x7efc4c10de10 2024-09-21T03:37:46.449 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:46.445+0000 7efc4a7fc640 1 -- 172.21.15.89:0/540067781 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc44030b70 con 0x7efc4c10de10 2024-09-21T03:37:46.449 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:46.445+0000 7efc5343b640 1 -- 172.21.15.89:0/540067781 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efc4c1a8f80 con 0x7efc4c10de10 2024-09-21T03:37:46.450 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:46.446+0000 7efc5343b640 1 -- 172.21.15.89:0/540067781 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efc4c104dc0 con 0x7efc4c10de10 2024-09-21T03:37:46.450 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:46.448+0000 7efc4a7fc640 1 -- 172.21.15.89:0/540067781 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 17) ==== 154512+0+0 (secure 0 0 0) 0x7efc44030d10 con 0x7efc4c10de10 2024-09-21T03:37:46.451 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:46.449+0000 7efc4a7fc640 1 --2- 172.21.15.89:0/540067781 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7efc340b6510 0x7efc340b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:46.451 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:46.449+0000 7efc511d7640 1 --2- 172.21.15.89:0/540067781 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7efc340b6510 0x7efc340b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:46.451 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:46.449+0000 7efc4a7fc640 1 -- 172.21.15.89:0/540067781 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8802+0+0 (secure 0 0 0) 0x7efc44108890 con 0x7efc4c10de10 2024-09-21T03:37:46.451 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:46.449+0000 7efc511d7640 1 --2- 172.21.15.89:0/540067781 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7efc340b6510 0x7efc340b8a20 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7efc4c06bc60 tx=0x7efc3c00b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:37:46.454 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:46.452+0000 7efc4a7fc640 1 -- 172.21.15.89:0/540067781 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7efc440ceec0 con 0x7efc4c10de10 2024-09-21T03:37:46.460 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:37:45 smithi114 ceph-mon[34140]: from='client.? 172.21.15.89:0/1490091146' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-09-21T03:37:46.461 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:37:45 smithi114 ceph-mon[34140]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-21T03:37:46.671 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:46.669+0000 7efc5343b640 1 -- 172.21.15.89:0/540067781 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"} v 0) -- 0x7efc4c1a9150 con 0x7efc4c10de10 2024-09-21T03:37:46.993 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:46.990+0000 7efc4a7fc640 1 -- 172.21.15.89:0/540067781 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]=0 v19) ==== 105+0+0 (secure 0 0 0) 0x7efc440d32a0 con 0x7efc4c10de10 2024-09-21T03:37:46.995 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:46.992+0000 7efc5343b640 1 -- 172.21.15.89:0/540067781 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7efc340b6510 msgr2=0x7efc340b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:46.995 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:46.993+0000 7efc5343b640 1 --2- 172.21.15.89:0/540067781 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7efc340b6510 0x7efc340b8a20 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7efc4c06bc60 tx=0x7efc3c00b390 comp rx=0 tx=0).stop 2024-09-21T03:37:46.995 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:46.993+0000 7efc5343b640 1 -- 172.21.15.89:0/540067781 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efc4c10de10 msgr2=0x7efc4c1a1e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:46.995 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:46.993+0000 7efc5343b640 1 --2- 172.21.15.89:0/540067781 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efc4c10de10 0x7efc4c1a1e00 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7efc44005cc0 tx=0x7efc44032a10 comp rx=0 tx=0).stop 2024-09-21T03:37:46.995 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:46.993+0000 7efc5343b640 1 -- 172.21.15.89:0/540067781 shutdown_connections 2024-09-21T03:37:46.995 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:46.993+0000 7efc5343b640 1 --2- 172.21.15.89:0/540067781 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7efc340b6510 0x7efc340b8a20 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:46.995 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:46.993+0000 7efc5343b640 1 --2- 172.21.15.89:0/540067781 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efc4c10de10 0x7efc4c1a1e00 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:46.995 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:46.993+0000 7efc5343b640 1 --2- 172.21.15.89:0/540067781 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7efc4c105300 0x7efc4c19cb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:46.995 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:46.993+0000 7efc5343b640 1 --2- 172.21.15.89:0/540067781 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7efc4c1049b0 0x7efc4c19c5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:46.995 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:46.993+0000 7efc5343b640 1 -- 172.21.15.89:0/540067781 >> 172.21.15.89:0/540067781 conn(0x7efc4c079730 msgr2=0x7efc4c1036c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:37:46.996 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:46.993+0000 7efc5343b640 1 -- 172.21.15.89:0/540067781 shutdown_connections 2024-09-21T03:37:46.996 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:46.993+0000 7efc5343b640 1 -- 172.21.15.89:0/540067781 wait complete. 2024-09-21T03:37:47.250 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:37:46 smithi110 ceph-mon[34000]: pgmap v307: 129 pgs: 129 active+clean; 581 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-09-21T03:37:47.250 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:37:46 smithi110 ceph-mon[34000]: from='client.? 172.21.15.89:0/540067781' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"} : dispatch 2024-09-21T03:37:47.311 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:37:46 smithi089 ceph-mon[30928]: pgmap v307: 129 pgs: 129 active+clean; 581 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-09-21T03:37:47.311 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:37:46 smithi089 ceph-mon[30928]: from='client.? 172.21.15.89:0/540067781' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"} : dispatch 2024-09-21T03:37:47.461 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:37:46 smithi114 ceph-mon[34140]: pgmap v307: 129 pgs: 129 active+clean; 581 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-09-21T03:37:47.461 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:37:46 smithi114 ceph-mon[34140]: from='client.? 172.21.15.89:0/540067781' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"} : dispatch 2024-09-21T03:37:47.721 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 7c374098-77c9-11ef-baf5-efdc52797490 -- ceph fs set cephfs allow_standby_replay true 2024-09-21T03:37:47.967 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/7c374098-77c9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:37:48.269 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:37:48 smithi089 ceph-mon[30928]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-09-21T03:37:48.269 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:37:48 smithi089 ceph-mon[30928]: from='client.? 172.21.15.89:0/540067781' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-09-21T03:37:48.269 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:37:48 smithi089 ceph-mon[30928]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-09-21T03:37:48.269 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:37:48 smithi089 ceph-mon[30928]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-21T03:37:48.269 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:37:48 smithi089 ceph-mon[30928]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2024-09-21T03:37:48.269 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:37:48 smithi089 ceph-mon[30928]: daemon mds.i is now active in filesystem cephfs as rank 1 2024-09-21T03:37:48.461 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:37:48 smithi114 ceph-mon[34140]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-09-21T03:37:48.461 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:37:48 smithi114 ceph-mon[34140]: from='client.? 172.21.15.89:0/540067781' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-09-21T03:37:48.461 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:37:48 smithi114 ceph-mon[34140]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-09-21T03:37:48.461 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:37:48 smithi114 ceph-mon[34140]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-21T03:37:48.461 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:37:48 smithi114 ceph-mon[34140]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2024-09-21T03:37:48.461 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:37:48 smithi114 ceph-mon[34140]: daemon mds.i is now active in filesystem cephfs as rank 1 2024-09-21T03:37:48.500 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:37:48 smithi110 ceph-mon[34000]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-09-21T03:37:48.500 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:37:48 smithi110 ceph-mon[34000]: from='client.? 172.21.15.89:0/540067781' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-09-21T03:37:48.500 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:37:48 smithi110 ceph-mon[34000]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-09-21T03:37:48.500 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:37:48 smithi110 ceph-mon[34000]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-21T03:37:48.500 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:37:48 smithi110 ceph-mon[34000]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2024-09-21T03:37:48.500 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:37:48 smithi110 ceph-mon[34000]: daemon mds.i is now active in filesystem cephfs as rank 1 2024-09-21T03:37:48.513 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:48.510+0000 7fe8d4a9d640 1 -- 172.21.15.89:0/1135987681 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe8d01083a0 msgr2=0x7fe8d01087b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:48.513 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:48.510+0000 7fe8d4a9d640 1 --2- 172.21.15.89:0/1135987681 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe8d01083a0 0x7fe8d01087b0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fe8b8009900 tx=0x7fe8b802a3f0 comp rx=0 tx=0).stop 2024-09-21T03:37:48.513 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:48.510+0000 7fe8d4a9d640 1 -- 172.21.15.89:0/1135987681 shutdown_connections 2024-09-21T03:37:48.513 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:48.510+0000 7fe8d4a9d640 1 --2- 172.21.15.89:0/1135987681 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe8d0108cf0 0x7fe8d010f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:48.513 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:48.510+0000 7fe8d4a9d640 1 --2- 172.21.15.89:0/1135987681 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe8d01083a0 0x7fe8d01087b0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:48.513 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:48.510+0000 7fe8d4a9d640 1 --2- 172.21.15.89:0/1135987681 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe8d0102a60 0x7fe8d0102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:48.513 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:48.510+0000 7fe8d4a9d640 1 -- 172.21.15.89:0/1135987681 >> 172.21.15.89:0/1135987681 conn(0x7fe8d0079730 msgr2=0x7fe8d0079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:37:48.513 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:48.510+0000 7fe8d4a9d640 1 -- 172.21.15.89:0/1135987681 shutdown_connections 2024-09-21T03:37:48.513 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:48.510+0000 7fe8d4a9d640 1 -- 172.21.15.89:0/1135987681 wait complete. 2024-09-21T03:37:48.515 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:48.511+0000 7fe8d4a9d640 1 Processor -- start 2024-09-21T03:37:48.515 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:48.511+0000 7fe8d4a9d640 1 -- start start 2024-09-21T03:37:48.515 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:48.512+0000 7fe8d4a9d640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe8d0102a60 0x7fe8d019a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:48.515 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:48.512+0000 7fe8d4a9d640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe8d01083a0 0x7fe8d019aa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:48.515 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:48.512+0000 7fe8d4a9d640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe8d0108cf0 0x7fe8d019fd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:48.515 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:48.512+0000 7fe8d4a9d640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fe8d0112810 con 0x7fe8d01083a0 2024-09-21T03:37:48.516 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:48.512+0000 7fe8d4a9d640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fe8d0112690 con 0x7fe8d0102a60 2024-09-21T03:37:48.516 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:48.512+0000 7fe8d4a9d640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fe8d0112990 con 0x7fe8d0108cf0 2024-09-21T03:37:48.516 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:48.512+0000 7fe8c7fff640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe8d01083a0 0x7fe8d019aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:48.516 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:48.512+0000 7fe8c7fff640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe8d01083a0 0x7fe8d019aa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:38478/0 (socket says 172.21.15.89:38478) 2024-09-21T03:37:48.516 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:48.512+0000 7fe8c7fff640 1 -- 172.21.15.89:0/1956670549 learned_addr learned my addr 172.21.15.89:0/1956670549 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-21T03:37:48.516 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:48.512+0000 7fe8ced9d640 1 --2- 172.21.15.89:0/1956670549 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe8d0108cf0 0x7fe8d019fd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:48.516 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:48.512+0000 7fe8ce59c640 1 --2- 172.21.15.89:0/1956670549 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe8d0102a60 0x7fe8d019a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:48.516 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:48.512+0000 7fe8c7fff640 1 -- 172.21.15.89:0/1956670549 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe8d0108cf0 msgr2=0x7fe8d019fd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:48.516 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:48.512+0000 7fe8c7fff640 1 --2- 172.21.15.89:0/1956670549 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe8d0108cf0 0x7fe8d019fd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:48.516 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:48.512+0000 7fe8c7fff640 1 -- 172.21.15.89:0/1956670549 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe8d0102a60 msgr2=0x7fe8d019a4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:48.516 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:48.512+0000 7fe8c7fff640 1 --2- 172.21.15.89:0/1956670549 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe8d0102a60 0x7fe8d019a4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:48.516 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:48.512+0000 7fe8c7fff640 1 -- 172.21.15.89:0/1956670549 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8d01a0380 con 0x7fe8d01083a0 2024-09-21T03:37:48.516 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:48.512+0000 7fe8c7fff640 1 --2- 172.21.15.89:0/1956670549 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe8d01083a0 0x7fe8d019aa70 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fe8b8005940 tx=0x7fe8b80042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:37:48.516 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:48.512+0000 7fe8c77fe640 1 -- 172.21.15.89:0/1956670549 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8b803f030 con 0x7fe8d01083a0 2024-09-21T03:37:48.517 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:48.513+0000 7fe8c77fe640 1 -- 172.21.15.89:0/1956670549 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe8b8003c90 con 0x7fe8d01083a0 2024-09-21T03:37:48.517 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:48.513+0000 7fe8c77fe640 1 -- 172.21.15.89:0/1956670549 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8b8035070 con 0x7fe8d01083a0 2024-09-21T03:37:48.517 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:48.513+0000 7fe8d4a9d640 1 -- 172.21.15.89:0/1956670549 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe8d01a0610 con 0x7fe8d01083a0 2024-09-21T03:37:48.517 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:48.513+0000 7fe8d4a9d640 1 -- 172.21.15.89:0/1956670549 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8d01a0aa0 con 0x7fe8d01083a0 2024-09-21T03:37:48.517 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:48.514+0000 7fe8d4a9d640 1 -- 172.21.15.89:0/1956670549 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe8d0103ce0 con 0x7fe8d01083a0 2024-09-21T03:37:48.517 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:48.515+0000 7fe8c77fe640 1 -- 172.21.15.89:0/1956670549 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 17) ==== 154512+0+0 (secure 0 0 0) 0x7fe8b8042030 con 0x7fe8d01083a0 2024-09-21T03:37:48.518 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:48.516+0000 7fe8c77fe640 1 --2- 172.21.15.89:0/1956670549 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7fe8bc0b6540 0x7fe8bc0b8a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:48.518 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:48.516+0000 7fe8ce59c640 1 --2- 172.21.15.89:0/1956670549 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7fe8bc0b6540 0x7fe8bc0b8a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:48.518 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:48.516+0000 7fe8c77fe640 1 -- 172.21.15.89:0/1956670549 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8802+0+0 (secure 0 0 0) 0x7fe8b8107fa0 con 0x7fe8d01083a0 2024-09-21T03:37:48.519 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:48.516+0000 7fe8ce59c640 1 --2- 172.21.15.89:0/1956670549 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7fe8bc0b6540 0x7fe8bc0b8a50 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fe8c0005e10 tx=0x7fe8c0005da0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:37:48.521 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:48.519+0000 7fe8c77fe640 1 -- 172.21.15.89:0/1956670549 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fe8b80ce5f0 con 0x7fe8d01083a0 2024-09-21T03:37:48.739 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:48.737+0000 7fe8d4a9d640 1 -- 172.21.15.89:0/1956670549 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} v 0) -- 0x7fe8d0102e70 con 0x7fe8d01083a0 2024-09-21T03:37:49.000 INFO:journalctl@ceph.mds.b.smithi110.stdout:Sep 21 03:37:48 smithi110 ceph-7c374098-77c9-11ef-baf5-efdc52797490-mds-b[62090]: 2024-09-21T03:37:48.565+0000 7f39912bb640 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2024-09-21T03:37:49.013 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:49.011+0000 7fe8c77fe640 1 -- 172.21.15.89:0/1956670549 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]=0 v23) ==== 121+0+0 (secure 0 0 0) 0x7fe8b80d29d0 con 0x7fe8d01083a0 2024-09-21T03:37:49.017 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:49.015+0000 7fe8d4a9d640 1 -- 172.21.15.89:0/1956670549 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7fe8bc0b6540 msgr2=0x7fe8bc0b8a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:49.017 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:49.015+0000 7fe8d4a9d640 1 --2- 172.21.15.89:0/1956670549 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7fe8bc0b6540 0x7fe8bc0b8a50 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fe8c0005e10 tx=0x7fe8c0005da0 comp rx=0 tx=0).stop 2024-09-21T03:37:49.017 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:49.015+0000 7fe8d4a9d640 1 -- 172.21.15.89:0/1956670549 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe8d01083a0 msgr2=0x7fe8d019aa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:49.018 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:49.015+0000 7fe8d4a9d640 1 --2- 172.21.15.89:0/1956670549 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe8d01083a0 0x7fe8d019aa70 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fe8b8005940 tx=0x7fe8b80042f0 comp rx=0 tx=0).stop 2024-09-21T03:37:49.018 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:49.015+0000 7fe8d4a9d640 1 -- 172.21.15.89:0/1956670549 shutdown_connections 2024-09-21T03:37:49.018 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:49.015+0000 7fe8d4a9d640 1 --2- 172.21.15.89:0/1956670549 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7fe8bc0b6540 0x7fe8bc0b8a50 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:49.018 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:49.015+0000 7fe8d4a9d640 1 --2- 172.21.15.89:0/1956670549 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe8d0108cf0 0x7fe8d019fd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:49.018 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:49.015+0000 7fe8d4a9d640 1 --2- 172.21.15.89:0/1956670549 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe8d01083a0 0x7fe8d019aa70 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:49.018 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:49.015+0000 7fe8d4a9d640 1 --2- 172.21.15.89:0/1956670549 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe8d0102a60 0x7fe8d019a4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:49.018 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:49.015+0000 7fe8d4a9d640 1 -- 172.21.15.89:0/1956670549 >> 172.21.15.89:0/1956670549 conn(0x7fe8d0079730 msgr2=0x7fe8d0101b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:37:49.018 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:49.015+0000 7fe8d4a9d640 1 -- 172.21.15.89:0/1956670549 shutdown_connections 2024-09-21T03:37:49.018 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:49.015+0000 7fe8d4a9d640 1 -- 172.21.15.89:0/1956670549 wait complete. 2024-09-21T03:37:49.311 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:37:49 smithi089 ceph-mon[30928]: pgmap v308: 129 pgs: 129 active+clean; 582 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s wr, 1 op/s 2024-09-21T03:37:49.311 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:37:49 smithi089 ceph-mon[30928]: mds.? [v2:172.21.15.114:6838/3427369944,v1:172.21.15.114:6839/3427369944] up:active 2024-09-21T03:37:49.311 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:37:49 smithi089 ceph-mon[30928]: daemon mds.h assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-09-21T03:37:49.311 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:37:49 smithi089 ceph-mon[30928]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-09-21T03:37:49.311 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:37:49 smithi089 ceph-mon[30928]: Cluster is now healthy 2024-09-21T03:37:49.311 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:37:49 smithi089 ceph-mon[30928]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 10 up:standby 2024-09-21T03:37:49.311 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:37:49 smithi089 ceph-mon[30928]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:creating} 9 up:standby 2024-09-21T03:37:49.311 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:37:49 smithi089 ceph-mon[30928]: daemon mds.h is now active in filesystem cephfs as rank 2 2024-09-21T03:37:49.311 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:37:49 smithi089 ceph-mon[30928]: from='client.? 172.21.15.89:0/1956670549' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} : dispatch 2024-09-21T03:37:49.461 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:37:49 smithi114 ceph-mon[34140]: pgmap v308: 129 pgs: 129 active+clean; 582 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s wr, 1 op/s 2024-09-21T03:37:49.461 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:37:49 smithi114 ceph-mon[34140]: mds.? [v2:172.21.15.114:6838/3427369944,v1:172.21.15.114:6839/3427369944] up:active 2024-09-21T03:37:49.461 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:37:49 smithi114 ceph-mon[34140]: daemon mds.h assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-09-21T03:37:49.461 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:37:49 smithi114 ceph-mon[34140]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-09-21T03:37:49.461 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:37:49 smithi114 ceph-mon[34140]: Cluster is now healthy 2024-09-21T03:37:49.461 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:37:49 smithi114 ceph-mon[34140]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 10 up:standby 2024-09-21T03:37:49.461 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:37:49 smithi114 ceph-mon[34140]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:creating} 9 up:standby 2024-09-21T03:37:49.461 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:37:49 smithi114 ceph-mon[34140]: daemon mds.h is now active in filesystem cephfs as rank 2 2024-09-21T03:37:49.462 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:37:49 smithi114 ceph-mon[34140]: from='client.? 172.21.15.89:0/1956670549' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} : dispatch 2024-09-21T03:37:49.500 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:37:49 smithi110 ceph-mon[34000]: pgmap v308: 129 pgs: 129 active+clean; 582 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s wr, 1 op/s 2024-09-21T03:37:49.500 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:37:49 smithi110 ceph-mon[34000]: mds.? [v2:172.21.15.114:6838/3427369944,v1:172.21.15.114:6839/3427369944] up:active 2024-09-21T03:37:49.500 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:37:49 smithi110 ceph-mon[34000]: daemon mds.h assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-09-21T03:37:49.500 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:37:49 smithi110 ceph-mon[34000]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-09-21T03:37:49.500 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:37:49 smithi110 ceph-mon[34000]: Cluster is now healthy 2024-09-21T03:37:49.500 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:37:49 smithi110 ceph-mon[34000]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 10 up:standby 2024-09-21T03:37:49.500 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:37:49 smithi110 ceph-mon[34000]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:creating} 9 up:standby 2024-09-21T03:37:49.500 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:37:49 smithi110 ceph-mon[34000]: daemon mds.h is now active in filesystem cephfs as rank 2 2024-09-21T03:37:49.500 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:37:49 smithi110 ceph-mon[34000]: from='client.? 172.21.15.89:0/1956670549' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} : dispatch 2024-09-21T03:37:49.743 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 7c374098-77c9-11ef-baf5-efdc52797490 -- ceph fs set cephfs session_timeout 300 2024-09-21T03:37:49.989 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/7c374098-77c9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:37:50.273 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:37:50 smithi089 ceph-mon[30928]: from='client.? 172.21.15.89:0/1956670549' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-09-21T03:37:50.273 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:37:50 smithi089 ceph-mon[30928]: mds.? [v2:172.21.15.110:6836/4047124589,v1:172.21.15.110:6837/4047124589] up:active 2024-09-21T03:37:50.273 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:37:50 smithi089 ceph-mon[30928]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 9 up:standby 2024-09-21T03:37:50.274 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:37:50 smithi089 ceph-mon[30928]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-09-21T03:37:50.461 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:37:50 smithi114 ceph-mon[34140]: from='client.? 172.21.15.89:0/1956670549' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-09-21T03:37:50.461 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:37:50 smithi114 ceph-mon[34140]: mds.? [v2:172.21.15.110:6836/4047124589,v1:172.21.15.110:6837/4047124589] up:active 2024-09-21T03:37:50.461 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:37:50 smithi114 ceph-mon[34140]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 9 up:standby 2024-09-21T03:37:50.461 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:37:50 smithi114 ceph-mon[34140]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-09-21T03:37:50.496 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:50.494+0000 7f7e588a3640 1 -- 172.21.15.89:0/64186066 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7e541083c0 msgr2=0x7f7e541087d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:50.496 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:50.494+0000 7f7e588a3640 1 --2- 172.21.15.89:0/64186066 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7e541083c0 0x7f7e541087d0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f7e48009900 tx=0x7f7e4802a410 comp rx=0 tx=0).stop 2024-09-21T03:37:50.497 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:50.494+0000 7f7e588a3640 1 -- 172.21.15.89:0/64186066 shutdown_connections 2024-09-21T03:37:50.497 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:50.494+0000 7f7e588a3640 1 --2- 172.21.15.89:0/64186066 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7e54108d10 0x7f7e5410f540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:50.497 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:50.494+0000 7f7e588a3640 1 --2- 172.21.15.89:0/64186066 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7e541083c0 0x7f7e541087d0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:50.497 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:50.494+0000 7f7e588a3640 1 --2- 172.21.15.89:0/64186066 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7e54102a80 0x7f7e54102e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:50.497 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:50.494+0000 7f7e588a3640 1 -- 172.21.15.89:0/64186066 >> 172.21.15.89:0/64186066 conn(0x7f7e54079740 msgr2=0x7f7e54079b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:37:50.497 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:50.494+0000 7f7e588a3640 1 -- 172.21.15.89:0/64186066 shutdown_connections 2024-09-21T03:37:50.497 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:50.494+0000 7f7e588a3640 1 -- 172.21.15.89:0/64186066 wait complete. 2024-09-21T03:37:50.499 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:50.495+0000 7f7e588a3640 1 Processor -- start 2024-09-21T03:37:50.499 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:50.495+0000 7f7e588a3640 1 -- start start 2024-09-21T03:37:50.499 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:50.495+0000 7f7e588a3640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7e54102a80 0x7f7e5419c610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:50.499 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:50.495+0000 7f7e588a3640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7e541083c0 0x7f7e5419cbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:50.499 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:50.495+0000 7f7e588a3640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7e54108d10 0x7f7e541a1e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:50.500 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:50.495+0000 7f7e588a3640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f7e54112830 con 0x7f7e541083c0 2024-09-21T03:37:50.500 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:50.495+0000 7f7e588a3640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f7e541126b0 con 0x7f7e54102a80 2024-09-21T03:37:50.500 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:50.495+0000 7f7e588a3640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f7e541129b0 con 0x7f7e54108d10 2024-09-21T03:37:50.500 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:50.495+0000 7f7e51d9b640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7e541083c0 0x7f7e5419cbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:50.500 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:50.495+0000 7f7e51d9b640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7e541083c0 0x7f7e5419cbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:38504/0 (socket says 172.21.15.89:38504) 2024-09-21T03:37:50.500 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:50.495+0000 7f7e51d9b640 1 -- 172.21.15.89:0/649259593 learned_addr learned my addr 172.21.15.89:0/649259593 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-21T03:37:50.500 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:50.495+0000 7f7e52d9d640 1 --2- 172.21.15.89:0/649259593 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7e54108d10 0x7f7e541a1e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:50.500 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:50.495+0000 7f7e5259c640 1 --2- 172.21.15.89:0/649259593 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7e54102a80 0x7f7e5419c610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:50.500 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:50.495+0000 7f7e51d9b640 1 -- 172.21.15.89:0/649259593 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7e54108d10 msgr2=0x7f7e541a1e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:50.500 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:50.495+0000 7f7e51d9b640 1 --2- 172.21.15.89:0/649259593 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7e54108d10 0x7f7e541a1e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:50.500 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:50.495+0000 7f7e51d9b640 1 -- 172.21.15.89:0/649259593 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7e54102a80 msgr2=0x7f7e5419c610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:50.500 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:50.495+0000 7f7e51d9b640 1 --2- 172.21.15.89:0/649259593 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7e54102a80 0x7f7e5419c610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:50.500 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:50.495+0000 7f7e51d9b640 1 -- 172.21.15.89:0/649259593 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e541a2580 con 0x7f7e541083c0 2024-09-21T03:37:50.500 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:50.496+0000 7f7e51d9b640 1 --2- 172.21.15.89:0/649259593 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7e541083c0 0x7f7e5419cbb0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f7e48005940 tx=0x7f7e480042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:37:50.501 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:50.496+0000 7f7e437fe640 1 -- 172.21.15.89:0/649259593 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e4803f030 con 0x7f7e541083c0 2024-09-21T03:37:50.501 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:50.496+0000 7f7e588a3640 1 -- 172.21.15.89:0/649259593 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7e541a2810 con 0x7f7e541083c0 2024-09-21T03:37:50.501 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:50.496+0000 7f7e437fe640 1 -- 172.21.15.89:0/649259593 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7e48003c90 con 0x7f7e541083c0 2024-09-21T03:37:50.501 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:50.496+0000 7f7e437fe640 1 -- 172.21.15.89:0/649259593 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e48035070 con 0x7f7e541083c0 2024-09-21T03:37:50.501 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:50.496+0000 7f7e588a3640 1 -- 172.21.15.89:0/649259593 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7e541a8fd0 con 0x7f7e541083c0 2024-09-21T03:37:50.501 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:50.497+0000 7f7e588a3640 1 -- 172.21.15.89:0/649259593 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7e54103d00 con 0x7f7e541083c0 2024-09-21T03:37:50.502 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:37:50 smithi110 ceph-mon[34000]: from='client.? 172.21.15.89:0/1956670549' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-09-21T03:37:50.502 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:37:50 smithi110 ceph-mon[34000]: mds.? [v2:172.21.15.110:6836/4047124589,v1:172.21.15.110:6837/4047124589] up:active 2024-09-21T03:37:50.502 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:37:50 smithi110 ceph-mon[34000]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 9 up:standby 2024-09-21T03:37:50.502 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:37:50 smithi110 ceph-mon[34000]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-09-21T03:37:50.502 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:50.498+0000 7f7e437fe640 1 -- 172.21.15.89:0/649259593 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 17) ==== 154512+0+0 (secure 0 0 0) 0x7f7e48042030 con 0x7f7e541083c0 2024-09-21T03:37:50.502 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:50.499+0000 7f7e437fe640 1 --2- 172.21.15.89:0/649259593 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f7e3c0b6510 0x7f7e3c0b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:50.503 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:50.499+0000 7f7e5259c640 1 --2- 172.21.15.89:0/649259593 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f7e3c0b6510 0x7f7e3c0b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:50.503 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:50.499+0000 7f7e437fe640 1 -- 172.21.15.89:0/649259593 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8802+0+0 (secure 0 0 0) 0x7f7e48107e70 con 0x7f7e541083c0 2024-09-21T03:37:50.503 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:50.500+0000 7f7e5259c640 1 --2- 172.21.15.89:0/649259593 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f7e3c0b6510 0x7f7e3c0b8a20 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f7e440058c0 tx=0x7f7e4400a2d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:37:50.504 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:50.502+0000 7f7e437fe640 1 -- 172.21.15.89:0/649259593 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f7e480d28a0 con 0x7f7e541083c0 2024-09-21T03:37:50.719 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:50.717+0000 7f7e588a3640 1 -- 172.21.15.89:0/649259593 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} v 0) -- 0x7f7e54102e90 con 0x7f7e541083c0 2024-09-21T03:37:51.022 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:51.019+0000 7f7e437fe640 1 -- 172.21.15.89:0/649259593 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]=0 v25) ==== 115+0+0 (secure 0 0 0) 0x7f7e54102e90 con 0x7f7e541083c0 2024-09-21T03:37:51.024 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:51.022+0000 7f7e588a3640 1 -- 172.21.15.89:0/649259593 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f7e3c0b6510 msgr2=0x7f7e3c0b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:51.025 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:51.022+0000 7f7e588a3640 1 --2- 172.21.15.89:0/649259593 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f7e3c0b6510 0x7f7e3c0b8a20 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f7e440058c0 tx=0x7f7e4400a2d0 comp rx=0 tx=0).stop 2024-09-21T03:37:51.025 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:51.022+0000 7f7e588a3640 1 -- 172.21.15.89:0/649259593 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7e541083c0 msgr2=0x7f7e5419cbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:51.025 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:51.022+0000 7f7e588a3640 1 --2- 172.21.15.89:0/649259593 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7e541083c0 0x7f7e5419cbb0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f7e48005940 tx=0x7f7e480042f0 comp rx=0 tx=0).stop 2024-09-21T03:37:51.025 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:51.022+0000 7f7e588a3640 1 -- 172.21.15.89:0/649259593 shutdown_connections 2024-09-21T03:37:51.025 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:51.022+0000 7f7e588a3640 1 --2- 172.21.15.89:0/649259593 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f7e3c0b6510 0x7f7e3c0b8a20 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:51.025 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:51.022+0000 7f7e588a3640 1 --2- 172.21.15.89:0/649259593 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7e54108d10 0x7f7e541a1e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:51.025 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:51.022+0000 7f7e588a3640 1 --2- 172.21.15.89:0/649259593 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7e541083c0 0x7f7e5419cbb0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:51.025 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:51.022+0000 7f7e588a3640 1 --2- 172.21.15.89:0/649259593 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7e54102a80 0x7f7e5419c610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:51.025 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:51.022+0000 7f7e588a3640 1 -- 172.21.15.89:0/649259593 >> 172.21.15.89:0/649259593 conn(0x7f7e54079740 msgr2=0x7f7e5406be20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:37:51.025 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:51.022+0000 7f7e588a3640 1 -- 172.21.15.89:0/649259593 shutdown_connections 2024-09-21T03:37:51.025 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:51.022+0000 7f7e588a3640 1 -- 172.21.15.89:0/649259593 wait complete. 2024-09-21T03:37:51.303 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:37:51 smithi089 ceph-mon[30928]: pgmap v309: 129 pgs: 129 active+clean; 582 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 682 B/s rd, 1.4 KiB/s wr, 5 op/s 2024-09-21T03:37:51.303 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:37:51 smithi089 ceph-mon[30928]: from='client.? 172.21.15.89:0/649259593' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} : dispatch 2024-09-21T03:37:51.461 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:37:51 smithi114 ceph-mon[34140]: pgmap v309: 129 pgs: 129 active+clean; 582 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 682 B/s rd, 1.4 KiB/s wr, 5 op/s 2024-09-21T03:37:51.461 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:37:51 smithi114 ceph-mon[34140]: from='client.? 172.21.15.89:0/649259593' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} : dispatch 2024-09-21T03:37:51.500 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:37:51 smithi110 ceph-mon[34000]: pgmap v309: 129 pgs: 129 active+clean; 582 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 682 B/s rd, 1.4 KiB/s wr, 5 op/s 2024-09-21T03:37:51.500 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:37:51 smithi110 ceph-mon[34000]: from='client.? 172.21.15.89:0/649259593' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} : dispatch 2024-09-21T03:37:51.683 DEBUG:tasks.cephfs.filesystem:Creating {'create': 2} subvols for filesystem 'cephfs' 2024-09-21T03:37:51.683 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 7c374098-77c9-11ef-baf5-efdc52797490 -- ceph fs subvolumegroup create cephfs qa 2024-09-21T03:37:51.941 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/7c374098-77c9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:37:52.302 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:37:52 smithi089 ceph-mon[30928]: from='client.? 172.21.15.89:0/649259593' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-09-21T03:37:52.302 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:37:52 smithi089 ceph-mon[30928]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-09-21T03:37:52.302 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:37:52 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.6", "id": [8, 11]} : dispatch 2024-09-21T03:37:52.302 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:37:52 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.c", "id": [2, 4]} : dispatch 2024-09-21T03:37:52.302 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:37:52 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 1]} : dispatch 2024-09-21T03:37:52.302 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:37:52 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [8, 11]} : dispatch 2024-09-21T03:37:52.303 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:37:52 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 9]} : dispatch 2024-09-21T03:37:52.303 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:37:52 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.34", "id": [4, 9]} : dispatch 2024-09-21T03:37:52.303 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:37:52 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.6", "id": [9, 11]} : dispatch 2024-09-21T03:37:52.303 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:37:52 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.12", "id": [9, 10]} : dispatch 2024-09-21T03:37:52.303 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:37:52 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.13", "id": [9, 8]} : dispatch 2024-09-21T03:37:52.303 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:37:52 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [5, 8]} : dispatch 2024-09-21T03:37:52.441 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:52.433+0000 7f1e543fb640 1 -- 172.21.15.89:0/2286466117 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1e4c108d20 msgr2=0x7f1e4c10f550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:52.441 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:52.433+0000 7f1e543fb640 1 --2- 172.21.15.89:0/2286466117 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1e4c108d20 0x7f1e4c10f550 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f1e44009ca0 tx=0x7f1e4402a8c0 comp rx=0 tx=0).stop 2024-09-21T03:37:52.441 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:52.433+0000 7f1e543fb640 1 -- 172.21.15.89:0/2286466117 shutdown_connections 2024-09-21T03:37:52.441 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:52.433+0000 7f1e543fb640 1 --2- 172.21.15.89:0/2286466117 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1e4c1083d0 0x7f1e4c1087e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:52.441 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:52.433+0000 7f1e543fb640 1 --2- 172.21.15.89:0/2286466117 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1e4c108d20 0x7f1e4c10f550 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:52.441 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:52.433+0000 7f1e543fb640 1 --2- 172.21.15.89:0/2286466117 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1e4c102a90 0x7f1e4c102ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:52.441 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:52.433+0000 7f1e543fb640 1 -- 172.21.15.89:0/2286466117 >> 172.21.15.89:0/2286466117 conn(0x7f1e4c079750 msgr2=0x7f1e4c079b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:37:52.441 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:52.434+0000 7f1e543fb640 1 -- 172.21.15.89:0/2286466117 shutdown_connections 2024-09-21T03:37:52.441 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:52.434+0000 7f1e543fb640 1 -- 172.21.15.89:0/2286466117 wait complete. 2024-09-21T03:37:52.442 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:52.434+0000 7f1e543fb640 1 Processor -- start 2024-09-21T03:37:52.442 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:52.434+0000 7f1e543fb640 1 -- start start 2024-09-21T03:37:52.442 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:52.435+0000 7f1e543fb640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1e4c102a90 0x7f1e4c199300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:52.442 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:52.435+0000 7f1e543fb640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1e4c1083d0 0x7f1e4c1998a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:52.442 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:52.435+0000 7f1e543fb640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1e4c108d20 0x7f1e4c19eb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:52.442 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:52.435+0000 7f1e543fb640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f1e4c112890 con 0x7f1e4c108d20 2024-09-21T03:37:52.442 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:52.435+0000 7f1e543fb640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f1e4c112710 con 0x7f1e4c102a90 2024-09-21T03:37:52.442 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:52.435+0000 7f1e543fb640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f1e4c112a10 con 0x7f1e4c1083d0 2024-09-21T03:37:52.442 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:52.435+0000 7f1e52998640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1e4c108d20 0x7f1e4c19eb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:52.442 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:52.435+0000 7f1e52998640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1e4c108d20 0x7f1e4c19eb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:38524/0 (socket says 172.21.15.89:38524) 2024-09-21T03:37:52.442 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:52.435+0000 7f1e52998640 1 -- 172.21.15.89:0/3628671223 learned_addr learned my addr 172.21.15.89:0/3628671223 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-21T03:37:52.442 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:52.435+0000 7f1e51996640 1 --2- 172.21.15.89:0/3628671223 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1e4c1083d0 0x7f1e4c1998a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:52.442 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:52.435+0000 7f1e52197640 1 --2- 172.21.15.89:0/3628671223 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1e4c102a90 0x7f1e4c199300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:52.442 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:52.435+0000 7f1e52998640 1 -- 172.21.15.89:0/3628671223 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1e4c1083d0 msgr2=0x7f1e4c1998a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:52.442 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:52.435+0000 7f1e52998640 1 --2- 172.21.15.89:0/3628671223 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1e4c1083d0 0x7f1e4c1998a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:52.443 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:52.435+0000 7f1e52998640 1 -- 172.21.15.89:0/3628671223 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1e4c102a90 msgr2=0x7f1e4c199300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:52.443 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:52.435+0000 7f1e52998640 1 --2- 172.21.15.89:0/3628671223 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1e4c102a90 0x7f1e4c199300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:52.443 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:52.435+0000 7f1e52998640 1 -- 172.21.15.89:0/3628671223 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e4c19f270 con 0x7f1e4c108d20 2024-09-21T03:37:52.443 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:52.435+0000 7f1e52197640 1 --2- 172.21.15.89:0/3628671223 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1e4c102a90 0x7f1e4c199300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:37:52.443 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:52.435+0000 7f1e52998640 1 --2- 172.21.15.89:0/3628671223 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1e4c108d20 0x7f1e4c19eb40 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f1e44005cc0 tx=0x7f1e44032c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:37:52.443 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:52.435+0000 7f1e437fe640 1 -- 172.21.15.89:0/3628671223 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e44004040 con 0x7f1e4c108d20 2024-09-21T03:37:52.443 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:52.436+0000 7f1e437fe640 1 -- 172.21.15.89:0/3628671223 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1e44004670 con 0x7f1e4c108d20 2024-09-21T03:37:52.443 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:52.436+0000 7f1e437fe640 1 -- 172.21.15.89:0/3628671223 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e44030cc0 con 0x7f1e4c108d20 2024-09-21T03:37:52.443 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:52.436+0000 7f1e543fb640 1 -- 172.21.15.89:0/3628671223 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1e4c19f500 con 0x7f1e4c108d20 2024-09-21T03:37:52.443 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:52.436+0000 7f1e543fb640 1 -- 172.21.15.89:0/3628671223 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1e4c1a5cc0 con 0x7f1e4c108d20 2024-09-21T03:37:52.443 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:52.437+0000 7f1e543fb640 1 -- 172.21.15.89:0/3628671223 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1e4c103d10 con 0x7f1e4c108d20 2024-09-21T03:37:52.443 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:52.438+0000 7f1e437fe640 1 -- 172.21.15.89:0/3628671223 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 17) ==== 154512+0+0 (secure 0 0 0) 0x7f1e440041e0 con 0x7f1e4c108d20 2024-09-21T03:37:52.444 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:52.439+0000 7f1e437fe640 1 --2- 172.21.15.89:0/3628671223 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f1e380b6510 0x7f1e380b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:52.444 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:52.439+0000 7f1e52197640 1 --2- 172.21.15.89:0/3628671223 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f1e380b6510 0x7f1e380b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:52.444 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:52.439+0000 7f1e437fe640 1 -- 172.21.15.89:0/3628671223 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 9092+0+0 (secure 0 0 0) 0x7f1e44108c10 con 0x7f1e4c108d20 2024-09-21T03:37:52.444 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:52.440+0000 7f1e52197640 1 --2- 172.21.15.89:0/3628671223 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f1e380b6510 0x7f1e380b8a20 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f1e48005fd0 tx=0x7f1e48005ea0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:37:52.445 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:52.443+0000 7f1e437fe640 1 -- 172.21.15.89:0/3628671223 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f1e440cf140 con 0x7f1e4c108d20 2024-09-21T03:37:52.461 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:37:52 smithi114 ceph-mon[34140]: from='client.? 172.21.15.89:0/649259593' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-09-21T03:37:52.461 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:37:52 smithi114 ceph-mon[34140]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-09-21T03:37:52.462 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:37:52 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.6", "id": [8, 11]} : dispatch 2024-09-21T03:37:52.462 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:37:52 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.c", "id": [2, 4]} : dispatch 2024-09-21T03:37:52.462 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:37:52 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 1]} : dispatch 2024-09-21T03:37:52.462 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:37:52 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [8, 11]} : dispatch 2024-09-21T03:37:52.462 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:37:52 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 9]} : dispatch 2024-09-21T03:37:52.462 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:37:52 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.34", "id": [4, 9]} : dispatch 2024-09-21T03:37:52.462 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:37:52 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.6", "id": [9, 11]} : dispatch 2024-09-21T03:37:52.462 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:37:52 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.12", "id": [9, 10]} : dispatch 2024-09-21T03:37:52.462 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:37:52 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.13", "id": [9, 8]} : dispatch 2024-09-21T03:37:52.462 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:37:52 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [5, 8]} : dispatch 2024-09-21T03:37:52.500 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:37:52 smithi110 ceph-mon[34000]: from='client.? 172.21.15.89:0/649259593' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-09-21T03:37:52.500 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:37:52 smithi110 ceph-mon[34000]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-09-21T03:37:52.500 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:37:52 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.6", "id": [8, 11]} : dispatch 2024-09-21T03:37:52.500 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:37:52 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.c", "id": [2, 4]} : dispatch 2024-09-21T03:37:52.500 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:37:52 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 1]} : dispatch 2024-09-21T03:37:52.500 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:37:52 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [8, 11]} : dispatch 2024-09-21T03:37:52.500 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:37:52 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 9]} : dispatch 2024-09-21T03:37:52.500 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:37:52 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.34", "id": [4, 9]} : dispatch 2024-09-21T03:37:52.500 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:37:52 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.6", "id": [9, 11]} : dispatch 2024-09-21T03:37:52.500 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:37:52 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.12", "id": [9, 10]} : dispatch 2024-09-21T03:37:52.500 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:37:52 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.13", "id": [9, 8]} : dispatch 2024-09-21T03:37:52.500 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:37:52 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [5, 8]} : dispatch 2024-09-21T03:37:52.648 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:52.645+0000 7f1e543fb640 1 -- 172.21.15.89:0/3628671223 --> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f1e4c10ab20 con 0x7f1e380b6510 2024-09-21T03:37:52.675 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:52.672+0000 7f1e437fe640 1 -- 172.21.15.89:0/3628671223 <== mgr.14152 v2:172.21.15.89:6800/1591532358 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f1e4c10ab20 con 0x7f1e380b6510 2024-09-21T03:37:52.677 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:52.674+0000 7f1e543fb640 1 -- 172.21.15.89:0/3628671223 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f1e380b6510 msgr2=0x7f1e380b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:52.677 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:52.674+0000 7f1e543fb640 1 --2- 172.21.15.89:0/3628671223 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f1e380b6510 0x7f1e380b8a20 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f1e48005fd0 tx=0x7f1e48005ea0 comp rx=0 tx=0).stop 2024-09-21T03:37:52.677 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:52.675+0000 7f1e543fb640 1 -- 172.21.15.89:0/3628671223 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1e4c108d20 msgr2=0x7f1e4c19eb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:52.677 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:52.675+0000 7f1e543fb640 1 --2- 172.21.15.89:0/3628671223 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1e4c108d20 0x7f1e4c19eb40 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f1e44005cc0 tx=0x7f1e44032c40 comp rx=0 tx=0).stop 2024-09-21T03:37:52.677 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:52.675+0000 7f1e543fb640 1 -- 172.21.15.89:0/3628671223 shutdown_connections 2024-09-21T03:37:52.677 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:52.675+0000 7f1e543fb640 1 --2- 172.21.15.89:0/3628671223 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f1e380b6510 0x7f1e380b8a20 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:52.677 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:52.675+0000 7f1e543fb640 1 --2- 172.21.15.89:0/3628671223 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1e4c108d20 0x7f1e4c19eb40 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:52.677 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:52.675+0000 7f1e543fb640 1 --2- 172.21.15.89:0/3628671223 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1e4c1083d0 0x7f1e4c1998a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:52.677 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:52.675+0000 7f1e543fb640 1 --2- 172.21.15.89:0/3628671223 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1e4c102a90 0x7f1e4c199300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:52.677 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:52.675+0000 7f1e543fb640 1 -- 172.21.15.89:0/3628671223 >> 172.21.15.89:0/3628671223 conn(0x7f1e4c079750 msgr2=0x7f1e4c104450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:37:52.677 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:52.675+0000 7f1e543fb640 1 -- 172.21.15.89:0/3628671223 shutdown_connections 2024-09-21T03:37:52.677 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:52.675+0000 7f1e543fb640 1 -- 172.21.15.89:0/3628671223 wait complete. 2024-09-21T03:37:53.036 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:37:53 smithi089 ceph-mon[30928]: pgmap v310: 129 pgs: 129 active+clean; 582 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 682 B/s rd, 1.1 KiB/s wr, 5 op/s 2024-09-21T03:37:53.036 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:37:53 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.6", "id": [8, 11]}]': finished 2024-09-21T03:37:53.036 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:37:53 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.c", "id": [2, 4]}]': finished 2024-09-21T03:37:53.036 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:37:53 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 1]}]': finished 2024-09-21T03:37:53.036 INFO:journalctl@ceph.mgr.x.smithi089.stdout:Sep 21 03:37:52 smithi089 ceph-7c374098-77c9-11ef-baf5-efdc52797490-mgr-x[31154]: 2024-09-21T03:37:52.651+0000 7f54ad47b640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T03:37:53.036 INFO:journalctl@ceph.mgr.x.smithi089.stdout:Sep 21 03:37:52 smithi089 ceph-7c374098-77c9-11ef-baf5-efdc52797490-mgr-x[31154]: 2024-09-21T03:37:52.651+0000 7f54ad47b640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T03:37:53.036 INFO:journalctl@ceph.mgr.x.smithi089.stdout:Sep 21 03:37:52 smithi089 ceph-7c374098-77c9-11ef-baf5-efdc52797490-mgr-x[31154]: 2024-09-21T03:37:52.651+0000 7f54ad47b640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T03:37:53.036 INFO:journalctl@ceph.mgr.x.smithi089.stdout:Sep 21 03:37:52 smithi089 ceph-7c374098-77c9-11ef-baf5-efdc52797490-mgr-x[31154]: 2024-09-21T03:37:52.651+0000 7f54ad47b640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T03:37:53.037 INFO:journalctl@ceph.mgr.x.smithi089.stdout:Sep 21 03:37:52 smithi089 ceph-7c374098-77c9-11ef-baf5-efdc52797490-mgr-x[31154]: 2024-09-21T03:37:52.651+0000 7f54ad47b640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T03:37:53.311 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:37:53 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [8, 11]}]': finished 2024-09-21T03:37:53.311 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:37:53 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 9]}]': finished 2024-09-21T03:37:53.311 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:37:53 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.34", "id": [4, 9]}]': finished 2024-09-21T03:37:53.311 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:37:53 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.6", "id": [9, 11]}]': finished 2024-09-21T03:37:53.311 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:37:53 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.12", "id": [9, 10]}]': finished 2024-09-21T03:37:53.312 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:37:53 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.13", "id": [9, 8]}]': finished 2024-09-21T03:37:53.312 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:37:53 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [5, 8]}]': finished 2024-09-21T03:37:53.312 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:37:53 smithi089 ceph-mon[30928]: osdmap e81: 12 total, 12 up, 12 in 2024-09-21T03:37:53.435 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 7c374098-77c9-11ef-baf5-efdc52797490 -- ceph fs subvolume create cephfs sv_0 --group_name qa 2024-09-21T03:37:53.461 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:37:53 smithi114 ceph-mon[34140]: pgmap v310: 129 pgs: 129 active+clean; 582 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 682 B/s rd, 1.1 KiB/s wr, 5 op/s 2024-09-21T03:37:53.461 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:37:53 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.6", "id": [8, 11]}]': finished 2024-09-21T03:37:53.461 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:37:53 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.c", "id": [2, 4]}]': finished 2024-09-21T03:37:53.461 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:37:53 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 1]}]': finished 2024-09-21T03:37:53.461 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:37:53 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [8, 11]}]': finished 2024-09-21T03:37:53.461 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:37:53 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 9]}]': finished 2024-09-21T03:37:53.461 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:37:53 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.34", "id": [4, 9]}]': finished 2024-09-21T03:37:53.461 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:37:53 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.6", "id": [9, 11]}]': finished 2024-09-21T03:37:53.461 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:37:53 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.12", "id": [9, 10]}]': finished 2024-09-21T03:37:53.461 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:37:53 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.13", "id": [9, 8]}]': finished 2024-09-21T03:37:53.461 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:37:53 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [5, 8]}]': finished 2024-09-21T03:37:53.461 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:37:53 smithi114 ceph-mon[34140]: osdmap e81: 12 total, 12 up, 12 in 2024-09-21T03:37:53.500 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:37:53 smithi110 ceph-mon[34000]: pgmap v310: 129 pgs: 129 active+clean; 582 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 682 B/s rd, 1.1 KiB/s wr, 5 op/s 2024-09-21T03:37:53.500 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:37:53 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.6", "id": [8, 11]}]': finished 2024-09-21T03:37:53.500 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:37:53 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.c", "id": [2, 4]}]': finished 2024-09-21T03:37:53.500 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:37:53 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 1]}]': finished 2024-09-21T03:37:53.500 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:37:53 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [8, 11]}]': finished 2024-09-21T03:37:53.500 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:37:53 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 9]}]': finished 2024-09-21T03:37:53.500 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:37:53 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.34", "id": [4, 9]}]': finished 2024-09-21T03:37:53.500 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:37:53 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.6", "id": [9, 11]}]': finished 2024-09-21T03:37:53.500 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:37:53 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.12", "id": [9, 10]}]': finished 2024-09-21T03:37:53.500 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:37:53 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.13", "id": [9, 8]}]': finished 2024-09-21T03:37:53.501 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:37:53 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [5, 8]}]': finished 2024-09-21T03:37:53.501 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:37:53 smithi110 ceph-mon[34000]: osdmap e81: 12 total, 12 up, 12 in 2024-09-21T03:37:53.683 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/7c374098-77c9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:37:54.061 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:37:54 smithi089 ceph-mon[30928]: from='client.14973 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:37:54.061 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:37:54 smithi089 ceph-mon[30928]: osdmap e82: 12 total, 12 up, 12 in 2024-09-21T03:37:54.164 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:54.162+0000 7f9a3b5c8640 1 -- 172.21.15.89:0/7517103 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9a341084b0 msgr2=0x7f9a3410ece0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:54.164 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:54.162+0000 7f9a3b5c8640 1 --2- 172.21.15.89:0/7517103 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9a341084b0 0x7f9a3410ece0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f9a2c009900 tx=0x7f9a2c02a410 comp rx=0 tx=0).stop 2024-09-21T03:37:54.165 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:54.162+0000 7f9a3b5c8640 1 -- 172.21.15.89:0/7517103 shutdown_connections 2024-09-21T03:37:54.165 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:54.162+0000 7f9a3b5c8640 1 --2- 172.21.15.89:0/7517103 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9a341084b0 0x7f9a3410ece0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:54.165 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:54.162+0000 7f9a3b5c8640 1 --2- 172.21.15.89:0/7517103 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9a34107b60 0x7f9a34107f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:54.165 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:54.162+0000 7f9a3b5c8640 1 --2- 172.21.15.89:0/7517103 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9a34102220 0x7f9a34102630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:54.165 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:54.162+0000 7f9a3b5c8640 1 -- 172.21.15.89:0/7517103 >> 172.21.15.89:0/7517103 conn(0x7f9a34079e30 msgr2=0x7f9a3407a240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:37:54.165 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:54.162+0000 7f9a3b5c8640 1 -- 172.21.15.89:0/7517103 shutdown_connections 2024-09-21T03:37:54.165 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:54.162+0000 7f9a3b5c8640 1 -- 172.21.15.89:0/7517103 wait complete. 2024-09-21T03:37:54.166 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:54.163+0000 7f9a3b5c8640 1 Processor -- start 2024-09-21T03:37:54.166 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:54.163+0000 7f9a3b5c8640 1 -- start start 2024-09-21T03:37:54.167 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:54.163+0000 7f9a3b5c8640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9a34102220 0x7f9a3419b1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:54.167 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:54.163+0000 7f9a3b5c8640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9a34107b60 0x7f9a3419b6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:54.167 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:54.163+0000 7f9a3b5c8640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9a341084b0 0x7f9a3419bc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:54.167 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:54.163+0000 7f9a3b5c8640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f9a341128a0 con 0x7f9a341084b0 2024-09-21T03:37:54.167 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:54.163+0000 7f9a3b5c8640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f9a34112720 con 0x7f9a34107b60 2024-09-21T03:37:54.167 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:54.163+0000 7f9a3b5c8640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f9a34112a20 con 0x7f9a34102220 2024-09-21T03:37:54.167 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:54.163+0000 7f9a39b65640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9a341084b0 0x7f9a3419bc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:54.167 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:54.163+0000 7f9a39364640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9a34102220 0x7f9a3419b1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:54.167 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:54.163+0000 7f9a39b65640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9a341084b0 0x7f9a3419bc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:38546/0 (socket says 172.21.15.89:38546) 2024-09-21T03:37:54.167 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:54.163+0000 7f9a39b65640 1 -- 172.21.15.89:0/2697801336 learned_addr learned my addr 172.21.15.89:0/2697801336 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-21T03:37:54.167 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:54.164+0000 7f9a38b63640 1 --2- 172.21.15.89:0/2697801336 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9a34107b60 0x7f9a3419b6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:54.167 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:54.164+0000 7f9a39b65640 1 -- 172.21.15.89:0/2697801336 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9a34102220 msgr2=0x7f9a3419b1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:54.167 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:54.164+0000 7f9a39b65640 1 --2- 172.21.15.89:0/2697801336 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9a34102220 0x7f9a3419b1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:54.167 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:54.164+0000 7f9a39b65640 1 -- 172.21.15.89:0/2697801336 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9a34107b60 msgr2=0x7f9a3419b6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:54.168 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:54.164+0000 7f9a39b65640 1 --2- 172.21.15.89:0/2697801336 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9a34107b60 0x7f9a3419b6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:54.168 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:54.164+0000 7f9a39b65640 1 -- 172.21.15.89:0/2697801336 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a341a6440 con 0x7f9a341084b0 2024-09-21T03:37:54.168 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:54.164+0000 7f9a39b65640 1 --2- 172.21.15.89:0/2697801336 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9a341084b0 0x7f9a3419bc30 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f9a2c004770 tx=0x7f9a2c004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:37:54.168 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:54.164+0000 7f9a327fc640 1 -- 172.21.15.89:0/2697801336 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a2c03f030 con 0x7f9a341084b0 2024-09-21T03:37:54.168 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:54.164+0000 7f9a3b5c8640 1 -- 172.21.15.89:0/2697801336 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9a341a66d0 con 0x7f9a341084b0 2024-09-21T03:37:54.168 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:54.164+0000 7f9a327fc640 1 -- 172.21.15.89:0/2697801336 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9a2c004440 con 0x7f9a341084b0 2024-09-21T03:37:54.168 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:54.164+0000 7f9a327fc640 1 -- 172.21.15.89:0/2697801336 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a2c035070 con 0x7f9a341084b0 2024-09-21T03:37:54.168 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:54.164+0000 7f9a3b5c8640 1 -- 172.21.15.89:0/2697801336 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9a341a6c00 con 0x7f9a341084b0 2024-09-21T03:37:54.169 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:54.165+0000 7f9a3b5c8640 1 -- 172.21.15.89:0/2697801336 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9a34102630 con 0x7f9a341084b0 2024-09-21T03:37:54.170 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:54.168+0000 7f9a327fc640 1 -- 172.21.15.89:0/2697801336 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 18) ==== 154563+0+0 (secure 0 0 0) 0x7f9a2c042030 con 0x7f9a341084b0 2024-09-21T03:37:54.171 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:54.168+0000 7f9a327fc640 1 --2- 172.21.15.89:0/2697801336 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f9a1c0b6650 0x7f9a1c0b8b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:54.171 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:54.169+0000 7f9a39364640 1 --2- 172.21.15.89:0/2697801336 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f9a1c0b6650 0x7f9a1c0b8b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:54.171 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:54.169+0000 7f9a327fc640 1 -- 172.21.15.89:0/2697801336 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 9092+0+0 (secure 0 0 0) 0x7f9a2c107ed0 con 0x7f9a341084b0 2024-09-21T03:37:54.171 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:54.169+0000 7f9a39364640 1 --2- 172.21.15.89:0/2697801336 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f9a1c0b6650 0x7f9a1c0b8b60 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f9a34110200 tx=0x7f9a24009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:37:54.174 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:54.172+0000 7f9a327fc640 1 -- 172.21.15.89:0/2697801336 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f9a2c0d2680 con 0x7f9a341084b0 2024-09-21T03:37:54.378 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:54.376+0000 7f9a3b5c8640 1 -- 172.21.15.89:0/2697801336 --> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f9a34063800 con 0x7f9a1c0b6650 2024-09-21T03:37:54.405 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:54.403+0000 7f9a327fc640 1 -- 172.21.15.89:0/2697801336 <== mgr.14152 v2:172.21.15.89:6800/1591532358 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f9a34063800 con 0x7f9a1c0b6650 2024-09-21T03:37:54.407 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:54.405+0000 7f9a3b5c8640 1 -- 172.21.15.89:0/2697801336 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f9a1c0b6650 msgr2=0x7f9a1c0b8b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:54.408 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:54.405+0000 7f9a3b5c8640 1 --2- 172.21.15.89:0/2697801336 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f9a1c0b6650 0x7f9a1c0b8b60 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f9a34110200 tx=0x7f9a24009500 comp rx=0 tx=0).stop 2024-09-21T03:37:54.408 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:54.405+0000 7f9a3b5c8640 1 -- 172.21.15.89:0/2697801336 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9a341084b0 msgr2=0x7f9a3419bc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:54.408 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:54.405+0000 7f9a3b5c8640 1 --2- 172.21.15.89:0/2697801336 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9a341084b0 0x7f9a3419bc30 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f9a2c004770 tx=0x7f9a2c004040 comp rx=0 tx=0).stop 2024-09-21T03:37:54.408 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:54.405+0000 7f9a3b5c8640 1 -- 172.21.15.89:0/2697801336 shutdown_connections 2024-09-21T03:37:54.408 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:54.405+0000 7f9a3b5c8640 1 --2- 172.21.15.89:0/2697801336 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f9a1c0b6650 0x7f9a1c0b8b60 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:54.408 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:54.405+0000 7f9a3b5c8640 1 --2- 172.21.15.89:0/2697801336 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9a34107b60 0x7f9a3419b6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:54.408 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:54.405+0000 7f9a3b5c8640 1 --2- 172.21.15.89:0/2697801336 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9a341084b0 0x7f9a3419bc30 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:54.408 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:54.405+0000 7f9a3b5c8640 1 --2- 172.21.15.89:0/2697801336 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9a34102220 0x7f9a3419b1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:54.408 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:54.405+0000 7f9a3b5c8640 1 -- 172.21.15.89:0/2697801336 >> 172.21.15.89:0/2697801336 conn(0x7f9a34079e30 msgr2=0x7f9a34105bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:37:54.408 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:54.405+0000 7f9a3b5c8640 1 -- 172.21.15.89:0/2697801336 shutdown_connections 2024-09-21T03:37:54.408 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:54.405+0000 7f9a3b5c8640 1 -- 172.21.15.89:0/2697801336 wait complete. 2024-09-21T03:37:54.460 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:37:54 smithi114 ceph-mon[34140]: from='client.14973 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:37:54.461 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:37:54 smithi114 ceph-mon[34140]: osdmap e82: 12 total, 12 up, 12 in 2024-09-21T03:37:54.500 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:37:54 smithi110 ceph-mon[34000]: from='client.14973 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:37:54.500 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:37:54 smithi110 ceph-mon[34000]: osdmap e82: 12 total, 12 up, 12 in 2024-09-21T03:37:55.145 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 7c374098-77c9-11ef-baf5-efdc52797490 -- ceph fs subvolume getpath cephfs --group_name qa sv_0 2024-09-21T03:37:55.311 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:37:55 smithi089 ceph-mon[30928]: pgmap v313: 129 pgs: 3 peering, 126 active+clean; 584 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 2.9 KiB/s wr, 13 op/s; 0 B/s, 0 objects/s recovering 2024-09-21T03:37:55.311 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:37:55 smithi089 ceph-mon[30928]: mgrmap e18: x(active, since 8m), standbys: z, y 2024-09-21T03:37:55.311 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:37:55 smithi089 ceph-mon[30928]: from='client.14985 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:37:55.390 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/7c374098-77c9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:37:55.461 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:37:55 smithi114 ceph-mon[34140]: pgmap v313: 129 pgs: 3 peering, 126 active+clean; 584 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 2.9 KiB/s wr, 13 op/s; 0 B/s, 0 objects/s recovering 2024-09-21T03:37:55.461 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:37:55 smithi114 ceph-mon[34140]: mgrmap e18: x(active, since 8m), standbys: z, y 2024-09-21T03:37:55.461 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:37:55 smithi114 ceph-mon[34140]: from='client.14985 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:37:55.500 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:37:55 smithi110 ceph-mon[34000]: pgmap v313: 129 pgs: 3 peering, 126 active+clean; 584 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 2.9 KiB/s wr, 13 op/s; 0 B/s, 0 objects/s recovering 2024-09-21T03:37:55.500 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:37:55 smithi110 ceph-mon[34000]: mgrmap e18: x(active, since 8m), standbys: z, y 2024-09-21T03:37:55.500 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:37:55 smithi110 ceph-mon[34000]: from='client.14985 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:37:55.883 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:55.881+0000 7f7549ea2640 1 -- 172.21.15.89:0/1841107090 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7544109e40 msgr2=0x7f7544110670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:55.883 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:55.881+0000 7f7549ea2640 1 --2- 172.21.15.89:0/1841107090 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7544109e40 0x7f7544110670 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f753c009900 tx=0x7f753c02a3f0 comp rx=0 tx=0).stop 2024-09-21T03:37:55.883 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:55.881+0000 7f7549ea2640 1 -- 172.21.15.89:0/1841107090 shutdown_connections 2024-09-21T03:37:55.883 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:55.881+0000 7f7549ea2640 1 --2- 172.21.15.89:0/1841107090 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7544109e40 0x7f7544110670 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:55.884 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:55.881+0000 7f7549ea2640 1 --2- 172.21.15.89:0/1841107090 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f75441094f0 0x7f7544109900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:55.884 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:55.881+0000 7f7549ea2640 1 --2- 172.21.15.89:0/1841107090 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7544103bb0 0x7f7544103fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:55.884 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:55.881+0000 7f7549ea2640 1 -- 172.21.15.89:0/1841107090 >> 172.21.15.89:0/1841107090 conn(0x7f75440ff600 msgr2=0x7f7544101a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:37:55.884 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:55.881+0000 7f7549ea2640 1 -- 172.21.15.89:0/1841107090 shutdown_connections 2024-09-21T03:37:55.884 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:55.881+0000 7f7549ea2640 1 -- 172.21.15.89:0/1841107090 wait complete. 2024-09-21T03:37:55.886 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:55.881+0000 7f7549ea2640 1 Processor -- start 2024-09-21T03:37:55.886 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:55.882+0000 7f7549ea2640 1 -- start start 2024-09-21T03:37:55.886 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:55.882+0000 7f7549ea2640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7544103bb0 0x7f754419a320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:55.886 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:55.882+0000 7f7549ea2640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f75441094f0 0x7f754419a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:55.886 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:55.882+0000 7f7549ea2640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7544109e40 0x7f754419fb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:55.886 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:55.882+0000 7f7549ea2640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f75441139b0 con 0x7f7544109e40 2024-09-21T03:37:55.886 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:55.882+0000 7f7549ea2640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f7544113830 con 0x7f75441094f0 2024-09-21T03:37:55.886 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:55.882+0000 7f7549ea2640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f7544113b30 con 0x7f7544103bb0 2024-09-21T03:37:55.887 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:55.882+0000 7f7543fff640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7544109e40 0x7f754419fb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:55.887 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:55.882+0000 7f7543fff640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7544109e40 0x7f754419fb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:38556/0 (socket says 172.21.15.89:38556) 2024-09-21T03:37:55.887 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:55.882+0000 7f7543fff640 1 -- 172.21.15.89:0/910102268 learned_addr learned my addr 172.21.15.89:0/910102268 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-21T03:37:55.887 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:55.882+0000 7f7542ffd640 1 --2- 172.21.15.89:0/910102268 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f75441094f0 0x7f754419a8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:55.887 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:55.882+0000 7f75437fe640 1 --2- 172.21.15.89:0/910102268 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7544103bb0 0x7f754419a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:55.887 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:55.882+0000 7f7543fff640 1 -- 172.21.15.89:0/910102268 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7544103bb0 msgr2=0x7f754419a320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:55.887 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:55.882+0000 7f7543fff640 1 --2- 172.21.15.89:0/910102268 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7544103bb0 0x7f754419a320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:55.887 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:55.882+0000 7f7543fff640 1 -- 172.21.15.89:0/910102268 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f75441094f0 msgr2=0x7f754419a8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:55.887 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:55.882+0000 7f7543fff640 1 --2- 172.21.15.89:0/910102268 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f75441094f0 0x7f754419a8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:55.887 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:55.882+0000 7f7543fff640 1 -- 172.21.15.89:0/910102268 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75441a01d0 con 0x7f7544109e40 2024-09-21T03:37:55.887 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:55.882+0000 7f75437fe640 1 --2- 172.21.15.89:0/910102268 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7544103bb0 0x7f754419a320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:37:55.887 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:55.883+0000 7f7543fff640 1 --2- 172.21.15.89:0/910102268 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7544109e40 0x7f754419fb60 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f753c005ab0 tx=0x7f753c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:37:55.887 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:55.883+0000 7f7540ff9640 1 -- 172.21.15.89:0/910102268 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f753c03f030 con 0x7f7544109e40 2024-09-21T03:37:55.887 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:55.883+0000 7f7540ff9640 1 -- 172.21.15.89:0/910102268 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f753c003c90 con 0x7f7544109e40 2024-09-21T03:37:55.887 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:55.883+0000 7f7540ff9640 1 -- 172.21.15.89:0/910102268 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f753c035070 con 0x7f7544109e40 2024-09-21T03:37:55.888 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:55.883+0000 7f7549ea2640 1 -- 172.21.15.89:0/910102268 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f75441a0460 con 0x7f7544109e40 2024-09-21T03:37:55.888 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:55.883+0000 7f7549ea2640 1 -- 172.21.15.89:0/910102268 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75441a08f0 con 0x7f7544109e40 2024-09-21T03:37:55.888 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:55.884+0000 7f7549ea2640 1 -- 172.21.15.89:0/910102268 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7544103fc0 con 0x7f7544109e40 2024-09-21T03:37:55.888 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:55.886+0000 7f7540ff9640 1 -- 172.21.15.89:0/910102268 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 18) ==== 154563+0+0 (secure 0 0 0) 0x7f753c042030 con 0x7f7544109e40 2024-09-21T03:37:55.888 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:55.886+0000 7f7540ff9640 1 --2- 172.21.15.89:0/910102268 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f752c0b6650 0x7f752c0b8b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:55.888 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:55.886+0000 7f75437fe640 1 --2- 172.21.15.89:0/910102268 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f752c0b6650 0x7f752c0b8b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:55.888 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:55.886+0000 7f7540ff9640 1 -- 172.21.15.89:0/910102268 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 9092+0+0 (secure 0 0 0) 0x7f753c108190 con 0x7f7544109e40 2024-09-21T03:37:55.889 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:55.887+0000 7f75437fe640 1 --2- 172.21.15.89:0/910102268 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f752c0b6650 0x7f752c0b8b60 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f753400abe0 tx=0x7f753400a5c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:37:55.891 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:55.890+0000 7f7540ff9640 1 -- 172.21.15.89:0/910102268 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f753c0ce560 con 0x7f7544109e40 2024-09-21T03:37:56.098 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:56.096+0000 7f7549ea2640 1 -- 172.21.15.89:0/910102268 --> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}) -- 0x7f7544063800 con 0x7f752c0b6650 2024-09-21T03:37:56.104 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:56.102+0000 7f7540ff9640 1 -- 172.21.15.89:0/910102268 <== mgr.14152 v2:172.21.15.89:6800/1591532358 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7f7544063800 con 0x7f752c0b6650 2024-09-21T03:37:56.104 INFO:teuthology.orchestra.run.smithi089.stdout:/volumes/qa/sv_0/d474bdba-6c8a-4432-8887-89da76119cad 2024-09-21T03:37:56.107 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:56.105+0000 7f7549ea2640 1 -- 172.21.15.89:0/910102268 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f752c0b6650 msgr2=0x7f752c0b8b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:56.107 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:56.105+0000 7f7549ea2640 1 --2- 172.21.15.89:0/910102268 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f752c0b6650 0x7f752c0b8b60 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f753400abe0 tx=0x7f753400a5c0 comp rx=0 tx=0).stop 2024-09-21T03:37:56.107 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:56.105+0000 7f7549ea2640 1 -- 172.21.15.89:0/910102268 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7544109e40 msgr2=0x7f754419fb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:56.107 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:56.105+0000 7f7549ea2640 1 --2- 172.21.15.89:0/910102268 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7544109e40 0x7f754419fb60 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f753c005ab0 tx=0x7f753c0042f0 comp rx=0 tx=0).stop 2024-09-21T03:37:56.107 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:56.105+0000 7f7549ea2640 1 -- 172.21.15.89:0/910102268 shutdown_connections 2024-09-21T03:37:56.108 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:56.105+0000 7f7549ea2640 1 --2- 172.21.15.89:0/910102268 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f752c0b6650 0x7f752c0b8b60 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:56.108 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:56.105+0000 7f7549ea2640 1 --2- 172.21.15.89:0/910102268 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f75441094f0 0x7f754419a8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:56.108 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:56.105+0000 7f7549ea2640 1 --2- 172.21.15.89:0/910102268 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7544109e40 0x7f754419fb60 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:56.108 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:56.105+0000 7f7549ea2640 1 --2- 172.21.15.89:0/910102268 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7544103bb0 0x7f754419a320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:56.108 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:56.105+0000 7f7549ea2640 1 -- 172.21.15.89:0/910102268 >> 172.21.15.89:0/910102268 conn(0x7f75440ff600 msgr2=0x7f7544101a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:37:56.108 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:56.105+0000 7f7549ea2640 1 -- 172.21.15.89:0/910102268 shutdown_connections 2024-09-21T03:37:56.108 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:56.105+0000 7f7549ea2640 1 -- 172.21.15.89:0/910102268 wait complete. 2024-09-21T03:37:56.883 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 7c374098-77c9-11ef-baf5-efdc52797490 -- ceph fs subvolume create cephfs sv_1 --group_name qa 2024-09-21T03:37:57.129 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/7c374098-77c9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:37:57.250 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:37:57 smithi089 ceph-mon[30928]: pgmap v314: 129 pgs: 3 peering, 126 active+clean; 590 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 3.7 KiB/s wr, 14 op/s; 13 B/s, 1 objects/s recovering 2024-09-21T03:37:57.250 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:37:57 smithi089 ceph-mon[30928]: from='client.14991 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:37:57.461 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:37:57 smithi114 ceph-mon[34140]: pgmap v314: 129 pgs: 3 peering, 126 active+clean; 590 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 3.7 KiB/s wr, 14 op/s; 13 B/s, 1 objects/s recovering 2024-09-21T03:37:57.461 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:37:57 smithi114 ceph-mon[34140]: from='client.14991 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:37:57.500 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:37:57 smithi110 ceph-mon[34000]: pgmap v314: 129 pgs: 3 peering, 126 active+clean; 590 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 3.7 KiB/s wr, 14 op/s; 13 B/s, 1 objects/s recovering 2024-09-21T03:37:57.500 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:37:57 smithi110 ceph-mon[34000]: from='client.14991 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:37:57.648 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:57.645+0000 7fd9f7ff0640 1 -- 172.21.15.89:0/1741155059 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd9f0108740 msgr2=0x7fd9f0108b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:57.648 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:57.645+0000 7fd9f7ff0640 1 --2- 172.21.15.89:0/1741155059 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd9f0108740 0x7fd9f0108b50 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fd9ec009900 tx=0x7fd9ec02a410 comp rx=0 tx=0).stop 2024-09-21T03:37:57.648 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:57.645+0000 7fd9f7ff0640 1 -- 172.21.15.89:0/1741155059 shutdown_connections 2024-09-21T03:37:57.648 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:57.645+0000 7fd9f7ff0640 1 --2- 172.21.15.89:0/1741155059 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd9f0101e30 0x7fd9f0102240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:57.648 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:57.645+0000 7fd9f7ff0640 1 --2- 172.21.15.89:0/1741155059 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd9f0102780 0x7fd9f01105b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:57.648 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:57.645+0000 7fd9f7ff0640 1 --2- 172.21.15.89:0/1741155059 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd9f0108740 0x7fd9f0108b50 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:57.648 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:57.645+0000 7fd9f7ff0640 1 -- 172.21.15.89:0/1741155059 >> 172.21.15.89:0/1741155059 conn(0x7fd9f0079f90 msgr2=0x7fd9f007a3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:37:57.648 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:57.645+0000 7fd9f7ff0640 1 -- 172.21.15.89:0/1741155059 shutdown_connections 2024-09-21T03:37:57.648 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:57.645+0000 7fd9f7ff0640 1 -- 172.21.15.89:0/1741155059 wait complete. 2024-09-21T03:37:57.649 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:57.646+0000 7fd9f7ff0640 1 Processor -- start 2024-09-21T03:37:57.649 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:57.646+0000 7fd9f7ff0640 1 -- start start 2024-09-21T03:37:57.650 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:57.646+0000 7fd9f7ff0640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd9f0101e30 0x7fd9f019a450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:57.650 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:57.646+0000 7fd9f7ff0640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd9f0102780 0x7fd9f019a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:57.651 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:57.646+0000 7fd9f7ff0640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd9f0108740 0x7fd9f019fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:57.651 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:57.646+0000 7fd9f7ff0640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fd9f01139a0 con 0x7fd9f0102780 2024-09-21T03:37:57.651 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:57.646+0000 7fd9f7ff0640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fd9f0113820 con 0x7fd9f0101e30 2024-09-21T03:37:57.651 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:57.646+0000 7fd9f7ff0640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fd9f0113b20 con 0x7fd9f0108740 2024-09-21T03:37:57.651 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:57.646+0000 7fd9f558b640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd9f0102780 0x7fd9f019a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:57.651 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:57.646+0000 7fd9f558b640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd9f0102780 0x7fd9f019a9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:38562/0 (socket says 172.21.15.89:38562) 2024-09-21T03:37:57.651 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:57.646+0000 7fd9f558b640 1 -- 172.21.15.89:0/2643361643 learned_addr learned my addr 172.21.15.89:0/2643361643 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-21T03:37:57.651 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:57.646+0000 7fd9f5d8c640 1 --2- 172.21.15.89:0/2643361643 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd9f0101e30 0x7fd9f019a450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:57.651 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:57.647+0000 7fd9f658d640 1 --2- 172.21.15.89:0/2643361643 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd9f0108740 0x7fd9f019fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:57.651 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:57.647+0000 7fd9f558b640 1 -- 172.21.15.89:0/2643361643 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd9f0108740 msgr2=0x7fd9f019fc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:57.651 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:57.647+0000 7fd9f558b640 1 --2- 172.21.15.89:0/2643361643 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd9f0108740 0x7fd9f019fc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:57.651 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:57.647+0000 7fd9f558b640 1 -- 172.21.15.89:0/2643361643 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd9f0101e30 msgr2=0x7fd9f019a450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:57.651 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:57.647+0000 7fd9f558b640 1 --2- 172.21.15.89:0/2643361643 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd9f0101e30 0x7fd9f019a450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:57.652 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:57.647+0000 7fd9f558b640 1 -- 172.21.15.89:0/2643361643 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9f01a03c0 con 0x7fd9f0102780 2024-09-21T03:37:57.652 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:57.647+0000 7fd9f658d640 1 --2- 172.21.15.89:0/2643361643 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd9f0108740 0x7fd9f019fc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:37:57.652 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:57.647+0000 7fd9f558b640 1 --2- 172.21.15.89:0/2643361643 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd9f0102780 0x7fd9f019a9f0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fd9e000d740 tx=0x7fd9e000dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:37:57.652 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:57.647+0000 7fd9e6ffd640 1 -- 172.21.15.89:0/2643361643 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9e0017070 con 0x7fd9f0102780 2024-09-21T03:37:57.652 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:57.647+0000 7fd9e6ffd640 1 -- 172.21.15.89:0/2643361643 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd9e000de50 con 0x7fd9f0102780 2024-09-21T03:37:57.652 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:57.647+0000 7fd9f7ff0640 1 -- 172.21.15.89:0/2643361643 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd9f01a06b0 con 0x7fd9f0102780 2024-09-21T03:37:57.652 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:57.647+0000 7fd9e6ffd640 1 -- 172.21.15.89:0/2643361643 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9e0003bb0 con 0x7fd9f0102780 2024-09-21T03:37:57.652 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:57.647+0000 7fd9f7ff0640 1 -- 172.21.15.89:0/2643361643 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd9f01a6e10 con 0x7fd9f0102780 2024-09-21T03:37:57.652 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:57.648+0000 7fd9f7ff0640 1 -- 172.21.15.89:0/2643361643 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd9f0102240 con 0x7fd9f0102780 2024-09-21T03:37:57.653 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:57.650+0000 7fd9e6ffd640 1 -- 172.21.15.89:0/2643361643 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 18) ==== 154563+0+0 (secure 0 0 0) 0x7fd9e0003d50 con 0x7fd9f0102780 2024-09-21T03:37:57.653 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:57.650+0000 7fd9e6ffd640 1 --2- 172.21.15.89:0/2643361643 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7fd9dc0b65a0 0x7fd9dc0b8ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:57.653 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:57.651+0000 7fd9f5d8c640 1 --2- 172.21.15.89:0/2643361643 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7fd9dc0b65a0 0x7fd9dc0b8ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:57.653 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:57.651+0000 7fd9e6ffd640 1 -- 172.21.15.89:0/2643361643 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 9092+0+0 (secure 0 0 0) 0x7fd9e00e75c0 con 0x7fd9f0102780 2024-09-21T03:37:57.653 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:57.651+0000 7fd9f5d8c640 1 --2- 172.21.15.89:0/2643361643 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7fd9dc0b65a0 0x7fd9dc0b8ab0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fd9ec0057c0 tx=0x7fd9ec0058d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:37:57.656 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:57.654+0000 7fd9e6ffd640 1 -- 172.21.15.89:0/2643361643 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fd9e00ad990 con 0x7fd9f0102780 2024-09-21T03:37:57.864 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:57.862+0000 7fd9f7ff0640 1 -- 172.21.15.89:0/2643361643 --> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7fd9f010bc20 con 0x7fd9dc0b65a0 2024-09-21T03:37:57.892 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:57.890+0000 7fd9e6ffd640 1 -- 172.21.15.89:0/2643361643 <== mgr.14152 v2:172.21.15.89:6800/1591532358 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fd9f010bc20 con 0x7fd9dc0b65a0 2024-09-21T03:37:57.895 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:57.892+0000 7fd9f7ff0640 1 -- 172.21.15.89:0/2643361643 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7fd9dc0b65a0 msgr2=0x7fd9dc0b8ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:57.895 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:57.892+0000 7fd9f7ff0640 1 --2- 172.21.15.89:0/2643361643 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7fd9dc0b65a0 0x7fd9dc0b8ab0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fd9ec0057c0 tx=0x7fd9ec0058d0 comp rx=0 tx=0).stop 2024-09-21T03:37:57.895 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:57.892+0000 7fd9f7ff0640 1 -- 172.21.15.89:0/2643361643 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd9f0102780 msgr2=0x7fd9f019a9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:57.895 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:57.892+0000 7fd9f7ff0640 1 --2- 172.21.15.89:0/2643361643 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd9f0102780 0x7fd9f019a9f0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fd9e000d740 tx=0x7fd9e000dc00 comp rx=0 tx=0).stop 2024-09-21T03:37:57.895 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:57.892+0000 7fd9f7ff0640 1 -- 172.21.15.89:0/2643361643 shutdown_connections 2024-09-21T03:37:57.895 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:57.892+0000 7fd9f7ff0640 1 --2- 172.21.15.89:0/2643361643 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7fd9dc0b65a0 0x7fd9dc0b8ab0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:57.895 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:57.892+0000 7fd9f7ff0640 1 --2- 172.21.15.89:0/2643361643 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd9f0108740 0x7fd9f019fc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:57.895 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:57.892+0000 7fd9f7ff0640 1 --2- 172.21.15.89:0/2643361643 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd9f0102780 0x7fd9f019a9f0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:57.895 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:57.892+0000 7fd9f7ff0640 1 --2- 172.21.15.89:0/2643361643 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd9f0101e30 0x7fd9f019a450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:57.895 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:57.892+0000 7fd9f7ff0640 1 -- 172.21.15.89:0/2643361643 >> 172.21.15.89:0/2643361643 conn(0x7fd9f0079f90 msgr2=0x7fd9f0107610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:37:57.895 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:57.892+0000 7fd9f7ff0640 1 -- 172.21.15.89:0/2643361643 shutdown_connections 2024-09-21T03:37:57.896 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:57.893+0000 7fd9f7ff0640 1 -- 172.21.15.89:0/2643361643 wait complete. 2024-09-21T03:37:58.589 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 7c374098-77c9-11ef-baf5-efdc52797490 -- ceph fs subvolume getpath cephfs --group_name qa sv_1 2024-09-21T03:37:58.838 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/7c374098-77c9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:37:59.262 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:37:59 smithi089 ceph-mon[30928]: pgmap v315: 129 pgs: 3 peering, 126 active+clean; 591 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 2.9 KiB/s wr, 9 op/s; 13 B/s, 1 objects/s recovering 2024-09-21T03:37:59.263 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:37:59 smithi089 ceph-mon[30928]: from='client.14997 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:37:59.364 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:59.356+0000 7f1ed02fd640 1 -- 172.21.15.89:0/212065017 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1ec8102a60 msgr2=0x7f1ec8102e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:59.364 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:59.356+0000 7f1ed02fd640 1 --2- 172.21.15.89:0/212065017 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1ec8102a60 0x7f1ec8102e70 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f1eb8009ca0 tx=0x7f1eb802a8c0 comp rx=0 tx=0).stop 2024-09-21T03:37:59.364 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:59.356+0000 7f1ed02fd640 1 -- 172.21.15.89:0/212065017 shutdown_connections 2024-09-21T03:37:59.364 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:59.356+0000 7f1ed02fd640 1 --2- 172.21.15.89:0/212065017 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1ec81083a0 0x7f1ec81087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:59.364 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:59.356+0000 7f1ed02fd640 1 --2- 172.21.15.89:0/212065017 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1ec8108cf0 0x7f1ec810f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:59.364 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:59.356+0000 7f1ed02fd640 1 --2- 172.21.15.89:0/212065017 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1ec8102a60 0x7f1ec8102e70 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:59.364 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:59.356+0000 7f1ed02fd640 1 -- 172.21.15.89:0/212065017 >> 172.21.15.89:0/212065017 conn(0x7f1ec8079730 msgr2=0x7f1ec8079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:37:59.364 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:59.356+0000 7f1ed02fd640 1 -- 172.21.15.89:0/212065017 shutdown_connections 2024-09-21T03:37:59.364 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:59.356+0000 7f1ed02fd640 1 -- 172.21.15.89:0/212065017 wait complete. 2024-09-21T03:37:59.364 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:59.357+0000 7f1ed02fd640 1 Processor -- start 2024-09-21T03:37:59.364 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:59.357+0000 7f1ed02fd640 1 -- start start 2024-09-21T03:37:59.364 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:59.357+0000 7f1ed02fd640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1ec8102a60 0x7f1ec819a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:59.365 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:59.357+0000 7f1ed02fd640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1ec81083a0 0x7f1ec819aa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:59.365 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:59.357+0000 7f1ed02fd640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1ec8108cf0 0x7f1ec819fca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:59.365 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:59.357+0000 7f1ed02fd640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f1ec8112860 con 0x7f1ec81083a0 2024-09-21T03:37:59.365 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:59.357+0000 7f1ed02fd640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f1ec81126e0 con 0x7f1ec8108cf0 2024-09-21T03:37:59.365 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:59.357+0000 7f1ed02fd640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f1ec81129e0 con 0x7f1ec8102a60 2024-09-21T03:37:59.365 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:59.357+0000 7f1ecd898640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1ec81083a0 0x7f1ec819aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:59.365 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:59.357+0000 7f1ecd898640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1ec81083a0 0x7f1ec819aa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:50748/0 (socket says 172.21.15.89:50748) 2024-09-21T03:37:59.365 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:59.357+0000 7f1ecd898640 1 -- 172.21.15.89:0/2116065006 learned_addr learned my addr 172.21.15.89:0/2116065006 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-21T03:37:59.365 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:59.357+0000 7f1ece099640 1 --2- 172.21.15.89:0/2116065006 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1ec8102a60 0x7f1ec819a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:59.365 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:59.357+0000 7f1ece89a640 1 --2- 172.21.15.89:0/2116065006 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1ec8108cf0 0x7f1ec819fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:59.365 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:59.357+0000 7f1ecd898640 1 -- 172.21.15.89:0/2116065006 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1ec8102a60 msgr2=0x7f1ec819a460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:59.365 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:59.357+0000 7f1ecd898640 1 --2- 172.21.15.89:0/2116065006 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1ec8102a60 0x7f1ec819a460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:59.365 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:59.357+0000 7f1ecd898640 1 -- 172.21.15.89:0/2116065006 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1ec8108cf0 msgr2=0x7f1ec819fca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:59.365 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:59.357+0000 7f1ecd898640 1 --2- 172.21.15.89:0/2116065006 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1ec8108cf0 0x7f1ec819fca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:59.365 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:59.357+0000 7f1ecd898640 1 -- 172.21.15.89:0/2116065006 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ec81a03d0 con 0x7f1ec81083a0 2024-09-21T03:37:59.365 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:59.357+0000 7f1ece099640 1 --2- 172.21.15.89:0/2116065006 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1ec8102a60 0x7f1ec819a460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:37:59.366 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:59.357+0000 7f1ece89a640 1 --2- 172.21.15.89:0/2116065006 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1ec8108cf0 0x7f1ec819fca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:37:59.366 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:59.358+0000 7f1ecd898640 1 --2- 172.21.15.89:0/2116065006 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1ec81083a0 0x7f1ec819aa00 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f1ebc005fd0 tx=0x7f1ebc00dde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:37:59.366 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:59.358+0000 7f1ec77fe640 1 -- 172.21.15.89:0/2116065006 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ebc017070 con 0x7f1ec81083a0 2024-09-21T03:37:59.366 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:59.358+0000 7f1ec77fe640 1 -- 172.21.15.89:0/2116065006 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1ebc009c30 con 0x7f1ec81083a0 2024-09-21T03:37:59.366 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:59.358+0000 7f1ed02fd640 1 -- 172.21.15.89:0/2116065006 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ec81a06c0 con 0x7f1ec81083a0 2024-09-21T03:37:59.366 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:59.358+0000 7f1ec77fe640 1 -- 172.21.15.89:0/2116065006 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ebc003c70 con 0x7f1ec81083a0 2024-09-21T03:37:59.366 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:59.358+0000 7f1ed02fd640 1 -- 172.21.15.89:0/2116065006 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ec81a6e20 con 0x7f1ec81083a0 2024-09-21T03:37:59.366 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:59.359+0000 7f1ed02fd640 1 -- 172.21.15.89:0/2116065006 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1ec8102e70 con 0x7f1ec81083a0 2024-09-21T03:37:59.366 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:59.360+0000 7f1ec77fe640 1 -- 172.21.15.89:0/2116065006 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 18) ==== 154563+0+0 (secure 0 0 0) 0x7f1ebc003e10 con 0x7f1ec81083a0 2024-09-21T03:37:59.366 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:59.361+0000 7f1ec77fe640 1 --2- 172.21.15.89:0/2116065006 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f1eb00b6620 0x7f1eb00b8b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:59.366 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:59.361+0000 7f1ece099640 1 --2- 172.21.15.89:0/2116065006 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f1eb00b6620 0x7f1eb00b8b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:59.366 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:59.361+0000 7f1ec77fe640 1 -- 172.21.15.89:0/2116065006 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 9092+0+0 (secure 0 0 0) 0x7f1ebc0e74a0 con 0x7f1ec81083a0 2024-09-21T03:37:59.366 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:59.362+0000 7f1ece099640 1 --2- 172.21.15.89:0/2116065006 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f1eb00b6620 0x7f1eb00b8b30 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f1eb8000f80 tx=0x7f1eb8009410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:37:59.368 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:59.366+0000 7f1ec77fe640 1 -- 172.21.15.89:0/2116065006 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f1ebc020030 con 0x7f1ec81083a0 2024-09-21T03:37:59.461 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:37:59 smithi114 ceph-mon[34140]: pgmap v315: 129 pgs: 3 peering, 126 active+clean; 591 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 2.9 KiB/s wr, 9 op/s; 13 B/s, 1 objects/s recovering 2024-09-21T03:37:59.461 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:37:59 smithi114 ceph-mon[34140]: from='client.14997 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:37:59.500 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:37:59 smithi110 ceph-mon[34000]: pgmap v315: 129 pgs: 3 peering, 126 active+clean; 591 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 2.9 KiB/s wr, 9 op/s; 13 B/s, 1 objects/s recovering 2024-09-21T03:37:59.500 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:37:59 smithi110 ceph-mon[34000]: from='client.14997 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:37:59.574 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:59.572+0000 7f1ed02fd640 1 -- 172.21.15.89:0/2116065006 --> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}) -- 0x7f1ec8106340 con 0x7f1eb00b6620 2024-09-21T03:37:59.578 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:59.576+0000 7f1ec77fe640 1 -- 172.21.15.89:0/2116065006 <== mgr.14152 v2:172.21.15.89:6800/1591532358 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7f1ec8106340 con 0x7f1eb00b6620 2024-09-21T03:37:59.578 INFO:teuthology.orchestra.run.smithi089.stdout:/volumes/qa/sv_1/fc2be6f8-9566-4456-8b95-56413e5916bc 2024-09-21T03:37:59.581 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:59.578+0000 7f1ed02fd640 1 -- 172.21.15.89:0/2116065006 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f1eb00b6620 msgr2=0x7f1eb00b8b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:59.581 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:59.578+0000 7f1ed02fd640 1 --2- 172.21.15.89:0/2116065006 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f1eb00b6620 0x7f1eb00b8b30 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f1eb8000f80 tx=0x7f1eb8009410 comp rx=0 tx=0).stop 2024-09-21T03:37:59.581 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:59.578+0000 7f1ed02fd640 1 -- 172.21.15.89:0/2116065006 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1ec81083a0 msgr2=0x7f1ec819aa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:59.581 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:59.578+0000 7f1ed02fd640 1 --2- 172.21.15.89:0/2116065006 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1ec81083a0 0x7f1ec819aa00 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f1ebc005fd0 tx=0x7f1ebc00dde0 comp rx=0 tx=0).stop 2024-09-21T03:37:59.581 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:59.578+0000 7f1ed02fd640 1 -- 172.21.15.89:0/2116065006 shutdown_connections 2024-09-21T03:37:59.581 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:59.578+0000 7f1ed02fd640 1 --2- 172.21.15.89:0/2116065006 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f1eb00b6620 0x7f1eb00b8b30 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:59.581 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:59.578+0000 7f1ed02fd640 1 --2- 172.21.15.89:0/2116065006 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1ec8108cf0 0x7f1ec819fca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:59.581 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:59.578+0000 7f1ed02fd640 1 --2- 172.21.15.89:0/2116065006 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1ec81083a0 0x7f1ec819aa00 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:59.581 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:59.578+0000 7f1ed02fd640 1 --2- 172.21.15.89:0/2116065006 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1ec8102a60 0x7f1ec819a460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:59.581 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:59.578+0000 7f1ed02fd640 1 -- 172.21.15.89:0/2116065006 >> 172.21.15.89:0/2116065006 conn(0x7f1ec8079730 msgr2=0x7f1ec8101bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:37:59.581 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:59.579+0000 7f1ed02fd640 1 -- 172.21.15.89:0/2116065006 shutdown_connections 2024-09-21T03:37:59.581 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:37:59.579+0000 7f1ed02fd640 1 -- 172.21.15.89:0/2116065006 wait complete. 2024-09-21T03:38:00.212 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 7c374098-77c9-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T03:38:00.311 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:38:00 smithi089 ceph-mon[30928]: from='client.15003 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:38:00.456 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/7c374098-77c9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:38:00.460 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:38:00 smithi114 ceph-mon[34140]: from='client.15003 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:38:00.500 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:38:00 smithi110 ceph-mon[34000]: from='client.15003 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:38:00.932 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:00.929+0000 7fb8ae7f8640 1 -- 172.21.15.89:0/3380754090 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb8a8109df0 msgr2=0x7fb8a8110620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:00.932 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:00.929+0000 7fb8ae7f8640 1 --2- 172.21.15.89:0/3380754090 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb8a8109df0 0x7fb8a8110620 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fb8a0009ca0 tx=0x7fb8a002a8c0 comp rx=0 tx=0).stop 2024-09-21T03:38:00.932 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:00.929+0000 7fb8ae7f8640 1 -- 172.21.15.89:0/3380754090 shutdown_connections 2024-09-21T03:38:00.932 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:00.929+0000 7fb8ae7f8640 1 --2- 172.21.15.89:0/3380754090 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb8a8109df0 0x7fb8a8110620 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:00.932 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:00.929+0000 7fb8ae7f8640 1 --2- 172.21.15.89:0/3380754090 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb8a81094a0 0x7fb8a81098b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:00.932 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:00.929+0000 7fb8ae7f8640 1 --2- 172.21.15.89:0/3380754090 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb8a8103b60 0x7fb8a8103f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:00.932 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:00.929+0000 7fb8ae7f8640 1 -- 172.21.15.89:0/3380754090 >> 172.21.15.89:0/3380754090 conn(0x7fb8a807b850 msgr2=0x7fb8a807bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:38:00.932 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:00.929+0000 7fb8ae7f8640 1 -- 172.21.15.89:0/3380754090 shutdown_connections 2024-09-21T03:38:00.932 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:00.929+0000 7fb8ae7f8640 1 -- 172.21.15.89:0/3380754090 wait complete. 2024-09-21T03:38:00.934 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:00.930+0000 7fb8ae7f8640 1 Processor -- start 2024-09-21T03:38:00.934 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:00.930+0000 7fb8ae7f8640 1 -- start start 2024-09-21T03:38:00.934 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:00.930+0000 7fb8ae7f8640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb8a8103b60 0x7fb8a819a3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:00.934 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:00.930+0000 7fb8ae7f8640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb8a81094a0 0x7fb8a819a940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:00.935 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:00.930+0000 7fb8ae7f8640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb8a8109df0 0x7fb8a819fbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:00.935 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:00.930+0000 7fb8ae7f8640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fb8a81139b0 con 0x7fb8a81094a0 2024-09-21T03:38:00.935 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:00.930+0000 7fb8ae7f8640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fb8a8113830 con 0x7fb8a8103b60 2024-09-21T03:38:00.935 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:00.930+0000 7fb8ae7f8640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fb8a8113b30 con 0x7fb8a8109df0 2024-09-21T03:38:00.935 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:00.931+0000 7fb8a77fe640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb8a81094a0 0x7fb8a819a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:00.935 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:00.931+0000 7fb8a77fe640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb8a81094a0 0x7fb8a819a940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:50762/0 (socket says 172.21.15.89:50762) 2024-09-21T03:38:00.935 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:00.931+0000 7fb8a77fe640 1 -- 172.21.15.89:0/3937749796 learned_addr learned my addr 172.21.15.89:0/3937749796 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-21T03:38:00.935 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:00.931+0000 7fb8a7fff640 1 --2- 172.21.15.89:0/3937749796 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb8a8103b60 0x7fb8a819a3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:00.935 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:00.931+0000 7fb8acd95640 1 --2- 172.21.15.89:0/3937749796 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb8a8109df0 0x7fb8a819fbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:00.935 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:00.931+0000 7fb8a77fe640 1 -- 172.21.15.89:0/3937749796 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb8a8109df0 msgr2=0x7fb8a819fbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:00.935 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:00.931+0000 7fb8a77fe640 1 --2- 172.21.15.89:0/3937749796 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb8a8109df0 0x7fb8a819fbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:00.935 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:00.931+0000 7fb8a77fe640 1 -- 172.21.15.89:0/3937749796 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb8a8103b60 msgr2=0x7fb8a819a3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:00.935 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:00.931+0000 7fb8a77fe640 1 --2- 172.21.15.89:0/3937749796 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb8a8103b60 0x7fb8a819a3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:00.936 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:00.931+0000 7fb8a77fe640 1 -- 172.21.15.89:0/3937749796 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8a81a0250 con 0x7fb8a81094a0 2024-09-21T03:38:00.936 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:00.931+0000 7fb8a7fff640 1 --2- 172.21.15.89:0/3937749796 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb8a8103b60 0x7fb8a819a3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:38:00.936 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:00.931+0000 7fb8a77fe640 1 --2- 172.21.15.89:0/3937749796 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb8a81094a0 0x7fb8a819a940 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fb89000c8c0 tx=0x7fb89000cd80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:38:00.936 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:00.931+0000 7fb8a57fa640 1 -- 172.21.15.89:0/3937749796 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb890007bb0 con 0x7fb8a81094a0 2024-09-21T03:38:00.936 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:00.931+0000 7fb8ae7f8640 1 -- 172.21.15.89:0/3937749796 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb8a81a0540 con 0x7fb8a81094a0 2024-09-21T03:38:00.936 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:00.931+0000 7fb8a57fa640 1 -- 172.21.15.89:0/3937749796 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb890007d50 con 0x7fb8a81094a0 2024-09-21T03:38:00.936 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:00.931+0000 7fb8a57fa640 1 -- 172.21.15.89:0/3937749796 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb890018660 con 0x7fb8a81094a0 2024-09-21T03:38:00.936 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:00.931+0000 7fb8ae7f8640 1 -- 172.21.15.89:0/3937749796 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8a81a6d60 con 0x7fb8a81094a0 2024-09-21T03:38:00.936 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:00.933+0000 7fb8ae7f8640 1 -- 172.21.15.89:0/3937749796 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb8a8103f70 con 0x7fb8a81094a0 2024-09-21T03:38:00.937 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:00.934+0000 7fb8a57fa640 1 -- 172.21.15.89:0/3937749796 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 18) ==== 154563+0+0 (secure 0 0 0) 0x7fb89000f450 con 0x7fb8a81094a0 2024-09-21T03:38:00.937 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:00.935+0000 7fb8a57fa640 1 --2- 172.21.15.89:0/3937749796 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7fb8940b65d0 0x7fb8940b8ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:00.937 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:00.935+0000 7fb8a7fff640 1 --2- 172.21.15.89:0/3937749796 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7fb8940b65d0 0x7fb8940b8ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:00.937 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:00.935+0000 7fb8a57fa640 1 -- 172.21.15.89:0/3937749796 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 9092+0+0 (secure 0 0 0) 0x7fb8900e77e0 con 0x7fb8a81094a0 2024-09-21T03:38:00.938 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:00.935+0000 7fb8a7fff640 1 --2- 172.21.15.89:0/3937749796 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7fb8940b65d0 0x7fb8940b8ae0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fb898009fc0 tx=0x7fb898005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:38:00.941 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:00.939+0000 7fb8a57fa640 1 -- 172.21.15.89:0/3937749796 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fb890014030 con 0x7fb8a81094a0 2024-09-21T03:38:01.158 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:01.155+0000 7fb8ae7f8640 1 -- 172.21.15.89:0/3937749796 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb8a810bc10 con 0x7fb8a81094a0 2024-09-21T03:38:01.158 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:38:01 smithi089 ceph-mon[30928]: pgmap v316: 129 pgs: 129 active+clean; 633 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 9.0 KiB/s wr, 14 op/s; 86 B/s, 1 objects/s recovering 2024-09-21T03:38:01.158 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:01.156+0000 7fb8a57fa640 1 -- 172.21.15.89:0/3937749796 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11912 (secure 0 0 0) 0x7fb8900adbb0 con 0x7fb8a81094a0 2024-09-21T03:38:01.159 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-09-21T03:38:01.159 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":25,"btime":"2024-09-21T03:37:51:019398+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14721,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6835/2341538514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":2341538514},{"type":"v1","addr":"172.21.15.89:6835","nonce":2341538514}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14733,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6837/146708592","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6836","nonce":146708592},{"type":"v1","addr":"172.21.15.89:6837","nonce":146708592}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14781,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6839/2269965499","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6838","nonce":2269965499},{"type":"v1","addr":"172.21.15.89:6839","nonce":2269965499}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14793,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6841/650850383","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6840","nonce":650850383},{"type":"v1","addr":"172.21.15.89:6841","nonce":650850383}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24565,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6833/752336912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6832","nonce":752336912},{"type":"v1","addr":"172.21.15.114:6833","nonce":752336912}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24575,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6833/127607331","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6832","nonce":127607331},{"type":"v1","addr":"172.21.15.110:6833","nonce":127607331}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T03:37:13.570694+0000","modified":"2024-09-21T03:37:50.719106+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24638,"mds_1":24631,"mds_2":24614},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24593":{"gid":24593,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.110:6835/671820654","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6834","nonce":671820654},{"type":"v1","addr":"172.21.15.110:6835","nonce":671820654}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24604":{"gid":24604,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.114:6835/1049714548","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6834","nonce":1049714548},{"type":"v1","addr":"172.21.15.114:6835","nonce":1049714548}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24610":{"gid":24610,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.114:6837/1941213818","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6836","nonce":1941213818},{"type":"v1","addr":"172.21.15.114:6837","nonce":1941213818}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24614":{"gid":24614,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":15,"addr":"172.21.15.110:6837/4047124589","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6836","nonce":4047124589},{"type":"v1","addr":"172.21.15.110:6837","nonce":4047124589}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24631":{"gid":24631,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":15,"addr":"172.21.15.114:6839/3427369944","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6838","nonce":3427369944},{"type":"v1","addr":"172.21.15.114:6839","nonce":3427369944}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24638":{"gid":24638,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.110:6839/132366709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6838","nonce":132366709},{"type":"v1","addr":"172.21.15.110:6839","nonce":132366709}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24638,"qdb_cluster":[24614,24631,24638]},"id":1}]} 2024-09-21T03:38:01.159 INFO:teuthology.orchestra.run.smithi089.stderr:dumped fsmap epoch 25 2024-09-21T03:38:01.161 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:01.159+0000 7fb8ae7f8640 1 -- 172.21.15.89:0/3937749796 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7fb8940b65d0 msgr2=0x7fb8940b8ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:01.161 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:01.159+0000 7fb8ae7f8640 1 --2- 172.21.15.89:0/3937749796 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7fb8940b65d0 0x7fb8940b8ae0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fb898009fc0 tx=0x7fb898005f90 comp rx=0 tx=0).stop 2024-09-21T03:38:01.161 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:01.159+0000 7fb8ae7f8640 1 -- 172.21.15.89:0/3937749796 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb8a81094a0 msgr2=0x7fb8a819a940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:01.161 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:01.159+0000 7fb8ae7f8640 1 --2- 172.21.15.89:0/3937749796 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb8a81094a0 0x7fb8a819a940 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fb89000c8c0 tx=0x7fb89000cd80 comp rx=0 tx=0).stop 2024-09-21T03:38:01.161 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:01.159+0000 7fb8ae7f8640 1 -- 172.21.15.89:0/3937749796 shutdown_connections 2024-09-21T03:38:01.161 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:01.159+0000 7fb8ae7f8640 1 --2- 172.21.15.89:0/3937749796 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7fb8940b65d0 0x7fb8940b8ae0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:01.161 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:01.159+0000 7fb8ae7f8640 1 --2- 172.21.15.89:0/3937749796 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb8a8109df0 0x7fb8a819fbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:01.161 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:01.159+0000 7fb8ae7f8640 1 --2- 172.21.15.89:0/3937749796 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb8a81094a0 0x7fb8a819a940 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:01.161 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:01.159+0000 7fb8ae7f8640 1 --2- 172.21.15.89:0/3937749796 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb8a8103b60 0x7fb8a819a3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:01.161 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:01.159+0000 7fb8ae7f8640 1 -- 172.21.15.89:0/3937749796 >> 172.21.15.89:0/3937749796 conn(0x7fb8a807b850 msgr2=0x7fb8a81054b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:38:01.162 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:01.159+0000 7fb8ae7f8640 1 -- 172.21.15.89:0/3937749796 shutdown_connections 2024-09-21T03:38:01.162 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:01.159+0000 7fb8ae7f8640 1 -- 172.21.15.89:0/3937749796 wait complete. 2024-09-21T03:38:01.460 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:38:01 smithi114 ceph-mon[34140]: pgmap v316: 129 pgs: 129 active+clean; 633 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 9.0 KiB/s wr, 14 op/s; 86 B/s, 1 objects/s recovering 2024-09-21T03:38:01.500 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:38:01 smithi110 ceph-mon[34000]: pgmap v316: 129 pgs: 129 active+clean; 633 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 9.0 KiB/s wr, 14 op/s; 86 B/s, 1 objects/s recovering 2024-09-21T03:38:01.902 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 7c374098-77c9-11ef-baf5-efdc52797490 -- ceph osd dump --format=json 2024-09-21T03:38:02.149 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/7c374098-77c9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:38:02.363 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:38:02 smithi089 ceph-mon[30928]: from='client.? 172.21.15.89:0/3937749796' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:38:02.460 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:38:02 smithi114 ceph-mon[34140]: from='client.? 172.21.15.89:0/3937749796' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:38:02.500 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:38:02 smithi110 ceph-mon[34000]: from='client.? 172.21.15.89:0/3937749796' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:38:02.633 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:02.605+0000 7f50b1195640 1 -- 172.21.15.89:0/3347599278 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f50ac108cf0 msgr2=0x7f50ac10f520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:02.633 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:02.605+0000 7f50b1195640 1 --2- 172.21.15.89:0/3347599278 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f50ac108cf0 0x7f50ac10f520 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f50a4009ca0 tx=0x7f50a402a8c0 comp rx=0 tx=0).stop 2024-09-21T03:38:02.633 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:02.605+0000 7f50b1195640 1 -- 172.21.15.89:0/3347599278 shutdown_connections 2024-09-21T03:38:02.633 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:02.605+0000 7f50b1195640 1 --2- 172.21.15.89:0/3347599278 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f50ac108cf0 0x7f50ac10f520 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:02.633 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:02.605+0000 7f50b1195640 1 --2- 172.21.15.89:0/3347599278 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f50ac1083a0 0x7f50ac1087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:02.633 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:02.605+0000 7f50b1195640 1 --2- 172.21.15.89:0/3347599278 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f50ac102a60 0x7f50ac102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:02.633 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:02.605+0000 7f50b1195640 1 -- 172.21.15.89:0/3347599278 >> 172.21.15.89:0/3347599278 conn(0x7f50ac079730 msgr2=0x7f50ac079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:38:02.633 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:02.605+0000 7f50b1195640 1 -- 172.21.15.89:0/3347599278 shutdown_connections 2024-09-21T03:38:02.633 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:02.605+0000 7f50b1195640 1 -- 172.21.15.89:0/3347599278 wait complete. 2024-09-21T03:38:02.633 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:02.606+0000 7f50b1195640 1 Processor -- start 2024-09-21T03:38:02.633 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:02.606+0000 7f50b1195640 1 -- start start 2024-09-21T03:38:02.633 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:02.606+0000 7f50b1195640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f50ac102a60 0x7f50ac199250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:02.634 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:02.606+0000 7f50b1195640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f50ac1083a0 0x7f50ac1997f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:02.634 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:02.606+0000 7f50b1195640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f50ac108cf0 0x7f50ac19ea90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:02.634 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:02.606+0000 7f50b1195640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f50ac112860 con 0x7f50ac108cf0 2024-09-21T03:38:02.634 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:02.606+0000 7f50b1195640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f50ac1126e0 con 0x7f50ac1083a0 2024-09-21T03:38:02.634 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:02.606+0000 7f50b1195640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f50ac1129e0 con 0x7f50ac102a60 2024-09-21T03:38:02.634 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:02.606+0000 7f50ab59e640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f50ac108cf0 0x7f50ac19ea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:02.634 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:02.606+0000 7f50ab59e640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f50ac108cf0 0x7f50ac19ea90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:50778/0 (socket says 172.21.15.89:50778) 2024-09-21T03:38:02.634 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:02.606+0000 7f50ab59e640 1 -- 172.21.15.89:0/3696661315 learned_addr learned my addr 172.21.15.89:0/3696661315 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-21T03:38:02.634 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:02.607+0000 7f50aa59c640 1 --2- 172.21.15.89:0/3696661315 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f50ac1083a0 0x7f50ac1997f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:02.634 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:02.607+0000 7f50aad9d640 1 --2- 172.21.15.89:0/3696661315 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f50ac102a60 0x7f50ac199250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:02.634 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:02.607+0000 7f50ab59e640 1 -- 172.21.15.89:0/3696661315 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f50ac102a60 msgr2=0x7f50ac199250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:02.634 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:02.607+0000 7f50ab59e640 1 --2- 172.21.15.89:0/3696661315 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f50ac102a60 0x7f50ac199250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:02.634 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:02.607+0000 7f50ab59e640 1 -- 172.21.15.89:0/3696661315 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f50ac1083a0 msgr2=0x7f50ac1997f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:02.635 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:02.607+0000 7f50ab59e640 1 --2- 172.21.15.89:0/3696661315 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f50ac1083a0 0x7f50ac1997f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:02.635 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:02.607+0000 7f50ab59e640 1 -- 172.21.15.89:0/3696661315 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50ac19f1c0 con 0x7f50ac108cf0 2024-09-21T03:38:02.635 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:02.607+0000 7f50aad9d640 1 --2- 172.21.15.89:0/3696661315 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f50ac102a60 0x7f50ac199250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:38:02.635 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:02.607+0000 7f50ab59e640 1 --2- 172.21.15.89:0/3696661315 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f50ac108cf0 0x7f50ac19ea90 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f50a4005cc0 tx=0x7f50a4032a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:38:02.635 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:02.607+0000 7f50a3fff640 1 -- 172.21.15.89:0/3696661315 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50a4032e00 con 0x7f50ac108cf0 2024-09-21T03:38:02.635 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:02.607+0000 7f50a3fff640 1 -- 172.21.15.89:0/3696661315 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f50a40044d0 con 0x7f50ac108cf0 2024-09-21T03:38:02.635 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:02.607+0000 7f50b1195640 1 -- 172.21.15.89:0/3696661315 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f50ac19f450 con 0x7f50ac108cf0 2024-09-21T03:38:02.635 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:02.607+0000 7f50a3fff640 1 -- 172.21.15.89:0/3696661315 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50a4030b70 con 0x7f50ac108cf0 2024-09-21T03:38:02.635 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:02.607+0000 7f50b1195640 1 -- 172.21.15.89:0/3696661315 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50ac1a5c10 con 0x7f50ac108cf0 2024-09-21T03:38:02.635 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:02.608+0000 7f50b1195640 1 -- 172.21.15.89:0/3696661315 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f50ac102e70 con 0x7f50ac108cf0 2024-09-21T03:38:02.635 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:02.610+0000 7f50a3fff640 1 -- 172.21.15.89:0/3696661315 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 18) ==== 154563+0+0 (secure 0 0 0) 0x7f50a4030d10 con 0x7f50ac108cf0 2024-09-21T03:38:02.635 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:02.610+0000 7f50a3fff640 1 --2- 172.21.15.89:0/3696661315 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f50980b65a0 0x7f50980b8ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:02.635 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:02.610+0000 7f50aad9d640 1 --2- 172.21.15.89:0/3696661315 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f50980b65a0 0x7f50980b8ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:02.636 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:02.610+0000 7f50a3fff640 1 -- 172.21.15.89:0/3696661315 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 9092+0+0 (secure 0 0 0) 0x7f50a4108c60 con 0x7f50ac108cf0 2024-09-21T03:38:02.636 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:02.611+0000 7f50aad9d640 1 --2- 172.21.15.89:0/3696661315 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f50980b65a0 0x7f50980b8ab0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f50940058c0 tx=0x7f5094005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:38:02.636 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:02.613+0000 7f50a3fff640 1 -- 172.21.15.89:0/3696661315 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f50a40cf030 con 0x7f50ac108cf0 2024-09-21T03:38:02.794 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:02.792+0000 7f50b1195640 1 -- 172.21.15.89:0/3696661315 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f50ac19a380 con 0x7f50ac108cf0 2024-09-21T03:38:02.796 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:02.794+0000 7f50a3fff640 1 -- 172.21.15.89:0/3696661315 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) ==== 74+0+24763 (secure 0 0 0) 0x7f50ac19a380 con 0x7f50ac108cf0 2024-09-21T03:38:02.796 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-09-21T03:38:02.796 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":82,"fsid":"7c374098-77c9-11ef-baf5-efdc52797490","created":"2024-09-21T03:28:56.681458+0000","modified":"2024-09-21T03:37:53.033682+0000","last_up_change":"2024-09-21T03:36:23.368379+0000","last_in_change":"2024-09-21T03:36:00.131331+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T03:32:37.834339+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-21T03:37:09.443341+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"74","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-21T03:37:11.467821+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"74","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-21T03:37:17.884066+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"27ca5ed8-17d1-4f8e-a7ae-084a914721a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6802","nonce":990539042},{"type":"v1","addr":"172.21.15.89:6803","nonce":990539042}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6804","nonce":990539042},{"type":"v1","addr":"172.21.15.89:6805","nonce":990539042}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6808","nonce":990539042},{"type":"v1","addr":"172.21.15.89:6809","nonce":990539042}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6806","nonce":990539042},{"type":"v1","addr":"172.21.15.89:6807","nonce":990539042}]},"public_addr":"172.21.15.89:6803/990539042","cluster_addr":"172.21.15.89:6805/990539042","heartbeat_back_addr":"172.21.15.89:6809/990539042","heartbeat_front_addr":"172.21.15.89:6807/990539042","state":["exists","up"]},{"osd":1,"uuid":"5176a910-5cf9-49d7-aedb-a20cf36ea075","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6800","nonce":3235852628},{"type":"v1","addr":"172.21.15.110:6801","nonce":3235852628}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6802","nonce":3235852628},{"type":"v1","addr":"172.21.15.110:6803","nonce":3235852628}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6806","nonce":3235852628},{"type":"v1","addr":"172.21.15.110:6807","nonce":3235852628}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6804","nonce":3235852628},{"type":"v1","addr":"172.21.15.110:6805","nonce":3235852628}]},"public_addr":"172.21.15.110:6801/3235852628","cluster_addr":"172.21.15.110:6803/3235852628","heartbeat_back_addr":"172.21.15.110:6807/3235852628","heartbeat_front_addr":"172.21.15.110:6805/3235852628","state":["exists","up"]},{"osd":2,"uuid":"d4977df1-b118-4d7e-b9a9-626a8accc812","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6800","nonce":169452122},{"type":"v1","addr":"172.21.15.114:6801","nonce":169452122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6802","nonce":169452122},{"type":"v1","addr":"172.21.15.114:6803","nonce":169452122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6806","nonce":169452122},{"type":"v1","addr":"172.21.15.114:6807","nonce":169452122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6804","nonce":169452122},{"type":"v1","addr":"172.21.15.114:6805","nonce":169452122}]},"public_addr":"172.21.15.114:6801/169452122","cluster_addr":"172.21.15.114:6803/169452122","heartbeat_back_addr":"172.21.15.114:6807/169452122","heartbeat_front_addr":"172.21.15.114:6805/169452122","state":["exists","up"]},{"osd":3,"uuid":"2c7ae1b4-0bf2-4954-81ed-55d03eab82c6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6810","nonce":3016980877},{"type":"v1","addr":"172.21.15.89:6811","nonce":3016980877}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6812","nonce":3016980877},{"type":"v1","addr":"172.21.15.89:6813","nonce":3016980877}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6816","nonce":3016980877},{"type":"v1","addr":"172.21.15.89:6817","nonce":3016980877}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6814","nonce":3016980877},{"type":"v1","addr":"172.21.15.89:6815","nonce":3016980877}]},"public_addr":"172.21.15.89:6811/3016980877","cluster_addr":"172.21.15.89:6813/3016980877","heartbeat_back_addr":"172.21.15.89:6817/3016980877","heartbeat_front_addr":"172.21.15.89:6815/3016980877","state":["exists","up"]},{"osd":4,"uuid":"8e65399d-6e55-4da4-98ba-f805f0c655a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":30,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6808","nonce":3217880476},{"type":"v1","addr":"172.21.15.110:6809","nonce":3217880476}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6810","nonce":3217880476},{"type":"v1","addr":"172.21.15.110:6811","nonce":3217880476}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6814","nonce":3217880476},{"type":"v1","addr":"172.21.15.110:6815","nonce":3217880476}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6812","nonce":3217880476},{"type":"v1","addr":"172.21.15.110:6813","nonce":3217880476}]},"public_addr":"172.21.15.110:6809/3217880476","cluster_addr":"172.21.15.110:6811/3217880476","heartbeat_back_addr":"172.21.15.110:6815/3217880476","heartbeat_front_addr":"172.21.15.110:6813/3217880476","state":["exists","up"]},{"osd":5,"uuid":"b9ec2889-46bd-40e5-8446-73ab28271c67","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6808","nonce":1144140707},{"type":"v1","addr":"172.21.15.114:6809","nonce":1144140707}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6810","nonce":1144140707},{"type":"v1","addr":"172.21.15.114:6811","nonce":1144140707}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6814","nonce":1144140707},{"type":"v1","addr":"172.21.15.114:6815","nonce":1144140707}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6812","nonce":1144140707},{"type":"v1","addr":"172.21.15.114:6813","nonce":1144140707}]},"public_addr":"172.21.15.114:6809/1144140707","cluster_addr":"172.21.15.114:6811/1144140707","heartbeat_back_addr":"172.21.15.114:6815/1144140707","heartbeat_front_addr":"172.21.15.114:6813/1144140707","state":["exists","up"]},{"osd":6,"uuid":"9b79989e-adb0-42b2-b2f6-a2dce56b66e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":41,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6818","nonce":1522586255},{"type":"v1","addr":"172.21.15.89:6819","nonce":1522586255}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6820","nonce":1522586255},{"type":"v1","addr":"172.21.15.89:6821","nonce":1522586255}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6824","nonce":1522586255},{"type":"v1","addr":"172.21.15.89:6825","nonce":1522586255}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6822","nonce":1522586255},{"type":"v1","addr":"172.21.15.89:6823","nonce":1522586255}]},"public_addr":"172.21.15.89:6819/1522586255","cluster_addr":"172.21.15.89:6821/1522586255","heartbeat_back_addr":"172.21.15.89:6825/1522586255","heartbeat_front_addr":"172.21.15.89:6823/1522586255","state":["exists","up"]},{"osd":7,"uuid":"b381821f-141c-47a1-be65-f57ca730391b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6816","nonce":1331818323},{"type":"v1","addr":"172.21.15.110:6817","nonce":1331818323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6818","nonce":1331818323},{"type":"v1","addr":"172.21.15.110:6819","nonce":1331818323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6822","nonce":1331818323},{"type":"v1","addr":"172.21.15.110:6823","nonce":1331818323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6820","nonce":1331818323},{"type":"v1","addr":"172.21.15.110:6821","nonce":1331818323}]},"public_addr":"172.21.15.110:6817/1331818323","cluster_addr":"172.21.15.110:6819/1331818323","heartbeat_back_addr":"172.21.15.110:6823/1331818323","heartbeat_front_addr":"172.21.15.110:6821/1331818323","state":["exists","up"]},{"osd":8,"uuid":"1bc2a5b4-15f3-456f-9d67-e51307a27bcc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6816","nonce":2138556699},{"type":"v1","addr":"172.21.15.114:6817","nonce":2138556699}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6818","nonce":2138556699},{"type":"v1","addr":"172.21.15.114:6819","nonce":2138556699}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6822","nonce":2138556699},{"type":"v1","addr":"172.21.15.114:6823","nonce":2138556699}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6820","nonce":2138556699},{"type":"v1","addr":"172.21.15.114:6821","nonce":2138556699}]},"public_addr":"172.21.15.114:6817/2138556699","cluster_addr":"172.21.15.114:6819/2138556699","heartbeat_back_addr":"172.21.15.114:6823/2138556699","heartbeat_front_addr":"172.21.15.114:6821/2138556699","state":["exists","up"]},{"osd":9,"uuid":"91f5fd29-d4a7-4800-ac7a-e83b13add89e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6826","nonce":1730253068},{"type":"v1","addr":"172.21.15.89:6827","nonce":1730253068}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6828","nonce":1730253068},{"type":"v1","addr":"172.21.15.89:6829","nonce":1730253068}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6832","nonce":1730253068},{"type":"v1","addr":"172.21.15.89:6833","nonce":1730253068}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6830","nonce":1730253068},{"type":"v1","addr":"172.21.15.89:6831","nonce":1730253068}]},"public_addr":"172.21.15.89:6827/1730253068","cluster_addr":"172.21.15.89:6829/1730253068","heartbeat_back_addr":"172.21.15.89:6833/1730253068","heartbeat_front_addr":"172.21.15.89:6831/1730253068","state":["exists","up"]},{"osd":10,"uuid":"01390499-2a4d-4188-b57a-fcb10c668e2c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6824","nonce":2904016117},{"type":"v1","addr":"172.21.15.110:6825","nonce":2904016117}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6826","nonce":2904016117},{"type":"v1","addr":"172.21.15.110:6827","nonce":2904016117}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6830","nonce":2904016117},{"type":"v1","addr":"172.21.15.110:6831","nonce":2904016117}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6828","nonce":2904016117},{"type":"v1","addr":"172.21.15.110:6829","nonce":2904016117}]},"public_addr":"172.21.15.110:6825/2904016117","cluster_addr":"172.21.15.110:6827/2904016117","heartbeat_back_addr":"172.21.15.110:6831/2904016117","heartbeat_front_addr":"172.21.15.110:6829/2904016117","state":["exists","up"]},{"osd":11,"uuid":"c3c0f38a-c3d3-415d-a9f8-875d5b756e35","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6824","nonce":3749854486},{"type":"v1","addr":"172.21.15.114:6825","nonce":3749854486}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6826","nonce":3749854486},{"type":"v1","addr":"172.21.15.114:6827","nonce":3749854486}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6830","nonce":3749854486},{"type":"v1","addr":"172.21.15.114:6831","nonce":3749854486}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6828","nonce":3749854486},{"type":"v1","addr":"172.21.15.114:6829","nonce":3749854486}]},"public_addr":"172.21.15.114:6825/3749854486","cluster_addr":"172.21.15.114:6827/3749854486","heartbeat_back_addr":"172.21.15.114:6831/3749854486","heartbeat_front_addr":"172.21.15.114:6829/3749854486","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:31:46.218908+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:32:10.338230+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:32:34.830596+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:33:00.816102+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:33:25.223838+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:33:50.188230+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:34:16.564255+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:34:41.498230+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:35:05.826322+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:35:31.234450+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:35:56.474373+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:36:21.741254+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.c","mappings":[{"from":2,"to":4}]},{"pgid":"2.e","mappings":[{"from":2,"to":1}]},{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]},{"pgid":"4.6","mappings":[{"from":9,"to":11}]},{"pgid":"4.12","mappings":[{"from":9,"to":10}]},{"pgid":"4.13","mappings":[{"from":9,"to":8}]},{"pgid":"4.1b","mappings":[{"from":5,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.89:6801/655110910":"2024-09-22T03:29:51.519106+0000","172.21.15.89:0/1618544983":"2024-09-22T03:29:51.519106+0000","172.21.15.89:0/1265502758":"2024-09-22T03:29:51.519106+0000","172.21.15.89:0/4017684419":"2024-09-22T03:29:51.519106+0000","172.21.15.89:6801/3312712420":"2024-09-22T03:29:26.004686+0000","172.21.15.89:6800/3312712420":"2024-09-22T03:29:26.004686+0000","172.21.15.89:0/2056576483":"2024-09-22T03:29:26.004686+0000","172.21.15.89:6800/655110910":"2024-09-22T03:29:51.519106+0000","172.21.15.89:0/2382866302":"2024-09-22T03:29:26.004686+0000","172.21.15.89:0/1584351801":"2024-09-22T03:29:26.004686+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T03:38:02.798 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:02.796+0000 7f50b1195640 1 -- 172.21.15.89:0/3696661315 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f50980b65a0 msgr2=0x7f50980b8ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:02.798 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:02.796+0000 7f50b1195640 1 --2- 172.21.15.89:0/3696661315 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f50980b65a0 0x7f50980b8ab0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f50940058c0 tx=0x7f5094005850 comp rx=0 tx=0).stop 2024-09-21T03:38:02.799 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:02.796+0000 7f50b1195640 1 -- 172.21.15.89:0/3696661315 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f50ac108cf0 msgr2=0x7f50ac19ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:02.799 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:02.796+0000 7f50b1195640 1 --2- 172.21.15.89:0/3696661315 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f50ac108cf0 0x7f50ac19ea90 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f50a4005cc0 tx=0x7f50a4032a10 comp rx=0 tx=0).stop 2024-09-21T03:38:02.799 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:02.796+0000 7f50b1195640 1 -- 172.21.15.89:0/3696661315 shutdown_connections 2024-09-21T03:38:02.799 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:02.796+0000 7f50b1195640 1 --2- 172.21.15.89:0/3696661315 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f50980b65a0 0x7f50980b8ab0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:02.799 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:02.796+0000 7f50b1195640 1 --2- 172.21.15.89:0/3696661315 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f50ac1083a0 0x7f50ac1997f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:02.799 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:02.796+0000 7f50b1195640 1 --2- 172.21.15.89:0/3696661315 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f50ac108cf0 0x7f50ac19ea90 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:02.799 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:02.796+0000 7f50b1195640 1 --2- 172.21.15.89:0/3696661315 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f50ac102a60 0x7f50ac199250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:02.799 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:02.796+0000 7f50b1195640 1 -- 172.21.15.89:0/3696661315 >> 172.21.15.89:0/3696661315 conn(0x7f50ac079730 msgr2=0x7f50ac1043d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:38:02.799 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:02.796+0000 7f50b1195640 1 -- 172.21.15.89:0/3696661315 shutdown_connections 2024-09-21T03:38:02.799 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:02.796+0000 7f50b1195640 1 -- 172.21.15.89:0/3696661315 wait complete. 2024-09-21T03:38:03.311 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:38:03 smithi089 ceph-mon[30928]: pgmap v317: 129 pgs: 129 active+clean; 633 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 7.4 KiB/s wr, 11 op/s; 72 B/s, 1 objects/s recovering 2024-09-21T03:38:03.311 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:38:03 smithi089 ceph-mon[30928]: from='client.? 172.21.15.89:0/3696661315' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T03:38:03.362 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:38:03 smithi114 ceph-mon[34140]: pgmap v317: 129 pgs: 129 active+clean; 633 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 7.4 KiB/s wr, 11 op/s; 72 B/s, 1 objects/s recovering 2024-09-21T03:38:03.362 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:38:03 smithi114 ceph-mon[34140]: from='client.? 172.21.15.89:0/3696661315' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T03:38:03.379 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-09-21T03:38:03.379 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 7c374098-77c9-11ef-baf5-efdc52797490 -- ceph pg dump --format=json 2024-09-21T03:38:03.401 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:38:03 smithi110 ceph-mon[34000]: pgmap v317: 129 pgs: 129 active+clean; 633 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 7.4 KiB/s wr, 11 op/s; 72 B/s, 1 objects/s recovering 2024-09-21T03:38:03.401 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:38:03 smithi110 ceph-mon[34000]: from='client.? 172.21.15.89:0/3696661315' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T03:38:03.628 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/7c374098-77c9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:38:04.096 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:04.091+0000 7fc93845d640 1 -- 172.21.15.89:0/4080273900 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc930103b60 msgr2=0x7fc930103f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:04.096 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:04.091+0000 7fc93845d640 1 --2- 172.21.15.89:0/4080273900 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc930103b60 0x7fc930103f70 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fc924009900 tx=0x7fc92402a410 comp rx=0 tx=0).stop 2024-09-21T03:38:04.096 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:04.091+0000 7fc93845d640 1 -- 172.21.15.89:0/4080273900 shutdown_connections 2024-09-21T03:38:04.096 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:04.091+0000 7fc93845d640 1 --2- 172.21.15.89:0/4080273900 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc930109df0 0x7fc930110620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:04.096 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:04.091+0000 7fc93845d640 1 --2- 172.21.15.89:0/4080273900 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc9301094a0 0x7fc9301098b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:04.096 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:04.091+0000 7fc93845d640 1 --2- 172.21.15.89:0/4080273900 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc930103b60 0x7fc930103f70 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:04.096 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:04.091+0000 7fc93845d640 1 -- 172.21.15.89:0/4080273900 >> 172.21.15.89:0/4080273900 conn(0x7fc93007b850 msgr2=0x7fc93007bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:38:04.096 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:04.091+0000 7fc93845d640 1 -- 172.21.15.89:0/4080273900 shutdown_connections 2024-09-21T03:38:04.096 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:04.091+0000 7fc93845d640 1 -- 172.21.15.89:0/4080273900 wait complete. 2024-09-21T03:38:04.097 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:04.092+0000 7fc93845d640 1 Processor -- start 2024-09-21T03:38:04.097 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:04.092+0000 7fc93845d640 1 -- start start 2024-09-21T03:38:04.097 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:04.093+0000 7fc93845d640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc930103b60 0x7fc93019a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:04.097 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:04.093+0000 7fc93845d640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc9301094a0 0x7fc93019a950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:04.097 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:04.093+0000 7fc93845d640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc930109df0 0x7fc93019fbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:04.097 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:04.093+0000 7fc93845d640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fc930113960 con 0x7fc9301094a0 2024-09-21T03:38:04.097 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:04.093+0000 7fc93845d640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fc9301137e0 con 0x7fc930109df0 2024-09-21T03:38:04.097 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:04.093+0000 7fc93845d640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fc930113ae0 con 0x7fc930103b60 2024-09-21T03:38:04.097 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:04.093+0000 7fc9359f8640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc9301094a0 0x7fc93019a950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:04.097 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:04.093+0000 7fc9359f8640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc9301094a0 0x7fc93019a950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:50790/0 (socket says 172.21.15.89:50790) 2024-09-21T03:38:04.097 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:04.093+0000 7fc9359f8640 1 -- 172.21.15.89:0/3884029574 learned_addr learned my addr 172.21.15.89:0/3884029574 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-21T03:38:04.097 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:04.093+0000 7fc9361f9640 1 --2- 172.21.15.89:0/3884029574 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc930103b60 0x7fc93019a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:04.097 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:04.093+0000 7fc9369fa640 1 --2- 172.21.15.89:0/3884029574 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc930109df0 0x7fc93019fbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:04.097 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:04.093+0000 7fc9359f8640 1 -- 172.21.15.89:0/3884029574 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc930103b60 msgr2=0x7fc93019a3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:04.097 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:04.093+0000 7fc9359f8640 1 --2- 172.21.15.89:0/3884029574 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc930103b60 0x7fc93019a3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:04.098 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:04.093+0000 7fc9359f8640 1 -- 172.21.15.89:0/3884029574 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc930109df0 msgr2=0x7fc93019fbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:04.098 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:04.093+0000 7fc9359f8640 1 --2- 172.21.15.89:0/3884029574 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc930109df0 0x7fc93019fbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:04.098 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:04.093+0000 7fc9359f8640 1 -- 172.21.15.89:0/3884029574 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9301a0320 con 0x7fc9301094a0 2024-09-21T03:38:04.098 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:04.093+0000 7fc9359f8640 1 --2- 172.21.15.89:0/3884029574 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc9301094a0 0x7fc93019a950 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fc92c00e9c0 tx=0x7fc92c00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:38:04.098 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:04.094+0000 7fc9237fe640 1 -- 172.21.15.89:0/3884029574 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc92c00ccc0 con 0x7fc9301094a0 2024-09-21T03:38:04.098 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:04.094+0000 7fc93845d640 1 -- 172.21.15.89:0/3884029574 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc9301a0610 con 0x7fc9301094a0 2024-09-21T03:38:04.098 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:04.094+0000 7fc9237fe640 1 -- 172.21.15.89:0/3884029574 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc92c0054b0 con 0x7fc9301094a0 2024-09-21T03:38:04.098 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:04.094+0000 7fc9237fe640 1 -- 172.21.15.89:0/3884029574 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc92c010430 con 0x7fc9301094a0 2024-09-21T03:38:04.098 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:04.094+0000 7fc93845d640 1 -- 172.21.15.89:0/3884029574 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9301a6d70 con 0x7fc9301094a0 2024-09-21T03:38:04.098 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:04.095+0000 7fc93845d640 1 -- 172.21.15.89:0/3884029574 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc930103f70 con 0x7fc9301094a0 2024-09-21T03:38:04.098 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:04.096+0000 7fc9237fe640 1 -- 172.21.15.89:0/3884029574 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 18) ==== 154563+0+0 (secure 0 0 0) 0x7fc92c004f20 con 0x7fc9301094a0 2024-09-21T03:38:04.099 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:04.097+0000 7fc9237fe640 1 --2- 172.21.15.89:0/3884029574 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7fc91c0b65a0 0x7fc91c0b8ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:04.099 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:04.097+0000 7fc9361f9640 1 --2- 172.21.15.89:0/3884029574 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7fc91c0b65a0 0x7fc91c0b8ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:04.099 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:04.097+0000 7fc9237fe640 1 -- 172.21.15.89:0/3884029574 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 9092+0+0 (secure 0 0 0) 0x7fc92c0e7b70 con 0x7fc9301094a0 2024-09-21T03:38:04.099 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:04.097+0000 7fc9361f9640 1 --2- 172.21.15.89:0/3884029574 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7fc91c0b65a0 0x7fc91c0b8ab0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fc9240057c0 tx=0x7fc9240058d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:38:04.103 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:04.101+0000 7fc9237fe640 1 -- 172.21.15.89:0/3884029574 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fc92c0adf40 con 0x7fc9301094a0 2024-09-21T03:38:04.281 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:04.278+0000 7fc93845d640 1 -- 172.21.15.89:0/3884029574 --> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fc930063800 con 0x7fc91c0b65a0 2024-09-21T03:38:04.286 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:04.284+0000 7fc9237fe640 1 -- 172.21.15.89:0/3884029574 <== mgr.14152 v2:172.21.15.89:6800/1591532358 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340626 (secure 0 0 0) 0x7fc930063800 con 0x7fc91c0b65a0 2024-09-21T03:38:04.290 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-09-21T03:38:04.291 INFO:teuthology.orchestra.run.smithi089.stderr:dumped all 2024-09-21T03:38:04.293 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:04.288+0000 7fc93845d640 1 -- 172.21.15.89:0/3884029574 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7fc91c0b65a0 msgr2=0x7fc91c0b8ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:04.294 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:04.288+0000 7fc93845d640 1 --2- 172.21.15.89:0/3884029574 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7fc91c0b65a0 0x7fc91c0b8ab0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fc9240057c0 tx=0x7fc9240058d0 comp rx=0 tx=0).stop 2024-09-21T03:38:04.294 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:04.288+0000 7fc93845d640 1 -- 172.21.15.89:0/3884029574 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc9301094a0 msgr2=0x7fc93019a950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:04.294 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:04.288+0000 7fc93845d640 1 --2- 172.21.15.89:0/3884029574 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc9301094a0 0x7fc93019a950 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fc92c00e9c0 tx=0x7fc92c00ee80 comp rx=0 tx=0).stop 2024-09-21T03:38:04.294 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:04.289+0000 7fc93845d640 1 -- 172.21.15.89:0/3884029574 shutdown_connections 2024-09-21T03:38:04.294 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:04.289+0000 7fc93845d640 1 --2- 172.21.15.89:0/3884029574 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7fc91c0b65a0 0x7fc91c0b8ab0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:04.294 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:04.289+0000 7fc93845d640 1 --2- 172.21.15.89:0/3884029574 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc930109df0 0x7fc93019fbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:04.294 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:04.289+0000 7fc93845d640 1 --2- 172.21.15.89:0/3884029574 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc9301094a0 0x7fc93019a950 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:04.294 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:04.289+0000 7fc93845d640 1 --2- 172.21.15.89:0/3884029574 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc930103b60 0x7fc93019a3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:04.294 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:04.289+0000 7fc93845d640 1 -- 172.21.15.89:0/3884029574 >> 172.21.15.89:0/3884029574 conn(0x7fc93007b850 msgr2=0x7fc9301075a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:38:04.294 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:04.289+0000 7fc93845d640 1 -- 172.21.15.89:0/3884029574 shutdown_connections 2024-09-21T03:38:04.294 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:04.289+0000 7fc93845d640 1 -- 172.21.15.89:0/3884029574 wait complete. 2024-09-21T03:38:04.979 INFO:teuthology.orchestra.run.smithi089.stdout:{"pg_ready":true,"pg_map":{"version":318,"stamp":"2024-09-21T03:38:03.622180+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":647837,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":170,"num_read_kb":169,"num_write":279,"num_write_kb":2254,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":23,"num_bytes_recovered":2952534,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":227,"ondisk_log_size":227,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":376888,"kb_used_data":10976,"kb_used_omap":18,"kb_used_meta":365677,"kb_avail":1124564936,"statfs":{"total":1151940427776,"available":1151554494464,"internally_reserved":0,"allocated":11239424,"data_stored":7454717,"data_compressed":390510,"data_compressed_allocated":528384,"data_compressed_original":2555904,"omap_allocated":19143,"internal_metadata":374453561},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":50129,"num_objects":5,"num_object_clones":0,"num_object_copies":17,"num_objects_missing_on_primary":0,"num_objects_missing":-2,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":66,"num_read_kb":82,"num_write":32,"num_write_kb":63,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":694,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.580703"},"pg_stats":[{"pgid":"2.1f","version":"80'9","reported_seq":45,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.047700+0000","last_change":"2024-09-21T03:37:10.836953+0000","last_active":"2024-09-21T03:37:53.047700+0000","last_peered":"2024-09-21T03:37:53.047700+0000","last_clean":"2024-09-21T03:37:53.047700+0000","last_became_active":"2024-09-21T03:37:10.836566+0000","last_became_peered":"2024-09-21T03:37:10.836566+0000","last_unstale":"2024-09-21T03:37:53.047700+0000","last_undegraded":"2024-09-21T03:37:53.047700+0000","last_fullsized":"2024-09-21T03:37:53.047700+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_clean_scrub_stamp":"2024-09-21T03:37:09.809805+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:15:45.591225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.046350+0000","last_change":"2024-09-21T03:37:12.866931+0000","last_active":"2024-09-21T03:37:53.046350+0000","last_peered":"2024-09-21T03:37:53.046350+0000","last_clean":"2024-09-21T03:37:53.046350+0000","last_became_active":"2024-09-21T03:37:12.866452+0000","last_became_peered":"2024-09-21T03:37:12.866452+0000","last_unstale":"2024-09-21T03:37:53.046350+0000","last_undegraded":"2024-09-21T03:37:53.046350+0000","last_fullsized":"2024-09-21T03:37:53.046350+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:37:14.324548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.044138+0000","last_change":"2024-09-21T03:37:19.912154+0000","last_active":"2024-09-21T03:37:53.044138+0000","last_peered":"2024-09-21T03:37:53.044138+0000","last_clean":"2024-09-21T03:37:53.044138+0000","last_became_active":"2024-09-21T03:37:19.911686+0000","last_became_peered":"2024-09-21T03:37:19.911686+0000","last_unstale":"2024-09-21T03:37:53.044138+0000","last_undegraded":"2024-09-21T03:37:53.044138+0000","last_fullsized":"2024-09-21T03:37:53.044138+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_clean_scrub_stamp":"2024-09-21T03:37:18.876933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:29:20.189418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"80'2","reported_seq":37,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.046676+0000","last_change":"2024-09-21T03:37:11.034501+0000","last_active":"2024-09-21T03:37:53.046676+0000","last_peered":"2024-09-21T03:37:53.046676+0000","last_clean":"2024-09-21T03:37:53.046676+0000","last_became_active":"2024-09-21T03:37:11.034110+0000","last_became_peered":"2024-09-21T03:37:11.034110+0000","last_unstale":"2024-09-21T03:37:53.046676+0000","last_undegraded":"2024-09-21T03:37:53.046676+0000","last_fullsized":"2024-09-21T03:37:53.046676+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_clean_scrub_stamp":"2024-09-21T03:37:09.809805+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:03:12.854469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.049598+0000","last_change":"2024-09-21T03:37:12.867330+0000","last_active":"2024-09-21T03:37:53.049598+0000","last_peered":"2024-09-21T03:37:53.049598+0000","last_clean":"2024-09-21T03:37:53.049598+0000","last_became_active":"2024-09-21T03:37:12.866952+0000","last_became_peered":"2024-09-21T03:37:12.866952+0000","last_unstale":"2024-09-21T03:37:53.049598+0000","last_undegraded":"2024-09-21T03:37:53.049598+0000","last_fullsized":"2024-09-21T03:37:53.049598+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:40:33.826546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.043412+0000","last_change":"2024-09-21T03:37:19.910661+0000","last_active":"2024-09-21T03:37:53.043412+0000","last_peered":"2024-09-21T03:37:53.043412+0000","last_clean":"2024-09-21T03:37:53.043412+0000","last_became_active":"2024-09-21T03:37:19.910108+0000","last_became_peered":"2024-09-21T03:37:19.910108+0000","last_unstale":"2024-09-21T03:37:53.043412+0000","last_undegraded":"2024-09-21T03:37:53.043412+0000","last_fullsized":"2024-09-21T03:37:53.043412+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_clean_scrub_stamp":"2024-09-21T03:37:18.876933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:18:05.654426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"80'4","reported_seq":39,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.050839+0000","last_change":"2024-09-21T03:37:10.835336+0000","last_active":"2024-09-21T03:37:53.050839+0000","last_peered":"2024-09-21T03:37:53.050839+0000","last_clean":"2024-09-21T03:37:53.050839+0000","last_became_active":"2024-09-21T03:37:10.834928+0000","last_became_peered":"2024-09-21T03:37:10.834928+0000","last_unstale":"2024-09-21T03:37:53.050839+0000","last_undegraded":"2024-09-21T03:37:53.050839+0000","last_fullsized":"2024-09-21T03:37:53.050839+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_clean_scrub_stamp":"2024-09-21T03:37:09.809805+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:02:14.717520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.111510+0000","last_change":"2024-09-21T03:37:12.861271+0000","last_active":"2024-09-21T03:37:53.111510+0000","last_peered":"2024-09-21T03:37:53.111510+0000","last_clean":"2024-09-21T03:37:53.111510+0000","last_became_active":"2024-09-21T03:37:12.860932+0000","last_became_peered":"2024-09-21T03:37:12.860932+0000","last_unstale":"2024-09-21T03:37:53.111510+0000","last_undegraded":"2024-09-21T03:37:53.111510+0000","last_fullsized":"2024-09-21T03:37:53.111510+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:46:08.993250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.053171+0000","last_change":"2024-09-21T03:37:53.053045+0000","last_active":"2024-09-21T03:37:53.053171+0000","last_peered":"2024-09-21T03:37:53.053171+0000","last_clean":"2024-09-21T03:37:53.053171+0000","last_became_active":"2024-09-21T03:37:53.052382+0000","last_became_peered":"2024-09-21T03:37:53.052382+0000","last_unstale":"2024-09-21T03:37:53.053171+0000","last_undegraded":"2024-09-21T03:37:53.053171+0000","last_fullsized":"2024-09-21T03:37:53.053171+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_clean_scrub_stamp":"2024-09-21T03:37:18.876933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:02:44.300768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,8],"acting":[4,3,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"74'2","reported_seq":37,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.050328+0000","last_change":"2024-09-21T03:37:10.833985+0000","last_active":"2024-09-21T03:37:53.050328+0000","last_peered":"2024-09-21T03:37:53.050328+0000","last_clean":"2024-09-21T03:37:53.050328+0000","last_became_active":"2024-09-21T03:37:10.833562+0000","last_became_peered":"2024-09-21T03:37:10.833562+0000","last_unstale":"2024-09-21T03:37:53.050328+0000","last_undegraded":"2024-09-21T03:37:53.050328+0000","last_fullsized":"2024-09-21T03:37:53.050328+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_clean_scrub_stamp":"2024-09-21T03:37:09.809805+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:49:32.617768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.047066+0000","last_change":"2024-09-21T03:37:12.865408+0000","last_active":"2024-09-21T03:37:53.047066+0000","last_peered":"2024-09-21T03:37:53.047066+0000","last_clean":"2024-09-21T03:37:53.047066+0000","last_became_active":"2024-09-21T03:37:12.864972+0000","last_became_peered":"2024-09-21T03:37:12.864972+0000","last_unstale":"2024-09-21T03:37:53.047066+0000","last_undegraded":"2024-09-21T03:37:53.047066+0000","last_fullsized":"2024-09-21T03:37:53.047066+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:03:16.384260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.043495+0000","last_change":"2024-09-21T03:37:19.912105+0000","last_active":"2024-09-21T03:37:53.043495+0000","last_peered":"2024-09-21T03:37:53.043495+0000","last_clean":"2024-09-21T03:37:53.043495+0000","last_became_active":"2024-09-21T03:37:19.911416+0000","last_became_peered":"2024-09-21T03:37:19.911416+0000","last_unstale":"2024-09-21T03:37:53.043495+0000","last_undegraded":"2024-09-21T03:37:53.043495+0000","last_fullsized":"2024-09-21T03:37:53.043495+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_clean_scrub_stamp":"2024-09-21T03:37:18.876933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:20:16.492231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.431254+0000","last_change":"2024-09-21T03:37:12.862354+0000","last_active":"2024-09-21T03:37:53.431254+0000","last_peered":"2024-09-21T03:37:53.431254+0000","last_clean":"2024-09-21T03:37:53.431254+0000","last_became_active":"2024-09-21T03:37:12.861977+0000","last_became_peered":"2024-09-21T03:37:12.861977+0000","last_unstale":"2024-09-21T03:37:53.431254+0000","last_undegraded":"2024-09-21T03:37:53.431254+0000","last_fullsized":"2024-09-21T03:37:53.431254+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:53:52.725524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"80'4","reported_seq":40,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.051416+0000","last_change":"2024-09-21T03:37:10.832140+0000","last_active":"2024-09-21T03:37:53.051416+0000","last_peered":"2024-09-21T03:37:53.051416+0000","last_clean":"2024-09-21T03:37:53.051416+0000","last_became_active":"2024-09-21T03:37:10.831395+0000","last_became_peered":"2024-09-21T03:37:10.831395+0000","last_unstale":"2024-09-21T03:37:53.051416+0000","last_undegraded":"2024-09-21T03:37:53.051416+0000","last_fullsized":"2024-09-21T03:37:53.051416+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_clean_scrub_stamp":"2024-09-21T03:37:09.809805+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:55:20.440890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.044184+0000","last_change":"2024-09-21T03:37:20.160850+0000","last_active":"2024-09-21T03:37:53.044184+0000","last_peered":"2024-09-21T03:37:53.044184+0000","last_clean":"2024-09-21T03:37:53.044184+0000","last_became_active":"2024-09-21T03:37:20.160305+0000","last_became_peered":"2024-09-21T03:37:20.160305+0000","last_unstale":"2024-09-21T03:37:53.044184+0000","last_undegraded":"2024-09-21T03:37:53.044184+0000","last_fullsized":"2024-09-21T03:37:53.044184+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_clean_scrub_stamp":"2024-09-21T03:37:18.876933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:51:45.971984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.046377+0000","last_change":"2024-09-21T03:37:12.859485+0000","last_active":"2024-09-21T03:37:53.046377+0000","last_peered":"2024-09-21T03:37:53.046377+0000","last_clean":"2024-09-21T03:37:53.046377+0000","last_became_active":"2024-09-21T03:37:12.859097+0000","last_became_peered":"2024-09-21T03:37:12.859097+0000","last_unstale":"2024-09-21T03:37:53.046377+0000","last_undegraded":"2024-09-21T03:37:53.046377+0000","last_fullsized":"2024-09-21T03:37:53.046377+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:16:05.438360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"80'1","reported_seq":36,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.044264+0000","last_change":"2024-09-21T03:37:10.832441+0000","last_active":"2024-09-21T03:37:53.044264+0000","last_peered":"2024-09-21T03:37:53.044264+0000","last_clean":"2024-09-21T03:37:53.044264+0000","last_became_active":"2024-09-21T03:37:10.832073+0000","last_became_peered":"2024-09-21T03:37:10.832073+0000","last_unstale":"2024-09-21T03:37:53.044264+0000","last_undegraded":"2024-09-21T03:37:53.044264+0000","last_fullsized":"2024-09-21T03:37:53.044264+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_clean_scrub_stamp":"2024-09-21T03:37:09.809805+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:32:19.411097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"82'1","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.043995+0000","last_change":"2024-09-21T03:37:19.905490+0000","last_active":"2024-09-21T03:37:53.043995+0000","last_peered":"2024-09-21T03:37:53.043995+0000","last_clean":"2024-09-21T03:37:53.043995+0000","last_became_active":"2024-09-21T03:37:19.904971+0000","last_became_peered":"2024-09-21T03:37:19.904971+0000","last_unstale":"2024-09-21T03:37:53.043995+0000","last_undegraded":"2024-09-21T03:37:53.043995+0000","last_fullsized":"2024-09-21T03:37:53.043995+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_clean_scrub_stamp":"2024-09-21T03:37:18.876933+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:44:20.655538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.048957+0000","last_change":"2024-09-21T03:37:12.863597+0000","last_active":"2024-09-21T03:37:53.048957+0000","last_peered":"2024-09-21T03:37:53.048957+0000","last_clean":"2024-09-21T03:37:53.048957+0000","last_became_active":"2024-09-21T03:37:12.863223+0000","last_became_peered":"2024-09-21T03:37:12.863223+0000","last_unstale":"2024-09-21T03:37:53.048957+0000","last_undegraded":"2024-09-21T03:37:53.048957+0000","last_fullsized":"2024-09-21T03:37:53.048957+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:29:28.878964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.9","version":"80'3","reported_seq":38,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.045808+0000","last_change":"2024-09-21T03:37:10.834648+0000","last_active":"2024-09-21T03:37:53.045808+0000","last_peered":"2024-09-21T03:37:53.045808+0000","last_clean":"2024-09-21T03:37:53.045808+0000","last_became_active":"2024-09-21T03:37:10.834293+0000","last_became_peered":"2024-09-21T03:37:10.834293+0000","last_unstale":"2024-09-21T03:37:53.045808+0000","last_undegraded":"2024-09-21T03:37:53.045808+0000","last_fullsized":"2024-09-21T03:37:53.045808+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_clean_scrub_stamp":"2024-09-21T03:37:09.809805+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:10:41.248733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.047492+0000","last_change":"2024-09-21T03:37:19.912526+0000","last_active":"2024-09-21T03:37:53.047492+0000","last_peered":"2024-09-21T03:37:53.047492+0000","last_clean":"2024-09-21T03:37:53.047492+0000","last_became_active":"2024-09-21T03:37:19.912062+0000","last_became_peered":"2024-09-21T03:37:19.912062+0000","last_unstale":"2024-09-21T03:37:53.047492+0000","last_undegraded":"2024-09-21T03:37:53.047492+0000","last_fullsized":"2024-09-21T03:37:53.047492+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_clean_scrub_stamp":"2024-09-21T03:37:18.876933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:52:40.395834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.049515+0000","last_change":"2024-09-21T03:37:12.855672+0000","last_active":"2024-09-21T03:37:53.049515+0000","last_peered":"2024-09-21T03:37:53.049515+0000","last_clean":"2024-09-21T03:37:53.049515+0000","last_became_active":"2024-09-21T03:37:12.855191+0000","last_became_peered":"2024-09-21T03:37:12.855191+0000","last_unstale":"2024-09-21T03:37:53.049515+0000","last_undegraded":"2024-09-21T03:37:53.049515+0000","last_fullsized":"2024-09-21T03:37:53.049515+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:26:56.115697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.8","version":"80'2","reported_seq":37,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.111296+0000","last_change":"2024-09-21T03:37:11.034752+0000","last_active":"2024-09-21T03:37:53.111296+0000","last_peered":"2024-09-21T03:37:53.111296+0000","last_clean":"2024-09-21T03:37:53.111296+0000","last_became_active":"2024-09-21T03:37:11.034386+0000","last_became_peered":"2024-09-21T03:37:11.034386+0000","last_unstale":"2024-09-21T03:37:53.111296+0000","last_undegraded":"2024-09-21T03:37:53.111296+0000","last_fullsized":"2024-09-21T03:37:53.111296+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_clean_scrub_stamp":"2024-09-21T03:37:09.809805+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:17:23.647411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.044213+0000","last_change":"2024-09-21T03:37:20.162051+0000","last_active":"2024-09-21T03:37:53.044213+0000","last_peered":"2024-09-21T03:37:53.044213+0000","last_clean":"2024-09-21T03:37:53.044213+0000","last_became_active":"2024-09-21T03:37:20.161452+0000","last_became_peered":"2024-09-21T03:37:20.161452+0000","last_unstale":"2024-09-21T03:37:53.044213+0000","last_undegraded":"2024-09-21T03:37:53.044213+0000","last_fullsized":"2024-09-21T03:37:53.044213+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_clean_scrub_stamp":"2024-09-21T03:37:18.876933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:29:22.544800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.049528+0000","last_change":"2024-09-21T03:37:12.866058+0000","last_active":"2024-09-21T03:37:53.049528+0000","last_peered":"2024-09-21T03:37:53.049528+0000","last_clean":"2024-09-21T03:37:53.049528+0000","last_became_active":"2024-09-21T03:37:12.865694+0000","last_became_peered":"2024-09-21T03:37:12.865694+0000","last_unstale":"2024-09-21T03:37:53.049528+0000","last_undegraded":"2024-09-21T03:37:53.049528+0000","last_fullsized":"2024-09-21T03:37:53.049528+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:56:33.629873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"80'4","reported_seq":39,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.044782+0000","last_change":"2024-09-21T03:37:11.358619+0000","last_active":"2024-09-21T03:37:53.044782+0000","last_peered":"2024-09-21T03:37:53.044782+0000","last_clean":"2024-09-21T03:37:53.044782+0000","last_became_active":"2024-09-21T03:37:11.358196+0000","last_became_peered":"2024-09-21T03:37:11.358196+0000","last_unstale":"2024-09-21T03:37:53.044782+0000","last_undegraded":"2024-09-21T03:37:53.044782+0000","last_fullsized":"2024-09-21T03:37:53.044782+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_clean_scrub_stamp":"2024-09-21T03:37:09.809805+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:57:13.476641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.044378+0000","last_change":"2024-09-21T03:37:19.905711+0000","last_active":"2024-09-21T03:37:53.044378+0000","last_peered":"2024-09-21T03:37:53.044378+0000","last_clean":"2024-09-21T03:37:53.044378+0000","last_became_active":"2024-09-21T03:37:19.905260+0000","last_became_peered":"2024-09-21T03:37:19.905260+0000","last_unstale":"2024-09-21T03:37:53.044378+0000","last_undegraded":"2024-09-21T03:37:53.044378+0000","last_fullsized":"2024-09-21T03:37:53.044378+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_clean_scrub_stamp":"2024-09-21T03:37:18.876933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:21:09.476082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.046425+0000","last_change":"2024-09-21T03:37:12.854376+0000","last_active":"2024-09-21T03:37:53.046425+0000","last_peered":"2024-09-21T03:37:53.046425+0000","last_clean":"2024-09-21T03:37:53.046425+0000","last_became_active":"2024-09-21T03:37:12.852942+0000","last_became_peered":"2024-09-21T03:37:12.852942+0000","last_unstale":"2024-09-21T03:37:53.046425+0000","last_undegraded":"2024-09-21T03:37:53.046425+0000","last_fullsized":"2024-09-21T03:37:53.046425+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:06:49.319002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"82'8","reported_seq":56,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:59.865741+0000","last_change":"2024-09-21T03:37:53.056889+0000","last_active":"2024-09-21T03:37:59.865741+0000","last_peered":"2024-09-21T03:37:59.865741+0000","last_clean":"2024-09-21T03:37:59.865741+0000","last_became_active":"2024-09-21T03:37:53.053054+0000","last_became_peered":"2024-09-21T03:37:53.053054+0000","last_unstale":"2024-09-21T03:37:59.865741+0000","last_undegraded":"2024-09-21T03:37:59.865741+0000","last_fullsized":"2024-09-21T03:37:59.865741+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_clean_scrub_stamp":"2024-09-21T03:37:09.809805+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:16:05.539111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,3],"acting":[9,11,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.045339+0000","last_change":"2024-09-21T03:37:20.161515+0000","last_active":"2024-09-21T03:37:53.045339+0000","last_peered":"2024-09-21T03:37:53.045339+0000","last_clean":"2024-09-21T03:37:53.045339+0000","last_became_active":"2024-09-21T03:37:20.160548+0000","last_became_peered":"2024-09-21T03:37:20.160548+0000","last_unstale":"2024-09-21T03:37:53.045339+0000","last_undegraded":"2024-09-21T03:37:53.045339+0000","last_fullsized":"2024-09-21T03:37:53.045339+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_clean_scrub_stamp":"2024-09-21T03:37:18.876933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:17:51.144859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.042971+0000","last_change":"2024-09-21T03:37:12.848305+0000","last_active":"2024-09-21T03:37:53.042971+0000","last_peered":"2024-09-21T03:37:53.042971+0000","last_clean":"2024-09-21T03:37:53.042971+0000","last_became_active":"2024-09-21T03:37:12.847916+0000","last_became_peered":"2024-09-21T03:37:12.847916+0000","last_unstale":"2024-09-21T03:37:53.042971+0000","last_undegraded":"2024-09-21T03:37:53.042971+0000","last_fullsized":"2024-09-21T03:37:53.042971+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:48:39.922805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"80'1","reported_seq":37,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.051068+0000","last_change":"2024-09-21T03:37:11.035964+0000","last_active":"2024-09-21T03:37:53.051068+0000","last_peered":"2024-09-21T03:37:53.051068+0000","last_clean":"2024-09-21T03:37:53.051068+0000","last_became_active":"2024-09-21T03:37:11.035427+0000","last_became_peered":"2024-09-21T03:37:11.035427+0000","last_unstale":"2024-09-21T03:37:53.051068+0000","last_undegraded":"2024-09-21T03:37:53.051068+0000","last_fullsized":"2024-09-21T03:37:53.051068+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_clean_scrub_stamp":"2024-09-21T03:37:09.809805+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:04:51.026753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.047032+0000","last_change":"2024-09-21T03:37:19.909428+0000","last_active":"2024-09-21T03:37:53.047032+0000","last_peered":"2024-09-21T03:37:53.047032+0000","last_clean":"2024-09-21T03:37:53.047032+0000","last_became_active":"2024-09-21T03:37:19.908904+0000","last_became_peered":"2024-09-21T03:37:19.908904+0000","last_unstale":"2024-09-21T03:37:53.047032+0000","last_undegraded":"2024-09-21T03:37:53.047032+0000","last_fullsized":"2024-09-21T03:37:53.047032+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_clean_scrub_stamp":"2024-09-21T03:37:18.876933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:12:07.564058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.046987+0000","last_change":"2024-09-21T03:37:12.855592+0000","last_active":"2024-09-21T03:37:53.046987+0000","last_peered":"2024-09-21T03:37:53.046987+0000","last_clean":"2024-09-21T03:37:53.046987+0000","last_became_active":"2024-09-21T03:37:12.855244+0000","last_became_peered":"2024-09-21T03:37:12.855244+0000","last_unstale":"2024-09-21T03:37:53.046987+0000","last_undegraded":"2024-09-21T03:37:53.046987+0000","last_fullsized":"2024-09-21T03:37:53.046987+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:21:34.603141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":35,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.047381+0000","last_change":"2024-09-21T03:37:11.036401+0000","last_active":"2024-09-21T03:37:53.047381+0000","last_peered":"2024-09-21T03:37:53.047381+0000","last_clean":"2024-09-21T03:37:53.047381+0000","last_became_active":"2024-09-21T03:37:11.036003+0000","last_became_peered":"2024-09-21T03:37:11.036003+0000","last_unstale":"2024-09-21T03:37:53.047381+0000","last_undegraded":"2024-09-21T03:37:53.047381+0000","last_fullsized":"2024-09-21T03:37:53.047381+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_clean_scrub_stamp":"2024-09-21T03:37:09.809805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:35:50.549060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.050609+0000","last_change":"2024-09-21T03:37:20.160668+0000","last_active":"2024-09-21T03:37:53.050609+0000","last_peered":"2024-09-21T03:37:53.050609+0000","last_clean":"2024-09-21T03:37:53.050609+0000","last_became_active":"2024-09-21T03:37:20.160158+0000","last_became_peered":"2024-09-21T03:37:20.160158+0000","last_unstale":"2024-09-21T03:37:53.050609+0000","last_undegraded":"2024-09-21T03:37:53.050609+0000","last_fullsized":"2024-09-21T03:37:53.050609+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_clean_scrub_stamp":"2024-09-21T03:37:18.876933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:45:25.677667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":37,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.050453+0000","last_change":"2024-09-21T03:37:53.050351+0000","last_active":"2024-09-21T03:37:53.050453+0000","last_peered":"2024-09-21T03:37:53.050453+0000","last_clean":"2024-09-21T03:37:53.050453+0000","last_became_active":"2024-09-21T03:37:53.050003+0000","last_became_peered":"2024-09-21T03:37:53.050003+0000","last_unstale":"2024-09-21T03:37:53.050453+0000","last_undegraded":"2024-09-21T03:37:53.050453+0000","last_fullsized":"2024-09-21T03:37:53.050453+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:49:03.313002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,0],"acting":[10,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"80'1","reported_seq":36,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.431360+0000","last_change":"2024-09-21T03:37:10.832635+0000","last_active":"2024-09-21T03:37:53.431360+0000","last_peered":"2024-09-21T03:37:53.431360+0000","last_clean":"2024-09-21T03:37:53.431360+0000","last_became_active":"2024-09-21T03:37:10.832280+0000","last_became_peered":"2024-09-21T03:37:10.832280+0000","last_unstale":"2024-09-21T03:37:53.431360+0000","last_undegraded":"2024-09-21T03:37:53.431360+0000","last_fullsized":"2024-09-21T03:37:53.431360+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_clean_scrub_stamp":"2024-09-21T03:37:09.809805+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:54:52.744486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.042822+0000","last_change":"2024-09-21T03:37:19.904112+0000","last_active":"2024-09-21T03:37:53.042822+0000","last_peered":"2024-09-21T03:37:53.042822+0000","last_clean":"2024-09-21T03:37:53.042822+0000","last_became_active":"2024-09-21T03:37:19.903677+0000","last_became_peered":"2024-09-21T03:37:19.903677+0000","last_unstale":"2024-09-21T03:37:53.042822+0000","last_undegraded":"2024-09-21T03:37:53.042822+0000","last_fullsized":"2024-09-21T03:37:53.042822+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_clean_scrub_stamp":"2024-09-21T03:37:18.876933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:32:20.096850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.046073+0000","last_change":"2024-09-21T03:37:12.864082+0000","last_active":"2024-09-21T03:37:53.046073+0000","last_peered":"2024-09-21T03:37:53.046073+0000","last_clean":"2024-09-21T03:37:53.046073+0000","last_became_active":"2024-09-21T03:37:12.862745+0000","last_became_peered":"2024-09-21T03:37:12.862745+0000","last_unstale":"2024-09-21T03:37:53.046073+0000","last_undegraded":"2024-09-21T03:37:53.046073+0000","last_fullsized":"2024-09-21T03:37:53.046073+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:25:35.759719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"80'4","reported_seq":50,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:58.042189+0000","last_change":"2024-09-21T03:37:11.034463+0000","last_active":"2024-09-21T03:37:58.042189+0000","last_peered":"2024-09-21T03:37:58.042189+0000","last_clean":"2024-09-21T03:37:58.042189+0000","last_became_active":"2024-09-21T03:37:11.033932+0000","last_became_peered":"2024-09-21T03:37:11.033932+0000","last_unstale":"2024-09-21T03:37:58.042189+0000","last_undegraded":"2024-09-21T03:37:58.042189+0000","last_fullsized":"2024-09-21T03:37:58.042189+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_clean_scrub_stamp":"2024-09-21T03:37:09.809805+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:15:18.595442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":11,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.043133+0000","last_change":"2024-09-21T03:37:20.160789+0000","last_active":"2024-09-21T03:37:53.043133+0000","last_peered":"2024-09-21T03:37:53.043133+0000","last_clean":"2024-09-21T03:37:53.043133+0000","last_became_active":"2024-09-21T03:37:20.160231+0000","last_became_peered":"2024-09-21T03:37:20.160231+0000","last_unstale":"2024-09-21T03:37:53.043133+0000","last_undegraded":"2024-09-21T03:37:53.043133+0000","last_fullsized":"2024-09-21T03:37:53.043133+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_clean_scrub_stamp":"2024-09-21T03:37:18.876933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:07:16.762249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.042510+0000","last_change":"2024-09-21T03:37:12.853178+0000","last_active":"2024-09-21T03:37:53.042510+0000","last_peered":"2024-09-21T03:37:53.042510+0000","last_clean":"2024-09-21T03:37:53.042510+0000","last_became_active":"2024-09-21T03:37:12.852822+0000","last_became_peered":"2024-09-21T03:37:12.852822+0000","last_unstale":"2024-09-21T03:37:53.042510+0000","last_undegraded":"2024-09-21T03:37:53.042510+0000","last_fullsized":"2024-09-21T03:37:53.042510+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:48:26.146593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"80'3","reported_seq":38,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.430945+0000","last_change":"2024-09-21T03:37:10.838939+0000","last_active":"2024-09-21T03:37:53.430945+0000","last_peered":"2024-09-21T03:37:53.430945+0000","last_clean":"2024-09-21T03:37:53.430945+0000","last_became_active":"2024-09-21T03:37:10.838562+0000","last_became_peered":"2024-09-21T03:37:10.838562+0000","last_unstale":"2024-09-21T03:37:53.430945+0000","last_undegraded":"2024-09-21T03:37:53.430945+0000","last_fullsized":"2024-09-21T03:37:53.430945+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_clean_scrub_stamp":"2024-09-21T03:37:09.809805+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:35:32.936824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.053062+0000","last_change":"2024-09-21T03:37:53.052940+0000","last_active":"2024-09-21T03:37:53.053062+0000","last_peered":"2024-09-21T03:37:53.053062+0000","last_clean":"2024-09-21T03:37:53.053062+0000","last_became_active":"2024-09-21T03:37:53.052310+0000","last_became_peered":"2024-09-21T03:37:53.052310+0000","last_unstale":"2024-09-21T03:37:53.053062+0000","last_undegraded":"2024-09-21T03:37:53.053062+0000","last_fullsized":"2024-09-21T03:37:53.053062+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_clean_scrub_stamp":"2024-09-21T03:37:18.876933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:31:35.353226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,11,6,10],"acting":[5,11,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.046076+0000","last_change":"2024-09-21T03:37:12.869795+0000","last_active":"2024-09-21T03:37:53.046076+0000","last_peered":"2024-09-21T03:37:53.046076+0000","last_clean":"2024-09-21T03:37:53.046076+0000","last_became_active":"2024-09-21T03:37:12.869381+0000","last_became_peered":"2024-09-21T03:37:12.869381+0000","last_unstale":"2024-09-21T03:37:53.046076+0000","last_undegraded":"2024-09-21T03:37:53.046076+0000","last_fullsized":"2024-09-21T03:37:53.046076+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:39:29.437552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"80'3","reported_seq":40,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.044859+0000","last_change":"2024-09-21T03:37:10.834853+0000","last_active":"2024-09-21T03:37:53.044859+0000","last_peered":"2024-09-21T03:37:53.044859+0000","last_clean":"2024-09-21T03:37:53.044859+0000","last_became_active":"2024-09-21T03:37:10.834402+0000","last_became_peered":"2024-09-21T03:37:10.834402+0000","last_unstale":"2024-09-21T03:37:53.044859+0000","last_undegraded":"2024-09-21T03:37:53.044859+0000","last_fullsized":"2024-09-21T03:37:53.044859+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_clean_scrub_stamp":"2024-09-21T03:37:09.809805+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:57:29.668570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"20'92","reported_seq":108,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.430997+0000","last_change":"2024-09-21T03:36:00.521781+0000","last_active":"2024-09-21T03:37:53.430997+0000","last_peered":"2024-09-21T03:37:53.430997+0000","last_clean":"2024-09-21T03:37:53.430997+0000","last_became_active":"2024-09-21T03:36:00.516468+0000","last_became_peered":"2024-09-21T03:36:00.516468+0000","last_unstale":"2024-09-21T03:37:53.430997+0000","last_undegraded":"2024-09-21T03:37:53.430997+0000","last_fullsized":"2024-09-21T03:37:53.430997+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:32:38.215432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:32:38.215432+0000","last_clean_scrub_stamp":"2024-09-21T03:32:38.215432+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:02:16.252544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"82'1","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.431151+0000","last_change":"2024-09-21T03:37:19.911309+0000","last_active":"2024-09-21T03:37:53.431151+0000","last_peered":"2024-09-21T03:37:53.431151+0000","last_clean":"2024-09-21T03:37:53.431151+0000","last_became_active":"2024-09-21T03:37:19.910778+0000","last_became_peered":"2024-09-21T03:37:19.910778+0000","last_unstale":"2024-09-21T03:37:53.431151+0000","last_undegraded":"2024-09-21T03:37:53.431151+0000","last_fullsized":"2024-09-21T03:37:53.431151+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_clean_scrub_stamp":"2024-09-21T03:37:18.876933+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:52:23.307983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.431249+0000","last_change":"2024-09-21T03:37:12.860707+0000","last_active":"2024-09-21T03:37:53.431249+0000","last_peered":"2024-09-21T03:37:53.431249+0000","last_clean":"2024-09-21T03:37:53.431249+0000","last_became_active":"2024-09-21T03:37:12.860341+0000","last_became_peered":"2024-09-21T03:37:12.860341+0000","last_unstale":"2024-09-21T03:37:53.431249+0000","last_undegraded":"2024-09-21T03:37:53.431249+0000","last_fullsized":"2024-09-21T03:37:53.431249+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:55:15.771259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"80'3","reported_seq":18,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.071234+0000","last_change":"2024-09-21T03:37:53.071125+0000","last_active":"2024-09-21T03:37:53.071234+0000","last_peered":"2024-09-21T03:37:53.071234+0000","last_clean":"2024-09-21T03:37:53.071234+0000","last_became_active":"2024-09-21T03:37:53.052237+0000","last_became_peered":"2024-09-21T03:37:53.052237+0000","last_unstale":"2024-09-21T03:37:53.071234+0000","last_undegraded":"2024-09-21T03:37:53.071234+0000","last_fullsized":"2024-09-21T03:37:53.071234+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_clean_scrub_stamp":"2024-09-21T03:37:09.809805+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:47:17.707045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,0],"acting":[4,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.111355+0000","last_change":"2024-09-21T03:37:19.909292+0000","last_active":"2024-09-21T03:37:53.111355+0000","last_peered":"2024-09-21T03:37:53.111355+0000","last_clean":"2024-09-21T03:37:53.111355+0000","last_became_active":"2024-09-21T03:37:19.908681+0000","last_became_peered":"2024-09-21T03:37:19.908681+0000","last_unstale":"2024-09-21T03:37:53.111355+0000","last_undegraded":"2024-09-21T03:37:53.111355+0000","last_fullsized":"2024-09-21T03:37:53.111355+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_clean_scrub_stamp":"2024-09-21T03:37:18.876933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:15:49.623109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.431136+0000","last_change":"2024-09-21T03:37:12.857678+0000","last_active":"2024-09-21T03:37:53.431136+0000","last_peered":"2024-09-21T03:37:53.431136+0000","last_clean":"2024-09-21T03:37:53.431136+0000","last_became_active":"2024-09-21T03:37:12.857266+0000","last_became_peered":"2024-09-21T03:37:12.857266+0000","last_unstale":"2024-09-21T03:37:53.431136+0000","last_undegraded":"2024-09-21T03:37:53.431136+0000","last_fullsized":"2024-09-21T03:37:53.431136+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:21:14.135784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"74'1","reported_seq":37,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.047567+0000","last_change":"2024-09-21T03:37:10.835323+0000","last_active":"2024-09-21T03:37:53.047567+0000","last_peered":"2024-09-21T03:37:53.047567+0000","last_clean":"2024-09-21T03:37:53.047567+0000","last_became_active":"2024-09-21T03:37:10.834950+0000","last_became_peered":"2024-09-21T03:37:10.834950+0000","last_unstale":"2024-09-21T03:37:53.047567+0000","last_undegraded":"2024-09-21T03:37:53.047567+0000","last_fullsized":"2024-09-21T03:37:53.047567+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_clean_scrub_stamp":"2024-09-21T03:37:09.809805+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:40:27.386342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.049010+0000","last_change":"2024-09-21T03:37:19.906360+0000","last_active":"2024-09-21T03:37:53.049010+0000","last_peered":"2024-09-21T03:37:53.049010+0000","last_clean":"2024-09-21T03:37:53.049010+0000","last_became_active":"2024-09-21T03:37:19.905888+0000","last_became_peered":"2024-09-21T03:37:19.905888+0000","last_unstale":"2024-09-21T03:37:53.049010+0000","last_undegraded":"2024-09-21T03:37:53.049010+0000","last_fullsized":"2024-09-21T03:37:53.049010+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_clean_scrub_stamp":"2024-09-21T03:37:18.876933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:13:55.885538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.431442+0000","last_change":"2024-09-21T03:37:12.849106+0000","last_active":"2024-09-21T03:37:53.431442+0000","last_peered":"2024-09-21T03:37:53.431442+0000","last_clean":"2024-09-21T03:37:53.431442+0000","last_became_active":"2024-09-21T03:37:12.848505+0000","last_became_peered":"2024-09-21T03:37:12.848505+0000","last_unstale":"2024-09-21T03:37:53.431442+0000","last_undegraded":"2024-09-21T03:37:53.431442+0000","last_fullsized":"2024-09-21T03:37:53.431442+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:50:19.569004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"74'2","reported_seq":48,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.115238+0000","last_change":"2024-09-21T03:37:53.115158+0000","last_active":"2024-09-21T03:37:53.115238+0000","last_peered":"2024-09-21T03:37:53.115238+0000","last_clean":"2024-09-21T03:37:53.115238+0000","last_became_active":"2024-09-21T03:37:53.112603+0000","last_became_peered":"2024-09-21T03:37:53.112603+0000","last_unstale":"2024-09-21T03:37:53.115238+0000","last_undegraded":"2024-09-21T03:37:53.115238+0000","last_fullsized":"2024-09-21T03:37:53.115238+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_clean_scrub_stamp":"2024-09-21T03:37:09.809805+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:23:21.525972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,1],"acting":[6,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.045614+0000","last_change":"2024-09-21T03:37:19.909040+0000","last_active":"2024-09-21T03:37:53.045614+0000","last_peered":"2024-09-21T03:37:53.045614+0000","last_clean":"2024-09-21T03:37:53.045614+0000","last_became_active":"2024-09-21T03:37:19.908530+0000","last_became_peered":"2024-09-21T03:37:19.908530+0000","last_unstale":"2024-09-21T03:37:53.045614+0000","last_undegraded":"2024-09-21T03:37:53.045614+0000","last_fullsized":"2024-09-21T03:37:53.045614+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_clean_scrub_stamp":"2024-09-21T03:37:18.876933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:35:20.919260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.049078+0000","last_change":"2024-09-21T03:37:12.858984+0000","last_active":"2024-09-21T03:37:53.049078+0000","last_peered":"2024-09-21T03:37:53.049078+0000","last_clean":"2024-09-21T03:37:53.049078+0000","last_became_active":"2024-09-21T03:37:12.858603+0000","last_became_peered":"2024-09-21T03:37:12.858603+0000","last_unstale":"2024-09-21T03:37:53.049078+0000","last_undegraded":"2024-09-21T03:37:53.049078+0000","last_fullsized":"2024-09-21T03:37:53.049078+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:16:45.025304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.f","version":"80'2","reported_seq":38,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.050780+0000","last_change":"2024-09-21T03:37:10.834844+0000","last_active":"2024-09-21T03:37:53.050780+0000","last_peered":"2024-09-21T03:37:53.050780+0000","last_clean":"2024-09-21T03:37:53.050780+0000","last_became_active":"2024-09-21T03:37:10.833664+0000","last_became_peered":"2024-09-21T03:37:10.833664+0000","last_unstale":"2024-09-21T03:37:53.050780+0000","last_undegraded":"2024-09-21T03:37:53.050780+0000","last_fullsized":"2024-09-21T03:37:53.050780+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_clean_scrub_stamp":"2024-09-21T03:37:09.809805+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:03:49.006414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.046180+0000","last_change":"2024-09-21T03:37:19.913277+0000","last_active":"2024-09-21T03:37:53.046180+0000","last_peered":"2024-09-21T03:37:53.046180+0000","last_clean":"2024-09-21T03:37:53.046180+0000","last_became_active":"2024-09-21T03:37:19.912781+0000","last_became_peered":"2024-09-21T03:37:19.912781+0000","last_unstale":"2024-09-21T03:37:53.046180+0000","last_undegraded":"2024-09-21T03:37:53.046180+0000","last_fullsized":"2024-09-21T03:37:53.046180+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_clean_scrub_stamp":"2024-09-21T03:37:18.876933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:18:27.980622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.046359+0000","last_change":"2024-09-21T03:37:12.862322+0000","last_active":"2024-09-21T03:37:53.046359+0000","last_peered":"2024-09-21T03:37:53.046359+0000","last_clean":"2024-09-21T03:37:53.046359+0000","last_became_active":"2024-09-21T03:37:12.861938+0000","last_became_peered":"2024-09-21T03:37:12.861938+0000","last_unstale":"2024-09-21T03:37:53.046359+0000","last_undegraded":"2024-09-21T03:37:53.046359+0000","last_fullsized":"2024-09-21T03:37:53.046359+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:01:48.944169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.10","version":"80'15","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.067663+0000","last_change":"2024-09-21T03:37:53.067130+0000","last_active":"2024-09-21T03:37:53.067663+0000","last_peered":"2024-09-21T03:37:53.067663+0000","last_clean":"2024-09-21T03:37:53.067663+0000","last_became_active":"2024-09-21T03:37:53.051596+0000","last_became_peered":"2024-09-21T03:37:53.051596+0000","last_unstale":"2024-09-21T03:37:53.067663+0000","last_undegraded":"2024-09-21T03:37:53.067663+0000","last_fullsized":"2024-09-21T03:37:53.067663+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_clean_scrub_stamp":"2024-09-21T03:37:09.809805+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:02:22.852015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.043129+0000","last_change":"2024-09-21T03:37:19.913206+0000","last_active":"2024-09-21T03:37:53.043129+0000","last_peered":"2024-09-21T03:37:53.043129+0000","last_clean":"2024-09-21T03:37:53.043129+0000","last_became_active":"2024-09-21T03:37:19.910718+0000","last_became_peered":"2024-09-21T03:37:19.910718+0000","last_unstale":"2024-09-21T03:37:53.043129+0000","last_undegraded":"2024-09-21T03:37:53.043129+0000","last_fullsized":"2024-09-21T03:37:53.043129+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_clean_scrub_stamp":"2024-09-21T03:37:18.876933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:11:31.431237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.430692+0000","last_change":"2024-09-21T03:37:12.857505+0000","last_active":"2024-09-21T03:37:53.430692+0000","last_peered":"2024-09-21T03:37:53.430692+0000","last_clean":"2024-09-21T03:37:53.430692+0000","last_became_active":"2024-09-21T03:37:12.857155+0000","last_became_peered":"2024-09-21T03:37:12.857155+0000","last_unstale":"2024-09-21T03:37:53.430692+0000","last_undegraded":"2024-09-21T03:37:53.430692+0000","last_fullsized":"2024-09-21T03:37:53.430692+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:05:50.387651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"80'3","reported_seq":54,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:38:01.047784+0000","last_change":"2024-09-21T03:37:10.832364+0000","last_active":"2024-09-21T03:38:01.047784+0000","last_peered":"2024-09-21T03:38:01.047784+0000","last_clean":"2024-09-21T03:38:01.047784+0000","last_became_active":"2024-09-21T03:37:10.832004+0000","last_became_peered":"2024-09-21T03:37:10.832004+0000","last_unstale":"2024-09-21T03:38:01.047784+0000","last_undegraded":"2024-09-21T03:38:01.047784+0000","last_fullsized":"2024-09-21T03:38:01.047784+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_clean_scrub_stamp":"2024-09-21T03:37:09.809805+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:21:10.895732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":17,"num_read_kb":3,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.047568+0000","last_change":"2024-09-21T03:37:19.912297+0000","last_active":"2024-09-21T03:37:53.047568+0000","last_peered":"2024-09-21T03:37:53.047568+0000","last_clean":"2024-09-21T03:37:53.047568+0000","last_became_active":"2024-09-21T03:37:19.911903+0000","last_became_peered":"2024-09-21T03:37:19.911903+0000","last_unstale":"2024-09-21T03:37:53.047568+0000","last_undegraded":"2024-09-21T03:37:53.047568+0000","last_fullsized":"2024-09-21T03:37:53.047568+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_clean_scrub_stamp":"2024-09-21T03:37:18.876933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:05:56.747246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.431059+0000","last_change":"2024-09-21T03:37:12.856230+0000","last_active":"2024-09-21T03:37:53.431059+0000","last_peered":"2024-09-21T03:37:53.431059+0000","last_clean":"2024-09-21T03:37:53.431059+0000","last_became_active":"2024-09-21T03:37:12.855819+0000","last_became_peered":"2024-09-21T03:37:12.855819+0000","last_unstale":"2024-09-21T03:37:53.431059+0000","last_undegraded":"2024-09-21T03:37:53.431059+0000","last_fullsized":"2024-09-21T03:37:53.431059+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:38:18.618579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"80'3","reported_seq":38,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.046856+0000","last_change":"2024-09-21T03:37:10.832401+0000","last_active":"2024-09-21T03:37:53.046856+0000","last_peered":"2024-09-21T03:37:53.046856+0000","last_clean":"2024-09-21T03:37:53.046856+0000","last_became_active":"2024-09-21T03:37:10.832033+0000","last_became_peered":"2024-09-21T03:37:10.832033+0000","last_unstale":"2024-09-21T03:37:53.046856+0000","last_undegraded":"2024-09-21T03:37:53.046856+0000","last_fullsized":"2024-09-21T03:37:53.046856+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_clean_scrub_stamp":"2024-09-21T03:37:09.809805+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:25:20.751529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.048479+0000","last_change":"2024-09-21T03:37:19.905825+0000","last_active":"2024-09-21T03:37:53.048479+0000","last_peered":"2024-09-21T03:37:53.048479+0000","last_clean":"2024-09-21T03:37:53.048479+0000","last_became_active":"2024-09-21T03:37:19.905352+0000","last_became_peered":"2024-09-21T03:37:19.905352+0000","last_unstale":"2024-09-21T03:37:53.048479+0000","last_undegraded":"2024-09-21T03:37:53.048479+0000","last_fullsized":"2024-09-21T03:37:53.048479+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_clean_scrub_stamp":"2024-09-21T03:37:18.876933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:25:05.861981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.042684+0000","last_change":"2024-09-21T03:37:12.852006+0000","last_active":"2024-09-21T03:37:53.042684+0000","last_peered":"2024-09-21T03:37:53.042684+0000","last_clean":"2024-09-21T03:37:53.042684+0000","last_became_active":"2024-09-21T03:37:12.851670+0000","last_became_peered":"2024-09-21T03:37:12.851670+0000","last_unstale":"2024-09-21T03:37:53.042684+0000","last_undegraded":"2024-09-21T03:37:53.042684+0000","last_fullsized":"2024-09-21T03:37:53.042684+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:41:50.177349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"80'6","reported_seq":42,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.044608+0000","last_change":"2024-09-21T03:37:10.832472+0000","last_active":"2024-09-21T03:37:53.044608+0000","last_peered":"2024-09-21T03:37:53.044608+0000","last_clean":"2024-09-21T03:37:53.044608+0000","last_became_active":"2024-09-21T03:37:10.832085+0000","last_became_peered":"2024-09-21T03:37:10.832085+0000","last_unstale":"2024-09-21T03:37:53.044608+0000","last_undegraded":"2024-09-21T03:37:53.044608+0000","last_fullsized":"2024-09-21T03:37:53.044608+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_clean_scrub_stamp":"2024-09-21T03:37:09.809805+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:06:41.251218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.046632+0000","last_change":"2024-09-21T03:37:19.914042+0000","last_active":"2024-09-21T03:37:53.046632+0000","last_peered":"2024-09-21T03:37:53.046632+0000","last_clean":"2024-09-21T03:37:53.046632+0000","last_became_active":"2024-09-21T03:37:19.913531+0000","last_became_peered":"2024-09-21T03:37:19.913531+0000","last_unstale":"2024-09-21T03:37:53.046632+0000","last_undegraded":"2024-09-21T03:37:53.046632+0000","last_fullsized":"2024-09-21T03:37:53.046632+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_clean_scrub_stamp":"2024-09-21T03:37:18.876933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:16:40.163359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.050691+0000","last_change":"2024-09-21T03:37:12.852893+0000","last_active":"2024-09-21T03:37:53.050691+0000","last_peered":"2024-09-21T03:37:53.050691+0000","last_clean":"2024-09-21T03:37:53.050691+0000","last_became_active":"2024-09-21T03:37:12.852452+0000","last_became_peered":"2024-09-21T03:37:12.852452+0000","last_unstale":"2024-09-21T03:37:53.050691+0000","last_undegraded":"2024-09-21T03:37:53.050691+0000","last_fullsized":"2024-09-21T03:37:53.050691+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:46:59.393901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.14","version":"82'26","reported_seq":89,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:59.046711+0000","last_change":"2024-09-21T03:37:10.837654+0000","last_active":"2024-09-21T03:37:59.046711+0000","last_peered":"2024-09-21T03:37:59.046711+0000","last_clean":"2024-09-21T03:37:59.046711+0000","last_became_active":"2024-09-21T03:37:10.837251+0000","last_became_peered":"2024-09-21T03:37:10.837251+0000","last_unstale":"2024-09-21T03:37:59.046711+0000","last_undegraded":"2024-09-21T03:37:59.046711+0000","last_fullsized":"2024-09-21T03:37:59.046711+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_clean_scrub_stamp":"2024-09-21T03:37:09.809805+0000","objects_scrubbed":0,"log_size":26,"log_dups_size":0,"ondisk_log_size":26,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:43:32.986371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":53234,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":28,"num_read_kb":66,"num_write":27,"num_write_kb":63,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.053980+0000","last_change":"2024-09-21T03:37:53.053405+0000","last_active":"2024-09-21T03:37:53.053980+0000","last_peered":"2024-09-21T03:37:53.053980+0000","last_clean":"2024-09-21T03:37:53.053980+0000","last_became_active":"2024-09-21T03:37:53.052818+0000","last_became_peered":"2024-09-21T03:37:53.052818+0000","last_unstale":"2024-09-21T03:37:53.053980+0000","last_undegraded":"2024-09-21T03:37:53.053980+0000","last_fullsized":"2024-09-21T03:37:53.053980+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_clean_scrub_stamp":"2024-09-21T03:37:18.876933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:05:09.121036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,11,5],"acting":[10,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.111585+0000","last_change":"2024-09-21T03:37:12.866588+0000","last_active":"2024-09-21T03:37:53.111585+0000","last_peered":"2024-09-21T03:37:53.111585+0000","last_clean":"2024-09-21T03:37:53.111585+0000","last_became_active":"2024-09-21T03:37:12.866075+0000","last_became_peered":"2024-09-21T03:37:12.866075+0000","last_unstale":"2024-09-21T03:37:53.111585+0000","last_undegraded":"2024-09-21T03:37:53.111585+0000","last_fullsized":"2024-09-21T03:37:53.111585+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:18:48.817562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"80'4","reported_seq":40,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.047298+0000","last_change":"2024-09-21T03:37:11.036751+0000","last_active":"2024-09-21T03:37:53.047298+0000","last_peered":"2024-09-21T03:37:53.047298+0000","last_clean":"2024-09-21T03:37:53.047298+0000","last_became_active":"2024-09-21T03:37:11.036125+0000","last_became_peered":"2024-09-21T03:37:11.036125+0000","last_unstale":"2024-09-21T03:37:53.047298+0000","last_undegraded":"2024-09-21T03:37:53.047298+0000","last_fullsized":"2024-09-21T03:37:53.047298+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_clean_scrub_stamp":"2024-09-21T03:37:09.809805+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:46:13.569422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.112980+0000","last_change":"2024-09-21T03:37:53.112845+0000","last_active":"2024-09-21T03:37:53.112980+0000","last_peered":"2024-09-21T03:37:53.112980+0000","last_clean":"2024-09-21T03:37:53.112980+0000","last_became_active":"2024-09-21T03:37:53.112286+0000","last_became_peered":"2024-09-21T03:37:53.112286+0000","last_unstale":"2024-09-21T03:37:53.112980+0000","last_undegraded":"2024-09-21T03:37:53.112980+0000","last_fullsized":"2024-09-21T03:37:53.112980+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_clean_scrub_stamp":"2024-09-21T03:37:18.876933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:29:58.062684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,8],"acting":[4,1,2,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.049603+0000","last_change":"2024-09-21T03:37:12.865011+0000","last_active":"2024-09-21T03:37:53.049603+0000","last_peered":"2024-09-21T03:37:53.049603+0000","last_clean":"2024-09-21T03:37:53.049603+0000","last_became_active":"2024-09-21T03:37:12.864645+0000","last_became_peered":"2024-09-21T03:37:12.864645+0000","last_unstale":"2024-09-21T03:37:53.049603+0000","last_undegraded":"2024-09-21T03:37:53.049603+0000","last_fullsized":"2024-09-21T03:37:53.049603+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:27:42.471227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"80'2","reported_seq":37,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.431360+0000","last_change":"2024-09-21T03:37:10.835171+0000","last_active":"2024-09-21T03:37:53.431360+0000","last_peered":"2024-09-21T03:37:53.431360+0000","last_clean":"2024-09-21T03:37:53.431360+0000","last_became_active":"2024-09-21T03:37:10.834744+0000","last_became_peered":"2024-09-21T03:37:10.834744+0000","last_unstale":"2024-09-21T03:37:53.431360+0000","last_undegraded":"2024-09-21T03:37:53.431360+0000","last_fullsized":"2024-09-21T03:37:53.431360+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_clean_scrub_stamp":"2024-09-21T03:37:09.809805+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:02:37.720457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.046301+0000","last_change":"2024-09-21T03:37:19.906734+0000","last_active":"2024-09-21T03:37:53.046301+0000","last_peered":"2024-09-21T03:37:53.046301+0000","last_clean":"2024-09-21T03:37:53.046301+0000","last_became_active":"2024-09-21T03:37:19.906327+0000","last_became_peered":"2024-09-21T03:37:19.906327+0000","last_unstale":"2024-09-21T03:37:53.046301+0000","last_undegraded":"2024-09-21T03:37:53.046301+0000","last_fullsized":"2024-09-21T03:37:53.046301+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_clean_scrub_stamp":"2024-09-21T03:37:18.876933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:55:11.603042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.111948+0000","last_change":"2024-09-21T03:37:12.866285+0000","last_active":"2024-09-21T03:37:53.111948+0000","last_peered":"2024-09-21T03:37:53.111948+0000","last_clean":"2024-09-21T03:37:53.111948+0000","last_became_active":"2024-09-21T03:37:12.865854+0000","last_became_peered":"2024-09-21T03:37:12.865854+0000","last_unstale":"2024-09-21T03:37:53.111948+0000","last_undegraded":"2024-09-21T03:37:53.111948+0000","last_fullsized":"2024-09-21T03:37:53.111948+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:00:18.075248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"80'3","reported_seq":49,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:58.042911+0000","last_change":"2024-09-21T03:37:10.835808+0000","last_active":"2024-09-21T03:37:58.042911+0000","last_peered":"2024-09-21T03:37:58.042911+0000","last_clean":"2024-09-21T03:37:58.042911+0000","last_became_active":"2024-09-21T03:37:10.835434+0000","last_became_peered":"2024-09-21T03:37:10.835434+0000","last_unstale":"2024-09-21T03:37:58.042911+0000","last_undegraded":"2024-09-21T03:37:58.042911+0000","last_fullsized":"2024-09-21T03:37:58.042911+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_clean_scrub_stamp":"2024-09-21T03:37:09.809805+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:35:39.817312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":11,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.045411+0000","last_change":"2024-09-21T03:37:19.915986+0000","last_active":"2024-09-21T03:37:53.045411+0000","last_peered":"2024-09-21T03:37:53.045411+0000","last_clean":"2024-09-21T03:37:53.045411+0000","last_became_active":"2024-09-21T03:37:19.915561+0000","last_became_peered":"2024-09-21T03:37:19.915561+0000","last_unstale":"2024-09-21T03:37:53.045411+0000","last_undegraded":"2024-09-21T03:37:53.045411+0000","last_fullsized":"2024-09-21T03:37:53.045411+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_clean_scrub_stamp":"2024-09-21T03:37:18.876933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:21:13.789664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.049328+0000","last_change":"2024-09-21T03:37:19.908230+0000","last_active":"2024-09-21T03:37:53.049328+0000","last_peered":"2024-09-21T03:37:53.049328+0000","last_clean":"2024-09-21T03:37:53.049328+0000","last_became_active":"2024-09-21T03:37:19.907843+0000","last_became_peered":"2024-09-21T03:37:19.907843+0000","last_unstale":"2024-09-21T03:37:53.049328+0000","last_undegraded":"2024-09-21T03:37:53.049328+0000","last_fullsized":"2024-09-21T03:37:53.049328+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_clean_scrub_stamp":"2024-09-21T03:37:18.876933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:15:58.142127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.046673+0000","last_change":"2024-09-21T03:37:12.856709+0000","last_active":"2024-09-21T03:37:53.046673+0000","last_peered":"2024-09-21T03:37:53.046673+0000","last_clean":"2024-09-21T03:37:53.046673+0000","last_became_active":"2024-09-21T03:37:12.856278+0000","last_became_peered":"2024-09-21T03:37:12.856278+0000","last_unstale":"2024-09-21T03:37:53.046673+0000","last_undegraded":"2024-09-21T03:37:53.046673+0000","last_fullsized":"2024-09-21T03:37:53.046673+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:16:13.884648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"80'1","reported_seq":36,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.046329+0000","last_change":"2024-09-21T03:37:10.832456+0000","last_active":"2024-09-21T03:37:53.046329+0000","last_peered":"2024-09-21T03:37:53.046329+0000","last_clean":"2024-09-21T03:37:53.046329+0000","last_became_active":"2024-09-21T03:37:10.832082+0000","last_became_peered":"2024-09-21T03:37:10.832082+0000","last_unstale":"2024-09-21T03:37:53.046329+0000","last_undegraded":"2024-09-21T03:37:53.046329+0000","last_fullsized":"2024-09-21T03:37:53.046329+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_clean_scrub_stamp":"2024-09-21T03:37:09.809805+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:17:12.170616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.048472+0000","last_change":"2024-09-21T03:37:19.908084+0000","last_active":"2024-09-21T03:37:53.048472+0000","last_peered":"2024-09-21T03:37:53.048472+0000","last_clean":"2024-09-21T03:37:53.048472+0000","last_became_active":"2024-09-21T03:37:19.907307+0000","last_became_peered":"2024-09-21T03:37:19.907307+0000","last_unstale":"2024-09-21T03:37:53.048472+0000","last_undegraded":"2024-09-21T03:37:53.048472+0000","last_fullsized":"2024-09-21T03:37:53.048472+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_clean_scrub_stamp":"2024-09-21T03:37:18.876933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:45:02.541015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.045568+0000","last_change":"2024-09-21T03:37:12.852577+0000","last_active":"2024-09-21T03:37:53.045568+0000","last_peered":"2024-09-21T03:37:53.045568+0000","last_clean":"2024-09-21T03:37:53.045568+0000","last_became_active":"2024-09-21T03:37:12.852186+0000","last_became_peered":"2024-09-21T03:37:12.852186+0000","last_unstale":"2024-09-21T03:37:53.045568+0000","last_undegraded":"2024-09-21T03:37:53.045568+0000","last_fullsized":"2024-09-21T03:37:53.045568+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:51:21.631029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"80'2","reported_seq":37,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.042546+0000","last_change":"2024-09-21T03:37:10.833587+0000","last_active":"2024-09-21T03:37:53.042546+0000","last_peered":"2024-09-21T03:37:53.042546+0000","last_clean":"2024-09-21T03:37:53.042546+0000","last_became_active":"2024-09-21T03:37:10.833194+0000","last_became_peered":"2024-09-21T03:37:10.833194+0000","last_unstale":"2024-09-21T03:37:53.042546+0000","last_undegraded":"2024-09-21T03:37:53.042546+0000","last_fullsized":"2024-09-21T03:37:53.042546+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_clean_scrub_stamp":"2024-09-21T03:37:09.809805+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:20:20.272108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"80'3","reported_seq":38,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.047496+0000","last_change":"2024-09-21T03:37:10.839103+0000","last_active":"2024-09-21T03:37:53.047496+0000","last_peered":"2024-09-21T03:37:53.047496+0000","last_clean":"2024-09-21T03:37:53.047496+0000","last_became_active":"2024-09-21T03:37:10.837192+0000","last_became_peered":"2024-09-21T03:37:10.837192+0000","last_unstale":"2024-09-21T03:37:53.047496+0000","last_undegraded":"2024-09-21T03:37:53.047496+0000","last_fullsized":"2024-09-21T03:37:53.047496+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_clean_scrub_stamp":"2024-09-21T03:37:09.809805+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:59:35.128171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.047232+0000","last_change":"2024-09-21T03:37:12.866626+0000","last_active":"2024-09-21T03:37:53.047232+0000","last_peered":"2024-09-21T03:37:53.047232+0000","last_clean":"2024-09-21T03:37:53.047232+0000","last_became_active":"2024-09-21T03:37:12.866254+0000","last_became_peered":"2024-09-21T03:37:12.866254+0000","last_unstale":"2024-09-21T03:37:53.047232+0000","last_undegraded":"2024-09-21T03:37:53.047232+0000","last_fullsized":"2024-09-21T03:37:53.047232+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:39:00.361254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.044954+0000","last_change":"2024-09-21T03:37:19.909415+0000","last_active":"2024-09-21T03:37:53.044954+0000","last_peered":"2024-09-21T03:37:53.044954+0000","last_clean":"2024-09-21T03:37:53.044954+0000","last_became_active":"2024-09-21T03:37:19.908955+0000","last_became_peered":"2024-09-21T03:37:19.908955+0000","last_unstale":"2024-09-21T03:37:53.044954+0000","last_undegraded":"2024-09-21T03:37:53.044954+0000","last_fullsized":"2024-09-21T03:37:53.044954+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_clean_scrub_stamp":"2024-09-21T03:37:18.876933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:31:18.878725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"81'6","reported_seq":57,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:38:00.047096+0000","last_change":"2024-09-21T03:37:10.836389+0000","last_active":"2024-09-21T03:38:00.047096+0000","last_peered":"2024-09-21T03:38:00.047096+0000","last_clean":"2024-09-21T03:38:00.047096+0000","last_became_active":"2024-09-21T03:37:10.835986+0000","last_became_peered":"2024-09-21T03:37:10.835986+0000","last_unstale":"2024-09-21T03:38:00.047096+0000","last_undegraded":"2024-09-21T03:38:00.047096+0000","last_fullsized":"2024-09-21T03:38:00.047096+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_clean_scrub_stamp":"2024-09-21T03:37:09.809805+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:05:17.963609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1013,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":16,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.050861+0000","last_change":"2024-09-21T03:37:12.856008+0000","last_active":"2024-09-21T03:37:53.050861+0000","last_peered":"2024-09-21T03:37:53.050861+0000","last_clean":"2024-09-21T03:37:53.050861+0000","last_became_active":"2024-09-21T03:37:12.855603+0000","last_became_peered":"2024-09-21T03:37:12.855603+0000","last_unstale":"2024-09-21T03:37:53.050861+0000","last_undegraded":"2024-09-21T03:37:53.050861+0000","last_fullsized":"2024-09-21T03:37:53.050861+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:11:41.920114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.045051+0000","last_change":"2024-09-21T03:37:19.911507+0000","last_active":"2024-09-21T03:37:53.045051+0000","last_peered":"2024-09-21T03:37:53.045051+0000","last_clean":"2024-09-21T03:37:53.045051+0000","last_became_active":"2024-09-21T03:37:19.911053+0000","last_became_peered":"2024-09-21T03:37:19.911053+0000","last_unstale":"2024-09-21T03:37:53.045051+0000","last_undegraded":"2024-09-21T03:37:53.045051+0000","last_fullsized":"2024-09-21T03:37:53.045051+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_clean_scrub_stamp":"2024-09-21T03:37:18.876933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:42:11.964829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.048840+0000","last_change":"2024-09-21T03:37:12.862576+0000","last_active":"2024-09-21T03:37:53.048840+0000","last_peered":"2024-09-21T03:37:53.048840+0000","last_clean":"2024-09-21T03:37:53.048840+0000","last_became_active":"2024-09-21T03:37:12.862146+0000","last_became_peered":"2024-09-21T03:37:12.862146+0000","last_unstale":"2024-09-21T03:37:53.048840+0000","last_undegraded":"2024-09-21T03:37:53.048840+0000","last_fullsized":"2024-09-21T03:37:53.048840+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:30:33.272415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.044731+0000","last_change":"2024-09-21T03:37:12.859395+0000","last_active":"2024-09-21T03:37:53.044731+0000","last_peered":"2024-09-21T03:37:53.044731+0000","last_clean":"2024-09-21T03:37:53.044731+0000","last_became_active":"2024-09-21T03:37:12.858914+0000","last_became_peered":"2024-09-21T03:37:12.858914+0000","last_unstale":"2024-09-21T03:37:53.044731+0000","last_undegraded":"2024-09-21T03:37:53.044731+0000","last_fullsized":"2024-09-21T03:37:53.044731+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:13:06.319896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.044203+0000","last_change":"2024-09-21T03:37:12.857467+0000","last_active":"2024-09-21T03:37:53.044203+0000","last_peered":"2024-09-21T03:37:53.044203+0000","last_clean":"2024-09-21T03:37:53.044203+0000","last_became_active":"2024-09-21T03:37:12.856214+0000","last_became_peered":"2024-09-21T03:37:12.856214+0000","last_unstale":"2024-09-21T03:37:53.044203+0000","last_undegraded":"2024-09-21T03:37:53.044203+0000","last_fullsized":"2024-09-21T03:37:53.044203+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:21:58.780018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.049029+0000","last_change":"2024-09-21T03:37:12.858052+0000","last_active":"2024-09-21T03:37:53.049029+0000","last_peered":"2024-09-21T03:37:53.049029+0000","last_clean":"2024-09-21T03:37:53.049029+0000","last_became_active":"2024-09-21T03:37:12.857445+0000","last_became_peered":"2024-09-21T03:37:12.857445+0000","last_unstale":"2024-09-21T03:37:53.049029+0000","last_undegraded":"2024-09-21T03:37:53.049029+0000","last_fullsized":"2024-09-21T03:37:53.049029+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:22:28.058846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.042880+0000","last_change":"2024-09-21T03:37:12.856499+0000","last_active":"2024-09-21T03:37:53.042880+0000","last_peered":"2024-09-21T03:37:53.042880+0000","last_clean":"2024-09-21T03:37:53.042880+0000","last_became_active":"2024-09-21T03:37:12.856146+0000","last_became_peered":"2024-09-21T03:37:12.856146+0000","last_unstale":"2024-09-21T03:37:53.042880+0000","last_undegraded":"2024-09-21T03:37:53.042880+0000","last_fullsized":"2024-09-21T03:37:53.042880+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:03:26.142740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.048901+0000","last_change":"2024-09-21T03:37:12.860830+0000","last_active":"2024-09-21T03:37:53.048901+0000","last_peered":"2024-09-21T03:37:53.048901+0000","last_clean":"2024-09-21T03:37:53.048901+0000","last_became_active":"2024-09-21T03:37:12.860468+0000","last_became_peered":"2024-09-21T03:37:12.860468+0000","last_unstale":"2024-09-21T03:37:53.048901+0000","last_undegraded":"2024-09-21T03:37:53.048901+0000","last_fullsized":"2024-09-21T03:37:53.048901+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:48:15.057622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.046546+0000","last_change":"2024-09-21T03:37:12.864399+0000","last_active":"2024-09-21T03:37:53.046546+0000","last_peered":"2024-09-21T03:37:53.046546+0000","last_clean":"2024-09-21T03:37:53.046546+0000","last_became_active":"2024-09-21T03:37:12.863892+0000","last_became_peered":"2024-09-21T03:37:12.863892+0000","last_unstale":"2024-09-21T03:37:53.046546+0000","last_undegraded":"2024-09-21T03:37:53.046546+0000","last_fullsized":"2024-09-21T03:37:53.046546+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:58:26.647896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.046313+0000","last_change":"2024-09-21T03:37:12.861716+0000","last_active":"2024-09-21T03:37:53.046313+0000","last_peered":"2024-09-21T03:37:53.046313+0000","last_clean":"2024-09-21T03:37:53.046313+0000","last_became_active":"2024-09-21T03:37:12.861392+0000","last_became_peered":"2024-09-21T03:37:12.861392+0000","last_unstale":"2024-09-21T03:37:53.046313+0000","last_undegraded":"2024-09-21T03:37:53.046313+0000","last_fullsized":"2024-09-21T03:37:53.046313+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:28:55.751107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.045975+0000","last_change":"2024-09-21T03:37:12.856167+0000","last_active":"2024-09-21T03:37:53.045975+0000","last_peered":"2024-09-21T03:37:53.045975+0000","last_clean":"2024-09-21T03:37:53.045975+0000","last_became_active":"2024-09-21T03:37:12.855752+0000","last_became_peered":"2024-09-21T03:37:12.855752+0000","last_unstale":"2024-09-21T03:37:53.045975+0000","last_undegraded":"2024-09-21T03:37:53.045975+0000","last_fullsized":"2024-09-21T03:37:53.045975+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:26:30.271342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.045615+0000","last_change":"2024-09-21T03:37:12.862782+0000","last_active":"2024-09-21T03:37:53.045615+0000","last_peered":"2024-09-21T03:37:53.045615+0000","last_clean":"2024-09-21T03:37:53.045615+0000","last_became_active":"2024-09-21T03:37:12.862462+0000","last_became_peered":"2024-09-21T03:37:12.862462+0000","last_unstale":"2024-09-21T03:37:53.045615+0000","last_undegraded":"2024-09-21T03:37:53.045615+0000","last_fullsized":"2024-09-21T03:37:53.045615+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:57:37.744531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.111623+0000","last_change":"2024-09-21T03:37:12.862312+0000","last_active":"2024-09-21T03:37:53.111623+0000","last_peered":"2024-09-21T03:37:53.111623+0000","last_clean":"2024-09-21T03:37:53.111623+0000","last_became_active":"2024-09-21T03:37:12.861940+0000","last_became_peered":"2024-09-21T03:37:12.861940+0000","last_unstale":"2024-09-21T03:37:53.111623+0000","last_undegraded":"2024-09-21T03:37:53.111623+0000","last_fullsized":"2024-09-21T03:37:53.111623+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:08:49.165522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.042830+0000","last_change":"2024-09-21T03:37:12.857012+0000","last_active":"2024-09-21T03:37:53.042830+0000","last_peered":"2024-09-21T03:37:53.042830+0000","last_clean":"2024-09-21T03:37:53.042830+0000","last_became_active":"2024-09-21T03:37:12.856390+0000","last_became_peered":"2024-09-21T03:37:12.856390+0000","last_unstale":"2024-09-21T03:37:53.042830+0000","last_undegraded":"2024-09-21T03:37:53.042830+0000","last_fullsized":"2024-09-21T03:37:53.042830+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:51:00.839709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.048487+0000","last_change":"2024-09-21T03:37:12.857207+0000","last_active":"2024-09-21T03:37:53.048487+0000","last_peered":"2024-09-21T03:37:53.048487+0000","last_clean":"2024-09-21T03:37:53.048487+0000","last_became_active":"2024-09-21T03:37:12.856770+0000","last_became_peered":"2024-09-21T03:37:12.856770+0000","last_unstale":"2024-09-21T03:37:53.048487+0000","last_undegraded":"2024-09-21T03:37:53.048487+0000","last_fullsized":"2024-09-21T03:37:53.048487+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:37:18.272321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.045688+0000","last_change":"2024-09-21T03:37:12.858540+0000","last_active":"2024-09-21T03:37:53.045688+0000","last_peered":"2024-09-21T03:37:53.045688+0000","last_clean":"2024-09-21T03:37:53.045688+0000","last_became_active":"2024-09-21T03:37:12.857340+0000","last_became_peered":"2024-09-21T03:37:12.857340+0000","last_unstale":"2024-09-21T03:37:53.045688+0000","last_undegraded":"2024-09-21T03:37:53.045688+0000","last_fullsized":"2024-09-21T03:37:53.045688+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:45:48.982132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.047908+0000","last_change":"2024-09-21T03:37:12.864408+0000","last_active":"2024-09-21T03:37:53.047908+0000","last_peered":"2024-09-21T03:37:53.047908+0000","last_clean":"2024-09-21T03:37:53.047908+0000","last_became_active":"2024-09-21T03:37:12.864082+0000","last_became_peered":"2024-09-21T03:37:12.864082+0000","last_unstale":"2024-09-21T03:37:53.047908+0000","last_undegraded":"2024-09-21T03:37:53.047908+0000","last_fullsized":"2024-09-21T03:37:53.047908+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:19:32.723589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.049781+0000","last_change":"2024-09-21T03:37:12.869445+0000","last_active":"2024-09-21T03:37:53.049781+0000","last_peered":"2024-09-21T03:37:53.049781+0000","last_clean":"2024-09-21T03:37:53.049781+0000","last_became_active":"2024-09-21T03:37:12.869058+0000","last_became_peered":"2024-09-21T03:37:12.869058+0000","last_unstale":"2024-09-21T03:37:53.049781+0000","last_undegraded":"2024-09-21T03:37:53.049781+0000","last_fullsized":"2024-09-21T03:37:53.049781+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:37:57.750665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.111172+0000","last_change":"2024-09-21T03:37:12.858431+0000","last_active":"2024-09-21T03:37:53.111172+0000","last_peered":"2024-09-21T03:37:53.111172+0000","last_clean":"2024-09-21T03:37:53.111172+0000","last_became_active":"2024-09-21T03:37:12.858047+0000","last_became_peered":"2024-09-21T03:37:12.858047+0000","last_unstale":"2024-09-21T03:37:53.111172+0000","last_undegraded":"2024-09-21T03:37:53.111172+0000","last_fullsized":"2024-09-21T03:37:53.111172+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:14:05.363743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.048838+0000","last_change":"2024-09-21T03:37:12.866784+0000","last_active":"2024-09-21T03:37:53.048838+0000","last_peered":"2024-09-21T03:37:53.048838+0000","last_clean":"2024-09-21T03:37:53.048838+0000","last_became_active":"2024-09-21T03:37:12.866435+0000","last_became_peered":"2024-09-21T03:37:12.866435+0000","last_unstale":"2024-09-21T03:37:53.048838+0000","last_undegraded":"2024-09-21T03:37:53.048838+0000","last_fullsized":"2024-09-21T03:37:53.048838+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:09:06.156030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.048908+0000","last_change":"2024-09-21T03:37:12.857054+0000","last_active":"2024-09-21T03:37:53.048908+0000","last_peered":"2024-09-21T03:37:53.048908+0000","last_clean":"2024-09-21T03:37:53.048908+0000","last_became_active":"2024-09-21T03:37:12.856671+0000","last_became_peered":"2024-09-21T03:37:12.856671+0000","last_unstale":"2024-09-21T03:37:53.048908+0000","last_undegraded":"2024-09-21T03:37:53.048908+0000","last_fullsized":"2024-09-21T03:37:53.048908+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:38:27.474550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.042778+0000","last_change":"2024-09-21T03:37:12.861501+0000","last_active":"2024-09-21T03:37:53.042778+0000","last_peered":"2024-09-21T03:37:53.042778+0000","last_clean":"2024-09-21T03:37:53.042778+0000","last_became_active":"2024-09-21T03:37:12.861146+0000","last_became_peered":"2024-09-21T03:37:12.861146+0000","last_unstale":"2024-09-21T03:37:53.042778+0000","last_undegraded":"2024-09-21T03:37:53.042778+0000","last_fullsized":"2024-09-21T03:37:53.042778+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:30:55.073854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.052333+0000","last_change":"2024-09-21T03:37:53.051912+0000","last_active":"2024-09-21T03:37:53.052333+0000","last_peered":"2024-09-21T03:37:53.052333+0000","last_clean":"2024-09-21T03:37:53.052333+0000","last_became_active":"2024-09-21T03:37:53.051640+0000","last_became_peered":"2024-09-21T03:37:53.051640+0000","last_unstale":"2024-09-21T03:37:53.052333+0000","last_undegraded":"2024-09-21T03:37:53.052333+0000","last_fullsized":"2024-09-21T03:37:53.052333+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:08:09.046867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,10,8],"acting":[9,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.431260+0000","last_change":"2024-09-21T03:37:12.865862+0000","last_active":"2024-09-21T03:37:53.431260+0000","last_peered":"2024-09-21T03:37:53.431260+0000","last_clean":"2024-09-21T03:37:53.431260+0000","last_became_active":"2024-09-21T03:37:12.865468+0000","last_became_peered":"2024-09-21T03:37:12.865468+0000","last_unstale":"2024-09-21T03:37:53.431260+0000","last_undegraded":"2024-09-21T03:37:53.431260+0000","last_fullsized":"2024-09-21T03:37:53.431260+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:18:14.496495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.047575+0000","last_change":"2024-09-21T03:37:12.862688+0000","last_active":"2024-09-21T03:37:53.047575+0000","last_peered":"2024-09-21T03:37:53.047575+0000","last_clean":"2024-09-21T03:37:53.047575+0000","last_became_active":"2024-09-21T03:37:12.862249+0000","last_became_peered":"2024-09-21T03:37:12.862249+0000","last_unstale":"2024-09-21T03:37:53.047575+0000","last_undegraded":"2024-09-21T03:37:53.047575+0000","last_fullsized":"2024-09-21T03:37:53.047575+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:55:54.731640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.112095+0000","last_change":"2024-09-21T03:37:12.863595+0000","last_active":"2024-09-21T03:37:53.112095+0000","last_peered":"2024-09-21T03:37:53.112095+0000","last_clean":"2024-09-21T03:37:53.112095+0000","last_became_active":"2024-09-21T03:37:12.863260+0000","last_became_peered":"2024-09-21T03:37:12.863260+0000","last_unstale":"2024-09-21T03:37:53.112095+0000","last_undegraded":"2024-09-21T03:37:53.112095+0000","last_fullsized":"2024-09-21T03:37:53.112095+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:10:07.995690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.045457+0000","last_change":"2024-09-21T03:37:12.865770+0000","last_active":"2024-09-21T03:37:53.045457+0000","last_peered":"2024-09-21T03:37:53.045457+0000","last_clean":"2024-09-21T03:37:53.045457+0000","last_became_active":"2024-09-21T03:37:12.865405+0000","last_became_peered":"2024-09-21T03:37:12.865405+0000","last_unstale":"2024-09-21T03:37:53.045457+0000","last_undegraded":"2024-09-21T03:37:53.045457+0000","last_fullsized":"2024-09-21T03:37:53.045457+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:46:25.255913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.430807+0000","last_change":"2024-09-21T03:37:12.856214+0000","last_active":"2024-09-21T03:37:53.430807+0000","last_peered":"2024-09-21T03:37:53.430807+0000","last_clean":"2024-09-21T03:37:53.430807+0000","last_became_active":"2024-09-21T03:37:12.853924+0000","last_became_peered":"2024-09-21T03:37:12.853924+0000","last_unstale":"2024-09-21T03:37:53.430807+0000","last_undegraded":"2024-09-21T03:37:53.430807+0000","last_fullsized":"2024-09-21T03:37:53.430807+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:10:32.182892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.047154+0000","last_change":"2024-09-21T03:37:12.858851+0000","last_active":"2024-09-21T03:37:53.047154+0000","last_peered":"2024-09-21T03:37:53.047154+0000","last_clean":"2024-09-21T03:37:53.047154+0000","last_became_active":"2024-09-21T03:37:12.858037+0000","last_became_peered":"2024-09-21T03:37:12.858037+0000","last_unstale":"2024-09-21T03:37:53.047154+0000","last_undegraded":"2024-09-21T03:37:53.047154+0000","last_fullsized":"2024-09-21T03:37:53.047154+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:32:23.134789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.051281+0000","last_change":"2024-09-21T03:37:12.852339+0000","last_active":"2024-09-21T03:37:53.051281+0000","last_peered":"2024-09-21T03:37:53.051281+0000","last_clean":"2024-09-21T03:37:53.051281+0000","last_became_active":"2024-09-21T03:37:12.851911+0000","last_became_peered":"2024-09-21T03:37:12.851911+0000","last_unstale":"2024-09-21T03:37:53.051281+0000","last_undegraded":"2024-09-21T03:37:53.051281+0000","last_fullsized":"2024-09-21T03:37:53.051281+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:03:30.739275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.044201+0000","last_change":"2024-09-21T03:37:12.861133+0000","last_active":"2024-09-21T03:37:53.044201+0000","last_peered":"2024-09-21T03:37:53.044201+0000","last_clean":"2024-09-21T03:37:53.044201+0000","last_became_active":"2024-09-21T03:37:12.860719+0000","last_became_peered":"2024-09-21T03:37:12.860719+0000","last_unstale":"2024-09-21T03:37:53.044201+0000","last_undegraded":"2024-09-21T03:37:53.044201+0000","last_fullsized":"2024-09-21T03:37:53.044201+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:21:25.241935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.049308+0000","last_change":"2024-09-21T03:37:12.865694+0000","last_active":"2024-09-21T03:37:53.049308+0000","last_peered":"2024-09-21T03:37:53.049308+0000","last_clean":"2024-09-21T03:37:53.049308+0000","last_became_active":"2024-09-21T03:37:12.864983+0000","last_became_peered":"2024-09-21T03:37:12.864983+0000","last_unstale":"2024-09-21T03:37:53.049308+0000","last_undegraded":"2024-09-21T03:37:53.049308+0000","last_fullsized":"2024-09-21T03:37:53.049308+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:52:28.583056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.049444+0000","last_change":"2024-09-21T03:37:12.860665+0000","last_active":"2024-09-21T03:37:53.049444+0000","last_peered":"2024-09-21T03:37:53.049444+0000","last_clean":"2024-09-21T03:37:53.049444+0000","last_became_active":"2024-09-21T03:37:12.860316+0000","last_became_peered":"2024-09-21T03:37:12.860316+0000","last_unstale":"2024-09-21T03:37:53.049444+0000","last_undegraded":"2024-09-21T03:37:53.049444+0000","last_fullsized":"2024-09-21T03:37:53.049444+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:13:07.316422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.048840+0000","last_change":"2024-09-21T03:37:12.869577+0000","last_active":"2024-09-21T03:37:53.048840+0000","last_peered":"2024-09-21T03:37:53.048840+0000","last_clean":"2024-09-21T03:37:53.048840+0000","last_became_active":"2024-09-21T03:37:12.869164+0000","last_became_peered":"2024-09-21T03:37:12.869164+0000","last_unstale":"2024-09-21T03:37:53.048840+0000","last_undegraded":"2024-09-21T03:37:53.048840+0000","last_fullsized":"2024-09-21T03:37:53.048840+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:40:34.167005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":57235,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":94,"num_read_kb":105,"num_write":154,"num_write_kb":102,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":694,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":393216,"data_stored":171705,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":133,"ondisk_log_size":133,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":285501,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":68,"seq":292057776150,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32072,"kb_used_data":884,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713080,"statfs":{"total":95995035648,"available":95962193920,"internally_reserved":0,"allocated":905216,"data_stored":461385,"data_compressed":23469,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":63,"seq":270582939675,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36352,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708800,"statfs":{"total":95995035648,"available":95957811200,"internally_reserved":0,"allocated":1032192,"data_stored":1101775,"data_compressed":59763,"data_compressed_allocated":77824,"data_compressed_original":655360,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":58,"seq":249108103201,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27920,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717232,"statfs":{"total":95995035648,"available":95966445568,"internally_reserved":0,"allocated":917504,"data_stored":465605,"data_compressed":23469,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":52,"seq":223338299429,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27908,"kb_used_data":884,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717244,"statfs":{"total":95995035648,"available":95966457856,"internally_reserved":0,"allocated":905216,"data_stored":461317,"data_compressed":23469,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":46,"seq":197568495659,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36284,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708868,"statfs":{"total":95995035648,"available":95957880832,"internally_reserved":0,"allocated":962560,"data_stored":1057338,"data_compressed":59763,"data_compressed_allocated":77824,"data_compressed_original":655360,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":41,"seq":176093659183,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32064,"kb_used_data":872,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713088,"statfs":{"total":95995035648,"available":95962202112,"internally_reserved":0,"allocated":892928,"data_stored":456685,"data_compressed":23469,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":35,"seq":150323855412,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32140,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713012,"statfs":{"total":95995035648,"available":95962124288,"internally_reserved":0,"allocated":970752,"data_stored":1052926,"data_compressed":59763,"data_compressed_allocated":77824,"data_compressed_original":655360,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":30,"seq":128849018937,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27912,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717240,"statfs":{"total":95995035648,"available":95966453760,"internally_reserved":0,"allocated":909312,"data_stored":461917,"data_compressed":23469,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":24,"seq":103079215165,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27908,"kb_used_data":884,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717244,"statfs":{"total":95995035648,"available":95966457856,"internally_reserved":0,"allocated":905216,"data_stored":456879,"data_compressed":23469,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":17,"seq":73014444100,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27972,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717180,"statfs":{"total":95995035648,"available":95966392320,"internally_reserved":0,"allocated":970752,"data_stored":510988,"data_compressed":23469,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574920,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36272,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708880,"statfs":{"total":95995035648,"available":95957893120,"internally_reserved":0,"allocated":950272,"data_stored":509829,"data_compressed":23469,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738446,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32084,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713068,"statfs":{"total":95995035648,"available":95962181632,"internally_reserved":0,"allocated":917504,"data_stored":458073,"data_compressed":23469,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34677,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34677,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34677,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36294,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36294,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36294,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36294,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36294,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":53234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":54393,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1226,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1867,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2183,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":626,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":54812,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T03:38:04.984 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 7c374098-77c9-11ef-baf5-efdc52797490 -- ceph pg dump --format=json 2024-09-21T03:38:05.231 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/7c374098-77c9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:38:05.311 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:38:05 smithi089 ceph-mon[30928]: pgmap v318: 129 pgs: 129 active+clean; 633 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 6.0 KiB/s wr, 9 op/s; 65 B/s, 1 objects/s recovering 2024-09-21T03:38:05.311 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:38:05 smithi089 ceph-mon[30928]: from='client.15021 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T03:38:05.460 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:38:05 smithi114 ceph-mon[34140]: pgmap v318: 129 pgs: 129 active+clean; 633 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 6.0 KiB/s wr, 9 op/s; 65 B/s, 1 objects/s recovering 2024-09-21T03:38:05.461 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:38:05 smithi114 ceph-mon[34140]: from='client.15021 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T03:38:05.500 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:38:05 smithi110 ceph-mon[34000]: pgmap v318: 129 pgs: 129 active+clean; 633 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 6.0 KiB/s wr, 9 op/s; 65 B/s, 1 objects/s recovering 2024-09-21T03:38:05.500 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:38:05 smithi110 ceph-mon[34000]: from='client.15021 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T03:38:05.731 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:05.726+0000 7fa39163d640 1 -- 172.21.15.89:0/2938747101 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa38c1083a0 msgr2=0x7fa38c1087b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:05.731 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:05.726+0000 7fa39163d640 1 --2- 172.21.15.89:0/2938747101 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa38c1083a0 0x7fa38c1087b0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fa374009900 tx=0x7fa37402a410 comp rx=0 tx=0).stop 2024-09-21T03:38:05.731 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:05.726+0000 7fa39163d640 1 -- 172.21.15.89:0/2938747101 shutdown_connections 2024-09-21T03:38:05.731 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:05.726+0000 7fa39163d640 1 --2- 172.21.15.89:0/2938747101 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa38c108cf0 0x7fa38c10f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:05.732 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:05.726+0000 7fa39163d640 1 --2- 172.21.15.89:0/2938747101 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa38c1083a0 0x7fa38c1087b0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:05.732 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:05.726+0000 7fa39163d640 1 --2- 172.21.15.89:0/2938747101 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa38c102a60 0x7fa38c102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:05.732 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:05.726+0000 7fa39163d640 1 -- 172.21.15.89:0/2938747101 >> 172.21.15.89:0/2938747101 conn(0x7fa38c079730 msgr2=0x7fa38c079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:38:05.732 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:05.726+0000 7fa39163d640 1 -- 172.21.15.89:0/2938747101 shutdown_connections 2024-09-21T03:38:05.732 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:05.726+0000 7fa39163d640 1 -- 172.21.15.89:0/2938747101 wait complete. 2024-09-21T03:38:05.732 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:05.726+0000 7fa39163d640 1 Processor -- start 2024-09-21T03:38:05.732 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:05.727+0000 7fa39163d640 1 -- start start 2024-09-21T03:38:05.732 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:05.727+0000 7fa39163d640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa38c102a60 0x7fa38c19a380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:05.732 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:05.727+0000 7fa39163d640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa38c1083a0 0x7fa38c19a920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:05.732 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:05.727+0000 7fa39163d640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa38c108cf0 0x7fa38c19fbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:05.732 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:05.727+0000 7fa39163d640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fa38c113930 con 0x7fa38c108cf0 2024-09-21T03:38:05.732 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:05.727+0000 7fa39163d640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fa38c1137b0 con 0x7fa38c102a60 2024-09-21T03:38:05.732 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:05.727+0000 7fa39163d640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fa38c113ab0 con 0x7fa38c1083a0 2024-09-21T03:38:05.732 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:05.727+0000 7fa38b7fe640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa38c108cf0 0x7fa38c19fbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:05.733 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:05.727+0000 7fa38b7fe640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa38c108cf0 0x7fa38c19fbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:50818/0 (socket says 172.21.15.89:50818) 2024-09-21T03:38:05.733 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:05.727+0000 7fa38b7fe640 1 -- 172.21.15.89:0/3938504108 learned_addr learned my addr 172.21.15.89:0/3938504108 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-21T03:38:05.733 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:05.727+0000 7fa38affd640 1 --2- 172.21.15.89:0/3938504108 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa38c102a60 0x7fa38c19a380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:05.733 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:05.727+0000 7fa3827fc640 1 --2- 172.21.15.89:0/3938504108 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa38c1083a0 0x7fa38c19a920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:05.733 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:05.727+0000 7fa38b7fe640 1 -- 172.21.15.89:0/3938504108 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa38c1083a0 msgr2=0x7fa38c19a920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:05.733 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:05.727+0000 7fa38b7fe640 1 --2- 172.21.15.89:0/3938504108 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa38c1083a0 0x7fa38c19a920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:05.733 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:05.727+0000 7fa38b7fe640 1 -- 172.21.15.89:0/3938504108 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa38c102a60 msgr2=0x7fa38c19a380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:05.733 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:05.727+0000 7fa38b7fe640 1 --2- 172.21.15.89:0/3938504108 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa38c102a60 0x7fa38c19a380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:05.733 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:05.727+0000 7fa38b7fe640 1 -- 172.21.15.89:0/3938504108 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa38c1a02f0 con 0x7fa38c108cf0 2024-09-21T03:38:05.733 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:05.727+0000 7fa38affd640 1 --2- 172.21.15.89:0/3938504108 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa38c102a60 0x7fa38c19a380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:38:05.733 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:05.728+0000 7fa38b7fe640 1 --2- 172.21.15.89:0/3938504108 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa38c108cf0 0x7fa38c19fbc0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fa38400eaa0 tx=0x7fa38400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:38:05.733 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:05.728+0000 7fa388ff9640 1 -- 172.21.15.89:0/3938504108 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa38400cdb0 con 0x7fa38c108cf0 2024-09-21T03:38:05.733 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:05.728+0000 7fa39163d640 1 -- 172.21.15.89:0/3938504108 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa38c1a05e0 con 0x7fa38c108cf0 2024-09-21T03:38:05.733 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:05.728+0000 7fa388ff9640 1 -- 172.21.15.89:0/3938504108 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa384005020 con 0x7fa38c108cf0 2024-09-21T03:38:05.734 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:05.728+0000 7fa388ff9640 1 -- 172.21.15.89:0/3938504108 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3840116d0 con 0x7fa38c108cf0 2024-09-21T03:38:05.734 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:05.728+0000 7fa39163d640 1 -- 172.21.15.89:0/3938504108 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa38c1a6d40 con 0x7fa38c108cf0 2024-09-21T03:38:05.734 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:05.729+0000 7fa39163d640 1 -- 172.21.15.89:0/3938504108 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa38c102e70 con 0x7fa38c108cf0 2024-09-21T03:38:05.734 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:05.731+0000 7fa388ff9640 1 -- 172.21.15.89:0/3938504108 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 18) ==== 154563+0+0 (secure 0 0 0) 0x7fa384004a40 con 0x7fa38c108cf0 2024-09-21T03:38:05.734 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:05.731+0000 7fa388ff9640 1 --2- 172.21.15.89:0/3938504108 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7fa3780b65a0 0x7fa3780b8ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:05.735 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:05.732+0000 7fa38affd640 1 --2- 172.21.15.89:0/3938504108 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7fa3780b65a0 0x7fa3780b8ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:05.735 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:05.732+0000 7fa388ff9640 1 -- 172.21.15.89:0/3938504108 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 9092+0+0 (secure 0 0 0) 0x7fa3840e7e20 con 0x7fa38c108cf0 2024-09-21T03:38:05.735 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:05.732+0000 7fa38affd640 1 --2- 172.21.15.89:0/3938504108 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7fa3780b65a0 0x7fa3780b8ab0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fa37c009c20 tx=0x7fa37c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:38:05.736 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:05.734+0000 7fa388ff9640 1 -- 172.21.15.89:0/3938504108 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fa384016030 con 0x7fa38c108cf0 2024-09-21T03:38:05.911 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:05.908+0000 7fa39163d640 1 -- 172.21.15.89:0/3938504108 --> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fa38c063800 con 0x7fa3780b65a0 2024-09-21T03:38:05.916 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:05.914+0000 7fa388ff9640 1 -- 172.21.15.89:0/3938504108 <== mgr.14152 v2:172.21.15.89:6800/1591532358 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340630 (secure 0 0 0) 0x7fa38c063800 con 0x7fa3780b65a0 2024-09-21T03:38:05.919 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-09-21T03:38:05.921 INFO:teuthology.orchestra.run.smithi089.stderr:dumped all 2024-09-21T03:38:05.923 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:05.918+0000 7fa39163d640 1 -- 172.21.15.89:0/3938504108 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7fa3780b65a0 msgr2=0x7fa3780b8ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:05.923 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:05.918+0000 7fa39163d640 1 --2- 172.21.15.89:0/3938504108 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7fa3780b65a0 0x7fa3780b8ab0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fa37c009c20 tx=0x7fa37c005f90 comp rx=0 tx=0).stop 2024-09-21T03:38:05.923 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:05.918+0000 7fa39163d640 1 -- 172.21.15.89:0/3938504108 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa38c108cf0 msgr2=0x7fa38c19fbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:05.923 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:05.918+0000 7fa39163d640 1 --2- 172.21.15.89:0/3938504108 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa38c108cf0 0x7fa38c19fbc0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fa38400eaa0 tx=0x7fa38400ef60 comp rx=0 tx=0).stop 2024-09-21T03:38:05.923 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:05.918+0000 7fa39163d640 1 -- 172.21.15.89:0/3938504108 shutdown_connections 2024-09-21T03:38:05.923 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:05.918+0000 7fa39163d640 1 --2- 172.21.15.89:0/3938504108 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7fa3780b65a0 0x7fa3780b8ab0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:05.923 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:05.918+0000 7fa39163d640 1 --2- 172.21.15.89:0/3938504108 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa38c108cf0 0x7fa38c19fbc0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:05.923 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:05.918+0000 7fa39163d640 1 --2- 172.21.15.89:0/3938504108 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa38c1083a0 0x7fa38c19a920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:05.923 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:05.918+0000 7fa39163d640 1 --2- 172.21.15.89:0/3938504108 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa38c102a60 0x7fa38c19a380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:05.924 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:05.918+0000 7fa39163d640 1 -- 172.21.15.89:0/3938504108 >> 172.21.15.89:0/3938504108 conn(0x7fa38c079730 msgr2=0x7fa38c1052f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:38:05.924 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:05.919+0000 7fa39163d640 1 -- 172.21.15.89:0/3938504108 shutdown_connections 2024-09-21T03:38:05.924 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:05.919+0000 7fa39163d640 1 -- 172.21.15.89:0/3938504108 wait complete. 2024-09-21T03:38:06.564 INFO:teuthology.orchestra.run.smithi089.stdout:{"pg_ready":true,"pg_map":{"version":319,"stamp":"2024-09-21T03:38:05.622846+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":655488,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":191,"num_read_kb":192,"num_write":282,"num_write_kb":2263,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":23,"num_bytes_recovered":2952534,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":230,"ondisk_log_size":230,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":376892,"kb_used_data":10984,"kb_used_omap":18,"kb_used_meta":365677,"kb_avail":1124564932,"statfs":{"total":1151940427776,"available":1151554490368,"internally_reserved":0,"allocated":11247616,"data_stored":7462368,"data_compressed":390510,"data_compressed_allocated":528384,"data_compressed_original":2555904,"omap_allocated":19143,"internal_metadata":374453561},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":57780,"num_objects":5,"num_object_clones":0,"num_object_copies":17,"num_objects_missing_on_primary":0,"num_objects_missing":-2,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":86,"num_read_kb":104,"num_write":33,"num_write_kb":71,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":694,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004001"},"pg_stats":[{"pgid":"2.1f","version":"80'9","reported_seq":45,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.047700+0000","last_change":"2024-09-21T03:37:10.836953+0000","last_active":"2024-09-21T03:37:53.047700+0000","last_peered":"2024-09-21T03:37:53.047700+0000","last_clean":"2024-09-21T03:37:53.047700+0000","last_became_active":"2024-09-21T03:37:10.836566+0000","last_became_peered":"2024-09-21T03:37:10.836566+0000","last_unstale":"2024-09-21T03:37:53.047700+0000","last_undegraded":"2024-09-21T03:37:53.047700+0000","last_fullsized":"2024-09-21T03:37:53.047700+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_clean_scrub_stamp":"2024-09-21T03:37:09.809805+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:15:45.591225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.046350+0000","last_change":"2024-09-21T03:37:12.866931+0000","last_active":"2024-09-21T03:37:53.046350+0000","last_peered":"2024-09-21T03:37:53.046350+0000","last_clean":"2024-09-21T03:37:53.046350+0000","last_became_active":"2024-09-21T03:37:12.866452+0000","last_became_peered":"2024-09-21T03:37:12.866452+0000","last_unstale":"2024-09-21T03:37:53.046350+0000","last_undegraded":"2024-09-21T03:37:53.046350+0000","last_fullsized":"2024-09-21T03:37:53.046350+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:37:14.324548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.044138+0000","last_change":"2024-09-21T03:37:19.912154+0000","last_active":"2024-09-21T03:37:53.044138+0000","last_peered":"2024-09-21T03:37:53.044138+0000","last_clean":"2024-09-21T03:37:53.044138+0000","last_became_active":"2024-09-21T03:37:19.911686+0000","last_became_peered":"2024-09-21T03:37:19.911686+0000","last_unstale":"2024-09-21T03:37:53.044138+0000","last_undegraded":"2024-09-21T03:37:53.044138+0000","last_fullsized":"2024-09-21T03:37:53.044138+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_clean_scrub_stamp":"2024-09-21T03:37:18.876933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:29:20.189418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"80'2","reported_seq":37,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.046676+0000","last_change":"2024-09-21T03:37:11.034501+0000","last_active":"2024-09-21T03:37:53.046676+0000","last_peered":"2024-09-21T03:37:53.046676+0000","last_clean":"2024-09-21T03:37:53.046676+0000","last_became_active":"2024-09-21T03:37:11.034110+0000","last_became_peered":"2024-09-21T03:37:11.034110+0000","last_unstale":"2024-09-21T03:37:53.046676+0000","last_undegraded":"2024-09-21T03:37:53.046676+0000","last_fullsized":"2024-09-21T03:37:53.046676+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_clean_scrub_stamp":"2024-09-21T03:37:09.809805+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:03:12.854469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.049598+0000","last_change":"2024-09-21T03:37:12.867330+0000","last_active":"2024-09-21T03:37:53.049598+0000","last_peered":"2024-09-21T03:37:53.049598+0000","last_clean":"2024-09-21T03:37:53.049598+0000","last_became_active":"2024-09-21T03:37:12.866952+0000","last_became_peered":"2024-09-21T03:37:12.866952+0000","last_unstale":"2024-09-21T03:37:53.049598+0000","last_undegraded":"2024-09-21T03:37:53.049598+0000","last_fullsized":"2024-09-21T03:37:53.049598+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:40:33.826546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.043412+0000","last_change":"2024-09-21T03:37:19.910661+0000","last_active":"2024-09-21T03:37:53.043412+0000","last_peered":"2024-09-21T03:37:53.043412+0000","last_clean":"2024-09-21T03:37:53.043412+0000","last_became_active":"2024-09-21T03:37:19.910108+0000","last_became_peered":"2024-09-21T03:37:19.910108+0000","last_unstale":"2024-09-21T03:37:53.043412+0000","last_undegraded":"2024-09-21T03:37:53.043412+0000","last_fullsized":"2024-09-21T03:37:53.043412+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_clean_scrub_stamp":"2024-09-21T03:37:18.876933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:18:05.654426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"80'4","reported_seq":39,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.050839+0000","last_change":"2024-09-21T03:37:10.835336+0000","last_active":"2024-09-21T03:37:53.050839+0000","last_peered":"2024-09-21T03:37:53.050839+0000","last_clean":"2024-09-21T03:37:53.050839+0000","last_became_active":"2024-09-21T03:37:10.834928+0000","last_became_peered":"2024-09-21T03:37:10.834928+0000","last_unstale":"2024-09-21T03:37:53.050839+0000","last_undegraded":"2024-09-21T03:37:53.050839+0000","last_fullsized":"2024-09-21T03:37:53.050839+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_clean_scrub_stamp":"2024-09-21T03:37:09.809805+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:02:14.717520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.111510+0000","last_change":"2024-09-21T03:37:12.861271+0000","last_active":"2024-09-21T03:37:53.111510+0000","last_peered":"2024-09-21T03:37:53.111510+0000","last_clean":"2024-09-21T03:37:53.111510+0000","last_became_active":"2024-09-21T03:37:12.860932+0000","last_became_peered":"2024-09-21T03:37:12.860932+0000","last_unstale":"2024-09-21T03:37:53.111510+0000","last_undegraded":"2024-09-21T03:37:53.111510+0000","last_fullsized":"2024-09-21T03:37:53.111510+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:46:08.993250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.053171+0000","last_change":"2024-09-21T03:37:53.053045+0000","last_active":"2024-09-21T03:37:53.053171+0000","last_peered":"2024-09-21T03:37:53.053171+0000","last_clean":"2024-09-21T03:37:53.053171+0000","last_became_active":"2024-09-21T03:37:53.052382+0000","last_became_peered":"2024-09-21T03:37:53.052382+0000","last_unstale":"2024-09-21T03:37:53.053171+0000","last_undegraded":"2024-09-21T03:37:53.053171+0000","last_fullsized":"2024-09-21T03:37:53.053171+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_clean_scrub_stamp":"2024-09-21T03:37:18.876933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:02:44.300768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,8],"acting":[4,3,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"74'2","reported_seq":37,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.050328+0000","last_change":"2024-09-21T03:37:10.833985+0000","last_active":"2024-09-21T03:37:53.050328+0000","last_peered":"2024-09-21T03:37:53.050328+0000","last_clean":"2024-09-21T03:37:53.050328+0000","last_became_active":"2024-09-21T03:37:10.833562+0000","last_became_peered":"2024-09-21T03:37:10.833562+0000","last_unstale":"2024-09-21T03:37:53.050328+0000","last_undegraded":"2024-09-21T03:37:53.050328+0000","last_fullsized":"2024-09-21T03:37:53.050328+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_clean_scrub_stamp":"2024-09-21T03:37:09.809805+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:49:32.617768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.047066+0000","last_change":"2024-09-21T03:37:12.865408+0000","last_active":"2024-09-21T03:37:53.047066+0000","last_peered":"2024-09-21T03:37:53.047066+0000","last_clean":"2024-09-21T03:37:53.047066+0000","last_became_active":"2024-09-21T03:37:12.864972+0000","last_became_peered":"2024-09-21T03:37:12.864972+0000","last_unstale":"2024-09-21T03:37:53.047066+0000","last_undegraded":"2024-09-21T03:37:53.047066+0000","last_fullsized":"2024-09-21T03:37:53.047066+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:03:16.384260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.043495+0000","last_change":"2024-09-21T03:37:19.912105+0000","last_active":"2024-09-21T03:37:53.043495+0000","last_peered":"2024-09-21T03:37:53.043495+0000","last_clean":"2024-09-21T03:37:53.043495+0000","last_became_active":"2024-09-21T03:37:19.911416+0000","last_became_peered":"2024-09-21T03:37:19.911416+0000","last_unstale":"2024-09-21T03:37:53.043495+0000","last_undegraded":"2024-09-21T03:37:53.043495+0000","last_fullsized":"2024-09-21T03:37:53.043495+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_clean_scrub_stamp":"2024-09-21T03:37:18.876933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:20:16.492231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.431254+0000","last_change":"2024-09-21T03:37:12.862354+0000","last_active":"2024-09-21T03:37:53.431254+0000","last_peered":"2024-09-21T03:37:53.431254+0000","last_clean":"2024-09-21T03:37:53.431254+0000","last_became_active":"2024-09-21T03:37:12.861977+0000","last_became_peered":"2024-09-21T03:37:12.861977+0000","last_unstale":"2024-09-21T03:37:53.431254+0000","last_undegraded":"2024-09-21T03:37:53.431254+0000","last_fullsized":"2024-09-21T03:37:53.431254+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:53:52.725524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"80'4","reported_seq":40,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.051416+0000","last_change":"2024-09-21T03:37:10.832140+0000","last_active":"2024-09-21T03:37:53.051416+0000","last_peered":"2024-09-21T03:37:53.051416+0000","last_clean":"2024-09-21T03:37:53.051416+0000","last_became_active":"2024-09-21T03:37:10.831395+0000","last_became_peered":"2024-09-21T03:37:10.831395+0000","last_unstale":"2024-09-21T03:37:53.051416+0000","last_undegraded":"2024-09-21T03:37:53.051416+0000","last_fullsized":"2024-09-21T03:37:53.051416+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_clean_scrub_stamp":"2024-09-21T03:37:09.809805+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:55:20.440890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.044184+0000","last_change":"2024-09-21T03:37:20.160850+0000","last_active":"2024-09-21T03:37:53.044184+0000","last_peered":"2024-09-21T03:37:53.044184+0000","last_clean":"2024-09-21T03:37:53.044184+0000","last_became_active":"2024-09-21T03:37:20.160305+0000","last_became_peered":"2024-09-21T03:37:20.160305+0000","last_unstale":"2024-09-21T03:37:53.044184+0000","last_undegraded":"2024-09-21T03:37:53.044184+0000","last_fullsized":"2024-09-21T03:37:53.044184+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_clean_scrub_stamp":"2024-09-21T03:37:18.876933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:51:45.971984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.046377+0000","last_change":"2024-09-21T03:37:12.859485+0000","last_active":"2024-09-21T03:37:53.046377+0000","last_peered":"2024-09-21T03:37:53.046377+0000","last_clean":"2024-09-21T03:37:53.046377+0000","last_became_active":"2024-09-21T03:37:12.859097+0000","last_became_peered":"2024-09-21T03:37:12.859097+0000","last_unstale":"2024-09-21T03:37:53.046377+0000","last_undegraded":"2024-09-21T03:37:53.046377+0000","last_fullsized":"2024-09-21T03:37:53.046377+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:16:05.438360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"80'1","reported_seq":36,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.044264+0000","last_change":"2024-09-21T03:37:10.832441+0000","last_active":"2024-09-21T03:37:53.044264+0000","last_peered":"2024-09-21T03:37:53.044264+0000","last_clean":"2024-09-21T03:37:53.044264+0000","last_became_active":"2024-09-21T03:37:10.832073+0000","last_became_peered":"2024-09-21T03:37:10.832073+0000","last_unstale":"2024-09-21T03:37:53.044264+0000","last_undegraded":"2024-09-21T03:37:53.044264+0000","last_fullsized":"2024-09-21T03:37:53.044264+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_clean_scrub_stamp":"2024-09-21T03:37:09.809805+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:32:19.411097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"82'1","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.043995+0000","last_change":"2024-09-21T03:37:19.905490+0000","last_active":"2024-09-21T03:37:53.043995+0000","last_peered":"2024-09-21T03:37:53.043995+0000","last_clean":"2024-09-21T03:37:53.043995+0000","last_became_active":"2024-09-21T03:37:19.904971+0000","last_became_peered":"2024-09-21T03:37:19.904971+0000","last_unstale":"2024-09-21T03:37:53.043995+0000","last_undegraded":"2024-09-21T03:37:53.043995+0000","last_fullsized":"2024-09-21T03:37:53.043995+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_clean_scrub_stamp":"2024-09-21T03:37:18.876933+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:44:20.655538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.048957+0000","last_change":"2024-09-21T03:37:12.863597+0000","last_active":"2024-09-21T03:37:53.048957+0000","last_peered":"2024-09-21T03:37:53.048957+0000","last_clean":"2024-09-21T03:37:53.048957+0000","last_became_active":"2024-09-21T03:37:12.863223+0000","last_became_peered":"2024-09-21T03:37:12.863223+0000","last_unstale":"2024-09-21T03:37:53.048957+0000","last_undegraded":"2024-09-21T03:37:53.048957+0000","last_fullsized":"2024-09-21T03:37:53.048957+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:29:28.878964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.9","version":"80'3","reported_seq":38,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.045808+0000","last_change":"2024-09-21T03:37:10.834648+0000","last_active":"2024-09-21T03:37:53.045808+0000","last_peered":"2024-09-21T03:37:53.045808+0000","last_clean":"2024-09-21T03:37:53.045808+0000","last_became_active":"2024-09-21T03:37:10.834293+0000","last_became_peered":"2024-09-21T03:37:10.834293+0000","last_unstale":"2024-09-21T03:37:53.045808+0000","last_undegraded":"2024-09-21T03:37:53.045808+0000","last_fullsized":"2024-09-21T03:37:53.045808+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_clean_scrub_stamp":"2024-09-21T03:37:09.809805+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:10:41.248733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.047492+0000","last_change":"2024-09-21T03:37:19.912526+0000","last_active":"2024-09-21T03:37:53.047492+0000","last_peered":"2024-09-21T03:37:53.047492+0000","last_clean":"2024-09-21T03:37:53.047492+0000","last_became_active":"2024-09-21T03:37:19.912062+0000","last_became_peered":"2024-09-21T03:37:19.912062+0000","last_unstale":"2024-09-21T03:37:53.047492+0000","last_undegraded":"2024-09-21T03:37:53.047492+0000","last_fullsized":"2024-09-21T03:37:53.047492+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_clean_scrub_stamp":"2024-09-21T03:37:18.876933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:52:40.395834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.049515+0000","last_change":"2024-09-21T03:37:12.855672+0000","last_active":"2024-09-21T03:37:53.049515+0000","last_peered":"2024-09-21T03:37:53.049515+0000","last_clean":"2024-09-21T03:37:53.049515+0000","last_became_active":"2024-09-21T03:37:12.855191+0000","last_became_peered":"2024-09-21T03:37:12.855191+0000","last_unstale":"2024-09-21T03:37:53.049515+0000","last_undegraded":"2024-09-21T03:37:53.049515+0000","last_fullsized":"2024-09-21T03:37:53.049515+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:26:56.115697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.8","version":"80'2","reported_seq":37,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.111296+0000","last_change":"2024-09-21T03:37:11.034752+0000","last_active":"2024-09-21T03:37:53.111296+0000","last_peered":"2024-09-21T03:37:53.111296+0000","last_clean":"2024-09-21T03:37:53.111296+0000","last_became_active":"2024-09-21T03:37:11.034386+0000","last_became_peered":"2024-09-21T03:37:11.034386+0000","last_unstale":"2024-09-21T03:37:53.111296+0000","last_undegraded":"2024-09-21T03:37:53.111296+0000","last_fullsized":"2024-09-21T03:37:53.111296+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_clean_scrub_stamp":"2024-09-21T03:37:09.809805+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:17:23.647411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.044213+0000","last_change":"2024-09-21T03:37:20.162051+0000","last_active":"2024-09-21T03:37:53.044213+0000","last_peered":"2024-09-21T03:37:53.044213+0000","last_clean":"2024-09-21T03:37:53.044213+0000","last_became_active":"2024-09-21T03:37:20.161452+0000","last_became_peered":"2024-09-21T03:37:20.161452+0000","last_unstale":"2024-09-21T03:37:53.044213+0000","last_undegraded":"2024-09-21T03:37:53.044213+0000","last_fullsized":"2024-09-21T03:37:53.044213+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_clean_scrub_stamp":"2024-09-21T03:37:18.876933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:29:22.544800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.049528+0000","last_change":"2024-09-21T03:37:12.866058+0000","last_active":"2024-09-21T03:37:53.049528+0000","last_peered":"2024-09-21T03:37:53.049528+0000","last_clean":"2024-09-21T03:37:53.049528+0000","last_became_active":"2024-09-21T03:37:12.865694+0000","last_became_peered":"2024-09-21T03:37:12.865694+0000","last_unstale":"2024-09-21T03:37:53.049528+0000","last_undegraded":"2024-09-21T03:37:53.049528+0000","last_fullsized":"2024-09-21T03:37:53.049528+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:56:33.629873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"80'4","reported_seq":39,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.044782+0000","last_change":"2024-09-21T03:37:11.358619+0000","last_active":"2024-09-21T03:37:53.044782+0000","last_peered":"2024-09-21T03:37:53.044782+0000","last_clean":"2024-09-21T03:37:53.044782+0000","last_became_active":"2024-09-21T03:37:11.358196+0000","last_became_peered":"2024-09-21T03:37:11.358196+0000","last_unstale":"2024-09-21T03:37:53.044782+0000","last_undegraded":"2024-09-21T03:37:53.044782+0000","last_fullsized":"2024-09-21T03:37:53.044782+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_clean_scrub_stamp":"2024-09-21T03:37:09.809805+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:57:13.476641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.044378+0000","last_change":"2024-09-21T03:37:19.905711+0000","last_active":"2024-09-21T03:37:53.044378+0000","last_peered":"2024-09-21T03:37:53.044378+0000","last_clean":"2024-09-21T03:37:53.044378+0000","last_became_active":"2024-09-21T03:37:19.905260+0000","last_became_peered":"2024-09-21T03:37:19.905260+0000","last_unstale":"2024-09-21T03:37:53.044378+0000","last_undegraded":"2024-09-21T03:37:53.044378+0000","last_fullsized":"2024-09-21T03:37:53.044378+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_clean_scrub_stamp":"2024-09-21T03:37:18.876933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:21:09.476082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.046425+0000","last_change":"2024-09-21T03:37:12.854376+0000","last_active":"2024-09-21T03:37:53.046425+0000","last_peered":"2024-09-21T03:37:53.046425+0000","last_clean":"2024-09-21T03:37:53.046425+0000","last_became_active":"2024-09-21T03:37:12.852942+0000","last_became_peered":"2024-09-21T03:37:12.852942+0000","last_unstale":"2024-09-21T03:37:53.046425+0000","last_undegraded":"2024-09-21T03:37:53.046425+0000","last_fullsized":"2024-09-21T03:37:53.046425+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:06:49.319002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"82'8","reported_seq":56,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:59.865741+0000","last_change":"2024-09-21T03:37:53.056889+0000","last_active":"2024-09-21T03:37:59.865741+0000","last_peered":"2024-09-21T03:37:59.865741+0000","last_clean":"2024-09-21T03:37:59.865741+0000","last_became_active":"2024-09-21T03:37:53.053054+0000","last_became_peered":"2024-09-21T03:37:53.053054+0000","last_unstale":"2024-09-21T03:37:59.865741+0000","last_undegraded":"2024-09-21T03:37:59.865741+0000","last_fullsized":"2024-09-21T03:37:59.865741+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_clean_scrub_stamp":"2024-09-21T03:37:09.809805+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:16:05.539111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,3],"acting":[9,11,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.045339+0000","last_change":"2024-09-21T03:37:20.161515+0000","last_active":"2024-09-21T03:37:53.045339+0000","last_peered":"2024-09-21T03:37:53.045339+0000","last_clean":"2024-09-21T03:37:53.045339+0000","last_became_active":"2024-09-21T03:37:20.160548+0000","last_became_peered":"2024-09-21T03:37:20.160548+0000","last_unstale":"2024-09-21T03:37:53.045339+0000","last_undegraded":"2024-09-21T03:37:53.045339+0000","last_fullsized":"2024-09-21T03:37:53.045339+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_clean_scrub_stamp":"2024-09-21T03:37:18.876933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:17:51.144859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.042971+0000","last_change":"2024-09-21T03:37:12.848305+0000","last_active":"2024-09-21T03:37:53.042971+0000","last_peered":"2024-09-21T03:37:53.042971+0000","last_clean":"2024-09-21T03:37:53.042971+0000","last_became_active":"2024-09-21T03:37:12.847916+0000","last_became_peered":"2024-09-21T03:37:12.847916+0000","last_unstale":"2024-09-21T03:37:53.042971+0000","last_undegraded":"2024-09-21T03:37:53.042971+0000","last_fullsized":"2024-09-21T03:37:53.042971+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:48:39.922805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"80'1","reported_seq":37,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.051068+0000","last_change":"2024-09-21T03:37:11.035964+0000","last_active":"2024-09-21T03:37:53.051068+0000","last_peered":"2024-09-21T03:37:53.051068+0000","last_clean":"2024-09-21T03:37:53.051068+0000","last_became_active":"2024-09-21T03:37:11.035427+0000","last_became_peered":"2024-09-21T03:37:11.035427+0000","last_unstale":"2024-09-21T03:37:53.051068+0000","last_undegraded":"2024-09-21T03:37:53.051068+0000","last_fullsized":"2024-09-21T03:37:53.051068+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_clean_scrub_stamp":"2024-09-21T03:37:09.809805+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:04:51.026753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.047032+0000","last_change":"2024-09-21T03:37:19.909428+0000","last_active":"2024-09-21T03:37:53.047032+0000","last_peered":"2024-09-21T03:37:53.047032+0000","last_clean":"2024-09-21T03:37:53.047032+0000","last_became_active":"2024-09-21T03:37:19.908904+0000","last_became_peered":"2024-09-21T03:37:19.908904+0000","last_unstale":"2024-09-21T03:37:53.047032+0000","last_undegraded":"2024-09-21T03:37:53.047032+0000","last_fullsized":"2024-09-21T03:37:53.047032+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_clean_scrub_stamp":"2024-09-21T03:37:18.876933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:12:07.564058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.046987+0000","last_change":"2024-09-21T03:37:12.855592+0000","last_active":"2024-09-21T03:37:53.046987+0000","last_peered":"2024-09-21T03:37:53.046987+0000","last_clean":"2024-09-21T03:37:53.046987+0000","last_became_active":"2024-09-21T03:37:12.855244+0000","last_became_peered":"2024-09-21T03:37:12.855244+0000","last_unstale":"2024-09-21T03:37:53.046987+0000","last_undegraded":"2024-09-21T03:37:53.046987+0000","last_fullsized":"2024-09-21T03:37:53.046987+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:21:34.603141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":35,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.047381+0000","last_change":"2024-09-21T03:37:11.036401+0000","last_active":"2024-09-21T03:37:53.047381+0000","last_peered":"2024-09-21T03:37:53.047381+0000","last_clean":"2024-09-21T03:37:53.047381+0000","last_became_active":"2024-09-21T03:37:11.036003+0000","last_became_peered":"2024-09-21T03:37:11.036003+0000","last_unstale":"2024-09-21T03:37:53.047381+0000","last_undegraded":"2024-09-21T03:37:53.047381+0000","last_fullsized":"2024-09-21T03:37:53.047381+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_clean_scrub_stamp":"2024-09-21T03:37:09.809805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:35:50.549060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.050609+0000","last_change":"2024-09-21T03:37:20.160668+0000","last_active":"2024-09-21T03:37:53.050609+0000","last_peered":"2024-09-21T03:37:53.050609+0000","last_clean":"2024-09-21T03:37:53.050609+0000","last_became_active":"2024-09-21T03:37:20.160158+0000","last_became_peered":"2024-09-21T03:37:20.160158+0000","last_unstale":"2024-09-21T03:37:53.050609+0000","last_undegraded":"2024-09-21T03:37:53.050609+0000","last_fullsized":"2024-09-21T03:37:53.050609+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_clean_scrub_stamp":"2024-09-21T03:37:18.876933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:45:25.677667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":37,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.050453+0000","last_change":"2024-09-21T03:37:53.050351+0000","last_active":"2024-09-21T03:37:53.050453+0000","last_peered":"2024-09-21T03:37:53.050453+0000","last_clean":"2024-09-21T03:37:53.050453+0000","last_became_active":"2024-09-21T03:37:53.050003+0000","last_became_peered":"2024-09-21T03:37:53.050003+0000","last_unstale":"2024-09-21T03:37:53.050453+0000","last_undegraded":"2024-09-21T03:37:53.050453+0000","last_fullsized":"2024-09-21T03:37:53.050453+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:49:03.313002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,0],"acting":[10,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"80'1","reported_seq":36,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.431360+0000","last_change":"2024-09-21T03:37:10.832635+0000","last_active":"2024-09-21T03:37:53.431360+0000","last_peered":"2024-09-21T03:37:53.431360+0000","last_clean":"2024-09-21T03:37:53.431360+0000","last_became_active":"2024-09-21T03:37:10.832280+0000","last_became_peered":"2024-09-21T03:37:10.832280+0000","last_unstale":"2024-09-21T03:37:53.431360+0000","last_undegraded":"2024-09-21T03:37:53.431360+0000","last_fullsized":"2024-09-21T03:37:53.431360+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_clean_scrub_stamp":"2024-09-21T03:37:09.809805+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:54:52.744486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.042822+0000","last_change":"2024-09-21T03:37:19.904112+0000","last_active":"2024-09-21T03:37:53.042822+0000","last_peered":"2024-09-21T03:37:53.042822+0000","last_clean":"2024-09-21T03:37:53.042822+0000","last_became_active":"2024-09-21T03:37:19.903677+0000","last_became_peered":"2024-09-21T03:37:19.903677+0000","last_unstale":"2024-09-21T03:37:53.042822+0000","last_undegraded":"2024-09-21T03:37:53.042822+0000","last_fullsized":"2024-09-21T03:37:53.042822+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_clean_scrub_stamp":"2024-09-21T03:37:18.876933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:32:20.096850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.046073+0000","last_change":"2024-09-21T03:37:12.864082+0000","last_active":"2024-09-21T03:37:53.046073+0000","last_peered":"2024-09-21T03:37:53.046073+0000","last_clean":"2024-09-21T03:37:53.046073+0000","last_became_active":"2024-09-21T03:37:12.862745+0000","last_became_peered":"2024-09-21T03:37:12.862745+0000","last_unstale":"2024-09-21T03:37:53.046073+0000","last_undegraded":"2024-09-21T03:37:53.046073+0000","last_fullsized":"2024-09-21T03:37:53.046073+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:25:35.759719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"80'4","reported_seq":55,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:38:03.050249+0000","last_change":"2024-09-21T03:37:11.034463+0000","last_active":"2024-09-21T03:38:03.050249+0000","last_peered":"2024-09-21T03:38:03.050249+0000","last_clean":"2024-09-21T03:38:03.050249+0000","last_became_active":"2024-09-21T03:37:11.033932+0000","last_became_peered":"2024-09-21T03:37:11.033932+0000","last_unstale":"2024-09-21T03:38:03.050249+0000","last_undegraded":"2024-09-21T03:38:03.050249+0000","last_fullsized":"2024-09-21T03:38:03.050249+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_clean_scrub_stamp":"2024-09-21T03:37:09.809805+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:15:18.595442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":16,"num_read_kb":16,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.043133+0000","last_change":"2024-09-21T03:37:20.160789+0000","last_active":"2024-09-21T03:37:53.043133+0000","last_peered":"2024-09-21T03:37:53.043133+0000","last_clean":"2024-09-21T03:37:53.043133+0000","last_became_active":"2024-09-21T03:37:20.160231+0000","last_became_peered":"2024-09-21T03:37:20.160231+0000","last_unstale":"2024-09-21T03:37:53.043133+0000","last_undegraded":"2024-09-21T03:37:53.043133+0000","last_fullsized":"2024-09-21T03:37:53.043133+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_clean_scrub_stamp":"2024-09-21T03:37:18.876933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:07:16.762249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.042510+0000","last_change":"2024-09-21T03:37:12.853178+0000","last_active":"2024-09-21T03:37:53.042510+0000","last_peered":"2024-09-21T03:37:53.042510+0000","last_clean":"2024-09-21T03:37:53.042510+0000","last_became_active":"2024-09-21T03:37:12.852822+0000","last_became_peered":"2024-09-21T03:37:12.852822+0000","last_unstale":"2024-09-21T03:37:53.042510+0000","last_undegraded":"2024-09-21T03:37:53.042510+0000","last_fullsized":"2024-09-21T03:37:53.042510+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:48:26.146593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"80'3","reported_seq":38,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.430945+0000","last_change":"2024-09-21T03:37:10.838939+0000","last_active":"2024-09-21T03:37:53.430945+0000","last_peered":"2024-09-21T03:37:53.430945+0000","last_clean":"2024-09-21T03:37:53.430945+0000","last_became_active":"2024-09-21T03:37:10.838562+0000","last_became_peered":"2024-09-21T03:37:10.838562+0000","last_unstale":"2024-09-21T03:37:53.430945+0000","last_undegraded":"2024-09-21T03:37:53.430945+0000","last_fullsized":"2024-09-21T03:37:53.430945+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_clean_scrub_stamp":"2024-09-21T03:37:09.809805+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:35:32.936824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.053062+0000","last_change":"2024-09-21T03:37:53.052940+0000","last_active":"2024-09-21T03:37:53.053062+0000","last_peered":"2024-09-21T03:37:53.053062+0000","last_clean":"2024-09-21T03:37:53.053062+0000","last_became_active":"2024-09-21T03:37:53.052310+0000","last_became_peered":"2024-09-21T03:37:53.052310+0000","last_unstale":"2024-09-21T03:37:53.053062+0000","last_undegraded":"2024-09-21T03:37:53.053062+0000","last_fullsized":"2024-09-21T03:37:53.053062+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_clean_scrub_stamp":"2024-09-21T03:37:18.876933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:31:35.353226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,11,6,10],"acting":[5,11,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.046076+0000","last_change":"2024-09-21T03:37:12.869795+0000","last_active":"2024-09-21T03:37:53.046076+0000","last_peered":"2024-09-21T03:37:53.046076+0000","last_clean":"2024-09-21T03:37:53.046076+0000","last_became_active":"2024-09-21T03:37:12.869381+0000","last_became_peered":"2024-09-21T03:37:12.869381+0000","last_unstale":"2024-09-21T03:37:53.046076+0000","last_undegraded":"2024-09-21T03:37:53.046076+0000","last_fullsized":"2024-09-21T03:37:53.046076+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:39:29.437552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"80'3","reported_seq":40,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.044859+0000","last_change":"2024-09-21T03:37:10.834853+0000","last_active":"2024-09-21T03:37:53.044859+0000","last_peered":"2024-09-21T03:37:53.044859+0000","last_clean":"2024-09-21T03:37:53.044859+0000","last_became_active":"2024-09-21T03:37:10.834402+0000","last_became_peered":"2024-09-21T03:37:10.834402+0000","last_unstale":"2024-09-21T03:37:53.044859+0000","last_undegraded":"2024-09-21T03:37:53.044859+0000","last_fullsized":"2024-09-21T03:37:53.044859+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_clean_scrub_stamp":"2024-09-21T03:37:09.809805+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:57:29.668570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"20'92","reported_seq":108,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.430997+0000","last_change":"2024-09-21T03:36:00.521781+0000","last_active":"2024-09-21T03:37:53.430997+0000","last_peered":"2024-09-21T03:37:53.430997+0000","last_clean":"2024-09-21T03:37:53.430997+0000","last_became_active":"2024-09-21T03:36:00.516468+0000","last_became_peered":"2024-09-21T03:36:00.516468+0000","last_unstale":"2024-09-21T03:37:53.430997+0000","last_undegraded":"2024-09-21T03:37:53.430997+0000","last_fullsized":"2024-09-21T03:37:53.430997+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:32:38.215432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:32:38.215432+0000","last_clean_scrub_stamp":"2024-09-21T03:32:38.215432+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:02:16.252544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"82'1","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.431151+0000","last_change":"2024-09-21T03:37:19.911309+0000","last_active":"2024-09-21T03:37:53.431151+0000","last_peered":"2024-09-21T03:37:53.431151+0000","last_clean":"2024-09-21T03:37:53.431151+0000","last_became_active":"2024-09-21T03:37:19.910778+0000","last_became_peered":"2024-09-21T03:37:19.910778+0000","last_unstale":"2024-09-21T03:37:53.431151+0000","last_undegraded":"2024-09-21T03:37:53.431151+0000","last_fullsized":"2024-09-21T03:37:53.431151+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_clean_scrub_stamp":"2024-09-21T03:37:18.876933+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:52:23.307983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.431249+0000","last_change":"2024-09-21T03:37:12.860707+0000","last_active":"2024-09-21T03:37:53.431249+0000","last_peered":"2024-09-21T03:37:53.431249+0000","last_clean":"2024-09-21T03:37:53.431249+0000","last_became_active":"2024-09-21T03:37:12.860341+0000","last_became_peered":"2024-09-21T03:37:12.860341+0000","last_unstale":"2024-09-21T03:37:53.431249+0000","last_undegraded":"2024-09-21T03:37:53.431249+0000","last_fullsized":"2024-09-21T03:37:53.431249+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:55:15.771259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"80'3","reported_seq":18,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.071234+0000","last_change":"2024-09-21T03:37:53.071125+0000","last_active":"2024-09-21T03:37:53.071234+0000","last_peered":"2024-09-21T03:37:53.071234+0000","last_clean":"2024-09-21T03:37:53.071234+0000","last_became_active":"2024-09-21T03:37:53.052237+0000","last_became_peered":"2024-09-21T03:37:53.052237+0000","last_unstale":"2024-09-21T03:37:53.071234+0000","last_undegraded":"2024-09-21T03:37:53.071234+0000","last_fullsized":"2024-09-21T03:37:53.071234+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_clean_scrub_stamp":"2024-09-21T03:37:09.809805+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:47:17.707045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,0],"acting":[4,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.111355+0000","last_change":"2024-09-21T03:37:19.909292+0000","last_active":"2024-09-21T03:37:53.111355+0000","last_peered":"2024-09-21T03:37:53.111355+0000","last_clean":"2024-09-21T03:37:53.111355+0000","last_became_active":"2024-09-21T03:37:19.908681+0000","last_became_peered":"2024-09-21T03:37:19.908681+0000","last_unstale":"2024-09-21T03:37:53.111355+0000","last_undegraded":"2024-09-21T03:37:53.111355+0000","last_fullsized":"2024-09-21T03:37:53.111355+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_clean_scrub_stamp":"2024-09-21T03:37:18.876933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:15:49.623109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.431136+0000","last_change":"2024-09-21T03:37:12.857678+0000","last_active":"2024-09-21T03:37:53.431136+0000","last_peered":"2024-09-21T03:37:53.431136+0000","last_clean":"2024-09-21T03:37:53.431136+0000","last_became_active":"2024-09-21T03:37:12.857266+0000","last_became_peered":"2024-09-21T03:37:12.857266+0000","last_unstale":"2024-09-21T03:37:53.431136+0000","last_undegraded":"2024-09-21T03:37:53.431136+0000","last_fullsized":"2024-09-21T03:37:53.431136+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:21:14.135784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"74'1","reported_seq":37,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.047567+0000","last_change":"2024-09-21T03:37:10.835323+0000","last_active":"2024-09-21T03:37:53.047567+0000","last_peered":"2024-09-21T03:37:53.047567+0000","last_clean":"2024-09-21T03:37:53.047567+0000","last_became_active":"2024-09-21T03:37:10.834950+0000","last_became_peered":"2024-09-21T03:37:10.834950+0000","last_unstale":"2024-09-21T03:37:53.047567+0000","last_undegraded":"2024-09-21T03:37:53.047567+0000","last_fullsized":"2024-09-21T03:37:53.047567+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_clean_scrub_stamp":"2024-09-21T03:37:09.809805+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:40:27.386342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.049010+0000","last_change":"2024-09-21T03:37:19.906360+0000","last_active":"2024-09-21T03:37:53.049010+0000","last_peered":"2024-09-21T03:37:53.049010+0000","last_clean":"2024-09-21T03:37:53.049010+0000","last_became_active":"2024-09-21T03:37:19.905888+0000","last_became_peered":"2024-09-21T03:37:19.905888+0000","last_unstale":"2024-09-21T03:37:53.049010+0000","last_undegraded":"2024-09-21T03:37:53.049010+0000","last_fullsized":"2024-09-21T03:37:53.049010+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_clean_scrub_stamp":"2024-09-21T03:37:18.876933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:13:55.885538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.431442+0000","last_change":"2024-09-21T03:37:12.849106+0000","last_active":"2024-09-21T03:37:53.431442+0000","last_peered":"2024-09-21T03:37:53.431442+0000","last_clean":"2024-09-21T03:37:53.431442+0000","last_became_active":"2024-09-21T03:37:12.848505+0000","last_became_peered":"2024-09-21T03:37:12.848505+0000","last_unstale":"2024-09-21T03:37:53.431442+0000","last_undegraded":"2024-09-21T03:37:53.431442+0000","last_fullsized":"2024-09-21T03:37:53.431442+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:50:19.569004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"74'2","reported_seq":48,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.115238+0000","last_change":"2024-09-21T03:37:53.115158+0000","last_active":"2024-09-21T03:37:53.115238+0000","last_peered":"2024-09-21T03:37:53.115238+0000","last_clean":"2024-09-21T03:37:53.115238+0000","last_became_active":"2024-09-21T03:37:53.112603+0000","last_became_peered":"2024-09-21T03:37:53.112603+0000","last_unstale":"2024-09-21T03:37:53.115238+0000","last_undegraded":"2024-09-21T03:37:53.115238+0000","last_fullsized":"2024-09-21T03:37:53.115238+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_clean_scrub_stamp":"2024-09-21T03:37:09.809805+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:23:21.525972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,1],"acting":[6,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.045614+0000","last_change":"2024-09-21T03:37:19.909040+0000","last_active":"2024-09-21T03:37:53.045614+0000","last_peered":"2024-09-21T03:37:53.045614+0000","last_clean":"2024-09-21T03:37:53.045614+0000","last_became_active":"2024-09-21T03:37:19.908530+0000","last_became_peered":"2024-09-21T03:37:19.908530+0000","last_unstale":"2024-09-21T03:37:53.045614+0000","last_undegraded":"2024-09-21T03:37:53.045614+0000","last_fullsized":"2024-09-21T03:37:53.045614+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_clean_scrub_stamp":"2024-09-21T03:37:18.876933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:35:20.919260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.049078+0000","last_change":"2024-09-21T03:37:12.858984+0000","last_active":"2024-09-21T03:37:53.049078+0000","last_peered":"2024-09-21T03:37:53.049078+0000","last_clean":"2024-09-21T03:37:53.049078+0000","last_became_active":"2024-09-21T03:37:12.858603+0000","last_became_peered":"2024-09-21T03:37:12.858603+0000","last_unstale":"2024-09-21T03:37:53.049078+0000","last_undegraded":"2024-09-21T03:37:53.049078+0000","last_fullsized":"2024-09-21T03:37:53.049078+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:16:45.025304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.f","version":"80'2","reported_seq":38,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.050780+0000","last_change":"2024-09-21T03:37:10.834844+0000","last_active":"2024-09-21T03:37:53.050780+0000","last_peered":"2024-09-21T03:37:53.050780+0000","last_clean":"2024-09-21T03:37:53.050780+0000","last_became_active":"2024-09-21T03:37:10.833664+0000","last_became_peered":"2024-09-21T03:37:10.833664+0000","last_unstale":"2024-09-21T03:37:53.050780+0000","last_undegraded":"2024-09-21T03:37:53.050780+0000","last_fullsized":"2024-09-21T03:37:53.050780+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_clean_scrub_stamp":"2024-09-21T03:37:09.809805+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:03:49.006414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.046180+0000","last_change":"2024-09-21T03:37:19.913277+0000","last_active":"2024-09-21T03:37:53.046180+0000","last_peered":"2024-09-21T03:37:53.046180+0000","last_clean":"2024-09-21T03:37:53.046180+0000","last_became_active":"2024-09-21T03:37:19.912781+0000","last_became_peered":"2024-09-21T03:37:19.912781+0000","last_unstale":"2024-09-21T03:37:53.046180+0000","last_undegraded":"2024-09-21T03:37:53.046180+0000","last_fullsized":"2024-09-21T03:37:53.046180+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_clean_scrub_stamp":"2024-09-21T03:37:18.876933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:18:27.980622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.046359+0000","last_change":"2024-09-21T03:37:12.862322+0000","last_active":"2024-09-21T03:37:53.046359+0000","last_peered":"2024-09-21T03:37:53.046359+0000","last_clean":"2024-09-21T03:37:53.046359+0000","last_became_active":"2024-09-21T03:37:12.861938+0000","last_became_peered":"2024-09-21T03:37:12.861938+0000","last_unstale":"2024-09-21T03:37:53.046359+0000","last_undegraded":"2024-09-21T03:37:53.046359+0000","last_fullsized":"2024-09-21T03:37:53.046359+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:01:48.944169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.10","version":"80'15","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.067663+0000","last_change":"2024-09-21T03:37:53.067130+0000","last_active":"2024-09-21T03:37:53.067663+0000","last_peered":"2024-09-21T03:37:53.067663+0000","last_clean":"2024-09-21T03:37:53.067663+0000","last_became_active":"2024-09-21T03:37:53.051596+0000","last_became_peered":"2024-09-21T03:37:53.051596+0000","last_unstale":"2024-09-21T03:37:53.067663+0000","last_undegraded":"2024-09-21T03:37:53.067663+0000","last_fullsized":"2024-09-21T03:37:53.067663+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_clean_scrub_stamp":"2024-09-21T03:37:09.809805+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:02:22.852015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.043129+0000","last_change":"2024-09-21T03:37:19.913206+0000","last_active":"2024-09-21T03:37:53.043129+0000","last_peered":"2024-09-21T03:37:53.043129+0000","last_clean":"2024-09-21T03:37:53.043129+0000","last_became_active":"2024-09-21T03:37:19.910718+0000","last_became_peered":"2024-09-21T03:37:19.910718+0000","last_unstale":"2024-09-21T03:37:53.043129+0000","last_undegraded":"2024-09-21T03:37:53.043129+0000","last_fullsized":"2024-09-21T03:37:53.043129+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_clean_scrub_stamp":"2024-09-21T03:37:18.876933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:11:31.431237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.430692+0000","last_change":"2024-09-21T03:37:12.857505+0000","last_active":"2024-09-21T03:37:53.430692+0000","last_peered":"2024-09-21T03:37:53.430692+0000","last_clean":"2024-09-21T03:37:53.430692+0000","last_became_active":"2024-09-21T03:37:12.857155+0000","last_became_peered":"2024-09-21T03:37:12.857155+0000","last_unstale":"2024-09-21T03:37:53.430692+0000","last_undegraded":"2024-09-21T03:37:53.430692+0000","last_fullsized":"2024-09-21T03:37:53.430692+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:05:50.387651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"80'3","reported_seq":54,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:38:01.047784+0000","last_change":"2024-09-21T03:37:10.832364+0000","last_active":"2024-09-21T03:38:01.047784+0000","last_peered":"2024-09-21T03:38:01.047784+0000","last_clean":"2024-09-21T03:38:01.047784+0000","last_became_active":"2024-09-21T03:37:10.832004+0000","last_became_peered":"2024-09-21T03:37:10.832004+0000","last_unstale":"2024-09-21T03:38:01.047784+0000","last_undegraded":"2024-09-21T03:38:01.047784+0000","last_fullsized":"2024-09-21T03:38:01.047784+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_clean_scrub_stamp":"2024-09-21T03:37:09.809805+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:21:10.895732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":17,"num_read_kb":3,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.047568+0000","last_change":"2024-09-21T03:37:19.912297+0000","last_active":"2024-09-21T03:37:53.047568+0000","last_peered":"2024-09-21T03:37:53.047568+0000","last_clean":"2024-09-21T03:37:53.047568+0000","last_became_active":"2024-09-21T03:37:19.911903+0000","last_became_peered":"2024-09-21T03:37:19.911903+0000","last_unstale":"2024-09-21T03:37:53.047568+0000","last_undegraded":"2024-09-21T03:37:53.047568+0000","last_fullsized":"2024-09-21T03:37:53.047568+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_clean_scrub_stamp":"2024-09-21T03:37:18.876933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:05:56.747246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.431059+0000","last_change":"2024-09-21T03:37:12.856230+0000","last_active":"2024-09-21T03:37:53.431059+0000","last_peered":"2024-09-21T03:37:53.431059+0000","last_clean":"2024-09-21T03:37:53.431059+0000","last_became_active":"2024-09-21T03:37:12.855819+0000","last_became_peered":"2024-09-21T03:37:12.855819+0000","last_unstale":"2024-09-21T03:37:53.431059+0000","last_undegraded":"2024-09-21T03:37:53.431059+0000","last_fullsized":"2024-09-21T03:37:53.431059+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:38:18.618579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"80'3","reported_seq":38,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.046856+0000","last_change":"2024-09-21T03:37:10.832401+0000","last_active":"2024-09-21T03:37:53.046856+0000","last_peered":"2024-09-21T03:37:53.046856+0000","last_clean":"2024-09-21T03:37:53.046856+0000","last_became_active":"2024-09-21T03:37:10.832033+0000","last_became_peered":"2024-09-21T03:37:10.832033+0000","last_unstale":"2024-09-21T03:37:53.046856+0000","last_undegraded":"2024-09-21T03:37:53.046856+0000","last_fullsized":"2024-09-21T03:37:53.046856+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_clean_scrub_stamp":"2024-09-21T03:37:09.809805+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:25:20.751529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.048479+0000","last_change":"2024-09-21T03:37:19.905825+0000","last_active":"2024-09-21T03:37:53.048479+0000","last_peered":"2024-09-21T03:37:53.048479+0000","last_clean":"2024-09-21T03:37:53.048479+0000","last_became_active":"2024-09-21T03:37:19.905352+0000","last_became_peered":"2024-09-21T03:37:19.905352+0000","last_unstale":"2024-09-21T03:37:53.048479+0000","last_undegraded":"2024-09-21T03:37:53.048479+0000","last_fullsized":"2024-09-21T03:37:53.048479+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_clean_scrub_stamp":"2024-09-21T03:37:18.876933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:25:05.861981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.042684+0000","last_change":"2024-09-21T03:37:12.852006+0000","last_active":"2024-09-21T03:37:53.042684+0000","last_peered":"2024-09-21T03:37:53.042684+0000","last_clean":"2024-09-21T03:37:53.042684+0000","last_became_active":"2024-09-21T03:37:12.851670+0000","last_became_peered":"2024-09-21T03:37:12.851670+0000","last_unstale":"2024-09-21T03:37:53.042684+0000","last_undegraded":"2024-09-21T03:37:53.042684+0000","last_fullsized":"2024-09-21T03:37:53.042684+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:41:50.177349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"80'6","reported_seq":42,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.044608+0000","last_change":"2024-09-21T03:37:10.832472+0000","last_active":"2024-09-21T03:37:53.044608+0000","last_peered":"2024-09-21T03:37:53.044608+0000","last_clean":"2024-09-21T03:37:53.044608+0000","last_became_active":"2024-09-21T03:37:10.832085+0000","last_became_peered":"2024-09-21T03:37:10.832085+0000","last_unstale":"2024-09-21T03:37:53.044608+0000","last_undegraded":"2024-09-21T03:37:53.044608+0000","last_fullsized":"2024-09-21T03:37:53.044608+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_clean_scrub_stamp":"2024-09-21T03:37:09.809805+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:06:41.251218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.046632+0000","last_change":"2024-09-21T03:37:19.914042+0000","last_active":"2024-09-21T03:37:53.046632+0000","last_peered":"2024-09-21T03:37:53.046632+0000","last_clean":"2024-09-21T03:37:53.046632+0000","last_became_active":"2024-09-21T03:37:19.913531+0000","last_became_peered":"2024-09-21T03:37:19.913531+0000","last_unstale":"2024-09-21T03:37:53.046632+0000","last_undegraded":"2024-09-21T03:37:53.046632+0000","last_fullsized":"2024-09-21T03:37:53.046632+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_clean_scrub_stamp":"2024-09-21T03:37:18.876933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:16:40.163359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.050691+0000","last_change":"2024-09-21T03:37:12.852893+0000","last_active":"2024-09-21T03:37:53.050691+0000","last_peered":"2024-09-21T03:37:53.050691+0000","last_clean":"2024-09-21T03:37:53.050691+0000","last_became_active":"2024-09-21T03:37:12.852452+0000","last_became_peered":"2024-09-21T03:37:12.852452+0000","last_unstale":"2024-09-21T03:37:53.050691+0000","last_undegraded":"2024-09-21T03:37:53.050691+0000","last_fullsized":"2024-09-21T03:37:53.050691+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:46:59.393901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.14","version":"82'29","reported_seq":103,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:38:04.053675+0000","last_change":"2024-09-21T03:37:10.837654+0000","last_active":"2024-09-21T03:38:04.053675+0000","last_peered":"2024-09-21T03:38:04.053675+0000","last_clean":"2024-09-21T03:38:04.053675+0000","last_became_active":"2024-09-21T03:37:10.837251+0000","last_became_peered":"2024-09-21T03:37:10.837251+0000","last_unstale":"2024-09-21T03:38:04.053675+0000","last_undegraded":"2024-09-21T03:38:04.053675+0000","last_fullsized":"2024-09-21T03:38:04.053675+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_clean_scrub_stamp":"2024-09-21T03:37:09.809805+0000","objects_scrubbed":0,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:43:32.986371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":60885,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":39,"num_read_kb":79,"num_write":30,"num_write_kb":72,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.053980+0000","last_change":"2024-09-21T03:37:53.053405+0000","last_active":"2024-09-21T03:37:53.053980+0000","last_peered":"2024-09-21T03:37:53.053980+0000","last_clean":"2024-09-21T03:37:53.053980+0000","last_became_active":"2024-09-21T03:37:53.052818+0000","last_became_peered":"2024-09-21T03:37:53.052818+0000","last_unstale":"2024-09-21T03:37:53.053980+0000","last_undegraded":"2024-09-21T03:37:53.053980+0000","last_fullsized":"2024-09-21T03:37:53.053980+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_clean_scrub_stamp":"2024-09-21T03:37:18.876933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:05:09.121036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,11,5],"acting":[10,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.111585+0000","last_change":"2024-09-21T03:37:12.866588+0000","last_active":"2024-09-21T03:37:53.111585+0000","last_peered":"2024-09-21T03:37:53.111585+0000","last_clean":"2024-09-21T03:37:53.111585+0000","last_became_active":"2024-09-21T03:37:12.866075+0000","last_became_peered":"2024-09-21T03:37:12.866075+0000","last_unstale":"2024-09-21T03:37:53.111585+0000","last_undegraded":"2024-09-21T03:37:53.111585+0000","last_fullsized":"2024-09-21T03:37:53.111585+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:18:48.817562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"80'4","reported_seq":40,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.047298+0000","last_change":"2024-09-21T03:37:11.036751+0000","last_active":"2024-09-21T03:37:53.047298+0000","last_peered":"2024-09-21T03:37:53.047298+0000","last_clean":"2024-09-21T03:37:53.047298+0000","last_became_active":"2024-09-21T03:37:11.036125+0000","last_became_peered":"2024-09-21T03:37:11.036125+0000","last_unstale":"2024-09-21T03:37:53.047298+0000","last_undegraded":"2024-09-21T03:37:53.047298+0000","last_fullsized":"2024-09-21T03:37:53.047298+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_clean_scrub_stamp":"2024-09-21T03:37:09.809805+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:46:13.569422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.112980+0000","last_change":"2024-09-21T03:37:53.112845+0000","last_active":"2024-09-21T03:37:53.112980+0000","last_peered":"2024-09-21T03:37:53.112980+0000","last_clean":"2024-09-21T03:37:53.112980+0000","last_became_active":"2024-09-21T03:37:53.112286+0000","last_became_peered":"2024-09-21T03:37:53.112286+0000","last_unstale":"2024-09-21T03:37:53.112980+0000","last_undegraded":"2024-09-21T03:37:53.112980+0000","last_fullsized":"2024-09-21T03:37:53.112980+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_clean_scrub_stamp":"2024-09-21T03:37:18.876933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:29:58.062684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,8],"acting":[4,1,2,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.049603+0000","last_change":"2024-09-21T03:37:12.865011+0000","last_active":"2024-09-21T03:37:53.049603+0000","last_peered":"2024-09-21T03:37:53.049603+0000","last_clean":"2024-09-21T03:37:53.049603+0000","last_became_active":"2024-09-21T03:37:12.864645+0000","last_became_peered":"2024-09-21T03:37:12.864645+0000","last_unstale":"2024-09-21T03:37:53.049603+0000","last_undegraded":"2024-09-21T03:37:53.049603+0000","last_fullsized":"2024-09-21T03:37:53.049603+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:27:42.471227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"80'2","reported_seq":37,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.431360+0000","last_change":"2024-09-21T03:37:10.835171+0000","last_active":"2024-09-21T03:37:53.431360+0000","last_peered":"2024-09-21T03:37:53.431360+0000","last_clean":"2024-09-21T03:37:53.431360+0000","last_became_active":"2024-09-21T03:37:10.834744+0000","last_became_peered":"2024-09-21T03:37:10.834744+0000","last_unstale":"2024-09-21T03:37:53.431360+0000","last_undegraded":"2024-09-21T03:37:53.431360+0000","last_fullsized":"2024-09-21T03:37:53.431360+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_clean_scrub_stamp":"2024-09-21T03:37:09.809805+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:02:37.720457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.046301+0000","last_change":"2024-09-21T03:37:19.906734+0000","last_active":"2024-09-21T03:37:53.046301+0000","last_peered":"2024-09-21T03:37:53.046301+0000","last_clean":"2024-09-21T03:37:53.046301+0000","last_became_active":"2024-09-21T03:37:19.906327+0000","last_became_peered":"2024-09-21T03:37:19.906327+0000","last_unstale":"2024-09-21T03:37:53.046301+0000","last_undegraded":"2024-09-21T03:37:53.046301+0000","last_fullsized":"2024-09-21T03:37:53.046301+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_clean_scrub_stamp":"2024-09-21T03:37:18.876933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:55:11.603042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.111948+0000","last_change":"2024-09-21T03:37:12.866285+0000","last_active":"2024-09-21T03:37:53.111948+0000","last_peered":"2024-09-21T03:37:53.111948+0000","last_clean":"2024-09-21T03:37:53.111948+0000","last_became_active":"2024-09-21T03:37:12.865854+0000","last_became_peered":"2024-09-21T03:37:12.865854+0000","last_unstale":"2024-09-21T03:37:53.111948+0000","last_undegraded":"2024-09-21T03:37:53.111948+0000","last_fullsized":"2024-09-21T03:37:53.111948+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:00:18.075248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"80'3","reported_seq":54,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:38:03.051279+0000","last_change":"2024-09-21T03:37:10.835808+0000","last_active":"2024-09-21T03:38:03.051279+0000","last_peered":"2024-09-21T03:38:03.051279+0000","last_clean":"2024-09-21T03:38:03.051279+0000","last_became_active":"2024-09-21T03:37:10.835434+0000","last_became_peered":"2024-09-21T03:37:10.835434+0000","last_unstale":"2024-09-21T03:38:03.051279+0000","last_undegraded":"2024-09-21T03:38:03.051279+0000","last_fullsized":"2024-09-21T03:38:03.051279+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_clean_scrub_stamp":"2024-09-21T03:37:09.809805+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:35:39.817312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":16,"num_read_kb":16,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.045411+0000","last_change":"2024-09-21T03:37:19.915986+0000","last_active":"2024-09-21T03:37:53.045411+0000","last_peered":"2024-09-21T03:37:53.045411+0000","last_clean":"2024-09-21T03:37:53.045411+0000","last_became_active":"2024-09-21T03:37:19.915561+0000","last_became_peered":"2024-09-21T03:37:19.915561+0000","last_unstale":"2024-09-21T03:37:53.045411+0000","last_undegraded":"2024-09-21T03:37:53.045411+0000","last_fullsized":"2024-09-21T03:37:53.045411+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_clean_scrub_stamp":"2024-09-21T03:37:18.876933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:21:13.789664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.049328+0000","last_change":"2024-09-21T03:37:19.908230+0000","last_active":"2024-09-21T03:37:53.049328+0000","last_peered":"2024-09-21T03:37:53.049328+0000","last_clean":"2024-09-21T03:37:53.049328+0000","last_became_active":"2024-09-21T03:37:19.907843+0000","last_became_peered":"2024-09-21T03:37:19.907843+0000","last_unstale":"2024-09-21T03:37:53.049328+0000","last_undegraded":"2024-09-21T03:37:53.049328+0000","last_fullsized":"2024-09-21T03:37:53.049328+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_clean_scrub_stamp":"2024-09-21T03:37:18.876933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:15:58.142127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.046673+0000","last_change":"2024-09-21T03:37:12.856709+0000","last_active":"2024-09-21T03:37:53.046673+0000","last_peered":"2024-09-21T03:37:53.046673+0000","last_clean":"2024-09-21T03:37:53.046673+0000","last_became_active":"2024-09-21T03:37:12.856278+0000","last_became_peered":"2024-09-21T03:37:12.856278+0000","last_unstale":"2024-09-21T03:37:53.046673+0000","last_undegraded":"2024-09-21T03:37:53.046673+0000","last_fullsized":"2024-09-21T03:37:53.046673+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:16:13.884648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"80'1","reported_seq":36,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.046329+0000","last_change":"2024-09-21T03:37:10.832456+0000","last_active":"2024-09-21T03:37:53.046329+0000","last_peered":"2024-09-21T03:37:53.046329+0000","last_clean":"2024-09-21T03:37:53.046329+0000","last_became_active":"2024-09-21T03:37:10.832082+0000","last_became_peered":"2024-09-21T03:37:10.832082+0000","last_unstale":"2024-09-21T03:37:53.046329+0000","last_undegraded":"2024-09-21T03:37:53.046329+0000","last_fullsized":"2024-09-21T03:37:53.046329+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_clean_scrub_stamp":"2024-09-21T03:37:09.809805+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:17:12.170616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.048472+0000","last_change":"2024-09-21T03:37:19.908084+0000","last_active":"2024-09-21T03:37:53.048472+0000","last_peered":"2024-09-21T03:37:53.048472+0000","last_clean":"2024-09-21T03:37:53.048472+0000","last_became_active":"2024-09-21T03:37:19.907307+0000","last_became_peered":"2024-09-21T03:37:19.907307+0000","last_unstale":"2024-09-21T03:37:53.048472+0000","last_undegraded":"2024-09-21T03:37:53.048472+0000","last_fullsized":"2024-09-21T03:37:53.048472+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_clean_scrub_stamp":"2024-09-21T03:37:18.876933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:45:02.541015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.045568+0000","last_change":"2024-09-21T03:37:12.852577+0000","last_active":"2024-09-21T03:37:53.045568+0000","last_peered":"2024-09-21T03:37:53.045568+0000","last_clean":"2024-09-21T03:37:53.045568+0000","last_became_active":"2024-09-21T03:37:12.852186+0000","last_became_peered":"2024-09-21T03:37:12.852186+0000","last_unstale":"2024-09-21T03:37:53.045568+0000","last_undegraded":"2024-09-21T03:37:53.045568+0000","last_fullsized":"2024-09-21T03:37:53.045568+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:51:21.631029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"80'2","reported_seq":37,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.042546+0000","last_change":"2024-09-21T03:37:10.833587+0000","last_active":"2024-09-21T03:37:53.042546+0000","last_peered":"2024-09-21T03:37:53.042546+0000","last_clean":"2024-09-21T03:37:53.042546+0000","last_became_active":"2024-09-21T03:37:10.833194+0000","last_became_peered":"2024-09-21T03:37:10.833194+0000","last_unstale":"2024-09-21T03:37:53.042546+0000","last_undegraded":"2024-09-21T03:37:53.042546+0000","last_fullsized":"2024-09-21T03:37:53.042546+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_clean_scrub_stamp":"2024-09-21T03:37:09.809805+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:20:20.272108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"80'3","reported_seq":38,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.047496+0000","last_change":"2024-09-21T03:37:10.839103+0000","last_active":"2024-09-21T03:37:53.047496+0000","last_peered":"2024-09-21T03:37:53.047496+0000","last_clean":"2024-09-21T03:37:53.047496+0000","last_became_active":"2024-09-21T03:37:10.837192+0000","last_became_peered":"2024-09-21T03:37:10.837192+0000","last_unstale":"2024-09-21T03:37:53.047496+0000","last_undegraded":"2024-09-21T03:37:53.047496+0000","last_fullsized":"2024-09-21T03:37:53.047496+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_clean_scrub_stamp":"2024-09-21T03:37:09.809805+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:59:35.128171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.047232+0000","last_change":"2024-09-21T03:37:12.866626+0000","last_active":"2024-09-21T03:37:53.047232+0000","last_peered":"2024-09-21T03:37:53.047232+0000","last_clean":"2024-09-21T03:37:53.047232+0000","last_became_active":"2024-09-21T03:37:12.866254+0000","last_became_peered":"2024-09-21T03:37:12.866254+0000","last_unstale":"2024-09-21T03:37:53.047232+0000","last_undegraded":"2024-09-21T03:37:53.047232+0000","last_fullsized":"2024-09-21T03:37:53.047232+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:39:00.361254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.044954+0000","last_change":"2024-09-21T03:37:19.909415+0000","last_active":"2024-09-21T03:37:53.044954+0000","last_peered":"2024-09-21T03:37:53.044954+0000","last_clean":"2024-09-21T03:37:53.044954+0000","last_became_active":"2024-09-21T03:37:19.908955+0000","last_became_peered":"2024-09-21T03:37:19.908955+0000","last_unstale":"2024-09-21T03:37:53.044954+0000","last_undegraded":"2024-09-21T03:37:53.044954+0000","last_fullsized":"2024-09-21T03:37:53.044954+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_clean_scrub_stamp":"2024-09-21T03:37:18.876933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:31:18.878725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"81'6","reported_seq":57,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:38:00.047096+0000","last_change":"2024-09-21T03:37:10.836389+0000","last_active":"2024-09-21T03:38:00.047096+0000","last_peered":"2024-09-21T03:38:00.047096+0000","last_clean":"2024-09-21T03:38:00.047096+0000","last_became_active":"2024-09-21T03:37:10.835986+0000","last_became_peered":"2024-09-21T03:37:10.835986+0000","last_unstale":"2024-09-21T03:38:00.047096+0000","last_undegraded":"2024-09-21T03:38:00.047096+0000","last_fullsized":"2024-09-21T03:38:00.047096+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_clean_scrub_stamp":"2024-09-21T03:37:09.809805+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:05:17.963609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1013,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":16,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.050861+0000","last_change":"2024-09-21T03:37:12.856008+0000","last_active":"2024-09-21T03:37:53.050861+0000","last_peered":"2024-09-21T03:37:53.050861+0000","last_clean":"2024-09-21T03:37:53.050861+0000","last_became_active":"2024-09-21T03:37:12.855603+0000","last_became_peered":"2024-09-21T03:37:12.855603+0000","last_unstale":"2024-09-21T03:37:53.050861+0000","last_undegraded":"2024-09-21T03:37:53.050861+0000","last_fullsized":"2024-09-21T03:37:53.050861+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:11:41.920114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.045051+0000","last_change":"2024-09-21T03:37:19.911507+0000","last_active":"2024-09-21T03:37:53.045051+0000","last_peered":"2024-09-21T03:37:53.045051+0000","last_clean":"2024-09-21T03:37:53.045051+0000","last_became_active":"2024-09-21T03:37:19.911053+0000","last_became_peered":"2024-09-21T03:37:19.911053+0000","last_unstale":"2024-09-21T03:37:53.045051+0000","last_undegraded":"2024-09-21T03:37:53.045051+0000","last_fullsized":"2024-09-21T03:37:53.045051+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_clean_scrub_stamp":"2024-09-21T03:37:18.876933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:42:11.964829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.048840+0000","last_change":"2024-09-21T03:37:12.862576+0000","last_active":"2024-09-21T03:37:53.048840+0000","last_peered":"2024-09-21T03:37:53.048840+0000","last_clean":"2024-09-21T03:37:53.048840+0000","last_became_active":"2024-09-21T03:37:12.862146+0000","last_became_peered":"2024-09-21T03:37:12.862146+0000","last_unstale":"2024-09-21T03:37:53.048840+0000","last_undegraded":"2024-09-21T03:37:53.048840+0000","last_fullsized":"2024-09-21T03:37:53.048840+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:30:33.272415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.044731+0000","last_change":"2024-09-21T03:37:12.859395+0000","last_active":"2024-09-21T03:37:53.044731+0000","last_peered":"2024-09-21T03:37:53.044731+0000","last_clean":"2024-09-21T03:37:53.044731+0000","last_became_active":"2024-09-21T03:37:12.858914+0000","last_became_peered":"2024-09-21T03:37:12.858914+0000","last_unstale":"2024-09-21T03:37:53.044731+0000","last_undegraded":"2024-09-21T03:37:53.044731+0000","last_fullsized":"2024-09-21T03:37:53.044731+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:13:06.319896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.044203+0000","last_change":"2024-09-21T03:37:12.857467+0000","last_active":"2024-09-21T03:37:53.044203+0000","last_peered":"2024-09-21T03:37:53.044203+0000","last_clean":"2024-09-21T03:37:53.044203+0000","last_became_active":"2024-09-21T03:37:12.856214+0000","last_became_peered":"2024-09-21T03:37:12.856214+0000","last_unstale":"2024-09-21T03:37:53.044203+0000","last_undegraded":"2024-09-21T03:37:53.044203+0000","last_fullsized":"2024-09-21T03:37:53.044203+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:21:58.780018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.049029+0000","last_change":"2024-09-21T03:37:12.858052+0000","last_active":"2024-09-21T03:37:53.049029+0000","last_peered":"2024-09-21T03:37:53.049029+0000","last_clean":"2024-09-21T03:37:53.049029+0000","last_became_active":"2024-09-21T03:37:12.857445+0000","last_became_peered":"2024-09-21T03:37:12.857445+0000","last_unstale":"2024-09-21T03:37:53.049029+0000","last_undegraded":"2024-09-21T03:37:53.049029+0000","last_fullsized":"2024-09-21T03:37:53.049029+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:22:28.058846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.042880+0000","last_change":"2024-09-21T03:37:12.856499+0000","last_active":"2024-09-21T03:37:53.042880+0000","last_peered":"2024-09-21T03:37:53.042880+0000","last_clean":"2024-09-21T03:37:53.042880+0000","last_became_active":"2024-09-21T03:37:12.856146+0000","last_became_peered":"2024-09-21T03:37:12.856146+0000","last_unstale":"2024-09-21T03:37:53.042880+0000","last_undegraded":"2024-09-21T03:37:53.042880+0000","last_fullsized":"2024-09-21T03:37:53.042880+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:03:26.142740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.048901+0000","last_change":"2024-09-21T03:37:12.860830+0000","last_active":"2024-09-21T03:37:53.048901+0000","last_peered":"2024-09-21T03:37:53.048901+0000","last_clean":"2024-09-21T03:37:53.048901+0000","last_became_active":"2024-09-21T03:37:12.860468+0000","last_became_peered":"2024-09-21T03:37:12.860468+0000","last_unstale":"2024-09-21T03:37:53.048901+0000","last_undegraded":"2024-09-21T03:37:53.048901+0000","last_fullsized":"2024-09-21T03:37:53.048901+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:48:15.057622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.046546+0000","last_change":"2024-09-21T03:37:12.864399+0000","last_active":"2024-09-21T03:37:53.046546+0000","last_peered":"2024-09-21T03:37:53.046546+0000","last_clean":"2024-09-21T03:37:53.046546+0000","last_became_active":"2024-09-21T03:37:12.863892+0000","last_became_peered":"2024-09-21T03:37:12.863892+0000","last_unstale":"2024-09-21T03:37:53.046546+0000","last_undegraded":"2024-09-21T03:37:53.046546+0000","last_fullsized":"2024-09-21T03:37:53.046546+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:58:26.647896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.046313+0000","last_change":"2024-09-21T03:37:12.861716+0000","last_active":"2024-09-21T03:37:53.046313+0000","last_peered":"2024-09-21T03:37:53.046313+0000","last_clean":"2024-09-21T03:37:53.046313+0000","last_became_active":"2024-09-21T03:37:12.861392+0000","last_became_peered":"2024-09-21T03:37:12.861392+0000","last_unstale":"2024-09-21T03:37:53.046313+0000","last_undegraded":"2024-09-21T03:37:53.046313+0000","last_fullsized":"2024-09-21T03:37:53.046313+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:28:55.751107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.045975+0000","last_change":"2024-09-21T03:37:12.856167+0000","last_active":"2024-09-21T03:37:53.045975+0000","last_peered":"2024-09-21T03:37:53.045975+0000","last_clean":"2024-09-21T03:37:53.045975+0000","last_became_active":"2024-09-21T03:37:12.855752+0000","last_became_peered":"2024-09-21T03:37:12.855752+0000","last_unstale":"2024-09-21T03:37:53.045975+0000","last_undegraded":"2024-09-21T03:37:53.045975+0000","last_fullsized":"2024-09-21T03:37:53.045975+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:26:30.271342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.045615+0000","last_change":"2024-09-21T03:37:12.862782+0000","last_active":"2024-09-21T03:37:53.045615+0000","last_peered":"2024-09-21T03:37:53.045615+0000","last_clean":"2024-09-21T03:37:53.045615+0000","last_became_active":"2024-09-21T03:37:12.862462+0000","last_became_peered":"2024-09-21T03:37:12.862462+0000","last_unstale":"2024-09-21T03:37:53.045615+0000","last_undegraded":"2024-09-21T03:37:53.045615+0000","last_fullsized":"2024-09-21T03:37:53.045615+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:57:37.744531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.111623+0000","last_change":"2024-09-21T03:37:12.862312+0000","last_active":"2024-09-21T03:37:53.111623+0000","last_peered":"2024-09-21T03:37:53.111623+0000","last_clean":"2024-09-21T03:37:53.111623+0000","last_became_active":"2024-09-21T03:37:12.861940+0000","last_became_peered":"2024-09-21T03:37:12.861940+0000","last_unstale":"2024-09-21T03:37:53.111623+0000","last_undegraded":"2024-09-21T03:37:53.111623+0000","last_fullsized":"2024-09-21T03:37:53.111623+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:08:49.165522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.042830+0000","last_change":"2024-09-21T03:37:12.857012+0000","last_active":"2024-09-21T03:37:53.042830+0000","last_peered":"2024-09-21T03:37:53.042830+0000","last_clean":"2024-09-21T03:37:53.042830+0000","last_became_active":"2024-09-21T03:37:12.856390+0000","last_became_peered":"2024-09-21T03:37:12.856390+0000","last_unstale":"2024-09-21T03:37:53.042830+0000","last_undegraded":"2024-09-21T03:37:53.042830+0000","last_fullsized":"2024-09-21T03:37:53.042830+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:51:00.839709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.048487+0000","last_change":"2024-09-21T03:37:12.857207+0000","last_active":"2024-09-21T03:37:53.048487+0000","last_peered":"2024-09-21T03:37:53.048487+0000","last_clean":"2024-09-21T03:37:53.048487+0000","last_became_active":"2024-09-21T03:37:12.856770+0000","last_became_peered":"2024-09-21T03:37:12.856770+0000","last_unstale":"2024-09-21T03:37:53.048487+0000","last_undegraded":"2024-09-21T03:37:53.048487+0000","last_fullsized":"2024-09-21T03:37:53.048487+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:37:18.272321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.045688+0000","last_change":"2024-09-21T03:37:12.858540+0000","last_active":"2024-09-21T03:37:53.045688+0000","last_peered":"2024-09-21T03:37:53.045688+0000","last_clean":"2024-09-21T03:37:53.045688+0000","last_became_active":"2024-09-21T03:37:12.857340+0000","last_became_peered":"2024-09-21T03:37:12.857340+0000","last_unstale":"2024-09-21T03:37:53.045688+0000","last_undegraded":"2024-09-21T03:37:53.045688+0000","last_fullsized":"2024-09-21T03:37:53.045688+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:45:48.982132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.047908+0000","last_change":"2024-09-21T03:37:12.864408+0000","last_active":"2024-09-21T03:37:53.047908+0000","last_peered":"2024-09-21T03:37:53.047908+0000","last_clean":"2024-09-21T03:37:53.047908+0000","last_became_active":"2024-09-21T03:37:12.864082+0000","last_became_peered":"2024-09-21T03:37:12.864082+0000","last_unstale":"2024-09-21T03:37:53.047908+0000","last_undegraded":"2024-09-21T03:37:53.047908+0000","last_fullsized":"2024-09-21T03:37:53.047908+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:19:32.723589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.049781+0000","last_change":"2024-09-21T03:37:12.869445+0000","last_active":"2024-09-21T03:37:53.049781+0000","last_peered":"2024-09-21T03:37:53.049781+0000","last_clean":"2024-09-21T03:37:53.049781+0000","last_became_active":"2024-09-21T03:37:12.869058+0000","last_became_peered":"2024-09-21T03:37:12.869058+0000","last_unstale":"2024-09-21T03:37:53.049781+0000","last_undegraded":"2024-09-21T03:37:53.049781+0000","last_fullsized":"2024-09-21T03:37:53.049781+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:37:57.750665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.111172+0000","last_change":"2024-09-21T03:37:12.858431+0000","last_active":"2024-09-21T03:37:53.111172+0000","last_peered":"2024-09-21T03:37:53.111172+0000","last_clean":"2024-09-21T03:37:53.111172+0000","last_became_active":"2024-09-21T03:37:12.858047+0000","last_became_peered":"2024-09-21T03:37:12.858047+0000","last_unstale":"2024-09-21T03:37:53.111172+0000","last_undegraded":"2024-09-21T03:37:53.111172+0000","last_fullsized":"2024-09-21T03:37:53.111172+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:14:05.363743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.048838+0000","last_change":"2024-09-21T03:37:12.866784+0000","last_active":"2024-09-21T03:37:53.048838+0000","last_peered":"2024-09-21T03:37:53.048838+0000","last_clean":"2024-09-21T03:37:53.048838+0000","last_became_active":"2024-09-21T03:37:12.866435+0000","last_became_peered":"2024-09-21T03:37:12.866435+0000","last_unstale":"2024-09-21T03:37:53.048838+0000","last_undegraded":"2024-09-21T03:37:53.048838+0000","last_fullsized":"2024-09-21T03:37:53.048838+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:09:06.156030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.048908+0000","last_change":"2024-09-21T03:37:12.857054+0000","last_active":"2024-09-21T03:37:53.048908+0000","last_peered":"2024-09-21T03:37:53.048908+0000","last_clean":"2024-09-21T03:37:53.048908+0000","last_became_active":"2024-09-21T03:37:12.856671+0000","last_became_peered":"2024-09-21T03:37:12.856671+0000","last_unstale":"2024-09-21T03:37:53.048908+0000","last_undegraded":"2024-09-21T03:37:53.048908+0000","last_fullsized":"2024-09-21T03:37:53.048908+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:38:27.474550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.042778+0000","last_change":"2024-09-21T03:37:12.861501+0000","last_active":"2024-09-21T03:37:53.042778+0000","last_peered":"2024-09-21T03:37:53.042778+0000","last_clean":"2024-09-21T03:37:53.042778+0000","last_became_active":"2024-09-21T03:37:12.861146+0000","last_became_peered":"2024-09-21T03:37:12.861146+0000","last_unstale":"2024-09-21T03:37:53.042778+0000","last_undegraded":"2024-09-21T03:37:53.042778+0000","last_fullsized":"2024-09-21T03:37:53.042778+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:30:55.073854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.052333+0000","last_change":"2024-09-21T03:37:53.051912+0000","last_active":"2024-09-21T03:37:53.052333+0000","last_peered":"2024-09-21T03:37:53.052333+0000","last_clean":"2024-09-21T03:37:53.052333+0000","last_became_active":"2024-09-21T03:37:53.051640+0000","last_became_peered":"2024-09-21T03:37:53.051640+0000","last_unstale":"2024-09-21T03:37:53.052333+0000","last_undegraded":"2024-09-21T03:37:53.052333+0000","last_fullsized":"2024-09-21T03:37:53.052333+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:08:09.046867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,10,8],"acting":[9,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.431260+0000","last_change":"2024-09-21T03:37:12.865862+0000","last_active":"2024-09-21T03:37:53.431260+0000","last_peered":"2024-09-21T03:37:53.431260+0000","last_clean":"2024-09-21T03:37:53.431260+0000","last_became_active":"2024-09-21T03:37:12.865468+0000","last_became_peered":"2024-09-21T03:37:12.865468+0000","last_unstale":"2024-09-21T03:37:53.431260+0000","last_undegraded":"2024-09-21T03:37:53.431260+0000","last_fullsized":"2024-09-21T03:37:53.431260+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:18:14.496495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.047575+0000","last_change":"2024-09-21T03:37:12.862688+0000","last_active":"2024-09-21T03:37:53.047575+0000","last_peered":"2024-09-21T03:37:53.047575+0000","last_clean":"2024-09-21T03:37:53.047575+0000","last_became_active":"2024-09-21T03:37:12.862249+0000","last_became_peered":"2024-09-21T03:37:12.862249+0000","last_unstale":"2024-09-21T03:37:53.047575+0000","last_undegraded":"2024-09-21T03:37:53.047575+0000","last_fullsized":"2024-09-21T03:37:53.047575+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:55:54.731640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.112095+0000","last_change":"2024-09-21T03:37:12.863595+0000","last_active":"2024-09-21T03:37:53.112095+0000","last_peered":"2024-09-21T03:37:53.112095+0000","last_clean":"2024-09-21T03:37:53.112095+0000","last_became_active":"2024-09-21T03:37:12.863260+0000","last_became_peered":"2024-09-21T03:37:12.863260+0000","last_unstale":"2024-09-21T03:37:53.112095+0000","last_undegraded":"2024-09-21T03:37:53.112095+0000","last_fullsized":"2024-09-21T03:37:53.112095+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:10:07.995690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.045457+0000","last_change":"2024-09-21T03:37:12.865770+0000","last_active":"2024-09-21T03:37:53.045457+0000","last_peered":"2024-09-21T03:37:53.045457+0000","last_clean":"2024-09-21T03:37:53.045457+0000","last_became_active":"2024-09-21T03:37:12.865405+0000","last_became_peered":"2024-09-21T03:37:12.865405+0000","last_unstale":"2024-09-21T03:37:53.045457+0000","last_undegraded":"2024-09-21T03:37:53.045457+0000","last_fullsized":"2024-09-21T03:37:53.045457+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:46:25.255913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.430807+0000","last_change":"2024-09-21T03:37:12.856214+0000","last_active":"2024-09-21T03:37:53.430807+0000","last_peered":"2024-09-21T03:37:53.430807+0000","last_clean":"2024-09-21T03:37:53.430807+0000","last_became_active":"2024-09-21T03:37:12.853924+0000","last_became_peered":"2024-09-21T03:37:12.853924+0000","last_unstale":"2024-09-21T03:37:53.430807+0000","last_undegraded":"2024-09-21T03:37:53.430807+0000","last_fullsized":"2024-09-21T03:37:53.430807+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:10:32.182892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.047154+0000","last_change":"2024-09-21T03:37:12.858851+0000","last_active":"2024-09-21T03:37:53.047154+0000","last_peered":"2024-09-21T03:37:53.047154+0000","last_clean":"2024-09-21T03:37:53.047154+0000","last_became_active":"2024-09-21T03:37:12.858037+0000","last_became_peered":"2024-09-21T03:37:12.858037+0000","last_unstale":"2024-09-21T03:37:53.047154+0000","last_undegraded":"2024-09-21T03:37:53.047154+0000","last_fullsized":"2024-09-21T03:37:53.047154+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:32:23.134789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.051281+0000","last_change":"2024-09-21T03:37:12.852339+0000","last_active":"2024-09-21T03:37:53.051281+0000","last_peered":"2024-09-21T03:37:53.051281+0000","last_clean":"2024-09-21T03:37:53.051281+0000","last_became_active":"2024-09-21T03:37:12.851911+0000","last_became_peered":"2024-09-21T03:37:12.851911+0000","last_unstale":"2024-09-21T03:37:53.051281+0000","last_undegraded":"2024-09-21T03:37:53.051281+0000","last_fullsized":"2024-09-21T03:37:53.051281+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:03:30.739275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.044201+0000","last_change":"2024-09-21T03:37:12.861133+0000","last_active":"2024-09-21T03:37:53.044201+0000","last_peered":"2024-09-21T03:37:53.044201+0000","last_clean":"2024-09-21T03:37:53.044201+0000","last_became_active":"2024-09-21T03:37:12.860719+0000","last_became_peered":"2024-09-21T03:37:12.860719+0000","last_unstale":"2024-09-21T03:37:53.044201+0000","last_undegraded":"2024-09-21T03:37:53.044201+0000","last_fullsized":"2024-09-21T03:37:53.044201+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:21:25.241935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.049308+0000","last_change":"2024-09-21T03:37:12.865694+0000","last_active":"2024-09-21T03:37:53.049308+0000","last_peered":"2024-09-21T03:37:53.049308+0000","last_clean":"2024-09-21T03:37:53.049308+0000","last_became_active":"2024-09-21T03:37:12.864983+0000","last_became_peered":"2024-09-21T03:37:12.864983+0000","last_unstale":"2024-09-21T03:37:53.049308+0000","last_undegraded":"2024-09-21T03:37:53.049308+0000","last_fullsized":"2024-09-21T03:37:53.049308+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:52:28.583056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.049444+0000","last_change":"2024-09-21T03:37:12.860665+0000","last_active":"2024-09-21T03:37:53.049444+0000","last_peered":"2024-09-21T03:37:53.049444+0000","last_clean":"2024-09-21T03:37:53.049444+0000","last_became_active":"2024-09-21T03:37:12.860316+0000","last_became_peered":"2024-09-21T03:37:12.860316+0000","last_unstale":"2024-09-21T03:37:53.049444+0000","last_undegraded":"2024-09-21T03:37:53.049444+0000","last_fullsized":"2024-09-21T03:37:53.049444+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:13:07.316422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.048840+0000","last_change":"2024-09-21T03:37:12.869577+0000","last_active":"2024-09-21T03:37:53.048840+0000","last_peered":"2024-09-21T03:37:53.048840+0000","last_clean":"2024-09-21T03:37:53.048840+0000","last_became_active":"2024-09-21T03:37:12.869164+0000","last_became_peered":"2024-09-21T03:37:12.869164+0000","last_unstale":"2024-09-21T03:37:53.048840+0000","last_undegraded":"2024-09-21T03:37:53.048840+0000","last_fullsized":"2024-09-21T03:37:53.048840+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:40:34.167005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":64886,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":115,"num_read_kb":128,"num_write":157,"num_write_kb":111,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":694,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":401408,"data_stored":179356,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":136,"ondisk_log_size":136,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":285501,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":68,"seq":292057776151,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32068,"kb_used_data":884,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713084,"statfs":{"total":95995035648,"available":95962198016,"internally_reserved":0,"allocated":905216,"data_stored":461385,"data_compressed":23469,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":63,"seq":270582939676,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36360,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708792,"statfs":{"total":95995035648,"available":95957803008,"internally_reserved":0,"allocated":1040384,"data_stored":1109426,"data_compressed":59763,"data_compressed_allocated":77824,"data_compressed_original":655360,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":58,"seq":249108103201,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27920,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717232,"statfs":{"total":95995035648,"available":95966445568,"internally_reserved":0,"allocated":917504,"data_stored":465605,"data_compressed":23469,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":52,"seq":223338299429,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27908,"kb_used_data":884,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717244,"statfs":{"total":95995035648,"available":95966457856,"internally_reserved":0,"allocated":905216,"data_stored":461317,"data_compressed":23469,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":46,"seq":197568495659,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36284,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708868,"statfs":{"total":95995035648,"available":95957880832,"internally_reserved":0,"allocated":962560,"data_stored":1057338,"data_compressed":59763,"data_compressed_allocated":77824,"data_compressed_original":655360,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":41,"seq":176093659184,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32064,"kb_used_data":872,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713088,"statfs":{"total":95995035648,"available":95962202112,"internally_reserved":0,"allocated":892928,"data_stored":456685,"data_compressed":23469,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":35,"seq":150323855412,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32140,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713012,"statfs":{"total":95995035648,"available":95962124288,"internally_reserved":0,"allocated":970752,"data_stored":1052926,"data_compressed":59763,"data_compressed_allocated":77824,"data_compressed_original":655360,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":30,"seq":128849018937,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27912,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717240,"statfs":{"total":95995035648,"available":95966453760,"internally_reserved":0,"allocated":909312,"data_stored":461917,"data_compressed":23469,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":24,"seq":103079215165,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27908,"kb_used_data":884,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717244,"statfs":{"total":95995035648,"available":95966457856,"internally_reserved":0,"allocated":905216,"data_stored":456879,"data_compressed":23469,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":17,"seq":73014444100,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27972,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717180,"statfs":{"total":95995035648,"available":95966392320,"internally_reserved":0,"allocated":970752,"data_stored":510988,"data_compressed":23469,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574920,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36272,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708880,"statfs":{"total":95995035648,"available":95957893120,"internally_reserved":0,"allocated":950272,"data_stored":509829,"data_compressed":23469,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738446,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32084,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713068,"statfs":{"total":95995035648,"available":95962181632,"internally_reserved":0,"allocated":917504,"data_stored":458073,"data_compressed":23469,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34677,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34677,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34677,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36294,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36294,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36294,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36294,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36294,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":53234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":54393,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1226,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1867,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2183,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":626,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":62463,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T03:38:06.569 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-09-21T03:38:06.569 INFO:tasks.cephadm:Setting up client nodes... 2024-09-21T03:38:06.570 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7c374098-77c9-11ef-baf5-efdc52797490 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-09-21T03:38:06.816 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/7c374098-77c9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:38:07.215 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:38:07 smithi089 ceph-mon[30928]: pgmap v319: 129 pgs: 129 active+clean; 640 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 5.9 KiB/s wr, 9 op/s; 57 B/s, 0 objects/s recovering 2024-09-21T03:38:07.215 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:38:07 smithi089 ceph-mon[30928]: from='client.15027 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T03:38:07.314 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:07.312+0000 7f52c56b4640 1 -- 172.21.15.89:0/442596204 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f52c0109490 msgr2=0x7f52c010b9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:07.314 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:07.312+0000 7f52c56b4640 1 --2- 172.21.15.89:0/442596204 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f52c0109490 0x7f52c010b9a0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f52b8009ca0 tx=0x7f52b802a8c0 comp rx=0 tx=0).stop 2024-09-21T03:38:07.314 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:07.312+0000 7f52c56b4640 1 -- 172.21.15.89:0/442596204 shutdown_connections 2024-09-21T03:38:07.314 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:07.312+0000 7f52c56b4640 1 --2- 172.21.15.89:0/442596204 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f52c0109490 0x7f52c010b9a0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:07.314 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:07.312+0000 7f52c56b4640 1 --2- 172.21.15.89:0/442596204 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f52c0106950 0x7f52c0108d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:07.314 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:07.312+0000 7f52c56b4640 1 --2- 172.21.15.89:0/442596204 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f52c006c580 0x7f52c0106410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:07.314 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:07.312+0000 7f52c56b4640 1 -- 172.21.15.89:0/442596204 >> 172.21.15.89:0/442596204 conn(0x7f52c00fdfd0 msgr2=0x7f52c0100480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:38:07.314 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:07.312+0000 7f52c56b4640 1 -- 172.21.15.89:0/442596204 shutdown_connections 2024-09-21T03:38:07.315 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:07.312+0000 7f52c56b4640 1 -- 172.21.15.89:0/442596204 wait complete. 2024-09-21T03:38:07.316 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:07.312+0000 7f52c56b4640 1 Processor -- start 2024-09-21T03:38:07.316 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:07.313+0000 7f52c56b4640 1 -- start start 2024-09-21T03:38:07.316 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:07.313+0000 7f52c56b4640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f52c006c580 0x7f52c0103a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:07.316 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:07.313+0000 7f52c56b4640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f52c0106950 0x7f52c0102310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:07.316 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:07.313+0000 7f52c56b4640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f52c0109490 0x7f52c0102850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:07.316 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:07.313+0000 7f52c56b4640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f52c010f280 con 0x7f52c006c580 2024-09-21T03:38:07.316 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:07.313+0000 7f52c56b4640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f52c010f100 con 0x7f52c0109490 2024-09-21T03:38:07.316 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:07.313+0000 7f52c56b4640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f52c010f400 con 0x7f52c0106950 2024-09-21T03:38:07.316 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:07.313+0000 7f52beffd640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f52c006c580 0x7f52c0103a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:07.317 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:07.313+0000 7f52beffd640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f52c006c580 0x7f52c0103a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:50846/0 (socket says 172.21.15.89:50846) 2024-09-21T03:38:07.317 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:07.313+0000 7f52beffd640 1 -- 172.21.15.89:0/1652843944 learned_addr learned my addr 172.21.15.89:0/1652843944 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-21T03:38:07.317 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:07.313+0000 7f52beffd640 1 -- 172.21.15.89:0/1652843944 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f52c0106950 msgr2=0x7f52c0102310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:07.317 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:07.313+0000 7f52be7fc640 1 --2- 172.21.15.89:0/1652843944 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f52c0106950 0x7f52c0102310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:07.317 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:07.313+0000 7f52bf7fe640 1 --2- 172.21.15.89:0/1652843944 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f52c0109490 0x7f52c0102850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:07.317 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:07.313+0000 7f52beffd640 1 --2- 172.21.15.89:0/1652843944 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f52c0106950 0x7f52c0102310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:07.317 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:07.313+0000 7f52beffd640 1 -- 172.21.15.89:0/1652843944 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f52c0109490 msgr2=0x7f52c0102850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:07.317 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:07.313+0000 7f52beffd640 1 --2- 172.21.15.89:0/1652843944 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f52c0109490 0x7f52c0102850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:07.317 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:07.313+0000 7f52beffd640 1 -- 172.21.15.89:0/1652843944 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52c019dc80 con 0x7f52c006c580 2024-09-21T03:38:07.317 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:07.313+0000 7f52beffd640 1 --2- 172.21.15.89:0/1652843944 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f52c006c580 0x7f52c0103a40 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f52b000c890 tx=0x7f52b000cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:38:07.317 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:07.313+0000 7f52b7fff640 1 -- 172.21.15.89:0/1652843944 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52b0004220 con 0x7f52c006c580 2024-09-21T03:38:07.317 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:07.314+0000 7f52b7fff640 1 -- 172.21.15.89:0/1652843944 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f52b0007e40 con 0x7f52c006c580 2024-09-21T03:38:07.317 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:07.314+0000 7f52b7fff640 1 -- 172.21.15.89:0/1652843944 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52b0004a00 con 0x7f52c006c580 2024-09-21T03:38:07.317 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:07.314+0000 7f52c56b4640 1 -- 172.21.15.89:0/1652843944 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f52c0102e50 con 0x7f52c006c580 2024-09-21T03:38:07.317 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:07.314+0000 7f52c56b4640 1 -- 172.21.15.89:0/1652843944 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52c01a88a0 con 0x7f52c006c580 2024-09-21T03:38:07.318 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:07.315+0000 7f52c56b4640 1 -- 172.21.15.89:0/1652843944 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f52c00ffaf0 con 0x7f52c006c580 2024-09-21T03:38:07.318 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:07.316+0000 7f52b7fff640 1 -- 172.21.15.89:0/1652843944 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 18) ==== 154563+0+0 (secure 0 0 0) 0x7f52b00079b0 con 0x7f52c006c580 2024-09-21T03:38:07.318 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:07.317+0000 7f52b7fff640 1 --2- 172.21.15.89:0/1652843944 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f52ac0b65a0 0x7f52ac0b8ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:07.319 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:07.317+0000 7f52be7fc640 1 --2- 172.21.15.89:0/1652843944 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f52ac0b65a0 0x7f52ac0b8ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:07.319 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:07.317+0000 7f52b7fff640 1 -- 172.21.15.89:0/1652843944 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 9092+0+0 (secure 0 0 0) 0x7f52b00e79f0 con 0x7f52c006c580 2024-09-21T03:38:07.319 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:07.317+0000 7f52be7fc640 1 --2- 172.21.15.89:0/1652843944 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f52ac0b65a0 0x7f52ac0b8ab0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f52a8005fd0 tx=0x7f52a8005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:38:07.322 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:07.320+0000 7f52b7fff640 1 -- 172.21.15.89:0/1652843944 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f52b0014030 con 0x7f52c006c580 2024-09-21T03:38:07.460 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:38:07 smithi114 ceph-mon[34140]: pgmap v319: 129 pgs: 129 active+clean; 640 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 5.9 KiB/s wr, 9 op/s; 57 B/s, 0 objects/s recovering 2024-09-21T03:38:07.461 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:38:07 smithi114 ceph-mon[34140]: from='client.15027 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T03:38:07.500 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:38:07 smithi110 ceph-mon[34000]: pgmap v319: 129 pgs: 129 active+clean; 640 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 5.9 KiB/s wr, 9 op/s; 57 B/s, 0 objects/s recovering 2024-09-21T03:38:07.500 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:38:07 smithi110 ceph-mon[34000]: from='client.15027 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T03:38:07.567 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:07.565+0000 7f52c56b4640 1 -- 172.21.15.89:0/1652843944 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) -- 0x7f52c002e640 con 0x7f52c006c580 2024-09-21T03:38:07.572 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:07.570+0000 7f52b7fff640 1 -- 172.21.15.89:0/1652843944 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v32) ==== 170+0+59 (secure 0 0 0) 0x7f52b00addc0 con 0x7f52c006c580 2024-09-21T03:38:07.572 INFO:teuthology.orchestra.run.smithi089.stdout:[client.0] 2024-09-21T03:38:07.572 INFO:teuthology.orchestra.run.smithi089.stdout: key = AQCfP+5mNy7UIRAAjDA/htOgymFiVQFIp55Xsw== 2024-09-21T03:38:07.575 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:07.572+0000 7f52c56b4640 1 -- 172.21.15.89:0/1652843944 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f52ac0b65a0 msgr2=0x7f52ac0b8ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:07.575 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:07.572+0000 7f52c56b4640 1 --2- 172.21.15.89:0/1652843944 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f52ac0b65a0 0x7f52ac0b8ab0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f52a8005fd0 tx=0x7f52a8005850 comp rx=0 tx=0).stop 2024-09-21T03:38:07.575 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:07.572+0000 7f52c56b4640 1 -- 172.21.15.89:0/1652843944 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f52c006c580 msgr2=0x7f52c0103a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:07.575 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:07.572+0000 7f52c56b4640 1 --2- 172.21.15.89:0/1652843944 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f52c006c580 0x7f52c0103a40 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f52b000c890 tx=0x7f52b000cd50 comp rx=0 tx=0).stop 2024-09-21T03:38:07.575 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:07.572+0000 7f52c56b4640 1 -- 172.21.15.89:0/1652843944 shutdown_connections 2024-09-21T03:38:07.575 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:07.572+0000 7f52c56b4640 1 --2- 172.21.15.89:0/1652843944 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f52ac0b65a0 0x7f52ac0b8ab0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:07.575 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:07.572+0000 7f52c56b4640 1 --2- 172.21.15.89:0/1652843944 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f52c0109490 0x7f52c0102850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:07.575 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:07.572+0000 7f52c56b4640 1 --2- 172.21.15.89:0/1652843944 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f52c0106950 0x7f52c0102310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:07.575 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:07.572+0000 7f52c56b4640 1 --2- 172.21.15.89:0/1652843944 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f52c006c580 0x7f52c0103a40 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:07.575 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:07.572+0000 7f52c56b4640 1 -- 172.21.15.89:0/1652843944 >> 172.21.15.89:0/1652843944 conn(0x7f52c00fdfd0 msgr2=0x7f52c0104e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:38:07.575 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:07.573+0000 7f52c56b4640 1 -- 172.21.15.89:0/1652843944 shutdown_connections 2024-09-21T03:38:07.575 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:07.573+0000 7f52c56b4640 1 -- 172.21.15.89:0/1652843944 wait complete. 2024-09-21T03:38:08.170 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:38:08 smithi089 ceph-mon[30928]: from='client.? 172.21.15.89:0/1652843944' entity='client.admin' cmd={"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} : dispatch 2024-09-21T03:38:08.171 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:38:08 smithi089 ceph-mon[30928]: from='client.? 172.21.15.89:0/1652843944' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-21T03:38:08.197 DEBUG:teuthology.orchestra.run.smithi089:> set -ex 2024-09-21T03:38:08.197 DEBUG:teuthology.orchestra.run.smithi089:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-09-21T03:38:08.197 DEBUG:teuthology.orchestra.run.smithi089:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-09-21T03:38:08.234 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 7c374098-77c9-11ef-baf5-efdc52797490 -- ceph config log 1 --format=json 2024-09-21T03:38:08.461 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:38:08 smithi114 ceph-mon[34140]: from='client.? 172.21.15.89:0/1652843944' entity='client.admin' cmd={"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} : dispatch 2024-09-21T03:38:08.461 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:38:08 smithi114 ceph-mon[34140]: from='client.? 172.21.15.89:0/1652843944' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-21T03:38:08.500 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:38:08 smithi110 ceph-mon[34000]: from='client.? 172.21.15.89:0/1652843944' entity='client.admin' cmd={"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} : dispatch 2024-09-21T03:38:08.500 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:38:08 smithi110 ceph-mon[34000]: from='client.? 172.21.15.89:0/1652843944' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-21T03:38:08.517 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/7c374098-77c9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:38:08.966 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:08.956+0000 7fe36a347640 1 -- 172.21.15.89:0/2445059303 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe364109df0 msgr2=0x7fe364110620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:08.967 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:08.956+0000 7fe36a347640 1 --2- 172.21.15.89:0/2445059303 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe364109df0 0x7fe364110620 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fe35c009ca0 tx=0x7fe35c02a8c0 comp rx=0 tx=0).stop 2024-09-21T03:38:08.967 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:08.956+0000 7fe36a347640 1 -- 172.21.15.89:0/2445059303 shutdown_connections 2024-09-21T03:38:08.967 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:08.956+0000 7fe36a347640 1 --2- 172.21.15.89:0/2445059303 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe364109df0 0x7fe364110620 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:08.967 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:08.956+0000 7fe36a347640 1 --2- 172.21.15.89:0/2445059303 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe3641094a0 0x7fe3641098b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:08.967 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:08.956+0000 7fe36a347640 1 --2- 172.21.15.89:0/2445059303 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe364103b60 0x7fe364103f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:08.967 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:08.956+0000 7fe36a347640 1 -- 172.21.15.89:0/2445059303 >> 172.21.15.89:0/2445059303 conn(0x7fe36407b850 msgr2=0x7fe36407bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:38:08.967 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:08.957+0000 7fe36a347640 1 -- 172.21.15.89:0/2445059303 shutdown_connections 2024-09-21T03:38:08.967 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:08.957+0000 7fe36a347640 1 -- 172.21.15.89:0/2445059303 wait complete. 2024-09-21T03:38:08.967 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:08.957+0000 7fe36a347640 1 Processor -- start 2024-09-21T03:38:08.967 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:08.958+0000 7fe36a347640 1 -- start start 2024-09-21T03:38:08.967 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:08.958+0000 7fe36a347640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe364103b60 0x7fe36419c5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:08.967 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:08.958+0000 7fe36a347640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe3641094a0 0x7fe36419cb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:08.968 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:08.958+0000 7fe36a347640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe364109df0 0x7fe3641a1e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:08.968 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:08.958+0000 7fe36a347640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fe3640760f0 con 0x7fe364109df0 2024-09-21T03:38:08.968 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:08.958+0000 7fe36a347640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fe364075f70 con 0x7fe364103b60 2024-09-21T03:38:08.968 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:08.958+0000 7fe36a347640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fe364076270 con 0x7fe3641094a0 2024-09-21T03:38:08.968 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:08.958+0000 7fe3688e4640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe364109df0 0x7fe3641a1e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:08.968 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:08.958+0000 7fe3637fe640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe3641094a0 0x7fe36419cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:08.968 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:08.958+0000 7fe363fff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe364103b60 0x7fe36419c5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:08.968 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:08.958+0000 7fe3637fe640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe3641094a0 0x7fe36419cb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.89:38262/0 (socket says 172.21.15.89:38262) 2024-09-21T03:38:08.968 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:08.958+0000 7fe3688e4640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe364109df0 0x7fe3641a1e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:38850/0 (socket says 172.21.15.89:38850) 2024-09-21T03:38:08.968 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:08.958+0000 7fe3637fe640 1 -- 172.21.15.89:0/2876754531 learned_addr learned my addr 172.21.15.89:0/2876754531 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-21T03:38:08.968 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:08.958+0000 7fe363fff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe364103b60 0x7fe36419c5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.89:40762/0 (socket says 172.21.15.89:40762) 2024-09-21T03:38:08.968 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:08.959+0000 7fe363fff640 1 -- 172.21.15.89:0/2876754531 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe3641094a0 msgr2=0x7fe36419cb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:08.968 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:08.959+0000 7fe363fff640 1 --2- 172.21.15.89:0/2876754531 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe3641094a0 0x7fe36419cb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:08.968 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:08.959+0000 7fe363fff640 1 -- 172.21.15.89:0/2876754531 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe364109df0 msgr2=0x7fe3641a1e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:08.969 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:08.959+0000 7fe363fff640 1 --2- 172.21.15.89:0/2876754531 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe364109df0 0x7fe3641a1e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:08.969 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:08.959+0000 7fe363fff640 1 -- 172.21.15.89:0/2876754531 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3641a2530 con 0x7fe364103b60 2024-09-21T03:38:08.969 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:08.959+0000 7fe3637fe640 1 --2- 172.21.15.89:0/2876754531 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe3641094a0 0x7fe36419cb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:38:08.969 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:08.959+0000 7fe363fff640 1 --2- 172.21.15.89:0/2876754531 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe364103b60 0x7fe36419c5d0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fe35400e9f0 tx=0x7fe35400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:38:08.969 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:08.959+0000 7fe3617fa640 1 -- 172.21.15.89:0/2876754531 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe35400ccc0 con 0x7fe364103b60 2024-09-21T03:38:08.969 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:08.959+0000 7fe36a347640 1 -- 172.21.15.89:0/2876754531 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe3641a2820 con 0x7fe364103b60 2024-09-21T03:38:08.969 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:08.959+0000 7fe3617fa640 1 -- 172.21.15.89:0/2876754531 <== mon.2 v2:172.21.15.110:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe3540108c0 con 0x7fe364103b60 2024-09-21T03:38:08.969 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:08.959+0000 7fe3617fa640 1 -- 172.21.15.89:0/2876754531 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3540187c0 con 0x7fe364103b60 2024-09-21T03:38:08.969 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:08.959+0000 7fe36a347640 1 -- 172.21.15.89:0/2876754531 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe3641a8f80 con 0x7fe364103b60 2024-09-21T03:38:08.969 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:08.960+0000 7fe36a347640 1 -- 172.21.15.89:0/2876754531 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe364103f70 con 0x7fe364103b60 2024-09-21T03:38:08.969 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:08.962+0000 7fe3617fa640 1 -- 172.21.15.89:0/2876754531 <== mon.2 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 18) ==== 154563+0+0 (secure 0 0 0) 0x7fe354010430 con 0x7fe364103b60 2024-09-21T03:38:08.969 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:08.962+0000 7fe3617fa640 1 --2- 172.21.15.89:0/2876754531 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7fe34c0b65a0 0x7fe34c0b8ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:08.969 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:08.962+0000 7fe3637fe640 1 --2- 172.21.15.89:0/2876754531 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7fe34c0b65a0 0x7fe34c0b8ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:08.969 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:08.963+0000 7fe3617fa640 1 -- 172.21.15.89:0/2876754531 <== mon.2 v2:172.21.15.110:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 9092+0+0 (secure 0 0 0) 0x7fe3540e7ae0 con 0x7fe364103b60 2024-09-21T03:38:08.970 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:08.963+0000 7fe3637fe640 1 --2- 172.21.15.89:0/2876754531 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7fe34c0b65a0 0x7fe34c0b8ab0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fe364105820 tx=0x7fe358005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:38:08.970 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:08.967+0000 7fe3617fa640 1 -- 172.21.15.89:0/2876754531 <== mon.2 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fe3540b2290 con 0x7fe364103b60 2024-09-21T03:38:09.169 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:09.167+0000 7fe36a347640 1 -- 172.21.15.89:0/2876754531 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "config log", "num": 1, "format": "json"} v 0) -- 0x7fe36419d6d0 con 0x7fe364103b60 2024-09-21T03:38:09.170 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:09.168+0000 7fe3617fa640 1 -- 172.21.15.89:0/2876754531 <== mon.2 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "config log", "num": 1, "format": "json"}]=0 v53) ==== 86+0+133 (secure 0 0 0) 0x7fe3540b71d0 con 0x7fe364103b60 2024-09-21T03:38:09.170 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-09-21T03:38:09.173 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:09.170+0000 7fe36a347640 1 -- 172.21.15.89:0/2876754531 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7fe34c0b65a0 msgr2=0x7fe34c0b8ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:09.173 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:09.170+0000 7fe36a347640 1 --2- 172.21.15.89:0/2876754531 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7fe34c0b65a0 0x7fe34c0b8ab0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fe364105820 tx=0x7fe358005f90 comp rx=0 tx=0).stop 2024-09-21T03:38:09.173 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:09.170+0000 7fe36a347640 1 -- 172.21.15.89:0/2876754531 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe364103b60 msgr2=0x7fe36419c5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:09.173 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:09.170+0000 7fe36a347640 1 --2- 172.21.15.89:0/2876754531 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe364103b60 0x7fe36419c5d0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fe35400e9f0 tx=0x7fe35400eeb0 comp rx=0 tx=0).stop 2024-09-21T03:38:09.173 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:09.170+0000 7fe36a347640 1 -- 172.21.15.89:0/2876754531 shutdown_connections 2024-09-21T03:38:09.173 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:09.170+0000 7fe36a347640 1 --2- 172.21.15.89:0/2876754531 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7fe34c0b65a0 0x7fe34c0b8ab0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:09.174 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:09.170+0000 7fe36a347640 1 --2- 172.21.15.89:0/2876754531 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe364109df0 0x7fe3641a1e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:09.174 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:09.170+0000 7fe36a347640 1 --2- 172.21.15.89:0/2876754531 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe3641094a0 0x7fe36419cb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:09.174 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:09.170+0000 7fe36a347640 1 --2- 172.21.15.89:0/2876754531 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe364103b60 0x7fe36419c5d0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:09.174 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:09.170+0000 7fe36a347640 1 -- 172.21.15.89:0/2876754531 >> 172.21.15.89:0/2876754531 conn(0x7fe36407b850 msgr2=0x7fe364102640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:38:09.174 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:09.170+0000 7fe36a347640 1 -- 172.21.15.89:0/2876754531 shutdown_connections 2024-09-21T03:38:09.174 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:09.171+0000 7fe36a347640 1 -- 172.21.15.89:0/2876754531 wait complete. 2024-09-21T03:38:09.811 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:38:09 smithi089 ceph-mon[30928]: pgmap v320: 129 pgs: 129 active+clean; 640 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 5.0 KiB/s wr, 7 op/s; 48 B/s, 0 objects/s recovering 2024-09-21T03:38:09.811 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:38:09 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:38:09.811 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:38:09 smithi089 ceph-mon[30928]: from='client.? 172.21.15.89:0/2876754531' entity='client.admin' cmd={"prefix": "config log", "num": 1, "format": "json"} : dispatch 2024-09-21T03:38:09.961 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:38:09 smithi114 ceph-mon[34140]: pgmap v320: 129 pgs: 129 active+clean; 640 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 5.0 KiB/s wr, 7 op/s; 48 B/s, 0 objects/s recovering 2024-09-21T03:38:09.961 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:38:09 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:38:09.961 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:38:09 smithi114 ceph-mon[34140]: from='client.? 172.21.15.89:0/2876754531' entity='client.admin' cmd={"prefix": "config log", "num": 1, "format": "json"} : dispatch 2024-09-21T03:38:10.000 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:38:09 smithi110 ceph-mon[34000]: pgmap v320: 129 pgs: 129 active+clean; 640 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 5.0 KiB/s wr, 7 op/s; 48 B/s, 0 objects/s recovering 2024-09-21T03:38:10.000 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:38:09 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:38:10.000 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:38:09 smithi110 ceph-mon[34000]: from='client.? 172.21.15.89:0/2876754531' entity='client.admin' cmd={"prefix": "config log", "num": 1, "format": "json"} : dispatch 2024-09-21T03:38:10.463 INFO:teuthology.orchestra.run.smithi089.stdout:[{"version":53,"timestamp":"2024-09-21T03:37:03.535696+0000","name":"","changes":[{"name":"mds.all/mds_join_fs","new_value":"all"}]}] 2024-09-21T03:38:10.463 INFO:tasks.ceph_manager:config epoch is 53 2024-09-21T03:38:10.463 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-09-21T03:38:10.464 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-09-21T03:38:10.464 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 7c374098-77c9-11ef-baf5-efdc52797490 -- ceph mgr dump --format=json 2024-09-21T03:38:10.743 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/7c374098-77c9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:38:11.139 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:38:11 smithi089 ceph-mon[30928]: pgmap v321: 129 pgs: 129 active+clean; 644 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 5.4 KiB/s wr, 9 op/s; 48 B/s, 0 objects/s recovering 2024-09-21T03:38:11.139 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:38:11 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:38:11.139 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:38:11 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:38:11.139 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:38:11 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:38:11.139 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:38:11 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:38:11.232 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:11.230+0000 7f8008bd5640 1 -- 172.21.15.89:0/3231154707 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8004109df0 msgr2=0x7f8004110620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:11.232 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:11.230+0000 7f8008bd5640 1 --2- 172.21.15.89:0/3231154707 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8004109df0 0x7f8004110620 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f7ffc009ca0 tx=0x7f7ffc02a8c0 comp rx=0 tx=0).stop 2024-09-21T03:38:11.233 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:11.230+0000 7f8008bd5640 1 -- 172.21.15.89:0/3231154707 shutdown_connections 2024-09-21T03:38:11.233 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:11.230+0000 7f8008bd5640 1 --2- 172.21.15.89:0/3231154707 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8004109df0 0x7f8004110620 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:11.233 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:11.230+0000 7f8008bd5640 1 --2- 172.21.15.89:0/3231154707 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f80041094a0 0x7f80041098b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:11.233 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:11.230+0000 7f8008bd5640 1 --2- 172.21.15.89:0/3231154707 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8004103b60 0x7f8004103f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:11.233 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:11.230+0000 7f8008bd5640 1 -- 172.21.15.89:0/3231154707 >> 172.21.15.89:0/3231154707 conn(0x7f800407b850 msgr2=0x7f800407bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:38:11.233 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:11.230+0000 7f8008bd5640 1 -- 172.21.15.89:0/3231154707 shutdown_connections 2024-09-21T03:38:11.233 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:11.230+0000 7f8008bd5640 1 -- 172.21.15.89:0/3231154707 wait complete. 2024-09-21T03:38:11.235 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:11.231+0000 7f8008bd5640 1 Processor -- start 2024-09-21T03:38:11.235 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:11.231+0000 7f8008bd5640 1 -- start start 2024-09-21T03:38:11.235 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:11.231+0000 7f8008bd5640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8004103b60 0x7f800419a400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:11.235 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:11.231+0000 7f8008bd5640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f80041094a0 0x7f800419a9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:11.235 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:11.231+0000 7f8008bd5640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8004109df0 0x7f800419fc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:11.235 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:11.231+0000 7f8008bd5640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f80041139b0 con 0x7f80041094a0 2024-09-21T03:38:11.235 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:11.231+0000 7f8008bd5640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f8004113830 con 0x7f8004109df0 2024-09-21T03:38:11.235 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:11.231+0000 7f8008bd5640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f8004113b30 con 0x7f8004103b60 2024-09-21T03:38:11.235 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:11.232+0000 7f8001d9b640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f80041094a0 0x7f800419a9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:11.235 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:11.232+0000 7f8001d9b640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f80041094a0 0x7f800419a9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:38858/0 (socket says 172.21.15.89:38858) 2024-09-21T03:38:11.235 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:11.232+0000 7f8001d9b640 1 -- 172.21.15.89:0/2529557969 learned_addr learned my addr 172.21.15.89:0/2529557969 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-21T03:38:11.235 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:11.232+0000 7f800259c640 1 --2- 172.21.15.89:0/2529557969 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8004103b60 0x7f800419a400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:11.236 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:11.232+0000 7f8001d9b640 1 -- 172.21.15.89:0/2529557969 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8004103b60 msgr2=0x7f800419a400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:11.236 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:11.232+0000 7f8001d9b640 1 --2- 172.21.15.89:0/2529557969 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8004103b60 0x7f800419a400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:11.236 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:11.232+0000 7f8001d9b640 1 -- 172.21.15.89:0/2529557969 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8004109df0 msgr2=0x7f800419fc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:11.236 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:11.232+0000 7f8001d9b640 1 --2- 172.21.15.89:0/2529557969 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8004109df0 0x7f800419fc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:11.236 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:11.232+0000 7f8001d9b640 1 -- 172.21.15.89:0/2529557969 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80041a0370 con 0x7f80041094a0 2024-09-21T03:38:11.236 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:11.232+0000 7f8001d9b640 1 --2- 172.21.15.89:0/2529557969 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f80041094a0 0x7f800419a9a0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f7ff800e9c0 tx=0x7f7ff800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:38:11.236 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:11.233+0000 7f7ff37fe640 1 -- 172.21.15.89:0/2529557969 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ff800ccc0 con 0x7f80041094a0 2024-09-21T03:38:11.236 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:11.233+0000 7f7ff37fe640 1 -- 172.21.15.89:0/2529557969 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7ff80054b0 con 0x7f80041094a0 2024-09-21T03:38:11.236 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:11.233+0000 7f8008bd5640 1 -- 172.21.15.89:0/2529557969 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f80041a0660 con 0x7f80041094a0 2024-09-21T03:38:11.236 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:11.233+0000 7f7ff37fe640 1 -- 172.21.15.89:0/2529557969 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ff8010430 con 0x7f80041094a0 2024-09-21T03:38:11.236 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:11.233+0000 7f8008bd5640 1 -- 172.21.15.89:0/2529557969 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80041a6dc0 con 0x7f80041094a0 2024-09-21T03:38:11.237 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:11.234+0000 7f8008bd5640 1 -- 172.21.15.89:0/2529557969 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8004103f70 con 0x7f80041094a0 2024-09-21T03:38:11.237 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:11.236+0000 7f7ff37fe640 1 -- 172.21.15.89:0/2529557969 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 18) ==== 154563+0+0 (secure 0 0 0) 0x7f7ff8004f00 con 0x7f80041094a0 2024-09-21T03:38:11.239 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:11.237+0000 7f7ff37fe640 1 --2- 172.21.15.89:0/2529557969 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f7fec0b6620 0x7f7fec0b8b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:11.239 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:11.237+0000 7f800259c640 1 --2- 172.21.15.89:0/2529557969 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f7fec0b6620 0x7f7fec0b8b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:11.239 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:11.237+0000 7f7ff37fe640 1 -- 172.21.15.89:0/2529557969 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 9092+0+0 (secure 0 0 0) 0x7f7ff80e7e20 con 0x7f80041094a0 2024-09-21T03:38:11.239 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:11.237+0000 7f800259c640 1 --2- 172.21.15.89:0/2529557969 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f7fec0b6620 0x7f7fec0b8b30 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f7ff4009b60 tx=0x7f7ff4005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:38:11.243 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:11.241+0000 7f7ff37fe640 1 -- 172.21.15.89:0/2529557969 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f7ff8056710 con 0x7f80041094a0 2024-09-21T03:38:11.461 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:38:11 smithi114 ceph-mon[34140]: pgmap v321: 129 pgs: 129 active+clean; 644 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 5.4 KiB/s wr, 9 op/s; 48 B/s, 0 objects/s recovering 2024-09-21T03:38:11.461 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:38:11 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:38:11.461 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:38:11 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:38:11.461 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:38:11 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:38:11.461 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:38:11 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:38:11.475 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:11.473+0000 7f8008bd5640 1 -- 172.21.15.89:0/2529557969 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7f800410b650 con 0x7f80041094a0 2024-09-21T03:38:11.481 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:11.479+0000 7f7ff37fe640 1 -- 172.21.15.89:0/2529557969 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v18) ==== 74+0+296437 (secure 0 0 0) 0x7f7ff80ae190 con 0x7f80041094a0 2024-09-21T03:38:11.484 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-09-21T03:38:11.486 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:11.482+0000 7f8008bd5640 1 -- 172.21.15.89:0/2529557969 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f7fec0b6620 msgr2=0x7f7fec0b8b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:11.486 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:11.482+0000 7f8008bd5640 1 --2- 172.21.15.89:0/2529557969 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f7fec0b6620 0x7f7fec0b8b30 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f7ff4009b60 tx=0x7f7ff4005f90 comp rx=0 tx=0).stop 2024-09-21T03:38:11.487 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:11.482+0000 7f8008bd5640 1 -- 172.21.15.89:0/2529557969 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f80041094a0 msgr2=0x7f800419a9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:11.487 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:11.482+0000 7f8008bd5640 1 --2- 172.21.15.89:0/2529557969 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f80041094a0 0x7f800419a9a0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f7ff800e9c0 tx=0x7f7ff800ee80 comp rx=0 tx=0).stop 2024-09-21T03:38:11.487 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:11.482+0000 7f8008bd5640 1 -- 172.21.15.89:0/2529557969 shutdown_connections 2024-09-21T03:38:11.487 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:11.482+0000 7f8008bd5640 1 --2- 172.21.15.89:0/2529557969 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f7fec0b6620 0x7f7fec0b8b30 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:11.487 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:11.482+0000 7f8008bd5640 1 --2- 172.21.15.89:0/2529557969 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8004109df0 0x7f800419fc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:11.487 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:11.482+0000 7f8008bd5640 1 --2- 172.21.15.89:0/2529557969 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f80041094a0 0x7f800419a9a0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:11.487 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:11.482+0000 7f8008bd5640 1 --2- 172.21.15.89:0/2529557969 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8004103b60 0x7f800419a400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:11.487 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:11.482+0000 7f8008bd5640 1 -- 172.21.15.89:0/2529557969 >> 172.21.15.89:0/2529557969 conn(0x7f800407b850 msgr2=0x7f80041075e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:38:11.487 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:11.482+0000 7f8008bd5640 1 -- 172.21.15.89:0/2529557969 shutdown_connections 2024-09-21T03:38:11.487 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:11.482+0000 7f8008bd5640 1 -- 172.21.15.89:0/2529557969 wait complete. 2024-09-21T03:38:11.500 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:38:11 smithi110 ceph-mon[34000]: pgmap v321: 129 pgs: 129 active+clean; 644 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 5.4 KiB/s wr, 9 op/s; 48 B/s, 0 objects/s recovering 2024-09-21T03:38:11.500 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:38:11 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:38:11.500 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:38:11 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:38:11.500 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:38:11 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:38:11.500 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:38:11 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:38:12.146 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:38:12 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:38:12.146 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:38:12 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:38:12.146 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:38:12 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:38:12.146 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:38:12 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:38:12.146 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:38:12 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:38:12.146 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:38:12 smithi089 ceph-mon[30928]: from='client.? 172.21.15.89:0/2529557969' entity='client.admin' cmd={"prefix": "mgr dump", "format": "json"} : dispatch 2024-09-21T03:38:12.174 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":18,"flags":0,"active_gid":14152,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6800","nonce":1591532358},{"type":"v1","addr":"172.21.15.89:6801","nonce":1591532358}]},"active_addr":"172.21.15.89:6801/1591532358","active_change":"2024-09-21T03:29:51.519235+0000","active_mgr_features":4540719139924082687,"available":true,"standbys":[{"gid":24112,"name":"z","mgr_features":4540719139924082687,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to, use commas to separate multiple","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.27.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:10.4.8","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_nginx":{"name":"container_image_nginx","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nginx:sclorg-nginx-126","min":"","max":"","enum_allowed":[],"desc":"Nginx container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.7.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.17","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_oauth2_proxy":{"name":"container_image_oauth2_proxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/oauth2-proxy/oauth2-proxy:v7.6.0","min":"","max":"","enum_allowed":[],"desc":"oauth2-proxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.51.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba_metrics":{"name":"container_image_samba_metrics","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-metrics:latest","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB metrics exporter container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"ssh_keepalive_count_max":{"name":"ssh_keepalive_count_max","type":"int","level":"advanced","flags":0,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"How many times ssh connections can fail liveness checks before the host is marked offline","long_desc":"","tags":[],"see_also":[]},"ssh_keepalive_interval":{"name":"ssh_keepalive_interval","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"How often ssh connections are checked for liveness","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sso_oauth2":{"name":"sso_oauth2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_requests":{"name":"max_requests","type":"int","level":"advanced","flags":0,"default_value":"500","min":"","max":"","enum_allowed":[],"desc":"Maximum number of requests to keep in memory. When new request comes in, the oldest request will be removed if the number of requests exceeds the max request number.if un-finished request is removed, error message will be logged in the ceph-mgr log.","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"secondary_zone_period_retry_limit":{"name":"secondary_zone_period_retry_limit","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"RGW module period update retry limit for secondary site","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"smb","can_run":true,"error_string":"","module_options":{"internal_store_backend":{"name":"internal_store_backend","type":"str","level":"dev","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"set internal store backend. for develoment and testing only","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_orchestration":{"name":"update_orchestration","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically update orchestration when smb resources are changed","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]},{"gid":24134,"name":"y","mgr_features":4540719139924082687,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to, use commas to separate multiple","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.27.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:10.4.8","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_nginx":{"name":"container_image_nginx","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nginx:sclorg-nginx-126","min":"","max":"","enum_allowed":[],"desc":"Nginx container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.7.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.17","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_oauth2_proxy":{"name":"container_image_oauth2_proxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/oauth2-proxy/oauth2-proxy:v7.6.0","min":"","max":"","enum_allowed":[],"desc":"oauth2-proxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.51.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba_metrics":{"name":"container_image_samba_metrics","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-metrics:latest","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB metrics exporter container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"ssh_keepalive_count_max":{"name":"ssh_keepalive_count_max","type":"int","level":"advanced","flags":0,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"How many times ssh connections can fail liveness checks before the host is marked offline","long_desc":"","tags":[],"see_also":[]},"ssh_keepalive_interval":{"name":"ssh_keepalive_interval","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"How often ssh connections are checked for liveness","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sso_oauth2":{"name":"sso_oauth2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_requests":{"name":"max_requests","type":"int","level":"advanced","flags":0,"default_value":"500","min":"","max":"","enum_allowed":[],"desc":"Maximum number of requests to keep in memory. When new request comes in, the oldest request will be removed if the number of requests exceeds the max request number.if un-finished request is removed, error message will be logged in the ceph-mgr log.","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"secondary_zone_period_retry_limit":{"name":"secondary_zone_period_retry_limit","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"RGW module period update retry limit for secondary site","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"smb","can_run":true,"error_string":"","module_options":{"internal_store_backend":{"name":"internal_store_backend","type":"str","level":"dev","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"set internal store backend. for develoment and testing only","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_orchestration":{"name":"update_orchestration","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically update orchestration when smb resources are changed","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to, use commas to separate multiple","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.27.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:10.4.8","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_nginx":{"name":"container_image_nginx","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nginx:sclorg-nginx-126","min":"","max":"","enum_allowed":[],"desc":"Nginx container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.7.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.17","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_oauth2_proxy":{"name":"container_image_oauth2_proxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/oauth2-proxy/oauth2-proxy:v7.6.0","min":"","max":"","enum_allowed":[],"desc":"oauth2-proxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.51.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba_metrics":{"name":"container_image_samba_metrics","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-metrics:latest","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB metrics exporter container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"ssh_keepalive_count_max":{"name":"ssh_keepalive_count_max","type":"int","level":"advanced","flags":0,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"How many times ssh connections can fail liveness checks before the host is marked offline","long_desc":"","tags":[],"see_also":[]},"ssh_keepalive_interval":{"name":"ssh_keepalive_interval","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"How often ssh connections are checked for liveness","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sso_oauth2":{"name":"sso_oauth2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_requests":{"name":"max_requests","type":"int","level":"advanced","flags":0,"default_value":"500","min":"","max":"","enum_allowed":[],"desc":"Maximum number of requests to keep in memory. When new request comes in, the oldest request will be removed if the number of requests exceeds the max request number.if un-finished request is removed, error message will be logged in the ceph-mgr log.","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"secondary_zone_period_retry_limit":{"name":"secondary_zone_period_retry_limit","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"RGW module period update retry limit for secondary site","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"smb","can_run":true,"error_string":"","module_options":{"internal_store_backend":{"name":"internal_store_backend","type":"str","level":"dev","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"set internal store backend. for develoment and testing only","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_orchestration":{"name":"update_orchestration","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically update orchestration when smb resources are changed","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.89:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":3,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.89:0","nonce":3874566624}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.89:0","nonce":1821892963}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.89:0","nonce":2892236499}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.89:0","nonce":1955004802}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.89:0","nonce":837308050}]}]} 2024-09-21T03:38:12.181 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-09-21T03:38:12.181 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-09-21T03:38:12.181 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 7c374098-77c9-11ef-baf5-efdc52797490 -- ceph osd dump --format=json 2024-09-21T03:38:12.428 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/7c374098-77c9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:38:12.460 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:38:12 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:38:12.461 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:38:12 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:38:12.461 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:38:12 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:38:12.461 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:38:12 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:38:12.461 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:38:12 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:38:12.461 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:38:12 smithi114 ceph-mon[34140]: from='client.? 172.21.15.89:0/2529557969' entity='client.admin' cmd={"prefix": "mgr dump", "format": "json"} : dispatch 2024-09-21T03:38:12.500 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:38:12 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:38:12.500 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:38:12 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:38:12.500 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:38:12 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:38:12.500 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:38:12 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:38:12.500 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:38:12 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:38:12.500 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:38:12 smithi110 ceph-mon[34000]: from='client.? 172.21.15.89:0/2529557969' entity='client.admin' cmd={"prefix": "mgr dump", "format": "json"} : dispatch 2024-09-21T03:38:12.865 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:12.862+0000 7fe72420d640 1 -- 172.21.15.89:0/2902441492 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe71c108420 msgr2=0x7fe71c108830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:12.865 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:12.862+0000 7fe72420d640 1 --2- 172.21.15.89:0/2902441492 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe71c108420 0x7fe71c108830 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fe70c009900 tx=0x7fe70c02a410 comp rx=0 tx=0).stop 2024-09-21T03:38:12.865 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:12.862+0000 7fe72420d640 1 -- 172.21.15.89:0/2902441492 shutdown_connections 2024-09-21T03:38:12.865 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:12.862+0000 7fe72420d640 1 --2- 172.21.15.89:0/2902441492 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe71c108d70 0x7fe71c10f5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:12.866 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:12.862+0000 7fe72420d640 1 --2- 172.21.15.89:0/2902441492 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe71c108420 0x7fe71c108830 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:12.866 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:12.862+0000 7fe72420d640 1 --2- 172.21.15.89:0/2902441492 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe71c102ae0 0x7fe71c102ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:12.866 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:12.862+0000 7fe72420d640 1 -- 172.21.15.89:0/2902441492 >> 172.21.15.89:0/2902441492 conn(0x7fe71c0796b0 msgr2=0x7fe71c079ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:38:12.866 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:12.863+0000 7fe72420d640 1 -- 172.21.15.89:0/2902441492 shutdown_connections 2024-09-21T03:38:12.866 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:12.863+0000 7fe72420d640 1 -- 172.21.15.89:0/2902441492 wait complete. 2024-09-21T03:38:12.866 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:12.863+0000 7fe72420d640 1 Processor -- start 2024-09-21T03:38:12.866 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:12.863+0000 7fe72420d640 1 -- start start 2024-09-21T03:38:12.867 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:12.863+0000 7fe72420d640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe71c102ae0 0x7fe71c19a520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:12.867 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:12.863+0000 7fe72420d640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe71c108420 0x7fe71c19aac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:12.867 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:12.864+0000 7fe72420d640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe71c108d70 0x7fe71c19fd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:12.868 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:12.864+0000 7fe72420d640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fe71c112920 con 0x7fe71c102ae0 2024-09-21T03:38:12.868 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:12.864+0000 7fe72420d640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fe71c1127a0 con 0x7fe71c108420 2024-09-21T03:38:12.868 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:12.864+0000 7fe72420d640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fe71c112aa0 con 0x7fe71c108d70 2024-09-21T03:38:12.868 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:12.864+0000 7fe721fa9640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe71c102ae0 0x7fe71c19a520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:12.868 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:12.864+0000 7fe721fa9640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe71c102ae0 0x7fe71c19a520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:38872/0 (socket says 172.21.15.89:38872) 2024-09-21T03:38:12.868 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:12.864+0000 7fe721fa9640 1 -- 172.21.15.89:0/1390772879 learned_addr learned my addr 172.21.15.89:0/1390772879 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-21T03:38:12.868 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:12.864+0000 7fe7227aa640 1 --2- 172.21.15.89:0/1390772879 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe71c108d70 0x7fe71c19fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:12.868 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:12.864+0000 7fe7217a8640 1 --2- 172.21.15.89:0/1390772879 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe71c108420 0x7fe71c19aac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:12.868 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:12.864+0000 7fe721fa9640 1 -- 172.21.15.89:0/1390772879 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe71c108d70 msgr2=0x7fe71c19fd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:12.868 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:12.864+0000 7fe721fa9640 1 --2- 172.21.15.89:0/1390772879 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe71c108d70 0x7fe71c19fd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:12.868 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:12.864+0000 7fe721fa9640 1 -- 172.21.15.89:0/1390772879 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe71c108420 msgr2=0x7fe71c19aac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:12.868 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:12.864+0000 7fe721fa9640 1 --2- 172.21.15.89:0/1390772879 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe71c108420 0x7fe71c19aac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:12.868 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:12.864+0000 7fe721fa9640 1 -- 172.21.15.89:0/1390772879 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe71c1a0490 con 0x7fe71c102ae0 2024-09-21T03:38:12.868 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:12.864+0000 7fe7217a8640 1 --2- 172.21.15.89:0/1390772879 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe71c108420 0x7fe71c19aac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:38:12.868 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:12.864+0000 7fe721fa9640 1 --2- 172.21.15.89:0/1390772879 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe71c102ae0 0x7fe71c19a520 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fe71800e9f0 tx=0x7fe71800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:38:12.869 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:12.864+0000 7fe712ffd640 1 -- 172.21.15.89:0/1390772879 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe71800cd80 con 0x7fe71c102ae0 2024-09-21T03:38:12.869 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:12.864+0000 7fe712ffd640 1 -- 172.21.15.89:0/1390772879 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe7180054b0 con 0x7fe71c102ae0 2024-09-21T03:38:12.869 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:12.864+0000 7fe72420d640 1 -- 172.21.15.89:0/1390772879 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe71c1a0780 con 0x7fe71c102ae0 2024-09-21T03:38:12.869 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:12.864+0000 7fe712ffd640 1 -- 172.21.15.89:0/1390772879 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe718010430 con 0x7fe71c102ae0 2024-09-21T03:38:12.869 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:12.864+0000 7fe72420d640 1 -- 172.21.15.89:0/1390772879 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe71c1a6ee0 con 0x7fe71c102ae0 2024-09-21T03:38:12.869 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:12.865+0000 7fe72420d640 1 -- 172.21.15.89:0/1390772879 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe71c103d60 con 0x7fe71c102ae0 2024-09-21T03:38:12.869 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:12.867+0000 7fe712ffd640 1 -- 172.21.15.89:0/1390772879 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 18) ==== 154563+0+0 (secure 0 0 0) 0x7fe7180046c0 con 0x7fe71c102ae0 2024-09-21T03:38:12.870 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:12.868+0000 7fe712ffd640 1 --2- 172.21.15.89:0/1390772879 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7fe7080b65a0 0x7fe7080b8ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:12.870 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:12.868+0000 7fe712ffd640 1 -- 172.21.15.89:0/1390772879 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 9092+0+0 (secure 0 0 0) 0x7fe7180e7e10 con 0x7fe71c102ae0 2024-09-21T03:38:12.870 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:12.868+0000 7fe7217a8640 1 --2- 172.21.15.89:0/1390772879 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7fe7080b65a0 0x7fe7080b8ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:12.870 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:12.868+0000 7fe7217a8640 1 --2- 172.21.15.89:0/1390772879 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7fe7080b65a0 0x7fe7080b8ab0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fe70c006010 tx=0x7fe70c000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:38:12.873 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:12.871+0000 7fe712ffd640 1 -- 172.21.15.89:0/1390772879 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fe718056710 con 0x7fe71c102ae0 2024-09-21T03:38:13.053 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:13.051+0000 7fe72420d640 1 -- 172.21.15.89:0/1390772879 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe71c10b5e0 con 0x7fe71c102ae0 2024-09-21T03:38:13.055 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:13.053+0000 7fe712ffd640 1 -- 172.21.15.89:0/1390772879 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) ==== 74+0+24763 (secure 0 0 0) 0x7fe7180ae1e0 con 0x7fe71c102ae0 2024-09-21T03:38:13.055 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-09-21T03:38:13.056 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":82,"fsid":"7c374098-77c9-11ef-baf5-efdc52797490","created":"2024-09-21T03:28:56.681458+0000","modified":"2024-09-21T03:37:53.033682+0000","last_up_change":"2024-09-21T03:36:23.368379+0000","last_in_change":"2024-09-21T03:36:00.131331+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T03:32:37.834339+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-21T03:37:09.443341+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"74","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-21T03:37:11.467821+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"74","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-21T03:37:17.884066+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"27ca5ed8-17d1-4f8e-a7ae-084a914721a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6802","nonce":990539042},{"type":"v1","addr":"172.21.15.89:6803","nonce":990539042}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6804","nonce":990539042},{"type":"v1","addr":"172.21.15.89:6805","nonce":990539042}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6808","nonce":990539042},{"type":"v1","addr":"172.21.15.89:6809","nonce":990539042}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6806","nonce":990539042},{"type":"v1","addr":"172.21.15.89:6807","nonce":990539042}]},"public_addr":"172.21.15.89:6803/990539042","cluster_addr":"172.21.15.89:6805/990539042","heartbeat_back_addr":"172.21.15.89:6809/990539042","heartbeat_front_addr":"172.21.15.89:6807/990539042","state":["exists","up"]},{"osd":1,"uuid":"5176a910-5cf9-49d7-aedb-a20cf36ea075","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6800","nonce":3235852628},{"type":"v1","addr":"172.21.15.110:6801","nonce":3235852628}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6802","nonce":3235852628},{"type":"v1","addr":"172.21.15.110:6803","nonce":3235852628}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6806","nonce":3235852628},{"type":"v1","addr":"172.21.15.110:6807","nonce":3235852628}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6804","nonce":3235852628},{"type":"v1","addr":"172.21.15.110:6805","nonce":3235852628}]},"public_addr":"172.21.15.110:6801/3235852628","cluster_addr":"172.21.15.110:6803/3235852628","heartbeat_back_addr":"172.21.15.110:6807/3235852628","heartbeat_front_addr":"172.21.15.110:6805/3235852628","state":["exists","up"]},{"osd":2,"uuid":"d4977df1-b118-4d7e-b9a9-626a8accc812","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6800","nonce":169452122},{"type":"v1","addr":"172.21.15.114:6801","nonce":169452122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6802","nonce":169452122},{"type":"v1","addr":"172.21.15.114:6803","nonce":169452122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6806","nonce":169452122},{"type":"v1","addr":"172.21.15.114:6807","nonce":169452122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6804","nonce":169452122},{"type":"v1","addr":"172.21.15.114:6805","nonce":169452122}]},"public_addr":"172.21.15.114:6801/169452122","cluster_addr":"172.21.15.114:6803/169452122","heartbeat_back_addr":"172.21.15.114:6807/169452122","heartbeat_front_addr":"172.21.15.114:6805/169452122","state":["exists","up"]},{"osd":3,"uuid":"2c7ae1b4-0bf2-4954-81ed-55d03eab82c6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6810","nonce":3016980877},{"type":"v1","addr":"172.21.15.89:6811","nonce":3016980877}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6812","nonce":3016980877},{"type":"v1","addr":"172.21.15.89:6813","nonce":3016980877}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6816","nonce":3016980877},{"type":"v1","addr":"172.21.15.89:6817","nonce":3016980877}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6814","nonce":3016980877},{"type":"v1","addr":"172.21.15.89:6815","nonce":3016980877}]},"public_addr":"172.21.15.89:6811/3016980877","cluster_addr":"172.21.15.89:6813/3016980877","heartbeat_back_addr":"172.21.15.89:6817/3016980877","heartbeat_front_addr":"172.21.15.89:6815/3016980877","state":["exists","up"]},{"osd":4,"uuid":"8e65399d-6e55-4da4-98ba-f805f0c655a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":30,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6808","nonce":3217880476},{"type":"v1","addr":"172.21.15.110:6809","nonce":3217880476}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6810","nonce":3217880476},{"type":"v1","addr":"172.21.15.110:6811","nonce":3217880476}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6814","nonce":3217880476},{"type":"v1","addr":"172.21.15.110:6815","nonce":3217880476}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6812","nonce":3217880476},{"type":"v1","addr":"172.21.15.110:6813","nonce":3217880476}]},"public_addr":"172.21.15.110:6809/3217880476","cluster_addr":"172.21.15.110:6811/3217880476","heartbeat_back_addr":"172.21.15.110:6815/3217880476","heartbeat_front_addr":"172.21.15.110:6813/3217880476","state":["exists","up"]},{"osd":5,"uuid":"b9ec2889-46bd-40e5-8446-73ab28271c67","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6808","nonce":1144140707},{"type":"v1","addr":"172.21.15.114:6809","nonce":1144140707}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6810","nonce":1144140707},{"type":"v1","addr":"172.21.15.114:6811","nonce":1144140707}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6814","nonce":1144140707},{"type":"v1","addr":"172.21.15.114:6815","nonce":1144140707}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6812","nonce":1144140707},{"type":"v1","addr":"172.21.15.114:6813","nonce":1144140707}]},"public_addr":"172.21.15.114:6809/1144140707","cluster_addr":"172.21.15.114:6811/1144140707","heartbeat_back_addr":"172.21.15.114:6815/1144140707","heartbeat_front_addr":"172.21.15.114:6813/1144140707","state":["exists","up"]},{"osd":6,"uuid":"9b79989e-adb0-42b2-b2f6-a2dce56b66e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":41,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6818","nonce":1522586255},{"type":"v1","addr":"172.21.15.89:6819","nonce":1522586255}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6820","nonce":1522586255},{"type":"v1","addr":"172.21.15.89:6821","nonce":1522586255}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6824","nonce":1522586255},{"type":"v1","addr":"172.21.15.89:6825","nonce":1522586255}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6822","nonce":1522586255},{"type":"v1","addr":"172.21.15.89:6823","nonce":1522586255}]},"public_addr":"172.21.15.89:6819/1522586255","cluster_addr":"172.21.15.89:6821/1522586255","heartbeat_back_addr":"172.21.15.89:6825/1522586255","heartbeat_front_addr":"172.21.15.89:6823/1522586255","state":["exists","up"]},{"osd":7,"uuid":"b381821f-141c-47a1-be65-f57ca730391b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6816","nonce":1331818323},{"type":"v1","addr":"172.21.15.110:6817","nonce":1331818323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6818","nonce":1331818323},{"type":"v1","addr":"172.21.15.110:6819","nonce":1331818323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6822","nonce":1331818323},{"type":"v1","addr":"172.21.15.110:6823","nonce":1331818323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6820","nonce":1331818323},{"type":"v1","addr":"172.21.15.110:6821","nonce":1331818323}]},"public_addr":"172.21.15.110:6817/1331818323","cluster_addr":"172.21.15.110:6819/1331818323","heartbeat_back_addr":"172.21.15.110:6823/1331818323","heartbeat_front_addr":"172.21.15.110:6821/1331818323","state":["exists","up"]},{"osd":8,"uuid":"1bc2a5b4-15f3-456f-9d67-e51307a27bcc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6816","nonce":2138556699},{"type":"v1","addr":"172.21.15.114:6817","nonce":2138556699}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6818","nonce":2138556699},{"type":"v1","addr":"172.21.15.114:6819","nonce":2138556699}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6822","nonce":2138556699},{"type":"v1","addr":"172.21.15.114:6823","nonce":2138556699}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6820","nonce":2138556699},{"type":"v1","addr":"172.21.15.114:6821","nonce":2138556699}]},"public_addr":"172.21.15.114:6817/2138556699","cluster_addr":"172.21.15.114:6819/2138556699","heartbeat_back_addr":"172.21.15.114:6823/2138556699","heartbeat_front_addr":"172.21.15.114:6821/2138556699","state":["exists","up"]},{"osd":9,"uuid":"91f5fd29-d4a7-4800-ac7a-e83b13add89e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6826","nonce":1730253068},{"type":"v1","addr":"172.21.15.89:6827","nonce":1730253068}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6828","nonce":1730253068},{"type":"v1","addr":"172.21.15.89:6829","nonce":1730253068}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6832","nonce":1730253068},{"type":"v1","addr":"172.21.15.89:6833","nonce":1730253068}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6830","nonce":1730253068},{"type":"v1","addr":"172.21.15.89:6831","nonce":1730253068}]},"public_addr":"172.21.15.89:6827/1730253068","cluster_addr":"172.21.15.89:6829/1730253068","heartbeat_back_addr":"172.21.15.89:6833/1730253068","heartbeat_front_addr":"172.21.15.89:6831/1730253068","state":["exists","up"]},{"osd":10,"uuid":"01390499-2a4d-4188-b57a-fcb10c668e2c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6824","nonce":2904016117},{"type":"v1","addr":"172.21.15.110:6825","nonce":2904016117}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6826","nonce":2904016117},{"type":"v1","addr":"172.21.15.110:6827","nonce":2904016117}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6830","nonce":2904016117},{"type":"v1","addr":"172.21.15.110:6831","nonce":2904016117}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6828","nonce":2904016117},{"type":"v1","addr":"172.21.15.110:6829","nonce":2904016117}]},"public_addr":"172.21.15.110:6825/2904016117","cluster_addr":"172.21.15.110:6827/2904016117","heartbeat_back_addr":"172.21.15.110:6831/2904016117","heartbeat_front_addr":"172.21.15.110:6829/2904016117","state":["exists","up"]},{"osd":11,"uuid":"c3c0f38a-c3d3-415d-a9f8-875d5b756e35","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6824","nonce":3749854486},{"type":"v1","addr":"172.21.15.114:6825","nonce":3749854486}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6826","nonce":3749854486},{"type":"v1","addr":"172.21.15.114:6827","nonce":3749854486}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6830","nonce":3749854486},{"type":"v1","addr":"172.21.15.114:6831","nonce":3749854486}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6828","nonce":3749854486},{"type":"v1","addr":"172.21.15.114:6829","nonce":3749854486}]},"public_addr":"172.21.15.114:6825/3749854486","cluster_addr":"172.21.15.114:6827/3749854486","heartbeat_back_addr":"172.21.15.114:6831/3749854486","heartbeat_front_addr":"172.21.15.114:6829/3749854486","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:31:46.218908+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:32:10.338230+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:32:34.830596+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:33:00.816102+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:33:25.223838+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:33:50.188230+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:34:16.564255+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:34:41.498230+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:35:05.826322+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:35:31.234450+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:35:56.474373+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:36:21.741254+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.c","mappings":[{"from":2,"to":4}]},{"pgid":"2.e","mappings":[{"from":2,"to":1}]},{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]},{"pgid":"4.6","mappings":[{"from":9,"to":11}]},{"pgid":"4.12","mappings":[{"from":9,"to":10}]},{"pgid":"4.13","mappings":[{"from":9,"to":8}]},{"pgid":"4.1b","mappings":[{"from":5,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.89:6801/655110910":"2024-09-22T03:29:51.519106+0000","172.21.15.89:0/1618544983":"2024-09-22T03:29:51.519106+0000","172.21.15.89:0/1265502758":"2024-09-22T03:29:51.519106+0000","172.21.15.89:0/4017684419":"2024-09-22T03:29:51.519106+0000","172.21.15.89:6801/3312712420":"2024-09-22T03:29:26.004686+0000","172.21.15.89:6800/3312712420":"2024-09-22T03:29:26.004686+0000","172.21.15.89:0/2056576483":"2024-09-22T03:29:26.004686+0000","172.21.15.89:6800/655110910":"2024-09-22T03:29:51.519106+0000","172.21.15.89:0/2382866302":"2024-09-22T03:29:26.004686+0000","172.21.15.89:0/1584351801":"2024-09-22T03:29:26.004686+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T03:38:13.057 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:13.055+0000 7fe72420d640 1 -- 172.21.15.89:0/1390772879 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7fe7080b65a0 msgr2=0x7fe7080b8ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:13.057 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:13.055+0000 7fe72420d640 1 --2- 172.21.15.89:0/1390772879 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7fe7080b65a0 0x7fe7080b8ab0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fe70c006010 tx=0x7fe70c000bc0 comp rx=0 tx=0).stop 2024-09-21T03:38:13.058 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:13.055+0000 7fe72420d640 1 -- 172.21.15.89:0/1390772879 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe71c102ae0 msgr2=0x7fe71c19a520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:13.058 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:13.055+0000 7fe72420d640 1 --2- 172.21.15.89:0/1390772879 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe71c102ae0 0x7fe71c19a520 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fe71800e9f0 tx=0x7fe71800edf0 comp rx=0 tx=0).stop 2024-09-21T03:38:13.058 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:13.055+0000 7fe72420d640 1 -- 172.21.15.89:0/1390772879 shutdown_connections 2024-09-21T03:38:13.058 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:13.055+0000 7fe72420d640 1 --2- 172.21.15.89:0/1390772879 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7fe7080b65a0 0x7fe7080b8ab0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:13.058 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:13.055+0000 7fe72420d640 1 --2- 172.21.15.89:0/1390772879 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe71c108420 0x7fe71c19aac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:13.058 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:13.055+0000 7fe72420d640 1 --2- 172.21.15.89:0/1390772879 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe71c108d70 0x7fe71c19fd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:13.058 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:13.055+0000 7fe72420d640 1 --2- 172.21.15.89:0/1390772879 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe71c102ae0 0x7fe71c19a520 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:13.058 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:13.055+0000 7fe72420d640 1 -- 172.21.15.89:0/1390772879 >> 172.21.15.89:0/1390772879 conn(0x7fe71c0796b0 msgr2=0x7fe71c101c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:38:13.058 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:13.055+0000 7fe72420d640 1 -- 172.21.15.89:0/1390772879 shutdown_connections 2024-09-21T03:38:13.058 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:13.055+0000 7fe72420d640 1 -- 172.21.15.89:0/1390772879 wait complete. 2024-09-21T03:38:13.688 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:38:13 smithi089 ceph-mon[30928]: pgmap v322: 129 pgs: 129 active+clean; 644 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 1.3 KiB/s wr, 6 op/s 2024-09-21T03:38:13.688 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:38:13 smithi089 ceph-mon[30928]: from='client.? 172.21.15.89:0/1390772879' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T03:38:13.710 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:38:13 smithi114 ceph-mon[34140]: pgmap v322: 129 pgs: 129 active+clean; 644 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 1.3 KiB/s wr, 6 op/s 2024-09-21T03:38:13.711 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:38:13 smithi114 ceph-mon[34140]: from='client.? 172.21.15.89:0/1390772879' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T03:38:13.717 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-09-21T03:38:13.717 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 7c374098-77c9-11ef-baf5-efdc52797490 -- ceph osd dump --format=json 2024-09-21T03:38:13.750 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:38:13 smithi110 ceph-mon[34000]: pgmap v322: 129 pgs: 129 active+clean; 644 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 1.3 KiB/s wr, 6 op/s 2024-09-21T03:38:13.750 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:38:13 smithi110 ceph-mon[34000]: from='client.? 172.21.15.89:0/1390772879' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T03:38:13.965 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/7c374098-77c9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:38:14.440 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:14.438+0000 7f2c015ca640 1 -- 172.21.15.89:0/2364630609 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2bfc1083a0 msgr2=0x7f2bfc1087b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:14.440 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:14.438+0000 7f2c015ca640 1 --2- 172.21.15.89:0/2364630609 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2bfc1083a0 0x7f2bfc1087b0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f2bf0009ca0 tx=0x7f2bf002a8c0 comp rx=0 tx=0).stop 2024-09-21T03:38:14.441 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:14.438+0000 7f2c015ca640 1 -- 172.21.15.89:0/2364630609 shutdown_connections 2024-09-21T03:38:14.441 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:14.438+0000 7f2c015ca640 1 --2- 172.21.15.89:0/2364630609 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2bfc108cf0 0x7f2bfc10f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:14.441 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:14.438+0000 7f2c015ca640 1 --2- 172.21.15.89:0/2364630609 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2bfc1083a0 0x7f2bfc1087b0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:14.441 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:14.438+0000 7f2c015ca640 1 --2- 172.21.15.89:0/2364630609 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2bfc102a60 0x7f2bfc102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:14.441 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:14.438+0000 7f2c015ca640 1 -- 172.21.15.89:0/2364630609 >> 172.21.15.89:0/2364630609 conn(0x7f2bfc079730 msgr2=0x7f2bfc079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:38:14.441 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:14.438+0000 7f2c015ca640 1 -- 172.21.15.89:0/2364630609 shutdown_connections 2024-09-21T03:38:14.441 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:14.438+0000 7f2c015ca640 1 -- 172.21.15.89:0/2364630609 wait complete. 2024-09-21T03:38:14.443 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:14.439+0000 7f2c015ca640 1 Processor -- start 2024-09-21T03:38:14.443 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:14.439+0000 7f2c015ca640 1 -- start start 2024-09-21T03:38:14.443 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:14.439+0000 7f2c015ca640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2bfc102a60 0x7f2bfc19a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:14.443 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:14.439+0000 7f2c015ca640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2bfc1083a0 0x7f2bfc19a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:14.443 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:14.439+0000 7f2c015ca640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2bfc108cf0 0x7f2bfc19fc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:14.443 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:14.439+0000 7f2c015ca640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f2bfc113960 con 0x7f2bfc1083a0 2024-09-21T03:38:14.443 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:14.439+0000 7f2c015ca640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f2bfc1137e0 con 0x7f2bfc102a60 2024-09-21T03:38:14.443 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:14.439+0000 7f2c015ca640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f2bfc113ae0 con 0x7f2bfc108cf0 2024-09-21T03:38:14.443 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:14.439+0000 7f2bfa7fc640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2bfc1083a0 0x7f2bfc19a9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:14.443 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:14.439+0000 7f2bfa7fc640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2bfc1083a0 0x7f2bfc19a9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:38888/0 (socket says 172.21.15.89:38888) 2024-09-21T03:38:14.443 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:14.439+0000 7f2bfa7fc640 1 -- 172.21.15.89:0/3976286753 learned_addr learned my addr 172.21.15.89:0/3976286753 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-21T03:38:14.443 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:14.440+0000 7f2bfb7fe640 1 --2- 172.21.15.89:0/3976286753 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2bfc108cf0 0x7f2bfc19fc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:14.443 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:14.440+0000 7f2bfaffd640 1 --2- 172.21.15.89:0/3976286753 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2bfc102a60 0x7f2bfc19a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:14.444 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:14.440+0000 7f2bfa7fc640 1 -- 172.21.15.89:0/3976286753 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2bfc108cf0 msgr2=0x7f2bfc19fc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:14.444 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:14.440+0000 7f2bfa7fc640 1 --2- 172.21.15.89:0/3976286753 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2bfc108cf0 0x7f2bfc19fc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:14.444 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:14.440+0000 7f2bfa7fc640 1 -- 172.21.15.89:0/3976286753 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2bfc102a60 msgr2=0x7f2bfc19a430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:14.444 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:14.440+0000 7f2bfa7fc640 1 --2- 172.21.15.89:0/3976286753 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2bfc102a60 0x7f2bfc19a430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:14.444 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:14.440+0000 7f2bfa7fc640 1 -- 172.21.15.89:0/3976286753 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2bfc1a02e0 con 0x7f2bfc1083a0 2024-09-21T03:38:14.444 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:14.440+0000 7f2bfa7fc640 1 --2- 172.21.15.89:0/3976286753 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2bfc1083a0 0x7f2bfc19a9d0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f2bf0032a10 tx=0x7f2bf0032a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:38:14.444 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:14.440+0000 7f2be3fff640 1 -- 172.21.15.89:0/3976286753 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bf0032e60 con 0x7f2bfc1083a0 2024-09-21T03:38:14.444 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:14.440+0000 7f2be3fff640 1 -- 172.21.15.89:0/3976286753 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2bf00044d0 con 0x7f2bfc1083a0 2024-09-21T03:38:14.444 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:14.440+0000 7f2c015ca640 1 -- 172.21.15.89:0/3976286753 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2bfc1a0570 con 0x7f2bfc1083a0 2024-09-21T03:38:14.444 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:14.440+0000 7f2be3fff640 1 -- 172.21.15.89:0/3976286753 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bf0030c00 con 0x7f2bfc1083a0 2024-09-21T03:38:14.444 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:14.440+0000 7f2c015ca640 1 -- 172.21.15.89:0/3976286753 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2bfc1a0ab0 con 0x7f2bfc1083a0 2024-09-21T03:38:14.445 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:14.441+0000 7f2c015ca640 1 -- 172.21.15.89:0/3976286753 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2bfc102e70 con 0x7f2bfc1083a0 2024-09-21T03:38:14.445 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:14.443+0000 7f2be3fff640 1 -- 172.21.15.89:0/3976286753 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 18) ==== 154563+0+0 (secure 0 0 0) 0x7f2bf0004040 con 0x7f2bfc1083a0 2024-09-21T03:38:14.446 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:14.443+0000 7f2be3fff640 1 --2- 172.21.15.89:0/3976286753 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f2be40b65d0 0x7f2be40b8ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:14.446 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:14.444+0000 7f2bfaffd640 1 --2- 172.21.15.89:0/3976286753 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f2be40b65d0 0x7f2be40b8ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:14.446 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:14.444+0000 7f2be3fff640 1 -- 172.21.15.89:0/3976286753 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 9092+0+0 (secure 0 0 0) 0x7f2bf0108bb0 con 0x7f2bfc1083a0 2024-09-21T03:38:14.446 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:14.444+0000 7f2bfaffd640 1 --2- 172.21.15.89:0/3976286753 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f2be40b65d0 0x7f2be40b8ae0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f2bf400bbe0 tx=0x7f2bf400b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:38:14.449 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:14.447+0000 7f2be3fff640 1 -- 172.21.15.89:0/3976286753 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f2bf00cef80 con 0x7f2bfc1083a0 2024-09-21T03:38:14.627 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:14.625+0000 7f2c015ca640 1 -- 172.21.15.89:0/3976286753 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f2bfc10ab20 con 0x7f2bfc1083a0 2024-09-21T03:38:14.628 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:14.626+0000 7f2be3fff640 1 -- 172.21.15.89:0/3976286753 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) ==== 74+0+24763 (secure 0 0 0) 0x7f2bf00098b0 con 0x7f2bfc1083a0 2024-09-21T03:38:14.629 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-09-21T03:38:14.629 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":82,"fsid":"7c374098-77c9-11ef-baf5-efdc52797490","created":"2024-09-21T03:28:56.681458+0000","modified":"2024-09-21T03:37:53.033682+0000","last_up_change":"2024-09-21T03:36:23.368379+0000","last_in_change":"2024-09-21T03:36:00.131331+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T03:32:37.834339+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-21T03:37:09.443341+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"74","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-21T03:37:11.467821+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"74","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-21T03:37:17.884066+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"27ca5ed8-17d1-4f8e-a7ae-084a914721a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6802","nonce":990539042},{"type":"v1","addr":"172.21.15.89:6803","nonce":990539042}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6804","nonce":990539042},{"type":"v1","addr":"172.21.15.89:6805","nonce":990539042}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6808","nonce":990539042},{"type":"v1","addr":"172.21.15.89:6809","nonce":990539042}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6806","nonce":990539042},{"type":"v1","addr":"172.21.15.89:6807","nonce":990539042}]},"public_addr":"172.21.15.89:6803/990539042","cluster_addr":"172.21.15.89:6805/990539042","heartbeat_back_addr":"172.21.15.89:6809/990539042","heartbeat_front_addr":"172.21.15.89:6807/990539042","state":["exists","up"]},{"osd":1,"uuid":"5176a910-5cf9-49d7-aedb-a20cf36ea075","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6800","nonce":3235852628},{"type":"v1","addr":"172.21.15.110:6801","nonce":3235852628}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6802","nonce":3235852628},{"type":"v1","addr":"172.21.15.110:6803","nonce":3235852628}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6806","nonce":3235852628},{"type":"v1","addr":"172.21.15.110:6807","nonce":3235852628}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6804","nonce":3235852628},{"type":"v1","addr":"172.21.15.110:6805","nonce":3235852628}]},"public_addr":"172.21.15.110:6801/3235852628","cluster_addr":"172.21.15.110:6803/3235852628","heartbeat_back_addr":"172.21.15.110:6807/3235852628","heartbeat_front_addr":"172.21.15.110:6805/3235852628","state":["exists","up"]},{"osd":2,"uuid":"d4977df1-b118-4d7e-b9a9-626a8accc812","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6800","nonce":169452122},{"type":"v1","addr":"172.21.15.114:6801","nonce":169452122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6802","nonce":169452122},{"type":"v1","addr":"172.21.15.114:6803","nonce":169452122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6806","nonce":169452122},{"type":"v1","addr":"172.21.15.114:6807","nonce":169452122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6804","nonce":169452122},{"type":"v1","addr":"172.21.15.114:6805","nonce":169452122}]},"public_addr":"172.21.15.114:6801/169452122","cluster_addr":"172.21.15.114:6803/169452122","heartbeat_back_addr":"172.21.15.114:6807/169452122","heartbeat_front_addr":"172.21.15.114:6805/169452122","state":["exists","up"]},{"osd":3,"uuid":"2c7ae1b4-0bf2-4954-81ed-55d03eab82c6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6810","nonce":3016980877},{"type":"v1","addr":"172.21.15.89:6811","nonce":3016980877}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6812","nonce":3016980877},{"type":"v1","addr":"172.21.15.89:6813","nonce":3016980877}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6816","nonce":3016980877},{"type":"v1","addr":"172.21.15.89:6817","nonce":3016980877}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6814","nonce":3016980877},{"type":"v1","addr":"172.21.15.89:6815","nonce":3016980877}]},"public_addr":"172.21.15.89:6811/3016980877","cluster_addr":"172.21.15.89:6813/3016980877","heartbeat_back_addr":"172.21.15.89:6817/3016980877","heartbeat_front_addr":"172.21.15.89:6815/3016980877","state":["exists","up"]},{"osd":4,"uuid":"8e65399d-6e55-4da4-98ba-f805f0c655a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":30,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6808","nonce":3217880476},{"type":"v1","addr":"172.21.15.110:6809","nonce":3217880476}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6810","nonce":3217880476},{"type":"v1","addr":"172.21.15.110:6811","nonce":3217880476}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6814","nonce":3217880476},{"type":"v1","addr":"172.21.15.110:6815","nonce":3217880476}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6812","nonce":3217880476},{"type":"v1","addr":"172.21.15.110:6813","nonce":3217880476}]},"public_addr":"172.21.15.110:6809/3217880476","cluster_addr":"172.21.15.110:6811/3217880476","heartbeat_back_addr":"172.21.15.110:6815/3217880476","heartbeat_front_addr":"172.21.15.110:6813/3217880476","state":["exists","up"]},{"osd":5,"uuid":"b9ec2889-46bd-40e5-8446-73ab28271c67","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6808","nonce":1144140707},{"type":"v1","addr":"172.21.15.114:6809","nonce":1144140707}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6810","nonce":1144140707},{"type":"v1","addr":"172.21.15.114:6811","nonce":1144140707}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6814","nonce":1144140707},{"type":"v1","addr":"172.21.15.114:6815","nonce":1144140707}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6812","nonce":1144140707},{"type":"v1","addr":"172.21.15.114:6813","nonce":1144140707}]},"public_addr":"172.21.15.114:6809/1144140707","cluster_addr":"172.21.15.114:6811/1144140707","heartbeat_back_addr":"172.21.15.114:6815/1144140707","heartbeat_front_addr":"172.21.15.114:6813/1144140707","state":["exists","up"]},{"osd":6,"uuid":"9b79989e-adb0-42b2-b2f6-a2dce56b66e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":41,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6818","nonce":1522586255},{"type":"v1","addr":"172.21.15.89:6819","nonce":1522586255}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6820","nonce":1522586255},{"type":"v1","addr":"172.21.15.89:6821","nonce":1522586255}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6824","nonce":1522586255},{"type":"v1","addr":"172.21.15.89:6825","nonce":1522586255}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6822","nonce":1522586255},{"type":"v1","addr":"172.21.15.89:6823","nonce":1522586255}]},"public_addr":"172.21.15.89:6819/1522586255","cluster_addr":"172.21.15.89:6821/1522586255","heartbeat_back_addr":"172.21.15.89:6825/1522586255","heartbeat_front_addr":"172.21.15.89:6823/1522586255","state":["exists","up"]},{"osd":7,"uuid":"b381821f-141c-47a1-be65-f57ca730391b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6816","nonce":1331818323},{"type":"v1","addr":"172.21.15.110:6817","nonce":1331818323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6818","nonce":1331818323},{"type":"v1","addr":"172.21.15.110:6819","nonce":1331818323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6822","nonce":1331818323},{"type":"v1","addr":"172.21.15.110:6823","nonce":1331818323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6820","nonce":1331818323},{"type":"v1","addr":"172.21.15.110:6821","nonce":1331818323}]},"public_addr":"172.21.15.110:6817/1331818323","cluster_addr":"172.21.15.110:6819/1331818323","heartbeat_back_addr":"172.21.15.110:6823/1331818323","heartbeat_front_addr":"172.21.15.110:6821/1331818323","state":["exists","up"]},{"osd":8,"uuid":"1bc2a5b4-15f3-456f-9d67-e51307a27bcc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6816","nonce":2138556699},{"type":"v1","addr":"172.21.15.114:6817","nonce":2138556699}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6818","nonce":2138556699},{"type":"v1","addr":"172.21.15.114:6819","nonce":2138556699}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6822","nonce":2138556699},{"type":"v1","addr":"172.21.15.114:6823","nonce":2138556699}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6820","nonce":2138556699},{"type":"v1","addr":"172.21.15.114:6821","nonce":2138556699}]},"public_addr":"172.21.15.114:6817/2138556699","cluster_addr":"172.21.15.114:6819/2138556699","heartbeat_back_addr":"172.21.15.114:6823/2138556699","heartbeat_front_addr":"172.21.15.114:6821/2138556699","state":["exists","up"]},{"osd":9,"uuid":"91f5fd29-d4a7-4800-ac7a-e83b13add89e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6826","nonce":1730253068},{"type":"v1","addr":"172.21.15.89:6827","nonce":1730253068}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6828","nonce":1730253068},{"type":"v1","addr":"172.21.15.89:6829","nonce":1730253068}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6832","nonce":1730253068},{"type":"v1","addr":"172.21.15.89:6833","nonce":1730253068}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6830","nonce":1730253068},{"type":"v1","addr":"172.21.15.89:6831","nonce":1730253068}]},"public_addr":"172.21.15.89:6827/1730253068","cluster_addr":"172.21.15.89:6829/1730253068","heartbeat_back_addr":"172.21.15.89:6833/1730253068","heartbeat_front_addr":"172.21.15.89:6831/1730253068","state":["exists","up"]},{"osd":10,"uuid":"01390499-2a4d-4188-b57a-fcb10c668e2c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6824","nonce":2904016117},{"type":"v1","addr":"172.21.15.110:6825","nonce":2904016117}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6826","nonce":2904016117},{"type":"v1","addr":"172.21.15.110:6827","nonce":2904016117}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6830","nonce":2904016117},{"type":"v1","addr":"172.21.15.110:6831","nonce":2904016117}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6828","nonce":2904016117},{"type":"v1","addr":"172.21.15.110:6829","nonce":2904016117}]},"public_addr":"172.21.15.110:6825/2904016117","cluster_addr":"172.21.15.110:6827/2904016117","heartbeat_back_addr":"172.21.15.110:6831/2904016117","heartbeat_front_addr":"172.21.15.110:6829/2904016117","state":["exists","up"]},{"osd":11,"uuid":"c3c0f38a-c3d3-415d-a9f8-875d5b756e35","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6824","nonce":3749854486},{"type":"v1","addr":"172.21.15.114:6825","nonce":3749854486}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6826","nonce":3749854486},{"type":"v1","addr":"172.21.15.114:6827","nonce":3749854486}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6830","nonce":3749854486},{"type":"v1","addr":"172.21.15.114:6831","nonce":3749854486}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6828","nonce":3749854486},{"type":"v1","addr":"172.21.15.114:6829","nonce":3749854486}]},"public_addr":"172.21.15.114:6825/3749854486","cluster_addr":"172.21.15.114:6827/3749854486","heartbeat_back_addr":"172.21.15.114:6831/3749854486","heartbeat_front_addr":"172.21.15.114:6829/3749854486","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:31:46.218908+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:32:10.338230+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:32:34.830596+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:33:00.816102+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:33:25.223838+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:33:50.188230+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:34:16.564255+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:34:41.498230+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:35:05.826322+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:35:31.234450+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:35:56.474373+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:36:21.741254+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.c","mappings":[{"from":2,"to":4}]},{"pgid":"2.e","mappings":[{"from":2,"to":1}]},{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]},{"pgid":"4.6","mappings":[{"from":9,"to":11}]},{"pgid":"4.12","mappings":[{"from":9,"to":10}]},{"pgid":"4.13","mappings":[{"from":9,"to":8}]},{"pgid":"4.1b","mappings":[{"from":5,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.89:6801/655110910":"2024-09-22T03:29:51.519106+0000","172.21.15.89:0/1618544983":"2024-09-22T03:29:51.519106+0000","172.21.15.89:0/1265502758":"2024-09-22T03:29:51.519106+0000","172.21.15.89:0/4017684419":"2024-09-22T03:29:51.519106+0000","172.21.15.89:6801/3312712420":"2024-09-22T03:29:26.004686+0000","172.21.15.89:6800/3312712420":"2024-09-22T03:29:26.004686+0000","172.21.15.89:0/2056576483":"2024-09-22T03:29:26.004686+0000","172.21.15.89:6800/655110910":"2024-09-22T03:29:51.519106+0000","172.21.15.89:0/2382866302":"2024-09-22T03:29:26.004686+0000","172.21.15.89:0/1584351801":"2024-09-22T03:29:26.004686+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T03:38:14.631 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:14.629+0000 7f2c015ca640 1 -- 172.21.15.89:0/3976286753 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f2be40b65d0 msgr2=0x7f2be40b8ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:14.631 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:14.629+0000 7f2c015ca640 1 --2- 172.21.15.89:0/3976286753 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f2be40b65d0 0x7f2be40b8ae0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f2bf400bbe0 tx=0x7f2bf400b390 comp rx=0 tx=0).stop 2024-09-21T03:38:14.631 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:14.629+0000 7f2c015ca640 1 -- 172.21.15.89:0/3976286753 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2bfc1083a0 msgr2=0x7f2bfc19a9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:14.631 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:14.629+0000 7f2c015ca640 1 --2- 172.21.15.89:0/3976286753 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2bfc1083a0 0x7f2bfc19a9d0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f2bf0032a10 tx=0x7f2bf0032a40 comp rx=0 tx=0).stop 2024-09-21T03:38:14.631 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:14.629+0000 7f2c015ca640 1 -- 172.21.15.89:0/3976286753 shutdown_connections 2024-09-21T03:38:14.631 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:14.629+0000 7f2c015ca640 1 --2- 172.21.15.89:0/3976286753 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f2be40b65d0 0x7f2be40b8ae0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:14.631 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:14.629+0000 7f2c015ca640 1 --2- 172.21.15.89:0/3976286753 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2bfc108cf0 0x7f2bfc19fc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:14.632 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:14.629+0000 7f2c015ca640 1 --2- 172.21.15.89:0/3976286753 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2bfc1083a0 0x7f2bfc19a9d0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:14.632 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:14.629+0000 7f2c015ca640 1 --2- 172.21.15.89:0/3976286753 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2bfc102a60 0x7f2bfc19a430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:14.632 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:14.629+0000 7f2c015ca640 1 -- 172.21.15.89:0/3976286753 >> 172.21.15.89:0/3976286753 conn(0x7f2bfc079730 msgr2=0x7f2bfc1044c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:38:14.632 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:14.629+0000 7f2c015ca640 1 -- 172.21.15.89:0/3976286753 shutdown_connections 2024-09-21T03:38:14.632 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:14.629+0000 7f2c015ca640 1 -- 172.21.15.89:0/3976286753 wait complete. 2024-09-21T03:38:15.224 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 7c374098-77c9-11ef-baf5-efdc52797490 -- ceph tell osd.0 flush_pg_stats 2024-09-21T03:38:15.224 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 7c374098-77c9-11ef-baf5-efdc52797490 -- ceph tell osd.1 flush_pg_stats 2024-09-21T03:38:15.225 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 7c374098-77c9-11ef-baf5-efdc52797490 -- ceph tell osd.2 flush_pg_stats 2024-09-21T03:38:15.225 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 7c374098-77c9-11ef-baf5-efdc52797490 -- ceph tell osd.3 flush_pg_stats 2024-09-21T03:38:15.225 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 7c374098-77c9-11ef-baf5-efdc52797490 -- ceph tell osd.4 flush_pg_stats 2024-09-21T03:38:15.226 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 7c374098-77c9-11ef-baf5-efdc52797490 -- ceph tell osd.5 flush_pg_stats 2024-09-21T03:38:15.226 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 7c374098-77c9-11ef-baf5-efdc52797490 -- ceph tell osd.6 flush_pg_stats 2024-09-21T03:38:15.227 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 7c374098-77c9-11ef-baf5-efdc52797490 -- ceph tell osd.7 flush_pg_stats 2024-09-21T03:38:15.227 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 7c374098-77c9-11ef-baf5-efdc52797490 -- ceph tell osd.8 flush_pg_stats 2024-09-21T03:38:15.228 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 7c374098-77c9-11ef-baf5-efdc52797490 -- ceph tell osd.9 flush_pg_stats 2024-09-21T03:38:15.228 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 7c374098-77c9-11ef-baf5-efdc52797490 -- ceph tell osd.10 flush_pg_stats 2024-09-21T03:38:15.228 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 7c374098-77c9-11ef-baf5-efdc52797490 -- ceph tell osd.11 flush_pg_stats 2024-09-21T03:38:15.711 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:38:15 smithi114 ceph-mon[34140]: pgmap v323: 129 pgs: 129 active+clean; 644 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 1.3 KiB/s wr, 7 op/s 2024-09-21T03:38:15.711 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:38:15 smithi114 ceph-mon[34140]: from='client.? 172.21.15.89:0/3976286753' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T03:38:15.750 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:38:15 smithi110 ceph-mon[34000]: pgmap v323: 129 pgs: 129 active+clean; 644 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 1.3 KiB/s wr, 7 op/s 2024-09-21T03:38:15.750 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:38:15 smithi110 ceph-mon[34000]: from='client.? 172.21.15.89:0/3976286753' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T03:38:15.811 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:38:15 smithi089 ceph-mon[30928]: pgmap v323: 129 pgs: 129 active+clean; 644 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 1.3 KiB/s wr, 7 op/s 2024-09-21T03:38:15.811 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:38:15 smithi089 ceph-mon[30928]: from='client.? 172.21.15.89:0/3976286753' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T03:38:15.863 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/7c374098-77c9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:38:15.865 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/7c374098-77c9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:38:15.957 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/7c374098-77c9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:38:16.195 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/7c374098-77c9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:38:16.195 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/7c374098-77c9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:38:16.197 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/7c374098-77c9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:38:16.201 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/7c374098-77c9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:38:16.203 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/7c374098-77c9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:38:16.203 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/7c374098-77c9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:38:17.241 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:17.238+0000 7fe3308ca640 1 -- 172.21.15.89:0/3355848752 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe32c1049b0 msgr2=0x7fe32c104dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:17.241 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:17.238+0000 7fe3308ca640 1 --2- 172.21.15.89:0/3355848752 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe32c1049b0 0x7fe32c104dc0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fe31c009900 tx=0x7fe31c02a410 comp rx=0 tx=0).stop 2024-09-21T03:38:17.241 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:17.238+0000 7fe3308ca640 1 -- 172.21.15.89:0/3355848752 shutdown_connections 2024-09-21T03:38:17.241 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:17.238+0000 7fe3308ca640 1 --2- 172.21.15.89:0/3355848752 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe32c105300 0x7fe32c10d850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:17.241 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:17.238+0000 7fe3308ca640 1 --2- 172.21.15.89:0/3355848752 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe32c10de10 0x7fe32c110300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:17.241 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:17.238+0000 7fe3308ca640 1 --2- 172.21.15.89:0/3355848752 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe32c1049b0 0x7fe32c104dc0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:17.241 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:17.238+0000 7fe3308ca640 1 -- 172.21.15.89:0/3355848752 >> 172.21.15.89:0/3355848752 conn(0x7fe32c079730 msgr2=0x7fe32c079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:38:17.241 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:17.238+0000 7fe3308ca640 1 -- 172.21.15.89:0/3355848752 shutdown_connections 2024-09-21T03:38:17.241 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:17.238+0000 7fe3308ca640 1 -- 172.21.15.89:0/3355848752 wait complete. 2024-09-21T03:38:17.243 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:17.239+0000 7fe3308ca640 1 Processor -- start 2024-09-21T03:38:17.243 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:17.239+0000 7fe3308ca640 1 -- start start 2024-09-21T03:38:17.243 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:17.240+0000 7fe3308ca640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe32c1049b0 0x7fe32c19a470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:17.243 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:17.240+0000 7fe3308ca640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe32c105300 0x7fe32c19aa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:17.243 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:17.240+0000 7fe3308ca640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe32c10de10 0x7fe32c19fcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:17.243 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:17.240+0000 7fe3308ca640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fe32c113940 con 0x7fe32c1049b0 2024-09-21T03:38:17.244 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:17.240+0000 7fe3308ca640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fe32c1137c0 con 0x7fe32c10de10 2024-09-21T03:38:17.244 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:17.240+0000 7fe3308ca640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fe32c113ac0 con 0x7fe32c105300 2024-09-21T03:38:17.244 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:17.240+0000 7fe32a59c640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe32c1049b0 0x7fe32c19a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:17.244 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:17.240+0000 7fe32a59c640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe32c1049b0 0x7fe32c19a470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:38910/0 (socket says 172.21.15.89:38910) 2024-09-21T03:38:17.244 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:17.240+0000 7fe32a59c640 1 -- 172.21.15.89:0/4255345174 learned_addr learned my addr 172.21.15.89:0/4255345174 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-21T03:38:17.244 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:17.240+0000 7fe32ad9d640 1 --2- 172.21.15.89:0/4255345174 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe32c10de10 0x7fe32c19fcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:17.244 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:17.240+0000 7fe321d9b640 1 --2- 172.21.15.89:0/4255345174 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe32c105300 0x7fe32c19aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:17.244 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:17.240+0000 7fe32a59c640 1 -- 172.21.15.89:0/4255345174 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe32c105300 msgr2=0x7fe32c19aa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:17.244 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:17.240+0000 7fe32a59c640 1 --2- 172.21.15.89:0/4255345174 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe32c105300 0x7fe32c19aa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:17.244 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:17.240+0000 7fe32a59c640 1 -- 172.21.15.89:0/4255345174 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe32c10de10 msgr2=0x7fe32c19fcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:17.244 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:17.240+0000 7fe32a59c640 1 --2- 172.21.15.89:0/4255345174 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe32c10de10 0x7fe32c19fcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:17.244 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:17.240+0000 7fe32a59c640 1 -- 172.21.15.89:0/4255345174 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe32c1a03e0 con 0x7fe32c1049b0 2024-09-21T03:38:17.244 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:17.240+0000 7fe321d9b640 1 --2- 172.21.15.89:0/4255345174 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe32c105300 0x7fe32c19aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:38:17.244 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:17.240+0000 7fe32ad9d640 1 --2- 172.21.15.89:0/4255345174 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe32c10de10 0x7fe32c19fcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:38:17.244 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:17.240+0000 7fe32a59c640 1 --2- 172.21.15.89:0/4255345174 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe32c1049b0 0x7fe32c19a470 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fe31c005940 tx=0x7fe31c004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:38:17.244 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:17.241+0000 7fe323fff640 1 -- 172.21.15.89:0/4255345174 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe31c03f030 con 0x7fe32c1049b0 2024-09-21T03:38:17.245 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:17.241+0000 7fe323fff640 1 -- 172.21.15.89:0/4255345174 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe31c004440 con 0x7fe32c1049b0 2024-09-21T03:38:17.245 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:17.241+0000 7fe3308ca640 1 -- 172.21.15.89:0/4255345174 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe32c1a0670 con 0x7fe32c1049b0 2024-09-21T03:38:17.245 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:17.241+0000 7fe323fff640 1 -- 172.21.15.89:0/4255345174 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe31c035070 con 0x7fe32c1049b0 2024-09-21T03:38:17.245 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:17.241+0000 7fe3308ca640 1 -- 172.21.15.89:0/4255345174 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe32c1025d0 con 0x7fe32c1049b0 2024-09-21T03:38:17.245 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:17.242+0000 7fe3308ca640 1 -- 172.21.15.89:0/4255345174 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fe32c063b10 con 0x7fe32c1049b0 2024-09-21T03:38:17.245 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:17.243+0000 7fe323fff640 1 -- 172.21.15.89:0/4255345174 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 18) ==== 154563+0+0 (secure 0 0 0) 0x7fe31c042030 con 0x7fe32c1049b0 2024-09-21T03:38:17.246 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:17.244+0000 7fe323fff640 1 --2- 172.21.15.89:0/4255345174 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7fe3180b65a0 0x7fe3180b8ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:17.246 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:17.244+0000 7fe321d9b640 1 --2- 172.21.15.89:0/4255345174 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7fe3180b65a0 0x7fe3180b8ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:17.246 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:17.244+0000 7fe323fff640 1 -- 172.21.15.89:0/4255345174 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 9092+0+0 (secure 0 0 0) 0x7fe31c1081d0 con 0x7fe32c1049b0 2024-09-21T03:38:17.246 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:17.244+0000 7fe323fff640 1 --2- 172.21.15.89:0/4255345174 >> [v2:172.21.15.110:6824/2904016117,v1:172.21.15.110:6825/2904016117] conn(0x7fe3180c16b0 0x7fe3180c3ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:17.247 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:17.244+0000 7fe323fff640 1 -- 172.21.15.89:0/4255345174 --> [v2:172.21.15.110:6824/2904016117,v1:172.21.15.110:6825/2904016117] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fe3180c4270 con 0x7fe3180c16b0 2024-09-21T03:38:17.247 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:17.244+0000 7fe323fff640 1 -- 172.21.15.89:0/4255345174 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_get_version_reply(handle=1 version=82) ==== 24+0+0 (secure 0 0 0) 0x7fe31c108620 con 0x7fe32c1049b0 2024-09-21T03:38:17.247 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:17.245+0000 7fe321d9b640 1 --2- 172.21.15.89:0/4255345174 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7fe3180b65a0 0x7fe3180b8ab0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fe314005d00 tx=0x7fe314009550 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:38:17.247 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:17.245+0000 7fe32ad9d640 1 --2- 172.21.15.89:0/4255345174 >> [v2:172.21.15.110:6824/2904016117,v1:172.21.15.110:6825/2904016117] conn(0x7fe3180c16b0 0x7fe3180c3ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:17.247 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:17.245+0000 7fe32ad9d640 1 --2- 172.21.15.89:0/4255345174 >> [v2:172.21.15.110:6824/2904016117,v1:172.21.15.110:6825/2904016117] conn(0x7fe3180c16b0 0x7fe3180c3ba0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.10 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:38:17.248 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:17.246+0000 7fe323fff640 1 -- 172.21.15.89:0/4255345174 <== osd.10 v2:172.21.15.110:6824/2904016117 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fe3180c4270 con 0x7fe3180c16b0 2024-09-21T03:38:17.261 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:17.259+0000 7fe3308ca640 1 -- 172.21.15.89:0/4255345174 --> [v2:172.21.15.110:6824/2904016117,v1:172.21.15.110:6825/2904016117] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fe32c063f90 con 0x7fe3180c16b0 2024-09-21T03:38:17.262 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:17.260+0000 7fe323fff640 1 -- 172.21.15.89:0/4255345174 <== osd.10 v2:172.21.15.110:6824/2904016117 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fe32c063f90 con 0x7fe3180c16b0 2024-09-21T03:38:17.263 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:17.260+0000 7fe3308ca640 1 -- 172.21.15.89:0/4255345174 >> [v2:172.21.15.110:6824/2904016117,v1:172.21.15.110:6825/2904016117] conn(0x7fe3180c16b0 msgr2=0x7fe3180c3ba0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:17.263 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:17.260+0000 7fe3308ca640 1 --2- 172.21.15.89:0/4255345174 >> [v2:172.21.15.110:6824/2904016117,v1:172.21.15.110:6825/2904016117] conn(0x7fe3180c16b0 0x7fe3180c3ba0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:17.263 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:17.260+0000 7fe3308ca640 1 -- 172.21.15.89:0/4255345174 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7fe3180b65a0 msgr2=0x7fe3180b8ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:17.263 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:17.260+0000 7fe3308ca640 1 --2- 172.21.15.89:0/4255345174 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7fe3180b65a0 0x7fe3180b8ab0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fe314005d00 tx=0x7fe314009550 comp rx=0 tx=0).stop 2024-09-21T03:38:17.263 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:17.260+0000 7fe3308ca640 1 -- 172.21.15.89:0/4255345174 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe32c1049b0 msgr2=0x7fe32c19a470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:17.263 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:17.260+0000 7fe3308ca640 1 --2- 172.21.15.89:0/4255345174 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe32c1049b0 0x7fe32c19a470 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fe31c005940 tx=0x7fe31c004040 comp rx=0 tx=0).stop 2024-09-21T03:38:17.263 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:17.260+0000 7fe32ad9d640 1 -- 172.21.15.89:0/4255345174 reap_dead start 2024-09-21T03:38:17.263 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:17.260+0000 7fe3308ca640 1 -- 172.21.15.89:0/4255345174 shutdown_connections 2024-09-21T03:38:17.263 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:17.260+0000 7fe3308ca640 1 -- 172.21.15.89:0/4255345174 >> 172.21.15.89:0/4255345174 conn(0x7fe32c079730 msgr2=0x7fe32c100850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:38:17.263 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:17.261+0000 7fe3308ca640 1 -- 172.21.15.89:0/4255345174 shutdown_connections 2024-09-21T03:38:17.263 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:17.261+0000 7fe3308ca640 1 -- 172.21.15.89:0/4255345174 wait complete. 2024-09-21T03:38:17.346 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:17.148+0000 7f6564399640 1 -- 172.21.15.89:0/199447506 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f655c1083a0 msgr2=0x7f655c1087b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:17.346 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:17.148+0000 7f6564399640 1 --2- 172.21.15.89:0/199447506 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f655c1083a0 0x7f655c1087b0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f6558009900 tx=0x7f655802a410 comp rx=0 tx=0).stop 2024-09-21T03:38:17.346 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:17.148+0000 7f6564399640 1 -- 172.21.15.89:0/199447506 shutdown_connections 2024-09-21T03:38:17.346 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:17.148+0000 7f6564399640 1 --2- 172.21.15.89:0/199447506 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f655c108cf0 0x7f655c10f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:17.346 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:17.148+0000 7f6564399640 1 --2- 172.21.15.89:0/199447506 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f655c1083a0 0x7f655c1087b0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:17.346 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:17.148+0000 7f6564399640 1 --2- 172.21.15.89:0/199447506 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f655c102a60 0x7f655c102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:17.346 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:17.148+0000 7f6564399640 1 -- 172.21.15.89:0/199447506 >> 172.21.15.89:0/199447506 conn(0x7f655c079730 msgr2=0x7f655c079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:38:17.346 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:17.148+0000 7f6564399640 1 -- 172.21.15.89:0/199447506 shutdown_connections 2024-09-21T03:38:17.346 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:17.148+0000 7f6564399640 1 -- 172.21.15.89:0/199447506 wait complete. 2024-09-21T03:38:17.346 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:17.149+0000 7f6564399640 1 Processor -- start 2024-09-21T03:38:17.346 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:17.149+0000 7f6564399640 1 -- start start 2024-09-21T03:38:17.347 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:17.149+0000 7f6564399640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f655c102a60 0x7f655c19b1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:17.347 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:17.149+0000 7f6564399640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f655c1083a0 0x7f655c19b6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:17.347 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:17.149+0000 7f6564399640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f655c108cf0 0x7f655c19bc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:17.347 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:17.149+0000 7f6564399640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f655c112810 con 0x7f655c108cf0 2024-09-21T03:38:17.347 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:17.149+0000 7f6564399640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f655c112690 con 0x7f655c1083a0 2024-09-21T03:38:17.347 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:17.149+0000 7f6564399640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f655c112990 con 0x7f655c102a60 2024-09-21T03:38:17.347 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:17.149+0000 7f6562936640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f655c108cf0 0x7f655c19bc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:17.347 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:17.149+0000 7f6562936640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f655c108cf0 0x7f655c19bc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:38898/0 (socket says 172.21.15.89:38898) 2024-09-21T03:38:17.347 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:17.149+0000 7f6562936640 1 -- 172.21.15.89:0/3169841786 learned_addr learned my addr 172.21.15.89:0/3169841786 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-21T03:38:17.347 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:17.149+0000 7f6562135640 1 --2- 172.21.15.89:0/3169841786 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f655c102a60 0x7f655c19b1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:17.347 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:17.149+0000 7f6561934640 1 --2- 172.21.15.89:0/3169841786 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f655c1083a0 0x7f655c19b6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:17.347 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:17.149+0000 7f6562936640 1 -- 172.21.15.89:0/3169841786 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f655c102a60 msgr2=0x7f655c19b1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:17.347 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:17.149+0000 7f6562936640 1 --2- 172.21.15.89:0/3169841786 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f655c102a60 0x7f655c19b1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:17.347 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:17.149+0000 7f6562936640 1 -- 172.21.15.89:0/3169841786 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f655c1083a0 msgr2=0x7f655c19b6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:17.348 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:17.149+0000 7f6562936640 1 --2- 172.21.15.89:0/3169841786 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f655c1083a0 0x7f655c19b6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:17.348 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:17.149+0000 7f6562936640 1 -- 172.21.15.89:0/3169841786 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f655c1a64b0 con 0x7f655c108cf0 2024-09-21T03:38:17.348 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:17.149+0000 7f6561934640 1 --2- 172.21.15.89:0/3169841786 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f655c1083a0 0x7f655c19b6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:38:17.348 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:17.150+0000 7f6562936640 1 --2- 172.21.15.89:0/3169841786 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f655c108cf0 0x7f655c19bc20 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f655400cad0 tx=0x7f655400cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:38:17.348 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:17.150+0000 7f65537fe640 1 -- 172.21.15.89:0/3169841786 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f655400ae20 con 0x7f655c108cf0 2024-09-21T03:38:17.348 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:17.150+0000 7f6564399640 1 -- 172.21.15.89:0/3169841786 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f655c1a67a0 con 0x7f655c108cf0 2024-09-21T03:38:17.348 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:17.150+0000 7f65537fe640 1 -- 172.21.15.89:0/3169841786 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6554005020 con 0x7f655c108cf0 2024-09-21T03:38:17.348 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:17.150+0000 7f65537fe640 1 -- 172.21.15.89:0/3169841786 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f655400f6b0 con 0x7f655c108cf0 2024-09-21T03:38:17.348 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:17.150+0000 7f6564399640 1 -- 172.21.15.89:0/3169841786 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f655c1a6cb0 con 0x7f655c108cf0 2024-09-21T03:38:17.348 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:17.151+0000 7f6564399640 1 -- 172.21.15.89:0/3169841786 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f655c063b50 con 0x7f655c108cf0 2024-09-21T03:38:17.348 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:17.153+0000 7f65537fe640 1 -- 172.21.15.89:0/3169841786 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 18) ==== 154563+0+0 (secure 0 0 0) 0x7f65540051d0 con 0x7f655c108cf0 2024-09-21T03:38:17.348 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:17.153+0000 7f65537fe640 1 --2- 172.21.15.89:0/3169841786 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f65480b65a0 0x7f65480b8ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:17.348 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:17.154+0000 7f6562135640 1 --2- 172.21.15.89:0/3169841786 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f65480b65a0 0x7f65480b8ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:17.348 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:17.154+0000 7f65537fe640 1 -- 172.21.15.89:0/3169841786 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 9092+0+0 (secure 0 0 0) 0x7f65540e7da0 con 0x7f655c108cf0 2024-09-21T03:38:17.348 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:17.154+0000 7f65537fe640 1 --2- 172.21.15.89:0/3169841786 >> [v2:172.21.15.110:6800/3235852628,v1:172.21.15.110:6801/3235852628] conn(0x7f65480c16b0 0x7f65480c3ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:17.349 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:17.154+0000 7f65537fe640 1 -- 172.21.15.89:0/3169841786 --> [v2:172.21.15.110:6800/3235852628,v1:172.21.15.110:6801/3235852628] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f6554012ec0 con 0x7f65480c16b0 2024-09-21T03:38:17.349 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:17.154+0000 7f65537fe640 1 -- 172.21.15.89:0/3169841786 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_get_version_reply(handle=1 version=82) ==== 24+0+0 (secure 0 0 0) 0x7f65540e81f0 con 0x7f655c108cf0 2024-09-21T03:38:17.349 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:17.154+0000 7f6562135640 1 --2- 172.21.15.89:0/3169841786 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f65480b65a0 0x7f65480b8ab0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f655c06ba20 tx=0x7f654c00b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:38:17.349 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:17.154+0000 7f6561934640 1 --2- 172.21.15.89:0/3169841786 >> [v2:172.21.15.110:6800/3235852628,v1:172.21.15.110:6801/3235852628] conn(0x7f65480c16b0 0x7f65480c3ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:17.349 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:17.155+0000 7f6561934640 1 --2- 172.21.15.89:0/3169841786 >> [v2:172.21.15.110:6800/3235852628,v1:172.21.15.110:6801/3235852628] conn(0x7f65480c16b0 0x7f65480c3ba0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:38:17.349 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:17.155+0000 7f65537fe640 1 -- 172.21.15.89:0/3169841786 <== osd.1 v2:172.21.15.110:6800/3235852628 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f6554012ec0 con 0x7f65480c16b0 2024-09-21T03:38:17.349 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:17.183+0000 7f6564399640 1 -- 172.21.15.89:0/3169841786 --> [v2:172.21.15.110:6800/3235852628,v1:172.21.15.110:6801/3235852628] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f655c063fd0 con 0x7f65480c16b0 2024-09-21T03:38:17.349 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:17.184+0000 7f65537fe640 1 -- 172.21.15.89:0/3169841786 <== osd.1 v2:172.21.15.110:6800/3235852628 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f655c063fd0 con 0x7f65480c16b0 2024-09-21T03:38:17.349 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:17.184+0000 7f6564399640 1 -- 172.21.15.89:0/3169841786 >> [v2:172.21.15.110:6800/3235852628,v1:172.21.15.110:6801/3235852628] conn(0x7f65480c16b0 msgr2=0x7f65480c3ba0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:17.349 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:17.184+0000 7f6564399640 1 --2- 172.21.15.89:0/3169841786 >> [v2:172.21.15.110:6800/3235852628,v1:172.21.15.110:6801/3235852628] conn(0x7f65480c16b0 0x7f65480c3ba0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:17.349 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:17.184+0000 7f6564399640 1 -- 172.21.15.89:0/3169841786 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f65480b65a0 msgr2=0x7f65480b8ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:17.350 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:17.184+0000 7f6564399640 1 --2- 172.21.15.89:0/3169841786 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f65480b65a0 0x7f65480b8ab0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f655c06ba20 tx=0x7f654c00b390 comp rx=0 tx=0).stop 2024-09-21T03:38:17.350 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:17.184+0000 7f6564399640 1 -- 172.21.15.89:0/3169841786 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f655c108cf0 msgr2=0x7f655c19bc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:17.350 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:17.184+0000 7f6564399640 1 --2- 172.21.15.89:0/3169841786 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f655c108cf0 0x7f655c19bc20 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f655400cad0 tx=0x7f655400cf90 comp rx=0 tx=0).stop 2024-09-21T03:38:17.350 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:17.184+0000 7f6562936640 1 -- 172.21.15.89:0/3169841786 reap_dead start 2024-09-21T03:38:17.350 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:17.184+0000 7f6564399640 1 -- 172.21.15.89:0/3169841786 shutdown_connections 2024-09-21T03:38:17.350 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:17.184+0000 7f6564399640 1 -- 172.21.15.89:0/3169841786 >> 172.21.15.89:0/3169841786 conn(0x7f655c079730 msgr2=0x7f655c104510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:38:17.350 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:17.184+0000 7f6564399640 1 -- 172.21.15.89:0/3169841786 shutdown_connections 2024-09-21T03:38:17.350 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:17.184+0000 7f6564399640 1 -- 172.21.15.89:0/3169841786 wait complete. 2024-09-21T03:38:17.561 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:38:17 smithi089 ceph-mon[30928]: pgmap v324: 129 pgs: 129 active+clean; 646 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 1.4 KiB/s wr, 7 op/s 2024-09-21T03:38:17.711 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:38:17 smithi114 ceph-mon[34140]: pgmap v324: 129 pgs: 129 active+clean; 646 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 1.4 KiB/s wr, 7 op/s 2024-09-21T03:38:17.750 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:38:17 smithi110 ceph-mon[34000]: pgmap v324: 129 pgs: 129 active+clean; 646 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 1.4 KiB/s wr, 7 op/s 2024-09-21T03:38:17.849 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:17.841+0000 7f82bc2ac640 1 -- 172.21.15.89:0/4081368545 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f82b4109e20 msgr2=0x7f82b4110650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:17.850 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:17.841+0000 7f82bc2ac640 1 --2- 172.21.15.89:0/4081368545 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f82b4109e20 0x7f82b4110650 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f82ac009ca0 tx=0x7f82ac02a8c0 comp rx=0 tx=0).stop 2024-09-21T03:38:17.850 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:17.841+0000 7f82bc2ac640 1 -- 172.21.15.89:0/4081368545 shutdown_connections 2024-09-21T03:38:17.850 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:17.841+0000 7f82bc2ac640 1 --2- 172.21.15.89:0/4081368545 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f82b41094d0 0x7f82b41098e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:17.850 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:17.841+0000 7f82bc2ac640 1 --2- 172.21.15.89:0/4081368545 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f82b4109e20 0x7f82b4110650 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:17.850 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:17.841+0000 7f82bc2ac640 1 --2- 172.21.15.89:0/4081368545 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f82b4103b90 0x7f82b4103fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:17.850 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:17.841+0000 7f82bc2ac640 1 -- 172.21.15.89:0/4081368545 >> 172.21.15.89:0/4081368545 conn(0x7f82b40ff600 msgr2=0x7f82b4101a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:38:17.850 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:17.841+0000 7f82bc2ac640 1 -- 172.21.15.89:0/4081368545 shutdown_connections 2024-09-21T03:38:17.850 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:17.841+0000 7f82bc2ac640 1 -- 172.21.15.89:0/4081368545 wait complete. 2024-09-21T03:38:17.850 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:17.842+0000 7f82bc2ac640 1 Processor -- start 2024-09-21T03:38:17.850 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:17.842+0000 7f82bc2ac640 1 -- start start 2024-09-21T03:38:17.850 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:17.842+0000 7f82bc2ac640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f82b4103b90 0x7f82b419a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:17.851 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:17.842+0000 7f82bc2ac640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f82b41094d0 0x7f82b419a990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:17.851 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:17.842+0000 7f82bc2ac640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f82b4109e20 0x7f82b419fc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:17.851 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:17.842+0000 7f82bc2ac640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f82b4113a00 con 0x7f82b4103b90 2024-09-21T03:38:17.851 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:17.842+0000 7f82bc2ac640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f82b4113880 con 0x7f82b41094d0 2024-09-21T03:38:17.851 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:17.842+0000 7f82bc2ac640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f82b4113b80 con 0x7f82b4109e20 2024-09-21T03:38:17.851 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:17.842+0000 7f82ba048640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f82b4103b90 0x7f82b419a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:17.851 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:17.842+0000 7f82ba048640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f82b4103b90 0x7f82b419a3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:38934/0 (socket says 172.21.15.89:38934) 2024-09-21T03:38:17.851 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:17.842+0000 7f82ba048640 1 -- 172.21.15.89:0/755299676 learned_addr learned my addr 172.21.15.89:0/755299676 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-21T03:38:17.851 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:17.842+0000 7f82ba849640 1 --2- 172.21.15.89:0/755299676 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f82b4109e20 0x7f82b419fc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:17.851 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:17.842+0000 7f82b9847640 1 --2- 172.21.15.89:0/755299676 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f82b41094d0 0x7f82b419a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:17.851 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:17.843+0000 7f82ba048640 1 -- 172.21.15.89:0/755299676 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f82b4109e20 msgr2=0x7f82b419fc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:17.851 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:17.843+0000 7f82ba048640 1 --2- 172.21.15.89:0/755299676 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f82b4109e20 0x7f82b419fc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:17.851 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:17.843+0000 7f82ba048640 1 -- 172.21.15.89:0/755299676 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f82b41094d0 msgr2=0x7f82b419a990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:17.852 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:17.843+0000 7f82ba048640 1 --2- 172.21.15.89:0/755299676 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f82b41094d0 0x7f82b419a990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:17.852 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:17.843+0000 7f82ba048640 1 -- 172.21.15.89:0/755299676 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82b41a0360 con 0x7f82b4103b90 2024-09-21T03:38:17.852 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:17.843+0000 7f82ba048640 1 --2- 172.21.15.89:0/755299676 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f82b4103b90 0x7f82b419a3f0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f82b000e9f0 tx=0x7f82b000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:38:17.852 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:17.843+0000 7f82ab7fe640 1 -- 172.21.15.89:0/755299676 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82b000cd80 con 0x7f82b4103b90 2024-09-21T03:38:17.852 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:17.843+0000 7f82ab7fe640 1 -- 172.21.15.89:0/755299676 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f82b00054b0 con 0x7f82b4103b90 2024-09-21T03:38:17.852 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:17.843+0000 7f82bc2ac640 1 -- 172.21.15.89:0/755299676 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f82b41a0650 con 0x7f82b4103b90 2024-09-21T03:38:17.852 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:17.843+0000 7f82ab7fe640 1 -- 172.21.15.89:0/755299676 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82b0010430 con 0x7f82b4103b90 2024-09-21T03:38:17.852 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:17.843+0000 7f82bc2ac640 1 -- 172.21.15.89:0/755299676 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82b41a6db0 con 0x7f82b4103b90 2024-09-21T03:38:17.852 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:17.844+0000 7f82bc2ac640 1 -- 172.21.15.89:0/755299676 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f82b4063b10 con 0x7f82b4103b90 2024-09-21T03:38:17.852 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:17.846+0000 7f82ab7fe640 1 -- 172.21.15.89:0/755299676 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 18) ==== 154563+0+0 (secure 0 0 0) 0x7f82b00046c0 con 0x7f82b4103b90 2024-09-21T03:38:17.852 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:17.846+0000 7f82ab7fe640 1 --2- 172.21.15.89:0/755299676 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f82a00b65a0 0x7f82a00b8ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:17.852 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:17.846+0000 7f82b9847640 1 --2- 172.21.15.89:0/755299676 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f82a00b65a0 0x7f82a00b8ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:17.852 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:17.846+0000 7f82ab7fe640 1 -- 172.21.15.89:0/755299676 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 9092+0+0 (secure 0 0 0) 0x7f82b00e80f0 con 0x7f82b4103b90 2024-09-21T03:38:17.852 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:17.847+0000 7f82ab7fe640 1 --2- 172.21.15.89:0/755299676 >> [v2:172.21.15.89:6818/1522586255,v1:172.21.15.89:6819/1522586255] conn(0x7f82a00c16b0 0x7f82a00c3ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:17.853 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:17.847+0000 7f82ab7fe640 1 -- 172.21.15.89:0/755299676 --> [v2:172.21.15.89:6818/1522586255,v1:172.21.15.89:6819/1522586255] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f82b0011e50 con 0x7f82a00c16b0 2024-09-21T03:38:17.853 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:17.847+0000 7f82ab7fe640 1 -- 172.21.15.89:0/755299676 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_get_version_reply(handle=1 version=82) ==== 24+0+0 (secure 0 0 0) 0x7f82b00e8540 con 0x7f82b4103b90 2024-09-21T03:38:17.853 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:17.847+0000 7f82ba849640 1 --2- 172.21.15.89:0/755299676 >> [v2:172.21.15.89:6818/1522586255,v1:172.21.15.89:6819/1522586255] conn(0x7f82a00c16b0 0x7f82a00c3ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:17.853 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:17.847+0000 7f82b9847640 1 --2- 172.21.15.89:0/755299676 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f82a00b65a0 0x7f82a00b8ab0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f82a4005d70 tx=0x7f82a4005d00 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:38:17.853 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:17.847+0000 7f82ba849640 1 --2- 172.21.15.89:0/755299676 >> [v2:172.21.15.89:6818/1522586255,v1:172.21.15.89:6819/1522586255] conn(0x7f82a00c16b0 0x7f82a00c3ba0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:38:17.853 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:17.848+0000 7f82ab7fe640 1 -- 172.21.15.89:0/755299676 <== osd.6 v2:172.21.15.89:6818/1522586255 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f82b0011e50 con 0x7f82a00c16b0 2024-09-21T03:38:17.865 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:17.863+0000 7f82bc2ac640 1 -- 172.21.15.89:0/755299676 --> [v2:172.21.15.89:6818/1522586255,v1:172.21.15.89:6819/1522586255] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f82b4063f90 con 0x7f82a00c16b0 2024-09-21T03:38:17.865 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:17.863+0000 7f82ab7fe640 1 -- 172.21.15.89:0/755299676 <== osd.6 v2:172.21.15.89:6818/1522586255 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f82b4063f90 con 0x7f82a00c16b0 2024-09-21T03:38:17.866 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:17.863+0000 7f82bc2ac640 1 -- 172.21.15.89:0/755299676 >> [v2:172.21.15.89:6818/1522586255,v1:172.21.15.89:6819/1522586255] conn(0x7f82a00c16b0 msgr2=0x7f82a00c3ba0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:17.866 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:17.863+0000 7f82bc2ac640 1 --2- 172.21.15.89:0/755299676 >> [v2:172.21.15.89:6818/1522586255,v1:172.21.15.89:6819/1522586255] conn(0x7f82a00c16b0 0x7f82a00c3ba0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:17.866 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:17.864+0000 7f82bc2ac640 1 -- 172.21.15.89:0/755299676 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f82a00b65a0 msgr2=0x7f82a00b8ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:17.866 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:17.864+0000 7f82bc2ac640 1 --2- 172.21.15.89:0/755299676 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f82a00b65a0 0x7f82a00b8ab0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f82a4005d70 tx=0x7f82a4005d00 comp rx=0 tx=0).stop 2024-09-21T03:38:17.866 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:17.864+0000 7f82bc2ac640 1 -- 172.21.15.89:0/755299676 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f82b4103b90 msgr2=0x7f82b419a3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:17.867 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:17.864+0000 7f82bc2ac640 1 --2- 172.21.15.89:0/755299676 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f82b4103b90 0x7f82b419a3f0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f82b000e9f0 tx=0x7f82b000edf0 comp rx=0 tx=0).stop 2024-09-21T03:38:17.867 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:17.864+0000 7f82ba849640 1 -- 172.21.15.89:0/755299676 reap_dead start 2024-09-21T03:38:17.867 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:17.864+0000 7f82bc2ac640 1 -- 172.21.15.89:0/755299676 shutdown_connections 2024-09-21T03:38:17.867 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:17.864+0000 7f82bc2ac640 1 -- 172.21.15.89:0/755299676 >> 172.21.15.89:0/755299676 conn(0x7f82b40ff600 msgr2=0x7f82b4106a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:38:17.867 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:17.864+0000 7f82bc2ac640 1 -- 172.21.15.89:0/755299676 shutdown_connections 2024-09-21T03:38:17.867 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:17.864+0000 7f82bc2ac640 1 -- 172.21.15.89:0/755299676 wait complete. 2024-09-21T03:38:18.431 INFO:teuthology.orchestra.run.smithi089.stdout:55834574923 2024-09-21T03:38:18.431 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 7c374098-77c9-11ef-baf5-efdc52797490 -- ceph osd last-stat-seq osd.1 2024-09-21T03:38:18.614 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:18.612+0000 7fbb3c35d640 1 -- 172.21.15.89:0/1318037917 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbb34108d10 msgr2=0x7fbb3410f540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:18.615 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:18.612+0000 7fbb3c35d640 1 --2- 172.21.15.89:0/1318037917 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbb34108d10 0x7fbb3410f540 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fbb2c009ca0 tx=0x7fbb2c02a8c0 comp rx=0 tx=0).stop 2024-09-21T03:38:18.615 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:18.612+0000 7fbb3c35d640 1 -- 172.21.15.89:0/1318037917 shutdown_connections 2024-09-21T03:38:18.615 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:18.612+0000 7fbb3c35d640 1 --2- 172.21.15.89:0/1318037917 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbb341083c0 0x7fbb341087d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:18.615 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:18.612+0000 7fbb3c35d640 1 --2- 172.21.15.89:0/1318037917 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbb34108d10 0x7fbb3410f540 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:18.615 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:18.612+0000 7fbb3c35d640 1 --2- 172.21.15.89:0/1318037917 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbb34102a80 0x7fbb34102e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:18.615 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:18.612+0000 7fbb3c35d640 1 -- 172.21.15.89:0/1318037917 >> 172.21.15.89:0/1318037917 conn(0x7fbb34079740 msgr2=0x7fbb34079b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:38:18.615 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:18.612+0000 7fbb3c35d640 1 -- 172.21.15.89:0/1318037917 shutdown_connections 2024-09-21T03:38:18.615 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:18.612+0000 7fbb3c35d640 1 -- 172.21.15.89:0/1318037917 wait complete. 2024-09-21T03:38:18.617 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:18.613+0000 7fbb3c35d640 1 Processor -- start 2024-09-21T03:38:18.617 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:18.613+0000 7fbb3c35d640 1 -- start start 2024-09-21T03:38:18.617 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:18.613+0000 7fbb3c35d640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbb34102a80 0x7fbb340ff690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:18.617 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:18.613+0000 7fbb3c35d640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbb341083c0 0x7fbb340ffbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:18.617 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:18.613+0000 7fbb3c35d640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbb34108d10 0x7fbb34100110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:18.618 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:18.613+0000 7fbb3c35d640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fbb34112880 con 0x7fbb34102a80 2024-09-21T03:38:18.618 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:18.613+0000 7fbb3c35d640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fbb34112700 con 0x7fbb34108d10 2024-09-21T03:38:18.618 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:18.613+0000 7fbb3c35d640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fbb34112a00 con 0x7fbb341083c0 2024-09-21T03:38:18.618 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:18.613+0000 7fbb3a0f9640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbb34102a80 0x7fbb340ff690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:18.618 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:18.613+0000 7fbb3a0f9640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbb34102a80 0x7fbb340ff690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:47588/0 (socket says 172.21.15.89:47588) 2024-09-21T03:38:18.618 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:18.613+0000 7fbb3a0f9640 1 -- 172.21.15.89:0/2923972834 learned_addr learned my addr 172.21.15.89:0/2923972834 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-21T03:38:18.618 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:18.613+0000 7fbb398f8640 1 --2- 172.21.15.89:0/2923972834 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbb341083c0 0x7fbb340ffbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:18.618 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:18.613+0000 7fbb3a8fa640 1 --2- 172.21.15.89:0/2923972834 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbb34108d10 0x7fbb34100110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:18.618 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:18.614+0000 7fbb3a0f9640 1 -- 172.21.15.89:0/2923972834 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbb341083c0 msgr2=0x7fbb340ffbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:18.618 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:18.614+0000 7fbb3a0f9640 1 --2- 172.21.15.89:0/2923972834 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbb341083c0 0x7fbb340ffbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:18.618 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:18.614+0000 7fbb3a0f9640 1 -- 172.21.15.89:0/2923972834 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbb34108d10 msgr2=0x7fbb34100110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:18.618 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:18.614+0000 7fbb3a0f9640 1 --2- 172.21.15.89:0/2923972834 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbb34108d10 0x7fbb34100110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:18.618 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:18.614+0000 7fbb3a0f9640 1 -- 172.21.15.89:0/2923972834 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb341a85f0 con 0x7fbb34102a80 2024-09-21T03:38:18.618 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:18.614+0000 7fbb3a0f9640 1 --2- 172.21.15.89:0/2923972834 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbb34102a80 0x7fbb340ff690 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fbb3000c920 tx=0x7fbb3000cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:38:18.619 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:18.614+0000 7fbb2b7fe640 1 -- 172.21.15.89:0/2923972834 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb30007c70 con 0x7fbb34102a80 2024-09-21T03:38:18.619 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:18.614+0000 7fbb3c35d640 1 -- 172.21.15.89:0/2923972834 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbb341a88e0 con 0x7fbb34102a80 2024-09-21T03:38:18.619 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:18.614+0000 7fbb2b7fe640 1 -- 172.21.15.89:0/2923972834 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fbb30007e10 con 0x7fbb34102a80 2024-09-21T03:38:18.619 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:18.614+0000 7fbb3c35d640 1 -- 172.21.15.89:0/2923972834 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbb341a8e70 con 0x7fbb34102a80 2024-09-21T03:38:18.619 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:18.614+0000 7fbb2b7fe640 1 -- 172.21.15.89:0/2923972834 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb300186f0 con 0x7fbb34102a80 2024-09-21T03:38:18.619 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:18.615+0000 7fbb3c35d640 1 -- 172.21.15.89:0/2923972834 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fbb341044a0 con 0x7fbb34102a80 2024-09-21T03:38:18.619 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:18.617+0000 7fbb2b7fe640 1 -- 172.21.15.89:0/2923972834 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 18) ==== 154563+0+0 (secure 0 0 0) 0x7fbb3000f450 con 0x7fbb34102a80 2024-09-21T03:38:18.620 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:18.618+0000 7fbb2b7fe640 1 --2- 172.21.15.89:0/2923972834 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7fbb200b65a0 0x7fbb200b8ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:18.620 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:18.618+0000 7fbb398f8640 1 --2- 172.21.15.89:0/2923972834 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7fbb200b65a0 0x7fbb200b8ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:18.620 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:18.618+0000 7fbb2b7fe640 1 -- 172.21.15.89:0/2923972834 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 9092+0+0 (secure 0 0 0) 0x7fbb300e7f40 con 0x7fbb34102a80 2024-09-21T03:38:18.621 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:18.618+0000 7fbb2b7fe640 1 --2- 172.21.15.89:0/2923972834 >> [v2:172.21.15.89:6802/990539042,v1:172.21.15.89:6803/990539042] conn(0x7fbb200c16b0 0x7fbb200c3ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:18.621 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:18.618+0000 7fbb2b7fe640 1 -- 172.21.15.89:0/2923972834 --> [v2:172.21.15.89:6802/990539042,v1:172.21.15.89:6803/990539042] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fbb3001ee50 con 0x7fbb200c16b0 2024-09-21T03:38:18.621 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:18.618+0000 7fbb2b7fe640 1 -- 172.21.15.89:0/2923972834 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_get_version_reply(handle=1 version=82) ==== 24+0+0 (secure 0 0 0) 0x7fbb300e8390 con 0x7fbb34102a80 2024-09-21T03:38:18.621 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:18.619+0000 7fbb398f8640 1 --2- 172.21.15.89:0/2923972834 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7fbb200b65a0 0x7fbb200b8ab0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fbb34105f40 tx=0x7fbb2400b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:38:18.621 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:18.619+0000 7fbb3a8fa640 1 --2- 172.21.15.89:0/2923972834 >> [v2:172.21.15.89:6802/990539042,v1:172.21.15.89:6803/990539042] conn(0x7fbb200c16b0 0x7fbb200c3ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:18.621 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:18.619+0000 7fbb3a8fa640 1 --2- 172.21.15.89:0/2923972834 >> [v2:172.21.15.89:6802/990539042,v1:172.21.15.89:6803/990539042] conn(0x7fbb200c16b0 0x7fbb200c3ba0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:38:18.622 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:18.620+0000 7fbb2b7fe640 1 -- 172.21.15.89:0/2923972834 <== osd.0 v2:172.21.15.89:6802/990539042 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fbb3001ee50 con 0x7fbb200c16b0 2024-09-21T03:38:18.636 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:18.634+0000 7fbb3c35d640 1 -- 172.21.15.89:0/2923972834 --> [v2:172.21.15.89:6802/990539042,v1:172.21.15.89:6803/990539042] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fbb3406b390 con 0x7fbb200c16b0 2024-09-21T03:38:18.637 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:18.635+0000 7fbb2b7fe640 1 -- 172.21.15.89:0/2923972834 <== osd.0 v2:172.21.15.89:6802/990539042 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fbb3406b390 con 0x7fbb200c16b0 2024-09-21T03:38:18.637 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:18.635+0000 7fbb3c35d640 1 -- 172.21.15.89:0/2923972834 >> [v2:172.21.15.89:6802/990539042,v1:172.21.15.89:6803/990539042] conn(0x7fbb200c16b0 msgr2=0x7fbb200c3ba0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:18.638 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:18.635+0000 7fbb3c35d640 1 --2- 172.21.15.89:0/2923972834 >> [v2:172.21.15.89:6802/990539042,v1:172.21.15.89:6803/990539042] conn(0x7fbb200c16b0 0x7fbb200c3ba0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:18.638 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:18.635+0000 7fbb3c35d640 1 -- 172.21.15.89:0/2923972834 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7fbb200b65a0 msgr2=0x7fbb200b8ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:18.638 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:18.635+0000 7fbb3c35d640 1 --2- 172.21.15.89:0/2923972834 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7fbb200b65a0 0x7fbb200b8ab0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fbb34105f40 tx=0x7fbb2400b390 comp rx=0 tx=0).stop 2024-09-21T03:38:18.638 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:18.635+0000 7fbb3c35d640 1 -- 172.21.15.89:0/2923972834 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbb34102a80 msgr2=0x7fbb340ff690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:18.638 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:18.635+0000 7fbb3c35d640 1 --2- 172.21.15.89:0/2923972834 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbb34102a80 0x7fbb340ff690 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fbb3000c920 tx=0x7fbb3000cde0 comp rx=0 tx=0).stop 2024-09-21T03:38:18.638 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:18.635+0000 7fbb3a8fa640 1 -- 172.21.15.89:0/2923972834 reap_dead start 2024-09-21T03:38:18.638 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:18.635+0000 7fbb3c35d640 1 -- 172.21.15.89:0/2923972834 shutdown_connections 2024-09-21T03:38:18.638 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:18.635+0000 7fbb3c35d640 1 -- 172.21.15.89:0/2923972834 >> 172.21.15.89:0/2923972834 conn(0x7fbb34079740 msgr2=0x7fbb34101a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:38:18.638 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:18.635+0000 7fbb3c35d640 1 -- 172.21.15.89:0/2923972834 shutdown_connections 2024-09-21T03:38:18.638 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:18.635+0000 7fbb3c35d640 1 -- 172.21.15.89:0/2923972834 wait complete. 2024-09-21T03:38:19.439 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:38:19 smithi089 ceph-mon[30928]: pgmap v325: 129 pgs: 129 active+clean; 646 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 682 B/s wr, 6 op/s 2024-09-21T03:38:19.633 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:19.622+0000 7f46db40e640 1 -- 172.21.15.89:0/360493745 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f46d41049b0 msgr2=0x7f46d4104dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:19.633 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:19.622+0000 7f46db40e640 1 --2- 172.21.15.89:0/360493745 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f46d41049b0 0x7f46d4104dc0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f46c4009ca0 tx=0x7f46c402a8c0 comp rx=0 tx=0).stop 2024-09-21T03:38:19.634 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:19.622+0000 7f46db40e640 1 -- 172.21.15.89:0/360493745 shutdown_connections 2024-09-21T03:38:19.634 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:19.622+0000 7f46db40e640 1 --2- 172.21.15.89:0/360493745 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f46d4105300 0x7f46d410d850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:19.634 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:19.622+0000 7f46db40e640 1 --2- 172.21.15.89:0/360493745 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f46d410de10 0x7f46d4110300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:19.634 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:19.622+0000 7f46db40e640 1 --2- 172.21.15.89:0/360493745 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f46d41049b0 0x7f46d4104dc0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:19.634 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:19.622+0000 7f46db40e640 1 -- 172.21.15.89:0/360493745 >> 172.21.15.89:0/360493745 conn(0x7f46d4079730 msgr2=0x7f46d4079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:38:19.634 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:19.622+0000 7f46db40e640 1 -- 172.21.15.89:0/360493745 shutdown_connections 2024-09-21T03:38:19.634 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:19.622+0000 7f46db40e640 1 -- 172.21.15.89:0/360493745 wait complete. 2024-09-21T03:38:19.634 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:19.623+0000 7f46db40e640 1 Processor -- start 2024-09-21T03:38:19.634 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:19.623+0000 7f46db40e640 1 -- start start 2024-09-21T03:38:19.634 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:19.623+0000 7f46db40e640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f46d41049b0 0x7f46d419a400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:19.634 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:19.623+0000 7f46db40e640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f46d4105300 0x7f46d419a9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:19.634 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:19.623+0000 7f46db40e640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f46d410de10 0x7f46d419fc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:19.635 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:19.623+0000 7f46db40e640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f46d41139f0 con 0x7f46d4105300 2024-09-21T03:38:19.635 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:19.623+0000 7f46db40e640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f46d4113870 con 0x7f46d41049b0 2024-09-21T03:38:19.635 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:19.623+0000 7f46db40e640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f46d4113b70 con 0x7f46d410de10 2024-09-21T03:38:19.635 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:19.623+0000 7f46d89a9640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f46d4105300 0x7f46d419a9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:19.635 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:19.623+0000 7f46d89a9640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f46d4105300 0x7f46d419a9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:47606/0 (socket says 172.21.15.89:47606) 2024-09-21T03:38:19.635 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:19.623+0000 7f46d89a9640 1 -- 172.21.15.89:0/3782888310 learned_addr learned my addr 172.21.15.89:0/3782888310 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-21T03:38:19.635 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:19.623+0000 7f46d91aa640 1 --2- 172.21.15.89:0/3782888310 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f46d41049b0 0x7f46d419a400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:19.635 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:19.623+0000 7f46d89a9640 1 -- 172.21.15.89:0/3782888310 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f46d410de10 msgr2=0x7f46d419fc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:19.635 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:19.623+0000 7f46d99ab640 1 --2- 172.21.15.89:0/3782888310 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f46d410de10 0x7f46d419fc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:19.635 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:19.623+0000 7f46d89a9640 1 --2- 172.21.15.89:0/3782888310 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f46d410de10 0x7f46d419fc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:19.635 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:19.623+0000 7f46d89a9640 1 -- 172.21.15.89:0/3782888310 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f46d41049b0 msgr2=0x7f46d419a400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:19.636 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:19.623+0000 7f46d89a9640 1 --2- 172.21.15.89:0/3782888310 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f46d41049b0 0x7f46d419a400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:19.636 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:19.623+0000 7f46d89a9640 1 -- 172.21.15.89:0/3782888310 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46d41a0370 con 0x7f46d4105300 2024-09-21T03:38:19.636 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:19.623+0000 7f46d91aa640 1 --2- 172.21.15.89:0/3782888310 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f46d41049b0 0x7f46d419a400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:38:19.636 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:19.624+0000 7f46d89a9640 1 --2- 172.21.15.89:0/3782888310 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f46d4105300 0x7f46d419a9a0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f46c800c970 tx=0x7f46c800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:38:19.636 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:19.624+0000 7f46d27fc640 1 -- 172.21.15.89:0/3782888310 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46c8007bb0 con 0x7f46d4105300 2024-09-21T03:38:19.636 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:19.624+0000 7f46db40e640 1 -- 172.21.15.89:0/3782888310 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46d41a0660 con 0x7f46d4105300 2024-09-21T03:38:19.636 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:19.624+0000 7f46d27fc640 1 -- 172.21.15.89:0/3782888310 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f46c8007d50 con 0x7f46d4105300 2024-09-21T03:38:19.636 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:19.624+0000 7f46d27fc640 1 -- 172.21.15.89:0/3782888310 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46c8004d40 con 0x7f46d4105300 2024-09-21T03:38:19.636 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:19.624+0000 7f46db40e640 1 -- 172.21.15.89:0/3782888310 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46d41a6dc0 con 0x7f46d4105300 2024-09-21T03:38:19.636 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:19.625+0000 7f46db40e640 1 -- 172.21.15.89:0/3782888310 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f46d4063b50 con 0x7f46d4105300 2024-09-21T03:38:19.636 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:19.626+0000 7f46d27fc640 1 -- 172.21.15.89:0/3782888310 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 18) ==== 154563+0+0 (secure 0 0 0) 0x7f46c8004fe0 con 0x7f46d4105300 2024-09-21T03:38:19.636 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:19.627+0000 7f46d27fc640 1 --2- 172.21.15.89:0/3782888310 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f46bc0b65a0 0x7f46bc0b8ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:19.636 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:19.627+0000 7f46d91aa640 1 --2- 172.21.15.89:0/3782888310 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f46bc0b65a0 0x7f46bc0b8ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:19.636 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:19.627+0000 7f46d27fc640 1 -- 172.21.15.89:0/3782888310 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 9092+0+0 (secure 0 0 0) 0x7f46c80e7a00 con 0x7f46d4105300 2024-09-21T03:38:19.637 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:19.627+0000 7f46d27fc640 1 --2- 172.21.15.89:0/3782888310 >> [v2:172.21.15.114:6824/3749854486,v1:172.21.15.114:6825/3749854486] conn(0x7f46bc0c16b0 0x7f46bc0c3ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:19.637 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:19.627+0000 7f46d27fc640 1 -- 172.21.15.89:0/3782888310 --> [v2:172.21.15.114:6824/3749854486,v1:172.21.15.114:6825/3749854486] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f46c8005610 con 0x7f46bc0c16b0 2024-09-21T03:38:19.637 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:19.627+0000 7f46d27fc640 1 -- 172.21.15.89:0/3782888310 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_get_version_reply(handle=1 version=82) ==== 24+0+0 (secure 0 0 0) 0x7f46c80e7e50 con 0x7f46d4105300 2024-09-21T03:38:19.637 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:19.628+0000 7f46d91aa640 1 --2- 172.21.15.89:0/3782888310 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f46bc0b65a0 0x7f46bc0b8ab0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f46c4000f80 tx=0x7f46c4031000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:38:19.637 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:19.628+0000 7f46d99ab640 1 --2- 172.21.15.89:0/3782888310 >> [v2:172.21.15.114:6824/3749854486,v1:172.21.15.114:6825/3749854486] conn(0x7f46bc0c16b0 0x7f46bc0c3ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:19.637 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:19.628+0000 7f46d99ab640 1 --2- 172.21.15.89:0/3782888310 >> [v2:172.21.15.114:6824/3749854486,v1:172.21.15.114:6825/3749854486] conn(0x7f46bc0c16b0 0x7f46bc0c3ba0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.11 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:38:19.637 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:19.629+0000 7f46d27fc640 1 -- 172.21.15.89:0/3782888310 <== osd.11 v2:172.21.15.114:6824/3749854486 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f46c8005610 con 0x7f46bc0c16b0 2024-09-21T03:38:19.644 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:19.643+0000 7f46db40e640 1 -- 172.21.15.89:0/3782888310 --> [v2:172.21.15.114:6824/3749854486,v1:172.21.15.114:6825/3749854486] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f46d4063f50 con 0x7f46bc0c16b0 2024-09-21T03:38:19.644 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:19.643+0000 7f46d27fc640 1 -- 172.21.15.89:0/3782888310 <== osd.11 v2:172.21.15.114:6824/3749854486 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f46d4063f50 con 0x7f46bc0c16b0 2024-09-21T03:38:19.645 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:19.644+0000 7f46db40e640 1 -- 172.21.15.89:0/3782888310 >> [v2:172.21.15.114:6824/3749854486,v1:172.21.15.114:6825/3749854486] conn(0x7f46bc0c16b0 msgr2=0x7f46bc0c3ba0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:19.645 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:19.644+0000 7f46db40e640 1 --2- 172.21.15.89:0/3782888310 >> [v2:172.21.15.114:6824/3749854486,v1:172.21.15.114:6825/3749854486] conn(0x7f46bc0c16b0 0x7f46bc0c3ba0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:19.645 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:19.644+0000 7f46db40e640 1 -- 172.21.15.89:0/3782888310 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f46bc0b65a0 msgr2=0x7f46bc0b8ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:19.645 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:19.644+0000 7f46db40e640 1 --2- 172.21.15.89:0/3782888310 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f46bc0b65a0 0x7f46bc0b8ab0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f46c4000f80 tx=0x7f46c4031000 comp rx=0 tx=0).stop 2024-09-21T03:38:19.646 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:19.644+0000 7f46db40e640 1 -- 172.21.15.89:0/3782888310 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f46d4105300 msgr2=0x7f46d419a9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:19.646 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:19.644+0000 7f46db40e640 1 --2- 172.21.15.89:0/3782888310 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f46d4105300 0x7f46d419a9a0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f46c800c970 tx=0x7f46c800ce30 comp rx=0 tx=0).stop 2024-09-21T03:38:19.646 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:19.644+0000 7f46d99ab640 1 -- 172.21.15.89:0/3782888310 reap_dead start 2024-09-21T03:38:19.646 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:19.644+0000 7f46db40e640 1 -- 172.21.15.89:0/3782888310 shutdown_connections 2024-09-21T03:38:19.646 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:19.644+0000 7f46db40e640 1 -- 172.21.15.89:0/3782888310 >> 172.21.15.89:0/3782888310 conn(0x7f46d4079730 msgr2=0x7f46d4100770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:38:19.646 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:19.644+0000 7f46db40e640 1 -- 172.21.15.89:0/3782888310 shutdown_connections 2024-09-21T03:38:19.646 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:19.644+0000 7f46db40e640 1 -- 172.21.15.89:0/3782888310 wait complete. 2024-09-21T03:38:19.711 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:38:19 smithi114 ceph-mon[34140]: pgmap v325: 129 pgs: 129 active+clean; 646 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 682 B/s wr, 6 op/s 2024-09-21T03:38:19.750 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:38:19 smithi110 ceph-mon[34000]: pgmap v325: 129 pgs: 129 active+clean; 646 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 682 B/s wr, 6 op/s 2024-09-21T03:38:19.827 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:19.823+0000 7f0b5aff6640 1 -- 172.21.15.89:0/204750166 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0b541083c0 msgr2=0x7f0b541087d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:19.827 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:19.823+0000 7f0b5aff6640 1 --2- 172.21.15.89:0/204750166 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0b541083c0 0x7f0b541087d0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f0b40009900 tx=0x7f0b4002a410 comp rx=0 tx=0).stop 2024-09-21T03:38:19.827 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:19.823+0000 7f0b5aff6640 1 -- 172.21.15.89:0/204750166 shutdown_connections 2024-09-21T03:38:19.827 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:19.823+0000 7f0b5aff6640 1 --2- 172.21.15.89:0/204750166 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0b54108d10 0x7f0b5410f540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:19.827 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:19.823+0000 7f0b5aff6640 1 --2- 172.21.15.89:0/204750166 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0b541083c0 0x7f0b541087d0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:19.827 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:19.823+0000 7f0b5aff6640 1 --2- 172.21.15.89:0/204750166 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0b54102a80 0x7f0b54102e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:19.827 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:19.823+0000 7f0b5aff6640 1 -- 172.21.15.89:0/204750166 >> 172.21.15.89:0/204750166 conn(0x7f0b54079740 msgr2=0x7f0b54079b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:38:19.827 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:19.824+0000 7f0b5aff6640 1 -- 172.21.15.89:0/204750166 shutdown_connections 2024-09-21T03:38:19.827 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:19.824+0000 7f0b5aff6640 1 -- 172.21.15.89:0/204750166 wait complete. 2024-09-21T03:38:19.827 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:19.824+0000 7f0b5aff6640 1 Processor -- start 2024-09-21T03:38:19.827 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:19.824+0000 7f0b5aff6640 1 -- start start 2024-09-21T03:38:19.828 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:19.824+0000 7f0b5aff6640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0b54102a80 0x7f0b540ff690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:19.828 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:19.824+0000 7f0b5aff6640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0b541083c0 0x7f0b540ffbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:19.828 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:19.825+0000 7f0b5aff6640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0b54108d10 0x7f0b54100110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:19.828 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:19.825+0000 7f0b5aff6640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f0b54112830 con 0x7f0b541083c0 2024-09-21T03:38:19.828 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:19.825+0000 7f0b5aff6640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f0b541126b0 con 0x7f0b54102a80 2024-09-21T03:38:19.828 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:19.825+0000 7f0b5aff6640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f0b541129b0 con 0x7f0b54108d10 2024-09-21T03:38:19.828 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:19.825+0000 7f0b53fff640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0b541083c0 0x7f0b540ffbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:19.828 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:19.825+0000 7f0b53fff640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0b541083c0 0x7f0b540ffbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:47628/0 (socket says 172.21.15.89:47628) 2024-09-21T03:38:19.828 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:19.825+0000 7f0b53fff640 1 -- 172.21.15.89:0/1365084226 learned_addr learned my addr 172.21.15.89:0/1365084226 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-21T03:38:19.828 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:19.825+0000 7f0b58d92640 1 --2- 172.21.15.89:0/1365084226 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0b54102a80 0x7f0b540ff690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:19.828 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:19.825+0000 7f0b59593640 1 --2- 172.21.15.89:0/1365084226 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0b54108d10 0x7f0b54100110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:19.828 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:19.825+0000 7f0b53fff640 1 -- 172.21.15.89:0/1365084226 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0b54108d10 msgr2=0x7f0b54100110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:19.828 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:19.825+0000 7f0b53fff640 1 --2- 172.21.15.89:0/1365084226 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0b54108d10 0x7f0b54100110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:19.828 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:19.825+0000 7f0b53fff640 1 -- 172.21.15.89:0/1365084226 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0b54102a80 msgr2=0x7f0b540ff690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:19.828 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:19.825+0000 7f0b53fff640 1 --2- 172.21.15.89:0/1365084226 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0b54102a80 0x7f0b540ff690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:19.829 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:19.825+0000 7f0b53fff640 1 -- 172.21.15.89:0/1365084226 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b541a86b0 con 0x7f0b541083c0 2024-09-21T03:38:19.829 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:19.825+0000 7f0b53fff640 1 --2- 172.21.15.89:0/1365084226 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0b541083c0 0x7f0b540ffbd0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f0b40005940 tx=0x7f0b40004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:38:19.829 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:19.825+0000 7f0b51ffb640 1 -- 172.21.15.89:0/1365084226 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b4003f030 con 0x7f0b541083c0 2024-09-21T03:38:19.829 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:19.825+0000 7f0b51ffb640 1 -- 172.21.15.89:0/1365084226 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0b40004440 con 0x7f0b541083c0 2024-09-21T03:38:19.829 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:19.825+0000 7f0b5aff6640 1 -- 172.21.15.89:0/1365084226 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0b541a8940 con 0x7f0b541083c0 2024-09-21T03:38:19.829 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:19.825+0000 7f0b51ffb640 1 -- 172.21.15.89:0/1365084226 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b40035070 con 0x7f0b541083c0 2024-09-21T03:38:19.829 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:19.826+0000 7f0b5aff6640 1 -- 172.21.15.89:0/1365084226 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0b541a8e70 con 0x7f0b541083c0 2024-09-21T03:38:19.830 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:19.827+0000 7f0b5aff6640 1 -- 172.21.15.89:0/1365084226 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f0b54063b50 con 0x7f0b541083c0 2024-09-21T03:38:19.830 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:19.828+0000 7f0b51ffb640 1 -- 172.21.15.89:0/1365084226 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 18) ==== 154563+0+0 (secure 0 0 0) 0x7f0b40042030 con 0x7f0b541083c0 2024-09-21T03:38:19.830 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:19.829+0000 7f0b51ffb640 1 --2- 172.21.15.89:0/1365084226 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f0b440b6620 0x7f0b440b8b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:19.830 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:19.829+0000 7f0b58d92640 1 --2- 172.21.15.89:0/1365084226 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f0b440b6620 0x7f0b440b8b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:19.830 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:19.829+0000 7f0b51ffb640 1 -- 172.21.15.89:0/1365084226 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 9092+0+0 (secure 0 0 0) 0x7f0b40108290 con 0x7f0b541083c0 2024-09-21T03:38:19.831 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:19.829+0000 7f0b51ffb640 1 --2- 172.21.15.89:0/1365084226 >> [v2:172.21.15.89:6826/1730253068,v1:172.21.15.89:6827/1730253068] conn(0x7f0b440c1730 0x7f0b440c3c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:19.831 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:19.829+0000 7f0b51ffb640 1 -- 172.21.15.89:0/1365084226 --> [v2:172.21.15.89:6826/1730253068,v1:172.21.15.89:6827/1730253068] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f0b440c42f0 con 0x7f0b440c1730 2024-09-21T03:38:19.831 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:19.829+0000 7f0b51ffb640 1 -- 172.21.15.89:0/1365084226 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_get_version_reply(handle=1 version=82) ==== 24+0+0 (secure 0 0 0) 0x7f0b401086e0 con 0x7f0b541083c0 2024-09-21T03:38:19.831 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:19.829+0000 7f0b59593640 1 --2- 172.21.15.89:0/1365084226 >> [v2:172.21.15.89:6826/1730253068,v1:172.21.15.89:6827/1730253068] conn(0x7f0b440c1730 0x7f0b440c3c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:19.831 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:19.829+0000 7f0b58d92640 1 --2- 172.21.15.89:0/1365084226 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f0b440b6620 0x7f0b440b8b30 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f0b541010e0 tx=0x7f0b48005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:38:19.831 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:19.830+0000 7f0b59593640 1 --2- 172.21.15.89:0/1365084226 >> [v2:172.21.15.89:6826/1730253068,v1:172.21.15.89:6827/1730253068] conn(0x7f0b440c1730 0x7f0b440c3c20 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.9 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:38:19.831 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:19.830+0000 7f0b51ffb640 1 -- 172.21.15.89:0/1365084226 <== osd.9 v2:172.21.15.89:6826/1730253068 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f0b440c42f0 con 0x7f0b440c1730 2024-09-21T03:38:19.845 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:19.844+0000 7f0b5aff6640 1 -- 172.21.15.89:0/1365084226 --> [v2:172.21.15.89:6826/1730253068,v1:172.21.15.89:6827/1730253068] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f0b54063f50 con 0x7f0b440c1730 2024-09-21T03:38:19.845 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:19.844+0000 7f0b51ffb640 1 -- 172.21.15.89:0/1365084226 <== osd.9 v2:172.21.15.89:6826/1730253068 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f0b54063f50 con 0x7f0b440c1730 2024-09-21T03:38:19.846 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:19.844+0000 7f0b5aff6640 1 -- 172.21.15.89:0/1365084226 >> [v2:172.21.15.89:6826/1730253068,v1:172.21.15.89:6827/1730253068] conn(0x7f0b440c1730 msgr2=0x7f0b440c3c20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:19.846 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:19.844+0000 7f0b5aff6640 1 --2- 172.21.15.89:0/1365084226 >> [v2:172.21.15.89:6826/1730253068,v1:172.21.15.89:6827/1730253068] conn(0x7f0b440c1730 0x7f0b440c3c20 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:19.846 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:19.844+0000 7f0b5aff6640 1 -- 172.21.15.89:0/1365084226 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f0b440b6620 msgr2=0x7f0b440b8b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:19.847 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:19.844+0000 7f0b5aff6640 1 --2- 172.21.15.89:0/1365084226 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f0b440b6620 0x7f0b440b8b30 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f0b541010e0 tx=0x7f0b48005f90 comp rx=0 tx=0).stop 2024-09-21T03:38:19.847 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:19.844+0000 7f0b5aff6640 1 -- 172.21.15.89:0/1365084226 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0b541083c0 msgr2=0x7f0b540ffbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:19.847 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:19.844+0000 7f0b5aff6640 1 --2- 172.21.15.89:0/1365084226 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0b541083c0 0x7f0b540ffbd0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f0b40005940 tx=0x7f0b40004040 comp rx=0 tx=0).stop 2024-09-21T03:38:19.847 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:19.844+0000 7f0b59593640 1 -- 172.21.15.89:0/1365084226 reap_dead start 2024-09-21T03:38:19.847 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:19.845+0000 7f0b5aff6640 1 -- 172.21.15.89:0/1365084226 shutdown_connections 2024-09-21T03:38:19.847 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:19.845+0000 7f0b5aff6640 1 -- 172.21.15.89:0/1365084226 >> 172.21.15.89:0/1365084226 conn(0x7f0b54079740 msgr2=0x7f0b5406c010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:38:19.847 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:19.845+0000 7f0b5aff6640 1 -- 172.21.15.89:0/1365084226 shutdown_connections 2024-09-21T03:38:19.847 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:19.845+0000 7f0b5aff6640 1 -- 172.21.15.89:0/1365084226 wait complete. 2024-09-21T03:38:20.678 INFO:teuthology.orchestra.run.smithi089.stdout:34359738450 2024-09-21T03:38:20.678 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 7c374098-77c9-11ef-baf5-efdc52797490 -- ceph osd last-stat-seq osd.0 2024-09-21T03:38:21.201 INFO:teuthology.orchestra.run.smithi089.stdout:270582939679 2024-09-21T03:38:21.201 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 7c374098-77c9-11ef-baf5-efdc52797490 -- ceph osd last-stat-seq osd.10 2024-09-21T03:38:21.376 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/7c374098-77c9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:38:21.376 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/7c374098-77c9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:38:21.475 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:38:21 smithi089 ceph-mon[30928]: pgmap v326: 129 pgs: 129 active+clean; 646 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 682 B/s wr, 7 op/s 2024-09-21T03:38:21.600 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:21.569+0000 7f7554e42640 1 -- 172.21.15.89:0/72121059 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7550109e00 msgr2=0x7f7550110630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:21.600 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:21.569+0000 7f7554e42640 1 --2- 172.21.15.89:0/72121059 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7550109e00 0x7f7550110630 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f7548009ca0 tx=0x7f754802a8c0 comp rx=0 tx=0).stop 2024-09-21T03:38:21.600 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:21.569+0000 7f7554e42640 1 -- 172.21.15.89:0/72121059 shutdown_connections 2024-09-21T03:38:21.600 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:21.569+0000 7f7554e42640 1 --2- 172.21.15.89:0/72121059 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7550109e00 0x7f7550110630 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:21.601 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:21.569+0000 7f7554e42640 1 --2- 172.21.15.89:0/72121059 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f75501094b0 0x7f75501098c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:21.601 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:21.569+0000 7f7554e42640 1 --2- 172.21.15.89:0/72121059 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7550103bc0 0x7f7550103fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:21.601 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:21.569+0000 7f7554e42640 1 -- 172.21.15.89:0/72121059 >> 172.21.15.89:0/72121059 conn(0x7f755007b910 msgr2=0x7f755007bd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:38:21.601 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:21.569+0000 7f7554e42640 1 -- 172.21.15.89:0/72121059 shutdown_connections 2024-09-21T03:38:21.601 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:21.569+0000 7f7554e42640 1 -- 172.21.15.89:0/72121059 wait complete. 2024-09-21T03:38:21.601 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:21.570+0000 7f7554e42640 1 Processor -- start 2024-09-21T03:38:21.601 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:21.570+0000 7f7554e42640 1 -- start start 2024-09-21T03:38:21.601 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:21.570+0000 7f7554e42640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7550103bc0 0x7f755019c550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:21.601 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:21.570+0000 7f7554e42640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f75501094b0 0x7f755019caf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:21.601 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:21.570+0000 7f7554e42640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7550109e00 0x7f75501a1d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:21.601 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:21.570+0000 7f7554e42640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f75501139c0 con 0x7f75501094b0 2024-09-21T03:38:21.601 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:21.570+0000 7f7554e42640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f7550113840 con 0x7f7550109e00 2024-09-21T03:38:21.601 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:21.570+0000 7f7554e42640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f7550113b40 con 0x7f7550103bc0 2024-09-21T03:38:21.602 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:21.570+0000 7f754dd9b640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f75501094b0 0x7f755019caf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:21.602 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:21.570+0000 7f754dd9b640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f75501094b0 0x7f755019caf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:47640/0 (socket says 172.21.15.89:47640) 2024-09-21T03:38:21.602 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:21.570+0000 7f754dd9b640 1 -- 172.21.15.89:0/1782202762 learned_addr learned my addr 172.21.15.89:0/1782202762 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-21T03:38:21.602 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:21.570+0000 7f754ed9d640 1 --2- 172.21.15.89:0/1782202762 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7550109e00 0x7f75501a1d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:21.602 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:21.571+0000 7f754e59c640 1 --2- 172.21.15.89:0/1782202762 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7550103bc0 0x7f755019c550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:21.602 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:21.571+0000 7f754dd9b640 1 -- 172.21.15.89:0/1782202762 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7550103bc0 msgr2=0x7f755019c550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:21.602 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:21.571+0000 7f754dd9b640 1 --2- 172.21.15.89:0/1782202762 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7550103bc0 0x7f755019c550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:21.602 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:21.571+0000 7f754dd9b640 1 -- 172.21.15.89:0/1782202762 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7550109e00 msgr2=0x7f75501a1d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:21.602 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:21.571+0000 7f754dd9b640 1 --2- 172.21.15.89:0/1782202762 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7550109e00 0x7f75501a1d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:21.602 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:21.571+0000 7f754dd9b640 1 -- 172.21.15.89:0/1782202762 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75501a24c0 con 0x7f75501094b0 2024-09-21T03:38:21.602 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:21.571+0000 7f754dd9b640 1 --2- 172.21.15.89:0/1782202762 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f75501094b0 0x7f755019caf0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f753800d950 tx=0x7f753800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:38:21.602 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:21.571+0000 7f75477fe640 1 -- 172.21.15.89:0/1782202762 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7538017070 con 0x7f75501094b0 2024-09-21T03:38:21.602 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:21.571+0000 7f7554e42640 1 -- 172.21.15.89:0/1782202762 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f75501a27b0 con 0x7f75501094b0 2024-09-21T03:38:21.603 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:21.571+0000 7f7554e42640 1 -- 172.21.15.89:0/1782202762 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75501a8f10 con 0x7f75501094b0 2024-09-21T03:38:21.603 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:21.571+0000 7f75477fe640 1 -- 172.21.15.89:0/1782202762 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7538009c30 con 0x7f75501094b0 2024-09-21T03:38:21.603 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:21.571+0000 7f75477fe640 1 -- 172.21.15.89:0/1782202762 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7538004cd0 con 0x7f75501094b0 2024-09-21T03:38:21.603 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:21.572+0000 7f7554e42640 1 -- 172.21.15.89:0/1782202762 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f7550105d40 con 0x7f75501094b0 2024-09-21T03:38:21.603 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:21.575+0000 7f75477fe640 1 -- 172.21.15.89:0/1782202762 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 18) ==== 154563+0+0 (secure 0 0 0) 0x7f7538004020 con 0x7f75501094b0 2024-09-21T03:38:21.603 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:21.576+0000 7f75477fe640 1 --2- 172.21.15.89:0/1782202762 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f753c0b65a0 0x7f753c0b8ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:21.603 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:21.576+0000 7f754e59c640 1 --2- 172.21.15.89:0/1782202762 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f753c0b65a0 0x7f753c0b8ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:21.603 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:21.576+0000 7f75477fe640 1 -- 172.21.15.89:0/1782202762 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 9092+0+0 (secure 0 0 0) 0x7f75380e7ab0 con 0x7f75501094b0 2024-09-21T03:38:21.603 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:21.576+0000 7f75477fe640 1 --2- 172.21.15.89:0/1782202762 >> [v2:172.21.15.110:6808/3217880476,v1:172.21.15.110:6809/3217880476] conn(0x7f753c0c16b0 0x7f753c0c3ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:21.603 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:21.576+0000 7f75477fe640 1 -- 172.21.15.89:0/1782202762 --> [v2:172.21.15.110:6808/3217880476,v1:172.21.15.110:6809/3217880476] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f75380056e0 con 0x7f753c0c16b0 2024-09-21T03:38:21.603 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:21.576+0000 7f75477fe640 1 -- 172.21.15.89:0/1782202762 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_get_version_reply(handle=1 version=82) ==== 24+0+0 (secure 0 0 0) 0x7f75380e7f00 con 0x7f75501094b0 2024-09-21T03:38:21.603 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:21.576+0000 7f754e59c640 1 --2- 172.21.15.89:0/1782202762 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f753c0b65a0 0x7f753c0b8ab0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f7540009bd0 tx=0x7f7540005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:38:21.603 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:21.576+0000 7f754ed9d640 1 --2- 172.21.15.89:0/1782202762 >> [v2:172.21.15.110:6808/3217880476,v1:172.21.15.110:6809/3217880476] conn(0x7f753c0c16b0 0x7f753c0c3ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:21.604 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:21.577+0000 7f754ed9d640 1 --2- 172.21.15.89:0/1782202762 >> [v2:172.21.15.110:6808/3217880476,v1:172.21.15.110:6809/3217880476] conn(0x7f753c0c16b0 0x7f753c0c3ba0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:38:21.604 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:21.578+0000 7f75477fe640 1 -- 172.21.15.89:0/1782202762 <== osd.4 v2:172.21.15.110:6808/3217880476 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f75380056e0 con 0x7f753c0c16b0 2024-09-21T03:38:21.604 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:21.591+0000 7f7554e42640 1 -- 172.21.15.89:0/1782202762 --> [v2:172.21.15.110:6808/3217880476,v1:172.21.15.110:6809/3217880476] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f7550104df0 con 0x7f753c0c16b0 2024-09-21T03:38:21.604 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:21.592+0000 7f75477fe640 1 -- 172.21.15.89:0/1782202762 <== osd.4 v2:172.21.15.110:6808/3217880476 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f7550104df0 con 0x7f753c0c16b0 2024-09-21T03:38:21.604 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:21.592+0000 7f7554e42640 1 -- 172.21.15.89:0/1782202762 >> [v2:172.21.15.110:6808/3217880476,v1:172.21.15.110:6809/3217880476] conn(0x7f753c0c16b0 msgr2=0x7f753c0c3ba0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:21.604 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:21.592+0000 7f7554e42640 1 --2- 172.21.15.89:0/1782202762 >> [v2:172.21.15.110:6808/3217880476,v1:172.21.15.110:6809/3217880476] conn(0x7f753c0c16b0 0x7f753c0c3ba0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:21.604 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:21.592+0000 7f7554e42640 1 -- 172.21.15.89:0/1782202762 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f753c0b65a0 msgr2=0x7f753c0b8ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:21.604 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:21.592+0000 7f7554e42640 1 --2- 172.21.15.89:0/1782202762 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f753c0b65a0 0x7f753c0b8ab0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f7540009bd0 tx=0x7f7540005f90 comp rx=0 tx=0).stop 2024-09-21T03:38:21.604 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:21.592+0000 7f7554e42640 1 -- 172.21.15.89:0/1782202762 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f75501094b0 msgr2=0x7f755019caf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:21.604 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:21.592+0000 7f7554e42640 1 --2- 172.21.15.89:0/1782202762 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f75501094b0 0x7f755019caf0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f753800d950 tx=0x7f753800de10 comp rx=0 tx=0).stop 2024-09-21T03:38:21.605 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:21.592+0000 7f754ed9d640 1 -- 172.21.15.89:0/1782202762 reap_dead start 2024-09-21T03:38:21.605 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:21.592+0000 7f7554e42640 1 -- 172.21.15.89:0/1782202762 shutdown_connections 2024-09-21T03:38:21.605 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:21.592+0000 7f7554e42640 1 -- 172.21.15.89:0/1782202762 >> 172.21.15.89:0/1782202762 conn(0x7f755007b910 msgr2=0x7f75501052e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:38:21.605 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:21.592+0000 7f7554e42640 1 -- 172.21.15.89:0/1782202762 shutdown_connections 2024-09-21T03:38:21.605 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:21.592+0000 7f7554e42640 1 -- 172.21.15.89:0/1782202762 wait complete. 2024-09-21T03:38:21.710 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:38:21 smithi114 ceph-mon[34140]: pgmap v326: 129 pgs: 129 active+clean; 646 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 682 B/s wr, 7 op/s 2024-09-21T03:38:21.750 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:38:21 smithi110 ceph-mon[34000]: pgmap v326: 129 pgs: 129 active+clean; 646 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 682 B/s wr, 7 op/s 2024-09-21T03:38:22.283 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:22.279+0000 7fea6eb29640 1 -- 172.21.15.89:0/1995795232 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fea681083a0 msgr2=0x7fea681087b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:22.283 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:22.279+0000 7fea6eb29640 1 --2- 172.21.15.89:0/1995795232 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fea681083a0 0x7fea681087b0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fea50009900 tx=0x7fea5002a410 comp rx=0 tx=0).stop 2024-09-21T03:38:22.283 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:22.279+0000 7fea6eb29640 1 -- 172.21.15.89:0/1995795232 shutdown_connections 2024-09-21T03:38:22.283 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:22.279+0000 7fea6eb29640 1 --2- 172.21.15.89:0/1995795232 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fea68108cf0 0x7fea6810f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:22.283 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:22.279+0000 7fea6eb29640 1 --2- 172.21.15.89:0/1995795232 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fea681083a0 0x7fea681087b0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:22.283 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:22.279+0000 7fea6eb29640 1 --2- 172.21.15.89:0/1995795232 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fea68102a60 0x7fea68102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:22.283 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:22.279+0000 7fea6eb29640 1 -- 172.21.15.89:0/1995795232 >> 172.21.15.89:0/1995795232 conn(0x7fea68079760 msgr2=0x7fea68079b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:38:22.283 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:22.279+0000 7fea6eb29640 1 -- 172.21.15.89:0/1995795232 shutdown_connections 2024-09-21T03:38:22.283 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:22.279+0000 7fea6eb29640 1 -- 172.21.15.89:0/1995795232 wait complete. 2024-09-21T03:38:22.283 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:22.280+0000 7fea6eb29640 1 Processor -- start 2024-09-21T03:38:22.283 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:22.280+0000 7fea6eb29640 1 -- start start 2024-09-21T03:38:22.283 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:22.280+0000 7fea6eb29640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fea68102a60 0x7fea680ff650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:22.284 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:22.280+0000 7fea6eb29640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fea681083a0 0x7fea680ffb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:22.284 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:22.280+0000 7fea6eb29640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fea68108cf0 0x7fea681000d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:22.284 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:22.280+0000 7fea6eb29640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fea68112860 con 0x7fea681083a0 2024-09-21T03:38:22.284 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:22.280+0000 7fea6eb29640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fea681126e0 con 0x7fea68108cf0 2024-09-21T03:38:22.284 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:22.280+0000 7fea6eb29640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fea681129e0 con 0x7fea68102a60 2024-09-21T03:38:22.284 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:22.280+0000 7fea67fff640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fea681083a0 0x7fea680ffb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:22.284 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:22.280+0000 7fea67fff640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fea681083a0 0x7fea680ffb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:47660/0 (socket says 172.21.15.89:47660) 2024-09-21T03:38:22.284 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:22.280+0000 7fea67fff640 1 -- 172.21.15.89:0/3724333707 learned_addr learned my addr 172.21.15.89:0/3724333707 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-21T03:38:22.284 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:22.280+0000 7fea6c8c5640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fea68102a60 0x7fea680ff650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:22.284 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:22.281+0000 7fea6d0c6640 1 --2- 172.21.15.89:0/3724333707 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fea68108cf0 0x7fea681000d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:22.284 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:22.281+0000 7fea6c8c5640 1 -- 172.21.15.89:0/3724333707 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fea68108cf0 msgr2=0x7fea681000d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:22.284 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:22.281+0000 7fea6c8c5640 1 --2- 172.21.15.89:0/3724333707 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fea68108cf0 0x7fea681000d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:22.284 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:22.281+0000 7fea6c8c5640 1 -- 172.21.15.89:0/3724333707 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fea681083a0 msgr2=0x7fea680ffb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:22.284 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:22.281+0000 7fea6c8c5640 1 --2- 172.21.15.89:0/3724333707 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fea681083a0 0x7fea680ffb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:22.284 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:22.281+0000 7fea6c8c5640 1 -- 172.21.15.89:0/3724333707 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea681a86c0 con 0x7fea68102a60 2024-09-21T03:38:22.284 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:22.281+0000 7fea67fff640 1 --2- 172.21.15.89:0/3724333707 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fea681083a0 0x7fea680ffb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:38:22.285 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:22.281+0000 7fea6c8c5640 1 --2- 172.21.15.89:0/3724333707 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fea68102a60 0x7fea680ff650 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fea58005fd0 tx=0x7fea5800dde0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:38:22.285 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:22.281+0000 7fea65ffb640 1 -- 172.21.15.89:0/3724333707 <== mon.1 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea58009c30 con 0x7fea68102a60 2024-09-21T03:38:22.285 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:22.281+0000 7fea6eb29640 1 -- 172.21.15.89:0/3724333707 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fea681a89b0 con 0x7fea68102a60 2024-09-21T03:38:22.285 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:22.281+0000 7fea65ffb640 1 -- 172.21.15.89:0/3724333707 <== mon.1 v2:172.21.15.114:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fea580044c0 con 0x7fea68102a60 2024-09-21T03:38:22.285 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:22.282+0000 7fea65ffb640 1 -- 172.21.15.89:0/3724333707 <== mon.1 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea58003db0 con 0x7fea68102a60 2024-09-21T03:38:22.285 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:22.282+0000 7fea6eb29640 1 -- 172.21.15.89:0/3724333707 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fea681a8f10 con 0x7fea68102a60 2024-09-21T03:38:22.285 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:22.282+0000 7fea6eb29640 1 -- 172.21.15.89:0/3724333707 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fea681064e0 con 0x7fea68102a60 2024-09-21T03:38:22.286 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:22.285+0000 7fea65ffb640 1 -- 172.21.15.89:0/3724333707 <== mon.1 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 18) ==== 154563+0+0 (secure 0 0 0) 0x7fea58004000 con 0x7fea68102a60 2024-09-21T03:38:22.288 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:22.286+0000 7fea65ffb640 1 --2- 172.21.15.89:0/3724333707 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7fea540b6620 0x7fea540b8b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:22.288 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:22.286+0000 7fea67fff640 1 --2- 172.21.15.89:0/3724333707 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7fea540b6620 0x7fea540b8b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:22.288 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:22.286+0000 7fea65ffb640 1 -- 172.21.15.89:0/3724333707 <== mon.1 v2:172.21.15.114:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 9092+0+0 (secure 0 0 0) 0x7fea580e76c0 con 0x7fea68102a60 2024-09-21T03:38:22.288 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:22.286+0000 7fea65ffb640 1 --2- 172.21.15.89:0/3724333707 >> [v2:172.21.15.114:6816/2138556699,v1:172.21.15.114:6817/2138556699] conn(0x7fea540c16b0 0x7fea540c3ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:22.288 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:22.286+0000 7fea65ffb640 1 -- 172.21.15.89:0/3724333707 --> [v2:172.21.15.114:6816/2138556699,v1:172.21.15.114:6817/2138556699] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fea58011e80 con 0x7fea540c16b0 2024-09-21T03:38:22.288 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:22.286+0000 7fea65ffb640 1 -- 172.21.15.89:0/3724333707 <== mon.1 v2:172.21.15.114:3300/0 6 ==== mon_get_version_reply(handle=1 version=82) ==== 24+0+0 (secure 0 0 0) 0x7fea580e7a70 con 0x7fea68102a60 2024-09-21T03:38:22.288 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:22.287+0000 7fea67fff640 1 --2- 172.21.15.89:0/3724333707 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7fea540b6620 0x7fea540b8b30 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fea50000c00 tx=0x7fea500058d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:38:22.288 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:22.287+0000 7fea6d0c6640 1 --2- 172.21.15.89:0/3724333707 >> [v2:172.21.15.114:6816/2138556699,v1:172.21.15.114:6817/2138556699] conn(0x7fea540c16b0 0x7fea540c3ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:22.288 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:22.287+0000 7fea6d0c6640 1 --2- 172.21.15.89:0/3724333707 >> [v2:172.21.15.114:6816/2138556699,v1:172.21.15.114:6817/2138556699] conn(0x7fea540c16b0 0x7fea540c3ba0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:38:22.289 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:22.288+0000 7fea65ffb640 1 -- 172.21.15.89:0/3724333707 <== osd.8 v2:172.21.15.114:6816/2138556699 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fea58011e80 con 0x7fea540c16b0 2024-09-21T03:38:22.303 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:22.302+0000 7fea6eb29640 1 -- 172.21.15.89:0/3724333707 --> [v2:172.21.15.114:6816/2138556699,v1:172.21.15.114:6817/2138556699] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fea6806b390 con 0x7fea540c16b0 2024-09-21T03:38:22.304 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:22.303+0000 7fea65ffb640 1 -- 172.21.15.89:0/3724333707 <== osd.8 v2:172.21.15.114:6816/2138556699 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fea6806b390 con 0x7fea540c16b0 2024-09-21T03:38:22.304 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:22.303+0000 7fea6eb29640 1 -- 172.21.15.89:0/3724333707 >> [v2:172.21.15.114:6816/2138556699,v1:172.21.15.114:6817/2138556699] conn(0x7fea540c16b0 msgr2=0x7fea540c3ba0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:22.305 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:22.303+0000 7fea6eb29640 1 --2- 172.21.15.89:0/3724333707 >> [v2:172.21.15.114:6816/2138556699,v1:172.21.15.114:6817/2138556699] conn(0x7fea540c16b0 0x7fea540c3ba0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:22.305 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:22.303+0000 7fea6eb29640 1 -- 172.21.15.89:0/3724333707 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7fea540b6620 msgr2=0x7fea540b8b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:22.305 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:22.303+0000 7fea6eb29640 1 --2- 172.21.15.89:0/3724333707 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7fea540b6620 0x7fea540b8b30 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fea50000c00 tx=0x7fea500058d0 comp rx=0 tx=0).stop 2024-09-21T03:38:22.305 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:22.303+0000 7fea6eb29640 1 -- 172.21.15.89:0/3724333707 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fea68102a60 msgr2=0x7fea680ff650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:22.305 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:22.303+0000 7fea6eb29640 1 --2- 172.21.15.89:0/3724333707 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fea68102a60 0x7fea680ff650 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fea58005fd0 tx=0x7fea5800dde0 comp rx=0 tx=0).stop 2024-09-21T03:38:22.305 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:22.303+0000 7fea6d0c6640 1 -- 172.21.15.89:0/3724333707 reap_dead start 2024-09-21T03:38:22.305 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:22.303+0000 7fea6eb29640 1 -- 172.21.15.89:0/3724333707 shutdown_connections 2024-09-21T03:38:22.305 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:22.303+0000 7fea6eb29640 1 -- 172.21.15.89:0/3724333707 >> 172.21.15.89:0/3724333707 conn(0x7fea68079760 msgr2=0x7fea68101a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:38:22.305 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:22.303+0000 7fea6eb29640 1 -- 172.21.15.89:0/3724333707 shutdown_connections 2024-09-21T03:38:22.305 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:22.303+0000 7fea6eb29640 1 -- 172.21.15.89:0/3724333707 wait complete. 2024-09-21T03:38:22.488 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/7c374098-77c9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:38:23.239 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:23.237+0000 7fc9b8b97640 1 -- 172.21.15.89:0/2438669106 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc9b4109560 msgr2=0x7fc9b4109970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:23.239 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:23.237+0000 7fc9b8b97640 1 --2- 172.21.15.89:0/2438669106 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc9b4109560 0x7fc9b4109970 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fc9a8009900 tx=0x7fc9a802a410 comp rx=0 tx=0).stop 2024-09-21T03:38:23.239 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:23.238+0000 7fc9b8b97640 1 -- 172.21.15.89:0/2438669106 shutdown_connections 2024-09-21T03:38:23.239 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:23.238+0000 7fc9b8b97640 1 --2- 172.21.15.89:0/2438669106 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc9b4109eb0 0x7fc9b41106e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:23.240 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:23.238+0000 7fc9b8b97640 1 --2- 172.21.15.89:0/2438669106 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc9b4109560 0x7fc9b4109970 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:23.240 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:23.238+0000 7fc9b8b97640 1 --2- 172.21.15.89:0/2438669106 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc9b4102080 0x7fc9b4102490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:23.240 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:23.238+0000 7fc9b8b97640 1 -- 172.21.15.89:0/2438669106 >> 172.21.15.89:0/2438669106 conn(0x7fc9b4079ec0 msgr2=0x7fc9b407a2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:38:23.240 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:23.238+0000 7fc9b8b97640 1 -- 172.21.15.89:0/2438669106 shutdown_connections 2024-09-21T03:38:23.240 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:23.238+0000 7fc9b8b97640 1 -- 172.21.15.89:0/2438669106 wait complete. 2024-09-21T03:38:23.242 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:23.239+0000 7fc9b8b97640 1 Processor -- start 2024-09-21T03:38:23.242 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:23.239+0000 7fc9b8b97640 1 -- start start 2024-09-21T03:38:23.242 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:23.239+0000 7fc9b8b97640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc9b4102080 0x7fc9b419a450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:23.242 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:23.239+0000 7fc9b8b97640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc9b4109560 0x7fc9b419a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:23.242 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:23.239+0000 7fc9b8b97640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc9b4109eb0 0x7fc9b419fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:23.242 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:23.239+0000 7fc9b8b97640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fc9b4113a20 con 0x7fc9b4102080 2024-09-21T03:38:23.242 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:23.239+0000 7fc9b8b97640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fc9b41138a0 con 0x7fc9b4109560 2024-09-21T03:38:23.242 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:23.239+0000 7fc9b8b97640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fc9b4113ba0 con 0x7fc9b4109eb0 2024-09-21T03:38:23.242 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:23.239+0000 7fc9b259c640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc9b4102080 0x7fc9b419a450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:23.242 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:23.239+0000 7fc9b259c640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc9b4102080 0x7fc9b419a450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:47670/0 (socket says 172.21.15.89:47670) 2024-09-21T03:38:23.243 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:23.239+0000 7fc9b259c640 1 -- 172.21.15.89:0/69947217 learned_addr learned my addr 172.21.15.89:0/69947217 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-21T03:38:23.243 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:23.239+0000 7fc9b1d9b640 1 --2- 172.21.15.89:0/69947217 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc9b4109560 0x7fc9b419a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:23.243 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:23.239+0000 7fc9b2d9d640 1 --2- 172.21.15.89:0/69947217 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc9b4109eb0 0x7fc9b419fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:23.243 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:23.240+0000 7fc9b259c640 1 -- 172.21.15.89:0/69947217 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc9b4109eb0 msgr2=0x7fc9b419fc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:23.243 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:23.240+0000 7fc9b259c640 1 --2- 172.21.15.89:0/69947217 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc9b4109eb0 0x7fc9b419fc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:23.243 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:23.240+0000 7fc9b259c640 1 -- 172.21.15.89:0/69947217 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc9b4109560 msgr2=0x7fc9b419a9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:23.243 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:23.240+0000 7fc9b259c640 1 --2- 172.21.15.89:0/69947217 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc9b4109560 0x7fc9b419a9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:23.243 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:23.240+0000 7fc9b259c640 1 -- 172.21.15.89:0/69947217 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9b41a03c0 con 0x7fc9b4102080 2024-09-21T03:38:23.243 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:23.240+0000 7fc9b2d9d640 1 --2- 172.21.15.89:0/69947217 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc9b4109eb0 0x7fc9b419fc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:38:23.243 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:23.240+0000 7fc9b259c640 1 --2- 172.21.15.89:0/69947217 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc9b4102080 0x7fc9b419a450 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fc9a400e9f0 tx=0x7fc9a400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:38:23.243 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:23.240+0000 7fc9a37fe640 1 -- 172.21.15.89:0/69947217 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9a400cd80 con 0x7fc9b4102080 2024-09-21T03:38:23.243 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:23.240+0000 7fc9b8b97640 1 -- 172.21.15.89:0/69947217 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc9b41a06b0 con 0x7fc9b4102080 2024-09-21T03:38:23.243 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:23.240+0000 7fc9a37fe640 1 -- 172.21.15.89:0/69947217 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc9a40108c0 con 0x7fc9b4102080 2024-09-21T03:38:23.243 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:23.240+0000 7fc9a37fe640 1 -- 172.21.15.89:0/69947217 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9a40187c0 con 0x7fc9b4102080 2024-09-21T03:38:23.243 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:23.240+0000 7fc9b8b97640 1 -- 172.21.15.89:0/69947217 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9b41a6e10 con 0x7fc9b4102080 2024-09-21T03:38:23.244 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:23.241+0000 7fc9b8b97640 1 -- 172.21.15.89:0/69947217 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fc9b4063ad0 con 0x7fc9b4102080 2024-09-21T03:38:23.244 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:23.243+0000 7fc9a37fe640 1 -- 172.21.15.89:0/69947217 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 18) ==== 154563+0+0 (secure 0 0 0) 0x7fc9a4010430 con 0x7fc9b4102080 2024-09-21T03:38:23.245 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:23.243+0000 7fc9a37fe640 1 --2- 172.21.15.89:0/69947217 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7fc99c0b6620 0x7fc99c0b8b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:23.245 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:23.244+0000 7fc9b1d9b640 1 --2- 172.21.15.89:0/69947217 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7fc99c0b6620 0x7fc99c0b8b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:23.245 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:23.244+0000 7fc9a37fe640 1 -- 172.21.15.89:0/69947217 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 9092+0+0 (secure 0 0 0) 0x7fc9a40e8740 con 0x7fc9b4102080 2024-09-21T03:38:23.245 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:23.244+0000 7fc9a37fe640 1 --2- 172.21.15.89:0/69947217 >> [v2:172.21.15.114:6808/1144140707,v1:172.21.15.114:6809/1144140707] conn(0x7fc99c0c1730 0x7fc99c0c3c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:23.245 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:23.244+0000 7fc9a37fe640 1 -- 172.21.15.89:0/69947217 --> [v2:172.21.15.114:6808/1144140707,v1:172.21.15.114:6809/1144140707] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fc9a4015e80 con 0x7fc99c0c1730 2024-09-21T03:38:23.245 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:23.244+0000 7fc9a37fe640 1 -- 172.21.15.89:0/69947217 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_get_version_reply(handle=1 version=82) ==== 24+0+0 (secure 0 0 0) 0x7fc9a40e8b90 con 0x7fc9b4102080 2024-09-21T03:38:23.246 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:23.244+0000 7fc9b1d9b640 1 --2- 172.21.15.89:0/69947217 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7fc99c0b6620 0x7fc99c0b8b30 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fc9a8005940 tx=0x7fc9a80058d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:38:23.246 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:23.244+0000 7fc9b2d9d640 1 --2- 172.21.15.89:0/69947217 >> [v2:172.21.15.114:6808/1144140707,v1:172.21.15.114:6809/1144140707] conn(0x7fc99c0c1730 0x7fc99c0c3c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:23.246 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:23.245+0000 7fc9b2d9d640 1 --2- 172.21.15.89:0/69947217 >> [v2:172.21.15.114:6808/1144140707,v1:172.21.15.114:6809/1144140707] conn(0x7fc99c0c1730 0x7fc99c0c3c20 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:38:23.247 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:23.246+0000 7fc9a37fe640 1 -- 172.21.15.89:0/69947217 <== osd.5 v2:172.21.15.114:6808/1144140707 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fc9a4015e80 con 0x7fc99c0c1730 2024-09-21T03:38:23.261 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:23.260+0000 7fc9b8b97640 1 -- 172.21.15.89:0/69947217 --> [v2:172.21.15.114:6808/1144140707,v1:172.21.15.114:6809/1144140707] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fc9b4063f00 con 0x7fc99c0c1730 2024-09-21T03:38:23.262 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:23.261+0000 7fc9a37fe640 1 -- 172.21.15.89:0/69947217 <== osd.5 v2:172.21.15.114:6808/1144140707 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fc9b4063f00 con 0x7fc99c0c1730 2024-09-21T03:38:23.263 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:23.261+0000 7fc9b8b97640 1 -- 172.21.15.89:0/69947217 >> [v2:172.21.15.114:6808/1144140707,v1:172.21.15.114:6809/1144140707] conn(0x7fc99c0c1730 msgr2=0x7fc99c0c3c20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:23.263 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:23.261+0000 7fc9b8b97640 1 --2- 172.21.15.89:0/69947217 >> [v2:172.21.15.114:6808/1144140707,v1:172.21.15.114:6809/1144140707] conn(0x7fc99c0c1730 0x7fc99c0c3c20 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:23.263 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:23.261+0000 7fc9b8b97640 1 -- 172.21.15.89:0/69947217 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7fc99c0b6620 msgr2=0x7fc99c0b8b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:23.263 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:23.261+0000 7fc9b8b97640 1 --2- 172.21.15.89:0/69947217 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7fc99c0b6620 0x7fc99c0b8b30 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fc9a8005940 tx=0x7fc9a80058d0 comp rx=0 tx=0).stop 2024-09-21T03:38:23.263 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:23.261+0000 7fc9b8b97640 1 -- 172.21.15.89:0/69947217 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc9b4102080 msgr2=0x7fc9b419a450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:23.263 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:23.261+0000 7fc9b8b97640 1 --2- 172.21.15.89:0/69947217 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc9b4102080 0x7fc9b419a450 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fc9a400e9f0 tx=0x7fc9a400eeb0 comp rx=0 tx=0).stop 2024-09-21T03:38:23.263 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:23.261+0000 7fc9b2d9d640 1 -- 172.21.15.89:0/69947217 reap_dead start 2024-09-21T03:38:23.263 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:23.261+0000 7fc9b8b97640 1 -- 172.21.15.89:0/69947217 shutdown_connections 2024-09-21T03:38:23.263 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:23.261+0000 7fc9b8b97640 1 -- 172.21.15.89:0/69947217 >> 172.21.15.89:0/69947217 conn(0x7fc9b4079ec0 msgr2=0x7fc9b4107660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:38:23.263 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:23.261+0000 7fc9b8b97640 1 -- 172.21.15.89:0/69947217 shutdown_connections 2024-09-21T03:38:23.263 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:23.261+0000 7fc9b8b97640 1 -- 172.21.15.89:0/69947217 wait complete. 2024-09-21T03:38:23.662 INFO:teuthology.orchestra.run.smithi089.stdout:292057776155 2024-09-21T03:38:23.662 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 7c374098-77c9-11ef-baf5-efdc52797490 -- ceph osd last-stat-seq osd.11 2024-09-21T03:38:23.685 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:38:23 smithi089 ceph-mon[30928]: pgmap v327: 129 pgs: 129 active+clean; 646 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 170 B/s wr, 5 op/s 2024-09-21T03:38:23.711 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:38:23 smithi114 ceph-mon[34140]: pgmap v327: 129 pgs: 129 active+clean; 646 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 170 B/s wr, 5 op/s 2024-09-21T03:38:23.750 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:38:23 smithi110 ceph-mon[34000]: pgmap v327: 129 pgs: 129 active+clean; 646 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 170 B/s wr, 5 op/s 2024-09-21T03:38:23.982 INFO:teuthology.orchestra.run.smithi089.stdout:128849018941 2024-09-21T03:38:23.983 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 7c374098-77c9-11ef-baf5-efdc52797490 -- ceph osd last-stat-seq osd.4 2024-09-21T03:38:24.467 INFO:teuthology.orchestra.run.smithi089.stdout:249108103205 2024-09-21T03:38:24.467 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 7c374098-77c9-11ef-baf5-efdc52797490 -- ceph osd last-stat-seq osd.9 2024-09-21T03:38:25.386 INFO:teuthology.orchestra.run.smithi089.stdout:176093659187 2024-09-21T03:38:25.387 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 7c374098-77c9-11ef-baf5-efdc52797490 -- ceph osd last-stat-seq osd.6 2024-09-21T03:38:25.561 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:38:25 smithi089 ceph-mon[30928]: pgmap v328: 129 pgs: 129 active+clean; 646 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 170 B/s wr, 6 op/s 2024-09-21T03:38:25.686 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:25.682+0000 7f8471593640 1 -- 172.21.15.89:0/1046411066 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f846c1094a0 msgr2=0x7f846c1098b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:25.686 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:25.682+0000 7f8471593640 1 --2- 172.21.15.89:0/1046411066 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f846c1094a0 0x7f846c1098b0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f8460009900 tx=0x7f846002a410 comp rx=0 tx=0).stop 2024-09-21T03:38:25.687 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:25.682+0000 7f8471593640 1 -- 172.21.15.89:0/1046411066 shutdown_connections 2024-09-21T03:38:25.687 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:25.682+0000 7f8471593640 1 --2- 172.21.15.89:0/1046411066 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f846c109df0 0x7f846c110620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:25.687 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:25.682+0000 7f8471593640 1 --2- 172.21.15.89:0/1046411066 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f846c1094a0 0x7f846c1098b0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:25.687 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:25.682+0000 7f8471593640 1 --2- 172.21.15.89:0/1046411066 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f846c103b60 0x7f846c103f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:25.687 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:25.682+0000 7f8471593640 1 -- 172.21.15.89:0/1046411066 >> 172.21.15.89:0/1046411066 conn(0x7f846c07b850 msgr2=0x7f846c07bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:38:25.687 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:25.682+0000 7f8471593640 1 -- 172.21.15.89:0/1046411066 shutdown_connections 2024-09-21T03:38:25.687 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:25.682+0000 7f8471593640 1 -- 172.21.15.89:0/1046411066 wait complete. 2024-09-21T03:38:25.687 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:25.682+0000 7f8471593640 1 Processor -- start 2024-09-21T03:38:25.687 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:25.683+0000 7f8471593640 1 -- start start 2024-09-21T03:38:25.687 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:25.683+0000 7f8471593640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f846c103b60 0x7f846c19a450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:25.687 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:25.683+0000 7f8471593640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f846c1094a0 0x7f846c19a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:25.687 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:25.683+0000 7f8471593640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f846c109df0 0x7f846c19fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:25.687 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:25.683+0000 7f8471593640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f846c113960 con 0x7f846c109df0 2024-09-21T03:38:25.688 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:25.683+0000 7f8471593640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f846c1137e0 con 0x7f846c1094a0 2024-09-21T03:38:25.688 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:25.683+0000 7f8471593640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f846c113ae0 con 0x7f846c103b60 2024-09-21T03:38:25.688 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:25.683+0000 7f846b7fe640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f846c109df0 0x7f846c19fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:25.688 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:25.683+0000 7f846b7fe640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f846c109df0 0x7f846c19fc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:47684/0 (socket says 172.21.15.89:47684) 2024-09-21T03:38:25.688 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:25.683+0000 7f846b7fe640 1 -- 172.21.15.89:0/2447586427 learned_addr learned my addr 172.21.15.89:0/2447586427 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-21T03:38:25.688 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:25.683+0000 7f846a7fc640 1 --2- 172.21.15.89:0/2447586427 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f846c1094a0 0x7f846c19a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:25.688 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:25.683+0000 7f846affd640 1 --2- 172.21.15.89:0/2447586427 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f846c103b60 0x7f846c19a450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:25.688 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:25.683+0000 7f846b7fe640 1 -- 172.21.15.89:0/2447586427 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f846c103b60 msgr2=0x7f846c19a450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:25.688 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:25.683+0000 7f846b7fe640 1 --2- 172.21.15.89:0/2447586427 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f846c103b60 0x7f846c19a450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:25.688 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:25.683+0000 7f846b7fe640 1 -- 172.21.15.89:0/2447586427 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f846c1094a0 msgr2=0x7f846c19a9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:25.688 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:25.683+0000 7f846b7fe640 1 --2- 172.21.15.89:0/2447586427 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f846c1094a0 0x7f846c19a9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:25.688 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:25.683+0000 7f846b7fe640 1 -- 172.21.15.89:0/2447586427 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f846c1a0300 con 0x7f846c109df0 2024-09-21T03:38:25.688 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:25.684+0000 7f846b7fe640 1 --2- 172.21.15.89:0/2447586427 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f846c109df0 0x7f846c19fc90 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f846400c970 tx=0x7f846400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:38:25.688 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:25.684+0000 7f8453fff640 1 -- 172.21.15.89:0/2447586427 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8464007bb0 con 0x7f846c109df0 2024-09-21T03:38:25.689 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:25.684+0000 7f8471593640 1 -- 172.21.15.89:0/2447586427 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f846c1a05f0 con 0x7f846c109df0 2024-09-21T03:38:25.689 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:25.684+0000 7f8453fff640 1 -- 172.21.15.89:0/2447586427 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8464007d50 con 0x7f846c109df0 2024-09-21T03:38:25.689 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:25.684+0000 7f8453fff640 1 -- 172.21.15.89:0/2447586427 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8464018660 con 0x7f846c109df0 2024-09-21T03:38:25.689 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:25.684+0000 7f8471593640 1 -- 172.21.15.89:0/2447586427 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f846c1a6e10 con 0x7f846c109df0 2024-09-21T03:38:25.689 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:25.685+0000 7f8471593640 1 -- 172.21.15.89:0/2447586427 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f846c063b50 con 0x7f846c109df0 2024-09-21T03:38:25.690 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:25.686+0000 7f8453fff640 1 -- 172.21.15.89:0/2447586427 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 18) ==== 154563+0+0 (secure 0 0 0) 0x7f846400f970 con 0x7f846c109df0 2024-09-21T03:38:25.690 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:25.687+0000 7f8453fff640 1 --2- 172.21.15.89:0/2447586427 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f84540b65d0 0x7f84540b8ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:25.690 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:25.687+0000 7f846affd640 1 --2- 172.21.15.89:0/2447586427 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f84540b65d0 0x7f84540b8ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:25.690 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:25.687+0000 7f8453fff640 1 -- 172.21.15.89:0/2447586427 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 9092+0+0 (secure 0 0 0) 0x7f84640e7b70 con 0x7f846c109df0 2024-09-21T03:38:25.690 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:25.687+0000 7f8453fff640 1 --2- 172.21.15.89:0/2447586427 >> [v2:172.21.15.110:6816/1331818323,v1:172.21.15.110:6817/1331818323] conn(0x7f84540c16e0 0x7f84540c3bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:25.690 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:25.688+0000 7f8453fff640 1 -- 172.21.15.89:0/2447586427 --> [v2:172.21.15.110:6816/1331818323,v1:172.21.15.110:6817/1331818323] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f846401ae50 con 0x7f84540c16e0 2024-09-21T03:38:25.690 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:25.688+0000 7f8453fff640 1 -- 172.21.15.89:0/2447586427 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_get_version_reply(handle=1 version=82) ==== 24+0+0 (secure 0 0 0) 0x7f84640e7fc0 con 0x7f846c109df0 2024-09-21T03:38:25.690 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:25.688+0000 7f846affd640 1 --2- 172.21.15.89:0/2447586427 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f84540b65d0 0x7f84540b8ae0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f845c005fd0 tx=0x7f845c009680 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:38:25.690 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:25.688+0000 7f846a7fc640 1 --2- 172.21.15.89:0/2447586427 >> [v2:172.21.15.110:6816/1331818323,v1:172.21.15.110:6817/1331818323] conn(0x7f84540c16e0 0x7f84540c3bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:25.691 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:25.688+0000 7f846a7fc640 1 --2- 172.21.15.89:0/2447586427 >> [v2:172.21.15.110:6816/1331818323,v1:172.21.15.110:6817/1331818323] conn(0x7f84540c16e0 0x7f84540c3bd0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:38:25.691 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:25.689+0000 7f8453fff640 1 -- 172.21.15.89:0/2447586427 <== osd.7 v2:172.21.15.110:6816/1331818323 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f846401ae50 con 0x7f84540c16e0 2024-09-21T03:38:25.705 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:25.704+0000 7f8471593640 1 -- 172.21.15.89:0/2447586427 --> [v2:172.21.15.110:6816/1331818323,v1:172.21.15.110:6817/1331818323] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f846c063fd0 con 0x7f84540c16e0 2024-09-21T03:38:25.705 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:25.704+0000 7f8453fff640 1 -- 172.21.15.89:0/2447586427 <== osd.7 v2:172.21.15.110:6816/1331818323 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f846c063fd0 con 0x7f84540c16e0 2024-09-21T03:38:25.706 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:25.705+0000 7f8471593640 1 -- 172.21.15.89:0/2447586427 >> [v2:172.21.15.110:6816/1331818323,v1:172.21.15.110:6817/1331818323] conn(0x7f84540c16e0 msgr2=0x7f84540c3bd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:25.706 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:25.705+0000 7f8471593640 1 --2- 172.21.15.89:0/2447586427 >> [v2:172.21.15.110:6816/1331818323,v1:172.21.15.110:6817/1331818323] conn(0x7f84540c16e0 0x7f84540c3bd0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:25.706 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:25.705+0000 7f8471593640 1 -- 172.21.15.89:0/2447586427 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f84540b65d0 msgr2=0x7f84540b8ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:25.706 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:25.705+0000 7f8471593640 1 --2- 172.21.15.89:0/2447586427 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f84540b65d0 0x7f84540b8ae0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f845c005fd0 tx=0x7f845c009680 comp rx=0 tx=0).stop 2024-09-21T03:38:25.706 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:25.705+0000 7f8471593640 1 -- 172.21.15.89:0/2447586427 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f846c109df0 msgr2=0x7f846c19fc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:25.706 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:25.705+0000 7f8471593640 1 --2- 172.21.15.89:0/2447586427 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f846c109df0 0x7f846c19fc90 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f846400c970 tx=0x7f846400ce30 comp rx=0 tx=0).stop 2024-09-21T03:38:25.706 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:25.705+0000 7f846b7fe640 1 -- 172.21.15.89:0/2447586427 reap_dead start 2024-09-21T03:38:25.707 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:25.705+0000 7f8471593640 1 -- 172.21.15.89:0/2447586427 shutdown_connections 2024-09-21T03:38:25.707 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:25.705+0000 7f8471593640 1 -- 172.21.15.89:0/2447586427 >> 172.21.15.89:0/2447586427 conn(0x7f846c07b850 msgr2=0x7f846c105620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:38:25.707 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:25.705+0000 7f8471593640 1 -- 172.21.15.89:0/2447586427 shutdown_connections 2024-09-21T03:38:25.707 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:25.705+0000 7f8471593640 1 -- 172.21.15.89:0/2447586427 wait complete. 2024-09-21T03:38:25.710 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:38:25 smithi114 ceph-mon[34140]: pgmap v328: 129 pgs: 129 active+clean; 646 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 170 B/s wr, 6 op/s 2024-09-21T03:38:25.750 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:38:25 smithi110 ceph-mon[34000]: pgmap v328: 129 pgs: 129 active+clean; 646 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 170 B/s wr, 6 op/s 2024-09-21T03:38:26.377 INFO:teuthology.orchestra.run.smithi089.stdout:197568495664 2024-09-21T03:38:26.378 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 7c374098-77c9-11ef-baf5-efdc52797490 -- ceph osd last-stat-seq osd.7 2024-09-21T03:38:26.908 INFO:teuthology.orchestra.run.smithi089.stdout:223338299433 2024-09-21T03:38:26.909 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 7c374098-77c9-11ef-baf5-efdc52797490 -- ceph osd last-stat-seq osd.8 2024-09-21T03:38:27.072 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:27.063+0000 7f2e37906640 1 -- 172.21.15.89:0/415574907 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2e301083a0 msgr2=0x7f2e301087b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:27.072 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:27.063+0000 7f2e37906640 1 --2- 172.21.15.89:0/415574907 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2e301083a0 0x7f2e301087b0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f2e24009900 tx=0x7f2e2402a410 comp rx=0 tx=0).stop 2024-09-21T03:38:27.072 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:27.063+0000 7f2e37906640 1 -- 172.21.15.89:0/415574907 shutdown_connections 2024-09-21T03:38:27.072 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:27.063+0000 7f2e37906640 1 --2- 172.21.15.89:0/415574907 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2e30108cf0 0x7f2e3010f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:27.073 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:27.063+0000 7f2e37906640 1 --2- 172.21.15.89:0/415574907 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2e301083a0 0x7f2e301087b0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:27.073 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:27.063+0000 7f2e37906640 1 --2- 172.21.15.89:0/415574907 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2e30102a60 0x7f2e30102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:27.073 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:27.063+0000 7f2e37906640 1 -- 172.21.15.89:0/415574907 >> 172.21.15.89:0/415574907 conn(0x7f2e30079730 msgr2=0x7f2e30079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:38:27.073 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:27.063+0000 7f2e37906640 1 -- 172.21.15.89:0/415574907 shutdown_connections 2024-09-21T03:38:27.073 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:27.063+0000 7f2e37906640 1 -- 172.21.15.89:0/415574907 wait complete. 2024-09-21T03:38:27.073 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:27.064+0000 7f2e37906640 1 Processor -- start 2024-09-21T03:38:27.073 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:27.064+0000 7f2e37906640 1 -- start start 2024-09-21T03:38:27.073 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:27.064+0000 7f2e37906640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2e30102a60 0x7f2e3019a370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:27.073 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:27.064+0000 7f2e37906640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2e301083a0 0x7f2e3019a910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:27.073 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:27.064+0000 7f2e37906640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2e30108cf0 0x7f2e3019fbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:27.073 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:27.064+0000 7f2e37906640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f2e30112810 con 0x7f2e30108cf0 2024-09-21T03:38:27.073 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:27.064+0000 7f2e37906640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f2e30112690 con 0x7f2e30102a60 2024-09-21T03:38:27.073 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:27.064+0000 7f2e37906640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f2e30112990 con 0x7f2e301083a0 2024-09-21T03:38:27.073 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:27.065+0000 7f2e35ea3640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2e30108cf0 0x7f2e3019fbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:27.074 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:27.065+0000 7f2e35ea3640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2e30108cf0 0x7f2e3019fbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:47710/0 (socket says 172.21.15.89:47710) 2024-09-21T03:38:27.074 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:27.065+0000 7f2e35ea3640 1 -- 172.21.15.89:0/2648040632 learned_addr learned my addr 172.21.15.89:0/2648040632 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-21T03:38:27.074 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:27.065+0000 7f2e34ea1640 1 --2- 172.21.15.89:0/2648040632 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2e301083a0 0x7f2e3019a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:27.074 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:27.065+0000 7f2e356a2640 1 --2- 172.21.15.89:0/2648040632 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2e30102a60 0x7f2e3019a370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:27.074 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:27.065+0000 7f2e35ea3640 1 -- 172.21.15.89:0/2648040632 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2e301083a0 msgr2=0x7f2e3019a910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:27.074 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:27.065+0000 7f2e35ea3640 1 --2- 172.21.15.89:0/2648040632 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2e301083a0 0x7f2e3019a910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:27.074 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:27.065+0000 7f2e35ea3640 1 -- 172.21.15.89:0/2648040632 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2e30102a60 msgr2=0x7f2e3019a370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:27.074 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:27.065+0000 7f2e35ea3640 1 --2- 172.21.15.89:0/2648040632 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2e30102a60 0x7f2e3019a370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:27.074 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:27.065+0000 7f2e35ea3640 1 -- 172.21.15.89:0/2648040632 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e301a02e0 con 0x7f2e30108cf0 2024-09-21T03:38:27.074 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:27.065+0000 7f2e35ea3640 1 --2- 172.21.15.89:0/2648040632 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2e30108cf0 0x7f2e3019fbb0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f2e2800eaa0 tx=0x7f2e2800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:38:27.074 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:27.065+0000 7f2e227fc640 1 -- 172.21.15.89:0/2648040632 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e2800cdb0 con 0x7f2e30108cf0 2024-09-21T03:38:27.074 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:27.065+0000 7f2e227fc640 1 -- 172.21.15.89:0/2648040632 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2e28005020 con 0x7f2e30108cf0 2024-09-21T03:38:27.074 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:27.065+0000 7f2e37906640 1 -- 172.21.15.89:0/2648040632 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2e301a05d0 con 0x7f2e30108cf0 2024-09-21T03:38:27.074 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:27.065+0000 7f2e227fc640 1 -- 172.21.15.89:0/2648040632 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e280116d0 con 0x7f2e30108cf0 2024-09-21T03:38:27.075 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:27.065+0000 7f2e37906640 1 -- 172.21.15.89:0/2648040632 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2e301a6d30 con 0x7f2e30108cf0 2024-09-21T03:38:27.075 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:27.066+0000 7f2e37906640 1 -- 172.21.15.89:0/2648040632 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f2e30104480 con 0x7f2e30108cf0 2024-09-21T03:38:27.075 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:27.068+0000 7f2e227fc640 1 -- 172.21.15.89:0/2648040632 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 18) ==== 154563+0+0 (secure 0 0 0) 0x7f2e280051d0 con 0x7f2e30108cf0 2024-09-21T03:38:27.075 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:27.069+0000 7f2e227fc640 1 --2- 172.21.15.89:0/2648040632 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f2e1c0b65a0 0x7f2e1c0b8ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:27.075 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:27.069+0000 7f2e227fc640 1 -- 172.21.15.89:0/2648040632 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 9092+0+0 (secure 0 0 0) 0x7f2e280e7cd0 con 0x7f2e30108cf0 2024-09-21T03:38:27.075 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:27.069+0000 7f2e356a2640 1 --2- 172.21.15.89:0/2648040632 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f2e1c0b65a0 0x7f2e1c0b8ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:27.075 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:27.069+0000 7f2e227fc640 1 --2- 172.21.15.89:0/2648040632 >> [v2:172.21.15.114:6800/169452122,v1:172.21.15.114:6801/169452122] conn(0x7f2e1c0c16b0 0x7f2e1c0c3ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:27.075 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:27.069+0000 7f2e227fc640 1 -- 172.21.15.89:0/2648040632 --> [v2:172.21.15.114:6800/169452122,v1:172.21.15.114:6801/169452122] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f2e1c0c4270 con 0x7f2e1c0c16b0 2024-09-21T03:38:27.075 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:27.069+0000 7f2e227fc640 1 -- 172.21.15.89:0/2648040632 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_get_version_reply(handle=1 version=82) ==== 24+0+0 (secure 0 0 0) 0x7f2e280e8120 con 0x7f2e30108cf0 2024-09-21T03:38:27.075 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:27.069+0000 7f2e34ea1640 1 --2- 172.21.15.89:0/2648040632 >> [v2:172.21.15.114:6800/169452122,v1:172.21.15.114:6801/169452122] conn(0x7f2e1c0c16b0 0x7f2e1c0c3ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:27.075 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:27.069+0000 7f2e356a2640 1 --2- 172.21.15.89:0/2648040632 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f2e1c0b65a0 0x7f2e1c0b8ab0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f2e2c0058c0 tx=0x7f2e2c00a2d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:38:27.075 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:27.070+0000 7f2e34ea1640 1 --2- 172.21.15.89:0/2648040632 >> [v2:172.21.15.114:6800/169452122,v1:172.21.15.114:6801/169452122] conn(0x7f2e1c0c16b0 0x7f2e1c0c3ba0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:38:27.075 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:27.071+0000 7f2e227fc640 1 -- 172.21.15.89:0/2648040632 <== osd.2 v2:172.21.15.114:6800/169452122 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f2e1c0c4270 con 0x7f2e1c0c16b0 2024-09-21T03:38:27.086 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:27.085+0000 7f2e37906640 1 -- 172.21.15.89:0/2648040632 --> [v2:172.21.15.114:6800/169452122,v1:172.21.15.114:6801/169452122] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f2e30103ce0 con 0x7f2e1c0c16b0 2024-09-21T03:38:27.086 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:27.085+0000 7f2e227fc640 1 -- 172.21.15.89:0/2648040632 <== osd.2 v2:172.21.15.114:6800/169452122 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f2e30103ce0 con 0x7f2e1c0c16b0 2024-09-21T03:38:27.087 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:27.086+0000 7f2e37906640 1 -- 172.21.15.89:0/2648040632 >> [v2:172.21.15.114:6800/169452122,v1:172.21.15.114:6801/169452122] conn(0x7f2e1c0c16b0 msgr2=0x7f2e1c0c3ba0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:27.087 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:27.086+0000 7f2e37906640 1 --2- 172.21.15.89:0/2648040632 >> [v2:172.21.15.114:6800/169452122,v1:172.21.15.114:6801/169452122] conn(0x7f2e1c0c16b0 0x7f2e1c0c3ba0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:27.087 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:27.086+0000 7f2e37906640 1 -- 172.21.15.89:0/2648040632 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f2e1c0b65a0 msgr2=0x7f2e1c0b8ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:27.087 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:27.086+0000 7f2e37906640 1 --2- 172.21.15.89:0/2648040632 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f2e1c0b65a0 0x7f2e1c0b8ab0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f2e2c0058c0 tx=0x7f2e2c00a2d0 comp rx=0 tx=0).stop 2024-09-21T03:38:27.087 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:27.086+0000 7f2e37906640 1 -- 172.21.15.89:0/2648040632 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2e30108cf0 msgr2=0x7f2e3019fbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:27.087 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:27.086+0000 7f2e37906640 1 --2- 172.21.15.89:0/2648040632 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2e30108cf0 0x7f2e3019fbb0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f2e2800eaa0 tx=0x7f2e2800ef60 comp rx=0 tx=0).stop 2024-09-21T03:38:27.088 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:27.086+0000 7f2e35ea3640 1 -- 172.21.15.89:0/2648040632 reap_dead start 2024-09-21T03:38:27.088 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:27.086+0000 7f2e37906640 1 -- 172.21.15.89:0/2648040632 shutdown_connections 2024-09-21T03:38:27.088 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:27.086+0000 7f2e37906640 1 -- 172.21.15.89:0/2648040632 >> 172.21.15.89:0/2648040632 conn(0x7f2e30079730 msgr2=0x7f2e30101a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:38:27.088 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:27.086+0000 7f2e37906640 1 -- 172.21.15.89:0/2648040632 shutdown_connections 2024-09-21T03:38:27.088 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:27.086+0000 7f2e37906640 1 -- 172.21.15.89:0/2648040632 wait complete. 2024-09-21T03:38:27.098 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/7c374098-77c9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:38:27.711 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:38:27 smithi114 ceph-mon[34140]: pgmap v329: 129 pgs: 129 active+clean; 646 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 170 B/s wr, 6 op/s 2024-09-21T03:38:27.750 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:38:27 smithi110 ceph-mon[34000]: pgmap v329: 129 pgs: 129 active+clean; 646 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 170 B/s wr, 6 op/s 2024-09-21T03:38:27.795 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:38:27 smithi089 ceph-mon[30928]: pgmap v329: 129 pgs: 129 active+clean; 646 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 170 B/s wr, 6 op/s 2024-09-21T03:38:27.822 INFO:teuthology.orchestra.run.smithi089.stdout:150323855417 2024-09-21T03:38:27.823 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 7c374098-77c9-11ef-baf5-efdc52797490 -- ceph osd last-stat-seq osd.5 2024-09-21T03:38:28.172 INFO:teuthology.orchestra.run.smithi089.stdout:73014444106 2024-09-21T03:38:28.172 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 7c374098-77c9-11ef-baf5-efdc52797490 -- ceph osd last-stat-seq osd.2 2024-09-21T03:38:28.669 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:28.656+0000 7f1456d84640 1 -- 172.21.15.89:0/4196966925 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1450108d20 msgr2=0x7f145010f550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:28.669 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:28.656+0000 7f1456d84640 1 --2- 172.21.15.89:0/4196966925 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1450108d20 0x7f145010f550 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f1448009900 tx=0x7f144802a410 comp rx=0 tx=0).stop 2024-09-21T03:38:28.669 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:28.656+0000 7f1456d84640 1 -- 172.21.15.89:0/4196966925 shutdown_connections 2024-09-21T03:38:28.669 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:28.656+0000 7f1456d84640 1 --2- 172.21.15.89:0/4196966925 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1450108d20 0x7f145010f550 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:28.669 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:28.656+0000 7f1456d84640 1 --2- 172.21.15.89:0/4196966925 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f14501083d0 0x7f14501087e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:28.669 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:28.656+0000 7f1456d84640 1 --2- 172.21.15.89:0/4196966925 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1450102ae0 0x7f1450102ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:28.669 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:28.656+0000 7f1456d84640 1 -- 172.21.15.89:0/4196966925 >> 172.21.15.89:0/4196966925 conn(0x7f14500797b0 msgr2=0x7f1450079bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:38:28.669 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:28.656+0000 7f1456d84640 1 -- 172.21.15.89:0/4196966925 shutdown_connections 2024-09-21T03:38:28.670 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:28.656+0000 7f1456d84640 1 -- 172.21.15.89:0/4196966925 wait complete. 2024-09-21T03:38:28.670 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:28.657+0000 7f1456d84640 1 Processor -- start 2024-09-21T03:38:28.670 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:28.657+0000 7f1456d84640 1 -- start start 2024-09-21T03:38:28.670 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:28.657+0000 7f1456d84640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1450102ae0 0x7f14501992a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:28.670 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:28.657+0000 7f1456d84640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f14501083d0 0x7f1450199840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:28.670 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:28.657+0000 7f1456d84640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1450108d20 0x7f145019eae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:28.670 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:28.657+0000 7f1456d84640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f14501128d0 con 0x7f14501083d0 2024-09-21T03:38:28.670 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:28.657+0000 7f1456d84640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f1450112750 con 0x7f1450102ae0 2024-09-21T03:38:28.670 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:28.657+0000 7f1456d84640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f1450112a50 con 0x7f1450108d20 2024-09-21T03:38:28.670 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:28.657+0000 7f144ffff640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f14501083d0 0x7f1450199840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:28.670 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:28.657+0000 7f144ffff640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f14501083d0 0x7f1450199840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:34850/0 (socket says 172.21.15.89:34850) 2024-09-21T03:38:28.670 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:28.657+0000 7f144ffff640 1 -- 172.21.15.89:0/681365648 learned_addr learned my addr 172.21.15.89:0/681365648 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-21T03:38:28.670 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:28.657+0000 7f1455321640 1 --2- 172.21.15.89:0/681365648 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1450108d20 0x7f145019eae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:28.670 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:28.657+0000 7f1454b20640 1 --2- 172.21.15.89:0/681365648 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1450102ae0 0x7f14501992a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:28.670 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:28.657+0000 7f144ffff640 1 -- 172.21.15.89:0/681365648 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1450108d20 msgr2=0x7f145019eae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:28.671 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:28.657+0000 7f144ffff640 1 --2- 172.21.15.89:0/681365648 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1450108d20 0x7f145019eae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:28.671 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:28.657+0000 7f144ffff640 1 -- 172.21.15.89:0/681365648 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1450102ae0 msgr2=0x7f14501992a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:28.671 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:28.657+0000 7f144ffff640 1 --2- 172.21.15.89:0/681365648 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1450102ae0 0x7f14501992a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:28.671 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:28.657+0000 7f144ffff640 1 -- 172.21.15.89:0/681365648 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f145019f210 con 0x7f14501083d0 2024-09-21T03:38:28.671 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:28.657+0000 7f1454b20640 1 --2- 172.21.15.89:0/681365648 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1450102ae0 0x7f14501992a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:38:28.671 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:28.657+0000 7f1455321640 1 --2- 172.21.15.89:0/681365648 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1450108d20 0x7f145019eae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:38:28.671 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:28.658+0000 7f144ffff640 1 --2- 172.21.15.89:0/681365648 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f14501083d0 0x7f1450199840 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f144000df40 tx=0x7f144000df70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:38:28.671 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:28.658+0000 7f144dffb640 1 -- 172.21.15.89:0/681365648 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f144000e980 con 0x7f14501083d0 2024-09-21T03:38:28.671 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:28.658+0000 7f144dffb640 1 -- 172.21.15.89:0/681365648 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1440009520 con 0x7f14501083d0 2024-09-21T03:38:28.671 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:28.658+0000 7f1456d84640 1 -- 172.21.15.89:0/681365648 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f145019f500 con 0x7f14501083d0 2024-09-21T03:38:28.671 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:28.658+0000 7f144dffb640 1 -- 172.21.15.89:0/681365648 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1440016650 con 0x7f14501083d0 2024-09-21T03:38:28.671 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:28.658+0000 7f1456d84640 1 -- 172.21.15.89:0/681365648 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f145006b820 con 0x7f14501083d0 2024-09-21T03:38:28.671 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:28.659+0000 7f1456d84640 1 -- 172.21.15.89:0/681365648 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1450102ef0 con 0x7f14501083d0 2024-09-21T03:38:28.671 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:28.661+0000 7f144dffb640 1 -- 172.21.15.89:0/681365648 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 18) ==== 154563+0+0 (secure 0 0 0) 0x7f144000eb20 con 0x7f14501083d0 2024-09-21T03:38:28.671 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:28.661+0000 7f144dffb640 1 --2- 172.21.15.89:0/681365648 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f143c0b65a0 0x7f143c0b8ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:28.672 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:28.661+0000 7f1454b20640 1 --2- 172.21.15.89:0/681365648 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f143c0b65a0 0x7f143c0b8ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:28.672 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:28.661+0000 7f144dffb640 1 -- 172.21.15.89:0/681365648 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 9092+0+0 (secure 0 0 0) 0x7f14400e8de0 con 0x7f14501083d0 2024-09-21T03:38:28.672 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:28.662+0000 7f1454b20640 1 --2- 172.21.15.89:0/681365648 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f143c0b65a0 0x7f143c0b8ab0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f1438009b50 tx=0x7f1438005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:38:28.672 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:28.666+0000 7f144dffb640 1 -- 172.21.15.89:0/681365648 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f14400af1b0 con 0x7f14501083d0 2024-09-21T03:38:28.851 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:28.848+0000 7f1456d84640 1 -- 172.21.15.89:0/681365648 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7f145006c320 con 0x7f14501083d0 2024-09-21T03:38:28.851 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:28.849+0000 7f144dffb640 1 -- 172.21.15.89:0/681365648 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f14400aa020 con 0x7f14501083d0 2024-09-21T03:38:28.851 INFO:teuthology.orchestra.run.smithi089.stdout:55834574925 2024-09-21T03:38:28.854 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:28.851+0000 7f1456d84640 1 -- 172.21.15.89:0/681365648 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f143c0b65a0 msgr2=0x7f143c0b8ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:28.854 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:28.851+0000 7f1456d84640 1 --2- 172.21.15.89:0/681365648 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f143c0b65a0 0x7f143c0b8ab0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f1438009b50 tx=0x7f1438005f90 comp rx=0 tx=0).stop 2024-09-21T03:38:28.854 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:28.851+0000 7f1456d84640 1 -- 172.21.15.89:0/681365648 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f14501083d0 msgr2=0x7f1450199840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:28.854 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:28.851+0000 7f1456d84640 1 --2- 172.21.15.89:0/681365648 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f14501083d0 0x7f1450199840 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f144000df40 tx=0x7f144000df70 comp rx=0 tx=0).stop 2024-09-21T03:38:28.854 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:28.851+0000 7f1456d84640 1 -- 172.21.15.89:0/681365648 shutdown_connections 2024-09-21T03:38:28.854 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:28.851+0000 7f1456d84640 1 --2- 172.21.15.89:0/681365648 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f143c0b65a0 0x7f143c0b8ab0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:28.854 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:28.851+0000 7f1456d84640 1 --2- 172.21.15.89:0/681365648 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1450108d20 0x7f145019eae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:28.854 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:28.851+0000 7f1456d84640 1 --2- 172.21.15.89:0/681365648 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f14501083d0 0x7f1450199840 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:28.854 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:28.851+0000 7f1456d84640 1 --2- 172.21.15.89:0/681365648 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1450102ae0 0x7f14501992a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:28.854 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:28.851+0000 7f1456d84640 1 -- 172.21.15.89:0/681365648 >> 172.21.15.89:0/681365648 conn(0x7f14500797b0 msgr2=0x7f1450106460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:38:28.854 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:28.852+0000 7f1456d84640 1 -- 172.21.15.89:0/681365648 shutdown_connections 2024-09-21T03:38:28.854 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:28.852+0000 7f1456d84640 1 -- 172.21.15.89:0/681365648 wait complete. 2024-09-21T03:38:28.945 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:28.940+0000 7f7fb944f640 1 -- 172.21.15.89:0/2368482742 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7fb41083a0 msgr2=0x7f7fb41087b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:28.945 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:28.940+0000 7f7fb944f640 1 --2- 172.21.15.89:0/2368482742 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7fb41083a0 0x7f7fb41087b0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f7f9c009860 tx=0x7f7f9c02a340 comp rx=0 tx=0).stop 2024-09-21T03:38:28.945 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:28.940+0000 7f7fb944f640 1 -- 172.21.15.89:0/2368482742 shutdown_connections 2024-09-21T03:38:28.945 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:28.940+0000 7f7fb944f640 1 --2- 172.21.15.89:0/2368482742 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7fb4108cf0 0x7f7fb410f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:28.945 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:28.940+0000 7f7fb944f640 1 --2- 172.21.15.89:0/2368482742 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7fb41083a0 0x7f7fb41087b0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:28.945 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:28.940+0000 7f7fb944f640 1 --2- 172.21.15.89:0/2368482742 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7fb4102a60 0x7f7fb4102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:28.945 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:28.940+0000 7f7fb944f640 1 -- 172.21.15.89:0/2368482742 >> 172.21.15.89:0/2368482742 conn(0x7f7fb4079730 msgr2=0x7f7fb4079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:38:28.945 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:28.940+0000 7f7fb944f640 1 -- 172.21.15.89:0/2368482742 shutdown_connections 2024-09-21T03:38:28.945 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:28.940+0000 7f7fb944f640 1 -- 172.21.15.89:0/2368482742 wait complete. 2024-09-21T03:38:28.945 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:28.941+0000 7f7fb944f640 1 Processor -- start 2024-09-21T03:38:28.946 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:28.941+0000 7f7fb944f640 1 -- start start 2024-09-21T03:38:28.946 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:28.941+0000 7f7fb944f640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7fb4102a60 0x7f7fb41992c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:28.946 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:28.941+0000 7f7fb944f640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7fb41083a0 0x7f7fb4199860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:28.946 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:28.941+0000 7f7fb944f640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7fb4108cf0 0x7f7fb419eb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:28.946 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:28.941+0000 7f7fb944f640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f7fb4112810 con 0x7f7fb4102a60 2024-09-21T03:38:28.946 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:28.941+0000 7f7fb944f640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f7fb4112690 con 0x7f7fb4108cf0 2024-09-21T03:38:28.946 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:28.941+0000 7f7fb944f640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f7fb4112990 con 0x7f7fb41083a0 2024-09-21T03:38:28.946 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:28.941+0000 7f7fb2ffd640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7fb4102a60 0x7f7fb41992c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:28.946 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:28.941+0000 7f7fb2ffd640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7fb4102a60 0x7f7fb41992c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:34872/0 (socket says 172.21.15.89:34872) 2024-09-21T03:38:28.946 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:28.941+0000 7f7fb2ffd640 1 -- 172.21.15.89:0/103521442 learned_addr learned my addr 172.21.15.89:0/103521442 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-21T03:38:28.946 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:28.942+0000 7f7fabfff640 1 --2- 172.21.15.89:0/103521442 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7fb41083a0 0x7f7fb4199860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:28.946 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:28.942+0000 7f7fb37fe640 1 --2- 172.21.15.89:0/103521442 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7fb4108cf0 0x7f7fb419eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:28.946 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:28.942+0000 7f7fb2ffd640 1 -- 172.21.15.89:0/103521442 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7fb41083a0 msgr2=0x7f7fb4199860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:28.946 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:28.942+0000 7f7fb2ffd640 1 --2- 172.21.15.89:0/103521442 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7fb41083a0 0x7f7fb4199860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:28.947 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:28.942+0000 7f7fb2ffd640 1 -- 172.21.15.89:0/103521442 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7fb4108cf0 msgr2=0x7f7fb419eb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:28.947 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:28.942+0000 7f7fb2ffd640 1 --2- 172.21.15.89:0/103521442 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7fb4108cf0 0x7f7fb419eb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:28.947 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:28.942+0000 7f7fb2ffd640 1 -- 172.21.15.89:0/103521442 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7fb419f230 con 0x7f7fb4102a60 2024-09-21T03:38:28.947 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:28.942+0000 7f7fb37fe640 1 --2- 172.21.15.89:0/103521442 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7fb4108cf0 0x7f7fb419eb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:38:28.947 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:28.942+0000 7f7fb2ffd640 1 --2- 172.21.15.89:0/103521442 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7fb4102a60 0x7f7fb41992c0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f7fa400d950 tx=0x7f7fa400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:38:28.947 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:28.942+0000 7f7fb0ff9640 1 -- 172.21.15.89:0/103521442 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fa4017030 con 0x7f7fb4102a60 2024-09-21T03:38:28.947 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:28.942+0000 7f7fb0ff9640 1 -- 172.21.15.89:0/103521442 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7fa4004020 con 0x7f7fb4102a60 2024-09-21T03:38:28.947 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:28.942+0000 7f7fb944f640 1 -- 172.21.15.89:0/103521442 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7fb419f520 con 0x7f7fb4102a60 2024-09-21T03:38:28.947 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:28.942+0000 7f7fb0ff9640 1 -- 172.21.15.89:0/103521442 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fa40055d0 con 0x7f7fb4102a60 2024-09-21T03:38:28.947 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:28.942+0000 7f7fb944f640 1 -- 172.21.15.89:0/103521442 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7fb41a5c80 con 0x7f7fb4102a60 2024-09-21T03:38:28.947 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:28.943+0000 7f7fb944f640 1 -- 172.21.15.89:0/103521442 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f7fb4063b50 con 0x7f7fb4102a60 2024-09-21T03:38:28.948 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:28.946+0000 7f7fb0ff9640 1 -- 172.21.15.89:0/103521442 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 18) ==== 154563+0+0 (secure 0 0 0) 0x7f7fa4009a30 con 0x7f7fb4102a60 2024-09-21T03:38:28.950 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:28.947+0000 7f7fb0ff9640 1 --2- 172.21.15.89:0/103521442 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f7fa00b6620 0x7f7fa00b8b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:28.950 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:28.948+0000 7f7fb0ff9640 1 -- 172.21.15.89:0/103521442 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 9092+0+0 (secure 0 0 0) 0x7f7fa40e78f0 con 0x7f7fb4102a60 2024-09-21T03:38:28.950 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:28.948+0000 7f7fb0ff9640 1 --2- 172.21.15.89:0/103521442 >> [v2:172.21.15.89:6810/3016980877,v1:172.21.15.89:6811/3016980877] conn(0x7f7fa00c1730 0x7f7fa00c3c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:28.950 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:28.948+0000 7f7fb0ff9640 1 -- 172.21.15.89:0/103521442 --> [v2:172.21.15.89:6810/3016980877,v1:172.21.15.89:6811/3016980877] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f7fa4022e50 con 0x7f7fa00c1730 2024-09-21T03:38:28.950 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:28.948+0000 7f7fb0ff9640 1 -- 172.21.15.89:0/103521442 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_get_version_reply(handle=1 version=82) ==== 24+0+0 (secure 0 0 0) 0x7f7fa40e7ca0 con 0x7f7fb4102a60 2024-09-21T03:38:28.950 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:28.948+0000 7f7fb37fe640 1 --2- 172.21.15.89:0/103521442 >> [v2:172.21.15.89:6810/3016980877,v1:172.21.15.89:6811/3016980877] conn(0x7f7fa00c1730 0x7f7fa00c3c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:28.951 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:28.948+0000 7f7fabfff640 1 --2- 172.21.15.89:0/103521442 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f7fa00b6620 0x7f7fa00b8b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:28.952 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:28.949+0000 7f7fb37fe640 1 --2- 172.21.15.89:0/103521442 >> [v2:172.21.15.89:6810/3016980877,v1:172.21.15.89:6811/3016980877] conn(0x7f7fa00c1730 0x7f7fa00c3c20 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:38:28.952 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:28.949+0000 7f7fabfff640 1 --2- 172.21.15.89:0/103521442 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f7fa00b6620 0x7f7fa00b8b30 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f7f9c005940 tx=0x7f7f9c0058b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:38:28.952 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:28.950+0000 7f7fb0ff9640 1 -- 172.21.15.89:0/103521442 <== osd.3 v2:172.21.15.89:6810/3016980877 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f7fa4022e50 con 0x7f7fa00c1730 2024-09-21T03:38:28.957 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/7c374098-77c9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:38:28.980 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:28.978+0000 7f7fb944f640 1 -- 172.21.15.89:0/103521442 --> [v2:172.21.15.89:6810/3016980877,v1:172.21.15.89:6811/3016980877] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f7fb4063f80 con 0x7f7fa00c1730 2024-09-21T03:38:28.981 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:28.978+0000 7f7fb0ff9640 1 -- 172.21.15.89:0/103521442 <== osd.3 v2:172.21.15.89:6810/3016980877 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f7fb4063f80 con 0x7f7fa00c1730 2024-09-21T03:38:28.983 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:28.980+0000 7f7fb944f640 1 -- 172.21.15.89:0/103521442 >> [v2:172.21.15.89:6810/3016980877,v1:172.21.15.89:6811/3016980877] conn(0x7f7fa00c1730 msgr2=0x7f7fa00c3c20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:28.983 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:28.980+0000 7f7fb944f640 1 --2- 172.21.15.89:0/103521442 >> [v2:172.21.15.89:6810/3016980877,v1:172.21.15.89:6811/3016980877] conn(0x7f7fa00c1730 0x7f7fa00c3c20 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:28.983 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:28.981+0000 7f7fb944f640 1 -- 172.21.15.89:0/103521442 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f7fa00b6620 msgr2=0x7f7fa00b8b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:28.983 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:28.981+0000 7f7fb944f640 1 --2- 172.21.15.89:0/103521442 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f7fa00b6620 0x7f7fa00b8b30 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f7f9c005940 tx=0x7f7f9c0058b0 comp rx=0 tx=0).stop 2024-09-21T03:38:28.983 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:28.981+0000 7f7fb944f640 1 -- 172.21.15.89:0/103521442 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7fb4102a60 msgr2=0x7f7fb41992c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:28.983 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:28.981+0000 7f7fb944f640 1 --2- 172.21.15.89:0/103521442 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7fb4102a60 0x7f7fb41992c0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f7fa400d950 tx=0x7f7fa400de10 comp rx=0 tx=0).stop 2024-09-21T03:38:28.984 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:28.981+0000 7f7fb37fe640 1 -- 172.21.15.89:0/103521442 reap_dead start 2024-09-21T03:38:28.985 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:28.982+0000 7f7fb944f640 1 -- 172.21.15.89:0/103521442 shutdown_connections 2024-09-21T03:38:28.985 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:28.982+0000 7f7fb944f640 1 -- 172.21.15.89:0/103521442 >> 172.21.15.89:0/103521442 conn(0x7f7fb4079730 msgr2=0x7f7fb41044d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:38:28.985 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:28.983+0000 7f7fb944f640 1 -- 172.21.15.89:0/103521442 shutdown_connections 2024-09-21T03:38:28.985 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:28.983+0000 7f7fb944f640 1 -- 172.21.15.89:0/103521442 wait complete. 2024-09-21T03:38:29.561 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:38:29 smithi089 ceph-mon[30928]: pgmap v330: 129 pgs: 129 active+clean; 646 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-21T03:38:29.561 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:38:29 smithi089 ceph-mon[30928]: from='client.? 172.21.15.89:0/681365648' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 1} : dispatch 2024-09-21T03:38:29.664 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834574923 got 55834574925 for osd.1 2024-09-21T03:38:29.664 DEBUG:teuthology.parallel:result is None 2024-09-21T03:38:29.711 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:38:29 smithi114 ceph-mon[34140]: pgmap v330: 129 pgs: 129 active+clean; 646 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-21T03:38:29.711 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:38:29 smithi114 ceph-mon[34140]: from='client.? 172.21.15.89:0/681365648' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 1} : dispatch 2024-09-21T03:38:29.750 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:38:29 smithi110 ceph-mon[34000]: pgmap v330: 129 pgs: 129 active+clean; 646 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-21T03:38:29.750 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:38:29 smithi110 ceph-mon[34000]: from='client.? 172.21.15.89:0/681365648' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 1} : dispatch 2024-09-21T03:38:29.949 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/7c374098-77c9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:38:29.949 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/7c374098-77c9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:38:30.352 INFO:teuthology.orchestra.run.smithi089.stdout:103079215171 2024-09-21T03:38:30.352 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 7c374098-77c9-11ef-baf5-efdc52797490 -- ceph osd last-stat-seq osd.3 2024-09-21T03:38:30.597 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/7c374098-77c9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:38:30.597 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/7c374098-77c9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:38:30.599 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/7c374098-77c9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:38:30.599 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/7c374098-77c9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:38:30.601 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/7c374098-77c9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:38:31.136 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:31.134+0000 7ffa0665f640 1 -- 172.21.15.89:0/3713553130 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ffa00103b60 msgr2=0x7ffa00103f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:31.136 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:31.134+0000 7ffa0665f640 1 --2- 172.21.15.89:0/3713553130 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ffa00103b60 0x7ffa00103f70 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7ff9f0009900 tx=0x7ff9f002a410 comp rx=0 tx=0).stop 2024-09-21T03:38:31.136 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:31.134+0000 7ffa0665f640 1 -- 172.21.15.89:0/3713553130 shutdown_connections 2024-09-21T03:38:31.136 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:31.134+0000 7ffa0665f640 1 --2- 172.21.15.89:0/3713553130 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ffa001094a0 0x7ffa001098b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:31.136 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:31.134+0000 7ffa0665f640 1 --2- 172.21.15.89:0/3713553130 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ffa00109df0 0x7ffa00110620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:31.136 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:31.134+0000 7ffa0665f640 1 --2- 172.21.15.89:0/3713553130 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ffa00103b60 0x7ffa00103f70 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:31.137 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:31.134+0000 7ffa0665f640 1 -- 172.21.15.89:0/3713553130 >> 172.21.15.89:0/3713553130 conn(0x7ffa0007b850 msgr2=0x7ffa0007bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:38:31.137 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:31.134+0000 7ffa0665f640 1 -- 172.21.15.89:0/3713553130 shutdown_connections 2024-09-21T03:38:31.137 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:31.134+0000 7ffa0665f640 1 -- 172.21.15.89:0/3713553130 wait complete. 2024-09-21T03:38:31.138 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:31.135+0000 7ffa0665f640 1 Processor -- start 2024-09-21T03:38:31.138 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:31.135+0000 7ffa0665f640 1 -- start start 2024-09-21T03:38:31.138 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:31.135+0000 7ffa0665f640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ffa00103b60 0x7ffa0019a3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:31.138 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:31.135+0000 7ffa0665f640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ffa001094a0 0x7ffa0019a970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:31.138 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:31.135+0000 7ffa0665f640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ffa00109df0 0x7ffa0019fc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:31.138 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:31.135+0000 7ffa0665f640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7ffa00113960 con 0x7ffa00103b60 2024-09-21T03:38:31.138 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:31.135+0000 7ffa0665f640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7ffa001137e0 con 0x7ffa00109df0 2024-09-21T03:38:31.139 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:31.135+0000 7ffa0665f640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7ffa00113ae0 con 0x7ffa001094a0 2024-09-21T03:38:31.139 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:31.135+0000 7ff9fffff640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ffa00103b60 0x7ffa0019a3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:31.139 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:31.135+0000 7ff9fffff640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ffa00103b60 0x7ffa0019a3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:34886/0 (socket says 172.21.15.89:34886) 2024-09-21T03:38:31.139 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:31.135+0000 7ff9fffff640 1 -- 172.21.15.89:0/2561893227 learned_addr learned my addr 172.21.15.89:0/2561893227 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-21T03:38:31.139 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:31.135+0000 7ff9f7fff640 1 --2- 172.21.15.89:0/2561893227 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ffa001094a0 0x7ffa0019a970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:31.139 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:31.135+0000 7ffa04bfc640 1 --2- 172.21.15.89:0/2561893227 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ffa00109df0 0x7ffa0019fc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:31.139 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:31.135+0000 7ff9fffff640 1 -- 172.21.15.89:0/2561893227 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ffa001094a0 msgr2=0x7ffa0019a970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:31.139 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:31.135+0000 7ff9fffff640 1 --2- 172.21.15.89:0/2561893227 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ffa001094a0 0x7ffa0019a970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:31.139 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:31.135+0000 7ff9fffff640 1 -- 172.21.15.89:0/2561893227 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ffa00109df0 msgr2=0x7ffa0019fc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:31.139 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:31.135+0000 7ff9fffff640 1 --2- 172.21.15.89:0/2561893227 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ffa00109df0 0x7ffa0019fc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:31.139 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:31.135+0000 7ff9fffff640 1 -- 172.21.15.89:0/2561893227 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa001a0340 con 0x7ffa00103b60 2024-09-21T03:38:31.139 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:31.135+0000 7ffa04bfc640 1 --2- 172.21.15.89:0/2561893227 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ffa00109df0 0x7ffa0019fc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:38:31.139 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:31.136+0000 7ff9fffff640 1 --2- 172.21.15.89:0/2561893227 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ffa00103b60 0x7ffa0019a3d0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7ff9f0009230 tx=0x7ff9f0004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:38:31.139 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:31.136+0000 7ff9fdffb640 1 -- 172.21.15.89:0/2561893227 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9f003f030 con 0x7ffa00103b60 2024-09-21T03:38:31.139 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:31.136+0000 7ff9fdffb640 1 -- 172.21.15.89:0/2561893227 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff9f0003a80 con 0x7ffa00103b60 2024-09-21T03:38:31.139 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:31.136+0000 7ffa0665f640 1 -- 172.21.15.89:0/2561893227 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa001a05d0 con 0x7ffa00103b60 2024-09-21T03:38:31.140 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:31.136+0000 7ff9fdffb640 1 -- 172.21.15.89:0/2561893227 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9f0035070 con 0x7ffa00103b60 2024-09-21T03:38:31.140 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:31.136+0000 7ffa0665f640 1 -- 172.21.15.89:0/2561893227 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa001a6d90 con 0x7ffa00103b60 2024-09-21T03:38:31.140 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:31.137+0000 7ffa0665f640 1 -- 172.21.15.89:0/2561893227 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffa00103f70 con 0x7ffa00103b60 2024-09-21T03:38:31.140 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:31.138+0000 7ff9fdffb640 1 -- 172.21.15.89:0/2561893227 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 18) ==== 154563+0+0 (secure 0 0 0) 0x7ff9f0042030 con 0x7ffa00103b60 2024-09-21T03:38:31.141 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:31.139+0000 7ff9fdffb640 1 --2- 172.21.15.89:0/2561893227 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7ff9ec0b65a0 0x7ff9ec0b8ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:31.141 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:31.139+0000 7ff9f7fff640 1 --2- 172.21.15.89:0/2561893227 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7ff9ec0b65a0 0x7ff9ec0b8ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:31.141 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:31.139+0000 7ff9fdffb640 1 -- 172.21.15.89:0/2561893227 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 9092+0+0 (secure 0 0 0) 0x7ff9f0108380 con 0x7ffa00103b60 2024-09-21T03:38:31.142 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:31.139+0000 7ff9f7fff640 1 --2- 172.21.15.89:0/2561893227 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7ff9ec0b65a0 0x7ff9ec0b8ab0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7ffa001065a0 tx=0x7ff9e800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:38:31.145 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:31.143+0000 7ff9fdffb640 1 -- 172.21.15.89:0/2561893227 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7ff9f00ce750 con 0x7ffa00103b60 2024-09-21T03:38:31.328 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:31.325+0000 7ffa0665f640 1 -- 172.21.15.89:0/2561893227 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7ffa0019b500 con 0x7ffa00103b60 2024-09-21T03:38:31.328 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:31.326+0000 7ff9fdffb640 1 -- 172.21.15.89:0/2561893227 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7ff9f00d2b30 con 0x7ffa00103b60 2024-09-21T03:38:31.328 INFO:teuthology.orchestra.run.smithi089.stdout:128849018943 2024-09-21T03:38:31.330 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:31.328+0000 7ffa0665f640 1 -- 172.21.15.89:0/2561893227 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7ff9ec0b65a0 msgr2=0x7ff9ec0b8ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:31.330 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:31.328+0000 7ffa0665f640 1 --2- 172.21.15.89:0/2561893227 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7ff9ec0b65a0 0x7ff9ec0b8ab0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7ffa001065a0 tx=0x7ff9e800b410 comp rx=0 tx=0).stop 2024-09-21T03:38:31.331 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:31.328+0000 7ffa0665f640 1 -- 172.21.15.89:0/2561893227 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ffa00103b60 msgr2=0x7ffa0019a3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:31.331 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:31.328+0000 7ffa0665f640 1 --2- 172.21.15.89:0/2561893227 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ffa00103b60 0x7ffa0019a3d0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7ff9f0009230 tx=0x7ff9f0004270 comp rx=0 tx=0).stop 2024-09-21T03:38:31.331 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:31.328+0000 7ffa0665f640 1 -- 172.21.15.89:0/2561893227 shutdown_connections 2024-09-21T03:38:31.331 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:31.328+0000 7ffa0665f640 1 --2- 172.21.15.89:0/2561893227 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7ff9ec0b65a0 0x7ff9ec0b8ab0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:31.331 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:31.328+0000 7ffa0665f640 1 --2- 172.21.15.89:0/2561893227 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ffa00109df0 0x7ffa0019fc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:31.331 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:31.328+0000 7ffa0665f640 1 --2- 172.21.15.89:0/2561893227 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ffa001094a0 0x7ffa0019a970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:31.331 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:31.328+0000 7ffa0665f640 1 --2- 172.21.15.89:0/2561893227 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ffa00103b60 0x7ffa0019a3d0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:31.331 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:31.328+0000 7ffa0665f640 1 -- 172.21.15.89:0/2561893227 >> 172.21.15.89:0/2561893227 conn(0x7ffa0007b850 msgr2=0x7ffa001075e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:38:31.331 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:31.328+0000 7ffa0665f640 1 -- 172.21.15.89:0/2561893227 shutdown_connections 2024-09-21T03:38:31.331 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:31.328+0000 7ffa0665f640 1 -- 172.21.15.89:0/2561893227 wait complete. 2024-09-21T03:38:31.561 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:38:31 smithi089 ceph-mon[30928]: pgmap v331: 129 pgs: 129 active+clean; 646 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-09-21T03:38:31.561 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:38:31 smithi089 ceph-mon[30928]: from='client.? 172.21.15.89:0/2561893227' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 4} : dispatch 2024-09-21T03:38:31.711 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:38:31 smithi114 ceph-mon[34140]: pgmap v331: 129 pgs: 129 active+clean; 646 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-09-21T03:38:31.711 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:38:31 smithi114 ceph-mon[34140]: from='client.? 172.21.15.89:0/2561893227' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 4} : dispatch 2024-09-21T03:38:31.750 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:38:31 smithi110 ceph-mon[34000]: pgmap v331: 129 pgs: 129 active+clean; 646 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-09-21T03:38:31.750 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:38:31 smithi110 ceph-mon[34000]: from='client.? 172.21.15.89:0/2561893227' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 4} : dispatch 2024-09-21T03:38:31.818 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:31.806+0000 7f6f8bd58640 1 -- 172.21.15.89:0/3443123362 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6f8410de90 msgr2=0x7f6f84110380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:31.818 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:31.806+0000 7f6f8bd58640 1 --2- 172.21.15.89:0/3443123362 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6f8410de90 0x7f6f84110380 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f6f7c009900 tx=0x7f6f7c02a3f0 comp rx=0 tx=0).stop 2024-09-21T03:38:31.818 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:31.807+0000 7f6f8bd58640 1 -- 172.21.15.89:0/3443123362 shutdown_connections 2024-09-21T03:38:31.818 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:31.807+0000 7f6f8bd58640 1 --2- 172.21.15.89:0/3443123362 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6f84105310 0x7f6f8410d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:31.819 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:31.807+0000 7f6f8bd58640 1 --2- 172.21.15.89:0/3443123362 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6f8410de90 0x7f6f84110380 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:31.819 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:31.807+0000 7f6f8bd58640 1 --2- 172.21.15.89:0/3443123362 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6f841048e0 0x7f6f84104cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:31.819 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:31.807+0000 7f6f8bd58640 1 -- 172.21.15.89:0/3443123362 >> 172.21.15.89:0/3443123362 conn(0x7f6f84079770 msgr2=0x7f6f84079b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:38:31.819 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:31.807+0000 7f6f8bd58640 1 -- 172.21.15.89:0/3443123362 shutdown_connections 2024-09-21T03:38:31.819 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:31.807+0000 7f6f8bd58640 1 -- 172.21.15.89:0/3443123362 wait complete. 2024-09-21T03:38:31.819 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:31.808+0000 7f6f8bd58640 1 Processor -- start 2024-09-21T03:38:31.819 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:31.808+0000 7f6f8bd58640 1 -- start start 2024-09-21T03:38:31.819 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:31.808+0000 7f6f8bd58640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6f841048e0 0x7f6f8419a330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:31.819 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:31.808+0000 7f6f8bd58640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6f84105310 0x7f6f8419a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:31.819 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:31.808+0000 7f6f8bd58640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6f8410de90 0x7f6f8419fb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:31.819 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:31.808+0000 7f6f8bd58640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f6f841139a0 con 0x7f6f8410de90 2024-09-21T03:38:31.819 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:31.808+0000 7f6f8bd58640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f6f84113820 con 0x7f6f84105310 2024-09-21T03:38:31.819 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:31.808+0000 7f6f8bd58640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f6f84113b20 con 0x7f6f841048e0 2024-09-21T03:38:31.819 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:31.808+0000 7f6f8a2f5640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6f8410de90 0x7f6f8419fb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:31.820 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:31.808+0000 7f6f8a2f5640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6f8410de90 0x7f6f8419fb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:34908/0 (socket says 172.21.15.89:34908) 2024-09-21T03:38:31.820 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:31.808+0000 7f6f8a2f5640 1 -- 172.21.15.89:0/3795943835 learned_addr learned my addr 172.21.15.89:0/3795943835 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-21T03:38:31.820 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:31.808+0000 7f6f89af4640 1 --2- 172.21.15.89:0/3795943835 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6f841048e0 0x7f6f8419a330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:31.820 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:31.808+0000 7f6f892f3640 1 --2- 172.21.15.89:0/3795943835 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6f84105310 0x7f6f8419a8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:31.820 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:31.808+0000 7f6f8a2f5640 1 -- 172.21.15.89:0/3795943835 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6f841048e0 msgr2=0x7f6f8419a330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:31.820 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:31.808+0000 7f6f8a2f5640 1 --2- 172.21.15.89:0/3795943835 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6f841048e0 0x7f6f8419a330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:31.820 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:31.808+0000 7f6f8a2f5640 1 -- 172.21.15.89:0/3795943835 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6f84105310 msgr2=0x7f6f8419a8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:31.820 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:31.808+0000 7f6f8a2f5640 1 --2- 172.21.15.89:0/3795943835 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6f84105310 0x7f6f8419a8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:31.820 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:31.808+0000 7f6f8a2f5640 1 -- 172.21.15.89:0/3795943835 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f841a01e0 con 0x7f6f8410de90 2024-09-21T03:38:31.820 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:31.808+0000 7f6f892f3640 1 --2- 172.21.15.89:0/3795943835 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6f84105310 0x7f6f8419a8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:38:31.820 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:31.809+0000 7f6f8a2f5640 1 --2- 172.21.15.89:0/3795943835 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6f8410de90 0x7f6f8419fb70 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f6f7c005ab0 tx=0x7f6f7c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:38:31.820 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:31.809+0000 7f6f82ffd640 1 -- 172.21.15.89:0/3795943835 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f7c03f030 con 0x7f6f8410de90 2024-09-21T03:38:31.820 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:31.809+0000 7f6f82ffd640 1 -- 172.21.15.89:0/3795943835 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6f7c003c90 con 0x7f6f8410de90 2024-09-21T03:38:31.820 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:31.809+0000 7f6f8bd58640 1 -- 172.21.15.89:0/3795943835 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6f841a0470 con 0x7f6f8410de90 2024-09-21T03:38:31.821 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:31.809+0000 7f6f82ffd640 1 -- 172.21.15.89:0/3795943835 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f7c035070 con 0x7f6f8410de90 2024-09-21T03:38:31.821 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:31.809+0000 7f6f8bd58640 1 -- 172.21.15.89:0/3795943835 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6f841a0950 con 0x7f6f8410de90 2024-09-21T03:38:31.821 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:31.810+0000 7f6f8bd58640 1 -- 172.21.15.89:0/3795943835 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6f8406c1e0 con 0x7f6f8410de90 2024-09-21T03:38:31.821 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:31.811+0000 7f6f82ffd640 1 -- 172.21.15.89:0/3795943835 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 18) ==== 154563+0+0 (secure 0 0 0) 0x7f6f7c042030 con 0x7f6f8410de90 2024-09-21T03:38:31.821 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:31.812+0000 7f6f82ffd640 1 --2- 172.21.15.89:0/3795943835 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f6f6c0b65d0 0x7f6f6c0b8ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:31.821 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:31.812+0000 7f6f89af4640 1 --2- 172.21.15.89:0/3795943835 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f6f6c0b65d0 0x7f6f6c0b8ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:31.821 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:31.812+0000 7f6f82ffd640 1 -- 172.21.15.89:0/3795943835 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 9092+0+0 (secure 0 0 0) 0x7f6f7c107e50 con 0x7f6f8410de90 2024-09-21T03:38:31.821 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:31.812+0000 7f6f89af4640 1 --2- 172.21.15.89:0/3795943835 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f6f6c0b65d0 0x7f6f6c0b8ae0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f6f74007350 tx=0x7f6f740072e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:38:31.821 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:31.816+0000 7f6f82ffd640 1 -- 172.21.15.89:0/3795943835 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f6f7c0ce220 con 0x7f6f8410de90 2024-09-21T03:38:32.005 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:32.002+0000 7f6f8bd58640 1 -- 172.21.15.89:0/3795943835 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 10} v 0) -- 0x7f6f8419b460 con 0x7f6f8410de90 2024-09-21T03:38:32.005 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:32.003+0000 7f6f82ffd640 1 -- 172.21.15.89:0/3795943835 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 10}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7f6f8419b460 con 0x7f6f8410de90 2024-09-21T03:38:32.005 INFO:teuthology.orchestra.run.smithi089.stdout:270582939682 2024-09-21T03:38:32.007 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:32.005+0000 7f6f8bd58640 1 -- 172.21.15.89:0/3795943835 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f6f6c0b65d0 msgr2=0x7f6f6c0b8ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:32.007 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:32.005+0000 7f6f8bd58640 1 --2- 172.21.15.89:0/3795943835 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f6f6c0b65d0 0x7f6f6c0b8ae0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f6f74007350 tx=0x7f6f740072e0 comp rx=0 tx=0).stop 2024-09-21T03:38:32.007 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:32.005+0000 7f6f8bd58640 1 -- 172.21.15.89:0/3795943835 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6f8410de90 msgr2=0x7f6f8419fb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:32.008 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:32.005+0000 7f6f8bd58640 1 --2- 172.21.15.89:0/3795943835 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6f8410de90 0x7f6f8419fb70 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f6f7c005ab0 tx=0x7f6f7c0042f0 comp rx=0 tx=0).stop 2024-09-21T03:38:32.008 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:32.005+0000 7f6f8bd58640 1 -- 172.21.15.89:0/3795943835 shutdown_connections 2024-09-21T03:38:32.008 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:32.005+0000 7f6f8bd58640 1 --2- 172.21.15.89:0/3795943835 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f6f6c0b65d0 0x7f6f6c0b8ae0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:32.008 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:32.005+0000 7f6f8bd58640 1 --2- 172.21.15.89:0/3795943835 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6f84105310 0x7f6f8419a8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:32.008 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:32.005+0000 7f6f8bd58640 1 --2- 172.21.15.89:0/3795943835 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6f8410de90 0x7f6f8419fb70 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:32.008 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:32.005+0000 7f6f8bd58640 1 --2- 172.21.15.89:0/3795943835 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6f841048e0 0x7f6f8419a330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:32.008 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:32.005+0000 7f6f8bd58640 1 -- 172.21.15.89:0/3795943835 >> 172.21.15.89:0/3795943835 conn(0x7f6f84079770 msgr2=0x7f6f841079a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:38:32.008 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:32.006+0000 7f6f8bd58640 1 -- 172.21.15.89:0/3795943835 shutdown_connections 2024-09-21T03:38:32.008 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:32.006+0000 7f6f8bd58640 1 -- 172.21.15.89:0/3795943835 wait complete. 2024-09-21T03:38:32.092 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:32.089+0000 7fdb4f873640 1 -- 172.21.15.89:0/1053409936 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdb48108da0 msgr2=0x7fdb4810f5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:32.092 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:32.089+0000 7fdb4f873640 1 --2- 172.21.15.89:0/1053409936 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdb48108da0 0x7fdb4810f5d0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fdb40009ca0 tx=0x7fdb4002a8c0 comp rx=0 tx=0).stop 2024-09-21T03:38:32.092 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:32.089+0000 7fdb4f873640 1 -- 172.21.15.89:0/1053409936 shutdown_connections 2024-09-21T03:38:32.092 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:32.089+0000 7fdb4f873640 1 --2- 172.21.15.89:0/1053409936 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdb48108da0 0x7fdb4810f5d0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:32.092 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:32.089+0000 7fdb4f873640 1 --2- 172.21.15.89:0/1053409936 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fdb48108450 0x7fdb48108860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:32.092 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:32.089+0000 7fdb4f873640 1 --2- 172.21.15.89:0/1053409936 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdb48102b10 0x7fdb48102f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:32.092 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:32.089+0000 7fdb4f873640 1 -- 172.21.15.89:0/1053409936 >> 172.21.15.89:0/1053409936 conn(0x7fdb480796d0 msgr2=0x7fdb48079ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:38:32.092 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:32.090+0000 7fdb4f873640 1 -- 172.21.15.89:0/1053409936 shutdown_connections 2024-09-21T03:38:32.092 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:32.090+0000 7fdb4f873640 1 -- 172.21.15.89:0/1053409936 wait complete. 2024-09-21T03:38:32.093 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:32.090+0000 7fdb4f873640 1 Processor -- start 2024-09-21T03:38:32.094 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:32.090+0000 7fdb4f873640 1 -- start start 2024-09-21T03:38:32.094 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:32.091+0000 7fdb4f873640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fdb48102b10 0x7fdb4819b1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:32.094 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:32.091+0000 7fdb4f873640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdb48108450 0x7fdb4819b6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:32.094 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:32.091+0000 7fdb4f873640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdb48108da0 0x7fdb4819bc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:32.094 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:32.091+0000 7fdb4f873640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fdb48112960 con 0x7fdb48108da0 2024-09-21T03:38:32.094 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:32.091+0000 7fdb4f873640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fdb481127e0 con 0x7fdb48108450 2024-09-21T03:38:32.094 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:32.091+0000 7fdb4f873640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fdb48112ae0 con 0x7fdb48102b10 2024-09-21T03:38:32.094 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:32.091+0000 7fdb4de10640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdb48108da0 0x7fdb4819bc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:32.094 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:32.091+0000 7fdb4de10640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdb48108da0 0x7fdb4819bc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:34924/0 (socket says 172.21.15.89:34924) 2024-09-21T03:38:32.094 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:32.091+0000 7fdb4de10640 1 -- 172.21.15.89:0/2779448220 learned_addr learned my addr 172.21.15.89:0/2779448220 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-21T03:38:32.094 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:32.091+0000 7fdb4d60f640 1 --2- 172.21.15.89:0/2779448220 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fdb48102b10 0x7fdb4819b1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:32.094 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:32.091+0000 7fdb4ce0e640 1 --2- 172.21.15.89:0/2779448220 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdb48108450 0x7fdb4819b6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:32.094 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:32.091+0000 7fdb4de10640 1 -- 172.21.15.89:0/2779448220 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fdb48102b10 msgr2=0x7fdb4819b1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:32.094 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:32.091+0000 7fdb4de10640 1 --2- 172.21.15.89:0/2779448220 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fdb48102b10 0x7fdb4819b1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:32.094 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:32.091+0000 7fdb4de10640 1 -- 172.21.15.89:0/2779448220 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdb48108450 msgr2=0x7fdb4819b6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:32.095 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:32.091+0000 7fdb4de10640 1 --2- 172.21.15.89:0/2779448220 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdb48108450 0x7fdb4819b6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:32.095 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:32.091+0000 7fdb4de10640 1 -- 172.21.15.89:0/2779448220 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb481a64c0 con 0x7fdb48108da0 2024-09-21T03:38:32.095 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:32.092+0000 7fdb4de10640 1 --2- 172.21.15.89:0/2779448220 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdb48108da0 0x7fdb4819bc30 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fdb40005e30 tx=0x7fdb400093d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:38:32.095 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:32.092+0000 7fdb3e7fc640 1 -- 172.21.15.89:0/2779448220 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb40032d10 con 0x7fdb48108da0 2024-09-21T03:38:32.095 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:32.092+0000 7fdb3e7fc640 1 -- 172.21.15.89:0/2779448220 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fdb40032eb0 con 0x7fdb48108da0 2024-09-21T03:38:32.096 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:32.092+0000 7fdb3e7fc640 1 -- 172.21.15.89:0/2779448220 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb40030b20 con 0x7fdb48108da0 2024-09-21T03:38:32.096 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:32.092+0000 7fdb4f873640 1 -- 172.21.15.89:0/2779448220 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdb481a6750 con 0x7fdb48108da0 2024-09-21T03:38:32.096 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:32.092+0000 7fdb4f873640 1 -- 172.21.15.89:0/2779448220 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdb481a6c90 con 0x7fdb48108da0 2024-09-21T03:38:32.096 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:32.093+0000 7fdb4f873640 1 -- 172.21.15.89:0/2779448220 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdb48102f20 con 0x7fdb48108da0 2024-09-21T03:38:32.096 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:32.094+0000 7fdb3e7fc640 1 -- 172.21.15.89:0/2779448220 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 18) ==== 154563+0+0 (secure 0 0 0) 0x7fdb40030cc0 con 0x7fdb48108da0 2024-09-21T03:38:32.098 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:32.096+0000 7fdb3e7fc640 1 --2- 172.21.15.89:0/2779448220 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7fdb340b65a0 0x7fdb340b8ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:32.098 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:32.096+0000 7fdb4d60f640 1 --2- 172.21.15.89:0/2779448220 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7fdb340b65a0 0x7fdb340b8ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:32.098 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:32.096+0000 7fdb3e7fc640 1 -- 172.21.15.89:0/2779448220 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 9092+0+0 (secure 0 0 0) 0x7fdb40108b60 con 0x7fdb48108da0 2024-09-21T03:38:32.098 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:32.096+0000 7fdb4d60f640 1 --2- 172.21.15.89:0/2779448220 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7fdb340b65a0 0x7fdb340b8ab0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fdb4806c720 tx=0x7fdb4400b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:38:32.102 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:32.099+0000 7fdb3e7fc640 1 -- 172.21.15.89:0/2779448220 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fdb400ceed0 con 0x7fdb48108da0 2024-09-21T03:38:32.282 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:32.280+0000 7fdb4f873640 1 -- 172.21.15.89:0/2779448220 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7fdb4806bfd0 con 0x7fdb48108da0 2024-09-21T03:38:32.283 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:32.281+0000 7fdb3e7fc640 1 -- 172.21.15.89:0/2779448220 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fdb400d32b0 con 0x7fdb48108da0 2024-09-21T03:38:32.283 INFO:teuthology.orchestra.run.smithi089.stdout:34359738452 2024-09-21T03:38:32.286 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:32.283+0000 7fdb4f873640 1 -- 172.21.15.89:0/2779448220 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7fdb340b65a0 msgr2=0x7fdb340b8ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:32.286 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:32.283+0000 7fdb4f873640 1 --2- 172.21.15.89:0/2779448220 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7fdb340b65a0 0x7fdb340b8ab0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fdb4806c720 tx=0x7fdb4400b390 comp rx=0 tx=0).stop 2024-09-21T03:38:32.286 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:32.283+0000 7fdb4f873640 1 -- 172.21.15.89:0/2779448220 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdb48108da0 msgr2=0x7fdb4819bc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:32.286 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:32.283+0000 7fdb4f873640 1 --2- 172.21.15.89:0/2779448220 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdb48108da0 0x7fdb4819bc30 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fdb40005e30 tx=0x7fdb400093d0 comp rx=0 tx=0).stop 2024-09-21T03:38:32.286 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:32.283+0000 7fdb4f873640 1 -- 172.21.15.89:0/2779448220 shutdown_connections 2024-09-21T03:38:32.286 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:32.283+0000 7fdb4f873640 1 --2- 172.21.15.89:0/2779448220 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7fdb340b65a0 0x7fdb340b8ab0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:32.286 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:32.283+0000 7fdb4f873640 1 --2- 172.21.15.89:0/2779448220 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdb48108450 0x7fdb4819b6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:32.286 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:32.283+0000 7fdb4f873640 1 --2- 172.21.15.89:0/2779448220 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdb48108da0 0x7fdb4819bc30 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:32.286 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:32.283+0000 7fdb4f873640 1 --2- 172.21.15.89:0/2779448220 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fdb48102b10 0x7fdb4819b1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:32.286 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:32.283+0000 7fdb4f873640 1 -- 172.21.15.89:0/2779448220 >> 172.21.15.89:0/2779448220 conn(0x7fdb480796d0 msgr2=0x7fdb48106460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:38:32.287 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:32.283+0000 7fdb4f873640 1 -- 172.21.15.89:0/2779448220 shutdown_connections 2024-09-21T03:38:32.287 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:32.283+0000 7fdb4f873640 1 -- 172.21.15.89:0/2779448220 wait complete. 2024-09-21T03:38:32.561 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:38:32 smithi089 ceph-mon[30928]: from='client.? 172.21.15.89:0/3795943835' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 10} : dispatch 2024-09-21T03:38:32.561 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:38:32 smithi089 ceph-mon[30928]: from='client.? 172.21.15.89:0/2779448220' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 0} : dispatch 2024-09-21T03:38:32.711 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:38:32 smithi114 ceph-mon[34140]: from='client.? 172.21.15.89:0/3795943835' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 10} : dispatch 2024-09-21T03:38:32.711 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:38:32 smithi114 ceph-mon[34140]: from='client.? 172.21.15.89:0/2779448220' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 0} : dispatch 2024-09-21T03:38:32.750 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:38:32 smithi110 ceph-mon[34000]: from='client.? 172.21.15.89:0/3795943835' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 10} : dispatch 2024-09-21T03:38:32.750 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:38:32 smithi110 ceph-mon[34000]: from='client.? 172.21.15.89:0/2779448220' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 0} : dispatch 2024-09-21T03:38:33.711 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:38:33 smithi114 ceph-mon[34140]: pgmap v332: 129 pgs: 129 active+clean; 646 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-21T03:38:33.744 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:38:33 smithi089 ceph-mon[30928]: pgmap v332: 129 pgs: 129 active+clean; 646 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-21T03:38:33.750 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:38:33 smithi110 ceph-mon[34000]: pgmap v332: 129 pgs: 129 active+clean; 646 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-21T03:38:33.771 INFO:tasks.cephadm.ceph_manager.ceph:need seq 128849018941 got 128849018943 for osd.4 2024-09-21T03:38:33.771 DEBUG:teuthology.parallel:result is None 2024-09-21T03:38:33.961 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:33.959+0000 7f30d2597640 1 -- 172.21.15.89:0/533941969 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f30cc109df0 msgr2=0x7f30cc110620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:33.961 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:33.959+0000 7f30d2597640 1 --2- 172.21.15.89:0/533941969 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f30cc109df0 0x7f30cc110620 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f30c4009ca0 tx=0x7f30c402a8c0 comp rx=0 tx=0).stop 2024-09-21T03:38:33.961 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:33.959+0000 7f30d2597640 1 -- 172.21.15.89:0/533941969 shutdown_connections 2024-09-21T03:38:33.961 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:33.959+0000 7f30d2597640 1 --2- 172.21.15.89:0/533941969 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f30cc1094a0 0x7f30cc1098b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:33.962 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:33.959+0000 7f30d2597640 1 --2- 172.21.15.89:0/533941969 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f30cc109df0 0x7f30cc110620 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:33.962 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:33.959+0000 7f30d2597640 1 --2- 172.21.15.89:0/533941969 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f30cc103b60 0x7f30cc103f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:33.962 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:33.959+0000 7f30d2597640 1 -- 172.21.15.89:0/533941969 >> 172.21.15.89:0/533941969 conn(0x7f30cc07b850 msgr2=0x7f30cc07bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:38:33.962 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:33.959+0000 7f30d2597640 1 -- 172.21.15.89:0/533941969 shutdown_connections 2024-09-21T03:38:33.962 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:33.959+0000 7f30d2597640 1 -- 172.21.15.89:0/533941969 wait complete. 2024-09-21T03:38:33.964 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:33.960+0000 7f30d2597640 1 Processor -- start 2024-09-21T03:38:33.964 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:33.960+0000 7f30d2597640 1 -- start start 2024-09-21T03:38:33.964 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:33.960+0000 7f30d2597640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f30cc103b60 0x7f30cc19a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:33.964 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:33.960+0000 7f30d2597640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f30cc1094a0 0x7f30cc19aa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:33.964 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:33.960+0000 7f30d2597640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f30cc109df0 0x7f30cc19fd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:33.964 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:33.960+0000 7f30d2597640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f30cc1139b0 con 0x7f30cc1094a0 2024-09-21T03:38:33.964 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:33.960+0000 7f30d2597640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f30cc113830 con 0x7f30cc109df0 2024-09-21T03:38:33.964 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:33.960+0000 7f30d2597640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f30cc113b30 con 0x7f30cc103b60 2024-09-21T03:38:33.964 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:33.960+0000 7f30c3fff640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f30cc1094a0 0x7f30cc19aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:33.964 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:33.960+0000 7f30c3fff640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f30cc1094a0 0x7f30cc19aa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:34932/0 (socket says 172.21.15.89:34932) 2024-09-21T03:38:33.964 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:33.960+0000 7f30c3fff640 1 -- 172.21.15.89:0/3692055992 learned_addr learned my addr 172.21.15.89:0/3692055992 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-21T03:38:33.964 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:33.960+0000 7f30d0b34640 1 --2- 172.21.15.89:0/3692055992 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f30cc109df0 0x7f30cc19fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:33.964 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:33.960+0000 7f30cbfff640 1 --2- 172.21.15.89:0/3692055992 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f30cc103b60 0x7f30cc19a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:33.965 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:33.960+0000 7f30c3fff640 1 -- 172.21.15.89:0/3692055992 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f30cc103b60 msgr2=0x7f30cc19a4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:33.965 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:33.960+0000 7f30c3fff640 1 --2- 172.21.15.89:0/3692055992 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f30cc103b60 0x7f30cc19a4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:33.965 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:33.960+0000 7f30c3fff640 1 -- 172.21.15.89:0/3692055992 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f30cc109df0 msgr2=0x7f30cc19fd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:33.965 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:33.960+0000 7f30c3fff640 1 --2- 172.21.15.89:0/3692055992 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f30cc109df0 0x7f30cc19fd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:33.965 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:33.960+0000 7f30c3fff640 1 -- 172.21.15.89:0/3692055992 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30cc1a0430 con 0x7f30cc1094a0 2024-09-21T03:38:33.965 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:33.961+0000 7f30c3fff640 1 --2- 172.21.15.89:0/3692055992 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f30cc1094a0 0x7f30cc19aa60 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f30b400d740 tx=0x7f30b400dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:38:33.965 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:33.961+0000 7f30c9ffb640 1 -- 172.21.15.89:0/3692055992 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30b4017070 con 0x7f30cc1094a0 2024-09-21T03:38:33.965 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:33.961+0000 7f30c9ffb640 1 -- 172.21.15.89:0/3692055992 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f30b400de50 con 0x7f30cc1094a0 2024-09-21T03:38:33.965 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:33.961+0000 7f30c9ffb640 1 -- 172.21.15.89:0/3692055992 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30b400f620 con 0x7f30cc1094a0 2024-09-21T03:38:33.965 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:33.961+0000 7f30d2597640 1 -- 172.21.15.89:0/3692055992 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f30cc1a0720 con 0x7f30cc1094a0 2024-09-21T03:38:33.965 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:33.961+0000 7f30d2597640 1 -- 172.21.15.89:0/3692055992 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f30cc1a6e80 con 0x7f30cc1094a0 2024-09-21T03:38:33.966 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:33.962+0000 7f30d2597640 1 -- 172.21.15.89:0/3692055992 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f30cc103f70 con 0x7f30cc1094a0 2024-09-21T03:38:33.966 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:33.963+0000 7f30c9ffb640 1 -- 172.21.15.89:0/3692055992 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 18) ==== 154563+0+0 (secure 0 0 0) 0x7f30b401a070 con 0x7f30cc1094a0 2024-09-21T03:38:33.967 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:33.964+0000 7f30c9ffb640 1 --2- 172.21.15.89:0/3692055992 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f30b80b6620 0x7f30b80b8b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:33.967 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:33.965+0000 7f30cbfff640 1 --2- 172.21.15.89:0/3692055992 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f30b80b6620 0x7f30b80b8b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:33.967 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:33.965+0000 7f30c9ffb640 1 -- 172.21.15.89:0/3692055992 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 9092+0+0 (secure 0 0 0) 0x7f30b40e70b0 con 0x7f30cc1094a0 2024-09-21T03:38:33.967 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:33.965+0000 7f30cbfff640 1 --2- 172.21.15.89:0/3692055992 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f30b80b6620 0x7f30b80b8b30 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f30bc005fd0 tx=0x7f30bc00b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:38:33.971 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:33.969+0000 7f30c9ffb640 1 -- 172.21.15.89:0/3692055992 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f30b40b1860 con 0x7f30cc1094a0 2024-09-21T03:38:34.158 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:34.156+0000 7f30d2597640 1 -- 172.21.15.89:0/3692055992 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7f30cc10bbe0 con 0x7f30cc1094a0 2024-09-21T03:38:34.159 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:34.156+0000 7f30c9ffb640 1 -- 172.21.15.89:0/3692055992 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f30b40b67a0 con 0x7f30cc1094a0 2024-09-21T03:38:34.159 INFO:teuthology.orchestra.run.smithi089.stdout:197568495665 2024-09-21T03:38:34.161 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:34.159+0000 7f30d2597640 1 -- 172.21.15.89:0/3692055992 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f30b80b6620 msgr2=0x7f30b80b8b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:34.161 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:34.159+0000 7f30d2597640 1 --2- 172.21.15.89:0/3692055992 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f30b80b6620 0x7f30b80b8b30 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f30bc005fd0 tx=0x7f30bc00b390 comp rx=0 tx=0).stop 2024-09-21T03:38:34.161 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:34.159+0000 7f30d2597640 1 -- 172.21.15.89:0/3692055992 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f30cc1094a0 msgr2=0x7f30cc19aa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:34.161 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:34.159+0000 7f30d2597640 1 --2- 172.21.15.89:0/3692055992 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f30cc1094a0 0x7f30cc19aa60 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f30b400d740 tx=0x7f30b400dc00 comp rx=0 tx=0).stop 2024-09-21T03:38:34.161 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:34.159+0000 7f30d2597640 1 -- 172.21.15.89:0/3692055992 shutdown_connections 2024-09-21T03:38:34.161 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:34.159+0000 7f30d2597640 1 --2- 172.21.15.89:0/3692055992 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f30b80b6620 0x7f30b80b8b30 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:34.161 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:34.159+0000 7f30d2597640 1 --2- 172.21.15.89:0/3692055992 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f30cc109df0 0x7f30cc19fd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:34.161 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:34.159+0000 7f30d2597640 1 --2- 172.21.15.89:0/3692055992 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f30cc1094a0 0x7f30cc19aa60 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:34.162 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:34.159+0000 7f30d2597640 1 --2- 172.21.15.89:0/3692055992 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f30cc103b60 0x7f30cc19a4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:34.162 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:34.159+0000 7f30d2597640 1 -- 172.21.15.89:0/3692055992 >> 172.21.15.89:0/3692055992 conn(0x7f30cc07b850 msgr2=0x7f30cc105610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:38:34.162 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:34.159+0000 7f30d2597640 1 -- 172.21.15.89:0/3692055992 shutdown_connections 2024-09-21T03:38:34.162 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:34.159+0000 7f30d2597640 1 -- 172.21.15.89:0/3692055992 wait complete. 2024-09-21T03:38:34.444 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/7c374098-77c9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:38:34.458 INFO:tasks.cephadm.ceph_manager.ceph:need seq 270582939679 got 270582939682 for osd.10 2024-09-21T03:38:34.458 DEBUG:teuthology.parallel:result is None 2024-09-21T03:38:34.711 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:38:34 smithi114 ceph-mon[34140]: from='client.? 172.21.15.89:0/3692055992' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 7} : dispatch 2024-09-21T03:38:34.750 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:38:34 smithi110 ceph-mon[34000]: from='client.? 172.21.15.89:0/3692055992' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 7} : dispatch 2024-09-21T03:38:34.811 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:38:34 smithi089 ceph-mon[30928]: from='client.? 172.21.15.89:0/3692055992' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 7} : dispatch 2024-09-21T03:38:34.856 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/7c374098-77c9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:38:34.893 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738450 got 34359738452 for osd.0 2024-09-21T03:38:34.893 DEBUG:teuthology.parallel:result is None 2024-09-21T03:38:35.705 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:38:35 smithi089 ceph-mon[30928]: pgmap v333: 129 pgs: 129 active+clean; 646 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-21T03:38:35.710 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:38:35 smithi114 ceph-mon[34140]: pgmap v333: 129 pgs: 129 active+clean; 646 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-21T03:38:35.750 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:38:35 smithi110 ceph-mon[34000]: pgmap v333: 129 pgs: 129 active+clean; 646 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-21T03:38:35.805 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:35.795+0000 7f333da4b640 1 -- 172.21.15.89:0/3665590330 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3338108cf0 msgr2=0x7f333810f520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:35.805 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:35.795+0000 7f333da4b640 1 --2- 172.21.15.89:0/3665590330 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3338108cf0 0x7f333810f520 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f3328009900 tx=0x7f332802a410 comp rx=0 tx=0).stop 2024-09-21T03:38:35.805 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:35.795+0000 7f333da4b640 1 -- 172.21.15.89:0/3665590330 shutdown_connections 2024-09-21T03:38:35.805 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:35.795+0000 7f333da4b640 1 --2- 172.21.15.89:0/3665590330 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3338108cf0 0x7f333810f520 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:35.805 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:35.795+0000 7f333da4b640 1 --2- 172.21.15.89:0/3665590330 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f33381083a0 0x7f33381087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:35.805 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:35.795+0000 7f333da4b640 1 --2- 172.21.15.89:0/3665590330 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3338102a60 0x7f3338102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:35.805 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:35.795+0000 7f333da4b640 1 -- 172.21.15.89:0/3665590330 >> 172.21.15.89:0/3665590330 conn(0x7f3338079730 msgr2=0x7f3338079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:38:35.805 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:35.795+0000 7f333da4b640 1 -- 172.21.15.89:0/3665590330 shutdown_connections 2024-09-21T03:38:35.806 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:35.795+0000 7f333da4b640 1 -- 172.21.15.89:0/3665590330 wait complete. 2024-09-21T03:38:35.806 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:35.796+0000 7f333da4b640 1 Processor -- start 2024-09-21T03:38:35.806 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:35.796+0000 7f333da4b640 1 -- start start 2024-09-21T03:38:35.806 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:35.796+0000 7f333da4b640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3338102a60 0x7f333819b160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:35.806 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:35.796+0000 7f333da4b640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f33381083a0 0x7f333819b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:35.806 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:35.796+0000 7f333da4b640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3338108cf0 0x7f333819bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:35.806 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:35.796+0000 7f333da4b640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f3338112810 con 0x7f33381083a0 2024-09-21T03:38:35.806 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:35.796+0000 7f333da4b640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f3338112690 con 0x7f3338102a60 2024-09-21T03:38:35.806 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:35.796+0000 7f333da4b640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f3338112990 con 0x7f3338108cf0 2024-09-21T03:38:35.806 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:35.797+0000 7f33367fc640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f33381083a0 0x7f333819b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:35.806 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:35.797+0000 7f33367fc640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f33381083a0 0x7f333819b6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:34980/0 (socket says 172.21.15.89:34980) 2024-09-21T03:38:35.806 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:35.797+0000 7f33367fc640 1 -- 172.21.15.89:0/1524383019 learned_addr learned my addr 172.21.15.89:0/1524383019 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-21T03:38:35.806 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:35.797+0000 7f3336ffd640 1 --2- 172.21.15.89:0/1524383019 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3338102a60 0x7f333819b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:35.807 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:35.797+0000 7f33377fe640 1 --2- 172.21.15.89:0/1524383019 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3338108cf0 0x7f333819bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:35.807 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:35.797+0000 7f33367fc640 1 -- 172.21.15.89:0/1524383019 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3338108cf0 msgr2=0x7f333819bbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:35.807 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:35.797+0000 7f33367fc640 1 --2- 172.21.15.89:0/1524383019 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3338108cf0 0x7f333819bbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:35.807 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:35.797+0000 7f33367fc640 1 -- 172.21.15.89:0/1524383019 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3338102a60 msgr2=0x7f333819b160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:35.807 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:35.797+0000 7f33367fc640 1 --2- 172.21.15.89:0/1524383019 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3338102a60 0x7f333819b160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:35.807 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:35.797+0000 7f33367fc640 1 -- 172.21.15.89:0/1524383019 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33381a63f0 con 0x7f33381083a0 2024-09-21T03:38:35.807 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:35.797+0000 7f33367fc640 1 --2- 172.21.15.89:0/1524383019 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f33381083a0 0x7f333819b6a0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f332c00ea50 tx=0x7f332c00ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:38:35.807 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:35.797+0000 7f333ca49640 1 -- 172.21.15.89:0/1524383019 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f332c00cdc0 con 0x7f33381083a0 2024-09-21T03:38:35.807 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:35.797+0000 7f333da4b640 1 -- 172.21.15.89:0/1524383019 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f33381a66e0 con 0x7f33381083a0 2024-09-21T03:38:35.807 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:35.797+0000 7f333ca49640 1 -- 172.21.15.89:0/1524383019 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f332c005020 con 0x7f33381083a0 2024-09-21T03:38:35.807 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:35.797+0000 7f333ca49640 1 -- 172.21.15.89:0/1524383019 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f332c0116d0 con 0x7f33381083a0 2024-09-21T03:38:35.807 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:35.797+0000 7f333da4b640 1 -- 172.21.15.89:0/1524383019 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f33381a6c20 con 0x7f33381083a0 2024-09-21T03:38:35.807 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:35.798+0000 7f333da4b640 1 -- 172.21.15.89:0/1524383019 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3338102e70 con 0x7f33381083a0 2024-09-21T03:38:35.807 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:35.800+0000 7f333ca49640 1 -- 172.21.15.89:0/1524383019 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 18) ==== 154563+0+0 (secure 0 0 0) 0x7f332c004a40 con 0x7f33381083a0 2024-09-21T03:38:35.808 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:35.801+0000 7f333ca49640 1 --2- 172.21.15.89:0/1524383019 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f33200b6650 0x7f33200b8b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:35.808 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:35.801+0000 7f333ca49640 1 -- 172.21.15.89:0/1524383019 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 9092+0+0 (secure 0 0 0) 0x7f332c0e7d10 con 0x7f33381083a0 2024-09-21T03:38:35.808 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:35.801+0000 7f3336ffd640 1 --2- 172.21.15.89:0/1524383019 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f33200b6650 0x7f33200b8b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:35.808 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:35.801+0000 7f3336ffd640 1 --2- 172.21.15.89:0/1524383019 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f33200b6650 0x7f33200b8b60 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f333806b9c0 tx=0x7f3330005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:38:35.812 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:35.809+0000 7f333ca49640 1 -- 172.21.15.89:0/1524383019 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f332c016030 con 0x7f33381083a0 2024-09-21T03:38:35.998 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:35.996+0000 7f333da4b640 1 -- 172.21.15.89:0/1524383019 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7f333802e640 con 0x7f33381083a0 2024-09-21T03:38:35.998 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:35.996+0000 7f333ca49640 1 -- 172.21.15.89:0/1524383019 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f332c0ae160 con 0x7f33381083a0 2024-09-21T03:38:35.998 INFO:teuthology.orchestra.run.smithi089.stdout:176093659190 2024-09-21T03:38:36.001 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:35.999+0000 7f333da4b640 1 -- 172.21.15.89:0/1524383019 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f33200b6650 msgr2=0x7f33200b8b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:36.001 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:35.999+0000 7f333da4b640 1 --2- 172.21.15.89:0/1524383019 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f33200b6650 0x7f33200b8b60 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f333806b9c0 tx=0x7f3330005f90 comp rx=0 tx=0).stop 2024-09-21T03:38:36.001 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:35.999+0000 7f333da4b640 1 -- 172.21.15.89:0/1524383019 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f33381083a0 msgr2=0x7f333819b6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:36.001 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:35.999+0000 7f333da4b640 1 --2- 172.21.15.89:0/1524383019 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f33381083a0 0x7f333819b6a0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f332c00ea50 tx=0x7f332c00ef10 comp rx=0 tx=0).stop 2024-09-21T03:38:36.001 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:35.999+0000 7f333da4b640 1 -- 172.21.15.89:0/1524383019 shutdown_connections 2024-09-21T03:38:36.001 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:35.999+0000 7f333da4b640 1 --2- 172.21.15.89:0/1524383019 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f33200b6650 0x7f33200b8b60 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:36.001 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:35.999+0000 7f333da4b640 1 --2- 172.21.15.89:0/1524383019 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3338108cf0 0x7f333819bbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:36.001 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:35.999+0000 7f333da4b640 1 --2- 172.21.15.89:0/1524383019 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f33381083a0 0x7f333819b6a0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:36.001 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:35.999+0000 7f333da4b640 1 --2- 172.21.15.89:0/1524383019 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3338102a60 0x7f333819b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:36.002 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:35.999+0000 7f333da4b640 1 -- 172.21.15.89:0/1524383019 >> 172.21.15.89:0/1524383019 conn(0x7f3338079730 msgr2=0x7f3338104420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:38:36.002 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:35.999+0000 7f333da4b640 1 -- 172.21.15.89:0/1524383019 shutdown_connections 2024-09-21T03:38:36.002 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:35.999+0000 7f333da4b640 1 -- 172.21.15.89:0/1524383019 wait complete. 2024-09-21T03:38:36.024 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:36.011+0000 7f311edea640 1 -- 172.21.15.89:0/19249499 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3118109ec0 msgr2=0x7f31181106f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:36.024 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:36.011+0000 7f311edea640 1 --2- 172.21.15.89:0/19249499 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3118109ec0 0x7f31181106f0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f3110009ca0 tx=0x7f311002a8e0 comp rx=0 tx=0).stop 2024-09-21T03:38:36.024 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:36.011+0000 7f311edea640 1 -- 172.21.15.89:0/19249499 shutdown_connections 2024-09-21T03:38:36.024 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:36.011+0000 7f311edea640 1 --2- 172.21.15.89:0/19249499 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3118109ec0 0x7f31181106f0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:36.024 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:36.011+0000 7f311edea640 1 --2- 172.21.15.89:0/19249499 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3118109570 0x7f3118109980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:36.024 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:36.011+0000 7f311edea640 1 --2- 172.21.15.89:0/19249499 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3118103c30 0x7f3118104040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:36.024 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:36.011+0000 7f311edea640 1 -- 172.21.15.89:0/19249499 >> 172.21.15.89:0/19249499 conn(0x7f311807b8d0 msgr2=0x7f311807bce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:38:36.024 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:36.011+0000 7f311edea640 1 -- 172.21.15.89:0/19249499 shutdown_connections 2024-09-21T03:38:36.024 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:36.011+0000 7f311edea640 1 -- 172.21.15.89:0/19249499 wait complete. 2024-09-21T03:38:36.024 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:36.012+0000 7f311edea640 1 Processor -- start 2024-09-21T03:38:36.025 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:36.012+0000 7f311edea640 1 -- start start 2024-09-21T03:38:36.025 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:36.012+0000 7f311edea640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3118103c30 0x7f311819c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:36.025 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:36.012+0000 7f311edea640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3118109570 0x7f311819cb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:36.025 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:36.012+0000 7f311edea640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3118109ec0 0x7f31181a1de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:36.025 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:36.012+0000 7f311edea640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f31181139d0 con 0x7f3118109ec0 2024-09-21T03:38:36.025 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:36.012+0000 7f311edea640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f3118113850 con 0x7f3118109570 2024-09-21T03:38:36.025 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:36.012+0000 7f311edea640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f3118113b50 con 0x7f3118103c30 2024-09-21T03:38:36.025 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:36.013+0000 7f311d387640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3118109ec0 0x7f31181a1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:36.025 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:36.013+0000 7f311d387640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3118109ec0 0x7f31181a1de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:35000/0 (socket says 172.21.15.89:35000) 2024-09-21T03:38:36.025 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:36.013+0000 7f311d387640 1 -- 172.21.15.89:0/2873899721 learned_addr learned my addr 172.21.15.89:0/2873899721 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-21T03:38:36.025 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:36.013+0000 7f311cb86640 1 --2- 172.21.15.89:0/2873899721 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3118103c30 0x7f311819c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:36.025 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:36.013+0000 7f3117fff640 1 --2- 172.21.15.89:0/2873899721 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3118109570 0x7f311819cb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:36.025 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:36.013+0000 7f311d387640 1 -- 172.21.15.89:0/2873899721 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3118103c30 msgr2=0x7f311819c5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:36.025 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:36.013+0000 7f311d387640 1 --2- 172.21.15.89:0/2873899721 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3118103c30 0x7f311819c5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:36.025 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:36.013+0000 7f311d387640 1 -- 172.21.15.89:0/2873899721 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3118109570 msgr2=0x7f311819cb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:36.026 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:36.013+0000 7f311d387640 1 --2- 172.21.15.89:0/2873899721 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3118109570 0x7f311819cb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:36.026 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:36.013+0000 7f311d387640 1 -- 172.21.15.89:0/2873899721 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31181a2450 con 0x7f3118109ec0 2024-09-21T03:38:36.026 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:36.013+0000 7f311cb86640 1 --2- 172.21.15.89:0/2873899721 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3118103c30 0x7f311819c5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:38:36.026 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:36.013+0000 7f311d387640 1 --2- 172.21.15.89:0/2873899721 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3118109ec0 0x7f31181a1de0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f3110006010 tx=0x7f3110003c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:38:36.026 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:36.014+0000 7f3115ffb640 1 -- 172.21.15.89:0/2873899721 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3110004240 con 0x7f3118109ec0 2024-09-21T03:38:36.026 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:36.014+0000 7f311edea640 1 -- 172.21.15.89:0/2873899721 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f31181a26e0 con 0x7f3118109ec0 2024-09-21T03:38:36.026 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:36.014+0000 7f3115ffb640 1 -- 172.21.15.89:0/2873899721 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f31100305e0 con 0x7f3118109ec0 2024-09-21T03:38:36.026 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:36.014+0000 7f3115ffb640 1 -- 172.21.15.89:0/2873899721 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f311003b660 con 0x7f3118109ec0 2024-09-21T03:38:36.026 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:36.014+0000 7f311edea640 1 -- 172.21.15.89:0/2873899721 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f31181a8f60 con 0x7f3118109ec0 2024-09-21T03:38:36.026 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:36.015+0000 7f311edea640 1 -- 172.21.15.89:0/2873899721 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3118104040 con 0x7f3118109ec0 2024-09-21T03:38:36.026 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:36.017+0000 7f3115ffb640 1 -- 172.21.15.89:0/2873899721 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 18) ==== 154563+0+0 (secure 0 0 0) 0x7f3110030790 con 0x7f3118109ec0 2024-09-21T03:38:36.026 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:36.017+0000 7f3115ffb640 1 --2- 172.21.15.89:0/2873899721 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f31080b65d0 0x7f31080b8ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:36.026 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:36.018+0000 7f311cb86640 1 --2- 172.21.15.89:0/2873899721 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f31080b65d0 0x7f31080b8ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:36.027 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:36.018+0000 7f3115ffb640 1 -- 172.21.15.89:0/2873899721 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 9092+0+0 (secure 0 0 0) 0x7f3110108cb0 con 0x7f3118109ec0 2024-09-21T03:38:36.027 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:36.018+0000 7f311cb86640 1 --2- 172.21.15.89:0/2873899721 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f31080b65d0 0x7f31080b8ae0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f310c00afb0 tx=0x7f310c00a2d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:38:36.027 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:36.021+0000 7f3115ffb640 1 -- 172.21.15.89:0/2873899721 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f31100cf080 con 0x7f3118109ec0 2024-09-21T03:38:36.208 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:36.205+0000 7f311edea640 1 -- 172.21.15.89:0/2873899721 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 9} v 0) -- 0x7f31181a9130 con 0x7f3118109ec0 2024-09-21T03:38:36.208 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:36.206+0000 7f3115ffb640 1 -- 172.21.15.89:0/2873899721 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 9}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f31100d3460 con 0x7f3118109ec0 2024-09-21T03:38:36.208 INFO:teuthology.orchestra.run.smithi089.stdout:249108103208 2024-09-21T03:38:36.211 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:36.208+0000 7f311edea640 1 -- 172.21.15.89:0/2873899721 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f31080b65d0 msgr2=0x7f31080b8ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:36.211 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:36.208+0000 7f311edea640 1 --2- 172.21.15.89:0/2873899721 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f31080b65d0 0x7f31080b8ae0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f310c00afb0 tx=0x7f310c00a2d0 comp rx=0 tx=0).stop 2024-09-21T03:38:36.211 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:36.208+0000 7f311edea640 1 -- 172.21.15.89:0/2873899721 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3118109ec0 msgr2=0x7f31181a1de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:36.211 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:36.208+0000 7f311edea640 1 --2- 172.21.15.89:0/2873899721 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3118109ec0 0x7f31181a1de0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f3110006010 tx=0x7f3110003c90 comp rx=0 tx=0).stop 2024-09-21T03:38:36.211 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:36.208+0000 7f311edea640 1 -- 172.21.15.89:0/2873899721 shutdown_connections 2024-09-21T03:38:36.211 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:36.208+0000 7f311edea640 1 --2- 172.21.15.89:0/2873899721 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f31080b65d0 0x7f31080b8ae0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:36.211 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:36.208+0000 7f311edea640 1 --2- 172.21.15.89:0/2873899721 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3118109570 0x7f311819cb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:36.211 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:36.208+0000 7f311edea640 1 --2- 172.21.15.89:0/2873899721 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3118109ec0 0x7f31181a1de0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:36.211 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:36.208+0000 7f311edea640 1 --2- 172.21.15.89:0/2873899721 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3118103c30 0x7f311819c5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:36.211 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:36.208+0000 7f311edea640 1 -- 172.21.15.89:0/2873899721 >> 172.21.15.89:0/2873899721 conn(0x7f311807b8d0 msgr2=0x7f3118107380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:38:36.211 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:36.208+0000 7f311edea640 1 -- 172.21.15.89:0/2873899721 shutdown_connections 2024-09-21T03:38:36.211 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:36.209+0000 7f311edea640 1 -- 172.21.15.89:0/2873899721 wait complete. 2024-09-21T03:38:36.711 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:38:36 smithi114 ceph-mon[34140]: from='client.? 172.21.15.89:0/3219121747' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 5} : dispatch 2024-09-21T03:38:36.711 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:38:36 smithi114 ceph-mon[34140]: from='client.? 172.21.15.89:0/1524383019' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 6} : dispatch 2024-09-21T03:38:36.711 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:38:36 smithi114 ceph-mon[34140]: from='client.? 172.21.15.89:0/2873899721' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 9} : dispatch 2024-09-21T03:38:36.749 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:38:36 smithi110 ceph-mon[34000]: from='client.? 172.21.15.89:0/3219121747' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 5} : dispatch 2024-09-21T03:38:36.750 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:38:36 smithi110 ceph-mon[34000]: from='client.? 172.21.15.89:0/1524383019' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 6} : dispatch 2024-09-21T03:38:36.750 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:38:36 smithi110 ceph-mon[34000]: from='client.? 172.21.15.89:0/2873899721' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 9} : dispatch 2024-09-21T03:38:36.763 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:38:36 smithi089 ceph-mon[30928]: from='client.? 172.21.15.89:0/3219121747' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 5} : dispatch 2024-09-21T03:38:36.763 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:38:36 smithi089 ceph-mon[30928]: from='client.? 172.21.15.89:0/1524383019' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 6} : dispatch 2024-09-21T03:38:36.763 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:38:36 smithi089 ceph-mon[30928]: from='client.? 172.21.15.89:0/2873899721' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 9} : dispatch 2024-09-21T03:38:36.972 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:35.715+0000 7fe01c425640 1 -- 172.21.15.89:0/3673448973 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe014104880 msgr2=0x7fe014104c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:36.972 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:35.715+0000 7fe01c425640 1 --2- 172.21.15.89:0/3673448973 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe014104880 0x7fe014104c90 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fe004009900 tx=0x7fe00402a410 comp rx=0 tx=0).stop 2024-09-21T03:38:36.972 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:35.715+0000 7fe01c425640 1 -- 172.21.15.89:0/3673448973 shutdown_connections 2024-09-21T03:38:36.972 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:35.715+0000 7fe01c425640 1 --2- 172.21.15.89:0/3673448973 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe0141052b0 0x7fe01410d8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:36.972 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:35.715+0000 7fe01c425640 1 --2- 172.21.15.89:0/3673448973 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe01410de30 0x7fe014110340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:36.972 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:35.715+0000 7fe01c425640 1 --2- 172.21.15.89:0/3673448973 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe014104880 0x7fe014104c90 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:36.972 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:35.715+0000 7fe01c425640 1 -- 172.21.15.89:0/3673448973 >> 172.21.15.89:0/3673448973 conn(0x7fe0140797e0 msgr2=0x7fe014079bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:38:36.972 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:35.716+0000 7fe01c425640 1 -- 172.21.15.89:0/3673448973 shutdown_connections 2024-09-21T03:38:36.972 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:35.716+0000 7fe01c425640 1 -- 172.21.15.89:0/3673448973 wait complete. 2024-09-21T03:38:36.972 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:35.716+0000 7fe01c425640 1 Processor -- start 2024-09-21T03:38:36.973 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:35.716+0000 7fe01c425640 1 -- start start 2024-09-21T03:38:36.973 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:35.717+0000 7fe01c425640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe014104880 0x7fe01419a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:36.973 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:35.717+0000 7fe01c425640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe0141052b0 0x7fe01419a990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:36.973 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:35.717+0000 7fe01c425640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe01410de30 0x7fe01419fc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:36.973 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:35.717+0000 7fe01c425640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fe0141139c0 con 0x7fe0141052b0 2024-09-21T03:38:36.973 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:35.717+0000 7fe01c425640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fe014113840 con 0x7fe014104880 2024-09-21T03:38:36.973 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:35.717+0000 7fe01c425640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fe014113b40 con 0x7fe01410de30 2024-09-21T03:38:36.973 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:35.717+0000 7fe0199c0640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe0141052b0 0x7fe01419a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:36.973 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:35.717+0000 7fe0199c0640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe0141052b0 0x7fe01419a990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:34952/0 (socket says 172.21.15.89:34952) 2024-09-21T03:38:36.973 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:35.717+0000 7fe0199c0640 1 -- 172.21.15.89:0/3219121747 learned_addr learned my addr 172.21.15.89:0/3219121747 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-21T03:38:36.973 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:35.717+0000 7fe01a1c1640 1 --2- 172.21.15.89:0/3219121747 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe014104880 0x7fe01419a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:36.973 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:35.717+0000 7fe01a9c2640 1 --2- 172.21.15.89:0/3219121747 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe01410de30 0x7fe01419fc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:36.973 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:35.717+0000 7fe0199c0640 1 -- 172.21.15.89:0/3219121747 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe01410de30 msgr2=0x7fe01419fc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:36.973 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:35.717+0000 7fe0199c0640 1 --2- 172.21.15.89:0/3219121747 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe01410de30 0x7fe01419fc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:36.973 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:35.717+0000 7fe0199c0640 1 -- 172.21.15.89:0/3219121747 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe014104880 msgr2=0x7fe01419a3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:36.974 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:35.717+0000 7fe0199c0640 1 --2- 172.21.15.89:0/3219121747 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe014104880 0x7fe01419a3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:36.974 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:35.717+0000 7fe0199c0640 1 -- 172.21.15.89:0/3219121747 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0141a0360 con 0x7fe0141052b0 2024-09-21T03:38:36.974 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:35.717+0000 7fe01a9c2640 1 --2- 172.21.15.89:0/3219121747 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe01410de30 0x7fe01419fc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:38:36.974 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:35.717+0000 7fe0199c0640 1 --2- 172.21.15.89:0/3219121747 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe0141052b0 0x7fe01419a990 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fe010006fd0 tx=0x7fe01000b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:38:36.974 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:35.717+0000 7fe00b7fe640 1 -- 172.21.15.89:0/3219121747 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe010011830 con 0x7fe0141052b0 2024-09-21T03:38:36.974 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:35.718+0000 7fe00b7fe640 1 -- 172.21.15.89:0/3219121747 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe010016070 con 0x7fe0141052b0 2024-09-21T03:38:36.974 INFO:teuthology.orchestra.run.smithi089.stdout:150323855419 2024-09-21T03:38:36.974 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:35.718+0000 7fe01c425640 1 -- 172.21.15.89:0/3219121747 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe0141a0650 con 0x7fe0141052b0 2024-09-21T03:38:36.974 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:35.718+0000 7fe00b7fe640 1 -- 172.21.15.89:0/3219121747 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe010010d30 con 0x7fe0141052b0 2024-09-21T03:38:36.975 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:35.718+0000 7fe01c425640 1 -- 172.21.15.89:0/3219121747 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe0141a6db0 con 0x7fe0141052b0 2024-09-21T03:38:36.975 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:35.719+0000 7fe01c425640 1 -- 172.21.15.89:0/3219121747 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe014104c90 con 0x7fe0141052b0 2024-09-21T03:38:36.975 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:35.720+0000 7fe00b7fe640 1 -- 172.21.15.89:0/3219121747 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 18) ==== 154563+0+0 (secure 0 0 0) 0x7fe0100119d0 con 0x7fe0141052b0 2024-09-21T03:38:36.975 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:35.721+0000 7fe00b7fe640 1 --2- 172.21.15.89:0/3219121747 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7fe0000b65a0 0x7fe0000b8ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:36.975 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:35.721+0000 7fe01a1c1640 1 --2- 172.21.15.89:0/3219121747 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7fe0000b65a0 0x7fe0000b8ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:36.975 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:35.721+0000 7fe00b7fe640 1 -- 172.21.15.89:0/3219121747 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 9092+0+0 (secure 0 0 0) 0x7fe0100e7400 con 0x7fe0141052b0 2024-09-21T03:38:36.975 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:35.722+0000 7fe01a1c1640 1 --2- 172.21.15.89:0/3219121747 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7fe0000b65a0 0x7fe0000b8ab0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fe004004770 tx=0x7fe0040058d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:38:36.975 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:35.724+0000 7fe00b7fe640 1 -- 172.21.15.89:0/3219121747 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fe0100ad7d0 con 0x7fe0141052b0 2024-09-21T03:38:36.975 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:35.914+0000 7fe01c425640 1 -- 172.21.15.89:0/3219121747 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7fe01419b520 con 0x7fe0141052b0 2024-09-21T03:38:36.975 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:35.914+0000 7fe00b7fe640 1 -- 172.21.15.89:0/3219121747 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fe01419b520 con 0x7fe0141052b0 2024-09-21T03:38:36.975 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:35.917+0000 7fe01c425640 1 -- 172.21.15.89:0/3219121747 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7fe0000b65a0 msgr2=0x7fe0000b8ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:36.975 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:35.917+0000 7fe01c425640 1 --2- 172.21.15.89:0/3219121747 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7fe0000b65a0 0x7fe0000b8ab0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fe004004770 tx=0x7fe0040058d0 comp rx=0 tx=0).stop 2024-09-21T03:38:36.975 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:35.917+0000 7fe01c425640 1 -- 172.21.15.89:0/3219121747 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe0141052b0 msgr2=0x7fe01419a990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:36.975 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:35.917+0000 7fe01c425640 1 --2- 172.21.15.89:0/3219121747 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe0141052b0 0x7fe01419a990 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fe010006fd0 tx=0x7fe01000b8e0 comp rx=0 tx=0).stop 2024-09-21T03:38:36.976 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:35.917+0000 7fe01c425640 1 -- 172.21.15.89:0/3219121747 shutdown_connections 2024-09-21T03:38:36.976 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:35.917+0000 7fe01c425640 1 --2- 172.21.15.89:0/3219121747 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7fe0000b65a0 0x7fe0000b8ab0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:36.976 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:35.917+0000 7fe01c425640 1 --2- 172.21.15.89:0/3219121747 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe01410de30 0x7fe01419fc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:36.976 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:35.917+0000 7fe01c425640 1 --2- 172.21.15.89:0/3219121747 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe0141052b0 0x7fe01419a990 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:36.976 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:35.917+0000 7fe01c425640 1 --2- 172.21.15.89:0/3219121747 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe014104880 0x7fe01419a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:36.976 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:35.917+0000 7fe01c425640 1 -- 172.21.15.89:0/3219121747 >> 172.21.15.89:0/3219121747 conn(0x7fe0140797e0 msgr2=0x7fe014100900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:38:36.976 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:35.917+0000 7fe01c425640 1 -- 172.21.15.89:0/3219121747 shutdown_connections 2024-09-21T03:38:36.976 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:35.917+0000 7fe01c425640 1 -- 172.21.15.89:0/3219121747 wait complete. 2024-09-21T03:38:37.368 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:37.365+0000 7f28d781f640 1 -- 172.21.15.89:0/1171046643 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f28d0108d00 msgr2=0x7f28d010f530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:37.368 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:37.365+0000 7f28d781f640 1 --2- 172.21.15.89:0/1171046643 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f28d0108d00 0x7f28d010f530 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f28c8009ca0 tx=0x7f28c802a8c0 comp rx=0 tx=0).stop 2024-09-21T03:38:37.368 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:37.365+0000 7f28d781f640 1 -- 172.21.15.89:0/1171046643 shutdown_connections 2024-09-21T03:38:37.368 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:37.365+0000 7f28d781f640 1 --2- 172.21.15.89:0/1171046643 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f28d01083b0 0x7f28d01087c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:37.368 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:37.365+0000 7f28d781f640 1 --2- 172.21.15.89:0/1171046643 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f28d0108d00 0x7f28d010f530 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:37.368 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:37.365+0000 7f28d781f640 1 --2- 172.21.15.89:0/1171046643 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f28d0103140 0x7f28d0103550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:37.368 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:37.365+0000 7f28d781f640 1 -- 172.21.15.89:0/1171046643 >> 172.21.15.89:0/1171046643 conn(0x7f28d0077c60 msgr2=0x7f28d00780a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:38:37.368 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:37.365+0000 7f28d781f640 1 -- 172.21.15.89:0/1171046643 shutdown_connections 2024-09-21T03:38:37.368 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:37.366+0000 7f28d781f640 1 -- 172.21.15.89:0/1171046643 wait complete. 2024-09-21T03:38:37.370 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:37.366+0000 7f28d781f640 1 Processor -- start 2024-09-21T03:38:37.370 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:37.366+0000 7f28d781f640 1 -- start start 2024-09-21T03:38:37.370 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:37.366+0000 7f28d781f640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f28d0103140 0x7f28d019a340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:37.370 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:37.366+0000 7f28d781f640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f28d01083b0 0x7f28d019a8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:37.370 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:37.366+0000 7f28d781f640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f28d0108d00 0x7f28d019fb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:37.370 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:37.366+0000 7f28d781f640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f28d01128c0 con 0x7f28d01083b0 2024-09-21T03:38:37.371 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:37.366+0000 7f28d781f640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f28d0112740 con 0x7f28d0108d00 2024-09-21T03:38:37.371 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:37.366+0000 7f28d781f640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f28d0112a40 con 0x7f28d0103140 2024-09-21T03:38:37.371 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:37.367+0000 7f28d4dba640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f28d01083b0 0x7f28d019a8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:37.371 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:37.367+0000 7f28d4dba640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f28d01083b0 0x7f28d019a8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:35012/0 (socket says 172.21.15.89:35012) 2024-09-21T03:38:37.371 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:37.367+0000 7f28d4dba640 1 -- 172.21.15.89:0/1026961921 learned_addr learned my addr 172.21.15.89:0/1026961921 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-21T03:38:37.371 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:37.367+0000 7f28d5dbc640 1 --2- 172.21.15.89:0/1026961921 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f28d0108d00 0x7f28d019fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:37.371 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:37.367+0000 7f28d55bb640 1 --2- 172.21.15.89:0/1026961921 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f28d0103140 0x7f28d019a340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:37.371 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:37.367+0000 7f28d4dba640 1 -- 172.21.15.89:0/1026961921 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f28d0103140 msgr2=0x7f28d019a340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:37.371 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:37.367+0000 7f28d4dba640 1 --2- 172.21.15.89:0/1026961921 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f28d0103140 0x7f28d019a340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:37.371 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:37.367+0000 7f28d4dba640 1 -- 172.21.15.89:0/1026961921 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f28d0108d00 msgr2=0x7f28d019fb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:37.371 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:37.367+0000 7f28d4dba640 1 --2- 172.21.15.89:0/1026961921 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f28d0108d00 0x7f28d019fb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:37.371 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:37.367+0000 7f28d4dba640 1 -- 172.21.15.89:0/1026961921 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28d01a02b0 con 0x7f28d01083b0 2024-09-21T03:38:37.371 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:37.367+0000 7f28d5dbc640 1 --2- 172.21.15.89:0/1026961921 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f28d0108d00 0x7f28d019fb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:38:37.371 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:37.367+0000 7f28d4dba640 1 --2- 172.21.15.89:0/1026961921 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f28d01083b0 0x7f28d019a8e0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f28c000d740 tx=0x7f28c000dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:38:37.371 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:37.367+0000 7f28c67fc640 1 -- 172.21.15.89:0/1026961921 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28c0017070 con 0x7f28d01083b0 2024-09-21T03:38:37.372 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:37.367+0000 7f28d781f640 1 -- 172.21.15.89:0/1026961921 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f28d01a05a0 con 0x7f28d01083b0 2024-09-21T03:38:37.372 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:37.367+0000 7f28c67fc640 1 -- 172.21.15.89:0/1026961921 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f28c000de50 con 0x7f28d01083b0 2024-09-21T03:38:37.372 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:37.367+0000 7f28c67fc640 1 -- 172.21.15.89:0/1026961921 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28c000f620 con 0x7f28d01083b0 2024-09-21T03:38:37.372 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:37.367+0000 7f28d781f640 1 -- 172.21.15.89:0/1026961921 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28d01a6d00 con 0x7f28d01083b0 2024-09-21T03:38:37.372 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:37.368+0000 7f28d781f640 1 -- 172.21.15.89:0/1026961921 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f28d0103550 con 0x7f28d01083b0 2024-09-21T03:38:37.372 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:37.370+0000 7f28c67fc640 1 -- 172.21.15.89:0/1026961921 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 18) ==== 154563+0+0 (secure 0 0 0) 0x7f28c001a070 con 0x7f28d01083b0 2024-09-21T03:38:37.373 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:37.371+0000 7f28c67fc640 1 --2- 172.21.15.89:0/1026961921 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f28bc0b65a0 0x7f28bc0b8ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:37.373 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:37.371+0000 7f28d55bb640 1 --2- 172.21.15.89:0/1026961921 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f28bc0b65a0 0x7f28bc0b8ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:37.373 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:37.371+0000 7f28c67fc640 1 -- 172.21.15.89:0/1026961921 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 9092+0+0 (secure 0 0 0) 0x7f28c00e72a0 con 0x7f28d01083b0 2024-09-21T03:38:37.373 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:37.371+0000 7f28d55bb640 1 --2- 172.21.15.89:0/1026961921 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f28bc0b65a0 0x7f28bc0b8ab0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f28cc009b60 tx=0x7f28cc005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:38:37.376 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:37.374+0000 7f28c67fc640 1 -- 172.21.15.89:0/1026961921 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f28c00ad670 con 0x7f28d01083b0 2024-09-21T03:38:37.561 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:38:37 smithi089 ceph-mon[30928]: pgmap v334: 129 pgs: 129 active+clean; 646 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-09-21T03:38:37.561 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:37.559+0000 7f28d781f640 1 -- 172.21.15.89:0/1026961921 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 11} v 0) -- 0x7f28d010bb30 con 0x7f28d01083b0 2024-09-21T03:38:37.561 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:37.559+0000 7f28c67fc640 1 -- 172.21.15.89:0/1026961921 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 11}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7f28c00b1a50 con 0x7f28d01083b0 2024-09-21T03:38:37.562 INFO:teuthology.orchestra.run.smithi089.stdout:292057776158 2024-09-21T03:38:37.564 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:37.562+0000 7f28d781f640 1 -- 172.21.15.89:0/1026961921 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f28bc0b65a0 msgr2=0x7f28bc0b8ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:37.564 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:37.562+0000 7f28d781f640 1 --2- 172.21.15.89:0/1026961921 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f28bc0b65a0 0x7f28bc0b8ab0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f28cc009b60 tx=0x7f28cc005f90 comp rx=0 tx=0).stop 2024-09-21T03:38:37.564 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:37.562+0000 7f28d781f640 1 -- 172.21.15.89:0/1026961921 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f28d01083b0 msgr2=0x7f28d019a8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:37.564 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:37.562+0000 7f28d781f640 1 --2- 172.21.15.89:0/1026961921 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f28d01083b0 0x7f28d019a8e0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f28c000d740 tx=0x7f28c000dc00 comp rx=0 tx=0).stop 2024-09-21T03:38:37.564 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:37.562+0000 7f28d781f640 1 -- 172.21.15.89:0/1026961921 shutdown_connections 2024-09-21T03:38:37.564 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:37.562+0000 7f28d781f640 1 --2- 172.21.15.89:0/1026961921 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f28bc0b65a0 0x7f28bc0b8ab0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:37.564 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:37.562+0000 7f28d781f640 1 --2- 172.21.15.89:0/1026961921 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f28d0108d00 0x7f28d019fb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:37.564 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:37.562+0000 7f28d781f640 1 --2- 172.21.15.89:0/1026961921 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f28d01083b0 0x7f28d019a8e0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:37.564 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:37.562+0000 7f28d781f640 1 --2- 172.21.15.89:0/1026961921 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f28d0103140 0x7f28d019a340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:37.564 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:37.562+0000 7f28d781f640 1 -- 172.21.15.89:0/1026961921 >> 172.21.15.89:0/1026961921 conn(0x7f28d0077c60 msgr2=0x7f28d0101200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:38:37.565 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:37.562+0000 7f28d781f640 1 -- 172.21.15.89:0/1026961921 shutdown_connections 2024-09-21T03:38:37.565 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:37.562+0000 7f28d781f640 1 -- 172.21.15.89:0/1026961921 wait complete. 2024-09-21T03:38:37.711 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:38:37 smithi114 ceph-mon[34140]: pgmap v334: 129 pgs: 129 active+clean; 646 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-09-21T03:38:37.750 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:38:37 smithi110 ceph-mon[34000]: pgmap v334: 129 pgs: 129 active+clean; 646 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-09-21T03:38:37.813 INFO:tasks.cephadm.ceph_manager.ceph:need seq 197568495664 got 197568495665 for osd.7 2024-09-21T03:38:37.813 DEBUG:teuthology.parallel:result is None 2024-09-21T03:38:37.994 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:37.966+0000 7fe7f363a640 1 -- 172.21.15.89:0/3939990538 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe7ec102a60 msgr2=0x7fe7ec102e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:37.995 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:37.966+0000 7fe7f363a640 1 --2- 172.21.15.89:0/3939990538 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe7ec102a60 0x7fe7ec102e70 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fe7dc009900 tx=0x7fe7dc02a410 comp rx=0 tx=0).stop 2024-09-21T03:38:37.995 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:37.966+0000 7fe7f363a640 1 -- 172.21.15.89:0/3939990538 shutdown_connections 2024-09-21T03:38:37.995 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:37.966+0000 7fe7f363a640 1 --2- 172.21.15.89:0/3939990538 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe7ec1083a0 0x7fe7ec1087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:37.995 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:37.966+0000 7fe7f363a640 1 --2- 172.21.15.89:0/3939990538 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe7ec108cf0 0x7fe7ec10f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:37.995 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:37.966+0000 7fe7f363a640 1 --2- 172.21.15.89:0/3939990538 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe7ec102a60 0x7fe7ec102e70 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:37.995 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:37.966+0000 7fe7f363a640 1 -- 172.21.15.89:0/3939990538 >> 172.21.15.89:0/3939990538 conn(0x7fe7ec079730 msgr2=0x7fe7ec079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:38:37.995 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:37.966+0000 7fe7f363a640 1 -- 172.21.15.89:0/3939990538 shutdown_connections 2024-09-21T03:38:37.995 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:37.966+0000 7fe7f363a640 1 -- 172.21.15.89:0/3939990538 wait complete. 2024-09-21T03:38:37.995 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:37.966+0000 7fe7f363a640 1 Processor -- start 2024-09-21T03:38:37.995 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:37.966+0000 7fe7f363a640 1 -- start start 2024-09-21T03:38:37.995 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:37.967+0000 7fe7f363a640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe7ec102a60 0x7fe7ec199360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:37.995 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:37.967+0000 7fe7f363a640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe7ec1083a0 0x7fe7ec199900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:37.996 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:37.967+0000 7fe7f363a640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe7ec108cf0 0x7fe7ec19eba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:37.996 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:37.967+0000 7fe7f363a640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fe7ec112870 con 0x7fe7ec108cf0 2024-09-21T03:38:37.996 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:37.967+0000 7fe7f363a640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fe7ec1126f0 con 0x7fe7ec102a60 2024-09-21T03:38:37.996 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:37.967+0000 7fe7f363a640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fe7ec1129f0 con 0x7fe7ec1083a0 2024-09-21T03:38:37.996 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:37.967+0000 7fe7f1bd7640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe7ec108cf0 0x7fe7ec19eba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:37.996 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:37.967+0000 7fe7f1bd7640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe7ec108cf0 0x7fe7ec19eba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:35044/0 (socket says 172.21.15.89:35044) 2024-09-21T03:38:37.996 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:37.967+0000 7fe7f1bd7640 1 -- 172.21.15.89:0/3434285416 learned_addr learned my addr 172.21.15.89:0/3434285416 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-21T03:38:37.996 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:37.967+0000 7fe7f13d6640 1 --2- 172.21.15.89:0/3434285416 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe7ec102a60 0x7fe7ec199360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:37.996 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:37.967+0000 7fe7f0bd5640 1 --2- 172.21.15.89:0/3434285416 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe7ec1083a0 0x7fe7ec199900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:37.996 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:37.967+0000 7fe7f1bd7640 1 -- 172.21.15.89:0/3434285416 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe7ec1083a0 msgr2=0x7fe7ec199900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:37.996 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:37.967+0000 7fe7f1bd7640 1 --2- 172.21.15.89:0/3434285416 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe7ec1083a0 0x7fe7ec199900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:37.996 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:37.967+0000 7fe7f1bd7640 1 -- 172.21.15.89:0/3434285416 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe7ec102a60 msgr2=0x7fe7ec199360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:37.996 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:37.967+0000 7fe7f1bd7640 1 --2- 172.21.15.89:0/3434285416 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe7ec102a60 0x7fe7ec199360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:37.996 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:37.967+0000 7fe7f1bd7640 1 -- 172.21.15.89:0/3434285416 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7ec19f2d0 con 0x7fe7ec108cf0 2024-09-21T03:38:37.997 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:37.967+0000 7fe7f13d6640 1 --2- 172.21.15.89:0/3434285416 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe7ec102a60 0x7fe7ec199360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:38:37.997 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:37.967+0000 7fe7f1bd7640 1 --2- 172.21.15.89:0/3434285416 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe7ec108cf0 0x7fe7ec19eba0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fe7e400b970 tx=0x7fe7e400be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:38:37.997 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:37.968+0000 7fe7e27fc640 1 -- 172.21.15.89:0/3434285416 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7e4019070 con 0x7fe7ec108cf0 2024-09-21T03:38:37.997 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:37.968+0000 7fe7e27fc640 1 -- 172.21.15.89:0/3434285416 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe7e4014440 con 0x7fe7ec108cf0 2024-09-21T03:38:37.997 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:37.968+0000 7fe7f363a640 1 -- 172.21.15.89:0/3434285416 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe7ec19f5c0 con 0x7fe7ec108cf0 2024-09-21T03:38:37.997 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:37.968+0000 7fe7e27fc640 1 -- 172.21.15.89:0/3434285416 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7e40075d0 con 0x7fe7ec108cf0 2024-09-21T03:38:37.997 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:37.968+0000 7fe7f363a640 1 -- 172.21.15.89:0/3434285416 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7ec1a5d20 con 0x7fe7ec108cf0 2024-09-21T03:38:37.997 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:37.969+0000 7fe7f363a640 1 -- 172.21.15.89:0/3434285416 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe7ec102e70 con 0x7fe7ec108cf0 2024-09-21T03:38:37.997 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:37.971+0000 7fe7e27fc640 1 -- 172.21.15.89:0/3434285416 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 18) ==== 154563+0+0 (secure 0 0 0) 0x7fe7e401c070 con 0x7fe7ec108cf0 2024-09-21T03:38:37.997 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:37.972+0000 7fe7e27fc640 1 --2- 172.21.15.89:0/3434285416 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7fe7d80b65a0 0x7fe7d80b8ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:37.997 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:37.972+0000 7fe7f13d6640 1 --2- 172.21.15.89:0/3434285416 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7fe7d80b65a0 0x7fe7d80b8ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:37.997 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:37.972+0000 7fe7e27fc640 1 -- 172.21.15.89:0/3434285416 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 9092+0+0 (secure 0 0 0) 0x7fe7e40e7190 con 0x7fe7ec108cf0 2024-09-21T03:38:37.997 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:37.973+0000 7fe7f13d6640 1 --2- 172.21.15.89:0/3434285416 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7fe7d80b65a0 0x7fe7d80b8ab0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fe7dc000c00 tx=0x7fe7dc005a20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:38:37.997 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:37.975+0000 7fe7e27fc640 1 -- 172.21.15.89:0/3434285416 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fe7e40ad560 con 0x7fe7ec108cf0 2024-09-21T03:38:38.161 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:38.159+0000 7fe7f363a640 1 -- 172.21.15.89:0/3434285416 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7fe7ec02e640 con 0x7fe7ec108cf0 2024-09-21T03:38:38.161 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:38.159+0000 7fe7e27fc640 1 -- 172.21.15.89:0/3434285416 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fe7e40b1940 con 0x7fe7ec108cf0 2024-09-21T03:38:38.161 INFO:teuthology.orchestra.run.smithi089.stdout:73014444107 2024-09-21T03:38:38.163 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:38.161+0000 7fe7f363a640 1 -- 172.21.15.89:0/3434285416 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7fe7d80b65a0 msgr2=0x7fe7d80b8ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:38.164 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:38.161+0000 7fe7f363a640 1 --2- 172.21.15.89:0/3434285416 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7fe7d80b65a0 0x7fe7d80b8ab0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fe7dc000c00 tx=0x7fe7dc005a20 comp rx=0 tx=0).stop 2024-09-21T03:38:38.164 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:38.161+0000 7fe7f363a640 1 -- 172.21.15.89:0/3434285416 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe7ec108cf0 msgr2=0x7fe7ec19eba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:38.164 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:38.161+0000 7fe7f363a640 1 --2- 172.21.15.89:0/3434285416 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe7ec108cf0 0x7fe7ec19eba0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fe7e400b970 tx=0x7fe7e400be30 comp rx=0 tx=0).stop 2024-09-21T03:38:38.164 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:38.161+0000 7fe7f363a640 1 -- 172.21.15.89:0/3434285416 shutdown_connections 2024-09-21T03:38:38.164 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:38.161+0000 7fe7f363a640 1 --2- 172.21.15.89:0/3434285416 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7fe7d80b65a0 0x7fe7d80b8ab0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:38.164 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:38.161+0000 7fe7f363a640 1 --2- 172.21.15.89:0/3434285416 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe7ec108cf0 0x7fe7ec19eba0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:38.164 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:38.161+0000 7fe7f363a640 1 --2- 172.21.15.89:0/3434285416 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe7ec1083a0 0x7fe7ec199900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:38.164 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:38.161+0000 7fe7f363a640 1 --2- 172.21.15.89:0/3434285416 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe7ec102a60 0x7fe7ec199360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:38.164 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:38.161+0000 7fe7f363a640 1 -- 172.21.15.89:0/3434285416 >> 172.21.15.89:0/3434285416 conn(0x7fe7ec079730 msgr2=0x7fe7ec1015b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:38:38.164 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:38.162+0000 7fe7f363a640 1 -- 172.21.15.89:0/3434285416 shutdown_connections 2024-09-21T03:38:38.164 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:38.162+0000 7fe7f363a640 1 -- 172.21.15.89:0/3434285416 wait complete. 2024-09-21T03:38:38.711 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:38:38 smithi114 ceph-mon[34140]: from='client.? 172.21.15.89:0/1026961921' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 11} : dispatch 2024-09-21T03:38:38.711 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:38:38 smithi114 ceph-mon[34140]: from='client.? 172.21.15.89:0/3434285416' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 2} : dispatch 2024-09-21T03:38:38.750 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:38:38 smithi110 ceph-mon[34000]: from='client.? 172.21.15.89:0/1026961921' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 11} : dispatch 2024-09-21T03:38:38.750 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:38:38 smithi110 ceph-mon[34000]: from='client.? 172.21.15.89:0/3434285416' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 2} : dispatch 2024-09-21T03:38:38.811 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:38:38 smithi089 ceph-mon[30928]: from='client.? 172.21.15.89:0/1026961921' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 11} : dispatch 2024-09-21T03:38:38.811 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:38:38 smithi089 ceph-mon[30928]: from='client.? 172.21.15.89:0/3434285416' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 2} : dispatch 2024-09-21T03:38:38.843 INFO:tasks.cephadm.ceph_manager.ceph:need seq 249108103205 got 249108103208 for osd.9 2024-09-21T03:38:38.843 DEBUG:teuthology.parallel:result is None 2024-09-21T03:38:39.198 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:39.076+0000 7f651ae08640 1 -- 172.21.15.89:0/3020036407 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6514102b20 msgr2=0x7f6514102f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:39.198 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:39.076+0000 7f651ae08640 1 --2- 172.21.15.89:0/3020036407 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6514102b20 0x7f6514102f30 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f6508009900 tx=0x7f650802a410 comp rx=0 tx=0).stop 2024-09-21T03:38:39.198 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:39.076+0000 7f651ae08640 1 -- 172.21.15.89:0/3020036407 shutdown_connections 2024-09-21T03:38:39.198 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:39.076+0000 7f651ae08640 1 --2- 172.21.15.89:0/3020036407 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6514108d60 0x7f651410f590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:39.198 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:39.076+0000 7f651ae08640 1 --2- 172.21.15.89:0/3020036407 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6514108410 0x7f6514108820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:39.198 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:39.076+0000 7f651ae08640 1 --2- 172.21.15.89:0/3020036407 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6514102b20 0x7f6514102f30 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:39.198 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:39.076+0000 7f651ae08640 1 -- 172.21.15.89:0/3020036407 >> 172.21.15.89:0/3020036407 conn(0x7f6514079700 msgr2=0x7f6514079b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:38:39.198 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:39.076+0000 7f651ae08640 1 -- 172.21.15.89:0/3020036407 shutdown_connections 2024-09-21T03:38:39.198 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:39.076+0000 7f651ae08640 1 -- 172.21.15.89:0/3020036407 wait complete. 2024-09-21T03:38:39.198 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:39.077+0000 7f651ae08640 1 Processor -- start 2024-09-21T03:38:39.198 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:39.077+0000 7f651ae08640 1 -- start start 2024-09-21T03:38:39.199 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:39.077+0000 7f651ae08640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6514102b20 0x7f651419b2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:39.199 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:39.078+0000 7f651ae08640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6514108410 0x7f651419b7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:39.199 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:39.078+0000 7f651ae08640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6514108d60 0x7f651419bd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:39.199 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:39.078+0000 7f651ae08640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f65141128d0 con 0x7f6514108d60 2024-09-21T03:38:39.199 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:39.078+0000 7f651ae08640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f6514112750 con 0x7f6514102b20 2024-09-21T03:38:39.199 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:39.078+0000 7f651ae08640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f6514112a50 con 0x7f6514108410 2024-09-21T03:38:39.199 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:39.078+0000 7f65193a5640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6514108d60 0x7f651419bd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:39.199 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:39.078+0000 7f65193a5640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6514108d60 0x7f651419bd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:51122/0 (socket says 172.21.15.89:51122) 2024-09-21T03:38:39.199 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:39.078+0000 7f65193a5640 1 -- 172.21.15.89:0/2673531433 learned_addr learned my addr 172.21.15.89:0/2673531433 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-21T03:38:39.199 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:39.078+0000 7f6518ba4640 1 --2- 172.21.15.89:0/2673531433 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6514102b20 0x7f651419b2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:39.199 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:39.078+0000 7f6513fff640 1 --2- 172.21.15.89:0/2673531433 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6514108410 0x7f651419b7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:39.199 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:39.078+0000 7f65193a5640 1 -- 172.21.15.89:0/2673531433 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6514108410 msgr2=0x7f651419b7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:39.199 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:39.078+0000 7f65193a5640 1 --2- 172.21.15.89:0/2673531433 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6514108410 0x7f651419b7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:39.199 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:39.078+0000 7f65193a5640 1 -- 172.21.15.89:0/2673531433 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6514102b20 msgr2=0x7f651419b2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:39.199 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:39.078+0000 7f65193a5640 1 --2- 172.21.15.89:0/2673531433 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6514102b20 0x7f651419b2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:39.199 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:39.078+0000 7f65193a5640 1 -- 172.21.15.89:0/2673531433 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65141a6560 con 0x7f6514108d60 2024-09-21T03:38:39.200 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:39.078+0000 7f65193a5640 1 --2- 172.21.15.89:0/2673531433 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6514108d60 0x7f651419bd20 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f650c00cad0 tx=0x7f650c00cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:38:39.200 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:39.078+0000 7f6511ffb640 1 -- 172.21.15.89:0/2673531433 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f650c00ae20 con 0x7f6514108d60 2024-09-21T03:38:39.200 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:39.078+0000 7f651ae08640 1 -- 172.21.15.89:0/2673531433 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f65141a6850 con 0x7f6514108d60 2024-09-21T03:38:39.200 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:39.078+0000 7f6511ffb640 1 -- 172.21.15.89:0/2673531433 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f650c005020 con 0x7f6514108d60 2024-09-21T03:38:39.200 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:39.078+0000 7f6511ffb640 1 -- 172.21.15.89:0/2673531433 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f650c00f6b0 con 0x7f6514108d60 2024-09-21T03:38:39.200 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:39.079+0000 7f651ae08640 1 -- 172.21.15.89:0/2673531433 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65141a6d90 con 0x7f6514108d60 2024-09-21T03:38:39.200 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:39.080+0000 7f651ae08640 1 -- 172.21.15.89:0/2673531433 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6514103d50 con 0x7f6514108d60 2024-09-21T03:38:39.200 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:39.081+0000 7f6511ffb640 1 -- 172.21.15.89:0/2673531433 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 18) ==== 154563+0+0 (secure 0 0 0) 0x7f650c0051d0 con 0x7f6514108d60 2024-09-21T03:38:39.200 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:39.082+0000 7f6511ffb640 1 --2- 172.21.15.89:0/2673531433 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f65040b65a0 0x7f65040b8ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:39.200 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:39.082+0000 7f6518ba4640 1 --2- 172.21.15.89:0/2673531433 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f65040b65a0 0x7f65040b8ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:39.200 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:39.082+0000 7f6511ffb640 1 -- 172.21.15.89:0/2673531433 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 9092+0+0 (secure 0 0 0) 0x7f650c0e7b00 con 0x7f6514108d60 2024-09-21T03:38:39.200 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:39.082+0000 7f6518ba4640 1 --2- 172.21.15.89:0/2673531433 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f65040b65a0 0x7f65040b8ab0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f651406c6e0 tx=0x7f65080058d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:38:39.200 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:39.086+0000 7f6511ffb640 1 -- 172.21.15.89:0/2673531433 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f650c014030 con 0x7f6514108d60 2024-09-21T03:38:39.272 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:39.270+0000 7f651ae08640 1 -- 172.21.15.89:0/2673531433 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 8} v 0) -- 0x7f651410a5a0 con 0x7f6514108d60 2024-09-21T03:38:39.272 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:39.270+0000 7f6511ffb640 1 -- 172.21.15.89:0/2673531433 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 8}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f650c0b22b0 con 0x7f6514108d60 2024-09-21T03:38:39.273 INFO:teuthology.orchestra.run.smithi089.stdout:223338299437 2024-09-21T03:38:39.275 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:39.273+0000 7f651ae08640 1 -- 172.21.15.89:0/2673531433 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f65040b65a0 msgr2=0x7f65040b8ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:39.275 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:39.273+0000 7f651ae08640 1 --2- 172.21.15.89:0/2673531433 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f65040b65a0 0x7f65040b8ab0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f651406c6e0 tx=0x7f65080058d0 comp rx=0 tx=0).stop 2024-09-21T03:38:39.275 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:39.273+0000 7f651ae08640 1 -- 172.21.15.89:0/2673531433 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6514108d60 msgr2=0x7f651419bd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:39.275 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:39.273+0000 7f651ae08640 1 --2- 172.21.15.89:0/2673531433 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6514108d60 0x7f651419bd20 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f650c00cad0 tx=0x7f650c00cf90 comp rx=0 tx=0).stop 2024-09-21T03:38:39.275 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:39.273+0000 7f651ae08640 1 -- 172.21.15.89:0/2673531433 shutdown_connections 2024-09-21T03:38:39.275 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:39.273+0000 7f651ae08640 1 --2- 172.21.15.89:0/2673531433 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f65040b65a0 0x7f65040b8ab0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:39.275 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:39.273+0000 7f651ae08640 1 --2- 172.21.15.89:0/2673531433 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6514108d60 0x7f651419bd20 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:39.276 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:39.273+0000 7f651ae08640 1 --2- 172.21.15.89:0/2673531433 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6514108410 0x7f651419b7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:39.276 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:39.273+0000 7f651ae08640 1 --2- 172.21.15.89:0/2673531433 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6514102b20 0x7f651419b2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:39.276 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:39.273+0000 7f651ae08640 1 -- 172.21.15.89:0/2673531433 >> 172.21.15.89:0/2673531433 conn(0x7f6514079700 msgr2=0x7f65141065c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:38:39.276 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:39.273+0000 7f651ae08640 1 -- 172.21.15.89:0/2673531433 shutdown_connections 2024-09-21T03:38:39.276 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:39.273+0000 7f651ae08640 1 -- 172.21.15.89:0/2673531433 wait complete. 2024-09-21T03:38:39.618 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:38:39 smithi089 ceph-mon[30928]: pgmap v335: 129 pgs: 129 active+clean; 646 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-21T03:38:39.618 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:38:39 smithi089 ceph-mon[30928]: from='client.? 172.21.15.89:0/2673531433' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 8} : dispatch 2024-09-21T03:38:39.645 INFO:tasks.cephadm.ceph_manager.ceph:need seq 176093659187 got 176093659190 for osd.6 2024-09-21T03:38:39.645 DEBUG:teuthology.parallel:result is None 2024-09-21T03:38:39.656 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/7c374098-77c9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:38:39.711 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:38:39 smithi114 ceph-mon[34140]: pgmap v335: 129 pgs: 129 active+clean; 646 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-21T03:38:39.711 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:38:39 smithi114 ceph-mon[34140]: from='client.? 172.21.15.89:0/2673531433' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 8} : dispatch 2024-09-21T03:38:39.750 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:38:39 smithi110 ceph-mon[34000]: pgmap v335: 129 pgs: 129 active+clean; 646 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-21T03:38:39.750 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:38:39 smithi110 ceph-mon[34000]: from='client.? 172.21.15.89:0/2673531433' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 8} : dispatch 2024-09-21T03:38:40.048 INFO:tasks.cephadm.ceph_manager.ceph:need seq 150323855417 got 150323855419 for osd.5 2024-09-21T03:38:40.048 DEBUG:teuthology.parallel:result is None 2024-09-21T03:38:40.479 INFO:tasks.cephadm.ceph_manager.ceph:need seq 73014444106 got 73014444107 for osd.2 2024-09-21T03:38:40.479 DEBUG:teuthology.parallel:result is None 2024-09-21T03:38:40.939 INFO:tasks.cephadm.ceph_manager.ceph:need seq 292057776155 got 292057776158 for osd.11 2024-09-21T03:38:40.939 DEBUG:teuthology.parallel:result is None 2024-09-21T03:38:41.504 INFO:tasks.cephadm.ceph_manager.ceph:need seq 223338299433 got 223338299437 for osd.8 2024-09-21T03:38:41.504 DEBUG:teuthology.parallel:result is None 2024-09-21T03:38:41.711 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:38:41 smithi114 ceph-mon[34140]: pgmap v336: 129 pgs: 129 active+clean; 646 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-21T03:38:41.750 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:38:41 smithi110 ceph-mon[34000]: pgmap v336: 129 pgs: 129 active+clean; 646 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-21T03:38:41.751 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:38:41 smithi089 ceph-mon[30928]: pgmap v336: 129 pgs: 129 active+clean; 646 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-21T03:38:41.979 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:41.975+0000 7f52017bb640 1 -- 172.21.15.89:0/2071889312 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f51fc103bc0 msgr2=0x7f51fc103fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:41.980 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:41.975+0000 7f52017bb640 1 --2- 172.21.15.89:0/2071889312 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f51fc103bc0 0x7f51fc103fd0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f51ec009900 tx=0x7f51ec02a410 comp rx=0 tx=0).stop 2024-09-21T03:38:41.980 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:41.975+0000 7f52017bb640 1 -- 172.21.15.89:0/2071889312 shutdown_connections 2024-09-21T03:38:41.980 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:41.975+0000 7f52017bb640 1 --2- 172.21.15.89:0/2071889312 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f51fc1094b0 0x7f51fc1098c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:41.980 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:41.975+0000 7f52017bb640 1 --2- 172.21.15.89:0/2071889312 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f51fc109e00 0x7f51fc110630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:41.980 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:41.975+0000 7f52017bb640 1 --2- 172.21.15.89:0/2071889312 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f51fc103bc0 0x7f51fc103fd0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:41.980 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:41.975+0000 7f52017bb640 1 -- 172.21.15.89:0/2071889312 >> 172.21.15.89:0/2071889312 conn(0x7f51fc07b910 msgr2=0x7f51fc07bd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:38:41.980 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:41.975+0000 7f52017bb640 1 -- 172.21.15.89:0/2071889312 shutdown_connections 2024-09-21T03:38:41.980 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:41.975+0000 7f52017bb640 1 -- 172.21.15.89:0/2071889312 wait complete. 2024-09-21T03:38:41.980 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:41.976+0000 7f52017bb640 1 Processor -- start 2024-09-21T03:38:41.980 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:41.976+0000 7f52017bb640 1 -- start start 2024-09-21T03:38:41.980 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:41.976+0000 7f52017bb640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f51fc103bc0 0x7f51fc19a420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:41.980 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:41.976+0000 7f52017bb640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f51fc1094b0 0x7f51fc19a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:41.980 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:41.976+0000 7f52017bb640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f51fc109e00 0x7f51fc19fc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:41.981 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:41.976+0000 7f52017bb640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f51fc1139d0 con 0x7f51fc109e00 2024-09-21T03:38:41.981 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:41.976+0000 7f52017bb640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f51fc113850 con 0x7f51fc103bc0 2024-09-21T03:38:41.981 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:41.976+0000 7f52017bb640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f51fc113b50 con 0x7f51fc1094b0 2024-09-21T03:38:41.981 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:41.976+0000 7f51fb7fe640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f51fc109e00 0x7f51fc19fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:41.981 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:41.976+0000 7f51fb7fe640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f51fc109e00 0x7f51fc19fc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:51132/0 (socket says 172.21.15.89:51132) 2024-09-21T03:38:41.981 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:41.976+0000 7f51fb7fe640 1 -- 172.21.15.89:0/1363641548 learned_addr learned my addr 172.21.15.89:0/1363641548 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-21T03:38:41.981 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:41.976+0000 7f51faffd640 1 --2- 172.21.15.89:0/1363641548 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f51fc103bc0 0x7f51fc19a420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:41.981 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:41.976+0000 7f51fa7fc640 1 --2- 172.21.15.89:0/1363641548 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f51fc1094b0 0x7f51fc19a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:41.981 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:41.977+0000 7f51fb7fe640 1 -- 172.21.15.89:0/1363641548 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f51fc1094b0 msgr2=0x7f51fc19a9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:41.981 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:41.977+0000 7f51fb7fe640 1 --2- 172.21.15.89:0/1363641548 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f51fc1094b0 0x7f51fc19a9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:41.981 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:41.977+0000 7f51fb7fe640 1 -- 172.21.15.89:0/1363641548 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f51fc103bc0 msgr2=0x7f51fc19a420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:41.981 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:41.977+0000 7f51fb7fe640 1 --2- 172.21.15.89:0/1363641548 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f51fc103bc0 0x7f51fc19a420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:41.981 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:41.977+0000 7f51fb7fe640 1 -- 172.21.15.89:0/1363641548 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51fc1a0390 con 0x7f51fc109e00 2024-09-21T03:38:41.981 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:41.977+0000 7f51faffd640 1 --2- 172.21.15.89:0/1363641548 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f51fc103bc0 0x7f51fc19a420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:38:41.982 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:41.977+0000 7f51fb7fe640 1 --2- 172.21.15.89:0/1363641548 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f51fc109e00 0x7f51fc19fc60 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f51f400b970 tx=0x7f51f400be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:38:41.982 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:41.977+0000 7f51e3fff640 1 -- 172.21.15.89:0/1363641548 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51f4019070 con 0x7f51fc109e00 2024-09-21T03:38:41.982 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:41.977+0000 7f51e3fff640 1 -- 172.21.15.89:0/1363641548 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f51f4014440 con 0x7f51fc109e00 2024-09-21T03:38:41.982 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:41.977+0000 7f52017bb640 1 -- 172.21.15.89:0/1363641548 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f51fc1a0680 con 0x7f51fc109e00 2024-09-21T03:38:41.982 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:41.977+0000 7f51e3fff640 1 -- 172.21.15.89:0/1363641548 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51f40075d0 con 0x7f51fc109e00 2024-09-21T03:38:41.982 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:41.977+0000 7f52017bb640 1 -- 172.21.15.89:0/1363641548 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51fc1a6de0 con 0x7f51fc109e00 2024-09-21T03:38:41.983 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:41.978+0000 7f52017bb640 1 -- 172.21.15.89:0/1363641548 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f51fc103fd0 con 0x7f51fc109e00 2024-09-21T03:38:41.983 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:41.980+0000 7f51e3fff640 1 -- 172.21.15.89:0/1363641548 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 18) ==== 154563+0+0 (secure 0 0 0) 0x7f51f401c070 con 0x7f51fc109e00 2024-09-21T03:38:41.983 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:41.980+0000 7f51e3fff640 1 --2- 172.21.15.89:0/1363641548 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f51e40b65a0 0x7f51e40b8ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:41.983 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:41.981+0000 7f51faffd640 1 --2- 172.21.15.89:0/1363641548 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f51e40b65a0 0x7f51e40b8ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:41.983 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:41.981+0000 7f51e3fff640 1 -- 172.21.15.89:0/1363641548 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 9092+0+0 (secure 0 0 0) 0x7f51f40e7150 con 0x7f51fc109e00 2024-09-21T03:38:41.983 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:41.981+0000 7f51faffd640 1 --2- 172.21.15.89:0/1363641548 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f51e40b65a0 0x7f51e40b8ab0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f51ec0057c0 tx=0x7f51ec0058d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:38:41.986 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:41.984+0000 7f51e3fff640 1 -- 172.21.15.89:0/1363641548 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f51f40b1900 con 0x7f51fc109e00 2024-09-21T03:38:42.167 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:42.165+0000 7f52017bb640 1 -- 172.21.15.89:0/1363641548 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7f51fc19b550 con 0x7f51fc109e00 2024-09-21T03:38:42.168 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:42.165+0000 7f51e3fff640 1 -- 172.21.15.89:0/1363641548 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f51fc19b550 con 0x7f51fc109e00 2024-09-21T03:38:42.168 INFO:teuthology.orchestra.run.smithi089.stdout:103079215173 2024-09-21T03:38:42.170 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:42.168+0000 7f52017bb640 1 -- 172.21.15.89:0/1363641548 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f51e40b65a0 msgr2=0x7f51e40b8ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:42.170 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:42.168+0000 7f52017bb640 1 --2- 172.21.15.89:0/1363641548 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f51e40b65a0 0x7f51e40b8ab0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f51ec0057c0 tx=0x7f51ec0058d0 comp rx=0 tx=0).stop 2024-09-21T03:38:42.170 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:42.168+0000 7f52017bb640 1 -- 172.21.15.89:0/1363641548 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f51fc109e00 msgr2=0x7f51fc19fc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:42.170 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:42.168+0000 7f52017bb640 1 --2- 172.21.15.89:0/1363641548 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f51fc109e00 0x7f51fc19fc60 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f51f400b970 tx=0x7f51f400be30 comp rx=0 tx=0).stop 2024-09-21T03:38:42.170 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:42.168+0000 7f52017bb640 1 -- 172.21.15.89:0/1363641548 shutdown_connections 2024-09-21T03:38:42.170 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:42.168+0000 7f52017bb640 1 --2- 172.21.15.89:0/1363641548 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f51e40b65a0 0x7f51e40b8ab0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:42.171 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:42.168+0000 7f52017bb640 1 --2- 172.21.15.89:0/1363641548 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f51fc109e00 0x7f51fc19fc60 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:42.171 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:42.168+0000 7f52017bb640 1 --2- 172.21.15.89:0/1363641548 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f51fc1094b0 0x7f51fc19a9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:42.171 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:42.168+0000 7f52017bb640 1 --2- 172.21.15.89:0/1363641548 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f51fc103bc0 0x7f51fc19a420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:42.171 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:42.168+0000 7f52017bb640 1 -- 172.21.15.89:0/1363641548 >> 172.21.15.89:0/1363641548 conn(0x7f51fc07b910 msgr2=0x7f51fc107600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:38:42.171 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:42.168+0000 7f52017bb640 1 -- 172.21.15.89:0/1363641548 shutdown_connections 2024-09-21T03:38:42.171 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:42.168+0000 7f52017bb640 1 -- 172.21.15.89:0/1363641548 wait complete. 2024-09-21T03:38:42.750 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:38:42 smithi110 ceph-mon[34000]: from='client.? 172.21.15.89:0/1363641548' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 3} : dispatch 2024-09-21T03:38:42.784 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:38:42 smithi089 ceph-mon[30928]: from='client.? 172.21.15.89:0/1363641548' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 3} : dispatch 2024-09-21T03:38:42.814 INFO:tasks.cephadm.ceph_manager.ceph:need seq 103079215171 got 103079215173 for osd.3 2024-09-21T03:38:42.815 DEBUG:teuthology.parallel:result is None 2024-09-21T03:38:42.815 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-09-21T03:38:42.815 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 7c374098-77c9-11ef-baf5-efdc52797490 -- ceph pg dump --format=json 2024-09-21T03:38:42.960 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:38:42 smithi114 ceph-mon[34140]: from='client.? 172.21.15.89:0/1363641548' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 3} : dispatch 2024-09-21T03:38:43.063 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/7c374098-77c9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:38:43.567 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:43.534+0000 7f15ea869640 1 -- 172.21.15.89:0/2903930878 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f15e4108d70 msgr2=0x7f15e410f5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:43.568 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:43.534+0000 7f15ea869640 1 --2- 172.21.15.89:0/2903930878 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f15e4108d70 0x7f15e410f5a0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f15dc009ca0 tx=0x7f15dc02a8c0 comp rx=0 tx=0).stop 2024-09-21T03:38:43.568 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:43.534+0000 7f15ea869640 1 -- 172.21.15.89:0/2903930878 shutdown_connections 2024-09-21T03:38:43.568 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:43.534+0000 7f15ea869640 1 --2- 172.21.15.89:0/2903930878 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f15e4108d70 0x7f15e410f5a0 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:43.568 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:43.534+0000 7f15ea869640 1 --2- 172.21.15.89:0/2903930878 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f15e4108420 0x7f15e4108830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:43.568 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:43.534+0000 7f15ea869640 1 --2- 172.21.15.89:0/2903930878 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f15e4102ae0 0x7f15e4102ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:43.568 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:43.534+0000 7f15ea869640 1 -- 172.21.15.89:0/2903930878 >> 172.21.15.89:0/2903930878 conn(0x7f15e4079770 msgr2=0x7f15e4079b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:38:43.568 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:43.534+0000 7f15ea869640 1 -- 172.21.15.89:0/2903930878 shutdown_connections 2024-09-21T03:38:43.568 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:43.534+0000 7f15ea869640 1 -- 172.21.15.89:0/2903930878 wait complete. 2024-09-21T03:38:43.568 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:43.535+0000 7f15ea869640 1 Processor -- start 2024-09-21T03:38:43.568 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:43.535+0000 7f15ea869640 1 -- start start 2024-09-21T03:38:43.568 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:43.535+0000 7f15ea869640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f15e4102ae0 0x7f15e419a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:43.568 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:43.535+0000 7f15ea869640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f15e4108420 0x7f15e419aa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:43.569 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:43.535+0000 7f15ea869640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f15e4108d70 0x7f15e419fca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:43.569 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:43.535+0000 7f15ea869640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f15e4113a30 con 0x7f15e4108d70 2024-09-21T03:38:43.569 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:43.535+0000 7f15ea869640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f15e41138b0 con 0x7f15e4108420 2024-09-21T03:38:43.569 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:43.535+0000 7f15ea869640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f15e4113bb0 con 0x7f15e4102ae0 2024-09-21T03:38:43.569 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:43.535+0000 7f15e8e06640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f15e4108d70 0x7f15e419fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:43.569 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:43.535+0000 7f15e8e06640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f15e4108d70 0x7f15e419fca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:51154/0 (socket says 172.21.15.89:51154) 2024-09-21T03:38:43.569 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:43.535+0000 7f15e8e06640 1 -- 172.21.15.89:0/2988518107 learned_addr learned my addr 172.21.15.89:0/2988518107 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-21T03:38:43.569 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:43.535+0000 7f15e3fff640 1 --2- 172.21.15.89:0/2988518107 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f15e4102ae0 0x7f15e419a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:43.569 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:43.535+0000 7f15e37fe640 1 --2- 172.21.15.89:0/2988518107 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f15e4108420 0x7f15e419aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:43.569 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:43.536+0000 7f15e8e06640 1 -- 172.21.15.89:0/2988518107 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f15e4102ae0 msgr2=0x7f15e419a460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:43.569 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:43.536+0000 7f15e8e06640 1 --2- 172.21.15.89:0/2988518107 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f15e4102ae0 0x7f15e419a460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:43.569 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:43.536+0000 7f15e8e06640 1 -- 172.21.15.89:0/2988518107 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f15e4108420 msgr2=0x7f15e419aa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:43.569 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:43.536+0000 7f15e8e06640 1 --2- 172.21.15.89:0/2988518107 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f15e4108420 0x7f15e419aa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:43.569 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:43.536+0000 7f15e8e06640 1 -- 172.21.15.89:0/2988518107 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15e41a03d0 con 0x7f15e4108d70 2024-09-21T03:38:43.570 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:43.536+0000 7f15e8e06640 1 --2- 172.21.15.89:0/2988518107 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f15e4108d70 0x7f15e419fca0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f15dc005cc0 tx=0x7f15dc032a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:38:43.570 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:43.536+0000 7f15e17fa640 1 -- 172.21.15.89:0/2988518107 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15dc032e00 con 0x7f15e4108d70 2024-09-21T03:38:43.570 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:43.536+0000 7f15e17fa640 1 -- 172.21.15.89:0/2988518107 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f15dc0044d0 con 0x7f15e4108d70 2024-09-21T03:38:43.570 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:43.536+0000 7f15ea869640 1 -- 172.21.15.89:0/2988518107 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f15e41a0660 con 0x7f15e4108d70 2024-09-21T03:38:43.570 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:43.536+0000 7f15e17fa640 1 -- 172.21.15.89:0/2988518107 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15dc030b70 con 0x7f15e4108d70 2024-09-21T03:38:43.570 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:43.536+0000 7f15ea869640 1 -- 172.21.15.89:0/2988518107 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15e41a6e20 con 0x7f15e4108d70 2024-09-21T03:38:43.570 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:43.537+0000 7f15ea869640 1 -- 172.21.15.89:0/2988518107 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f15e4103d60 con 0x7f15e4108d70 2024-09-21T03:38:43.570 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:43.538+0000 7f15e17fa640 1 -- 172.21.15.89:0/2988518107 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 18) ==== 154563+0+0 (secure 0 0 0) 0x7f15dc004680 con 0x7f15e4108d70 2024-09-21T03:38:43.570 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:43.539+0000 7f15e17fa640 1 --2- 172.21.15.89:0/2988518107 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f15cc0b6620 0x7f15cc0b8b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:43.570 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:43.539+0000 7f15e3fff640 1 --2- 172.21.15.89:0/2988518107 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f15cc0b6620 0x7f15cc0b8b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:43.570 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:43.539+0000 7f15e17fa640 1 -- 172.21.15.89:0/2988518107 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 9092+0+0 (secure 0 0 0) 0x7f15dc108930 con 0x7f15e4108d70 2024-09-21T03:38:43.570 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:43.540+0000 7f15e3fff640 1 --2- 172.21.15.89:0/2988518107 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f15cc0b6620 0x7f15cc0b8b30 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f15d400bbe0 tx=0x7f15d400b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:38:43.570 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:43.544+0000 7f15e17fa640 1 -- 172.21.15.89:0/2988518107 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f15dc0ced00 con 0x7f15e4108d70 2024-09-21T03:38:43.722 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:43.720+0000 7f15ea869640 1 -- 172.21.15.89:0/2988518107 --> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f15e410abc0 con 0x7f15cc0b6620 2024-09-21T03:38:43.722 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:38:43 smithi089 ceph-mon[30928]: pgmap v337: 129 pgs: 129 active+clean; 646 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-21T03:38:43.728 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:43.726+0000 7f15e17fa640 1 -- 172.21.15.89:0/2988518107 <== mgr.14152 v2:172.21.15.89:6800/1591532358 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340623 (secure 0 0 0) 0x7f15e410abc0 con 0x7f15cc0b6620 2024-09-21T03:38:43.731 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-09-21T03:38:43.733 INFO:teuthology.orchestra.run.smithi089.stderr:dumped all 2024-09-21T03:38:43.735 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:43.729+0000 7f15ea869640 1 -- 172.21.15.89:0/2988518107 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f15cc0b6620 msgr2=0x7f15cc0b8b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:43.735 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:43.729+0000 7f15ea869640 1 --2- 172.21.15.89:0/2988518107 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f15cc0b6620 0x7f15cc0b8b30 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f15d400bbe0 tx=0x7f15d400b390 comp rx=0 tx=0).stop 2024-09-21T03:38:43.735 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:43.729+0000 7f15ea869640 1 -- 172.21.15.89:0/2988518107 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f15e4108d70 msgr2=0x7f15e419fca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:43.735 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:43.729+0000 7f15ea869640 1 --2- 172.21.15.89:0/2988518107 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f15e4108d70 0x7f15e419fca0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f15dc005cc0 tx=0x7f15dc032a10 comp rx=0 tx=0).stop 2024-09-21T03:38:43.735 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:43.729+0000 7f15ea869640 1 -- 172.21.15.89:0/2988518107 shutdown_connections 2024-09-21T03:38:43.735 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:43.729+0000 7f15ea869640 1 --2- 172.21.15.89:0/2988518107 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f15cc0b6620 0x7f15cc0b8b30 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:43.735 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:43.729+0000 7f15ea869640 1 --2- 172.21.15.89:0/2988518107 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f15e4108420 0x7f15e419aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:43.735 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:43.729+0000 7f15ea869640 1 --2- 172.21.15.89:0/2988518107 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f15e4108d70 0x7f15e419fca0 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:43.735 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:43.729+0000 7f15ea869640 1 --2- 172.21.15.89:0/2988518107 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f15e4102ae0 0x7f15e419a460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:43.736 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:43.729+0000 7f15ea869640 1 -- 172.21.15.89:0/2988518107 >> 172.21.15.89:0/2988518107 conn(0x7f15e4079770 msgr2=0x7f15e4106510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:38:43.736 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:43.729+0000 7f15ea869640 1 -- 172.21.15.89:0/2988518107 shutdown_connections 2024-09-21T03:38:43.736 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:43.729+0000 7f15ea869640 1 -- 172.21.15.89:0/2988518107 wait complete. 2024-09-21T03:38:43.750 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:38:43 smithi110 ceph-mon[34000]: pgmap v337: 129 pgs: 129 active+clean; 646 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-21T03:38:43.961 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:38:43 smithi114 ceph-mon[34140]: pgmap v337: 129 pgs: 129 active+clean; 646 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-21T03:38:44.406 INFO:teuthology.orchestra.run.smithi089.stdout:{"pg_ready":true,"pg_map":{"version":338,"stamp":"2024-09-21T03:38:43.634020+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":661741,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":413,"num_read_kb":304,"num_write":289,"num_write_kb":2271,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":23,"num_bytes_recovered":2952534,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":235,"ondisk_log_size":235,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":376932,"kb_used_data":11024,"kb_used_omap":18,"kb_used_meta":365677,"kb_avail":1124564892,"statfs":{"total":1151940427776,"available":1151554449408,"internally_reserved":0,"allocated":11288576,"data_stored":7496429,"data_compressed":390510,"data_compressed_allocated":528384,"data_compressed_original":2555904,"omap_allocated":19143,"internal_metadata":374453561},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":70,"num_read_kb":30,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003166"},"pg_stats":[{"pgid":"2.1f","version":"80'9","reported_seq":45,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.047700+0000","last_change":"2024-09-21T03:37:10.836953+0000","last_active":"2024-09-21T03:37:53.047700+0000","last_peered":"2024-09-21T03:37:53.047700+0000","last_clean":"2024-09-21T03:37:53.047700+0000","last_became_active":"2024-09-21T03:37:10.836566+0000","last_became_peered":"2024-09-21T03:37:10.836566+0000","last_unstale":"2024-09-21T03:37:53.047700+0000","last_undegraded":"2024-09-21T03:37:53.047700+0000","last_fullsized":"2024-09-21T03:37:53.047700+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_clean_scrub_stamp":"2024-09-21T03:37:09.809805+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:15:45.591225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.046350+0000","last_change":"2024-09-21T03:37:12.866931+0000","last_active":"2024-09-21T03:37:53.046350+0000","last_peered":"2024-09-21T03:37:53.046350+0000","last_clean":"2024-09-21T03:37:53.046350+0000","last_became_active":"2024-09-21T03:37:12.866452+0000","last_became_peered":"2024-09-21T03:37:12.866452+0000","last_unstale":"2024-09-21T03:37:53.046350+0000","last_undegraded":"2024-09-21T03:37:53.046350+0000","last_fullsized":"2024-09-21T03:37:53.046350+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:37:14.324548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.044138+0000","last_change":"2024-09-21T03:37:19.912154+0000","last_active":"2024-09-21T03:37:53.044138+0000","last_peered":"2024-09-21T03:37:53.044138+0000","last_clean":"2024-09-21T03:37:53.044138+0000","last_became_active":"2024-09-21T03:37:19.911686+0000","last_became_peered":"2024-09-21T03:37:19.911686+0000","last_unstale":"2024-09-21T03:37:53.044138+0000","last_undegraded":"2024-09-21T03:37:53.044138+0000","last_fullsized":"2024-09-21T03:37:53.044138+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_clean_scrub_stamp":"2024-09-21T03:37:18.876933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:29:20.189418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"80'2","reported_seq":37,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.046676+0000","last_change":"2024-09-21T03:37:11.034501+0000","last_active":"2024-09-21T03:37:53.046676+0000","last_peered":"2024-09-21T03:37:53.046676+0000","last_clean":"2024-09-21T03:37:53.046676+0000","last_became_active":"2024-09-21T03:37:11.034110+0000","last_became_peered":"2024-09-21T03:37:11.034110+0000","last_unstale":"2024-09-21T03:37:53.046676+0000","last_undegraded":"2024-09-21T03:37:53.046676+0000","last_fullsized":"2024-09-21T03:37:53.046676+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_clean_scrub_stamp":"2024-09-21T03:37:09.809805+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:03:12.854469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.049598+0000","last_change":"2024-09-21T03:37:12.867330+0000","last_active":"2024-09-21T03:37:53.049598+0000","last_peered":"2024-09-21T03:37:53.049598+0000","last_clean":"2024-09-21T03:37:53.049598+0000","last_became_active":"2024-09-21T03:37:12.866952+0000","last_became_peered":"2024-09-21T03:37:12.866952+0000","last_unstale":"2024-09-21T03:37:53.049598+0000","last_undegraded":"2024-09-21T03:37:53.049598+0000","last_fullsized":"2024-09-21T03:37:53.049598+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:40:33.826546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.043412+0000","last_change":"2024-09-21T03:37:19.910661+0000","last_active":"2024-09-21T03:37:53.043412+0000","last_peered":"2024-09-21T03:37:53.043412+0000","last_clean":"2024-09-21T03:37:53.043412+0000","last_became_active":"2024-09-21T03:37:19.910108+0000","last_became_peered":"2024-09-21T03:37:19.910108+0000","last_unstale":"2024-09-21T03:37:53.043412+0000","last_undegraded":"2024-09-21T03:37:53.043412+0000","last_fullsized":"2024-09-21T03:37:53.043412+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_clean_scrub_stamp":"2024-09-21T03:37:18.876933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:18:05.654426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"80'4","reported_seq":39,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.050839+0000","last_change":"2024-09-21T03:37:10.835336+0000","last_active":"2024-09-21T03:37:53.050839+0000","last_peered":"2024-09-21T03:37:53.050839+0000","last_clean":"2024-09-21T03:37:53.050839+0000","last_became_active":"2024-09-21T03:37:10.834928+0000","last_became_peered":"2024-09-21T03:37:10.834928+0000","last_unstale":"2024-09-21T03:37:53.050839+0000","last_undegraded":"2024-09-21T03:37:53.050839+0000","last_fullsized":"2024-09-21T03:37:53.050839+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_clean_scrub_stamp":"2024-09-21T03:37:09.809805+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:02:14.717520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.111510+0000","last_change":"2024-09-21T03:37:12.861271+0000","last_active":"2024-09-21T03:37:53.111510+0000","last_peered":"2024-09-21T03:37:53.111510+0000","last_clean":"2024-09-21T03:37:53.111510+0000","last_became_active":"2024-09-21T03:37:12.860932+0000","last_became_peered":"2024-09-21T03:37:12.860932+0000","last_unstale":"2024-09-21T03:37:53.111510+0000","last_undegraded":"2024-09-21T03:37:53.111510+0000","last_fullsized":"2024-09-21T03:37:53.111510+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:46:08.993250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.053171+0000","last_change":"2024-09-21T03:37:53.053045+0000","last_active":"2024-09-21T03:37:53.053171+0000","last_peered":"2024-09-21T03:37:53.053171+0000","last_clean":"2024-09-21T03:37:53.053171+0000","last_became_active":"2024-09-21T03:37:53.052382+0000","last_became_peered":"2024-09-21T03:37:53.052382+0000","last_unstale":"2024-09-21T03:37:53.053171+0000","last_undegraded":"2024-09-21T03:37:53.053171+0000","last_fullsized":"2024-09-21T03:37:53.053171+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_clean_scrub_stamp":"2024-09-21T03:37:18.876933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:02:44.300768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,8],"acting":[4,3,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"74'2","reported_seq":37,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.050328+0000","last_change":"2024-09-21T03:37:10.833985+0000","last_active":"2024-09-21T03:37:53.050328+0000","last_peered":"2024-09-21T03:37:53.050328+0000","last_clean":"2024-09-21T03:37:53.050328+0000","last_became_active":"2024-09-21T03:37:10.833562+0000","last_became_peered":"2024-09-21T03:37:10.833562+0000","last_unstale":"2024-09-21T03:37:53.050328+0000","last_undegraded":"2024-09-21T03:37:53.050328+0000","last_fullsized":"2024-09-21T03:37:53.050328+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_clean_scrub_stamp":"2024-09-21T03:37:09.809805+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:49:32.617768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.047066+0000","last_change":"2024-09-21T03:37:12.865408+0000","last_active":"2024-09-21T03:37:53.047066+0000","last_peered":"2024-09-21T03:37:53.047066+0000","last_clean":"2024-09-21T03:37:53.047066+0000","last_became_active":"2024-09-21T03:37:12.864972+0000","last_became_peered":"2024-09-21T03:37:12.864972+0000","last_unstale":"2024-09-21T03:37:53.047066+0000","last_undegraded":"2024-09-21T03:37:53.047066+0000","last_fullsized":"2024-09-21T03:37:53.047066+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:03:16.384260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.043495+0000","last_change":"2024-09-21T03:37:19.912105+0000","last_active":"2024-09-21T03:37:53.043495+0000","last_peered":"2024-09-21T03:37:53.043495+0000","last_clean":"2024-09-21T03:37:53.043495+0000","last_became_active":"2024-09-21T03:37:19.911416+0000","last_became_peered":"2024-09-21T03:37:19.911416+0000","last_unstale":"2024-09-21T03:37:53.043495+0000","last_undegraded":"2024-09-21T03:37:53.043495+0000","last_fullsized":"2024-09-21T03:37:53.043495+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_clean_scrub_stamp":"2024-09-21T03:37:18.876933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:20:16.492231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.431254+0000","last_change":"2024-09-21T03:37:12.862354+0000","last_active":"2024-09-21T03:37:53.431254+0000","last_peered":"2024-09-21T03:37:53.431254+0000","last_clean":"2024-09-21T03:37:53.431254+0000","last_became_active":"2024-09-21T03:37:12.861977+0000","last_became_peered":"2024-09-21T03:37:12.861977+0000","last_unstale":"2024-09-21T03:37:53.431254+0000","last_undegraded":"2024-09-21T03:37:53.431254+0000","last_fullsized":"2024-09-21T03:37:53.431254+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:53:52.725524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"80'4","reported_seq":40,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.051416+0000","last_change":"2024-09-21T03:37:10.832140+0000","last_active":"2024-09-21T03:37:53.051416+0000","last_peered":"2024-09-21T03:37:53.051416+0000","last_clean":"2024-09-21T03:37:53.051416+0000","last_became_active":"2024-09-21T03:37:10.831395+0000","last_became_peered":"2024-09-21T03:37:10.831395+0000","last_unstale":"2024-09-21T03:37:53.051416+0000","last_undegraded":"2024-09-21T03:37:53.051416+0000","last_fullsized":"2024-09-21T03:37:53.051416+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_clean_scrub_stamp":"2024-09-21T03:37:09.809805+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:55:20.440890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.044184+0000","last_change":"2024-09-21T03:37:20.160850+0000","last_active":"2024-09-21T03:37:53.044184+0000","last_peered":"2024-09-21T03:37:53.044184+0000","last_clean":"2024-09-21T03:37:53.044184+0000","last_became_active":"2024-09-21T03:37:20.160305+0000","last_became_peered":"2024-09-21T03:37:20.160305+0000","last_unstale":"2024-09-21T03:37:53.044184+0000","last_undegraded":"2024-09-21T03:37:53.044184+0000","last_fullsized":"2024-09-21T03:37:53.044184+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_clean_scrub_stamp":"2024-09-21T03:37:18.876933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:51:45.971984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.046377+0000","last_change":"2024-09-21T03:37:12.859485+0000","last_active":"2024-09-21T03:37:53.046377+0000","last_peered":"2024-09-21T03:37:53.046377+0000","last_clean":"2024-09-21T03:37:53.046377+0000","last_became_active":"2024-09-21T03:37:12.859097+0000","last_became_peered":"2024-09-21T03:37:12.859097+0000","last_unstale":"2024-09-21T03:37:53.046377+0000","last_undegraded":"2024-09-21T03:37:53.046377+0000","last_fullsized":"2024-09-21T03:37:53.046377+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:16:05.438360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"80'1","reported_seq":36,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.044264+0000","last_change":"2024-09-21T03:37:10.832441+0000","last_active":"2024-09-21T03:37:53.044264+0000","last_peered":"2024-09-21T03:37:53.044264+0000","last_clean":"2024-09-21T03:37:53.044264+0000","last_became_active":"2024-09-21T03:37:10.832073+0000","last_became_peered":"2024-09-21T03:37:10.832073+0000","last_unstale":"2024-09-21T03:37:53.044264+0000","last_undegraded":"2024-09-21T03:37:53.044264+0000","last_fullsized":"2024-09-21T03:37:53.044264+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_clean_scrub_stamp":"2024-09-21T03:37:09.809805+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:32:19.411097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"82'1","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.043995+0000","last_change":"2024-09-21T03:37:19.905490+0000","last_active":"2024-09-21T03:37:53.043995+0000","last_peered":"2024-09-21T03:37:53.043995+0000","last_clean":"2024-09-21T03:37:53.043995+0000","last_became_active":"2024-09-21T03:37:19.904971+0000","last_became_peered":"2024-09-21T03:37:19.904971+0000","last_unstale":"2024-09-21T03:37:53.043995+0000","last_undegraded":"2024-09-21T03:37:53.043995+0000","last_fullsized":"2024-09-21T03:37:53.043995+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_clean_scrub_stamp":"2024-09-21T03:37:18.876933+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:44:20.655538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.048957+0000","last_change":"2024-09-21T03:37:12.863597+0000","last_active":"2024-09-21T03:37:53.048957+0000","last_peered":"2024-09-21T03:37:53.048957+0000","last_clean":"2024-09-21T03:37:53.048957+0000","last_became_active":"2024-09-21T03:37:12.863223+0000","last_became_peered":"2024-09-21T03:37:12.863223+0000","last_unstale":"2024-09-21T03:37:53.048957+0000","last_undegraded":"2024-09-21T03:37:53.048957+0000","last_fullsized":"2024-09-21T03:37:53.048957+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:29:28.878964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.9","version":"80'3","reported_seq":38,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.045808+0000","last_change":"2024-09-21T03:37:10.834648+0000","last_active":"2024-09-21T03:37:53.045808+0000","last_peered":"2024-09-21T03:37:53.045808+0000","last_clean":"2024-09-21T03:37:53.045808+0000","last_became_active":"2024-09-21T03:37:10.834293+0000","last_became_peered":"2024-09-21T03:37:10.834293+0000","last_unstale":"2024-09-21T03:37:53.045808+0000","last_undegraded":"2024-09-21T03:37:53.045808+0000","last_fullsized":"2024-09-21T03:37:53.045808+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_clean_scrub_stamp":"2024-09-21T03:37:09.809805+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:10:41.248733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.047492+0000","last_change":"2024-09-21T03:37:19.912526+0000","last_active":"2024-09-21T03:37:53.047492+0000","last_peered":"2024-09-21T03:37:53.047492+0000","last_clean":"2024-09-21T03:37:53.047492+0000","last_became_active":"2024-09-21T03:37:19.912062+0000","last_became_peered":"2024-09-21T03:37:19.912062+0000","last_unstale":"2024-09-21T03:37:53.047492+0000","last_undegraded":"2024-09-21T03:37:53.047492+0000","last_fullsized":"2024-09-21T03:37:53.047492+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_clean_scrub_stamp":"2024-09-21T03:37:18.876933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:52:40.395834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.049515+0000","last_change":"2024-09-21T03:37:12.855672+0000","last_active":"2024-09-21T03:37:53.049515+0000","last_peered":"2024-09-21T03:37:53.049515+0000","last_clean":"2024-09-21T03:37:53.049515+0000","last_became_active":"2024-09-21T03:37:12.855191+0000","last_became_peered":"2024-09-21T03:37:12.855191+0000","last_unstale":"2024-09-21T03:37:53.049515+0000","last_undegraded":"2024-09-21T03:37:53.049515+0000","last_fullsized":"2024-09-21T03:37:53.049515+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:26:56.115697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.8","version":"80'2","reported_seq":37,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.111296+0000","last_change":"2024-09-21T03:37:11.034752+0000","last_active":"2024-09-21T03:37:53.111296+0000","last_peered":"2024-09-21T03:37:53.111296+0000","last_clean":"2024-09-21T03:37:53.111296+0000","last_became_active":"2024-09-21T03:37:11.034386+0000","last_became_peered":"2024-09-21T03:37:11.034386+0000","last_unstale":"2024-09-21T03:37:53.111296+0000","last_undegraded":"2024-09-21T03:37:53.111296+0000","last_fullsized":"2024-09-21T03:37:53.111296+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_clean_scrub_stamp":"2024-09-21T03:37:09.809805+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:17:23.647411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.044213+0000","last_change":"2024-09-21T03:37:20.162051+0000","last_active":"2024-09-21T03:37:53.044213+0000","last_peered":"2024-09-21T03:37:53.044213+0000","last_clean":"2024-09-21T03:37:53.044213+0000","last_became_active":"2024-09-21T03:37:20.161452+0000","last_became_peered":"2024-09-21T03:37:20.161452+0000","last_unstale":"2024-09-21T03:37:53.044213+0000","last_undegraded":"2024-09-21T03:37:53.044213+0000","last_fullsized":"2024-09-21T03:37:53.044213+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_clean_scrub_stamp":"2024-09-21T03:37:18.876933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:29:22.544800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.049528+0000","last_change":"2024-09-21T03:37:12.866058+0000","last_active":"2024-09-21T03:37:53.049528+0000","last_peered":"2024-09-21T03:37:53.049528+0000","last_clean":"2024-09-21T03:37:53.049528+0000","last_became_active":"2024-09-21T03:37:12.865694+0000","last_became_peered":"2024-09-21T03:37:12.865694+0000","last_unstale":"2024-09-21T03:37:53.049528+0000","last_undegraded":"2024-09-21T03:37:53.049528+0000","last_fullsized":"2024-09-21T03:37:53.049528+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:56:33.629873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"80'4","reported_seq":39,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.044782+0000","last_change":"2024-09-21T03:37:11.358619+0000","last_active":"2024-09-21T03:37:53.044782+0000","last_peered":"2024-09-21T03:37:53.044782+0000","last_clean":"2024-09-21T03:37:53.044782+0000","last_became_active":"2024-09-21T03:37:11.358196+0000","last_became_peered":"2024-09-21T03:37:11.358196+0000","last_unstale":"2024-09-21T03:37:53.044782+0000","last_undegraded":"2024-09-21T03:37:53.044782+0000","last_fullsized":"2024-09-21T03:37:53.044782+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_clean_scrub_stamp":"2024-09-21T03:37:09.809805+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:57:13.476641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.044378+0000","last_change":"2024-09-21T03:37:19.905711+0000","last_active":"2024-09-21T03:37:53.044378+0000","last_peered":"2024-09-21T03:37:53.044378+0000","last_clean":"2024-09-21T03:37:53.044378+0000","last_became_active":"2024-09-21T03:37:19.905260+0000","last_became_peered":"2024-09-21T03:37:19.905260+0000","last_unstale":"2024-09-21T03:37:53.044378+0000","last_undegraded":"2024-09-21T03:37:53.044378+0000","last_fullsized":"2024-09-21T03:37:53.044378+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_clean_scrub_stamp":"2024-09-21T03:37:18.876933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:21:09.476082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.046425+0000","last_change":"2024-09-21T03:37:12.854376+0000","last_active":"2024-09-21T03:37:53.046425+0000","last_peered":"2024-09-21T03:37:53.046425+0000","last_clean":"2024-09-21T03:37:53.046425+0000","last_became_active":"2024-09-21T03:37:12.852942+0000","last_became_peered":"2024-09-21T03:37:12.852942+0000","last_unstale":"2024-09-21T03:37:53.046425+0000","last_undegraded":"2024-09-21T03:37:53.046425+0000","last_fullsized":"2024-09-21T03:37:53.046425+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:06:49.319002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"82'10","reported_seq":58,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:38:03.865942+0000","last_change":"2024-09-21T03:37:53.056889+0000","last_active":"2024-09-21T03:38:03.865942+0000","last_peered":"2024-09-21T03:38:03.865942+0000","last_clean":"2024-09-21T03:38:03.865942+0000","last_became_active":"2024-09-21T03:37:53.053054+0000","last_became_peered":"2024-09-21T03:37:53.053054+0000","last_unstale":"2024-09-21T03:38:03.865942+0000","last_undegraded":"2024-09-21T03:38:03.865942+0000","last_fullsized":"2024-09-21T03:38:03.865942+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_clean_scrub_stamp":"2024-09-21T03:37:09.809805+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:16:05.539111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,3],"acting":[9,11,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.045339+0000","last_change":"2024-09-21T03:37:20.161515+0000","last_active":"2024-09-21T03:37:53.045339+0000","last_peered":"2024-09-21T03:37:53.045339+0000","last_clean":"2024-09-21T03:37:53.045339+0000","last_became_active":"2024-09-21T03:37:20.160548+0000","last_became_peered":"2024-09-21T03:37:20.160548+0000","last_unstale":"2024-09-21T03:37:53.045339+0000","last_undegraded":"2024-09-21T03:37:53.045339+0000","last_fullsized":"2024-09-21T03:37:53.045339+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_clean_scrub_stamp":"2024-09-21T03:37:18.876933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:17:51.144859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.042971+0000","last_change":"2024-09-21T03:37:12.848305+0000","last_active":"2024-09-21T03:37:53.042971+0000","last_peered":"2024-09-21T03:37:53.042971+0000","last_clean":"2024-09-21T03:37:53.042971+0000","last_became_active":"2024-09-21T03:37:12.847916+0000","last_became_peered":"2024-09-21T03:37:12.847916+0000","last_unstale":"2024-09-21T03:37:53.042971+0000","last_undegraded":"2024-09-21T03:37:53.042971+0000","last_fullsized":"2024-09-21T03:37:53.042971+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:48:39.922805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"80'1","reported_seq":37,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.051068+0000","last_change":"2024-09-21T03:37:11.035964+0000","last_active":"2024-09-21T03:37:53.051068+0000","last_peered":"2024-09-21T03:37:53.051068+0000","last_clean":"2024-09-21T03:37:53.051068+0000","last_became_active":"2024-09-21T03:37:11.035427+0000","last_became_peered":"2024-09-21T03:37:11.035427+0000","last_unstale":"2024-09-21T03:37:53.051068+0000","last_undegraded":"2024-09-21T03:37:53.051068+0000","last_fullsized":"2024-09-21T03:37:53.051068+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_clean_scrub_stamp":"2024-09-21T03:37:09.809805+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:04:51.026753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.047032+0000","last_change":"2024-09-21T03:37:19.909428+0000","last_active":"2024-09-21T03:37:53.047032+0000","last_peered":"2024-09-21T03:37:53.047032+0000","last_clean":"2024-09-21T03:37:53.047032+0000","last_became_active":"2024-09-21T03:37:19.908904+0000","last_became_peered":"2024-09-21T03:37:19.908904+0000","last_unstale":"2024-09-21T03:37:53.047032+0000","last_undegraded":"2024-09-21T03:37:53.047032+0000","last_fullsized":"2024-09-21T03:37:53.047032+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_clean_scrub_stamp":"2024-09-21T03:37:18.876933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:12:07.564058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.046987+0000","last_change":"2024-09-21T03:37:12.855592+0000","last_active":"2024-09-21T03:37:53.046987+0000","last_peered":"2024-09-21T03:37:53.046987+0000","last_clean":"2024-09-21T03:37:53.046987+0000","last_became_active":"2024-09-21T03:37:12.855244+0000","last_became_peered":"2024-09-21T03:37:12.855244+0000","last_unstale":"2024-09-21T03:37:53.046987+0000","last_undegraded":"2024-09-21T03:37:53.046987+0000","last_fullsized":"2024-09-21T03:37:53.046987+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:21:34.603141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":35,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.047381+0000","last_change":"2024-09-21T03:37:11.036401+0000","last_active":"2024-09-21T03:37:53.047381+0000","last_peered":"2024-09-21T03:37:53.047381+0000","last_clean":"2024-09-21T03:37:53.047381+0000","last_became_active":"2024-09-21T03:37:11.036003+0000","last_became_peered":"2024-09-21T03:37:11.036003+0000","last_unstale":"2024-09-21T03:37:53.047381+0000","last_undegraded":"2024-09-21T03:37:53.047381+0000","last_fullsized":"2024-09-21T03:37:53.047381+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_clean_scrub_stamp":"2024-09-21T03:37:09.809805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:35:50.549060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.050609+0000","last_change":"2024-09-21T03:37:20.160668+0000","last_active":"2024-09-21T03:37:53.050609+0000","last_peered":"2024-09-21T03:37:53.050609+0000","last_clean":"2024-09-21T03:37:53.050609+0000","last_became_active":"2024-09-21T03:37:20.160158+0000","last_became_peered":"2024-09-21T03:37:20.160158+0000","last_unstale":"2024-09-21T03:37:53.050609+0000","last_undegraded":"2024-09-21T03:37:53.050609+0000","last_fullsized":"2024-09-21T03:37:53.050609+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_clean_scrub_stamp":"2024-09-21T03:37:18.876933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:45:25.677667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":37,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.050453+0000","last_change":"2024-09-21T03:37:53.050351+0000","last_active":"2024-09-21T03:37:53.050453+0000","last_peered":"2024-09-21T03:37:53.050453+0000","last_clean":"2024-09-21T03:37:53.050453+0000","last_became_active":"2024-09-21T03:37:53.050003+0000","last_became_peered":"2024-09-21T03:37:53.050003+0000","last_unstale":"2024-09-21T03:37:53.050453+0000","last_undegraded":"2024-09-21T03:37:53.050453+0000","last_fullsized":"2024-09-21T03:37:53.050453+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:49:03.313002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,0],"acting":[10,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"80'1","reported_seq":36,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.431360+0000","last_change":"2024-09-21T03:37:10.832635+0000","last_active":"2024-09-21T03:37:53.431360+0000","last_peered":"2024-09-21T03:37:53.431360+0000","last_clean":"2024-09-21T03:37:53.431360+0000","last_became_active":"2024-09-21T03:37:10.832280+0000","last_became_peered":"2024-09-21T03:37:10.832280+0000","last_unstale":"2024-09-21T03:37:53.431360+0000","last_undegraded":"2024-09-21T03:37:53.431360+0000","last_fullsized":"2024-09-21T03:37:53.431360+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_clean_scrub_stamp":"2024-09-21T03:37:09.809805+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:54:52.744486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.042822+0000","last_change":"2024-09-21T03:37:19.904112+0000","last_active":"2024-09-21T03:37:53.042822+0000","last_peered":"2024-09-21T03:37:53.042822+0000","last_clean":"2024-09-21T03:37:53.042822+0000","last_became_active":"2024-09-21T03:37:19.903677+0000","last_became_peered":"2024-09-21T03:37:19.903677+0000","last_unstale":"2024-09-21T03:37:53.042822+0000","last_undegraded":"2024-09-21T03:37:53.042822+0000","last_fullsized":"2024-09-21T03:37:53.042822+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_clean_scrub_stamp":"2024-09-21T03:37:18.876933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:32:20.096850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.046073+0000","last_change":"2024-09-21T03:37:12.864082+0000","last_active":"2024-09-21T03:37:53.046073+0000","last_peered":"2024-09-21T03:37:53.046073+0000","last_clean":"2024-09-21T03:37:53.046073+0000","last_became_active":"2024-09-21T03:37:12.862745+0000","last_became_peered":"2024-09-21T03:37:12.862745+0000","last_unstale":"2024-09-21T03:37:53.046073+0000","last_undegraded":"2024-09-21T03:37:53.046073+0000","last_fullsized":"2024-09-21T03:37:53.046073+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:25:35.759719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"80'4","reported_seq":90,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:38:38.106317+0000","last_change":"2024-09-21T03:37:11.034463+0000","last_active":"2024-09-21T03:38:38.106317+0000","last_peered":"2024-09-21T03:38:38.106317+0000","last_clean":"2024-09-21T03:38:38.106317+0000","last_became_active":"2024-09-21T03:37:11.033932+0000","last_became_peered":"2024-09-21T03:37:11.033932+0000","last_unstale":"2024-09-21T03:38:38.106317+0000","last_undegraded":"2024-09-21T03:38:38.106317+0000","last_fullsized":"2024-09-21T03:38:38.106317+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_clean_scrub_stamp":"2024-09-21T03:37:09.809805+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:15:18.595442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":51,"num_read_kb":51,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.043133+0000","last_change":"2024-09-21T03:37:20.160789+0000","last_active":"2024-09-21T03:37:53.043133+0000","last_peered":"2024-09-21T03:37:53.043133+0000","last_clean":"2024-09-21T03:37:53.043133+0000","last_became_active":"2024-09-21T03:37:20.160231+0000","last_became_peered":"2024-09-21T03:37:20.160231+0000","last_unstale":"2024-09-21T03:37:53.043133+0000","last_undegraded":"2024-09-21T03:37:53.043133+0000","last_fullsized":"2024-09-21T03:37:53.043133+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_clean_scrub_stamp":"2024-09-21T03:37:18.876933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:07:16.762249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.042510+0000","last_change":"2024-09-21T03:37:12.853178+0000","last_active":"2024-09-21T03:37:53.042510+0000","last_peered":"2024-09-21T03:37:53.042510+0000","last_clean":"2024-09-21T03:37:53.042510+0000","last_became_active":"2024-09-21T03:37:12.852822+0000","last_became_peered":"2024-09-21T03:37:12.852822+0000","last_unstale":"2024-09-21T03:37:53.042510+0000","last_undegraded":"2024-09-21T03:37:53.042510+0000","last_fullsized":"2024-09-21T03:37:53.042510+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:48:26.146593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"80'3","reported_seq":38,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.430945+0000","last_change":"2024-09-21T03:37:10.838939+0000","last_active":"2024-09-21T03:37:53.430945+0000","last_peered":"2024-09-21T03:37:53.430945+0000","last_clean":"2024-09-21T03:37:53.430945+0000","last_became_active":"2024-09-21T03:37:10.838562+0000","last_became_peered":"2024-09-21T03:37:10.838562+0000","last_unstale":"2024-09-21T03:37:53.430945+0000","last_undegraded":"2024-09-21T03:37:53.430945+0000","last_fullsized":"2024-09-21T03:37:53.430945+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_clean_scrub_stamp":"2024-09-21T03:37:09.809805+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:35:32.936824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.053062+0000","last_change":"2024-09-21T03:37:53.052940+0000","last_active":"2024-09-21T03:37:53.053062+0000","last_peered":"2024-09-21T03:37:53.053062+0000","last_clean":"2024-09-21T03:37:53.053062+0000","last_became_active":"2024-09-21T03:37:53.052310+0000","last_became_peered":"2024-09-21T03:37:53.052310+0000","last_unstale":"2024-09-21T03:37:53.053062+0000","last_undegraded":"2024-09-21T03:37:53.053062+0000","last_fullsized":"2024-09-21T03:37:53.053062+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_clean_scrub_stamp":"2024-09-21T03:37:18.876933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:31:35.353226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,11,6,10],"acting":[5,11,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.046076+0000","last_change":"2024-09-21T03:37:12.869795+0000","last_active":"2024-09-21T03:37:53.046076+0000","last_peered":"2024-09-21T03:37:53.046076+0000","last_clean":"2024-09-21T03:37:53.046076+0000","last_became_active":"2024-09-21T03:37:12.869381+0000","last_became_peered":"2024-09-21T03:37:12.869381+0000","last_unstale":"2024-09-21T03:37:53.046076+0000","last_undegraded":"2024-09-21T03:37:53.046076+0000","last_fullsized":"2024-09-21T03:37:53.046076+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:39:29.437552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"80'3","reported_seq":40,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.044859+0000","last_change":"2024-09-21T03:37:10.834853+0000","last_active":"2024-09-21T03:37:53.044859+0000","last_peered":"2024-09-21T03:37:53.044859+0000","last_clean":"2024-09-21T03:37:53.044859+0000","last_became_active":"2024-09-21T03:37:10.834402+0000","last_became_peered":"2024-09-21T03:37:10.834402+0000","last_unstale":"2024-09-21T03:37:53.044859+0000","last_undegraded":"2024-09-21T03:37:53.044859+0000","last_fullsized":"2024-09-21T03:37:53.044859+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_clean_scrub_stamp":"2024-09-21T03:37:09.809805+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:57:29.668570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"20'92","reported_seq":108,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.430997+0000","last_change":"2024-09-21T03:36:00.521781+0000","last_active":"2024-09-21T03:37:53.430997+0000","last_peered":"2024-09-21T03:37:53.430997+0000","last_clean":"2024-09-21T03:37:53.430997+0000","last_became_active":"2024-09-21T03:36:00.516468+0000","last_became_peered":"2024-09-21T03:36:00.516468+0000","last_unstale":"2024-09-21T03:37:53.430997+0000","last_undegraded":"2024-09-21T03:37:53.430997+0000","last_fullsized":"2024-09-21T03:37:53.430997+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:32:38.215432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:32:38.215432+0000","last_clean_scrub_stamp":"2024-09-21T03:32:38.215432+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:02:16.252544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"82'1","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.431151+0000","last_change":"2024-09-21T03:37:19.911309+0000","last_active":"2024-09-21T03:37:53.431151+0000","last_peered":"2024-09-21T03:37:53.431151+0000","last_clean":"2024-09-21T03:37:53.431151+0000","last_became_active":"2024-09-21T03:37:19.910778+0000","last_became_peered":"2024-09-21T03:37:19.910778+0000","last_unstale":"2024-09-21T03:37:53.431151+0000","last_undegraded":"2024-09-21T03:37:53.431151+0000","last_fullsized":"2024-09-21T03:37:53.431151+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_clean_scrub_stamp":"2024-09-21T03:37:18.876933+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:52:23.307983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.431249+0000","last_change":"2024-09-21T03:37:12.860707+0000","last_active":"2024-09-21T03:37:53.431249+0000","last_peered":"2024-09-21T03:37:53.431249+0000","last_clean":"2024-09-21T03:37:53.431249+0000","last_became_active":"2024-09-21T03:37:12.860341+0000","last_became_peered":"2024-09-21T03:37:12.860341+0000","last_unstale":"2024-09-21T03:37:53.431249+0000","last_undegraded":"2024-09-21T03:37:53.431249+0000","last_fullsized":"2024-09-21T03:37:53.431249+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:55:15.771259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"80'3","reported_seq":18,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.071234+0000","last_change":"2024-09-21T03:37:53.071125+0000","last_active":"2024-09-21T03:37:53.071234+0000","last_peered":"2024-09-21T03:37:53.071234+0000","last_clean":"2024-09-21T03:37:53.071234+0000","last_became_active":"2024-09-21T03:37:53.052237+0000","last_became_peered":"2024-09-21T03:37:53.052237+0000","last_unstale":"2024-09-21T03:37:53.071234+0000","last_undegraded":"2024-09-21T03:37:53.071234+0000","last_fullsized":"2024-09-21T03:37:53.071234+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_clean_scrub_stamp":"2024-09-21T03:37:09.809805+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:47:17.707045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,0],"acting":[4,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.111355+0000","last_change":"2024-09-21T03:37:19.909292+0000","last_active":"2024-09-21T03:37:53.111355+0000","last_peered":"2024-09-21T03:37:53.111355+0000","last_clean":"2024-09-21T03:37:53.111355+0000","last_became_active":"2024-09-21T03:37:19.908681+0000","last_became_peered":"2024-09-21T03:37:19.908681+0000","last_unstale":"2024-09-21T03:37:53.111355+0000","last_undegraded":"2024-09-21T03:37:53.111355+0000","last_fullsized":"2024-09-21T03:37:53.111355+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_clean_scrub_stamp":"2024-09-21T03:37:18.876933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:15:49.623109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.431136+0000","last_change":"2024-09-21T03:37:12.857678+0000","last_active":"2024-09-21T03:37:53.431136+0000","last_peered":"2024-09-21T03:37:53.431136+0000","last_clean":"2024-09-21T03:37:53.431136+0000","last_became_active":"2024-09-21T03:37:12.857266+0000","last_became_peered":"2024-09-21T03:37:12.857266+0000","last_unstale":"2024-09-21T03:37:53.431136+0000","last_undegraded":"2024-09-21T03:37:53.431136+0000","last_fullsized":"2024-09-21T03:37:53.431136+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:21:14.135784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"74'1","reported_seq":37,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.047567+0000","last_change":"2024-09-21T03:37:10.835323+0000","last_active":"2024-09-21T03:37:53.047567+0000","last_peered":"2024-09-21T03:37:53.047567+0000","last_clean":"2024-09-21T03:37:53.047567+0000","last_became_active":"2024-09-21T03:37:10.834950+0000","last_became_peered":"2024-09-21T03:37:10.834950+0000","last_unstale":"2024-09-21T03:37:53.047567+0000","last_undegraded":"2024-09-21T03:37:53.047567+0000","last_fullsized":"2024-09-21T03:37:53.047567+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_clean_scrub_stamp":"2024-09-21T03:37:09.809805+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:40:27.386342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.049010+0000","last_change":"2024-09-21T03:37:19.906360+0000","last_active":"2024-09-21T03:37:53.049010+0000","last_peered":"2024-09-21T03:37:53.049010+0000","last_clean":"2024-09-21T03:37:53.049010+0000","last_became_active":"2024-09-21T03:37:19.905888+0000","last_became_peered":"2024-09-21T03:37:19.905888+0000","last_unstale":"2024-09-21T03:37:53.049010+0000","last_undegraded":"2024-09-21T03:37:53.049010+0000","last_fullsized":"2024-09-21T03:37:53.049010+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_clean_scrub_stamp":"2024-09-21T03:37:18.876933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:13:55.885538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.431442+0000","last_change":"2024-09-21T03:37:12.849106+0000","last_active":"2024-09-21T03:37:53.431442+0000","last_peered":"2024-09-21T03:37:53.431442+0000","last_clean":"2024-09-21T03:37:53.431442+0000","last_became_active":"2024-09-21T03:37:12.848505+0000","last_became_peered":"2024-09-21T03:37:12.848505+0000","last_unstale":"2024-09-21T03:37:53.431442+0000","last_undegraded":"2024-09-21T03:37:53.431442+0000","last_fullsized":"2024-09-21T03:37:53.431442+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:50:19.569004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"74'2","reported_seq":48,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.115238+0000","last_change":"2024-09-21T03:37:53.115158+0000","last_active":"2024-09-21T03:37:53.115238+0000","last_peered":"2024-09-21T03:37:53.115238+0000","last_clean":"2024-09-21T03:37:53.115238+0000","last_became_active":"2024-09-21T03:37:53.112603+0000","last_became_peered":"2024-09-21T03:37:53.112603+0000","last_unstale":"2024-09-21T03:37:53.115238+0000","last_undegraded":"2024-09-21T03:37:53.115238+0000","last_fullsized":"2024-09-21T03:37:53.115238+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_clean_scrub_stamp":"2024-09-21T03:37:09.809805+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:23:21.525972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,1],"acting":[6,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.045614+0000","last_change":"2024-09-21T03:37:19.909040+0000","last_active":"2024-09-21T03:37:53.045614+0000","last_peered":"2024-09-21T03:37:53.045614+0000","last_clean":"2024-09-21T03:37:53.045614+0000","last_became_active":"2024-09-21T03:37:19.908530+0000","last_became_peered":"2024-09-21T03:37:19.908530+0000","last_unstale":"2024-09-21T03:37:53.045614+0000","last_undegraded":"2024-09-21T03:37:53.045614+0000","last_fullsized":"2024-09-21T03:37:53.045614+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_clean_scrub_stamp":"2024-09-21T03:37:18.876933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:35:20.919260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.049078+0000","last_change":"2024-09-21T03:37:12.858984+0000","last_active":"2024-09-21T03:37:53.049078+0000","last_peered":"2024-09-21T03:37:53.049078+0000","last_clean":"2024-09-21T03:37:53.049078+0000","last_became_active":"2024-09-21T03:37:12.858603+0000","last_became_peered":"2024-09-21T03:37:12.858603+0000","last_unstale":"2024-09-21T03:37:53.049078+0000","last_undegraded":"2024-09-21T03:37:53.049078+0000","last_fullsized":"2024-09-21T03:37:53.049078+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:16:45.025304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.f","version":"80'2","reported_seq":38,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.050780+0000","last_change":"2024-09-21T03:37:10.834844+0000","last_active":"2024-09-21T03:37:53.050780+0000","last_peered":"2024-09-21T03:37:53.050780+0000","last_clean":"2024-09-21T03:37:53.050780+0000","last_became_active":"2024-09-21T03:37:10.833664+0000","last_became_peered":"2024-09-21T03:37:10.833664+0000","last_unstale":"2024-09-21T03:37:53.050780+0000","last_undegraded":"2024-09-21T03:37:53.050780+0000","last_fullsized":"2024-09-21T03:37:53.050780+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_clean_scrub_stamp":"2024-09-21T03:37:09.809805+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:03:49.006414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.046180+0000","last_change":"2024-09-21T03:37:19.913277+0000","last_active":"2024-09-21T03:37:53.046180+0000","last_peered":"2024-09-21T03:37:53.046180+0000","last_clean":"2024-09-21T03:37:53.046180+0000","last_became_active":"2024-09-21T03:37:19.912781+0000","last_became_peered":"2024-09-21T03:37:19.912781+0000","last_unstale":"2024-09-21T03:37:53.046180+0000","last_undegraded":"2024-09-21T03:37:53.046180+0000","last_fullsized":"2024-09-21T03:37:53.046180+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_clean_scrub_stamp":"2024-09-21T03:37:18.876933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:18:27.980622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.046359+0000","last_change":"2024-09-21T03:37:12.862322+0000","last_active":"2024-09-21T03:37:53.046359+0000","last_peered":"2024-09-21T03:37:53.046359+0000","last_clean":"2024-09-21T03:37:53.046359+0000","last_became_active":"2024-09-21T03:37:12.861938+0000","last_became_peered":"2024-09-21T03:37:12.861938+0000","last_unstale":"2024-09-21T03:37:53.046359+0000","last_undegraded":"2024-09-21T03:37:53.046359+0000","last_fullsized":"2024-09-21T03:37:53.046359+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:01:48.944169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.10","version":"80'15","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.067663+0000","last_change":"2024-09-21T03:37:53.067130+0000","last_active":"2024-09-21T03:37:53.067663+0000","last_peered":"2024-09-21T03:37:53.067663+0000","last_clean":"2024-09-21T03:37:53.067663+0000","last_became_active":"2024-09-21T03:37:53.051596+0000","last_became_peered":"2024-09-21T03:37:53.051596+0000","last_unstale":"2024-09-21T03:37:53.067663+0000","last_undegraded":"2024-09-21T03:37:53.067663+0000","last_fullsized":"2024-09-21T03:37:53.067663+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_clean_scrub_stamp":"2024-09-21T03:37:09.809805+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:02:22.852015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.043129+0000","last_change":"2024-09-21T03:37:19.913206+0000","last_active":"2024-09-21T03:37:53.043129+0000","last_peered":"2024-09-21T03:37:53.043129+0000","last_clean":"2024-09-21T03:37:53.043129+0000","last_became_active":"2024-09-21T03:37:19.910718+0000","last_became_peered":"2024-09-21T03:37:19.910718+0000","last_unstale":"2024-09-21T03:37:53.043129+0000","last_undegraded":"2024-09-21T03:37:53.043129+0000","last_fullsized":"2024-09-21T03:37:53.043129+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_clean_scrub_stamp":"2024-09-21T03:37:18.876933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:11:31.431237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.430692+0000","last_change":"2024-09-21T03:37:12.857505+0000","last_active":"2024-09-21T03:37:53.430692+0000","last_peered":"2024-09-21T03:37:53.430692+0000","last_clean":"2024-09-21T03:37:53.430692+0000","last_became_active":"2024-09-21T03:37:12.857155+0000","last_became_peered":"2024-09-21T03:37:12.857155+0000","last_unstale":"2024-09-21T03:37:53.430692+0000","last_undegraded":"2024-09-21T03:37:53.430692+0000","last_fullsized":"2024-09-21T03:37:53.430692+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:05:50.387651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"80'3","reported_seq":94,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:38:41.111964+0000","last_change":"2024-09-21T03:37:10.832364+0000","last_active":"2024-09-21T03:38:41.111964+0000","last_peered":"2024-09-21T03:38:41.111964+0000","last_clean":"2024-09-21T03:38:41.111964+0000","last_became_active":"2024-09-21T03:37:10.832004+0000","last_became_peered":"2024-09-21T03:37:10.832004+0000","last_unstale":"2024-09-21T03:38:41.111964+0000","last_undegraded":"2024-09-21T03:38:41.111964+0000","last_fullsized":"2024-09-21T03:38:41.111964+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_clean_scrub_stamp":"2024-09-21T03:37:09.809805+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:21:10.895732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":57,"num_read_kb":3,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.047568+0000","last_change":"2024-09-21T03:37:19.912297+0000","last_active":"2024-09-21T03:37:53.047568+0000","last_peered":"2024-09-21T03:37:53.047568+0000","last_clean":"2024-09-21T03:37:53.047568+0000","last_became_active":"2024-09-21T03:37:19.911903+0000","last_became_peered":"2024-09-21T03:37:19.911903+0000","last_unstale":"2024-09-21T03:37:53.047568+0000","last_undegraded":"2024-09-21T03:37:53.047568+0000","last_fullsized":"2024-09-21T03:37:53.047568+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_clean_scrub_stamp":"2024-09-21T03:37:18.876933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:05:56.747246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.431059+0000","last_change":"2024-09-21T03:37:12.856230+0000","last_active":"2024-09-21T03:37:53.431059+0000","last_peered":"2024-09-21T03:37:53.431059+0000","last_clean":"2024-09-21T03:37:53.431059+0000","last_became_active":"2024-09-21T03:37:12.855819+0000","last_became_peered":"2024-09-21T03:37:12.855819+0000","last_unstale":"2024-09-21T03:37:53.431059+0000","last_undegraded":"2024-09-21T03:37:53.431059+0000","last_fullsized":"2024-09-21T03:37:53.431059+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:38:18.618579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"80'3","reported_seq":38,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.046856+0000","last_change":"2024-09-21T03:37:10.832401+0000","last_active":"2024-09-21T03:37:53.046856+0000","last_peered":"2024-09-21T03:37:53.046856+0000","last_clean":"2024-09-21T03:37:53.046856+0000","last_became_active":"2024-09-21T03:37:10.832033+0000","last_became_peered":"2024-09-21T03:37:10.832033+0000","last_unstale":"2024-09-21T03:37:53.046856+0000","last_undegraded":"2024-09-21T03:37:53.046856+0000","last_fullsized":"2024-09-21T03:37:53.046856+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_clean_scrub_stamp":"2024-09-21T03:37:09.809805+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:25:20.751529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.048479+0000","last_change":"2024-09-21T03:37:19.905825+0000","last_active":"2024-09-21T03:37:53.048479+0000","last_peered":"2024-09-21T03:37:53.048479+0000","last_clean":"2024-09-21T03:37:53.048479+0000","last_became_active":"2024-09-21T03:37:19.905352+0000","last_became_peered":"2024-09-21T03:37:19.905352+0000","last_unstale":"2024-09-21T03:37:53.048479+0000","last_undegraded":"2024-09-21T03:37:53.048479+0000","last_fullsized":"2024-09-21T03:37:53.048479+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_clean_scrub_stamp":"2024-09-21T03:37:18.876933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:25:05.861981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.042684+0000","last_change":"2024-09-21T03:37:12.852006+0000","last_active":"2024-09-21T03:37:53.042684+0000","last_peered":"2024-09-21T03:37:53.042684+0000","last_clean":"2024-09-21T03:37:53.042684+0000","last_became_active":"2024-09-21T03:37:12.851670+0000","last_became_peered":"2024-09-21T03:37:12.851670+0000","last_unstale":"2024-09-21T03:37:53.042684+0000","last_undegraded":"2024-09-21T03:37:53.042684+0000","last_fullsized":"2024-09-21T03:37:53.042684+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:41:50.177349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"80'6","reported_seq":42,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.044608+0000","last_change":"2024-09-21T03:37:10.832472+0000","last_active":"2024-09-21T03:37:53.044608+0000","last_peered":"2024-09-21T03:37:53.044608+0000","last_clean":"2024-09-21T03:37:53.044608+0000","last_became_active":"2024-09-21T03:37:10.832085+0000","last_became_peered":"2024-09-21T03:37:10.832085+0000","last_unstale":"2024-09-21T03:37:53.044608+0000","last_undegraded":"2024-09-21T03:37:53.044608+0000","last_fullsized":"2024-09-21T03:37:53.044608+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_clean_scrub_stamp":"2024-09-21T03:37:09.809805+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:06:41.251218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.046632+0000","last_change":"2024-09-21T03:37:19.914042+0000","last_active":"2024-09-21T03:37:53.046632+0000","last_peered":"2024-09-21T03:37:53.046632+0000","last_clean":"2024-09-21T03:37:53.046632+0000","last_became_active":"2024-09-21T03:37:19.913531+0000","last_became_peered":"2024-09-21T03:37:19.913531+0000","last_unstale":"2024-09-21T03:37:53.046632+0000","last_undegraded":"2024-09-21T03:37:53.046632+0000","last_fullsized":"2024-09-21T03:37:53.046632+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_clean_scrub_stamp":"2024-09-21T03:37:18.876933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:16:40.163359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.050691+0000","last_change":"2024-09-21T03:37:12.852893+0000","last_active":"2024-09-21T03:37:53.050691+0000","last_peered":"2024-09-21T03:37:53.050691+0000","last_clean":"2024-09-21T03:37:53.050691+0000","last_became_active":"2024-09-21T03:37:12.852452+0000","last_became_peered":"2024-09-21T03:37:12.852452+0000","last_unstale":"2024-09-21T03:37:53.050691+0000","last_undegraded":"2024-09-21T03:37:53.050691+0000","last_fullsized":"2024-09-21T03:37:53.050691+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:46:59.393901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.14","version":"82'32","reported_seq":178,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:38:39.108865+0000","last_change":"2024-09-21T03:37:10.837654+0000","last_active":"2024-09-21T03:38:39.108865+0000","last_peered":"2024-09-21T03:38:39.108865+0000","last_clean":"2024-09-21T03:38:39.108865+0000","last_became_active":"2024-09-21T03:37:10.837251+0000","last_became_peered":"2024-09-21T03:37:10.837251+0000","last_unstale":"2024-09-21T03:38:39.108865+0000","last_undegraded":"2024-09-21T03:38:39.108865+0000","last_fullsized":"2024-09-21T03:38:39.108865+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_clean_scrub_stamp":"2024-09-21T03:37:09.809805+0000","objects_scrubbed":0,"log_size":32,"log_dups_size":0,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:43:32.986371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":67138,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":111,"num_read_kb":121,"num_write":33,"num_write_kb":80,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.053980+0000","last_change":"2024-09-21T03:37:53.053405+0000","last_active":"2024-09-21T03:37:53.053980+0000","last_peered":"2024-09-21T03:37:53.053980+0000","last_clean":"2024-09-21T03:37:53.053980+0000","last_became_active":"2024-09-21T03:37:53.052818+0000","last_became_peered":"2024-09-21T03:37:53.052818+0000","last_unstale":"2024-09-21T03:37:53.053980+0000","last_undegraded":"2024-09-21T03:37:53.053980+0000","last_fullsized":"2024-09-21T03:37:53.053980+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_clean_scrub_stamp":"2024-09-21T03:37:18.876933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:05:09.121036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,11,5],"acting":[10,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.111585+0000","last_change":"2024-09-21T03:37:12.866588+0000","last_active":"2024-09-21T03:37:53.111585+0000","last_peered":"2024-09-21T03:37:53.111585+0000","last_clean":"2024-09-21T03:37:53.111585+0000","last_became_active":"2024-09-21T03:37:12.866075+0000","last_became_peered":"2024-09-21T03:37:12.866075+0000","last_unstale":"2024-09-21T03:37:53.111585+0000","last_undegraded":"2024-09-21T03:37:53.111585+0000","last_fullsized":"2024-09-21T03:37:53.111585+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:18:48.817562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"80'4","reported_seq":40,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.047298+0000","last_change":"2024-09-21T03:37:11.036751+0000","last_active":"2024-09-21T03:37:53.047298+0000","last_peered":"2024-09-21T03:37:53.047298+0000","last_clean":"2024-09-21T03:37:53.047298+0000","last_became_active":"2024-09-21T03:37:11.036125+0000","last_became_peered":"2024-09-21T03:37:11.036125+0000","last_unstale":"2024-09-21T03:37:53.047298+0000","last_undegraded":"2024-09-21T03:37:53.047298+0000","last_fullsized":"2024-09-21T03:37:53.047298+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_clean_scrub_stamp":"2024-09-21T03:37:09.809805+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:46:13.569422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.112980+0000","last_change":"2024-09-21T03:37:53.112845+0000","last_active":"2024-09-21T03:37:53.112980+0000","last_peered":"2024-09-21T03:37:53.112980+0000","last_clean":"2024-09-21T03:37:53.112980+0000","last_became_active":"2024-09-21T03:37:53.112286+0000","last_became_peered":"2024-09-21T03:37:53.112286+0000","last_unstale":"2024-09-21T03:37:53.112980+0000","last_undegraded":"2024-09-21T03:37:53.112980+0000","last_fullsized":"2024-09-21T03:37:53.112980+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_clean_scrub_stamp":"2024-09-21T03:37:18.876933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:29:58.062684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,8],"acting":[4,1,2,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.049603+0000","last_change":"2024-09-21T03:37:12.865011+0000","last_active":"2024-09-21T03:37:53.049603+0000","last_peered":"2024-09-21T03:37:53.049603+0000","last_clean":"2024-09-21T03:37:53.049603+0000","last_became_active":"2024-09-21T03:37:12.864645+0000","last_became_peered":"2024-09-21T03:37:12.864645+0000","last_unstale":"2024-09-21T03:37:53.049603+0000","last_undegraded":"2024-09-21T03:37:53.049603+0000","last_fullsized":"2024-09-21T03:37:53.049603+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:27:42.471227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"80'2","reported_seq":37,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.431360+0000","last_change":"2024-09-21T03:37:10.835171+0000","last_active":"2024-09-21T03:37:53.431360+0000","last_peered":"2024-09-21T03:37:53.431360+0000","last_clean":"2024-09-21T03:37:53.431360+0000","last_became_active":"2024-09-21T03:37:10.834744+0000","last_became_peered":"2024-09-21T03:37:10.834744+0000","last_unstale":"2024-09-21T03:37:53.431360+0000","last_undegraded":"2024-09-21T03:37:53.431360+0000","last_fullsized":"2024-09-21T03:37:53.431360+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_clean_scrub_stamp":"2024-09-21T03:37:09.809805+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:02:37.720457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.046301+0000","last_change":"2024-09-21T03:37:19.906734+0000","last_active":"2024-09-21T03:37:53.046301+0000","last_peered":"2024-09-21T03:37:53.046301+0000","last_clean":"2024-09-21T03:37:53.046301+0000","last_became_active":"2024-09-21T03:37:19.906327+0000","last_became_peered":"2024-09-21T03:37:19.906327+0000","last_unstale":"2024-09-21T03:37:53.046301+0000","last_undegraded":"2024-09-21T03:37:53.046301+0000","last_fullsized":"2024-09-21T03:37:53.046301+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_clean_scrub_stamp":"2024-09-21T03:37:18.876933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:55:11.603042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.111948+0000","last_change":"2024-09-21T03:37:12.866285+0000","last_active":"2024-09-21T03:37:53.111948+0000","last_peered":"2024-09-21T03:37:53.111948+0000","last_clean":"2024-09-21T03:37:53.111948+0000","last_became_active":"2024-09-21T03:37:12.865854+0000","last_became_peered":"2024-09-21T03:37:12.865854+0000","last_unstale":"2024-09-21T03:37:53.111948+0000","last_undegraded":"2024-09-21T03:37:53.111948+0000","last_fullsized":"2024-09-21T03:37:53.111948+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:00:18.075248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"80'3","reported_seq":89,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:38:38.107167+0000","last_change":"2024-09-21T03:37:10.835808+0000","last_active":"2024-09-21T03:38:38.107167+0000","last_peered":"2024-09-21T03:38:38.107167+0000","last_clean":"2024-09-21T03:38:38.107167+0000","last_became_active":"2024-09-21T03:37:10.835434+0000","last_became_peered":"2024-09-21T03:37:10.835434+0000","last_unstale":"2024-09-21T03:38:38.107167+0000","last_undegraded":"2024-09-21T03:38:38.107167+0000","last_fullsized":"2024-09-21T03:38:38.107167+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_clean_scrub_stamp":"2024-09-21T03:37:09.809805+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:35:39.817312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":51,"num_read_kb":51,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.045411+0000","last_change":"2024-09-21T03:37:19.915986+0000","last_active":"2024-09-21T03:37:53.045411+0000","last_peered":"2024-09-21T03:37:53.045411+0000","last_clean":"2024-09-21T03:37:53.045411+0000","last_became_active":"2024-09-21T03:37:19.915561+0000","last_became_peered":"2024-09-21T03:37:19.915561+0000","last_unstale":"2024-09-21T03:37:53.045411+0000","last_undegraded":"2024-09-21T03:37:53.045411+0000","last_fullsized":"2024-09-21T03:37:53.045411+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_clean_scrub_stamp":"2024-09-21T03:37:18.876933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:21:13.789664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.049328+0000","last_change":"2024-09-21T03:37:19.908230+0000","last_active":"2024-09-21T03:37:53.049328+0000","last_peered":"2024-09-21T03:37:53.049328+0000","last_clean":"2024-09-21T03:37:53.049328+0000","last_became_active":"2024-09-21T03:37:19.907843+0000","last_became_peered":"2024-09-21T03:37:19.907843+0000","last_unstale":"2024-09-21T03:37:53.049328+0000","last_undegraded":"2024-09-21T03:37:53.049328+0000","last_fullsized":"2024-09-21T03:37:53.049328+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_clean_scrub_stamp":"2024-09-21T03:37:18.876933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:15:58.142127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.046673+0000","last_change":"2024-09-21T03:37:12.856709+0000","last_active":"2024-09-21T03:37:53.046673+0000","last_peered":"2024-09-21T03:37:53.046673+0000","last_clean":"2024-09-21T03:37:53.046673+0000","last_became_active":"2024-09-21T03:37:12.856278+0000","last_became_peered":"2024-09-21T03:37:12.856278+0000","last_unstale":"2024-09-21T03:37:53.046673+0000","last_undegraded":"2024-09-21T03:37:53.046673+0000","last_fullsized":"2024-09-21T03:37:53.046673+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:16:13.884648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"80'1","reported_seq":36,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.046329+0000","last_change":"2024-09-21T03:37:10.832456+0000","last_active":"2024-09-21T03:37:53.046329+0000","last_peered":"2024-09-21T03:37:53.046329+0000","last_clean":"2024-09-21T03:37:53.046329+0000","last_became_active":"2024-09-21T03:37:10.832082+0000","last_became_peered":"2024-09-21T03:37:10.832082+0000","last_unstale":"2024-09-21T03:37:53.046329+0000","last_undegraded":"2024-09-21T03:37:53.046329+0000","last_fullsized":"2024-09-21T03:37:53.046329+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_clean_scrub_stamp":"2024-09-21T03:37:09.809805+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:17:12.170616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.048472+0000","last_change":"2024-09-21T03:37:19.908084+0000","last_active":"2024-09-21T03:37:53.048472+0000","last_peered":"2024-09-21T03:37:53.048472+0000","last_clean":"2024-09-21T03:37:53.048472+0000","last_became_active":"2024-09-21T03:37:19.907307+0000","last_became_peered":"2024-09-21T03:37:19.907307+0000","last_unstale":"2024-09-21T03:37:53.048472+0000","last_undegraded":"2024-09-21T03:37:53.048472+0000","last_fullsized":"2024-09-21T03:37:53.048472+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_clean_scrub_stamp":"2024-09-21T03:37:18.876933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:45:02.541015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.045568+0000","last_change":"2024-09-21T03:37:12.852577+0000","last_active":"2024-09-21T03:37:53.045568+0000","last_peered":"2024-09-21T03:37:53.045568+0000","last_clean":"2024-09-21T03:37:53.045568+0000","last_became_active":"2024-09-21T03:37:12.852186+0000","last_became_peered":"2024-09-21T03:37:12.852186+0000","last_unstale":"2024-09-21T03:37:53.045568+0000","last_undegraded":"2024-09-21T03:37:53.045568+0000","last_fullsized":"2024-09-21T03:37:53.045568+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:51:21.631029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"80'2","reported_seq":37,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.042546+0000","last_change":"2024-09-21T03:37:10.833587+0000","last_active":"2024-09-21T03:37:53.042546+0000","last_peered":"2024-09-21T03:37:53.042546+0000","last_clean":"2024-09-21T03:37:53.042546+0000","last_became_active":"2024-09-21T03:37:10.833194+0000","last_became_peered":"2024-09-21T03:37:10.833194+0000","last_unstale":"2024-09-21T03:37:53.042546+0000","last_undegraded":"2024-09-21T03:37:53.042546+0000","last_fullsized":"2024-09-21T03:37:53.042546+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_clean_scrub_stamp":"2024-09-21T03:37:09.809805+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:20:20.272108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"80'3","reported_seq":38,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.047496+0000","last_change":"2024-09-21T03:37:10.839103+0000","last_active":"2024-09-21T03:37:53.047496+0000","last_peered":"2024-09-21T03:37:53.047496+0000","last_clean":"2024-09-21T03:37:53.047496+0000","last_became_active":"2024-09-21T03:37:10.837192+0000","last_became_peered":"2024-09-21T03:37:10.837192+0000","last_unstale":"2024-09-21T03:37:53.047496+0000","last_undegraded":"2024-09-21T03:37:53.047496+0000","last_fullsized":"2024-09-21T03:37:53.047496+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_clean_scrub_stamp":"2024-09-21T03:37:09.809805+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:59:35.128171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.047232+0000","last_change":"2024-09-21T03:37:12.866626+0000","last_active":"2024-09-21T03:37:53.047232+0000","last_peered":"2024-09-21T03:37:53.047232+0000","last_clean":"2024-09-21T03:37:53.047232+0000","last_became_active":"2024-09-21T03:37:12.866254+0000","last_became_peered":"2024-09-21T03:37:12.866254+0000","last_unstale":"2024-09-21T03:37:53.047232+0000","last_undegraded":"2024-09-21T03:37:53.047232+0000","last_fullsized":"2024-09-21T03:37:53.047232+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:39:00.361254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.044954+0000","last_change":"2024-09-21T03:37:19.909415+0000","last_active":"2024-09-21T03:37:53.044954+0000","last_peered":"2024-09-21T03:37:53.044954+0000","last_clean":"2024-09-21T03:37:53.044954+0000","last_became_active":"2024-09-21T03:37:19.908955+0000","last_became_peered":"2024-09-21T03:37:19.908955+0000","last_unstale":"2024-09-21T03:37:53.044954+0000","last_undegraded":"2024-09-21T03:37:53.044954+0000","last_fullsized":"2024-09-21T03:37:53.044954+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_clean_scrub_stamp":"2024-09-21T03:37:18.876933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:31:18.878725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"81'6","reported_seq":97,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:38:40.110836+0000","last_change":"2024-09-21T03:37:10.836389+0000","last_active":"2024-09-21T03:38:40.110836+0000","last_peered":"2024-09-21T03:38:40.110836+0000","last_clean":"2024-09-21T03:38:40.110836+0000","last_became_active":"2024-09-21T03:37:10.835986+0000","last_became_peered":"2024-09-21T03:37:10.835986+0000","last_unstale":"2024-09-21T03:38:40.110836+0000","last_undegraded":"2024-09-21T03:38:40.110836+0000","last_fullsized":"2024-09-21T03:38:40.110836+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_clean_scrub_stamp":"2024-09-21T03:37:09.809805+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:05:17.963609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1013,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":56,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.050861+0000","last_change":"2024-09-21T03:37:12.856008+0000","last_active":"2024-09-21T03:37:53.050861+0000","last_peered":"2024-09-21T03:37:53.050861+0000","last_clean":"2024-09-21T03:37:53.050861+0000","last_became_active":"2024-09-21T03:37:12.855603+0000","last_became_peered":"2024-09-21T03:37:12.855603+0000","last_unstale":"2024-09-21T03:37:53.050861+0000","last_undegraded":"2024-09-21T03:37:53.050861+0000","last_fullsized":"2024-09-21T03:37:53.050861+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:11:41.920114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.045051+0000","last_change":"2024-09-21T03:37:19.911507+0000","last_active":"2024-09-21T03:37:53.045051+0000","last_peered":"2024-09-21T03:37:53.045051+0000","last_clean":"2024-09-21T03:37:53.045051+0000","last_became_active":"2024-09-21T03:37:19.911053+0000","last_became_peered":"2024-09-21T03:37:19.911053+0000","last_unstale":"2024-09-21T03:37:53.045051+0000","last_undegraded":"2024-09-21T03:37:53.045051+0000","last_fullsized":"2024-09-21T03:37:53.045051+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_clean_scrub_stamp":"2024-09-21T03:37:18.876933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:42:11.964829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.048840+0000","last_change":"2024-09-21T03:37:12.862576+0000","last_active":"2024-09-21T03:37:53.048840+0000","last_peered":"2024-09-21T03:37:53.048840+0000","last_clean":"2024-09-21T03:37:53.048840+0000","last_became_active":"2024-09-21T03:37:12.862146+0000","last_became_peered":"2024-09-21T03:37:12.862146+0000","last_unstale":"2024-09-21T03:37:53.048840+0000","last_undegraded":"2024-09-21T03:37:53.048840+0000","last_fullsized":"2024-09-21T03:37:53.048840+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:30:33.272415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.044731+0000","last_change":"2024-09-21T03:37:12.859395+0000","last_active":"2024-09-21T03:37:53.044731+0000","last_peered":"2024-09-21T03:37:53.044731+0000","last_clean":"2024-09-21T03:37:53.044731+0000","last_became_active":"2024-09-21T03:37:12.858914+0000","last_became_peered":"2024-09-21T03:37:12.858914+0000","last_unstale":"2024-09-21T03:37:53.044731+0000","last_undegraded":"2024-09-21T03:37:53.044731+0000","last_fullsized":"2024-09-21T03:37:53.044731+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:13:06.319896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.044203+0000","last_change":"2024-09-21T03:37:12.857467+0000","last_active":"2024-09-21T03:37:53.044203+0000","last_peered":"2024-09-21T03:37:53.044203+0000","last_clean":"2024-09-21T03:37:53.044203+0000","last_became_active":"2024-09-21T03:37:12.856214+0000","last_became_peered":"2024-09-21T03:37:12.856214+0000","last_unstale":"2024-09-21T03:37:53.044203+0000","last_undegraded":"2024-09-21T03:37:53.044203+0000","last_fullsized":"2024-09-21T03:37:53.044203+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:21:58.780018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.049029+0000","last_change":"2024-09-21T03:37:12.858052+0000","last_active":"2024-09-21T03:37:53.049029+0000","last_peered":"2024-09-21T03:37:53.049029+0000","last_clean":"2024-09-21T03:37:53.049029+0000","last_became_active":"2024-09-21T03:37:12.857445+0000","last_became_peered":"2024-09-21T03:37:12.857445+0000","last_unstale":"2024-09-21T03:37:53.049029+0000","last_undegraded":"2024-09-21T03:37:53.049029+0000","last_fullsized":"2024-09-21T03:37:53.049029+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:22:28.058846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.042880+0000","last_change":"2024-09-21T03:37:12.856499+0000","last_active":"2024-09-21T03:37:53.042880+0000","last_peered":"2024-09-21T03:37:53.042880+0000","last_clean":"2024-09-21T03:37:53.042880+0000","last_became_active":"2024-09-21T03:37:12.856146+0000","last_became_peered":"2024-09-21T03:37:12.856146+0000","last_unstale":"2024-09-21T03:37:53.042880+0000","last_undegraded":"2024-09-21T03:37:53.042880+0000","last_fullsized":"2024-09-21T03:37:53.042880+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:03:26.142740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.048901+0000","last_change":"2024-09-21T03:37:12.860830+0000","last_active":"2024-09-21T03:37:53.048901+0000","last_peered":"2024-09-21T03:37:53.048901+0000","last_clean":"2024-09-21T03:37:53.048901+0000","last_became_active":"2024-09-21T03:37:12.860468+0000","last_became_peered":"2024-09-21T03:37:12.860468+0000","last_unstale":"2024-09-21T03:37:53.048901+0000","last_undegraded":"2024-09-21T03:37:53.048901+0000","last_fullsized":"2024-09-21T03:37:53.048901+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:48:15.057622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.046546+0000","last_change":"2024-09-21T03:37:12.864399+0000","last_active":"2024-09-21T03:37:53.046546+0000","last_peered":"2024-09-21T03:37:53.046546+0000","last_clean":"2024-09-21T03:37:53.046546+0000","last_became_active":"2024-09-21T03:37:12.863892+0000","last_became_peered":"2024-09-21T03:37:12.863892+0000","last_unstale":"2024-09-21T03:37:53.046546+0000","last_undegraded":"2024-09-21T03:37:53.046546+0000","last_fullsized":"2024-09-21T03:37:53.046546+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:58:26.647896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.046313+0000","last_change":"2024-09-21T03:37:12.861716+0000","last_active":"2024-09-21T03:37:53.046313+0000","last_peered":"2024-09-21T03:37:53.046313+0000","last_clean":"2024-09-21T03:37:53.046313+0000","last_became_active":"2024-09-21T03:37:12.861392+0000","last_became_peered":"2024-09-21T03:37:12.861392+0000","last_unstale":"2024-09-21T03:37:53.046313+0000","last_undegraded":"2024-09-21T03:37:53.046313+0000","last_fullsized":"2024-09-21T03:37:53.046313+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:28:55.751107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.045975+0000","last_change":"2024-09-21T03:37:12.856167+0000","last_active":"2024-09-21T03:37:53.045975+0000","last_peered":"2024-09-21T03:37:53.045975+0000","last_clean":"2024-09-21T03:37:53.045975+0000","last_became_active":"2024-09-21T03:37:12.855752+0000","last_became_peered":"2024-09-21T03:37:12.855752+0000","last_unstale":"2024-09-21T03:37:53.045975+0000","last_undegraded":"2024-09-21T03:37:53.045975+0000","last_fullsized":"2024-09-21T03:37:53.045975+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:26:30.271342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.045615+0000","last_change":"2024-09-21T03:37:12.862782+0000","last_active":"2024-09-21T03:37:53.045615+0000","last_peered":"2024-09-21T03:37:53.045615+0000","last_clean":"2024-09-21T03:37:53.045615+0000","last_became_active":"2024-09-21T03:37:12.862462+0000","last_became_peered":"2024-09-21T03:37:12.862462+0000","last_unstale":"2024-09-21T03:37:53.045615+0000","last_undegraded":"2024-09-21T03:37:53.045615+0000","last_fullsized":"2024-09-21T03:37:53.045615+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:57:37.744531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.111623+0000","last_change":"2024-09-21T03:37:12.862312+0000","last_active":"2024-09-21T03:37:53.111623+0000","last_peered":"2024-09-21T03:37:53.111623+0000","last_clean":"2024-09-21T03:37:53.111623+0000","last_became_active":"2024-09-21T03:37:12.861940+0000","last_became_peered":"2024-09-21T03:37:12.861940+0000","last_unstale":"2024-09-21T03:37:53.111623+0000","last_undegraded":"2024-09-21T03:37:53.111623+0000","last_fullsized":"2024-09-21T03:37:53.111623+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:08:49.165522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.042830+0000","last_change":"2024-09-21T03:37:12.857012+0000","last_active":"2024-09-21T03:37:53.042830+0000","last_peered":"2024-09-21T03:37:53.042830+0000","last_clean":"2024-09-21T03:37:53.042830+0000","last_became_active":"2024-09-21T03:37:12.856390+0000","last_became_peered":"2024-09-21T03:37:12.856390+0000","last_unstale":"2024-09-21T03:37:53.042830+0000","last_undegraded":"2024-09-21T03:37:53.042830+0000","last_fullsized":"2024-09-21T03:37:53.042830+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:51:00.839709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.048487+0000","last_change":"2024-09-21T03:37:12.857207+0000","last_active":"2024-09-21T03:37:53.048487+0000","last_peered":"2024-09-21T03:37:53.048487+0000","last_clean":"2024-09-21T03:37:53.048487+0000","last_became_active":"2024-09-21T03:37:12.856770+0000","last_became_peered":"2024-09-21T03:37:12.856770+0000","last_unstale":"2024-09-21T03:37:53.048487+0000","last_undegraded":"2024-09-21T03:37:53.048487+0000","last_fullsized":"2024-09-21T03:37:53.048487+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:37:18.272321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.045688+0000","last_change":"2024-09-21T03:37:12.858540+0000","last_active":"2024-09-21T03:37:53.045688+0000","last_peered":"2024-09-21T03:37:53.045688+0000","last_clean":"2024-09-21T03:37:53.045688+0000","last_became_active":"2024-09-21T03:37:12.857340+0000","last_became_peered":"2024-09-21T03:37:12.857340+0000","last_unstale":"2024-09-21T03:37:53.045688+0000","last_undegraded":"2024-09-21T03:37:53.045688+0000","last_fullsized":"2024-09-21T03:37:53.045688+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:45:48.982132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.047908+0000","last_change":"2024-09-21T03:37:12.864408+0000","last_active":"2024-09-21T03:37:53.047908+0000","last_peered":"2024-09-21T03:37:53.047908+0000","last_clean":"2024-09-21T03:37:53.047908+0000","last_became_active":"2024-09-21T03:37:12.864082+0000","last_became_peered":"2024-09-21T03:37:12.864082+0000","last_unstale":"2024-09-21T03:37:53.047908+0000","last_undegraded":"2024-09-21T03:37:53.047908+0000","last_fullsized":"2024-09-21T03:37:53.047908+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:19:32.723589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.049781+0000","last_change":"2024-09-21T03:37:12.869445+0000","last_active":"2024-09-21T03:37:53.049781+0000","last_peered":"2024-09-21T03:37:53.049781+0000","last_clean":"2024-09-21T03:37:53.049781+0000","last_became_active":"2024-09-21T03:37:12.869058+0000","last_became_peered":"2024-09-21T03:37:12.869058+0000","last_unstale":"2024-09-21T03:37:53.049781+0000","last_undegraded":"2024-09-21T03:37:53.049781+0000","last_fullsized":"2024-09-21T03:37:53.049781+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:37:57.750665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.111172+0000","last_change":"2024-09-21T03:37:12.858431+0000","last_active":"2024-09-21T03:37:53.111172+0000","last_peered":"2024-09-21T03:37:53.111172+0000","last_clean":"2024-09-21T03:37:53.111172+0000","last_became_active":"2024-09-21T03:37:12.858047+0000","last_became_peered":"2024-09-21T03:37:12.858047+0000","last_unstale":"2024-09-21T03:37:53.111172+0000","last_undegraded":"2024-09-21T03:37:53.111172+0000","last_fullsized":"2024-09-21T03:37:53.111172+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:14:05.363743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.048838+0000","last_change":"2024-09-21T03:37:12.866784+0000","last_active":"2024-09-21T03:37:53.048838+0000","last_peered":"2024-09-21T03:37:53.048838+0000","last_clean":"2024-09-21T03:37:53.048838+0000","last_became_active":"2024-09-21T03:37:12.866435+0000","last_became_peered":"2024-09-21T03:37:12.866435+0000","last_unstale":"2024-09-21T03:37:53.048838+0000","last_undegraded":"2024-09-21T03:37:53.048838+0000","last_fullsized":"2024-09-21T03:37:53.048838+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:09:06.156030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.048908+0000","last_change":"2024-09-21T03:37:12.857054+0000","last_active":"2024-09-21T03:37:53.048908+0000","last_peered":"2024-09-21T03:37:53.048908+0000","last_clean":"2024-09-21T03:37:53.048908+0000","last_became_active":"2024-09-21T03:37:12.856671+0000","last_became_peered":"2024-09-21T03:37:12.856671+0000","last_unstale":"2024-09-21T03:37:53.048908+0000","last_undegraded":"2024-09-21T03:37:53.048908+0000","last_fullsized":"2024-09-21T03:37:53.048908+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:38:27.474550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.042778+0000","last_change":"2024-09-21T03:37:12.861501+0000","last_active":"2024-09-21T03:37:53.042778+0000","last_peered":"2024-09-21T03:37:53.042778+0000","last_clean":"2024-09-21T03:37:53.042778+0000","last_became_active":"2024-09-21T03:37:12.861146+0000","last_became_peered":"2024-09-21T03:37:12.861146+0000","last_unstale":"2024-09-21T03:37:53.042778+0000","last_undegraded":"2024-09-21T03:37:53.042778+0000","last_fullsized":"2024-09-21T03:37:53.042778+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:30:55.073854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.052333+0000","last_change":"2024-09-21T03:37:53.051912+0000","last_active":"2024-09-21T03:37:53.052333+0000","last_peered":"2024-09-21T03:37:53.052333+0000","last_clean":"2024-09-21T03:37:53.052333+0000","last_became_active":"2024-09-21T03:37:53.051640+0000","last_became_peered":"2024-09-21T03:37:53.051640+0000","last_unstale":"2024-09-21T03:37:53.052333+0000","last_undegraded":"2024-09-21T03:37:53.052333+0000","last_fullsized":"2024-09-21T03:37:53.052333+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:08:09.046867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,10,8],"acting":[9,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.431260+0000","last_change":"2024-09-21T03:37:12.865862+0000","last_active":"2024-09-21T03:37:53.431260+0000","last_peered":"2024-09-21T03:37:53.431260+0000","last_clean":"2024-09-21T03:37:53.431260+0000","last_became_active":"2024-09-21T03:37:12.865468+0000","last_became_peered":"2024-09-21T03:37:12.865468+0000","last_unstale":"2024-09-21T03:37:53.431260+0000","last_undegraded":"2024-09-21T03:37:53.431260+0000","last_fullsized":"2024-09-21T03:37:53.431260+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:18:14.496495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.047575+0000","last_change":"2024-09-21T03:37:12.862688+0000","last_active":"2024-09-21T03:37:53.047575+0000","last_peered":"2024-09-21T03:37:53.047575+0000","last_clean":"2024-09-21T03:37:53.047575+0000","last_became_active":"2024-09-21T03:37:12.862249+0000","last_became_peered":"2024-09-21T03:37:12.862249+0000","last_unstale":"2024-09-21T03:37:53.047575+0000","last_undegraded":"2024-09-21T03:37:53.047575+0000","last_fullsized":"2024-09-21T03:37:53.047575+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:55:54.731640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.112095+0000","last_change":"2024-09-21T03:37:12.863595+0000","last_active":"2024-09-21T03:37:53.112095+0000","last_peered":"2024-09-21T03:37:53.112095+0000","last_clean":"2024-09-21T03:37:53.112095+0000","last_became_active":"2024-09-21T03:37:12.863260+0000","last_became_peered":"2024-09-21T03:37:12.863260+0000","last_unstale":"2024-09-21T03:37:53.112095+0000","last_undegraded":"2024-09-21T03:37:53.112095+0000","last_fullsized":"2024-09-21T03:37:53.112095+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:10:07.995690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.045457+0000","last_change":"2024-09-21T03:37:12.865770+0000","last_active":"2024-09-21T03:37:53.045457+0000","last_peered":"2024-09-21T03:37:53.045457+0000","last_clean":"2024-09-21T03:37:53.045457+0000","last_became_active":"2024-09-21T03:37:12.865405+0000","last_became_peered":"2024-09-21T03:37:12.865405+0000","last_unstale":"2024-09-21T03:37:53.045457+0000","last_undegraded":"2024-09-21T03:37:53.045457+0000","last_fullsized":"2024-09-21T03:37:53.045457+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:46:25.255913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.430807+0000","last_change":"2024-09-21T03:37:12.856214+0000","last_active":"2024-09-21T03:37:53.430807+0000","last_peered":"2024-09-21T03:37:53.430807+0000","last_clean":"2024-09-21T03:37:53.430807+0000","last_became_active":"2024-09-21T03:37:12.853924+0000","last_became_peered":"2024-09-21T03:37:12.853924+0000","last_unstale":"2024-09-21T03:37:53.430807+0000","last_undegraded":"2024-09-21T03:37:53.430807+0000","last_fullsized":"2024-09-21T03:37:53.430807+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:10:32.182892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.047154+0000","last_change":"2024-09-21T03:37:12.858851+0000","last_active":"2024-09-21T03:37:53.047154+0000","last_peered":"2024-09-21T03:37:53.047154+0000","last_clean":"2024-09-21T03:37:53.047154+0000","last_became_active":"2024-09-21T03:37:12.858037+0000","last_became_peered":"2024-09-21T03:37:12.858037+0000","last_unstale":"2024-09-21T03:37:53.047154+0000","last_undegraded":"2024-09-21T03:37:53.047154+0000","last_fullsized":"2024-09-21T03:37:53.047154+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:32:23.134789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.051281+0000","last_change":"2024-09-21T03:37:12.852339+0000","last_active":"2024-09-21T03:37:53.051281+0000","last_peered":"2024-09-21T03:37:53.051281+0000","last_clean":"2024-09-21T03:37:53.051281+0000","last_became_active":"2024-09-21T03:37:12.851911+0000","last_became_peered":"2024-09-21T03:37:12.851911+0000","last_unstale":"2024-09-21T03:37:53.051281+0000","last_undegraded":"2024-09-21T03:37:53.051281+0000","last_fullsized":"2024-09-21T03:37:53.051281+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:03:30.739275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.044201+0000","last_change":"2024-09-21T03:37:12.861133+0000","last_active":"2024-09-21T03:37:53.044201+0000","last_peered":"2024-09-21T03:37:53.044201+0000","last_clean":"2024-09-21T03:37:53.044201+0000","last_became_active":"2024-09-21T03:37:12.860719+0000","last_became_peered":"2024-09-21T03:37:12.860719+0000","last_unstale":"2024-09-21T03:37:53.044201+0000","last_undegraded":"2024-09-21T03:37:53.044201+0000","last_fullsized":"2024-09-21T03:37:53.044201+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:21:25.241935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.049308+0000","last_change":"2024-09-21T03:37:12.865694+0000","last_active":"2024-09-21T03:37:53.049308+0000","last_peered":"2024-09-21T03:37:53.049308+0000","last_clean":"2024-09-21T03:37:53.049308+0000","last_became_active":"2024-09-21T03:37:12.864983+0000","last_became_peered":"2024-09-21T03:37:12.864983+0000","last_unstale":"2024-09-21T03:37:53.049308+0000","last_undegraded":"2024-09-21T03:37:53.049308+0000","last_fullsized":"2024-09-21T03:37:53.049308+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:52:28.583056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.049444+0000","last_change":"2024-09-21T03:37:12.860665+0000","last_active":"2024-09-21T03:37:53.049444+0000","last_peered":"2024-09-21T03:37:53.049444+0000","last_clean":"2024-09-21T03:37:53.049444+0000","last_became_active":"2024-09-21T03:37:12.860316+0000","last_became_peered":"2024-09-21T03:37:12.860316+0000","last_unstale":"2024-09-21T03:37:53.049444+0000","last_undegraded":"2024-09-21T03:37:53.049444+0000","last_fullsized":"2024-09-21T03:37:53.049444+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:13:07.316422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.048840+0000","last_change":"2024-09-21T03:37:12.869577+0000","last_active":"2024-09-21T03:37:53.048840+0000","last_peered":"2024-09-21T03:37:53.048840+0000","last_clean":"2024-09-21T03:37:53.048840+0000","last_became_active":"2024-09-21T03:37:12.869164+0000","last_became_peered":"2024-09-21T03:37:12.869164+0000","last_unstale":"2024-09-21T03:37:53.048840+0000","last_undegraded":"2024-09-21T03:37:53.048840+0000","last_fullsized":"2024-09-21T03:37:53.048840+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:40:34.167005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":71139,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":337,"num_read_kb":240,"num_write":164,"num_write_kb":119,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":694,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":213417,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":141,"ondisk_log_size":141,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":285501,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":68,"seq":292057776159,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32068,"kb_used_data":884,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713084,"statfs":{"total":95995035648,"available":95962198016,"internally_reserved":0,"allocated":905216,"data_stored":461385,"data_compressed":23469,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":63,"seq":270582939684,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36368,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708784,"statfs":{"total":95995035648,"available":95957794816,"internally_reserved":0,"allocated":1048576,"data_stored":1115679,"data_compressed":59763,"data_compressed_allocated":77824,"data_compressed_original":655360,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":58,"seq":249108103210,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27920,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717232,"statfs":{"total":95995035648,"available":95966445568,"internally_reserved":0,"allocated":917504,"data_stored":465605,"data_compressed":23469,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":52,"seq":223338299438,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27908,"kb_used_data":884,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717244,"statfs":{"total":95995035648,"available":95966457856,"internally_reserved":0,"allocated":905216,"data_stored":461317,"data_compressed":23469,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":46,"seq":197568495668,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36284,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708868,"statfs":{"total":95995035648,"available":95957880832,"internally_reserved":0,"allocated":962560,"data_stored":1057338,"data_compressed":59763,"data_compressed_allocated":77824,"data_compressed_original":655360,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":41,"seq":176093659192,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32064,"kb_used_data":872,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713088,"statfs":{"total":95995035648,"available":95962202112,"internally_reserved":0,"allocated":892928,"data_stored":456685,"data_compressed":23469,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":35,"seq":150323855421,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32140,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713012,"statfs":{"total":95995035648,"available":95962124288,"internally_reserved":0,"allocated":970752,"data_stored":1052926,"data_compressed":59763,"data_compressed_allocated":77824,"data_compressed_original":655360,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":30,"seq":128849018946,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27912,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717240,"statfs":{"total":95995035648,"available":95966453760,"internally_reserved":0,"allocated":909312,"data_stored":461917,"data_compressed":23469,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":24,"seq":103079215174,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27908,"kb_used_data":884,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717244,"statfs":{"total":95995035648,"available":95966457856,"internally_reserved":0,"allocated":905216,"data_stored":456879,"data_compressed":23469,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":17,"seq":73014444109,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27988,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717164,"statfs":{"total":95995035648,"available":95966375936,"internally_reserved":0,"allocated":987136,"data_stored":524892,"data_compressed":23469,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574929,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36288,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708864,"statfs":{"total":95995035648,"available":95957876736,"internally_reserved":0,"allocated":966656,"data_stored":523733,"data_compressed":23469,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738455,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32084,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713068,"statfs":{"total":95995035648,"available":95962181632,"internally_reserved":0,"allocated":917504,"data_stored":458073,"data_compressed":23469,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34677,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34677,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34677,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36294,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36294,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36294,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36294,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36294,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":67138,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":68297,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1226,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1867,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2183,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":626,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":68716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T03:38:44.411 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 7c374098-77c9-11ef-baf5-efdc52797490 -- ceph pg dump --format=json 2024-09-21T03:38:44.657 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/7c374098-77c9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:38:45.103 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:45.100+0000 7f6e53766640 1 -- 172.21.15.89:0/824737354 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6e4c1094a0 msgr2=0x7f6e4c1098b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:45.103 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:45.100+0000 7f6e53766640 1 --2- 172.21.15.89:0/824737354 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6e4c1094a0 0x7f6e4c1098b0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f6e40009900 tx=0x7f6e4002a410 comp rx=0 tx=0).stop 2024-09-21T03:38:45.104 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:45.101+0000 7f6e53766640 1 -- 172.21.15.89:0/824737354 shutdown_connections 2024-09-21T03:38:45.104 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:45.101+0000 7f6e53766640 1 --2- 172.21.15.89:0/824737354 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6e4c109df0 0x7f6e4c110620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:45.104 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:45.101+0000 7f6e53766640 1 --2- 172.21.15.89:0/824737354 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6e4c1094a0 0x7f6e4c1098b0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:45.104 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:45.101+0000 7f6e53766640 1 --2- 172.21.15.89:0/824737354 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6e4c103b60 0x7f6e4c103f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:45.104 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:45.101+0000 7f6e53766640 1 -- 172.21.15.89:0/824737354 >> 172.21.15.89:0/824737354 conn(0x7f6e4c07b850 msgr2=0x7f6e4c07bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:38:45.104 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:45.101+0000 7f6e53766640 1 -- 172.21.15.89:0/824737354 shutdown_connections 2024-09-21T03:38:45.104 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:45.101+0000 7f6e53766640 1 -- 172.21.15.89:0/824737354 wait complete. 2024-09-21T03:38:45.104 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:45.101+0000 7f6e53766640 1 Processor -- start 2024-09-21T03:38:45.104 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:45.101+0000 7f6e53766640 1 -- start start 2024-09-21T03:38:45.106 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:45.102+0000 7f6e53766640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6e4c103b60 0x7f6e4c19a3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:45.106 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:45.102+0000 7f6e53766640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6e4c1094a0 0x7f6e4c19a970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:45.106 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:45.102+0000 7f6e53766640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6e4c109df0 0x7f6e4c19fc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:45.106 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:45.102+0000 7f6e53766640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f6e4c113960 con 0x7f6e4c1094a0 2024-09-21T03:38:45.106 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:45.102+0000 7f6e53766640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f6e4c1137e0 con 0x7f6e4c109df0 2024-09-21T03:38:45.106 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:45.102+0000 7f6e53766640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f6e4c113ae0 con 0x7f6e4c103b60 2024-09-21T03:38:45.106 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:45.102+0000 7f6e50d01640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6e4c1094a0 0x7f6e4c19a970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:45.106 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:45.102+0000 7f6e50d01640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6e4c1094a0 0x7f6e4c19a970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:51162/0 (socket says 172.21.15.89:51162) 2024-09-21T03:38:45.106 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:45.102+0000 7f6e50d01640 1 -- 172.21.15.89:0/681659777 learned_addr learned my addr 172.21.15.89:0/681659777 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-21T03:38:45.106 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:45.102+0000 7f6e51d03640 1 --2- 172.21.15.89:0/681659777 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6e4c109df0 0x7f6e4c19fc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:45.106 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:45.102+0000 7f6e51502640 1 --2- 172.21.15.89:0/681659777 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6e4c103b60 0x7f6e4c19a3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:45.107 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:45.102+0000 7f6e50d01640 1 -- 172.21.15.89:0/681659777 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6e4c103b60 msgr2=0x7f6e4c19a3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:45.107 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:45.102+0000 7f6e50d01640 1 --2- 172.21.15.89:0/681659777 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6e4c103b60 0x7f6e4c19a3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:45.107 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:45.102+0000 7f6e50d01640 1 -- 172.21.15.89:0/681659777 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6e4c109df0 msgr2=0x7f6e4c19fc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:45.107 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:45.102+0000 7f6e50d01640 1 --2- 172.21.15.89:0/681659777 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6e4c109df0 0x7f6e4c19fc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:45.107 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:45.102+0000 7f6e50d01640 1 -- 172.21.15.89:0/681659777 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e4c1a0340 con 0x7f6e4c1094a0 2024-09-21T03:38:45.107 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:45.102+0000 7f6e50d01640 1 --2- 172.21.15.89:0/681659777 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6e4c1094a0 0x7f6e4c19a970 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f6e40005940 tx=0x7f6e400040e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:38:45.107 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:45.103+0000 7f6e3e7fc640 1 -- 172.21.15.89:0/681659777 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e4003f030 con 0x7f6e4c1094a0 2024-09-21T03:38:45.107 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:45.103+0000 7f6e3e7fc640 1 -- 172.21.15.89:0/681659777 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6e40003a80 con 0x7f6e4c1094a0 2024-09-21T03:38:45.107 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:45.103+0000 7f6e3e7fc640 1 -- 172.21.15.89:0/681659777 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e40035070 con 0x7f6e4c1094a0 2024-09-21T03:38:45.107 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:45.103+0000 7f6e53766640 1 -- 172.21.15.89:0/681659777 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6e4c1a05d0 con 0x7f6e4c1094a0 2024-09-21T03:38:45.107 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:45.103+0000 7f6e53766640 1 -- 172.21.15.89:0/681659777 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6e4c1a6d90 con 0x7f6e4c1094a0 2024-09-21T03:38:45.107 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:45.104+0000 7f6e53766640 1 -- 172.21.15.89:0/681659777 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6e4c103f70 con 0x7f6e4c1094a0 2024-09-21T03:38:45.108 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:45.105+0000 7f6e3e7fc640 1 -- 172.21.15.89:0/681659777 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 18) ==== 154563+0+0 (secure 0 0 0) 0x7f6e40042030 con 0x7f6e4c1094a0 2024-09-21T03:38:45.108 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:45.106+0000 7f6e3e7fc640 1 --2- 172.21.15.89:0/681659777 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f6e380b65a0 0x7f6e380b8ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:45.109 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:45.106+0000 7f6e51502640 1 --2- 172.21.15.89:0/681659777 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f6e380b65a0 0x7f6e380b8ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:45.109 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:45.106+0000 7f6e3e7fc640 1 -- 172.21.15.89:0/681659777 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 9092+0+0 (secure 0 0 0) 0x7f6e40108130 con 0x7f6e4c1094a0 2024-09-21T03:38:45.109 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:45.107+0000 7f6e51502640 1 --2- 172.21.15.89:0/681659777 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f6e380b65a0 0x7f6e380b8ab0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f6e4800bbe0 tx=0x7f6e4800b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:38:45.112 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:45.110+0000 7f6e3e7fc640 1 -- 172.21.15.89:0/681659777 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f6e400d2880 con 0x7f6e4c1094a0 2024-09-21T03:38:45.284 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:45.282+0000 7f6e53766640 1 -- 172.21.15.89:0/681659777 --> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f6e4c10bc20 con 0x7f6e380b65a0 2024-09-21T03:38:45.290 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:45.288+0000 7f6e3e7fc640 1 -- 172.21.15.89:0/681659777 <== mgr.14152 v2:172.21.15.89:6800/1591532358 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340623 (secure 0 0 0) 0x7f6e4c10bc20 con 0x7f6e380b65a0 2024-09-21T03:38:45.293 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-09-21T03:38:45.294 INFO:teuthology.orchestra.run.smithi089.stderr:dumped all 2024-09-21T03:38:45.296 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:45.291+0000 7f6e53766640 1 -- 172.21.15.89:0/681659777 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f6e380b65a0 msgr2=0x7f6e380b8ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:45.296 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:45.291+0000 7f6e53766640 1 --2- 172.21.15.89:0/681659777 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f6e380b65a0 0x7f6e380b8ab0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f6e4800bbe0 tx=0x7f6e4800b390 comp rx=0 tx=0).stop 2024-09-21T03:38:45.296 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:45.291+0000 7f6e53766640 1 -- 172.21.15.89:0/681659777 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6e4c1094a0 msgr2=0x7f6e4c19a970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:45.296 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:45.291+0000 7f6e53766640 1 --2- 172.21.15.89:0/681659777 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6e4c1094a0 0x7f6e4c19a970 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f6e40005940 tx=0x7f6e400040e0 comp rx=0 tx=0).stop 2024-09-21T03:38:45.297 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:45.291+0000 7f6e53766640 1 -- 172.21.15.89:0/681659777 shutdown_connections 2024-09-21T03:38:45.297 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:45.291+0000 7f6e53766640 1 --2- 172.21.15.89:0/681659777 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f6e380b65a0 0x7f6e380b8ab0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:45.297 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:45.291+0000 7f6e53766640 1 --2- 172.21.15.89:0/681659777 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6e4c109df0 0x7f6e4c19fc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:45.297 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:45.291+0000 7f6e53766640 1 --2- 172.21.15.89:0/681659777 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6e4c1094a0 0x7f6e4c19a970 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:45.297 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:45.291+0000 7f6e53766640 1 --2- 172.21.15.89:0/681659777 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6e4c103b60 0x7f6e4c19a3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:45.297 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:45.291+0000 7f6e53766640 1 -- 172.21.15.89:0/681659777 >> 172.21.15.89:0/681659777 conn(0x7f6e4c07b850 msgr2=0x7f6e4c1055c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:38:45.297 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:45.291+0000 7f6e53766640 1 -- 172.21.15.89:0/681659777 shutdown_connections 2024-09-21T03:38:45.297 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:45.291+0000 7f6e53766640 1 -- 172.21.15.89:0/681659777 wait complete. 2024-09-21T03:38:45.750 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:38:45 smithi110 ceph-mon[34000]: pgmap v338: 129 pgs: 129 active+clean; 646 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-21T03:38:45.750 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:38:45 smithi110 ceph-mon[34000]: from='client.15207 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T03:38:45.811 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:38:45 smithi089 ceph-mon[30928]: pgmap v338: 129 pgs: 129 active+clean; 646 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-21T03:38:45.811 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:38:45 smithi089 ceph-mon[30928]: from='client.15207 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T03:38:45.855 INFO:teuthology.orchestra.run.smithi089.stdout:{"pg_ready":true,"pg_map":{"version":338,"stamp":"2024-09-21T03:38:43.634020+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":661741,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":413,"num_read_kb":304,"num_write":289,"num_write_kb":2271,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":23,"num_bytes_recovered":2952534,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":235,"ondisk_log_size":235,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":376932,"kb_used_data":11024,"kb_used_omap":18,"kb_used_meta":365677,"kb_avail":1124564892,"statfs":{"total":1151940427776,"available":1151554449408,"internally_reserved":0,"allocated":11288576,"data_stored":7496429,"data_compressed":390510,"data_compressed_allocated":528384,"data_compressed_original":2555904,"omap_allocated":19143,"internal_metadata":374453561},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":70,"num_read_kb":30,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003166"},"pg_stats":[{"pgid":"2.1f","version":"80'9","reported_seq":45,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.047700+0000","last_change":"2024-09-21T03:37:10.836953+0000","last_active":"2024-09-21T03:37:53.047700+0000","last_peered":"2024-09-21T03:37:53.047700+0000","last_clean":"2024-09-21T03:37:53.047700+0000","last_became_active":"2024-09-21T03:37:10.836566+0000","last_became_peered":"2024-09-21T03:37:10.836566+0000","last_unstale":"2024-09-21T03:37:53.047700+0000","last_undegraded":"2024-09-21T03:37:53.047700+0000","last_fullsized":"2024-09-21T03:37:53.047700+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_clean_scrub_stamp":"2024-09-21T03:37:09.809805+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:15:45.591225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.046350+0000","last_change":"2024-09-21T03:37:12.866931+0000","last_active":"2024-09-21T03:37:53.046350+0000","last_peered":"2024-09-21T03:37:53.046350+0000","last_clean":"2024-09-21T03:37:53.046350+0000","last_became_active":"2024-09-21T03:37:12.866452+0000","last_became_peered":"2024-09-21T03:37:12.866452+0000","last_unstale":"2024-09-21T03:37:53.046350+0000","last_undegraded":"2024-09-21T03:37:53.046350+0000","last_fullsized":"2024-09-21T03:37:53.046350+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:37:14.324548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.044138+0000","last_change":"2024-09-21T03:37:19.912154+0000","last_active":"2024-09-21T03:37:53.044138+0000","last_peered":"2024-09-21T03:37:53.044138+0000","last_clean":"2024-09-21T03:37:53.044138+0000","last_became_active":"2024-09-21T03:37:19.911686+0000","last_became_peered":"2024-09-21T03:37:19.911686+0000","last_unstale":"2024-09-21T03:37:53.044138+0000","last_undegraded":"2024-09-21T03:37:53.044138+0000","last_fullsized":"2024-09-21T03:37:53.044138+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_clean_scrub_stamp":"2024-09-21T03:37:18.876933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:29:20.189418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"80'2","reported_seq":37,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.046676+0000","last_change":"2024-09-21T03:37:11.034501+0000","last_active":"2024-09-21T03:37:53.046676+0000","last_peered":"2024-09-21T03:37:53.046676+0000","last_clean":"2024-09-21T03:37:53.046676+0000","last_became_active":"2024-09-21T03:37:11.034110+0000","last_became_peered":"2024-09-21T03:37:11.034110+0000","last_unstale":"2024-09-21T03:37:53.046676+0000","last_undegraded":"2024-09-21T03:37:53.046676+0000","last_fullsized":"2024-09-21T03:37:53.046676+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_clean_scrub_stamp":"2024-09-21T03:37:09.809805+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:03:12.854469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.049598+0000","last_change":"2024-09-21T03:37:12.867330+0000","last_active":"2024-09-21T03:37:53.049598+0000","last_peered":"2024-09-21T03:37:53.049598+0000","last_clean":"2024-09-21T03:37:53.049598+0000","last_became_active":"2024-09-21T03:37:12.866952+0000","last_became_peered":"2024-09-21T03:37:12.866952+0000","last_unstale":"2024-09-21T03:37:53.049598+0000","last_undegraded":"2024-09-21T03:37:53.049598+0000","last_fullsized":"2024-09-21T03:37:53.049598+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:40:33.826546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.043412+0000","last_change":"2024-09-21T03:37:19.910661+0000","last_active":"2024-09-21T03:37:53.043412+0000","last_peered":"2024-09-21T03:37:53.043412+0000","last_clean":"2024-09-21T03:37:53.043412+0000","last_became_active":"2024-09-21T03:37:19.910108+0000","last_became_peered":"2024-09-21T03:37:19.910108+0000","last_unstale":"2024-09-21T03:37:53.043412+0000","last_undegraded":"2024-09-21T03:37:53.043412+0000","last_fullsized":"2024-09-21T03:37:53.043412+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_clean_scrub_stamp":"2024-09-21T03:37:18.876933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:18:05.654426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"80'4","reported_seq":39,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.050839+0000","last_change":"2024-09-21T03:37:10.835336+0000","last_active":"2024-09-21T03:37:53.050839+0000","last_peered":"2024-09-21T03:37:53.050839+0000","last_clean":"2024-09-21T03:37:53.050839+0000","last_became_active":"2024-09-21T03:37:10.834928+0000","last_became_peered":"2024-09-21T03:37:10.834928+0000","last_unstale":"2024-09-21T03:37:53.050839+0000","last_undegraded":"2024-09-21T03:37:53.050839+0000","last_fullsized":"2024-09-21T03:37:53.050839+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_clean_scrub_stamp":"2024-09-21T03:37:09.809805+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:02:14.717520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.111510+0000","last_change":"2024-09-21T03:37:12.861271+0000","last_active":"2024-09-21T03:37:53.111510+0000","last_peered":"2024-09-21T03:37:53.111510+0000","last_clean":"2024-09-21T03:37:53.111510+0000","last_became_active":"2024-09-21T03:37:12.860932+0000","last_became_peered":"2024-09-21T03:37:12.860932+0000","last_unstale":"2024-09-21T03:37:53.111510+0000","last_undegraded":"2024-09-21T03:37:53.111510+0000","last_fullsized":"2024-09-21T03:37:53.111510+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:46:08.993250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.053171+0000","last_change":"2024-09-21T03:37:53.053045+0000","last_active":"2024-09-21T03:37:53.053171+0000","last_peered":"2024-09-21T03:37:53.053171+0000","last_clean":"2024-09-21T03:37:53.053171+0000","last_became_active":"2024-09-21T03:37:53.052382+0000","last_became_peered":"2024-09-21T03:37:53.052382+0000","last_unstale":"2024-09-21T03:37:53.053171+0000","last_undegraded":"2024-09-21T03:37:53.053171+0000","last_fullsized":"2024-09-21T03:37:53.053171+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_clean_scrub_stamp":"2024-09-21T03:37:18.876933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:02:44.300768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,8],"acting":[4,3,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"74'2","reported_seq":37,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.050328+0000","last_change":"2024-09-21T03:37:10.833985+0000","last_active":"2024-09-21T03:37:53.050328+0000","last_peered":"2024-09-21T03:37:53.050328+0000","last_clean":"2024-09-21T03:37:53.050328+0000","last_became_active":"2024-09-21T03:37:10.833562+0000","last_became_peered":"2024-09-21T03:37:10.833562+0000","last_unstale":"2024-09-21T03:37:53.050328+0000","last_undegraded":"2024-09-21T03:37:53.050328+0000","last_fullsized":"2024-09-21T03:37:53.050328+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_clean_scrub_stamp":"2024-09-21T03:37:09.809805+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:49:32.617768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.047066+0000","last_change":"2024-09-21T03:37:12.865408+0000","last_active":"2024-09-21T03:37:53.047066+0000","last_peered":"2024-09-21T03:37:53.047066+0000","last_clean":"2024-09-21T03:37:53.047066+0000","last_became_active":"2024-09-21T03:37:12.864972+0000","last_became_peered":"2024-09-21T03:37:12.864972+0000","last_unstale":"2024-09-21T03:37:53.047066+0000","last_undegraded":"2024-09-21T03:37:53.047066+0000","last_fullsized":"2024-09-21T03:37:53.047066+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:03:16.384260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.043495+0000","last_change":"2024-09-21T03:37:19.912105+0000","last_active":"2024-09-21T03:37:53.043495+0000","last_peered":"2024-09-21T03:37:53.043495+0000","last_clean":"2024-09-21T03:37:53.043495+0000","last_became_active":"2024-09-21T03:37:19.911416+0000","last_became_peered":"2024-09-21T03:37:19.911416+0000","last_unstale":"2024-09-21T03:37:53.043495+0000","last_undegraded":"2024-09-21T03:37:53.043495+0000","last_fullsized":"2024-09-21T03:37:53.043495+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_clean_scrub_stamp":"2024-09-21T03:37:18.876933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:20:16.492231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.431254+0000","last_change":"2024-09-21T03:37:12.862354+0000","last_active":"2024-09-21T03:37:53.431254+0000","last_peered":"2024-09-21T03:37:53.431254+0000","last_clean":"2024-09-21T03:37:53.431254+0000","last_became_active":"2024-09-21T03:37:12.861977+0000","last_became_peered":"2024-09-21T03:37:12.861977+0000","last_unstale":"2024-09-21T03:37:53.431254+0000","last_undegraded":"2024-09-21T03:37:53.431254+0000","last_fullsized":"2024-09-21T03:37:53.431254+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:53:52.725524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"80'4","reported_seq":40,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.051416+0000","last_change":"2024-09-21T03:37:10.832140+0000","last_active":"2024-09-21T03:37:53.051416+0000","last_peered":"2024-09-21T03:37:53.051416+0000","last_clean":"2024-09-21T03:37:53.051416+0000","last_became_active":"2024-09-21T03:37:10.831395+0000","last_became_peered":"2024-09-21T03:37:10.831395+0000","last_unstale":"2024-09-21T03:37:53.051416+0000","last_undegraded":"2024-09-21T03:37:53.051416+0000","last_fullsized":"2024-09-21T03:37:53.051416+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_clean_scrub_stamp":"2024-09-21T03:37:09.809805+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:55:20.440890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.044184+0000","last_change":"2024-09-21T03:37:20.160850+0000","last_active":"2024-09-21T03:37:53.044184+0000","last_peered":"2024-09-21T03:37:53.044184+0000","last_clean":"2024-09-21T03:37:53.044184+0000","last_became_active":"2024-09-21T03:37:20.160305+0000","last_became_peered":"2024-09-21T03:37:20.160305+0000","last_unstale":"2024-09-21T03:37:53.044184+0000","last_undegraded":"2024-09-21T03:37:53.044184+0000","last_fullsized":"2024-09-21T03:37:53.044184+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_clean_scrub_stamp":"2024-09-21T03:37:18.876933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:51:45.971984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.046377+0000","last_change":"2024-09-21T03:37:12.859485+0000","last_active":"2024-09-21T03:37:53.046377+0000","last_peered":"2024-09-21T03:37:53.046377+0000","last_clean":"2024-09-21T03:37:53.046377+0000","last_became_active":"2024-09-21T03:37:12.859097+0000","last_became_peered":"2024-09-21T03:37:12.859097+0000","last_unstale":"2024-09-21T03:37:53.046377+0000","last_undegraded":"2024-09-21T03:37:53.046377+0000","last_fullsized":"2024-09-21T03:37:53.046377+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:16:05.438360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"80'1","reported_seq":36,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.044264+0000","last_change":"2024-09-21T03:37:10.832441+0000","last_active":"2024-09-21T03:37:53.044264+0000","last_peered":"2024-09-21T03:37:53.044264+0000","last_clean":"2024-09-21T03:37:53.044264+0000","last_became_active":"2024-09-21T03:37:10.832073+0000","last_became_peered":"2024-09-21T03:37:10.832073+0000","last_unstale":"2024-09-21T03:37:53.044264+0000","last_undegraded":"2024-09-21T03:37:53.044264+0000","last_fullsized":"2024-09-21T03:37:53.044264+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_clean_scrub_stamp":"2024-09-21T03:37:09.809805+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:32:19.411097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"82'1","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.043995+0000","last_change":"2024-09-21T03:37:19.905490+0000","last_active":"2024-09-21T03:37:53.043995+0000","last_peered":"2024-09-21T03:37:53.043995+0000","last_clean":"2024-09-21T03:37:53.043995+0000","last_became_active":"2024-09-21T03:37:19.904971+0000","last_became_peered":"2024-09-21T03:37:19.904971+0000","last_unstale":"2024-09-21T03:37:53.043995+0000","last_undegraded":"2024-09-21T03:37:53.043995+0000","last_fullsized":"2024-09-21T03:37:53.043995+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_clean_scrub_stamp":"2024-09-21T03:37:18.876933+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:44:20.655538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.048957+0000","last_change":"2024-09-21T03:37:12.863597+0000","last_active":"2024-09-21T03:37:53.048957+0000","last_peered":"2024-09-21T03:37:53.048957+0000","last_clean":"2024-09-21T03:37:53.048957+0000","last_became_active":"2024-09-21T03:37:12.863223+0000","last_became_peered":"2024-09-21T03:37:12.863223+0000","last_unstale":"2024-09-21T03:37:53.048957+0000","last_undegraded":"2024-09-21T03:37:53.048957+0000","last_fullsized":"2024-09-21T03:37:53.048957+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:29:28.878964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.9","version":"80'3","reported_seq":38,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.045808+0000","last_change":"2024-09-21T03:37:10.834648+0000","last_active":"2024-09-21T03:37:53.045808+0000","last_peered":"2024-09-21T03:37:53.045808+0000","last_clean":"2024-09-21T03:37:53.045808+0000","last_became_active":"2024-09-21T03:37:10.834293+0000","last_became_peered":"2024-09-21T03:37:10.834293+0000","last_unstale":"2024-09-21T03:37:53.045808+0000","last_undegraded":"2024-09-21T03:37:53.045808+0000","last_fullsized":"2024-09-21T03:37:53.045808+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_clean_scrub_stamp":"2024-09-21T03:37:09.809805+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:10:41.248733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.047492+0000","last_change":"2024-09-21T03:37:19.912526+0000","last_active":"2024-09-21T03:37:53.047492+0000","last_peered":"2024-09-21T03:37:53.047492+0000","last_clean":"2024-09-21T03:37:53.047492+0000","last_became_active":"2024-09-21T03:37:19.912062+0000","last_became_peered":"2024-09-21T03:37:19.912062+0000","last_unstale":"2024-09-21T03:37:53.047492+0000","last_undegraded":"2024-09-21T03:37:53.047492+0000","last_fullsized":"2024-09-21T03:37:53.047492+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_clean_scrub_stamp":"2024-09-21T03:37:18.876933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:52:40.395834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.049515+0000","last_change":"2024-09-21T03:37:12.855672+0000","last_active":"2024-09-21T03:37:53.049515+0000","last_peered":"2024-09-21T03:37:53.049515+0000","last_clean":"2024-09-21T03:37:53.049515+0000","last_became_active":"2024-09-21T03:37:12.855191+0000","last_became_peered":"2024-09-21T03:37:12.855191+0000","last_unstale":"2024-09-21T03:37:53.049515+0000","last_undegraded":"2024-09-21T03:37:53.049515+0000","last_fullsized":"2024-09-21T03:37:53.049515+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:26:56.115697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.8","version":"80'2","reported_seq":37,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.111296+0000","last_change":"2024-09-21T03:37:11.034752+0000","last_active":"2024-09-21T03:37:53.111296+0000","last_peered":"2024-09-21T03:37:53.111296+0000","last_clean":"2024-09-21T03:37:53.111296+0000","last_became_active":"2024-09-21T03:37:11.034386+0000","last_became_peered":"2024-09-21T03:37:11.034386+0000","last_unstale":"2024-09-21T03:37:53.111296+0000","last_undegraded":"2024-09-21T03:37:53.111296+0000","last_fullsized":"2024-09-21T03:37:53.111296+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_clean_scrub_stamp":"2024-09-21T03:37:09.809805+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:17:23.647411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.044213+0000","last_change":"2024-09-21T03:37:20.162051+0000","last_active":"2024-09-21T03:37:53.044213+0000","last_peered":"2024-09-21T03:37:53.044213+0000","last_clean":"2024-09-21T03:37:53.044213+0000","last_became_active":"2024-09-21T03:37:20.161452+0000","last_became_peered":"2024-09-21T03:37:20.161452+0000","last_unstale":"2024-09-21T03:37:53.044213+0000","last_undegraded":"2024-09-21T03:37:53.044213+0000","last_fullsized":"2024-09-21T03:37:53.044213+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_clean_scrub_stamp":"2024-09-21T03:37:18.876933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:29:22.544800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.049528+0000","last_change":"2024-09-21T03:37:12.866058+0000","last_active":"2024-09-21T03:37:53.049528+0000","last_peered":"2024-09-21T03:37:53.049528+0000","last_clean":"2024-09-21T03:37:53.049528+0000","last_became_active":"2024-09-21T03:37:12.865694+0000","last_became_peered":"2024-09-21T03:37:12.865694+0000","last_unstale":"2024-09-21T03:37:53.049528+0000","last_undegraded":"2024-09-21T03:37:53.049528+0000","last_fullsized":"2024-09-21T03:37:53.049528+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:56:33.629873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"80'4","reported_seq":39,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.044782+0000","last_change":"2024-09-21T03:37:11.358619+0000","last_active":"2024-09-21T03:37:53.044782+0000","last_peered":"2024-09-21T03:37:53.044782+0000","last_clean":"2024-09-21T03:37:53.044782+0000","last_became_active":"2024-09-21T03:37:11.358196+0000","last_became_peered":"2024-09-21T03:37:11.358196+0000","last_unstale":"2024-09-21T03:37:53.044782+0000","last_undegraded":"2024-09-21T03:37:53.044782+0000","last_fullsized":"2024-09-21T03:37:53.044782+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_clean_scrub_stamp":"2024-09-21T03:37:09.809805+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:57:13.476641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.044378+0000","last_change":"2024-09-21T03:37:19.905711+0000","last_active":"2024-09-21T03:37:53.044378+0000","last_peered":"2024-09-21T03:37:53.044378+0000","last_clean":"2024-09-21T03:37:53.044378+0000","last_became_active":"2024-09-21T03:37:19.905260+0000","last_became_peered":"2024-09-21T03:37:19.905260+0000","last_unstale":"2024-09-21T03:37:53.044378+0000","last_undegraded":"2024-09-21T03:37:53.044378+0000","last_fullsized":"2024-09-21T03:37:53.044378+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_clean_scrub_stamp":"2024-09-21T03:37:18.876933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:21:09.476082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.046425+0000","last_change":"2024-09-21T03:37:12.854376+0000","last_active":"2024-09-21T03:37:53.046425+0000","last_peered":"2024-09-21T03:37:53.046425+0000","last_clean":"2024-09-21T03:37:53.046425+0000","last_became_active":"2024-09-21T03:37:12.852942+0000","last_became_peered":"2024-09-21T03:37:12.852942+0000","last_unstale":"2024-09-21T03:37:53.046425+0000","last_undegraded":"2024-09-21T03:37:53.046425+0000","last_fullsized":"2024-09-21T03:37:53.046425+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:06:49.319002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"82'10","reported_seq":58,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:38:03.865942+0000","last_change":"2024-09-21T03:37:53.056889+0000","last_active":"2024-09-21T03:38:03.865942+0000","last_peered":"2024-09-21T03:38:03.865942+0000","last_clean":"2024-09-21T03:38:03.865942+0000","last_became_active":"2024-09-21T03:37:53.053054+0000","last_became_peered":"2024-09-21T03:37:53.053054+0000","last_unstale":"2024-09-21T03:38:03.865942+0000","last_undegraded":"2024-09-21T03:38:03.865942+0000","last_fullsized":"2024-09-21T03:38:03.865942+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_clean_scrub_stamp":"2024-09-21T03:37:09.809805+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:16:05.539111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,3],"acting":[9,11,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.045339+0000","last_change":"2024-09-21T03:37:20.161515+0000","last_active":"2024-09-21T03:37:53.045339+0000","last_peered":"2024-09-21T03:37:53.045339+0000","last_clean":"2024-09-21T03:37:53.045339+0000","last_became_active":"2024-09-21T03:37:20.160548+0000","last_became_peered":"2024-09-21T03:37:20.160548+0000","last_unstale":"2024-09-21T03:37:53.045339+0000","last_undegraded":"2024-09-21T03:37:53.045339+0000","last_fullsized":"2024-09-21T03:37:53.045339+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_clean_scrub_stamp":"2024-09-21T03:37:18.876933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:17:51.144859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.042971+0000","last_change":"2024-09-21T03:37:12.848305+0000","last_active":"2024-09-21T03:37:53.042971+0000","last_peered":"2024-09-21T03:37:53.042971+0000","last_clean":"2024-09-21T03:37:53.042971+0000","last_became_active":"2024-09-21T03:37:12.847916+0000","last_became_peered":"2024-09-21T03:37:12.847916+0000","last_unstale":"2024-09-21T03:37:53.042971+0000","last_undegraded":"2024-09-21T03:37:53.042971+0000","last_fullsized":"2024-09-21T03:37:53.042971+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:48:39.922805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"80'1","reported_seq":37,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.051068+0000","last_change":"2024-09-21T03:37:11.035964+0000","last_active":"2024-09-21T03:37:53.051068+0000","last_peered":"2024-09-21T03:37:53.051068+0000","last_clean":"2024-09-21T03:37:53.051068+0000","last_became_active":"2024-09-21T03:37:11.035427+0000","last_became_peered":"2024-09-21T03:37:11.035427+0000","last_unstale":"2024-09-21T03:37:53.051068+0000","last_undegraded":"2024-09-21T03:37:53.051068+0000","last_fullsized":"2024-09-21T03:37:53.051068+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_clean_scrub_stamp":"2024-09-21T03:37:09.809805+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:04:51.026753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.047032+0000","last_change":"2024-09-21T03:37:19.909428+0000","last_active":"2024-09-21T03:37:53.047032+0000","last_peered":"2024-09-21T03:37:53.047032+0000","last_clean":"2024-09-21T03:37:53.047032+0000","last_became_active":"2024-09-21T03:37:19.908904+0000","last_became_peered":"2024-09-21T03:37:19.908904+0000","last_unstale":"2024-09-21T03:37:53.047032+0000","last_undegraded":"2024-09-21T03:37:53.047032+0000","last_fullsized":"2024-09-21T03:37:53.047032+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_clean_scrub_stamp":"2024-09-21T03:37:18.876933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:12:07.564058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.046987+0000","last_change":"2024-09-21T03:37:12.855592+0000","last_active":"2024-09-21T03:37:53.046987+0000","last_peered":"2024-09-21T03:37:53.046987+0000","last_clean":"2024-09-21T03:37:53.046987+0000","last_became_active":"2024-09-21T03:37:12.855244+0000","last_became_peered":"2024-09-21T03:37:12.855244+0000","last_unstale":"2024-09-21T03:37:53.046987+0000","last_undegraded":"2024-09-21T03:37:53.046987+0000","last_fullsized":"2024-09-21T03:37:53.046987+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:21:34.603141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":35,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.047381+0000","last_change":"2024-09-21T03:37:11.036401+0000","last_active":"2024-09-21T03:37:53.047381+0000","last_peered":"2024-09-21T03:37:53.047381+0000","last_clean":"2024-09-21T03:37:53.047381+0000","last_became_active":"2024-09-21T03:37:11.036003+0000","last_became_peered":"2024-09-21T03:37:11.036003+0000","last_unstale":"2024-09-21T03:37:53.047381+0000","last_undegraded":"2024-09-21T03:37:53.047381+0000","last_fullsized":"2024-09-21T03:37:53.047381+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_clean_scrub_stamp":"2024-09-21T03:37:09.809805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:35:50.549060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.050609+0000","last_change":"2024-09-21T03:37:20.160668+0000","last_active":"2024-09-21T03:37:53.050609+0000","last_peered":"2024-09-21T03:37:53.050609+0000","last_clean":"2024-09-21T03:37:53.050609+0000","last_became_active":"2024-09-21T03:37:20.160158+0000","last_became_peered":"2024-09-21T03:37:20.160158+0000","last_unstale":"2024-09-21T03:37:53.050609+0000","last_undegraded":"2024-09-21T03:37:53.050609+0000","last_fullsized":"2024-09-21T03:37:53.050609+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_clean_scrub_stamp":"2024-09-21T03:37:18.876933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:45:25.677667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":37,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.050453+0000","last_change":"2024-09-21T03:37:53.050351+0000","last_active":"2024-09-21T03:37:53.050453+0000","last_peered":"2024-09-21T03:37:53.050453+0000","last_clean":"2024-09-21T03:37:53.050453+0000","last_became_active":"2024-09-21T03:37:53.050003+0000","last_became_peered":"2024-09-21T03:37:53.050003+0000","last_unstale":"2024-09-21T03:37:53.050453+0000","last_undegraded":"2024-09-21T03:37:53.050453+0000","last_fullsized":"2024-09-21T03:37:53.050453+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:49:03.313002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,0],"acting":[10,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"80'1","reported_seq":36,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.431360+0000","last_change":"2024-09-21T03:37:10.832635+0000","last_active":"2024-09-21T03:37:53.431360+0000","last_peered":"2024-09-21T03:37:53.431360+0000","last_clean":"2024-09-21T03:37:53.431360+0000","last_became_active":"2024-09-21T03:37:10.832280+0000","last_became_peered":"2024-09-21T03:37:10.832280+0000","last_unstale":"2024-09-21T03:37:53.431360+0000","last_undegraded":"2024-09-21T03:37:53.431360+0000","last_fullsized":"2024-09-21T03:37:53.431360+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_clean_scrub_stamp":"2024-09-21T03:37:09.809805+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:54:52.744486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.042822+0000","last_change":"2024-09-21T03:37:19.904112+0000","last_active":"2024-09-21T03:37:53.042822+0000","last_peered":"2024-09-21T03:37:53.042822+0000","last_clean":"2024-09-21T03:37:53.042822+0000","last_became_active":"2024-09-21T03:37:19.903677+0000","last_became_peered":"2024-09-21T03:37:19.903677+0000","last_unstale":"2024-09-21T03:37:53.042822+0000","last_undegraded":"2024-09-21T03:37:53.042822+0000","last_fullsized":"2024-09-21T03:37:53.042822+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_clean_scrub_stamp":"2024-09-21T03:37:18.876933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:32:20.096850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.046073+0000","last_change":"2024-09-21T03:37:12.864082+0000","last_active":"2024-09-21T03:37:53.046073+0000","last_peered":"2024-09-21T03:37:53.046073+0000","last_clean":"2024-09-21T03:37:53.046073+0000","last_became_active":"2024-09-21T03:37:12.862745+0000","last_became_peered":"2024-09-21T03:37:12.862745+0000","last_unstale":"2024-09-21T03:37:53.046073+0000","last_undegraded":"2024-09-21T03:37:53.046073+0000","last_fullsized":"2024-09-21T03:37:53.046073+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:25:35.759719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"80'4","reported_seq":90,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:38:38.106317+0000","last_change":"2024-09-21T03:37:11.034463+0000","last_active":"2024-09-21T03:38:38.106317+0000","last_peered":"2024-09-21T03:38:38.106317+0000","last_clean":"2024-09-21T03:38:38.106317+0000","last_became_active":"2024-09-21T03:37:11.033932+0000","last_became_peered":"2024-09-21T03:37:11.033932+0000","last_unstale":"2024-09-21T03:38:38.106317+0000","last_undegraded":"2024-09-21T03:38:38.106317+0000","last_fullsized":"2024-09-21T03:38:38.106317+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_clean_scrub_stamp":"2024-09-21T03:37:09.809805+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:15:18.595442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":51,"num_read_kb":51,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.043133+0000","last_change":"2024-09-21T03:37:20.160789+0000","last_active":"2024-09-21T03:37:53.043133+0000","last_peered":"2024-09-21T03:37:53.043133+0000","last_clean":"2024-09-21T03:37:53.043133+0000","last_became_active":"2024-09-21T03:37:20.160231+0000","last_became_peered":"2024-09-21T03:37:20.160231+0000","last_unstale":"2024-09-21T03:37:53.043133+0000","last_undegraded":"2024-09-21T03:37:53.043133+0000","last_fullsized":"2024-09-21T03:37:53.043133+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_clean_scrub_stamp":"2024-09-21T03:37:18.876933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:07:16.762249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.042510+0000","last_change":"2024-09-21T03:37:12.853178+0000","last_active":"2024-09-21T03:37:53.042510+0000","last_peered":"2024-09-21T03:37:53.042510+0000","last_clean":"2024-09-21T03:37:53.042510+0000","last_became_active":"2024-09-21T03:37:12.852822+0000","last_became_peered":"2024-09-21T03:37:12.852822+0000","last_unstale":"2024-09-21T03:37:53.042510+0000","last_undegraded":"2024-09-21T03:37:53.042510+0000","last_fullsized":"2024-09-21T03:37:53.042510+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:48:26.146593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"80'3","reported_seq":38,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.430945+0000","last_change":"2024-09-21T03:37:10.838939+0000","last_active":"2024-09-21T03:37:53.430945+0000","last_peered":"2024-09-21T03:37:53.430945+0000","last_clean":"2024-09-21T03:37:53.430945+0000","last_became_active":"2024-09-21T03:37:10.838562+0000","last_became_peered":"2024-09-21T03:37:10.838562+0000","last_unstale":"2024-09-21T03:37:53.430945+0000","last_undegraded":"2024-09-21T03:37:53.430945+0000","last_fullsized":"2024-09-21T03:37:53.430945+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_clean_scrub_stamp":"2024-09-21T03:37:09.809805+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:35:32.936824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.053062+0000","last_change":"2024-09-21T03:37:53.052940+0000","last_active":"2024-09-21T03:37:53.053062+0000","last_peered":"2024-09-21T03:37:53.053062+0000","last_clean":"2024-09-21T03:37:53.053062+0000","last_became_active":"2024-09-21T03:37:53.052310+0000","last_became_peered":"2024-09-21T03:37:53.052310+0000","last_unstale":"2024-09-21T03:37:53.053062+0000","last_undegraded":"2024-09-21T03:37:53.053062+0000","last_fullsized":"2024-09-21T03:37:53.053062+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_clean_scrub_stamp":"2024-09-21T03:37:18.876933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:31:35.353226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,11,6,10],"acting":[5,11,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.046076+0000","last_change":"2024-09-21T03:37:12.869795+0000","last_active":"2024-09-21T03:37:53.046076+0000","last_peered":"2024-09-21T03:37:53.046076+0000","last_clean":"2024-09-21T03:37:53.046076+0000","last_became_active":"2024-09-21T03:37:12.869381+0000","last_became_peered":"2024-09-21T03:37:12.869381+0000","last_unstale":"2024-09-21T03:37:53.046076+0000","last_undegraded":"2024-09-21T03:37:53.046076+0000","last_fullsized":"2024-09-21T03:37:53.046076+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:39:29.437552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"80'3","reported_seq":40,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.044859+0000","last_change":"2024-09-21T03:37:10.834853+0000","last_active":"2024-09-21T03:37:53.044859+0000","last_peered":"2024-09-21T03:37:53.044859+0000","last_clean":"2024-09-21T03:37:53.044859+0000","last_became_active":"2024-09-21T03:37:10.834402+0000","last_became_peered":"2024-09-21T03:37:10.834402+0000","last_unstale":"2024-09-21T03:37:53.044859+0000","last_undegraded":"2024-09-21T03:37:53.044859+0000","last_fullsized":"2024-09-21T03:37:53.044859+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_clean_scrub_stamp":"2024-09-21T03:37:09.809805+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:57:29.668570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"20'92","reported_seq":108,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.430997+0000","last_change":"2024-09-21T03:36:00.521781+0000","last_active":"2024-09-21T03:37:53.430997+0000","last_peered":"2024-09-21T03:37:53.430997+0000","last_clean":"2024-09-21T03:37:53.430997+0000","last_became_active":"2024-09-21T03:36:00.516468+0000","last_became_peered":"2024-09-21T03:36:00.516468+0000","last_unstale":"2024-09-21T03:37:53.430997+0000","last_undegraded":"2024-09-21T03:37:53.430997+0000","last_fullsized":"2024-09-21T03:37:53.430997+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:32:38.215432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:32:38.215432+0000","last_clean_scrub_stamp":"2024-09-21T03:32:38.215432+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:02:16.252544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"82'1","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.431151+0000","last_change":"2024-09-21T03:37:19.911309+0000","last_active":"2024-09-21T03:37:53.431151+0000","last_peered":"2024-09-21T03:37:53.431151+0000","last_clean":"2024-09-21T03:37:53.431151+0000","last_became_active":"2024-09-21T03:37:19.910778+0000","last_became_peered":"2024-09-21T03:37:19.910778+0000","last_unstale":"2024-09-21T03:37:53.431151+0000","last_undegraded":"2024-09-21T03:37:53.431151+0000","last_fullsized":"2024-09-21T03:37:53.431151+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_clean_scrub_stamp":"2024-09-21T03:37:18.876933+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:52:23.307983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.431249+0000","last_change":"2024-09-21T03:37:12.860707+0000","last_active":"2024-09-21T03:37:53.431249+0000","last_peered":"2024-09-21T03:37:53.431249+0000","last_clean":"2024-09-21T03:37:53.431249+0000","last_became_active":"2024-09-21T03:37:12.860341+0000","last_became_peered":"2024-09-21T03:37:12.860341+0000","last_unstale":"2024-09-21T03:37:53.431249+0000","last_undegraded":"2024-09-21T03:37:53.431249+0000","last_fullsized":"2024-09-21T03:37:53.431249+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:55:15.771259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"80'3","reported_seq":18,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.071234+0000","last_change":"2024-09-21T03:37:53.071125+0000","last_active":"2024-09-21T03:37:53.071234+0000","last_peered":"2024-09-21T03:37:53.071234+0000","last_clean":"2024-09-21T03:37:53.071234+0000","last_became_active":"2024-09-21T03:37:53.052237+0000","last_became_peered":"2024-09-21T03:37:53.052237+0000","last_unstale":"2024-09-21T03:37:53.071234+0000","last_undegraded":"2024-09-21T03:37:53.071234+0000","last_fullsized":"2024-09-21T03:37:53.071234+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_clean_scrub_stamp":"2024-09-21T03:37:09.809805+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:47:17.707045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,0],"acting":[4,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.111355+0000","last_change":"2024-09-21T03:37:19.909292+0000","last_active":"2024-09-21T03:37:53.111355+0000","last_peered":"2024-09-21T03:37:53.111355+0000","last_clean":"2024-09-21T03:37:53.111355+0000","last_became_active":"2024-09-21T03:37:19.908681+0000","last_became_peered":"2024-09-21T03:37:19.908681+0000","last_unstale":"2024-09-21T03:37:53.111355+0000","last_undegraded":"2024-09-21T03:37:53.111355+0000","last_fullsized":"2024-09-21T03:37:53.111355+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_clean_scrub_stamp":"2024-09-21T03:37:18.876933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:15:49.623109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.431136+0000","last_change":"2024-09-21T03:37:12.857678+0000","last_active":"2024-09-21T03:37:53.431136+0000","last_peered":"2024-09-21T03:37:53.431136+0000","last_clean":"2024-09-21T03:37:53.431136+0000","last_became_active":"2024-09-21T03:37:12.857266+0000","last_became_peered":"2024-09-21T03:37:12.857266+0000","last_unstale":"2024-09-21T03:37:53.431136+0000","last_undegraded":"2024-09-21T03:37:53.431136+0000","last_fullsized":"2024-09-21T03:37:53.431136+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:21:14.135784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"74'1","reported_seq":37,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.047567+0000","last_change":"2024-09-21T03:37:10.835323+0000","last_active":"2024-09-21T03:37:53.047567+0000","last_peered":"2024-09-21T03:37:53.047567+0000","last_clean":"2024-09-21T03:37:53.047567+0000","last_became_active":"2024-09-21T03:37:10.834950+0000","last_became_peered":"2024-09-21T03:37:10.834950+0000","last_unstale":"2024-09-21T03:37:53.047567+0000","last_undegraded":"2024-09-21T03:37:53.047567+0000","last_fullsized":"2024-09-21T03:37:53.047567+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_clean_scrub_stamp":"2024-09-21T03:37:09.809805+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:40:27.386342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.049010+0000","last_change":"2024-09-21T03:37:19.906360+0000","last_active":"2024-09-21T03:37:53.049010+0000","last_peered":"2024-09-21T03:37:53.049010+0000","last_clean":"2024-09-21T03:37:53.049010+0000","last_became_active":"2024-09-21T03:37:19.905888+0000","last_became_peered":"2024-09-21T03:37:19.905888+0000","last_unstale":"2024-09-21T03:37:53.049010+0000","last_undegraded":"2024-09-21T03:37:53.049010+0000","last_fullsized":"2024-09-21T03:37:53.049010+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_clean_scrub_stamp":"2024-09-21T03:37:18.876933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:13:55.885538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.431442+0000","last_change":"2024-09-21T03:37:12.849106+0000","last_active":"2024-09-21T03:37:53.431442+0000","last_peered":"2024-09-21T03:37:53.431442+0000","last_clean":"2024-09-21T03:37:53.431442+0000","last_became_active":"2024-09-21T03:37:12.848505+0000","last_became_peered":"2024-09-21T03:37:12.848505+0000","last_unstale":"2024-09-21T03:37:53.431442+0000","last_undegraded":"2024-09-21T03:37:53.431442+0000","last_fullsized":"2024-09-21T03:37:53.431442+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:50:19.569004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"74'2","reported_seq":48,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.115238+0000","last_change":"2024-09-21T03:37:53.115158+0000","last_active":"2024-09-21T03:37:53.115238+0000","last_peered":"2024-09-21T03:37:53.115238+0000","last_clean":"2024-09-21T03:37:53.115238+0000","last_became_active":"2024-09-21T03:37:53.112603+0000","last_became_peered":"2024-09-21T03:37:53.112603+0000","last_unstale":"2024-09-21T03:37:53.115238+0000","last_undegraded":"2024-09-21T03:37:53.115238+0000","last_fullsized":"2024-09-21T03:37:53.115238+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_clean_scrub_stamp":"2024-09-21T03:37:09.809805+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:23:21.525972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,1],"acting":[6,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.045614+0000","last_change":"2024-09-21T03:37:19.909040+0000","last_active":"2024-09-21T03:37:53.045614+0000","last_peered":"2024-09-21T03:37:53.045614+0000","last_clean":"2024-09-21T03:37:53.045614+0000","last_became_active":"2024-09-21T03:37:19.908530+0000","last_became_peered":"2024-09-21T03:37:19.908530+0000","last_unstale":"2024-09-21T03:37:53.045614+0000","last_undegraded":"2024-09-21T03:37:53.045614+0000","last_fullsized":"2024-09-21T03:37:53.045614+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_clean_scrub_stamp":"2024-09-21T03:37:18.876933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:35:20.919260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.049078+0000","last_change":"2024-09-21T03:37:12.858984+0000","last_active":"2024-09-21T03:37:53.049078+0000","last_peered":"2024-09-21T03:37:53.049078+0000","last_clean":"2024-09-21T03:37:53.049078+0000","last_became_active":"2024-09-21T03:37:12.858603+0000","last_became_peered":"2024-09-21T03:37:12.858603+0000","last_unstale":"2024-09-21T03:37:53.049078+0000","last_undegraded":"2024-09-21T03:37:53.049078+0000","last_fullsized":"2024-09-21T03:37:53.049078+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:16:45.025304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.f","version":"80'2","reported_seq":38,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.050780+0000","last_change":"2024-09-21T03:37:10.834844+0000","last_active":"2024-09-21T03:37:53.050780+0000","last_peered":"2024-09-21T03:37:53.050780+0000","last_clean":"2024-09-21T03:37:53.050780+0000","last_became_active":"2024-09-21T03:37:10.833664+0000","last_became_peered":"2024-09-21T03:37:10.833664+0000","last_unstale":"2024-09-21T03:37:53.050780+0000","last_undegraded":"2024-09-21T03:37:53.050780+0000","last_fullsized":"2024-09-21T03:37:53.050780+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_clean_scrub_stamp":"2024-09-21T03:37:09.809805+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:03:49.006414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.046180+0000","last_change":"2024-09-21T03:37:19.913277+0000","last_active":"2024-09-21T03:37:53.046180+0000","last_peered":"2024-09-21T03:37:53.046180+0000","last_clean":"2024-09-21T03:37:53.046180+0000","last_became_active":"2024-09-21T03:37:19.912781+0000","last_became_peered":"2024-09-21T03:37:19.912781+0000","last_unstale":"2024-09-21T03:37:53.046180+0000","last_undegraded":"2024-09-21T03:37:53.046180+0000","last_fullsized":"2024-09-21T03:37:53.046180+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_clean_scrub_stamp":"2024-09-21T03:37:18.876933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:18:27.980622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.046359+0000","last_change":"2024-09-21T03:37:12.862322+0000","last_active":"2024-09-21T03:37:53.046359+0000","last_peered":"2024-09-21T03:37:53.046359+0000","last_clean":"2024-09-21T03:37:53.046359+0000","last_became_active":"2024-09-21T03:37:12.861938+0000","last_became_peered":"2024-09-21T03:37:12.861938+0000","last_unstale":"2024-09-21T03:37:53.046359+0000","last_undegraded":"2024-09-21T03:37:53.046359+0000","last_fullsized":"2024-09-21T03:37:53.046359+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:01:48.944169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.10","version":"80'15","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.067663+0000","last_change":"2024-09-21T03:37:53.067130+0000","last_active":"2024-09-21T03:37:53.067663+0000","last_peered":"2024-09-21T03:37:53.067663+0000","last_clean":"2024-09-21T03:37:53.067663+0000","last_became_active":"2024-09-21T03:37:53.051596+0000","last_became_peered":"2024-09-21T03:37:53.051596+0000","last_unstale":"2024-09-21T03:37:53.067663+0000","last_undegraded":"2024-09-21T03:37:53.067663+0000","last_fullsized":"2024-09-21T03:37:53.067663+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_clean_scrub_stamp":"2024-09-21T03:37:09.809805+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:02:22.852015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.043129+0000","last_change":"2024-09-21T03:37:19.913206+0000","last_active":"2024-09-21T03:37:53.043129+0000","last_peered":"2024-09-21T03:37:53.043129+0000","last_clean":"2024-09-21T03:37:53.043129+0000","last_became_active":"2024-09-21T03:37:19.910718+0000","last_became_peered":"2024-09-21T03:37:19.910718+0000","last_unstale":"2024-09-21T03:37:53.043129+0000","last_undegraded":"2024-09-21T03:37:53.043129+0000","last_fullsized":"2024-09-21T03:37:53.043129+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_clean_scrub_stamp":"2024-09-21T03:37:18.876933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:11:31.431237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.430692+0000","last_change":"2024-09-21T03:37:12.857505+0000","last_active":"2024-09-21T03:37:53.430692+0000","last_peered":"2024-09-21T03:37:53.430692+0000","last_clean":"2024-09-21T03:37:53.430692+0000","last_became_active":"2024-09-21T03:37:12.857155+0000","last_became_peered":"2024-09-21T03:37:12.857155+0000","last_unstale":"2024-09-21T03:37:53.430692+0000","last_undegraded":"2024-09-21T03:37:53.430692+0000","last_fullsized":"2024-09-21T03:37:53.430692+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:05:50.387651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"80'3","reported_seq":94,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:38:41.111964+0000","last_change":"2024-09-21T03:37:10.832364+0000","last_active":"2024-09-21T03:38:41.111964+0000","last_peered":"2024-09-21T03:38:41.111964+0000","last_clean":"2024-09-21T03:38:41.111964+0000","last_became_active":"2024-09-21T03:37:10.832004+0000","last_became_peered":"2024-09-21T03:37:10.832004+0000","last_unstale":"2024-09-21T03:38:41.111964+0000","last_undegraded":"2024-09-21T03:38:41.111964+0000","last_fullsized":"2024-09-21T03:38:41.111964+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_clean_scrub_stamp":"2024-09-21T03:37:09.809805+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:21:10.895732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":57,"num_read_kb":3,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.047568+0000","last_change":"2024-09-21T03:37:19.912297+0000","last_active":"2024-09-21T03:37:53.047568+0000","last_peered":"2024-09-21T03:37:53.047568+0000","last_clean":"2024-09-21T03:37:53.047568+0000","last_became_active":"2024-09-21T03:37:19.911903+0000","last_became_peered":"2024-09-21T03:37:19.911903+0000","last_unstale":"2024-09-21T03:37:53.047568+0000","last_undegraded":"2024-09-21T03:37:53.047568+0000","last_fullsized":"2024-09-21T03:37:53.047568+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_clean_scrub_stamp":"2024-09-21T03:37:18.876933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:05:56.747246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.431059+0000","last_change":"2024-09-21T03:37:12.856230+0000","last_active":"2024-09-21T03:37:53.431059+0000","last_peered":"2024-09-21T03:37:53.431059+0000","last_clean":"2024-09-21T03:37:53.431059+0000","last_became_active":"2024-09-21T03:37:12.855819+0000","last_became_peered":"2024-09-21T03:37:12.855819+0000","last_unstale":"2024-09-21T03:37:53.431059+0000","last_undegraded":"2024-09-21T03:37:53.431059+0000","last_fullsized":"2024-09-21T03:37:53.431059+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:38:18.618579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"80'3","reported_seq":38,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.046856+0000","last_change":"2024-09-21T03:37:10.832401+0000","last_active":"2024-09-21T03:37:53.046856+0000","last_peered":"2024-09-21T03:37:53.046856+0000","last_clean":"2024-09-21T03:37:53.046856+0000","last_became_active":"2024-09-21T03:37:10.832033+0000","last_became_peered":"2024-09-21T03:37:10.832033+0000","last_unstale":"2024-09-21T03:37:53.046856+0000","last_undegraded":"2024-09-21T03:37:53.046856+0000","last_fullsized":"2024-09-21T03:37:53.046856+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_clean_scrub_stamp":"2024-09-21T03:37:09.809805+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:25:20.751529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.048479+0000","last_change":"2024-09-21T03:37:19.905825+0000","last_active":"2024-09-21T03:37:53.048479+0000","last_peered":"2024-09-21T03:37:53.048479+0000","last_clean":"2024-09-21T03:37:53.048479+0000","last_became_active":"2024-09-21T03:37:19.905352+0000","last_became_peered":"2024-09-21T03:37:19.905352+0000","last_unstale":"2024-09-21T03:37:53.048479+0000","last_undegraded":"2024-09-21T03:37:53.048479+0000","last_fullsized":"2024-09-21T03:37:53.048479+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_clean_scrub_stamp":"2024-09-21T03:37:18.876933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:25:05.861981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.042684+0000","last_change":"2024-09-21T03:37:12.852006+0000","last_active":"2024-09-21T03:37:53.042684+0000","last_peered":"2024-09-21T03:37:53.042684+0000","last_clean":"2024-09-21T03:37:53.042684+0000","last_became_active":"2024-09-21T03:37:12.851670+0000","last_became_peered":"2024-09-21T03:37:12.851670+0000","last_unstale":"2024-09-21T03:37:53.042684+0000","last_undegraded":"2024-09-21T03:37:53.042684+0000","last_fullsized":"2024-09-21T03:37:53.042684+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:41:50.177349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"80'6","reported_seq":42,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.044608+0000","last_change":"2024-09-21T03:37:10.832472+0000","last_active":"2024-09-21T03:37:53.044608+0000","last_peered":"2024-09-21T03:37:53.044608+0000","last_clean":"2024-09-21T03:37:53.044608+0000","last_became_active":"2024-09-21T03:37:10.832085+0000","last_became_peered":"2024-09-21T03:37:10.832085+0000","last_unstale":"2024-09-21T03:37:53.044608+0000","last_undegraded":"2024-09-21T03:37:53.044608+0000","last_fullsized":"2024-09-21T03:37:53.044608+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_clean_scrub_stamp":"2024-09-21T03:37:09.809805+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:06:41.251218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.046632+0000","last_change":"2024-09-21T03:37:19.914042+0000","last_active":"2024-09-21T03:37:53.046632+0000","last_peered":"2024-09-21T03:37:53.046632+0000","last_clean":"2024-09-21T03:37:53.046632+0000","last_became_active":"2024-09-21T03:37:19.913531+0000","last_became_peered":"2024-09-21T03:37:19.913531+0000","last_unstale":"2024-09-21T03:37:53.046632+0000","last_undegraded":"2024-09-21T03:37:53.046632+0000","last_fullsized":"2024-09-21T03:37:53.046632+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_clean_scrub_stamp":"2024-09-21T03:37:18.876933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:16:40.163359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.050691+0000","last_change":"2024-09-21T03:37:12.852893+0000","last_active":"2024-09-21T03:37:53.050691+0000","last_peered":"2024-09-21T03:37:53.050691+0000","last_clean":"2024-09-21T03:37:53.050691+0000","last_became_active":"2024-09-21T03:37:12.852452+0000","last_became_peered":"2024-09-21T03:37:12.852452+0000","last_unstale":"2024-09-21T03:37:53.050691+0000","last_undegraded":"2024-09-21T03:37:53.050691+0000","last_fullsized":"2024-09-21T03:37:53.050691+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:46:59.393901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.14","version":"82'32","reported_seq":178,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:38:39.108865+0000","last_change":"2024-09-21T03:37:10.837654+0000","last_active":"2024-09-21T03:38:39.108865+0000","last_peered":"2024-09-21T03:38:39.108865+0000","last_clean":"2024-09-21T03:38:39.108865+0000","last_became_active":"2024-09-21T03:37:10.837251+0000","last_became_peered":"2024-09-21T03:37:10.837251+0000","last_unstale":"2024-09-21T03:38:39.108865+0000","last_undegraded":"2024-09-21T03:38:39.108865+0000","last_fullsized":"2024-09-21T03:38:39.108865+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_clean_scrub_stamp":"2024-09-21T03:37:09.809805+0000","objects_scrubbed":0,"log_size":32,"log_dups_size":0,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:43:32.986371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":67138,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":111,"num_read_kb":121,"num_write":33,"num_write_kb":80,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.053980+0000","last_change":"2024-09-21T03:37:53.053405+0000","last_active":"2024-09-21T03:37:53.053980+0000","last_peered":"2024-09-21T03:37:53.053980+0000","last_clean":"2024-09-21T03:37:53.053980+0000","last_became_active":"2024-09-21T03:37:53.052818+0000","last_became_peered":"2024-09-21T03:37:53.052818+0000","last_unstale":"2024-09-21T03:37:53.053980+0000","last_undegraded":"2024-09-21T03:37:53.053980+0000","last_fullsized":"2024-09-21T03:37:53.053980+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_clean_scrub_stamp":"2024-09-21T03:37:18.876933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:05:09.121036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,11,5],"acting":[10,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.111585+0000","last_change":"2024-09-21T03:37:12.866588+0000","last_active":"2024-09-21T03:37:53.111585+0000","last_peered":"2024-09-21T03:37:53.111585+0000","last_clean":"2024-09-21T03:37:53.111585+0000","last_became_active":"2024-09-21T03:37:12.866075+0000","last_became_peered":"2024-09-21T03:37:12.866075+0000","last_unstale":"2024-09-21T03:37:53.111585+0000","last_undegraded":"2024-09-21T03:37:53.111585+0000","last_fullsized":"2024-09-21T03:37:53.111585+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:18:48.817562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"80'4","reported_seq":40,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.047298+0000","last_change":"2024-09-21T03:37:11.036751+0000","last_active":"2024-09-21T03:37:53.047298+0000","last_peered":"2024-09-21T03:37:53.047298+0000","last_clean":"2024-09-21T03:37:53.047298+0000","last_became_active":"2024-09-21T03:37:11.036125+0000","last_became_peered":"2024-09-21T03:37:11.036125+0000","last_unstale":"2024-09-21T03:37:53.047298+0000","last_undegraded":"2024-09-21T03:37:53.047298+0000","last_fullsized":"2024-09-21T03:37:53.047298+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_clean_scrub_stamp":"2024-09-21T03:37:09.809805+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:46:13.569422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.112980+0000","last_change":"2024-09-21T03:37:53.112845+0000","last_active":"2024-09-21T03:37:53.112980+0000","last_peered":"2024-09-21T03:37:53.112980+0000","last_clean":"2024-09-21T03:37:53.112980+0000","last_became_active":"2024-09-21T03:37:53.112286+0000","last_became_peered":"2024-09-21T03:37:53.112286+0000","last_unstale":"2024-09-21T03:37:53.112980+0000","last_undegraded":"2024-09-21T03:37:53.112980+0000","last_fullsized":"2024-09-21T03:37:53.112980+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_clean_scrub_stamp":"2024-09-21T03:37:18.876933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:29:58.062684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,8],"acting":[4,1,2,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.049603+0000","last_change":"2024-09-21T03:37:12.865011+0000","last_active":"2024-09-21T03:37:53.049603+0000","last_peered":"2024-09-21T03:37:53.049603+0000","last_clean":"2024-09-21T03:37:53.049603+0000","last_became_active":"2024-09-21T03:37:12.864645+0000","last_became_peered":"2024-09-21T03:37:12.864645+0000","last_unstale":"2024-09-21T03:37:53.049603+0000","last_undegraded":"2024-09-21T03:37:53.049603+0000","last_fullsized":"2024-09-21T03:37:53.049603+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:27:42.471227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"80'2","reported_seq":37,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.431360+0000","last_change":"2024-09-21T03:37:10.835171+0000","last_active":"2024-09-21T03:37:53.431360+0000","last_peered":"2024-09-21T03:37:53.431360+0000","last_clean":"2024-09-21T03:37:53.431360+0000","last_became_active":"2024-09-21T03:37:10.834744+0000","last_became_peered":"2024-09-21T03:37:10.834744+0000","last_unstale":"2024-09-21T03:37:53.431360+0000","last_undegraded":"2024-09-21T03:37:53.431360+0000","last_fullsized":"2024-09-21T03:37:53.431360+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_clean_scrub_stamp":"2024-09-21T03:37:09.809805+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:02:37.720457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.046301+0000","last_change":"2024-09-21T03:37:19.906734+0000","last_active":"2024-09-21T03:37:53.046301+0000","last_peered":"2024-09-21T03:37:53.046301+0000","last_clean":"2024-09-21T03:37:53.046301+0000","last_became_active":"2024-09-21T03:37:19.906327+0000","last_became_peered":"2024-09-21T03:37:19.906327+0000","last_unstale":"2024-09-21T03:37:53.046301+0000","last_undegraded":"2024-09-21T03:37:53.046301+0000","last_fullsized":"2024-09-21T03:37:53.046301+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_clean_scrub_stamp":"2024-09-21T03:37:18.876933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:55:11.603042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.111948+0000","last_change":"2024-09-21T03:37:12.866285+0000","last_active":"2024-09-21T03:37:53.111948+0000","last_peered":"2024-09-21T03:37:53.111948+0000","last_clean":"2024-09-21T03:37:53.111948+0000","last_became_active":"2024-09-21T03:37:12.865854+0000","last_became_peered":"2024-09-21T03:37:12.865854+0000","last_unstale":"2024-09-21T03:37:53.111948+0000","last_undegraded":"2024-09-21T03:37:53.111948+0000","last_fullsized":"2024-09-21T03:37:53.111948+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:00:18.075248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"80'3","reported_seq":89,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:38:38.107167+0000","last_change":"2024-09-21T03:37:10.835808+0000","last_active":"2024-09-21T03:38:38.107167+0000","last_peered":"2024-09-21T03:38:38.107167+0000","last_clean":"2024-09-21T03:38:38.107167+0000","last_became_active":"2024-09-21T03:37:10.835434+0000","last_became_peered":"2024-09-21T03:37:10.835434+0000","last_unstale":"2024-09-21T03:38:38.107167+0000","last_undegraded":"2024-09-21T03:38:38.107167+0000","last_fullsized":"2024-09-21T03:38:38.107167+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_clean_scrub_stamp":"2024-09-21T03:37:09.809805+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:35:39.817312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":51,"num_read_kb":51,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.045411+0000","last_change":"2024-09-21T03:37:19.915986+0000","last_active":"2024-09-21T03:37:53.045411+0000","last_peered":"2024-09-21T03:37:53.045411+0000","last_clean":"2024-09-21T03:37:53.045411+0000","last_became_active":"2024-09-21T03:37:19.915561+0000","last_became_peered":"2024-09-21T03:37:19.915561+0000","last_unstale":"2024-09-21T03:37:53.045411+0000","last_undegraded":"2024-09-21T03:37:53.045411+0000","last_fullsized":"2024-09-21T03:37:53.045411+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_clean_scrub_stamp":"2024-09-21T03:37:18.876933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:21:13.789664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.049328+0000","last_change":"2024-09-21T03:37:19.908230+0000","last_active":"2024-09-21T03:37:53.049328+0000","last_peered":"2024-09-21T03:37:53.049328+0000","last_clean":"2024-09-21T03:37:53.049328+0000","last_became_active":"2024-09-21T03:37:19.907843+0000","last_became_peered":"2024-09-21T03:37:19.907843+0000","last_unstale":"2024-09-21T03:37:53.049328+0000","last_undegraded":"2024-09-21T03:37:53.049328+0000","last_fullsized":"2024-09-21T03:37:53.049328+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_clean_scrub_stamp":"2024-09-21T03:37:18.876933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:15:58.142127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.046673+0000","last_change":"2024-09-21T03:37:12.856709+0000","last_active":"2024-09-21T03:37:53.046673+0000","last_peered":"2024-09-21T03:37:53.046673+0000","last_clean":"2024-09-21T03:37:53.046673+0000","last_became_active":"2024-09-21T03:37:12.856278+0000","last_became_peered":"2024-09-21T03:37:12.856278+0000","last_unstale":"2024-09-21T03:37:53.046673+0000","last_undegraded":"2024-09-21T03:37:53.046673+0000","last_fullsized":"2024-09-21T03:37:53.046673+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:16:13.884648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"80'1","reported_seq":36,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.046329+0000","last_change":"2024-09-21T03:37:10.832456+0000","last_active":"2024-09-21T03:37:53.046329+0000","last_peered":"2024-09-21T03:37:53.046329+0000","last_clean":"2024-09-21T03:37:53.046329+0000","last_became_active":"2024-09-21T03:37:10.832082+0000","last_became_peered":"2024-09-21T03:37:10.832082+0000","last_unstale":"2024-09-21T03:37:53.046329+0000","last_undegraded":"2024-09-21T03:37:53.046329+0000","last_fullsized":"2024-09-21T03:37:53.046329+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_clean_scrub_stamp":"2024-09-21T03:37:09.809805+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:17:12.170616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.048472+0000","last_change":"2024-09-21T03:37:19.908084+0000","last_active":"2024-09-21T03:37:53.048472+0000","last_peered":"2024-09-21T03:37:53.048472+0000","last_clean":"2024-09-21T03:37:53.048472+0000","last_became_active":"2024-09-21T03:37:19.907307+0000","last_became_peered":"2024-09-21T03:37:19.907307+0000","last_unstale":"2024-09-21T03:37:53.048472+0000","last_undegraded":"2024-09-21T03:37:53.048472+0000","last_fullsized":"2024-09-21T03:37:53.048472+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_clean_scrub_stamp":"2024-09-21T03:37:18.876933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:45:02.541015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.045568+0000","last_change":"2024-09-21T03:37:12.852577+0000","last_active":"2024-09-21T03:37:53.045568+0000","last_peered":"2024-09-21T03:37:53.045568+0000","last_clean":"2024-09-21T03:37:53.045568+0000","last_became_active":"2024-09-21T03:37:12.852186+0000","last_became_peered":"2024-09-21T03:37:12.852186+0000","last_unstale":"2024-09-21T03:37:53.045568+0000","last_undegraded":"2024-09-21T03:37:53.045568+0000","last_fullsized":"2024-09-21T03:37:53.045568+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:51:21.631029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"80'2","reported_seq":37,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.042546+0000","last_change":"2024-09-21T03:37:10.833587+0000","last_active":"2024-09-21T03:37:53.042546+0000","last_peered":"2024-09-21T03:37:53.042546+0000","last_clean":"2024-09-21T03:37:53.042546+0000","last_became_active":"2024-09-21T03:37:10.833194+0000","last_became_peered":"2024-09-21T03:37:10.833194+0000","last_unstale":"2024-09-21T03:37:53.042546+0000","last_undegraded":"2024-09-21T03:37:53.042546+0000","last_fullsized":"2024-09-21T03:37:53.042546+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_clean_scrub_stamp":"2024-09-21T03:37:09.809805+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:20:20.272108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"80'3","reported_seq":38,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.047496+0000","last_change":"2024-09-21T03:37:10.839103+0000","last_active":"2024-09-21T03:37:53.047496+0000","last_peered":"2024-09-21T03:37:53.047496+0000","last_clean":"2024-09-21T03:37:53.047496+0000","last_became_active":"2024-09-21T03:37:10.837192+0000","last_became_peered":"2024-09-21T03:37:10.837192+0000","last_unstale":"2024-09-21T03:37:53.047496+0000","last_undegraded":"2024-09-21T03:37:53.047496+0000","last_fullsized":"2024-09-21T03:37:53.047496+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_clean_scrub_stamp":"2024-09-21T03:37:09.809805+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:59:35.128171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.047232+0000","last_change":"2024-09-21T03:37:12.866626+0000","last_active":"2024-09-21T03:37:53.047232+0000","last_peered":"2024-09-21T03:37:53.047232+0000","last_clean":"2024-09-21T03:37:53.047232+0000","last_became_active":"2024-09-21T03:37:12.866254+0000","last_became_peered":"2024-09-21T03:37:12.866254+0000","last_unstale":"2024-09-21T03:37:53.047232+0000","last_undegraded":"2024-09-21T03:37:53.047232+0000","last_fullsized":"2024-09-21T03:37:53.047232+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:39:00.361254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.044954+0000","last_change":"2024-09-21T03:37:19.909415+0000","last_active":"2024-09-21T03:37:53.044954+0000","last_peered":"2024-09-21T03:37:53.044954+0000","last_clean":"2024-09-21T03:37:53.044954+0000","last_became_active":"2024-09-21T03:37:19.908955+0000","last_became_peered":"2024-09-21T03:37:19.908955+0000","last_unstale":"2024-09-21T03:37:53.044954+0000","last_undegraded":"2024-09-21T03:37:53.044954+0000","last_fullsized":"2024-09-21T03:37:53.044954+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_clean_scrub_stamp":"2024-09-21T03:37:18.876933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:31:18.878725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"81'6","reported_seq":97,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:38:40.110836+0000","last_change":"2024-09-21T03:37:10.836389+0000","last_active":"2024-09-21T03:38:40.110836+0000","last_peered":"2024-09-21T03:38:40.110836+0000","last_clean":"2024-09-21T03:38:40.110836+0000","last_became_active":"2024-09-21T03:37:10.835986+0000","last_became_peered":"2024-09-21T03:37:10.835986+0000","last_unstale":"2024-09-21T03:38:40.110836+0000","last_undegraded":"2024-09-21T03:38:40.110836+0000","last_fullsized":"2024-09-21T03:38:40.110836+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_clean_scrub_stamp":"2024-09-21T03:37:09.809805+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:05:17.963609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1013,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":56,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.050861+0000","last_change":"2024-09-21T03:37:12.856008+0000","last_active":"2024-09-21T03:37:53.050861+0000","last_peered":"2024-09-21T03:37:53.050861+0000","last_clean":"2024-09-21T03:37:53.050861+0000","last_became_active":"2024-09-21T03:37:12.855603+0000","last_became_peered":"2024-09-21T03:37:12.855603+0000","last_unstale":"2024-09-21T03:37:53.050861+0000","last_undegraded":"2024-09-21T03:37:53.050861+0000","last_fullsized":"2024-09-21T03:37:53.050861+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:11:41.920114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.045051+0000","last_change":"2024-09-21T03:37:19.911507+0000","last_active":"2024-09-21T03:37:53.045051+0000","last_peered":"2024-09-21T03:37:53.045051+0000","last_clean":"2024-09-21T03:37:53.045051+0000","last_became_active":"2024-09-21T03:37:19.911053+0000","last_became_peered":"2024-09-21T03:37:19.911053+0000","last_unstale":"2024-09-21T03:37:53.045051+0000","last_undegraded":"2024-09-21T03:37:53.045051+0000","last_fullsized":"2024-09-21T03:37:53.045051+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_clean_scrub_stamp":"2024-09-21T03:37:18.876933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:42:11.964829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.048840+0000","last_change":"2024-09-21T03:37:12.862576+0000","last_active":"2024-09-21T03:37:53.048840+0000","last_peered":"2024-09-21T03:37:53.048840+0000","last_clean":"2024-09-21T03:37:53.048840+0000","last_became_active":"2024-09-21T03:37:12.862146+0000","last_became_peered":"2024-09-21T03:37:12.862146+0000","last_unstale":"2024-09-21T03:37:53.048840+0000","last_undegraded":"2024-09-21T03:37:53.048840+0000","last_fullsized":"2024-09-21T03:37:53.048840+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:30:33.272415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.044731+0000","last_change":"2024-09-21T03:37:12.859395+0000","last_active":"2024-09-21T03:37:53.044731+0000","last_peered":"2024-09-21T03:37:53.044731+0000","last_clean":"2024-09-21T03:37:53.044731+0000","last_became_active":"2024-09-21T03:37:12.858914+0000","last_became_peered":"2024-09-21T03:37:12.858914+0000","last_unstale":"2024-09-21T03:37:53.044731+0000","last_undegraded":"2024-09-21T03:37:53.044731+0000","last_fullsized":"2024-09-21T03:37:53.044731+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:13:06.319896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.044203+0000","last_change":"2024-09-21T03:37:12.857467+0000","last_active":"2024-09-21T03:37:53.044203+0000","last_peered":"2024-09-21T03:37:53.044203+0000","last_clean":"2024-09-21T03:37:53.044203+0000","last_became_active":"2024-09-21T03:37:12.856214+0000","last_became_peered":"2024-09-21T03:37:12.856214+0000","last_unstale":"2024-09-21T03:37:53.044203+0000","last_undegraded":"2024-09-21T03:37:53.044203+0000","last_fullsized":"2024-09-21T03:37:53.044203+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:21:58.780018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.049029+0000","last_change":"2024-09-21T03:37:12.858052+0000","last_active":"2024-09-21T03:37:53.049029+0000","last_peered":"2024-09-21T03:37:53.049029+0000","last_clean":"2024-09-21T03:37:53.049029+0000","last_became_active":"2024-09-21T03:37:12.857445+0000","last_became_peered":"2024-09-21T03:37:12.857445+0000","last_unstale":"2024-09-21T03:37:53.049029+0000","last_undegraded":"2024-09-21T03:37:53.049029+0000","last_fullsized":"2024-09-21T03:37:53.049029+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:22:28.058846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.042880+0000","last_change":"2024-09-21T03:37:12.856499+0000","last_active":"2024-09-21T03:37:53.042880+0000","last_peered":"2024-09-21T03:37:53.042880+0000","last_clean":"2024-09-21T03:37:53.042880+0000","last_became_active":"2024-09-21T03:37:12.856146+0000","last_became_peered":"2024-09-21T03:37:12.856146+0000","last_unstale":"2024-09-21T03:37:53.042880+0000","last_undegraded":"2024-09-21T03:37:53.042880+0000","last_fullsized":"2024-09-21T03:37:53.042880+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:03:26.142740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.048901+0000","last_change":"2024-09-21T03:37:12.860830+0000","last_active":"2024-09-21T03:37:53.048901+0000","last_peered":"2024-09-21T03:37:53.048901+0000","last_clean":"2024-09-21T03:37:53.048901+0000","last_became_active":"2024-09-21T03:37:12.860468+0000","last_became_peered":"2024-09-21T03:37:12.860468+0000","last_unstale":"2024-09-21T03:37:53.048901+0000","last_undegraded":"2024-09-21T03:37:53.048901+0000","last_fullsized":"2024-09-21T03:37:53.048901+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:48:15.057622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.046546+0000","last_change":"2024-09-21T03:37:12.864399+0000","last_active":"2024-09-21T03:37:53.046546+0000","last_peered":"2024-09-21T03:37:53.046546+0000","last_clean":"2024-09-21T03:37:53.046546+0000","last_became_active":"2024-09-21T03:37:12.863892+0000","last_became_peered":"2024-09-21T03:37:12.863892+0000","last_unstale":"2024-09-21T03:37:53.046546+0000","last_undegraded":"2024-09-21T03:37:53.046546+0000","last_fullsized":"2024-09-21T03:37:53.046546+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:58:26.647896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.046313+0000","last_change":"2024-09-21T03:37:12.861716+0000","last_active":"2024-09-21T03:37:53.046313+0000","last_peered":"2024-09-21T03:37:53.046313+0000","last_clean":"2024-09-21T03:37:53.046313+0000","last_became_active":"2024-09-21T03:37:12.861392+0000","last_became_peered":"2024-09-21T03:37:12.861392+0000","last_unstale":"2024-09-21T03:37:53.046313+0000","last_undegraded":"2024-09-21T03:37:53.046313+0000","last_fullsized":"2024-09-21T03:37:53.046313+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:28:55.751107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.045975+0000","last_change":"2024-09-21T03:37:12.856167+0000","last_active":"2024-09-21T03:37:53.045975+0000","last_peered":"2024-09-21T03:37:53.045975+0000","last_clean":"2024-09-21T03:37:53.045975+0000","last_became_active":"2024-09-21T03:37:12.855752+0000","last_became_peered":"2024-09-21T03:37:12.855752+0000","last_unstale":"2024-09-21T03:37:53.045975+0000","last_undegraded":"2024-09-21T03:37:53.045975+0000","last_fullsized":"2024-09-21T03:37:53.045975+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:26:30.271342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.045615+0000","last_change":"2024-09-21T03:37:12.862782+0000","last_active":"2024-09-21T03:37:53.045615+0000","last_peered":"2024-09-21T03:37:53.045615+0000","last_clean":"2024-09-21T03:37:53.045615+0000","last_became_active":"2024-09-21T03:37:12.862462+0000","last_became_peered":"2024-09-21T03:37:12.862462+0000","last_unstale":"2024-09-21T03:37:53.045615+0000","last_undegraded":"2024-09-21T03:37:53.045615+0000","last_fullsized":"2024-09-21T03:37:53.045615+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:57:37.744531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.111623+0000","last_change":"2024-09-21T03:37:12.862312+0000","last_active":"2024-09-21T03:37:53.111623+0000","last_peered":"2024-09-21T03:37:53.111623+0000","last_clean":"2024-09-21T03:37:53.111623+0000","last_became_active":"2024-09-21T03:37:12.861940+0000","last_became_peered":"2024-09-21T03:37:12.861940+0000","last_unstale":"2024-09-21T03:37:53.111623+0000","last_undegraded":"2024-09-21T03:37:53.111623+0000","last_fullsized":"2024-09-21T03:37:53.111623+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:08:49.165522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.042830+0000","last_change":"2024-09-21T03:37:12.857012+0000","last_active":"2024-09-21T03:37:53.042830+0000","last_peered":"2024-09-21T03:37:53.042830+0000","last_clean":"2024-09-21T03:37:53.042830+0000","last_became_active":"2024-09-21T03:37:12.856390+0000","last_became_peered":"2024-09-21T03:37:12.856390+0000","last_unstale":"2024-09-21T03:37:53.042830+0000","last_undegraded":"2024-09-21T03:37:53.042830+0000","last_fullsized":"2024-09-21T03:37:53.042830+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:51:00.839709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.048487+0000","last_change":"2024-09-21T03:37:12.857207+0000","last_active":"2024-09-21T03:37:53.048487+0000","last_peered":"2024-09-21T03:37:53.048487+0000","last_clean":"2024-09-21T03:37:53.048487+0000","last_became_active":"2024-09-21T03:37:12.856770+0000","last_became_peered":"2024-09-21T03:37:12.856770+0000","last_unstale":"2024-09-21T03:37:53.048487+0000","last_undegraded":"2024-09-21T03:37:53.048487+0000","last_fullsized":"2024-09-21T03:37:53.048487+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:37:18.272321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.045688+0000","last_change":"2024-09-21T03:37:12.858540+0000","last_active":"2024-09-21T03:37:53.045688+0000","last_peered":"2024-09-21T03:37:53.045688+0000","last_clean":"2024-09-21T03:37:53.045688+0000","last_became_active":"2024-09-21T03:37:12.857340+0000","last_became_peered":"2024-09-21T03:37:12.857340+0000","last_unstale":"2024-09-21T03:37:53.045688+0000","last_undegraded":"2024-09-21T03:37:53.045688+0000","last_fullsized":"2024-09-21T03:37:53.045688+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:45:48.982132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.047908+0000","last_change":"2024-09-21T03:37:12.864408+0000","last_active":"2024-09-21T03:37:53.047908+0000","last_peered":"2024-09-21T03:37:53.047908+0000","last_clean":"2024-09-21T03:37:53.047908+0000","last_became_active":"2024-09-21T03:37:12.864082+0000","last_became_peered":"2024-09-21T03:37:12.864082+0000","last_unstale":"2024-09-21T03:37:53.047908+0000","last_undegraded":"2024-09-21T03:37:53.047908+0000","last_fullsized":"2024-09-21T03:37:53.047908+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:19:32.723589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.049781+0000","last_change":"2024-09-21T03:37:12.869445+0000","last_active":"2024-09-21T03:37:53.049781+0000","last_peered":"2024-09-21T03:37:53.049781+0000","last_clean":"2024-09-21T03:37:53.049781+0000","last_became_active":"2024-09-21T03:37:12.869058+0000","last_became_peered":"2024-09-21T03:37:12.869058+0000","last_unstale":"2024-09-21T03:37:53.049781+0000","last_undegraded":"2024-09-21T03:37:53.049781+0000","last_fullsized":"2024-09-21T03:37:53.049781+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:37:57.750665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.111172+0000","last_change":"2024-09-21T03:37:12.858431+0000","last_active":"2024-09-21T03:37:53.111172+0000","last_peered":"2024-09-21T03:37:53.111172+0000","last_clean":"2024-09-21T03:37:53.111172+0000","last_became_active":"2024-09-21T03:37:12.858047+0000","last_became_peered":"2024-09-21T03:37:12.858047+0000","last_unstale":"2024-09-21T03:37:53.111172+0000","last_undegraded":"2024-09-21T03:37:53.111172+0000","last_fullsized":"2024-09-21T03:37:53.111172+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:14:05.363743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.048838+0000","last_change":"2024-09-21T03:37:12.866784+0000","last_active":"2024-09-21T03:37:53.048838+0000","last_peered":"2024-09-21T03:37:53.048838+0000","last_clean":"2024-09-21T03:37:53.048838+0000","last_became_active":"2024-09-21T03:37:12.866435+0000","last_became_peered":"2024-09-21T03:37:12.866435+0000","last_unstale":"2024-09-21T03:37:53.048838+0000","last_undegraded":"2024-09-21T03:37:53.048838+0000","last_fullsized":"2024-09-21T03:37:53.048838+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:09:06.156030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.048908+0000","last_change":"2024-09-21T03:37:12.857054+0000","last_active":"2024-09-21T03:37:53.048908+0000","last_peered":"2024-09-21T03:37:53.048908+0000","last_clean":"2024-09-21T03:37:53.048908+0000","last_became_active":"2024-09-21T03:37:12.856671+0000","last_became_peered":"2024-09-21T03:37:12.856671+0000","last_unstale":"2024-09-21T03:37:53.048908+0000","last_undegraded":"2024-09-21T03:37:53.048908+0000","last_fullsized":"2024-09-21T03:37:53.048908+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:38:27.474550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.042778+0000","last_change":"2024-09-21T03:37:12.861501+0000","last_active":"2024-09-21T03:37:53.042778+0000","last_peered":"2024-09-21T03:37:53.042778+0000","last_clean":"2024-09-21T03:37:53.042778+0000","last_became_active":"2024-09-21T03:37:12.861146+0000","last_became_peered":"2024-09-21T03:37:12.861146+0000","last_unstale":"2024-09-21T03:37:53.042778+0000","last_undegraded":"2024-09-21T03:37:53.042778+0000","last_fullsized":"2024-09-21T03:37:53.042778+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:30:55.073854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.052333+0000","last_change":"2024-09-21T03:37:53.051912+0000","last_active":"2024-09-21T03:37:53.052333+0000","last_peered":"2024-09-21T03:37:53.052333+0000","last_clean":"2024-09-21T03:37:53.052333+0000","last_became_active":"2024-09-21T03:37:53.051640+0000","last_became_peered":"2024-09-21T03:37:53.051640+0000","last_unstale":"2024-09-21T03:37:53.052333+0000","last_undegraded":"2024-09-21T03:37:53.052333+0000","last_fullsized":"2024-09-21T03:37:53.052333+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:08:09.046867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,10,8],"acting":[9,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.431260+0000","last_change":"2024-09-21T03:37:12.865862+0000","last_active":"2024-09-21T03:37:53.431260+0000","last_peered":"2024-09-21T03:37:53.431260+0000","last_clean":"2024-09-21T03:37:53.431260+0000","last_became_active":"2024-09-21T03:37:12.865468+0000","last_became_peered":"2024-09-21T03:37:12.865468+0000","last_unstale":"2024-09-21T03:37:53.431260+0000","last_undegraded":"2024-09-21T03:37:53.431260+0000","last_fullsized":"2024-09-21T03:37:53.431260+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:18:14.496495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.047575+0000","last_change":"2024-09-21T03:37:12.862688+0000","last_active":"2024-09-21T03:37:53.047575+0000","last_peered":"2024-09-21T03:37:53.047575+0000","last_clean":"2024-09-21T03:37:53.047575+0000","last_became_active":"2024-09-21T03:37:12.862249+0000","last_became_peered":"2024-09-21T03:37:12.862249+0000","last_unstale":"2024-09-21T03:37:53.047575+0000","last_undegraded":"2024-09-21T03:37:53.047575+0000","last_fullsized":"2024-09-21T03:37:53.047575+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:55:54.731640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.112095+0000","last_change":"2024-09-21T03:37:12.863595+0000","last_active":"2024-09-21T03:37:53.112095+0000","last_peered":"2024-09-21T03:37:53.112095+0000","last_clean":"2024-09-21T03:37:53.112095+0000","last_became_active":"2024-09-21T03:37:12.863260+0000","last_became_peered":"2024-09-21T03:37:12.863260+0000","last_unstale":"2024-09-21T03:37:53.112095+0000","last_undegraded":"2024-09-21T03:37:53.112095+0000","last_fullsized":"2024-09-21T03:37:53.112095+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:10:07.995690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.045457+0000","last_change":"2024-09-21T03:37:12.865770+0000","last_active":"2024-09-21T03:37:53.045457+0000","last_peered":"2024-09-21T03:37:53.045457+0000","last_clean":"2024-09-21T03:37:53.045457+0000","last_became_active":"2024-09-21T03:37:12.865405+0000","last_became_peered":"2024-09-21T03:37:12.865405+0000","last_unstale":"2024-09-21T03:37:53.045457+0000","last_undegraded":"2024-09-21T03:37:53.045457+0000","last_fullsized":"2024-09-21T03:37:53.045457+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:46:25.255913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.430807+0000","last_change":"2024-09-21T03:37:12.856214+0000","last_active":"2024-09-21T03:37:53.430807+0000","last_peered":"2024-09-21T03:37:53.430807+0000","last_clean":"2024-09-21T03:37:53.430807+0000","last_became_active":"2024-09-21T03:37:12.853924+0000","last_became_peered":"2024-09-21T03:37:12.853924+0000","last_unstale":"2024-09-21T03:37:53.430807+0000","last_undegraded":"2024-09-21T03:37:53.430807+0000","last_fullsized":"2024-09-21T03:37:53.430807+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:10:32.182892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.047154+0000","last_change":"2024-09-21T03:37:12.858851+0000","last_active":"2024-09-21T03:37:53.047154+0000","last_peered":"2024-09-21T03:37:53.047154+0000","last_clean":"2024-09-21T03:37:53.047154+0000","last_became_active":"2024-09-21T03:37:12.858037+0000","last_became_peered":"2024-09-21T03:37:12.858037+0000","last_unstale":"2024-09-21T03:37:53.047154+0000","last_undegraded":"2024-09-21T03:37:53.047154+0000","last_fullsized":"2024-09-21T03:37:53.047154+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:32:23.134789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.051281+0000","last_change":"2024-09-21T03:37:12.852339+0000","last_active":"2024-09-21T03:37:53.051281+0000","last_peered":"2024-09-21T03:37:53.051281+0000","last_clean":"2024-09-21T03:37:53.051281+0000","last_became_active":"2024-09-21T03:37:12.851911+0000","last_became_peered":"2024-09-21T03:37:12.851911+0000","last_unstale":"2024-09-21T03:37:53.051281+0000","last_undegraded":"2024-09-21T03:37:53.051281+0000","last_fullsized":"2024-09-21T03:37:53.051281+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:03:30.739275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.044201+0000","last_change":"2024-09-21T03:37:12.861133+0000","last_active":"2024-09-21T03:37:53.044201+0000","last_peered":"2024-09-21T03:37:53.044201+0000","last_clean":"2024-09-21T03:37:53.044201+0000","last_became_active":"2024-09-21T03:37:12.860719+0000","last_became_peered":"2024-09-21T03:37:12.860719+0000","last_unstale":"2024-09-21T03:37:53.044201+0000","last_undegraded":"2024-09-21T03:37:53.044201+0000","last_fullsized":"2024-09-21T03:37:53.044201+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:21:25.241935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.049308+0000","last_change":"2024-09-21T03:37:12.865694+0000","last_active":"2024-09-21T03:37:53.049308+0000","last_peered":"2024-09-21T03:37:53.049308+0000","last_clean":"2024-09-21T03:37:53.049308+0000","last_became_active":"2024-09-21T03:37:12.864983+0000","last_became_peered":"2024-09-21T03:37:12.864983+0000","last_unstale":"2024-09-21T03:37:53.049308+0000","last_undegraded":"2024-09-21T03:37:53.049308+0000","last_fullsized":"2024-09-21T03:37:53.049308+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:52:28.583056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.049444+0000","last_change":"2024-09-21T03:37:12.860665+0000","last_active":"2024-09-21T03:37:53.049444+0000","last_peered":"2024-09-21T03:37:53.049444+0000","last_clean":"2024-09-21T03:37:53.049444+0000","last_became_active":"2024-09-21T03:37:12.860316+0000","last_became_peered":"2024-09-21T03:37:12.860316+0000","last_unstale":"2024-09-21T03:37:53.049444+0000","last_undegraded":"2024-09-21T03:37:53.049444+0000","last_fullsized":"2024-09-21T03:37:53.049444+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:13:07.316422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T03:37:53.048840+0000","last_change":"2024-09-21T03:37:12.869577+0000","last_active":"2024-09-21T03:37:53.048840+0000","last_peered":"2024-09-21T03:37:53.048840+0000","last_clean":"2024-09-21T03:37:53.048840+0000","last_became_active":"2024-09-21T03:37:12.869164+0000","last_became_peered":"2024-09-21T03:37:12.869164+0000","last_unstale":"2024-09-21T03:37:53.048840+0000","last_undegraded":"2024-09-21T03:37:53.048840+0000","last_fullsized":"2024-09-21T03:37:53.048840+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:40:34.167005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":71139,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":337,"num_read_kb":240,"num_write":164,"num_write_kb":119,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":694,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":213417,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":141,"ondisk_log_size":141,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":285501,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":68,"seq":292057776159,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32068,"kb_used_data":884,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713084,"statfs":{"total":95995035648,"available":95962198016,"internally_reserved":0,"allocated":905216,"data_stored":461385,"data_compressed":23469,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":63,"seq":270582939684,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36368,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708784,"statfs":{"total":95995035648,"available":95957794816,"internally_reserved":0,"allocated":1048576,"data_stored":1115679,"data_compressed":59763,"data_compressed_allocated":77824,"data_compressed_original":655360,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":58,"seq":249108103210,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27920,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717232,"statfs":{"total":95995035648,"available":95966445568,"internally_reserved":0,"allocated":917504,"data_stored":465605,"data_compressed":23469,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":52,"seq":223338299438,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27908,"kb_used_data":884,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717244,"statfs":{"total":95995035648,"available":95966457856,"internally_reserved":0,"allocated":905216,"data_stored":461317,"data_compressed":23469,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":46,"seq":197568495668,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36284,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708868,"statfs":{"total":95995035648,"available":95957880832,"internally_reserved":0,"allocated":962560,"data_stored":1057338,"data_compressed":59763,"data_compressed_allocated":77824,"data_compressed_original":655360,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":41,"seq":176093659192,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32064,"kb_used_data":872,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713088,"statfs":{"total":95995035648,"available":95962202112,"internally_reserved":0,"allocated":892928,"data_stored":456685,"data_compressed":23469,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":35,"seq":150323855421,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32140,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713012,"statfs":{"total":95995035648,"available":95962124288,"internally_reserved":0,"allocated":970752,"data_stored":1052926,"data_compressed":59763,"data_compressed_allocated":77824,"data_compressed_original":655360,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":30,"seq":128849018946,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27912,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717240,"statfs":{"total":95995035648,"available":95966453760,"internally_reserved":0,"allocated":909312,"data_stored":461917,"data_compressed":23469,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":24,"seq":103079215174,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27908,"kb_used_data":884,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717244,"statfs":{"total":95995035648,"available":95966457856,"internally_reserved":0,"allocated":905216,"data_stored":456879,"data_compressed":23469,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":17,"seq":73014444109,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27988,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717164,"statfs":{"total":95995035648,"available":95966375936,"internally_reserved":0,"allocated":987136,"data_stored":524892,"data_compressed":23469,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574929,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36288,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708864,"statfs":{"total":95995035648,"available":95957876736,"internally_reserved":0,"allocated":966656,"data_stored":523733,"data_compressed":23469,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738455,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32084,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713068,"statfs":{"total":95995035648,"available":95962181632,"internally_reserved":0,"allocated":917504,"data_stored":458073,"data_compressed":23469,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34677,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34677,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34677,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36294,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36294,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36294,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36294,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36294,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":67138,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":68297,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1226,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1867,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2183,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":626,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":68716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T03:38:45.860 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-09-21T03:38:45.860 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-09-21T03:38:45.860 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-09-21T03:38:45.860 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 7c374098-77c9-11ef-baf5-efdc52797490 -- ceph health --format=json 2024-09-21T03:38:45.961 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:38:45 smithi114 ceph-mon[34140]: pgmap v338: 129 pgs: 129 active+clean; 646 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-21T03:38:45.961 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:38:45 smithi114 ceph-mon[34140]: from='client.15207 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T03:38:46.106 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/7c374098-77c9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:38:46.584 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:46.579+0000 7f6557fe1640 1 -- 172.21.15.89:0/709220929 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6550108d70 msgr2=0x7f655010f5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:46.584 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:46.579+0000 7f6557fe1640 1 --2- 172.21.15.89:0/709220929 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6550108d70 0x7f655010f5a0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f6548009ca0 tx=0x7f654802a8c0 comp rx=0 tx=0).stop 2024-09-21T03:38:46.584 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:46.579+0000 7f6557fe1640 1 -- 172.21.15.89:0/709220929 shutdown_connections 2024-09-21T03:38:46.584 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:46.579+0000 7f6557fe1640 1 --2- 172.21.15.89:0/709220929 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6550108d70 0x7f655010f5a0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:46.584 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:46.579+0000 7f6557fe1640 1 --2- 172.21.15.89:0/709220929 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6550108420 0x7f6550108830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:46.584 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:46.579+0000 7f6557fe1640 1 --2- 172.21.15.89:0/709220929 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6550102ae0 0x7f6550102ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:46.584 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:46.579+0000 7f6557fe1640 1 -- 172.21.15.89:0/709220929 >> 172.21.15.89:0/709220929 conn(0x7f6550079770 msgr2=0x7f6550079b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:38:46.584 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:46.579+0000 7f6557fe1640 1 -- 172.21.15.89:0/709220929 shutdown_connections 2024-09-21T03:38:46.584 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:46.579+0000 7f6557fe1640 1 -- 172.21.15.89:0/709220929 wait complete. 2024-09-21T03:38:46.585 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:46.580+0000 7f6557fe1640 1 Processor -- start 2024-09-21T03:38:46.585 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:46.580+0000 7f6557fe1640 1 -- start start 2024-09-21T03:38:46.585 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:46.580+0000 7f6557fe1640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6550102ae0 0x7f655019a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:46.585 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:46.581+0000 7f6557fe1640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6550108420 0x7f655019aa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:46.585 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:46.581+0000 7f6557fe1640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6550108d70 0x7f655019fca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:46.585 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:46.581+0000 7f6557fe1640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f6550113a30 con 0x7f6550102ae0 2024-09-21T03:38:46.585 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:46.581+0000 7f6557fe1640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f65501138b0 con 0x7f6550108420 2024-09-21T03:38:46.585 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:46.581+0000 7f6557fe1640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f6550113bb0 con 0x7f6550108d70 2024-09-21T03:38:46.585 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:46.581+0000 7f6555d7d640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6550102ae0 0x7f655019a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:46.585 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:46.581+0000 7f6555d7d640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6550102ae0 0x7f655019a460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:51194/0 (socket says 172.21.15.89:51194) 2024-09-21T03:38:46.585 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:46.581+0000 7f6555d7d640 1 -- 172.21.15.89:0/18151746 learned_addr learned my addr 172.21.15.89:0/18151746 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-21T03:38:46.585 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:46.581+0000 7f655557c640 1 --2- 172.21.15.89:0/18151746 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6550108420 0x7f655019aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:46.585 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:46.581+0000 7f655657e640 1 --2- 172.21.15.89:0/18151746 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6550108d70 0x7f655019fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:46.586 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:46.581+0000 7f6555d7d640 1 -- 172.21.15.89:0/18151746 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6550108d70 msgr2=0x7f655019fca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:46.586 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:46.581+0000 7f6555d7d640 1 --2- 172.21.15.89:0/18151746 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6550108d70 0x7f655019fca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:46.586 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:46.581+0000 7f6555d7d640 1 -- 172.21.15.89:0/18151746 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6550108420 msgr2=0x7f655019aa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:46.586 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:46.581+0000 7f6555d7d640 1 --2- 172.21.15.89:0/18151746 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6550108420 0x7f655019aa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:46.586 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:46.581+0000 7f6555d7d640 1 -- 172.21.15.89:0/18151746 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65501a03d0 con 0x7f6550102ae0 2024-09-21T03:38:46.586 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:46.581+0000 7f655657e640 1 --2- 172.21.15.89:0/18151746 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6550108d70 0x7f655019fca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:38:46.586 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:46.581+0000 7f6555d7d640 1 --2- 172.21.15.89:0/18151746 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6550102ae0 0x7f655019a460 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f654c00c890 tx=0x7f654c00cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:38:46.586 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:46.581+0000 7f6546ffd640 1 -- 172.21.15.89:0/18151746 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f654c004220 con 0x7f6550102ae0 2024-09-21T03:38:46.586 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:46.581+0000 7f6557fe1640 1 -- 172.21.15.89:0/18151746 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f65501a06c0 con 0x7f6550102ae0 2024-09-21T03:38:46.586 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:46.581+0000 7f6546ffd640 1 -- 172.21.15.89:0/18151746 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f654c007e40 con 0x7f6550102ae0 2024-09-21T03:38:46.586 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:46.581+0000 7f6546ffd640 1 -- 172.21.15.89:0/18151746 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f654c004a00 con 0x7f6550102ae0 2024-09-21T03:38:46.586 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:46.581+0000 7f6557fe1640 1 -- 172.21.15.89:0/18151746 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f655010ff60 con 0x7f6550102ae0 2024-09-21T03:38:46.587 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:46.583+0000 7f6557fe1640 1 -- 172.21.15.89:0/18151746 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6550103d60 con 0x7f6550102ae0 2024-09-21T03:38:46.587 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:46.584+0000 7f6546ffd640 1 -- 172.21.15.89:0/18151746 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 18) ==== 154563+0+0 (secure 0 0 0) 0x7f654c0079b0 con 0x7f6550102ae0 2024-09-21T03:38:46.587 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:46.585+0000 7f6546ffd640 1 --2- 172.21.15.89:0/18151746 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f653c0b65a0 0x7f653c0b8ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:46.587 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:46.585+0000 7f655557c640 1 --2- 172.21.15.89:0/18151746 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f653c0b65a0 0x7f653c0b8ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:46.587 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:46.585+0000 7f6546ffd640 1 -- 172.21.15.89:0/18151746 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 9092+0+0 (secure 0 0 0) 0x7f654c0e7a10 con 0x7f6550102ae0 2024-09-21T03:38:46.588 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:46.585+0000 7f655557c640 1 --2- 172.21.15.89:0/18151746 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f653c0b65a0 0x7f653c0b8ab0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f6540005fd0 tx=0x7f6540005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:38:46.590 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:46.588+0000 7f6546ffd640 1 -- 172.21.15.89:0/18151746 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f654c014030 con 0x7f6550102ae0 2024-09-21T03:38:46.750 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:38:46 smithi110 ceph-mon[34000]: from='client.15213 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T03:38:46.805 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:46.802+0000 7f6557fe1640 1 -- 172.21.15.89:0/18151746 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f6550102ef0 con 0x7f6550102ae0 2024-09-21T03:38:46.805 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:38:46 smithi089 ceph-mon[30928]: from='client.15213 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T03:38:46.805 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:46.803+0000 7f6546ffd640 1 -- 172.21.15.89:0/18151746 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7f654c0adde0 con 0x7f6550102ae0 2024-09-21T03:38:46.805 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-09-21T03:38:46.806 INFO:teuthology.orchestra.run.smithi089.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-09-21T03:38:46.808 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:46.805+0000 7f6557fe1640 1 -- 172.21.15.89:0/18151746 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f653c0b65a0 msgr2=0x7f653c0b8ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:46.808 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:46.805+0000 7f6557fe1640 1 --2- 172.21.15.89:0/18151746 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f653c0b65a0 0x7f653c0b8ab0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f6540005fd0 tx=0x7f6540005850 comp rx=0 tx=0).stop 2024-09-21T03:38:46.808 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:46.805+0000 7f6557fe1640 1 -- 172.21.15.89:0/18151746 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6550102ae0 msgr2=0x7f655019a460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:46.808 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:46.805+0000 7f6557fe1640 1 --2- 172.21.15.89:0/18151746 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6550102ae0 0x7f655019a460 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f654c00c890 tx=0x7f654c00cd50 comp rx=0 tx=0).stop 2024-09-21T03:38:46.808 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:46.805+0000 7f6557fe1640 1 -- 172.21.15.89:0/18151746 shutdown_connections 2024-09-21T03:38:46.808 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:46.805+0000 7f6557fe1640 1 --2- 172.21.15.89:0/18151746 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f653c0b65a0 0x7f653c0b8ab0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:46.808 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:46.805+0000 7f6557fe1640 1 --2- 172.21.15.89:0/18151746 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6550108420 0x7f655019aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:46.808 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:46.805+0000 7f6557fe1640 1 --2- 172.21.15.89:0/18151746 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6550108d70 0x7f655019fca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:46.808 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:46.805+0000 7f6557fe1640 1 --2- 172.21.15.89:0/18151746 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6550102ae0 0x7f655019a460 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:46.808 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:46.805+0000 7f6557fe1640 1 -- 172.21.15.89:0/18151746 >> 172.21.15.89:0/18151746 conn(0x7f6550079770 msgr2=0x7f6550106510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:38:46.808 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:46.806+0000 7f6557fe1640 1 -- 172.21.15.89:0/18151746 shutdown_connections 2024-09-21T03:38:46.808 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:46.806+0000 7f6557fe1640 1 -- 172.21.15.89:0/18151746 wait complete. 2024-09-21T03:38:46.961 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:38:46 smithi114 ceph-mon[34140]: from='client.15213 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T03:38:47.435 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-09-21T03:38:47.435 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 7c374098-77c9-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T03:38:47.685 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/7c374098-77c9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:38:47.750 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:38:47 smithi110 ceph-mon[34000]: pgmap v339: 129 pgs: 129 active+clean; 646 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-09-21T03:38:47.750 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:38:47 smithi110 ceph-mon[34000]: from='client.? 172.21.15.89:0/18151746' entity='client.admin' cmd={"prefix": "health", "format": "json"} : dispatch 2024-09-21T03:38:47.806 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:38:47 smithi089 ceph-mon[30928]: pgmap v339: 129 pgs: 129 active+clean; 646 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-09-21T03:38:47.806 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:38:47 smithi089 ceph-mon[30928]: from='client.? 172.21.15.89:0/18151746' entity='client.admin' cmd={"prefix": "health", "format": "json"} : dispatch 2024-09-21T03:38:47.960 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:38:47 smithi114 ceph-mon[34140]: pgmap v339: 129 pgs: 129 active+clean; 646 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-09-21T03:38:47.961 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:38:47 smithi114 ceph-mon[34140]: from='client.? 172.21.15.89:0/18151746' entity='client.admin' cmd={"prefix": "health", "format": "json"} : dispatch 2024-09-21T03:38:48.170 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:48.161+0000 7fc7eedc4640 1 -- 172.21.15.89:0/735072291 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc7e8106330 msgr2=0x7fc7e8106740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:48.170 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:48.161+0000 7fc7eedc4640 1 --2- 172.21.15.89:0/735072291 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc7e8106330 0x7fc7e8106740 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fc7d4009900 tx=0x7fc7d402a410 comp rx=0 tx=0).stop 2024-09-21T03:38:48.170 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:48.161+0000 7fc7eedc4640 1 -- 172.21.15.89:0/735072291 shutdown_connections 2024-09-21T03:38:48.170 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:48.161+0000 7fc7eedc4640 1 --2- 172.21.15.89:0/735072291 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc7e8106c80 0x7fc7e8110710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:48.170 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:48.161+0000 7fc7eedc4640 1 --2- 172.21.15.89:0/735072291 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc7e8106330 0x7fc7e8106740 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:48.170 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:48.161+0000 7fc7eedc4640 1 --2- 172.21.15.89:0/735072291 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc7e8102140 0x7fc7e8102550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:48.170 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:48.161+0000 7fc7eedc4640 1 -- 172.21.15.89:0/735072291 >> 172.21.15.89:0/735072291 conn(0x7fc7e8079ec0 msgr2=0x7fc7e807a2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:38:48.171 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:48.161+0000 7fc7eedc4640 1 -- 172.21.15.89:0/735072291 shutdown_connections 2024-09-21T03:38:48.171 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:48.162+0000 7fc7eedc4640 1 -- 172.21.15.89:0/735072291 wait complete. 2024-09-21T03:38:48.171 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:48.162+0000 7fc7eedc4640 1 Processor -- start 2024-09-21T03:38:48.171 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:48.163+0000 7fc7eedc4640 1 -- start start 2024-09-21T03:38:48.171 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:48.163+0000 7fc7eedc4640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc7e8102140 0x7fc7e819a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:48.171 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:48.163+0000 7fc7eedc4640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc7e8106330 0x7fc7e819a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:48.171 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:48.163+0000 7fc7eedc4640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc7e8106c80 0x7fc7e819fc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:48.171 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:48.163+0000 7fc7eedc4640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fc7e8113a00 con 0x7fc7e8106330 2024-09-21T03:38:48.171 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:48.163+0000 7fc7eedc4640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fc7e8113880 con 0x7fc7e8102140 2024-09-21T03:38:48.171 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:48.163+0000 7fc7eedc4640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fc7e8113b80 con 0x7fc7e8106c80 2024-09-21T03:38:48.171 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:48.163+0000 7fc7e7fff640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc7e8106330 0x7fc7e819a9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:48.171 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:48.163+0000 7fc7e7fff640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc7e8106330 0x7fc7e819a9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:51222/0 (socket says 172.21.15.89:51222) 2024-09-21T03:38:48.171 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:48.163+0000 7fc7e7fff640 1 -- 172.21.15.89:0/1646015252 learned_addr learned my addr 172.21.15.89:0/1646015252 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-21T03:38:48.171 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:48.163+0000 7fc7ecb60640 1 --2- 172.21.15.89:0/1646015252 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc7e8102140 0x7fc7e819a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:48.171 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:48.163+0000 7fc7ed361640 1 --2- 172.21.15.89:0/1646015252 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc7e8106c80 0x7fc7e819fc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:48.172 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:48.163+0000 7fc7e7fff640 1 -- 172.21.15.89:0/1646015252 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc7e8106c80 msgr2=0x7fc7e819fc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:48.172 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:48.163+0000 7fc7e7fff640 1 --2- 172.21.15.89:0/1646015252 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc7e8106c80 0x7fc7e819fc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:48.172 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:48.163+0000 7fc7e7fff640 1 -- 172.21.15.89:0/1646015252 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc7e8102140 msgr2=0x7fc7e819a430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:48.172 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:48.163+0000 7fc7e7fff640 1 --2- 172.21.15.89:0/1646015252 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc7e8102140 0x7fc7e819a430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:48.172 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:48.163+0000 7fc7e7fff640 1 -- 172.21.15.89:0/1646015252 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7e81a03a0 con 0x7fc7e8106330 2024-09-21T03:38:48.172 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:48.163+0000 7fc7e7fff640 1 --2- 172.21.15.89:0/1646015252 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc7e8106330 0x7fc7e819a9d0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fc7d4005940 tx=0x7fc7d4032f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:38:48.172 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:48.164+0000 7fc7e5ffb640 1 -- 172.21.15.89:0/1646015252 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7d403f030 con 0x7fc7e8106330 2024-09-21T03:38:48.172 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:48.164+0000 7fc7e5ffb640 1 -- 172.21.15.89:0/1646015252 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc7d4004450 con 0x7fc7e8106330 2024-09-21T03:38:48.172 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:48.164+0000 7fc7eedc4640 1 -- 172.21.15.89:0/1646015252 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc7e81a0630 con 0x7fc7e8106330 2024-09-21T03:38:48.172 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:48.164+0000 7fc7e5ffb640 1 -- 172.21.15.89:0/1646015252 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7d4035070 con 0x7fc7e8106330 2024-09-21T03:38:48.172 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:48.164+0000 7fc7eedc4640 1 -- 172.21.15.89:0/1646015252 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7e81a6df0 con 0x7fc7e8106330 2024-09-21T03:38:48.172 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:48.165+0000 7fc7eedc4640 1 -- 172.21.15.89:0/1646015252 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc7e806b830 con 0x7fc7e8106330 2024-09-21T03:38:48.172 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:48.167+0000 7fc7e5ffb640 1 -- 172.21.15.89:0/1646015252 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 18) ==== 154563+0+0 (secure 0 0 0) 0x7fc7d4042030 con 0x7fc7e8106330 2024-09-21T03:38:48.173 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:48.167+0000 7fc7e5ffb640 1 --2- 172.21.15.89:0/1646015252 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7fc7d80b65a0 0x7fc7d80b8ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:48.173 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:48.167+0000 7fc7ecb60640 1 --2- 172.21.15.89:0/1646015252 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7fc7d80b65a0 0x7fc7d80b8ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:48.173 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:48.167+0000 7fc7e5ffb640 1 -- 172.21.15.89:0/1646015252 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 9092+0+0 (secure 0 0 0) 0x7fc7d4108050 con 0x7fc7e8106330 2024-09-21T03:38:48.173 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:48.168+0000 7fc7ecb60640 1 --2- 172.21.15.89:0/1646015252 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7fc7d80b65a0 0x7fc7d80b8ab0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fc7e806bb30 tx=0x7fc7dc00b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:38:48.173 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:48.171+0000 7fc7e5ffb640 1 -- 172.21.15.89:0/1646015252 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fc7d40d2800 con 0x7fc7e8106330 2024-09-21T03:38:48.389 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:48.387+0000 7fc7eedc4640 1 -- 172.21.15.89:0/1646015252 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc7e810bce0 con 0x7fc7e8106330 2024-09-21T03:38:48.390 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:48.388+0000 7fc7e5ffb640 1 -- 172.21.15.89:0/1646015252 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11912 (secure 0 0 0) 0x7fc7d40d7740 con 0x7fc7e8106330 2024-09-21T03:38:48.390 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-09-21T03:38:48.390 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":25,"btime":"2024-09-21T03:37:51:019398+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14721,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6835/2341538514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":2341538514},{"type":"v1","addr":"172.21.15.89:6835","nonce":2341538514}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14733,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6837/146708592","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6836","nonce":146708592},{"type":"v1","addr":"172.21.15.89:6837","nonce":146708592}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14781,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6839/2269965499","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6838","nonce":2269965499},{"type":"v1","addr":"172.21.15.89:6839","nonce":2269965499}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14793,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6841/650850383","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6840","nonce":650850383},{"type":"v1","addr":"172.21.15.89:6841","nonce":650850383}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24565,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6833/752336912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6832","nonce":752336912},{"type":"v1","addr":"172.21.15.114:6833","nonce":752336912}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24575,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6833/127607331","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6832","nonce":127607331},{"type":"v1","addr":"172.21.15.110:6833","nonce":127607331}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T03:37:13.570694+0000","modified":"2024-09-21T03:37:50.719106+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24638,"mds_1":24631,"mds_2":24614},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24593":{"gid":24593,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.110:6835/671820654","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6834","nonce":671820654},{"type":"v1","addr":"172.21.15.110:6835","nonce":671820654}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24604":{"gid":24604,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.114:6835/1049714548","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6834","nonce":1049714548},{"type":"v1","addr":"172.21.15.114:6835","nonce":1049714548}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24610":{"gid":24610,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.114:6837/1941213818","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6836","nonce":1941213818},{"type":"v1","addr":"172.21.15.114:6837","nonce":1941213818}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24614":{"gid":24614,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":15,"addr":"172.21.15.110:6837/4047124589","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6836","nonce":4047124589},{"type":"v1","addr":"172.21.15.110:6837","nonce":4047124589}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24631":{"gid":24631,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":15,"addr":"172.21.15.114:6839/3427369944","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6838","nonce":3427369944},{"type":"v1","addr":"172.21.15.114:6839","nonce":3427369944}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24638":{"gid":24638,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.110:6839/132366709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6838","nonce":132366709},{"type":"v1","addr":"172.21.15.110:6839","nonce":132366709}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24638,"qdb_cluster":[24614,24631,24638]},"id":1}]} 2024-09-21T03:38:48.391 INFO:teuthology.orchestra.run.smithi089.stderr:dumped fsmap epoch 25 2024-09-21T03:38:48.393 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:48.390+0000 7fc7eedc4640 1 -- 172.21.15.89:0/1646015252 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7fc7d80b65a0 msgr2=0x7fc7d80b8ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:48.393 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:48.390+0000 7fc7eedc4640 1 --2- 172.21.15.89:0/1646015252 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7fc7d80b65a0 0x7fc7d80b8ab0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fc7e806bb30 tx=0x7fc7dc00b390 comp rx=0 tx=0).stop 2024-09-21T03:38:48.393 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:48.390+0000 7fc7eedc4640 1 -- 172.21.15.89:0/1646015252 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc7e8106330 msgr2=0x7fc7e819a9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:48.393 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:48.390+0000 7fc7eedc4640 1 --2- 172.21.15.89:0/1646015252 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc7e8106330 0x7fc7e819a9d0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fc7d4005940 tx=0x7fc7d4032f90 comp rx=0 tx=0).stop 2024-09-21T03:38:48.393 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:48.390+0000 7fc7eedc4640 1 -- 172.21.15.89:0/1646015252 shutdown_connections 2024-09-21T03:38:48.393 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:48.390+0000 7fc7eedc4640 1 --2- 172.21.15.89:0/1646015252 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7fc7d80b65a0 0x7fc7d80b8ab0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:48.393 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:48.390+0000 7fc7eedc4640 1 --2- 172.21.15.89:0/1646015252 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc7e8106c80 0x7fc7e819fc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:48.393 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:48.390+0000 7fc7eedc4640 1 --2- 172.21.15.89:0/1646015252 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc7e8106330 0x7fc7e819a9d0 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:48.393 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:48.390+0000 7fc7eedc4640 1 --2- 172.21.15.89:0/1646015252 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc7e8102140 0x7fc7e819a430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:48.393 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:48.391+0000 7fc7eedc4640 1 -- 172.21.15.89:0/1646015252 >> 172.21.15.89:0/1646015252 conn(0x7fc7e8079ec0 msgr2=0x7fc7e8107620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:38:48.394 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:48.391+0000 7fc7eedc4640 1 -- 172.21.15.89:0/1646015252 shutdown_connections 2024-09-21T03:38:48.394 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:48.391+0000 7fc7eedc4640 1 -- 172.21.15.89:0/1646015252 wait complete. 2024-09-21T03:38:48.750 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:38:48 smithi110 ceph-mon[34000]: from='client.? 172.21.15.89:0/1646015252' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:38:48.811 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:38:48 smithi089 ceph-mon[30928]: from='client.? 172.21.15.89:0/1646015252' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:38:48.961 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:38:48 smithi114 ceph-mon[34140]: from='client.? 172.21.15.89:0/1646015252' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:38:49.117 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 7c374098-77c9-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T03:38:49.362 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/7c374098-77c9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:38:49.750 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:38:49 smithi110 ceph-mon[34000]: pgmap v340: 129 pgs: 129 active+clean; 646 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-21T03:38:49.802 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:38:49 smithi089 ceph-mon[30928]: pgmap v340: 129 pgs: 129 active+clean; 646 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-21T03:38:49.815 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:49.813+0000 7fc7db6c9640 1 -- 172.21.15.89:0/820993517 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc7d4108cf0 msgr2=0x7fc7d410f520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:49.815 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:49.813+0000 7fc7db6c9640 1 --2- 172.21.15.89:0/820993517 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc7d4108cf0 0x7fc7d410f520 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fc7cc009ca0 tx=0x7fc7cc02a8c0 comp rx=0 tx=0).stop 2024-09-21T03:38:49.815 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:49.813+0000 7fc7db6c9640 1 -- 172.21.15.89:0/820993517 shutdown_connections 2024-09-21T03:38:49.815 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:49.813+0000 7fc7db6c9640 1 --2- 172.21.15.89:0/820993517 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc7d4108cf0 0x7fc7d410f520 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:49.815 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:49.813+0000 7fc7db6c9640 1 --2- 172.21.15.89:0/820993517 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc7d41083a0 0x7fc7d41087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:49.816 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:49.813+0000 7fc7db6c9640 1 --2- 172.21.15.89:0/820993517 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc7d4102a60 0x7fc7d4102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:49.816 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:49.813+0000 7fc7db6c9640 1 -- 172.21.15.89:0/820993517 >> 172.21.15.89:0/820993517 conn(0x7fc7d4079730 msgr2=0x7fc7d4079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:38:49.816 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:49.813+0000 7fc7db6c9640 1 -- 172.21.15.89:0/820993517 shutdown_connections 2024-09-21T03:38:49.816 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:49.813+0000 7fc7db6c9640 1 -- 172.21.15.89:0/820993517 wait complete. 2024-09-21T03:38:49.817 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:49.814+0000 7fc7db6c9640 1 Processor -- start 2024-09-21T03:38:49.817 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:49.814+0000 7fc7db6c9640 1 -- start start 2024-09-21T03:38:49.818 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:49.814+0000 7fc7db6c9640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc7d4102a60 0x7fc7d41050e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:49.818 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:49.814+0000 7fc7db6c9640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc7d41083a0 0x7fc7d4105620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:49.818 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:49.814+0000 7fc7db6c9640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc7d4108cf0 0x7fc7d4105b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:49.818 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:49.814+0000 7fc7db6c9640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fc7d41128e0 con 0x7fc7d4108cf0 2024-09-21T03:38:49.818 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:49.814+0000 7fc7db6c9640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fc7d4112760 con 0x7fc7d41083a0 2024-09-21T03:38:49.818 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:49.814+0000 7fc7db6c9640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fc7d4112a60 con 0x7fc7d4102a60 2024-09-21T03:38:49.818 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:49.814+0000 7fc7d9c66640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc7d4108cf0 0x7fc7d4105b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:49.818 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:49.814+0000 7fc7d9c66640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc7d4108cf0 0x7fc7d4105b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:55222/0 (socket says 172.21.15.89:55222) 2024-09-21T03:38:49.818 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:49.814+0000 7fc7d9c66640 1 -- 172.21.15.89:0/2431747325 learned_addr learned my addr 172.21.15.89:0/2431747325 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-21T03:38:49.818 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:49.814+0000 7fc7d8c64640 1 --2- 172.21.15.89:0/2431747325 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc7d41083a0 0x7fc7d4105620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:49.819 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:49.814+0000 7fc7d9465640 1 --2- 172.21.15.89:0/2431747325 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc7d4102a60 0x7fc7d41050e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:49.819 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:49.815+0000 7fc7d9c66640 1 -- 172.21.15.89:0/2431747325 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc7d4102a60 msgr2=0x7fc7d41050e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:49.819 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:49.815+0000 7fc7d9c66640 1 --2- 172.21.15.89:0/2431747325 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc7d4102a60 0x7fc7d41050e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:49.819 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:49.815+0000 7fc7d9c66640 1 -- 172.21.15.89:0/2431747325 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc7d41083a0 msgr2=0x7fc7d4105620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:49.819 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:49.815+0000 7fc7d9c66640 1 --2- 172.21.15.89:0/2431747325 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc7d41083a0 0x7fc7d4105620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:49.819 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:49.815+0000 7fc7d9c66640 1 -- 172.21.15.89:0/2431747325 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7d41a86d0 con 0x7fc7d4108cf0 2024-09-21T03:38:49.819 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:49.815+0000 7fc7d9c66640 1 --2- 172.21.15.89:0/2431747325 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc7d4108cf0 0x7fc7d4105b60 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fc7cc005cc0 tx=0x7fc7cc032c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:38:49.819 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:49.815+0000 7fc7ca7fc640 1 -- 172.21.15.89:0/2431747325 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7cc004040 con 0x7fc7d4108cf0 2024-09-21T03:38:49.819 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:49.815+0000 7fc7ca7fc640 1 -- 172.21.15.89:0/2431747325 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc7cc004670 con 0x7fc7d4108cf0 2024-09-21T03:38:49.819 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:49.815+0000 7fc7db6c9640 1 -- 172.21.15.89:0/2431747325 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc7d41a8960 con 0x7fc7d4108cf0 2024-09-21T03:38:49.819 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:49.815+0000 7fc7ca7fc640 1 -- 172.21.15.89:0/2431747325 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7cc030cc0 con 0x7fc7d4108cf0 2024-09-21T03:38:49.819 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:49.815+0000 7fc7db6c9640 1 -- 172.21.15.89:0/2431747325 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7d41a8ea0 con 0x7fc7d4108cf0 2024-09-21T03:38:49.820 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:49.816+0000 7fc7db6c9640 1 -- 172.21.15.89:0/2431747325 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc7d4102e70 con 0x7fc7d4108cf0 2024-09-21T03:38:49.821 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:49.819+0000 7fc7ca7fc640 1 -- 172.21.15.89:0/2431747325 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 18) ==== 154563+0+0 (secure 0 0 0) 0x7fc7cc0041e0 con 0x7fc7d4108cf0 2024-09-21T03:38:49.821 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:49.819+0000 7fc7ca7fc640 1 --2- 172.21.15.89:0/2431747325 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7fc7c00b65a0 0x7fc7c00b8ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:49.822 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:49.819+0000 7fc7d9465640 1 --2- 172.21.15.89:0/2431747325 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7fc7c00b65a0 0x7fc7c00b8ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:49.822 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:49.819+0000 7fc7ca7fc640 1 -- 172.21.15.89:0/2431747325 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 9092+0+0 (secure 0 0 0) 0x7fc7cc108d90 con 0x7fc7d4108cf0 2024-09-21T03:38:49.822 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:49.820+0000 7fc7d9465640 1 --2- 172.21.15.89:0/2431747325 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7fc7c00b65a0 0x7fc7c00b8ab0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fc7d4106b30 tx=0x7fc7d000a250 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:38:49.824 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:49.823+0000 7fc7ca7fc640 1 -- 172.21.15.89:0/2431747325 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fc7cc0cf160 con 0x7fc7d4108cf0 2024-09-21T03:38:49.961 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:38:49 smithi114 ceph-mon[34140]: pgmap v340: 129 pgs: 129 active+clean; 646 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-21T03:38:50.038 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:50.036+0000 7fc7db6c9640 1 -- 172.21.15.89:0/2431747325 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc7d41a9230 con 0x7fc7d4108cf0 2024-09-21T03:38:50.039 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:50.037+0000 7fc7ca7fc640 1 -- 172.21.15.89:0/2431747325 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11912 (secure 0 0 0) 0x7fc7cc0f20e0 con 0x7fc7d4108cf0 2024-09-21T03:38:50.039 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-09-21T03:38:50.039 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":25,"btime":"2024-09-21T03:37:51:019398+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14721,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6835/2341538514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":2341538514},{"type":"v1","addr":"172.21.15.89:6835","nonce":2341538514}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14733,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6837/146708592","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6836","nonce":146708592},{"type":"v1","addr":"172.21.15.89:6837","nonce":146708592}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14781,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6839/2269965499","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6838","nonce":2269965499},{"type":"v1","addr":"172.21.15.89:6839","nonce":2269965499}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14793,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6841/650850383","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6840","nonce":650850383},{"type":"v1","addr":"172.21.15.89:6841","nonce":650850383}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24565,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6833/752336912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6832","nonce":752336912},{"type":"v1","addr":"172.21.15.114:6833","nonce":752336912}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24575,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6833/127607331","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6832","nonce":127607331},{"type":"v1","addr":"172.21.15.110:6833","nonce":127607331}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T03:37:13.570694+0000","modified":"2024-09-21T03:37:50.719106+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24638,"mds_1":24631,"mds_2":24614},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24593":{"gid":24593,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.110:6835/671820654","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6834","nonce":671820654},{"type":"v1","addr":"172.21.15.110:6835","nonce":671820654}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24604":{"gid":24604,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.114:6835/1049714548","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6834","nonce":1049714548},{"type":"v1","addr":"172.21.15.114:6835","nonce":1049714548}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24610":{"gid":24610,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.114:6837/1941213818","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6836","nonce":1941213818},{"type":"v1","addr":"172.21.15.114:6837","nonce":1941213818}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24614":{"gid":24614,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":15,"addr":"172.21.15.110:6837/4047124589","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6836","nonce":4047124589},{"type":"v1","addr":"172.21.15.110:6837","nonce":4047124589}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24631":{"gid":24631,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":15,"addr":"172.21.15.114:6839/3427369944","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6838","nonce":3427369944},{"type":"v1","addr":"172.21.15.114:6839","nonce":3427369944}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24638":{"gid":24638,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.110:6839/132366709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6838","nonce":132366709},{"type":"v1","addr":"172.21.15.110:6839","nonce":132366709}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24638,"qdb_cluster":[24614,24631,24638]},"id":1}]} 2024-09-21T03:38:50.040 INFO:teuthology.orchestra.run.smithi089.stderr:dumped fsmap epoch 25 2024-09-21T03:38:50.041 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:50.039+0000 7fc7db6c9640 1 -- 172.21.15.89:0/2431747325 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7fc7c00b65a0 msgr2=0x7fc7c00b8ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:50.042 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:50.039+0000 7fc7db6c9640 1 --2- 172.21.15.89:0/2431747325 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7fc7c00b65a0 0x7fc7c00b8ab0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fc7d4106b30 tx=0x7fc7d000a250 comp rx=0 tx=0).stop 2024-09-21T03:38:50.042 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:50.039+0000 7fc7db6c9640 1 -- 172.21.15.89:0/2431747325 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc7d4108cf0 msgr2=0x7fc7d4105b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:50.042 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:50.039+0000 7fc7db6c9640 1 --2- 172.21.15.89:0/2431747325 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc7d4108cf0 0x7fc7d4105b60 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fc7cc005cc0 tx=0x7fc7cc032c40 comp rx=0 tx=0).stop 2024-09-21T03:38:50.042 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:50.039+0000 7fc7db6c9640 1 -- 172.21.15.89:0/2431747325 shutdown_connections 2024-09-21T03:38:50.042 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:50.039+0000 7fc7db6c9640 1 --2- 172.21.15.89:0/2431747325 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7fc7c00b65a0 0x7fc7c00b8ab0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:50.042 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:50.039+0000 7fc7db6c9640 1 --2- 172.21.15.89:0/2431747325 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc7d41083a0 0x7fc7d4105620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:50.042 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:50.039+0000 7fc7db6c9640 1 --2- 172.21.15.89:0/2431747325 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc7d4108cf0 0x7fc7d4105b60 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:50.042 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:50.039+0000 7fc7db6c9640 1 --2- 172.21.15.89:0/2431747325 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc7d4102a60 0x7fc7d41050e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:50.042 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:50.039+0000 7fc7db6c9640 1 -- 172.21.15.89:0/2431747325 >> 172.21.15.89:0/2431747325 conn(0x7fc7d4079730 msgr2=0x7fc7d4100c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:38:50.042 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:50.039+0000 7fc7db6c9640 1 -- 172.21.15.89:0/2431747325 shutdown_connections 2024-09-21T03:38:50.042 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:50.039+0000 7fc7db6c9640 1 -- 172.21.15.89:0/2431747325 wait complete. 2024-09-21T03:38:50.750 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:38:50 smithi110 ceph-mon[34000]: from='client.? 172.21.15.89:0/2431747325' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:38:50.811 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:38:50 smithi089 ceph-mon[30928]: from='client.? 172.21.15.89:0/2431747325' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:38:50.844 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 7c374098-77c9-11ef-baf5-efdc52797490 -- ceph osd dump --format=json 2024-09-21T03:38:50.961 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:38:50 smithi114 ceph-mon[34140]: from='client.? 172.21.15.89:0/2431747325' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:38:51.104 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/7c374098-77c9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:38:51.638 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:51.627+0000 7ff1d72f7640 1 -- 172.21.15.89:0/3665663212 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff1d0109df0 msgr2=0x7ff1d0110620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:51.638 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:51.627+0000 7ff1d72f7640 1 --2- 172.21.15.89:0/3665663212 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff1d0109df0 0x7ff1d0110620 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7ff1c8009ca0 tx=0x7ff1c802a8c0 comp rx=0 tx=0).stop 2024-09-21T03:38:51.638 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:51.627+0000 7ff1d72f7640 1 -- 172.21.15.89:0/3665663212 shutdown_connections 2024-09-21T03:38:51.638 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:51.627+0000 7ff1d72f7640 1 --2- 172.21.15.89:0/3665663212 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff1d0109df0 0x7ff1d0110620 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:51.638 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:51.627+0000 7ff1d72f7640 1 --2- 172.21.15.89:0/3665663212 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff1d01094a0 0x7ff1d01098b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:51.638 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:51.627+0000 7ff1d72f7640 1 --2- 172.21.15.89:0/3665663212 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff1d0103b60 0x7ff1d0103f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:51.639 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:51.627+0000 7ff1d72f7640 1 -- 172.21.15.89:0/3665663212 >> 172.21.15.89:0/3665663212 conn(0x7ff1d007b850 msgr2=0x7ff1d007bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:38:51.639 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:51.627+0000 7ff1d72f7640 1 -- 172.21.15.89:0/3665663212 shutdown_connections 2024-09-21T03:38:51.639 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:51.627+0000 7ff1d72f7640 1 -- 172.21.15.89:0/3665663212 wait complete. 2024-09-21T03:38:51.639 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:51.628+0000 7ff1d72f7640 1 Processor -- start 2024-09-21T03:38:51.639 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:51.628+0000 7ff1d72f7640 1 -- start start 2024-09-21T03:38:51.639 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:51.628+0000 7ff1d72f7640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff1d0103b60 0x7ff1d019a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:51.639 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:51.628+0000 7ff1d72f7640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff1d01094a0 0x7ff1d019aa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:51.639 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:51.628+0000 7ff1d72f7640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff1d0109df0 0x7ff1d019fd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:51.639 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:51.628+0000 7ff1d72f7640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7ff1d01139b0 con 0x7ff1d01094a0 2024-09-21T03:38:51.640 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:51.628+0000 7ff1d72f7640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7ff1d0113830 con 0x7ff1d0109df0 2024-09-21T03:38:51.640 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:51.628+0000 7ff1d72f7640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7ff1d0113b30 con 0x7ff1d0103b60 2024-09-21T03:38:51.640 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:51.628+0000 7ff1d4892640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff1d01094a0 0x7ff1d019aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:51.640 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:51.629+0000 7ff1d4892640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff1d01094a0 0x7ff1d019aa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:55232/0 (socket says 172.21.15.89:55232) 2024-09-21T03:38:51.640 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:51.629+0000 7ff1d4892640 1 -- 172.21.15.89:0/107549950 learned_addr learned my addr 172.21.15.89:0/107549950 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-21T03:38:51.640 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:51.629+0000 7ff1d5093640 1 --2- 172.21.15.89:0/107549950 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff1d0103b60 0x7ff1d019a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:51.640 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:51.629+0000 7ff1d5894640 1 --2- 172.21.15.89:0/107549950 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff1d0109df0 0x7ff1d019fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:51.640 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:51.629+0000 7ff1d4892640 1 -- 172.21.15.89:0/107549950 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff1d0103b60 msgr2=0x7ff1d019a4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:51.640 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:51.629+0000 7ff1d4892640 1 --2- 172.21.15.89:0/107549950 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff1d0103b60 0x7ff1d019a4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:51.640 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:51.629+0000 7ff1d4892640 1 -- 172.21.15.89:0/107549950 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff1d0109df0 msgr2=0x7ff1d019fd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:51.640 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:51.629+0000 7ff1d4892640 1 --2- 172.21.15.89:0/107549950 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff1d0109df0 0x7ff1d019fd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:51.640 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:51.629+0000 7ff1d4892640 1 -- 172.21.15.89:0/107549950 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1d01a0430 con 0x7ff1d01094a0 2024-09-21T03:38:51.640 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:51.629+0000 7ff1d5894640 1 --2- 172.21.15.89:0/107549950 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff1d0109df0 0x7ff1d019fd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:38:51.640 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:51.629+0000 7ff1d4892640 1 --2- 172.21.15.89:0/107549950 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff1d01094a0 0x7ff1d019aa60 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7ff1cc00d750 tx=0x7ff1cc00dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:38:51.641 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:51.629+0000 7ff1c27fc640 1 -- 172.21.15.89:0/107549950 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1cc017070 con 0x7ff1d01094a0 2024-09-21T03:38:51.641 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:51.629+0000 7ff1c27fc640 1 -- 172.21.15.89:0/107549950 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff1cc0054b0 con 0x7ff1d01094a0 2024-09-21T03:38:51.641 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:51.629+0000 7ff1d72f7640 1 -- 172.21.15.89:0/107549950 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff1d01a0720 con 0x7ff1d01094a0 2024-09-21T03:38:51.641 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:51.629+0000 7ff1c27fc640 1 -- 172.21.15.89:0/107549950 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1cc004bc0 con 0x7ff1d01094a0 2024-09-21T03:38:51.641 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:51.629+0000 7ff1d72f7640 1 -- 172.21.15.89:0/107549950 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1d01a6e80 con 0x7ff1d01094a0 2024-09-21T03:38:51.641 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:51.630+0000 7ff1d72f7640 1 -- 172.21.15.89:0/107549950 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff1d0103f70 con 0x7ff1d01094a0 2024-09-21T03:38:51.641 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:51.632+0000 7ff1c27fc640 1 -- 172.21.15.89:0/107549950 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 18) ==== 154563+0+0 (secure 0 0 0) 0x7ff1cc009c30 con 0x7ff1d01094a0 2024-09-21T03:38:51.641 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:51.633+0000 7ff1c27fc640 1 --2- 172.21.15.89:0/107549950 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7ff1bc0b6620 0x7ff1bc0b8b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:51.641 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:51.633+0000 7ff1d5093640 1 --2- 172.21.15.89:0/107549950 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7ff1bc0b6620 0x7ff1bc0b8b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:51.642 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:51.633+0000 7ff1c27fc640 1 -- 172.21.15.89:0/107549950 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 9092+0+0 (secure 0 0 0) 0x7ff1cc0e7520 con 0x7ff1d01094a0 2024-09-21T03:38:51.642 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:51.633+0000 7ff1d5093640 1 --2- 172.21.15.89:0/107549950 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7ff1bc0b6620 0x7ff1bc0b8b30 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7ff1c400bbe0 tx=0x7ff1c400b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:38:51.642 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:51.636+0000 7ff1c27fc640 1 -- 172.21.15.89:0/107549950 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7ff1cc0ad970 con 0x7ff1d01094a0 2024-09-21T03:38:51.750 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:38:51 smithi110 ceph-mon[34000]: pgmap v341: 129 pgs: 129 active+clean; 646 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-21T03:38:51.811 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:38:51 smithi089 ceph-mon[30928]: pgmap v341: 129 pgs: 129 active+clean; 646 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-21T03:38:51.824 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:51.822+0000 7ff1d72f7640 1 -- 172.21.15.89:0/107549950 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff1d019b5f0 con 0x7ff1d01094a0 2024-09-21T03:38:51.825 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:51.823+0000 7ff1c27fc640 1 -- 172.21.15.89:0/107549950 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) ==== 74+0+24763 (secure 0 0 0) 0x7ff1d019b5f0 con 0x7ff1d01094a0 2024-09-21T03:38:51.826 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-09-21T03:38:51.826 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":82,"fsid":"7c374098-77c9-11ef-baf5-efdc52797490","created":"2024-09-21T03:28:56.681458+0000","modified":"2024-09-21T03:37:53.033682+0000","last_up_change":"2024-09-21T03:36:23.368379+0000","last_in_change":"2024-09-21T03:36:00.131331+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T03:32:37.834339+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-21T03:37:09.443341+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"74","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-21T03:37:11.467821+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"74","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-21T03:37:17.884066+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"27ca5ed8-17d1-4f8e-a7ae-084a914721a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6802","nonce":990539042},{"type":"v1","addr":"172.21.15.89:6803","nonce":990539042}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6804","nonce":990539042},{"type":"v1","addr":"172.21.15.89:6805","nonce":990539042}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6808","nonce":990539042},{"type":"v1","addr":"172.21.15.89:6809","nonce":990539042}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6806","nonce":990539042},{"type":"v1","addr":"172.21.15.89:6807","nonce":990539042}]},"public_addr":"172.21.15.89:6803/990539042","cluster_addr":"172.21.15.89:6805/990539042","heartbeat_back_addr":"172.21.15.89:6809/990539042","heartbeat_front_addr":"172.21.15.89:6807/990539042","state":["exists","up"]},{"osd":1,"uuid":"5176a910-5cf9-49d7-aedb-a20cf36ea075","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6800","nonce":3235852628},{"type":"v1","addr":"172.21.15.110:6801","nonce":3235852628}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6802","nonce":3235852628},{"type":"v1","addr":"172.21.15.110:6803","nonce":3235852628}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6806","nonce":3235852628},{"type":"v1","addr":"172.21.15.110:6807","nonce":3235852628}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6804","nonce":3235852628},{"type":"v1","addr":"172.21.15.110:6805","nonce":3235852628}]},"public_addr":"172.21.15.110:6801/3235852628","cluster_addr":"172.21.15.110:6803/3235852628","heartbeat_back_addr":"172.21.15.110:6807/3235852628","heartbeat_front_addr":"172.21.15.110:6805/3235852628","state":["exists","up"]},{"osd":2,"uuid":"d4977df1-b118-4d7e-b9a9-626a8accc812","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6800","nonce":169452122},{"type":"v1","addr":"172.21.15.114:6801","nonce":169452122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6802","nonce":169452122},{"type":"v1","addr":"172.21.15.114:6803","nonce":169452122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6806","nonce":169452122},{"type":"v1","addr":"172.21.15.114:6807","nonce":169452122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6804","nonce":169452122},{"type":"v1","addr":"172.21.15.114:6805","nonce":169452122}]},"public_addr":"172.21.15.114:6801/169452122","cluster_addr":"172.21.15.114:6803/169452122","heartbeat_back_addr":"172.21.15.114:6807/169452122","heartbeat_front_addr":"172.21.15.114:6805/169452122","state":["exists","up"]},{"osd":3,"uuid":"2c7ae1b4-0bf2-4954-81ed-55d03eab82c6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6810","nonce":3016980877},{"type":"v1","addr":"172.21.15.89:6811","nonce":3016980877}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6812","nonce":3016980877},{"type":"v1","addr":"172.21.15.89:6813","nonce":3016980877}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6816","nonce":3016980877},{"type":"v1","addr":"172.21.15.89:6817","nonce":3016980877}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6814","nonce":3016980877},{"type":"v1","addr":"172.21.15.89:6815","nonce":3016980877}]},"public_addr":"172.21.15.89:6811/3016980877","cluster_addr":"172.21.15.89:6813/3016980877","heartbeat_back_addr":"172.21.15.89:6817/3016980877","heartbeat_front_addr":"172.21.15.89:6815/3016980877","state":["exists","up"]},{"osd":4,"uuid":"8e65399d-6e55-4da4-98ba-f805f0c655a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":30,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6808","nonce":3217880476},{"type":"v1","addr":"172.21.15.110:6809","nonce":3217880476}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6810","nonce":3217880476},{"type":"v1","addr":"172.21.15.110:6811","nonce":3217880476}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6814","nonce":3217880476},{"type":"v1","addr":"172.21.15.110:6815","nonce":3217880476}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6812","nonce":3217880476},{"type":"v1","addr":"172.21.15.110:6813","nonce":3217880476}]},"public_addr":"172.21.15.110:6809/3217880476","cluster_addr":"172.21.15.110:6811/3217880476","heartbeat_back_addr":"172.21.15.110:6815/3217880476","heartbeat_front_addr":"172.21.15.110:6813/3217880476","state":["exists","up"]},{"osd":5,"uuid":"b9ec2889-46bd-40e5-8446-73ab28271c67","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6808","nonce":1144140707},{"type":"v1","addr":"172.21.15.114:6809","nonce":1144140707}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6810","nonce":1144140707},{"type":"v1","addr":"172.21.15.114:6811","nonce":1144140707}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6814","nonce":1144140707},{"type":"v1","addr":"172.21.15.114:6815","nonce":1144140707}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6812","nonce":1144140707},{"type":"v1","addr":"172.21.15.114:6813","nonce":1144140707}]},"public_addr":"172.21.15.114:6809/1144140707","cluster_addr":"172.21.15.114:6811/1144140707","heartbeat_back_addr":"172.21.15.114:6815/1144140707","heartbeat_front_addr":"172.21.15.114:6813/1144140707","state":["exists","up"]},{"osd":6,"uuid":"9b79989e-adb0-42b2-b2f6-a2dce56b66e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":41,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6818","nonce":1522586255},{"type":"v1","addr":"172.21.15.89:6819","nonce":1522586255}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6820","nonce":1522586255},{"type":"v1","addr":"172.21.15.89:6821","nonce":1522586255}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6824","nonce":1522586255},{"type":"v1","addr":"172.21.15.89:6825","nonce":1522586255}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6822","nonce":1522586255},{"type":"v1","addr":"172.21.15.89:6823","nonce":1522586255}]},"public_addr":"172.21.15.89:6819/1522586255","cluster_addr":"172.21.15.89:6821/1522586255","heartbeat_back_addr":"172.21.15.89:6825/1522586255","heartbeat_front_addr":"172.21.15.89:6823/1522586255","state":["exists","up"]},{"osd":7,"uuid":"b381821f-141c-47a1-be65-f57ca730391b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6816","nonce":1331818323},{"type":"v1","addr":"172.21.15.110:6817","nonce":1331818323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6818","nonce":1331818323},{"type":"v1","addr":"172.21.15.110:6819","nonce":1331818323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6822","nonce":1331818323},{"type":"v1","addr":"172.21.15.110:6823","nonce":1331818323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6820","nonce":1331818323},{"type":"v1","addr":"172.21.15.110:6821","nonce":1331818323}]},"public_addr":"172.21.15.110:6817/1331818323","cluster_addr":"172.21.15.110:6819/1331818323","heartbeat_back_addr":"172.21.15.110:6823/1331818323","heartbeat_front_addr":"172.21.15.110:6821/1331818323","state":["exists","up"]},{"osd":8,"uuid":"1bc2a5b4-15f3-456f-9d67-e51307a27bcc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6816","nonce":2138556699},{"type":"v1","addr":"172.21.15.114:6817","nonce":2138556699}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6818","nonce":2138556699},{"type":"v1","addr":"172.21.15.114:6819","nonce":2138556699}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6822","nonce":2138556699},{"type":"v1","addr":"172.21.15.114:6823","nonce":2138556699}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6820","nonce":2138556699},{"type":"v1","addr":"172.21.15.114:6821","nonce":2138556699}]},"public_addr":"172.21.15.114:6817/2138556699","cluster_addr":"172.21.15.114:6819/2138556699","heartbeat_back_addr":"172.21.15.114:6823/2138556699","heartbeat_front_addr":"172.21.15.114:6821/2138556699","state":["exists","up"]},{"osd":9,"uuid":"91f5fd29-d4a7-4800-ac7a-e83b13add89e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6826","nonce":1730253068},{"type":"v1","addr":"172.21.15.89:6827","nonce":1730253068}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6828","nonce":1730253068},{"type":"v1","addr":"172.21.15.89:6829","nonce":1730253068}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6832","nonce":1730253068},{"type":"v1","addr":"172.21.15.89:6833","nonce":1730253068}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6830","nonce":1730253068},{"type":"v1","addr":"172.21.15.89:6831","nonce":1730253068}]},"public_addr":"172.21.15.89:6827/1730253068","cluster_addr":"172.21.15.89:6829/1730253068","heartbeat_back_addr":"172.21.15.89:6833/1730253068","heartbeat_front_addr":"172.21.15.89:6831/1730253068","state":["exists","up"]},{"osd":10,"uuid":"01390499-2a4d-4188-b57a-fcb10c668e2c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6824","nonce":2904016117},{"type":"v1","addr":"172.21.15.110:6825","nonce":2904016117}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6826","nonce":2904016117},{"type":"v1","addr":"172.21.15.110:6827","nonce":2904016117}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6830","nonce":2904016117},{"type":"v1","addr":"172.21.15.110:6831","nonce":2904016117}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6828","nonce":2904016117},{"type":"v1","addr":"172.21.15.110:6829","nonce":2904016117}]},"public_addr":"172.21.15.110:6825/2904016117","cluster_addr":"172.21.15.110:6827/2904016117","heartbeat_back_addr":"172.21.15.110:6831/2904016117","heartbeat_front_addr":"172.21.15.110:6829/2904016117","state":["exists","up"]},{"osd":11,"uuid":"c3c0f38a-c3d3-415d-a9f8-875d5b756e35","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6824","nonce":3749854486},{"type":"v1","addr":"172.21.15.114:6825","nonce":3749854486}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6826","nonce":3749854486},{"type":"v1","addr":"172.21.15.114:6827","nonce":3749854486}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6830","nonce":3749854486},{"type":"v1","addr":"172.21.15.114:6831","nonce":3749854486}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6828","nonce":3749854486},{"type":"v1","addr":"172.21.15.114:6829","nonce":3749854486}]},"public_addr":"172.21.15.114:6825/3749854486","cluster_addr":"172.21.15.114:6827/3749854486","heartbeat_back_addr":"172.21.15.114:6831/3749854486","heartbeat_front_addr":"172.21.15.114:6829/3749854486","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:31:46.218908+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:32:10.338230+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:32:34.830596+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:33:00.816102+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:33:25.223838+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:33:50.188230+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:34:16.564255+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:34:41.498230+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:35:05.826322+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:35:31.234450+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:35:56.474373+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:36:21.741254+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.c","mappings":[{"from":2,"to":4}]},{"pgid":"2.e","mappings":[{"from":2,"to":1}]},{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]},{"pgid":"4.6","mappings":[{"from":9,"to":11}]},{"pgid":"4.12","mappings":[{"from":9,"to":10}]},{"pgid":"4.13","mappings":[{"from":9,"to":8}]},{"pgid":"4.1b","mappings":[{"from":5,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.89:6801/655110910":"2024-09-22T03:29:51.519106+0000","172.21.15.89:0/1618544983":"2024-09-22T03:29:51.519106+0000","172.21.15.89:0/1265502758":"2024-09-22T03:29:51.519106+0000","172.21.15.89:0/4017684419":"2024-09-22T03:29:51.519106+0000","172.21.15.89:6801/3312712420":"2024-09-22T03:29:26.004686+0000","172.21.15.89:6800/3312712420":"2024-09-22T03:29:26.004686+0000","172.21.15.89:0/2056576483":"2024-09-22T03:29:26.004686+0000","172.21.15.89:6800/655110910":"2024-09-22T03:29:51.519106+0000","172.21.15.89:0/2382866302":"2024-09-22T03:29:26.004686+0000","172.21.15.89:0/1584351801":"2024-09-22T03:29:26.004686+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T03:38:51.828 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:51.825+0000 7ff1d72f7640 1 -- 172.21.15.89:0/107549950 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7ff1bc0b6620 msgr2=0x7ff1bc0b8b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:51.828 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:51.825+0000 7ff1d72f7640 1 --2- 172.21.15.89:0/107549950 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7ff1bc0b6620 0x7ff1bc0b8b30 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7ff1c400bbe0 tx=0x7ff1c400b390 comp rx=0 tx=0).stop 2024-09-21T03:38:51.828 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:51.825+0000 7ff1d72f7640 1 -- 172.21.15.89:0/107549950 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff1d01094a0 msgr2=0x7ff1d019aa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:51.828 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:51.825+0000 7ff1d72f7640 1 --2- 172.21.15.89:0/107549950 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff1d01094a0 0x7ff1d019aa60 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7ff1cc00d750 tx=0x7ff1cc00dc10 comp rx=0 tx=0).stop 2024-09-21T03:38:51.828 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:51.826+0000 7ff1d72f7640 1 -- 172.21.15.89:0/107549950 shutdown_connections 2024-09-21T03:38:51.828 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:51.826+0000 7ff1d72f7640 1 --2- 172.21.15.89:0/107549950 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7ff1bc0b6620 0x7ff1bc0b8b30 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:51.828 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:51.826+0000 7ff1d72f7640 1 --2- 172.21.15.89:0/107549950 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff1d0109df0 0x7ff1d019fd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:51.828 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:51.826+0000 7ff1d72f7640 1 --2- 172.21.15.89:0/107549950 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff1d01094a0 0x7ff1d019aa60 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:51.828 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:51.826+0000 7ff1d72f7640 1 --2- 172.21.15.89:0/107549950 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff1d0103b60 0x7ff1d019a4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:51.828 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:51.826+0000 7ff1d72f7640 1 -- 172.21.15.89:0/107549950 >> 172.21.15.89:0/107549950 conn(0x7ff1d007b850 msgr2=0x7ff1d0107690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:38:51.829 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:51.826+0000 7ff1d72f7640 1 -- 172.21.15.89:0/107549950 shutdown_connections 2024-09-21T03:38:51.829 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:51.826+0000 7ff1d72f7640 1 -- 172.21.15.89:0/107549950 wait complete. 2024-09-21T03:38:51.960 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:38:51 smithi114 ceph-mon[34140]: pgmap v341: 129 pgs: 129 active+clean; 646 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-21T03:38:52.564 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 7c374098-77c9-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T03:38:52.811 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:38:52 smithi089 ceph-mon[30928]: pgmap v342: 129 pgs: 129 active+clean; 646 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-21T03:38:52.811 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:38:52 smithi089 ceph-mon[30928]: from='client.? 172.21.15.89:0/107549950' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T03:38:52.811 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:38:52 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [5, 9]} : dispatch 2024-09-21T03:38:52.816 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/7c374098-77c9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:38:52.961 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:38:52 smithi114 ceph-mon[34140]: pgmap v342: 129 pgs: 129 active+clean; 646 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-21T03:38:52.961 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:38:52 smithi114 ceph-mon[34140]: from='client.? 172.21.15.89:0/107549950' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T03:38:52.961 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:38:52 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [5, 9]} : dispatch 2024-09-21T03:38:53.000 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:38:52 smithi110 ceph-mon[34000]: pgmap v342: 129 pgs: 129 active+clean; 646 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-21T03:38:53.000 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:38:52 smithi110 ceph-mon[34000]: from='client.? 172.21.15.89:0/107549950' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T03:38:53.000 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:38:52 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [5, 9]} : dispatch 2024-09-21T03:38:53.314 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:53.293+0000 7effc91f6640 1 -- 172.21.15.89:0/705477854 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7effc41094a0 msgr2=0x7effc41098b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:53.315 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:53.293+0000 7effc91f6640 1 --2- 172.21.15.89:0/705477854 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7effc41094a0 0x7effc41098b0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7effb8009900 tx=0x7effb802a410 comp rx=0 tx=0).stop 2024-09-21T03:38:53.315 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:53.293+0000 7effc91f6640 1 -- 172.21.15.89:0/705477854 shutdown_connections 2024-09-21T03:38:53.315 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:53.293+0000 7effc91f6640 1 --2- 172.21.15.89:0/705477854 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7effc4109df0 0x7effc4110620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:53.315 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:53.293+0000 7effc91f6640 1 --2- 172.21.15.89:0/705477854 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7effc41094a0 0x7effc41098b0 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:53.315 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:53.293+0000 7effc91f6640 1 --2- 172.21.15.89:0/705477854 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7effc4103b60 0x7effc4103f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:53.315 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:53.293+0000 7effc91f6640 1 -- 172.21.15.89:0/705477854 >> 172.21.15.89:0/705477854 conn(0x7effc407b850 msgr2=0x7effc407bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:38:53.315 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:53.293+0000 7effc91f6640 1 -- 172.21.15.89:0/705477854 shutdown_connections 2024-09-21T03:38:53.315 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:53.293+0000 7effc91f6640 1 -- 172.21.15.89:0/705477854 wait complete. 2024-09-21T03:38:53.315 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:53.294+0000 7effc91f6640 1 Processor -- start 2024-09-21T03:38:53.315 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:53.294+0000 7effc91f6640 1 -- start start 2024-09-21T03:38:53.315 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:53.294+0000 7effc91f6640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7effc4103b60 0x7effc419a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:53.315 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:53.294+0000 7effc91f6640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7effc41094a0 0x7effc419aa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:53.316 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:53.294+0000 7effc91f6640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7effc4109df0 0x7effc419fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:53.316 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:53.294+0000 7effc91f6640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7effc41139b0 con 0x7effc4103b60 2024-09-21T03:38:53.316 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:53.294+0000 7effc91f6640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7effc4113830 con 0x7effc41094a0 2024-09-21T03:38:53.316 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:53.294+0000 7effc91f6640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7effc4113b30 con 0x7effc4109df0 2024-09-21T03:38:53.316 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:53.295+0000 7effc2d9d640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7effc4103b60 0x7effc419a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:53.316 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:53.295+0000 7effc2d9d640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7effc4103b60 0x7effc419a480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:55240/0 (socket says 172.21.15.89:55240) 2024-09-21T03:38:53.316 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:53.295+0000 7effc2d9d640 1 -- 172.21.15.89:0/3472647264 learned_addr learned my addr 172.21.15.89:0/3472647264 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-21T03:38:53.316 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:53.295+0000 7effc259c640 1 --2- 172.21.15.89:0/3472647264 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7effc41094a0 0x7effc419aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:53.316 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:53.295+0000 7effc359e640 1 --2- 172.21.15.89:0/3472647264 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7effc4109df0 0x7effc419fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:53.316 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:53.295+0000 7effc2d9d640 1 -- 172.21.15.89:0/3472647264 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7effc4109df0 msgr2=0x7effc419fcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:53.316 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:53.295+0000 7effc2d9d640 1 --2- 172.21.15.89:0/3472647264 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7effc4109df0 0x7effc419fcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:53.316 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:53.295+0000 7effc2d9d640 1 -- 172.21.15.89:0/3472647264 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7effc41094a0 msgr2=0x7effc419aa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:53.316 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:53.295+0000 7effc2d9d640 1 --2- 172.21.15.89:0/3472647264 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7effc41094a0 0x7effc419aa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:53.316 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:53.295+0000 7effc2d9d640 1 -- 172.21.15.89:0/3472647264 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7effc41a03f0 con 0x7effc4103b60 2024-09-21T03:38:53.316 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:53.295+0000 7effc2d9d640 1 --2- 172.21.15.89:0/3472647264 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7effc4103b60 0x7effc419a480 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7effb400d770 tx=0x7effb400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:38:53.317 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:53.295+0000 7effabfff640 1 -- 172.21.15.89:0/3472647264 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effb4017070 con 0x7effc4103b60 2024-09-21T03:38:53.317 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:53.295+0000 7effc91f6640 1 -- 172.21.15.89:0/3472647264 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7effc41a06e0 con 0x7effc4103b60 2024-09-21T03:38:53.317 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:53.295+0000 7effabfff640 1 -- 172.21.15.89:0/3472647264 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7effb40054b0 con 0x7effc4103b60 2024-09-21T03:38:53.317 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:53.295+0000 7effabfff640 1 -- 172.21.15.89:0/3472647264 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effb4004c60 con 0x7effc4103b60 2024-09-21T03:38:53.317 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:53.295+0000 7effc91f6640 1 -- 172.21.15.89:0/3472647264 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7effc41a6e40 con 0x7effc4103b60 2024-09-21T03:38:53.317 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:53.296+0000 7effc91f6640 1 -- 172.21.15.89:0/3472647264 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7effc4104de0 con 0x7effc4103b60 2024-09-21T03:38:53.317 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:53.298+0000 7effabfff640 1 -- 172.21.15.89:0/3472647264 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 18) ==== 154563+0+0 (secure 0 0 0) 0x7effb4004e00 con 0x7effc4103b60 2024-09-21T03:38:53.317 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:53.299+0000 7effabfff640 1 --2- 172.21.15.89:0/3472647264 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7effac0b65a0 0x7effac0b8ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:53.317 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:53.299+0000 7effc259c640 1 --2- 172.21.15.89:0/3472647264 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7effac0b65a0 0x7effac0b8ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:53.317 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:53.299+0000 7effabfff640 1 -- 172.21.15.89:0/3472647264 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 9121+0+0 (secure 0 0 0) 0x7effb40e7690 con 0x7effc4103b60 2024-09-21T03:38:53.317 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:53.300+0000 7effc259c640 1 --2- 172.21.15.89:0/3472647264 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7effac0b65a0 0x7effac0b8ab0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7effb802a930 tx=0x7effb8005a40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:38:53.317 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:53.302+0000 7effabfff640 1 -- 172.21.15.89:0/3472647264 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7effb40ada40 con 0x7effc4103b60 2024-09-21T03:38:53.520 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:53.518+0000 7effc91f6640 1 -- 172.21.15.89:0/3472647264 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7effc4103f70 con 0x7effc4103b60 2024-09-21T03:38:53.521 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:53.519+0000 7effabfff640 1 -- 172.21.15.89:0/3472647264 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11912 (secure 0 0 0) 0x7effc4103f70 con 0x7effc4103b60 2024-09-21T03:38:53.521 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-09-21T03:38:53.522 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":25,"btime":"2024-09-21T03:37:51:019398+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14721,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6835/2341538514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":2341538514},{"type":"v1","addr":"172.21.15.89:6835","nonce":2341538514}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14733,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6837/146708592","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6836","nonce":146708592},{"type":"v1","addr":"172.21.15.89:6837","nonce":146708592}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14781,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6839/2269965499","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6838","nonce":2269965499},{"type":"v1","addr":"172.21.15.89:6839","nonce":2269965499}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14793,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6841/650850383","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6840","nonce":650850383},{"type":"v1","addr":"172.21.15.89:6841","nonce":650850383}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24565,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6833/752336912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6832","nonce":752336912},{"type":"v1","addr":"172.21.15.114:6833","nonce":752336912}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24575,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6833/127607331","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6832","nonce":127607331},{"type":"v1","addr":"172.21.15.110:6833","nonce":127607331}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T03:37:13.570694+0000","modified":"2024-09-21T03:37:50.719106+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24638,"mds_1":24631,"mds_2":24614},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24593":{"gid":24593,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.110:6835/671820654","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6834","nonce":671820654},{"type":"v1","addr":"172.21.15.110:6835","nonce":671820654}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24604":{"gid":24604,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.114:6835/1049714548","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6834","nonce":1049714548},{"type":"v1","addr":"172.21.15.114:6835","nonce":1049714548}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24610":{"gid":24610,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.114:6837/1941213818","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6836","nonce":1941213818},{"type":"v1","addr":"172.21.15.114:6837","nonce":1941213818}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24614":{"gid":24614,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":15,"addr":"172.21.15.110:6837/4047124589","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6836","nonce":4047124589},{"type":"v1","addr":"172.21.15.110:6837","nonce":4047124589}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24631":{"gid":24631,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":15,"addr":"172.21.15.114:6839/3427369944","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6838","nonce":3427369944},{"type":"v1","addr":"172.21.15.114:6839","nonce":3427369944}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24638":{"gid":24638,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.110:6839/132366709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6838","nonce":132366709},{"type":"v1","addr":"172.21.15.110:6839","nonce":132366709}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24638,"qdb_cluster":[24614,24631,24638]},"id":1}]} 2024-09-21T03:38:53.522 INFO:teuthology.orchestra.run.smithi089.stderr:dumped fsmap epoch 25 2024-09-21T03:38:53.523 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:53.521+0000 7effc91f6640 1 -- 172.21.15.89:0/3472647264 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7effac0b65a0 msgr2=0x7effac0b8ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:53.524 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:53.521+0000 7effc91f6640 1 --2- 172.21.15.89:0/3472647264 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7effac0b65a0 0x7effac0b8ab0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7effb802a930 tx=0x7effb8005a40 comp rx=0 tx=0).stop 2024-09-21T03:38:53.524 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:53.521+0000 7effc91f6640 1 -- 172.21.15.89:0/3472647264 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7effc4103b60 msgr2=0x7effc419a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:53.524 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:53.521+0000 7effc91f6640 1 --2- 172.21.15.89:0/3472647264 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7effc4103b60 0x7effc419a480 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7effb400d770 tx=0x7effb400dc30 comp rx=0 tx=0).stop 2024-09-21T03:38:53.524 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:53.521+0000 7effc91f6640 1 -- 172.21.15.89:0/3472647264 shutdown_connections 2024-09-21T03:38:53.524 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:53.521+0000 7effc91f6640 1 --2- 172.21.15.89:0/3472647264 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7effac0b65a0 0x7effac0b8ab0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:53.524 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:53.521+0000 7effc91f6640 1 --2- 172.21.15.89:0/3472647264 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7effc41094a0 0x7effc419aa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:53.524 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:53.521+0000 7effc91f6640 1 --2- 172.21.15.89:0/3472647264 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7effc4109df0 0x7effc419fcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:53.524 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:53.521+0000 7effc91f6640 1 --2- 172.21.15.89:0/3472647264 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7effc4103b60 0x7effc419a480 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:53.524 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:53.521+0000 7effc91f6640 1 -- 172.21.15.89:0/3472647264 >> 172.21.15.89:0/3472647264 conn(0x7effc407b850 msgr2=0x7effc4107640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:38:53.524 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:53.521+0000 7effc91f6640 1 -- 172.21.15.89:0/3472647264 shutdown_connections 2024-09-21T03:38:53.524 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:53.521+0000 7effc91f6640 1 -- 172.21.15.89:0/3472647264 wait complete. 2024-09-21T03:38:53.811 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:38:53 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [5, 9]}]': finished 2024-09-21T03:38:53.811 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:38:53 smithi089 ceph-mon[30928]: osdmap e83: 12 total, 12 up, 12 in 2024-09-21T03:38:53.811 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:38:53 smithi089 ceph-mon[30928]: from='client.? 172.21.15.89:0/3472647264' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:38:53.961 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:38:53 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [5, 9]}]': finished 2024-09-21T03:38:53.961 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:38:53 smithi114 ceph-mon[34140]: osdmap e83: 12 total, 12 up, 12 in 2024-09-21T03:38:53.961 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:38:53 smithi114 ceph-mon[34140]: from='client.? 172.21.15.89:0/3472647264' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:38:54.000 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:38:53 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [5, 9]}]': finished 2024-09-21T03:38:54.000 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:38:53 smithi110 ceph-mon[34000]: osdmap e83: 12 total, 12 up, 12 in 2024-09-21T03:38:54.000 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:38:53 smithi110 ceph-mon[34000]: from='client.? 172.21.15.89:0/3472647264' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:38:54.122 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 25, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-09-21T03:37:13.570694+0000', 'modified': '2024-09-21T03:37:50.719106+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 24638, 'mds_1': 24631, 'mds_2': 24614}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24593': {'gid': 24593, 'name': 'e', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.110:6835/671820654', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.110:6834', 'nonce': 671820654}, {'type': 'v1', 'addr': '172.21.15.110:6835', 'nonce': 671820654}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24604': {'gid': 24604, 'name': 'f', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.114:6835/1049714548', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.114:6834', 'nonce': 1049714548}, {'type': 'v1', 'addr': '172.21.15.114:6835', 'nonce': 1049714548}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24610': {'gid': 24610, 'name': 'c', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.114:6837/1941213818', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.114:6836', 'nonce': 1941213818}, {'type': 'v1', 'addr': '172.21.15.114:6837', 'nonce': 1941213818}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24614': {'gid': 24614, 'name': 'h', 'rank': 2, 'incarnation': 22, 'state': 'up:active', 'state_seq': 15, 'addr': '172.21.15.110:6837/4047124589', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.110:6836', 'nonce': 4047124589}, {'type': 'v1', 'addr': '172.21.15.110:6837', 'nonce': 4047124589}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24631': {'gid': 24631, 'name': 'i', 'rank': 1, 'incarnation': 20, 'state': 'up:active', 'state_seq': 15, 'addr': '172.21.15.114:6839/3427369944', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.114:6838', 'nonce': 3427369944}, {'type': 'v1', 'addr': '172.21.15.114:6839', 'nonce': 3427369944}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24638': {'gid': 24638, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.110:6839/132366709', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.110:6838', 'nonce': 132366709}, {'type': 'v1', 'addr': '172.21.15.110:6839', 'nonce': 132366709}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24638, 'qdb_cluster': [24614, 24631, 24638]} 2024-09-21T03:38:54.123 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2024-09-21T03:38:54.123 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 7c374098-77c9-11ef-baf5-efdc52797490 -- ceph tell mds.h status 2024-09-21T03:38:54.368 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/7c374098-77c9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:38:54.811 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:38:54 smithi089 ceph-mon[30928]: pgmap v344: 129 pgs: 1 peering, 128 active+clean; 646 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-09-21T03:38:54.811 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:38:54 smithi089 ceph-mon[30928]: Health check failed: Reduced data availability: 1 pg inactive, 1 pg peering (PG_AVAILABILITY) 2024-09-21T03:38:54.811 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:38:54 smithi089 ceph-mon[30928]: osdmap e84: 12 total, 12 up, 12 in 2024-09-21T03:38:54.918 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:54.916+0000 7ff491f41640 1 -- 172.21.15.89:0/435513385 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff48c102a60 msgr2=0x7ff48c102e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:54.919 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:54.916+0000 7ff491f41640 1 --2- 172.21.15.89:0/435513385 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff48c102a60 0x7ff48c102e70 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7ff47c009900 tx=0x7ff47c02a410 comp rx=0 tx=0).stop 2024-09-21T03:38:54.919 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:54.916+0000 7ff491f41640 1 -- 172.21.15.89:0/435513385 shutdown_connections 2024-09-21T03:38:54.919 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:54.916+0000 7ff491f41640 1 --2- 172.21.15.89:0/435513385 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff48c108cf0 0x7ff48c10f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:54.919 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:54.916+0000 7ff491f41640 1 --2- 172.21.15.89:0/435513385 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff48c1083a0 0x7ff48c1087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:54.919 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:54.916+0000 7ff491f41640 1 --2- 172.21.15.89:0/435513385 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff48c102a60 0x7ff48c102e70 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:54.919 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:54.916+0000 7ff491f41640 1 -- 172.21.15.89:0/435513385 >> 172.21.15.89:0/435513385 conn(0x7ff48c079730 msgr2=0x7ff48c079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:38:54.919 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:54.916+0000 7ff491f41640 1 -- 172.21.15.89:0/435513385 shutdown_connections 2024-09-21T03:38:54.919 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:54.916+0000 7ff491f41640 1 -- 172.21.15.89:0/435513385 wait complete. 2024-09-21T03:38:54.921 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:54.917+0000 7ff491f41640 1 Processor -- start 2024-09-21T03:38:54.921 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:54.917+0000 7ff491f41640 1 -- start start 2024-09-21T03:38:54.921 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:54.917+0000 7ff491f41640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff48c102a60 0x7ff48c199280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:54.921 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:54.917+0000 7ff491f41640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff48c1083a0 0x7ff48c199820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:54.921 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:54.917+0000 7ff491f41640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff48c108cf0 0x7ff48c19eac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:54.921 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:54.917+0000 7ff491f41640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7ff48c112810 con 0x7ff48c108cf0 2024-09-21T03:38:54.921 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:54.917+0000 7ff491f41640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7ff48c112690 con 0x7ff48c1083a0 2024-09-21T03:38:54.921 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:54.917+0000 7ff491f41640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7ff48c112990 con 0x7ff48c102a60 2024-09-21T03:38:54.921 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:54.917+0000 7ff48bfff640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff48c108cf0 0x7ff48c19eac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:54.921 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:54.917+0000 7ff48bfff640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff48c108cf0 0x7ff48c19eac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:55258/0 (socket says 172.21.15.89:55258) 2024-09-21T03:38:54.921 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:54.917+0000 7ff48bfff640 1 -- 172.21.15.89:0/2275350451 learned_addr learned my addr 172.21.15.89:0/2275350451 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-21T03:38:54.922 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:54.918+0000 7ff48affd640 1 --2- 172.21.15.89:0/2275350451 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff48c1083a0 0x7ff48c199820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:54.922 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:54.918+0000 7ff48b7fe640 1 --2- 172.21.15.89:0/2275350451 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff48c102a60 0x7ff48c199280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:54.922 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:54.918+0000 7ff48bfff640 1 -- 172.21.15.89:0/2275350451 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff48c102a60 msgr2=0x7ff48c199280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:54.922 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:54.918+0000 7ff48bfff640 1 --2- 172.21.15.89:0/2275350451 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff48c102a60 0x7ff48c199280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:54.922 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:54.918+0000 7ff48bfff640 1 -- 172.21.15.89:0/2275350451 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff48c1083a0 msgr2=0x7ff48c199820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:54.922 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:54.918+0000 7ff48bfff640 1 --2- 172.21.15.89:0/2275350451 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff48c1083a0 0x7ff48c199820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:54.922 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:54.918+0000 7ff48bfff640 1 -- 172.21.15.89:0/2275350451 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff48c19f1f0 con 0x7ff48c108cf0 2024-09-21T03:38:54.922 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:54.918+0000 7ff48bfff640 1 --2- 172.21.15.89:0/2275350451 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff48c108cf0 0x7ff48c19eac0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7ff48400eb10 tx=0x7ff48400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:38:54.922 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:54.918+0000 7ff488ff9640 1 -- 172.21.15.89:0/2275350451 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff484004b20 con 0x7ff48c108cf0 2024-09-21T03:38:54.922 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:54.918+0000 7ff491f41640 1 -- 172.21.15.89:0/2275350451 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff48c19f4e0 con 0x7ff48c108cf0 2024-09-21T03:38:54.922 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:54.918+0000 7ff488ff9640 1 -- 172.21.15.89:0/2275350451 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff484004cc0 con 0x7ff48c108cf0 2024-09-21T03:38:54.922 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:54.918+0000 7ff488ff9640 1 -- 172.21.15.89:0/2275350451 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff484010690 con 0x7ff48c108cf0 2024-09-21T03:38:54.922 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:54.918+0000 7ff491f41640 1 -- 172.21.15.89:0/2275350451 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff48c1a5c40 con 0x7ff48c108cf0 2024-09-21T03:38:54.923 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:54.921+0000 7ff488ff9640 1 -- 172.21.15.89:0/2275350451 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 18) ==== 154563+0+0 (secure 0 0 0) 0x7ff484005020 con 0x7ff48c108cf0 2024-09-21T03:38:54.923 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:54.921+0000 7ff4984b5740 1 Processor -- start 2024-09-21T03:38:54.923 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:54.921+0000 7ff4984b5740 1 -- start start 2024-09-21T03:38:54.925 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:54.921+0000 7ff4984b5740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556028516f60 0x5560288bcf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:54.925 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:54.921+0000 7ff4984b5740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5560284ee0a0 0x5560288bd440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:54.925 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:54.921+0000 7ff4984b5740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5560284f6280 0x5560288bd980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:54.925 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:54.921+0000 7ff4984b5740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x556028834c10 con 0x5560284ee0a0 2024-09-21T03:38:54.925 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:54.921+0000 7ff4984b5740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x556028824040 con 0x556028516f60 2024-09-21T03:38:54.925 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:54.921+0000 7ff4984b5740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x556028827e50 con 0x5560284f6280 2024-09-21T03:38:54.925 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:54.921+0000 7ff48b7fe640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5560284ee0a0 0x5560288bd440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:54.925 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:54.922+0000 7ff48b7fe640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5560284ee0a0 0x5560288bd440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:55270/0 (socket says 172.21.15.89:55270) 2024-09-21T03:38:54.925 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:54.922+0000 7ff48b7fe640 1 -- 172.21.15.89:0/3308368329 learned_addr learned my addr 172.21.15.89:0/3308368329 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-21T03:38:54.925 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:54.922+0000 7ff48affd640 1 --2- 172.21.15.89:0/3308368329 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556028516f60 0x5560288bcf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:54.925 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:54.922+0000 7ff48affd640 1 --2- 172.21.15.89:0/3308368329 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5560284f6280 0x5560288bd980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:54.925 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:54.922+0000 7ff48b7fe640 1 -- 172.21.15.89:0/3308368329 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5560284f6280 msgr2=0x5560288bd980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:38:54.925 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:54.922+0000 7ff48b7fe640 1 --2- 172.21.15.89:0/3308368329 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5560284f6280 0x5560288bd980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:54.925 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:54.922+0000 7ff48b7fe640 1 -- 172.21.15.89:0/3308368329 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556028516f60 msgr2=0x5560288bcf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:38:54.926 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:54.922+0000 7ff48b7fe640 1 --2- 172.21.15.89:0/3308368329 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556028516f60 0x5560288bcf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:54.926 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:54.922+0000 7ff48b7fe640 1 -- 172.21.15.89:0/3308368329 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556028582540 con 0x5560284ee0a0 2024-09-21T03:38:54.926 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:54.922+0000 7ff48affd640 1 --2- 172.21.15.89:0/3308368329 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556028516f60 0x5560288bcf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:38:54.926 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:54.922+0000 7ff488ff9640 1 --2- 172.21.15.89:0/2275350451 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7ff4740b65a0 0x7ff4740b8ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:54.926 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:54.922+0000 7ff48affd640 1 --2- 172.21.15.89:0/2275350451 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7ff4740b65a0 0x7ff4740b8ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:54.926 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:54.922+0000 7ff48b7fe640 1 --2- 172.21.15.89:0/3308368329 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5560284ee0a0 0x5560288bd440 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7ff47c02a930 tx=0x7ff47c031af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c78f07b6604db9cb server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:38:54.926 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:54.922+0000 7ff488ff9640 1 -- 172.21.15.89:0/2275350451 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 9121+0+0 (secure 0 0 0) 0x7ff4840e7e10 con 0x7ff48c108cf0 2024-09-21T03:38:54.926 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:54.922+0000 7ff471ffb640 1 -- 172.21.15.89:0/3308368329 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff47c035030 con 0x5560284ee0a0 2024-09-21T03:38:54.926 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:54.922+0000 7ff471ffb640 1 -- 172.21.15.89:0/3308368329 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff47c040070 con 0x5560284ee0a0 2024-09-21T03:38:54.926 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:54.922+0000 7ff471ffb640 1 -- 172.21.15.89:0/3308368329 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff47c0038c0 con 0x5560284ee0a0 2024-09-21T03:38:54.929 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:54.923+0000 7ff4984b5740 1 -- 172.21.15.89:0/3308368329 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5560284ee0a0 msgr2=0x5560288bd440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:54.929 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:54.923+0000 7ff4984b5740 1 --2- 172.21.15.89:0/3308368329 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5560284ee0a0 0x5560288bd440 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7ff47c02a930 tx=0x7ff47c031af0 comp rx=0 tx=0).stop 2024-09-21T03:38:54.929 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:54.923+0000 7ff48affd640 1 --2- 172.21.15.89:0/2275350451 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7ff4740b65a0 0x7ff4740b8ab0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7ff480005d00 tx=0x7ff480014040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:38:54.929 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:54.923+0000 7ff4984b5740 1 -- 172.21.15.89:0/3308368329 shutdown_connections 2024-09-21T03:38:54.929 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:54.923+0000 7ff4984b5740 1 --2- 172.21.15.89:0/3308368329 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5560284f6280 0x5560288bd980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:54.929 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:54.923+0000 7ff4984b5740 1 --2- 172.21.15.89:0/3308368329 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5560284ee0a0 0x5560288bd440 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:54.929 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:54.923+0000 7ff4984b5740 1 --2- 172.21.15.89:0/3308368329 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556028516f60 0x5560288bcf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:54.929 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:54.923+0000 7ff4984b5740 1 -- 172.21.15.89:0/3308368329 >> 172.21.15.89:0/3308368329 conn(0x55602850b350 msgr2=0x5560288badf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:38:54.929 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:54.923+0000 7ff4984b5740 1 -- 172.21.15.89:0/3308368329 shutdown_connections 2024-09-21T03:38:54.930 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:54.923+0000 7ff4984b5740 1 -- 172.21.15.89:0/3308368329 wait complete. 2024-09-21T03:38:54.930 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:54.924+0000 7ff4984b5740 1 -- start start 2024-09-21T03:38:54.930 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:54.924+0000 7ff4984b5740 1 Processor -- start 2024-09-21T03:38:54.930 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:54.924+0000 7ff4984b5740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5560284ee0a0 0x5560288c8a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:54.930 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:54.924+0000 7ff4984b5740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5560284f6280 0x5560288c8f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:54.930 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:54.924+0000 7ff4984b5740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556028516f60 0x5560288d4970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:54.930 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:54.924+0000 7ff4984b5740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x5560286855f0 con 0x556028516f60 2024-09-21T03:38:54.930 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:54.924+0000 7ff4984b5740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x5560287bf010 con 0x5560284f6280 2024-09-21T03:38:54.930 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:54.924+0000 7ff4984b5740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x5560288cb680 con 0x5560284ee0a0 2024-09-21T03:38:54.930 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:54.924+0000 7ff4984b5740 10 client.0 _ll_register_callbacks cb 0x55602881e340 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:38:54.930 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:54.924+0000 7ff48bfff640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556028516f60 0x5560288d4970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:54.930 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:54.925+0000 7ff48bfff640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556028516f60 0x5560288d4970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:55286/0 (socket says 172.21.15.89:55286) 2024-09-21T03:38:54.930 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:54.925+0000 7ff48bfff640 1 -- 172.21.15.89:0/277124418 learned_addr learned my addr 172.21.15.89:0/277124418 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-21T03:38:54.931 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:54.925+0000 7ff48affd640 1 --2- 172.21.15.89:0/277124418 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5560284f6280 0x5560288c8f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:54.931 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:54.925+0000 7ff48b7fe640 1 --2- 172.21.15.89:0/277124418 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5560284ee0a0 0x5560288c8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:54.931 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:54.925+0000 7ff48bfff640 1 -- 172.21.15.89:0/277124418 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5560284ee0a0 msgr2=0x5560288c8a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:38:54.931 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:54.925+0000 7ff48bfff640 1 --2- 172.21.15.89:0/277124418 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5560284ee0a0 0x5560288c8a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:54.931 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:54.925+0000 7ff48bfff640 1 -- 172.21.15.89:0/277124418 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5560284f6280 msgr2=0x5560288c8f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:38:54.931 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:54.925+0000 7ff48bfff640 1 --2- 172.21.15.89:0/277124418 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5560284f6280 0x5560288c8f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:54.931 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:54.925+0000 7ff48bfff640 1 -- 172.21.15.89:0/277124418 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5560287eca00 con 0x556028516f60 2024-09-21T03:38:54.931 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:54.925+0000 7ff48b7fe640 1 --2- 172.21.15.89:0/277124418 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5560284ee0a0 0x5560288c8a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:38:54.931 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:54.925+0000 7ff48bfff640 1 --2- 172.21.15.89:0/277124418 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556028516f60 0x5560288d4970 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7ff4840057c0 tx=0x7ff4840ad950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dbd2211391ae3dc1 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:38:54.931 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:54.925+0000 7ff46affd640 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2024-09-21T03:38:54.931 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:54.925+0000 7ff46affd640 1 -- 172.21.15.89:0/277124418 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4840b2070 con 0x556028516f60 2024-09-21T03:38:54.932 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:54.925+0000 7ff46affd640 1 -- 172.21.15.89:0/277124418 <== mon.0 v2:172.21.15.89:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 9121+0+0 (secure 0 0 0) 0x7ff4840b7050 con 0x556028516f60 2024-09-21T03:38:54.932 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:54.925+0000 7ff4984b5740 1 -- 172.21.15.89:0/277124418 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff47c040070 con 0x556028516f60 2024-09-21T03:38:54.932 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:54.925+0000 7ff46affd640 1 -- 172.21.15.89:0/277124418 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7ff4840ae1c0 con 0x556028516f60 2024-09-21T03:38:54.932 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:54.925+0000 7ff4984b5740 10 client.15255 fetch_fsmap learned FSMap version 25 2024-09-21T03:38:54.932 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:54.925+0000 7ff4984b5740 1 -- 172.21.15.89:0/277124418 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5560288d75c0 con 0x556028516f60 2024-09-21T03:38:54.932 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:54.926+0000 7ff46affd640 1 -- 172.21.15.89:0/277124418 <== mon.0 v2:172.21.15.89:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff47c040070 con 0x556028516f60 2024-09-21T03:38:54.932 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:54.926+0000 7ff46affd640 1 -- 172.21.15.89:0/277124418 <== mon.0 v2:172.21.15.89:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7ff4840a9410 con 0x556028516f60 2024-09-21T03:38:54.932 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:54.926+0000 7ff4984b5740 10 client.15255 fetch_fsmap finished waiting for FSMap version 25 2024-09-21T03:38:54.932 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:54.926+0000 7ff4984b5740 10 client.15255 resolve_mds: resolved name 'h' to daemon mds.h 2024-09-21T03:38:54.932 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:54.926+0000 7ff4984b5740 20 client.15255 populate_metadata read hostname 'smithi089' 2024-09-21T03:38:54.932 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:54.926+0000 7ff4984b5740 1 --2- 172.21.15.89:0/277124418 >> [v2:172.21.15.110:6836/4047124589,v1:172.21.15.110:6837/4047124589] conn(0x5560284d58e0 0x5560288db060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:54.932 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:54.926+0000 7ff4984b5740 4 client.15255 mds_command: new command op to 24614 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:38:54.932 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:54.926+0000 7ff4984b5740 1 -- 172.21.15.89:0/277124418 --> [v2:172.21.15.110:6836/4047124589,v1:172.21.15.110:6837/4047124589] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556028836b60 con 0x5560284d58e0 2024-09-21T03:38:54.932 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:54.926+0000 7ff48b7fe640 1 --2- 172.21.15.89:0/277124418 >> [v2:172.21.15.110:6836/4047124589,v1:172.21.15.110:6837/4047124589] conn(0x5560284d58e0 0x5560288db060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:54.933 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:54.927+0000 7ff46affd640 10 client.15255 ms_handle_connect on v2:172.21.15.110:6836/4047124589 2024-09-21T03:38:54.933 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:54.927+0000 7ff48b7fe640 1 --2- 172.21.15.89:0/277124418 >> [v2:172.21.15.110:6836/4047124589,v1:172.21.15.110:6837/4047124589] conn(0x5560284d58e0 0x5560288db060 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=3ee5d346498ce9b2 server_cookie=488895dadc2d3d4 in_seq=0 out_seq=0 2024-09-21T03:38:54.934 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:54.927+0000 7ff46affd640 1 -- 172.21.15.89:0/277124418 <== mds.2 v2:172.21.15.110:6836/4047124589 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x556028836b60 con 0x5560284d58e0 2024-09-21T03:38:54.934 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:54.927+0000 7ff46affd640 10 client.15255 handle_command_reply: tid=0 2024-09-21T03:38:54.934 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:54.928+0000 7ff4984b5740 1 client.15255 shutdown 2024-09-21T03:38:54.934 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:54.928+0000 7ff4984b5740 1 -- 172.21.15.89:0/277124418 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556028516f60 msgr2=0x5560288d4970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:54.934 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:54.928+0000 7ff4984b5740 1 --2- 172.21.15.89:0/277124418 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556028516f60 0x5560288d4970 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7ff4840057c0 tx=0x7ff4840ad950 comp rx=0 tx=0).stop 2024-09-21T03:38:54.934 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:54.928+0000 7ff4984b5740 1 -- 172.21.15.89:0/277124418 shutdown_connections 2024-09-21T03:38:54.934 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:54.928+0000 7ff4984b5740 1 --2- 172.21.15.89:0/277124418 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5560284f6280 0x5560288c8f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:54.934 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:54.928+0000 7ff4984b5740 1 --2- 172.21.15.89:0/277124418 >> [v2:172.21.15.110:6836/4047124589,v1:172.21.15.110:6837/4047124589] conn(0x5560284d58e0 0x5560288db060 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:54.934 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:54.928+0000 7ff4984b5740 1 --2- 172.21.15.89:0/277124418 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556028516f60 0x5560288d4970 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:54.934 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:54.928+0000 7ff4984b5740 1 --2- 172.21.15.89:0/277124418 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5560284ee0a0 0x5560288c8a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:54.934 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:54.928+0000 7ff4984b5740 1 -- 172.21.15.89:0/277124418 >> 172.21.15.89:0/277124418 conn(0x55602850b350 msgr2=0x5560288c00e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:38:54.934 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:54.928+0000 7ff46affd640 0 client.15255 ms_handle_reset on v2:172.21.15.110:6836/4047124589 2024-09-21T03:38:54.934 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:54.928+0000 7ff4984b5740 1 -- 172.21.15.89:0/277124418 shutdown_connections 2024-09-21T03:38:54.935 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:54.928+0000 7ff4984b5740 1 -- 172.21.15.89:0/277124418 wait complete. 2024-09-21T03:38:54.935 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:54.928+0000 7ff4984b5740 20 client.15255 trim_cache size 0 max 16384 2024-09-21T03:38:54.949 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:54.946+0000 7ff4984b5740 1 Processor -- start 2024-09-21T03:38:54.949 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:54.946+0000 7ff4984b5740 1 -- start start 2024-09-21T03:38:54.949 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:54.946+0000 7ff4984b5740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5560284ee0a0 0x5560288c3020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:54.949 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:54.946+0000 7ff4984b5740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5560284f6280 0x5560288d48e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:54.949 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:54.946+0000 7ff4984b5740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556028516f60 0x5560288d4e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:54.949 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:54.946+0000 7ff4984b5740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x5560288d93a0 con 0x5560284f6280 2024-09-21T03:38:54.949 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:54.946+0000 7ff4984b5740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x5560288d9a70 con 0x5560284ee0a0 2024-09-21T03:38:54.950 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:54.946+0000 7ff4984b5740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x5560288da140 con 0x556028516f60 2024-09-21T03:38:54.950 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:54.946+0000 7ff48affd640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5560284f6280 0x5560288d48e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:54.950 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:54.946+0000 7ff48affd640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5560284f6280 0x5560288d48e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:55294/0 (socket says 172.21.15.89:55294) 2024-09-21T03:38:54.950 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:54.946+0000 7ff48affd640 1 -- 172.21.15.89:0/3521423125 learned_addr learned my addr 172.21.15.89:0/3521423125 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-21T03:38:54.950 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:54.946+0000 7ff48b7fe640 1 --2- 172.21.15.89:0/3521423125 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5560284ee0a0 0x5560288c3020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:54.950 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:54.947+0000 7ff48bfff640 1 --2- 172.21.15.89:0/3521423125 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556028516f60 0x5560288d4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:54.950 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:54.947+0000 7ff48affd640 1 -- 172.21.15.89:0/3521423125 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556028516f60 msgr2=0x5560288d4e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:38:54.950 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:54.947+0000 7ff48affd640 1 --2- 172.21.15.89:0/3521423125 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556028516f60 0x5560288d4e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:54.950 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:54.947+0000 7ff48affd640 1 -- 172.21.15.89:0/3521423125 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5560284ee0a0 msgr2=0x5560288c3020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:38:54.950 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:54.947+0000 7ff48affd640 1 --2- 172.21.15.89:0/3521423125 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5560284ee0a0 0x5560288c3020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:54.950 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:54.947+0000 7ff48affd640 1 -- 172.21.15.89:0/3521423125 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5560288b5080 con 0x5560284f6280 2024-09-21T03:38:54.950 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:54.947+0000 7ff48b7fe640 1 --2- 172.21.15.89:0/3521423125 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5560284ee0a0 0x5560288c3020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:38:54.950 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:54.947+0000 7ff48affd640 1 --2- 172.21.15.89:0/3521423125 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5560284f6280 0x5560288d48e0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7ff4800043b0 tx=0x7ff4800046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11f137b41f3b8a16 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:38:54.950 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:54.947+0000 7ff4717fa640 1 -- 172.21.15.89:0/3521423125 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff480018030 con 0x5560284f6280 2024-09-21T03:38:54.951 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:54.947+0000 7ff4717fa640 1 -- 172.21.15.89:0/3521423125 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff48001a070 con 0x5560284f6280 2024-09-21T03:38:54.951 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:54.947+0000 7ff4717fa640 1 -- 172.21.15.89:0/3521423125 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4800206a0 con 0x5560284f6280 2024-09-21T03:38:54.954 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:54.948+0000 7ff4984b5740 1 -- 172.21.15.89:0/3521423125 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5560284f6280 msgr2=0x5560288d48e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:54.954 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:54.948+0000 7ff4984b5740 1 --2- 172.21.15.89:0/3521423125 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5560284f6280 0x5560288d48e0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7ff4800043b0 tx=0x7ff4800046c0 comp rx=0 tx=0).stop 2024-09-21T03:38:54.954 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:54.948+0000 7ff4984b5740 1 -- 172.21.15.89:0/3521423125 shutdown_connections 2024-09-21T03:38:54.954 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:54.948+0000 7ff4984b5740 1 --2- 172.21.15.89:0/3521423125 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556028516f60 0x5560288d4e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:54.955 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:54.948+0000 7ff4984b5740 1 --2- 172.21.15.89:0/3521423125 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5560284f6280 0x5560288d48e0 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:54.955 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:54.948+0000 7ff4984b5740 1 --2- 172.21.15.89:0/3521423125 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5560284ee0a0 0x5560288c3020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:54.955 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:54.948+0000 7ff4984b5740 1 -- 172.21.15.89:0/3521423125 >> 172.21.15.89:0/3521423125 conn(0x55602850b350 msgr2=0x5560288c39d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:38:54.955 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:54.948+0000 7ff4984b5740 1 -- 172.21.15.89:0/3521423125 shutdown_connections 2024-09-21T03:38:54.955 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:54.948+0000 7ff4984b5740 1 -- 172.21.15.89:0/3521423125 wait complete. 2024-09-21T03:38:54.955 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:54.948+0000 7ff4984b5740 1 -- start start 2024-09-21T03:38:54.955 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:54.948+0000 7ff4984b5740 1 Processor -- start 2024-09-21T03:38:54.955 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:54.949+0000 7ff4984b5740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5560284ee0a0 0x5560288e3cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:54.955 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:54.949+0000 7ff4984b5740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5560284f6280 0x5560288c58f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:54.955 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:54.949+0000 7ff4984b5740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556028516f60 0x5560288c6000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:54.955 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:54.949+0000 7ff4984b5740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x5560288daee0 con 0x5560284ee0a0 2024-09-21T03:38:54.955 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:54.949+0000 7ff4984b5740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x5560288da810 con 0x5560284f6280 2024-09-21T03:38:54.955 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:54.949+0000 7ff4984b5740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x5560288db5b0 con 0x556028516f60 2024-09-21T03:38:54.955 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:54.949+0000 7ff4984b5740 10 client.0 _ll_register_callbacks cb 0x7ff484005710 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:38:54.956 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:54.949+0000 7ff48b7fe640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5560284ee0a0 0x5560288e3cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:54.956 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:54.949+0000 7ff48b7fe640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5560284ee0a0 0x5560288e3cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:55302/0 (socket says 172.21.15.89:55302) 2024-09-21T03:38:54.956 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:54.949+0000 7ff48b7fe640 1 -- 172.21.15.89:0/1372257566 learned_addr learned my addr 172.21.15.89:0/1372257566 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-21T03:38:54.956 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:54.949+0000 7ff48bfff640 1 --2- 172.21.15.89:0/1372257566 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556028516f60 0x5560288c6000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:54.956 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:54.949+0000 7ff48affd640 1 --2- 172.21.15.89:0/1372257566 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5560284f6280 0x5560288c58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:54.956 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:54.949+0000 7ff48b7fe640 1 -- 172.21.15.89:0/1372257566 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556028516f60 msgr2=0x5560288c6000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:38:54.956 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:54.949+0000 7ff48b7fe640 1 --2- 172.21.15.89:0/1372257566 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556028516f60 0x5560288c6000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:54.956 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:54.949+0000 7ff48b7fe640 1 -- 172.21.15.89:0/1372257566 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5560284f6280 msgr2=0x5560288c58f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:38:54.956 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:54.949+0000 7ff48b7fe640 1 --2- 172.21.15.89:0/1372257566 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5560284f6280 0x5560288c58f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:54.956 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:54.949+0000 7ff48b7fe640 1 -- 172.21.15.89:0/1372257566 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5560288c93c0 con 0x5560284ee0a0 2024-09-21T03:38:54.956 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:54.949+0000 7ff48affd640 1 --2- 172.21.15.89:0/1372257566 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5560284f6280 0x5560288c58f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:38:54.956 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:54.949+0000 7ff48b7fe640 1 --2- 172.21.15.89:0/1372257566 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5560284ee0a0 0x5560288e3cb0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7ff47c005f70 tx=0x7ff47c02aba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2cea975a07437f63 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:38:54.956 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:54.949+0000 7ff46affd640 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2024-09-21T03:38:54.956 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:54.950+0000 7ff46affd640 1 -- 172.21.15.89:0/1372257566 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff47c035040 con 0x5560284ee0a0 2024-09-21T03:38:54.956 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:54.950+0000 7ff46affd640 1 -- 172.21.15.89:0/1372257566 <== mon.0 v2:172.21.15.89:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 9121+0+0 (secure 0 0 0) 0x7ff47c004040 con 0x5560284ee0a0 2024-09-21T03:38:54.957 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:54.950+0000 7ff4984b5740 1 -- 172.21.15.89:0/1372257566 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff48001a070 con 0x5560284ee0a0 2024-09-21T03:38:54.957 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:54.950+0000 7ff46affd640 1 -- 172.21.15.89:0/1372257566 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7ff47c05c050 con 0x5560284ee0a0 2024-09-21T03:38:54.957 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:54.950+0000 7ff4984b5740 10 client.15261 fetch_fsmap learned FSMap version 25 2024-09-21T03:38:54.957 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:54.950+0000 7ff4984b5740 1 -- 172.21.15.89:0/1372257566 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x556028946530 con 0x5560284ee0a0 2024-09-21T03:38:54.957 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:54.950+0000 7ff46affd640 1 -- 172.21.15.89:0/1372257566 <== mon.0 v2:172.21.15.89:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff48001a070 con 0x5560284ee0a0 2024-09-21T03:38:54.957 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:54.950+0000 7ff46affd640 1 -- 172.21.15.89:0/1372257566 <== mon.0 v2:172.21.15.89:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7ff47c03c4d0 con 0x5560284ee0a0 2024-09-21T03:38:54.957 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:54.950+0000 7ff4984b5740 10 client.15261 fetch_fsmap finished waiting for FSMap version 25 2024-09-21T03:38:54.957 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:54.950+0000 7ff4984b5740 10 client.15261 resolve_mds: resolved name 'h' to daemon mds.h 2024-09-21T03:38:54.957 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:54.950+0000 7ff4984b5740 20 client.15261 populate_metadata read hostname 'smithi089' 2024-09-21T03:38:54.957 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:54.950+0000 7ff4984b5740 1 --2- 172.21.15.89:0/1372257566 >> [v2:172.21.15.110:6836/4047124589,v1:172.21.15.110:6837/4047124589] conn(0x5560284aac90 0x556028949950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:54.957 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:54.950+0000 7ff4984b5740 4 client.15261 mds_command: new command op to 24614 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-21T03:38:54.957 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:54.950+0000 7ff4984b5740 1 -- 172.21.15.89:0/1372257566 --> [v2:172.21.15.110:6836/4047124589,v1:172.21.15.110:6837/4047124589] -- command(tid 0: {"prefix": "status"}) -- 0x556028832610 con 0x5560284aac90 2024-09-21T03:38:54.957 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:54.951+0000 7ff48affd640 1 --2- 172.21.15.89:0/1372257566 >> [v2:172.21.15.110:6836/4047124589,v1:172.21.15.110:6837/4047124589] conn(0x5560284aac90 0x556028949950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:54.957 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:54.951+0000 7ff48affd640 1 --2- 172.21.15.89:0/1372257566 >> [v2:172.21.15.110:6836/4047124589,v1:172.21.15.110:6837/4047124589] conn(0x5560284aac90 0x556028949950 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=967ba6db4622f3da server_cookie=c9a681017e286e44 in_seq=0 out_seq=0 2024-09-21T03:38:54.958 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:54.951+0000 7ff46affd640 10 client.15261 ms_handle_connect on v2:172.21.15.110:6836/4047124589 2024-09-21T03:38:54.958 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:54.951+0000 7ff46affd640 1 -- 172.21.15.89:0/1372257566 <== mds.2 v2:172.21.15.110:6836/4047124589 1 ==== command_reply(tid 0: 0 ) ==== 8+0+326 (crc 0 0 0) 0x556028832610 con 0x5560284aac90 2024-09-21T03:38:54.958 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:54.952+0000 7ff46affd640 10 client.15261 handle_command_reply: tid=0 2024-09-21T03:38:54.958 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:54.952+0000 7ff4984b5740 1 client.15261 shutdown 2024-09-21T03:38:54.958 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:54.952+0000 7ff4984b5740 1 -- 172.21.15.89:0/1372257566 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5560284ee0a0 msgr2=0x5560288e3cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:54.958 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:54.952+0000 7ff4984b5740 1 --2- 172.21.15.89:0/1372257566 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5560284ee0a0 0x5560288e3cb0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7ff47c005f70 tx=0x7ff47c02aba0 comp rx=0 tx=0).stop 2024-09-21T03:38:54.958 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:54.952+0000 7ff4984b5740 1 -- 172.21.15.89:0/1372257566 shutdown_connections 2024-09-21T03:38:54.958 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:54.952+0000 7ff4984b5740 1 --2- 172.21.15.89:0/1372257566 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5560284f6280 0x5560288c58f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:54.958 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:54.952+0000 7ff4984b5740 1 --2- 172.21.15.89:0/1372257566 >> [v2:172.21.15.110:6836/4047124589,v1:172.21.15.110:6837/4047124589] conn(0x5560284aac90 0x556028949950 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:54.958 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:54.952+0000 7ff4984b5740 1 --2- 172.21.15.89:0/1372257566 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556028516f60 0x5560288c6000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:54.958 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:54.952+0000 7ff46affd640 0 client.15261 ms_handle_reset on v2:172.21.15.110:6836/4047124589 2024-09-21T03:38:54.958 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:54.952+0000 7ff4984b5740 1 --2- 172.21.15.89:0/1372257566 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5560284ee0a0 0x5560288e3cb0 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:54.958 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:54.952+0000 7ff4984b5740 1 -- 172.21.15.89:0/1372257566 >> 172.21.15.89:0/1372257566 conn(0x55602850b350 msgr2=0x5560288ba600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:38:54.958 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:54.952+0000 7ff4984b5740 1 -- 172.21.15.89:0/1372257566 shutdown_connections 2024-09-21T03:38:54.958 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:54.952+0000 7ff4984b5740 1 -- 172.21.15.89:0/1372257566 wait complete. 2024-09-21T03:38:54.959 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:54.952+0000 7ff4984b5740 20 client.15261 trim_cache size 0 max 16384 2024-09-21T03:38:54.959 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-09-21T03:38:54.959 INFO:teuthology.orchestra.run.smithi089.stdout: "cluster_fsid": "7c374098-77c9-11ef-baf5-efdc52797490", 2024-09-21T03:38:54.959 INFO:teuthology.orchestra.run.smithi089.stdout: "whoami": 2, 2024-09-21T03:38:54.959 INFO:teuthology.orchestra.run.smithi089.stdout: "id": 24614, 2024-09-21T03:38:54.959 INFO:teuthology.orchestra.run.smithi089.stdout: "want_state": "up:active", 2024-09-21T03:38:54.959 INFO:teuthology.orchestra.run.smithi089.stdout: "state": "up:active", 2024-09-21T03:38:54.959 INFO:teuthology.orchestra.run.smithi089.stdout: "fs_name": "cephfs", 2024-09-21T03:38:54.959 INFO:teuthology.orchestra.run.smithi089.stdout: "rank_uptime": 66.940068498000002, 2024-09-21T03:38:54.959 INFO:teuthology.orchestra.run.smithi089.stdout: "mdsmap_epoch": 25, 2024-09-21T03:38:54.960 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap_epoch": 84, 2024-09-21T03:38:54.960 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap_epoch_barrier": 80, 2024-09-21T03:38:54.960 INFO:teuthology.orchestra.run.smithi089.stdout: "uptime": 122.93016179 2024-09-21T03:38:54.960 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-09-21T03:38:54.960 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:54.952+0000 7ff491f41640 1 -- 172.21.15.89:0/2275350451 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7ff4740b65a0 msgr2=0x7ff4740b8ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:54.960 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:54.952+0000 7ff491f41640 1 --2- 172.21.15.89:0/2275350451 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7ff4740b65a0 0x7ff4740b8ab0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7ff480005d00 tx=0x7ff480014040 comp rx=0 tx=0).stop 2024-09-21T03:38:54.961 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:54.952+0000 7ff491f41640 1 -- 172.21.15.89:0/2275350451 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff48c108cf0 msgr2=0x7ff48c19eac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:54.961 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:54.952+0000 7ff491f41640 1 --2- 172.21.15.89:0/2275350451 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff48c108cf0 0x7ff48c19eac0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7ff48400eb10 tx=0x7ff48400c2d0 comp rx=0 tx=0).stop 2024-09-21T03:38:54.961 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:54.953+0000 7ff491f41640 1 -- 172.21.15.89:0/2275350451 shutdown_connections 2024-09-21T03:38:54.961 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:54.953+0000 7ff491f41640 1 --2- 172.21.15.89:0/2275350451 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7ff4740b65a0 0x7ff4740b8ab0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:54.961 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:54.953+0000 7ff491f41640 1 --2- 172.21.15.89:0/2275350451 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff48c1083a0 0x7ff48c199820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:54.961 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:54.953+0000 7ff491f41640 1 --2- 172.21.15.89:0/2275350451 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff48c108cf0 0x7ff48c19eac0 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:54.961 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:54.953+0000 7ff491f41640 1 --2- 172.21.15.89:0/2275350451 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff48c102a60 0x7ff48c199280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:54.961 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:54.953+0000 7ff491f41640 1 -- 172.21.15.89:0/2275350451 >> 172.21.15.89:0/2275350451 conn(0x7ff48c079730 msgr2=0x7ff48c101510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:38:54.961 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:54.953+0000 7ff491f41640 1 -- 172.21.15.89:0/2275350451 shutdown_connections 2024-09-21T03:38:54.961 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:54.953+0000 7ff491f41640 1 -- 172.21.15.89:0/2275350451 wait complete. 2024-09-21T03:38:54.962 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:38:54 smithi114 ceph-mon[34140]: pgmap v344: 129 pgs: 1 peering, 128 active+clean; 646 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-09-21T03:38:54.962 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:38:54 smithi114 ceph-mon[34140]: Health check failed: Reduced data availability: 1 pg inactive, 1 pg peering (PG_AVAILABILITY) 2024-09-21T03:38:54.962 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:38:54 smithi114 ceph-mon[34140]: osdmap e84: 12 total, 12 up, 12 in 2024-09-21T03:38:55.000 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:38:54 smithi110 ceph-mon[34000]: pgmap v344: 129 pgs: 1 peering, 128 active+clean; 646 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-09-21T03:38:55.000 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:38:54 smithi110 ceph-mon[34000]: Health check failed: Reduced data availability: 1 pg inactive, 1 pg peering (PG_AVAILABILITY) 2024-09-21T03:38:55.000 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:38:54 smithi110 ceph-mon[34000]: osdmap e84: 12 total, 12 up, 12 in 2024-09-21T03:38:55.674 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 7c374098-77c9-11ef-baf5-efdc52797490 -- ceph tell mds.i status 2024-09-21T03:38:55.922 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/7c374098-77c9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:38:56.453 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:56.441+0000 7ff6d549c640 1 -- 172.21.15.89:0/2892402826 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff6d01094c0 msgr2=0x7ff6d01098d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:56.453 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:56.441+0000 7ff6d549c640 1 --2- 172.21.15.89:0/2892402826 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff6d01094c0 0x7ff6d01098d0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7ff6b8009900 tx=0x7ff6b802a410 comp rx=0 tx=0).stop 2024-09-21T03:38:56.454 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:56.442+0000 7ff6d549c640 1 -- 172.21.15.89:0/2892402826 shutdown_connections 2024-09-21T03:38:56.454 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:56.442+0000 7ff6d549c640 1 --2- 172.21.15.89:0/2892402826 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff6d0109e10 0x7ff6d0110640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:56.454 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:56.442+0000 7ff6d549c640 1 --2- 172.21.15.89:0/2892402826 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff6d01094c0 0x7ff6d01098d0 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:56.454 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:56.442+0000 7ff6d549c640 1 --2- 172.21.15.89:0/2892402826 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff6d0103b80 0x7ff6d0103f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:56.454 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:56.442+0000 7ff6d549c640 1 -- 172.21.15.89:0/2892402826 >> 172.21.15.89:0/2892402826 conn(0x7ff6d00ff590 msgr2=0x7ff6d01019e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:38:56.454 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:56.442+0000 7ff6d549c640 1 -- 172.21.15.89:0/2892402826 shutdown_connections 2024-09-21T03:38:56.454 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:56.442+0000 7ff6d549c640 1 -- 172.21.15.89:0/2892402826 wait complete. 2024-09-21T03:38:56.454 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:56.442+0000 7ff6d549c640 1 Processor -- start 2024-09-21T03:38:56.454 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:56.442+0000 7ff6d549c640 1 -- start start 2024-09-21T03:38:56.454 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:56.443+0000 7ff6d549c640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff6d0103b80 0x7ff6d019a250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:56.454 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:56.443+0000 7ff6d549c640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff6d01094c0 0x7ff6d019a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:56.454 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:56.443+0000 7ff6d549c640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff6d0109e10 0x7ff6d019fa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:56.454 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:56.443+0000 7ff6d549c640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7ff6d0113980 con 0x7ff6d0109e10 2024-09-21T03:38:56.455 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:56.443+0000 7ff6d549c640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7ff6d0113800 con 0x7ff6d0103b80 2024-09-21T03:38:56.455 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:56.443+0000 7ff6d549c640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7ff6d0113b00 con 0x7ff6d01094c0 2024-09-21T03:38:56.455 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:56.443+0000 7ff6cf7fe640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff6d0109e10 0x7ff6d019fa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:56.455 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:56.443+0000 7ff6cf7fe640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff6d0109e10 0x7ff6d019fa90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:55318/0 (socket says 172.21.15.89:55318) 2024-09-21T03:38:56.455 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:56.443+0000 7ff6cf7fe640 1 -- 172.21.15.89:0/2127595091 learned_addr learned my addr 172.21.15.89:0/2127595091 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-21T03:38:56.455 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:56.443+0000 7ff6ceffd640 1 --2- 172.21.15.89:0/2127595091 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff6d0103b80 0x7ff6d019a250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:56.455 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:56.443+0000 7ff6cf7fe640 1 -- 172.21.15.89:0/2127595091 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff6d01094c0 msgr2=0x7ff6d019a7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:56.455 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:56.443+0000 7ff6c7fff640 1 --2- 172.21.15.89:0/2127595091 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff6d01094c0 0x7ff6d019a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:56.455 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:56.443+0000 7ff6cf7fe640 1 --2- 172.21.15.89:0/2127595091 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff6d01094c0 0x7ff6d019a7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:56.455 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:56.443+0000 7ff6cf7fe640 1 -- 172.21.15.89:0/2127595091 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff6d0103b80 msgr2=0x7ff6d019a250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:56.455 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:56.443+0000 7ff6cf7fe640 1 --2- 172.21.15.89:0/2127595091 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff6d0103b80 0x7ff6d019a250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:56.455 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:56.443+0000 7ff6cf7fe640 1 -- 172.21.15.89:0/2127595091 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6d01a01c0 con 0x7ff6d0109e10 2024-09-21T03:38:56.455 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:56.443+0000 7ff6cf7fe640 1 --2- 172.21.15.89:0/2127595091 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff6d0109e10 0x7ff6d019fa90 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7ff6c800cad0 tx=0x7ff6c800cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:38:56.455 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:56.443+0000 7ff6ccff9640 1 -- 172.21.15.89:0/2127595091 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6c800ae20 con 0x7ff6d0109e10 2024-09-21T03:38:56.456 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:56.443+0000 7ff6ccff9640 1 -- 172.21.15.89:0/2127595091 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff6c8005020 con 0x7ff6d0109e10 2024-09-21T03:38:56.456 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:56.443+0000 7ff6d549c640 1 -- 172.21.15.89:0/2127595091 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff6d01a04b0 con 0x7ff6d0109e10 2024-09-21T03:38:56.456 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:56.443+0000 7ff6ccff9640 1 -- 172.21.15.89:0/2127595091 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6c800f6b0 con 0x7ff6d0109e10 2024-09-21T03:38:56.456 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:56.443+0000 7ff6d549c640 1 -- 172.21.15.89:0/2127595091 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff6d01a6c10 con 0x7ff6d0109e10 2024-09-21T03:38:56.456 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:56.446+0000 7ff6dba10740 1 Processor -- start 2024-09-21T03:38:56.456 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:56.447+0000 7ff6ccff9640 1 -- 172.21.15.89:0/2127595091 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 18) ==== 154563+0+0 (secure 0 0 0) 0x7ff6c80051d0 con 0x7ff6d0109e10 2024-09-21T03:38:56.456 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:56.447+0000 7ff6dba10740 1 -- start start 2024-09-21T03:38:56.456 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:56.447+0000 7ff6dba10740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55681ebadf60 0x55681ef53f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:56.456 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:56.447+0000 7ff6dba10740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55681eb850a0 0x55681ef54440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:56.456 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:56.447+0000 7ff6dba10740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55681eb8d280 0x55681ef54980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:56.456 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:56.447+0000 7ff6dba10740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55681eecbc10 con 0x55681eb8d280 2024-09-21T03:38:56.456 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:56.447+0000 7ff6dba10740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55681eebb040 con 0x55681ebadf60 2024-09-21T03:38:56.456 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:56.447+0000 7ff6dba10740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55681eebee50 con 0x55681eb850a0 2024-09-21T03:38:56.456 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:56.447+0000 7ff6c7fff640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55681eb8d280 0x55681ef54980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:56.457 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:56.447+0000 7ff6c7fff640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55681eb8d280 0x55681ef54980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:55322/0 (socket says 172.21.15.89:55322) 2024-09-21T03:38:56.457 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:56.447+0000 7ff6c7fff640 1 -- 172.21.15.89:0/54080633 learned_addr learned my addr 172.21.15.89:0/54080633 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-21T03:38:56.457 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:56.447+0000 7ff6c7fff640 1 --2- 172.21.15.89:0/54080633 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55681ebadf60 0x55681ef53f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:56.457 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:56.447+0000 7ff6ceffd640 1 --2- 172.21.15.89:0/54080633 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55681eb850a0 0x55681ef54440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:56.457 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:56.448+0000 7ff6c7fff640 1 -- 172.21.15.89:0/54080633 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55681eb850a0 msgr2=0x55681ef54440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:38:56.457 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:56.448+0000 7ff6c7fff640 1 --2- 172.21.15.89:0/54080633 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55681eb850a0 0x55681ef54440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:56.457 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:56.448+0000 7ff6c7fff640 1 -- 172.21.15.89:0/54080633 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55681ebadf60 msgr2=0x55681ef53f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:38:56.457 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:56.448+0000 7ff6c7fff640 1 --2- 172.21.15.89:0/54080633 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55681ebadf60 0x55681ef53f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:56.457 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:56.448+0000 7ff6c7fff640 1 -- 172.21.15.89:0/54080633 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55681ec19540 con 0x55681eb8d280 2024-09-21T03:38:56.457 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:56.448+0000 7ff6ccff9640 1 --2- 172.21.15.89:0/2127595091 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7ff6bc0b65a0 0x7ff6bc0b8ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:56.457 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:56.448+0000 7ff6c7fff640 1 --2- 172.21.15.89:0/54080633 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55681eb8d280 0x55681ef54980 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7ff6b8005ab0 tx=0x7ff6b80058d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3a50384be8286d9 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:38:56.457 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:56.448+0000 7ff6ceffd640 1 --2- 172.21.15.89:0/2127595091 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7ff6bc0b65a0 0x7ff6bc0b8ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:56.457 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:56.448+0000 7ff6ccff9640 1 -- 172.21.15.89:0/2127595091 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 9121+0+0 (secure 0 0 0) 0x7ff6c80e7c80 con 0x7ff6d0109e10 2024-09-21T03:38:56.457 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:56.448+0000 7ff6c57fa640 1 -- 172.21.15.89:0/54080633 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6b8035040 con 0x55681eb8d280 2024-09-21T03:38:56.457 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:56.448+0000 7ff6c57fa640 1 -- 172.21.15.89:0/54080633 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff6b80054c0 con 0x55681eb8d280 2024-09-21T03:38:56.458 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:56.448+0000 7ff6c57fa640 1 -- 172.21.15.89:0/54080633 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6b8004cc0 con 0x55681eb8d280 2024-09-21T03:38:56.458 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:56.449+0000 7ff6dba10740 1 -- 172.21.15.89:0/54080633 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55681eb8d280 msgr2=0x55681ef54980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:56.458 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:56.449+0000 7ff6dba10740 1 --2- 172.21.15.89:0/54080633 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55681eb8d280 0x55681ef54980 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7ff6b8005ab0 tx=0x7ff6b80058d0 comp rx=0 tx=0).stop 2024-09-21T03:38:56.458 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:56.449+0000 7ff6ceffd640 1 --2- 172.21.15.89:0/2127595091 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7ff6bc0b65a0 0x7ff6bc0b8ab0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x55681ef52b80 tx=0x7ff6c000b4d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:38:56.458 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:56.449+0000 7ff6dba10740 1 -- 172.21.15.89:0/54080633 shutdown_connections 2024-09-21T03:38:56.458 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:56.449+0000 7ff6dba10740 1 --2- 172.21.15.89:0/54080633 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55681eb8d280 0x55681ef54980 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:56.458 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:56.449+0000 7ff6dba10740 1 --2- 172.21.15.89:0/54080633 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55681eb850a0 0x55681ef54440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:56.458 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:56.449+0000 7ff6dba10740 1 --2- 172.21.15.89:0/54080633 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55681ebadf60 0x55681ef53f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:56.458 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:56.449+0000 7ff6dba10740 1 -- 172.21.15.89:0/54080633 >> 172.21.15.89:0/54080633 conn(0x55681eba2350 msgr2=0x55681ef51df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:38:56.458 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:56.449+0000 7ff6dba10740 1 -- 172.21.15.89:0/54080633 shutdown_connections 2024-09-21T03:38:56.458 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:56.449+0000 7ff6dba10740 1 -- 172.21.15.89:0/54080633 wait complete. 2024-09-21T03:38:56.458 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:56.450+0000 7ff6dba10740 1 -- start start 2024-09-21T03:38:56.458 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:56.450+0000 7ff6dba10740 1 Processor -- start 2024-09-21T03:38:56.458 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:56.450+0000 7ff6dba10740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55681eb850a0 0x55681ef5f9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:56.459 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:56.450+0000 7ff6dba10740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55681eb8d280 0x55681ef5fee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:56.459 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:56.450+0000 7ff6dba10740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55681ebadf60 0x55681ef6b970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:56.459 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:56.450+0000 7ff6dba10740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55681ed1c5f0 con 0x55681eb8d280 2024-09-21T03:38:56.459 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:56.450+0000 7ff6dba10740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55681ee56010 con 0x55681ebadf60 2024-09-21T03:38:56.459 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:56.450+0000 7ff6dba10740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55681ef62680 con 0x55681eb850a0 2024-09-21T03:38:56.459 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:56.450+0000 7ff6dba10740 10 client.0 _ll_register_callbacks cb 0x55681eeb5340 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:38:56.459 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:56.450+0000 7ff6c7fff640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55681eb8d280 0x55681ef5fee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:56.459 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:56.450+0000 7ff6c7fff640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55681eb8d280 0x55681ef5fee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:55334/0 (socket says 172.21.15.89:55334) 2024-09-21T03:38:56.459 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:56.450+0000 7ff6c7fff640 1 -- 172.21.15.89:0/2887183836 learned_addr learned my addr 172.21.15.89:0/2887183836 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-21T03:38:56.459 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:56.450+0000 7ff6ceffd640 1 --2- 172.21.15.89:0/2887183836 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55681eb850a0 0x55681ef5f9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:56.459 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:56.450+0000 7ff6cf7fe640 1 --2- 172.21.15.89:0/2887183836 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55681ebadf60 0x55681ef6b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:56.459 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:56.451+0000 7ff6c7fff640 1 -- 172.21.15.89:0/2887183836 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55681eb850a0 msgr2=0x55681ef5f9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:38:56.459 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:56.451+0000 7ff6c7fff640 1 --2- 172.21.15.89:0/2887183836 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55681eb850a0 0x55681ef5f9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:56.459 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:56.451+0000 7ff6c7fff640 1 -- 172.21.15.89:0/2887183836 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55681ebadf60 msgr2=0x55681ef6b970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:38:56.460 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:56.451+0000 7ff6c7fff640 1 --2- 172.21.15.89:0/2887183836 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55681ebadf60 0x55681ef6b970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:56.460 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:56.451+0000 7ff6c7fff640 1 -- 172.21.15.89:0/2887183836 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55681ee83a00 con 0x55681eb8d280 2024-09-21T03:38:56.460 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:56.451+0000 7ff6c7fff640 1 --2- 172.21.15.89:0/2887183836 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55681eb8d280 0x55681ef5fee0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7ff6b80057c0 tx=0x7ff6b8032f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44f4dbd8bb48a369 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:38:56.460 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:56.451+0000 7ff6b67fc640 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2024-09-21T03:38:56.460 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:56.451+0000 7ff6b67fc640 1 -- 172.21.15.89:0/2887183836 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6b804d070 con 0x55681eb8d280 2024-09-21T03:38:56.460 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:56.451+0000 7ff6dba10740 1 -- 172.21.15.89:0/2887183836 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff6b80054c0 con 0x55681eb8d280 2024-09-21T03:38:56.460 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:56.451+0000 7ff6b67fc640 1 -- 172.21.15.89:0/2887183836 <== mon.0 v2:172.21.15.89:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 9121+0+0 (secure 0 0 0) 0x7ff6b8004e50 con 0x55681eb8d280 2024-09-21T03:38:56.460 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:56.451+0000 7ff6b67fc640 1 -- 172.21.15.89:0/2887183836 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7ff6b8035030 con 0x55681eb8d280 2024-09-21T03:38:56.460 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:56.451+0000 7ff6dba10740 10 client.15273 fetch_fsmap learned FSMap version 25 2024-09-21T03:38:56.460 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:56.451+0000 7ff6dba10740 1 -- 172.21.15.89:0/2887183836 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55681ef6e570 con 0x55681eb8d280 2024-09-21T03:38:56.460 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:56.451+0000 7ff6b67fc640 1 -- 172.21.15.89:0/2887183836 <== mon.0 v2:172.21.15.89:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff6b80054c0 con 0x55681eb8d280 2024-09-21T03:38:56.462 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:56.452+0000 7ff6b67fc640 1 -- 172.21.15.89:0/2887183836 <== mon.0 v2:172.21.15.89:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7ff6b803a040 con 0x55681eb8d280 2024-09-21T03:38:56.462 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:56.452+0000 7ff6dba10740 10 client.15273 fetch_fsmap finished waiting for FSMap version 25 2024-09-21T03:38:56.462 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:56.452+0000 7ff6dba10740 10 client.15273 resolve_mds: resolved name 'i' to daemon mds.i 2024-09-21T03:38:56.462 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:56.452+0000 7ff6dba10740 20 client.15273 populate_metadata read hostname 'smithi089' 2024-09-21T03:38:56.462 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:56.452+0000 7ff6dba10740 1 --2- 172.21.15.89:0/2887183836 >> [v2:172.21.15.114:6838/3427369944,v1:172.21.15.114:6839/3427369944] conn(0x55681eb6c8e0 0x55681ef71a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:56.462 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:56.452+0000 7ff6dba10740 4 client.15273 mds_command: new command op to 24631 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:38:56.462 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:56.452+0000 7ff6dba10740 1 -- 172.21.15.89:0/2887183836 --> [v2:172.21.15.114:6838/3427369944,v1:172.21.15.114:6839/3427369944] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55681eecdb60 con 0x55681eb6c8e0 2024-09-21T03:38:56.462 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:56.452+0000 7ff6ceffd640 1 --2- 172.21.15.89:0/2887183836 >> [v2:172.21.15.114:6838/3427369944,v1:172.21.15.114:6839/3427369944] conn(0x55681eb6c8e0 0x55681ef71a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:56.462 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:56.453+0000 7ff6ceffd640 1 --2- 172.21.15.89:0/2887183836 >> [v2:172.21.15.114:6838/3427369944,v1:172.21.15.114:6839/3427369944] conn(0x55681eb6c8e0 0x55681ef71a10 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=c87239ad648bd2c0 server_cookie=4ae55d0e04ec6378 in_seq=0 out_seq=0 2024-09-21T03:38:56.463 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:56.453+0000 7ff6b67fc640 10 client.15273 ms_handle_connect on v2:172.21.15.114:6838/3427369944 2024-09-21T03:38:56.463 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:56.453+0000 7ff6b67fc640 1 -- 172.21.15.89:0/2887183836 <== mds.1 v2:172.21.15.114:6838/3427369944 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55681eecdb60 con 0x55681eb6c8e0 2024-09-21T03:38:56.463 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:56.453+0000 7ff6b67fc640 10 client.15273 handle_command_reply: tid=0 2024-09-21T03:38:56.463 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:56.453+0000 7ff6dba10740 1 client.15273 shutdown 2024-09-21T03:38:56.463 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:56.454+0000 7ff6dba10740 1 -- 172.21.15.89:0/2887183836 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55681eb8d280 msgr2=0x55681ef5fee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:56.463 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:56.454+0000 7ff6dba10740 1 --2- 172.21.15.89:0/2887183836 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55681eb8d280 0x55681ef5fee0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7ff6b80057c0 tx=0x7ff6b8032f60 comp rx=0 tx=0).stop 2024-09-21T03:38:56.463 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:56.454+0000 7ff6dba10740 1 -- 172.21.15.89:0/2887183836 shutdown_connections 2024-09-21T03:38:56.463 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:56.454+0000 7ff6dba10740 1 --2- 172.21.15.89:0/2887183836 >> [v2:172.21.15.114:6838/3427369944,v1:172.21.15.114:6839/3427369944] conn(0x55681eb6c8e0 0x55681ef71a10 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:56.463 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:56.454+0000 7ff6dba10740 1 --2- 172.21.15.89:0/2887183836 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55681ebadf60 0x55681ef6b970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:56.463 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:56.454+0000 7ff6dba10740 1 --2- 172.21.15.89:0/2887183836 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55681eb8d280 0x55681ef5fee0 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:56.463 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:56.454+0000 7ff6dba10740 1 --2- 172.21.15.89:0/2887183836 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55681eb850a0 0x55681ef5f9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:56.463 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:56.454+0000 7ff6dba10740 1 -- 172.21.15.89:0/2887183836 >> 172.21.15.89:0/2887183836 conn(0x55681eba2350 msgr2=0x55681ef570e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:38:56.463 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:56.454+0000 7ff6b67fc640 0 client.15273 ms_handle_reset on v2:172.21.15.114:6838/3427369944 2024-09-21T03:38:56.463 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:56.454+0000 7ff6dba10740 1 -- 172.21.15.89:0/2887183836 shutdown_connections 2024-09-21T03:38:56.463 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:56.454+0000 7ff6dba10740 1 -- 172.21.15.89:0/2887183836 wait complete. 2024-09-21T03:38:56.464 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:56.454+0000 7ff6dba10740 20 client.15273 trim_cache size 0 max 16384 2024-09-21T03:38:56.473 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:56.471+0000 7ff6dba10740 1 Processor -- start 2024-09-21T03:38:56.474 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:56.471+0000 7ff6dba10740 1 -- start start 2024-09-21T03:38:56.475 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:56.472+0000 7ff6dba10740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55681eb850a0 0x55681ef6b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:56.475 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:56.472+0000 7ff6dba10740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55681eb8d280 0x55681ef6be20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:56.475 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:56.472+0000 7ff6dba10740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55681ebadf60 0x55681ef51600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:56.475 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:56.472+0000 7ff6dba10740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55681ef6fd50 con 0x55681eb8d280 2024-09-21T03:38:56.475 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:56.472+0000 7ff6dba10740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55681ef70420 con 0x55681ebadf60 2024-09-21T03:38:56.475 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:56.472+0000 7ff6dba10740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55681ef70af0 con 0x55681eb850a0 2024-09-21T03:38:56.475 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:56.472+0000 7ff6c7fff640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55681eb8d280 0x55681ef6be20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:56.475 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:56.472+0000 7ff6c7fff640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55681eb8d280 0x55681ef6be20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:55344/0 (socket says 172.21.15.89:55344) 2024-09-21T03:38:56.475 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:56.472+0000 7ff6c7fff640 1 -- 172.21.15.89:0/2609795836 learned_addr learned my addr 172.21.15.89:0/2609795836 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-21T03:38:56.476 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:56.472+0000 7ff6ceffd640 1 --2- 172.21.15.89:0/2609795836 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55681eb850a0 0x55681ef6b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:56.476 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:56.472+0000 7ff6cf7fe640 1 --2- 172.21.15.89:0/2609795836 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55681ebadf60 0x55681ef51600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:56.476 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:56.472+0000 7ff6c7fff640 1 -- 172.21.15.89:0/2609795836 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55681eb850a0 msgr2=0x55681ef6b8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:38:56.476 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:56.472+0000 7ff6c7fff640 1 --2- 172.21.15.89:0/2609795836 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55681eb850a0 0x55681ef6b8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:56.476 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:56.472+0000 7ff6c7fff640 1 -- 172.21.15.89:0/2609795836 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55681ebadf60 msgr2=0x55681ef51600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:38:56.476 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:56.472+0000 7ff6c7fff640 1 --2- 172.21.15.89:0/2609795836 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55681ebadf60 0x55681ef51600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:56.476 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:56.472+0000 7ff6c7fff640 1 -- 172.21.15.89:0/2609795836 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55681ef561f0 con 0x55681eb8d280 2024-09-21T03:38:56.476 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:56.473+0000 7ff6c7fff640 1 --2- 172.21.15.89:0/2609795836 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55681eb8d280 0x55681ef6be20 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7ff6b8032330 tx=0x7ff6b802ab70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ebc151f612f03d9 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:38:56.476 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:56.473+0000 7ff6c4ff9640 1 -- 172.21.15.89:0/2609795836 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6b804d040 con 0x55681eb8d280 2024-09-21T03:38:56.476 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:56.473+0000 7ff6c4ff9640 1 -- 172.21.15.89:0/2609795836 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff6b8042030 con 0x55681eb8d280 2024-09-21T03:38:56.476 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:56.473+0000 7ff6c4ff9640 1 -- 172.21.15.89:0/2609795836 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6b8004c50 con 0x55681eb8d280 2024-09-21T03:38:56.480 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:56.473+0000 7ff6dba10740 1 -- 172.21.15.89:0/2609795836 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55681eb8d280 msgr2=0x55681ef6be20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:56.480 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:56.473+0000 7ff6dba10740 1 --2- 172.21.15.89:0/2609795836 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55681eb8d280 0x55681ef6be20 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7ff6b8032330 tx=0x7ff6b802ab70 comp rx=0 tx=0).stop 2024-09-21T03:38:56.480 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:56.474+0000 7ff6dba10740 1 -- 172.21.15.89:0/2609795836 shutdown_connections 2024-09-21T03:38:56.480 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:56.474+0000 7ff6dba10740 1 --2- 172.21.15.89:0/2609795836 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55681ebadf60 0x55681ef51600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:56.480 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:56.474+0000 7ff6dba10740 1 --2- 172.21.15.89:0/2609795836 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55681eb8d280 0x55681ef6be20 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:56.480 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:56.474+0000 7ff6dba10740 1 --2- 172.21.15.89:0/2609795836 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55681eb850a0 0x55681ef6b8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:56.480 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:56.474+0000 7ff6dba10740 1 -- 172.21.15.89:0/2609795836 >> 172.21.15.89:0/2609795836 conn(0x55681eba2350 msgr2=0x55681ef5a980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:38:56.480 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:56.474+0000 7ff6dba10740 1 -- 172.21.15.89:0/2609795836 shutdown_connections 2024-09-21T03:38:56.481 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:56.474+0000 7ff6dba10740 1 -- 172.21.15.89:0/2609795836 wait complete. 2024-09-21T03:38:56.481 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:56.474+0000 7ff6dba10740 1 -- start start 2024-09-21T03:38:56.481 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:56.474+0000 7ff6dba10740 1 Processor -- start 2024-09-21T03:38:56.481 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:56.474+0000 7ff6dba10740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55681eb850a0 0x55681ef7a2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:56.481 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:56.474+0000 7ff6dba10740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55681eb8d280 0x55681ef73760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:56.481 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:56.474+0000 7ff6dba10740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55681ebadf60 0x55681ef73ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:56.481 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:56.474+0000 7ff6dba10740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55681ef71890 con 0x55681ebadf60 2024-09-21T03:38:56.481 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:56.474+0000 7ff6dba10740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55681ef711c0 con 0x55681eb850a0 2024-09-21T03:38:56.481 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:56.474+0000 7ff6dba10740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55681ef71f60 con 0x55681eb8d280 2024-09-21T03:38:56.481 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:56.474+0000 7ff6dba10740 10 client.0 _ll_register_callbacks cb 0x7ff6b8032a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:38:56.481 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:56.475+0000 7ff6cf7fe640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55681ebadf60 0x55681ef73ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:56.481 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:56.475+0000 7ff6cf7fe640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55681ebadf60 0x55681ef73ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:55346/0 (socket says 172.21.15.89:55346) 2024-09-21T03:38:56.481 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:56.475+0000 7ff6cf7fe640 1 -- 172.21.15.89:0/3788148470 learned_addr learned my addr 172.21.15.89:0/3788148470 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-21T03:38:56.481 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:56.475+0000 7ff6c7fff640 1 --2- 172.21.15.89:0/3788148470 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55681eb8d280 0x55681ef73760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:56.482 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:56.475+0000 7ff6ceffd640 1 --2- 172.21.15.89:0/3788148470 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55681eb850a0 0x55681ef7a2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:56.482 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:56.475+0000 7ff6cf7fe640 1 -- 172.21.15.89:0/3788148470 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55681eb8d280 msgr2=0x55681ef73760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:38:56.482 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:56.475+0000 7ff6cf7fe640 1 --2- 172.21.15.89:0/3788148470 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55681eb8d280 0x55681ef73760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:56.482 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:56.475+0000 7ff6cf7fe640 1 -- 172.21.15.89:0/3788148470 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55681eb850a0 msgr2=0x55681ef7a2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:38:56.482 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:56.475+0000 7ff6cf7fe640 1 --2- 172.21.15.89:0/3788148470 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55681eb850a0 0x55681ef7a2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:56.482 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:56.475+0000 7ff6cf7fe640 1 -- 172.21.15.89:0/3788148470 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55681ef74260 con 0x55681ebadf60 2024-09-21T03:38:56.482 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:56.475+0000 7ff6ceffd640 1 --2- 172.21.15.89:0/3788148470 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55681eb850a0 0x55681ef7a2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:38:56.482 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:56.475+0000 7ff6cf7fe640 1 --2- 172.21.15.89:0/3788148470 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55681ebadf60 0x55681ef73ca0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7ff6c80aa0f0 tx=0x7ff6c80e5730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e7f0739716b361d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:38:56.482 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:56.475+0000 7ff6b67fc640 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2024-09-21T03:38:56.482 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:56.475+0000 7ff6b67fc640 1 -- 172.21.15.89:0/3788148470 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6c801d040 con 0x55681ebadf60 2024-09-21T03:38:56.482 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:56.475+0000 7ff6b67fc640 1 -- 172.21.15.89:0/3788148470 <== mon.0 v2:172.21.15.89:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 9121+0+0 (secure 0 0 0) 0x7ff6c80acd90 con 0x55681ebadf60 2024-09-21T03:38:56.482 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:56.475+0000 7ff6dba10740 1 -- 172.21.15.89:0/3788148470 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff6b8042030 con 0x55681ebadf60 2024-09-21T03:38:56.482 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:56.475+0000 7ff6b67fc640 1 -- 172.21.15.89:0/3788148470 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7ff6c80ae030 con 0x55681ebadf60 2024-09-21T03:38:56.482 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:56.475+0000 7ff6dba10740 10 client.15279 fetch_fsmap learned FSMap version 25 2024-09-21T03:38:56.483 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:56.475+0000 7ff6dba10740 1 -- 172.21.15.89:0/3788148470 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55681ef79ca0 con 0x55681ebadf60 2024-09-21T03:38:56.483 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:56.475+0000 7ff6b67fc640 1 -- 172.21.15.89:0/3788148470 <== mon.0 v2:172.21.15.89:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff6b8042030 con 0x55681ebadf60 2024-09-21T03:38:56.483 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:56.476+0000 7ff6b67fc640 1 -- 172.21.15.89:0/3788148470 <== mon.0 v2:172.21.15.89:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7ff6c80b35a0 con 0x55681ebadf60 2024-09-21T03:38:56.483 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:56.476+0000 7ff6dba10740 10 client.15279 fetch_fsmap finished waiting for FSMap version 25 2024-09-21T03:38:56.483 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:56.476+0000 7ff6dba10740 10 client.15279 resolve_mds: resolved name 'i' to daemon mds.i 2024-09-21T03:38:56.483 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:56.476+0000 7ff6dba10740 20 client.15279 populate_metadata read hostname 'smithi089' 2024-09-21T03:38:56.483 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:56.476+0000 7ff6dba10740 1 --2- 172.21.15.89:0/3788148470 >> [v2:172.21.15.114:6838/3427369944,v1:172.21.15.114:6839/3427369944] conn(0x55681eb41c90 0x55681efe05d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:56.483 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:56.476+0000 7ff6dba10740 4 client.15279 mds_command: new command op to 24631 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-21T03:38:56.483 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:56.476+0000 7ff6dba10740 1 -- 172.21.15.89:0/3788148470 --> [v2:172.21.15.114:6838/3427369944,v1:172.21.15.114:6839/3427369944] -- command(tid 0: {"prefix": "status"}) -- 0x55681eec9610 con 0x55681eb41c90 2024-09-21T03:38:56.483 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:56.476+0000 7ff6ceffd640 1 --2- 172.21.15.89:0/3788148470 >> [v2:172.21.15.114:6838/3427369944,v1:172.21.15.114:6839/3427369944] conn(0x55681eb41c90 0x55681efe05d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:56.483 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:56.477+0000 7ff6b67fc640 10 client.15279 ms_handle_connect on v2:172.21.15.114:6838/3427369944 2024-09-21T03:38:56.483 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:56.477+0000 7ff6ceffd640 1 --2- 172.21.15.89:0/3788148470 >> [v2:172.21.15.114:6838/3427369944,v1:172.21.15.114:6839/3427369944] conn(0x55681eb41c90 0x55681efe05d0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=6a71cd0680d14ccf server_cookie=55fa2167d9673eda in_seq=0 out_seq=0 2024-09-21T03:38:56.483 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:56.477+0000 7ff6b67fc640 1 -- 172.21.15.89:0/3788148470 <== mds.1 v2:172.21.15.114:6838/3427369944 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55681eec9610 con 0x55681eb41c90 2024-09-21T03:38:56.483 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:56.477+0000 7ff6b67fc640 10 client.15279 handle_command_reply: tid=0 2024-09-21T03:38:56.484 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:56.477+0000 7ff6dba10740 1 client.15279 shutdown 2024-09-21T03:38:56.484 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:56.477+0000 7ff6dba10740 1 -- 172.21.15.89:0/3788148470 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55681ebadf60 msgr2=0x55681ef73ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:56.484 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:56.477+0000 7ff6dba10740 1 --2- 172.21.15.89:0/3788148470 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55681ebadf60 0x55681ef73ca0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7ff6c80aa0f0 tx=0x7ff6c80e5730 comp rx=0 tx=0).stop 2024-09-21T03:38:56.484 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:56.477+0000 7ff6dba10740 1 -- 172.21.15.89:0/3788148470 shutdown_connections 2024-09-21T03:38:56.484 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:56.477+0000 7ff6dba10740 1 --2- 172.21.15.89:0/3788148470 >> [v2:172.21.15.114:6838/3427369944,v1:172.21.15.114:6839/3427369944] conn(0x55681eb41c90 0x55681efe05d0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:56.484 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:56.477+0000 7ff6dba10740 1 --2- 172.21.15.89:0/3788148470 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55681ebadf60 0x55681ef73ca0 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:56.484 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:56.477+0000 7ff6dba10740 1 --2- 172.21.15.89:0/3788148470 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55681eb8d280 0x55681ef73760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:56.484 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:56.477+0000 7ff6dba10740 1 --2- 172.21.15.89:0/3788148470 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55681eb850a0 0x55681ef7a2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:56.484 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:56.477+0000 7ff6b67fc640 0 client.15279 ms_handle_reset on v2:172.21.15.114:6838/3427369944 2024-09-21T03:38:56.484 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:56.477+0000 7ff6dba10740 1 -- 172.21.15.89:0/3788148470 >> 172.21.15.89:0/3788148470 conn(0x55681eba2350 msgr2=0x55681ef51600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:38:56.484 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:56.477+0000 7ff6dba10740 1 -- 172.21.15.89:0/3788148470 shutdown_connections 2024-09-21T03:38:56.484 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:56.477+0000 7ff6dba10740 1 -- 172.21.15.89:0/3788148470 wait complete. 2024-09-21T03:38:56.484 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:56.477+0000 7ff6dba10740 20 client.15279 trim_cache size 0 max 16384 2024-09-21T03:38:56.485 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-09-21T03:38:56.485 INFO:teuthology.orchestra.run.smithi089.stdout: "cluster_fsid": "7c374098-77c9-11ef-baf5-efdc52797490", 2024-09-21T03:38:56.485 INFO:teuthology.orchestra.run.smithi089.stdout: "whoami": 1, 2024-09-21T03:38:56.485 INFO:teuthology.orchestra.run.smithi089.stdout: "id": 24631, 2024-09-21T03:38:56.485 INFO:teuthology.orchestra.run.smithi089.stdout: "want_state": "up:active", 2024-09-21T03:38:56.485 INFO:teuthology.orchestra.run.smithi089.stdout: "state": "up:active", 2024-09-21T03:38:56.485 INFO:teuthology.orchestra.run.smithi089.stdout: "fs_name": "cephfs", 2024-09-21T03:38:56.485 INFO:teuthology.orchestra.run.smithi089.stdout: "rank_uptime": 69.479395499999995, 2024-09-21T03:38:56.485 INFO:teuthology.orchestra.run.smithi089.stdout: "mdsmap_epoch": 25, 2024-09-21T03:38:56.485 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap_epoch": 84, 2024-09-21T03:38:56.485 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap_epoch_barrier": 80, 2024-09-21T03:38:56.485 INFO:teuthology.orchestra.run.smithi089.stdout: "uptime": 121.832467088 2024-09-21T03:38:56.486 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-09-21T03:38:56.486 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:56.478+0000 7ff6d549c640 1 -- 172.21.15.89:0/2127595091 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7ff6bc0b65a0 msgr2=0x7ff6bc0b8ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:56.486 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:56.478+0000 7ff6d549c640 1 --2- 172.21.15.89:0/2127595091 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7ff6bc0b65a0 0x7ff6bc0b8ab0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x55681ef52b80 tx=0x7ff6c000b4d0 comp rx=0 tx=0).stop 2024-09-21T03:38:56.486 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:56.478+0000 7ff6d549c640 1 -- 172.21.15.89:0/2127595091 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff6d0109e10 msgr2=0x7ff6d019fa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:56.486 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:56.478+0000 7ff6d549c640 1 --2- 172.21.15.89:0/2127595091 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff6d0109e10 0x7ff6d019fa90 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7ff6c800cad0 tx=0x7ff6c800cf90 comp rx=0 tx=0).stop 2024-09-21T03:38:56.487 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:56.478+0000 7ff6d549c640 1 -- 172.21.15.89:0/2127595091 shutdown_connections 2024-09-21T03:38:56.487 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:56.478+0000 7ff6d549c640 1 --2- 172.21.15.89:0/2127595091 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7ff6bc0b65a0 0x7ff6bc0b8ab0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:56.487 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:56.478+0000 7ff6d549c640 1 --2- 172.21.15.89:0/2127595091 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff6d0109e10 0x7ff6d019fa90 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:56.487 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:56.478+0000 7ff6d549c640 1 --2- 172.21.15.89:0/2127595091 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff6d01094c0 0x7ff6d019a7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:56.487 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:56.478+0000 7ff6d549c640 1 --2- 172.21.15.89:0/2127595091 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff6d0103b80 0x7ff6d019a250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:56.487 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:56.478+0000 7ff6d549c640 1 -- 172.21.15.89:0/2127595091 >> 172.21.15.89:0/2127595091 conn(0x7ff6d00ff590 msgr2=0x7ff6d0106990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:38:56.487 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:56.478+0000 7ff6d549c640 1 -- 172.21.15.89:0/2127595091 shutdown_connections 2024-09-21T03:38:56.487 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:56.478+0000 7ff6d549c640 1 -- 172.21.15.89:0/2127595091 wait complete. 2024-09-21T03:38:56.961 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:38:56 smithi114 ceph-mon[34140]: pgmap v346: 129 pgs: 1 peering, 128 active+clean; 646 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-09-21T03:38:57.000 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:38:56 smithi110 ceph-mon[34000]: pgmap v346: 129 pgs: 1 peering, 128 active+clean; 646 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-09-21T03:38:57.061 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:38:56 smithi089 ceph-mon[30928]: pgmap v346: 129 pgs: 1 peering, 128 active+clean; 646 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-09-21T03:38:57.183 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 7c374098-77c9-11ef-baf5-efdc52797490 -- ceph tell mds.b status 2024-09-21T03:38:57.427 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/7c374098-77c9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:38:57.927 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:57.924+0000 7f0f596ca640 1 -- 172.21.15.89:0/2639590577 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0f54103160 msgr2=0x7f0f54103570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:57.960 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:57.924+0000 7f0f596ca640 1 --2- 172.21.15.89:0/2639590577 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0f54103160 0x7f0f54103570 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f0f44009ca0 tx=0x7f0f4402a8c0 comp rx=0 tx=0).stop 2024-09-21T03:38:57.960 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:57.924+0000 7f0f596ca640 1 -- 172.21.15.89:0/2639590577 shutdown_connections 2024-09-21T03:38:57.960 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:57.924+0000 7f0f596ca640 1 --2- 172.21.15.89:0/2639590577 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0f54105ac0 0x7f0f5410f530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:57.960 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:57.924+0000 7f0f596ca640 1 --2- 172.21.15.89:0/2639590577 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0f54105170 0x7f0f54105580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:57.960 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:57.924+0000 7f0f596ca640 1 --2- 172.21.15.89:0/2639590577 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0f54103160 0x7f0f54103570 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:57.961 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:57.924+0000 7f0f596ca640 1 -- 172.21.15.89:0/2639590577 >> 172.21.15.89:0/2639590577 conn(0x7f0f54077c60 msgr2=0x7f0f540780a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:38:57.961 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:57.924+0000 7f0f596ca640 1 -- 172.21.15.89:0/2639590577 shutdown_connections 2024-09-21T03:38:57.961 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:57.924+0000 7f0f596ca640 1 -- 172.21.15.89:0/2639590577 wait complete. 2024-09-21T03:38:57.961 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:57.925+0000 7f0f596ca640 1 Processor -- start 2024-09-21T03:38:57.961 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:57.925+0000 7f0f596ca640 1 -- start start 2024-09-21T03:38:57.971 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:57.925+0000 7f0f596ca640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0f54103160 0x7f0f541992f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:57.971 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:57.925+0000 7f0f596ca640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0f54105170 0x7f0f54199890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:57.972 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:57.925+0000 7f0f596ca640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0f54105ac0 0x7f0f5419eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:57.972 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:57.925+0000 7f0f596ca640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f0f541128c0 con 0x7f0f54105170 2024-09-21T03:38:57.972 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:57.925+0000 7f0f596ca640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f0f54112740 con 0x7f0f54103160 2024-09-21T03:38:57.972 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:57.925+0000 7f0f596ca640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f0f54112a40 con 0x7f0f54105ac0 2024-09-21T03:38:57.972 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:57.925+0000 7f0f527fc640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0f54105170 0x7f0f54199890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:57.972 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:57.925+0000 7f0f527fc640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0f54105170 0x7f0f54199890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:55364/0 (socket says 172.21.15.89:55364) 2024-09-21T03:38:57.972 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:57.925+0000 7f0f527fc640 1 -- 172.21.15.89:0/2427177713 learned_addr learned my addr 172.21.15.89:0/2427177713 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-21T03:38:57.972 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:57.925+0000 7f0f52ffd640 1 --2- 172.21.15.89:0/2427177713 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0f54103160 0x7f0f541992f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:57.972 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:57.926+0000 7f0f537fe640 1 --2- 172.21.15.89:0/2427177713 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0f54105ac0 0x7f0f5419eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:57.972 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:57.926+0000 7f0f527fc640 1 -- 172.21.15.89:0/2427177713 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0f54105ac0 msgr2=0x7f0f5419eb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:57.972 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:57.926+0000 7f0f527fc640 1 --2- 172.21.15.89:0/2427177713 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0f54105ac0 0x7f0f5419eb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:57.972 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:57.926+0000 7f0f527fc640 1 -- 172.21.15.89:0/2427177713 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0f54103160 msgr2=0x7f0f541992f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:57.972 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:57.926+0000 7f0f527fc640 1 --2- 172.21.15.89:0/2427177713 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0f54103160 0x7f0f541992f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:57.972 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:57.926+0000 7f0f527fc640 1 -- 172.21.15.89:0/2427177713 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f5419f260 con 0x7f0f54105170 2024-09-21T03:38:57.973 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:57.926+0000 7f0f527fc640 1 --2- 172.21.15.89:0/2427177713 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0f54105170 0x7f0f54199890 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f0f3c00c970 tx=0x7f0f3c00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:38:57.973 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:57.926+0000 7f0f4bfff640 1 -- 172.21.15.89:0/2427177713 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f3c007bb0 con 0x7f0f54105170 2024-09-21T03:38:57.973 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:57.926+0000 7f0f596ca640 1 -- 172.21.15.89:0/2427177713 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0f5419f550 con 0x7f0f54105170 2024-09-21T03:38:57.973 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:57.926+0000 7f0f4bfff640 1 -- 172.21.15.89:0/2427177713 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0f3c007d50 con 0x7f0f54105170 2024-09-21T03:38:57.973 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:57.926+0000 7f0f4bfff640 1 -- 172.21.15.89:0/2427177713 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f3c004d40 con 0x7f0f54105170 2024-09-21T03:38:57.973 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:57.926+0000 7f0f596ca640 1 -- 172.21.15.89:0/2427177713 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0f54101dc0 con 0x7f0f54105170 2024-09-21T03:38:57.973 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:57.929+0000 7f0f5fc3e740 1 Processor -- start 2024-09-21T03:38:57.973 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:57.929+0000 7f0f5fc3e740 1 -- start start 2024-09-21T03:38:57.973 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:57.929+0000 7f0f5fc3e740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c001793f60 0x55c001b39f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:57.973 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:57.929+0000 7f0f5fc3e740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c00176b0a0 0x55c001b3a440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:57.973 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:57.929+0000 7f0f4bfff640 1 -- 172.21.15.89:0/2427177713 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 18) ==== 154563+0+0 (secure 0 0 0) 0x7f0f3c00f450 con 0x7f0f54105170 2024-09-21T03:38:57.973 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:57.929+0000 7f0f5fc3e740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c001773280 0x55c001b3a980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:57.973 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:57.929+0000 7f0f5fc3e740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55c001ab1c10 con 0x55c001793f60 2024-09-21T03:38:57.973 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:57.929+0000 7f0f5fc3e740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55c001aa1040 con 0x55c001773280 2024-09-21T03:38:57.973 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:57.929+0000 7f0f5fc3e740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55c001aa4e50 con 0x55c00176b0a0 2024-09-21T03:38:57.974 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:57.929+0000 7f0f537fe640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c001793f60 0x55c001b39f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:57.974 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:57.929+0000 7f0f537fe640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c001793f60 0x55c001b39f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:55378/0 (socket says 172.21.15.89:55378) 2024-09-21T03:38:57.974 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:57.929+0000 7f0f537fe640 1 -- 172.21.15.89:0/3598702689 learned_addr learned my addr 172.21.15.89:0/3598702689 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-21T03:38:57.974 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:57.930+0000 7f0f52ffd640 1 --2- 172.21.15.89:0/3598702689 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c00176b0a0 0x55c001b3a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:57.974 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:57.930+0000 7f0f527fc640 1 --2- 172.21.15.89:0/3598702689 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c001773280 0x55c001b3a980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:57.974 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:57.930+0000 7f0f537fe640 1 -- 172.21.15.89:0/3598702689 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c00176b0a0 msgr2=0x55c001b3a440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:38:57.974 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:57.930+0000 7f0f537fe640 1 --2- 172.21.15.89:0/3598702689 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c00176b0a0 0x55c001b3a440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:57.974 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:57.930+0000 7f0f537fe640 1 -- 172.21.15.89:0/3598702689 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c001773280 msgr2=0x55c001b3a980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:38:57.974 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:57.930+0000 7f0f537fe640 1 --2- 172.21.15.89:0/3598702689 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c001773280 0x55c001b3a980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:57.974 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:57.930+0000 7f0f537fe640 1 -- 172.21.15.89:0/3598702689 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c0017ff540 con 0x55c001793f60 2024-09-21T03:38:57.974 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:57.930+0000 7f0f537fe640 1 --2- 172.21.15.89:0/3598702689 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c001793f60 0x55c001b39f00 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f0f4c00ab80 tx=0x7f0f4c00f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9a318137e79ab55 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:38:57.974 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:57.930+0000 7f0f497fa640 1 -- 172.21.15.89:0/3598702689 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f4c01d070 con 0x55c001793f60 2024-09-21T03:38:57.974 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:57.930+0000 7f0f4bfff640 1 --2- 172.21.15.89:0/2427177713 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f0f400b65a0 0x7f0f400b8ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:57.975 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:57.930+0000 7f0f497fa640 1 -- 172.21.15.89:0/3598702689 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0f4c018ad0 con 0x55c001793f60 2024-09-21T03:38:57.975 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:57.930+0000 7f0f497fa640 1 -- 172.21.15.89:0/3598702689 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f4c0108a0 con 0x55c001793f60 2024-09-21T03:38:57.975 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:57.930+0000 7f0f52ffd640 1 --2- 172.21.15.89:0/2427177713 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f0f400b65a0 0x7f0f400b8ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:57.975 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:57.931+0000 7f0f4bfff640 1 -- 172.21.15.89:0/2427177713 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 9121+0+0 (secure 0 0 0) 0x7f0f3c0e7490 con 0x7f0f54105170 2024-09-21T03:38:57.975 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:57.931+0000 7f0f5fc3e740 1 -- 172.21.15.89:0/3598702689 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c001793f60 msgr2=0x55c001b39f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:57.975 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:57.931+0000 7f0f52ffd640 1 --2- 172.21.15.89:0/2427177713 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f0f400b65a0 0x7f0f400b8ab0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f0f440098d0 tx=0x7f0f44031000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:38:57.975 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:57.931+0000 7f0f5fc3e740 1 --2- 172.21.15.89:0/3598702689 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c001793f60 0x55c001b39f00 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f0f4c00ab80 tx=0x7f0f4c00f790 comp rx=0 tx=0).stop 2024-09-21T03:38:57.975 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:57.931+0000 7f0f5fc3e740 1 -- 172.21.15.89:0/3598702689 shutdown_connections 2024-09-21T03:38:57.975 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:57.931+0000 7f0f5fc3e740 1 --2- 172.21.15.89:0/3598702689 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c001773280 0x55c001b3a980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:57.975 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:57.931+0000 7f0f5fc3e740 1 --2- 172.21.15.89:0/3598702689 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c00176b0a0 0x55c001b3a440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:57.975 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:57.931+0000 7f0f5fc3e740 1 --2- 172.21.15.89:0/3598702689 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c001793f60 0x55c001b39f00 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:57.975 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:57.931+0000 7f0f5fc3e740 1 -- 172.21.15.89:0/3598702689 >> 172.21.15.89:0/3598702689 conn(0x55c001788350 msgr2=0x55c001b37df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:38:57.975 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:57.931+0000 7f0f5fc3e740 1 -- 172.21.15.89:0/3598702689 shutdown_connections 2024-09-21T03:38:57.975 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:57.931+0000 7f0f5fc3e740 1 -- 172.21.15.89:0/3598702689 wait complete. 2024-09-21T03:38:57.976 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:57.932+0000 7f0f5fc3e740 1 -- start start 2024-09-21T03:38:57.976 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:57.932+0000 7f0f5fc3e740 1 Processor -- start 2024-09-21T03:38:57.976 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:57.932+0000 7f0f5fc3e740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c00176b0a0 0x55c001b459e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:57.976 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:57.932+0000 7f0f5fc3e740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c001773280 0x55c001b45f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:57.976 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:57.932+0000 7f0f5fc3e740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c001793f60 0x55c001b519d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:57.976 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:57.932+0000 7f0f5fc3e740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55c0019025f0 con 0x55c001793f60 2024-09-21T03:38:57.976 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:57.932+0000 7f0f5fc3e740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55c001a3c010 con 0x55c00176b0a0 2024-09-21T03:38:57.976 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:57.932+0000 7f0f5fc3e740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55c001b486e0 con 0x55c001773280 2024-09-21T03:38:57.976 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:57.932+0000 7f0f5fc3e740 10 client.0 _ll_register_callbacks cb 0x55c001a9b340 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:38:57.976 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:57.932+0000 7f0f537fe640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c001793f60 0x55c001b519d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:57.976 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:57.932+0000 7f0f537fe640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c001793f60 0x55c001b519d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:55386/0 (socket says 172.21.15.89:55386) 2024-09-21T03:38:57.976 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:57.932+0000 7f0f537fe640 1 -- 172.21.15.89:0/1157840487 learned_addr learned my addr 172.21.15.89:0/1157840487 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-21T03:38:57.976 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:57.932+0000 7f0f527fc640 1 --2- 172.21.15.89:0/1157840487 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c001773280 0x55c001b45f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:57.977 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:57.932+0000 7f0f52ffd640 1 --2- 172.21.15.89:0/1157840487 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c00176b0a0 0x55c001b459e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:57.977 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:57.933+0000 7f0f537fe640 1 -- 172.21.15.89:0/1157840487 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c001773280 msgr2=0x55c001b45f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:38:57.977 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:57.933+0000 7f0f537fe640 1 --2- 172.21.15.89:0/1157840487 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c001773280 0x55c001b45f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:57.977 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:57.933+0000 7f0f537fe640 1 -- 172.21.15.89:0/1157840487 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c00176b0a0 msgr2=0x55c001b459e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:38:57.977 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:57.933+0000 7f0f537fe640 1 --2- 172.21.15.89:0/1157840487 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c00176b0a0 0x55c001b459e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:57.977 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:57.933+0000 7f0f537fe640 1 -- 172.21.15.89:0/1157840487 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c001a69a00 con 0x55c001793f60 2024-09-21T03:38:57.977 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:57.933+0000 7f0f52ffd640 1 --2- 172.21.15.89:0/1157840487 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c00176b0a0 0x55c001b459e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:38:57.977 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:57.933+0000 7f0f537fe640 1 --2- 172.21.15.89:0/1157840487 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c001793f60 0x55c001b519d0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f0f4c01b040 tx=0x7f0f4c0186f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1a0983206515e70 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:38:57.977 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:57.933+0000 7f0f3a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2024-09-21T03:38:57.977 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:57.933+0000 7f0f3a7fc640 1 -- 172.21.15.89:0/1157840487 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f4c027020 con 0x55c001793f60 2024-09-21T03:38:57.977 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:57.933+0000 7f0f3a7fc640 1 -- 172.21.15.89:0/1157840487 <== mon.0 v2:172.21.15.89:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 9121+0+0 (secure 0 0 0) 0x7f0f4c005240 con 0x55c001793f60 2024-09-21T03:38:57.977 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:57.933+0000 7f0f5fc3e740 1 -- 172.21.15.89:0/1157840487 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0f4c018ad0 con 0x55c001793f60 2024-09-21T03:38:57.977 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:57.933+0000 7f0f3a7fc640 1 -- 172.21.15.89:0/1157840487 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f0f4c032050 con 0x55c001793f60 2024-09-21T03:38:57.977 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:57.933+0000 7f0f5fc3e740 10 client.15291 fetch_fsmap learned FSMap version 25 2024-09-21T03:38:57.978 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:57.933+0000 7f0f5fc3e740 1 -- 172.21.15.89:0/1157840487 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55c001b54620 con 0x55c001793f60 2024-09-21T03:38:57.978 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:57.933+0000 7f0f3a7fc640 1 -- 172.21.15.89:0/1157840487 <== mon.0 v2:172.21.15.89:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0f4c018ad0 con 0x55c001793f60 2024-09-21T03:38:57.978 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:57.934+0000 7f0f3a7fc640 1 -- 172.21.15.89:0/1157840487 <== mon.0 v2:172.21.15.89:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f0f4c0105a0 con 0x55c001793f60 2024-09-21T03:38:57.978 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:57.934+0000 7f0f5fc3e740 10 client.15291 fetch_fsmap finished waiting for FSMap version 25 2024-09-21T03:38:57.978 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:57.934+0000 7f0f5fc3e740 10 client.15291 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-21T03:38:57.978 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:57.934+0000 7f0f5fc3e740 20 client.15291 populate_metadata read hostname 'smithi089' 2024-09-21T03:38:57.978 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:57.934+0000 7f0f5fc3e740 1 --2- 172.21.15.89:0/1157840487 >> [v2:172.21.15.110:6838/132366709,v1:172.21.15.110:6839/132366709] conn(0x55c0017528e0 0x55c001b57cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:57.978 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:57.934+0000 7f0f5fc3e740 4 client.15291 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:38:57.978 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:57.934+0000 7f0f5fc3e740 1 -- 172.21.15.89:0/1157840487 --> [v2:172.21.15.110:6838/132366709,v1:172.21.15.110:6839/132366709] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c001ab3b60 con 0x55c0017528e0 2024-09-21T03:38:57.978 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:57.934+0000 7f0f52ffd640 1 --2- 172.21.15.89:0/1157840487 >> [v2:172.21.15.110:6838/132366709,v1:172.21.15.110:6839/132366709] conn(0x55c0017528e0 0x55c001b57cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:57.978 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:57.935+0000 7f0f3a7fc640 10 client.15291 ms_handle_connect on v2:172.21.15.110:6838/132366709 2024-09-21T03:38:57.978 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:57.935+0000 7f0f52ffd640 1 --2- 172.21.15.89:0/1157840487 >> [v2:172.21.15.110:6838/132366709,v1:172.21.15.110:6839/132366709] conn(0x55c0017528e0 0x55c001b57cb0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e68e99da9df7074a server_cookie=627102bd49132eda in_seq=0 out_seq=0 2024-09-21T03:38:57.978 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:57.935+0000 7f0f3a7fc640 1 -- 172.21.15.89:0/1157840487 <== mds.0 v2:172.21.15.110:6838/132366709 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55c001ab3b60 con 0x55c0017528e0 2024-09-21T03:38:57.978 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:57.935+0000 7f0f3a7fc640 10 client.15291 handle_command_reply: tid=0 2024-09-21T03:38:57.979 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:57.935+0000 7f0f5fc3e740 1 client.15291 shutdown 2024-09-21T03:38:57.979 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:57.935+0000 7f0f5fc3e740 1 -- 172.21.15.89:0/1157840487 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c001793f60 msgr2=0x55c001b519d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:57.979 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:57.935+0000 7f0f5fc3e740 1 --2- 172.21.15.89:0/1157840487 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c001793f60 0x55c001b519d0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f0f4c01b040 tx=0x7f0f4c0186f0 comp rx=0 tx=0).stop 2024-09-21T03:38:57.979 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:57.935+0000 7f0f5fc3e740 1 -- 172.21.15.89:0/1157840487 shutdown_connections 2024-09-21T03:38:57.979 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:57.935+0000 7f0f5fc3e740 1 --2- 172.21.15.89:0/1157840487 >> [v2:172.21.15.110:6838/132366709,v1:172.21.15.110:6839/132366709] conn(0x55c0017528e0 0x55c001b57cb0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:57.979 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:57.935+0000 7f0f5fc3e740 1 --2- 172.21.15.89:0/1157840487 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c001793f60 0x55c001b519d0 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:57.979 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:57.935+0000 7f0f5fc3e740 1 --2- 172.21.15.89:0/1157840487 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c001773280 0x55c001b45f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:57.979 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:57.936+0000 7f0f5fc3e740 1 --2- 172.21.15.89:0/1157840487 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c00176b0a0 0x55c001b459e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:57.979 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:57.936+0000 7f0f5fc3e740 1 -- 172.21.15.89:0/1157840487 >> 172.21.15.89:0/1157840487 conn(0x55c001788350 msgr2=0x55c001b3d0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:38:57.979 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:57.936+0000 7f0f3a7fc640 0 client.15291 ms_handle_reset on v2:172.21.15.110:6838/132366709 2024-09-21T03:38:57.979 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:57.936+0000 7f0f5fc3e740 1 -- 172.21.15.89:0/1157840487 shutdown_connections 2024-09-21T03:38:57.979 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:57.936+0000 7f0f5fc3e740 1 -- 172.21.15.89:0/1157840487 wait complete. 2024-09-21T03:38:57.979 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:57.936+0000 7f0f5fc3e740 20 client.15291 trim_cache size 0 max 16384 2024-09-21T03:38:57.979 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:57.953+0000 7f0f5fc3e740 1 Processor -- start 2024-09-21T03:38:57.980 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:57.954+0000 7f0f5fc3e740 1 -- start start 2024-09-21T03:38:57.980 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:57.954+0000 7f0f5fc3e740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c00176b0a0 0x55c001b57cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:57.980 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:57.954+0000 7f0f5fc3e740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c001773280 0x55c001b51940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:57.980 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:57.954+0000 7f0f5fc3e740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c001793f60 0x55c001b51e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:57.980 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:57.954+0000 7f0f5fc3e740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55c001b55ff0 con 0x55c001773280 2024-09-21T03:38:57.980 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:57.954+0000 7f0f5fc3e740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55c001b566c0 con 0x55c001793f60 2024-09-21T03:38:57.980 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:57.954+0000 7f0f5fc3e740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55c001b56d90 con 0x55c00176b0a0 2024-09-21T03:38:57.980 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:57.954+0000 7f0f527fc640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c001773280 0x55c001b51940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:57.980 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:57.954+0000 7f0f527fc640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c001773280 0x55c001b51940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:55388/0 (socket says 172.21.15.89:55388) 2024-09-21T03:38:57.980 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:57.954+0000 7f0f527fc640 1 -- 172.21.15.89:0/4216097130 learned_addr learned my addr 172.21.15.89:0/4216097130 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-21T03:38:57.980 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:57.954+0000 7f0f52ffd640 1 --2- 172.21.15.89:0/4216097130 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c00176b0a0 0x55c001b57cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:57.980 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:57.954+0000 7f0f527fc640 1 -- 172.21.15.89:0/4216097130 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c00176b0a0 msgr2=0x55c001b57cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:38:57.980 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:57.954+0000 7f0f527fc640 1 --2- 172.21.15.89:0/4216097130 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c00176b0a0 0x55c001b57cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:57.980 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:57.954+0000 7f0f527fc640 1 -- 172.21.15.89:0/4216097130 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c001793f60 msgr2=0x55c001b51e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:38:57.980 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:57.954+0000 7f0f527fc640 1 --2- 172.21.15.89:0/4216097130 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c001793f60 0x55c001b51e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:57.981 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:57.954+0000 7f0f527fc640 1 -- 172.21.15.89:0/4216097130 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c001b46aa0 con 0x55c001773280 2024-09-21T03:38:57.981 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:57.955+0000 7f0f527fc640 1 --2- 172.21.15.89:0/4216097130 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c001773280 0x55c001b51940 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f0f3c0acef0 tx=0x7f0f3c0a9090 comp rx=0 tx=0).ready entity=mon.0 client_cookie=caac4fb14299fc8b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:38:57.981 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:57.955+0000 7f0f48ff9640 1 -- 172.21.15.89:0/4216097130 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f3c01d040 con 0x55c001773280 2024-09-21T03:38:57.981 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:57.955+0000 7f0f48ff9640 1 -- 172.21.15.89:0/4216097130 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0f3c0acd00 con 0x55c001773280 2024-09-21T03:38:57.981 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:57.955+0000 7f0f48ff9640 1 -- 172.21.15.89:0/4216097130 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f3c0a9460 con 0x55c001773280 2024-09-21T03:38:57.981 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:57.956+0000 7f0f5fc3e740 1 -- 172.21.15.89:0/4216097130 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c001773280 msgr2=0x55c001b51940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:57.981 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:57.956+0000 7f0f5fc3e740 1 --2- 172.21.15.89:0/4216097130 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c001773280 0x55c001b51940 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f0f3c0acef0 tx=0x7f0f3c0a9090 comp rx=0 tx=0).stop 2024-09-21T03:38:57.981 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:57.956+0000 7f0f5fc3e740 1 -- 172.21.15.89:0/4216097130 shutdown_connections 2024-09-21T03:38:57.981 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:57.956+0000 7f0f5fc3e740 1 --2- 172.21.15.89:0/4216097130 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c001793f60 0x55c001b51e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:57.981 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:57.956+0000 7f0f5fc3e740 1 --2- 172.21.15.89:0/4216097130 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c001773280 0x55c001b51940 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:57.981 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:57.956+0000 7f0f5fc3e740 1 --2- 172.21.15.89:0/4216097130 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c00176b0a0 0x55c001b57cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:57.981 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:57.956+0000 7f0f5fc3e740 1 -- 172.21.15.89:0/4216097130 >> 172.21.15.89:0/4216097130 conn(0x55c001788350 msgr2=0x55c001b409c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:38:57.981 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:57.956+0000 7f0f5fc3e740 1 -- 172.21.15.89:0/4216097130 shutdown_connections 2024-09-21T03:38:57.981 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:57.956+0000 7f0f5fc3e740 1 -- 172.21.15.89:0/4216097130 wait complete. 2024-09-21T03:38:57.981 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:57.956+0000 7f0f5fc3e740 1 -- start start 2024-09-21T03:38:57.982 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:57.956+0000 7f0f5fc3e740 1 Processor -- start 2024-09-21T03:38:57.982 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:57.956+0000 7f0f5fc3e740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c00176b0a0 0x55c001b608c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:57.982 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:57.956+0000 7f0f5fc3e740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c001773280 0x55c001b5dd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:57.982 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:57.956+0000 7f0f5fc3e740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c001793f60 0x55c001b5e430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:57.982 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:57.957+0000 7f0f5fc3e740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55c001b57b30 con 0x55c001773280 2024-09-21T03:38:57.982 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:57.957+0000 7f0f5fc3e740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55c001b57460 con 0x55c00176b0a0 2024-09-21T03:38:57.982 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:57.957+0000 7f0f5fc3e740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55c001b58200 con 0x55c001793f60 2024-09-21T03:38:57.982 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:57.957+0000 7f0f5fc3e740 10 client.0 _ll_register_callbacks cb 0x7f0f4c01b070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:38:57.982 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:57.957+0000 7f0f527fc640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c001773280 0x55c001b5dd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:57.982 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:57.957+0000 7f0f527fc640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c001773280 0x55c001b5dd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:55396/0 (socket says 172.21.15.89:55396) 2024-09-21T03:38:57.982 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:57.957+0000 7f0f527fc640 1 -- 172.21.15.89:0/589188584 learned_addr learned my addr 172.21.15.89:0/589188584 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-21T03:38:57.982 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:57.957+0000 7f0f52ffd640 1 --2- 172.21.15.89:0/589188584 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c00176b0a0 0x55c001b608c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:57.982 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:57.957+0000 7f0f527fc640 1 -- 172.21.15.89:0/589188584 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c001793f60 msgr2=0x55c001b5e430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:38:57.983 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:57.957+0000 7f0f527fc640 1 --2- 172.21.15.89:0/589188584 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c001793f60 0x55c001b5e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:57.983 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:57.957+0000 7f0f527fc640 1 -- 172.21.15.89:0/589188584 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c00176b0a0 msgr2=0x55c001b608c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:38:57.983 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:57.957+0000 7f0f527fc640 1 --2- 172.21.15.89:0/589188584 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c00176b0a0 0x55c001b608c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:57.983 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:57.957+0000 7f0f527fc640 1 -- 172.21.15.89:0/589188584 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c001b5ebc0 con 0x55c001773280 2024-09-21T03:38:57.983 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:57.957+0000 7f0f527fc640 1 --2- 172.21.15.89:0/589188584 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c001773280 0x55c001b5dd90 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f0f3c00c940 tx=0x7f0f3c0b6bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8523ce19ced2f3fb server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:38:57.983 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:57.957+0000 7f0f3a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2024-09-21T03:38:57.983 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:57.957+0000 7f0f3a7fc640 1 -- 172.21.15.89:0/589188584 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f3c0d4020 con 0x55c001773280 2024-09-21T03:38:57.983 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:57.957+0000 7f0f3a7fc640 1 -- 172.21.15.89:0/589188584 <== mon.0 v2:172.21.15.89:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 9121+0+0 (secure 0 0 0) 0x7f0f3c0d0ac0 con 0x55c001773280 2024-09-21T03:38:57.983 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:57.957+0000 7f0f5fc3e740 1 -- 172.21.15.89:0/589188584 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0f3c0acd00 con 0x55c001773280 2024-09-21T03:38:57.983 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:57.957+0000 7f0f3a7fc640 1 -- 172.21.15.89:0/589188584 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f0f3c01d070 con 0x55c001773280 2024-09-21T03:38:57.983 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:57.958+0000 7f0f5fc3e740 10 client.15297 fetch_fsmap learned FSMap version 25 2024-09-21T03:38:57.983 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:57.958+0000 7f0f5fc3e740 1 -- 172.21.15.89:0/589188584 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55c001bc7260 con 0x55c001773280 2024-09-21T03:38:57.983 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:57.958+0000 7f0f3a7fc640 1 -- 172.21.15.89:0/589188584 <== mon.0 v2:172.21.15.89:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0f3c0acd00 con 0x55c001773280 2024-09-21T03:38:57.983 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:57.958+0000 7f0f3a7fc640 1 -- 172.21.15.89:0/589188584 <== mon.0 v2:172.21.15.89:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f0f3c0b2db0 con 0x55c001773280 2024-09-21T03:38:57.984 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:57.958+0000 7f0f5fc3e740 10 client.15297 fetch_fsmap finished waiting for FSMap version 25 2024-09-21T03:38:57.984 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:57.958+0000 7f0f5fc3e740 10 client.15297 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-21T03:38:57.984 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:57.958+0000 7f0f5fc3e740 20 client.15297 populate_metadata read hostname 'smithi089' 2024-09-21T03:38:57.984 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:57.958+0000 7f0f5fc3e740 1 --2- 172.21.15.89:0/589188584 >> [v2:172.21.15.110:6838/132366709,v1:172.21.15.110:6839/132366709] conn(0x55c001727c90 0x55c001bca650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:57.984 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:57.958+0000 7f0f5fc3e740 4 client.15297 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-21T03:38:57.984 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:57.958+0000 7f0f5fc3e740 1 -- 172.21.15.89:0/589188584 --> [v2:172.21.15.110:6838/132366709,v1:172.21.15.110:6839/132366709] -- command(tid 0: {"prefix": "status"}) -- 0x55c001aaf610 con 0x55c001727c90 2024-09-21T03:38:57.984 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:57.958+0000 7f0f52ffd640 1 --2- 172.21.15.89:0/589188584 >> [v2:172.21.15.110:6838/132366709,v1:172.21.15.110:6839/132366709] conn(0x55c001727c90 0x55c001bca650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:57.984 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:57.959+0000 7f0f3a7fc640 10 client.15297 ms_handle_connect on v2:172.21.15.110:6838/132366709 2024-09-21T03:38:57.984 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:57.959+0000 7f0f52ffd640 1 --2- 172.21.15.89:0/589188584 >> [v2:172.21.15.110:6838/132366709,v1:172.21.15.110:6839/132366709] conn(0x55c001727c90 0x55c001bca650 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dc2c8164eb178887 server_cookie=70ea82e124c4b41b in_seq=0 out_seq=0 2024-09-21T03:38:57.984 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:57.959+0000 7f0f3a7fc640 1 -- 172.21.15.89:0/589188584 <== mds.0 v2:172.21.15.110:6838/132366709 1 ==== command_reply(tid 0: 0 ) ==== 8+0+322 (crc 0 0 0) 0x55c001aaf610 con 0x55c001727c90 2024-09-21T03:38:57.984 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:57.959+0000 7f0f3a7fc640 10 client.15297 handle_command_reply: tid=0 2024-09-21T03:38:57.984 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:57.959+0000 7f0f5fc3e740 1 client.15297 shutdown 2024-09-21T03:38:57.985 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:57.959+0000 7f0f5fc3e740 1 -- 172.21.15.89:0/589188584 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c001773280 msgr2=0x55c001b5dd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:57.985 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:57.959+0000 7f0f5fc3e740 1 --2- 172.21.15.89:0/589188584 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c001773280 0x55c001b5dd90 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f0f3c00c940 tx=0x7f0f3c0b6bf0 comp rx=0 tx=0).stop 2024-09-21T03:38:57.985 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:57.959+0000 7f0f5fc3e740 1 -- 172.21.15.89:0/589188584 shutdown_connections 2024-09-21T03:38:57.985 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:57.959+0000 7f0f5fc3e740 1 --2- 172.21.15.89:0/589188584 >> [v2:172.21.15.110:6838/132366709,v1:172.21.15.110:6839/132366709] conn(0x55c001727c90 0x55c001bca650 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:57.985 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:57.959+0000 7f0f5fc3e740 1 --2- 172.21.15.89:0/589188584 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c001793f60 0x55c001b5e430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:57.985 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:57.959+0000 7f0f5fc3e740 1 --2- 172.21.15.89:0/589188584 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c001773280 0x55c001b5dd90 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:57.985 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:57.959+0000 7f0f3a7fc640 0 client.15297 ms_handle_reset on v2:172.21.15.110:6838/132366709 2024-09-21T03:38:57.985 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:57.959+0000 7f0f5fc3e740 1 --2- 172.21.15.89:0/589188584 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c00176b0a0 0x55c001b608c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:57.985 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:57.959+0000 7f0f5fc3e740 1 -- 172.21.15.89:0/589188584 >> 172.21.15.89:0/589188584 conn(0x55c001788350 msgr2=0x55c001b459c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:38:57.985 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:57.959+0000 7f0f5fc3e740 1 -- 172.21.15.89:0/589188584 shutdown_connections 2024-09-21T03:38:57.985 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:57.959+0000 7f0f5fc3e740 1 -- 172.21.15.89:0/589188584 wait complete. 2024-09-21T03:38:57.985 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:57.959+0000 7f0f5fc3e740 20 client.15297 trim_cache size 0 max 16384 2024-09-21T03:38:57.986 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-09-21T03:38:57.986 INFO:teuthology.orchestra.run.smithi089.stdout: "cluster_fsid": "7c374098-77c9-11ef-baf5-efdc52797490", 2024-09-21T03:38:57.986 INFO:teuthology.orchestra.run.smithi089.stdout: "whoami": 0, 2024-09-21T03:38:57.986 INFO:teuthology.orchestra.run.smithi089.stdout: "id": 24638, 2024-09-21T03:38:57.986 INFO:teuthology.orchestra.run.smithi089.stdout: "want_state": "up:active", 2024-09-21T03:38:57.986 INFO:teuthology.orchestra.run.smithi089.stdout: "state": "up:active", 2024-09-21T03:38:57.986 INFO:teuthology.orchestra.run.smithi089.stdout: "fs_name": "cephfs", 2024-09-21T03:38:57.986 INFO:teuthology.orchestra.run.smithi089.stdout: "rank_uptime": 104.106170392, 2024-09-21T03:38:57.986 INFO:teuthology.orchestra.run.smithi089.stdout: "mdsmap_epoch": 25, 2024-09-21T03:38:57.986 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap_epoch": 84, 2024-09-21T03:38:57.986 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap_epoch_barrier": 80, 2024-09-21T03:38:57.986 INFO:teuthology.orchestra.run.smithi089.stdout: "uptime": 114.573951968 2024-09-21T03:38:57.986 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-09-21T03:38:57.987 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:57.960+0000 7f0f596ca640 1 -- 172.21.15.89:0/2427177713 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f0f400b65a0 msgr2=0x7f0f400b8ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:57.987 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:57.960+0000 7f0f596ca640 1 --2- 172.21.15.89:0/2427177713 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f0f400b65a0 0x7f0f400b8ab0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f0f440098d0 tx=0x7f0f44031000 comp rx=0 tx=0).stop 2024-09-21T03:38:57.987 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:57.960+0000 7f0f596ca640 1 -- 172.21.15.89:0/2427177713 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0f54105170 msgr2=0x7f0f54199890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:57.988 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:57.960+0000 7f0f596ca640 1 --2- 172.21.15.89:0/2427177713 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0f54105170 0x7f0f54199890 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f0f3c00c970 tx=0x7f0f3c00ce30 comp rx=0 tx=0).stop 2024-09-21T03:38:57.988 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:57.960+0000 7f0f596ca640 1 -- 172.21.15.89:0/2427177713 shutdown_connections 2024-09-21T03:38:57.988 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:57.960+0000 7f0f596ca640 1 --2- 172.21.15.89:0/2427177713 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f0f400b65a0 0x7f0f400b8ab0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:57.988 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:57.960+0000 7f0f596ca640 1 --2- 172.21.15.89:0/2427177713 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0f54105ac0 0x7f0f5419eb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:57.988 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:57.960+0000 7f0f596ca640 1 --2- 172.21.15.89:0/2427177713 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0f54105170 0x7f0f54199890 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:57.988 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:57.960+0000 7f0f596ca640 1 --2- 172.21.15.89:0/2427177713 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0f54103160 0x7f0f541992f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:57.988 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:57.960+0000 7f0f596ca640 1 -- 172.21.15.89:0/2427177713 >> 172.21.15.89:0/2427177713 conn(0x7f0f54077c60 msgr2=0x7f0f54107080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:38:57.988 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:57.960+0000 7f0f596ca640 1 -- 172.21.15.89:0/2427177713 shutdown_connections 2024-09-21T03:38:57.988 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:57.960+0000 7f0f596ca640 1 -- 172.21.15.89:0/2427177713 wait complete. 2024-09-21T03:38:58.739 INFO:tasks.cephadm:Setup complete, yielding 2024-09-21T03:38:58.739 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-09-21T03:38:58.747 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi089.front.sepia.ceph.com 2024-09-21T03:38:58.748 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7c374098-77c9-11ef-baf5-efdc52797490 -- bash -c 'ceph orch status' 2024-09-21T03:38:58.961 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:38:58 smithi114 ceph-mon[34140]: pgmap v347: 129 pgs: 1 peering, 128 active+clean; 646 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 4 op/s 2024-09-21T03:38:58.995 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/7c374098-77c9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:38:59.000 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:38:58 smithi110 ceph-mon[34000]: pgmap v347: 129 pgs: 1 peering, 128 active+clean; 646 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 4 op/s 2024-09-21T03:38:59.061 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:38:58 smithi089 ceph-mon[30928]: pgmap v347: 129 pgs: 1 peering, 128 active+clean; 646 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 4 op/s 2024-09-21T03:38:59.521 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:59.504+0000 7f7837cb4640 1 -- 172.21.15.89:0/1526135206 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7830105210 msgr2=0x7f783010bbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:59.521 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:59.504+0000 7f7837cb4640 1 --2- 172.21.15.89:0/1526135206 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7830105210 0x7f783010bbb0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f7828009900 tx=0x7f782802a480 comp rx=0 tx=0).stop 2024-09-21T03:38:59.521 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:59.504+0000 7f7837cb4640 1 -- 172.21.15.89:0/1526135206 shutdown_connections 2024-09-21T03:38:59.522 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:59.504+0000 7f7837cb4640 1 --2- 172.21.15.89:0/1526135206 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7830104870 0x7f7830104c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:59.522 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:59.504+0000 7f7837cb4640 1 --2- 172.21.15.89:0/1526135206 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7830105210 0x7f783010bbb0 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:59.522 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:59.504+0000 7f7837cb4640 1 --2- 172.21.15.89:0/1526135206 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f78301037e0 0x7f7830103c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:59.522 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:59.504+0000 7f7837cb4640 1 -- 172.21.15.89:0/1526135206 >> 172.21.15.89:0/1526135206 conn(0x7f78300ff150 msgr2=0x7f7830101600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:38:59.522 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:59.504+0000 7f7837cb4640 1 -- 172.21.15.89:0/1526135206 shutdown_connections 2024-09-21T03:38:59.522 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:59.504+0000 7f7837cb4640 1 -- 172.21.15.89:0/1526135206 wait complete. 2024-09-21T03:38:59.522 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:59.505+0000 7f7837cb4640 1 Processor -- start 2024-09-21T03:38:59.522 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:59.505+0000 7f7837cb4640 1 -- start start 2024-09-21T03:38:59.522 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:59.505+0000 7f7837cb4640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f78301037e0 0x7f7830199f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:59.522 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:59.505+0000 7f7837cb4640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7830104870 0x7f783019a5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:59.522 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:59.505+0000 7f7837cb4640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7830105210 0x7f783019b0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:59.522 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:59.505+0000 7f7837cb4640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f783010f270 con 0x7f78301037e0 2024-09-21T03:38:59.522 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:59.505+0000 7f7837cb4640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f783010f0f0 con 0x7f7830105210 2024-09-21T03:38:59.522 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:59.505+0000 7f7837cb4640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f783010f3f0 con 0x7f7830104870 2024-09-21T03:38:59.523 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:59.505+0000 7f7835a50640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f78301037e0 0x7f7830199f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:59.523 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:59.505+0000 7f7835a50640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f78301037e0 0x7f7830199f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:47690/0 (socket says 172.21.15.89:47690) 2024-09-21T03:38:59.523 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:59.505+0000 7f7835a50640 1 -- 172.21.15.89:0/3192793198 learned_addr learned my addr 172.21.15.89:0/3192793198 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-21T03:38:59.523 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:59.505+0000 7f783524f640 1 --2- 172.21.15.89:0/3192793198 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7830104870 0x7f783019a5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:59.523 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:59.505+0000 7f7836251640 1 --2- 172.21.15.89:0/3192793198 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7830105210 0x7f783019b0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:59.523 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:59.505+0000 7f7835a50640 1 -- 172.21.15.89:0/3192793198 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7830104870 msgr2=0x7f783019a5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:59.523 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:59.505+0000 7f7835a50640 1 --2- 172.21.15.89:0/3192793198 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7830104870 0x7f783019a5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:59.523 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:59.505+0000 7f7835a50640 1 -- 172.21.15.89:0/3192793198 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7830105210 msgr2=0x7f783019b0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:59.523 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:59.505+0000 7f7835a50640 1 --2- 172.21.15.89:0/3192793198 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7830105210 0x7f783019b0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:59.523 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:59.505+0000 7f7835a50640 1 -- 172.21.15.89:0/3192793198 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f783019b910 con 0x7f78301037e0 2024-09-21T03:38:59.523 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:59.505+0000 7f7836251640 1 --2- 172.21.15.89:0/3192793198 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7830105210 0x7f783019b0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:38:59.523 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:59.506+0000 7f7835a50640 1 --2- 172.21.15.89:0/3192793198 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f78301037e0 0x7f7830199f50 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f782000ea80 tx=0x7f782000ef40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:38:59.523 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:59.506+0000 7f782effd640 1 -- 172.21.15.89:0/3192793198 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f782000ce40 con 0x7f78301037e0 2024-09-21T03:38:59.523 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:59.506+0000 7f782effd640 1 -- 172.21.15.89:0/3192793198 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7820005020 con 0x7f78301037e0 2024-09-21T03:38:59.524 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:59.506+0000 7f7837cb4640 1 -- 172.21.15.89:0/3192793198 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f783019bc00 con 0x7f78301037e0 2024-09-21T03:38:59.524 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:59.506+0000 7f782effd640 1 -- 172.21.15.89:0/3192793198 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78200116d0 con 0x7f78301037e0 2024-09-21T03:38:59.524 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:59.506+0000 7f7837cb4640 1 -- 172.21.15.89:0/3192793198 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78301a2300 con 0x7f78301037e0 2024-09-21T03:38:59.524 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:59.507+0000 7f7837cb4640 1 -- 172.21.15.89:0/3192793198 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f783010a9c0 con 0x7f78301037e0 2024-09-21T03:38:59.524 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:59.508+0000 7f782effd640 1 -- 172.21.15.89:0/3192793198 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 18) ==== 154563+0+0 (secure 0 0 0) 0x7f7820004a40 con 0x7f78301037e0 2024-09-21T03:38:59.524 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:59.509+0000 7f782effd640 1 --2- 172.21.15.89:0/3192793198 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f78180b65a0 0x7f78180b8ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:59.524 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:59.509+0000 7f783524f640 1 --2- 172.21.15.89:0/3192793198 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f78180b65a0 0x7f78180b8ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:59.524 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:59.509+0000 7f782effd640 1 -- 172.21.15.89:0/3192793198 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 9121+0+0 (secure 0 0 0) 0x7f78200e7e20 con 0x7f78301037e0 2024-09-21T03:38:59.524 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:59.509+0000 7f783524f640 1 --2- 172.21.15.89:0/3192793198 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f78180b65a0 0x7f78180b8ab0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f7824005d00 tx=0x7f782400b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:38:59.524 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:59.512+0000 7f782effd640 1 -- 172.21.15.89:0/3192793198 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f7820016030 con 0x7f78301037e0 2024-09-21T03:38:59.715 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:59.713+0000 7f7837cb4640 1 -- 172.21.15.89:0/3192793198 --> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) -- 0x7f7830063800 con 0x7f78180b65a0 2024-09-21T03:38:59.716 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:59.714+0000 7f782effd640 1 -- 172.21.15.89:0/3192793198 <== mgr.14152 v2:172.21.15.89:6800/1591532358 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+43 (secure 0 0 0) 0x7f7830063800 con 0x7f78180b65a0 2024-09-21T03:38:59.716 INFO:teuthology.orchestra.run.smithi089.stdout:Backend: cephadm 2024-09-21T03:38:59.716 INFO:teuthology.orchestra.run.smithi089.stdout:Available: Yes 2024-09-21T03:38:59.716 INFO:teuthology.orchestra.run.smithi089.stdout:Paused: No 2024-09-21T03:38:59.720 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:59.717+0000 7f7837cb4640 1 -- 172.21.15.89:0/3192793198 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f78180b65a0 msgr2=0x7f78180b8ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:59.720 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:59.717+0000 7f7837cb4640 1 --2- 172.21.15.89:0/3192793198 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f78180b65a0 0x7f78180b8ab0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f7824005d00 tx=0x7f782400b410 comp rx=0 tx=0).stop 2024-09-21T03:38:59.720 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:59.717+0000 7f7837cb4640 1 -- 172.21.15.89:0/3192793198 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f78301037e0 msgr2=0x7f7830199f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:59.720 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:59.717+0000 7f7837cb4640 1 --2- 172.21.15.89:0/3192793198 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f78301037e0 0x7f7830199f50 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f782000ea80 tx=0x7f782000ef40 comp rx=0 tx=0).stop 2024-09-21T03:38:59.720 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:59.717+0000 7f7837cb4640 1 -- 172.21.15.89:0/3192793198 shutdown_connections 2024-09-21T03:38:59.720 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:59.717+0000 7f7837cb4640 1 --2- 172.21.15.89:0/3192793198 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f78180b65a0 0x7f78180b8ab0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:59.720 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:59.717+0000 7f7837cb4640 1 --2- 172.21.15.89:0/3192793198 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7830105210 0x7f783019b0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:59.720 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:59.717+0000 7f7837cb4640 1 --2- 172.21.15.89:0/3192793198 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7830104870 0x7f783019a5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:59.720 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:59.717+0000 7f7837cb4640 1 --2- 172.21.15.89:0/3192793198 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f78301037e0 0x7f7830199f50 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:59.720 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:59.717+0000 7f7837cb4640 1 -- 172.21.15.89:0/3192793198 >> 172.21.15.89:0/3192793198 conn(0x7f78300ff150 msgr2=0x7f7830107b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:38:59.720 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:59.717+0000 7f7837cb4640 1 -- 172.21.15.89:0/3192793198 shutdown_connections 2024-09-21T03:38:59.720 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:38:59.717+0000 7f7837cb4640 1 -- 172.21.15.89:0/3192793198 wait complete. 2024-09-21T03:39:00.000 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:38:59 smithi110 ceph-mon[34000]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg inactive, 1 pg peering) 2024-09-21T03:39:00.000 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:38:59 smithi110 ceph-mon[34000]: Cluster is now healthy 2024-09-21T03:39:00.061 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:38:59 smithi089 ceph-mon[30928]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg inactive, 1 pg peering) 2024-09-21T03:39:00.061 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:38:59 smithi089 ceph-mon[30928]: Cluster is now healthy 2024-09-21T03:39:00.211 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:38:59 smithi114 ceph-mon[34140]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg inactive, 1 pg peering) 2024-09-21T03:39:00.211 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:38:59 smithi114 ceph-mon[34140]: Cluster is now healthy 2024-09-21T03:39:00.442 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7c374098-77c9-11ef-baf5-efdc52797490 -- bash -c 'ceph orch ps' 2024-09-21T03:39:00.689 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/7c374098-77c9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:39:01.000 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:39:00 smithi110 ceph-mon[34000]: pgmap v348: 129 pgs: 129 active+clean; 646 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-21T03:39:01.000 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:39:00 smithi110 ceph-mon[34000]: from='client.15303 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:39:01.061 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:39:00 smithi089 ceph-mon[30928]: pgmap v348: 129 pgs: 129 active+clean; 646 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-21T03:39:01.061 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:39:00 smithi089 ceph-mon[30928]: from='client.15303 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:39:01.187 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:01.185+0000 7f9658895640 1 -- 172.21.15.89:0/196626272 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9654103750 msgr2=0x7f9654103bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:01.188 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:01.185+0000 7f9658895640 1 --2- 172.21.15.89:0/196626272 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9654103750 0x7f9654103bc0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f9644009900 tx=0x7f964402a480 comp rx=0 tx=0).stop 2024-09-21T03:39:01.188 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:01.185+0000 7f9658895640 1 -- 172.21.15.89:0/196626272 shutdown_connections 2024-09-21T03:39:01.188 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:01.185+0000 7f9658895640 1 --2- 172.21.15.89:0/196626272 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9654105180 0x7f965410bb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:01.188 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:01.185+0000 7f9658895640 1 --2- 172.21.15.89:0/196626272 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f96541047e0 0x7f9654104bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:01.188 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:01.185+0000 7f9658895640 1 --2- 172.21.15.89:0/196626272 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9654103750 0x7f9654103bc0 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:01.188 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:01.185+0000 7f9658895640 1 -- 172.21.15.89:0/196626272 >> 172.21.15.89:0/196626272 conn(0x7f96540ff0e0 msgr2=0x7f9654101570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:39:01.188 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:01.185+0000 7f9658895640 1 -- 172.21.15.89:0/196626272 shutdown_connections 2024-09-21T03:39:01.188 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:01.185+0000 7f9658895640 1 -- 172.21.15.89:0/196626272 wait complete. 2024-09-21T03:39:01.190 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:01.186+0000 7f9658895640 1 Processor -- start 2024-09-21T03:39:01.190 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:01.186+0000 7f9658895640 1 -- start start 2024-09-21T03:39:01.190 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:01.186+0000 7f9658895640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9654103750 0x7f9654199f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:01.190 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:01.186+0000 7f9658895640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f96541047e0 0x7f965419a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:01.190 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:01.186+0000 7f9658895640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9654105180 0x7f965419b150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:01.190 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:01.186+0000 7f9658895640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f965410f220 con 0x7f96541047e0 2024-09-21T03:39:01.190 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:01.186+0000 7f9658895640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f965410f0a0 con 0x7f9654103750 2024-09-21T03:39:01.190 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:01.186+0000 7f9658895640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f965410f3a0 con 0x7f9654105180 2024-09-21T03:39:01.190 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:01.186+0000 7f964bfff640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f96541047e0 0x7f965419a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:01.190 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:01.186+0000 7f964bfff640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f96541047e0 0x7f965419a5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:47708/0 (socket says 172.21.15.89:47708) 2024-09-21T03:39:01.190 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:01.186+0000 7f964bfff640 1 -- 172.21.15.89:0/1440015117 learned_addr learned my addr 172.21.15.89:0/1440015117 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-21T03:39:01.190 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:01.186+0000 7f965259c640 1 --2- 172.21.15.89:0/1440015117 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9654103750 0x7f9654199f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:01.190 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:01.186+0000 7f9652d9d640 1 --2- 172.21.15.89:0/1440015117 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9654105180 0x7f965419b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:01.191 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:01.187+0000 7f964bfff640 1 -- 172.21.15.89:0/1440015117 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9654105180 msgr2=0x7f965419b150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:01.191 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:01.187+0000 7f964bfff640 1 --2- 172.21.15.89:0/1440015117 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9654105180 0x7f965419b150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:01.191 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:01.187+0000 7f964bfff640 1 -- 172.21.15.89:0/1440015117 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9654103750 msgr2=0x7f9654199f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:01.191 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:01.187+0000 7f964bfff640 1 --2- 172.21.15.89:0/1440015117 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9654103750 0x7f9654199f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:01.191 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:01.187+0000 7f964bfff640 1 -- 172.21.15.89:0/1440015117 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f965419b8e0 con 0x7f96541047e0 2024-09-21T03:39:01.191 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:01.187+0000 7f964bfff640 1 --2- 172.21.15.89:0/1440015117 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f96541047e0 0x7f965419a5b0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f963c00c890 tx=0x7f963c00cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:39:01.191 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:01.187+0000 7f964b7fe640 1 -- 172.21.15.89:0/1440015117 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f963c004220 con 0x7f96541047e0 2024-09-21T03:39:01.191 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:01.187+0000 7f9658895640 1 -- 172.21.15.89:0/1440015117 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f965419bbd0 con 0x7f96541047e0 2024-09-21T03:39:01.191 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:01.187+0000 7f964b7fe640 1 -- 172.21.15.89:0/1440015117 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f963c007e40 con 0x7f96541047e0 2024-09-21T03:39:01.191 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:01.187+0000 7f964b7fe640 1 -- 172.21.15.89:0/1440015117 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f963c004a00 con 0x7f96541047e0 2024-09-21T03:39:01.191 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:01.187+0000 7f9658895640 1 -- 172.21.15.89:0/1440015117 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96541a2360 con 0x7f96541047e0 2024-09-21T03:39:01.192 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:01.188+0000 7f9658895640 1 -- 172.21.15.89:0/1440015117 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f965410a930 con 0x7f96541047e0 2024-09-21T03:39:01.192 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:01.190+0000 7f964b7fe640 1 -- 172.21.15.89:0/1440015117 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 18) ==== 154563+0+0 (secure 0 0 0) 0x7f963c0079b0 con 0x7f96541047e0 2024-09-21T03:39:01.193 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:01.190+0000 7f964b7fe640 1 --2- 172.21.15.89:0/1440015117 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f96400b65a0 0x7f96400b8ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:01.193 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:01.191+0000 7f965259c640 1 --2- 172.21.15.89:0/1440015117 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f96400b65a0 0x7f96400b8ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:01.193 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:01.191+0000 7f964b7fe640 1 -- 172.21.15.89:0/1440015117 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 9121+0+0 (secure 0 0 0) 0x7f963c0e78b0 con 0x7f96541047e0 2024-09-21T03:39:01.193 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:01.191+0000 7f965259c640 1 --2- 172.21.15.89:0/1440015117 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f96400b65a0 0x7f96400b8ab0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f96440057c0 tx=0x7f9644000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:39:01.196 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:01.194+0000 7f964b7fe640 1 -- 172.21.15.89:0/1440015117 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f963c014030 con 0x7f96541047e0 2024-09-21T03:39:01.210 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:39:00 smithi114 ceph-mon[34140]: pgmap v348: 129 pgs: 129 active+clean; 646 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-21T03:39:01.211 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:39:00 smithi114 ceph-mon[34140]: from='client.15303 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:39:01.382 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:01.380+0000 7f9658895640 1 -- 172.21.15.89:0/1440015117 --> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7f9654063800 con 0x7f96400b65a0 2024-09-21T03:39:01.395 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:01.393+0000 7f964b7fe640 1 -- 172.21.15.89:0/1440015117 <== mgr.14152 v2:172.21.15.89:6800/1591532358 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+4185 (secure 0 0 0) 0x7f9654063800 con 0x7f96400b65a0 2024-09-21T03:39:01.395 INFO:teuthology.orchestra.run.smithi089.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-21T03:39:01.395 INFO:teuthology.orchestra.run.smithi089.stdout:mds.a smithi089 running (2m) 50s ago 2m 83.3M - 19.3.0-5120-g18432218 ddce307e1f32 a9b21a569206 2024-09-21T03:39:01.395 INFO:teuthology.orchestra.run.smithi089.stdout:mds.b smithi110 running (118s) 51s ago 117s 94.0M - 19.3.0-5120-g18432218 ddce307e1f32 469eb07a1cfb 2024-09-21T03:39:01.395 INFO:teuthology.orchestra.run.smithi089.stdout:mds.c smithi114 running (2m) 51s ago 2m 84.0M - 19.3.0-5120-g18432218 ddce307e1f32 bf3e0de614f4 2024-09-21T03:39:01.395 INFO:teuthology.orchestra.run.smithi089.stdout:mds.d smithi089 running (2m) 50s ago 2m 81.9M - 19.3.0-5120-g18432218 ddce307e1f32 a3f28b104710 2024-09-21T03:39:01.395 INFO:teuthology.orchestra.run.smithi089.stdout:mds.e smithi110 running (2m) 51s ago 2m 85.9M - 19.3.0-5120-g18432218 ddce307e1f32 368b2198fd45 2024-09-21T03:39:01.395 INFO:teuthology.orchestra.run.smithi089.stdout:mds.f smithi114 running (2m) 51s ago 2m 83.4M - 19.3.0-5120-g18432218 ddce307e1f32 843d5ceede5f 2024-09-21T03:39:01.395 INFO:teuthology.orchestra.run.smithi089.stdout:mds.g smithi089 running (2m) 50s ago 2m 82.2M - 19.3.0-5120-g18432218 ddce307e1f32 bdceadcf48c0 2024-09-21T03:39:01.395 INFO:teuthology.orchestra.run.smithi089.stdout:mds.h smithi110 running (2m) 51s ago 2m 84.2M - 19.3.0-5120-g18432218 ddce307e1f32 9ea35882772d 2024-09-21T03:39:01.396 INFO:teuthology.orchestra.run.smithi089.stdout:mds.i smithi114 running (2m) 51s ago 2m 87.2M - 19.3.0-5120-g18432218 ddce307e1f32 16eb77ff6645 2024-09-21T03:39:01.396 INFO:teuthology.orchestra.run.smithi089.stdout:mds.j smithi089 running (2m) 50s ago 2m 84.5M - 19.3.0-5120-g18432218 ddce307e1f32 67a53e1dba19 2024-09-21T03:39:01.396 INFO:teuthology.orchestra.run.smithi089.stdout:mds.k smithi110 running (2m) 51s ago 2m 81.7M - 19.3.0-5120-g18432218 ddce307e1f32 c0c7a27c8983 2024-09-21T03:39:01.396 INFO:teuthology.orchestra.run.smithi089.stdout:mds.l smithi114 running (2m) 51s ago 2m 82.2M - 19.3.0-5120-g18432218 ddce307e1f32 6dbceef3df18 2024-09-21T03:39:01.396 INFO:teuthology.orchestra.run.smithi089.stdout:mgr.x smithi089 *:9283,8765 running (9m) 50s ago 9m 592M - 19.3.0-5120-g18432218 ddce307e1f32 5ac528282a8c 2024-09-21T03:39:01.396 INFO:teuthology.orchestra.run.smithi089.stdout:mgr.y smithi110 *:8443,8765 running (8m) 51s ago 8m 523M - 19.3.0-5120-g18432218 ddce307e1f32 a8ac88b4b1dd 2024-09-21T03:39:01.396 INFO:teuthology.orchestra.run.smithi089.stdout:mgr.z smithi114 *:8443,8765 running (8m) 51s ago 8m 524M - 19.3.0-5120-g18432218 ddce307e1f32 78bce9af8cd4 2024-09-21T03:39:01.396 INFO:teuthology.orchestra.run.smithi089.stdout:mon.a smithi089 running (10m) 50s ago 10m 121M 2048M 19.3.0-5120-g18432218 ddce307e1f32 768910a1020e 2024-09-21T03:39:01.396 INFO:teuthology.orchestra.run.smithi089.stdout:mon.b smithi110 running (8m) 51s ago 8m 109M 2048M 19.3.0-5120-g18432218 ddce307e1f32 7e32224c6f9a 2024-09-21T03:39:01.396 INFO:teuthology.orchestra.run.smithi089.stdout:mon.c smithi114 running (8m) 51s ago 8m 109M 2048M 19.3.0-5120-g18432218 ddce307e1f32 1205e7833f22 2024-09-21T03:39:01.396 INFO:teuthology.orchestra.run.smithi089.stdout:osd.0 smithi089 running (7m) 50s ago 7m 125M 4271M 19.3.0-5120-g18432218 ddce307e1f32 0b2ede32cfc7 2024-09-21T03:39:01.397 INFO:teuthology.orchestra.run.smithi089.stdout:osd.1 smithi110 running (6m) 51s ago 6m 124M 4272M 19.3.0-5120-g18432218 ddce307e1f32 63ad62231ff2 2024-09-21T03:39:01.397 INFO:teuthology.orchestra.run.smithi089.stdout:osd.2 smithi114 running (6m) 51s ago 6m 115M 4272M 19.3.0-5120-g18432218 ddce307e1f32 3edce8a43c5b 2024-09-21T03:39:01.397 INFO:teuthology.orchestra.run.smithi089.stdout:osd.3 smithi089 running (6m) 50s ago 6m 111M 4271M 19.3.0-5120-g18432218 ddce307e1f32 ebca35608a2e 2024-09-21T03:39:01.397 INFO:teuthology.orchestra.run.smithi089.stdout:osd.4 smithi110 running (5m) 51s ago 5m 110M 4272M 19.3.0-5120-g18432218 ddce307e1f32 a2f1b2e2b128 2024-09-21T03:39:01.397 INFO:teuthology.orchestra.run.smithi089.stdout:osd.5 smithi114 running (5m) 51s ago 5m 120M 4272M 19.3.0-5120-g18432218 ddce307e1f32 45bdbb1a6250 2024-09-21T03:39:01.397 INFO:teuthology.orchestra.run.smithi089.stdout:osd.6 smithi089 running (4m) 50s ago 4m 118M 4271M 19.3.0-5120-g18432218 ddce307e1f32 3a375799f736 2024-09-21T03:39:01.397 INFO:teuthology.orchestra.run.smithi089.stdout:osd.7 smithi110 running (4m) 51s ago 4m 123M 4272M 19.3.0-5120-g18432218 ddce307e1f32 5223b04ac522 2024-09-21T03:39:01.397 INFO:teuthology.orchestra.run.smithi089.stdout:osd.8 smithi114 running (4m) 51s ago 4m 115M 4272M 19.3.0-5120-g18432218 ddce307e1f32 275dd45675fc 2024-09-21T03:39:01.397 INFO:teuthology.orchestra.run.smithi089.stdout:osd.9 smithi089 running (3m) 50s ago 3m 109M 4271M 19.3.0-5120-g18432218 ddce307e1f32 31b294330e44 2024-09-21T03:39:01.397 INFO:teuthology.orchestra.run.smithi089.stdout:osd.10 smithi110 running (3m) 51s ago 3m 119M 4272M 19.3.0-5120-g18432218 ddce307e1f32 b0203de153a7 2024-09-21T03:39:01.397 INFO:teuthology.orchestra.run.smithi089.stdout:osd.11 smithi114 running (2m) 51s ago 2m 120M 4272M 19.3.0-5120-g18432218 ddce307e1f32 1c802259437b 2024-09-21T03:39:01.398 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:01.395+0000 7f9658895640 1 -- 172.21.15.89:0/1440015117 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f96400b65a0 msgr2=0x7f96400b8ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:01.398 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:01.395+0000 7f9658895640 1 --2- 172.21.15.89:0/1440015117 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f96400b65a0 0x7f96400b8ab0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f96440057c0 tx=0x7f9644000bc0 comp rx=0 tx=0).stop 2024-09-21T03:39:01.398 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:01.395+0000 7f9658895640 1 -- 172.21.15.89:0/1440015117 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f96541047e0 msgr2=0x7f965419a5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:01.398 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:01.395+0000 7f9658895640 1 --2- 172.21.15.89:0/1440015117 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f96541047e0 0x7f965419a5b0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f963c00c890 tx=0x7f963c00cd50 comp rx=0 tx=0).stop 2024-09-21T03:39:01.398 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:01.395+0000 7f9658895640 1 -- 172.21.15.89:0/1440015117 shutdown_connections 2024-09-21T03:39:01.398 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:01.395+0000 7f9658895640 1 --2- 172.21.15.89:0/1440015117 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f96400b65a0 0x7f96400b8ab0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:01.398 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:01.395+0000 7f9658895640 1 --2- 172.21.15.89:0/1440015117 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9654105180 0x7f965419b150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:01.398 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:01.395+0000 7f9658895640 1 --2- 172.21.15.89:0/1440015117 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f96541047e0 0x7f965419a5b0 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:01.398 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:01.395+0000 7f9658895640 1 --2- 172.21.15.89:0/1440015117 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9654103750 0x7f9654199f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:01.398 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:01.395+0000 7f9658895640 1 -- 172.21.15.89:0/1440015117 >> 172.21.15.89:0/1440015117 conn(0x7f96540ff0e0 msgr2=0x7f9654107b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:39:01.398 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:01.395+0000 7f9658895640 1 -- 172.21.15.89:0/1440015117 shutdown_connections 2024-09-21T03:39:01.399 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:01.395+0000 7f9658895640 1 -- 172.21.15.89:0/1440015117 wait complete. 2024-09-21T03:39:02.000 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:39:01 smithi110 ceph-mon[34000]: from='client.15309 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:39:02.061 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:39:01 smithi089 ceph-mon[30928]: from='client.15309 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:39:02.133 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7c374098-77c9-11ef-baf5-efdc52797490 -- bash -c 'ceph orch ls' 2024-09-21T03:39:02.211 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:39:01 smithi114 ceph-mon[34140]: from='client.15309 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:39:02.379 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/7c374098-77c9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:39:02.875 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:02.867+0000 7fc8fc80a640 1 -- 172.21.15.89:0/2740112740 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc8f8102650 msgr2=0x7fc8f8102ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:02.875 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:02.867+0000 7fc8fc80a640 1 --2- 172.21.15.89:0/2740112740 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc8f8102650 0x7fc8f8102ac0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fc8e0009900 tx=0x7fc8e002a480 comp rx=0 tx=0).stop 2024-09-21T03:39:02.876 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:02.867+0000 7fc8fc80a640 1 -- 172.21.15.89:0/2740112740 shutdown_connections 2024-09-21T03:39:02.876 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:02.867+0000 7fc8fc80a640 1 --2- 172.21.15.89:0/2740112740 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc8f8104080 0x7fc8f810aa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:02.876 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:02.867+0000 7fc8fc80a640 1 --2- 172.21.15.89:0/2740112740 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc8f81036e0 0x7fc8f8103af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:02.876 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:02.867+0000 7fc8fc80a640 1 --2- 172.21.15.89:0/2740112740 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc8f8102650 0x7fc8f8102ac0 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:02.876 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:02.867+0000 7fc8fc80a640 1 -- 172.21.15.89:0/2740112740 >> 172.21.15.89:0/2740112740 conn(0x7fc8f80fdfe0 msgr2=0x7fc8f8100470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:39:02.876 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:02.867+0000 7fc8fc80a640 1 -- 172.21.15.89:0/2740112740 shutdown_connections 2024-09-21T03:39:02.876 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:02.867+0000 7fc8fc80a640 1 -- 172.21.15.89:0/2740112740 wait complete. 2024-09-21T03:39:02.876 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:02.868+0000 7fc8fc80a640 1 Processor -- start 2024-09-21T03:39:02.876 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:02.868+0000 7fc8fc80a640 1 -- start start 2024-09-21T03:39:02.876 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:02.868+0000 7fc8fc80a640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc8f8102650 0x7fc8f8198e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:02.876 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:02.868+0000 7fc8fc80a640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc8f81036e0 0x7fc8f81994d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:02.876 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:02.868+0000 7fc8fc80a640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc8f8104080 0x7fc8f819a070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:02.876 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:02.868+0000 7fc8fc80a640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fc8f810e1a0 con 0x7fc8f81036e0 2024-09-21T03:39:02.877 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:02.868+0000 7fc8fc80a640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fc8f810e020 con 0x7fc8f8102650 2024-09-21T03:39:02.877 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:02.868+0000 7fc8fc80a640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fc8f810e320 con 0x7fc8f8104080 2024-09-21T03:39:02.877 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:02.868+0000 7fc8f5d9b640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc8f81036e0 0x7fc8f81994d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:02.877 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:02.868+0000 7fc8f5d9b640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc8f81036e0 0x7fc8f81994d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:47718/0 (socket says 172.21.15.89:47718) 2024-09-21T03:39:02.877 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:02.868+0000 7fc8f5d9b640 1 -- 172.21.15.89:0/434569286 learned_addr learned my addr 172.21.15.89:0/434569286 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-21T03:39:02.877 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:02.868+0000 7fc8f659c640 1 --2- 172.21.15.89:0/434569286 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc8f8102650 0x7fc8f8198e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:02.877 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:02.869+0000 7fc8f6d9d640 1 --2- 172.21.15.89:0/434569286 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc8f8104080 0x7fc8f819a070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:02.877 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:02.869+0000 7fc8f5d9b640 1 -- 172.21.15.89:0/434569286 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc8f8104080 msgr2=0x7fc8f819a070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:02.877 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:02.869+0000 7fc8f5d9b640 1 --2- 172.21.15.89:0/434569286 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc8f8104080 0x7fc8f819a070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:02.877 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:02.869+0000 7fc8f5d9b640 1 -- 172.21.15.89:0/434569286 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc8f8102650 msgr2=0x7fc8f8198e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:02.877 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:02.869+0000 7fc8f5d9b640 1 --2- 172.21.15.89:0/434569286 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc8f8102650 0x7fc8f8198e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:02.877 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:02.869+0000 7fc8f5d9b640 1 -- 172.21.15.89:0/434569286 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8f819a890 con 0x7fc8f81036e0 2024-09-21T03:39:02.877 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:02.869+0000 7fc8f6d9d640 1 --2- 172.21.15.89:0/434569286 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc8f8104080 0x7fc8f819a070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:39:02.878 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:02.869+0000 7fc8f5d9b640 1 --2- 172.21.15.89:0/434569286 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc8f81036e0 0x7fc8f81994d0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fc8e800b4a0 tx=0x7fc8e800b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:39:02.878 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:02.869+0000 7fc8ef7fe640 1 -- 172.21.15.89:0/434569286 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8e8014070 con 0x7fc8f81036e0 2024-09-21T03:39:02.878 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:02.869+0000 7fc8ef7fe640 1 -- 172.21.15.89:0/434569286 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc8e8010aa0 con 0x7fc8f81036e0 2024-09-21T03:39:02.878 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:02.869+0000 7fc8fc80a640 1 -- 172.21.15.89:0/434569286 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc8f819ab80 con 0x7fc8f81036e0 2024-09-21T03:39:02.878 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:02.869+0000 7fc8ef7fe640 1 -- 172.21.15.89:0/434569286 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8e8017b70 con 0x7fc8f81036e0 2024-09-21T03:39:02.878 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:02.869+0000 7fc8fc80a640 1 -- 172.21.15.89:0/434569286 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc8f81a1280 con 0x7fc8f81036e0 2024-09-21T03:39:02.878 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:02.870+0000 7fc8fc80a640 1 -- 172.21.15.89:0/434569286 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc8f8109830 con 0x7fc8f81036e0 2024-09-21T03:39:02.878 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:02.872+0000 7fc8ef7fe640 1 -- 172.21.15.89:0/434569286 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 18) ==== 154563+0+0 (secure 0 0 0) 0x7fc8e8017070 con 0x7fc8f81036e0 2024-09-21T03:39:02.878 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:02.873+0000 7fc8ef7fe640 1 --2- 172.21.15.89:0/434569286 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7fc8e40b6620 0x7fc8e40b8b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:02.878 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:02.873+0000 7fc8ef7fe640 1 -- 172.21.15.89:0/434569286 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 9121+0+0 (secure 0 0 0) 0x7fc8e80e6ec0 con 0x7fc8f81036e0 2024-09-21T03:39:02.878 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:02.873+0000 7fc8f659c640 1 --2- 172.21.15.89:0/434569286 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7fc8e40b6620 0x7fc8e40b8b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:02.878 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:02.873+0000 7fc8f659c640 1 --2- 172.21.15.89:0/434569286 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7fc8e40b6620 0x7fc8e40b8b30 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fc8e00057c0 tx=0x7fc8e0000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:39:02.879 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:02.876+0000 7fc8ef7fe640 1 -- 172.21.15.89:0/434569286 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fc8e80ad270 con 0x7fc8f81036e0 2024-09-21T03:39:03.061 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:39:02 smithi089 ceph-mon[30928]: pgmap v349: 129 pgs: 129 active+clean; 646 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-21T03:39:03.064 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:03.063+0000 7fc8fc80a640 1 -- 172.21.15.89:0/434569286 --> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) -- 0x7fc8f8063800 con 0x7fc8e40b6620 2024-09-21T03:39:03.069 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:03.068+0000 7fc8ef7fe640 1 -- 172.21.15.89:0/434569286 <== mgr.14152 v2:172.21.15.89:6800/1591532358 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+980 (secure 0 0 0) 0x7fc8f8063800 con 0x7fc8e40b6620 2024-09-21T03:39:03.070 INFO:teuthology.orchestra.run.smithi089.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2024-09-21T03:39:03.070 INFO:teuthology.orchestra.run.smithi089.stdout:mds.all 12/12 53s ago 2m smithi089=a;smithi089=d;smithi089=g;smithi089=j;smithi110=b;smithi110=e;smithi110=h;smithi110=k;smithi114=c;smithi114=f;smithi114=i;smithi114=l;count:12 2024-09-21T03:39:03.070 INFO:teuthology.orchestra.run.smithi089.stdout:mgr 3/3 53s ago 8m smithi089=x;smithi110=y;smithi114=z;count:3 2024-09-21T03:39:03.070 INFO:teuthology.orchestra.run.smithi089.stdout:mon 3/3 53s ago 8m smithi089:172.21.15.89=a;smithi110:172.21.15.110=b;smithi114:172.21.15.114=c;count:3 2024-09-21T03:39:03.070 INFO:teuthology.orchestra.run.smithi089.stdout:osd 12 53s ago - 2024-09-21T03:39:03.072 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:03.071+0000 7fc8fc80a640 1 -- 172.21.15.89:0/434569286 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7fc8e40b6620 msgr2=0x7fc8e40b8b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:03.072 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:03.071+0000 7fc8fc80a640 1 --2- 172.21.15.89:0/434569286 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7fc8e40b6620 0x7fc8e40b8b30 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fc8e00057c0 tx=0x7fc8e0000bc0 comp rx=0 tx=0).stop 2024-09-21T03:39:03.072 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:03.071+0000 7fc8fc80a640 1 -- 172.21.15.89:0/434569286 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc8f81036e0 msgr2=0x7fc8f81994d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:03.072 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:03.071+0000 7fc8fc80a640 1 --2- 172.21.15.89:0/434569286 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc8f81036e0 0x7fc8f81994d0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fc8e800b4a0 tx=0x7fc8e800b960 comp rx=0 tx=0).stop 2024-09-21T03:39:03.072 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:03.071+0000 7fc8fc80a640 1 -- 172.21.15.89:0/434569286 shutdown_connections 2024-09-21T03:39:03.072 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:03.071+0000 7fc8fc80a640 1 --2- 172.21.15.89:0/434569286 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7fc8e40b6620 0x7fc8e40b8b30 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:03.072 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:03.071+0000 7fc8fc80a640 1 --2- 172.21.15.89:0/434569286 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc8f8104080 0x7fc8f819a070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:03.072 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:03.071+0000 7fc8fc80a640 1 --2- 172.21.15.89:0/434569286 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc8f81036e0 0x7fc8f81994d0 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:03.073 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:03.071+0000 7fc8fc80a640 1 --2- 172.21.15.89:0/434569286 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc8f8102650 0x7fc8f8198e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:03.073 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:03.071+0000 7fc8fc80a640 1 -- 172.21.15.89:0/434569286 >> 172.21.15.89:0/434569286 conn(0x7fc8f80fdfe0 msgr2=0x7fc8f8106a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:39:03.073 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:03.071+0000 7fc8fc80a640 1 -- 172.21.15.89:0/434569286 shutdown_connections 2024-09-21T03:39:03.073 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:03.071+0000 7fc8fc80a640 1 -- 172.21.15.89:0/434569286 wait complete. 2024-09-21T03:39:03.112 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:39:02 smithi114 ceph-mon[34140]: pgmap v349: 129 pgs: 129 active+clean; 646 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-21T03:39:03.250 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:39:02 smithi110 ceph-mon[34000]: pgmap v349: 129 pgs: 129 active+clean; 646 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-21T03:39:03.852 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7c374098-77c9-11ef-baf5-efdc52797490 -- bash -c 'ceph orch host ls' 2024-09-21T03:39:04.061 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:39:03 smithi089 ceph-mon[30928]: from='client.15315 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:39:04.100 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/7c374098-77c9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:39:04.151 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:39:03 smithi110 ceph-mon[34000]: from='client.15315 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:39:04.211 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:39:03 smithi114 ceph-mon[34140]: from='client.15315 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:39:04.588 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:04.585+0000 7f737b901640 1 -- 172.21.15.89:0/3088014759 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f73741047e0 msgr2=0x7f7374104bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:04.588 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:04.585+0000 7f737b901640 1 --2- 172.21.15.89:0/3088014759 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f73741047e0 0x7f7374104bf0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f7364009900 tx=0x7f736402a480 comp rx=0 tx=0).stop 2024-09-21T03:39:04.588 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:04.585+0000 7f737b901640 1 -- 172.21.15.89:0/3088014759 shutdown_connections 2024-09-21T03:39:04.588 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:04.585+0000 7f737b901640 1 --2- 172.21.15.89:0/3088014759 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7374105180 0x7f737410bb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:04.588 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:04.585+0000 7f737b901640 1 --2- 172.21.15.89:0/3088014759 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f73741047e0 0x7f7374104bf0 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:04.588 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:04.585+0000 7f737b901640 1 --2- 172.21.15.89:0/3088014759 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7374103750 0x7f7374103bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:04.589 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:04.585+0000 7f737b901640 1 -- 172.21.15.89:0/3088014759 >> 172.21.15.89:0/3088014759 conn(0x7f73740ff0c0 msgr2=0x7f7374101570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:39:04.589 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:04.585+0000 7f737b901640 1 -- 172.21.15.89:0/3088014759 shutdown_connections 2024-09-21T03:39:04.589 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:04.585+0000 7f737b901640 1 -- 172.21.15.89:0/3088014759 wait complete. 2024-09-21T03:39:04.589 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:04.585+0000 7f737b901640 1 Processor -- start 2024-09-21T03:39:04.589 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:04.585+0000 7f737b901640 1 -- start start 2024-09-21T03:39:04.589 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:04.586+0000 7f737b901640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7374103750 0x7f7374199ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:04.589 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:04.586+0000 7f737b901640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f73741047e0 0x7f737419a530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:04.589 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:04.586+0000 7f737b901640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7374105180 0x7f737419b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:04.589 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:04.586+0000 7f737b901640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f737410f230 con 0x7f73741047e0 2024-09-21T03:39:04.589 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:04.586+0000 7f737b901640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f737410f0b0 con 0x7f7374105180 2024-09-21T03:39:04.589 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:04.586+0000 7f737b901640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f737410f3b0 con 0x7f7374103750 2024-09-21T03:39:04.589 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:04.586+0000 7f7378e9c640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f73741047e0 0x7f737419a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:04.589 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:04.586+0000 7f7378e9c640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f73741047e0 0x7f737419a530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:47730/0 (socket says 172.21.15.89:47730) 2024-09-21T03:39:04.589 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:04.586+0000 7f7378e9c640 1 -- 172.21.15.89:0/2971217478 learned_addr learned my addr 172.21.15.89:0/2971217478 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-21T03:39:04.589 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:04.586+0000 7f737969d640 1 --2- 172.21.15.89:0/2971217478 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7374103750 0x7f7374199ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:04.590 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:04.586+0000 7f7379e9e640 1 --2- 172.21.15.89:0/2971217478 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7374105180 0x7f737419b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:04.590 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:04.586+0000 7f7378e9c640 1 -- 172.21.15.89:0/2971217478 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7374103750 msgr2=0x7f7374199ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:04.590 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:04.586+0000 7f7378e9c640 1 --2- 172.21.15.89:0/2971217478 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7374103750 0x7f7374199ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:04.590 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:04.586+0000 7f7378e9c640 1 -- 172.21.15.89:0/2971217478 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7374105180 msgr2=0x7f737419b0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:04.590 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:04.586+0000 7f7378e9c640 1 --2- 172.21.15.89:0/2971217478 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7374105180 0x7f737419b0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:04.590 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:04.586+0000 7f7378e9c640 1 -- 172.21.15.89:0/2971217478 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f737419b8f0 con 0x7f73741047e0 2024-09-21T03:39:04.590 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:04.586+0000 7f7379e9e640 1 --2- 172.21.15.89:0/2971217478 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7374105180 0x7f737419b0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:39:04.590 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:04.586+0000 7f7378e9c640 1 --2- 172.21.15.89:0/2971217478 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f73741047e0 0x7f737419a530 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f7364009230 tx=0x7f73640042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:39:04.590 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:04.587+0000 7f736a7fc640 1 -- 172.21.15.89:0/2971217478 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f736403f030 con 0x7f73741047e0 2024-09-21T03:39:04.590 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:04.587+0000 7f737b901640 1 -- 172.21.15.89:0/2971217478 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f737419bb80 con 0x7f73741047e0 2024-09-21T03:39:04.590 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:04.587+0000 7f736a7fc640 1 -- 172.21.15.89:0/2971217478 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7364003c90 con 0x7f73741047e0 2024-09-21T03:39:04.590 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:04.587+0000 7f736a7fc640 1 -- 172.21.15.89:0/2971217478 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7364035070 con 0x7f73741047e0 2024-09-21T03:39:04.590 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:04.587+0000 7f737b901640 1 -- 172.21.15.89:0/2971217478 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f73741a2110 con 0x7f73741047e0 2024-09-21T03:39:04.591 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:04.588+0000 7f737b901640 1 -- 172.21.15.89:0/2971217478 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f737410a930 con 0x7f73741047e0 2024-09-21T03:39:04.591 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:04.589+0000 7f736a7fc640 1 -- 172.21.15.89:0/2971217478 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 18) ==== 154563+0+0 (secure 0 0 0) 0x7f7364042030 con 0x7f73741047e0 2024-09-21T03:39:04.591 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:04.590+0000 7f736a7fc640 1 --2- 172.21.15.89:0/2971217478 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f73600b65a0 0x7f73600b8ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:04.591 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:04.590+0000 7f737969d640 1 --2- 172.21.15.89:0/2971217478 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f73600b65a0 0x7f73600b8ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:04.591 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:04.590+0000 7f736a7fc640 1 -- 172.21.15.89:0/2971217478 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 9121+0+0 (secure 0 0 0) 0x7f73641082d0 con 0x7f73741047e0 2024-09-21T03:39:04.592 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:04.591+0000 7f737969d640 1 --2- 172.21.15.89:0/2971217478 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f73600b65a0 0x7f73600b8ab0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f73700058c0 tx=0x7f737000a2d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:39:04.594 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:04.593+0000 7f736a7fc640 1 -- 172.21.15.89:0/2971217478 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f73640ce680 con 0x7f73741047e0 2024-09-21T03:39:04.783 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:04.782+0000 7f737b901640 1 -- 172.21.15.89:0/2971217478 --> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) -- 0x7f7374063800 con 0x7f73600b65a0 2024-09-21T03:39:04.784 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:04.783+0000 7f736a7fc640 1 -- 172.21.15.89:0/2971217478 <== mgr.14152 v2:172.21.15.89:6800/1591532358 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+191 (secure 0 0 0) 0x7f7374063800 con 0x7f73600b65a0 2024-09-21T03:39:04.785 INFO:teuthology.orchestra.run.smithi089.stdout:HOST ADDR LABELS STATUS 2024-09-21T03:39:04.785 INFO:teuthology.orchestra.run.smithi089.stdout:smithi089 172.21.15.89 2024-09-21T03:39:04.785 INFO:teuthology.orchestra.run.smithi089.stdout:smithi110 172.21.15.110 2024-09-21T03:39:04.785 INFO:teuthology.orchestra.run.smithi089.stdout:smithi114 172.21.15.114 2024-09-21T03:39:04.785 INFO:teuthology.orchestra.run.smithi089.stdout:3 hosts in cluster 2024-09-21T03:39:04.787 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:04.786+0000 7f737b901640 1 -- 172.21.15.89:0/2971217478 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f73600b65a0 msgr2=0x7f73600b8ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:04.787 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:04.786+0000 7f737b901640 1 --2- 172.21.15.89:0/2971217478 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f73600b65a0 0x7f73600b8ab0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f73700058c0 tx=0x7f737000a2d0 comp rx=0 tx=0).stop 2024-09-21T03:39:04.787 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:04.786+0000 7f737b901640 1 -- 172.21.15.89:0/2971217478 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f73741047e0 msgr2=0x7f737419a530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:04.787 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:04.786+0000 7f737b901640 1 --2- 172.21.15.89:0/2971217478 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f73741047e0 0x7f737419a530 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f7364009230 tx=0x7f73640042f0 comp rx=0 tx=0).stop 2024-09-21T03:39:04.787 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:04.786+0000 7f737b901640 1 -- 172.21.15.89:0/2971217478 shutdown_connections 2024-09-21T03:39:04.788 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:04.786+0000 7f737b901640 1 --2- 172.21.15.89:0/2971217478 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f73600b65a0 0x7f73600b8ab0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:04.788 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:04.786+0000 7f737b901640 1 --2- 172.21.15.89:0/2971217478 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7374105180 0x7f737419b0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:04.788 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:04.786+0000 7f737b901640 1 --2- 172.21.15.89:0/2971217478 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f73741047e0 0x7f737419a530 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:04.788 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:04.786+0000 7f737b901640 1 --2- 172.21.15.89:0/2971217478 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7374103750 0x7f7374199ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:04.788 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:04.786+0000 7f737b901640 1 -- 172.21.15.89:0/2971217478 >> 172.21.15.89:0/2971217478 conn(0x7f73740ff0c0 msgr2=0x7f7374107b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:39:04.788 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:04.786+0000 7f737b901640 1 -- 172.21.15.89:0/2971217478 shutdown_connections 2024-09-21T03:39:04.788 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:04.786+0000 7f737b901640 1 -- 172.21.15.89:0/2971217478 wait complete. 2024-09-21T03:39:05.061 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:39:04 smithi089 ceph-mon[30928]: pgmap v350: 129 pgs: 129 active+clean; 646 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-09-21T03:39:05.211 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:39:04 smithi114 ceph-mon[34140]: pgmap v350: 129 pgs: 129 active+clean; 646 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-09-21T03:39:05.250 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:39:04 smithi110 ceph-mon[34000]: pgmap v350: 129 pgs: 129 active+clean; 646 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-09-21T03:39:05.531 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7c374098-77c9-11ef-baf5-efdc52797490 -- bash -c 'ceph orch device ls' 2024-09-21T03:39:05.780 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/7c374098-77c9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:39:06.054 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:39:05 smithi089 ceph-mon[30928]: from='client.15321 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:39:06.211 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:39:05 smithi114 ceph-mon[34140]: from='client.15321 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:39:06.250 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:39:05 smithi110 ceph-mon[34000]: from='client.15321 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:39:06.264 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:06.262+0000 7f5ad0051640 1 -- 172.21.15.89:0/2726537227 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5ac8105180 msgr2=0x7f5ac810bb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:06.264 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:06.262+0000 7f5ad0051640 1 --2- 172.21.15.89:0/2726537227 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5ac8105180 0x7f5ac810bb20 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f5ac0009ca0 tx=0x7f5ac002a950 comp rx=0 tx=0).stop 2024-09-21T03:39:06.264 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:06.262+0000 7f5ad0051640 1 -- 172.21.15.89:0/2726537227 shutdown_connections 2024-09-21T03:39:06.264 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:06.262+0000 7f5ad0051640 1 --2- 172.21.15.89:0/2726537227 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5ac8105180 0x7f5ac810bb20 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:06.264 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:06.262+0000 7f5ad0051640 1 --2- 172.21.15.89:0/2726537227 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5ac81047e0 0x7f5ac8104bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:06.264 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:06.262+0000 7f5ad0051640 1 --2- 172.21.15.89:0/2726537227 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5ac8103750 0x7f5ac8103bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:06.264 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:06.262+0000 7f5ad0051640 1 -- 172.21.15.89:0/2726537227 >> 172.21.15.89:0/2726537227 conn(0x7f5ac80ff0c0 msgr2=0x7f5ac8101570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:39:06.264 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:06.262+0000 7f5ad0051640 1 -- 172.21.15.89:0/2726537227 shutdown_connections 2024-09-21T03:39:06.265 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:06.262+0000 7f5ad0051640 1 -- 172.21.15.89:0/2726537227 wait complete. 2024-09-21T03:39:06.266 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:06.263+0000 7f5ad0051640 1 Processor -- start 2024-09-21T03:39:06.266 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:06.263+0000 7f5ad0051640 1 -- start start 2024-09-21T03:39:06.267 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:06.264+0000 7f5ad0051640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5ac8103750 0x7f5ac8199ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:06.267 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:06.264+0000 7f5ad0051640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5ac81047e0 0x7f5ac819a550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:06.267 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:06.264+0000 7f5ad0051640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5ac8105180 0x7f5ac819b0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:06.267 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:06.264+0000 7f5ad0051640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f5ac810f2b0 con 0x7f5ac8103750 2024-09-21T03:39:06.267 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:06.264+0000 7f5ad0051640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f5ac810f130 con 0x7f5ac8105180 2024-09-21T03:39:06.267 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:06.264+0000 7f5ad0051640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f5ac810f430 con 0x7f5ac81047e0 2024-09-21T03:39:06.267 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:06.264+0000 7f5acdded640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5ac8103750 0x7f5ac8199ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:06.267 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:06.264+0000 7f5acdded640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5ac8103750 0x7f5ac8199ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:47740/0 (socket says 172.21.15.89:47740) 2024-09-21T03:39:06.267 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:06.264+0000 7f5acdded640 1 -- 172.21.15.89:0/704448140 learned_addr learned my addr 172.21.15.89:0/704448140 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-21T03:39:06.267 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:06.264+0000 7f5acd5ec640 1 --2- 172.21.15.89:0/704448140 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5ac81047e0 0x7f5ac819a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:06.267 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:06.264+0000 7f5ace5ee640 1 --2- 172.21.15.89:0/704448140 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5ac8105180 0x7f5ac819b0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:06.267 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:06.264+0000 7f5acdded640 1 -- 172.21.15.89:0/704448140 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5ac81047e0 msgr2=0x7f5ac819a550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:06.267 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:06.264+0000 7f5acdded640 1 --2- 172.21.15.89:0/704448140 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5ac81047e0 0x7f5ac819a550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:06.267 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:06.264+0000 7f5acdded640 1 -- 172.21.15.89:0/704448140 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5ac8105180 msgr2=0x7f5ac819b0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:06.268 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:06.264+0000 7f5acdded640 1 --2- 172.21.15.89:0/704448140 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5ac8105180 0x7f5ac819b0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:06.268 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:06.264+0000 7f5acdded640 1 -- 172.21.15.89:0/704448140 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ac819b910 con 0x7f5ac8103750 2024-09-21T03:39:06.268 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:06.264+0000 7f5acd5ec640 1 --2- 172.21.15.89:0/704448140 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5ac81047e0 0x7f5ac819a550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:39:06.268 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:06.264+0000 7f5acdded640 1 --2- 172.21.15.89:0/704448140 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5ac8103750 0x7f5ac8199ee0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f5ab800c890 tx=0x7f5ab800cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:39:06.268 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:06.264+0000 7f5ac6ffd640 1 -- 172.21.15.89:0/704448140 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ab8004220 con 0x7f5ac8103750 2024-09-21T03:39:06.268 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:06.265+0000 7f5ad0051640 1 -- 172.21.15.89:0/704448140 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ac819bc00 con 0x7f5ac8103750 2024-09-21T03:39:06.268 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:06.265+0000 7f5ac6ffd640 1 -- 172.21.15.89:0/704448140 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5ab8007e40 con 0x7f5ac8103750 2024-09-21T03:39:06.268 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:06.265+0000 7f5ac6ffd640 1 -- 172.21.15.89:0/704448140 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ab8004a00 con 0x7f5ac8103750 2024-09-21T03:39:06.268 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:06.265+0000 7f5ad0051640 1 -- 172.21.15.89:0/704448140 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ac8075840 con 0x7f5ac8103750 2024-09-21T03:39:06.269 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:06.266+0000 7f5ad0051640 1 -- 172.21.15.89:0/704448140 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5ac8103bc0 con 0x7f5ac8103750 2024-09-21T03:39:06.269 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:06.267+0000 7f5ac6ffd640 1 -- 172.21.15.89:0/704448140 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 18) ==== 154563+0+0 (secure 0 0 0) 0x7f5ab80079b0 con 0x7f5ac8103750 2024-09-21T03:39:06.269 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:06.268+0000 7f5ac6ffd640 1 --2- 172.21.15.89:0/704448140 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f5ab00b65a0 0x7f5ab00b8ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:06.269 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:06.268+0000 7f5acd5ec640 1 --2- 172.21.15.89:0/704448140 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f5ab00b65a0 0x7f5ab00b8ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:06.269 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:06.268+0000 7f5ac6ffd640 1 -- 172.21.15.89:0/704448140 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 9121+0+0 (secure 0 0 0) 0x7f5ab80e7610 con 0x7f5ac8103750 2024-09-21T03:39:06.270 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:06.268+0000 7f5acd5ec640 1 --2- 172.21.15.89:0/704448140 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f5ab00b65a0 0x7f5ab00b8ab0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f5abc006fd0 tx=0x7f5abc008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:39:06.273 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:06.272+0000 7f5ac6ffd640 1 -- 172.21.15.89:0/704448140 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f5ab8014030 con 0x7f5ac8103750 2024-09-21T03:39:06.461 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:06.459+0000 7f5ad0051640 1 -- 172.21.15.89:0/704448140 --> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) -- 0x7f5ac802ea20 con 0x7f5ab00b65a0 2024-09-21T03:39:06.462 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:06.461+0000 7f5ac6ffd640 1 -- 172.21.15.89:0/704448140 <== mgr.14152 v2:172.21.15.89:6800/1591532358 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+532 (secure 0 0 0) 0x7f5ac802ea20 con 0x7f5ab00b65a0 2024-09-21T03:39:06.463 INFO:teuthology.orchestra.run.smithi089.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2024-09-21T03:39:06.463 INFO:teuthology.orchestra.run.smithi089.stdout:smithi089 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT62040126400BGN 372G No 3m ago Has a FileSystem, LVM detected 2024-09-21T03:39:06.463 INFO:teuthology.orchestra.run.smithi089.stdout:smithi110 /dev/nvme0n1 ssd INTEL_SSDPED1D480GA_PHMB7476002T480DGN 447G No 3m ago Has a FileSystem, LVM detected 2024-09-21T03:39:06.463 INFO:teuthology.orchestra.run.smithi089.stdout:smithi114 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT620400WB400BGN 372G No 2m ago Has a FileSystem, LVM detected 2024-09-21T03:39:06.465 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:06.463+0000 7f5ad0051640 1 -- 172.21.15.89:0/704448140 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f5ab00b65a0 msgr2=0x7f5ab00b8ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:06.465 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:06.463+0000 7f5ad0051640 1 --2- 172.21.15.89:0/704448140 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f5ab00b65a0 0x7f5ab00b8ab0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f5abc006fd0 tx=0x7f5abc008040 comp rx=0 tx=0).stop 2024-09-21T03:39:06.465 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:06.463+0000 7f5ad0051640 1 -- 172.21.15.89:0/704448140 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5ac8103750 msgr2=0x7f5ac8199ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:06.465 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:06.463+0000 7f5ad0051640 1 --2- 172.21.15.89:0/704448140 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5ac8103750 0x7f5ac8199ee0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f5ab800c890 tx=0x7f5ab800cd50 comp rx=0 tx=0).stop 2024-09-21T03:39:06.465 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:06.463+0000 7f5ad0051640 1 -- 172.21.15.89:0/704448140 shutdown_connections 2024-09-21T03:39:06.465 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:06.463+0000 7f5ad0051640 1 --2- 172.21.15.89:0/704448140 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f5ab00b65a0 0x7f5ab00b8ab0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:06.465 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:06.463+0000 7f5ad0051640 1 --2- 172.21.15.89:0/704448140 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5ac8105180 0x7f5ac819b0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:06.465 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:06.463+0000 7f5ad0051640 1 --2- 172.21.15.89:0/704448140 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5ac81047e0 0x7f5ac819a550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:06.466 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:06.463+0000 7f5ad0051640 1 --2- 172.21.15.89:0/704448140 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5ac8103750 0x7f5ac8199ee0 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:06.466 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:06.463+0000 7f5ad0051640 1 -- 172.21.15.89:0/704448140 >> 172.21.15.89:0/704448140 conn(0x7f5ac80ff0c0 msgr2=0x7f5ac8107b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:39:06.466 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:06.464+0000 7f5ad0051640 1 -- 172.21.15.89:0/704448140 shutdown_connections 2024-09-21T03:39:06.466 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:06.464+0000 7f5ad0051640 1 -- 172.21.15.89:0/704448140 wait complete. 2024-09-21T03:39:07.061 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:39:06 smithi089 ceph-mon[30928]: pgmap v351: 129 pgs: 129 active+clean; 646 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 6 op/s 2024-09-21T03:39:07.061 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:39:06 smithi089 ceph-mon[30928]: from='client.15327 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:39:07.113 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-09-21T03:39:07.121 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi089.front.sepia.ceph.com 2024-09-21T03:39:07.122 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7c374098-77c9-11ef-baf5-efdc52797490 -- bash -c 'ceph fs dump' 2024-09-21T03:39:07.211 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:39:06 smithi114 ceph-mon[34140]: pgmap v351: 129 pgs: 129 active+clean; 646 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 6 op/s 2024-09-21T03:39:07.211 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:39:06 smithi114 ceph-mon[34140]: from='client.15327 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:39:07.250 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:39:06 smithi110 ceph-mon[34000]: pgmap v351: 129 pgs: 129 active+clean; 646 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 6 op/s 2024-09-21T03:39:07.250 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:39:06 smithi110 ceph-mon[34000]: from='client.15327 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:39:07.368 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/7c374098-77c9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:39:07.848 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:07.845+0000 7f9ca9b48640 1 -- 172.21.15.89:0/478794869 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9ca4104070 msgr2=0x7f9ca410aa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:07.848 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:07.845+0000 7f9ca9b48640 1 --2- 172.21.15.89:0/478794869 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9ca4104070 0x7f9ca410aa10 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f9c9c009900 tx=0x7f9c9c02a480 comp rx=0 tx=0).stop 2024-09-21T03:39:07.849 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:07.845+0000 7f9ca9b48640 1 -- 172.21.15.89:0/478794869 shutdown_connections 2024-09-21T03:39:07.849 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:07.845+0000 7f9ca9b48640 1 --2- 172.21.15.89:0/478794869 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9ca41036d0 0x7f9ca4103ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:07.849 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:07.845+0000 7f9ca9b48640 1 --2- 172.21.15.89:0/478794869 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9ca4104070 0x7f9ca410aa10 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:07.849 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:07.845+0000 7f9ca9b48640 1 --2- 172.21.15.89:0/478794869 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9ca4102640 0x7f9ca4102ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:07.849 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:07.845+0000 7f9ca9b48640 1 -- 172.21.15.89:0/478794869 >> 172.21.15.89:0/478794869 conn(0x7f9ca40fdfb0 msgr2=0x7f9ca4100460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:39:07.849 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:07.845+0000 7f9ca9b48640 1 -- 172.21.15.89:0/478794869 shutdown_connections 2024-09-21T03:39:07.849 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:07.845+0000 7f9ca9b48640 1 -- 172.21.15.89:0/478794869 wait complete. 2024-09-21T03:39:07.849 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:07.845+0000 7f9ca9b48640 1 Processor -- start 2024-09-21T03:39:07.849 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:07.846+0000 7f9ca9b48640 1 -- start start 2024-09-21T03:39:07.849 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:07.846+0000 7f9ca9b48640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9ca4102640 0x7f9ca4198d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:07.849 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:07.846+0000 7f9ca9b48640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9ca41036d0 0x7f9ca41993f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:07.849 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:07.846+0000 7f9ca9b48640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9ca4104070 0x7f9ca4199f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:07.849 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:07.846+0000 7f9ca9b48640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f9ca410e120 con 0x7f9ca41036d0 2024-09-21T03:39:07.849 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:07.846+0000 7f9ca9b48640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f9ca410dfa0 con 0x7f9ca4104070 2024-09-21T03:39:07.850 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:07.846+0000 7f9ca9b48640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f9ca410e2a0 con 0x7f9ca4102640 2024-09-21T03:39:07.850 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:07.846+0000 7f9ca2ffd640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9ca41036d0 0x7f9ca41993f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:07.850 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:07.846+0000 7f9ca2ffd640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9ca41036d0 0x7f9ca41993f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:47746/0 (socket says 172.21.15.89:47746) 2024-09-21T03:39:07.850 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:07.846+0000 7f9ca2ffd640 1 -- 172.21.15.89:0/1455133245 learned_addr learned my addr 172.21.15.89:0/1455133245 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-21T03:39:07.850 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:07.846+0000 7f9ca3fff640 1 --2- 172.21.15.89:0/1455133245 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9ca4104070 0x7f9ca4199f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:07.850 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:07.846+0000 7f9ca37fe640 1 --2- 172.21.15.89:0/1455133245 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9ca4102640 0x7f9ca4198d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:07.850 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:07.846+0000 7f9ca2ffd640 1 -- 172.21.15.89:0/1455133245 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9ca4102640 msgr2=0x7f9ca4198d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:07.850 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:07.846+0000 7f9ca2ffd640 1 --2- 172.21.15.89:0/1455133245 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9ca4102640 0x7f9ca4198d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:07.850 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:07.846+0000 7f9ca2ffd640 1 -- 172.21.15.89:0/1455133245 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9ca4104070 msgr2=0x7f9ca4199f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:07.850 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:07.846+0000 7f9ca2ffd640 1 --2- 172.21.15.89:0/1455133245 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9ca4104070 0x7f9ca4199f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:07.850 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:07.846+0000 7f9ca2ffd640 1 -- 172.21.15.89:0/1455133245 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ca419a720 con 0x7f9ca41036d0 2024-09-21T03:39:07.850 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:07.846+0000 7f9ca37fe640 1 --2- 172.21.15.89:0/1455133245 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9ca4102640 0x7f9ca4198d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:39:07.850 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:07.847+0000 7f9ca2ffd640 1 --2- 172.21.15.89:0/1455133245 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9ca41036d0 0x7f9ca41993f0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f9c9800eab0 tx=0x7f9c9800ef70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:39:07.851 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:07.847+0000 7f9ca0ff9640 1 -- 172.21.15.89:0/1455133245 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c9800ce40 con 0x7f9ca41036d0 2024-09-21T03:39:07.851 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:07.847+0000 7f9ca9b48640 1 -- 172.21.15.89:0/1455133245 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ca419aa10 con 0x7f9ca41036d0 2024-09-21T03:39:07.851 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:07.847+0000 7f9ca0ff9640 1 -- 172.21.15.89:0/1455133245 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9c98011900 con 0x7f9ca41036d0 2024-09-21T03:39:07.851 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:07.847+0000 7f9ca0ff9640 1 -- 172.21.15.89:0/1455133245 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c98010990 con 0x7f9ca41036d0 2024-09-21T03:39:07.851 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:07.847+0000 7f9ca9b48640 1 -- 172.21.15.89:0/1455133245 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ca41a11a0 con 0x7f9ca41036d0 2024-09-21T03:39:07.852 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:07.848+0000 7f9ca9b48640 1 -- 172.21.15.89:0/1455133245 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9ca4102ab0 con 0x7f9ca41036d0 2024-09-21T03:39:07.852 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:07.849+0000 7f9ca0ff9640 1 -- 172.21.15.89:0/1455133245 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 18) ==== 154563+0+0 (secure 0 0 0) 0x7f9c98011470 con 0x7f9ca41036d0 2024-09-21T03:39:07.852 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:07.850+0000 7f9ca0ff9640 1 --2- 172.21.15.89:0/1455133245 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f9c8c0b65a0 0x7f9c8c0b8ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:07.852 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:07.850+0000 7f9ca37fe640 1 --2- 172.21.15.89:0/1455133245 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f9c8c0b65a0 0x7f9c8c0b8ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:07.852 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:07.850+0000 7f9ca0ff9640 1 -- 172.21.15.89:0/1455133245 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 9121+0+0 (secure 0 0 0) 0x7f9c980e7e30 con 0x7f9ca41036d0 2024-09-21T03:39:07.852 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:07.851+0000 7f9ca37fe640 1 --2- 172.21.15.89:0/1455133245 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f9c8c0b65a0 0x7f9c8c0b8ab0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f9c9400abe0 tx=0x7f9c9400a5c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:39:07.854 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:07.853+0000 7f9ca0ff9640 1 -- 172.21.15.89:0/1455133245 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f9c98016030 con 0x7f9ca41036d0 2024-09-21T03:39:08.072 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:08.071+0000 7f9ca9b48640 1 -- 172.21.15.89:0/1455133245 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs dump"} v 0) -- 0x7f9ca4063fc0 con 0x7f9ca41036d0 2024-09-21T03:39:08.072 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:08.071+0000 7f9ca0ff9640 1 -- 172.21.15.89:0/1455133245 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 25 v25) ==== 76+0+3088 (secure 0 0 0) 0x7f9c980ae1e0 con 0x7f9ca41036d0 2024-09-21T03:39:08.073 INFO:teuthology.orchestra.run.smithi089.stdout:e25 2024-09-21T03:39:08.073 INFO:teuthology.orchestra.run.smithi089.stdout:btime 2024-09-21T03:37:51:019398+0000 2024-09-21T03:39:08.073 INFO:teuthology.orchestra.run.smithi089.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-21T03:39:08.073 INFO:teuthology.orchestra.run.smithi089.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-21T03:39:08.073 INFO:teuthology.orchestra.run.smithi089.stdout:legacy client fscid: 1 2024-09-21T03:39:08.073 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-09-21T03:39:08.073 INFO:teuthology.orchestra.run.smithi089.stdout:Filesystem 'cephfs' (1) 2024-09-21T03:39:08.073 INFO:teuthology.orchestra.run.smithi089.stdout:fs_name cephfs 2024-09-21T03:39:08.073 INFO:teuthology.orchestra.run.smithi089.stdout:epoch 25 2024-09-21T03:39:08.073 INFO:teuthology.orchestra.run.smithi089.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-09-21T03:39:08.073 INFO:teuthology.orchestra.run.smithi089.stdout:created 2024-09-21T03:37:13.570694+0000 2024-09-21T03:39:08.074 INFO:teuthology.orchestra.run.smithi089.stdout:modified 2024-09-21T03:37:50.719106+0000 2024-09-21T03:39:08.074 INFO:teuthology.orchestra.run.smithi089.stdout:tableserver 0 2024-09-21T03:39:08.074 INFO:teuthology.orchestra.run.smithi089.stdout:root 0 2024-09-21T03:39:08.074 INFO:teuthology.orchestra.run.smithi089.stdout:session_timeout 300 2024-09-21T03:39:08.074 INFO:teuthology.orchestra.run.smithi089.stdout:session_autoclose 300 2024-09-21T03:39:08.074 INFO:teuthology.orchestra.run.smithi089.stdout:max_file_size 1099511627776 2024-09-21T03:39:08.074 INFO:teuthology.orchestra.run.smithi089.stdout:max_xattr_size 65536 2024-09-21T03:39:08.074 INFO:teuthology.orchestra.run.smithi089.stdout:required_client_features {} 2024-09-21T03:39:08.074 INFO:teuthology.orchestra.run.smithi089.stdout:last_failure 0 2024-09-21T03:39:08.074 INFO:teuthology.orchestra.run.smithi089.stdout:last_failure_osd_epoch 0 2024-09-21T03:39:08.074 INFO:teuthology.orchestra.run.smithi089.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-21T03:39:08.074 INFO:teuthology.orchestra.run.smithi089.stdout:max_mds 3 2024-09-21T03:39:08.074 INFO:teuthology.orchestra.run.smithi089.stdout:in 0,1,2 2024-09-21T03:39:08.074 INFO:teuthology.orchestra.run.smithi089.stdout:up {0=24638,1=24631,2=24614} 2024-09-21T03:39:08.075 INFO:teuthology.orchestra.run.smithi089.stdout:failed 2024-09-21T03:39:08.075 INFO:teuthology.orchestra.run.smithi089.stdout:damaged 2024-09-21T03:39:08.075 INFO:teuthology.orchestra.run.smithi089.stdout:stopped 2024-09-21T03:39:08.075 INFO:teuthology.orchestra.run.smithi089.stdout:data_pools [3,4] 2024-09-21T03:39:08.075 INFO:teuthology.orchestra.run.smithi089.stdout:metadata_pool 2 2024-09-21T03:39:08.075 INFO:teuthology.orchestra.run.smithi089.stdout:inline_data disabled 2024-09-21T03:39:08.075 INFO:teuthology.orchestra.run.smithi089.stdout:balancer 2024-09-21T03:39:08.075 INFO:teuthology.orchestra.run.smithi089.stdout:bal_rank_mask -1 2024-09-21T03:39:08.075 INFO:teuthology.orchestra.run.smithi089.stdout:standby_count_wanted 0 2024-09-21T03:39:08.075 INFO:teuthology.orchestra.run.smithi089.stdout:qdb_cluster leader: 24638 members: 24614,24631,24638 2024-09-21T03:39:08.075 INFO:teuthology.orchestra.run.smithi089.stdout:[mds.b{0:24638} state up:active seq 4 addr [v2:172.21.15.110:6838/132366709,v1:172.21.15.110:6839/132366709] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T03:39:08.075 INFO:teuthology.orchestra.run.smithi089.stdout:[mds.c{0:24610} state up:standby-replay seq 1 addr [v2:172.21.15.114:6836/1941213818,v1:172.21.15.114:6837/1941213818] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T03:39:08.075 INFO:teuthology.orchestra.run.smithi089.stdout:[mds.i{1:24631} state up:active seq 15 addr [v2:172.21.15.114:6838/3427369944,v1:172.21.15.114:6839/3427369944] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T03:39:08.075 INFO:teuthology.orchestra.run.smithi089.stdout:[mds.f{1:24604} state up:standby-replay seq 1 addr [v2:172.21.15.114:6834/1049714548,v1:172.21.15.114:6835/1049714548] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T03:39:08.075 INFO:teuthology.orchestra.run.smithi089.stdout:[mds.h{2:24614} state up:active seq 15 addr [v2:172.21.15.110:6836/4047124589,v1:172.21.15.110:6837/4047124589] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T03:39:08.076 INFO:teuthology.orchestra.run.smithi089.stdout:[mds.e{2:24593} state up:standby-replay seq 1 addr [v2:172.21.15.110:6834/671820654,v1:172.21.15.110:6835/671820654] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T03:39:08.076 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-09-21T03:39:08.076 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-09-21T03:39:08.076 INFO:teuthology.orchestra.run.smithi089.stdout:Standby daemons: 2024-09-21T03:39:08.076 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-09-21T03:39:08.076 INFO:teuthology.orchestra.run.smithi089.stdout:[mds.j{-1:14721} state up:standby seq 1 addr [v2:172.21.15.89:6834/2341538514,v1:172.21.15.89:6835/2341538514] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T03:39:08.076 INFO:teuthology.orchestra.run.smithi089.stdout:[mds.d{-1:14733} state up:standby seq 1 addr [v2:172.21.15.89:6836/146708592,v1:172.21.15.89:6837/146708592] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T03:39:08.076 INFO:teuthology.orchestra.run.smithi089.stdout:[mds.g{-1:14781} state up:standby seq 1 addr [v2:172.21.15.89:6838/2269965499,v1:172.21.15.89:6839/2269965499] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T03:39:08.076 INFO:teuthology.orchestra.run.smithi089.stdout:[mds.a{-1:14793} state up:standby seq 1 addr [v2:172.21.15.89:6840/650850383,v1:172.21.15.89:6841/650850383] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T03:39:08.076 INFO:teuthology.orchestra.run.smithi089.stdout:[mds.l{-1:24565} state up:standby seq 1 addr [v2:172.21.15.114:6832/752336912,v1:172.21.15.114:6833/752336912] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T03:39:08.076 INFO:teuthology.orchestra.run.smithi089.stdout:[mds.k{-1:24575} state up:standby seq 1 addr [v2:172.21.15.110:6832/127607331,v1:172.21.15.110:6833/127607331] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T03:39:08.077 INFO:teuthology.orchestra.run.smithi089.stderr:dumped fsmap epoch 25 2024-09-21T03:39:08.077 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:08.073+0000 7f9ca9b48640 1 -- 172.21.15.89:0/1455133245 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f9c8c0b65a0 msgr2=0x7f9c8c0b8ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:08.077 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:08.073+0000 7f9ca9b48640 1 --2- 172.21.15.89:0/1455133245 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f9c8c0b65a0 0x7f9c8c0b8ab0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f9c9400abe0 tx=0x7f9c9400a5c0 comp rx=0 tx=0).stop 2024-09-21T03:39:08.077 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:08.073+0000 7f9ca9b48640 1 -- 172.21.15.89:0/1455133245 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9ca41036d0 msgr2=0x7f9ca41993f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:08.077 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:08.073+0000 7f9ca9b48640 1 --2- 172.21.15.89:0/1455133245 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9ca41036d0 0x7f9ca41993f0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f9c9800eab0 tx=0x7f9c9800ef70 comp rx=0 tx=0).stop 2024-09-21T03:39:08.077 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:08.073+0000 7f9ca9b48640 1 -- 172.21.15.89:0/1455133245 shutdown_connections 2024-09-21T03:39:08.077 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:08.073+0000 7f9ca9b48640 1 --2- 172.21.15.89:0/1455133245 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f9c8c0b65a0 0x7f9c8c0b8ab0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:08.077 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:08.073+0000 7f9ca9b48640 1 --2- 172.21.15.89:0/1455133245 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9ca4104070 0x7f9ca4199f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:08.078 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:08.073+0000 7f9ca9b48640 1 --2- 172.21.15.89:0/1455133245 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9ca41036d0 0x7f9ca41993f0 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:08.078 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:08.073+0000 7f9ca9b48640 1 --2- 172.21.15.89:0/1455133245 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9ca4102640 0x7f9ca4198d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:08.078 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:08.073+0000 7f9ca9b48640 1 -- 172.21.15.89:0/1455133245 >> 172.21.15.89:0/1455133245 conn(0x7f9ca40fdfb0 msgr2=0x7f9ca41069f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:39:08.078 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:08.074+0000 7f9ca9b48640 1 -- 172.21.15.89:0/1455133245 shutdown_connections 2024-09-21T03:39:08.078 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:08.074+0000 7f9ca9b48640 1 -- 172.21.15.89:0/1455133245 wait complete. 2024-09-21T03:39:08.883 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7c374098-77c9-11ef-baf5-efdc52797490 -- bash -c 'ceph osd dump' 2024-09-21T03:39:09.061 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:39:08 smithi089 ceph-mon[30928]: pgmap v352: 129 pgs: 129 active+clean; 646 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-21T03:39:09.061 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:39:08 smithi089 ceph-mon[30928]: from='client.? 172.21.15.89:0/1455133245' entity='client.admin' cmd={"prefix": "fs dump"} : dispatch 2024-09-21T03:39:09.130 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/7c374098-77c9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:39:09.211 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:39:08 smithi114 ceph-mon[34140]: pgmap v352: 129 pgs: 129 active+clean; 646 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-21T03:39:09.211 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:39:08 smithi114 ceph-mon[34140]: from='client.? 172.21.15.89:0/1455133245' entity='client.admin' cmd={"prefix": "fs dump"} : dispatch 2024-09-21T03:39:09.250 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:39:08 smithi110 ceph-mon[34000]: pgmap v352: 129 pgs: 129 active+clean; 646 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-21T03:39:09.250 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:39:08 smithi110 ceph-mon[34000]: from='client.? 172.21.15.89:0/1455133245' entity='client.admin' cmd={"prefix": "fs dump"} : dispatch 2024-09-21T03:39:09.647 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:09.622+0000 7f4af5fe0640 1 -- 172.21.15.89:0/3722566984 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4af0109470 msgr2=0x7f4af010b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:09.647 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:09.622+0000 7f4af5fe0640 1 --2- 172.21.15.89:0/3722566984 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4af0109470 0x7f4af010b980 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f4ae8009ca0 tx=0x7f4ae802a8c0 comp rx=0 tx=0).stop 2024-09-21T03:39:09.648 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:09.622+0000 7f4af5fe0640 1 -- 172.21.15.89:0/3722566984 shutdown_connections 2024-09-21T03:39:09.648 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:09.622+0000 7f4af5fe0640 1 --2- 172.21.15.89:0/3722566984 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4af01068d0 0x7f4af0108d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:09.648 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:09.622+0000 7f4af5fe0640 1 --2- 172.21.15.89:0/3722566984 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4af0109470 0x7f4af010b980 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:09.648 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:09.622+0000 7f4af5fe0640 1 --2- 172.21.15.89:0/3722566984 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4af006b980 0x7f4af0106390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:09.648 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:09.622+0000 7f4af5fe0640 1 -- 172.21.15.89:0/3722566984 >> 172.21.15.89:0/3722566984 conn(0x7f4af00fdfb0 msgr2=0x7f4af0100460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:39:09.648 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:09.622+0000 7f4af5fe0640 1 -- 172.21.15.89:0/3722566984 shutdown_connections 2024-09-21T03:39:09.648 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:09.622+0000 7f4af5fe0640 1 -- 172.21.15.89:0/3722566984 wait complete. 2024-09-21T03:39:09.648 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:09.622+0000 7f4af5fe0640 1 Processor -- start 2024-09-21T03:39:09.648 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:09.623+0000 7f4af5fe0640 1 -- start start 2024-09-21T03:39:09.648 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:09.623+0000 7f4af5fe0640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4af006b980 0x7f4af01039c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:09.648 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:09.623+0000 7f4af5fe0640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4af01068d0 0x7f4af0102290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:09.648 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:09.623+0000 7f4af5fe0640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4af0109470 0x7f4af01027d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:09.648 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:09.623+0000 7f4af5fe0640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f4af010f270 con 0x7f4af006b980 2024-09-21T03:39:09.648 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:09.623+0000 7f4af5fe0640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f4af010f0f0 con 0x7f4af0109470 2024-09-21T03:39:09.649 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:09.623+0000 7f4af5fe0640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f4af010f3f0 con 0x7f4af01068d0 2024-09-21T03:39:09.649 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:09.623+0000 7f4aef7fe640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4af006b980 0x7f4af01039c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:09.649 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:09.623+0000 7f4aef7fe640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4af006b980 0x7f4af01039c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:36828/0 (socket says 172.21.15.89:36828) 2024-09-21T03:39:09.649 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:09.623+0000 7f4aef7fe640 1 -- 172.21.15.89:0/152679945 learned_addr learned my addr 172.21.15.89:0/152679945 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-21T03:39:09.649 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:09.623+0000 7f4aeffff640 1 --2- 172.21.15.89:0/152679945 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4af0109470 0x7f4af01027d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:09.649 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:09.623+0000 7f4ae6ffd640 1 --2- 172.21.15.89:0/152679945 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4af01068d0 0x7f4af0102290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:09.649 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:09.623+0000 7f4aef7fe640 1 -- 172.21.15.89:0/152679945 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4af01068d0 msgr2=0x7f4af0102290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:09.649 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:09.623+0000 7f4aef7fe640 1 --2- 172.21.15.89:0/152679945 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4af01068d0 0x7f4af0102290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:09.649 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:09.623+0000 7f4aef7fe640 1 -- 172.21.15.89:0/152679945 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4af0109470 msgr2=0x7f4af01027d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:09.649 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:09.623+0000 7f4aef7fe640 1 --2- 172.21.15.89:0/152679945 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4af0109470 0x7f4af01027d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:09.649 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:09.623+0000 7f4aef7fe640 1 -- 172.21.15.89:0/152679945 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4af0102e40 con 0x7f4af006b980 2024-09-21T03:39:09.649 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:09.624+0000 7f4aef7fe640 1 --2- 172.21.15.89:0/152679945 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4af006b980 0x7f4af01039c0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f4ae000c890 tx=0x7f4ae000cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:39:09.649 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:09.624+0000 7f4aed7fa640 1 -- 172.21.15.89:0/152679945 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ae0004220 con 0x7f4af006b980 2024-09-21T03:39:09.649 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:09.624+0000 7f4af5fe0640 1 -- 172.21.15.89:0/152679945 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4af01a3ef0 con 0x7f4af006b980 2024-09-21T03:39:09.649 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:09.624+0000 7f4aed7fa640 1 -- 172.21.15.89:0/152679945 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4ae0007e40 con 0x7f4af006b980 2024-09-21T03:39:09.650 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:09.624+0000 7f4aed7fa640 1 -- 172.21.15.89:0/152679945 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ae0004a00 con 0x7f4af006b980 2024-09-21T03:39:09.650 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:09.624+0000 7f4af5fe0640 1 -- 172.21.15.89:0/152679945 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4af01a4290 con 0x7f4af006b980 2024-09-21T03:39:09.650 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:09.625+0000 7f4af5fe0640 1 -- 172.21.15.89:0/152679945 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4af0106d20 con 0x7f4af006b980 2024-09-21T03:39:09.650 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:09.626+0000 7f4aed7fa640 1 -- 172.21.15.89:0/152679945 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 18) ==== 154563+0+0 (secure 0 0 0) 0x7f4ae00079b0 con 0x7f4af006b980 2024-09-21T03:39:09.650 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:09.627+0000 7f4aed7fa640 1 --2- 172.21.15.89:0/152679945 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f4adc0b65a0 0x7f4adc0b8ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:09.650 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:09.627+0000 7f4ae6ffd640 1 --2- 172.21.15.89:0/152679945 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f4adc0b65a0 0x7f4adc0b8ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:09.650 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:09.627+0000 7f4aed7fa640 1 -- 172.21.15.89:0/152679945 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 9121+0+0 (secure 0 0 0) 0x7f4ae00e7610 con 0x7f4af006b980 2024-09-21T03:39:09.650 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:09.628+0000 7f4ae6ffd640 1 --2- 172.21.15.89:0/152679945 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f4adc0b65a0 0x7f4adc0b8ab0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f4ad8005e10 tx=0x7f4ad8005da0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:39:09.650 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:09.631+0000 7f4aed7fa640 1 -- 172.21.15.89:0/152679945 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f4ae0014030 con 0x7f4af006b980 2024-09-21T03:39:09.815 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:09.814+0000 7f4af5fe0640 1 -- 172.21.15.89:0/152679945 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd dump"} v 0) -- 0x7f4af00ffad0 con 0x7f4af006b980 2024-09-21T03:39:09.816 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:09.815+0000 7f4aed7fa640 1 -- 172.21.15.89:0/152679945 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump"}]=0 v84) ==== 56+0+5632 (secure 0 0 0) 0x7f4ae00ad9c0 con 0x7f4af006b980 2024-09-21T03:39:09.816 INFO:teuthology.orchestra.run.smithi089.stdout:epoch 84 2024-09-21T03:39:09.816 INFO:teuthology.orchestra.run.smithi089.stdout:fsid 7c374098-77c9-11ef-baf5-efdc52797490 2024-09-21T03:39:09.816 INFO:teuthology.orchestra.run.smithi089.stdout:created 2024-09-21T03:28:56.681458+0000 2024-09-21T03:39:09.817 INFO:teuthology.orchestra.run.smithi089.stdout:modified 2024-09-21T03:38:53.661606+0000 2024-09-21T03:39:09.817 INFO:teuthology.orchestra.run.smithi089.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-09-21T03:39:09.817 INFO:teuthology.orchestra.run.smithi089.stdout:crush_version 27 2024-09-21T03:39:09.817 INFO:teuthology.orchestra.run.smithi089.stdout:full_ratio 0.95 2024-09-21T03:39:09.817 INFO:teuthology.orchestra.run.smithi089.stdout:backfillfull_ratio 0.9 2024-09-21T03:39:09.817 INFO:teuthology.orchestra.run.smithi089.stdout:nearfull_ratio 0.85 2024-09-21T03:39:09.817 INFO:teuthology.orchestra.run.smithi089.stdout:require_min_compat_client luminous 2024-09-21T03:39:09.817 INFO:teuthology.orchestra.run.smithi089.stdout:min_compat_client luminous 2024-09-21T03:39:09.817 INFO:teuthology.orchestra.run.smithi089.stdout:require_osd_release squid 2024-09-21T03:39:09.817 INFO:teuthology.orchestra.run.smithi089.stdout:stretch_mode_enabled false 2024-09-21T03:39:09.817 INFO:teuthology.orchestra.run.smithi089.stdout:pool 1 '.mgr' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 21 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 12.00 2024-09-21T03:39:09.817 INFO:teuthology.orchestra.run.smithi089.stdout:pool 2 'cephfs_metadata' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 74 flags hashpspool stripe_width 0 pg_autoscale_bias 4 pg_num_min 16 recovery_priority 5 application cephfs read_balance_score 1.88 2024-09-21T03:39:09.817 INFO:teuthology.orchestra.run.smithi089.stdout:pool 3 'cephfs_data' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 64 pgp_num 64 autoscale_mode off last_change 74 flags hashpspool stripe_width 0 pg_num_min 64 target_size_ratio 0.05 application cephfs read_balance_score 1.50 2024-09-21T03:39:09.818 INFO:teuthology.orchestra.run.smithi089.stdout:pool 4 'cephfs_data_ec' erasure profile cephfs_data_ec size 4 min_size 3 crush_rule 1 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 80 flags hashpspool,ec_overwrites stripe_width 8192 pg_num_min 64 target_size_ratio 0.9 application cephfs 2024-09-21T03:39:09.818 INFO:teuthology.orchestra.run.smithi089.stdout:max_osd 12 2024-09-21T03:39:09.818 INFO:teuthology.orchestra.run.smithi089.stdout:osd.0 up in weight 1 up_from 8 up_thru 77 down_at 0 last_clean_interval [0,0) [v2:172.21.15.89:6802/990539042,v1:172.21.15.89:6803/990539042] [v2:172.21.15.89:6804/990539042,v1:172.21.15.89:6805/990539042] exists,up 27ca5ed8-17d1-4f8e-a7ae-084a914721a0 2024-09-21T03:39:09.818 INFO:teuthology.orchestra.run.smithi089.stdout:osd.1 up in weight 1 up_from 13 up_thru 77 down_at 0 last_clean_interval [0,0) [v2:172.21.15.110:6800/3235852628,v1:172.21.15.110:6801/3235852628] [v2:172.21.15.110:6802/3235852628,v1:172.21.15.110:6803/3235852628] exists,up 5176a910-5cf9-49d7-aedb-a20cf36ea075 2024-09-21T03:39:09.818 INFO:teuthology.orchestra.run.smithi089.stdout:osd.2 up in weight 1 up_from 17 up_thru 77 down_at 0 last_clean_interval [0,0) [v2:172.21.15.114:6800/169452122,v1:172.21.15.114:6801/169452122] [v2:172.21.15.114:6802/169452122,v1:172.21.15.114:6803/169452122] exists,up d4977df1-b118-4d7e-b9a9-626a8accc812 2024-09-21T03:39:09.818 INFO:teuthology.orchestra.run.smithi089.stdout:osd.3 up in weight 1 up_from 24 up_thru 77 down_at 0 last_clean_interval [0,0) [v2:172.21.15.89:6810/3016980877,v1:172.21.15.89:6811/3016980877] [v2:172.21.15.89:6812/3016980877,v1:172.21.15.89:6813/3016980877] exists,up 2c7ae1b4-0bf2-4954-81ed-55d03eab82c6 2024-09-21T03:39:09.818 INFO:teuthology.orchestra.run.smithi089.stdout:osd.4 up in weight 1 up_from 30 up_thru 81 down_at 0 last_clean_interval [0,0) [v2:172.21.15.110:6808/3217880476,v1:172.21.15.110:6809/3217880476] [v2:172.21.15.110:6810/3217880476,v1:172.21.15.110:6811/3217880476] exists,up 8e65399d-6e55-4da4-98ba-f805f0c655a7 2024-09-21T03:39:09.818 INFO:teuthology.orchestra.run.smithi089.stdout:osd.5 up in weight 1 up_from 35 up_thru 81 down_at 0 last_clean_interval [0,0) [v2:172.21.15.114:6808/1144140707,v1:172.21.15.114:6809/1144140707] [v2:172.21.15.114:6810/1144140707,v1:172.21.15.114:6811/1144140707] exists,up b9ec2889-46bd-40e5-8446-73ab28271c67 2024-09-21T03:39:09.818 INFO:teuthology.orchestra.run.smithi089.stdout:osd.6 up in weight 1 up_from 41 up_thru 81 down_at 0 last_clean_interval [0,0) [v2:172.21.15.89:6818/1522586255,v1:172.21.15.89:6819/1522586255] [v2:172.21.15.89:6820/1522586255,v1:172.21.15.89:6821/1522586255] exists,up 9b79989e-adb0-42b2-b2f6-a2dce56b66e7 2024-09-21T03:39:09.818 INFO:teuthology.orchestra.run.smithi089.stdout:osd.7 up in weight 1 up_from 46 up_thru 77 down_at 0 last_clean_interval [0,0) [v2:172.21.15.110:6816/1331818323,v1:172.21.15.110:6817/1331818323] [v2:172.21.15.110:6818/1331818323,v1:172.21.15.110:6819/1331818323] exists,up b381821f-141c-47a1-be65-f57ca730391b 2024-09-21T03:39:09.818 INFO:teuthology.orchestra.run.smithi089.stdout:osd.8 up in weight 1 up_from 52 up_thru 77 down_at 0 last_clean_interval [0,0) [v2:172.21.15.114:6816/2138556699,v1:172.21.15.114:6817/2138556699] [v2:172.21.15.114:6818/2138556699,v1:172.21.15.114:6819/2138556699] exists,up 1bc2a5b4-15f3-456f-9d67-e51307a27bcc 2024-09-21T03:39:09.818 INFO:teuthology.orchestra.run.smithi089.stdout:osd.9 up in weight 1 up_from 58 up_thru 83 down_at 0 last_clean_interval [0,0) [v2:172.21.15.89:6826/1730253068,v1:172.21.15.89:6827/1730253068] [v2:172.21.15.89:6828/1730253068,v1:172.21.15.89:6829/1730253068] exists,up 91f5fd29-d4a7-4800-ac7a-e83b13add89e 2024-09-21T03:39:09.819 INFO:teuthology.orchestra.run.smithi089.stdout:osd.10 up in weight 1 up_from 63 up_thru 81 down_at 0 last_clean_interval [0,0) [v2:172.21.15.110:6824/2904016117,v1:172.21.15.110:6825/2904016117] [v2:172.21.15.110:6826/2904016117,v1:172.21.15.110:6827/2904016117] exists,up 01390499-2a4d-4188-b57a-fcb10c668e2c 2024-09-21T03:39:09.819 INFO:teuthology.orchestra.run.smithi089.stdout:osd.11 up in weight 1 up_from 68 up_thru 81 down_at 0 last_clean_interval [0,0) [v2:172.21.15.114:6824/3749854486,v1:172.21.15.114:6825/3749854486] [v2:172.21.15.114:6826/3749854486,v1:172.21.15.114:6827/3749854486] exists,up c3c0f38a-c3d3-415d-a9f8-875d5b756e35 2024-09-21T03:39:09.819 INFO:teuthology.orchestra.run.smithi089.stdout:pg_upmap_items 2.6 [8,11] 2024-09-21T03:39:09.819 INFO:teuthology.orchestra.run.smithi089.stdout:pg_upmap_items 2.c [2,4] 2024-09-21T03:39:09.819 INFO:teuthology.orchestra.run.smithi089.stdout:pg_upmap_items 2.e [2,1] 2024-09-21T03:39:09.819 INFO:teuthology.orchestra.run.smithi089.stdout:pg_upmap_items 2.10 [8,11] 2024-09-21T03:39:09.819 INFO:teuthology.orchestra.run.smithi089.stdout:pg_upmap_items 3.3 [4,9] 2024-09-21T03:39:09.819 INFO:teuthology.orchestra.run.smithi089.stdout:pg_upmap_items 3.1b [5,9] 2024-09-21T03:39:09.819 INFO:teuthology.orchestra.run.smithi089.stdout:pg_upmap_items 3.34 [4,9] 2024-09-21T03:39:09.819 INFO:teuthology.orchestra.run.smithi089.stdout:pg_upmap_items 4.6 [9,11] 2024-09-21T03:39:09.819 INFO:teuthology.orchestra.run.smithi089.stdout:pg_upmap_items 4.12 [9,10] 2024-09-21T03:39:09.819 INFO:teuthology.orchestra.run.smithi089.stdout:pg_upmap_items 4.13 [9,8] 2024-09-21T03:39:09.819 INFO:teuthology.orchestra.run.smithi089.stdout:pg_upmap_items 4.1b [5,8] 2024-09-21T03:39:09.819 INFO:teuthology.orchestra.run.smithi089.stdout:blocklist 172.21.15.89:6801/655110910 expires 2024-09-22T03:29:51.519106+0000 2024-09-21T03:39:09.819 INFO:teuthology.orchestra.run.smithi089.stdout:blocklist 172.21.15.89:0/1618544983 expires 2024-09-22T03:29:51.519106+0000 2024-09-21T03:39:09.820 INFO:teuthology.orchestra.run.smithi089.stdout:blocklist 172.21.15.89:0/1265502758 expires 2024-09-22T03:29:51.519106+0000 2024-09-21T03:39:09.820 INFO:teuthology.orchestra.run.smithi089.stdout:blocklist 172.21.15.89:0/4017684419 expires 2024-09-22T03:29:51.519106+0000 2024-09-21T03:39:09.820 INFO:teuthology.orchestra.run.smithi089.stdout:blocklist 172.21.15.89:6801/3312712420 expires 2024-09-22T03:29:26.004686+0000 2024-09-21T03:39:09.820 INFO:teuthology.orchestra.run.smithi089.stdout:blocklist 172.21.15.89:6800/3312712420 expires 2024-09-22T03:29:26.004686+0000 2024-09-21T03:39:09.820 INFO:teuthology.orchestra.run.smithi089.stdout:blocklist 172.21.15.89:0/2056576483 expires 2024-09-22T03:29:26.004686+0000 2024-09-21T03:39:09.820 INFO:teuthology.orchestra.run.smithi089.stdout:blocklist 172.21.15.89:6800/655110910 expires 2024-09-22T03:29:51.519106+0000 2024-09-21T03:39:09.820 INFO:teuthology.orchestra.run.smithi089.stdout:blocklist 172.21.15.89:0/2382866302 expires 2024-09-22T03:29:26.004686+0000 2024-09-21T03:39:09.820 INFO:teuthology.orchestra.run.smithi089.stdout:blocklist 172.21.15.89:0/1584351801 expires 2024-09-22T03:29:26.004686+0000 2024-09-21T03:39:09.821 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:09.817+0000 7f4af5fe0640 1 -- 172.21.15.89:0/152679945 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f4adc0b65a0 msgr2=0x7f4adc0b8ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:09.821 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:09.817+0000 7f4af5fe0640 1 --2- 172.21.15.89:0/152679945 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f4adc0b65a0 0x7f4adc0b8ab0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f4ad8005e10 tx=0x7f4ad8005da0 comp rx=0 tx=0).stop 2024-09-21T03:39:09.821 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:09.817+0000 7f4af5fe0640 1 -- 172.21.15.89:0/152679945 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4af006b980 msgr2=0x7f4af01039c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:09.821 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:09.817+0000 7f4af5fe0640 1 --2- 172.21.15.89:0/152679945 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4af006b980 0x7f4af01039c0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f4ae000c890 tx=0x7f4ae000cd50 comp rx=0 tx=0).stop 2024-09-21T03:39:09.821 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:09.817+0000 7f4af5fe0640 1 -- 172.21.15.89:0/152679945 shutdown_connections 2024-09-21T03:39:09.821 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:09.817+0000 7f4af5fe0640 1 --2- 172.21.15.89:0/152679945 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f4adc0b65a0 0x7f4adc0b8ab0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:09.821 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:09.817+0000 7f4af5fe0640 1 --2- 172.21.15.89:0/152679945 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4af0109470 0x7f4af01027d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:09.821 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:09.817+0000 7f4af5fe0640 1 --2- 172.21.15.89:0/152679945 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4af01068d0 0x7f4af0102290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:09.821 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:09.817+0000 7f4af5fe0640 1 --2- 172.21.15.89:0/152679945 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4af006b980 0x7f4af01039c0 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:09.821 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:09.817+0000 7f4af5fe0640 1 -- 172.21.15.89:0/152679945 >> 172.21.15.89:0/152679945 conn(0x7f4af00fdfb0 msgr2=0x7f4af0104de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:39:09.821 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:09.817+0000 7f4af5fe0640 1 -- 172.21.15.89:0/152679945 shutdown_connections 2024-09-21T03:39:09.822 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:09.817+0000 7f4af5fe0640 1 -- 172.21.15.89:0/152679945 wait complete. 2024-09-21T03:39:10.568 INFO:teuthology.run_tasks:Running task fs.ready... 2024-09-21T03:39:10.582 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 7c374098-77c9-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T03:39:10.811 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:39:10 smithi089 ceph-mon[30928]: pgmap v353: 129 pgs: 129 active+clean; 646 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-21T03:39:10.811 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:39:10 smithi089 ceph-mon[30928]: from='client.? 172.21.15.89:0/152679945' entity='client.admin' cmd={"prefix": "osd dump"} : dispatch 2024-09-21T03:39:10.829 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/7c374098-77c9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:39:11.211 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:39:10 smithi114 ceph-mon[34140]: pgmap v353: 129 pgs: 129 active+clean; 646 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-21T03:39:11.211 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:39:10 smithi114 ceph-mon[34140]: from='client.? 172.21.15.89:0/152679945' entity='client.admin' cmd={"prefix": "osd dump"} : dispatch 2024-09-21T03:39:11.250 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:39:10 smithi110 ceph-mon[34000]: pgmap v353: 129 pgs: 129 active+clean; 646 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-21T03:39:11.250 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:39:10 smithi110 ceph-mon[34000]: from='client.? 172.21.15.89:0/152679945' entity='client.admin' cmd={"prefix": "osd dump"} : dispatch 2024-09-21T03:39:11.335 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:11.308+0000 7f126d974640 1 -- 172.21.15.89:0/4072969136 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f12681094a0 msgr2=0x7f12681098b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:11.336 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:11.308+0000 7f126d974640 1 --2- 172.21.15.89:0/4072969136 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f12681094a0 0x7f12681098b0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f1250009900 tx=0x7f125002a410 comp rx=0 tx=0).stop 2024-09-21T03:39:11.336 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:11.308+0000 7f126d974640 1 -- 172.21.15.89:0/4072969136 shutdown_connections 2024-09-21T03:39:11.336 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:11.308+0000 7f126d974640 1 --2- 172.21.15.89:0/4072969136 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1268109df0 0x7f1268110620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:11.336 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:11.308+0000 7f126d974640 1 --2- 172.21.15.89:0/4072969136 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f12681094a0 0x7f12681098b0 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:11.336 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:11.308+0000 7f126d974640 1 --2- 172.21.15.89:0/4072969136 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1268103b60 0x7f1268103f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:11.336 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:11.308+0000 7f126d974640 1 -- 172.21.15.89:0/4072969136 >> 172.21.15.89:0/4072969136 conn(0x7f126807b850 msgr2=0x7f126807bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:39:11.336 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:11.308+0000 7f126d974640 1 -- 172.21.15.89:0/4072969136 shutdown_connections 2024-09-21T03:39:11.336 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:11.309+0000 7f126d974640 1 -- 172.21.15.89:0/4072969136 wait complete. 2024-09-21T03:39:11.336 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:11.309+0000 7f126d974640 1 Processor -- start 2024-09-21T03:39:11.336 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:11.309+0000 7f126d974640 1 -- start start 2024-09-21T03:39:11.336 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:11.310+0000 7f126d974640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1268103b60 0x7f126819a410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:11.336 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:11.310+0000 7f126d974640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f12681094a0 0x7f126819a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:11.336 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:11.310+0000 7f126d974640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1268109df0 0x7f126819fc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:11.337 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:11.310+0000 7f126d974640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f1268113960 con 0x7f1268109df0 2024-09-21T03:39:11.337 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:11.310+0000 7f126d974640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f12681137e0 con 0x7f1268103b60 2024-09-21T03:39:11.337 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:11.310+0000 7f126d974640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f1268113ae0 con 0x7f12681094a0 2024-09-21T03:39:11.337 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:11.310+0000 7f12677fe640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1268109df0 0x7f126819fc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:11.337 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:11.310+0000 7f12677fe640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1268109df0 0x7f126819fc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:36842/0 (socket says 172.21.15.89:36842) 2024-09-21T03:39:11.337 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:11.310+0000 7f12677fe640 1 -- 172.21.15.89:0/3846809228 learned_addr learned my addr 172.21.15.89:0/3846809228 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-21T03:39:11.337 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:11.310+0000 7f12667fc640 1 --2- 172.21.15.89:0/3846809228 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f12681094a0 0x7f126819a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:11.337 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:11.310+0000 7f1266ffd640 1 --2- 172.21.15.89:0/3846809228 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1268103b60 0x7f126819a410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:11.337 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:11.310+0000 7f12677fe640 1 -- 172.21.15.89:0/3846809228 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f12681094a0 msgr2=0x7f126819a9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:11.337 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:11.310+0000 7f12677fe640 1 --2- 172.21.15.89:0/3846809228 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f12681094a0 0x7f126819a9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:11.337 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:11.310+0000 7f12677fe640 1 -- 172.21.15.89:0/3846809228 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1268103b60 msgr2=0x7f126819a410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:11.337 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:11.310+0000 7f12677fe640 1 --2- 172.21.15.89:0/3846809228 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1268103b60 0x7f126819a410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:11.337 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:11.310+0000 7f12677fe640 1 -- 172.21.15.89:0/3846809228 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12681a0380 con 0x7f1268109df0 2024-09-21T03:39:11.338 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:11.310+0000 7f1266ffd640 1 --2- 172.21.15.89:0/3846809228 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1268103b60 0x7f126819a410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:39:11.338 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:11.310+0000 7f12677fe640 1 --2- 172.21.15.89:0/3846809228 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1268109df0 0x7f126819fc50 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f126000eaa0 tx=0x7f126000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:39:11.338 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:11.310+0000 7f126c972640 1 -- 172.21.15.89:0/3846809228 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f126000cdb0 con 0x7f1268109df0 2024-09-21T03:39:11.338 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:11.310+0000 7f126d974640 1 -- 172.21.15.89:0/3846809228 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f12681a0670 con 0x7f1268109df0 2024-09-21T03:39:11.338 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:11.310+0000 7f126c972640 1 -- 172.21.15.89:0/3846809228 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1260005020 con 0x7f1268109df0 2024-09-21T03:39:11.338 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:11.311+0000 7f126c972640 1 -- 172.21.15.89:0/3846809228 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12600116d0 con 0x7f1268109df0 2024-09-21T03:39:11.338 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:11.311+0000 7f126d974640 1 -- 172.21.15.89:0/3846809228 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f12681a6dd0 con 0x7f1268109df0 2024-09-21T03:39:11.338 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:11.311+0000 7f126d974640 1 -- 172.21.15.89:0/3846809228 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1268103f70 con 0x7f1268109df0 2024-09-21T03:39:11.338 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:11.313+0000 7f126c972640 1 -- 172.21.15.89:0/3846809228 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 18) ==== 154563+0+0 (secure 0 0 0) 0x7f12600051d0 con 0x7f1268109df0 2024-09-21T03:39:11.338 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:11.314+0000 7f126c972640 1 --2- 172.21.15.89:0/3846809228 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f12540b65a0 0x7f12540b8ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:11.338 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:11.314+0000 7f1266ffd640 1 --2- 172.21.15.89:0/3846809228 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f12540b65a0 0x7f12540b8ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:11.338 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:11.314+0000 7f126c972640 1 -- 172.21.15.89:0/3846809228 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 9121+0+0 (secure 0 0 0) 0x7f12600e7e20 con 0x7f1268109df0 2024-09-21T03:39:11.338 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:11.315+0000 7f1266ffd640 1 --2- 172.21.15.89:0/3846809228 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f12540b65a0 0x7f12540b8ab0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f1258009b40 tx=0x7f1258005e50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:39:11.338 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:11.317+0000 7f126c972640 1 -- 172.21.15.89:0/3846809228 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f1260016030 con 0x7f1268109df0 2024-09-21T03:39:11.542 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:11.539+0000 7f126d974640 1 -- 172.21.15.89:0/3846809228 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f126819b540 con 0x7f1268109df0 2024-09-21T03:39:11.542 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:11.539+0000 7f126c972640 1 -- 172.21.15.89:0/3846809228 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11912 (secure 0 0 0) 0x7f126819b540 con 0x7f1268109df0 2024-09-21T03:39:11.542 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-09-21T03:39:11.542 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":25,"btime":"2024-09-21T03:37:51:019398+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14721,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6835/2341538514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":2341538514},{"type":"v1","addr":"172.21.15.89:6835","nonce":2341538514}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14733,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6837/146708592","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6836","nonce":146708592},{"type":"v1","addr":"172.21.15.89:6837","nonce":146708592}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14781,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6839/2269965499","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6838","nonce":2269965499},{"type":"v1","addr":"172.21.15.89:6839","nonce":2269965499}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14793,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6841/650850383","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6840","nonce":650850383},{"type":"v1","addr":"172.21.15.89:6841","nonce":650850383}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24565,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6833/752336912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6832","nonce":752336912},{"type":"v1","addr":"172.21.15.114:6833","nonce":752336912}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24575,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6833/127607331","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6832","nonce":127607331},{"type":"v1","addr":"172.21.15.110:6833","nonce":127607331}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T03:37:13.570694+0000","modified":"2024-09-21T03:37:50.719106+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24638,"mds_1":24631,"mds_2":24614},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24593":{"gid":24593,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.110:6835/671820654","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6834","nonce":671820654},{"type":"v1","addr":"172.21.15.110:6835","nonce":671820654}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24604":{"gid":24604,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.114:6835/1049714548","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6834","nonce":1049714548},{"type":"v1","addr":"172.21.15.114:6835","nonce":1049714548}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24610":{"gid":24610,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.114:6837/1941213818","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6836","nonce":1941213818},{"type":"v1","addr":"172.21.15.114:6837","nonce":1941213818}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24614":{"gid":24614,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":15,"addr":"172.21.15.110:6837/4047124589","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6836","nonce":4047124589},{"type":"v1","addr":"172.21.15.110:6837","nonce":4047124589}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24631":{"gid":24631,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":15,"addr":"172.21.15.114:6839/3427369944","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6838","nonce":3427369944},{"type":"v1","addr":"172.21.15.114:6839","nonce":3427369944}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24638":{"gid":24638,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.110:6839/132366709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6838","nonce":132366709},{"type":"v1","addr":"172.21.15.110:6839","nonce":132366709}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24638,"qdb_cluster":[24614,24631,24638]},"id":1}]} 2024-09-21T03:39:11.543 INFO:teuthology.orchestra.run.smithi089.stderr:dumped fsmap epoch 25 2024-09-21T03:39:11.545 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:11.542+0000 7f126d974640 1 -- 172.21.15.89:0/3846809228 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f12540b65a0 msgr2=0x7f12540b8ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:11.545 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:11.542+0000 7f126d974640 1 --2- 172.21.15.89:0/3846809228 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f12540b65a0 0x7f12540b8ab0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f1258009b40 tx=0x7f1258005e50 comp rx=0 tx=0).stop 2024-09-21T03:39:11.545 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:11.542+0000 7f126d974640 1 -- 172.21.15.89:0/3846809228 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1268109df0 msgr2=0x7f126819fc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:11.545 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:11.542+0000 7f126d974640 1 --2- 172.21.15.89:0/3846809228 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1268109df0 0x7f126819fc50 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f126000eaa0 tx=0x7f126000ef60 comp rx=0 tx=0).stop 2024-09-21T03:39:11.545 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:11.542+0000 7f126d974640 1 -- 172.21.15.89:0/3846809228 shutdown_connections 2024-09-21T03:39:11.545 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:11.542+0000 7f126d974640 1 --2- 172.21.15.89:0/3846809228 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f12540b65a0 0x7f12540b8ab0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:11.545 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:11.542+0000 7f126d974640 1 --2- 172.21.15.89:0/3846809228 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1268109df0 0x7f126819fc50 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:11.546 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:11.542+0000 7f126d974640 1 --2- 172.21.15.89:0/3846809228 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f12681094a0 0x7f126819a9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:11.546 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:11.542+0000 7f126d974640 1 --2- 172.21.15.89:0/3846809228 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1268103b60 0x7f126819a410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:11.546 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:11.542+0000 7f126d974640 1 -- 172.21.15.89:0/3846809228 >> 172.21.15.89:0/3846809228 conn(0x7f126807b850 msgr2=0x7f12681055d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:39:11.546 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:11.543+0000 7f126d974640 1 -- 172.21.15.89:0/3846809228 shutdown_connections 2024-09-21T03:39:11.546 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:11.543+0000 7f126d974640 1 -- 172.21.15.89:0/3846809228 wait complete. 2024-09-21T03:39:11.811 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:39:11 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:39:11.811 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:39:11 smithi089 ceph-mon[30928]: from='client.? 172.21.15.89:0/3846809228' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:39:12.043 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:39:11 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:39:12.043 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:39:11 smithi114 ceph-mon[34140]: from='client.? 172.21.15.89:0/3846809228' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:39:12.181 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:39:11 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:39:12.182 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:39:11 smithi110 ceph-mon[34000]: from='client.? 172.21.15.89:0/3846809228' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:39:12.210 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 7c374098-77c9-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T03:39:12.978 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/7c374098-77c9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:39:13.061 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:39:12 smithi089 ceph-mon[30928]: pgmap v354: 129 pgs: 129 active+clean; 646 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-21T03:39:13.210 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:39:12 smithi114 ceph-mon[34140]: pgmap v354: 129 pgs: 129 active+clean; 646 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-21T03:39:13.250 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:39:12 smithi110 ceph-mon[34000]: pgmap v354: 129 pgs: 129 active+clean; 646 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-21T03:39:13.441 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:13.436+0000 7f4ebeb0f640 1 -- 172.21.15.89:0/3599620260 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4eb81062d0 msgr2=0x7f4eb81066e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:13.441 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:13.436+0000 7f4ebeb0f640 1 --2- 172.21.15.89:0/3599620260 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4eb81062d0 0x7f4eb81066e0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f4ea8009900 tx=0x7f4ea802a410 comp rx=0 tx=0).stop 2024-09-21T03:39:13.441 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:13.436+0000 7f4ebeb0f640 1 -- 172.21.15.89:0/3599620260 shutdown_connections 2024-09-21T03:39:13.441 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:13.436+0000 7f4ebeb0f640 1 --2- 172.21.15.89:0/3599620260 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4eb8106c20 0x7f4eb8110670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:13.441 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:13.436+0000 7f4ebeb0f640 1 --2- 172.21.15.89:0/3599620260 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4eb81062d0 0x7f4eb81066e0 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:13.441 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:13.436+0000 7f4ebeb0f640 1 --2- 172.21.15.89:0/3599620260 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4eb8101ff0 0x7f4eb8102400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:13.441 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:13.436+0000 7f4ebeb0f640 1 -- 172.21.15.89:0/3599620260 >> 172.21.15.89:0/3599620260 conn(0x7f4eb8079e30 msgr2=0x7f4eb807a240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:39:13.442 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:13.436+0000 7f4ebeb0f640 1 -- 172.21.15.89:0/3599620260 shutdown_connections 2024-09-21T03:39:13.442 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:13.436+0000 7f4ebeb0f640 1 -- 172.21.15.89:0/3599620260 wait complete. 2024-09-21T03:39:13.442 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:13.437+0000 7f4ebeb0f640 1 Processor -- start 2024-09-21T03:39:13.442 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:13.437+0000 7f4ebeb0f640 1 -- start start 2024-09-21T03:39:13.442 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:13.437+0000 7f4ebeb0f640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4eb8101ff0 0x7f4eb8102f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:13.442 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:13.437+0000 7f4ebeb0f640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4eb81062d0 0x7f4eb81034d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:13.442 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:13.437+0000 7f4ebeb0f640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4eb8106c20 0x7f4eb8103a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:13.442 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:13.437+0000 7f4ebeb0f640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f4eb8113970 con 0x7f4eb8106c20 2024-09-21T03:39:13.442 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:13.437+0000 7f4ebeb0f640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f4eb81137f0 con 0x7f4eb8101ff0 2024-09-21T03:39:13.442 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:13.437+0000 7f4ebeb0f640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f4eb8113af0 con 0x7f4eb81062d0 2024-09-21T03:39:13.442 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:13.438+0000 7f4ebd0ac640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4eb8106c20 0x7f4eb8103a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:13.442 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:13.438+0000 7f4ebd0ac640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4eb8106c20 0x7f4eb8103a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:36858/0 (socket says 172.21.15.89:36858) 2024-09-21T03:39:13.442 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:13.438+0000 7f4ebd0ac640 1 -- 172.21.15.89:0/2250202139 learned_addr learned my addr 172.21.15.89:0/2250202139 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-21T03:39:13.442 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:13.438+0000 7f4ebc8ab640 1 --2- 172.21.15.89:0/2250202139 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4eb8101ff0 0x7f4eb8102f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:13.443 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:13.438+0000 7f4eaffff640 1 --2- 172.21.15.89:0/2250202139 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4eb81062d0 0x7f4eb81034d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:13.443 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:13.438+0000 7f4ebd0ac640 1 -- 172.21.15.89:0/2250202139 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4eb81062d0 msgr2=0x7f4eb81034d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:13.443 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:13.438+0000 7f4ebd0ac640 1 --2- 172.21.15.89:0/2250202139 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4eb81062d0 0x7f4eb81034d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:13.443 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:13.438+0000 7f4ebd0ac640 1 -- 172.21.15.89:0/2250202139 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4eb8101ff0 msgr2=0x7f4eb8102f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:13.443 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:13.438+0000 7f4ebd0ac640 1 --2- 172.21.15.89:0/2250202139 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4eb8101ff0 0x7f4eb8102f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:13.443 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:13.438+0000 7f4ebd0ac640 1 -- 172.21.15.89:0/2250202139 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4eb81a8660 con 0x7f4eb8106c20 2024-09-21T03:39:13.443 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:13.438+0000 7f4ebd0ac640 1 --2- 172.21.15.89:0/2250202139 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4eb8106c20 0x7f4eb8103a10 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f4eb000eb10 tx=0x7f4eb000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:39:13.443 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:13.438+0000 7f4eadffb640 1 -- 172.21.15.89:0/2250202139 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4eb0004b20 con 0x7f4eb8106c20 2024-09-21T03:39:13.443 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:13.438+0000 7f4ebeb0f640 1 -- 172.21.15.89:0/2250202139 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4eb81a8950 con 0x7f4eb8106c20 2024-09-21T03:39:13.443 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:13.438+0000 7f4eadffb640 1 -- 172.21.15.89:0/2250202139 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4eb0004cc0 con 0x7f4eb8106c20 2024-09-21T03:39:13.443 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:13.438+0000 7f4eadffb640 1 -- 172.21.15.89:0/2250202139 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4eb0010690 con 0x7f4eb8106c20 2024-09-21T03:39:13.443 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:13.439+0000 7f4ebeb0f640 1 -- 172.21.15.89:0/2250202139 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4eb81a8e60 con 0x7f4eb8106c20 2024-09-21T03:39:13.444 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:13.440+0000 7f4ebeb0f640 1 -- 172.21.15.89:0/2250202139 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4eb8102400 con 0x7f4eb8106c20 2024-09-21T03:39:13.444 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:13.441+0000 7f4eadffb640 1 -- 172.21.15.89:0/2250202139 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 18) ==== 154563+0+0 (secure 0 0 0) 0x7f4eb0005000 con 0x7f4eb8106c20 2024-09-21T03:39:13.444 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:13.442+0000 7f4eadffb640 1 --2- 172.21.15.89:0/2250202139 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f4ea00b65a0 0x7f4ea00b8ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:13.444 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:13.442+0000 7f4eadffb640 1 -- 172.21.15.89:0/2250202139 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 9121+0+0 (secure 0 0 0) 0x7f4eb00e8000 con 0x7f4eb8106c20 2024-09-21T03:39:13.444 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:13.442+0000 7f4ebc8ab640 1 --2- 172.21.15.89:0/2250202139 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f4ea00b65a0 0x7f4ea00b8ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:13.445 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:13.442+0000 7f4ebc8ab640 1 --2- 172.21.15.89:0/2250202139 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f4ea00b65a0 0x7f4ea00b8ab0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f4eb81049e0 tx=0x7f4eb400b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:39:13.447 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:13.445+0000 7f4eadffb640 1 -- 172.21.15.89:0/2250202139 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f4eb0056710 con 0x7f4eb8106c20 2024-09-21T03:39:13.683 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:13.681+0000 7f4ebeb0f640 1 -- 172.21.15.89:0/2250202139 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4eb8104290 con 0x7f4eb8106c20 2024-09-21T03:39:13.684 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:13.682+0000 7f4eadffb640 1 -- 172.21.15.89:0/2250202139 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11912 (secure 0 0 0) 0x7f4eb8104290 con 0x7f4eb8106c20 2024-09-21T03:39:13.684 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-09-21T03:39:13.685 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":25,"btime":"2024-09-21T03:37:51:019398+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14721,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6835/2341538514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":2341538514},{"type":"v1","addr":"172.21.15.89:6835","nonce":2341538514}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14733,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6837/146708592","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6836","nonce":146708592},{"type":"v1","addr":"172.21.15.89:6837","nonce":146708592}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14781,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6839/2269965499","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6838","nonce":2269965499},{"type":"v1","addr":"172.21.15.89:6839","nonce":2269965499}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14793,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6841/650850383","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6840","nonce":650850383},{"type":"v1","addr":"172.21.15.89:6841","nonce":650850383}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24565,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6833/752336912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6832","nonce":752336912},{"type":"v1","addr":"172.21.15.114:6833","nonce":752336912}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24575,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6833/127607331","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6832","nonce":127607331},{"type":"v1","addr":"172.21.15.110:6833","nonce":127607331}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T03:37:13.570694+0000","modified":"2024-09-21T03:37:50.719106+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24638,"mds_1":24631,"mds_2":24614},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24593":{"gid":24593,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.110:6835/671820654","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6834","nonce":671820654},{"type":"v1","addr":"172.21.15.110:6835","nonce":671820654}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24604":{"gid":24604,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.114:6835/1049714548","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6834","nonce":1049714548},{"type":"v1","addr":"172.21.15.114:6835","nonce":1049714548}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24610":{"gid":24610,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.114:6837/1941213818","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6836","nonce":1941213818},{"type":"v1","addr":"172.21.15.114:6837","nonce":1941213818}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24614":{"gid":24614,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":15,"addr":"172.21.15.110:6837/4047124589","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6836","nonce":4047124589},{"type":"v1","addr":"172.21.15.110:6837","nonce":4047124589}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24631":{"gid":24631,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":15,"addr":"172.21.15.114:6839/3427369944","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6838","nonce":3427369944},{"type":"v1","addr":"172.21.15.114:6839","nonce":3427369944}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24638":{"gid":24638,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.110:6839/132366709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6838","nonce":132366709},{"type":"v1","addr":"172.21.15.110:6839","nonce":132366709}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24638,"qdb_cluster":[24614,24631,24638]},"id":1}]} 2024-09-21T03:39:13.685 INFO:teuthology.orchestra.run.smithi089.stderr:dumped fsmap epoch 25 2024-09-21T03:39:13.687 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:13.685+0000 7f4ebeb0f640 1 -- 172.21.15.89:0/2250202139 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f4ea00b65a0 msgr2=0x7f4ea00b8ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:13.687 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:13.685+0000 7f4ebeb0f640 1 --2- 172.21.15.89:0/2250202139 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f4ea00b65a0 0x7f4ea00b8ab0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f4eb81049e0 tx=0x7f4eb400b390 comp rx=0 tx=0).stop 2024-09-21T03:39:13.687 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:13.685+0000 7f4ebeb0f640 1 -- 172.21.15.89:0/2250202139 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4eb8106c20 msgr2=0x7f4eb8103a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:13.687 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:13.685+0000 7f4ebeb0f640 1 --2- 172.21.15.89:0/2250202139 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4eb8106c20 0x7f4eb8103a10 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f4eb000eb10 tx=0x7f4eb000c2d0 comp rx=0 tx=0).stop 2024-09-21T03:39:13.687 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:13.685+0000 7f4ebeb0f640 1 -- 172.21.15.89:0/2250202139 shutdown_connections 2024-09-21T03:39:13.688 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:13.685+0000 7f4ebeb0f640 1 --2- 172.21.15.89:0/2250202139 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f4ea00b65a0 0x7f4ea00b8ab0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:13.688 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:13.685+0000 7f4ebeb0f640 1 --2- 172.21.15.89:0/2250202139 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4eb8106c20 0x7f4eb8103a10 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:13.688 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:13.685+0000 7f4ebeb0f640 1 --2- 172.21.15.89:0/2250202139 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4eb81062d0 0x7f4eb81034d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:13.688 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:13.685+0000 7f4ebeb0f640 1 --2- 172.21.15.89:0/2250202139 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4eb8101ff0 0x7f4eb8102f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:13.688 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:13.685+0000 7f4ebeb0f640 1 -- 172.21.15.89:0/2250202139 >> 172.21.15.89:0/2250202139 conn(0x7f4eb8079e30 msgr2=0x7f4eb81075e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:39:13.688 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:13.685+0000 7f4ebeb0f640 1 -- 172.21.15.89:0/2250202139 shutdown_connections 2024-09-21T03:39:13.688 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:13.685+0000 7f4ebeb0f640 1 -- 172.21.15.89:0/2250202139 wait complete. 2024-09-21T03:39:14.061 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:39:13 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:39:14.061 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:39:13 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:39:14.061 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:39:13 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:39:14.061 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:39:13 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:39:14.061 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:39:13 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:39:14.061 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:39:13 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:39:14.061 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:39:13 smithi089 ceph-mon[30928]: from='client.? 172.21.15.89:0/2250202139' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:39:14.211 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:39:13 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:39:14.211 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:39:13 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:39:14.211 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:39:13 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:39:14.211 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:39:13 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:39:14.211 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:39:13 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:39:14.211 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:39:13 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:39:14.211 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:39:13 smithi114 ceph-mon[34140]: from='client.? 172.21.15.89:0/2250202139' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:39:14.250 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:39:13 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:39:14.250 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:39:13 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:39:14.250 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:39:13 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:39:14.250 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:39:13 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:39:14.250 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:39:13 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:39:14.250 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:39:13 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:39:14.250 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:39:13 smithi110 ceph-mon[34000]: from='client.? 172.21.15.89:0/2250202139' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:39:14.373 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 7c374098-77c9-11ef-baf5-efdc52797490 -- ceph osd dump --format=json 2024-09-21T03:39:14.619 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/7c374098-77c9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:39:15.102 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:15.080+0000 7f49d8dea640 1 -- 172.21.15.89:0/134677348 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f49d4109df0 msgr2=0x7f49d4110620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:15.102 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:15.080+0000 7f49d8dea640 1 --2- 172.21.15.89:0/134677348 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f49d4109df0 0x7f49d4110620 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f49c4009900 tx=0x7f49c402a410 comp rx=0 tx=0).stop 2024-09-21T03:39:15.102 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:15.080+0000 7f49d8dea640 1 -- 172.21.15.89:0/134677348 shutdown_connections 2024-09-21T03:39:15.102 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:15.080+0000 7f49d8dea640 1 --2- 172.21.15.89:0/134677348 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f49d4109df0 0x7f49d4110620 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:15.103 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:15.080+0000 7f49d8dea640 1 --2- 172.21.15.89:0/134677348 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f49d41094a0 0x7f49d41098b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:15.103 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:15.080+0000 7f49d8dea640 1 --2- 172.21.15.89:0/134677348 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f49d4103b60 0x7f49d4103f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:15.103 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:15.081+0000 7f49d8dea640 1 -- 172.21.15.89:0/134677348 >> 172.21.15.89:0/134677348 conn(0x7f49d407b850 msgr2=0x7f49d407bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:39:15.103 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:15.081+0000 7f49d8dea640 1 -- 172.21.15.89:0/134677348 shutdown_connections 2024-09-21T03:39:15.103 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:15.081+0000 7f49d8dea640 1 -- 172.21.15.89:0/134677348 wait complete. 2024-09-21T03:39:15.103 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:15.081+0000 7f49d8dea640 1 Processor -- start 2024-09-21T03:39:15.103 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:15.081+0000 7f49d8dea640 1 -- start start 2024-09-21T03:39:15.103 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:15.082+0000 7f49d8dea640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f49d4103b60 0x7f49d419c590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:15.103 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:15.082+0000 7f49d8dea640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f49d41094a0 0x7f49d419cb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:15.103 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:15.082+0000 7f49d8dea640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f49d4109df0 0x7f49d41a1dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:15.103 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:15.082+0000 7f49d8dea640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f49d4113960 con 0x7f49d41094a0 2024-09-21T03:39:15.103 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:15.082+0000 7f49d8dea640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f49d41137e0 con 0x7f49d4103b60 2024-09-21T03:39:15.103 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:15.082+0000 7f49d8dea640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f49d4113ae0 con 0x7f49d4109df0 2024-09-21T03:39:15.103 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:15.082+0000 7f49d1d9b640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f49d41094a0 0x7f49d419cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:15.103 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:15.082+0000 7f49d1d9b640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f49d41094a0 0x7f49d419cb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:36886/0 (socket says 172.21.15.89:36886) 2024-09-21T03:39:15.104 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:15.082+0000 7f49d1d9b640 1 -- 172.21.15.89:0/370027105 learned_addr learned my addr 172.21.15.89:0/370027105 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-21T03:39:15.104 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:15.082+0000 7f49d2d9d640 1 --2- 172.21.15.89:0/370027105 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f49d4109df0 0x7f49d41a1dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:15.104 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:15.082+0000 7f49d259c640 1 --2- 172.21.15.89:0/370027105 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f49d4103b60 0x7f49d419c590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:15.104 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:15.082+0000 7f49d1d9b640 1 -- 172.21.15.89:0/370027105 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f49d4109df0 msgr2=0x7f49d41a1dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:15.104 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:15.082+0000 7f49d1d9b640 1 --2- 172.21.15.89:0/370027105 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f49d4109df0 0x7f49d41a1dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:15.104 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:15.082+0000 7f49d1d9b640 1 -- 172.21.15.89:0/370027105 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f49d4103b60 msgr2=0x7f49d419c590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:15.104 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:15.082+0000 7f49d1d9b640 1 --2- 172.21.15.89:0/370027105 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f49d4103b60 0x7f49d419c590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:15.104 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:15.082+0000 7f49d1d9b640 1 -- 172.21.15.89:0/370027105 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49d41a2440 con 0x7f49d41094a0 2024-09-21T03:39:15.104 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:15.082+0000 7f49d1d9b640 1 --2- 172.21.15.89:0/370027105 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f49d41094a0 0x7f49d419cb30 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f49bc00d750 tx=0x7f49bc00dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:39:15.104 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:15.083+0000 7f49cb7fe640 1 -- 172.21.15.89:0/370027105 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49bc017070 con 0x7f49d41094a0 2024-09-21T03:39:15.104 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:15.083+0000 7f49cb7fe640 1 -- 172.21.15.89:0/370027105 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f49bc0054b0 con 0x7f49d41094a0 2024-09-21T03:39:15.104 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:15.083+0000 7f49d8dea640 1 -- 172.21.15.89:0/370027105 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f49d41a2730 con 0x7f49d41094a0 2024-09-21T03:39:15.104 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:15.083+0000 7f49cb7fe640 1 -- 172.21.15.89:0/370027105 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49bc004bc0 con 0x7f49d41094a0 2024-09-21T03:39:15.104 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:15.083+0000 7f49d8dea640 1 -- 172.21.15.89:0/370027105 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49d41a8f50 con 0x7f49d41094a0 2024-09-21T03:39:15.105 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:15.084+0000 7f49d8dea640 1 -- 172.21.15.89:0/370027105 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f49d4104de0 con 0x7f49d41094a0 2024-09-21T03:39:15.105 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:15.086+0000 7f49cb7fe640 1 -- 172.21.15.89:0/370027105 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 18) ==== 154563+0+0 (secure 0 0 0) 0x7f49bc009c30 con 0x7f49d41094a0 2024-09-21T03:39:15.105 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:15.087+0000 7f49cb7fe640 1 --2- 172.21.15.89:0/370027105 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f49c00b6650 0x7f49c00b8b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:15.105 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:15.087+0000 7f49d259c640 1 --2- 172.21.15.89:0/370027105 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f49c00b6650 0x7f49c00b8b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:15.105 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:15.087+0000 7f49cb7fe640 1 -- 172.21.15.89:0/370027105 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 9121+0+0 (secure 0 0 0) 0x7f49bc0e7510 con 0x7f49d41094a0 2024-09-21T03:39:15.105 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:15.087+0000 7f49d259c640 1 --2- 172.21.15.89:0/370027105 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f49c00b6650 0x7f49c00b8b60 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f49cc007ec0 tx=0x7f49cc0075b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:39:15.105 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:15.090+0000 7f49cb7fe640 1 -- 172.21.15.89:0/370027105 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f49bc0b1d20 con 0x7f49d41094a0 2024-09-21T03:39:15.270 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:15.268+0000 7f49d8dea640 1 -- 172.21.15.89:0/370027105 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f49d41a9120 con 0x7f49d41094a0 2024-09-21T03:39:15.270 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:39:15 smithi089 ceph-mon[30928]: pgmap v355: 129 pgs: 129 active+clean; 646 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-21T03:39:15.270 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:39:15 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:39:15.270 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:39:15 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:39:15.270 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:39:15 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:39:15.274 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:15.269+0000 7f49cb7fe640 1 -- 172.21.15.89:0/370027105 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v84) ==== 74+0+24810 (secure 0 0 0) 0x7f49bc0b6c60 con 0x7f49d41094a0 2024-09-21T03:39:15.274 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-09-21T03:39:15.274 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":84,"fsid":"7c374098-77c9-11ef-baf5-efdc52797490","created":"2024-09-21T03:28:56.681458+0000","modified":"2024-09-21T03:38:53.661606+0000","last_up_change":"2024-09-21T03:36:23.368379+0000","last_in_change":"2024-09-21T03:36:00.131331+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T03:32:37.834339+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-21T03:37:09.443341+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"74","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-21T03:37:11.467821+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"74","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-21T03:37:17.884066+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"27ca5ed8-17d1-4f8e-a7ae-084a914721a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6802","nonce":990539042},{"type":"v1","addr":"172.21.15.89:6803","nonce":990539042}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6804","nonce":990539042},{"type":"v1","addr":"172.21.15.89:6805","nonce":990539042}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6808","nonce":990539042},{"type":"v1","addr":"172.21.15.89:6809","nonce":990539042}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6806","nonce":990539042},{"type":"v1","addr":"172.21.15.89:6807","nonce":990539042}]},"public_addr":"172.21.15.89:6803/990539042","cluster_addr":"172.21.15.89:6805/990539042","heartbeat_back_addr":"172.21.15.89:6809/990539042","heartbeat_front_addr":"172.21.15.89:6807/990539042","state":["exists","up"]},{"osd":1,"uuid":"5176a910-5cf9-49d7-aedb-a20cf36ea075","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6800","nonce":3235852628},{"type":"v1","addr":"172.21.15.110:6801","nonce":3235852628}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6802","nonce":3235852628},{"type":"v1","addr":"172.21.15.110:6803","nonce":3235852628}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6806","nonce":3235852628},{"type":"v1","addr":"172.21.15.110:6807","nonce":3235852628}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6804","nonce":3235852628},{"type":"v1","addr":"172.21.15.110:6805","nonce":3235852628}]},"public_addr":"172.21.15.110:6801/3235852628","cluster_addr":"172.21.15.110:6803/3235852628","heartbeat_back_addr":"172.21.15.110:6807/3235852628","heartbeat_front_addr":"172.21.15.110:6805/3235852628","state":["exists","up"]},{"osd":2,"uuid":"d4977df1-b118-4d7e-b9a9-626a8accc812","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6800","nonce":169452122},{"type":"v1","addr":"172.21.15.114:6801","nonce":169452122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6802","nonce":169452122},{"type":"v1","addr":"172.21.15.114:6803","nonce":169452122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6806","nonce":169452122},{"type":"v1","addr":"172.21.15.114:6807","nonce":169452122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6804","nonce":169452122},{"type":"v1","addr":"172.21.15.114:6805","nonce":169452122}]},"public_addr":"172.21.15.114:6801/169452122","cluster_addr":"172.21.15.114:6803/169452122","heartbeat_back_addr":"172.21.15.114:6807/169452122","heartbeat_front_addr":"172.21.15.114:6805/169452122","state":["exists","up"]},{"osd":3,"uuid":"2c7ae1b4-0bf2-4954-81ed-55d03eab82c6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6810","nonce":3016980877},{"type":"v1","addr":"172.21.15.89:6811","nonce":3016980877}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6812","nonce":3016980877},{"type":"v1","addr":"172.21.15.89:6813","nonce":3016980877}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6816","nonce":3016980877},{"type":"v1","addr":"172.21.15.89:6817","nonce":3016980877}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6814","nonce":3016980877},{"type":"v1","addr":"172.21.15.89:6815","nonce":3016980877}]},"public_addr":"172.21.15.89:6811/3016980877","cluster_addr":"172.21.15.89:6813/3016980877","heartbeat_back_addr":"172.21.15.89:6817/3016980877","heartbeat_front_addr":"172.21.15.89:6815/3016980877","state":["exists","up"]},{"osd":4,"uuid":"8e65399d-6e55-4da4-98ba-f805f0c655a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":30,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6808","nonce":3217880476},{"type":"v1","addr":"172.21.15.110:6809","nonce":3217880476}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6810","nonce":3217880476},{"type":"v1","addr":"172.21.15.110:6811","nonce":3217880476}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6814","nonce":3217880476},{"type":"v1","addr":"172.21.15.110:6815","nonce":3217880476}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6812","nonce":3217880476},{"type":"v1","addr":"172.21.15.110:6813","nonce":3217880476}]},"public_addr":"172.21.15.110:6809/3217880476","cluster_addr":"172.21.15.110:6811/3217880476","heartbeat_back_addr":"172.21.15.110:6815/3217880476","heartbeat_front_addr":"172.21.15.110:6813/3217880476","state":["exists","up"]},{"osd":5,"uuid":"b9ec2889-46bd-40e5-8446-73ab28271c67","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6808","nonce":1144140707},{"type":"v1","addr":"172.21.15.114:6809","nonce":1144140707}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6810","nonce":1144140707},{"type":"v1","addr":"172.21.15.114:6811","nonce":1144140707}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6814","nonce":1144140707},{"type":"v1","addr":"172.21.15.114:6815","nonce":1144140707}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6812","nonce":1144140707},{"type":"v1","addr":"172.21.15.114:6813","nonce":1144140707}]},"public_addr":"172.21.15.114:6809/1144140707","cluster_addr":"172.21.15.114:6811/1144140707","heartbeat_back_addr":"172.21.15.114:6815/1144140707","heartbeat_front_addr":"172.21.15.114:6813/1144140707","state":["exists","up"]},{"osd":6,"uuid":"9b79989e-adb0-42b2-b2f6-a2dce56b66e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":41,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6818","nonce":1522586255},{"type":"v1","addr":"172.21.15.89:6819","nonce":1522586255}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6820","nonce":1522586255},{"type":"v1","addr":"172.21.15.89:6821","nonce":1522586255}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6824","nonce":1522586255},{"type":"v1","addr":"172.21.15.89:6825","nonce":1522586255}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6822","nonce":1522586255},{"type":"v1","addr":"172.21.15.89:6823","nonce":1522586255}]},"public_addr":"172.21.15.89:6819/1522586255","cluster_addr":"172.21.15.89:6821/1522586255","heartbeat_back_addr":"172.21.15.89:6825/1522586255","heartbeat_front_addr":"172.21.15.89:6823/1522586255","state":["exists","up"]},{"osd":7,"uuid":"b381821f-141c-47a1-be65-f57ca730391b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6816","nonce":1331818323},{"type":"v1","addr":"172.21.15.110:6817","nonce":1331818323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6818","nonce":1331818323},{"type":"v1","addr":"172.21.15.110:6819","nonce":1331818323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6822","nonce":1331818323},{"type":"v1","addr":"172.21.15.110:6823","nonce":1331818323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6820","nonce":1331818323},{"type":"v1","addr":"172.21.15.110:6821","nonce":1331818323}]},"public_addr":"172.21.15.110:6817/1331818323","cluster_addr":"172.21.15.110:6819/1331818323","heartbeat_back_addr":"172.21.15.110:6823/1331818323","heartbeat_front_addr":"172.21.15.110:6821/1331818323","state":["exists","up"]},{"osd":8,"uuid":"1bc2a5b4-15f3-456f-9d67-e51307a27bcc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6816","nonce":2138556699},{"type":"v1","addr":"172.21.15.114:6817","nonce":2138556699}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6818","nonce":2138556699},{"type":"v1","addr":"172.21.15.114:6819","nonce":2138556699}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6822","nonce":2138556699},{"type":"v1","addr":"172.21.15.114:6823","nonce":2138556699}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6820","nonce":2138556699},{"type":"v1","addr":"172.21.15.114:6821","nonce":2138556699}]},"public_addr":"172.21.15.114:6817/2138556699","cluster_addr":"172.21.15.114:6819/2138556699","heartbeat_back_addr":"172.21.15.114:6823/2138556699","heartbeat_front_addr":"172.21.15.114:6821/2138556699","state":["exists","up"]},{"osd":9,"uuid":"91f5fd29-d4a7-4800-ac7a-e83b13add89e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6826","nonce":1730253068},{"type":"v1","addr":"172.21.15.89:6827","nonce":1730253068}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6828","nonce":1730253068},{"type":"v1","addr":"172.21.15.89:6829","nonce":1730253068}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6832","nonce":1730253068},{"type":"v1","addr":"172.21.15.89:6833","nonce":1730253068}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6830","nonce":1730253068},{"type":"v1","addr":"172.21.15.89:6831","nonce":1730253068}]},"public_addr":"172.21.15.89:6827/1730253068","cluster_addr":"172.21.15.89:6829/1730253068","heartbeat_back_addr":"172.21.15.89:6833/1730253068","heartbeat_front_addr":"172.21.15.89:6831/1730253068","state":["exists","up"]},{"osd":10,"uuid":"01390499-2a4d-4188-b57a-fcb10c668e2c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6824","nonce":2904016117},{"type":"v1","addr":"172.21.15.110:6825","nonce":2904016117}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6826","nonce":2904016117},{"type":"v1","addr":"172.21.15.110:6827","nonce":2904016117}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6830","nonce":2904016117},{"type":"v1","addr":"172.21.15.110:6831","nonce":2904016117}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6828","nonce":2904016117},{"type":"v1","addr":"172.21.15.110:6829","nonce":2904016117}]},"public_addr":"172.21.15.110:6825/2904016117","cluster_addr":"172.21.15.110:6827/2904016117","heartbeat_back_addr":"172.21.15.110:6831/2904016117","heartbeat_front_addr":"172.21.15.110:6829/2904016117","state":["exists","up"]},{"osd":11,"uuid":"c3c0f38a-c3d3-415d-a9f8-875d5b756e35","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6824","nonce":3749854486},{"type":"v1","addr":"172.21.15.114:6825","nonce":3749854486}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6826","nonce":3749854486},{"type":"v1","addr":"172.21.15.114:6827","nonce":3749854486}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6830","nonce":3749854486},{"type":"v1","addr":"172.21.15.114:6831","nonce":3749854486}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6828","nonce":3749854486},{"type":"v1","addr":"172.21.15.114:6829","nonce":3749854486}]},"public_addr":"172.21.15.114:6825/3749854486","cluster_addr":"172.21.15.114:6827/3749854486","heartbeat_back_addr":"172.21.15.114:6831/3749854486","heartbeat_front_addr":"172.21.15.114:6829/3749854486","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:31:46.218908+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:32:10.338230+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:32:34.830596+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:33:00.816102+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:33:25.223838+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:33:50.188230+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:34:16.564255+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:34:41.498230+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:35:05.826322+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:35:31.234450+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:35:56.474373+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:36:21.741254+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.c","mappings":[{"from":2,"to":4}]},{"pgid":"2.e","mappings":[{"from":2,"to":1}]},{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.1b","mappings":[{"from":5,"to":9}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]},{"pgid":"4.6","mappings":[{"from":9,"to":11}]},{"pgid":"4.12","mappings":[{"from":9,"to":10}]},{"pgid":"4.13","mappings":[{"from":9,"to":8}]},{"pgid":"4.1b","mappings":[{"from":5,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.89:6801/655110910":"2024-09-22T03:29:51.519106+0000","172.21.15.89:0/1618544983":"2024-09-22T03:29:51.519106+0000","172.21.15.89:0/1265502758":"2024-09-22T03:29:51.519106+0000","172.21.15.89:0/4017684419":"2024-09-22T03:29:51.519106+0000","172.21.15.89:6801/3312712420":"2024-09-22T03:29:26.004686+0000","172.21.15.89:6800/3312712420":"2024-09-22T03:29:26.004686+0000","172.21.15.89:0/2056576483":"2024-09-22T03:29:26.004686+0000","172.21.15.89:6800/655110910":"2024-09-22T03:29:51.519106+0000","172.21.15.89:0/2382866302":"2024-09-22T03:29:26.004686+0000","172.21.15.89:0/1584351801":"2024-09-22T03:29:26.004686+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T03:39:15.275 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:15.271+0000 7f49d8dea640 1 -- 172.21.15.89:0/370027105 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f49c00b6650 msgr2=0x7f49c00b8b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:15.275 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:15.271+0000 7f49d8dea640 1 --2- 172.21.15.89:0/370027105 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f49c00b6650 0x7f49c00b8b60 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f49cc007ec0 tx=0x7f49cc0075b0 comp rx=0 tx=0).stop 2024-09-21T03:39:15.275 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:15.271+0000 7f49d8dea640 1 -- 172.21.15.89:0/370027105 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f49d41094a0 msgr2=0x7f49d419cb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:15.275 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:15.271+0000 7f49d8dea640 1 --2- 172.21.15.89:0/370027105 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f49d41094a0 0x7f49d419cb30 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f49bc00d750 tx=0x7f49bc00dc10 comp rx=0 tx=0).stop 2024-09-21T03:39:15.275 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:15.271+0000 7f49d8dea640 1 -- 172.21.15.89:0/370027105 shutdown_connections 2024-09-21T03:39:15.276 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:15.271+0000 7f49d8dea640 1 --2- 172.21.15.89:0/370027105 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f49c00b6650 0x7f49c00b8b60 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:15.276 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:15.271+0000 7f49d8dea640 1 --2- 172.21.15.89:0/370027105 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f49d4109df0 0x7f49d41a1dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:15.276 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:15.271+0000 7f49d8dea640 1 --2- 172.21.15.89:0/370027105 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f49d41094a0 0x7f49d419cb30 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:15.276 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:15.271+0000 7f49d8dea640 1 --2- 172.21.15.89:0/370027105 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f49d4103b60 0x7f49d419c590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:15.276 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:15.271+0000 7f49d8dea640 1 -- 172.21.15.89:0/370027105 >> 172.21.15.89:0/370027105 conn(0x7f49d407b850 msgr2=0x7f49d4107310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:39:15.276 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:15.271+0000 7f49d8dea640 1 -- 172.21.15.89:0/370027105 shutdown_connections 2024-09-21T03:39:15.276 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:15.272+0000 7f49d8dea640 1 -- 172.21.15.89:0/370027105 wait complete. 2024-09-21T03:39:15.461 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:39:15 smithi114 ceph-mon[34140]: pgmap v355: 129 pgs: 129 active+clean; 646 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-21T03:39:15.461 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:39:15 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:39:15.461 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:39:15 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:39:15.461 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:39:15 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:39:15.500 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:39:15 smithi110 ceph-mon[34000]: pgmap v355: 129 pgs: 129 active+clean; 646 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-21T03:39:15.500 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:39:15 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:39:15.500 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:39:15 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:39:15.500 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:39:15 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:39:15.902 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 25, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-09-21T03:37:13.570694+0000', 'modified': '2024-09-21T03:37:50.719106+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 24638, 'mds_1': 24631, 'mds_2': 24614}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24593': {'gid': 24593, 'name': 'e', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.110:6835/671820654', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.110:6834', 'nonce': 671820654}, {'type': 'v1', 'addr': '172.21.15.110:6835', 'nonce': 671820654}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24604': {'gid': 24604, 'name': 'f', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.114:6835/1049714548', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.114:6834', 'nonce': 1049714548}, {'type': 'v1', 'addr': '172.21.15.114:6835', 'nonce': 1049714548}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24610': {'gid': 24610, 'name': 'c', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.114:6837/1941213818', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.114:6836', 'nonce': 1941213818}, {'type': 'v1', 'addr': '172.21.15.114:6837', 'nonce': 1941213818}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24614': {'gid': 24614, 'name': 'h', 'rank': 2, 'incarnation': 22, 'state': 'up:active', 'state_seq': 15, 'addr': '172.21.15.110:6837/4047124589', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.110:6836', 'nonce': 4047124589}, {'type': 'v1', 'addr': '172.21.15.110:6837', 'nonce': 4047124589}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24631': {'gid': 24631, 'name': 'i', 'rank': 1, 'incarnation': 20, 'state': 'up:active', 'state_seq': 15, 'addr': '172.21.15.114:6839/3427369944', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.114:6838', 'nonce': 3427369944}, {'type': 'v1', 'addr': '172.21.15.114:6839', 'nonce': 3427369944}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24638': {'gid': 24638, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.110:6839/132366709', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.110:6838', 'nonce': 132366709}, {'type': 'v1', 'addr': '172.21.15.110:6839', 'nonce': 132366709}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24638, 'qdb_cluster': [24614, 24631, 24638]} 2024-09-21T03:39:15.902 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2024-09-21T03:39:15.902 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 7c374098-77c9-11ef-baf5-efdc52797490 -- ceph tell mds.h status 2024-09-21T03:39:16.151 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/7c374098-77c9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:39:16.241 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:39:16 smithi089 ceph-mon[30928]: from='client.? 172.21.15.89:0/370027105' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T03:39:16.461 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:39:16 smithi114 ceph-mon[34140]: from='client.? 172.21.15.89:0/370027105' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T03:39:16.500 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:39:16 smithi110 ceph-mon[34000]: from='client.? 172.21.15.89:0/370027105' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T03:39:16.611 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:16.606+0000 7f20d46bc640 1 -- 172.21.15.89:0/2034888353 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f20cc103b60 msgr2=0x7f20cc103f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:16.611 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:16.606+0000 7f20d46bc640 1 --2- 172.21.15.89:0/2034888353 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f20cc103b60 0x7f20cc103f70 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f20c8009900 tx=0x7f20c802a410 comp rx=0 tx=0).stop 2024-09-21T03:39:16.611 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:16.607+0000 7f20d46bc640 1 -- 172.21.15.89:0/2034888353 shutdown_connections 2024-09-21T03:39:16.611 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:16.607+0000 7f20d46bc640 1 --2- 172.21.15.89:0/2034888353 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f20cc109df0 0x7f20cc110620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:16.611 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:16.607+0000 7f20d46bc640 1 --2- 172.21.15.89:0/2034888353 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f20cc1094a0 0x7f20cc1098b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:16.611 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:16.607+0000 7f20d46bc640 1 --2- 172.21.15.89:0/2034888353 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f20cc103b60 0x7f20cc103f70 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:16.611 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:16.607+0000 7f20d46bc640 1 -- 172.21.15.89:0/2034888353 >> 172.21.15.89:0/2034888353 conn(0x7f20cc07b850 msgr2=0x7f20cc07bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:39:16.612 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:16.607+0000 7f20d46bc640 1 -- 172.21.15.89:0/2034888353 shutdown_connections 2024-09-21T03:39:16.612 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:16.607+0000 7f20d46bc640 1 -- 172.21.15.89:0/2034888353 wait complete. 2024-09-21T03:39:16.612 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:16.607+0000 7f20d46bc640 1 Processor -- start 2024-09-21T03:39:16.612 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:16.607+0000 7f20d46bc640 1 -- start start 2024-09-21T03:39:16.612 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:16.608+0000 7f20d46bc640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f20cc103b60 0x7f20cc19a450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:16.612 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:16.608+0000 7f20d46bc640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f20cc1094a0 0x7f20cc19a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:16.612 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:16.608+0000 7f20d46bc640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f20cc109df0 0x7f20cc19fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:16.612 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:16.608+0000 7f20d46bc640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f20cc113960 con 0x7f20cc1094a0 2024-09-21T03:39:16.612 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:16.608+0000 7f20d46bc640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f20cc1137e0 con 0x7f20cc109df0 2024-09-21T03:39:16.612 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:16.608+0000 7f20d46bc640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f20cc113ae0 con 0x7f20cc103b60 2024-09-21T03:39:16.612 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:16.608+0000 7f20d1c57640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f20cc1094a0 0x7f20cc19a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:16.612 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:16.608+0000 7f20d1c57640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f20cc1094a0 0x7f20cc19a9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:36908/0 (socket says 172.21.15.89:36908) 2024-09-21T03:39:16.612 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:16.608+0000 7f20d1c57640 1 -- 172.21.15.89:0/189568840 learned_addr learned my addr 172.21.15.89:0/189568840 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-21T03:39:16.612 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:16.608+0000 7f20d2458640 1 --2- 172.21.15.89:0/189568840 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f20cc103b60 0x7f20cc19a450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:16.613 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:16.608+0000 7f20d2c59640 1 --2- 172.21.15.89:0/189568840 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f20cc109df0 0x7f20cc19fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:16.613 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:16.608+0000 7f20d1c57640 1 -- 172.21.15.89:0/189568840 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f20cc103b60 msgr2=0x7f20cc19a450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:16.613 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:16.608+0000 7f20d1c57640 1 --2- 172.21.15.89:0/189568840 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f20cc103b60 0x7f20cc19a450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:16.613 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:16.608+0000 7f20d1c57640 1 -- 172.21.15.89:0/189568840 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f20cc109df0 msgr2=0x7f20cc19fc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:16.613 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:16.608+0000 7f20d1c57640 1 --2- 172.21.15.89:0/189568840 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f20cc109df0 0x7f20cc19fc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:16.613 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:16.608+0000 7f20d1c57640 1 -- 172.21.15.89:0/189568840 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20cc1a03c0 con 0x7f20cc1094a0 2024-09-21T03:39:16.613 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:16.608+0000 7f20d2c59640 1 --2- 172.21.15.89:0/189568840 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f20cc109df0 0x7f20cc19fc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:39:16.613 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:16.608+0000 7f20d1c57640 1 --2- 172.21.15.89:0/189568840 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f20cc1094a0 0x7f20cc19a9f0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f20bc00c970 tx=0x7f20bc00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:39:16.613 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:16.608+0000 7f20c37fe640 1 -- 172.21.15.89:0/189568840 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20bc007bb0 con 0x7f20cc1094a0 2024-09-21T03:39:16.613 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:16.609+0000 7f20d46bc640 1 -- 172.21.15.89:0/189568840 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f20cc1a06b0 con 0x7f20cc1094a0 2024-09-21T03:39:16.613 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:16.609+0000 7f20c37fe640 1 -- 172.21.15.89:0/189568840 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f20bc007d50 con 0x7f20cc1094a0 2024-09-21T03:39:16.613 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:16.609+0000 7f20c37fe640 1 -- 172.21.15.89:0/189568840 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20bc004d40 con 0x7f20cc1094a0 2024-09-21T03:39:16.613 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:16.609+0000 7f20d46bc640 1 -- 172.21.15.89:0/189568840 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20cc1a6e10 con 0x7f20cc1094a0 2024-09-21T03:39:16.615 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:16.611+0000 7f20dac30740 1 Processor -- start 2024-09-21T03:39:16.615 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:16.611+0000 7f20dac30740 1 -- start start 2024-09-21T03:39:16.615 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:16.611+0000 7f20c37fe640 1 -- 172.21.15.89:0/189568840 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 18) ==== 154563+0+0 (secure 0 0 0) 0x7f20bc00f450 con 0x7f20cc1094a0 2024-09-21T03:39:16.615 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:16.611+0000 7f20dac30740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55fde1a33f60 0x55fde1dd9f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:16.615 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:16.611+0000 7f20dac30740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fde1a0b0a0 0x55fde1dda440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:16.616 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:16.612+0000 7f20dac30740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55fde1a13280 0x55fde1dda980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:16.616 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:16.612+0000 7f20dac30740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55fde1d51c10 con 0x55fde1a13280 2024-09-21T03:39:16.616 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:16.612+0000 7f20dac30740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55fde1d41040 con 0x55fde1a33f60 2024-09-21T03:39:16.616 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:16.612+0000 7f20dac30740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55fde1d44e50 con 0x55fde1a0b0a0 2024-09-21T03:39:16.616 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:16.612+0000 7f20d1c57640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55fde1a13280 0x55fde1dda980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:16.616 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:16.612+0000 7f20d1c57640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55fde1a13280 0x55fde1dda980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:36918/0 (socket says 172.21.15.89:36918) 2024-09-21T03:39:16.616 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:16.612+0000 7f20d1c57640 1 -- 172.21.15.89:0/1302990277 learned_addr learned my addr 172.21.15.89:0/1302990277 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-21T03:39:16.616 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:16.612+0000 7f20d2458640 1 --2- 172.21.15.89:0/1302990277 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fde1a0b0a0 0x55fde1dda440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:16.616 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:16.612+0000 7f20d2c59640 1 --2- 172.21.15.89:0/1302990277 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55fde1a33f60 0x55fde1dd9f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:16.616 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:16.612+0000 7f20d1c57640 1 -- 172.21.15.89:0/1302990277 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fde1a0b0a0 msgr2=0x55fde1dda440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:39:16.616 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:16.612+0000 7f20d1c57640 1 --2- 172.21.15.89:0/1302990277 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fde1a0b0a0 0x55fde1dda440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:16.616 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:16.612+0000 7f20d1c57640 1 -- 172.21.15.89:0/1302990277 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55fde1a33f60 msgr2=0x55fde1dd9f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:39:16.616 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:16.612+0000 7f20d1c57640 1 --2- 172.21.15.89:0/1302990277 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55fde1a33f60 0x55fde1dd9f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:16.616 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:16.612+0000 7f20d1c57640 1 -- 172.21.15.89:0/1302990277 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fde1a9f540 con 0x55fde1a13280 2024-09-21T03:39:16.617 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:16.612+0000 7f20d1c57640 1 --2- 172.21.15.89:0/1302990277 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55fde1a13280 0x55fde1dda980 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f20bc0edbf0 tx=0x7f20bc0ef670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=285a358a5379c494 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:39:16.617 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:16.612+0000 7f20c37fe640 1 --2- 172.21.15.89:0/189568840 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f20b80b65a0 0x7f20b80b8ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:16.617 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:16.612+0000 7f20c0ff9640 1 -- 172.21.15.89:0/1302990277 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20bc0f6070 con 0x55fde1a13280 2024-09-21T03:39:16.617 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:16.613+0000 7f20d2c59640 1 --2- 172.21.15.89:0/189568840 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f20b80b65a0 0x7f20b80b8ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:16.617 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:16.613+0000 7f20c0ff9640 1 -- 172.21.15.89:0/1302990277 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f20bc0acd30 con 0x55fde1a13280 2024-09-21T03:39:16.617 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:16.613+0000 7f20c0ff9640 1 -- 172.21.15.89:0/1302990277 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20bc0acff0 con 0x55fde1a13280 2024-09-21T03:39:16.617 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:16.613+0000 7f20c37fe640 1 -- 172.21.15.89:0/189568840 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 9121+0+0 (secure 0 0 0) 0x7f20bc0e7570 con 0x7f20cc1094a0 2024-09-21T03:39:16.621 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:16.614+0000 7f20dac30740 1 -- 172.21.15.89:0/1302990277 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55fde1a13280 msgr2=0x55fde1dda980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:16.621 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:16.614+0000 7f20d2c59640 1 --2- 172.21.15.89:0/189568840 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f20b80b65a0 0x7f20b80b8ab0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f20c4005fd0 tx=0x7f20c40095a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:39:16.621 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:16.614+0000 7f20dac30740 1 --2- 172.21.15.89:0/1302990277 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55fde1a13280 0x55fde1dda980 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f20bc0edbf0 tx=0x7f20bc0ef670 comp rx=0 tx=0).stop 2024-09-21T03:39:16.621 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:16.614+0000 7f20dac30740 1 -- 172.21.15.89:0/1302990277 shutdown_connections 2024-09-21T03:39:16.621 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:16.614+0000 7f20dac30740 1 --2- 172.21.15.89:0/1302990277 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55fde1a13280 0x55fde1dda980 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:16.621 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:16.614+0000 7f20dac30740 1 --2- 172.21.15.89:0/1302990277 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fde1a0b0a0 0x55fde1dda440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:16.621 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:16.614+0000 7f20dac30740 1 --2- 172.21.15.89:0/1302990277 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55fde1a33f60 0x55fde1dd9f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:16.621 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:16.614+0000 7f20dac30740 1 -- 172.21.15.89:0/1302990277 >> 172.21.15.89:0/1302990277 conn(0x55fde1a28350 msgr2=0x55fde1dd7df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:39:16.622 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:16.614+0000 7f20dac30740 1 -- 172.21.15.89:0/1302990277 shutdown_connections 2024-09-21T03:39:16.622 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:16.614+0000 7f20dac30740 1 -- 172.21.15.89:0/1302990277 wait complete. 2024-09-21T03:39:16.622 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:16.614+0000 7f20dac30740 1 -- start start 2024-09-21T03:39:16.622 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:16.614+0000 7f20dac30740 1 Processor -- start 2024-09-21T03:39:16.622 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:16.615+0000 7f20dac30740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55fde1a0b0a0 0x55fde1de9af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:16.622 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:16.615+0000 7f20dac30740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55fde1a13280 0x55fde1dea050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:16.622 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:16.615+0000 7f20dac30740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fde1a33f60 0x55fde1df5d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:16.622 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:16.615+0000 7f20dac30740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55fde1ba25f0 con 0x55fde1a0b0a0 2024-09-21T03:39:16.622 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:16.615+0000 7f20dac30740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55fde1cdc010 con 0x55fde1a13280 2024-09-21T03:39:16.622 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:16.615+0000 7f20dac30740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55fde1deca10 con 0x55fde1a33f60 2024-09-21T03:39:16.622 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:16.615+0000 7f20dac30740 10 client.0 _ll_register_callbacks cb 0x55fde1d3b340 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:39:16.623 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:16.615+0000 7f20d2458640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55fde1a0b0a0 0x55fde1de9af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:16.623 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:16.615+0000 7f20d2458640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55fde1a0b0a0 0x55fde1de9af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:36930/0 (socket says 172.21.15.89:36930) 2024-09-21T03:39:16.623 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:16.615+0000 7f20d2458640 1 -- 172.21.15.89:0/1102791636 learned_addr learned my addr 172.21.15.89:0/1102791636 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-21T03:39:16.623 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:16.615+0000 7f20d1c57640 1 --2- 172.21.15.89:0/1102791636 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55fde1a13280 0x55fde1dea050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:16.623 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:16.615+0000 7f20d2c59640 1 --2- 172.21.15.89:0/1102791636 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fde1a33f60 0x55fde1df5d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:16.623 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:16.615+0000 7f20d2458640 1 -- 172.21.15.89:0/1102791636 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fde1a33f60 msgr2=0x55fde1df5d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:39:16.623 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:16.615+0000 7f20d2458640 1 --2- 172.21.15.89:0/1102791636 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fde1a33f60 0x55fde1df5d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:16.623 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:16.615+0000 7f20d2458640 1 -- 172.21.15.89:0/1102791636 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55fde1a13280 msgr2=0x55fde1dea050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:39:16.623 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:16.615+0000 7f20d2458640 1 --2- 172.21.15.89:0/1102791636 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55fde1a13280 0x55fde1dea050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:16.623 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:16.615+0000 7f20d2458640 1 -- 172.21.15.89:0/1102791636 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fde1d09a00 con 0x55fde1a0b0a0 2024-09-21T03:39:16.623 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:16.615+0000 7f20d2c59640 1 --2- 172.21.15.89:0/1102791636 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fde1a33f60 0x55fde1df5d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:39:16.623 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:16.615+0000 7f20d2458640 1 --2- 172.21.15.89:0/1102791636 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55fde1a0b0a0 0x55fde1de9af0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f20c802a930 tx=0x7f20c8031400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a4ffb22e8a2d3f1 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:39:16.623 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:16.615+0000 7f20b5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2024-09-21T03:39:16.623 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:16.615+0000 7f20b5ffb640 1 -- 172.21.15.89:0/1102791636 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20c8035040 con 0x55fde1a0b0a0 2024-09-21T03:39:16.624 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:16.615+0000 7f20b5ffb640 1 -- 172.21.15.89:0/1102791636 <== mon.0 v2:172.21.15.89:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 9121+0+0 (secure 0 0 0) 0x7f20c8004240 con 0x55fde1a0b0a0 2024-09-21T03:39:16.624 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:16.615+0000 7f20dac30740 1 -- 172.21.15.89:0/1102791636 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f20bc0acd30 con 0x55fde1a0b0a0 2024-09-21T03:39:16.624 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:16.616+0000 7f20b5ffb640 1 -- 172.21.15.89:0/1102791636 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f20c804f050 con 0x55fde1a0b0a0 2024-09-21T03:39:16.624 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:16.616+0000 7f20dac30740 10 client.15369 fetch_fsmap learned FSMap version 25 2024-09-21T03:39:16.624 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:16.616+0000 7f20dac30740 1 -- 172.21.15.89:0/1102791636 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55fde1df88c0 con 0x55fde1a0b0a0 2024-09-21T03:39:16.624 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:16.616+0000 7f20b5ffb640 1 -- 172.21.15.89:0/1102791636 <== mon.0 v2:172.21.15.89:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f20bc0acd30 con 0x55fde1a0b0a0 2024-09-21T03:39:16.625 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:16.616+0000 7f20b5ffb640 1 -- 172.21.15.89:0/1102791636 <== mon.0 v2:172.21.15.89:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f20c8044020 con 0x55fde1a0b0a0 2024-09-21T03:39:16.625 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:16.616+0000 7f20dac30740 10 client.15369 fetch_fsmap finished waiting for FSMap version 25 2024-09-21T03:39:16.625 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:16.616+0000 7f20dac30740 10 client.15369 resolve_mds: resolved name 'h' to daemon mds.h 2024-09-21T03:39:16.625 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:16.616+0000 7f20dac30740 20 client.15369 populate_metadata read hostname 'smithi089' 2024-09-21T03:39:16.626 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:16.616+0000 7f20dac30740 1 --2- 172.21.15.89:0/1102791636 >> [v2:172.21.15.110:6836/4047124589,v1:172.21.15.110:6837/4047124589] conn(0x55fde19f28e0 0x55fde1dfc390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:16.626 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:16.616+0000 7f20dac30740 4 client.15369 mds_command: new command op to 24614 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:39:16.626 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:16.616+0000 7f20dac30740 1 -- 172.21.15.89:0/1102791636 --> [v2:172.21.15.110:6836/4047124589,v1:172.21.15.110:6837/4047124589] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fde1d53b60 con 0x55fde19f28e0 2024-09-21T03:39:16.626 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:16.616+0000 7f20d1c57640 1 --2- 172.21.15.89:0/1102791636 >> [v2:172.21.15.110:6836/4047124589,v1:172.21.15.110:6837/4047124589] conn(0x55fde19f28e0 0x55fde1dfc390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:16.626 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:16.617+0000 7f20b5ffb640 10 client.15369 ms_handle_connect on v2:172.21.15.110:6836/4047124589 2024-09-21T03:39:16.626 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:16.617+0000 7f20d1c57640 1 --2- 172.21.15.89:0/1102791636 >> [v2:172.21.15.110:6836/4047124589,v1:172.21.15.110:6837/4047124589] conn(0x55fde19f28e0 0x55fde1dfc390 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=a81ed368df1a4f58 server_cookie=d0d7799c218761cd in_seq=0 out_seq=0 2024-09-21T03:39:16.626 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:16.618+0000 7f20b5ffb640 1 -- 172.21.15.89:0/1102791636 <== mds.2 v2:172.21.15.110:6836/4047124589 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55fde1d53b60 con 0x55fde19f28e0 2024-09-21T03:39:16.626 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:16.618+0000 7f20b5ffb640 10 client.15369 handle_command_reply: tid=0 2024-09-21T03:39:16.626 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:16.618+0000 7f20dac30740 1 client.15369 shutdown 2024-09-21T03:39:16.626 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:16.618+0000 7f20dac30740 1 -- 172.21.15.89:0/1102791636 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55fde1a0b0a0 msgr2=0x55fde1de9af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:16.626 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:16.618+0000 7f20dac30740 1 --2- 172.21.15.89:0/1102791636 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55fde1a0b0a0 0x55fde1de9af0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f20c802a930 tx=0x7f20c8031400 comp rx=0 tx=0).stop 2024-09-21T03:39:16.626 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:16.618+0000 7f20dac30740 1 -- 172.21.15.89:0/1102791636 shutdown_connections 2024-09-21T03:39:16.626 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:16.618+0000 7f20dac30740 1 --2- 172.21.15.89:0/1102791636 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55fde1a13280 0x55fde1dea050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:16.626 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:16.618+0000 7f20dac30740 1 --2- 172.21.15.89:0/1102791636 >> [v2:172.21.15.110:6836/4047124589,v1:172.21.15.110:6837/4047124589] conn(0x55fde19f28e0 0x55fde1dfc390 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:16.627 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:16.618+0000 7f20dac30740 1 --2- 172.21.15.89:0/1102791636 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fde1a33f60 0x55fde1df5d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:16.627 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:16.618+0000 7f20b5ffb640 0 client.15369 ms_handle_reset on v2:172.21.15.110:6836/4047124589 2024-09-21T03:39:16.627 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:16.618+0000 7f20dac30740 1 --2- 172.21.15.89:0/1102791636 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55fde1a0b0a0 0x55fde1de9af0 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:16.627 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:16.618+0000 7f20dac30740 1 -- 172.21.15.89:0/1102791636 >> 172.21.15.89:0/1102791636 conn(0x55fde1a28350 msgr2=0x55fde1ddd050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:39:16.627 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:16.618+0000 7f20dac30740 1 -- 172.21.15.89:0/1102791636 shutdown_connections 2024-09-21T03:39:16.627 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:16.618+0000 7f20dac30740 1 -- 172.21.15.89:0/1102791636 wait complete. 2024-09-21T03:39:16.627 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:16.618+0000 7f20dac30740 20 client.15369 trim_cache size 0 max 16384 2024-09-21T03:39:16.639 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:16.636+0000 7f20dac30740 1 Processor -- start 2024-09-21T03:39:16.639 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:16.636+0000 7f20dac30740 1 -- start start 2024-09-21T03:39:16.639 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:16.636+0000 7f20dac30740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55fde1a0b0a0 0x55fde1de4ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:16.639 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:16.636+0000 7f20dac30740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55fde1a13280 0x55fde1de51e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:16.639 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:16.636+0000 7f20dac30740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fde1a33f60 0x55fde1de8370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:16.639 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:16.636+0000 7f20dac30740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55fde1dfa6d0 con 0x55fde1a13280 2024-09-21T03:39:16.639 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:16.636+0000 7f20dac30740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55fde1dfada0 con 0x55fde1a0b0a0 2024-09-21T03:39:16.639 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:16.636+0000 7f20dac30740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55fde1dfb470 con 0x55fde1a33f60 2024-09-21T03:39:16.639 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:16.636+0000 7f20d1c57640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55fde1a13280 0x55fde1de51e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:16.640 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:16.636+0000 7f20d1c57640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55fde1a13280 0x55fde1de51e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:36932/0 (socket says 172.21.15.89:36932) 2024-09-21T03:39:16.640 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:16.636+0000 7f20d1c57640 1 -- 172.21.15.89:0/3555677149 learned_addr learned my addr 172.21.15.89:0/3555677149 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-21T03:39:16.640 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:16.636+0000 7f20d2c59640 1 --2- 172.21.15.89:0/3555677149 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fde1a33f60 0x55fde1de8370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:16.640 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:16.636+0000 7f20d2458640 1 --2- 172.21.15.89:0/3555677149 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55fde1a0b0a0 0x55fde1de4ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:16.640 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:16.636+0000 7f20d1c57640 1 -- 172.21.15.89:0/3555677149 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fde1a33f60 msgr2=0x55fde1de8370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:39:16.640 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:16.636+0000 7f20d1c57640 1 --2- 172.21.15.89:0/3555677149 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fde1a33f60 0x55fde1de8370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:16.640 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:16.636+0000 7f20d1c57640 1 -- 172.21.15.89:0/3555677149 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55fde1a0b0a0 msgr2=0x55fde1de4ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:39:16.640 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:16.636+0000 7f20d1c57640 1 --2- 172.21.15.89:0/3555677149 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55fde1a0b0a0 0x55fde1de4ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:16.640 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:16.636+0000 7f20d1c57640 1 -- 172.21.15.89:0/3555677149 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fde1ddace0 con 0x55fde1a13280 2024-09-21T03:39:16.640 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:16.636+0000 7f20d2458640 1 --2- 172.21.15.89:0/3555677149 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55fde1a0b0a0 0x55fde1de4ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:39:16.640 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:16.637+0000 7f20d1c57640 1 --2- 172.21.15.89:0/3555677149 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55fde1a13280 0x55fde1de51e0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f20bc003bb0 tx=0x7f20bc000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7a50d090b65f4c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:39:16.640 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:16.637+0000 7f20b7fff640 1 -- 172.21.15.89:0/3555677149 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20bc003800 con 0x55fde1a13280 2024-09-21T03:39:16.640 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:16.637+0000 7f20b7fff640 1 -- 172.21.15.89:0/3555677149 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f20bc0ed300 con 0x55fde1a13280 2024-09-21T03:39:16.640 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:16.637+0000 7f20b7fff640 1 -- 172.21.15.89:0/3555677149 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20bc0039f0 con 0x55fde1a13280 2024-09-21T03:39:16.643 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:16.638+0000 7f20dac30740 1 -- 172.21.15.89:0/3555677149 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55fde1a13280 msgr2=0x55fde1de51e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:16.643 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:16.638+0000 7f20dac30740 1 --2- 172.21.15.89:0/3555677149 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55fde1a13280 0x55fde1de51e0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f20bc003bb0 tx=0x7f20bc000bc0 comp rx=0 tx=0).stop 2024-09-21T03:39:16.643 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:16.638+0000 7f20dac30740 1 -- 172.21.15.89:0/3555677149 shutdown_connections 2024-09-21T03:39:16.643 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:16.638+0000 7f20dac30740 1 --2- 172.21.15.89:0/3555677149 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fde1a33f60 0x55fde1de8370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:16.644 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:16.638+0000 7f20dac30740 1 --2- 172.21.15.89:0/3555677149 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55fde1a13280 0x55fde1de51e0 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:16.644 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:16.638+0000 7f20dac30740 1 --2- 172.21.15.89:0/3555677149 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55fde1a0b0a0 0x55fde1de4ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:16.644 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:16.638+0000 7f20dac30740 1 -- 172.21.15.89:0/3555677149 >> 172.21.15.89:0/3555677149 conn(0x55fde1a28350 msgr2=0x55fde1df9d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:39:16.644 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:16.638+0000 7f20dac30740 1 -- 172.21.15.89:0/3555677149 shutdown_connections 2024-09-21T03:39:16.644 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:16.638+0000 7f20dac30740 1 -- 172.21.15.89:0/3555677149 wait complete. 2024-09-21T03:39:16.644 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:16.638+0000 7f20dac30740 1 -- start start 2024-09-21T03:39:16.644 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:16.639+0000 7f20dac30740 1 Processor -- start 2024-09-21T03:39:16.644 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:16.639+0000 7f20dac30740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fde1a0b0a0 0x55fde1dfeb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:16.644 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:16.639+0000 7f20dac30740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55fde1a13280 0x55fde1dff0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:16.644 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:16.639+0000 7f20dac30740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55fde1a33f60 0x55fde1dff5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:16.644 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:16.639+0000 7f20dac30740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55fde1dfc210 con 0x55fde1a13280 2024-09-21T03:39:16.644 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:16.639+0000 7f20dac30740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55fde1dfbb40 con 0x55fde1a33f60 2024-09-21T03:39:16.644 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:16.639+0000 7f20dac30740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55fde1dfc8e0 con 0x55fde1a0b0a0 2024-09-21T03:39:16.645 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:16.639+0000 7f20dac30740 10 client.0 _ll_register_callbacks cb 0x7f20c8031a70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:39:16.645 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:16.639+0000 7f20d1c57640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55fde1a13280 0x55fde1dff0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:16.645 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:16.639+0000 7f20d1c57640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55fde1a13280 0x55fde1dff0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:36946/0 (socket says 172.21.15.89:36946) 2024-09-21T03:39:16.645 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:16.639+0000 7f20d1c57640 1 -- 172.21.15.89:0/737803803 learned_addr learned my addr 172.21.15.89:0/737803803 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-21T03:39:16.645 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:16.639+0000 7f20d2c59640 1 --2- 172.21.15.89:0/737803803 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55fde1a33f60 0x55fde1dff5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:16.645 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:16.639+0000 7f20d2458640 1 --2- 172.21.15.89:0/737803803 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fde1a0b0a0 0x55fde1dfeb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:16.645 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:16.639+0000 7f20d1c57640 1 -- 172.21.15.89:0/737803803 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fde1a0b0a0 msgr2=0x55fde1dfeb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:39:16.645 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:16.639+0000 7f20d1c57640 1 --2- 172.21.15.89:0/737803803 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fde1a0b0a0 0x55fde1dfeb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:16.645 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:16.639+0000 7f20d1c57640 1 -- 172.21.15.89:0/737803803 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55fde1a33f60 msgr2=0x55fde1dff5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:39:16.645 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:16.639+0000 7f20d1c57640 1 --2- 172.21.15.89:0/737803803 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55fde1a33f60 0x55fde1dff5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:16.645 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:16.639+0000 7f20d1c57640 1 -- 172.21.15.89:0/737803803 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fde1e03cc0 con 0x55fde1a13280 2024-09-21T03:39:16.645 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:16.640+0000 7f20d1c57640 1 --2- 172.21.15.89:0/737803803 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55fde1a13280 0x55fde1dff0b0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f20bc007770 tx=0x7f20bc0edf70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44aab323b3043fcb server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:39:16.645 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:16.640+0000 7f20b5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2024-09-21T03:39:16.645 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:16.640+0000 7f20b5ffb640 1 -- 172.21.15.89:0/737803803 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20bc0f6070 con 0x55fde1a13280 2024-09-21T03:39:16.645 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:16.640+0000 7f20b5ffb640 1 -- 172.21.15.89:0/737803803 <== mon.0 v2:172.21.15.89:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 9121+0+0 (secure 0 0 0) 0x7f20bc0e72b0 con 0x55fde1a13280 2024-09-21T03:39:16.646 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:16.640+0000 7f20dac30740 1 -- 172.21.15.89:0/737803803 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f20bc0ed300 con 0x55fde1a13280 2024-09-21T03:39:16.646 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:16.640+0000 7f20b5ffb640 1 -- 172.21.15.89:0/737803803 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f20bc003be0 con 0x55fde1a13280 2024-09-21T03:39:16.646 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:16.640+0000 7f20dac30740 10 client.15375 fetch_fsmap learned FSMap version 25 2024-09-21T03:39:16.646 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:16.640+0000 7f20dac30740 1 -- 172.21.15.89:0/737803803 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55fde1e04eb0 con 0x55fde1a13280 2024-09-21T03:39:16.646 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:16.640+0000 7f20b5ffb640 1 -- 172.21.15.89:0/737803803 <== mon.0 v2:172.21.15.89:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f20bc0ed300 con 0x55fde1a13280 2024-09-21T03:39:16.646 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:16.640+0000 7f20b5ffb640 1 -- 172.21.15.89:0/737803803 <== mon.0 v2:172.21.15.89:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f20bc0a9490 con 0x55fde1a13280 2024-09-21T03:39:16.646 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:16.640+0000 7f20dac30740 10 client.15375 fetch_fsmap finished waiting for FSMap version 25 2024-09-21T03:39:16.646 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:16.641+0000 7f20dac30740 10 client.15375 resolve_mds: resolved name 'h' to daemon mds.h 2024-09-21T03:39:16.646 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:16.641+0000 7f20dac30740 20 client.15375 populate_metadata read hostname 'smithi089' 2024-09-21T03:39:16.646 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:16.641+0000 7f20dac30740 1 --2- 172.21.15.89:0/737803803 >> [v2:172.21.15.110:6836/4047124589,v1:172.21.15.110:6837/4047124589] conn(0x55fde19c7c90 0x55fde1e6a370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:16.646 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:16.641+0000 7f20dac30740 4 client.15375 mds_command: new command op to 24614 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-21T03:39:16.646 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:16.641+0000 7f20dac30740 1 -- 172.21.15.89:0/737803803 --> [v2:172.21.15.110:6836/4047124589,v1:172.21.15.110:6837/4047124589] -- command(tid 0: {"prefix": "status"}) -- 0x55fde1d4f610 con 0x55fde19c7c90 2024-09-21T03:39:16.646 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:16.641+0000 7f20d2458640 1 --2- 172.21.15.89:0/737803803 >> [v2:172.21.15.110:6836/4047124589,v1:172.21.15.110:6837/4047124589] conn(0x55fde19c7c90 0x55fde1e6a370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:16.646 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:16.641+0000 7f20b5ffb640 10 client.15375 ms_handle_connect on v2:172.21.15.110:6836/4047124589 2024-09-21T03:39:16.646 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:16.641+0000 7f20d2458640 1 --2- 172.21.15.89:0/737803803 >> [v2:172.21.15.110:6836/4047124589,v1:172.21.15.110:6837/4047124589] conn(0x55fde19c7c90 0x55fde1e6a370 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=93bb3152fe5c1e12 server_cookie=3c428ec4fa4cd50e in_seq=0 out_seq=0 2024-09-21T03:39:16.648 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:16.641+0000 7f20b5ffb640 1 -- 172.21.15.89:0/737803803 <== mds.2 v2:172.21.15.110:6836/4047124589 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55fde1d4f610 con 0x55fde19c7c90 2024-09-21T03:39:16.648 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:16.641+0000 7f20b5ffb640 10 client.15375 handle_command_reply: tid=0 2024-09-21T03:39:16.648 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:16.641+0000 7f20dac30740 1 client.15375 shutdown 2024-09-21T03:39:16.648 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:16.642+0000 7f20dac30740 1 -- 172.21.15.89:0/737803803 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55fde1a13280 msgr2=0x55fde1dff0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:16.648 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:16.642+0000 7f20dac30740 1 --2- 172.21.15.89:0/737803803 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55fde1a13280 0x55fde1dff0b0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f20bc007770 tx=0x7f20bc0edf70 comp rx=0 tx=0).stop 2024-09-21T03:39:16.648 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:16.642+0000 7f20dac30740 1 -- 172.21.15.89:0/737803803 shutdown_connections 2024-09-21T03:39:16.648 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:16.642+0000 7f20dac30740 1 --2- 172.21.15.89:0/737803803 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55fde1a33f60 0x55fde1dff5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:16.648 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:16.642+0000 7f20dac30740 1 --2- 172.21.15.89:0/737803803 >> [v2:172.21.15.110:6836/4047124589,v1:172.21.15.110:6837/4047124589] conn(0x55fde19c7c90 0x55fde1e6a370 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:16.648 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:16.642+0000 7f20dac30740 1 --2- 172.21.15.89:0/737803803 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55fde1a13280 0x55fde1dff0b0 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:16.648 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:16.642+0000 7f20dac30740 1 --2- 172.21.15.89:0/737803803 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fde1a0b0a0 0x55fde1dfeb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:16.648 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:16.642+0000 7f20dac30740 1 -- 172.21.15.89:0/737803803 >> 172.21.15.89:0/737803803 conn(0x55fde1a28350 msgr2=0x55fde1de4ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:39:16.649 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:16.642+0000 7f20b5ffb640 0 client.15375 ms_handle_reset on v2:172.21.15.110:6836/4047124589 2024-09-21T03:39:16.649 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:16.642+0000 7f20dac30740 1 -- 172.21.15.89:0/737803803 shutdown_connections 2024-09-21T03:39:16.649 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:16.642+0000 7f20dac30740 1 -- 172.21.15.89:0/737803803 wait complete. 2024-09-21T03:39:16.649 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:16.642+0000 7f20dac30740 20 client.15375 trim_cache size 0 max 16384 2024-09-21T03:39:16.649 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-09-21T03:39:16.649 INFO:teuthology.orchestra.run.smithi089.stdout: "cluster_fsid": "7c374098-77c9-11ef-baf5-efdc52797490", 2024-09-21T03:39:16.649 INFO:teuthology.orchestra.run.smithi089.stdout: "whoami": 2, 2024-09-21T03:39:16.649 INFO:teuthology.orchestra.run.smithi089.stdout: "id": 24614, 2024-09-21T03:39:16.649 INFO:teuthology.orchestra.run.smithi089.stdout: "want_state": "up:active", 2024-09-21T03:39:16.649 INFO:teuthology.orchestra.run.smithi089.stdout: "state": "up:active", 2024-09-21T03:39:16.649 INFO:teuthology.orchestra.run.smithi089.stdout: "fs_name": "cephfs", 2024-09-21T03:39:16.650 INFO:teuthology.orchestra.run.smithi089.stdout: "rank_uptime": 88.630044284999997, 2024-09-21T03:39:16.650 INFO:teuthology.orchestra.run.smithi089.stdout: "mdsmap_epoch": 25, 2024-09-21T03:39:16.650 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap_epoch": 84, 2024-09-21T03:39:16.650 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap_epoch_barrier": 80, 2024-09-21T03:39:16.650 INFO:teuthology.orchestra.run.smithi089.stdout: "uptime": 144.62013818899999 2024-09-21T03:39:16.650 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-09-21T03:39:16.650 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:16.642+0000 7f20d46bc640 1 -- 172.21.15.89:0/189568840 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f20b80b65a0 msgr2=0x7f20b80b8ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:16.651 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:16.642+0000 7f20d46bc640 1 --2- 172.21.15.89:0/189568840 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f20b80b65a0 0x7f20b80b8ab0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f20c4005fd0 tx=0x7f20c40095a0 comp rx=0 tx=0).stop 2024-09-21T03:39:16.651 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:16.642+0000 7f20d46bc640 1 -- 172.21.15.89:0/189568840 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f20cc1094a0 msgr2=0x7f20cc19a9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:16.651 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:16.642+0000 7f20d46bc640 1 --2- 172.21.15.89:0/189568840 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f20cc1094a0 0x7f20cc19a9f0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f20bc00c970 tx=0x7f20bc00ce30 comp rx=0 tx=0).stop 2024-09-21T03:39:16.651 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:16.643+0000 7f20d46bc640 1 -- 172.21.15.89:0/189568840 shutdown_connections 2024-09-21T03:39:16.651 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:16.643+0000 7f20d46bc640 1 --2- 172.21.15.89:0/189568840 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f20b80b65a0 0x7f20b80b8ab0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:16.651 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:16.643+0000 7f20d46bc640 1 --2- 172.21.15.89:0/189568840 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f20cc109df0 0x7f20cc19fc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:16.651 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:16.643+0000 7f20d46bc640 1 --2- 172.21.15.89:0/189568840 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f20cc1094a0 0x7f20cc19a9f0 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:16.651 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:16.643+0000 7f20d46bc640 1 --2- 172.21.15.89:0/189568840 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f20cc103b60 0x7f20cc19a450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:16.651 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:16.643+0000 7f20d46bc640 1 -- 172.21.15.89:0/189568840 >> 172.21.15.89:0/189568840 conn(0x7f20cc07b850 msgr2=0x7f20cc107690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:39:16.651 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:16.643+0000 7f20d46bc640 1 -- 172.21.15.89:0/189568840 shutdown_connections 2024-09-21T03:39:16.651 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:16.643+0000 7f20d46bc640 1 -- 172.21.15.89:0/189568840 wait complete. 2024-09-21T03:39:17.311 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:39:17 smithi089 ceph-mon[30928]: pgmap v356: 129 pgs: 129 active+clean; 646 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-09-21T03:39:17.389 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 7c374098-77c9-11ef-baf5-efdc52797490 -- ceph tell mds.i status 2024-09-21T03:39:17.461 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:39:17 smithi114 ceph-mon[34140]: pgmap v356: 129 pgs: 129 active+clean; 646 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-09-21T03:39:17.500 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:39:17 smithi110 ceph-mon[34000]: pgmap v356: 129 pgs: 129 active+clean; 646 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-09-21T03:39:17.636 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/7c374098-77c9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:39:18.147 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:18.133+0000 7fe19dc6b640 1 -- 172.21.15.89:0/1240285686 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe198102b10 msgr2=0x7fe198102f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:18.147 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:18.133+0000 7fe19dc6b640 1 --2- 172.21.15.89:0/1240285686 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe198102b10 0x7fe198102f20 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fe188009900 tx=0x7fe18802a410 comp rx=0 tx=0).stop 2024-09-21T03:39:18.147 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:18.133+0000 7fe19dc6b640 1 -- 172.21.15.89:0/1240285686 shutdown_connections 2024-09-21T03:39:18.148 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:18.133+0000 7fe19dc6b640 1 --2- 172.21.15.89:0/1240285686 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe198108da0 0x7fe19810f5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:18.148 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:18.133+0000 7fe19dc6b640 1 --2- 172.21.15.89:0/1240285686 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe198108450 0x7fe198108860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:18.148 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:18.133+0000 7fe19dc6b640 1 --2- 172.21.15.89:0/1240285686 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe198102b10 0x7fe198102f20 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:18.148 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:18.133+0000 7fe19dc6b640 1 -- 172.21.15.89:0/1240285686 >> 172.21.15.89:0/1240285686 conn(0x7fe1980796d0 msgr2=0x7fe198079ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:39:18.148 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:18.133+0000 7fe19dc6b640 1 -- 172.21.15.89:0/1240285686 shutdown_connections 2024-09-21T03:39:18.148 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:18.133+0000 7fe19dc6b640 1 -- 172.21.15.89:0/1240285686 wait complete. 2024-09-21T03:39:18.148 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:18.134+0000 7fe19dc6b640 1 Processor -- start 2024-09-21T03:39:18.148 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:18.134+0000 7fe19dc6b640 1 -- start start 2024-09-21T03:39:18.148 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:18.134+0000 7fe19dc6b640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe198102b10 0x7fe1981993a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:18.148 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:18.134+0000 7fe19dc6b640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe198108450 0x7fe198199940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:18.148 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:18.134+0000 7fe19dc6b640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe198108da0 0x7fe19819ebe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:18.148 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:18.134+0000 7fe19dc6b640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fe198112910 con 0x7fe198102b10 2024-09-21T03:39:18.148 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:18.134+0000 7fe19dc6b640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fe198112790 con 0x7fe198108450 2024-09-21T03:39:18.148 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:18.134+0000 7fe19dc6b640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fe198112a90 con 0x7fe198108da0 2024-09-21T03:39:18.149 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:18.134+0000 7fe1977fe640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe198102b10 0x7fe1981993a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:18.149 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:18.135+0000 7fe1977fe640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe198102b10 0x7fe1981993a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:36970/0 (socket says 172.21.15.89:36970) 2024-09-21T03:39:18.149 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:18.135+0000 7fe1977fe640 1 -- 172.21.15.89:0/3400475748 learned_addr learned my addr 172.21.15.89:0/3400475748 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-21T03:39:18.149 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:18.135+0000 7fe18effd640 1 --2- 172.21.15.89:0/3400475748 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe198108450 0x7fe198199940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:18.149 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:18.135+0000 7fe1977fe640 1 -- 172.21.15.89:0/3400475748 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe198108da0 msgr2=0x7fe19819ebe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:18.149 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:18.135+0000 7fe197fff640 1 --2- 172.21.15.89:0/3400475748 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe198108da0 0x7fe19819ebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:18.149 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:18.135+0000 7fe1977fe640 1 --2- 172.21.15.89:0/3400475748 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe198108da0 0x7fe19819ebe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:18.149 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:18.135+0000 7fe1977fe640 1 -- 172.21.15.89:0/3400475748 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe198108450 msgr2=0x7fe198199940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:18.149 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:18.135+0000 7fe1977fe640 1 --2- 172.21.15.89:0/3400475748 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe198108450 0x7fe198199940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:18.149 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:18.135+0000 7fe1977fe640 1 -- 172.21.15.89:0/3400475748 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe19819f310 con 0x7fe198102b10 2024-09-21T03:39:18.149 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:18.135+0000 7fe18effd640 1 --2- 172.21.15.89:0/3400475748 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe198108450 0x7fe198199940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:39:18.149 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:18.135+0000 7fe1977fe640 1 --2- 172.21.15.89:0/3400475748 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe198102b10 0x7fe1981993a0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fe188009230 tx=0x7fe188004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:39:18.149 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:18.135+0000 7fe1957fa640 1 -- 172.21.15.89:0/3400475748 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe18803f030 con 0x7fe198102b10 2024-09-21T03:39:18.149 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:18.135+0000 7fe1957fa640 1 -- 172.21.15.89:0/3400475748 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe188003a80 con 0x7fe198102b10 2024-09-21T03:39:18.150 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:18.135+0000 7fe19dc6b640 1 -- 172.21.15.89:0/3400475748 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe19819f5a0 con 0x7fe198102b10 2024-09-21T03:39:18.150 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:18.135+0000 7fe1957fa640 1 -- 172.21.15.89:0/3400475748 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe188035070 con 0x7fe198102b10 2024-09-21T03:39:18.150 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:18.135+0000 7fe19dc6b640 1 -- 172.21.15.89:0/3400475748 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1981a5d60 con 0x7fe198102b10 2024-09-21T03:39:18.150 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:18.138+0000 7fe1957fa640 1 -- 172.21.15.89:0/3400475748 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 18) ==== 154563+0+0 (secure 0 0 0) 0x7fe188042030 con 0x7fe198102b10 2024-09-21T03:39:18.150 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:18.138+0000 7fe1a41df740 1 Processor -- start 2024-09-21T03:39:18.150 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:18.138+0000 7fe1a41df740 1 -- start start 2024-09-21T03:39:18.150 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:18.138+0000 7fe1a41df740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5650c23edf60 0x5650c2793f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:18.150 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:18.138+0000 7fe1a41df740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5650c23c50a0 0x5650c2794440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:18.150 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:18.138+0000 7fe1a41df740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5650c23cd280 0x5650c2794980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:18.150 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:18.138+0000 7fe1a41df740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x5650c270bc10 con 0x5650c23cd280 2024-09-21T03:39:18.150 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:18.138+0000 7fe1a41df740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x5650c26fb040 con 0x5650c23edf60 2024-09-21T03:39:18.150 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:18.138+0000 7fe1a41df740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x5650c26fee50 con 0x5650c23c50a0 2024-09-21T03:39:18.150 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:18.138+0000 7fe18effd640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5650c23cd280 0x5650c2794980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:18.150 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:18.139+0000 7fe18effd640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5650c23cd280 0x5650c2794980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:36984/0 (socket says 172.21.15.89:36984) 2024-09-21T03:39:18.151 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:18.139+0000 7fe18effd640 1 -- 172.21.15.89:0/3467093885 learned_addr learned my addr 172.21.15.89:0/3467093885 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-21T03:39:18.151 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:18.139+0000 7fe1977fe640 1 --2- 172.21.15.89:0/3467093885 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5650c23c50a0 0x5650c2794440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:18.151 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:18.139+0000 7fe197fff640 1 --2- 172.21.15.89:0/3467093885 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5650c23edf60 0x5650c2793f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:18.151 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:18.139+0000 7fe1957fa640 1 --2- 172.21.15.89:0/3400475748 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7fe1840b65a0 0x7fe1840b8ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:18.151 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:18.139+0000 7fe18effd640 1 -- 172.21.15.89:0/3467093885 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5650c23c50a0 msgr2=0x5650c2794440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:39:18.151 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:18.139+0000 7fe18effd640 1 --2- 172.21.15.89:0/3467093885 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5650c23c50a0 0x5650c2794440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:18.151 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:18.139+0000 7fe18effd640 1 -- 172.21.15.89:0/3467093885 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5650c23edf60 msgr2=0x5650c2793f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:39:18.151 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:18.139+0000 7fe18effd640 1 --2- 172.21.15.89:0/3467093885 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5650c23edf60 0x5650c2793f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:18.151 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:18.139+0000 7fe18effd640 1 -- 172.21.15.89:0/3467093885 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5650c2459540 con 0x5650c23cd280 2024-09-21T03:39:18.151 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:18.139+0000 7fe197fff640 1 --2- 172.21.15.89:0/3400475748 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7fe1840b65a0 0x7fe1840b8ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:18.151 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:18.139+0000 7fe1957fa640 1 -- 172.21.15.89:0/3400475748 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 9121+0+0 (secure 0 0 0) 0x7fe188108020 con 0x7fe198102b10 2024-09-21T03:39:18.151 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:18.139+0000 7fe18effd640 1 --2- 172.21.15.89:0/3467093885 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5650c23cd280 0x5650c2794980 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fe180009bc0 tx=0x7fe18000e6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5e983e754030354 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:39:18.151 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:18.139+0000 7fe18dffb640 1 -- 172.21.15.89:0/3467093885 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe180016070 con 0x5650c23cd280 2024-09-21T03:39:18.151 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:18.139+0000 7fe18dffb640 1 -- 172.21.15.89:0/3467093885 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe180011900 con 0x5650c23cd280 2024-09-21T03:39:18.152 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:18.139+0000 7fe18dffb640 1 -- 172.21.15.89:0/3467093885 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe180010750 con 0x5650c23cd280 2024-09-21T03:39:18.152 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:18.140+0000 7fe1a41df740 1 -- 172.21.15.89:0/3467093885 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5650c23cd280 msgr2=0x5650c2794980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:18.152 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:18.140+0000 7fe197fff640 1 --2- 172.21.15.89:0/3400475748 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7fe1840b65a0 0x7fe1840b8ab0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fe190005fd0 tx=0x7fe190004a40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:39:18.152 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:18.140+0000 7fe1a41df740 1 --2- 172.21.15.89:0/3467093885 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5650c23cd280 0x5650c2794980 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fe180009bc0 tx=0x7fe18000e6c0 comp rx=0 tx=0).stop 2024-09-21T03:39:18.152 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:18.140+0000 7fe1a41df740 1 -- 172.21.15.89:0/3467093885 shutdown_connections 2024-09-21T03:39:18.152 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:18.140+0000 7fe1a41df740 1 --2- 172.21.15.89:0/3467093885 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5650c23cd280 0x5650c2794980 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:18.152 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:18.140+0000 7fe1a41df740 1 --2- 172.21.15.89:0/3467093885 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5650c23c50a0 0x5650c2794440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:18.152 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:18.140+0000 7fe1a41df740 1 --2- 172.21.15.89:0/3467093885 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5650c23edf60 0x5650c2793f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:18.152 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:18.140+0000 7fe1a41df740 1 -- 172.21.15.89:0/3467093885 >> 172.21.15.89:0/3467093885 conn(0x5650c23e2350 msgr2=0x5650c2791df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:39:18.152 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:18.140+0000 7fe1a41df740 1 -- 172.21.15.89:0/3467093885 shutdown_connections 2024-09-21T03:39:18.152 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:18.140+0000 7fe1a41df740 1 -- 172.21.15.89:0/3467093885 wait complete. 2024-09-21T03:39:18.152 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:18.141+0000 7fe1a41df740 1 -- start start 2024-09-21T03:39:18.152 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:18.141+0000 7fe1a41df740 1 Processor -- start 2024-09-21T03:39:18.152 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:18.141+0000 7fe1a41df740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5650c23c50a0 0x5650c279f9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:18.152 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:18.141+0000 7fe1a41df740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5650c23cd280 0x5650c279ff00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:18.153 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:18.141+0000 7fe1a41df740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5650c23edf60 0x5650c27ab9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:18.153 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:18.141+0000 7fe1a41df740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x5650c255c5f0 con 0x5650c23cd280 2024-09-21T03:39:18.153 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:18.141+0000 7fe1a41df740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x5650c2696010 con 0x5650c23edf60 2024-09-21T03:39:18.153 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:18.141+0000 7fe1a41df740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x5650c27a26e0 con 0x5650c23c50a0 2024-09-21T03:39:18.153 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:18.141+0000 7fe1a41df740 10 client.0 _ll_register_callbacks cb 0x5650c26f5340 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:39:18.153 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:18.141+0000 7fe18effd640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5650c23cd280 0x5650c279ff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:18.153 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:18.141+0000 7fe18effd640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5650c23cd280 0x5650c279ff00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:36994/0 (socket says 172.21.15.89:36994) 2024-09-21T03:39:18.153 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:18.141+0000 7fe18effd640 1 -- 172.21.15.89:0/3697482904 learned_addr learned my addr 172.21.15.89:0/3697482904 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-21T03:39:18.153 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:18.141+0000 7fe1977fe640 1 --2- 172.21.15.89:0/3697482904 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5650c23c50a0 0x5650c279f9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:18.153 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:18.142+0000 7fe197fff640 1 --2- 172.21.15.89:0/3697482904 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5650c23edf60 0x5650c27ab9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:18.153 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:18.142+0000 7fe18effd640 1 -- 172.21.15.89:0/3697482904 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5650c23c50a0 msgr2=0x5650c279f9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:39:18.153 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:18.142+0000 7fe18effd640 1 --2- 172.21.15.89:0/3697482904 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5650c23c50a0 0x5650c279f9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:18.153 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:18.142+0000 7fe18effd640 1 -- 172.21.15.89:0/3697482904 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5650c23edf60 msgr2=0x5650c27ab9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:39:18.153 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:18.142+0000 7fe18effd640 1 --2- 172.21.15.89:0/3697482904 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5650c23edf60 0x5650c27ab9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:18.153 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:18.142+0000 7fe18effd640 1 -- 172.21.15.89:0/3697482904 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5650c26c3a00 con 0x5650c23cd280 2024-09-21T03:39:18.154 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:18.142+0000 7fe18effd640 1 --2- 172.21.15.89:0/3697482904 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5650c23cd280 0x5650c279ff00 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fe180009ee0 tx=0x7fe1800097c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa1a5e3c49dc05ab server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:39:18.154 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:18.142+0000 7fe176ffd640 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2024-09-21T03:39:18.154 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:18.142+0000 7fe176ffd640 1 -- 172.21.15.89:0/3697482904 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe180028060 con 0x5650c23cd280 2024-09-21T03:39:18.154 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:18.142+0000 7fe176ffd640 1 -- 172.21.15.89:0/3697482904 <== mon.0 v2:172.21.15.89:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 9121+0+0 (secure 0 0 0) 0x7fe180005040 con 0x5650c23cd280 2024-09-21T03:39:18.154 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:18.142+0000 7fe1a41df740 1 -- 172.21.15.89:0/3697482904 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe180011900 con 0x5650c23cd280 2024-09-21T03:39:18.154 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:18.142+0000 7fe176ffd640 1 -- 172.21.15.89:0/3697482904 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fe18002f050 con 0x5650c23cd280 2024-09-21T03:39:18.154 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:18.142+0000 7fe1a41df740 10 client.15387 fetch_fsmap learned FSMap version 25 2024-09-21T03:39:18.154 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:18.142+0000 7fe1a41df740 1 -- 172.21.15.89:0/3697482904 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5650c27b0810 con 0x5650c23cd280 2024-09-21T03:39:18.154 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:18.142+0000 7fe176ffd640 1 -- 172.21.15.89:0/3697482904 <== mon.0 v2:172.21.15.89:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe180011900 con 0x5650c23cd280 2024-09-21T03:39:18.154 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:18.143+0000 7fe176ffd640 1 -- 172.21.15.89:0/3697482904 <== mon.0 v2:172.21.15.89:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7fe18000f040 con 0x5650c23cd280 2024-09-21T03:39:18.154 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:18.143+0000 7fe1a41df740 10 client.15387 fetch_fsmap finished waiting for FSMap version 25 2024-09-21T03:39:18.154 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:18.143+0000 7fe1a41df740 10 client.15387 resolve_mds: resolved name 'i' to daemon mds.i 2024-09-21T03:39:18.154 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:18.143+0000 7fe1a41df740 20 client.15387 populate_metadata read hostname 'smithi089' 2024-09-21T03:39:18.154 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:18.143+0000 7fe1a41df740 1 --2- 172.21.15.89:0/3697482904 >> [v2:172.21.15.114:6838/3427369944,v1:172.21.15.114:6839/3427369944] conn(0x5650c23ac8e0 0x5650c27b3f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:18.154 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:18.143+0000 7fe1a41df740 4 client.15387 mds_command: new command op to 24631 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:39:18.155 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:18.143+0000 7fe1a41df740 1 -- 172.21.15.89:0/3697482904 --> [v2:172.21.15.114:6838/3427369944,v1:172.21.15.114:6839/3427369944] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5650c270db60 con 0x5650c23ac8e0 2024-09-21T03:39:18.155 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:18.143+0000 7fe1977fe640 1 --2- 172.21.15.89:0/3697482904 >> [v2:172.21.15.114:6838/3427369944,v1:172.21.15.114:6839/3427369944] conn(0x5650c23ac8e0 0x5650c27b3f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:18.155 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:18.143+0000 7fe176ffd640 10 client.15387 ms_handle_connect on v2:172.21.15.114:6838/3427369944 2024-09-21T03:39:18.155 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:18.143+0000 7fe1977fe640 1 --2- 172.21.15.89:0/3697482904 >> [v2:172.21.15.114:6838/3427369944,v1:172.21.15.114:6839/3427369944] conn(0x5650c23ac8e0 0x5650c27b3f10 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e4ad32e2bb1e59ac server_cookie=b2000492f65b4f44 in_seq=0 out_seq=0 2024-09-21T03:39:18.155 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:18.144+0000 7fe176ffd640 1 -- 172.21.15.89:0/3697482904 <== mds.1 v2:172.21.15.114:6838/3427369944 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5650c270db60 con 0x5650c23ac8e0 2024-09-21T03:39:18.155 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:18.144+0000 7fe176ffd640 10 client.15387 handle_command_reply: tid=0 2024-09-21T03:39:18.155 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:18.145+0000 7fe1a41df740 1 client.15387 shutdown 2024-09-21T03:39:18.155 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:18.145+0000 7fe1a41df740 1 -- 172.21.15.89:0/3697482904 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5650c23cd280 msgr2=0x5650c279ff00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:18.155 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:18.145+0000 7fe1a41df740 1 --2- 172.21.15.89:0/3697482904 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5650c23cd280 0x5650c279ff00 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fe180009ee0 tx=0x7fe1800097c0 comp rx=0 tx=0).stop 2024-09-21T03:39:18.155 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:18.145+0000 7fe1a41df740 1 -- 172.21.15.89:0/3697482904 shutdown_connections 2024-09-21T03:39:18.155 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:18.145+0000 7fe1a41df740 1 --2- 172.21.15.89:0/3697482904 >> [v2:172.21.15.114:6838/3427369944,v1:172.21.15.114:6839/3427369944] conn(0x5650c23ac8e0 0x5650c27b3f10 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:18.155 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:18.145+0000 7fe1a41df740 1 --2- 172.21.15.89:0/3697482904 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5650c23edf60 0x5650c27ab9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:18.155 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:18.145+0000 7fe1a41df740 1 --2- 172.21.15.89:0/3697482904 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5650c23cd280 0x5650c279ff00 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:18.155 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:18.145+0000 7fe1a41df740 1 --2- 172.21.15.89:0/3697482904 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5650c23c50a0 0x5650c279f9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:18.155 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:18.145+0000 7fe1a41df740 1 -- 172.21.15.89:0/3697482904 >> 172.21.15.89:0/3697482904 conn(0x5650c23e2350 msgr2=0x5650c27970e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:39:18.156 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:18.145+0000 7fe176ffd640 0 client.15387 ms_handle_reset on v2:172.21.15.114:6838/3427369944 2024-09-21T03:39:18.156 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:18.145+0000 7fe1a41df740 1 -- 172.21.15.89:0/3697482904 shutdown_connections 2024-09-21T03:39:18.156 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:18.145+0000 7fe1a41df740 1 -- 172.21.15.89:0/3697482904 wait complete. 2024-09-21T03:39:18.156 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:18.145+0000 7fe1a41df740 20 client.15387 trim_cache size 0 max 16384 2024-09-21T03:39:18.166 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:18.163+0000 7fe1a41df740 1 Processor -- start 2024-09-21T03:39:18.166 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:18.163+0000 7fe1a41df740 1 -- start start 2024-09-21T03:39:18.166 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:18.163+0000 7fe1a41df740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5650c23c50a0 0x5650c27b2aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:18.166 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:18.163+0000 7fe1a41df740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5650c23cd280 0x5650c27a1cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:18.166 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:18.163+0000 7fe1a41df740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5650c23edf60 0x5650c279a9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:18.166 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:18.163+0000 7fe1a41df740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x5650c27b2250 con 0x5650c23edf60 2024-09-21T03:39:18.166 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:18.163+0000 7fe1a41df740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x5650c27b2920 con 0x5650c23c50a0 2024-09-21T03:39:18.166 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:18.163+0000 7fe1a41df740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x5650c27b2ff0 con 0x5650c23cd280 2024-09-21T03:39:18.167 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:18.163+0000 7fe197fff640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5650c23edf60 0x5650c279a9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:18.167 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:18.163+0000 7fe197fff640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5650c23edf60 0x5650c279a9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:36998/0 (socket says 172.21.15.89:36998) 2024-09-21T03:39:18.167 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:18.163+0000 7fe197fff640 1 -- 172.21.15.89:0/1499608997 learned_addr learned my addr 172.21.15.89:0/1499608997 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-21T03:39:18.167 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:18.163+0000 7fe1977fe640 1 --2- 172.21.15.89:0/1499608997 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5650c23c50a0 0x5650c27b2aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:18.167 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:18.163+0000 7fe18effd640 1 --2- 172.21.15.89:0/1499608997 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5650c23cd280 0x5650c27a1cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:18.167 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:18.164+0000 7fe197fff640 1 -- 172.21.15.89:0/1499608997 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5650c23cd280 msgr2=0x5650c27a1cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:39:18.167 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:18.164+0000 7fe197fff640 1 --2- 172.21.15.89:0/1499608997 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5650c23cd280 0x5650c27a1cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:18.167 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:18.164+0000 7fe197fff640 1 -- 172.21.15.89:0/1499608997 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5650c23c50a0 msgr2=0x5650c27b2aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:39:18.167 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:18.164+0000 7fe197fff640 1 --2- 172.21.15.89:0/1499608997 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5650c23c50a0 0x5650c27b2aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:18.167 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:18.164+0000 7fe197fff640 1 -- 172.21.15.89:0/1499608997 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5650c27a0a80 con 0x5650c23edf60 2024-09-21T03:39:18.167 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:18.164+0000 7fe197fff640 1 --2- 172.21.15.89:0/1499608997 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5650c23edf60 0x5650c279a9a0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fe190009f50 tx=0x7fe190014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ab8e581f6e48cb3 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:39:18.167 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:18.164+0000 7fe18d7fa640 1 -- 172.21.15.89:0/1499608997 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe19001a640 con 0x5650c23edf60 2024-09-21T03:39:18.167 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:18.164+0000 7fe18d7fa640 1 -- 172.21.15.89:0/1499608997 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe19001de50 con 0x5650c23edf60 2024-09-21T03:39:18.168 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:18.164+0000 7fe18d7fa640 1 -- 172.21.15.89:0/1499608997 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe190012af0 con 0x5650c23edf60 2024-09-21T03:39:18.171 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:18.165+0000 7fe1a41df740 1 -- 172.21.15.89:0/1499608997 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5650c23edf60 msgr2=0x5650c279a9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:18.171 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:18.165+0000 7fe1a41df740 1 --2- 172.21.15.89:0/1499608997 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5650c23edf60 0x5650c279a9a0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fe190009f50 tx=0x7fe190014000 comp rx=0 tx=0).stop 2024-09-21T03:39:18.172 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:18.165+0000 7fe1a41df740 1 -- 172.21.15.89:0/1499608997 shutdown_connections 2024-09-21T03:39:18.172 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:18.165+0000 7fe1a41df740 1 --2- 172.21.15.89:0/1499608997 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5650c23edf60 0x5650c279a9a0 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:18.172 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:18.165+0000 7fe1a41df740 1 --2- 172.21.15.89:0/1499608997 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5650c23cd280 0x5650c27a1cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:18.172 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:18.165+0000 7fe1a41df740 1 --2- 172.21.15.89:0/1499608997 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5650c23c50a0 0x5650c27b2aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:18.172 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:18.165+0000 7fe1a41df740 1 -- 172.21.15.89:0/1499608997 >> 172.21.15.89:0/1499608997 conn(0x5650c23e2350 msgr2=0x5650c279f9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:39:18.172 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:18.165+0000 7fe1a41df740 1 -- 172.21.15.89:0/1499608997 shutdown_connections 2024-09-21T03:39:18.172 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:18.165+0000 7fe1a41df740 1 -- 172.21.15.89:0/1499608997 wait complete. 2024-09-21T03:39:18.172 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:18.165+0000 7fe1a41df740 1 -- start start 2024-09-21T03:39:18.172 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:18.166+0000 7fe1a41df740 1 Processor -- start 2024-09-21T03:39:18.172 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:18.166+0000 7fe1a41df740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5650c23c50a0 0x5650c27bc4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:18.172 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:18.166+0000 7fe1a41df740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5650c23cd280 0x5650c27bcad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:18.172 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:18.166+0000 7fe1a41df740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5650c23edf60 0x5650c27b5a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:18.172 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:18.166+0000 7fe1a41df740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x5650c27b3d90 con 0x5650c23edf60 2024-09-21T03:39:18.173 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:18.166+0000 7fe1a41df740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x5650c27b36c0 con 0x5650c23cd280 2024-09-21T03:39:18.173 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:18.166+0000 7fe1a41df740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x5650c27b4460 con 0x5650c23c50a0 2024-09-21T03:39:18.173 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:18.166+0000 7fe1a41df740 10 client.0 _ll_register_callbacks cb 0x7fe18810ba40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:39:18.173 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:18.166+0000 7fe197fff640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5650c23edf60 0x5650c27b5a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:18.173 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:18.166+0000 7fe197fff640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5650c23edf60 0x5650c27b5a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:37010/0 (socket says 172.21.15.89:37010) 2024-09-21T03:39:18.173 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:18.166+0000 7fe197fff640 1 -- 172.21.15.89:0/1265190624 learned_addr learned my addr 172.21.15.89:0/1265190624 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-21T03:39:18.173 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:18.166+0000 7fe18effd640 1 --2- 172.21.15.89:0/1265190624 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5650c23cd280 0x5650c27bcad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:18.173 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:18.166+0000 7fe1977fe640 1 --2- 172.21.15.89:0/1265190624 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5650c23c50a0 0x5650c27bc4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:18.173 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:18.167+0000 7fe197fff640 1 -- 172.21.15.89:0/1265190624 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5650c23c50a0 msgr2=0x5650c27bc4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:39:18.173 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:18.167+0000 7fe197fff640 1 --2- 172.21.15.89:0/1265190624 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5650c23c50a0 0x5650c27bc4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:18.173 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:18.167+0000 7fe197fff640 1 -- 172.21.15.89:0/1265190624 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5650c23cd280 msgr2=0x5650c27bcad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:39:18.173 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:18.167+0000 7fe197fff640 1 --2- 172.21.15.89:0/1265190624 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5650c23cd280 0x5650c27bcad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:18.173 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:18.167+0000 7fe197fff640 1 -- 172.21.15.89:0/1265190624 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5650c27b61c0 con 0x5650c23edf60 2024-09-21T03:39:18.173 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:18.167+0000 7fe18effd640 1 --2- 172.21.15.89:0/1265190624 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5650c23cd280 0x5650c27bcad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:39:18.173 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:18.167+0000 7fe197fff640 1 --2- 172.21.15.89:0/1265190624 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5650c23edf60 0x5650c27b5a60 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7fe190009950 tx=0x7fe190012690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d25198368c7d4b7a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:39:18.173 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:18.167+0000 7fe176ffd640 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2024-09-21T03:39:18.173 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:18.167+0000 7fe176ffd640 1 -- 172.21.15.89:0/1265190624 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe190033060 con 0x5650c23edf60 2024-09-21T03:39:18.174 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:18.167+0000 7fe176ffd640 1 -- 172.21.15.89:0/1265190624 <== mon.0 v2:172.21.15.89:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 9121+0+0 (secure 0 0 0) 0x7fe190012a30 con 0x5650c23edf60 2024-09-21T03:39:18.174 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:18.167+0000 7fe1a41df740 1 -- 172.21.15.89:0/1265190624 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe19001de50 con 0x5650c23edf60 2024-09-21T03:39:18.174 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:18.167+0000 7fe176ffd640 1 -- 172.21.15.89:0/1265190624 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fe19003a050 con 0x5650c23edf60 2024-09-21T03:39:18.174 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:18.167+0000 7fe1a41df740 10 client.15393 fetch_fsmap learned FSMap version 25 2024-09-21T03:39:18.174 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:18.167+0000 7fe1a41df740 1 -- 172.21.15.89:0/1265190624 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5650c27bbbf0 con 0x5650c23edf60 2024-09-21T03:39:18.174 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:18.167+0000 7fe176ffd640 1 -- 172.21.15.89:0/1265190624 <== mon.0 v2:172.21.15.89:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe19001de50 con 0x5650c23edf60 2024-09-21T03:39:18.174 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:18.168+0000 7fe176ffd640 1 -- 172.21.15.89:0/1265190624 <== mon.0 v2:172.21.15.89:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7fe190005810 con 0x5650c23edf60 2024-09-21T03:39:18.174 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:18.168+0000 7fe1a41df740 10 client.15393 fetch_fsmap finished waiting for FSMap version 25 2024-09-21T03:39:18.174 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:18.168+0000 7fe1a41df740 10 client.15393 resolve_mds: resolved name 'i' to daemon mds.i 2024-09-21T03:39:18.174 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:18.168+0000 7fe1a41df740 20 client.15393 populate_metadata read hostname 'smithi089' 2024-09-21T03:39:18.174 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:18.168+0000 7fe1a41df740 1 --2- 172.21.15.89:0/1265190624 >> [v2:172.21.15.114:6838/3427369944,v1:172.21.15.114:6839/3427369944] conn(0x5650c2381c90 0x5650c28225d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:18.174 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:18.168+0000 7fe1a41df740 4 client.15393 mds_command: new command op to 24631 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-21T03:39:18.174 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:18.168+0000 7fe1a41df740 1 -- 172.21.15.89:0/1265190624 --> [v2:172.21.15.114:6838/3427369944,v1:172.21.15.114:6839/3427369944] -- command(tid 0: {"prefix": "status"}) -- 0x5650c2709610 con 0x5650c2381c90 2024-09-21T03:39:18.174 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:18.168+0000 7fe1977fe640 1 --2- 172.21.15.89:0/1265190624 >> [v2:172.21.15.114:6838/3427369944,v1:172.21.15.114:6839/3427369944] conn(0x5650c2381c90 0x5650c28225d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:18.175 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:18.168+0000 7fe1977fe640 1 --2- 172.21.15.89:0/1265190624 >> [v2:172.21.15.114:6838/3427369944,v1:172.21.15.114:6839/3427369944] conn(0x5650c2381c90 0x5650c28225d0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=433bbe288194fcc5 server_cookie=d389c7cf2716184f in_seq=0 out_seq=0 2024-09-21T03:39:18.175 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:18.168+0000 7fe176ffd640 10 client.15393 ms_handle_connect on v2:172.21.15.114:6838/3427369944 2024-09-21T03:39:18.175 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:18.169+0000 7fe176ffd640 1 -- 172.21.15.89:0/1265190624 <== mds.1 v2:172.21.15.114:6838/3427369944 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x5650c2709610 con 0x5650c2381c90 2024-09-21T03:39:18.175 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:18.169+0000 7fe176ffd640 10 client.15393 handle_command_reply: tid=0 2024-09-21T03:39:18.175 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:18.169+0000 7fe1a41df740 1 client.15393 shutdown 2024-09-21T03:39:18.175 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:18.169+0000 7fe1a41df740 1 -- 172.21.15.89:0/1265190624 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5650c23edf60 msgr2=0x5650c27b5a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:18.175 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:18.169+0000 7fe1a41df740 1 --2- 172.21.15.89:0/1265190624 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5650c23edf60 0x5650c27b5a60 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7fe190009950 tx=0x7fe190012690 comp rx=0 tx=0).stop 2024-09-21T03:39:18.175 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:18.169+0000 7fe1a41df740 1 -- 172.21.15.89:0/1265190624 shutdown_connections 2024-09-21T03:39:18.175 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:18.169+0000 7fe1a41df740 1 --2- 172.21.15.89:0/1265190624 >> [v2:172.21.15.114:6838/3427369944,v1:172.21.15.114:6839/3427369944] conn(0x5650c2381c90 0x5650c28225d0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:18.175 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:18.169+0000 7fe1a41df740 1 --2- 172.21.15.89:0/1265190624 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5650c23cd280 0x5650c27bcad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:18.175 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:18.169+0000 7fe1a41df740 1 --2- 172.21.15.89:0/1265190624 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5650c23edf60 0x5650c27b5a60 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:18.175 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:18.169+0000 7fe1a41df740 1 --2- 172.21.15.89:0/1265190624 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5650c23c50a0 0x5650c27bc4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:18.175 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:18.169+0000 7fe1a41df740 1 -- 172.21.15.89:0/1265190624 >> 172.21.15.89:0/1265190624 conn(0x5650c23e2350 msgr2=0x5650c279a9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:39:18.175 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:18.169+0000 7fe176ffd640 0 client.15393 ms_handle_reset on v2:172.21.15.114:6838/3427369944 2024-09-21T03:39:18.175 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:18.169+0000 7fe1a41df740 1 -- 172.21.15.89:0/1265190624 shutdown_connections 2024-09-21T03:39:18.176 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:18.169+0000 7fe1a41df740 1 -- 172.21.15.89:0/1265190624 wait complete. 2024-09-21T03:39:18.176 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:18.169+0000 7fe1a41df740 20 client.15393 trim_cache size 0 max 16384 2024-09-21T03:39:18.176 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-09-21T03:39:18.176 INFO:teuthology.orchestra.run.smithi089.stdout: "cluster_fsid": "7c374098-77c9-11ef-baf5-efdc52797490", 2024-09-21T03:39:18.176 INFO:teuthology.orchestra.run.smithi089.stdout: "whoami": 1, 2024-09-21T03:39:18.176 INFO:teuthology.orchestra.run.smithi089.stdout: "id": 24631, 2024-09-21T03:39:18.176 INFO:teuthology.orchestra.run.smithi089.stdout: "want_state": "up:active", 2024-09-21T03:39:18.176 INFO:teuthology.orchestra.run.smithi089.stdout: "state": "up:active", 2024-09-21T03:39:18.176 INFO:teuthology.orchestra.run.smithi089.stdout: "fs_name": "cephfs", 2024-09-21T03:39:18.176 INFO:teuthology.orchestra.run.smithi089.stdout: "rank_uptime": 91.171208738999994, 2024-09-21T03:39:18.176 INFO:teuthology.orchestra.run.smithi089.stdout: "mdsmap_epoch": 25, 2024-09-21T03:39:18.177 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap_epoch": 84, 2024-09-21T03:39:18.177 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap_epoch_barrier": 80, 2024-09-21T03:39:18.177 INFO:teuthology.orchestra.run.smithi089.stdout: "uptime": 143.524280105 2024-09-21T03:39:18.177 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-09-21T03:39:18.177 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:18.170+0000 7fe19dc6b640 1 -- 172.21.15.89:0/3400475748 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7fe1840b65a0 msgr2=0x7fe1840b8ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:18.177 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:18.170+0000 7fe19dc6b640 1 --2- 172.21.15.89:0/3400475748 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7fe1840b65a0 0x7fe1840b8ab0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fe190005fd0 tx=0x7fe190004a40 comp rx=0 tx=0).stop 2024-09-21T03:39:18.178 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:18.170+0000 7fe19dc6b640 1 -- 172.21.15.89:0/3400475748 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe198102b10 msgr2=0x7fe1981993a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:18.178 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:18.170+0000 7fe19dc6b640 1 --2- 172.21.15.89:0/3400475748 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe198102b10 0x7fe1981993a0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fe188009230 tx=0x7fe188004270 comp rx=0 tx=0).stop 2024-09-21T03:39:18.178 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:18.170+0000 7fe19dc6b640 1 -- 172.21.15.89:0/3400475748 shutdown_connections 2024-09-21T03:39:18.178 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:18.170+0000 7fe19dc6b640 1 --2- 172.21.15.89:0/3400475748 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7fe1840b65a0 0x7fe1840b8ab0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:18.178 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:18.170+0000 7fe19dc6b640 1 --2- 172.21.15.89:0/3400475748 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe198108450 0x7fe198199940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:18.178 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:18.170+0000 7fe19dc6b640 1 --2- 172.21.15.89:0/3400475748 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe198108da0 0x7fe19819ebe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:18.178 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:18.170+0000 7fe19dc6b640 1 --2- 172.21.15.89:0/3400475748 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe198102b10 0x7fe1981993a0 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:18.178 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:18.170+0000 7fe19dc6b640 1 -- 172.21.15.89:0/3400475748 >> 172.21.15.89:0/3400475748 conn(0x7fe1980796d0 msgr2=0x7fe1981065b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:39:18.178 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:18.170+0000 7fe19dc6b640 1 -- 172.21.15.89:0/3400475748 shutdown_connections 2024-09-21T03:39:18.178 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:18.170+0000 7fe19dc6b640 1 -- 172.21.15.89:0/3400475748 wait complete. 2024-09-21T03:39:18.828 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 7c374098-77c9-11ef-baf5-efdc52797490 -- ceph tell mds.b status 2024-09-21T03:39:19.042 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:39:19 smithi089 ceph-mon[30928]: pgmap v357: 129 pgs: 129 active+clean; 646 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-21T03:39:19.076 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/7c374098-77c9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:39:19.461 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:39:19 smithi114 ceph-mon[34140]: pgmap v357: 129 pgs: 129 active+clean; 646 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-21T03:39:19.500 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:39:19 smithi110 ceph-mon[34000]: pgmap v357: 129 pgs: 129 active+clean; 646 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-21T03:39:19.575 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:19.570+0000 7f824acf5640 1 -- 172.21.15.89:0/1599682062 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8244109570 msgr2=0x7f8244109980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:19.575 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:19.570+0000 7f824acf5640 1 --2- 172.21.15.89:0/1599682062 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8244109570 0x7f8244109980 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f8234009900 tx=0x7f823402a410 comp rx=0 tx=0).stop 2024-09-21T03:39:19.575 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:19.570+0000 7f824acf5640 1 -- 172.21.15.89:0/1599682062 shutdown_connections 2024-09-21T03:39:19.575 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:19.570+0000 7f824acf5640 1 --2- 172.21.15.89:0/1599682062 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8244109ec0 0x7f82441106f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:19.575 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:19.570+0000 7f824acf5640 1 --2- 172.21.15.89:0/1599682062 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8244109570 0x7f8244109980 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:19.576 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:19.570+0000 7f824acf5640 1 --2- 172.21.15.89:0/1599682062 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8244103c30 0x7f8244104040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:19.576 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:19.570+0000 7f824acf5640 1 -- 172.21.15.89:0/1599682062 >> 172.21.15.89:0/1599682062 conn(0x7f824407b8d0 msgr2=0x7f824407bce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:39:19.576 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:19.570+0000 7f824acf5640 1 -- 172.21.15.89:0/1599682062 shutdown_connections 2024-09-21T03:39:19.576 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:19.570+0000 7f824acf5640 1 -- 172.21.15.89:0/1599682062 wait complete. 2024-09-21T03:39:19.576 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:19.571+0000 7f824acf5640 1 Processor -- start 2024-09-21T03:39:19.576 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:19.571+0000 7f824acf5640 1 -- start start 2024-09-21T03:39:19.576 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:19.571+0000 7f824acf5640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8244103c30 0x7f824419a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:19.576 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:19.571+0000 7f824acf5640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8244109570 0x7f824419aa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:19.576 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:19.571+0000 7f824acf5640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8244109ec0 0x7f824419fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:19.576 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:19.571+0000 7f824acf5640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f8244113990 con 0x7f8244109ec0 2024-09-21T03:39:19.576 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:19.571+0000 7f824acf5640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f8244113810 con 0x7f8244103c30 2024-09-21T03:39:19.576 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:19.571+0000 7f824acf5640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f8244113b10 con 0x7f8244109570 2024-09-21T03:39:19.576 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:19.571+0000 7f8249292640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8244109ec0 0x7f824419fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:19.576 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:19.571+0000 7f8249292640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8244109ec0 0x7f824419fcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:53232/0 (socket says 172.21.15.89:53232) 2024-09-21T03:39:19.577 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:19.571+0000 7f8249292640 1 -- 172.21.15.89:0/2242522540 learned_addr learned my addr 172.21.15.89:0/2242522540 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-21T03:39:19.577 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:19.571+0000 7f8248a91640 1 --2- 172.21.15.89:0/2242522540 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8244103c30 0x7f824419a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:19.577 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:19.571+0000 7f823bfff640 1 --2- 172.21.15.89:0/2242522540 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8244109570 0x7f824419aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:19.577 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:19.571+0000 7f8249292640 1 -- 172.21.15.89:0/2242522540 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8244109570 msgr2=0x7f824419aa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:19.577 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:19.571+0000 7f8249292640 1 --2- 172.21.15.89:0/2242522540 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8244109570 0x7f824419aa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:19.577 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:19.571+0000 7f8249292640 1 -- 172.21.15.89:0/2242522540 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8244103c30 msgr2=0x7f824419a480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:19.577 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:19.571+0000 7f8249292640 1 --2- 172.21.15.89:0/2242522540 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8244103c30 0x7f824419a480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:19.577 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:19.571+0000 7f8249292640 1 -- 172.21.15.89:0/2242522540 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82441a03f0 con 0x7f8244109ec0 2024-09-21T03:39:19.577 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:19.572+0000 7f8249292640 1 --2- 172.21.15.89:0/2242522540 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8244109ec0 0x7f824419fcc0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f823c00eaa0 tx=0x7f823c00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:39:19.577 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:19.572+0000 7f8239ffb640 1 -- 172.21.15.89:0/2242522540 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f823c00cdb0 con 0x7f8244109ec0 2024-09-21T03:39:19.577 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:19.572+0000 7f8239ffb640 1 -- 172.21.15.89:0/2242522540 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f823c005020 con 0x7f8244109ec0 2024-09-21T03:39:19.577 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:19.572+0000 7f8239ffb640 1 -- 172.21.15.89:0/2242522540 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f823c0116d0 con 0x7f8244109ec0 2024-09-21T03:39:19.577 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:19.572+0000 7f824acf5640 1 -- 172.21.15.89:0/2242522540 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f82441a06e0 con 0x7f8244109ec0 2024-09-21T03:39:19.577 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:19.572+0000 7f824acf5640 1 -- 172.21.15.89:0/2242522540 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82441a6e40 con 0x7f8244109ec0 2024-09-21T03:39:19.582 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:19.575+0000 7f8239ffb640 1 -- 172.21.15.89:0/2242522540 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 18) ==== 154563+0+0 (secure 0 0 0) 0x7f823c0051d0 con 0x7f8244109ec0 2024-09-21T03:39:19.582 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:19.575+0000 7f8251269740 1 Processor -- start 2024-09-21T03:39:19.582 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:19.575+0000 7f8251269740 1 -- start start 2024-09-21T03:39:19.582 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:19.575+0000 7f8251269740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5564c35caf60 0x5564c3970f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:19.582 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:19.575+0000 7f8251269740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5564c35a20a0 0x5564c3971440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:19.583 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:19.575+0000 7f8251269740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5564c35aa280 0x5564c3971980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:19.583 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:19.575+0000 7f8251269740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x5564c38e8c10 con 0x5564c35aa280 2024-09-21T03:39:19.583 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:19.575+0000 7f8251269740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x5564c38d8040 con 0x5564c35caf60 2024-09-21T03:39:19.583 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:19.575+0000 7f8251269740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x5564c38dbe50 con 0x5564c35a20a0 2024-09-21T03:39:19.583 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:19.575+0000 7f823bfff640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5564c35aa280 0x5564c3971980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:19.583 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:19.575+0000 7f823bfff640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5564c35aa280 0x5564c3971980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:53234/0 (socket says 172.21.15.89:53234) 2024-09-21T03:39:19.583 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:19.575+0000 7f823bfff640 1 -- 172.21.15.89:0/2442288269 learned_addr learned my addr 172.21.15.89:0/2442288269 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-21T03:39:19.583 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:19.575+0000 7f8248a91640 1 --2- 172.21.15.89:0/2442288269 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5564c35a20a0 0x5564c3971440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:19.583 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:19.575+0000 7f823bfff640 1 --2- 172.21.15.89:0/2442288269 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5564c35caf60 0x5564c3970f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:19.583 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:19.575+0000 7f8248a91640 1 -- 172.21.15.89:0/2442288269 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5564c35caf60 msgr2=0x5564c3970f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:39:19.583 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:19.575+0000 7f8248a91640 1 --2- 172.21.15.89:0/2442288269 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5564c35caf60 0x5564c3970f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:19.583 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:19.575+0000 7f8248a91640 1 -- 172.21.15.89:0/2442288269 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5564c35aa280 msgr2=0x5564c3971980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:39:19.583 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:19.575+0000 7f8248a91640 1 --2- 172.21.15.89:0/2442288269 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5564c35aa280 0x5564c3971980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:19.583 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:19.575+0000 7f8248a91640 1 -- 172.21.15.89:0/2442288269 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5564c3636540 con 0x5564c35a20a0 2024-09-21T03:39:19.584 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:19.575+0000 7f823bfff640 1 --2- 172.21.15.89:0/2442288269 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5564c35aa280 0x5564c3971980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:39:19.584 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:19.576+0000 7f8239ffb640 1 --2- 172.21.15.89:0/2242522540 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f822c0b65a0 0x7f822c0b8ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:19.584 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:19.576+0000 7f8248a91640 1 --2- 172.21.15.89:0/2442288269 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5564c35a20a0 0x5564c3971440 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f8240009c00 tx=0x7f8240011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=94164b84b71946e4 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:39:19.584 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:19.576+0000 7f823bfff640 1 --2- 172.21.15.89:0/2242522540 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f822c0b65a0 0x7f822c0b8ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:19.584 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:19.576+0000 7f8239ffb640 1 -- 172.21.15.89:0/2242522540 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 9121+0+0 (secure 0 0 0) 0x7f823c0e79b0 con 0x7f8244109ec0 2024-09-21T03:39:19.584 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:19.576+0000 7f822affd640 1 -- 172.21.15.89:0/2442288269 <== mon.1 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8240004230 con 0x5564c35a20a0 2024-09-21T03:39:19.584 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:19.576+0000 7f822affd640 1 -- 172.21.15.89:0/2442288269 <== mon.1 v2:172.21.15.114:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f82400043d0 con 0x5564c35a20a0 2024-09-21T03:39:19.584 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:19.576+0000 7f823bfff640 1 --2- 172.21.15.89:0/2242522540 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f822c0b65a0 0x7f822c0b8ab0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f8234006010 tx=0x7f82340058b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:39:19.584 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:19.576+0000 7f822affd640 1 -- 172.21.15.89:0/2442288269 <== mon.1 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8240004bc0 con 0x5564c35a20a0 2024-09-21T03:39:19.584 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:19.577+0000 7f8251269740 1 -- 172.21.15.89:0/2442288269 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5564c35a20a0 msgr2=0x5564c3971440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:19.584 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:19.577+0000 7f8251269740 1 --2- 172.21.15.89:0/2442288269 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5564c35a20a0 0x5564c3971440 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f8240009c00 tx=0x7f8240011810 comp rx=0 tx=0).stop 2024-09-21T03:39:19.584 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:19.577+0000 7f8251269740 1 -- 172.21.15.89:0/2442288269 shutdown_connections 2024-09-21T03:39:19.584 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:19.577+0000 7f8251269740 1 --2- 172.21.15.89:0/2442288269 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5564c35aa280 0x5564c3971980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:19.584 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:19.577+0000 7f8251269740 1 --2- 172.21.15.89:0/2442288269 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5564c35a20a0 0x5564c3971440 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:19.584 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:19.577+0000 7f8251269740 1 --2- 172.21.15.89:0/2442288269 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5564c35caf60 0x5564c3970f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:19.585 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:19.577+0000 7f8251269740 1 -- 172.21.15.89:0/2442288269 >> 172.21.15.89:0/2442288269 conn(0x5564c35bf350 msgr2=0x5564c396edf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:39:19.585 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:19.577+0000 7f8251269740 1 -- 172.21.15.89:0/2442288269 shutdown_connections 2024-09-21T03:39:19.585 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:19.577+0000 7f8251269740 1 -- 172.21.15.89:0/2442288269 wait complete. 2024-09-21T03:39:19.585 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:19.577+0000 7f8251269740 1 -- start start 2024-09-21T03:39:19.585 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:19.578+0000 7f8251269740 1 Processor -- start 2024-09-21T03:39:19.585 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:19.578+0000 7f8251269740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5564c35a20a0 0x5564c397c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:19.585 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:19.578+0000 7f8251269740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5564c35aa280 0x5564c397cf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:19.585 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:19.578+0000 7f8251269740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5564c35caf60 0x5564c3988a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:19.585 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:19.578+0000 7f8251269740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x5564c37395f0 con 0x5564c35a20a0 2024-09-21T03:39:19.585 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:19.578+0000 7f8251269740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x5564c3873010 con 0x5564c35caf60 2024-09-21T03:39:19.585 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:19.578+0000 7f8251269740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x5564c397f6e0 con 0x5564c35aa280 2024-09-21T03:39:19.585 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:19.578+0000 7f8251269740 10 client.0 _ll_register_callbacks cb 0x5564c38d2340 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:39:19.585 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:19.578+0000 7f8248a91640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5564c35a20a0 0x5564c397c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:19.586 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:19.578+0000 7f8248a91640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5564c35a20a0 0x5564c397c9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:53246/0 (socket says 172.21.15.89:53246) 2024-09-21T03:39:19.586 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:19.578+0000 7f8248a91640 1 -- 172.21.15.89:0/1792809036 learned_addr learned my addr 172.21.15.89:0/1792809036 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-21T03:39:19.586 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:19.578+0000 7f823bfff640 1 --2- 172.21.15.89:0/1792809036 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5564c35aa280 0x5564c397cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:19.586 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:19.578+0000 7f8249292640 1 --2- 172.21.15.89:0/1792809036 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5564c35caf60 0x5564c3988a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:19.586 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:19.579+0000 7f8248a91640 1 -- 172.21.15.89:0/1792809036 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5564c35aa280 msgr2=0x5564c397cf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:39:19.586 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:19.579+0000 7f8248a91640 1 --2- 172.21.15.89:0/1792809036 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5564c35aa280 0x5564c397cf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:19.586 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:19.579+0000 7f8248a91640 1 -- 172.21.15.89:0/1792809036 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5564c35caf60 msgr2=0x5564c3988a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:39:19.586 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:19.579+0000 7f8248a91640 1 --2- 172.21.15.89:0/1792809036 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5564c35caf60 0x5564c3988a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:19.586 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:19.579+0000 7f8248a91640 1 -- 172.21.15.89:0/1792809036 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5564c38a0a00 con 0x5564c35a20a0 2024-09-21T03:39:19.586 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:19.579+0000 7f8248a91640 1 --2- 172.21.15.89:0/1792809036 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5564c35a20a0 0x5564c397c9c0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f8240005890 tx=0x7f8240019ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76934217a167f70 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:39:19.586 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:19.579+0000 7f8223fff640 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2024-09-21T03:39:19.586 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:19.579+0000 7f8223fff640 1 -- 172.21.15.89:0/1792809036 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8240028020 con 0x5564c35a20a0 2024-09-21T03:39:19.587 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:19.579+0000 7f8223fff640 1 -- 172.21.15.89:0/1792809036 <== mon.0 v2:172.21.15.89:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 9121+0+0 (secure 0 0 0) 0x7f8240024620 con 0x5564c35a20a0 2024-09-21T03:39:19.587 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:19.579+0000 7f8251269740 1 -- 172.21.15.89:0/1792809036 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f82400043d0 con 0x5564c35a20a0 2024-09-21T03:39:19.587 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:19.579+0000 7f8223fff640 1 -- 172.21.15.89:0/1792809036 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f8240033050 con 0x5564c35a20a0 2024-09-21T03:39:19.587 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:19.579+0000 7f8251269740 10 client.15405 fetch_fsmap learned FSMap version 25 2024-09-21T03:39:19.587 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:19.579+0000 7f8251269740 1 -- 172.21.15.89:0/1792809036 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5564c398b610 con 0x5564c35a20a0 2024-09-21T03:39:19.587 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:19.579+0000 7f8223fff640 1 -- 172.21.15.89:0/1792809036 <== mon.0 v2:172.21.15.89:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f82400043d0 con 0x5564c35a20a0 2024-09-21T03:39:19.587 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:19.580+0000 7f8223fff640 1 -- 172.21.15.89:0/1792809036 <== mon.0 v2:172.21.15.89:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f8240018070 con 0x5564c35a20a0 2024-09-21T03:39:19.587 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:19.580+0000 7f8251269740 10 client.15405 fetch_fsmap finished waiting for FSMap version 25 2024-09-21T03:39:19.587 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:19.580+0000 7f8251269740 10 client.15405 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-21T03:39:19.587 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:19.580+0000 7f8251269740 20 client.15405 populate_metadata read hostname 'smithi089' 2024-09-21T03:39:19.588 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:19.580+0000 7f8251269740 1 --2- 172.21.15.89:0/1792809036 >> [v2:172.21.15.110:6838/132366709,v1:172.21.15.110:6839/132366709] conn(0x5564c35898e0 0x5564c398eea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:19.588 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:19.580+0000 7f8251269740 4 client.15405 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:39:19.588 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:19.580+0000 7f8251269740 1 -- 172.21.15.89:0/1792809036 --> [v2:172.21.15.110:6838/132366709,v1:172.21.15.110:6839/132366709] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5564c38eab60 con 0x5564c35898e0 2024-09-21T03:39:19.589 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:19.580+0000 7f823bfff640 1 --2- 172.21.15.89:0/1792809036 >> [v2:172.21.15.110:6838/132366709,v1:172.21.15.110:6839/132366709] conn(0x5564c35898e0 0x5564c398eea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:19.589 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:19.581+0000 7f823bfff640 1 --2- 172.21.15.89:0/1792809036 >> [v2:172.21.15.110:6838/132366709,v1:172.21.15.110:6839/132366709] conn(0x5564c35898e0 0x5564c398eea0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1498bc239c6ce38a server_cookie=bcac36bea9ce5eef in_seq=0 out_seq=0 2024-09-21T03:39:19.589 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:19.581+0000 7f8223fff640 10 client.15405 ms_handle_connect on v2:172.21.15.110:6838/132366709 2024-09-21T03:39:19.590 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:19.581+0000 7f8223fff640 1 -- 172.21.15.89:0/1792809036 <== mds.0 v2:172.21.15.110:6838/132366709 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5564c38eab60 con 0x5564c35898e0 2024-09-21T03:39:19.590 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:19.581+0000 7f8223fff640 10 client.15405 handle_command_reply: tid=0 2024-09-21T03:39:19.590 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:19.581+0000 7f8251269740 1 client.15405 shutdown 2024-09-21T03:39:19.590 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:19.582+0000 7f8251269740 1 -- 172.21.15.89:0/1792809036 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5564c35a20a0 msgr2=0x5564c397c9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:19.590 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:19.582+0000 7f8251269740 1 --2- 172.21.15.89:0/1792809036 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5564c35a20a0 0x5564c397c9c0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f8240005890 tx=0x7f8240019ba0 comp rx=0 tx=0).stop 2024-09-21T03:39:19.590 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:19.582+0000 7f8251269740 1 -- 172.21.15.89:0/1792809036 shutdown_connections 2024-09-21T03:39:19.590 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:19.582+0000 7f8251269740 1 --2- 172.21.15.89:0/1792809036 >> [v2:172.21.15.110:6838/132366709,v1:172.21.15.110:6839/132366709] conn(0x5564c35898e0 0x5564c398eea0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:19.590 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:19.582+0000 7f8223fff640 0 client.15405 ms_handle_reset on v2:172.21.15.110:6838/132366709 2024-09-21T03:39:19.590 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:19.582+0000 7f8251269740 1 --2- 172.21.15.89:0/1792809036 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5564c35caf60 0x5564c3988a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:19.590 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:19.582+0000 7f8251269740 1 --2- 172.21.15.89:0/1792809036 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5564c35aa280 0x5564c397cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:19.590 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:19.582+0000 7f8251269740 1 --2- 172.21.15.89:0/1792809036 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5564c35a20a0 0x5564c397c9c0 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:19.590 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:19.582+0000 7f8251269740 1 -- 172.21.15.89:0/1792809036 >> 172.21.15.89:0/1792809036 conn(0x5564c35bf350 msgr2=0x5564c3974090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:39:19.590 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:19.582+0000 7f8251269740 1 -- 172.21.15.89:0/1792809036 shutdown_connections 2024-09-21T03:39:19.590 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:19.582+0000 7f8251269740 1 -- 172.21.15.89:0/1792809036 wait complete. 2024-09-21T03:39:19.591 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:19.582+0000 7f8251269740 20 client.15405 trim_cache size 0 max 16384 2024-09-21T03:39:19.603 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:19.600+0000 7f8251269740 1 Processor -- start 2024-09-21T03:39:19.603 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:19.600+0000 7f8251269740 1 -- start start 2024-09-21T03:39:19.603 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:19.600+0000 7f8251269740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5564c35898e0 0x5564c3977020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:19.603 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:19.600+0000 7f8251269740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5564c35a20a0 0x5564c39779e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:19.603 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:19.600+0000 7f8251269740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5564c35aa280 0x5564c3977f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:19.604 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:19.600+0000 7f8251269740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x5564c398d1e0 con 0x5564c35a20a0 2024-09-21T03:39:19.604 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:19.600+0000 7f8251269740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x5564c398d8b0 con 0x5564c35898e0 2024-09-21T03:39:19.604 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:19.600+0000 7f8251269740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x5564c398df80 con 0x5564c35aa280 2024-09-21T03:39:19.604 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:19.600+0000 7f823bfff640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5564c35a20a0 0x5564c39779e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:19.604 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:19.600+0000 7f823bfff640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5564c35a20a0 0x5564c39779e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:53258/0 (socket says 172.21.15.89:53258) 2024-09-21T03:39:19.604 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:19.600+0000 7f823bfff640 1 -- 172.21.15.89:0/981087836 learned_addr learned my addr 172.21.15.89:0/981087836 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-21T03:39:19.604 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:19.601+0000 7f8248a91640 1 --2- 172.21.15.89:0/981087836 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5564c35898e0 0x5564c3977020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:19.604 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:19.601+0000 7f823bfff640 1 -- 172.21.15.89:0/981087836 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5564c35aa280 msgr2=0x5564c3977f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:39:19.604 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:19.601+0000 7f823bfff640 1 --2- 172.21.15.89:0/981087836 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5564c35aa280 0x5564c3977f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:19.604 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:19.601+0000 7f823bfff640 1 -- 172.21.15.89:0/981087836 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5564c35898e0 msgr2=0x5564c3977020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:39:19.604 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:19.601+0000 7f823bfff640 1 --2- 172.21.15.89:0/981087836 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5564c35898e0 0x5564c3977020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:19.604 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:19.601+0000 7f823bfff640 1 -- 172.21.15.89:0/981087836 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5564c3969080 con 0x5564c35a20a0 2024-09-21T03:39:19.604 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:19.601+0000 7f8248a91640 1 --2- 172.21.15.89:0/981087836 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5564c35898e0 0x5564c3977020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:39:19.605 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:19.601+0000 7f823bfff640 1 --2- 172.21.15.89:0/981087836 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5564c35a20a0 0x5564c39779e0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f823402a930 tx=0x7f823403f070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e427a4aa3ce9b8 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:39:19.605 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:19.601+0000 7f822a7fc640 1 -- 172.21.15.89:0/981087836 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8234035040 con 0x5564c35a20a0 2024-09-21T03:39:19.605 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:19.601+0000 7f822a7fc640 1 -- 172.21.15.89:0/981087836 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8234059070 con 0x5564c35a20a0 2024-09-21T03:39:19.605 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:19.601+0000 7f822a7fc640 1 -- 172.21.15.89:0/981087836 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f823403d450 con 0x5564c35a20a0 2024-09-21T03:39:19.609 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:19.602+0000 7f8251269740 1 -- 172.21.15.89:0/981087836 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5564c35a20a0 msgr2=0x5564c39779e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:19.609 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:19.602+0000 7f8251269740 1 --2- 172.21.15.89:0/981087836 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5564c35a20a0 0x5564c39779e0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f823402a930 tx=0x7f823403f070 comp rx=0 tx=0).stop 2024-09-21T03:39:19.609 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:19.602+0000 7f8251269740 1 -- 172.21.15.89:0/981087836 shutdown_connections 2024-09-21T03:39:19.609 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:19.602+0000 7f8251269740 1 --2- 172.21.15.89:0/981087836 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5564c35aa280 0x5564c3977f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:19.609 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:19.602+0000 7f8251269740 1 --2- 172.21.15.89:0/981087836 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5564c35a20a0 0x5564c39779e0 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:19.609 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:19.602+0000 7f8251269740 1 --2- 172.21.15.89:0/981087836 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5564c35898e0 0x5564c3977020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:19.609 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:19.602+0000 7f8251269740 1 -- 172.21.15.89:0/981087836 >> 172.21.15.89:0/981087836 conn(0x5564c35bf350 msgr2=0x5564c397c9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:39:19.609 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:19.602+0000 7f8251269740 1 -- 172.21.15.89:0/981087836 shutdown_connections 2024-09-21T03:39:19.609 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:19.602+0000 7f8251269740 1 -- 172.21.15.89:0/981087836 wait complete. 2024-09-21T03:39:19.609 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:19.602+0000 7f8251269740 1 -- start start 2024-09-21T03:39:19.609 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:19.602+0000 7f8251269740 1 Processor -- start 2024-09-21T03:39:19.609 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:19.603+0000 7f8251269740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5564c35898e0 0x5564c39908f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:19.610 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:19.603+0000 7f8251269740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5564c35a20a0 0x5564c3990eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:19.610 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:19.603+0000 7f8251269740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5564c35aa280 0x5564c3995ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:19.610 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:19.603+0000 7f8251269740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x5564c398ed20 con 0x5564c35aa280 2024-09-21T03:39:19.610 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:19.603+0000 7f8251269740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x5564c398e650 con 0x5564c35a20a0 2024-09-21T03:39:19.610 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:19.603+0000 7f8251269740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x5564c398f3f0 con 0x5564c35898e0 2024-09-21T03:39:19.610 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:19.603+0000 7f8251269740 10 client.0 _ll_register_callbacks cb 0x7f8240005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:39:19.610 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:19.603+0000 7f8249292640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5564c35aa280 0x5564c3995ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:19.610 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:19.603+0000 7f8249292640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5564c35aa280 0x5564c3995ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:53270/0 (socket says 172.21.15.89:53270) 2024-09-21T03:39:19.610 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:19.603+0000 7f8249292640 1 -- 172.21.15.89:0/2481614828 learned_addr learned my addr 172.21.15.89:0/2481614828 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-21T03:39:19.610 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:19.603+0000 7f823bfff640 1 --2- 172.21.15.89:0/2481614828 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5564c35a20a0 0x5564c3990eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:19.610 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:19.603+0000 7f8248a91640 1 --2- 172.21.15.89:0/2481614828 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5564c35898e0 0x5564c39908f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:19.610 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:19.603+0000 7f8249292640 1 -- 172.21.15.89:0/2481614828 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5564c35898e0 msgr2=0x5564c39908f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:39:19.610 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:19.603+0000 7f8249292640 1 --2- 172.21.15.89:0/2481614828 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5564c35898e0 0x5564c39908f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:19.610 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:19.603+0000 7f8249292640 1 -- 172.21.15.89:0/2481614828 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5564c35a20a0 msgr2=0x5564c3990eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:39:19.611 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:19.603+0000 7f8249292640 1 --2- 172.21.15.89:0/2481614828 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5564c35a20a0 0x5564c3990eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:19.611 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:19.603+0000 7f8249292640 1 -- 172.21.15.89:0/2481614828 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5564c3996560 con 0x5564c35aa280 2024-09-21T03:39:19.611 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:19.603+0000 7f8248a91640 1 --2- 172.21.15.89:0/2481614828 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5564c35898e0 0x5564c39908f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:39:19.611 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:19.603+0000 7f8249292640 1 --2- 172.21.15.89:0/2481614828 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5564c35aa280 0x5564c3995ef0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f823c0a9e20 tx=0x7f823c005720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f59b225be170d0a9 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:39:19.611 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:19.603+0000 7f8223fff640 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2024-09-21T03:39:19.611 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:19.604+0000 7f8223fff640 1 -- 172.21.15.89:0/2481614828 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f823c01f070 con 0x5564c35aa280 2024-09-21T03:39:19.611 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:19.604+0000 7f8223fff640 1 -- 172.21.15.89:0/2481614828 <== mon.0 v2:172.21.15.89:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 9121+0+0 (secure 0 0 0) 0x7f823c0a8fb0 con 0x5564c35aa280 2024-09-21T03:39:19.611 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:19.604+0000 7f8251269740 1 -- 172.21.15.89:0/2481614828 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8234059070 con 0x5564c35aa280 2024-09-21T03:39:19.611 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:19.604+0000 7f8223fff640 1 -- 172.21.15.89:0/2481614828 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f823c0b2140 con 0x5564c35aa280 2024-09-21T03:39:19.611 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:19.604+0000 7f8251269740 10 client.15411 fetch_fsmap learned FSMap version 25 2024-09-21T03:39:19.611 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:19.604+0000 7f8251269740 1 -- 172.21.15.89:0/2481614828 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5564c3997890 con 0x5564c35aa280 2024-09-21T03:39:19.612 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:19.604+0000 7f8223fff640 1 -- 172.21.15.89:0/2481614828 <== mon.0 v2:172.21.15.89:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8234059070 con 0x5564c35aa280 2024-09-21T03:39:19.612 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:19.604+0000 7f8223fff640 1 -- 172.21.15.89:0/2481614828 <== mon.0 v2:172.21.15.89:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f823c00f040 con 0x5564c35aa280 2024-09-21T03:39:19.612 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:19.604+0000 7f8251269740 10 client.15411 fetch_fsmap finished waiting for FSMap version 25 2024-09-21T03:39:19.612 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:19.604+0000 7f8251269740 10 client.15411 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-21T03:39:19.612 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:19.604+0000 7f8251269740 20 client.15411 populate_metadata read hostname 'smithi089' 2024-09-21T03:39:19.612 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:19.604+0000 7f8251269740 1 --2- 172.21.15.89:0/2481614828 >> [v2:172.21.15.110:6838/132366709,v1:172.21.15.110:6839/132366709] conn(0x5564c35caf60 0x5564c39fd420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:19.612 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:19.604+0000 7f8251269740 4 client.15411 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-21T03:39:19.612 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:19.604+0000 7f8251269740 1 -- 172.21.15.89:0/2481614828 --> [v2:172.21.15.110:6838/132366709,v1:172.21.15.110:6839/132366709] -- command(tid 0: {"prefix": "status"}) -- 0x5564c38e6610 con 0x5564c35caf60 2024-09-21T03:39:19.612 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:19.605+0000 7f8248a91640 1 --2- 172.21.15.89:0/2481614828 >> [v2:172.21.15.110:6838/132366709,v1:172.21.15.110:6839/132366709] conn(0x5564c35caf60 0x5564c39fd420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:19.612 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:19.605+0000 7f8223fff640 10 client.15411 ms_handle_connect on v2:172.21.15.110:6838/132366709 2024-09-21T03:39:19.612 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:19.605+0000 7f8248a91640 1 --2- 172.21.15.89:0/2481614828 >> [v2:172.21.15.110:6838/132366709,v1:172.21.15.110:6839/132366709] conn(0x5564c35caf60 0x5564c39fd420 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=11b9255edda5000 server_cookie=f6ba55542b8ab725 in_seq=0 out_seq=0 2024-09-21T03:39:19.612 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:19.605+0000 7f8223fff640 1 -- 172.21.15.89:0/2481614828 <== mds.0 v2:172.21.15.110:6838/132366709 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x5564c38e6610 con 0x5564c35caf60 2024-09-21T03:39:19.612 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:19.605+0000 7f8223fff640 10 client.15411 handle_command_reply: tid=0 2024-09-21T03:39:19.613 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:19.605+0000 7f8251269740 1 client.15411 shutdown 2024-09-21T03:39:19.613 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:19.605+0000 7f8251269740 1 -- 172.21.15.89:0/2481614828 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5564c35aa280 msgr2=0x5564c3995ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:19.613 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:19.605+0000 7f8251269740 1 --2- 172.21.15.89:0/2481614828 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5564c35aa280 0x5564c3995ef0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f823c0a9e20 tx=0x7f823c005720 comp rx=0 tx=0).stop 2024-09-21T03:39:19.613 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:19.605+0000 7f8251269740 1 -- 172.21.15.89:0/2481614828 shutdown_connections 2024-09-21T03:39:19.613 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:19.605+0000 7f8251269740 1 --2- 172.21.15.89:0/2481614828 >> [v2:172.21.15.110:6838/132366709,v1:172.21.15.110:6839/132366709] conn(0x5564c35caf60 0x5564c39fd420 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:19.613 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:19.605+0000 7f8251269740 1 --2- 172.21.15.89:0/2481614828 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5564c35a20a0 0x5564c3990eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:19.613 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:19.605+0000 7f8251269740 1 --2- 172.21.15.89:0/2481614828 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5564c35aa280 0x5564c3995ef0 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:19.613 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:19.605+0000 7f8223fff640 0 client.15411 ms_handle_reset on v2:172.21.15.110:6838/132366709 2024-09-21T03:39:19.613 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:19.605+0000 7f8251269740 1 --2- 172.21.15.89:0/2481614828 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5564c35898e0 0x5564c39908f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:19.613 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:19.605+0000 7f8251269740 1 -- 172.21.15.89:0/2481614828 >> 172.21.15.89:0/2481614828 conn(0x5564c35bf350 msgr2=0x5564c397c9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:39:19.613 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:19.606+0000 7f8251269740 1 -- 172.21.15.89:0/2481614828 shutdown_connections 2024-09-21T03:39:19.613 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:19.606+0000 7f8251269740 1 -- 172.21.15.89:0/2481614828 wait complete. 2024-09-21T03:39:19.613 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:19.606+0000 7f8251269740 20 client.15411 trim_cache size 0 max 16384 2024-09-21T03:39:19.614 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-09-21T03:39:19.614 INFO:teuthology.orchestra.run.smithi089.stdout: "cluster_fsid": "7c374098-77c9-11ef-baf5-efdc52797490", 2024-09-21T03:39:19.614 INFO:teuthology.orchestra.run.smithi089.stdout: "whoami": 0, 2024-09-21T03:39:19.614 INFO:teuthology.orchestra.run.smithi089.stdout: "id": 24638, 2024-09-21T03:39:19.614 INFO:teuthology.orchestra.run.smithi089.stdout: "want_state": "up:active", 2024-09-21T03:39:19.614 INFO:teuthology.orchestra.run.smithi089.stdout: "state": "up:active", 2024-09-21T03:39:19.614 INFO:teuthology.orchestra.run.smithi089.stdout: "fs_name": "cephfs", 2024-09-21T03:39:19.614 INFO:teuthology.orchestra.run.smithi089.stdout: "rank_uptime": 125.75253645399999, 2024-09-21T03:39:19.614 INFO:teuthology.orchestra.run.smithi089.stdout: "mdsmap_epoch": 25, 2024-09-21T03:39:19.614 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap_epoch": 84, 2024-09-21T03:39:19.614 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap_epoch_barrier": 80, 2024-09-21T03:39:19.614 INFO:teuthology.orchestra.run.smithi089.stdout: "uptime": 136.220317932 2024-09-21T03:39:19.614 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-09-21T03:39:19.615 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:19.606+0000 7f824acf5640 1 -- 172.21.15.89:0/2242522540 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f822c0b65a0 msgr2=0x7f822c0b8ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:19.615 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:19.606+0000 7f824acf5640 1 --2- 172.21.15.89:0/2242522540 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f822c0b65a0 0x7f822c0b8ab0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f8234006010 tx=0x7f82340058b0 comp rx=0 tx=0).stop 2024-09-21T03:39:19.615 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:19.606+0000 7f824acf5640 1 -- 172.21.15.89:0/2242522540 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8244109ec0 msgr2=0x7f824419fcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:19.615 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:19.606+0000 7f824acf5640 1 --2- 172.21.15.89:0/2242522540 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8244109ec0 0x7f824419fcc0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f823c00eaa0 tx=0x7f823c00ef60 comp rx=0 tx=0).stop 2024-09-21T03:39:19.615 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:19.606+0000 7f824acf5640 1 -- 172.21.15.89:0/2242522540 shutdown_connections 2024-09-21T03:39:19.615 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:19.606+0000 7f824acf5640 1 --2- 172.21.15.89:0/2242522540 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f822c0b65a0 0x7f822c0b8ab0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:19.615 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:19.606+0000 7f824acf5640 1 --2- 172.21.15.89:0/2242522540 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8244109ec0 0x7f824419fcc0 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:19.615 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:19.606+0000 7f824acf5640 1 --2- 172.21.15.89:0/2242522540 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8244109570 0x7f824419aa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:19.616 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:19.606+0000 7f824acf5640 1 --2- 172.21.15.89:0/2242522540 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8244103c30 0x7f824419a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:19.616 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:19.606+0000 7f824acf5640 1 -- 172.21.15.89:0/2242522540 >> 172.21.15.89:0/2242522540 conn(0x7f824407b8d0 msgr2=0x7f82441076d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:39:19.616 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:19.606+0000 7f824acf5640 1 -- 172.21.15.89:0/2242522540 shutdown_connections 2024-09-21T03:39:19.616 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:19.606+0000 7f824acf5640 1 -- 172.21.15.89:0/2242522540 wait complete. 2024-09-21T03:39:20.332 INFO:teuthology.run_tasks:Running task ceph-fuse... 2024-09-21T03:39:20.346 INFO:tasks.ceph_fuse:Running ceph_fuse task... 2024-09-21T03:39:20.346 INFO:tasks.ceph_fuse:config is {'client.0': {}} 2024-09-21T03:39:20.346 INFO:tasks.ceph_fuse:client.0 config is {'mount_subvol_num': 0} 2024-09-21T03:39:20.346 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-09-21T03:39:20.346 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-21T03:39:20.346 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:39:20.346 DEBUG:teuthology.orchestra.run.smithi089:> ip netns list 2024-09-21T03:39:20.362 INFO:teuthology.orchestra.run.smithi089.stdout:ceph-ns--home-ubuntu-cephtest-mnt.admin (id: 0) 2024-09-21T03:39:20.363 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:39:20.363 DEBUG:teuthology.orchestra.run.smithi089:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-09-21T03:39:20.430 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:39:20.430 DEBUG:teuthology.orchestra.run.smithi089:> sudo ip link delete ceph-brx 2024-09-21T03:39:20.540 INFO:tasks.ceph_fuse:Mounting ceph-fuse clients... 2024-09-21T03:39:20.541 DEBUG:tasks.ceph_fuse:passing mntargs=[] 2024-09-21T03:39:20.541 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 7c374098-77c9-11ef-baf5-efdc52797490 -- ceph fs ls 2024-09-21T03:39:20.806 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/7c374098-77c9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:39:21.168 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:39:21 smithi089 ceph-mon[30928]: pgmap v358: 129 pgs: 129 active+clean; 646 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-21T03:39:21.276 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:21.257+0000 7fc9df670640 1 -- 172.21.15.89:0/1774359396 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc9d8102a60 msgr2=0x7fc9d8102e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:21.276 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:21.257+0000 7fc9df670640 1 --2- 172.21.15.89:0/1774359396 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc9d8102a60 0x7fc9d8102e70 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fc9d4009900 tx=0x7fc9d402a410 comp rx=0 tx=0).stop 2024-09-21T03:39:21.276 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:21.257+0000 7fc9df670640 1 -- 172.21.15.89:0/1774359396 shutdown_connections 2024-09-21T03:39:21.276 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:21.257+0000 7fc9df670640 1 --2- 172.21.15.89:0/1774359396 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc9d8108cf0 0x7fc9d810f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:21.276 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:21.257+0000 7fc9df670640 1 --2- 172.21.15.89:0/1774359396 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc9d81083a0 0x7fc9d81087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:21.276 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:21.257+0000 7fc9df670640 1 --2- 172.21.15.89:0/1774359396 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc9d8102a60 0x7fc9d8102e70 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:21.276 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:21.257+0000 7fc9df670640 1 -- 172.21.15.89:0/1774359396 >> 172.21.15.89:0/1774359396 conn(0x7fc9d8079730 msgr2=0x7fc9d8079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:39:21.276 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:21.257+0000 7fc9df670640 1 -- 172.21.15.89:0/1774359396 shutdown_connections 2024-09-21T03:39:21.276 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:21.257+0000 7fc9df670640 1 -- 172.21.15.89:0/1774359396 wait complete. 2024-09-21T03:39:21.276 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:21.258+0000 7fc9df670640 1 Processor -- start 2024-09-21T03:39:21.276 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:21.258+0000 7fc9df670640 1 -- start start 2024-09-21T03:39:21.277 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:21.258+0000 7fc9df670640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc9d8102a60 0x7fc9d8199300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:21.277 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:21.258+0000 7fc9df670640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc9d81083a0 0x7fc9d81998a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:21.277 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:21.258+0000 7fc9df670640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc9d8108cf0 0x7fc9d819eb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:21.277 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:21.258+0000 7fc9df670640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fc9d8112810 con 0x7fc9d81083a0 2024-09-21T03:39:21.277 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:21.258+0000 7fc9df670640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fc9d8112690 con 0x7fc9d8108cf0 2024-09-21T03:39:21.277 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:21.258+0000 7fc9df670640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fc9d8112990 con 0x7fc9d8102a60 2024-09-21T03:39:21.277 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:21.258+0000 7fc9dcc0b640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc9d81083a0 0x7fc9d81998a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:21.277 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:21.258+0000 7fc9dcc0b640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc9d81083a0 0x7fc9d81998a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:53292/0 (socket says 172.21.15.89:53292) 2024-09-21T03:39:21.277 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:21.258+0000 7fc9dcc0b640 1 -- 172.21.15.89:0/3276823920 learned_addr learned my addr 172.21.15.89:0/3276823920 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-21T03:39:21.277 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:21.258+0000 7fc9dd40c640 1 --2- 172.21.15.89:0/3276823920 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc9d8102a60 0x7fc9d8199300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:21.277 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:21.258+0000 7fc9ddc0d640 1 --2- 172.21.15.89:0/3276823920 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc9d8108cf0 0x7fc9d819eb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:21.277 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:21.259+0000 7fc9dcc0b640 1 -- 172.21.15.89:0/3276823920 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc9d8102a60 msgr2=0x7fc9d8199300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:21.278 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:21.259+0000 7fc9dcc0b640 1 --2- 172.21.15.89:0/3276823920 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc9d8102a60 0x7fc9d8199300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:21.278 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:21.259+0000 7fc9dcc0b640 1 -- 172.21.15.89:0/3276823920 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc9d8108cf0 msgr2=0x7fc9d819eb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:21.278 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:21.259+0000 7fc9dcc0b640 1 --2- 172.21.15.89:0/3276823920 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc9d8108cf0 0x7fc9d819eb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:21.278 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:21.259+0000 7fc9dcc0b640 1 -- 172.21.15.89:0/3276823920 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9d819f270 con 0x7fc9d81083a0 2024-09-21T03:39:21.278 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:21.259+0000 7fc9dcc0b640 1 --2- 172.21.15.89:0/3276823920 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc9d81083a0 0x7fc9d81998a0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7fc9c8009830 tx=0x7fc9c8009cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:39:21.278 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:21.259+0000 7fc9ce7fc640 1 -- 172.21.15.89:0/3276823920 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9c8005020 con 0x7fc9d81083a0 2024-09-21T03:39:21.278 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:21.259+0000 7fc9df670640 1 -- 172.21.15.89:0/3276823920 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc9d819f560 con 0x7fc9d81083a0 2024-09-21T03:39:21.278 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:21.259+0000 7fc9ce7fc640 1 -- 172.21.15.89:0/3276823920 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc9c80047a0 con 0x7fc9d81083a0 2024-09-21T03:39:21.278 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:21.259+0000 7fc9ce7fc640 1 -- 172.21.15.89:0/3276823920 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9c8007a00 con 0x7fc9d81083a0 2024-09-21T03:39:21.279 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:21.259+0000 7fc9df670640 1 -- 172.21.15.89:0/3276823920 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9d806ae60 con 0x7fc9d81083a0 2024-09-21T03:39:21.279 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:21.260+0000 7fc9df670640 1 -- 172.21.15.89:0/3276823920 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc9d8103ce0 con 0x7fc9d81083a0 2024-09-21T03:39:21.279 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:21.263+0000 7fc9ce7fc640 1 -- 172.21.15.89:0/3276823920 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 18) ==== 154563+0+0 (secure 0 0 0) 0x7fc9c8004950 con 0x7fc9d81083a0 2024-09-21T03:39:21.279 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:21.263+0000 7fc9ce7fc640 1 --2- 172.21.15.89:0/3276823920 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7fc9c40b65a0 0x7fc9c40b8ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:21.279 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:21.263+0000 7fc9dd40c640 1 --2- 172.21.15.89:0/3276823920 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7fc9c40b65a0 0x7fc9c40b8ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:21.279 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:21.263+0000 7fc9ce7fc640 1 -- 172.21.15.89:0/3276823920 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 9121+0+0 (secure 0 0 0) 0x7fc9c80e9320 con 0x7fc9d81083a0 2024-09-21T03:39:21.279 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:21.264+0000 7fc9dd40c640 1 --2- 172.21.15.89:0/3276823920 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7fc9c40b65a0 0x7fc9c40b8ab0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fc9d40057c0 tx=0x7fc9d40058d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:39:21.279 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:21.266+0000 7fc9ce7fc640 1 -- 172.21.15.89:0/3276823920 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fc9c80b3b50 con 0x7fc9d81083a0 2024-09-21T03:39:21.461 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:39:21 smithi114 ceph-mon[34140]: pgmap v358: 129 pgs: 129 active+clean; 646 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-21T03:39:21.495 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:21.493+0000 7fc9df670640 1 -- 172.21.15.89:0/3276823920 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fc9d8102e70 con 0x7fc9d81083a0 2024-09-21T03:39:21.495 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:21.493+0000 7fc9ce7fc640 1 -- 172.21.15.89:0/3276823920 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v25) ==== 53+0+88 (secure 0 0 0) 0x7fc9c80b8a90 con 0x7fc9d81083a0 2024-09-21T03:39:21.495 INFO:teuthology.orchestra.run.smithi089.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-09-21T03:39:21.498 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:21.495+0000 7fc9df670640 1 -- 172.21.15.89:0/3276823920 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7fc9c40b65a0 msgr2=0x7fc9c40b8ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:21.498 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:21.495+0000 7fc9df670640 1 --2- 172.21.15.89:0/3276823920 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7fc9c40b65a0 0x7fc9c40b8ab0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fc9d40057c0 tx=0x7fc9d40058d0 comp rx=0 tx=0).stop 2024-09-21T03:39:21.498 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:21.495+0000 7fc9df670640 1 -- 172.21.15.89:0/3276823920 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc9d81083a0 msgr2=0x7fc9d81998a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:21.498 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:21.495+0000 7fc9df670640 1 --2- 172.21.15.89:0/3276823920 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc9d81083a0 0x7fc9d81998a0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7fc9c8009830 tx=0x7fc9c8009cf0 comp rx=0 tx=0).stop 2024-09-21T03:39:21.498 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:21.495+0000 7fc9df670640 1 -- 172.21.15.89:0/3276823920 shutdown_connections 2024-09-21T03:39:21.498 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:21.495+0000 7fc9df670640 1 --2- 172.21.15.89:0/3276823920 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7fc9c40b65a0 0x7fc9c40b8ab0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:21.498 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:21.495+0000 7fc9df670640 1 --2- 172.21.15.89:0/3276823920 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc9d8108cf0 0x7fc9d819eb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:21.498 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:21.495+0000 7fc9df670640 1 --2- 172.21.15.89:0/3276823920 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc9d81083a0 0x7fc9d81998a0 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:21.498 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:21.495+0000 7fc9df670640 1 --2- 172.21.15.89:0/3276823920 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc9d8102a60 0x7fc9d8199300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:21.498 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:21.495+0000 7fc9df670640 1 -- 172.21.15.89:0/3276823920 >> 172.21.15.89:0/3276823920 conn(0x7fc9d8079730 msgr2=0x7fc9d8101560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:39:21.498 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:21.496+0000 7fc9df670640 1 -- 172.21.15.89:0/3276823920 shutdown_connections 2024-09-21T03:39:21.499 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:21.496+0000 7fc9df670640 1 -- 172.21.15.89:0/3276823920 wait complete. 2024-09-21T03:39:21.500 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:39:21 smithi110 ceph-mon[34000]: pgmap v358: 129 pgs: 129 active+clean; 646 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-21T03:39:22.181 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:39:22 smithi089 ceph-mon[30928]: from='client.? 172.21.15.89:0/3276823920' entity='client.admin' cmd={"prefix": "fs ls"} : dispatch 2024-09-21T03:39:22.209 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-21T03:39:22.209 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-21T03:39:22.209 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi089.front.sepia.ceph.com 2024-09-21T03:39:22.209 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-21T03:39:22.209 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-21T03:39:22.209 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-21T03:39:22.209 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-21T03:39:22.209 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-21T03:39:22.210 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-21T03:39:22.210 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:39:22.210 DEBUG:teuthology.orchestra.run.smithi089:> ip addr 2024-09-21T03:39:22.228 INFO:teuthology.orchestra.run.smithi089.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-21T03:39:22.228 INFO:teuthology.orchestra.run.smithi089.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-21T03:39:22.228 INFO:teuthology.orchestra.run.smithi089.stdout: inet 127.0.0.1/8 scope host lo 2024-09-21T03:39:22.228 INFO:teuthology.orchestra.run.smithi089.stdout: valid_lft forever preferred_lft forever 2024-09-21T03:39:22.228 INFO:teuthology.orchestra.run.smithi089.stdout: inet6 ::1/128 scope host 2024-09-21T03:39:22.228 INFO:teuthology.orchestra.run.smithi089.stdout: valid_lft forever preferred_lft forever 2024-09-21T03:39:22.228 INFO:teuthology.orchestra.run.smithi089.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T03:39:22.228 INFO:teuthology.orchestra.run.smithi089.stdout: link/ether 0c:c4:7a:d8:a4:c6 brd ff:ff:ff:ff:ff:ff 2024-09-21T03:39:22.228 INFO:teuthology.orchestra.run.smithi089.stdout: altname enp7s0f0 2024-09-21T03:39:22.228 INFO:teuthology.orchestra.run.smithi089.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T03:39:22.228 INFO:teuthology.orchestra.run.smithi089.stdout: link/ether 0c:c4:7a:d8:a4:c7 brd ff:ff:ff:ff:ff:ff 2024-09-21T03:39:22.229 INFO:teuthology.orchestra.run.smithi089.stdout: altname enp7s0f1 2024-09-21T03:39:22.229 INFO:teuthology.orchestra.run.smithi089.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T03:39:22.229 INFO:teuthology.orchestra.run.smithi089.stdout: link/ether 0c:c4:7a:8f:d5:00 brd ff:ff:ff:ff:ff:ff 2024-09-21T03:39:22.229 INFO:teuthology.orchestra.run.smithi089.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-21T03:39:22.229 INFO:teuthology.orchestra.run.smithi089.stdout: link/ether 0c:c4:7a:8f:d5:01 brd ff:ff:ff:ff:ff:ff 2024-09-21T03:39:22.229 INFO:teuthology.orchestra.run.smithi089.stdout: inet 172.21.15.89/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-09-21T03:39:22.229 INFO:teuthology.orchestra.run.smithi089.stdout: valid_lft 42159sec preferred_lft 42159sec 2024-09-21T03:39:22.229 INFO:teuthology.orchestra.run.smithi089.stdout: inet6 fe80::ec4:7aff:fe8f:d501/64 scope link noprefixroute 2024-09-21T03:39:22.229 INFO:teuthology.orchestra.run.smithi089.stdout: valid_lft forever preferred_lft forever 2024-09-21T03:39:22.229 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-09-21T03:39:22.229 DEBUG:teuthology.orchestra.run.smithi089:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T03:39:22.229 DEBUG:teuthology.orchestra.run.smithi089:> set -e 2024-09-21T03:39:22.230 DEBUG:teuthology.orchestra.run.smithi089:> sudo ip link add name ceph-brx type bridge 2024-09-21T03:39:22.230 DEBUG:teuthology.orchestra.run.smithi089:> sudo ip addr flush dev ceph-brx 2024-09-21T03:39:22.230 DEBUG:teuthology.orchestra.run.smithi089:> sudo ip link set ceph-brx up 2024-09-21T03:39:22.230 DEBUG:teuthology.orchestra.run.smithi089:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-09-21T03:39:22.230 DEBUG:teuthology.orchestra.run.smithi089:> ') 2024-09-21T03:39:22.316 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:22 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T03:39:22.361 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:22 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T03:39:22.367 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:39:22.368 DEBUG:teuthology.orchestra.run.smithi089:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-21T03:39:22.391 INFO:teuthology.orchestra.run.smithi089.stdout:1 2024-09-21T03:39:22.393 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:39:22.393 DEBUG:teuthology.orchestra.run.smithi089:> route 2024-09-21T03:39:22.450 INFO:teuthology.orchestra.run.smithi089.stdout:Kernel IP routing table 2024-09-21T03:39:22.450 INFO:teuthology.orchestra.run.smithi089.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-21T03:39:22.450 INFO:teuthology.orchestra.run.smithi089.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-21T03:39:22.450 INFO:teuthology.orchestra.run.smithi089.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-21T03:39:22.450 INFO:teuthology.orchestra.run.smithi089.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-21T03:39:22.450 DEBUG:teuthology.orchestra.run.smithi089:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T03:39:22.450 DEBUG:teuthology.orchestra.run.smithi089:> set -e 2024-09-21T03:39:22.450 DEBUG:teuthology.orchestra.run.smithi089:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-21T03:39:22.450 DEBUG:teuthology.orchestra.run.smithi089:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-21T03:39:22.450 DEBUG:teuthology.orchestra.run.smithi089:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-21T03:39:22.451 DEBUG:teuthology.orchestra.run.smithi089:> ') 2024-09-21T03:39:22.460 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:39:22 smithi114 ceph-mon[34140]: from='client.? 172.21.15.89:0/3276823920' entity='client.admin' cmd={"prefix": "fs ls"} : dispatch 2024-09-21T03:39:22.500 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:39:22 smithi110 ceph-mon[34000]: from='client.? 172.21.15.89:0/3276823920' entity='client.admin' cmd={"prefix": "fs ls"} : dispatch 2024-09-21T03:39:22.538 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:22 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T03:39:22.571 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:22 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T03:39:22.577 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:39:22.578 DEBUG:teuthology.orchestra.run.smithi089:> ip netns list 2024-09-21T03:39:22.593 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:39:22.593 DEBUG:teuthology.orchestra.run.smithi089:> ip netns list-id 2024-09-21T03:39:22.648 DEBUG:teuthology.orchestra.run.smithi089:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T03:39:22.648 DEBUG:teuthology.orchestra.run.smithi089:> set -e 2024-09-21T03:39:22.648 DEBUG:teuthology.orchestra.run.smithi089:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-21T03:39:22.648 DEBUG:teuthology.orchestra.run.smithi089:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-09-21T03:39:22.648 DEBUG:teuthology.orchestra.run.smithi089:> ') 2024-09-21T03:39:22.735 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:22 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T03:39:22.758 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:22 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T03:39:22.764 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-09-21T03:39:22.764 DEBUG:teuthology.orchestra.run.smithi089:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T03:39:22.764 DEBUG:teuthology.orchestra.run.smithi089:> set -e 2024-09-21T03:39:22.764 DEBUG:teuthology.orchestra.run.smithi089:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-09-21T03:39:22.764 DEBUG:teuthology.orchestra.run.smithi089:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-09-21T03:39:22.764 DEBUG:teuthology.orchestra.run.smithi089:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-09-21T03:39:22.764 DEBUG:teuthology.orchestra.run.smithi089:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-09-21T03:39:22.764 DEBUG:teuthology.orchestra.run.smithi089:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-09-21T03:39:22.764 DEBUG:teuthology.orchestra.run.smithi089:> ') 2024-09-21T03:39:22.812 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:22 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T03:39:22.870 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:22 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T03:39:22.876 DEBUG:teuthology.orchestra.run.smithi089:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T03:39:22.876 DEBUG:teuthology.orchestra.run.smithi089:> set -e 2024-09-21T03:39:22.876 DEBUG:teuthology.orchestra.run.smithi089:> sudo ip link set brx.0 up 2024-09-21T03:39:22.876 DEBUG:teuthology.orchestra.run.smithi089:> sudo ip link set dev brx.0 master ceph-brx 2024-09-21T03:39:22.876 DEBUG:teuthology.orchestra.run.smithi089:> ') 2024-09-21T03:39:22.924 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:22 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T03:39:22.946 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:22 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T03:39:22.952 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {'mount_subvol_num': 0} 2024-09-21T03:39:22.952 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-21T03:39:22.952 DEBUG:teuthology.orchestra.run.smithi089:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-21T03:39:22.967 INFO:teuthology.orchestra.run.smithi089.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-21T03:39:22.967 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-21T03:39:22.967 DEBUG:teuthology.orchestra.run.smithi089:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-21T03:39:23.022 DEBUG:teuthology.orchestra.run.smithi089:> sudo modprobe fuse 2024-09-21T03:39:23.083 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:39:23 smithi089 ceph-mon[30928]: pgmap v359: 129 pgs: 129 active+clean; 646 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-21T03:39:23.088 DEBUG:teuthology.orchestra.run.smithi089:> cat /proc/self/mounts | awk '{print $2}' 2024-09-21T03:39:23.144 INFO:teuthology.orchestra.run.smithi089.stdout:/proc 2024-09-21T03:39:23.144 INFO:teuthology.orchestra.run.smithi089.stdout:/sys 2024-09-21T03:39:23.144 INFO:teuthology.orchestra.run.smithi089.stdout:/dev 2024-09-21T03:39:23.144 INFO:teuthology.orchestra.run.smithi089.stdout:/sys/kernel/security 2024-09-21T03:39:23.145 INFO:teuthology.orchestra.run.smithi089.stdout:/dev/shm 2024-09-21T03:39:23.145 INFO:teuthology.orchestra.run.smithi089.stdout:/dev/pts 2024-09-21T03:39:23.145 INFO:teuthology.orchestra.run.smithi089.stdout:/run 2024-09-21T03:39:23.145 INFO:teuthology.orchestra.run.smithi089.stdout:/sys/fs/cgroup 2024-09-21T03:39:23.145 INFO:teuthology.orchestra.run.smithi089.stdout:/sys/fs/pstore 2024-09-21T03:39:23.145 INFO:teuthology.orchestra.run.smithi089.stdout:/sys/fs/bpf 2024-09-21T03:39:23.145 INFO:teuthology.orchestra.run.smithi089.stdout:/ 2024-09-21T03:39:23.145 INFO:teuthology.orchestra.run.smithi089.stdout:/sys/fs/selinux 2024-09-21T03:39:23.145 INFO:teuthology.orchestra.run.smithi089.stdout:/proc/sys/fs/binfmt_misc 2024-09-21T03:39:23.145 INFO:teuthology.orchestra.run.smithi089.stdout:/sys/kernel/debug 2024-09-21T03:39:23.145 INFO:teuthology.orchestra.run.smithi089.stdout:/dev/mqueue 2024-09-21T03:39:23.145 INFO:teuthology.orchestra.run.smithi089.stdout:/sys/kernel/tracing 2024-09-21T03:39:23.145 INFO:teuthology.orchestra.run.smithi089.stdout:/dev/hugepages 2024-09-21T03:39:23.145 INFO:teuthology.orchestra.run.smithi089.stdout:/sys/fs/fuse/connections 2024-09-21T03:39:23.145 INFO:teuthology.orchestra.run.smithi089.stdout:/sys/kernel/config 2024-09-21T03:39:23.146 INFO:teuthology.orchestra.run.smithi089.stdout:/run/credentials/systemd-sysctl.service 2024-09-21T03:39:23.146 INFO:teuthology.orchestra.run.smithi089.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-09-21T03:39:23.146 INFO:teuthology.orchestra.run.smithi089.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-09-21T03:39:23.146 INFO:teuthology.orchestra.run.smithi089.stdout:/var/lib/nfs/rpc_pipefs 2024-09-21T03:39:23.146 INFO:teuthology.orchestra.run.smithi089.stdout:/run/user/1000 2024-09-21T03:39:23.146 INFO:teuthology.orchestra.run.smithi089.stdout:/var/lib/ceph 2024-09-21T03:39:23.146 INFO:teuthology.orchestra.run.smithi089.stdout:/var/lib/containers/storage/overlay 2024-09-21T03:39:23.146 INFO:teuthology.orchestra.run.smithi089.stdout:/var/lib/containers/storage/overlay/0b1266360c1e947b91c951aa5a770f203724c9783f3817320d7b3f833fdd3623/merged 2024-09-21T03:39:23.146 INFO:teuthology.orchestra.run.smithi089.stdout:/var/lib/containers/storage/overlay/74f483b2de23aa3db5ef9fda5e7c9ec9d9869ed0e09e152a27d160b31080f0d7/merged 2024-09-21T03:39:23.146 INFO:teuthology.orchestra.run.smithi089.stdout:/run/user/0 2024-09-21T03:39:23.146 INFO:teuthology.orchestra.run.smithi089.stdout:/proc/sys/fs/binfmt_misc 2024-09-21T03:39:23.146 INFO:teuthology.orchestra.run.smithi089.stdout:/var/lib/containers/storage/overlay/0c33d786533fa7c6c9b4ece0dd8c3bfc33ee457df01a35b7a874175fb0d24091/merged 2024-09-21T03:39:23.146 INFO:teuthology.orchestra.run.smithi089.stdout:/var/lib/containers/storage/overlay/1e263b0a542f58eaf76d73e377dd5959e47135a86a85f6eb5eae463c0822e8c2/merged 2024-09-21T03:39:23.146 INFO:teuthology.orchestra.run.smithi089.stdout:/var/lib/containers/storage/overlay/2f136aa314ce51b9e309ae462848aa731dc41b3b9ee884ab006901dd9af81ad8/merged 2024-09-21T03:39:23.147 INFO:teuthology.orchestra.run.smithi089.stdout:/var/lib/containers/storage/overlay/64f7a59823e11e80a119f6f784180bf56bca738c73a5be4e7e05673bb3400150/merged 2024-09-21T03:39:23.147 INFO:teuthology.orchestra.run.smithi089.stdout:/var/lib/containers/storage/overlay/cc0f955263cae7210080df6836f384f710eb6af21a08203f2d154f023c022d18/merged 2024-09-21T03:39:23.147 INFO:teuthology.orchestra.run.smithi089.stdout:/var/lib/containers/storage/overlay/9a935757ca5b5da6d9a52aaebf493ea1b50d289d444a1fc297330740dbd3169d/merged 2024-09-21T03:39:23.147 INFO:teuthology.orchestra.run.smithi089.stdout:/var/lib/containers/storage/overlay/050ed91c932a05cf2dc3c502aa3bf5b8822fb5b205b8284f46e11cf4638c07a2/merged 2024-09-21T03:39:23.147 INFO:teuthology.orchestra.run.smithi089.stdout:/var/lib/containers/storage/overlay/34ca3c8ad8cdf365681f5fde2e944c6107cead8c5aea4405ffe1ac85eb695e9c/merged 2024-09-21T03:39:23.147 INFO:teuthology.orchestra.run.smithi089.stdout:/run/netns 2024-09-21T03:39:23.147 INFO:teuthology.orchestra.run.smithi089.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-21T03:39:23.147 INFO:teuthology.orchestra.run.smithi089.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-21T03:39:23.147 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:39:23.147 DEBUG:teuthology.orchestra.run.smithi089:> ls /sys/fs/fuse/connections 2024-09-21T03:39:23.200 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-09-21T03:39:23.200 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0 --client_mountpoint=/volumes/qa/sv_0/d474bdba-6c8a-4432-8887-89da76119cad --client_fs=cephfs) 2024-09-21T03:39:23.242 DEBUG:teuthology.orchestra.run.smithi089:> sudo modprobe fuse 2024-09-21T03:39:23.266 DEBUG:teuthology.orchestra.run.smithi089:> cat /proc/self/mounts | awk '{print $2}' 2024-09-21T03:39:23.323 INFO:teuthology.orchestra.run.smithi089.stdout:/proc 2024-09-21T03:39:23.323 INFO:teuthology.orchestra.run.smithi089.stdout:/sys 2024-09-21T03:39:23.323 INFO:teuthology.orchestra.run.smithi089.stdout:/dev 2024-09-21T03:39:23.324 INFO:teuthology.orchestra.run.smithi089.stdout:/sys/kernel/security 2024-09-21T03:39:23.324 INFO:teuthology.orchestra.run.smithi089.stdout:/dev/shm 2024-09-21T03:39:23.324 INFO:teuthology.orchestra.run.smithi089.stdout:/dev/pts 2024-09-21T03:39:23.324 INFO:teuthology.orchestra.run.smithi089.stdout:/run 2024-09-21T03:39:23.324 INFO:teuthology.orchestra.run.smithi089.stdout:/sys/fs/cgroup 2024-09-21T03:39:23.324 INFO:teuthology.orchestra.run.smithi089.stdout:/sys/fs/pstore 2024-09-21T03:39:23.324 INFO:teuthology.orchestra.run.smithi089.stdout:/sys/fs/bpf 2024-09-21T03:39:23.324 INFO:teuthology.orchestra.run.smithi089.stdout:/ 2024-09-21T03:39:23.324 INFO:teuthology.orchestra.run.smithi089.stdout:/sys/fs/selinux 2024-09-21T03:39:23.324 INFO:teuthology.orchestra.run.smithi089.stdout:/proc/sys/fs/binfmt_misc 2024-09-21T03:39:23.324 INFO:teuthology.orchestra.run.smithi089.stdout:/sys/kernel/debug 2024-09-21T03:39:23.324 INFO:teuthology.orchestra.run.smithi089.stdout:/dev/mqueue 2024-09-21T03:39:23.324 INFO:teuthology.orchestra.run.smithi089.stdout:/sys/kernel/tracing 2024-09-21T03:39:23.324 INFO:teuthology.orchestra.run.smithi089.stdout:/dev/hugepages 2024-09-21T03:39:23.325 INFO:teuthology.orchestra.run.smithi089.stdout:/sys/fs/fuse/connections 2024-09-21T03:39:23.325 INFO:teuthology.orchestra.run.smithi089.stdout:/sys/kernel/config 2024-09-21T03:39:23.325 INFO:teuthology.orchestra.run.smithi089.stdout:/run/credentials/systemd-sysctl.service 2024-09-21T03:39:23.325 INFO:teuthology.orchestra.run.smithi089.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-09-21T03:39:23.325 INFO:teuthology.orchestra.run.smithi089.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-09-21T03:39:23.325 INFO:teuthology.orchestra.run.smithi089.stdout:/var/lib/nfs/rpc_pipefs 2024-09-21T03:39:23.325 INFO:teuthology.orchestra.run.smithi089.stdout:/run/user/1000 2024-09-21T03:39:23.325 INFO:teuthology.orchestra.run.smithi089.stdout:/var/lib/ceph 2024-09-21T03:39:23.325 INFO:teuthology.orchestra.run.smithi089.stdout:/var/lib/containers/storage/overlay 2024-09-21T03:39:23.325 INFO:teuthology.orchestra.run.smithi089.stdout:/var/lib/containers/storage/overlay/0b1266360c1e947b91c951aa5a770f203724c9783f3817320d7b3f833fdd3623/merged 2024-09-21T03:39:23.325 INFO:teuthology.orchestra.run.smithi089.stdout:/var/lib/containers/storage/overlay/74f483b2de23aa3db5ef9fda5e7c9ec9d9869ed0e09e152a27d160b31080f0d7/merged 2024-09-21T03:39:23.325 INFO:teuthology.orchestra.run.smithi089.stdout:/run/user/0 2024-09-21T03:39:23.325 INFO:teuthology.orchestra.run.smithi089.stdout:/proc/sys/fs/binfmt_misc 2024-09-21T03:39:23.325 INFO:teuthology.orchestra.run.smithi089.stdout:/var/lib/containers/storage/overlay/0c33d786533fa7c6c9b4ece0dd8c3bfc33ee457df01a35b7a874175fb0d24091/merged 2024-09-21T03:39:23.325 INFO:teuthology.orchestra.run.smithi089.stdout:/var/lib/containers/storage/overlay/1e263b0a542f58eaf76d73e377dd5959e47135a86a85f6eb5eae463c0822e8c2/merged 2024-09-21T03:39:23.325 INFO:teuthology.orchestra.run.smithi089.stdout:/var/lib/containers/storage/overlay/2f136aa314ce51b9e309ae462848aa731dc41b3b9ee884ab006901dd9af81ad8/merged 2024-09-21T03:39:23.326 INFO:teuthology.orchestra.run.smithi089.stdout:/var/lib/containers/storage/overlay/64f7a59823e11e80a119f6f784180bf56bca738c73a5be4e7e05673bb3400150/merged 2024-09-21T03:39:23.326 INFO:teuthology.orchestra.run.smithi089.stdout:/var/lib/containers/storage/overlay/cc0f955263cae7210080df6836f384f710eb6af21a08203f2d154f023c022d18/merged 2024-09-21T03:39:23.326 INFO:teuthology.orchestra.run.smithi089.stdout:/var/lib/containers/storage/overlay/9a935757ca5b5da6d9a52aaebf493ea1b50d289d444a1fc297330740dbd3169d/merged 2024-09-21T03:39:23.326 INFO:teuthology.orchestra.run.smithi089.stdout:/var/lib/containers/storage/overlay/050ed91c932a05cf2dc3c502aa3bf5b8822fb5b205b8284f46e11cf4638c07a2/merged 2024-09-21T03:39:23.326 INFO:teuthology.orchestra.run.smithi089.stdout:/var/lib/containers/storage/overlay/34ca3c8ad8cdf365681f5fde2e944c6107cead8c5aea4405ffe1ac85eb695e9c/merged 2024-09-21T03:39:23.326 INFO:teuthology.orchestra.run.smithi089.stdout:/run/netns 2024-09-21T03:39:23.326 INFO:teuthology.orchestra.run.smithi089.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-21T03:39:23.326 INFO:teuthology.orchestra.run.smithi089.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-21T03:39:23.326 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:39:23.326 DEBUG:teuthology.orchestra.run.smithi089:> ls /sys/fs/fuse/connections 2024-09-21T03:39:23.334 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi089.stderr:2024-09-21T03:39:23.332+0000 7f9559537480 -1 init, newargv = 0x7f953c0046a0 newargc=15 2024-09-21T03:39:23.334 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi089.stderr:ceph-fuse[84548]: starting ceph client 2024-09-21T03:39:23.349 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi089.stderr:ceph-fuse[84548]: starting fuse 2024-09-21T03:39:23.460 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:39:23 smithi114 ceph-mon[34140]: pgmap v359: 129 pgs: 129 active+clean; 646 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-21T03:39:23.500 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:39:23 smithi110 ceph-mon[34000]: pgmap v359: 129 pgs: 129 active+clean; 646 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-21T03:39:24.450 DEBUG:teuthology.orchestra.run.smithi089:> sudo modprobe fuse 2024-09-21T03:39:24.478 DEBUG:teuthology.orchestra.run.smithi089:> cat /proc/self/mounts | awk '{print $2}' 2024-09-21T03:39:24.536 INFO:teuthology.orchestra.run.smithi089.stdout:/proc 2024-09-21T03:39:24.536 INFO:teuthology.orchestra.run.smithi089.stdout:/sys 2024-09-21T03:39:24.537 INFO:teuthology.orchestra.run.smithi089.stdout:/dev 2024-09-21T03:39:24.537 INFO:teuthology.orchestra.run.smithi089.stdout:/sys/kernel/security 2024-09-21T03:39:24.537 INFO:teuthology.orchestra.run.smithi089.stdout:/dev/shm 2024-09-21T03:39:24.537 INFO:teuthology.orchestra.run.smithi089.stdout:/dev/pts 2024-09-21T03:39:24.537 INFO:teuthology.orchestra.run.smithi089.stdout:/run 2024-09-21T03:39:24.537 INFO:teuthology.orchestra.run.smithi089.stdout:/sys/fs/cgroup 2024-09-21T03:39:24.537 INFO:teuthology.orchestra.run.smithi089.stdout:/sys/fs/pstore 2024-09-21T03:39:24.537 INFO:teuthology.orchestra.run.smithi089.stdout:/sys/fs/bpf 2024-09-21T03:39:24.537 INFO:teuthology.orchestra.run.smithi089.stdout:/ 2024-09-21T03:39:24.537 INFO:teuthology.orchestra.run.smithi089.stdout:/sys/fs/selinux 2024-09-21T03:39:24.537 INFO:teuthology.orchestra.run.smithi089.stdout:/proc/sys/fs/binfmt_misc 2024-09-21T03:39:24.537 INFO:teuthology.orchestra.run.smithi089.stdout:/sys/kernel/debug 2024-09-21T03:39:24.537 INFO:teuthology.orchestra.run.smithi089.stdout:/dev/mqueue 2024-09-21T03:39:24.538 INFO:teuthology.orchestra.run.smithi089.stdout:/sys/kernel/tracing 2024-09-21T03:39:24.538 INFO:teuthology.orchestra.run.smithi089.stdout:/dev/hugepages 2024-09-21T03:39:24.538 INFO:teuthology.orchestra.run.smithi089.stdout:/sys/fs/fuse/connections 2024-09-21T03:39:24.538 INFO:teuthology.orchestra.run.smithi089.stdout:/sys/kernel/config 2024-09-21T03:39:24.538 INFO:teuthology.orchestra.run.smithi089.stdout:/run/credentials/systemd-sysctl.service 2024-09-21T03:39:24.538 INFO:teuthology.orchestra.run.smithi089.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-09-21T03:39:24.538 INFO:teuthology.orchestra.run.smithi089.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-09-21T03:39:24.538 INFO:teuthology.orchestra.run.smithi089.stdout:/var/lib/nfs/rpc_pipefs 2024-09-21T03:39:24.538 INFO:teuthology.orchestra.run.smithi089.stdout:/run/user/1000 2024-09-21T03:39:24.538 INFO:teuthology.orchestra.run.smithi089.stdout:/var/lib/ceph 2024-09-21T03:39:24.538 INFO:teuthology.orchestra.run.smithi089.stdout:/var/lib/containers/storage/overlay 2024-09-21T03:39:24.538 INFO:teuthology.orchestra.run.smithi089.stdout:/var/lib/containers/storage/overlay/0b1266360c1e947b91c951aa5a770f203724c9783f3817320d7b3f833fdd3623/merged 2024-09-21T03:39:24.538 INFO:teuthology.orchestra.run.smithi089.stdout:/var/lib/containers/storage/overlay/74f483b2de23aa3db5ef9fda5e7c9ec9d9869ed0e09e152a27d160b31080f0d7/merged 2024-09-21T03:39:24.538 INFO:teuthology.orchestra.run.smithi089.stdout:/run/user/0 2024-09-21T03:39:24.539 INFO:teuthology.orchestra.run.smithi089.stdout:/proc/sys/fs/binfmt_misc 2024-09-21T03:39:24.539 INFO:teuthology.orchestra.run.smithi089.stdout:/var/lib/containers/storage/overlay/0c33d786533fa7c6c9b4ece0dd8c3bfc33ee457df01a35b7a874175fb0d24091/merged 2024-09-21T03:39:24.539 INFO:teuthology.orchestra.run.smithi089.stdout:/var/lib/containers/storage/overlay/1e263b0a542f58eaf76d73e377dd5959e47135a86a85f6eb5eae463c0822e8c2/merged 2024-09-21T03:39:24.539 INFO:teuthology.orchestra.run.smithi089.stdout:/var/lib/containers/storage/overlay/2f136aa314ce51b9e309ae462848aa731dc41b3b9ee884ab006901dd9af81ad8/merged 2024-09-21T03:39:24.539 INFO:teuthology.orchestra.run.smithi089.stdout:/var/lib/containers/storage/overlay/64f7a59823e11e80a119f6f784180bf56bca738c73a5be4e7e05673bb3400150/merged 2024-09-21T03:39:24.539 INFO:teuthology.orchestra.run.smithi089.stdout:/var/lib/containers/storage/overlay/cc0f955263cae7210080df6836f384f710eb6af21a08203f2d154f023c022d18/merged 2024-09-21T03:39:24.539 INFO:teuthology.orchestra.run.smithi089.stdout:/var/lib/containers/storage/overlay/9a935757ca5b5da6d9a52aaebf493ea1b50d289d444a1fc297330740dbd3169d/merged 2024-09-21T03:39:24.539 INFO:teuthology.orchestra.run.smithi089.stdout:/var/lib/containers/storage/overlay/050ed91c932a05cf2dc3c502aa3bf5b8822fb5b205b8284f46e11cf4638c07a2/merged 2024-09-21T03:39:24.539 INFO:teuthology.orchestra.run.smithi089.stdout:/var/lib/containers/storage/overlay/34ca3c8ad8cdf365681f5fde2e944c6107cead8c5aea4405ffe1ac85eb695e9c/merged 2024-09-21T03:39:24.539 INFO:teuthology.orchestra.run.smithi089.stdout:/run/netns 2024-09-21T03:39:24.539 INFO:teuthology.orchestra.run.smithi089.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-21T03:39:24.539 INFO:teuthology.orchestra.run.smithi089.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-21T03:39:24.539 INFO:teuthology.orchestra.run.smithi089.stdout:/home/ubuntu/cephtest/mnt.0 2024-09-21T03:39:24.540 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:39:24.540 DEBUG:teuthology.orchestra.run.smithi089:> ls /sys/fs/fuse/connections 2024-09-21T03:39:24.593 INFO:teuthology.orchestra.run.smithi089.stdout:120 2024-09-21T03:39:24.593 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [120] 2024-09-21T03:39:24.593 DEBUG:teuthology.orchestra.run.smithi089:> sudo stdin-killer -- python3 -c ' 2024-09-21T03:39:24.593 DEBUG:teuthology.orchestra.run.smithi089:> import glob 2024-09-21T03:39:24.593 DEBUG:teuthology.orchestra.run.smithi089:> import re 2024-09-21T03:39:24.593 DEBUG:teuthology.orchestra.run.smithi089:> import os 2024-09-21T03:39:24.594 DEBUG:teuthology.orchestra.run.smithi089:> import subprocess 2024-09-21T03:39:24.594 DEBUG:teuthology.orchestra.run.smithi089:> 2024-09-21T03:39:24.594 DEBUG:teuthology.orchestra.run.smithi089:> def _find_admin_socket(client_name): 2024-09-21T03:39:24.594 DEBUG:teuthology.orchestra.run.smithi089:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-09-21T03:39:24.594 DEBUG:teuthology.orchestra.run.smithi089:> files = glob.glob(asok_path) 2024-09-21T03:39:24.594 DEBUG:teuthology.orchestra.run.smithi089:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-09-21T03:39:24.594 DEBUG:teuthology.orchestra.run.smithi089:> 2024-09-21T03:39:24.594 DEBUG:teuthology.orchestra.run.smithi089:> # Given a non-glob path, it better be there 2024-09-21T03:39:24.594 DEBUG:teuthology.orchestra.run.smithi089:> if "*" not in asok_path: 2024-09-21T03:39:24.594 DEBUG:teuthology.orchestra.run.smithi089:> assert(len(files) == 1) 2024-09-21T03:39:24.594 DEBUG:teuthology.orchestra.run.smithi089:> return files[0] 2024-09-21T03:39:24.594 DEBUG:teuthology.orchestra.run.smithi089:> 2024-09-21T03:39:24.594 DEBUG:teuthology.orchestra.run.smithi089:> for f in files: 2024-09-21T03:39:24.594 DEBUG:teuthology.orchestra.run.smithi089:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-09-21T03:39:24.594 DEBUG:teuthology.orchestra.run.smithi089:> if os.path.exists("/proc/{0}".format(pid)): 2024-09-21T03:39:24.594 DEBUG:teuthology.orchestra.run.smithi089:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-09-21T03:39:24.595 DEBUG:teuthology.orchestra.run.smithi089:> contents = proc_f.read() 2024-09-21T03:39:24.595 DEBUG:teuthology.orchestra.run.smithi089:> if mountpoint in contents: 2024-09-21T03:39:24.595 DEBUG:teuthology.orchestra.run.smithi089:> return f 2024-09-21T03:39:24.595 DEBUG:teuthology.orchestra.run.smithi089:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-09-21T03:39:24.595 DEBUG:teuthology.orchestra.run.smithi089:> 2024-09-21T03:39:24.595 DEBUG:teuthology.orchestra.run.smithi089:> print(_find_admin_socket("client.0")) 2024-09-21T03:39:24.595 DEBUG:teuthology.orchestra.run.smithi089:> ' 2024-09-21T03:39:24.711 INFO:teuthology.orchestra.run.smithi089.stdout:/var/run/ceph/ceph-client.0.84548.asok 2024-09-21T03:39:24.714 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:24 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T03:39:24.721 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.84548.asok 2024-09-21T03:39:24.722 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:39:24.722 DEBUG:teuthology.orchestra.run.smithi089:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.84548.asok status 2024-09-21T03:39:24.864 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-09-21T03:39:24.864 INFO:teuthology.orchestra.run.smithi089.stdout: "metadata": { 2024-09-21T03:39:24.864 INFO:teuthology.orchestra.run.smithi089.stdout: "ceph_sha1": "1843221895adccfb3b692060251dd6a0183bc410", 2024-09-21T03:39:24.864 INFO:teuthology.orchestra.run.smithi089.stdout: "ceph_version": "ceph version 19.3.0-5120-g18432218 (1843221895adccfb3b692060251dd6a0183bc410) squid (dev)", 2024-09-21T03:39:24.864 INFO:teuthology.orchestra.run.smithi089.stdout: "entity_id": "0", 2024-09-21T03:39:24.864 INFO:teuthology.orchestra.run.smithi089.stdout: "hostname": "smithi089", 2024-09-21T03:39:24.864 INFO:teuthology.orchestra.run.smithi089.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-09-21T03:39:24.864 INFO:teuthology.orchestra.run.smithi089.stdout: "pid": "84548", 2024-09-21T03:39:24.865 INFO:teuthology.orchestra.run.smithi089.stdout: "root": "/volumes/qa/sv_0/d474bdba-6c8a-4432-8887-89da76119cad" 2024-09-21T03:39:24.865 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:39:24.865 INFO:teuthology.orchestra.run.smithi089.stdout: "dentry_count": 0, 2024-09-21T03:39:24.865 INFO:teuthology.orchestra.run.smithi089.stdout: "dentry_pinned_count": 0, 2024-09-21T03:39:24.865 INFO:teuthology.orchestra.run.smithi089.stdout: "id": 15423, 2024-09-21T03:39:24.865 INFO:teuthology.orchestra.run.smithi089.stdout: "inst": { 2024-09-21T03:39:24.865 INFO:teuthology.orchestra.run.smithi089.stdout: "name": { 2024-09-21T03:39:24.865 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "client", 2024-09-21T03:39:24.865 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 15423 2024-09-21T03:39:24.865 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:39:24.865 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": { 2024-09-21T03:39:24.865 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-09-21T03:39:24.865 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "192.168.0.1:0", 2024-09-21T03:39:24.865 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 1104611521 2024-09-21T03:39:24.865 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-21T03:39:24.866 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:39:24.866 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": { 2024-09-21T03:39:24.866 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-09-21T03:39:24.866 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "192.168.0.1:0", 2024-09-21T03:39:24.866 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 1104611521 2024-09-21T03:39:24.866 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:39:24.866 INFO:teuthology.orchestra.run.smithi089.stdout: "inst_str": "client.15423 192.168.0.1:0/1104611521", 2024-09-21T03:39:24.866 INFO:teuthology.orchestra.run.smithi089.stdout: "addr_str": "192.168.0.1:0/1104611521", 2024-09-21T03:39:24.866 INFO:teuthology.orchestra.run.smithi089.stdout: "inode_count": 5, 2024-09-21T03:39:24.866 INFO:teuthology.orchestra.run.smithi089.stdout: "mds_epoch": 25, 2024-09-21T03:39:24.866 INFO:teuthology.orchestra.run.smithi089.stdout: "osd_epoch": 84, 2024-09-21T03:39:24.866 INFO:teuthology.orchestra.run.smithi089.stdout: "osd_epoch_barrier": 0, 2024-09-21T03:39:24.866 INFO:teuthology.orchestra.run.smithi089.stdout: "blocklisted": false, 2024-09-21T03:39:24.866 INFO:teuthology.orchestra.run.smithi089.stdout: "fs_name": "cephfs" 2024-09-21T03:39:24.866 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-09-21T03:39:24.873 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:39:24.873 DEBUG:teuthology.orchestra.run.smithi089:> stat --file-system '--printf=%T 2024-09-21T03:39:24.873 DEBUG:teuthology.orchestra.run.smithi089:> ' -- /home/ubuntu/cephtest/mnt.0 2024-09-21T03:39:24.888 INFO:teuthology.orchestra.run.smithi089.stdout:fuseblk 2024-09-21T03:39:24.889 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-09-21T03:39:24.889 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:39:24.889 DEBUG:teuthology.orchestra.run.smithi089:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-21T03:39:24.956 INFO:teuthology.run_tasks:Running task exec... 2024-09-21T03:39:24.966 INFO:teuthology.task.exec:Executing custom commands... 2024-09-21T03:39:24.966 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi089.front.sepia.ceph.com 2024-09-21T03:39:24.966 DEBUG:teuthology.orchestra.run.smithi089:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs set cephfs balance_automate false' 2024-09-21T03:39:25.111 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:25.106+0000 7f03e5a5c640 1 -- 172.21.15.89:0/990574901 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f03e0104750 msgr2=0x7f03e0104b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:25.111 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:25.106+0000 7f03e5a5c640 1 --2- 172.21.15.89:0/990574901 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f03e0104750 0x7f03e0104b60 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f03c8009900 tx=0x7f03c802a410 comp rx=0 tx=0).stop 2024-09-21T03:39:25.111 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:25.106+0000 7f03e5a5c640 1 -- 172.21.15.89:0/990574901 shutdown_connections 2024-09-21T03:39:25.111 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:25.106+0000 7f03e5a5c640 1 --2- 172.21.15.89:0/990574901 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f03e01050f0 0x7f03e010ba90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:25.111 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:25.106+0000 7f03e5a5c640 1 --2- 172.21.15.89:0/990574901 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f03e0104750 0x7f03e0104b60 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:25.111 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:25.106+0000 7f03e5a5c640 1 --2- 172.21.15.89:0/990574901 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f03e01036d0 0x7f03e0103b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:25.111 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:25.106+0000 7f03e5a5c640 1 -- 172.21.15.89:0/990574901 >> 172.21.15.89:0/990574901 conn(0x7f03e00ff030 msgr2=0x7f03e01014c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:39:25.112 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:25.106+0000 7f03e5a5c640 1 -- 172.21.15.89:0/990574901 shutdown_connections 2024-09-21T03:39:25.112 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:25.106+0000 7f03e5a5c640 1 -- 172.21.15.89:0/990574901 wait complete. 2024-09-21T03:39:25.112 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:25.107+0000 7f03e5a5c640 1 Processor -- start 2024-09-21T03:39:25.112 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:25.107+0000 7f03e5a5c640 1 -- start start 2024-09-21T03:39:25.112 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:25.107+0000 7f03e5a5c640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f03e01036d0 0x7f03e0199e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:25.112 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:25.107+0000 7f03e5a5c640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f03e0104750 0x7f03e019a500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:25.112 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:25.107+0000 7f03e5a5c640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f03e01050f0 0x7f03e019b0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:25.112 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:25.107+0000 7f03e5a5c640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f03e010f1d0 con 0x7f03e01036d0 2024-09-21T03:39:25.112 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:25.107+0000 7f03e5a5c640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f03e010f050 con 0x7f03e0104750 2024-09-21T03:39:25.112 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:25.107+0000 7f03e5a5c640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f03e010f350 con 0x7f03e01050f0 2024-09-21T03:39:25.112 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:25.107+0000 7f03deffd640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f03e01036d0 0x7f03e0199e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:25.112 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:25.107+0000 7f03deffd640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f03e01036d0 0x7f03e0199e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:53300/0 (socket says 172.21.15.89:53300) 2024-09-21T03:39:25.112 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:25.107+0000 7f03deffd640 1 -- 172.21.15.89:0/1015357812 learned_addr learned my addr 172.21.15.89:0/1015357812 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-21T03:39:25.113 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:25.107+0000 7f03df7fe640 1 --2- 172.21.15.89:0/1015357812 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f03e01050f0 0x7f03e019b0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:25.113 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:25.107+0000 7f03de7fc640 1 --2- 172.21.15.89:0/1015357812 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f03e0104750 0x7f03e019a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:25.113 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:25.108+0000 7f03deffd640 1 -- 172.21.15.89:0/1015357812 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f03e01050f0 msgr2=0x7f03e019b0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:25.113 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:25.108+0000 7f03deffd640 1 --2- 172.21.15.89:0/1015357812 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f03e01050f0 0x7f03e019b0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:25.113 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:25.108+0000 7f03deffd640 1 -- 172.21.15.89:0/1015357812 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f03e0104750 msgr2=0x7f03e019a500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:25.113 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:25.108+0000 7f03deffd640 1 --2- 172.21.15.89:0/1015357812 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f03e0104750 0x7f03e019a500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:25.113 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:25.108+0000 7f03deffd640 1 -- 172.21.15.89:0/1015357812 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03e019b840 con 0x7f03e01036d0 2024-09-21T03:39:25.113 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:25.108+0000 7f03de7fc640 1 --2- 172.21.15.89:0/1015357812 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f03e0104750 0x7f03e019a500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:39:25.113 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:25.108+0000 7f03deffd640 1 --2- 172.21.15.89:0/1015357812 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f03e01036d0 0x7f03e0199e90 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f03d000ea80 tx=0x7f03d000ef40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:39:25.113 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:25.108+0000 7f03e4a5a640 1 -- 172.21.15.89:0/1015357812 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03d000ce40 con 0x7f03e01036d0 2024-09-21T03:39:25.113 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:25.108+0000 7f03e4a5a640 1 -- 172.21.15.89:0/1015357812 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f03d0005020 con 0x7f03e01036d0 2024-09-21T03:39:25.113 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:25.108+0000 7f03e5a5c640 1 -- 172.21.15.89:0/1015357812 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f03e019bb30 con 0x7f03e01036d0 2024-09-21T03:39:25.113 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:25.108+0000 7f03e4a5a640 1 -- 172.21.15.89:0/1015357812 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03d00116d0 con 0x7f03e01036d0 2024-09-21T03:39:25.113 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:25.108+0000 7f03e5a5c640 1 -- 172.21.15.89:0/1015357812 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03e01a20d0 con 0x7f03e01036d0 2024-09-21T03:39:25.114 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:25.109+0000 7f03e5a5c640 1 -- 172.21.15.89:0/1015357812 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f03e010a8a0 con 0x7f03e01036d0 2024-09-21T03:39:25.114 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:25.111+0000 7f03e4a5a640 1 -- 172.21.15.89:0/1015357812 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 19) ==== 154512+0+0 (secure 0 0 0) 0x7f03d0004a40 con 0x7f03e01036d0 2024-09-21T03:39:25.114 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:25.112+0000 7f03e4a5a640 1 --2- 172.21.15.89:0/1015357812 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f03cc0b6510 0x7f03cc0b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:25.114 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:25.112+0000 7f03de7fc640 1 --2- 172.21.15.89:0/1015357812 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f03cc0b6510 0x7f03cc0b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:25.115 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:25.112+0000 7f03e4a5a640 1 -- 172.21.15.89:0/1015357812 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 9121+0+0 (secure 0 0 0) 0x7f03d00e7cd0 con 0x7f03e01036d0 2024-09-21T03:39:25.115 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:25.113+0000 7f03de7fc640 1 --2- 172.21.15.89:0/1015357812 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f03cc0b6510 0x7f03cc0b8a20 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f03c8005f70 tx=0x7f03c8000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:39:25.117 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:25.115+0000 7f03e4a5a640 1 -- 172.21.15.89:0/1015357812 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f03d0016030 con 0x7f03e01036d0 2024-09-21T03:39:25.311 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:39:25 smithi089 ceph-mon[30928]: pgmap v360: 129 pgs: 129 active+clean; 646 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 85 B/s wr, 6 op/s 2024-09-21T03:39:25.311 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:39:25 smithi089 ceph-mon[30928]: mgrmap e19: x(active, since 9m), standbys: z, y 2024-09-21T03:39:25.334 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:25.332+0000 7f03e5a5c640 1 -- 172.21.15.89:0/1015357812 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"} v 0) -- 0x7f03e002e640 con 0x7f03e01036d0 2024-09-21T03:39:25.461 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:39:25 smithi114 ceph-mon[34140]: pgmap v360: 129 pgs: 129 active+clean; 646 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 85 B/s wr, 6 op/s 2024-09-21T03:39:25.461 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:39:25 smithi114 ceph-mon[34140]: mgrmap e19: x(active, since 9m), standbys: z, y 2024-09-21T03:39:25.500 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:39:25 smithi110 ceph-mon[34000]: pgmap v360: 129 pgs: 129 active+clean; 646 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 85 B/s wr, 6 op/s 2024-09-21T03:39:25.500 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:39:25 smithi110 ceph-mon[34000]: mgrmap e19: x(active, since 9m), standbys: z, y 2024-09-21T03:39:26.065 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:26.063+0000 7f03e4a5a640 1 -- 172.21.15.89:0/1015357812 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]=0 v26) ==== 118+0+0 (secure 0 0 0) 0x7f03d00ae1e0 con 0x7f03e01036d0 2024-09-21T03:39:26.068 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:26.065+0000 7f03e5a5c640 1 -- 172.21.15.89:0/1015357812 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f03cc0b6510 msgr2=0x7f03cc0b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:26.068 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:26.065+0000 7f03e5a5c640 1 --2- 172.21.15.89:0/1015357812 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f03cc0b6510 0x7f03cc0b8a20 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f03c8005f70 tx=0x7f03c8000bc0 comp rx=0 tx=0).stop 2024-09-21T03:39:26.068 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:26.065+0000 7f03e5a5c640 1 -- 172.21.15.89:0/1015357812 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f03e01036d0 msgr2=0x7f03e0199e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:26.068 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:26.066+0000 7f03e5a5c640 1 --2- 172.21.15.89:0/1015357812 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f03e01036d0 0x7f03e0199e90 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f03d000ea80 tx=0x7f03d000ef40 comp rx=0 tx=0).stop 2024-09-21T03:39:26.068 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:26.066+0000 7f03e5a5c640 1 -- 172.21.15.89:0/1015357812 shutdown_connections 2024-09-21T03:39:26.068 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:26.066+0000 7f03e5a5c640 1 --2- 172.21.15.89:0/1015357812 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f03cc0b6510 0x7f03cc0b8a20 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:26.068 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:26.066+0000 7f03e5a5c640 1 --2- 172.21.15.89:0/1015357812 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f03e0104750 0x7f03e019a500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:26.068 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:26.066+0000 7f03e5a5c640 1 --2- 172.21.15.89:0/1015357812 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f03e01050f0 0x7f03e019b0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:26.068 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:26.066+0000 7f03e5a5c640 1 --2- 172.21.15.89:0/1015357812 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f03e01036d0 0x7f03e0199e90 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:26.069 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:26.066+0000 7f03e5a5c640 1 -- 172.21.15.89:0/1015357812 >> 172.21.15.89:0/1015357812 conn(0x7f03e00ff030 msgr2=0x7f03e0107a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:39:26.069 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:26.066+0000 7f03e5a5c640 1 -- 172.21.15.89:0/1015357812 shutdown_connections 2024-09-21T03:39:26.069 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:26.066+0000 7f03e5a5c640 1 -- 172.21.15.89:0/1015357812 wait complete. 2024-09-21T03:39:26.080 DEBUG:teuthology.orchestra.run.smithi089:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs subvolumegroup pin cephfs qa random 0.10' 2024-09-21T03:39:26.193 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:26.190+0000 7f6d408bd640 1 -- 172.21.15.89:0/4186392215 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6d3c1050e0 msgr2=0x7f6d3c10ba80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:26.193 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:26.190+0000 7f6d408bd640 1 --2- 172.21.15.89:0/4186392215 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6d3c1050e0 0x7f6d3c10ba80 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f6d34009ca0 tx=0x7f6d3402a8c0 comp rx=0 tx=0).stop 2024-09-21T03:39:26.193 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:26.190+0000 7f6d408bd640 1 -- 172.21.15.89:0/4186392215 shutdown_connections 2024-09-21T03:39:26.193 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:26.190+0000 7f6d408bd640 1 --2- 172.21.15.89:0/4186392215 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6d3c1050e0 0x7f6d3c10ba80 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:26.193 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:26.190+0000 7f6d408bd640 1 --2- 172.21.15.89:0/4186392215 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6d3c104740 0x7f6d3c104b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:26.193 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:26.190+0000 7f6d408bd640 1 --2- 172.21.15.89:0/4186392215 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6d3c1036c0 0x7f6d3c103b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:26.193 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:26.190+0000 7f6d408bd640 1 -- 172.21.15.89:0/4186392215 >> 172.21.15.89:0/4186392215 conn(0x7f6d3c0ff020 msgr2=0x7f6d3c1014b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:39:26.193 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:26.190+0000 7f6d408bd640 1 -- 172.21.15.89:0/4186392215 shutdown_connections 2024-09-21T03:39:26.193 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:26.190+0000 7f6d408bd640 1 -- 172.21.15.89:0/4186392215 wait complete. 2024-09-21T03:39:26.195 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:26.191+0000 7f6d408bd640 1 Processor -- start 2024-09-21T03:39:26.195 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:26.191+0000 7f6d408bd640 1 -- start start 2024-09-21T03:39:26.195 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:26.191+0000 7f6d408bd640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6d3c1036c0 0x7f6d3c199e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:26.195 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:26.191+0000 7f6d408bd640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6d3c104740 0x7f6d3c19a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:26.195 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:26.191+0000 7f6d408bd640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6d3c1050e0 0x7f6d3c19b050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:26.196 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:26.191+0000 7f6d408bd640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f6d3c10f170 con 0x7f6d3c1036c0 2024-09-21T03:39:26.196 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:26.191+0000 7f6d408bd640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f6d3c10eff0 con 0x7f6d3c104740 2024-09-21T03:39:26.196 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:26.191+0000 7f6d408bd640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f6d3c10f2f0 con 0x7f6d3c1050e0 2024-09-21T03:39:26.196 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:26.191+0000 7f6d3a59c640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6d3c1036c0 0x7f6d3c199e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:26.196 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:26.191+0000 7f6d3a59c640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6d3c1036c0 0x7f6d3c199e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:53316/0 (socket says 172.21.15.89:53316) 2024-09-21T03:39:26.196 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:26.192+0000 7f6d3a59c640 1 -- 172.21.15.89:0/2444255740 learned_addr learned my addr 172.21.15.89:0/2444255740 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-21T03:39:26.196 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:26.192+0000 7f6d39d9b640 1 --2- 172.21.15.89:0/2444255740 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6d3c104740 0x7f6d3c19a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:26.196 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:26.192+0000 7f6d3ad9d640 1 --2- 172.21.15.89:0/2444255740 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6d3c1050e0 0x7f6d3c19b050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:26.196 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:26.192+0000 7f6d3a59c640 1 -- 172.21.15.89:0/2444255740 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6d3c1050e0 msgr2=0x7f6d3c19b050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:26.196 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:26.192+0000 7f6d3a59c640 1 --2- 172.21.15.89:0/2444255740 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6d3c1050e0 0x7f6d3c19b050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:26.196 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:26.192+0000 7f6d3a59c640 1 -- 172.21.15.89:0/2444255740 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6d3c104740 msgr2=0x7f6d3c19a4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:26.196 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:26.192+0000 7f6d3a59c640 1 --2- 172.21.15.89:0/2444255740 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6d3c104740 0x7f6d3c19a4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:26.196 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:26.192+0000 7f6d3a59c640 1 -- 172.21.15.89:0/2444255740 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d3c19b870 con 0x7f6d3c1036c0 2024-09-21T03:39:26.197 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:26.192+0000 7f6d3a59c640 1 --2- 172.21.15.89:0/2444255740 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6d3c1036c0 0x7f6d3c199e30 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f6d2c00e9f0 tx=0x7f6d2c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:39:26.197 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:26.192+0000 7f6d337fe640 1 -- 172.21.15.89:0/2444255740 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d2c00cd80 con 0x7f6d3c1036c0 2024-09-21T03:39:26.197 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:26.192+0000 7f6d337fe640 1 -- 172.21.15.89:0/2444255740 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6d2c0108c0 con 0x7f6d3c1036c0 2024-09-21T03:39:26.197 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:26.192+0000 7f6d337fe640 1 -- 172.21.15.89:0/2444255740 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d2c0187c0 con 0x7f6d3c1036c0 2024-09-21T03:39:26.197 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:26.192+0000 7f6d408bd640 1 -- 172.21.15.89:0/2444255740 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6d3c19bb60 con 0x7f6d3c1036c0 2024-09-21T03:39:26.197 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:26.192+0000 7f6d408bd640 1 -- 172.21.15.89:0/2444255740 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6d3c1a2250 con 0x7f6d3c1036c0 2024-09-21T03:39:26.197 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:26.193+0000 7f6d408bd640 1 -- 172.21.15.89:0/2444255740 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6d3c10a870 con 0x7f6d3c1036c0 2024-09-21T03:39:26.198 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:26.195+0000 7f6d337fe640 1 -- 172.21.15.89:0/2444255740 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 19) ==== 154512+0+0 (secure 0 0 0) 0x7f6d2c010430 con 0x7f6d3c1036c0 2024-09-21T03:39:26.198 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:26.195+0000 7f6d337fe640 1 --2- 172.21.15.89:0/2444255740 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f6d280b6510 0x7f6d280b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:26.198 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:26.195+0000 7f6d39d9b640 1 --2- 172.21.15.89:0/2444255740 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f6d280b6510 0x7f6d280b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:26.198 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:26.195+0000 7f6d337fe640 1 -- 172.21.15.89:0/2444255740 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 9121+0+0 (secure 0 0 0) 0x7f6d2c0e7f60 con 0x7f6d3c1036c0 2024-09-21T03:39:26.198 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:26.196+0000 7f6d39d9b640 1 --2- 172.21.15.89:0/2444255740 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f6d280b6510 0x7f6d280b8a20 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f6d24009bd0 tx=0x7f6d24005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:39:26.200 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:26.198+0000 7f6d337fe640 1 -- 172.21.15.89:0/2444255740 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f6d2c0ae470 con 0x7f6d3c1036c0 2024-09-21T03:39:26.405 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:26.403+0000 7f6d408bd640 1 -- 172.21.15.89:0/2444255740 --> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}) -- 0x7f6d3c063960 con 0x7f6d280b6510 2024-09-21T03:39:26.411 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:39:26 smithi089 ceph-mon[30928]: from='client.? 172.21.15.89:0/1015357812' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"} : dispatch 2024-09-21T03:39:26.426 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:26.424+0000 7f6d337fe640 1 -- 172.21.15.89:0/2444255740 <== mgr.14152 v2:172.21.15.89:6800/1591532358 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+3 (secure 0 0 0) 0x7f6d3c063960 con 0x7f6d280b6510 2024-09-21T03:39:26.426 INFO:teuthology.orchestra.run.smithi089.stdout:{} 2024-09-21T03:39:26.429 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:26.426+0000 7f6d408bd640 1 -- 172.21.15.89:0/2444255740 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f6d280b6510 msgr2=0x7f6d280b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:26.429 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:26.426+0000 7f6d408bd640 1 --2- 172.21.15.89:0/2444255740 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f6d280b6510 0x7f6d280b8a20 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f6d24009bd0 tx=0x7f6d24005f90 comp rx=0 tx=0).stop 2024-09-21T03:39:26.429 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:26.426+0000 7f6d408bd640 1 -- 172.21.15.89:0/2444255740 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6d3c1036c0 msgr2=0x7f6d3c199e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:26.429 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:26.426+0000 7f6d408bd640 1 --2- 172.21.15.89:0/2444255740 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6d3c1036c0 0x7f6d3c199e30 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f6d2c00e9f0 tx=0x7f6d2c00eeb0 comp rx=0 tx=0).stop 2024-09-21T03:39:26.429 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:26.426+0000 7f6d408bd640 1 -- 172.21.15.89:0/2444255740 shutdown_connections 2024-09-21T03:39:26.429 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:26.426+0000 7f6d408bd640 1 --2- 172.21.15.89:0/2444255740 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f6d280b6510 0x7f6d280b8a20 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:26.429 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:26.426+0000 7f6d408bd640 1 --2- 172.21.15.89:0/2444255740 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6d3c104740 0x7f6d3c19a4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:26.429 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:26.426+0000 7f6d408bd640 1 --2- 172.21.15.89:0/2444255740 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6d3c1050e0 0x7f6d3c19b050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:26.429 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:26.426+0000 7f6d408bd640 1 --2- 172.21.15.89:0/2444255740 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6d3c1036c0 0x7f6d3c199e30 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:26.429 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:26.426+0000 7f6d408bd640 1 -- 172.21.15.89:0/2444255740 >> 172.21.15.89:0/2444255740 conn(0x7f6d3c0ff020 msgr2=0x7f6d3c107a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:39:26.429 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:26.426+0000 7f6d408bd640 1 -- 172.21.15.89:0/2444255740 shutdown_connections 2024-09-21T03:39:26.429 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:26.427+0000 7f6d408bd640 1 -- 172.21.15.89:0/2444255740 wait complete. 2024-09-21T03:39:26.440 INFO:teuthology.run_tasks:Running task check-counter... 2024-09-21T03:39:26.456 INFO:teuthology.run_tasks:Running task background_exec... 2024-09-21T03:39:26.536 INFO:teuthology.task.background_exec:Running background command on role mon.a host ubuntu@smithi089.front.sepia.ceph.com 2024-09-21T03:39:26.536 DEBUG:teuthology.orchestra.run.smithi089:> sudo TESTDIR=/home/ubuntu/cephtest daemon-helper kill --kill-group bash -c 'while sleep 13; do ceph tell mds.cephfs:0 flush journal; done' 2024-09-21T03:39:26.537 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:39:26 smithi110 ceph-mon[34000]: from='client.? 172.21.15.89:0/1015357812' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"} : dispatch 2024-09-21T03:39:26.537 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:39:26 smithi114 ceph-mon[34140]: from='client.? 172.21.15.89:0/1015357812' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"} : dispatch 2024-09-21T03:39:26.539 INFO:teuthology.run_tasks:Running task quiescer... 2024-09-21T03:39:26.574 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-09-21T03:39:26.574 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 7c374098-77c9-11ef-baf5-efdc52797490 -- ceph pg dump --format=json 2024-09-21T03:39:26.811 INFO:journalctl@ceph.mgr.x.smithi089.stdout:Sep 21 03:39:26 smithi089 ceph-7c374098-77c9-11ef-baf5-efdc52797490-mgr-x[31154]: 2024-09-21T03:39:26.408+0000 7f54ad47b640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T03:39:26.811 INFO:journalctl@ceph.mgr.x.smithi089.stdout:Sep 21 03:39:26 smithi089 ceph-7c374098-77c9-11ef-baf5-efdc52797490-mgr-x[31154]: 2024-09-21T03:39:26.408+0000 7f54ad47b640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T03:39:26.811 INFO:journalctl@ceph.mgr.x.smithi089.stdout:Sep 21 03:39:26 smithi089 ceph-7c374098-77c9-11ef-baf5-efdc52797490-mgr-x[31154]: 2024-09-21T03:39:26.408+0000 7f54ad47b640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T03:39:26.811 INFO:journalctl@ceph.mgr.x.smithi089.stdout:Sep 21 03:39:26 smithi089 ceph-7c374098-77c9-11ef-baf5-efdc52797490-mgr-x[31154]: 2024-09-21T03:39:26.408+0000 7f54ad47b640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T03:39:26.811 INFO:journalctl@ceph.mgr.x.smithi089.stdout:Sep 21 03:39:26 smithi089 ceph-7c374098-77c9-11ef-baf5-efdc52797490-mgr-x[31154]: 2024-09-21T03:39:26.408+0000 7f54ad47b640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T03:39:26.825 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/7c374098-77c9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:39:27.230 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:39:27 smithi089 ceph-mon[30928]: pgmap v361: 129 pgs: 129 active+clean; 647 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 597 B/s wr, 7 op/s 2024-09-21T03:39:27.230 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:39:27 smithi089 ceph-mon[30928]: from='client.? 172.21.15.89:0/1015357812' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-09-21T03:39:27.230 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:39:27 smithi089 ceph-mon[30928]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-09-21T03:39:27.231 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:39:27 smithi089 ceph-mon[30928]: from='client.15435 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:39:27.330 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:27.318+0000 7f2357f85640 1 -- 172.21.15.89:0/989251819 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2350103ba0 msgr2=0x7f2350103fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:27.330 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:27.318+0000 7f2357f85640 1 --2- 172.21.15.89:0/989251819 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2350103ba0 0x7f2350103fb0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f2340009900 tx=0x7f234002a410 comp rx=0 tx=0).stop 2024-09-21T03:39:27.330 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:27.318+0000 7f2357f85640 1 -- 172.21.15.89:0/989251819 shutdown_connections 2024-09-21T03:39:27.330 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:27.318+0000 7f2357f85640 1 --2- 172.21.15.89:0/989251819 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2350109e30 0x7f2350110660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:27.330 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:27.318+0000 7f2357f85640 1 --2- 172.21.15.89:0/989251819 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f23501094e0 0x7f23501098f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:27.330 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:27.318+0000 7f2357f85640 1 --2- 172.21.15.89:0/989251819 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2350103ba0 0x7f2350103fb0 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:27.330 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:27.318+0000 7f2357f85640 1 -- 172.21.15.89:0/989251819 >> 172.21.15.89:0/989251819 conn(0x7f23500ff630 msgr2=0x7f2350101a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:39:27.330 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:27.318+0000 7f2357f85640 1 -- 172.21.15.89:0/989251819 shutdown_connections 2024-09-21T03:39:27.330 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:27.318+0000 7f2357f85640 1 -- 172.21.15.89:0/989251819 wait complete. 2024-09-21T03:39:27.331 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:27.319+0000 7f2357f85640 1 Processor -- start 2024-09-21T03:39:27.331 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:27.319+0000 7f2357f85640 1 -- start start 2024-09-21T03:39:27.331 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:27.320+0000 7f2357f85640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2350103ba0 0x7f235019a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:27.331 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:27.320+0000 7f2357f85640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f23501094e0 0x7f235019a990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:27.331 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:27.320+0000 7f2357f85640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2350109e30 0x7f235019fc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:27.331 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:27.320+0000 7f2357f85640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f23501139a0 con 0x7f23501094e0 2024-09-21T03:39:27.331 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:27.320+0000 7f2357f85640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f2350113820 con 0x7f2350103ba0 2024-09-21T03:39:27.331 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:27.320+0000 7f2357f85640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f2350113b20 con 0x7f2350109e30 2024-09-21T03:39:27.331 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:27.320+0000 7f2355520640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f23501094e0 0x7f235019a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:27.331 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:27.320+0000 7f2355520640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f23501094e0 0x7f235019a990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:53330/0 (socket says 172.21.15.89:53330) 2024-09-21T03:39:27.331 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:27.320+0000 7f2355520640 1 -- 172.21.15.89:0/2348139931 learned_addr learned my addr 172.21.15.89:0/2348139931 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-21T03:39:27.331 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:27.320+0000 7f2355d21640 1 --2- 172.21.15.89:0/2348139931 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2350103ba0 0x7f235019a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:27.331 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:27.320+0000 7f2356522640 1 --2- 172.21.15.89:0/2348139931 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2350109e30 0x7f235019fc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:27.332 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:27.320+0000 7f2355520640 1 -- 172.21.15.89:0/2348139931 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2350109e30 msgr2=0x7f235019fc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:27.332 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:27.320+0000 7f2355520640 1 --2- 172.21.15.89:0/2348139931 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2350109e30 0x7f235019fc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:27.332 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:27.320+0000 7f2355520640 1 -- 172.21.15.89:0/2348139931 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2350103ba0 msgr2=0x7f235019a3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:27.332 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:27.320+0000 7f2355520640 1 --2- 172.21.15.89:0/2348139931 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2350103ba0 0x7f235019a3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:27.332 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:27.320+0000 7f2355520640 1 -- 172.21.15.89:0/2348139931 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23501a0360 con 0x7f23501094e0 2024-09-21T03:39:27.332 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:27.320+0000 7f2356522640 1 --2- 172.21.15.89:0/2348139931 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2350109e30 0x7f235019fc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:39:27.332 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:27.320+0000 7f2355520640 1 --2- 172.21.15.89:0/2348139931 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f23501094e0 0x7f235019a990 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f234c00e9f0 tx=0x7f234c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:39:27.332 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:27.321+0000 7f2346ffd640 1 -- 172.21.15.89:0/2348139931 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f234c00ccc0 con 0x7f23501094e0 2024-09-21T03:39:27.332 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:27.321+0000 7f2346ffd640 1 -- 172.21.15.89:0/2348139931 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f234c0108c0 con 0x7f23501094e0 2024-09-21T03:39:27.332 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:27.321+0000 7f2357f85640 1 -- 172.21.15.89:0/2348139931 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f23501a0650 con 0x7f23501094e0 2024-09-21T03:39:27.332 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:27.321+0000 7f2346ffd640 1 -- 172.21.15.89:0/2348139931 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f234c018750 con 0x7f23501094e0 2024-09-21T03:39:27.332 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:27.321+0000 7f2357f85640 1 -- 172.21.15.89:0/2348139931 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f23501a6db0 con 0x7f23501094e0 2024-09-21T03:39:27.332 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:27.322+0000 7f2357f85640 1 -- 172.21.15.89:0/2348139931 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2350103fb0 con 0x7f23501094e0 2024-09-21T03:39:27.332 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:27.323+0000 7f2346ffd640 1 -- 172.21.15.89:0/2348139931 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 19) ==== 154512+0+0 (secure 0 0 0) 0x7f234c010430 con 0x7f23501094e0 2024-09-21T03:39:27.332 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:27.324+0000 7f2346ffd640 1 --2- 172.21.15.89:0/2348139931 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f233c0b6590 0x7f233c0b8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:27.333 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:27.324+0000 7f2355d21640 1 --2- 172.21.15.89:0/2348139931 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f233c0b6590 0x7f233c0b8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:27.333 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:27.324+0000 7f2346ffd640 1 -- 172.21.15.89:0/2348139931 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 9121+0+0 (secure 0 0 0) 0x7f234c0e7e40 con 0x7f23501094e0 2024-09-21T03:39:27.333 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:27.324+0000 7f2355d21640 1 --2- 172.21.15.89:0/2348139931 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f233c0b6590 0x7f233c0b8aa0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f23400057c0 tx=0x7f23400058d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:39:27.333 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:27.327+0000 7f2346ffd640 1 -- 172.21.15.89:0/2348139931 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f234c0ae350 con 0x7f23501094e0 2024-09-21T03:39:27.461 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:39:27 smithi114 ceph-mon[34140]: pgmap v361: 129 pgs: 129 active+clean; 647 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 597 B/s wr, 7 op/s 2024-09-21T03:39:27.461 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:39:27 smithi114 ceph-mon[34140]: from='client.? 172.21.15.89:0/1015357812' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-09-21T03:39:27.461 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:39:27 smithi114 ceph-mon[34140]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-09-21T03:39:27.461 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:39:27 smithi114 ceph-mon[34140]: from='client.15435 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:39:27.500 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:39:27 smithi110 ceph-mon[34000]: pgmap v361: 129 pgs: 129 active+clean; 647 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 597 B/s wr, 7 op/s 2024-09-21T03:39:27.500 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:39:27 smithi110 ceph-mon[34000]: from='client.? 172.21.15.89:0/1015357812' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-09-21T03:39:27.500 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:39:27 smithi110 ceph-mon[34000]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-09-21T03:39:27.500 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:39:27 smithi110 ceph-mon[34000]: from='client.15435 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:39:27.506 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:27.504+0000 7f2357f85640 1 -- 172.21.15.89:0/2348139931 --> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f2350063800 con 0x7f233c0b6590 2024-09-21T03:39:27.511 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:27.509+0000 7f2346ffd640 1 -- 172.21.15.89:0/2348139931 <== mgr.14152 v2:172.21.15.89:6800/1591532358 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340885 (secure 0 0 0) 0x7f2350063800 con 0x7f233c0b6590 2024-09-21T03:39:27.515 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-09-21T03:39:27.516 INFO:teuthology.orchestra.run.smithi089.stderr:dumped all 2024-09-21T03:39:27.518 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:27.514+0000 7f2357f85640 1 -- 172.21.15.89:0/2348139931 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f233c0b6590 msgr2=0x7f233c0b8aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:27.518 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:27.514+0000 7f2357f85640 1 --2- 172.21.15.89:0/2348139931 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f233c0b6590 0x7f233c0b8aa0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f23400057c0 tx=0x7f23400058d0 comp rx=0 tx=0).stop 2024-09-21T03:39:27.518 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:27.515+0000 7f2357f85640 1 -- 172.21.15.89:0/2348139931 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f23501094e0 msgr2=0x7f235019a990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:27.519 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:27.515+0000 7f2357f85640 1 --2- 172.21.15.89:0/2348139931 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f23501094e0 0x7f235019a990 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f234c00e9f0 tx=0x7f234c00eeb0 comp rx=0 tx=0).stop 2024-09-21T03:39:27.519 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:27.515+0000 7f2357f85640 1 -- 172.21.15.89:0/2348139931 shutdown_connections 2024-09-21T03:39:27.519 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:27.515+0000 7f2357f85640 1 --2- 172.21.15.89:0/2348139931 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f233c0b6590 0x7f233c0b8aa0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:27.519 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:27.515+0000 7f2357f85640 1 --2- 172.21.15.89:0/2348139931 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2350109e30 0x7f235019fc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:27.519 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:27.515+0000 7f2357f85640 1 --2- 172.21.15.89:0/2348139931 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f23501094e0 0x7f235019a990 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:27.519 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:27.515+0000 7f2357f85640 1 --2- 172.21.15.89:0/2348139931 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2350103ba0 0x7f235019a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:27.519 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:27.515+0000 7f2357f85640 1 -- 172.21.15.89:0/2348139931 >> 172.21.15.89:0/2348139931 conn(0x7f23500ff630 msgr2=0x7f2350101a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:39:27.519 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:27.515+0000 7f2357f85640 1 -- 172.21.15.89:0/2348139931 shutdown_connections 2024-09-21T03:39:27.519 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:27.515+0000 7f2357f85640 1 -- 172.21.15.89:0/2348139931 wait complete. 2024-09-21T03:39:28.247 INFO:teuthology.orchestra.run.smithi089.stdout:{"pg_ready":true,"pg_map":{"version":361,"stamp":"2024-09-21T03:39:25.664921+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":662624,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":675,"num_read_kb":441,"num_write":298,"num_write_kb":2278,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":23,"num_bytes_recovered":2952534,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":743,"ondisk_log_size":743,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":381372,"kb_used_data":11312,"kb_used_omap":18,"kb_used_meta":369837,"kb_avail":1124560452,"statfs":{"total":1151940427776,"available":1151549902848,"internally_reserved":0,"allocated":11583488,"data_stored":7723578,"data_compressed":461874,"data_compressed_allocated":626688,"data_compressed_original":2752512,"omap_allocated":19143,"internal_metadata":378713401},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":21,"apply_latency_ms":21,"commit_latency_ns":21000000,"apply_latency_ns":21000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":883,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":82,"num_read_kb":47,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003668"},"pg_stats":[{"pgid":"2.1f","version":"80'9","reported_seq":50,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:38:53.672984+0000","last_change":"2024-09-21T03:37:10.836953+0000","last_active":"2024-09-21T03:38:53.672984+0000","last_peered":"2024-09-21T03:38:53.672984+0000","last_clean":"2024-09-21T03:38:53.672984+0000","last_became_active":"2024-09-21T03:37:10.836566+0000","last_became_peered":"2024-09-21T03:37:10.836566+0000","last_unstale":"2024-09-21T03:38:53.672984+0000","last_undegraded":"2024-09-21T03:38:53.672984+0000","last_fullsized":"2024-09-21T03:38:53.672984+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_clean_scrub_stamp":"2024-09-21T03:37:09.809805+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:15:45.591225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"84'5","reported_seq":40,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:39:21.839616+0000","last_change":"2024-09-21T03:37:12.866931+0000","last_active":"2024-09-21T03:39:21.839616+0000","last_peered":"2024-09-21T03:39:21.839616+0000","last_clean":"2024-09-21T03:39:21.839616+0000","last_became_active":"2024-09-21T03:37:12.866452+0000","last_became_peered":"2024-09-21T03:37:12.866452+0000","last_unstale":"2024-09-21T03:39:21.839616+0000","last_undegraded":"2024-09-21T03:39:21.839616+0000","last_fullsized":"2024-09-21T03:39:21.839616+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:37:14.324548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:38:53.669589+0000","last_change":"2024-09-21T03:37:19.912154+0000","last_active":"2024-09-21T03:38:53.669589+0000","last_peered":"2024-09-21T03:38:53.669589+0000","last_clean":"2024-09-21T03:38:53.669589+0000","last_became_active":"2024-09-21T03:37:19.911686+0000","last_became_peered":"2024-09-21T03:37:19.911686+0000","last_unstale":"2024-09-21T03:38:53.669589+0000","last_undegraded":"2024-09-21T03:38:53.669589+0000","last_fullsized":"2024-09-21T03:38:53.669589+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_clean_scrub_stamp":"2024-09-21T03:37:18.876933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:29:20.189418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"80'2","reported_seq":42,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:38:53.972157+0000","last_change":"2024-09-21T03:37:11.034501+0000","last_active":"2024-09-21T03:38:53.972157+0000","last_peered":"2024-09-21T03:38:53.972157+0000","last_clean":"2024-09-21T03:38:53.972157+0000","last_became_active":"2024-09-21T03:37:11.034110+0000","last_became_peered":"2024-09-21T03:37:11.034110+0000","last_unstale":"2024-09-21T03:38:53.972157+0000","last_undegraded":"2024-09-21T03:38:53.972157+0000","last_fullsized":"2024-09-21T03:38:53.972157+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_clean_scrub_stamp":"2024-09-21T03:37:09.809805+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:03:12.854469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"84'6","reported_seq":41,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:39:21.858901+0000","last_change":"2024-09-21T03:37:12.867330+0000","last_active":"2024-09-21T03:39:21.858901+0000","last_peered":"2024-09-21T03:39:21.858901+0000","last_clean":"2024-09-21T03:39:21.858901+0000","last_became_active":"2024-09-21T03:37:12.866952+0000","last_became_peered":"2024-09-21T03:37:12.866952+0000","last_unstale":"2024-09-21T03:39:21.858901+0000","last_undegraded":"2024-09-21T03:39:21.858901+0000","last_fullsized":"2024-09-21T03:39:21.858901+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:40:33.826546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:38:54.095716+0000","last_change":"2024-09-21T03:37:19.910661+0000","last_active":"2024-09-21T03:38:54.095716+0000","last_peered":"2024-09-21T03:38:54.095716+0000","last_clean":"2024-09-21T03:38:54.095716+0000","last_became_active":"2024-09-21T03:37:19.910108+0000","last_became_peered":"2024-09-21T03:37:19.910108+0000","last_unstale":"2024-09-21T03:38:54.095716+0000","last_undegraded":"2024-09-21T03:38:54.095716+0000","last_fullsized":"2024-09-21T03:38:54.095716+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_clean_scrub_stamp":"2024-09-21T03:37:18.876933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:18:05.654426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"80'4","reported_seq":43,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:38:54.442879+0000","last_change":"2024-09-21T03:37:10.835336+0000","last_active":"2024-09-21T03:38:54.442879+0000","last_peered":"2024-09-21T03:38:54.442879+0000","last_clean":"2024-09-21T03:38:54.442879+0000","last_became_active":"2024-09-21T03:37:10.834928+0000","last_became_peered":"2024-09-21T03:37:10.834928+0000","last_unstale":"2024-09-21T03:38:54.442879+0000","last_undegraded":"2024-09-21T03:38:54.442879+0000","last_fullsized":"2024-09-21T03:38:54.442879+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_clean_scrub_stamp":"2024-09-21T03:37:09.809805+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:02:14.717520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"84'9","reported_seq":43,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:39:21.865718+0000","last_change":"2024-09-21T03:37:12.861271+0000","last_active":"2024-09-21T03:39:21.865718+0000","last_peered":"2024-09-21T03:39:21.865718+0000","last_clean":"2024-09-21T03:39:21.865718+0000","last_became_active":"2024-09-21T03:37:12.860932+0000","last_became_peered":"2024-09-21T03:37:12.860932+0000","last_unstale":"2024-09-21T03:39:21.865718+0000","last_undegraded":"2024-09-21T03:39:21.865718+0000","last_fullsized":"2024-09-21T03:39:21.865718+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:46:08.993250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":34,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:38:54.096300+0000","last_change":"2024-09-21T03:37:53.053045+0000","last_active":"2024-09-21T03:38:54.096300+0000","last_peered":"2024-09-21T03:38:54.096300+0000","last_clean":"2024-09-21T03:38:54.096300+0000","last_became_active":"2024-09-21T03:37:53.052382+0000","last_became_peered":"2024-09-21T03:37:53.052382+0000","last_unstale":"2024-09-21T03:38:54.096300+0000","last_undegraded":"2024-09-21T03:38:54.096300+0000","last_fullsized":"2024-09-21T03:38:54.096300+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_clean_scrub_stamp":"2024-09-21T03:37:18.876933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:02:44.300768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,8],"acting":[4,3,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"74'2","reported_seq":41,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:38:54.442781+0000","last_change":"2024-09-21T03:37:10.833985+0000","last_active":"2024-09-21T03:38:54.442781+0000","last_peered":"2024-09-21T03:38:54.442781+0000","last_clean":"2024-09-21T03:38:54.442781+0000","last_became_active":"2024-09-21T03:37:10.833562+0000","last_became_peered":"2024-09-21T03:37:10.833562+0000","last_unstale":"2024-09-21T03:38:54.442781+0000","last_undegraded":"2024-09-21T03:38:54.442781+0000","last_fullsized":"2024-09-21T03:38:54.442781+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_clean_scrub_stamp":"2024-09-21T03:37:09.809805+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:49:32.617768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"84'15","reported_seq":49,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:39:21.852488+0000","last_change":"2024-09-21T03:37:12.865408+0000","last_active":"2024-09-21T03:39:21.852488+0000","last_peered":"2024-09-21T03:39:21.852488+0000","last_clean":"2024-09-21T03:39:21.852488+0000","last_became_active":"2024-09-21T03:37:12.864972+0000","last_became_peered":"2024-09-21T03:37:12.864972+0000","last_unstale":"2024-09-21T03:39:21.852488+0000","last_undegraded":"2024-09-21T03:39:21.852488+0000","last_fullsized":"2024-09-21T03:39:21.852488+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:03:16.384260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:38:54.095856+0000","last_change":"2024-09-21T03:37:19.912105+0000","last_active":"2024-09-21T03:38:54.095856+0000","last_peered":"2024-09-21T03:38:54.095856+0000","last_clean":"2024-09-21T03:38:54.095856+0000","last_became_active":"2024-09-21T03:37:19.911416+0000","last_became_peered":"2024-09-21T03:37:19.911416+0000","last_unstale":"2024-09-21T03:38:54.095856+0000","last_undegraded":"2024-09-21T03:38:54.095856+0000","last_fullsized":"2024-09-21T03:38:54.095856+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_clean_scrub_stamp":"2024-09-21T03:37:18.876933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:20:16.492231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"84'5","reported_seq":40,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:39:21.865669+0000","last_change":"2024-09-21T03:37:12.862354+0000","last_active":"2024-09-21T03:39:21.865669+0000","last_peered":"2024-09-21T03:39:21.865669+0000","last_clean":"2024-09-21T03:39:21.865669+0000","last_became_active":"2024-09-21T03:37:12.861977+0000","last_became_peered":"2024-09-21T03:37:12.861977+0000","last_unstale":"2024-09-21T03:39:21.865669+0000","last_undegraded":"2024-09-21T03:39:21.865669+0000","last_fullsized":"2024-09-21T03:39:21.865669+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:53:52.725524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"80'4","reported_seq":44,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:38:54.442405+0000","last_change":"2024-09-21T03:37:10.832140+0000","last_active":"2024-09-21T03:38:54.442405+0000","last_peered":"2024-09-21T03:38:54.442405+0000","last_clean":"2024-09-21T03:38:54.442405+0000","last_became_active":"2024-09-21T03:37:10.831395+0000","last_became_peered":"2024-09-21T03:37:10.831395+0000","last_unstale":"2024-09-21T03:38:54.442405+0000","last_undegraded":"2024-09-21T03:38:54.442405+0000","last_fullsized":"2024-09-21T03:38:54.442405+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_clean_scrub_stamp":"2024-09-21T03:37:09.809805+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:55:20.440890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:38:54.096219+0000","last_change":"2024-09-21T03:37:20.160850+0000","last_active":"2024-09-21T03:38:54.096219+0000","last_peered":"2024-09-21T03:38:54.096219+0000","last_clean":"2024-09-21T03:38:54.096219+0000","last_became_active":"2024-09-21T03:37:20.160305+0000","last_became_peered":"2024-09-21T03:37:20.160305+0000","last_unstale":"2024-09-21T03:38:54.096219+0000","last_undegraded":"2024-09-21T03:38:54.096219+0000","last_fullsized":"2024-09-21T03:38:54.096219+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_clean_scrub_stamp":"2024-09-21T03:37:18.876933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:51:45.971984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"84'8","reported_seq":42,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:39:21.844639+0000","last_change":"2024-09-21T03:37:12.859485+0000","last_active":"2024-09-21T03:39:21.844639+0000","last_peered":"2024-09-21T03:39:21.844639+0000","last_clean":"2024-09-21T03:39:21.844639+0000","last_became_active":"2024-09-21T03:37:12.859097+0000","last_became_peered":"2024-09-21T03:37:12.859097+0000","last_unstale":"2024-09-21T03:39:21.844639+0000","last_undegraded":"2024-09-21T03:39:21.844639+0000","last_fullsized":"2024-09-21T03:39:21.844639+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:16:05.438360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"80'1","reported_seq":41,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:38:53.669958+0000","last_change":"2024-09-21T03:37:10.832441+0000","last_active":"2024-09-21T03:38:53.669958+0000","last_peered":"2024-09-21T03:38:53.669958+0000","last_clean":"2024-09-21T03:38:53.669958+0000","last_became_active":"2024-09-21T03:37:10.832073+0000","last_became_peered":"2024-09-21T03:37:10.832073+0000","last_unstale":"2024-09-21T03:38:53.669958+0000","last_undegraded":"2024-09-21T03:38:53.669958+0000","last_fullsized":"2024-09-21T03:38:53.669958+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_clean_scrub_stamp":"2024-09-21T03:37:09.809805+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:32:19.411097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"82'1","reported_seq":27,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:38:54.096331+0000","last_change":"2024-09-21T03:37:19.905490+0000","last_active":"2024-09-21T03:38:54.096331+0000","last_peered":"2024-09-21T03:38:54.096331+0000","last_clean":"2024-09-21T03:38:54.096331+0000","last_became_active":"2024-09-21T03:37:19.904971+0000","last_became_peered":"2024-09-21T03:37:19.904971+0000","last_unstale":"2024-09-21T03:38:54.096331+0000","last_undegraded":"2024-09-21T03:38:54.096331+0000","last_fullsized":"2024-09-21T03:38:54.096331+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_clean_scrub_stamp":"2024-09-21T03:37:18.876933+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:44:20.655538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"84'9","reported_seq":44,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:39:21.853193+0000","last_change":"2024-09-21T03:37:12.863597+0000","last_active":"2024-09-21T03:39:21.853193+0000","last_peered":"2024-09-21T03:39:21.853193+0000","last_clean":"2024-09-21T03:39:21.853193+0000","last_became_active":"2024-09-21T03:37:12.863223+0000","last_became_peered":"2024-09-21T03:37:12.863223+0000","last_unstale":"2024-09-21T03:39:21.853193+0000","last_undegraded":"2024-09-21T03:39:21.853193+0000","last_fullsized":"2024-09-21T03:39:21.853193+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:29:28.878964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.9","version":"80'3","reported_seq":43,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:38:53.971972+0000","last_change":"2024-09-21T03:37:10.834648+0000","last_active":"2024-09-21T03:38:53.971972+0000","last_peered":"2024-09-21T03:38:53.971972+0000","last_clean":"2024-09-21T03:38:53.971972+0000","last_became_active":"2024-09-21T03:37:10.834293+0000","last_became_peered":"2024-09-21T03:37:10.834293+0000","last_unstale":"2024-09-21T03:38:53.971972+0000","last_undegraded":"2024-09-21T03:38:53.971972+0000","last_fullsized":"2024-09-21T03:38:53.971972+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_clean_scrub_stamp":"2024-09-21T03:37:09.809805+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:10:41.248733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:38:53.672715+0000","last_change":"2024-09-21T03:37:19.912526+0000","last_active":"2024-09-21T03:38:53.672715+0000","last_peered":"2024-09-21T03:38:53.672715+0000","last_clean":"2024-09-21T03:38:53.672715+0000","last_became_active":"2024-09-21T03:37:19.912062+0000","last_became_peered":"2024-09-21T03:37:19.912062+0000","last_unstale":"2024-09-21T03:38:53.672715+0000","last_undegraded":"2024-09-21T03:38:53.672715+0000","last_fullsized":"2024-09-21T03:38:53.672715+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_clean_scrub_stamp":"2024-09-21T03:37:18.876933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:52:40.395834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.9","version":"84'2","reported_seq":37,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:39:21.852191+0000","last_change":"2024-09-21T03:37:12.855672+0000","last_active":"2024-09-21T03:39:21.852191+0000","last_peered":"2024-09-21T03:39:21.852191+0000","last_clean":"2024-09-21T03:39:21.852191+0000","last_became_active":"2024-09-21T03:37:12.855191+0000","last_became_peered":"2024-09-21T03:37:12.855191+0000","last_unstale":"2024-09-21T03:39:21.852191+0000","last_undegraded":"2024-09-21T03:39:21.852191+0000","last_fullsized":"2024-09-21T03:39:21.852191+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:26:56.115697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.8","version":"80'2","reported_seq":42,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:38:54.096441+0000","last_change":"2024-09-21T03:37:11.034752+0000","last_active":"2024-09-21T03:38:54.096441+0000","last_peered":"2024-09-21T03:38:54.096441+0000","last_clean":"2024-09-21T03:38:54.096441+0000","last_became_active":"2024-09-21T03:37:11.034386+0000","last_became_peered":"2024-09-21T03:37:11.034386+0000","last_unstale":"2024-09-21T03:38:54.096441+0000","last_undegraded":"2024-09-21T03:38:54.096441+0000","last_fullsized":"2024-09-21T03:38:54.096441+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_clean_scrub_stamp":"2024-09-21T03:37:09.809805+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:17:23.647411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:38:54.096423+0000","last_change":"2024-09-21T03:37:20.162051+0000","last_active":"2024-09-21T03:38:54.096423+0000","last_peered":"2024-09-21T03:38:54.096423+0000","last_clean":"2024-09-21T03:38:54.096423+0000","last_became_active":"2024-09-21T03:37:20.161452+0000","last_became_peered":"2024-09-21T03:37:20.161452+0000","last_unstale":"2024-09-21T03:38:54.096423+0000","last_undegraded":"2024-09-21T03:38:54.096423+0000","last_fullsized":"2024-09-21T03:38:54.096423+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_clean_scrub_stamp":"2024-09-21T03:37:18.876933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:29:22.544800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"84'7","reported_seq":42,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:39:21.851621+0000","last_change":"2024-09-21T03:37:12.866058+0000","last_active":"2024-09-21T03:39:21.851621+0000","last_peered":"2024-09-21T03:39:21.851621+0000","last_clean":"2024-09-21T03:39:21.851621+0000","last_became_active":"2024-09-21T03:37:12.865694+0000","last_became_peered":"2024-09-21T03:37:12.865694+0000","last_unstale":"2024-09-21T03:39:21.851621+0000","last_undegraded":"2024-09-21T03:39:21.851621+0000","last_fullsized":"2024-09-21T03:39:21.851621+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:56:33.629873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"80'4","reported_seq":44,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:38:54.096939+0000","last_change":"2024-09-21T03:37:11.358619+0000","last_active":"2024-09-21T03:38:54.096939+0000","last_peered":"2024-09-21T03:38:54.096939+0000","last_clean":"2024-09-21T03:38:54.096939+0000","last_became_active":"2024-09-21T03:37:11.358196+0000","last_became_peered":"2024-09-21T03:37:11.358196+0000","last_unstale":"2024-09-21T03:38:54.096939+0000","last_undegraded":"2024-09-21T03:38:54.096939+0000","last_fullsized":"2024-09-21T03:38:54.096939+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_clean_scrub_stamp":"2024-09-21T03:37:09.809805+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:57:13.476641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:38:53.669687+0000","last_change":"2024-09-21T03:37:19.905711+0000","last_active":"2024-09-21T03:38:53.669687+0000","last_peered":"2024-09-21T03:38:53.669687+0000","last_clean":"2024-09-21T03:38:53.669687+0000","last_became_active":"2024-09-21T03:37:19.905260+0000","last_became_peered":"2024-09-21T03:37:19.905260+0000","last_unstale":"2024-09-21T03:38:53.669687+0000","last_undegraded":"2024-09-21T03:38:53.669687+0000","last_fullsized":"2024-09-21T03:38:53.669687+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_clean_scrub_stamp":"2024-09-21T03:37:18.876933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:21:09.476082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"84'5","reported_seq":40,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:39:21.852928+0000","last_change":"2024-09-21T03:37:12.854376+0000","last_active":"2024-09-21T03:39:21.852928+0000","last_peered":"2024-09-21T03:39:21.852928+0000","last_clean":"2024-09-21T03:39:21.852928+0000","last_became_active":"2024-09-21T03:37:12.852942+0000","last_became_peered":"2024-09-21T03:37:12.852942+0000","last_unstale":"2024-09-21T03:39:21.852928+0000","last_undegraded":"2024-09-21T03:39:21.852928+0000","last_fullsized":"2024-09-21T03:39:21.852928+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:06:49.319002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"84'11","reported_seq":64,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:39:21.877193+0000","last_change":"2024-09-21T03:37:53.056889+0000","last_active":"2024-09-21T03:39:21.877193+0000","last_peered":"2024-09-21T03:39:21.877193+0000","last_clean":"2024-09-21T03:39:21.877193+0000","last_became_active":"2024-09-21T03:37:53.053054+0000","last_became_peered":"2024-09-21T03:37:53.053054+0000","last_unstale":"2024-09-21T03:39:21.877193+0000","last_undegraded":"2024-09-21T03:39:21.877193+0000","last_fullsized":"2024-09-21T03:39:21.877193+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_clean_scrub_stamp":"2024-09-21T03:37:09.809805+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:16:05.539111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,3],"acting":[9,11,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":27,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:38:53.971576+0000","last_change":"2024-09-21T03:37:20.161515+0000","last_active":"2024-09-21T03:38:53.971576+0000","last_peered":"2024-09-21T03:38:53.971576+0000","last_clean":"2024-09-21T03:38:53.971576+0000","last_became_active":"2024-09-21T03:37:20.160548+0000","last_became_peered":"2024-09-21T03:37:20.160548+0000","last_unstale":"2024-09-21T03:38:53.971576+0000","last_undegraded":"2024-09-21T03:38:53.971576+0000","last_fullsized":"2024-09-21T03:38:53.971576+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_clean_scrub_stamp":"2024-09-21T03:37:18.876933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:17:51.144859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"84'9","reported_seq":44,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:39:21.832796+0000","last_change":"2024-09-21T03:37:12.848305+0000","last_active":"2024-09-21T03:39:21.832796+0000","last_peered":"2024-09-21T03:39:21.832796+0000","last_clean":"2024-09-21T03:39:21.832796+0000","last_became_active":"2024-09-21T03:37:12.847916+0000","last_became_peered":"2024-09-21T03:37:12.847916+0000","last_unstale":"2024-09-21T03:39:21.832796+0000","last_undegraded":"2024-09-21T03:39:21.832796+0000","last_fullsized":"2024-09-21T03:39:21.832796+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:48:39.922805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"80'1","reported_seq":42,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:38:54.442955+0000","last_change":"2024-09-21T03:37:11.035964+0000","last_active":"2024-09-21T03:38:54.442955+0000","last_peered":"2024-09-21T03:38:54.442955+0000","last_clean":"2024-09-21T03:38:54.442955+0000","last_became_active":"2024-09-21T03:37:11.035427+0000","last_became_peered":"2024-09-21T03:37:11.035427+0000","last_unstale":"2024-09-21T03:38:54.442955+0000","last_undegraded":"2024-09-21T03:38:54.442955+0000","last_fullsized":"2024-09-21T03:38:54.442955+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_clean_scrub_stamp":"2024-09-21T03:37:09.809805+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:04:51.026753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":27,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:38:54.096205+0000","last_change":"2024-09-21T03:37:19.909428+0000","last_active":"2024-09-21T03:38:54.096205+0000","last_peered":"2024-09-21T03:38:54.096205+0000","last_clean":"2024-09-21T03:38:54.096205+0000","last_became_active":"2024-09-21T03:37:19.908904+0000","last_became_peered":"2024-09-21T03:37:19.908904+0000","last_unstale":"2024-09-21T03:38:54.096205+0000","last_undegraded":"2024-09-21T03:38:54.096205+0000","last_fullsized":"2024-09-21T03:38:54.096205+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_clean_scrub_stamp":"2024-09-21T03:37:18.876933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:12:07.564058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.5","version":"84'4","reported_seq":39,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:39:21.852816+0000","last_change":"2024-09-21T03:37:12.855592+0000","last_active":"2024-09-21T03:39:21.852816+0000","last_peered":"2024-09-21T03:39:21.852816+0000","last_clean":"2024-09-21T03:39:21.852816+0000","last_became_active":"2024-09-21T03:37:12.855244+0000","last_became_peered":"2024-09-21T03:37:12.855244+0000","last_unstale":"2024-09-21T03:39:21.852816+0000","last_undegraded":"2024-09-21T03:39:21.852816+0000","last_fullsized":"2024-09-21T03:39:21.852816+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:21:34.603141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":40,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:38:53.672861+0000","last_change":"2024-09-21T03:37:11.036401+0000","last_active":"2024-09-21T03:38:53.672861+0000","last_peered":"2024-09-21T03:38:53.672861+0000","last_clean":"2024-09-21T03:38:53.672861+0000","last_became_active":"2024-09-21T03:37:11.036003+0000","last_became_peered":"2024-09-21T03:37:11.036003+0000","last_unstale":"2024-09-21T03:38:53.672861+0000","last_undegraded":"2024-09-21T03:38:53.672861+0000","last_fullsized":"2024-09-21T03:38:53.672861+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_clean_scrub_stamp":"2024-09-21T03:37:09.809805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:35:50.549060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:38:54.442252+0000","last_change":"2024-09-21T03:37:20.160668+0000","last_active":"2024-09-21T03:38:54.442252+0000","last_peered":"2024-09-21T03:38:54.442252+0000","last_clean":"2024-09-21T03:38:54.442252+0000","last_became_active":"2024-09-21T03:37:20.160158+0000","last_became_peered":"2024-09-21T03:37:20.160158+0000","last_unstale":"2024-09-21T03:38:54.442252+0000","last_undegraded":"2024-09-21T03:38:54.442252+0000","last_fullsized":"2024-09-21T03:38:54.442252+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_clean_scrub_stamp":"2024-09-21T03:37:18.876933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:45:25.677667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3","version":"84'7","reported_seq":49,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:39:21.847550+0000","last_change":"2024-09-21T03:37:53.050351+0000","last_active":"2024-09-21T03:39:21.847550+0000","last_peered":"2024-09-21T03:39:21.847550+0000","last_clean":"2024-09-21T03:39:21.847550+0000","last_became_active":"2024-09-21T03:37:53.050003+0000","last_became_peered":"2024-09-21T03:37:53.050003+0000","last_unstale":"2024-09-21T03:39:21.847550+0000","last_undegraded":"2024-09-21T03:39:21.847550+0000","last_fullsized":"2024-09-21T03:39:21.847550+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:49:03.313002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,0],"acting":[10,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"80'1","reported_seq":41,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:38:53.942258+0000","last_change":"2024-09-21T03:37:10.832635+0000","last_active":"2024-09-21T03:38:53.942258+0000","last_peered":"2024-09-21T03:38:53.942258+0000","last_clean":"2024-09-21T03:38:53.942258+0000","last_became_active":"2024-09-21T03:37:10.832280+0000","last_became_peered":"2024-09-21T03:37:10.832280+0000","last_unstale":"2024-09-21T03:38:53.942258+0000","last_undegraded":"2024-09-21T03:38:53.942258+0000","last_fullsized":"2024-09-21T03:38:53.942258+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_clean_scrub_stamp":"2024-09-21T03:37:09.809805+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:54:52.744486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":27,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:38:53.668675+0000","last_change":"2024-09-21T03:37:19.904112+0000","last_active":"2024-09-21T03:38:53.668675+0000","last_peered":"2024-09-21T03:38:53.668675+0000","last_clean":"2024-09-21T03:38:53.668675+0000","last_became_active":"2024-09-21T03:37:19.903677+0000","last_became_peered":"2024-09-21T03:37:19.903677+0000","last_unstale":"2024-09-21T03:38:53.668675+0000","last_undegraded":"2024-09-21T03:38:53.668675+0000","last_fullsized":"2024-09-21T03:38:53.668675+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_clean_scrub_stamp":"2024-09-21T03:37:18.876933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:32:20.096850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"84'8","reported_seq":43,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:39:21.847619+0000","last_change":"2024-09-21T03:37:12.864082+0000","last_active":"2024-09-21T03:39:21.847619+0000","last_peered":"2024-09-21T03:39:21.847619+0000","last_clean":"2024-09-21T03:39:21.847619+0000","last_became_active":"2024-09-21T03:37:12.862745+0000","last_became_peered":"2024-09-21T03:37:12.862745+0000","last_unstale":"2024-09-21T03:39:21.847619+0000","last_undegraded":"2024-09-21T03:39:21.847619+0000","last_fullsized":"2024-09-21T03:39:21.847619+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:25:35.759719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"84'5","reported_seq":141,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:39:23.346185+0000","last_change":"2024-09-21T03:37:11.034463+0000","last_active":"2024-09-21T03:39:23.346185+0000","last_peered":"2024-09-21T03:39:23.346185+0000","last_clean":"2024-09-21T03:39:23.346185+0000","last_became_active":"2024-09-21T03:37:11.033932+0000","last_became_peered":"2024-09-21T03:37:11.033932+0000","last_unstale":"2024-09-21T03:39:23.346185+0000","last_undegraded":"2024-09-21T03:39:23.346185+0000","last_fullsized":"2024-09-21T03:39:23.346185+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_clean_scrub_stamp":"2024-09-21T03:37:09.809805+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:15:18.595442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":96,"num_read_kb":96,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:38:53.668706+0000","last_change":"2024-09-21T03:37:20.160789+0000","last_active":"2024-09-21T03:38:53.668706+0000","last_peered":"2024-09-21T03:38:53.668706+0000","last_clean":"2024-09-21T03:38:53.668706+0000","last_became_active":"2024-09-21T03:37:20.160231+0000","last_became_peered":"2024-09-21T03:37:20.160231+0000","last_unstale":"2024-09-21T03:38:53.668706+0000","last_undegraded":"2024-09-21T03:38:53.668706+0000","last_fullsized":"2024-09-21T03:38:53.668706+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_clean_scrub_stamp":"2024-09-21T03:37:18.876933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:07:16.762249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"84'5","reported_seq":40,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:39:21.852307+0000","last_change":"2024-09-21T03:37:12.853178+0000","last_active":"2024-09-21T03:39:21.852307+0000","last_peered":"2024-09-21T03:39:21.852307+0000","last_clean":"2024-09-21T03:39:21.852307+0000","last_became_active":"2024-09-21T03:37:12.852822+0000","last_became_peered":"2024-09-21T03:37:12.852822+0000","last_unstale":"2024-09-21T03:39:21.852307+0000","last_undegraded":"2024-09-21T03:39:21.852307+0000","last_fullsized":"2024-09-21T03:39:21.852307+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:48:26.146593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"80'3","reported_seq":43,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:38:53.941666+0000","last_change":"2024-09-21T03:37:10.838939+0000","last_active":"2024-09-21T03:38:53.941666+0000","last_peered":"2024-09-21T03:38:53.941666+0000","last_clean":"2024-09-21T03:38:53.941666+0000","last_became_active":"2024-09-21T03:37:10.838562+0000","last_became_peered":"2024-09-21T03:37:10.838562+0000","last_unstale":"2024-09-21T03:38:53.941666+0000","last_undegraded":"2024-09-21T03:38:53.941666+0000","last_fullsized":"2024-09-21T03:38:53.941666+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_clean_scrub_stamp":"2024-09-21T03:37:09.809805+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:35:32.936824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":34,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:38:54.097042+0000","last_change":"2024-09-21T03:37:53.052940+0000","last_active":"2024-09-21T03:38:54.097042+0000","last_peered":"2024-09-21T03:38:54.097042+0000","last_clean":"2024-09-21T03:38:54.097042+0000","last_became_active":"2024-09-21T03:37:53.052310+0000","last_became_peered":"2024-09-21T03:37:53.052310+0000","last_unstale":"2024-09-21T03:38:54.097042+0000","last_undegraded":"2024-09-21T03:38:54.097042+0000","last_fullsized":"2024-09-21T03:38:54.097042+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_clean_scrub_stamp":"2024-09-21T03:37:18.876933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:31:35.353226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,11,6,10],"acting":[5,11,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"84'8","reported_seq":43,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:39:21.847226+0000","last_change":"2024-09-21T03:37:12.869795+0000","last_active":"2024-09-21T03:39:21.847226+0000","last_peered":"2024-09-21T03:39:21.847226+0000","last_clean":"2024-09-21T03:39:21.847226+0000","last_became_active":"2024-09-21T03:37:12.869381+0000","last_became_peered":"2024-09-21T03:37:12.869381+0000","last_unstale":"2024-09-21T03:39:21.847226+0000","last_undegraded":"2024-09-21T03:39:21.847226+0000","last_fullsized":"2024-09-21T03:39:21.847226+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:39:29.437552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"80'3","reported_seq":45,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:38:54.096584+0000","last_change":"2024-09-21T03:37:10.834853+0000","last_active":"2024-09-21T03:38:54.096584+0000","last_peered":"2024-09-21T03:38:54.096584+0000","last_clean":"2024-09-21T03:38:54.096584+0000","last_became_active":"2024-09-21T03:37:10.834402+0000","last_became_peered":"2024-09-21T03:37:10.834402+0000","last_unstale":"2024-09-21T03:38:54.096584+0000","last_undegraded":"2024-09-21T03:38:54.096584+0000","last_fullsized":"2024-09-21T03:38:54.096584+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_clean_scrub_stamp":"2024-09-21T03:37:09.809805+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:57:29.668570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"20'92","reported_seq":113,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:38:53.941746+0000","last_change":"2024-09-21T03:36:00.521781+0000","last_active":"2024-09-21T03:38:53.941746+0000","last_peered":"2024-09-21T03:38:53.941746+0000","last_clean":"2024-09-21T03:38:53.941746+0000","last_became_active":"2024-09-21T03:36:00.516468+0000","last_became_peered":"2024-09-21T03:36:00.516468+0000","last_unstale":"2024-09-21T03:38:53.941746+0000","last_undegraded":"2024-09-21T03:38:53.941746+0000","last_fullsized":"2024-09-21T03:38:53.941746+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:32:38.215432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:32:38.215432+0000","last_clean_scrub_stamp":"2024-09-21T03:32:38.215432+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:02:16.252544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"82'1","reported_seq":27,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:38:53.941730+0000","last_change":"2024-09-21T03:37:19.911309+0000","last_active":"2024-09-21T03:38:53.941730+0000","last_peered":"2024-09-21T03:38:53.941730+0000","last_clean":"2024-09-21T03:38:53.941730+0000","last_became_active":"2024-09-21T03:37:19.910778+0000","last_became_peered":"2024-09-21T03:37:19.910778+0000","last_unstale":"2024-09-21T03:38:53.941730+0000","last_undegraded":"2024-09-21T03:38:53.941730+0000","last_fullsized":"2024-09-21T03:38:53.941730+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_clean_scrub_stamp":"2024-09-21T03:37:18.876933+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:52:23.307983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"84'7","reported_seq":41,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:39:21.864440+0000","last_change":"2024-09-21T03:37:12.860707+0000","last_active":"2024-09-21T03:39:21.864440+0000","last_peered":"2024-09-21T03:39:21.864440+0000","last_clean":"2024-09-21T03:39:21.864440+0000","last_became_active":"2024-09-21T03:37:12.860341+0000","last_became_peered":"2024-09-21T03:37:12.860341+0000","last_unstale":"2024-09-21T03:39:21.864440+0000","last_undegraded":"2024-09-21T03:39:21.864440+0000","last_fullsized":"2024-09-21T03:39:21.864440+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:55:15.771259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"80'3","reported_seq":22,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:38:54.095929+0000","last_change":"2024-09-21T03:37:53.071125+0000","last_active":"2024-09-21T03:38:54.095929+0000","last_peered":"2024-09-21T03:38:54.095929+0000","last_clean":"2024-09-21T03:38:54.095929+0000","last_became_active":"2024-09-21T03:37:53.052237+0000","last_became_peered":"2024-09-21T03:37:53.052237+0000","last_unstale":"2024-09-21T03:38:54.095929+0000","last_undegraded":"2024-09-21T03:38:54.095929+0000","last_fullsized":"2024-09-21T03:38:54.095929+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_clean_scrub_stamp":"2024-09-21T03:37:09.809805+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:47:17.707045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,0],"acting":[4,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:38:54.096435+0000","last_change":"2024-09-21T03:37:19.909292+0000","last_active":"2024-09-21T03:38:54.096435+0000","last_peered":"2024-09-21T03:38:54.096435+0000","last_clean":"2024-09-21T03:38:54.096435+0000","last_became_active":"2024-09-21T03:37:19.908681+0000","last_became_peered":"2024-09-21T03:37:19.908681+0000","last_unstale":"2024-09-21T03:38:54.096435+0000","last_undegraded":"2024-09-21T03:38:54.096435+0000","last_fullsized":"2024-09-21T03:38:54.096435+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_clean_scrub_stamp":"2024-09-21T03:37:18.876933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:15:49.623109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"84'6","reported_seq":40,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:39:21.852497+0000","last_change":"2024-09-21T03:37:12.857678+0000","last_active":"2024-09-21T03:39:21.852497+0000","last_peered":"2024-09-21T03:39:21.852497+0000","last_clean":"2024-09-21T03:39:21.852497+0000","last_became_active":"2024-09-21T03:37:12.857266+0000","last_became_peered":"2024-09-21T03:37:12.857266+0000","last_unstale":"2024-09-21T03:39:21.852497+0000","last_undegraded":"2024-09-21T03:39:21.852497+0000","last_fullsized":"2024-09-21T03:39:21.852497+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:21:14.135784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"74'1","reported_seq":41,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:38:54.097232+0000","last_change":"2024-09-21T03:37:10.835323+0000","last_active":"2024-09-21T03:38:54.097232+0000","last_peered":"2024-09-21T03:38:54.097232+0000","last_clean":"2024-09-21T03:38:54.097232+0000","last_became_active":"2024-09-21T03:37:10.834950+0000","last_became_peered":"2024-09-21T03:37:10.834950+0000","last_unstale":"2024-09-21T03:38:54.097232+0000","last_undegraded":"2024-09-21T03:38:54.097232+0000","last_fullsized":"2024-09-21T03:38:54.097232+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_clean_scrub_stamp":"2024-09-21T03:37:09.809805+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:40:27.386342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:38:54.095581+0000","last_change":"2024-09-21T03:37:19.906360+0000","last_active":"2024-09-21T03:38:54.095581+0000","last_peered":"2024-09-21T03:38:54.095581+0000","last_clean":"2024-09-21T03:38:54.095581+0000","last_became_active":"2024-09-21T03:37:19.905888+0000","last_became_peered":"2024-09-21T03:37:19.905888+0000","last_unstale":"2024-09-21T03:38:54.095581+0000","last_undegraded":"2024-09-21T03:38:54.095581+0000","last_fullsized":"2024-09-21T03:38:54.095581+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_clean_scrub_stamp":"2024-09-21T03:37:18.876933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:13:55.885538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"84'9","reported_seq":43,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:39:21.849514+0000","last_change":"2024-09-21T03:37:12.849106+0000","last_active":"2024-09-21T03:39:21.849514+0000","last_peered":"2024-09-21T03:39:21.849514+0000","last_clean":"2024-09-21T03:39:21.849514+0000","last_became_active":"2024-09-21T03:37:12.848505+0000","last_became_peered":"2024-09-21T03:37:12.848505+0000","last_unstale":"2024-09-21T03:39:21.849514+0000","last_undegraded":"2024-09-21T03:39:21.849514+0000","last_fullsized":"2024-09-21T03:39:21.849514+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:50:19.569004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"74'2","reported_seq":52,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:38:53.668984+0000","last_change":"2024-09-21T03:37:53.115158+0000","last_active":"2024-09-21T03:38:53.668984+0000","last_peered":"2024-09-21T03:38:53.668984+0000","last_clean":"2024-09-21T03:38:53.668984+0000","last_became_active":"2024-09-21T03:37:53.112603+0000","last_became_peered":"2024-09-21T03:37:53.112603+0000","last_unstale":"2024-09-21T03:38:53.668984+0000","last_undegraded":"2024-09-21T03:38:53.668984+0000","last_fullsized":"2024-09-21T03:38:53.668984+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_clean_scrub_stamp":"2024-09-21T03:37:09.809805+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:23:21.525972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,1],"acting":[6,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:38:54.095667+0000","last_change":"2024-09-21T03:37:19.909040+0000","last_active":"2024-09-21T03:38:54.095667+0000","last_peered":"2024-09-21T03:38:54.095667+0000","last_clean":"2024-09-21T03:38:54.095667+0000","last_became_active":"2024-09-21T03:37:19.908530+0000","last_became_peered":"2024-09-21T03:37:19.908530+0000","last_unstale":"2024-09-21T03:38:54.095667+0000","last_undegraded":"2024-09-21T03:38:54.095667+0000","last_fullsized":"2024-09-21T03:38:54.095667+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_clean_scrub_stamp":"2024-09-21T03:37:18.876933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:35:20.919260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"84'5","reported_seq":39,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:39:21.851823+0000","last_change":"2024-09-21T03:37:12.858984+0000","last_active":"2024-09-21T03:39:21.851823+0000","last_peered":"2024-09-21T03:39:21.851823+0000","last_clean":"2024-09-21T03:39:21.851823+0000","last_became_active":"2024-09-21T03:37:12.858603+0000","last_became_peered":"2024-09-21T03:37:12.858603+0000","last_unstale":"2024-09-21T03:39:21.851823+0000","last_undegraded":"2024-09-21T03:39:21.851823+0000","last_fullsized":"2024-09-21T03:39:21.851823+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:16:45.025304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.f","version":"80'2","reported_seq":42,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:38:54.442639+0000","last_change":"2024-09-21T03:37:10.834844+0000","last_active":"2024-09-21T03:38:54.442639+0000","last_peered":"2024-09-21T03:38:54.442639+0000","last_clean":"2024-09-21T03:38:54.442639+0000","last_became_active":"2024-09-21T03:37:10.833664+0000","last_became_peered":"2024-09-21T03:37:10.833664+0000","last_unstale":"2024-09-21T03:38:54.442639+0000","last_undegraded":"2024-09-21T03:38:54.442639+0000","last_fullsized":"2024-09-21T03:38:54.442639+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_clean_scrub_stamp":"2024-09-21T03:37:09.809805+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:03:49.006414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:38:54.095855+0000","last_change":"2024-09-21T03:37:19.913277+0000","last_active":"2024-09-21T03:38:54.095855+0000","last_peered":"2024-09-21T03:38:54.095855+0000","last_clean":"2024-09-21T03:38:54.095855+0000","last_became_active":"2024-09-21T03:37:19.912781+0000","last_became_peered":"2024-09-21T03:37:19.912781+0000","last_unstale":"2024-09-21T03:38:54.095855+0000","last_undegraded":"2024-09-21T03:38:54.095855+0000","last_fullsized":"2024-09-21T03:38:54.095855+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_clean_scrub_stamp":"2024-09-21T03:37:18.876933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:18:27.980622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"84'6","reported_seq":40,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:39:21.847384+0000","last_change":"2024-09-21T03:37:12.862322+0000","last_active":"2024-09-21T03:39:21.847384+0000","last_peered":"2024-09-21T03:39:21.847384+0000","last_clean":"2024-09-21T03:39:21.847384+0000","last_became_active":"2024-09-21T03:37:12.861938+0000","last_became_peered":"2024-09-21T03:37:12.861938+0000","last_unstale":"2024-09-21T03:39:21.847384+0000","last_undegraded":"2024-09-21T03:39:21.847384+0000","last_fullsized":"2024-09-21T03:39:21.847384+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:01:48.944169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.10","version":"80'15","reported_seq":34,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:38:54.096078+0000","last_change":"2024-09-21T03:37:53.067130+0000","last_active":"2024-09-21T03:38:54.096078+0000","last_peered":"2024-09-21T03:38:54.096078+0000","last_clean":"2024-09-21T03:38:54.096078+0000","last_became_active":"2024-09-21T03:37:53.051596+0000","last_became_peered":"2024-09-21T03:37:53.051596+0000","last_unstale":"2024-09-21T03:38:54.096078+0000","last_undegraded":"2024-09-21T03:38:54.096078+0000","last_fullsized":"2024-09-21T03:38:54.096078+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_clean_scrub_stamp":"2024-09-21T03:37:09.809805+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:02:22.852015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:38:53.668923+0000","last_change":"2024-09-21T03:37:19.913206+0000","last_active":"2024-09-21T03:38:53.668923+0000","last_peered":"2024-09-21T03:38:53.668923+0000","last_clean":"2024-09-21T03:38:53.668923+0000","last_became_active":"2024-09-21T03:37:19.910718+0000","last_became_peered":"2024-09-21T03:37:19.910718+0000","last_unstale":"2024-09-21T03:38:53.668923+0000","last_undegraded":"2024-09-21T03:38:53.668923+0000","last_fullsized":"2024-09-21T03:38:53.668923+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_clean_scrub_stamp":"2024-09-21T03:37:18.876933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:11:31.431237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"84'6","reported_seq":40,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:39:21.854210+0000","last_change":"2024-09-21T03:37:12.857505+0000","last_active":"2024-09-21T03:39:21.854210+0000","last_peered":"2024-09-21T03:39:21.854210+0000","last_clean":"2024-09-21T03:39:21.854210+0000","last_became_active":"2024-09-21T03:37:12.857155+0000","last_became_peered":"2024-09-21T03:37:12.857155+0000","last_unstale":"2024-09-21T03:39:21.854210+0000","last_undegraded":"2024-09-21T03:39:21.854210+0000","last_fullsized":"2024-09-21T03:39:21.854210+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:05:50.387651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"80'3","reported_seq":138,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:39:21.178872+0000","last_change":"2024-09-21T03:37:10.832364+0000","last_active":"2024-09-21T03:39:21.178872+0000","last_peered":"2024-09-21T03:39:21.178872+0000","last_clean":"2024-09-21T03:39:21.178872+0000","last_became_active":"2024-09-21T03:37:10.832004+0000","last_became_peered":"2024-09-21T03:37:10.832004+0000","last_unstale":"2024-09-21T03:39:21.178872+0000","last_undegraded":"2024-09-21T03:39:21.178872+0000","last_fullsized":"2024-09-21T03:39:21.178872+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_clean_scrub_stamp":"2024-09-21T03:37:09.809805+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:21:10.895732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":97,"num_read_kb":3,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:38:53.672765+0000","last_change":"2024-09-21T03:37:19.912297+0000","last_active":"2024-09-21T03:38:53.672765+0000","last_peered":"2024-09-21T03:38:53.672765+0000","last_clean":"2024-09-21T03:38:53.672765+0000","last_became_active":"2024-09-21T03:37:19.911903+0000","last_became_peered":"2024-09-21T03:37:19.911903+0000","last_unstale":"2024-09-21T03:38:53.672765+0000","last_undegraded":"2024-09-21T03:38:53.672765+0000","last_fullsized":"2024-09-21T03:38:53.672765+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_clean_scrub_stamp":"2024-09-21T03:37:18.876933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:05:56.747246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.13","version":"84'15","reported_seq":49,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:39:21.853966+0000","last_change":"2024-09-21T03:37:12.856230+0000","last_active":"2024-09-21T03:39:21.853966+0000","last_peered":"2024-09-21T03:39:21.853966+0000","last_clean":"2024-09-21T03:39:21.853966+0000","last_became_active":"2024-09-21T03:37:12.855819+0000","last_became_peered":"2024-09-21T03:37:12.855819+0000","last_unstale":"2024-09-21T03:39:21.853966+0000","last_undegraded":"2024-09-21T03:39:21.853966+0000","last_fullsized":"2024-09-21T03:39:21.853966+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:38:18.618579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"80'3","reported_seq":42,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:38:54.096706+0000","last_change":"2024-09-21T03:37:10.832401+0000","last_active":"2024-09-21T03:38:54.096706+0000","last_peered":"2024-09-21T03:38:54.096706+0000","last_clean":"2024-09-21T03:38:54.096706+0000","last_became_active":"2024-09-21T03:37:10.832033+0000","last_became_peered":"2024-09-21T03:37:10.832033+0000","last_unstale":"2024-09-21T03:38:54.096706+0000","last_undegraded":"2024-09-21T03:38:54.096706+0000","last_fullsized":"2024-09-21T03:38:54.096706+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_clean_scrub_stamp":"2024-09-21T03:37:09.809805+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:25:20.751529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:38:54.096414+0000","last_change":"2024-09-21T03:37:19.905825+0000","last_active":"2024-09-21T03:38:54.096414+0000","last_peered":"2024-09-21T03:38:54.096414+0000","last_clean":"2024-09-21T03:38:54.096414+0000","last_became_active":"2024-09-21T03:37:19.905352+0000","last_became_peered":"2024-09-21T03:37:19.905352+0000","last_unstale":"2024-09-21T03:38:54.096414+0000","last_undegraded":"2024-09-21T03:38:54.096414+0000","last_fullsized":"2024-09-21T03:38:54.096414+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_clean_scrub_stamp":"2024-09-21T03:37:18.876933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:25:05.861981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"84'9","reported_seq":43,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:39:21.847167+0000","last_change":"2024-09-21T03:37:12.852006+0000","last_active":"2024-09-21T03:39:21.847167+0000","last_peered":"2024-09-21T03:39:21.847167+0000","last_clean":"2024-09-21T03:39:21.847167+0000","last_became_active":"2024-09-21T03:37:12.851670+0000","last_became_peered":"2024-09-21T03:37:12.851670+0000","last_unstale":"2024-09-21T03:39:21.847167+0000","last_undegraded":"2024-09-21T03:39:21.847167+0000","last_fullsized":"2024-09-21T03:39:21.847167+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:41:50.177349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"80'6","reported_seq":46,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:38:53.670064+0000","last_change":"2024-09-21T03:37:10.832472+0000","last_active":"2024-09-21T03:38:53.670064+0000","last_peered":"2024-09-21T03:38:53.670064+0000","last_clean":"2024-09-21T03:38:53.670064+0000","last_became_active":"2024-09-21T03:37:10.832085+0000","last_became_peered":"2024-09-21T03:37:10.832085+0000","last_unstale":"2024-09-21T03:38:53.670064+0000","last_undegraded":"2024-09-21T03:38:53.670064+0000","last_fullsized":"2024-09-21T03:38:53.670064+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_clean_scrub_stamp":"2024-09-21T03:37:09.809805+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:06:41.251218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:38:54.096945+0000","last_change":"2024-09-21T03:37:19.914042+0000","last_active":"2024-09-21T03:38:54.096945+0000","last_peered":"2024-09-21T03:38:54.096945+0000","last_clean":"2024-09-21T03:38:54.096945+0000","last_became_active":"2024-09-21T03:37:19.913531+0000","last_became_peered":"2024-09-21T03:37:19.913531+0000","last_unstale":"2024-09-21T03:38:54.096945+0000","last_undegraded":"2024-09-21T03:38:54.096945+0000","last_fullsized":"2024-09-21T03:38:54.096945+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_clean_scrub_stamp":"2024-09-21T03:37:18.876933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:16:40.163359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"84'6","reported_seq":40,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:39:21.852857+0000","last_change":"2024-09-21T03:37:12.852893+0000","last_active":"2024-09-21T03:39:21.852857+0000","last_peered":"2024-09-21T03:39:21.852857+0000","last_clean":"2024-09-21T03:39:21.852857+0000","last_became_active":"2024-09-21T03:37:12.852452+0000","last_became_peered":"2024-09-21T03:37:12.852452+0000","last_unstale":"2024-09-21T03:39:21.852857+0000","last_undegraded":"2024-09-21T03:39:21.852857+0000","last_fullsized":"2024-09-21T03:39:21.852857+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:46:59.393901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.14","version":"84'36","reported_seq":278,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:39:24.185435+0000","last_change":"2024-09-21T03:37:10.837654+0000","last_active":"2024-09-21T03:39:24.185435+0000","last_peered":"2024-09-21T03:39:24.185435+0000","last_clean":"2024-09-21T03:39:24.185435+0000","last_became_active":"2024-09-21T03:37:10.837251+0000","last_became_peered":"2024-09-21T03:37:10.837251+0000","last_unstale":"2024-09-21T03:39:24.185435+0000","last_undegraded":"2024-09-21T03:39:24.185435+0000","last_fullsized":"2024-09-21T03:39:24.185435+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_clean_scrub_stamp":"2024-09-21T03:37:09.809805+0000","objects_scrubbed":0,"log_size":36,"log_dups_size":0,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:43:32.986371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":67880,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":203,"num_read_kb":168,"num_write":37,"num_write_kb":84,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:38:53.669528+0000","last_change":"2024-09-21T03:37:53.053405+0000","last_active":"2024-09-21T03:38:53.669528+0000","last_peered":"2024-09-21T03:38:53.669528+0000","last_clean":"2024-09-21T03:38:53.669528+0000","last_became_active":"2024-09-21T03:37:53.052818+0000","last_became_peered":"2024-09-21T03:37:53.052818+0000","last_unstale":"2024-09-21T03:38:53.669528+0000","last_undegraded":"2024-09-21T03:38:53.669528+0000","last_fullsized":"2024-09-21T03:38:53.669528+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_clean_scrub_stamp":"2024-09-21T03:37:18.876933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:05:09.121036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,11,5],"acting":[10,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.14","version":"84'11","reported_seq":45,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:39:21.857069+0000","last_change":"2024-09-21T03:37:12.866588+0000","last_active":"2024-09-21T03:39:21.857069+0000","last_peered":"2024-09-21T03:39:21.857069+0000","last_clean":"2024-09-21T03:39:21.857069+0000","last_became_active":"2024-09-21T03:37:12.866075+0000","last_became_peered":"2024-09-21T03:37:12.866075+0000","last_unstale":"2024-09-21T03:39:21.857069+0000","last_undegraded":"2024-09-21T03:39:21.857069+0000","last_fullsized":"2024-09-21T03:39:21.857069+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:18:48.817562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"80'4","reported_seq":44,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:38:54.097208+0000","last_change":"2024-09-21T03:37:11.036751+0000","last_active":"2024-09-21T03:38:54.097208+0000","last_peered":"2024-09-21T03:38:54.097208+0000","last_clean":"2024-09-21T03:38:54.097208+0000","last_became_active":"2024-09-21T03:37:11.036125+0000","last_became_peered":"2024-09-21T03:37:11.036125+0000","last_unstale":"2024-09-21T03:38:54.097208+0000","last_undegraded":"2024-09-21T03:38:54.097208+0000","last_fullsized":"2024-09-21T03:38:54.097208+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_clean_scrub_stamp":"2024-09-21T03:37:09.809805+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:46:13.569422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":34,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:38:54.095956+0000","last_change":"2024-09-21T03:37:53.112845+0000","last_active":"2024-09-21T03:38:54.095956+0000","last_peered":"2024-09-21T03:38:54.095956+0000","last_clean":"2024-09-21T03:38:54.095956+0000","last_became_active":"2024-09-21T03:37:53.112286+0000","last_became_peered":"2024-09-21T03:37:53.112286+0000","last_unstale":"2024-09-21T03:38:54.095956+0000","last_undegraded":"2024-09-21T03:38:54.095956+0000","last_fullsized":"2024-09-21T03:38:54.095956+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_clean_scrub_stamp":"2024-09-21T03:37:18.876933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:29:58.062684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,8],"acting":[4,1,2,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"84'9","reported_seq":43,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:39:21.851704+0000","last_change":"2024-09-21T03:37:12.865011+0000","last_active":"2024-09-21T03:39:21.851704+0000","last_peered":"2024-09-21T03:39:21.851704+0000","last_clean":"2024-09-21T03:39:21.851704+0000","last_became_active":"2024-09-21T03:37:12.864645+0000","last_became_peered":"2024-09-21T03:37:12.864645+0000","last_unstale":"2024-09-21T03:39:21.851704+0000","last_undegraded":"2024-09-21T03:39:21.851704+0000","last_fullsized":"2024-09-21T03:39:21.851704+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:27:42.471227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"80'2","reported_seq":41,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:38:53.941930+0000","last_change":"2024-09-21T03:37:10.835171+0000","last_active":"2024-09-21T03:38:53.941930+0000","last_peered":"2024-09-21T03:38:53.941930+0000","last_clean":"2024-09-21T03:38:53.941930+0000","last_became_active":"2024-09-21T03:37:10.834744+0000","last_became_peered":"2024-09-21T03:37:10.834744+0000","last_unstale":"2024-09-21T03:38:53.941930+0000","last_undegraded":"2024-09-21T03:38:53.941930+0000","last_fullsized":"2024-09-21T03:38:53.941930+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_clean_scrub_stamp":"2024-09-21T03:37:09.809805+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:02:37.720457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:38:53.672239+0000","last_change":"2024-09-21T03:37:19.906734+0000","last_active":"2024-09-21T03:38:53.672239+0000","last_peered":"2024-09-21T03:38:53.672239+0000","last_clean":"2024-09-21T03:38:53.672239+0000","last_became_active":"2024-09-21T03:37:19.906327+0000","last_became_peered":"2024-09-21T03:37:19.906327+0000","last_unstale":"2024-09-21T03:38:53.672239+0000","last_undegraded":"2024-09-21T03:38:53.672239+0000","last_fullsized":"2024-09-21T03:38:53.672239+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_clean_scrub_stamp":"2024-09-21T03:37:18.876933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:55:11.603042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.16","version":"84'10","reported_seq":44,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:39:21.866214+0000","last_change":"2024-09-21T03:37:12.866285+0000","last_active":"2024-09-21T03:39:21.866214+0000","last_peered":"2024-09-21T03:39:21.866214+0000","last_clean":"2024-09-21T03:39:21.866214+0000","last_became_active":"2024-09-21T03:37:12.865854+0000","last_became_peered":"2024-09-21T03:37:12.865854+0000","last_unstale":"2024-09-21T03:39:21.866214+0000","last_undegraded":"2024-09-21T03:39:21.866214+0000","last_fullsized":"2024-09-21T03:39:21.866214+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:00:18.075248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"84'4","reported_seq":139,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:39:23.182875+0000","last_change":"2024-09-21T03:37:10.835808+0000","last_active":"2024-09-21T03:39:23.182875+0000","last_peered":"2024-09-21T03:39:23.182875+0000","last_clean":"2024-09-21T03:39:23.182875+0000","last_became_active":"2024-09-21T03:37:10.835434+0000","last_became_peered":"2024-09-21T03:37:10.835434+0000","last_unstale":"2024-09-21T03:39:23.182875+0000","last_undegraded":"2024-09-21T03:39:23.182875+0000","last_fullsized":"2024-09-21T03:39:23.182875+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_clean_scrub_stamp":"2024-09-21T03:37:09.809805+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:35:39.817312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":96,"num_read_kb":96,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:38:53.971629+0000","last_change":"2024-09-21T03:37:19.915986+0000","last_active":"2024-09-21T03:38:53.971629+0000","last_peered":"2024-09-21T03:38:53.971629+0000","last_clean":"2024-09-21T03:38:53.971629+0000","last_became_active":"2024-09-21T03:37:19.915561+0000","last_became_peered":"2024-09-21T03:37:19.915561+0000","last_unstale":"2024-09-21T03:38:53.971629+0000","last_undegraded":"2024-09-21T03:38:53.971629+0000","last_fullsized":"2024-09-21T03:38:53.971629+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_clean_scrub_stamp":"2024-09-21T03:37:18.876933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:21:13.789664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":27,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:38:54.095727+0000","last_change":"2024-09-21T03:37:19.908230+0000","last_active":"2024-09-21T03:38:54.095727+0000","last_peered":"2024-09-21T03:38:54.095727+0000","last_clean":"2024-09-21T03:38:54.095727+0000","last_became_active":"2024-09-21T03:37:19.907843+0000","last_became_peered":"2024-09-21T03:37:19.907843+0000","last_unstale":"2024-09-21T03:38:54.095727+0000","last_undegraded":"2024-09-21T03:38:54.095727+0000","last_fullsized":"2024-09-21T03:38:54.095727+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_clean_scrub_stamp":"2024-09-21T03:37:18.876933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:15:58.142127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"84'11","reported_seq":45,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:39:21.851083+0000","last_change":"2024-09-21T03:37:12.856709+0000","last_active":"2024-09-21T03:39:21.851083+0000","last_peered":"2024-09-21T03:39:21.851083+0000","last_clean":"2024-09-21T03:39:21.851083+0000","last_became_active":"2024-09-21T03:37:12.856278+0000","last_became_peered":"2024-09-21T03:37:12.856278+0000","last_unstale":"2024-09-21T03:39:21.851083+0000","last_undegraded":"2024-09-21T03:39:21.851083+0000","last_fullsized":"2024-09-21T03:39:21.851083+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:16:13.884648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"80'1","reported_seq":40,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:38:54.096239+0000","last_change":"2024-09-21T03:37:10.832456+0000","last_active":"2024-09-21T03:38:54.096239+0000","last_peered":"2024-09-21T03:38:54.096239+0000","last_clean":"2024-09-21T03:38:54.096239+0000","last_became_active":"2024-09-21T03:37:10.832082+0000","last_became_peered":"2024-09-21T03:37:10.832082+0000","last_unstale":"2024-09-21T03:38:54.096239+0000","last_undegraded":"2024-09-21T03:38:54.096239+0000","last_fullsized":"2024-09-21T03:38:54.096239+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_clean_scrub_stamp":"2024-09-21T03:37:09.809805+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:17:12.170616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":27,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:38:54.096158+0000","last_change":"2024-09-21T03:37:19.908084+0000","last_active":"2024-09-21T03:38:54.096158+0000","last_peered":"2024-09-21T03:38:54.096158+0000","last_clean":"2024-09-21T03:38:54.096158+0000","last_became_active":"2024-09-21T03:37:19.907307+0000","last_became_peered":"2024-09-21T03:37:19.907307+0000","last_unstale":"2024-09-21T03:38:54.096158+0000","last_undegraded":"2024-09-21T03:38:54.096158+0000","last_fullsized":"2024-09-21T03:38:54.096158+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_clean_scrub_stamp":"2024-09-21T03:37:18.876933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:45:02.541015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.18","version":"84'8","reported_seq":42,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:39:21.852435+0000","last_change":"2024-09-21T03:37:12.852577+0000","last_active":"2024-09-21T03:39:21.852435+0000","last_peered":"2024-09-21T03:39:21.852435+0000","last_clean":"2024-09-21T03:39:21.852435+0000","last_became_active":"2024-09-21T03:37:12.852186+0000","last_became_peered":"2024-09-21T03:37:12.852186+0000","last_unstale":"2024-09-21T03:39:21.852435+0000","last_undegraded":"2024-09-21T03:39:21.852435+0000","last_fullsized":"2024-09-21T03:39:21.852435+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:51:21.631029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"80'2","reported_seq":41,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:38:53.668330+0000","last_change":"2024-09-21T03:37:10.833587+0000","last_active":"2024-09-21T03:38:53.668330+0000","last_peered":"2024-09-21T03:38:53.668330+0000","last_clean":"2024-09-21T03:38:53.668330+0000","last_became_active":"2024-09-21T03:37:10.833194+0000","last_became_peered":"2024-09-21T03:37:10.833194+0000","last_unstale":"2024-09-21T03:38:53.668330+0000","last_undegraded":"2024-09-21T03:38:53.668330+0000","last_fullsized":"2024-09-21T03:38:53.668330+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_clean_scrub_stamp":"2024-09-21T03:37:09.809805+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:20:20.272108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"80'3","reported_seq":42,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:38:53.669769+0000","last_change":"2024-09-21T03:37:10.839103+0000","last_active":"2024-09-21T03:38:53.669769+0000","last_peered":"2024-09-21T03:38:53.669769+0000","last_clean":"2024-09-21T03:38:53.669769+0000","last_became_active":"2024-09-21T03:37:10.837192+0000","last_became_peered":"2024-09-21T03:37:10.837192+0000","last_unstale":"2024-09-21T03:38:53.669769+0000","last_undegraded":"2024-09-21T03:38:53.669769+0000","last_fullsized":"2024-09-21T03:38:53.669769+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_clean_scrub_stamp":"2024-09-21T03:37:09.809805+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:59:35.128171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"84'8","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:39:21.852475+0000","last_change":"2024-09-21T03:38:54.097890+0000","last_active":"2024-09-21T03:39:21.852475+0000","last_peered":"2024-09-21T03:39:21.852475+0000","last_clean":"2024-09-21T03:39:21.852475+0000","last_became_active":"2024-09-21T03:38:54.097484+0000","last_became_peered":"2024-09-21T03:38:54.097484+0000","last_unstale":"2024-09-21T03:39:21.852475+0000","last_undegraded":"2024-09-21T03:39:21.852475+0000","last_fullsized":"2024-09-21T03:39:21.852475+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:06:16.098400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,3],"acting":[9,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:38:53.669938+0000","last_change":"2024-09-21T03:37:19.909415+0000","last_active":"2024-09-21T03:38:53.669938+0000","last_peered":"2024-09-21T03:38:53.669938+0000","last_clean":"2024-09-21T03:38:53.669938+0000","last_became_active":"2024-09-21T03:37:19.908955+0000","last_became_peered":"2024-09-21T03:37:19.908955+0000","last_unstale":"2024-09-21T03:38:53.669938+0000","last_undegraded":"2024-09-21T03:38:53.669938+0000","last_fullsized":"2024-09-21T03:38:53.669938+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_clean_scrub_stamp":"2024-09-21T03:37:18.876933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:31:18.878725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"84'7","reported_seq":142,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:39:21.179862+0000","last_change":"2024-09-21T03:37:10.836389+0000","last_active":"2024-09-21T03:39:21.179862+0000","last_peered":"2024-09-21T03:39:21.179862+0000","last_clean":"2024-09-21T03:39:21.179862+0000","last_became_active":"2024-09-21T03:37:10.835986+0000","last_became_peered":"2024-09-21T03:37:10.835986+0000","last_unstale":"2024-09-21T03:39:21.179862+0000","last_undegraded":"2024-09-21T03:39:21.179862+0000","last_fullsized":"2024-09-21T03:39:21.179862+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_clean_scrub_stamp":"2024-09-21T03:37:09.809805+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:05:17.963609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1154,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":96,"num_read_kb":3,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"84'7","reported_seq":41,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:39:21.834248+0000","last_change":"2024-09-21T03:37:12.856008+0000","last_active":"2024-09-21T03:39:21.834248+0000","last_peered":"2024-09-21T03:39:21.834248+0000","last_clean":"2024-09-21T03:39:21.834248+0000","last_became_active":"2024-09-21T03:37:12.855603+0000","last_became_peered":"2024-09-21T03:37:12.855603+0000","last_unstale":"2024-09-21T03:39:21.834248+0000","last_undegraded":"2024-09-21T03:39:21.834248+0000","last_fullsized":"2024-09-21T03:39:21.834248+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:11:41.920114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:38:53.669941+0000","last_change":"2024-09-21T03:37:19.911507+0000","last_active":"2024-09-21T03:38:53.669941+0000","last_peered":"2024-09-21T03:38:53.669941+0000","last_clean":"2024-09-21T03:38:53.669941+0000","last_became_active":"2024-09-21T03:37:19.911053+0000","last_became_peered":"2024-09-21T03:37:19.911053+0000","last_unstale":"2024-09-21T03:38:53.669941+0000","last_undegraded":"2024-09-21T03:38:53.669941+0000","last_fullsized":"2024-09-21T03:38:53.669941+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_clean_scrub_stamp":"2024-09-21T03:37:18.876933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:42:11.964829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"84'9","reported_seq":44,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:39:21.853090+0000","last_change":"2024-09-21T03:37:12.862576+0000","last_active":"2024-09-21T03:39:21.853090+0000","last_peered":"2024-09-21T03:39:21.853090+0000","last_clean":"2024-09-21T03:39:21.853090+0000","last_became_active":"2024-09-21T03:37:12.862146+0000","last_became_peered":"2024-09-21T03:37:12.862146+0000","last_unstale":"2024-09-21T03:39:21.853090+0000","last_undegraded":"2024-09-21T03:39:21.853090+0000","last_fullsized":"2024-09-21T03:39:21.853090+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:30:33.272415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"84'12","reported_seq":47,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:39:21.853470+0000","last_change":"2024-09-21T03:37:12.859395+0000","last_active":"2024-09-21T03:39:21.853470+0000","last_peered":"2024-09-21T03:39:21.853470+0000","last_clean":"2024-09-21T03:39:21.853470+0000","last_became_active":"2024-09-21T03:37:12.858914+0000","last_became_peered":"2024-09-21T03:37:12.858914+0000","last_unstale":"2024-09-21T03:39:21.853470+0000","last_undegraded":"2024-09-21T03:39:21.853470+0000","last_fullsized":"2024-09-21T03:39:21.853470+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:13:06.319896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"84'8","reported_seq":43,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:39:21.844893+0000","last_change":"2024-09-21T03:37:12.857467+0000","last_active":"2024-09-21T03:39:21.844893+0000","last_peered":"2024-09-21T03:39:21.844893+0000","last_clean":"2024-09-21T03:39:21.844893+0000","last_became_active":"2024-09-21T03:37:12.856214+0000","last_became_peered":"2024-09-21T03:37:12.856214+0000","last_unstale":"2024-09-21T03:39:21.844893+0000","last_undegraded":"2024-09-21T03:39:21.844893+0000","last_fullsized":"2024-09-21T03:39:21.844893+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:21:58.780018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"84'7","reported_seq":42,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:39:21.842894+0000","last_change":"2024-09-21T03:37:12.858052+0000","last_active":"2024-09-21T03:39:21.842894+0000","last_peered":"2024-09-21T03:39:21.842894+0000","last_clean":"2024-09-21T03:39:21.842894+0000","last_became_active":"2024-09-21T03:37:12.857445+0000","last_became_peered":"2024-09-21T03:37:12.857445+0000","last_unstale":"2024-09-21T03:39:21.842894+0000","last_undegraded":"2024-09-21T03:39:21.842894+0000","last_fullsized":"2024-09-21T03:39:21.842894+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:22:28.058846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"84'4","reported_seq":39,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:39:21.852798+0000","last_change":"2024-09-21T03:37:12.856499+0000","last_active":"2024-09-21T03:39:21.852798+0000","last_peered":"2024-09-21T03:39:21.852798+0000","last_clean":"2024-09-21T03:39:21.852798+0000","last_became_active":"2024-09-21T03:37:12.856146+0000","last_became_peered":"2024-09-21T03:37:12.856146+0000","last_unstale":"2024-09-21T03:39:21.852798+0000","last_undegraded":"2024-09-21T03:39:21.852798+0000","last_fullsized":"2024-09-21T03:39:21.852798+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:03:26.142740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"84'6","reported_seq":41,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:39:21.851873+0000","last_change":"2024-09-21T03:37:12.860830+0000","last_active":"2024-09-21T03:39:21.851873+0000","last_peered":"2024-09-21T03:39:21.851873+0000","last_clean":"2024-09-21T03:39:21.851873+0000","last_became_active":"2024-09-21T03:37:12.860468+0000","last_became_peered":"2024-09-21T03:37:12.860468+0000","last_unstale":"2024-09-21T03:39:21.851873+0000","last_undegraded":"2024-09-21T03:39:21.851873+0000","last_fullsized":"2024-09-21T03:39:21.851873+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:48:15.057622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"84'15","reported_seq":50,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:39:21.849602+0000","last_change":"2024-09-21T03:37:12.864399+0000","last_active":"2024-09-21T03:39:21.849602+0000","last_peered":"2024-09-21T03:39:21.849602+0000","last_clean":"2024-09-21T03:39:21.849602+0000","last_became_active":"2024-09-21T03:37:12.863892+0000","last_became_peered":"2024-09-21T03:37:12.863892+0000","last_unstale":"2024-09-21T03:39:21.849602+0000","last_undegraded":"2024-09-21T03:39:21.849602+0000","last_fullsized":"2024-09-21T03:39:21.849602+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:58:26.647896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"84'2","reported_seq":37,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:39:21.847416+0000","last_change":"2024-09-21T03:37:12.861716+0000","last_active":"2024-09-21T03:39:21.847416+0000","last_peered":"2024-09-21T03:39:21.847416+0000","last_clean":"2024-09-21T03:39:21.847416+0000","last_became_active":"2024-09-21T03:37:12.861392+0000","last_became_peered":"2024-09-21T03:37:12.861392+0000","last_unstale":"2024-09-21T03:39:21.847416+0000","last_undegraded":"2024-09-21T03:39:21.847416+0000","last_fullsized":"2024-09-21T03:39:21.847416+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:28:55.751107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"84'6","reported_seq":41,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:39:21.842051+0000","last_change":"2024-09-21T03:37:12.856167+0000","last_active":"2024-09-21T03:39:21.842051+0000","last_peered":"2024-09-21T03:39:21.842051+0000","last_clean":"2024-09-21T03:39:21.842051+0000","last_became_active":"2024-09-21T03:37:12.855752+0000","last_became_peered":"2024-09-21T03:37:12.855752+0000","last_unstale":"2024-09-21T03:39:21.842051+0000","last_undegraded":"2024-09-21T03:39:21.842051+0000","last_fullsized":"2024-09-21T03:39:21.842051+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:26:30.271342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"84'8","reported_seq":42,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:39:21.852755+0000","last_change":"2024-09-21T03:37:12.862782+0000","last_active":"2024-09-21T03:39:21.852755+0000","last_peered":"2024-09-21T03:39:21.852755+0000","last_clean":"2024-09-21T03:39:21.852755+0000","last_became_active":"2024-09-21T03:37:12.862462+0000","last_became_peered":"2024-09-21T03:37:12.862462+0000","last_unstale":"2024-09-21T03:39:21.852755+0000","last_undegraded":"2024-09-21T03:39:21.852755+0000","last_fullsized":"2024-09-21T03:39:21.852755+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:57:37.744531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"84'10","reported_seq":44,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:39:21.863800+0000","last_change":"2024-09-21T03:37:12.862312+0000","last_active":"2024-09-21T03:39:21.863800+0000","last_peered":"2024-09-21T03:39:21.863800+0000","last_clean":"2024-09-21T03:39:21.863800+0000","last_became_active":"2024-09-21T03:37:12.861940+0000","last_became_peered":"2024-09-21T03:37:12.861940+0000","last_unstale":"2024-09-21T03:39:21.863800+0000","last_undegraded":"2024-09-21T03:39:21.863800+0000","last_fullsized":"2024-09-21T03:39:21.863800+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:08:49.165522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"84'11","reported_seq":45,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:39:21.847875+0000","last_change":"2024-09-21T03:37:12.857012+0000","last_active":"2024-09-21T03:39:21.847875+0000","last_peered":"2024-09-21T03:39:21.847875+0000","last_clean":"2024-09-21T03:39:21.847875+0000","last_became_active":"2024-09-21T03:37:12.856390+0000","last_became_peered":"2024-09-21T03:37:12.856390+0000","last_unstale":"2024-09-21T03:39:21.847875+0000","last_undegraded":"2024-09-21T03:39:21.847875+0000","last_fullsized":"2024-09-21T03:39:21.847875+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:51:00.839709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"84'8","reported_seq":42,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:39:21.851665+0000","last_change":"2024-09-21T03:37:12.857207+0000","last_active":"2024-09-21T03:39:21.851665+0000","last_peered":"2024-09-21T03:39:21.851665+0000","last_clean":"2024-09-21T03:39:21.851665+0000","last_became_active":"2024-09-21T03:37:12.856770+0000","last_became_peered":"2024-09-21T03:37:12.856770+0000","last_unstale":"2024-09-21T03:39:21.851665+0000","last_undegraded":"2024-09-21T03:39:21.851665+0000","last_fullsized":"2024-09-21T03:39:21.851665+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:37:18.272321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2d","version":"84'7","reported_seq":41,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:39:21.844643+0000","last_change":"2024-09-21T03:37:12.858540+0000","last_active":"2024-09-21T03:39:21.844643+0000","last_peered":"2024-09-21T03:39:21.844643+0000","last_clean":"2024-09-21T03:39:21.844643+0000","last_became_active":"2024-09-21T03:37:12.857340+0000","last_became_peered":"2024-09-21T03:37:12.857340+0000","last_unstale":"2024-09-21T03:39:21.844643+0000","last_undegraded":"2024-09-21T03:39:21.844643+0000","last_fullsized":"2024-09-21T03:39:21.844643+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:45:48.982132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"84'10","reported_seq":44,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:39:21.851214+0000","last_change":"2024-09-21T03:37:12.864408+0000","last_active":"2024-09-21T03:39:21.851214+0000","last_peered":"2024-09-21T03:39:21.851214+0000","last_clean":"2024-09-21T03:39:21.851214+0000","last_became_active":"2024-09-21T03:37:12.864082+0000","last_became_peered":"2024-09-21T03:37:12.864082+0000","last_unstale":"2024-09-21T03:39:21.851214+0000","last_undegraded":"2024-09-21T03:39:21.851214+0000","last_fullsized":"2024-09-21T03:39:21.851214+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:19:32.723589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2f","version":"84'8","reported_seq":42,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:39:21.864236+0000","last_change":"2024-09-21T03:37:12.869445+0000","last_active":"2024-09-21T03:39:21.864236+0000","last_peered":"2024-09-21T03:39:21.864236+0000","last_clean":"2024-09-21T03:39:21.864236+0000","last_became_active":"2024-09-21T03:37:12.869058+0000","last_became_peered":"2024-09-21T03:37:12.869058+0000","last_unstale":"2024-09-21T03:39:21.864236+0000","last_undegraded":"2024-09-21T03:39:21.864236+0000","last_fullsized":"2024-09-21T03:39:21.864236+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:37:57.750665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.30","version":"84'9","reported_seq":43,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:39:21.865647+0000","last_change":"2024-09-21T03:37:12.858431+0000","last_active":"2024-09-21T03:39:21.865647+0000","last_peered":"2024-09-21T03:39:21.865647+0000","last_clean":"2024-09-21T03:39:21.865647+0000","last_became_active":"2024-09-21T03:37:12.858047+0000","last_became_peered":"2024-09-21T03:37:12.858047+0000","last_unstale":"2024-09-21T03:39:21.865647+0000","last_undegraded":"2024-09-21T03:39:21.865647+0000","last_fullsized":"2024-09-21T03:39:21.865647+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:14:05.363743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.31","version":"84'7","reported_seq":41,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:39:21.845694+0000","last_change":"2024-09-21T03:37:12.866784+0000","last_active":"2024-09-21T03:39:21.845694+0000","last_peered":"2024-09-21T03:39:21.845694+0000","last_clean":"2024-09-21T03:39:21.845694+0000","last_became_active":"2024-09-21T03:37:12.866435+0000","last_became_peered":"2024-09-21T03:37:12.866435+0000","last_unstale":"2024-09-21T03:39:21.845694+0000","last_undegraded":"2024-09-21T03:39:21.845694+0000","last_fullsized":"2024-09-21T03:39:21.845694+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:09:06.156030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.32","version":"84'7","reported_seq":41,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:39:21.850666+0000","last_change":"2024-09-21T03:37:12.857054+0000","last_active":"2024-09-21T03:39:21.850666+0000","last_peered":"2024-09-21T03:39:21.850666+0000","last_clean":"2024-09-21T03:39:21.850666+0000","last_became_active":"2024-09-21T03:37:12.856671+0000","last_became_peered":"2024-09-21T03:37:12.856671+0000","last_unstale":"2024-09-21T03:39:21.850666+0000","last_undegraded":"2024-09-21T03:39:21.850666+0000","last_fullsized":"2024-09-21T03:39:21.850666+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:38:27.474550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"84'10","reported_seq":44,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:39:21.850693+0000","last_change":"2024-09-21T03:37:12.861501+0000","last_active":"2024-09-21T03:39:21.850693+0000","last_peered":"2024-09-21T03:39:21.850693+0000","last_clean":"2024-09-21T03:39:21.850693+0000","last_became_active":"2024-09-21T03:37:12.861146+0000","last_became_peered":"2024-09-21T03:37:12.861146+0000","last_unstale":"2024-09-21T03:39:21.850693+0000","last_undegraded":"2024-09-21T03:39:21.850693+0000","last_fullsized":"2024-09-21T03:39:21.850693+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:30:55.073854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"84'7","reported_seq":23,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:39:21.850561+0000","last_change":"2024-09-21T03:37:53.051912+0000","last_active":"2024-09-21T03:39:21.850561+0000","last_peered":"2024-09-21T03:39:21.850561+0000","last_clean":"2024-09-21T03:39:21.850561+0000","last_became_active":"2024-09-21T03:37:53.051640+0000","last_became_peered":"2024-09-21T03:37:53.051640+0000","last_unstale":"2024-09-21T03:39:21.850561+0000","last_undegraded":"2024-09-21T03:39:21.850561+0000","last_fullsized":"2024-09-21T03:39:21.850561+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:08:09.046867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,10,8],"acting":[9,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.35","version":"84'6","reported_seq":40,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:39:21.864066+0000","last_change":"2024-09-21T03:37:12.865862+0000","last_active":"2024-09-21T03:39:21.864066+0000","last_peered":"2024-09-21T03:39:21.864066+0000","last_clean":"2024-09-21T03:39:21.864066+0000","last_became_active":"2024-09-21T03:37:12.865468+0000","last_became_peered":"2024-09-21T03:37:12.865468+0000","last_unstale":"2024-09-21T03:39:21.864066+0000","last_undegraded":"2024-09-21T03:39:21.864066+0000","last_fullsized":"2024-09-21T03:39:21.864066+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:18:14.496495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"84'16","reported_seq":50,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:39:21.856994+0000","last_change":"2024-09-21T03:37:12.862688+0000","last_active":"2024-09-21T03:39:21.856994+0000","last_peered":"2024-09-21T03:39:21.856994+0000","last_clean":"2024-09-21T03:39:21.856994+0000","last_became_active":"2024-09-21T03:37:12.862249+0000","last_became_peered":"2024-09-21T03:37:12.862249+0000","last_unstale":"2024-09-21T03:39:21.856994+0000","last_undegraded":"2024-09-21T03:39:21.856994+0000","last_fullsized":"2024-09-21T03:39:21.856994+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:55:54.731640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.37","version":"84'11","reported_seq":45,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:39:21.864822+0000","last_change":"2024-09-21T03:37:12.863595+0000","last_active":"2024-09-21T03:39:21.864822+0000","last_peered":"2024-09-21T03:39:21.864822+0000","last_clean":"2024-09-21T03:39:21.864822+0000","last_became_active":"2024-09-21T03:37:12.863260+0000","last_became_peered":"2024-09-21T03:37:12.863260+0000","last_unstale":"2024-09-21T03:39:21.864822+0000","last_undegraded":"2024-09-21T03:39:21.864822+0000","last_fullsized":"2024-09-21T03:39:21.864822+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:10:07.995690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"84'9","reported_seq":43,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:39:21.847848+0000","last_change":"2024-09-21T03:37:12.865770+0000","last_active":"2024-09-21T03:39:21.847848+0000","last_peered":"2024-09-21T03:39:21.847848+0000","last_clean":"2024-09-21T03:39:21.847848+0000","last_became_active":"2024-09-21T03:37:12.865405+0000","last_became_peered":"2024-09-21T03:37:12.865405+0000","last_unstale":"2024-09-21T03:39:21.847848+0000","last_undegraded":"2024-09-21T03:39:21.847848+0000","last_fullsized":"2024-09-21T03:39:21.847848+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:46:25.255913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"84'5","reported_seq":39,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:39:21.851267+0000","last_change":"2024-09-21T03:37:12.856214+0000","last_active":"2024-09-21T03:39:21.851267+0000","last_peered":"2024-09-21T03:39:21.851267+0000","last_clean":"2024-09-21T03:39:21.851267+0000","last_became_active":"2024-09-21T03:37:12.853924+0000","last_became_peered":"2024-09-21T03:37:12.853924+0000","last_unstale":"2024-09-21T03:39:21.851267+0000","last_undegraded":"2024-09-21T03:39:21.851267+0000","last_fullsized":"2024-09-21T03:39:21.851267+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:10:32.182892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"84'6","reported_seq":40,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:39:21.856643+0000","last_change":"2024-09-21T03:37:12.858851+0000","last_active":"2024-09-21T03:39:21.856643+0000","last_peered":"2024-09-21T03:39:21.856643+0000","last_clean":"2024-09-21T03:39:21.856643+0000","last_became_active":"2024-09-21T03:37:12.858037+0000","last_became_peered":"2024-09-21T03:37:12.858037+0000","last_unstale":"2024-09-21T03:39:21.856643+0000","last_undegraded":"2024-09-21T03:39:21.856643+0000","last_fullsized":"2024-09-21T03:39:21.856643+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:32:23.134789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.3b","version":"84'4","reported_seq":38,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:39:21.837281+0000","last_change":"2024-09-21T03:37:12.852339+0000","last_active":"2024-09-21T03:39:21.837281+0000","last_peered":"2024-09-21T03:39:21.837281+0000","last_clean":"2024-09-21T03:39:21.837281+0000","last_became_active":"2024-09-21T03:37:12.851911+0000","last_became_peered":"2024-09-21T03:37:12.851911+0000","last_unstale":"2024-09-21T03:39:21.837281+0000","last_undegraded":"2024-09-21T03:39:21.837281+0000","last_fullsized":"2024-09-21T03:39:21.837281+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:03:30.739275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3c","version":"84'8","reported_seq":43,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:39:21.851835+0000","last_change":"2024-09-21T03:37:12.861133+0000","last_active":"2024-09-21T03:39:21.851835+0000","last_peered":"2024-09-21T03:39:21.851835+0000","last_clean":"2024-09-21T03:39:21.851835+0000","last_became_active":"2024-09-21T03:37:12.860719+0000","last_became_peered":"2024-09-21T03:37:12.860719+0000","last_unstale":"2024-09-21T03:39:21.851835+0000","last_undegraded":"2024-09-21T03:39:21.851835+0000","last_fullsized":"2024-09-21T03:39:21.851835+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:21:25.241935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3d","version":"84'7","reported_seq":42,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:39:21.858925+0000","last_change":"2024-09-21T03:37:12.865694+0000","last_active":"2024-09-21T03:39:21.858925+0000","last_peered":"2024-09-21T03:39:21.858925+0000","last_clean":"2024-09-21T03:39:21.858925+0000","last_became_active":"2024-09-21T03:37:12.864983+0000","last_became_peered":"2024-09-21T03:37:12.864983+0000","last_unstale":"2024-09-21T03:39:21.858925+0000","last_undegraded":"2024-09-21T03:39:21.858925+0000","last_fullsized":"2024-09-21T03:39:21.858925+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:52:28.583056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"84'4","reported_seq":39,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:39:21.850375+0000","last_change":"2024-09-21T03:37:12.860665+0000","last_active":"2024-09-21T03:39:21.850375+0000","last_peered":"2024-09-21T03:39:21.850375+0000","last_clean":"2024-09-21T03:39:21.850375+0000","last_became_active":"2024-09-21T03:37:12.860316+0000","last_became_peered":"2024-09-21T03:37:12.860316+0000","last_unstale":"2024-09-21T03:39:21.850375+0000","last_undegraded":"2024-09-21T03:39:21.850375+0000","last_fullsized":"2024-09-21T03:39:21.850375+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:13:07.316422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3f","version":"84'8","reported_seq":43,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:39:21.864109+0000","last_change":"2024-09-21T03:37:12.869577+0000","last_active":"2024-09-21T03:39:21.864109+0000","last_peered":"2024-09-21T03:39:21.864109+0000","last_clean":"2024-09-21T03:39:21.864109+0000","last_became_active":"2024-09-21T03:37:12.869164+0000","last_became_peered":"2024-09-21T03:37:12.869164+0000","last_unstale":"2024-09-21T03:39:21.864109+0000","last_undegraded":"2024-09-21T03:39:21.864109+0000","last_fullsized":"2024-09-21T03:39:21.864109+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:40:34.167005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":72022,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":599,"num_read_kb":377,"num_write":173,"num_write_kb":126,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":694,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":216559,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":149,"ondisk_log_size":149,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":285501,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":68,"seq":292057776168,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32092,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713060,"statfs":{"total":95995035648,"available":95962173440,"internally_reserved":0,"allocated":929792,"data_stored":480064,"data_compressed":29416,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":10,"up_from":63,"seq":270582939693,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36392,"kb_used_data":1048,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708760,"statfs":{"total":95995035648,"available":95957770240,"internally_reserved":0,"allocated":1073152,"data_stored":1135593,"data_compressed":65710,"data_compressed_allocated":86016,"data_compressed_original":671744,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":9,"up_from":58,"seq":249108103218,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27944,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717208,"statfs":{"total":95995035648,"available":95966420992,"internally_reserved":0,"allocated":942080,"data_stored":484284,"data_compressed":29416,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":52,"seq":223338299446,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27936,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717216,"statfs":{"total":95995035648,"available":95966429184,"internally_reserved":0,"allocated":929792,"data_stored":479996,"data_compressed":29416,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":7,"up_from":46,"seq":197568495676,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36308,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708844,"statfs":{"total":95995035648,"available":95957856256,"internally_reserved":0,"allocated":987136,"data_stored":1077144,"data_compressed":65710,"data_compressed_allocated":86016,"data_compressed_original":671744,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":41,"seq":176093659201,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32080,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713072,"statfs":{"total":95995035648,"available":95962185728,"internally_reserved":0,"allocated":917504,"data_stored":475364,"data_compressed":29416,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":35,"seq":150323855429,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32164,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712988,"statfs":{"total":95995035648,"available":95962099712,"internally_reserved":0,"allocated":995328,"data_stored":1071605,"data_compressed":65710,"data_compressed_allocated":86016,"data_compressed_original":671744,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":30,"seq":128849018954,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27936,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717216,"statfs":{"total":95995035648,"available":95966429184,"internally_reserved":0,"allocated":933888,"data_stored":480596,"data_compressed":29416,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":24,"seq":103079215182,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27932,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717220,"statfs":{"total":95995035648,"available":95966433280,"internally_reserved":0,"allocated":929792,"data_stored":475558,"data_compressed":29416,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":17,"seq":73014444117,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28012,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717140,"statfs":{"total":95995035648,"available":95966351360,"internally_reserved":0,"allocated":1011712,"data_stored":543961,"data_compressed":29416,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574937,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36312,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708840,"statfs":{"total":95995035648,"available":95957852160,"internally_reserved":0,"allocated":991232,"data_stored":542661,"data_compressed":29416,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738463,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36264,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708888,"statfs":{"total":95995035648,"available":95957901312,"internally_reserved":0,"allocated":942080,"data_stored":476752,"data_compressed":29416,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34677,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34677,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34677,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36294,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36294,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36294,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36294,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36294,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":67387,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":68687,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1226,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2008,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":3310,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":626,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":69951,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T03:39:28.253 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 7c374098-77c9-11ef-baf5-efdc52797490 -- ceph pg dump --format=json 2024-09-21T03:39:28.461 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:39:28 smithi114 ceph-mon[34140]: from='client.15447 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T03:39:28.500 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:39:28 smithi110 ceph-mon[34000]: from='client.15447 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T03:39:28.539 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/7c374098-77c9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:39:28.561 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:39:28 smithi089 ceph-mon[30928]: from='client.15447 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T03:39:28.996 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:28.994+0000 7f32a2465640 1 -- 172.21.15.89:0/3520419028 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f329c108d10 msgr2=0x7f329c10f540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:28.996 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:28.994+0000 7f32a2465640 1 --2- 172.21.15.89:0/3520419028 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f329c108d10 0x7f329c10f540 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f3294009ca0 tx=0x7f329402a8c0 comp rx=0 tx=0).stop 2024-09-21T03:39:28.996 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:28.994+0000 7f32a2465640 1 -- 172.21.15.89:0/3520419028 shutdown_connections 2024-09-21T03:39:28.997 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:28.994+0000 7f32a2465640 1 --2- 172.21.15.89:0/3520419028 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f329c1083c0 0x7f329c1087d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:28.997 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:28.994+0000 7f32a2465640 1 --2- 172.21.15.89:0/3520419028 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f329c108d10 0x7f329c10f540 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:28.997 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:28.994+0000 7f32a2465640 1 --2- 172.21.15.89:0/3520419028 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f329c102a80 0x7f329c102e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:28.997 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:28.994+0000 7f32a2465640 1 -- 172.21.15.89:0/3520419028 >> 172.21.15.89:0/3520419028 conn(0x7f329c079740 msgr2=0x7f329c079b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:39:28.997 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:28.994+0000 7f32a2465640 1 -- 172.21.15.89:0/3520419028 shutdown_connections 2024-09-21T03:39:28.997 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:28.994+0000 7f32a2465640 1 -- 172.21.15.89:0/3520419028 wait complete. 2024-09-21T03:39:28.999 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:28.995+0000 7f32a2465640 1 Processor -- start 2024-09-21T03:39:28.999 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:28.995+0000 7f32a2465640 1 -- start start 2024-09-21T03:39:28.999 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:28.995+0000 7f32a2465640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f329c102a80 0x7f329c19a310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:28.999 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:28.995+0000 7f32a2465640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f329c1083c0 0x7f329c19a8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:28.999 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:28.995+0000 7f32a2465640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f329c108d10 0x7f329c19fb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:28.999 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:28.995+0000 7f32a2465640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f329c113980 con 0x7f329c108d10 2024-09-21T03:39:28.999 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:28.995+0000 7f32a2465640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f329c113800 con 0x7f329c102a80 2024-09-21T03:39:28.999 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:28.995+0000 7f32a2465640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f329c113b00 con 0x7f329c1083c0 2024-09-21T03:39:28.999 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:28.996+0000 7f32a0a02640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f329c108d10 0x7f329c19fb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:28.999 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:28.996+0000 7f32a0a02640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f329c108d10 0x7f329c19fb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:35722/0 (socket says 172.21.15.89:35722) 2024-09-21T03:39:28.999 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:28.996+0000 7f32a0a02640 1 -- 172.21.15.89:0/1292371036 learned_addr learned my addr 172.21.15.89:0/1292371036 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-21T03:39:28.999 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:28.996+0000 7f329bfff640 1 --2- 172.21.15.89:0/1292371036 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f329c102a80 0x7f329c19a310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:29.000 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:28.996+0000 7f329b7fe640 1 --2- 172.21.15.89:0/1292371036 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f329c1083c0 0x7f329c19a8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:29.000 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:28.996+0000 7f32a0a02640 1 -- 172.21.15.89:0/1292371036 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f329c1083c0 msgr2=0x7f329c19a8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:29.000 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:28.996+0000 7f32a0a02640 1 --2- 172.21.15.89:0/1292371036 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f329c1083c0 0x7f329c19a8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:29.000 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:28.996+0000 7f32a0a02640 1 -- 172.21.15.89:0/1292371036 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f329c102a80 msgr2=0x7f329c19a310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:29.000 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:28.996+0000 7f32a0a02640 1 --2- 172.21.15.89:0/1292371036 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f329c102a80 0x7f329c19a310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:29.000 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:28.996+0000 7f32a0a02640 1 -- 172.21.15.89:0/1292371036 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f329c1a0280 con 0x7f329c108d10 2024-09-21T03:39:29.000 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:28.996+0000 7f32a0a02640 1 --2- 172.21.15.89:0/1292371036 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f329c108d10 0x7f329c19fb50 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f3294031040 tx=0x7f3294003f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:39:29.000 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:28.996+0000 7f32997fa640 1 -- 172.21.15.89:0/1292371036 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3294032c90 con 0x7f329c108d10 2024-09-21T03:39:29.000 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:28.996+0000 7f32997fa640 1 -- 172.21.15.89:0/1292371036 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3294032e30 con 0x7f329c108d10 2024-09-21T03:39:29.000 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:28.996+0000 7f32a2465640 1 -- 172.21.15.89:0/1292371036 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f329c1a0510 con 0x7f329c108d10 2024-09-21T03:39:29.000 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:28.996+0000 7f32997fa640 1 -- 172.21.15.89:0/1292371036 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3294030af0 con 0x7f329c108d10 2024-09-21T03:39:29.000 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:28.997+0000 7f32a2465640 1 -- 172.21.15.89:0/1292371036 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f329c1a6cd0 con 0x7f329c108d10 2024-09-21T03:39:29.001 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:28.997+0000 7f32a2465640 1 -- 172.21.15.89:0/1292371036 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f329c103d00 con 0x7f329c108d10 2024-09-21T03:39:29.002 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:29.000+0000 7f32997fa640 1 -- 172.21.15.89:0/1292371036 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 20) ==== 154563+0+0 (secure 0 0 0) 0x7f3294030c90 con 0x7f329c108d10 2024-09-21T03:39:29.003 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:29.000+0000 7f32997fa640 1 --2- 172.21.15.89:0/1292371036 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f32840b6620 0x7f32840b8b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:29.003 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:29.001+0000 7f329bfff640 1 --2- 172.21.15.89:0/1292371036 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f32840b6620 0x7f32840b8b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:29.003 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:29.001+0000 7f32997fa640 1 -- 172.21.15.89:0/1292371036 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 9121+0+0 (secure 0 0 0) 0x7f3294108a60 con 0x7f329c108d10 2024-09-21T03:39:29.003 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:29.001+0000 7f329bfff640 1 --2- 172.21.15.89:0/1292371036 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f32840b6620 0x7f32840b8b30 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f328c009bd0 tx=0x7f328c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:39:29.006 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:29.004+0000 7f32997fa640 1 -- 172.21.15.89:0/1292371036 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f32940cee10 con 0x7f329c108d10 2024-09-21T03:39:29.183 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:29.181+0000 7f32a2465640 1 -- 172.21.15.89:0/1292371036 --> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f329c063800 con 0x7f32840b6620 2024-09-21T03:39:29.183 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:39:29 smithi089 ceph-mon[30928]: pgmap v362: 129 pgs: 129 active+clean; 648 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 938 B/s wr, 6 op/s 2024-09-21T03:39:29.183 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:39:29 smithi089 ceph-mon[30928]: mgrmap e20: x(active, since 9m), standbys: z, y 2024-09-21T03:39:29.188 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:29.187+0000 7f32997fa640 1 -- 172.21.15.89:0/1292371036 <== mgr.14152 v2:172.21.15.89:6800/1591532358 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340842 (secure 0 0 0) 0x7f329c063800 con 0x7f32840b6620 2024-09-21T03:39:29.192 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-09-21T03:39:29.193 INFO:teuthology.orchestra.run.smithi089.stderr:dumped all 2024-09-21T03:39:29.195 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:29.190+0000 7f32a2465640 1 -- 172.21.15.89:0/1292371036 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f32840b6620 msgr2=0x7f32840b8b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:29.195 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:29.190+0000 7f32a2465640 1 --2- 172.21.15.89:0/1292371036 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f32840b6620 0x7f32840b8b30 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f328c009bd0 tx=0x7f328c005f90 comp rx=0 tx=0).stop 2024-09-21T03:39:29.195 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:29.190+0000 7f32a2465640 1 -- 172.21.15.89:0/1292371036 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f329c108d10 msgr2=0x7f329c19fb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:29.195 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:29.190+0000 7f32a2465640 1 --2- 172.21.15.89:0/1292371036 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f329c108d10 0x7f329c19fb50 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f3294031040 tx=0x7f3294003f80 comp rx=0 tx=0).stop 2024-09-21T03:39:29.196 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:29.190+0000 7f32a2465640 1 -- 172.21.15.89:0/1292371036 shutdown_connections 2024-09-21T03:39:29.196 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:29.190+0000 7f32a2465640 1 --2- 172.21.15.89:0/1292371036 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f32840b6620 0x7f32840b8b30 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:29.196 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:29.190+0000 7f32a2465640 1 --2- 172.21.15.89:0/1292371036 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f329c108d10 0x7f329c19fb50 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:29.196 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:29.190+0000 7f32a2465640 1 --2- 172.21.15.89:0/1292371036 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f329c1083c0 0x7f329c19a8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:29.196 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:29.190+0000 7f32a2465640 1 --2- 172.21.15.89:0/1292371036 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f329c102a80 0x7f329c19a310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:29.196 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:29.190+0000 7f32a2465640 1 -- 172.21.15.89:0/1292371036 >> 172.21.15.89:0/1292371036 conn(0x7f329c079740 msgr2=0x7f329c1043d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:39:29.196 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:29.190+0000 7f32a2465640 1 -- 172.21.15.89:0/1292371036 shutdown_connections 2024-09-21T03:39:29.196 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:29.190+0000 7f32a2465640 1 -- 172.21.15.89:0/1292371036 wait complete. 2024-09-21T03:39:29.461 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:39:29 smithi114 ceph-mon[34140]: pgmap v362: 129 pgs: 129 active+clean; 648 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 938 B/s wr, 6 op/s 2024-09-21T03:39:29.461 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:39:29 smithi114 ceph-mon[34140]: mgrmap e20: x(active, since 9m), standbys: z, y 2024-09-21T03:39:29.500 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:39:29 smithi110 ceph-mon[34000]: pgmap v362: 129 pgs: 129 active+clean; 648 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 938 B/s wr, 6 op/s 2024-09-21T03:39:29.500 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:39:29 smithi110 ceph-mon[34000]: mgrmap e20: x(active, since 9m), standbys: z, y 2024-09-21T03:39:29.870 INFO:teuthology.orchestra.run.smithi089.stdout:{"pg_ready":true,"pg_map":{"version":362,"stamp":"2024-09-21T03:39:27.665392+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":663508,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":682,"num_read_kb":443,"num_write":304,"num_write_kb":2282,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":23,"num_bytes_recovered":2952534,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":747,"ondisk_log_size":747,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":381364,"kb_used_data":11316,"kb_used_omap":18,"kb_used_meta":369837,"kb_avail":1124560460,"statfs":{"total":1151940427776,"available":1151549911040,"internally_reserved":0,"allocated":11587584,"data_stored":7730329,"data_compressed":461874,"data_compressed_allocated":626688,"data_compressed_original":2752512,"omap_allocated":19143,"internal_metadata":378713401},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1767,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":69,"num_read_kb":34,"num_write":15,"num_write_kb":11,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003548"},"pg_stats":[{"pgid":"2.1f","version":"80'9","reported_seq":50,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:38:53.672984+0000","last_change":"2024-09-21T03:37:10.836953+0000","last_active":"2024-09-21T03:38:53.672984+0000","last_peered":"2024-09-21T03:38:53.672984+0000","last_clean":"2024-09-21T03:38:53.672984+0000","last_became_active":"2024-09-21T03:37:10.836566+0000","last_became_peered":"2024-09-21T03:37:10.836566+0000","last_unstale":"2024-09-21T03:38:53.672984+0000","last_undegraded":"2024-09-21T03:38:53.672984+0000","last_fullsized":"2024-09-21T03:38:53.672984+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_clean_scrub_stamp":"2024-09-21T03:37:09.809805+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:15:45.591225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"84'5","reported_seq":40,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:39:21.839616+0000","last_change":"2024-09-21T03:37:12.866931+0000","last_active":"2024-09-21T03:39:21.839616+0000","last_peered":"2024-09-21T03:39:21.839616+0000","last_clean":"2024-09-21T03:39:21.839616+0000","last_became_active":"2024-09-21T03:37:12.866452+0000","last_became_peered":"2024-09-21T03:37:12.866452+0000","last_unstale":"2024-09-21T03:39:21.839616+0000","last_undegraded":"2024-09-21T03:39:21.839616+0000","last_fullsized":"2024-09-21T03:39:21.839616+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:37:14.324548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:38:53.669589+0000","last_change":"2024-09-21T03:37:19.912154+0000","last_active":"2024-09-21T03:38:53.669589+0000","last_peered":"2024-09-21T03:38:53.669589+0000","last_clean":"2024-09-21T03:38:53.669589+0000","last_became_active":"2024-09-21T03:37:19.911686+0000","last_became_peered":"2024-09-21T03:37:19.911686+0000","last_unstale":"2024-09-21T03:38:53.669589+0000","last_undegraded":"2024-09-21T03:38:53.669589+0000","last_fullsized":"2024-09-21T03:38:53.669589+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_clean_scrub_stamp":"2024-09-21T03:37:18.876933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:29:20.189418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"80'2","reported_seq":42,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:38:53.972157+0000","last_change":"2024-09-21T03:37:11.034501+0000","last_active":"2024-09-21T03:38:53.972157+0000","last_peered":"2024-09-21T03:38:53.972157+0000","last_clean":"2024-09-21T03:38:53.972157+0000","last_became_active":"2024-09-21T03:37:11.034110+0000","last_became_peered":"2024-09-21T03:37:11.034110+0000","last_unstale":"2024-09-21T03:38:53.972157+0000","last_undegraded":"2024-09-21T03:38:53.972157+0000","last_fullsized":"2024-09-21T03:38:53.972157+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_clean_scrub_stamp":"2024-09-21T03:37:09.809805+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:03:12.854469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"84'6","reported_seq":41,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:39:21.858901+0000","last_change":"2024-09-21T03:37:12.867330+0000","last_active":"2024-09-21T03:39:21.858901+0000","last_peered":"2024-09-21T03:39:21.858901+0000","last_clean":"2024-09-21T03:39:21.858901+0000","last_became_active":"2024-09-21T03:37:12.866952+0000","last_became_peered":"2024-09-21T03:37:12.866952+0000","last_unstale":"2024-09-21T03:39:21.858901+0000","last_undegraded":"2024-09-21T03:39:21.858901+0000","last_fullsized":"2024-09-21T03:39:21.858901+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:40:33.826546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:38:54.095716+0000","last_change":"2024-09-21T03:37:19.910661+0000","last_active":"2024-09-21T03:38:54.095716+0000","last_peered":"2024-09-21T03:38:54.095716+0000","last_clean":"2024-09-21T03:38:54.095716+0000","last_became_active":"2024-09-21T03:37:19.910108+0000","last_became_peered":"2024-09-21T03:37:19.910108+0000","last_unstale":"2024-09-21T03:38:54.095716+0000","last_undegraded":"2024-09-21T03:38:54.095716+0000","last_fullsized":"2024-09-21T03:38:54.095716+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_clean_scrub_stamp":"2024-09-21T03:37:18.876933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:18:05.654426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"80'4","reported_seq":43,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:38:54.442879+0000","last_change":"2024-09-21T03:37:10.835336+0000","last_active":"2024-09-21T03:38:54.442879+0000","last_peered":"2024-09-21T03:38:54.442879+0000","last_clean":"2024-09-21T03:38:54.442879+0000","last_became_active":"2024-09-21T03:37:10.834928+0000","last_became_peered":"2024-09-21T03:37:10.834928+0000","last_unstale":"2024-09-21T03:38:54.442879+0000","last_undegraded":"2024-09-21T03:38:54.442879+0000","last_fullsized":"2024-09-21T03:38:54.442879+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_clean_scrub_stamp":"2024-09-21T03:37:09.809805+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:02:14.717520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"84'9","reported_seq":43,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:39:21.865718+0000","last_change":"2024-09-21T03:37:12.861271+0000","last_active":"2024-09-21T03:39:21.865718+0000","last_peered":"2024-09-21T03:39:21.865718+0000","last_clean":"2024-09-21T03:39:21.865718+0000","last_became_active":"2024-09-21T03:37:12.860932+0000","last_became_peered":"2024-09-21T03:37:12.860932+0000","last_unstale":"2024-09-21T03:39:21.865718+0000","last_undegraded":"2024-09-21T03:39:21.865718+0000","last_fullsized":"2024-09-21T03:39:21.865718+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:46:08.993250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":34,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:38:54.096300+0000","last_change":"2024-09-21T03:37:53.053045+0000","last_active":"2024-09-21T03:38:54.096300+0000","last_peered":"2024-09-21T03:38:54.096300+0000","last_clean":"2024-09-21T03:38:54.096300+0000","last_became_active":"2024-09-21T03:37:53.052382+0000","last_became_peered":"2024-09-21T03:37:53.052382+0000","last_unstale":"2024-09-21T03:38:54.096300+0000","last_undegraded":"2024-09-21T03:38:54.096300+0000","last_fullsized":"2024-09-21T03:38:54.096300+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_clean_scrub_stamp":"2024-09-21T03:37:18.876933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:02:44.300768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,8],"acting":[4,3,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"74'2","reported_seq":41,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:38:54.442781+0000","last_change":"2024-09-21T03:37:10.833985+0000","last_active":"2024-09-21T03:38:54.442781+0000","last_peered":"2024-09-21T03:38:54.442781+0000","last_clean":"2024-09-21T03:38:54.442781+0000","last_became_active":"2024-09-21T03:37:10.833562+0000","last_became_peered":"2024-09-21T03:37:10.833562+0000","last_unstale":"2024-09-21T03:38:54.442781+0000","last_undegraded":"2024-09-21T03:38:54.442781+0000","last_fullsized":"2024-09-21T03:38:54.442781+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_clean_scrub_stamp":"2024-09-21T03:37:09.809805+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:49:32.617768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"84'15","reported_seq":49,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:39:21.852488+0000","last_change":"2024-09-21T03:37:12.865408+0000","last_active":"2024-09-21T03:39:21.852488+0000","last_peered":"2024-09-21T03:39:21.852488+0000","last_clean":"2024-09-21T03:39:21.852488+0000","last_became_active":"2024-09-21T03:37:12.864972+0000","last_became_peered":"2024-09-21T03:37:12.864972+0000","last_unstale":"2024-09-21T03:39:21.852488+0000","last_undegraded":"2024-09-21T03:39:21.852488+0000","last_fullsized":"2024-09-21T03:39:21.852488+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:03:16.384260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:38:54.095856+0000","last_change":"2024-09-21T03:37:19.912105+0000","last_active":"2024-09-21T03:38:54.095856+0000","last_peered":"2024-09-21T03:38:54.095856+0000","last_clean":"2024-09-21T03:38:54.095856+0000","last_became_active":"2024-09-21T03:37:19.911416+0000","last_became_peered":"2024-09-21T03:37:19.911416+0000","last_unstale":"2024-09-21T03:38:54.095856+0000","last_undegraded":"2024-09-21T03:38:54.095856+0000","last_fullsized":"2024-09-21T03:38:54.095856+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_clean_scrub_stamp":"2024-09-21T03:37:18.876933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:20:16.492231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"84'5","reported_seq":40,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:39:21.865669+0000","last_change":"2024-09-21T03:37:12.862354+0000","last_active":"2024-09-21T03:39:21.865669+0000","last_peered":"2024-09-21T03:39:21.865669+0000","last_clean":"2024-09-21T03:39:21.865669+0000","last_became_active":"2024-09-21T03:37:12.861977+0000","last_became_peered":"2024-09-21T03:37:12.861977+0000","last_unstale":"2024-09-21T03:39:21.865669+0000","last_undegraded":"2024-09-21T03:39:21.865669+0000","last_fullsized":"2024-09-21T03:39:21.865669+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:53:52.725524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"80'4","reported_seq":45,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:38:54.442515+0000","last_change":"2024-09-21T03:37:10.832140+0000","last_active":"2024-09-21T03:38:54.442515+0000","last_peered":"2024-09-21T03:38:54.442515+0000","last_clean":"2024-09-21T03:38:54.442515+0000","last_became_active":"2024-09-21T03:37:10.831395+0000","last_became_peered":"2024-09-21T03:37:10.831395+0000","last_unstale":"2024-09-21T03:38:54.442515+0000","last_undegraded":"2024-09-21T03:38:54.442515+0000","last_fullsized":"2024-09-21T03:38:54.442515+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_clean_scrub_stamp":"2024-09-21T03:37:09.809805+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:55:20.440890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:38:54.096219+0000","last_change":"2024-09-21T03:37:20.160850+0000","last_active":"2024-09-21T03:38:54.096219+0000","last_peered":"2024-09-21T03:38:54.096219+0000","last_clean":"2024-09-21T03:38:54.096219+0000","last_became_active":"2024-09-21T03:37:20.160305+0000","last_became_peered":"2024-09-21T03:37:20.160305+0000","last_unstale":"2024-09-21T03:38:54.096219+0000","last_undegraded":"2024-09-21T03:38:54.096219+0000","last_fullsized":"2024-09-21T03:38:54.096219+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_clean_scrub_stamp":"2024-09-21T03:37:18.876933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:51:45.971984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"84'8","reported_seq":43,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:39:21.848436+0000","last_change":"2024-09-21T03:37:12.859485+0000","last_active":"2024-09-21T03:39:21.848436+0000","last_peered":"2024-09-21T03:39:21.848436+0000","last_clean":"2024-09-21T03:39:21.848436+0000","last_became_active":"2024-09-21T03:37:12.859097+0000","last_became_peered":"2024-09-21T03:37:12.859097+0000","last_unstale":"2024-09-21T03:39:21.848436+0000","last_undegraded":"2024-09-21T03:39:21.848436+0000","last_fullsized":"2024-09-21T03:39:21.848436+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:16:05.438360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"84'2","reported_seq":42,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:39:22.087392+0000","last_change":"2024-09-21T03:37:10.832441+0000","last_active":"2024-09-21T03:39:22.087392+0000","last_peered":"2024-09-21T03:39:22.087392+0000","last_clean":"2024-09-21T03:39:22.087392+0000","last_became_active":"2024-09-21T03:37:10.832073+0000","last_became_peered":"2024-09-21T03:37:10.832073+0000","last_unstale":"2024-09-21T03:39:22.087392+0000","last_undegraded":"2024-09-21T03:39:22.087392+0000","last_fullsized":"2024-09-21T03:39:22.087392+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_clean_scrub_stamp":"2024-09-21T03:37:09.809805+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:32:19.411097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"82'1","reported_seq":27,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:38:54.096331+0000","last_change":"2024-09-21T03:37:19.905490+0000","last_active":"2024-09-21T03:38:54.096331+0000","last_peered":"2024-09-21T03:38:54.096331+0000","last_clean":"2024-09-21T03:38:54.096331+0000","last_became_active":"2024-09-21T03:37:19.904971+0000","last_became_peered":"2024-09-21T03:37:19.904971+0000","last_unstale":"2024-09-21T03:38:54.096331+0000","last_undegraded":"2024-09-21T03:38:54.096331+0000","last_fullsized":"2024-09-21T03:38:54.096331+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_clean_scrub_stamp":"2024-09-21T03:37:18.876933+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:44:20.655538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"84'9","reported_seq":44,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:39:21.853193+0000","last_change":"2024-09-21T03:37:12.863597+0000","last_active":"2024-09-21T03:39:21.853193+0000","last_peered":"2024-09-21T03:39:21.853193+0000","last_clean":"2024-09-21T03:39:21.853193+0000","last_became_active":"2024-09-21T03:37:12.863223+0000","last_became_peered":"2024-09-21T03:37:12.863223+0000","last_unstale":"2024-09-21T03:39:21.853193+0000","last_undegraded":"2024-09-21T03:39:21.853193+0000","last_fullsized":"2024-09-21T03:39:21.853193+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:29:28.878964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.9","version":"80'3","reported_seq":43,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:38:53.971972+0000","last_change":"2024-09-21T03:37:10.834648+0000","last_active":"2024-09-21T03:38:53.971972+0000","last_peered":"2024-09-21T03:38:53.971972+0000","last_clean":"2024-09-21T03:38:53.971972+0000","last_became_active":"2024-09-21T03:37:10.834293+0000","last_became_peered":"2024-09-21T03:37:10.834293+0000","last_unstale":"2024-09-21T03:38:53.971972+0000","last_undegraded":"2024-09-21T03:38:53.971972+0000","last_fullsized":"2024-09-21T03:38:53.971972+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_clean_scrub_stamp":"2024-09-21T03:37:09.809805+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:10:41.248733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:38:53.672715+0000","last_change":"2024-09-21T03:37:19.912526+0000","last_active":"2024-09-21T03:38:53.672715+0000","last_peered":"2024-09-21T03:38:53.672715+0000","last_clean":"2024-09-21T03:38:53.672715+0000","last_became_active":"2024-09-21T03:37:19.912062+0000","last_became_peered":"2024-09-21T03:37:19.912062+0000","last_unstale":"2024-09-21T03:38:53.672715+0000","last_undegraded":"2024-09-21T03:38:53.672715+0000","last_fullsized":"2024-09-21T03:38:53.672715+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_clean_scrub_stamp":"2024-09-21T03:37:18.876933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:52:40.395834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.9","version":"84'2","reported_seq":37,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:39:21.852191+0000","last_change":"2024-09-21T03:37:12.855672+0000","last_active":"2024-09-21T03:39:21.852191+0000","last_peered":"2024-09-21T03:39:21.852191+0000","last_clean":"2024-09-21T03:39:21.852191+0000","last_became_active":"2024-09-21T03:37:12.855191+0000","last_became_peered":"2024-09-21T03:37:12.855191+0000","last_unstale":"2024-09-21T03:39:21.852191+0000","last_undegraded":"2024-09-21T03:39:21.852191+0000","last_fullsized":"2024-09-21T03:39:21.852191+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:26:56.115697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.8","version":"80'2","reported_seq":42,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:38:54.096441+0000","last_change":"2024-09-21T03:37:11.034752+0000","last_active":"2024-09-21T03:38:54.096441+0000","last_peered":"2024-09-21T03:38:54.096441+0000","last_clean":"2024-09-21T03:38:54.096441+0000","last_became_active":"2024-09-21T03:37:11.034386+0000","last_became_peered":"2024-09-21T03:37:11.034386+0000","last_unstale":"2024-09-21T03:38:54.096441+0000","last_undegraded":"2024-09-21T03:38:54.096441+0000","last_fullsized":"2024-09-21T03:38:54.096441+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_clean_scrub_stamp":"2024-09-21T03:37:09.809805+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:17:23.647411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:38:54.096423+0000","last_change":"2024-09-21T03:37:20.162051+0000","last_active":"2024-09-21T03:38:54.096423+0000","last_peered":"2024-09-21T03:38:54.096423+0000","last_clean":"2024-09-21T03:38:54.096423+0000","last_became_active":"2024-09-21T03:37:20.161452+0000","last_became_peered":"2024-09-21T03:37:20.161452+0000","last_unstale":"2024-09-21T03:38:54.096423+0000","last_undegraded":"2024-09-21T03:38:54.096423+0000","last_fullsized":"2024-09-21T03:38:54.096423+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_clean_scrub_stamp":"2024-09-21T03:37:18.876933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:29:22.544800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"84'7","reported_seq":42,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:39:21.851621+0000","last_change":"2024-09-21T03:37:12.866058+0000","last_active":"2024-09-21T03:39:21.851621+0000","last_peered":"2024-09-21T03:39:21.851621+0000","last_clean":"2024-09-21T03:39:21.851621+0000","last_became_active":"2024-09-21T03:37:12.865694+0000","last_became_peered":"2024-09-21T03:37:12.865694+0000","last_unstale":"2024-09-21T03:39:21.851621+0000","last_undegraded":"2024-09-21T03:39:21.851621+0000","last_fullsized":"2024-09-21T03:39:21.851621+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:56:33.629873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"80'4","reported_seq":44,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:38:54.096939+0000","last_change":"2024-09-21T03:37:11.358619+0000","last_active":"2024-09-21T03:38:54.096939+0000","last_peered":"2024-09-21T03:38:54.096939+0000","last_clean":"2024-09-21T03:38:54.096939+0000","last_became_active":"2024-09-21T03:37:11.358196+0000","last_became_peered":"2024-09-21T03:37:11.358196+0000","last_unstale":"2024-09-21T03:38:54.096939+0000","last_undegraded":"2024-09-21T03:38:54.096939+0000","last_fullsized":"2024-09-21T03:38:54.096939+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_clean_scrub_stamp":"2024-09-21T03:37:09.809805+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:57:13.476641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":27,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:38:53.669752+0000","last_change":"2024-09-21T03:37:19.905711+0000","last_active":"2024-09-21T03:38:53.669752+0000","last_peered":"2024-09-21T03:38:53.669752+0000","last_clean":"2024-09-21T03:38:53.669752+0000","last_became_active":"2024-09-21T03:37:19.905260+0000","last_became_peered":"2024-09-21T03:37:19.905260+0000","last_unstale":"2024-09-21T03:38:53.669752+0000","last_undegraded":"2024-09-21T03:38:53.669752+0000","last_fullsized":"2024-09-21T03:38:53.669752+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_clean_scrub_stamp":"2024-09-21T03:37:18.876933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:21:09.476082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"84'5","reported_seq":40,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:39:21.852928+0000","last_change":"2024-09-21T03:37:12.854376+0000","last_active":"2024-09-21T03:39:21.852928+0000","last_peered":"2024-09-21T03:39:21.852928+0000","last_clean":"2024-09-21T03:39:21.852928+0000","last_became_active":"2024-09-21T03:37:12.852942+0000","last_became_peered":"2024-09-21T03:37:12.852942+0000","last_unstale":"2024-09-21T03:39:21.852928+0000","last_undegraded":"2024-09-21T03:39:21.852928+0000","last_fullsized":"2024-09-21T03:39:21.852928+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:06:49.319002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"84'11","reported_seq":64,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:39:21.877193+0000","last_change":"2024-09-21T03:37:53.056889+0000","last_active":"2024-09-21T03:39:21.877193+0000","last_peered":"2024-09-21T03:39:21.877193+0000","last_clean":"2024-09-21T03:39:21.877193+0000","last_became_active":"2024-09-21T03:37:53.053054+0000","last_became_peered":"2024-09-21T03:37:53.053054+0000","last_unstale":"2024-09-21T03:39:21.877193+0000","last_undegraded":"2024-09-21T03:39:21.877193+0000","last_fullsized":"2024-09-21T03:39:21.877193+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_clean_scrub_stamp":"2024-09-21T03:37:09.809805+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:16:05.539111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,3],"acting":[9,11,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":27,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:38:53.971576+0000","last_change":"2024-09-21T03:37:20.161515+0000","last_active":"2024-09-21T03:38:53.971576+0000","last_peered":"2024-09-21T03:38:53.971576+0000","last_clean":"2024-09-21T03:38:53.971576+0000","last_became_active":"2024-09-21T03:37:20.160548+0000","last_became_peered":"2024-09-21T03:37:20.160548+0000","last_unstale":"2024-09-21T03:38:53.971576+0000","last_undegraded":"2024-09-21T03:38:53.971576+0000","last_fullsized":"2024-09-21T03:38:53.971576+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_clean_scrub_stamp":"2024-09-21T03:37:18.876933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:17:51.144859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"84'9","reported_seq":44,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:39:21.832796+0000","last_change":"2024-09-21T03:37:12.848305+0000","last_active":"2024-09-21T03:39:21.832796+0000","last_peered":"2024-09-21T03:39:21.832796+0000","last_clean":"2024-09-21T03:39:21.832796+0000","last_became_active":"2024-09-21T03:37:12.847916+0000","last_became_peered":"2024-09-21T03:37:12.847916+0000","last_unstale":"2024-09-21T03:39:21.832796+0000","last_undegraded":"2024-09-21T03:39:21.832796+0000","last_fullsized":"2024-09-21T03:39:21.832796+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:48:39.922805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"80'1","reported_seq":42,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:38:54.442955+0000","last_change":"2024-09-21T03:37:11.035964+0000","last_active":"2024-09-21T03:38:54.442955+0000","last_peered":"2024-09-21T03:38:54.442955+0000","last_clean":"2024-09-21T03:38:54.442955+0000","last_became_active":"2024-09-21T03:37:11.035427+0000","last_became_peered":"2024-09-21T03:37:11.035427+0000","last_unstale":"2024-09-21T03:38:54.442955+0000","last_undegraded":"2024-09-21T03:38:54.442955+0000","last_fullsized":"2024-09-21T03:38:54.442955+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_clean_scrub_stamp":"2024-09-21T03:37:09.809805+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:04:51.026753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":27,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:38:54.096205+0000","last_change":"2024-09-21T03:37:19.909428+0000","last_active":"2024-09-21T03:38:54.096205+0000","last_peered":"2024-09-21T03:38:54.096205+0000","last_clean":"2024-09-21T03:38:54.096205+0000","last_became_active":"2024-09-21T03:37:19.908904+0000","last_became_peered":"2024-09-21T03:37:19.908904+0000","last_unstale":"2024-09-21T03:38:54.096205+0000","last_undegraded":"2024-09-21T03:38:54.096205+0000","last_fullsized":"2024-09-21T03:38:54.096205+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_clean_scrub_stamp":"2024-09-21T03:37:18.876933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:12:07.564058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.5","version":"84'4","reported_seq":39,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:39:21.852816+0000","last_change":"2024-09-21T03:37:12.855592+0000","last_active":"2024-09-21T03:39:21.852816+0000","last_peered":"2024-09-21T03:39:21.852816+0000","last_clean":"2024-09-21T03:39:21.852816+0000","last_became_active":"2024-09-21T03:37:12.855244+0000","last_became_peered":"2024-09-21T03:37:12.855244+0000","last_unstale":"2024-09-21T03:39:21.852816+0000","last_undegraded":"2024-09-21T03:39:21.852816+0000","last_fullsized":"2024-09-21T03:39:21.852816+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:21:34.603141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":40,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:38:53.672861+0000","last_change":"2024-09-21T03:37:11.036401+0000","last_active":"2024-09-21T03:38:53.672861+0000","last_peered":"2024-09-21T03:38:53.672861+0000","last_clean":"2024-09-21T03:38:53.672861+0000","last_became_active":"2024-09-21T03:37:11.036003+0000","last_became_peered":"2024-09-21T03:37:11.036003+0000","last_unstale":"2024-09-21T03:38:53.672861+0000","last_undegraded":"2024-09-21T03:38:53.672861+0000","last_fullsized":"2024-09-21T03:38:53.672861+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_clean_scrub_stamp":"2024-09-21T03:37:09.809805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:35:50.549060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":27,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:38:54.442319+0000","last_change":"2024-09-21T03:37:20.160668+0000","last_active":"2024-09-21T03:38:54.442319+0000","last_peered":"2024-09-21T03:38:54.442319+0000","last_clean":"2024-09-21T03:38:54.442319+0000","last_became_active":"2024-09-21T03:37:20.160158+0000","last_became_peered":"2024-09-21T03:37:20.160158+0000","last_unstale":"2024-09-21T03:38:54.442319+0000","last_undegraded":"2024-09-21T03:38:54.442319+0000","last_fullsized":"2024-09-21T03:38:54.442319+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_clean_scrub_stamp":"2024-09-21T03:37:18.876933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:45:25.677667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3","version":"84'7","reported_seq":49,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:39:21.847550+0000","last_change":"2024-09-21T03:37:53.050351+0000","last_active":"2024-09-21T03:39:21.847550+0000","last_peered":"2024-09-21T03:39:21.847550+0000","last_clean":"2024-09-21T03:39:21.847550+0000","last_became_active":"2024-09-21T03:37:53.050003+0000","last_became_peered":"2024-09-21T03:37:53.050003+0000","last_unstale":"2024-09-21T03:39:21.847550+0000","last_undegraded":"2024-09-21T03:39:21.847550+0000","last_fullsized":"2024-09-21T03:39:21.847550+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:49:03.313002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,0],"acting":[10,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"80'1","reported_seq":41,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:38:53.942258+0000","last_change":"2024-09-21T03:37:10.832635+0000","last_active":"2024-09-21T03:38:53.942258+0000","last_peered":"2024-09-21T03:38:53.942258+0000","last_clean":"2024-09-21T03:38:53.942258+0000","last_became_active":"2024-09-21T03:37:10.832280+0000","last_became_peered":"2024-09-21T03:37:10.832280+0000","last_unstale":"2024-09-21T03:38:53.942258+0000","last_undegraded":"2024-09-21T03:38:53.942258+0000","last_fullsized":"2024-09-21T03:38:53.942258+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_clean_scrub_stamp":"2024-09-21T03:37:09.809805+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:54:52.744486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":27,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:38:53.668675+0000","last_change":"2024-09-21T03:37:19.904112+0000","last_active":"2024-09-21T03:38:53.668675+0000","last_peered":"2024-09-21T03:38:53.668675+0000","last_clean":"2024-09-21T03:38:53.668675+0000","last_became_active":"2024-09-21T03:37:19.903677+0000","last_became_peered":"2024-09-21T03:37:19.903677+0000","last_unstale":"2024-09-21T03:38:53.668675+0000","last_undegraded":"2024-09-21T03:38:53.668675+0000","last_fullsized":"2024-09-21T03:38:53.668675+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_clean_scrub_stamp":"2024-09-21T03:37:18.876933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:32:20.096850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"84'8","reported_seq":43,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:39:21.847619+0000","last_change":"2024-09-21T03:37:12.864082+0000","last_active":"2024-09-21T03:39:21.847619+0000","last_peered":"2024-09-21T03:39:21.847619+0000","last_clean":"2024-09-21T03:39:21.847619+0000","last_became_active":"2024-09-21T03:37:12.862745+0000","last_became_peered":"2024-09-21T03:37:12.862745+0000","last_unstale":"2024-09-21T03:39:21.847619+0000","last_undegraded":"2024-09-21T03:39:21.847619+0000","last_fullsized":"2024-09-21T03:39:21.847619+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:25:35.759719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"84'5","reported_seq":141,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:39:23.346185+0000","last_change":"2024-09-21T03:37:11.034463+0000","last_active":"2024-09-21T03:39:23.346185+0000","last_peered":"2024-09-21T03:39:23.346185+0000","last_clean":"2024-09-21T03:39:23.346185+0000","last_became_active":"2024-09-21T03:37:11.033932+0000","last_became_peered":"2024-09-21T03:37:11.033932+0000","last_unstale":"2024-09-21T03:39:23.346185+0000","last_undegraded":"2024-09-21T03:39:23.346185+0000","last_fullsized":"2024-09-21T03:39:23.346185+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_clean_scrub_stamp":"2024-09-21T03:37:09.809805+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:15:18.595442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":96,"num_read_kb":96,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:38:53.668706+0000","last_change":"2024-09-21T03:37:20.160789+0000","last_active":"2024-09-21T03:38:53.668706+0000","last_peered":"2024-09-21T03:38:53.668706+0000","last_clean":"2024-09-21T03:38:53.668706+0000","last_became_active":"2024-09-21T03:37:20.160231+0000","last_became_peered":"2024-09-21T03:37:20.160231+0000","last_unstale":"2024-09-21T03:38:53.668706+0000","last_undegraded":"2024-09-21T03:38:53.668706+0000","last_fullsized":"2024-09-21T03:38:53.668706+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_clean_scrub_stamp":"2024-09-21T03:37:18.876933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:07:16.762249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"84'5","reported_seq":40,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:39:21.852307+0000","last_change":"2024-09-21T03:37:12.853178+0000","last_active":"2024-09-21T03:39:21.852307+0000","last_peered":"2024-09-21T03:39:21.852307+0000","last_clean":"2024-09-21T03:39:21.852307+0000","last_became_active":"2024-09-21T03:37:12.852822+0000","last_became_peered":"2024-09-21T03:37:12.852822+0000","last_unstale":"2024-09-21T03:39:21.852307+0000","last_undegraded":"2024-09-21T03:39:21.852307+0000","last_fullsized":"2024-09-21T03:39:21.852307+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:48:26.146593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"80'3","reported_seq":43,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:38:53.941666+0000","last_change":"2024-09-21T03:37:10.838939+0000","last_active":"2024-09-21T03:38:53.941666+0000","last_peered":"2024-09-21T03:38:53.941666+0000","last_clean":"2024-09-21T03:38:53.941666+0000","last_became_active":"2024-09-21T03:37:10.838562+0000","last_became_peered":"2024-09-21T03:37:10.838562+0000","last_unstale":"2024-09-21T03:38:53.941666+0000","last_undegraded":"2024-09-21T03:38:53.941666+0000","last_fullsized":"2024-09-21T03:38:53.941666+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_clean_scrub_stamp":"2024-09-21T03:37:09.809805+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:35:32.936824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":34,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:38:54.097042+0000","last_change":"2024-09-21T03:37:53.052940+0000","last_active":"2024-09-21T03:38:54.097042+0000","last_peered":"2024-09-21T03:38:54.097042+0000","last_clean":"2024-09-21T03:38:54.097042+0000","last_became_active":"2024-09-21T03:37:53.052310+0000","last_became_peered":"2024-09-21T03:37:53.052310+0000","last_unstale":"2024-09-21T03:38:54.097042+0000","last_undegraded":"2024-09-21T03:38:54.097042+0000","last_fullsized":"2024-09-21T03:38:54.097042+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_clean_scrub_stamp":"2024-09-21T03:37:18.876933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:31:35.353226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,11,6,10],"acting":[5,11,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"84'8","reported_seq":43,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:39:21.847226+0000","last_change":"2024-09-21T03:37:12.869795+0000","last_active":"2024-09-21T03:39:21.847226+0000","last_peered":"2024-09-21T03:39:21.847226+0000","last_clean":"2024-09-21T03:39:21.847226+0000","last_became_active":"2024-09-21T03:37:12.869381+0000","last_became_peered":"2024-09-21T03:37:12.869381+0000","last_unstale":"2024-09-21T03:39:21.847226+0000","last_undegraded":"2024-09-21T03:39:21.847226+0000","last_fullsized":"2024-09-21T03:39:21.847226+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:39:29.437552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"80'3","reported_seq":45,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:38:54.096584+0000","last_change":"2024-09-21T03:37:10.834853+0000","last_active":"2024-09-21T03:38:54.096584+0000","last_peered":"2024-09-21T03:38:54.096584+0000","last_clean":"2024-09-21T03:38:54.096584+0000","last_became_active":"2024-09-21T03:37:10.834402+0000","last_became_peered":"2024-09-21T03:37:10.834402+0000","last_unstale":"2024-09-21T03:38:54.096584+0000","last_undegraded":"2024-09-21T03:38:54.096584+0000","last_fullsized":"2024-09-21T03:38:54.096584+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_clean_scrub_stamp":"2024-09-21T03:37:09.809805+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:57:29.668570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"20'92","reported_seq":113,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:38:53.941746+0000","last_change":"2024-09-21T03:36:00.521781+0000","last_active":"2024-09-21T03:38:53.941746+0000","last_peered":"2024-09-21T03:38:53.941746+0000","last_clean":"2024-09-21T03:38:53.941746+0000","last_became_active":"2024-09-21T03:36:00.516468+0000","last_became_peered":"2024-09-21T03:36:00.516468+0000","last_unstale":"2024-09-21T03:38:53.941746+0000","last_undegraded":"2024-09-21T03:38:53.941746+0000","last_fullsized":"2024-09-21T03:38:53.941746+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:32:38.215432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:32:38.215432+0000","last_clean_scrub_stamp":"2024-09-21T03:32:38.215432+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:02:16.252544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"82'1","reported_seq":27,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:38:53.941730+0000","last_change":"2024-09-21T03:37:19.911309+0000","last_active":"2024-09-21T03:38:53.941730+0000","last_peered":"2024-09-21T03:38:53.941730+0000","last_clean":"2024-09-21T03:38:53.941730+0000","last_became_active":"2024-09-21T03:37:19.910778+0000","last_became_peered":"2024-09-21T03:37:19.910778+0000","last_unstale":"2024-09-21T03:38:53.941730+0000","last_undegraded":"2024-09-21T03:38:53.941730+0000","last_fullsized":"2024-09-21T03:38:53.941730+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_clean_scrub_stamp":"2024-09-21T03:37:18.876933+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:52:23.307983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"84'7","reported_seq":41,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:39:21.864440+0000","last_change":"2024-09-21T03:37:12.860707+0000","last_active":"2024-09-21T03:39:21.864440+0000","last_peered":"2024-09-21T03:39:21.864440+0000","last_clean":"2024-09-21T03:39:21.864440+0000","last_became_active":"2024-09-21T03:37:12.860341+0000","last_became_peered":"2024-09-21T03:37:12.860341+0000","last_unstale":"2024-09-21T03:39:21.864440+0000","last_undegraded":"2024-09-21T03:39:21.864440+0000","last_fullsized":"2024-09-21T03:39:21.864440+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:55:15.771259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"80'3","reported_seq":22,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:38:54.095929+0000","last_change":"2024-09-21T03:37:53.071125+0000","last_active":"2024-09-21T03:38:54.095929+0000","last_peered":"2024-09-21T03:38:54.095929+0000","last_clean":"2024-09-21T03:38:54.095929+0000","last_became_active":"2024-09-21T03:37:53.052237+0000","last_became_peered":"2024-09-21T03:37:53.052237+0000","last_unstale":"2024-09-21T03:38:54.095929+0000","last_undegraded":"2024-09-21T03:38:54.095929+0000","last_fullsized":"2024-09-21T03:38:54.095929+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_clean_scrub_stamp":"2024-09-21T03:37:09.809805+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:47:17.707045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,0],"acting":[4,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:38:54.096435+0000","last_change":"2024-09-21T03:37:19.909292+0000","last_active":"2024-09-21T03:38:54.096435+0000","last_peered":"2024-09-21T03:38:54.096435+0000","last_clean":"2024-09-21T03:38:54.096435+0000","last_became_active":"2024-09-21T03:37:19.908681+0000","last_became_peered":"2024-09-21T03:37:19.908681+0000","last_unstale":"2024-09-21T03:38:54.096435+0000","last_undegraded":"2024-09-21T03:38:54.096435+0000","last_fullsized":"2024-09-21T03:38:54.096435+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_clean_scrub_stamp":"2024-09-21T03:37:18.876933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:15:49.623109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"84'6","reported_seq":40,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:39:21.852497+0000","last_change":"2024-09-21T03:37:12.857678+0000","last_active":"2024-09-21T03:39:21.852497+0000","last_peered":"2024-09-21T03:39:21.852497+0000","last_clean":"2024-09-21T03:39:21.852497+0000","last_became_active":"2024-09-21T03:37:12.857266+0000","last_became_peered":"2024-09-21T03:37:12.857266+0000","last_unstale":"2024-09-21T03:39:21.852497+0000","last_undegraded":"2024-09-21T03:39:21.852497+0000","last_fullsized":"2024-09-21T03:39:21.852497+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:21:14.135784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"74'1","reported_seq":42,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:38:54.097310+0000","last_change":"2024-09-21T03:37:10.835323+0000","last_active":"2024-09-21T03:38:54.097310+0000","last_peered":"2024-09-21T03:38:54.097310+0000","last_clean":"2024-09-21T03:38:54.097310+0000","last_became_active":"2024-09-21T03:37:10.834950+0000","last_became_peered":"2024-09-21T03:37:10.834950+0000","last_unstale":"2024-09-21T03:38:54.097310+0000","last_undegraded":"2024-09-21T03:38:54.097310+0000","last_fullsized":"2024-09-21T03:38:54.097310+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_clean_scrub_stamp":"2024-09-21T03:37:09.809805+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:40:27.386342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:38:54.095581+0000","last_change":"2024-09-21T03:37:19.906360+0000","last_active":"2024-09-21T03:38:54.095581+0000","last_peered":"2024-09-21T03:38:54.095581+0000","last_clean":"2024-09-21T03:38:54.095581+0000","last_became_active":"2024-09-21T03:37:19.905888+0000","last_became_peered":"2024-09-21T03:37:19.905888+0000","last_unstale":"2024-09-21T03:38:54.095581+0000","last_undegraded":"2024-09-21T03:38:54.095581+0000","last_fullsized":"2024-09-21T03:38:54.095581+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_clean_scrub_stamp":"2024-09-21T03:37:18.876933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:13:55.885538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"84'9","reported_seq":43,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:39:21.849514+0000","last_change":"2024-09-21T03:37:12.849106+0000","last_active":"2024-09-21T03:39:21.849514+0000","last_peered":"2024-09-21T03:39:21.849514+0000","last_clean":"2024-09-21T03:39:21.849514+0000","last_became_active":"2024-09-21T03:37:12.848505+0000","last_became_peered":"2024-09-21T03:37:12.848505+0000","last_unstale":"2024-09-21T03:39:21.849514+0000","last_undegraded":"2024-09-21T03:39:21.849514+0000","last_fullsized":"2024-09-21T03:39:21.849514+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:50:19.569004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"74'2","reported_seq":52,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:38:53.668984+0000","last_change":"2024-09-21T03:37:53.115158+0000","last_active":"2024-09-21T03:38:53.668984+0000","last_peered":"2024-09-21T03:38:53.668984+0000","last_clean":"2024-09-21T03:38:53.668984+0000","last_became_active":"2024-09-21T03:37:53.112603+0000","last_became_peered":"2024-09-21T03:37:53.112603+0000","last_unstale":"2024-09-21T03:38:53.668984+0000","last_undegraded":"2024-09-21T03:38:53.668984+0000","last_fullsized":"2024-09-21T03:38:53.668984+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_clean_scrub_stamp":"2024-09-21T03:37:09.809805+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:23:21.525972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,1],"acting":[6,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:38:54.095667+0000","last_change":"2024-09-21T03:37:19.909040+0000","last_active":"2024-09-21T03:38:54.095667+0000","last_peered":"2024-09-21T03:38:54.095667+0000","last_clean":"2024-09-21T03:38:54.095667+0000","last_became_active":"2024-09-21T03:37:19.908530+0000","last_became_peered":"2024-09-21T03:37:19.908530+0000","last_unstale":"2024-09-21T03:38:54.095667+0000","last_undegraded":"2024-09-21T03:38:54.095667+0000","last_fullsized":"2024-09-21T03:38:54.095667+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_clean_scrub_stamp":"2024-09-21T03:37:18.876933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:35:20.919260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"84'5","reported_seq":39,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:39:21.851823+0000","last_change":"2024-09-21T03:37:12.858984+0000","last_active":"2024-09-21T03:39:21.851823+0000","last_peered":"2024-09-21T03:39:21.851823+0000","last_clean":"2024-09-21T03:39:21.851823+0000","last_became_active":"2024-09-21T03:37:12.858603+0000","last_became_peered":"2024-09-21T03:37:12.858603+0000","last_unstale":"2024-09-21T03:39:21.851823+0000","last_undegraded":"2024-09-21T03:39:21.851823+0000","last_fullsized":"2024-09-21T03:39:21.851823+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:16:45.025304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.f","version":"84'3","reported_seq":44,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:39:27.032126+0000","last_change":"2024-09-21T03:37:10.834844+0000","last_active":"2024-09-21T03:39:27.032126+0000","last_peered":"2024-09-21T03:39:27.032126+0000","last_clean":"2024-09-21T03:39:27.032126+0000","last_became_active":"2024-09-21T03:37:10.833664+0000","last_became_peered":"2024-09-21T03:37:10.833664+0000","last_unstale":"2024-09-21T03:39:27.032126+0000","last_undegraded":"2024-09-21T03:39:27.032126+0000","last_fullsized":"2024-09-21T03:39:27.032126+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_clean_scrub_stamp":"2024-09-21T03:37:09.809805+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:03:49.006414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:38:54.095855+0000","last_change":"2024-09-21T03:37:19.913277+0000","last_active":"2024-09-21T03:38:54.095855+0000","last_peered":"2024-09-21T03:38:54.095855+0000","last_clean":"2024-09-21T03:38:54.095855+0000","last_became_active":"2024-09-21T03:37:19.912781+0000","last_became_peered":"2024-09-21T03:37:19.912781+0000","last_unstale":"2024-09-21T03:38:54.095855+0000","last_undegraded":"2024-09-21T03:38:54.095855+0000","last_fullsized":"2024-09-21T03:38:54.095855+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_clean_scrub_stamp":"2024-09-21T03:37:18.876933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:18:27.980622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"84'6","reported_seq":40,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:39:21.847384+0000","last_change":"2024-09-21T03:37:12.862322+0000","last_active":"2024-09-21T03:39:21.847384+0000","last_peered":"2024-09-21T03:39:21.847384+0000","last_clean":"2024-09-21T03:39:21.847384+0000","last_became_active":"2024-09-21T03:37:12.861938+0000","last_became_peered":"2024-09-21T03:37:12.861938+0000","last_unstale":"2024-09-21T03:39:21.847384+0000","last_undegraded":"2024-09-21T03:39:21.847384+0000","last_fullsized":"2024-09-21T03:39:21.847384+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:01:48.944169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.10","version":"80'15","reported_seq":34,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:38:54.096078+0000","last_change":"2024-09-21T03:37:53.067130+0000","last_active":"2024-09-21T03:38:54.096078+0000","last_peered":"2024-09-21T03:38:54.096078+0000","last_clean":"2024-09-21T03:38:54.096078+0000","last_became_active":"2024-09-21T03:37:53.051596+0000","last_became_peered":"2024-09-21T03:37:53.051596+0000","last_unstale":"2024-09-21T03:38:54.096078+0000","last_undegraded":"2024-09-21T03:38:54.096078+0000","last_fullsized":"2024-09-21T03:38:54.096078+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_clean_scrub_stamp":"2024-09-21T03:37:09.809805+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:02:22.852015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:38:53.668923+0000","last_change":"2024-09-21T03:37:19.913206+0000","last_active":"2024-09-21T03:38:53.668923+0000","last_peered":"2024-09-21T03:38:53.668923+0000","last_clean":"2024-09-21T03:38:53.668923+0000","last_became_active":"2024-09-21T03:37:19.910718+0000","last_became_peered":"2024-09-21T03:37:19.910718+0000","last_unstale":"2024-09-21T03:38:53.668923+0000","last_undegraded":"2024-09-21T03:38:53.668923+0000","last_fullsized":"2024-09-21T03:38:53.668923+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_clean_scrub_stamp":"2024-09-21T03:37:18.876933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:11:31.431237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"84'6","reported_seq":40,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:39:21.854210+0000","last_change":"2024-09-21T03:37:12.857505+0000","last_active":"2024-09-21T03:39:21.854210+0000","last_peered":"2024-09-21T03:39:21.854210+0000","last_clean":"2024-09-21T03:39:21.854210+0000","last_became_active":"2024-09-21T03:37:12.857155+0000","last_became_peered":"2024-09-21T03:37:12.857155+0000","last_unstale":"2024-09-21T03:39:21.854210+0000","last_undegraded":"2024-09-21T03:39:21.854210+0000","last_fullsized":"2024-09-21T03:39:21.854210+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:05:50.387651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"80'3","reported_seq":138,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:39:21.178872+0000","last_change":"2024-09-21T03:37:10.832364+0000","last_active":"2024-09-21T03:39:21.178872+0000","last_peered":"2024-09-21T03:39:21.178872+0000","last_clean":"2024-09-21T03:39:21.178872+0000","last_became_active":"2024-09-21T03:37:10.832004+0000","last_became_peered":"2024-09-21T03:37:10.832004+0000","last_unstale":"2024-09-21T03:39:21.178872+0000","last_undegraded":"2024-09-21T03:39:21.178872+0000","last_fullsized":"2024-09-21T03:39:21.178872+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_clean_scrub_stamp":"2024-09-21T03:37:09.809805+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:21:10.895732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":97,"num_read_kb":3,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:38:53.672765+0000","last_change":"2024-09-21T03:37:19.912297+0000","last_active":"2024-09-21T03:38:53.672765+0000","last_peered":"2024-09-21T03:38:53.672765+0000","last_clean":"2024-09-21T03:38:53.672765+0000","last_became_active":"2024-09-21T03:37:19.911903+0000","last_became_peered":"2024-09-21T03:37:19.911903+0000","last_unstale":"2024-09-21T03:38:53.672765+0000","last_undegraded":"2024-09-21T03:38:53.672765+0000","last_fullsized":"2024-09-21T03:38:53.672765+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_clean_scrub_stamp":"2024-09-21T03:37:18.876933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:05:56.747246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.13","version":"84'15","reported_seq":49,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:39:21.853966+0000","last_change":"2024-09-21T03:37:12.856230+0000","last_active":"2024-09-21T03:39:21.853966+0000","last_peered":"2024-09-21T03:39:21.853966+0000","last_clean":"2024-09-21T03:39:21.853966+0000","last_became_active":"2024-09-21T03:37:12.855819+0000","last_became_peered":"2024-09-21T03:37:12.855819+0000","last_unstale":"2024-09-21T03:39:21.853966+0000","last_undegraded":"2024-09-21T03:39:21.853966+0000","last_fullsized":"2024-09-21T03:39:21.853966+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:38:18.618579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"80'3","reported_seq":42,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:38:54.096706+0000","last_change":"2024-09-21T03:37:10.832401+0000","last_active":"2024-09-21T03:38:54.096706+0000","last_peered":"2024-09-21T03:38:54.096706+0000","last_clean":"2024-09-21T03:38:54.096706+0000","last_became_active":"2024-09-21T03:37:10.832033+0000","last_became_peered":"2024-09-21T03:37:10.832033+0000","last_unstale":"2024-09-21T03:38:54.096706+0000","last_undegraded":"2024-09-21T03:38:54.096706+0000","last_fullsized":"2024-09-21T03:38:54.096706+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_clean_scrub_stamp":"2024-09-21T03:37:09.809805+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:25:20.751529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:38:54.096414+0000","last_change":"2024-09-21T03:37:19.905825+0000","last_active":"2024-09-21T03:38:54.096414+0000","last_peered":"2024-09-21T03:38:54.096414+0000","last_clean":"2024-09-21T03:38:54.096414+0000","last_became_active":"2024-09-21T03:37:19.905352+0000","last_became_peered":"2024-09-21T03:37:19.905352+0000","last_unstale":"2024-09-21T03:38:54.096414+0000","last_undegraded":"2024-09-21T03:38:54.096414+0000","last_fullsized":"2024-09-21T03:38:54.096414+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_clean_scrub_stamp":"2024-09-21T03:37:18.876933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:25:05.861981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"84'9","reported_seq":43,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:39:21.847167+0000","last_change":"2024-09-21T03:37:12.852006+0000","last_active":"2024-09-21T03:39:21.847167+0000","last_peered":"2024-09-21T03:39:21.847167+0000","last_clean":"2024-09-21T03:39:21.847167+0000","last_became_active":"2024-09-21T03:37:12.851670+0000","last_became_peered":"2024-09-21T03:37:12.851670+0000","last_unstale":"2024-09-21T03:39:21.847167+0000","last_undegraded":"2024-09-21T03:39:21.847167+0000","last_fullsized":"2024-09-21T03:39:21.847167+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:41:50.177349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"80'6","reported_seq":46,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:38:53.670064+0000","last_change":"2024-09-21T03:37:10.832472+0000","last_active":"2024-09-21T03:38:53.670064+0000","last_peered":"2024-09-21T03:38:53.670064+0000","last_clean":"2024-09-21T03:38:53.670064+0000","last_became_active":"2024-09-21T03:37:10.832085+0000","last_became_peered":"2024-09-21T03:37:10.832085+0000","last_unstale":"2024-09-21T03:38:53.670064+0000","last_undegraded":"2024-09-21T03:38:53.670064+0000","last_fullsized":"2024-09-21T03:38:53.670064+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_clean_scrub_stamp":"2024-09-21T03:37:09.809805+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:06:41.251218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:38:54.096945+0000","last_change":"2024-09-21T03:37:19.914042+0000","last_active":"2024-09-21T03:38:54.096945+0000","last_peered":"2024-09-21T03:38:54.096945+0000","last_clean":"2024-09-21T03:38:54.096945+0000","last_became_active":"2024-09-21T03:37:19.913531+0000","last_became_peered":"2024-09-21T03:37:19.913531+0000","last_unstale":"2024-09-21T03:38:54.096945+0000","last_undegraded":"2024-09-21T03:38:54.096945+0000","last_fullsized":"2024-09-21T03:38:54.096945+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_clean_scrub_stamp":"2024-09-21T03:37:18.876933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:16:40.163359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"84'6","reported_seq":40,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:39:21.852857+0000","last_change":"2024-09-21T03:37:12.852893+0000","last_active":"2024-09-21T03:39:21.852857+0000","last_peered":"2024-09-21T03:39:21.852857+0000","last_clean":"2024-09-21T03:39:21.852857+0000","last_became_active":"2024-09-21T03:37:12.852452+0000","last_became_peered":"2024-09-21T03:37:12.852452+0000","last_unstale":"2024-09-21T03:39:21.852857+0000","last_undegraded":"2024-09-21T03:39:21.852857+0000","last_fullsized":"2024-09-21T03:39:21.852857+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:46:59.393901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.14","version":"84'36","reported_seq":278,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:39:24.185435+0000","last_change":"2024-09-21T03:37:10.837654+0000","last_active":"2024-09-21T03:39:24.185435+0000","last_peered":"2024-09-21T03:39:24.185435+0000","last_clean":"2024-09-21T03:39:24.185435+0000","last_became_active":"2024-09-21T03:37:10.837251+0000","last_became_peered":"2024-09-21T03:37:10.837251+0000","last_unstale":"2024-09-21T03:39:24.185435+0000","last_undegraded":"2024-09-21T03:39:24.185435+0000","last_fullsized":"2024-09-21T03:39:24.185435+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_clean_scrub_stamp":"2024-09-21T03:37:09.809805+0000","objects_scrubbed":0,"log_size":36,"log_dups_size":0,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:43:32.986371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":67880,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":203,"num_read_kb":168,"num_write":37,"num_write_kb":84,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:38:53.669528+0000","last_change":"2024-09-21T03:37:53.053405+0000","last_active":"2024-09-21T03:38:53.669528+0000","last_peered":"2024-09-21T03:38:53.669528+0000","last_clean":"2024-09-21T03:38:53.669528+0000","last_became_active":"2024-09-21T03:37:53.052818+0000","last_became_peered":"2024-09-21T03:37:53.052818+0000","last_unstale":"2024-09-21T03:38:53.669528+0000","last_undegraded":"2024-09-21T03:38:53.669528+0000","last_fullsized":"2024-09-21T03:38:53.669528+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_clean_scrub_stamp":"2024-09-21T03:37:18.876933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:05:09.121036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,11,5],"acting":[10,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.14","version":"84'11","reported_seq":45,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:39:21.857069+0000","last_change":"2024-09-21T03:37:12.866588+0000","last_active":"2024-09-21T03:39:21.857069+0000","last_peered":"2024-09-21T03:39:21.857069+0000","last_clean":"2024-09-21T03:39:21.857069+0000","last_became_active":"2024-09-21T03:37:12.866075+0000","last_became_peered":"2024-09-21T03:37:12.866075+0000","last_unstale":"2024-09-21T03:39:21.857069+0000","last_undegraded":"2024-09-21T03:39:21.857069+0000","last_fullsized":"2024-09-21T03:39:21.857069+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:18:48.817562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"80'4","reported_seq":44,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:38:54.097208+0000","last_change":"2024-09-21T03:37:11.036751+0000","last_active":"2024-09-21T03:38:54.097208+0000","last_peered":"2024-09-21T03:38:54.097208+0000","last_clean":"2024-09-21T03:38:54.097208+0000","last_became_active":"2024-09-21T03:37:11.036125+0000","last_became_peered":"2024-09-21T03:37:11.036125+0000","last_unstale":"2024-09-21T03:38:54.097208+0000","last_undegraded":"2024-09-21T03:38:54.097208+0000","last_fullsized":"2024-09-21T03:38:54.097208+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_clean_scrub_stamp":"2024-09-21T03:37:09.809805+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:46:13.569422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":34,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:38:54.095956+0000","last_change":"2024-09-21T03:37:53.112845+0000","last_active":"2024-09-21T03:38:54.095956+0000","last_peered":"2024-09-21T03:38:54.095956+0000","last_clean":"2024-09-21T03:38:54.095956+0000","last_became_active":"2024-09-21T03:37:53.112286+0000","last_became_peered":"2024-09-21T03:37:53.112286+0000","last_unstale":"2024-09-21T03:38:54.095956+0000","last_undegraded":"2024-09-21T03:38:54.095956+0000","last_fullsized":"2024-09-21T03:38:54.095956+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_clean_scrub_stamp":"2024-09-21T03:37:18.876933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:29:58.062684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,8],"acting":[4,1,2,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"84'9","reported_seq":43,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:39:21.851704+0000","last_change":"2024-09-21T03:37:12.865011+0000","last_active":"2024-09-21T03:39:21.851704+0000","last_peered":"2024-09-21T03:39:21.851704+0000","last_clean":"2024-09-21T03:39:21.851704+0000","last_became_active":"2024-09-21T03:37:12.864645+0000","last_became_peered":"2024-09-21T03:37:12.864645+0000","last_unstale":"2024-09-21T03:39:21.851704+0000","last_undegraded":"2024-09-21T03:39:21.851704+0000","last_fullsized":"2024-09-21T03:39:21.851704+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:27:42.471227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"80'2","reported_seq":41,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:38:53.941930+0000","last_change":"2024-09-21T03:37:10.835171+0000","last_active":"2024-09-21T03:38:53.941930+0000","last_peered":"2024-09-21T03:38:53.941930+0000","last_clean":"2024-09-21T03:38:53.941930+0000","last_became_active":"2024-09-21T03:37:10.834744+0000","last_became_peered":"2024-09-21T03:37:10.834744+0000","last_unstale":"2024-09-21T03:38:53.941930+0000","last_undegraded":"2024-09-21T03:38:53.941930+0000","last_fullsized":"2024-09-21T03:38:53.941930+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_clean_scrub_stamp":"2024-09-21T03:37:09.809805+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:02:37.720457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:38:53.672239+0000","last_change":"2024-09-21T03:37:19.906734+0000","last_active":"2024-09-21T03:38:53.672239+0000","last_peered":"2024-09-21T03:38:53.672239+0000","last_clean":"2024-09-21T03:38:53.672239+0000","last_became_active":"2024-09-21T03:37:19.906327+0000","last_became_peered":"2024-09-21T03:37:19.906327+0000","last_unstale":"2024-09-21T03:38:53.672239+0000","last_undegraded":"2024-09-21T03:38:53.672239+0000","last_fullsized":"2024-09-21T03:38:53.672239+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_clean_scrub_stamp":"2024-09-21T03:37:18.876933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:55:11.603042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.16","version":"84'10","reported_seq":44,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:39:21.866214+0000","last_change":"2024-09-21T03:37:12.866285+0000","last_active":"2024-09-21T03:39:21.866214+0000","last_peered":"2024-09-21T03:39:21.866214+0000","last_clean":"2024-09-21T03:39:21.866214+0000","last_became_active":"2024-09-21T03:37:12.865854+0000","last_became_peered":"2024-09-21T03:37:12.865854+0000","last_unstale":"2024-09-21T03:39:21.866214+0000","last_undegraded":"2024-09-21T03:39:21.866214+0000","last_fullsized":"2024-09-21T03:39:21.866214+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:00:18.075248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"84'4","reported_seq":139,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:39:23.182875+0000","last_change":"2024-09-21T03:37:10.835808+0000","last_active":"2024-09-21T03:39:23.182875+0000","last_peered":"2024-09-21T03:39:23.182875+0000","last_clean":"2024-09-21T03:39:23.182875+0000","last_became_active":"2024-09-21T03:37:10.835434+0000","last_became_peered":"2024-09-21T03:37:10.835434+0000","last_unstale":"2024-09-21T03:39:23.182875+0000","last_undegraded":"2024-09-21T03:39:23.182875+0000","last_fullsized":"2024-09-21T03:39:23.182875+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_clean_scrub_stamp":"2024-09-21T03:37:09.809805+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:35:39.817312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":96,"num_read_kb":96,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:38:53.971629+0000","last_change":"2024-09-21T03:37:19.915986+0000","last_active":"2024-09-21T03:38:53.971629+0000","last_peered":"2024-09-21T03:38:53.971629+0000","last_clean":"2024-09-21T03:38:53.971629+0000","last_became_active":"2024-09-21T03:37:19.915561+0000","last_became_peered":"2024-09-21T03:37:19.915561+0000","last_unstale":"2024-09-21T03:38:53.971629+0000","last_undegraded":"2024-09-21T03:38:53.971629+0000","last_fullsized":"2024-09-21T03:38:53.971629+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_clean_scrub_stamp":"2024-09-21T03:37:18.876933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:21:13.789664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":27,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:38:54.095727+0000","last_change":"2024-09-21T03:37:19.908230+0000","last_active":"2024-09-21T03:38:54.095727+0000","last_peered":"2024-09-21T03:38:54.095727+0000","last_clean":"2024-09-21T03:38:54.095727+0000","last_became_active":"2024-09-21T03:37:19.907843+0000","last_became_peered":"2024-09-21T03:37:19.907843+0000","last_unstale":"2024-09-21T03:38:54.095727+0000","last_undegraded":"2024-09-21T03:38:54.095727+0000","last_fullsized":"2024-09-21T03:38:54.095727+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_clean_scrub_stamp":"2024-09-21T03:37:18.876933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:15:58.142127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"84'11","reported_seq":45,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:39:21.851083+0000","last_change":"2024-09-21T03:37:12.856709+0000","last_active":"2024-09-21T03:39:21.851083+0000","last_peered":"2024-09-21T03:39:21.851083+0000","last_clean":"2024-09-21T03:39:21.851083+0000","last_became_active":"2024-09-21T03:37:12.856278+0000","last_became_peered":"2024-09-21T03:37:12.856278+0000","last_unstale":"2024-09-21T03:39:21.851083+0000","last_undegraded":"2024-09-21T03:39:21.851083+0000","last_fullsized":"2024-09-21T03:39:21.851083+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:16:13.884648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"80'1","reported_seq":40,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:38:54.096239+0000","last_change":"2024-09-21T03:37:10.832456+0000","last_active":"2024-09-21T03:38:54.096239+0000","last_peered":"2024-09-21T03:38:54.096239+0000","last_clean":"2024-09-21T03:38:54.096239+0000","last_became_active":"2024-09-21T03:37:10.832082+0000","last_became_peered":"2024-09-21T03:37:10.832082+0000","last_unstale":"2024-09-21T03:38:54.096239+0000","last_undegraded":"2024-09-21T03:38:54.096239+0000","last_fullsized":"2024-09-21T03:38:54.096239+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_clean_scrub_stamp":"2024-09-21T03:37:09.809805+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:17:12.170616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":27,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:38:54.096158+0000","last_change":"2024-09-21T03:37:19.908084+0000","last_active":"2024-09-21T03:38:54.096158+0000","last_peered":"2024-09-21T03:38:54.096158+0000","last_clean":"2024-09-21T03:38:54.096158+0000","last_became_active":"2024-09-21T03:37:19.907307+0000","last_became_peered":"2024-09-21T03:37:19.907307+0000","last_unstale":"2024-09-21T03:38:54.096158+0000","last_undegraded":"2024-09-21T03:38:54.096158+0000","last_fullsized":"2024-09-21T03:38:54.096158+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_clean_scrub_stamp":"2024-09-21T03:37:18.876933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:45:02.541015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.18","version":"84'8","reported_seq":42,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:39:21.852435+0000","last_change":"2024-09-21T03:37:12.852577+0000","last_active":"2024-09-21T03:39:21.852435+0000","last_peered":"2024-09-21T03:39:21.852435+0000","last_clean":"2024-09-21T03:39:21.852435+0000","last_became_active":"2024-09-21T03:37:12.852186+0000","last_became_peered":"2024-09-21T03:37:12.852186+0000","last_unstale":"2024-09-21T03:39:21.852435+0000","last_undegraded":"2024-09-21T03:39:21.852435+0000","last_fullsized":"2024-09-21T03:39:21.852435+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:51:21.631029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"80'2","reported_seq":41,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:38:53.668330+0000","last_change":"2024-09-21T03:37:10.833587+0000","last_active":"2024-09-21T03:38:53.668330+0000","last_peered":"2024-09-21T03:38:53.668330+0000","last_clean":"2024-09-21T03:38:53.668330+0000","last_became_active":"2024-09-21T03:37:10.833194+0000","last_became_peered":"2024-09-21T03:37:10.833194+0000","last_unstale":"2024-09-21T03:38:53.668330+0000","last_undegraded":"2024-09-21T03:38:53.668330+0000","last_fullsized":"2024-09-21T03:38:53.668330+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_clean_scrub_stamp":"2024-09-21T03:37:09.809805+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:20:20.272108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"80'3","reported_seq":42,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:38:53.669769+0000","last_change":"2024-09-21T03:37:10.839103+0000","last_active":"2024-09-21T03:38:53.669769+0000","last_peered":"2024-09-21T03:38:53.669769+0000","last_clean":"2024-09-21T03:38:53.669769+0000","last_became_active":"2024-09-21T03:37:10.837192+0000","last_became_peered":"2024-09-21T03:37:10.837192+0000","last_unstale":"2024-09-21T03:38:53.669769+0000","last_undegraded":"2024-09-21T03:38:53.669769+0000","last_fullsized":"2024-09-21T03:38:53.669769+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_clean_scrub_stamp":"2024-09-21T03:37:09.809805+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:59:35.128171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"84'8","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:39:21.852475+0000","last_change":"2024-09-21T03:38:54.097890+0000","last_active":"2024-09-21T03:39:21.852475+0000","last_peered":"2024-09-21T03:39:21.852475+0000","last_clean":"2024-09-21T03:39:21.852475+0000","last_became_active":"2024-09-21T03:38:54.097484+0000","last_became_peered":"2024-09-21T03:38:54.097484+0000","last_unstale":"2024-09-21T03:39:21.852475+0000","last_undegraded":"2024-09-21T03:39:21.852475+0000","last_fullsized":"2024-09-21T03:39:21.852475+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:06:16.098400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,3],"acting":[9,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:38:53.669938+0000","last_change":"2024-09-21T03:37:19.909415+0000","last_active":"2024-09-21T03:38:53.669938+0000","last_peered":"2024-09-21T03:38:53.669938+0000","last_clean":"2024-09-21T03:38:53.669938+0000","last_became_active":"2024-09-21T03:37:19.908955+0000","last_became_peered":"2024-09-21T03:37:19.908955+0000","last_unstale":"2024-09-21T03:38:53.669938+0000","last_undegraded":"2024-09-21T03:38:53.669938+0000","last_fullsized":"2024-09-21T03:38:53.669938+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_clean_scrub_stamp":"2024-09-21T03:37:18.876933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:31:18.878725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"84'9","reported_seq":151,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:39:26.189837+0000","last_change":"2024-09-21T03:37:10.836389+0000","last_active":"2024-09-21T03:39:26.189837+0000","last_peered":"2024-09-21T03:39:26.189837+0000","last_clean":"2024-09-21T03:39:26.189837+0000","last_became_active":"2024-09-21T03:37:10.835986+0000","last_became_peered":"2024-09-21T03:37:10.835986+0000","last_unstale":"2024-09-21T03:39:26.189837+0000","last_undegraded":"2024-09-21T03:39:26.189837+0000","last_fullsized":"2024-09-21T03:39:26.189837+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_clean_scrub_stamp":"2024-09-21T03:37:09.809805+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:05:17.963609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2038,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":103,"num_read_kb":5,"num_write":6,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"84'7","reported_seq":41,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:39:21.834248+0000","last_change":"2024-09-21T03:37:12.856008+0000","last_active":"2024-09-21T03:39:21.834248+0000","last_peered":"2024-09-21T03:39:21.834248+0000","last_clean":"2024-09-21T03:39:21.834248+0000","last_became_active":"2024-09-21T03:37:12.855603+0000","last_became_peered":"2024-09-21T03:37:12.855603+0000","last_unstale":"2024-09-21T03:39:21.834248+0000","last_undegraded":"2024-09-21T03:39:21.834248+0000","last_fullsized":"2024-09-21T03:39:21.834248+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:11:41.920114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:38:53.669941+0000","last_change":"2024-09-21T03:37:19.911507+0000","last_active":"2024-09-21T03:38:53.669941+0000","last_peered":"2024-09-21T03:38:53.669941+0000","last_clean":"2024-09-21T03:38:53.669941+0000","last_became_active":"2024-09-21T03:37:19.911053+0000","last_became_peered":"2024-09-21T03:37:19.911053+0000","last_unstale":"2024-09-21T03:38:53.669941+0000","last_undegraded":"2024-09-21T03:38:53.669941+0000","last_fullsized":"2024-09-21T03:38:53.669941+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_clean_scrub_stamp":"2024-09-21T03:37:18.876933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:42:11.964829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"84'9","reported_seq":44,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:39:21.853090+0000","last_change":"2024-09-21T03:37:12.862576+0000","last_active":"2024-09-21T03:39:21.853090+0000","last_peered":"2024-09-21T03:39:21.853090+0000","last_clean":"2024-09-21T03:39:21.853090+0000","last_became_active":"2024-09-21T03:37:12.862146+0000","last_became_peered":"2024-09-21T03:37:12.862146+0000","last_unstale":"2024-09-21T03:39:21.853090+0000","last_undegraded":"2024-09-21T03:39:21.853090+0000","last_fullsized":"2024-09-21T03:39:21.853090+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:30:33.272415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"84'12","reported_seq":47,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:39:21.853470+0000","last_change":"2024-09-21T03:37:12.859395+0000","last_active":"2024-09-21T03:39:21.853470+0000","last_peered":"2024-09-21T03:39:21.853470+0000","last_clean":"2024-09-21T03:39:21.853470+0000","last_became_active":"2024-09-21T03:37:12.858914+0000","last_became_peered":"2024-09-21T03:37:12.858914+0000","last_unstale":"2024-09-21T03:39:21.853470+0000","last_undegraded":"2024-09-21T03:39:21.853470+0000","last_fullsized":"2024-09-21T03:39:21.853470+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:13:06.319896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"84'8","reported_seq":43,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:39:21.844893+0000","last_change":"2024-09-21T03:37:12.857467+0000","last_active":"2024-09-21T03:39:21.844893+0000","last_peered":"2024-09-21T03:39:21.844893+0000","last_clean":"2024-09-21T03:39:21.844893+0000","last_became_active":"2024-09-21T03:37:12.856214+0000","last_became_peered":"2024-09-21T03:37:12.856214+0000","last_unstale":"2024-09-21T03:39:21.844893+0000","last_undegraded":"2024-09-21T03:39:21.844893+0000","last_fullsized":"2024-09-21T03:39:21.844893+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:21:58.780018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"84'7","reported_seq":42,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:39:21.842894+0000","last_change":"2024-09-21T03:37:12.858052+0000","last_active":"2024-09-21T03:39:21.842894+0000","last_peered":"2024-09-21T03:39:21.842894+0000","last_clean":"2024-09-21T03:39:21.842894+0000","last_became_active":"2024-09-21T03:37:12.857445+0000","last_became_peered":"2024-09-21T03:37:12.857445+0000","last_unstale":"2024-09-21T03:39:21.842894+0000","last_undegraded":"2024-09-21T03:39:21.842894+0000","last_fullsized":"2024-09-21T03:39:21.842894+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:22:28.058846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"84'4","reported_seq":39,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:39:21.852798+0000","last_change":"2024-09-21T03:37:12.856499+0000","last_active":"2024-09-21T03:39:21.852798+0000","last_peered":"2024-09-21T03:39:21.852798+0000","last_clean":"2024-09-21T03:39:21.852798+0000","last_became_active":"2024-09-21T03:37:12.856146+0000","last_became_peered":"2024-09-21T03:37:12.856146+0000","last_unstale":"2024-09-21T03:39:21.852798+0000","last_undegraded":"2024-09-21T03:39:21.852798+0000","last_fullsized":"2024-09-21T03:39:21.852798+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:03:26.142740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"84'6","reported_seq":41,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:39:21.851873+0000","last_change":"2024-09-21T03:37:12.860830+0000","last_active":"2024-09-21T03:39:21.851873+0000","last_peered":"2024-09-21T03:39:21.851873+0000","last_clean":"2024-09-21T03:39:21.851873+0000","last_became_active":"2024-09-21T03:37:12.860468+0000","last_became_peered":"2024-09-21T03:37:12.860468+0000","last_unstale":"2024-09-21T03:39:21.851873+0000","last_undegraded":"2024-09-21T03:39:21.851873+0000","last_fullsized":"2024-09-21T03:39:21.851873+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:48:15.057622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"84'15","reported_seq":50,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:39:21.849602+0000","last_change":"2024-09-21T03:37:12.864399+0000","last_active":"2024-09-21T03:39:21.849602+0000","last_peered":"2024-09-21T03:39:21.849602+0000","last_clean":"2024-09-21T03:39:21.849602+0000","last_became_active":"2024-09-21T03:37:12.863892+0000","last_became_peered":"2024-09-21T03:37:12.863892+0000","last_unstale":"2024-09-21T03:39:21.849602+0000","last_undegraded":"2024-09-21T03:39:21.849602+0000","last_fullsized":"2024-09-21T03:39:21.849602+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:58:26.647896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"84'2","reported_seq":37,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:39:21.847416+0000","last_change":"2024-09-21T03:37:12.861716+0000","last_active":"2024-09-21T03:39:21.847416+0000","last_peered":"2024-09-21T03:39:21.847416+0000","last_clean":"2024-09-21T03:39:21.847416+0000","last_became_active":"2024-09-21T03:37:12.861392+0000","last_became_peered":"2024-09-21T03:37:12.861392+0000","last_unstale":"2024-09-21T03:39:21.847416+0000","last_undegraded":"2024-09-21T03:39:21.847416+0000","last_fullsized":"2024-09-21T03:39:21.847416+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:28:55.751107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"84'6","reported_seq":41,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:39:21.842051+0000","last_change":"2024-09-21T03:37:12.856167+0000","last_active":"2024-09-21T03:39:21.842051+0000","last_peered":"2024-09-21T03:39:21.842051+0000","last_clean":"2024-09-21T03:39:21.842051+0000","last_became_active":"2024-09-21T03:37:12.855752+0000","last_became_peered":"2024-09-21T03:37:12.855752+0000","last_unstale":"2024-09-21T03:39:21.842051+0000","last_undegraded":"2024-09-21T03:39:21.842051+0000","last_fullsized":"2024-09-21T03:39:21.842051+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:26:30.271342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"84'8","reported_seq":43,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:39:21.852887+0000","last_change":"2024-09-21T03:37:12.862782+0000","last_active":"2024-09-21T03:39:21.852887+0000","last_peered":"2024-09-21T03:39:21.852887+0000","last_clean":"2024-09-21T03:39:21.852887+0000","last_became_active":"2024-09-21T03:37:12.862462+0000","last_became_peered":"2024-09-21T03:37:12.862462+0000","last_unstale":"2024-09-21T03:39:21.852887+0000","last_undegraded":"2024-09-21T03:39:21.852887+0000","last_fullsized":"2024-09-21T03:39:21.852887+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:57:37.744531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"84'10","reported_seq":44,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:39:21.863800+0000","last_change":"2024-09-21T03:37:12.862312+0000","last_active":"2024-09-21T03:39:21.863800+0000","last_peered":"2024-09-21T03:39:21.863800+0000","last_clean":"2024-09-21T03:39:21.863800+0000","last_became_active":"2024-09-21T03:37:12.861940+0000","last_became_peered":"2024-09-21T03:37:12.861940+0000","last_unstale":"2024-09-21T03:39:21.863800+0000","last_undegraded":"2024-09-21T03:39:21.863800+0000","last_fullsized":"2024-09-21T03:39:21.863800+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:08:49.165522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"84'11","reported_seq":45,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:39:21.847875+0000","last_change":"2024-09-21T03:37:12.857012+0000","last_active":"2024-09-21T03:39:21.847875+0000","last_peered":"2024-09-21T03:39:21.847875+0000","last_clean":"2024-09-21T03:39:21.847875+0000","last_became_active":"2024-09-21T03:37:12.856390+0000","last_became_peered":"2024-09-21T03:37:12.856390+0000","last_unstale":"2024-09-21T03:39:21.847875+0000","last_undegraded":"2024-09-21T03:39:21.847875+0000","last_fullsized":"2024-09-21T03:39:21.847875+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:51:00.839709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"84'8","reported_seq":42,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:39:21.851665+0000","last_change":"2024-09-21T03:37:12.857207+0000","last_active":"2024-09-21T03:39:21.851665+0000","last_peered":"2024-09-21T03:39:21.851665+0000","last_clean":"2024-09-21T03:39:21.851665+0000","last_became_active":"2024-09-21T03:37:12.856770+0000","last_became_peered":"2024-09-21T03:37:12.856770+0000","last_unstale":"2024-09-21T03:39:21.851665+0000","last_undegraded":"2024-09-21T03:39:21.851665+0000","last_fullsized":"2024-09-21T03:39:21.851665+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:37:18.272321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2d","version":"84'7","reported_seq":42,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:39:21.845170+0000","last_change":"2024-09-21T03:37:12.858540+0000","last_active":"2024-09-21T03:39:21.845170+0000","last_peered":"2024-09-21T03:39:21.845170+0000","last_clean":"2024-09-21T03:39:21.845170+0000","last_became_active":"2024-09-21T03:37:12.857340+0000","last_became_peered":"2024-09-21T03:37:12.857340+0000","last_unstale":"2024-09-21T03:39:21.845170+0000","last_undegraded":"2024-09-21T03:39:21.845170+0000","last_fullsized":"2024-09-21T03:39:21.845170+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:45:48.982132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"84'10","reported_seq":44,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:39:21.851214+0000","last_change":"2024-09-21T03:37:12.864408+0000","last_active":"2024-09-21T03:39:21.851214+0000","last_peered":"2024-09-21T03:39:21.851214+0000","last_clean":"2024-09-21T03:39:21.851214+0000","last_became_active":"2024-09-21T03:37:12.864082+0000","last_became_peered":"2024-09-21T03:37:12.864082+0000","last_unstale":"2024-09-21T03:39:21.851214+0000","last_undegraded":"2024-09-21T03:39:21.851214+0000","last_fullsized":"2024-09-21T03:39:21.851214+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:19:32.723589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2f","version":"84'8","reported_seq":42,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:39:21.864236+0000","last_change":"2024-09-21T03:37:12.869445+0000","last_active":"2024-09-21T03:39:21.864236+0000","last_peered":"2024-09-21T03:39:21.864236+0000","last_clean":"2024-09-21T03:39:21.864236+0000","last_became_active":"2024-09-21T03:37:12.869058+0000","last_became_peered":"2024-09-21T03:37:12.869058+0000","last_unstale":"2024-09-21T03:39:21.864236+0000","last_undegraded":"2024-09-21T03:39:21.864236+0000","last_fullsized":"2024-09-21T03:39:21.864236+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:37:57.750665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.30","version":"84'9","reported_seq":43,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:39:21.865647+0000","last_change":"2024-09-21T03:37:12.858431+0000","last_active":"2024-09-21T03:39:21.865647+0000","last_peered":"2024-09-21T03:39:21.865647+0000","last_clean":"2024-09-21T03:39:21.865647+0000","last_became_active":"2024-09-21T03:37:12.858047+0000","last_became_peered":"2024-09-21T03:37:12.858047+0000","last_unstale":"2024-09-21T03:39:21.865647+0000","last_undegraded":"2024-09-21T03:39:21.865647+0000","last_fullsized":"2024-09-21T03:39:21.865647+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:14:05.363743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.31","version":"84'7","reported_seq":41,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:39:21.845694+0000","last_change":"2024-09-21T03:37:12.866784+0000","last_active":"2024-09-21T03:39:21.845694+0000","last_peered":"2024-09-21T03:39:21.845694+0000","last_clean":"2024-09-21T03:39:21.845694+0000","last_became_active":"2024-09-21T03:37:12.866435+0000","last_became_peered":"2024-09-21T03:37:12.866435+0000","last_unstale":"2024-09-21T03:39:21.845694+0000","last_undegraded":"2024-09-21T03:39:21.845694+0000","last_fullsized":"2024-09-21T03:39:21.845694+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:09:06.156030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.32","version":"84'7","reported_seq":41,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:39:21.850666+0000","last_change":"2024-09-21T03:37:12.857054+0000","last_active":"2024-09-21T03:39:21.850666+0000","last_peered":"2024-09-21T03:39:21.850666+0000","last_clean":"2024-09-21T03:39:21.850666+0000","last_became_active":"2024-09-21T03:37:12.856671+0000","last_became_peered":"2024-09-21T03:37:12.856671+0000","last_unstale":"2024-09-21T03:39:21.850666+0000","last_undegraded":"2024-09-21T03:39:21.850666+0000","last_fullsized":"2024-09-21T03:39:21.850666+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:38:27.474550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"84'10","reported_seq":44,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:39:21.850693+0000","last_change":"2024-09-21T03:37:12.861501+0000","last_active":"2024-09-21T03:39:21.850693+0000","last_peered":"2024-09-21T03:39:21.850693+0000","last_clean":"2024-09-21T03:39:21.850693+0000","last_became_active":"2024-09-21T03:37:12.861146+0000","last_became_peered":"2024-09-21T03:37:12.861146+0000","last_unstale":"2024-09-21T03:39:21.850693+0000","last_undegraded":"2024-09-21T03:39:21.850693+0000","last_fullsized":"2024-09-21T03:39:21.850693+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:30:55.073854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"84'7","reported_seq":23,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:39:21.850561+0000","last_change":"2024-09-21T03:37:53.051912+0000","last_active":"2024-09-21T03:39:21.850561+0000","last_peered":"2024-09-21T03:39:21.850561+0000","last_clean":"2024-09-21T03:39:21.850561+0000","last_became_active":"2024-09-21T03:37:53.051640+0000","last_became_peered":"2024-09-21T03:37:53.051640+0000","last_unstale":"2024-09-21T03:39:21.850561+0000","last_undegraded":"2024-09-21T03:39:21.850561+0000","last_fullsized":"2024-09-21T03:39:21.850561+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:08:09.046867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,10,8],"acting":[9,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.35","version":"84'6","reported_seq":40,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:39:21.864066+0000","last_change":"2024-09-21T03:37:12.865862+0000","last_active":"2024-09-21T03:39:21.864066+0000","last_peered":"2024-09-21T03:39:21.864066+0000","last_clean":"2024-09-21T03:39:21.864066+0000","last_became_active":"2024-09-21T03:37:12.865468+0000","last_became_peered":"2024-09-21T03:37:12.865468+0000","last_unstale":"2024-09-21T03:39:21.864066+0000","last_undegraded":"2024-09-21T03:39:21.864066+0000","last_fullsized":"2024-09-21T03:39:21.864066+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:18:14.496495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"84'16","reported_seq":50,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:39:21.856994+0000","last_change":"2024-09-21T03:37:12.862688+0000","last_active":"2024-09-21T03:39:21.856994+0000","last_peered":"2024-09-21T03:39:21.856994+0000","last_clean":"2024-09-21T03:39:21.856994+0000","last_became_active":"2024-09-21T03:37:12.862249+0000","last_became_peered":"2024-09-21T03:37:12.862249+0000","last_unstale":"2024-09-21T03:39:21.856994+0000","last_undegraded":"2024-09-21T03:39:21.856994+0000","last_fullsized":"2024-09-21T03:39:21.856994+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:55:54.731640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.37","version":"84'11","reported_seq":45,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:39:21.864822+0000","last_change":"2024-09-21T03:37:12.863595+0000","last_active":"2024-09-21T03:39:21.864822+0000","last_peered":"2024-09-21T03:39:21.864822+0000","last_clean":"2024-09-21T03:39:21.864822+0000","last_became_active":"2024-09-21T03:37:12.863260+0000","last_became_peered":"2024-09-21T03:37:12.863260+0000","last_unstale":"2024-09-21T03:39:21.864822+0000","last_undegraded":"2024-09-21T03:39:21.864822+0000","last_fullsized":"2024-09-21T03:39:21.864822+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:10:07.995690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"84'9","reported_seq":43,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:39:21.847848+0000","last_change":"2024-09-21T03:37:12.865770+0000","last_active":"2024-09-21T03:39:21.847848+0000","last_peered":"2024-09-21T03:39:21.847848+0000","last_clean":"2024-09-21T03:39:21.847848+0000","last_became_active":"2024-09-21T03:37:12.865405+0000","last_became_peered":"2024-09-21T03:37:12.865405+0000","last_unstale":"2024-09-21T03:39:21.847848+0000","last_undegraded":"2024-09-21T03:39:21.847848+0000","last_fullsized":"2024-09-21T03:39:21.847848+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:46:25.255913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"84'5","reported_seq":39,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:39:21.851267+0000","last_change":"2024-09-21T03:37:12.856214+0000","last_active":"2024-09-21T03:39:21.851267+0000","last_peered":"2024-09-21T03:39:21.851267+0000","last_clean":"2024-09-21T03:39:21.851267+0000","last_became_active":"2024-09-21T03:37:12.853924+0000","last_became_peered":"2024-09-21T03:37:12.853924+0000","last_unstale":"2024-09-21T03:39:21.851267+0000","last_undegraded":"2024-09-21T03:39:21.851267+0000","last_fullsized":"2024-09-21T03:39:21.851267+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:10:32.182892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"84'6","reported_seq":40,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:39:21.856643+0000","last_change":"2024-09-21T03:37:12.858851+0000","last_active":"2024-09-21T03:39:21.856643+0000","last_peered":"2024-09-21T03:39:21.856643+0000","last_clean":"2024-09-21T03:39:21.856643+0000","last_became_active":"2024-09-21T03:37:12.858037+0000","last_became_peered":"2024-09-21T03:37:12.858037+0000","last_unstale":"2024-09-21T03:39:21.856643+0000","last_undegraded":"2024-09-21T03:39:21.856643+0000","last_fullsized":"2024-09-21T03:39:21.856643+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:32:23.134789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.3b","version":"84'4","reported_seq":38,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:39:21.837281+0000","last_change":"2024-09-21T03:37:12.852339+0000","last_active":"2024-09-21T03:39:21.837281+0000","last_peered":"2024-09-21T03:39:21.837281+0000","last_clean":"2024-09-21T03:39:21.837281+0000","last_became_active":"2024-09-21T03:37:12.851911+0000","last_became_peered":"2024-09-21T03:37:12.851911+0000","last_unstale":"2024-09-21T03:39:21.837281+0000","last_undegraded":"2024-09-21T03:39:21.837281+0000","last_fullsized":"2024-09-21T03:39:21.837281+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:03:30.739275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3c","version":"84'8","reported_seq":43,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:39:21.851835+0000","last_change":"2024-09-21T03:37:12.861133+0000","last_active":"2024-09-21T03:39:21.851835+0000","last_peered":"2024-09-21T03:39:21.851835+0000","last_clean":"2024-09-21T03:39:21.851835+0000","last_became_active":"2024-09-21T03:37:12.860719+0000","last_became_peered":"2024-09-21T03:37:12.860719+0000","last_unstale":"2024-09-21T03:39:21.851835+0000","last_undegraded":"2024-09-21T03:39:21.851835+0000","last_fullsized":"2024-09-21T03:39:21.851835+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:21:25.241935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3d","version":"84'7","reported_seq":42,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:39:21.858925+0000","last_change":"2024-09-21T03:37:12.865694+0000","last_active":"2024-09-21T03:39:21.858925+0000","last_peered":"2024-09-21T03:39:21.858925+0000","last_clean":"2024-09-21T03:39:21.858925+0000","last_became_active":"2024-09-21T03:37:12.864983+0000","last_became_peered":"2024-09-21T03:37:12.864983+0000","last_unstale":"2024-09-21T03:39:21.858925+0000","last_undegraded":"2024-09-21T03:39:21.858925+0000","last_fullsized":"2024-09-21T03:39:21.858925+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:52:28.583056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"84'4","reported_seq":39,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:39:21.850375+0000","last_change":"2024-09-21T03:37:12.860665+0000","last_active":"2024-09-21T03:39:21.850375+0000","last_peered":"2024-09-21T03:39:21.850375+0000","last_clean":"2024-09-21T03:39:21.850375+0000","last_became_active":"2024-09-21T03:37:12.860316+0000","last_became_peered":"2024-09-21T03:37:12.860316+0000","last_unstale":"2024-09-21T03:39:21.850375+0000","last_undegraded":"2024-09-21T03:39:21.850375+0000","last_fullsized":"2024-09-21T03:39:21.850375+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:13:07.316422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3f","version":"84'8","reported_seq":43,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:39:21.864109+0000","last_change":"2024-09-21T03:37:12.869577+0000","last_active":"2024-09-21T03:39:21.864109+0000","last_peered":"2024-09-21T03:39:21.864109+0000","last_clean":"2024-09-21T03:39:21.864109+0000","last_became_active":"2024-09-21T03:37:12.869164+0000","last_became_peered":"2024-09-21T03:37:12.869164+0000","last_unstale":"2024-09-21T03:39:21.864109+0000","last_undegraded":"2024-09-21T03:39:21.864109+0000","last_fullsized":"2024-09-21T03:39:21.864109+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:40:34.167005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":72906,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":606,"num_read_kb":379,"num_write":179,"num_write_kb":130,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":694,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":446464,"data_stored":223560,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":153,"ondisk_log_size":153,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":285501,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":68,"seq":292057776168,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32092,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713060,"statfs":{"total":95995035648,"available":95962173440,"internally_reserved":0,"allocated":929792,"data_stored":480064,"data_compressed":29416,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":10,"up_from":63,"seq":270582939693,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36392,"kb_used_data":1048,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708760,"statfs":{"total":95995035648,"available":95957770240,"internally_reserved":0,"allocated":1073152,"data_stored":1135593,"data_compressed":65710,"data_compressed_allocated":86016,"data_compressed_original":671744,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":9,"up_from":58,"seq":249108103218,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27944,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717208,"statfs":{"total":95995035648,"available":95966420992,"internally_reserved":0,"allocated":942080,"data_stored":484284,"data_compressed":29416,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":52,"seq":223338299447,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27932,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717220,"statfs":{"total":95995035648,"available":95966433280,"internally_reserved":0,"allocated":929792,"data_stored":479996,"data_compressed":29416,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":46,"seq":197568495676,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36308,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708844,"statfs":{"total":95995035648,"available":95957856256,"internally_reserved":0,"allocated":987136,"data_stored":1077144,"data_compressed":65710,"data_compressed_allocated":86016,"data_compressed_original":671744,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":41,"seq":176093659201,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32080,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713072,"statfs":{"total":95995035648,"available":95962185728,"internally_reserved":0,"allocated":917504,"data_stored":475364,"data_compressed":29416,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":35,"seq":150323855430,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32156,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712996,"statfs":{"total":95995035648,"available":95962107904,"internally_reserved":0,"allocated":995328,"data_stored":1072239,"data_compressed":65710,"data_compressed_allocated":86016,"data_compressed_original":671744,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":30,"seq":128849018954,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27936,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717216,"statfs":{"total":95995035648,"available":95966429184,"internally_reserved":0,"allocated":933888,"data_stored":480596,"data_compressed":29416,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":24,"seq":103079215183,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27932,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717220,"statfs":{"total":95995035648,"available":95966433280,"internally_reserved":0,"allocated":929792,"data_stored":475558,"data_compressed":29416,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":17,"seq":73014444118,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28016,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717136,"statfs":{"total":95995035648,"available":95966347264,"internally_reserved":0,"allocated":1015808,"data_stored":550078,"data_compressed":29416,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574937,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36312,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708840,"statfs":{"total":95995035648,"available":95957852160,"internally_reserved":0,"allocated":991232,"data_stored":542661,"data_compressed":29416,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738463,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36264,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708888,"statfs":{"total":95995035648,"available":95957901312,"internally_reserved":0,"allocated":942080,"data_stored":476752,"data_compressed":29416,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34677,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34677,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34677,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36294,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36294,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36294,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36294,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36294,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":67387,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":74804,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1226,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2892,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":3310,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":626,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":69951,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T03:39:29.875 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-09-21T03:39:29.876 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 7c374098-77c9-11ef-baf5-efdc52797490 -- ceph pg dump --format=json 2024-09-21T03:39:30.125 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/7c374098-77c9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:39:30.222 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:39:30 smithi089 ceph-mon[30928]: from='client.15453 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T03:39:30.461 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:39:30 smithi114 ceph-mon[34140]: from='client.15453 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T03:39:30.500 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:39:30 smithi110 ceph-mon[34000]: from='client.15453 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T03:39:30.550 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:30.528+0000 7fc856e99640 1 -- 172.21.15.89:0/1019538897 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc85010de10 msgr2=0x7fc8501102f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:30.550 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:30.528+0000 7fc856e99640 1 --2- 172.21.15.89:0/1019538897 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc85010de10 0x7fc8501102f0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fc848009ca0 tx=0x7fc84802a8c0 comp rx=0 tx=0).stop 2024-09-21T03:39:30.550 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:30.528+0000 7fc856e99640 1 -- 172.21.15.89:0/1019538897 shutdown_connections 2024-09-21T03:39:30.550 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:30.528+0000 7fc856e99640 1 --2- 172.21.15.89:0/1019538897 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc850105290 0x7fc85010d8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:30.550 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:30.528+0000 7fc856e99640 1 --2- 172.21.15.89:0/1019538897 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc85010de10 0x7fc8501102f0 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:30.550 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:30.528+0000 7fc856e99640 1 --2- 172.21.15.89:0/1019538897 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc850104860 0x7fc850104c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:30.550 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:30.528+0000 7fc856e99640 1 -- 172.21.15.89:0/1019538897 >> 172.21.15.89:0/1019538897 conn(0x7fc850079730 msgr2=0x7fc850079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:39:30.550 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:30.528+0000 7fc856e99640 1 -- 172.21.15.89:0/1019538897 shutdown_connections 2024-09-21T03:39:30.550 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:30.528+0000 7fc856e99640 1 -- 172.21.15.89:0/1019538897 wait complete. 2024-09-21T03:39:30.551 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:30.529+0000 7fc856e99640 1 Processor -- start 2024-09-21T03:39:30.551 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:30.529+0000 7fc856e99640 1 -- start start 2024-09-21T03:39:30.551 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:30.529+0000 7fc856e99640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc850104860 0x7fc85019c550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:30.551 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:30.529+0000 7fc856e99640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc850105290 0x7fc85019caf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:30.551 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:30.529+0000 7fc856e99640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc85010de10 0x7fc8501a1d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:30.551 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:30.529+0000 7fc856e99640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fc850113980 con 0x7fc85010de10 2024-09-21T03:39:30.551 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:30.529+0000 7fc856e99640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fc850113800 con 0x7fc850104860 2024-09-21T03:39:30.551 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:30.529+0000 7fc856e99640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fc850113b00 con 0x7fc850105290 2024-09-21T03:39:30.551 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:30.529+0000 7fc855436640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc85010de10 0x7fc8501a1d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:30.551 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:30.529+0000 7fc855436640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc85010de10 0x7fc8501a1d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:35732/0 (socket says 172.21.15.89:35732) 2024-09-21T03:39:30.551 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:30.529+0000 7fc855436640 1 -- 172.21.15.89:0/385643582 learned_addr learned my addr 172.21.15.89:0/385643582 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-21T03:39:30.551 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:30.529+0000 7fc854c35640 1 --2- 172.21.15.89:0/385643582 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc850104860 0x7fc85019c550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:30.551 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:30.529+0000 7fc847fff640 1 --2- 172.21.15.89:0/385643582 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc850105290 0x7fc85019caf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:30.552 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:30.530+0000 7fc855436640 1 -- 172.21.15.89:0/385643582 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc850105290 msgr2=0x7fc85019caf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:30.552 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:30.530+0000 7fc855436640 1 --2- 172.21.15.89:0/385643582 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc850105290 0x7fc85019caf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:30.552 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:30.530+0000 7fc855436640 1 -- 172.21.15.89:0/385643582 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc850104860 msgr2=0x7fc85019c550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:30.552 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:30.530+0000 7fc855436640 1 --2- 172.21.15.89:0/385643582 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc850104860 0x7fc85019c550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:30.552 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:30.530+0000 7fc855436640 1 -- 172.21.15.89:0/385643582 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8501a24c0 con 0x7fc85010de10 2024-09-21T03:39:30.552 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:30.530+0000 7fc855436640 1 --2- 172.21.15.89:0/385643582 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc85010de10 0x7fc8501a1d90 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fc848005e30 tx=0x7fc8480093d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:39:30.552 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:30.530+0000 7fc845ffb640 1 -- 172.21.15.89:0/385643582 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc848032d10 con 0x7fc85010de10 2024-09-21T03:39:30.552 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:30.530+0000 7fc845ffb640 1 -- 172.21.15.89:0/385643582 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc848032eb0 con 0x7fc85010de10 2024-09-21T03:39:30.552 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:30.530+0000 7fc856e99640 1 -- 172.21.15.89:0/385643582 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc8501a2750 con 0x7fc85010de10 2024-09-21T03:39:30.552 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:30.530+0000 7fc845ffb640 1 -- 172.21.15.89:0/385643582 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc848030b20 con 0x7fc85010de10 2024-09-21T03:39:30.552 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:30.530+0000 7fc856e99640 1 -- 172.21.15.89:0/385643582 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc8501a8f10 con 0x7fc85010de10 2024-09-21T03:39:30.552 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:30.531+0000 7fc856e99640 1 -- 172.21.15.89:0/385643582 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc850104c70 con 0x7fc85010de10 2024-09-21T03:39:30.552 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:30.533+0000 7fc845ffb640 1 -- 172.21.15.89:0/385643582 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 20) ==== 154563+0+0 (secure 0 0 0) 0x7fc848030cc0 con 0x7fc85010de10 2024-09-21T03:39:30.552 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:30.534+0000 7fc845ffb640 1 --2- 172.21.15.89:0/385643582 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7fc8380b6620 0x7fc8380b8b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:30.552 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:30.534+0000 7fc854c35640 1 --2- 172.21.15.89:0/385643582 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7fc8380b6620 0x7fc8380b8b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:30.552 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:30.534+0000 7fc845ffb640 1 -- 172.21.15.89:0/385643582 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 9121+0+0 (secure 0 0 0) 0x7fc848108a70 con 0x7fc85010de10 2024-09-21T03:39:30.553 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:30.534+0000 7fc854c35640 1 --2- 172.21.15.89:0/385643582 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7fc8380b6620 0x7fc8380b8b30 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fc84c009bd0 tx=0x7fc84c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:39:30.553 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:30.537+0000 7fc845ffb640 1 -- 172.21.15.89:0/385643582 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fc8480cee20 con 0x7fc85010de10 2024-09-21T03:39:30.713 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:30.711+0000 7fc856e99640 1 -- 172.21.15.89:0/385643582 --> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fc85010a3a0 con 0x7fc8380b6620 2024-09-21T03:39:30.719 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:30.717+0000 7fc845ffb640 1 -- 172.21.15.89:0/385643582 <== mgr.14152 v2:172.21.15.89:6800/1591532358 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340748 (secure 0 0 0) 0x7fc85010a3a0 con 0x7fc8380b6620 2024-09-21T03:39:30.722 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-09-21T03:39:30.724 INFO:teuthology.orchestra.run.smithi089.stderr:dumped all 2024-09-21T03:39:30.726 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:30.720+0000 7fc856e99640 1 -- 172.21.15.89:0/385643582 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7fc8380b6620 msgr2=0x7fc8380b8b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:30.726 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:30.721+0000 7fc856e99640 1 --2- 172.21.15.89:0/385643582 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7fc8380b6620 0x7fc8380b8b30 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fc84c009bd0 tx=0x7fc84c005f90 comp rx=0 tx=0).stop 2024-09-21T03:39:30.726 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:30.721+0000 7fc856e99640 1 -- 172.21.15.89:0/385643582 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc85010de10 msgr2=0x7fc8501a1d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:30.726 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:30.721+0000 7fc856e99640 1 --2- 172.21.15.89:0/385643582 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc85010de10 0x7fc8501a1d90 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fc848005e30 tx=0x7fc8480093d0 comp rx=0 tx=0).stop 2024-09-21T03:39:30.726 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:30.721+0000 7fc856e99640 1 -- 172.21.15.89:0/385643582 shutdown_connections 2024-09-21T03:39:30.726 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:30.721+0000 7fc856e99640 1 --2- 172.21.15.89:0/385643582 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7fc8380b6620 0x7fc8380b8b30 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:30.726 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:30.721+0000 7fc856e99640 1 --2- 172.21.15.89:0/385643582 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc85010de10 0x7fc8501a1d90 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:30.726 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:30.721+0000 7fc856e99640 1 --2- 172.21.15.89:0/385643582 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc850105290 0x7fc85019caf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:30.727 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:30.721+0000 7fc856e99640 1 --2- 172.21.15.89:0/385643582 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc850104860 0x7fc85019c550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:30.727 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:30.721+0000 7fc856e99640 1 -- 172.21.15.89:0/385643582 >> 172.21.15.89:0/385643582 conn(0x7fc850079730 msgr2=0x7fc850109c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:39:30.727 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:30.721+0000 7fc856e99640 1 -- 172.21.15.89:0/385643582 shutdown_connections 2024-09-21T03:39:30.727 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:30.721+0000 7fc856e99640 1 -- 172.21.15.89:0/385643582 wait complete. 2024-09-21T03:39:31.356 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:39:31 smithi089 ceph-mon[30928]: pgmap v363: 129 pgs: 129 active+clean; 656 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 1.9 KiB/s wr, 9 op/s 2024-09-21T03:39:31.383 INFO:teuthology.orchestra.run.smithi089.stdout:{"pg_ready":true,"pg_map":{"version":363,"stamp":"2024-09-21T03:39:29.665893+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":671652,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":711,"num_read_kb":468,"num_write":312,"num_write_kb":2294,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":23,"num_bytes_recovered":2952534,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":753,"ondisk_log_size":753,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":381376,"kb_used_data":11328,"kb_used_omap":18,"kb_used_meta":369837,"kb_avail":1124560448,"statfs":{"total":1151940427776,"available":1151549898752,"internally_reserved":0,"allocated":11599872,"data_stored":7744879,"data_compressed":461874,"data_compressed_allocated":626688,"data_compressed_original":2752512,"omap_allocated":19143,"internal_metadata":378713401},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":9911,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":93,"num_read_kb":59,"num_write":23,"num_write_kb":23,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003613"},"pg_stats":[{"pgid":"2.1f","version":"80'9","reported_seq":50,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:38:53.672984+0000","last_change":"2024-09-21T03:37:10.836953+0000","last_active":"2024-09-21T03:38:53.672984+0000","last_peered":"2024-09-21T03:38:53.672984+0000","last_clean":"2024-09-21T03:38:53.672984+0000","last_became_active":"2024-09-21T03:37:10.836566+0000","last_became_peered":"2024-09-21T03:37:10.836566+0000","last_unstale":"2024-09-21T03:38:53.672984+0000","last_undegraded":"2024-09-21T03:38:53.672984+0000","last_fullsized":"2024-09-21T03:38:53.672984+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_clean_scrub_stamp":"2024-09-21T03:37:09.809805+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:15:45.591225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"84'5","reported_seq":40,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:39:21.839616+0000","last_change":"2024-09-21T03:37:12.866931+0000","last_active":"2024-09-21T03:39:21.839616+0000","last_peered":"2024-09-21T03:39:21.839616+0000","last_clean":"2024-09-21T03:39:21.839616+0000","last_became_active":"2024-09-21T03:37:12.866452+0000","last_became_peered":"2024-09-21T03:37:12.866452+0000","last_unstale":"2024-09-21T03:39:21.839616+0000","last_undegraded":"2024-09-21T03:39:21.839616+0000","last_fullsized":"2024-09-21T03:39:21.839616+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:37:14.324548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:38:53.669589+0000","last_change":"2024-09-21T03:37:19.912154+0000","last_active":"2024-09-21T03:38:53.669589+0000","last_peered":"2024-09-21T03:38:53.669589+0000","last_clean":"2024-09-21T03:38:53.669589+0000","last_became_active":"2024-09-21T03:37:19.911686+0000","last_became_peered":"2024-09-21T03:37:19.911686+0000","last_unstale":"2024-09-21T03:38:53.669589+0000","last_undegraded":"2024-09-21T03:38:53.669589+0000","last_fullsized":"2024-09-21T03:38:53.669589+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_clean_scrub_stamp":"2024-09-21T03:37:18.876933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:29:20.189418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"80'2","reported_seq":42,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:38:53.972157+0000","last_change":"2024-09-21T03:37:11.034501+0000","last_active":"2024-09-21T03:38:53.972157+0000","last_peered":"2024-09-21T03:38:53.972157+0000","last_clean":"2024-09-21T03:38:53.972157+0000","last_became_active":"2024-09-21T03:37:11.034110+0000","last_became_peered":"2024-09-21T03:37:11.034110+0000","last_unstale":"2024-09-21T03:38:53.972157+0000","last_undegraded":"2024-09-21T03:38:53.972157+0000","last_fullsized":"2024-09-21T03:38:53.972157+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_clean_scrub_stamp":"2024-09-21T03:37:09.809805+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:03:12.854469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"84'6","reported_seq":41,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:39:21.858901+0000","last_change":"2024-09-21T03:37:12.867330+0000","last_active":"2024-09-21T03:39:21.858901+0000","last_peered":"2024-09-21T03:39:21.858901+0000","last_clean":"2024-09-21T03:39:21.858901+0000","last_became_active":"2024-09-21T03:37:12.866952+0000","last_became_peered":"2024-09-21T03:37:12.866952+0000","last_unstale":"2024-09-21T03:39:21.858901+0000","last_undegraded":"2024-09-21T03:39:21.858901+0000","last_fullsized":"2024-09-21T03:39:21.858901+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:40:33.826546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:38:54.095716+0000","last_change":"2024-09-21T03:37:19.910661+0000","last_active":"2024-09-21T03:38:54.095716+0000","last_peered":"2024-09-21T03:38:54.095716+0000","last_clean":"2024-09-21T03:38:54.095716+0000","last_became_active":"2024-09-21T03:37:19.910108+0000","last_became_peered":"2024-09-21T03:37:19.910108+0000","last_unstale":"2024-09-21T03:38:54.095716+0000","last_undegraded":"2024-09-21T03:38:54.095716+0000","last_fullsized":"2024-09-21T03:38:54.095716+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_clean_scrub_stamp":"2024-09-21T03:37:18.876933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:18:05.654426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"80'4","reported_seq":43,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:38:54.442879+0000","last_change":"2024-09-21T03:37:10.835336+0000","last_active":"2024-09-21T03:38:54.442879+0000","last_peered":"2024-09-21T03:38:54.442879+0000","last_clean":"2024-09-21T03:38:54.442879+0000","last_became_active":"2024-09-21T03:37:10.834928+0000","last_became_peered":"2024-09-21T03:37:10.834928+0000","last_unstale":"2024-09-21T03:38:54.442879+0000","last_undegraded":"2024-09-21T03:38:54.442879+0000","last_fullsized":"2024-09-21T03:38:54.442879+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_clean_scrub_stamp":"2024-09-21T03:37:09.809805+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:02:14.717520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"84'9","reported_seq":43,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:39:21.865718+0000","last_change":"2024-09-21T03:37:12.861271+0000","last_active":"2024-09-21T03:39:21.865718+0000","last_peered":"2024-09-21T03:39:21.865718+0000","last_clean":"2024-09-21T03:39:21.865718+0000","last_became_active":"2024-09-21T03:37:12.860932+0000","last_became_peered":"2024-09-21T03:37:12.860932+0000","last_unstale":"2024-09-21T03:39:21.865718+0000","last_undegraded":"2024-09-21T03:39:21.865718+0000","last_fullsized":"2024-09-21T03:39:21.865718+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:46:08.993250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":34,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:38:54.096300+0000","last_change":"2024-09-21T03:37:53.053045+0000","last_active":"2024-09-21T03:38:54.096300+0000","last_peered":"2024-09-21T03:38:54.096300+0000","last_clean":"2024-09-21T03:38:54.096300+0000","last_became_active":"2024-09-21T03:37:53.052382+0000","last_became_peered":"2024-09-21T03:37:53.052382+0000","last_unstale":"2024-09-21T03:38:54.096300+0000","last_undegraded":"2024-09-21T03:38:54.096300+0000","last_fullsized":"2024-09-21T03:38:54.096300+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_clean_scrub_stamp":"2024-09-21T03:37:18.876933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:02:44.300768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,8],"acting":[4,3,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"74'2","reported_seq":41,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:38:54.442781+0000","last_change":"2024-09-21T03:37:10.833985+0000","last_active":"2024-09-21T03:38:54.442781+0000","last_peered":"2024-09-21T03:38:54.442781+0000","last_clean":"2024-09-21T03:38:54.442781+0000","last_became_active":"2024-09-21T03:37:10.833562+0000","last_became_peered":"2024-09-21T03:37:10.833562+0000","last_unstale":"2024-09-21T03:38:54.442781+0000","last_undegraded":"2024-09-21T03:38:54.442781+0000","last_fullsized":"2024-09-21T03:38:54.442781+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_clean_scrub_stamp":"2024-09-21T03:37:09.809805+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:49:32.617768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"84'15","reported_seq":49,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:39:21.852488+0000","last_change":"2024-09-21T03:37:12.865408+0000","last_active":"2024-09-21T03:39:21.852488+0000","last_peered":"2024-09-21T03:39:21.852488+0000","last_clean":"2024-09-21T03:39:21.852488+0000","last_became_active":"2024-09-21T03:37:12.864972+0000","last_became_peered":"2024-09-21T03:37:12.864972+0000","last_unstale":"2024-09-21T03:39:21.852488+0000","last_undegraded":"2024-09-21T03:39:21.852488+0000","last_fullsized":"2024-09-21T03:39:21.852488+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:03:16.384260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:38:54.095856+0000","last_change":"2024-09-21T03:37:19.912105+0000","last_active":"2024-09-21T03:38:54.095856+0000","last_peered":"2024-09-21T03:38:54.095856+0000","last_clean":"2024-09-21T03:38:54.095856+0000","last_became_active":"2024-09-21T03:37:19.911416+0000","last_became_peered":"2024-09-21T03:37:19.911416+0000","last_unstale":"2024-09-21T03:38:54.095856+0000","last_undegraded":"2024-09-21T03:38:54.095856+0000","last_fullsized":"2024-09-21T03:38:54.095856+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_clean_scrub_stamp":"2024-09-21T03:37:18.876933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:20:16.492231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"84'5","reported_seq":40,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:39:21.865669+0000","last_change":"2024-09-21T03:37:12.862354+0000","last_active":"2024-09-21T03:39:21.865669+0000","last_peered":"2024-09-21T03:39:21.865669+0000","last_clean":"2024-09-21T03:39:21.865669+0000","last_became_active":"2024-09-21T03:37:12.861977+0000","last_became_peered":"2024-09-21T03:37:12.861977+0000","last_unstale":"2024-09-21T03:39:21.865669+0000","last_undegraded":"2024-09-21T03:39:21.865669+0000","last_fullsized":"2024-09-21T03:39:21.865669+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:53:52.725524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"80'4","reported_seq":45,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:38:54.442515+0000","last_change":"2024-09-21T03:37:10.832140+0000","last_active":"2024-09-21T03:38:54.442515+0000","last_peered":"2024-09-21T03:38:54.442515+0000","last_clean":"2024-09-21T03:38:54.442515+0000","last_became_active":"2024-09-21T03:37:10.831395+0000","last_became_peered":"2024-09-21T03:37:10.831395+0000","last_unstale":"2024-09-21T03:38:54.442515+0000","last_undegraded":"2024-09-21T03:38:54.442515+0000","last_fullsized":"2024-09-21T03:38:54.442515+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_clean_scrub_stamp":"2024-09-21T03:37:09.809805+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:55:20.440890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:38:54.096219+0000","last_change":"2024-09-21T03:37:20.160850+0000","last_active":"2024-09-21T03:38:54.096219+0000","last_peered":"2024-09-21T03:38:54.096219+0000","last_clean":"2024-09-21T03:38:54.096219+0000","last_became_active":"2024-09-21T03:37:20.160305+0000","last_became_peered":"2024-09-21T03:37:20.160305+0000","last_unstale":"2024-09-21T03:38:54.096219+0000","last_undegraded":"2024-09-21T03:38:54.096219+0000","last_fullsized":"2024-09-21T03:38:54.096219+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_clean_scrub_stamp":"2024-09-21T03:37:18.876933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:51:45.971984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"84'8","reported_seq":43,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:39:21.848436+0000","last_change":"2024-09-21T03:37:12.859485+0000","last_active":"2024-09-21T03:39:21.848436+0000","last_peered":"2024-09-21T03:39:21.848436+0000","last_clean":"2024-09-21T03:39:21.848436+0000","last_became_active":"2024-09-21T03:37:12.859097+0000","last_became_peered":"2024-09-21T03:37:12.859097+0000","last_unstale":"2024-09-21T03:39:21.848436+0000","last_undegraded":"2024-09-21T03:39:21.848436+0000","last_fullsized":"2024-09-21T03:39:21.848436+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:16:05.438360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"84'2","reported_seq":42,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:39:22.087392+0000","last_change":"2024-09-21T03:37:10.832441+0000","last_active":"2024-09-21T03:39:22.087392+0000","last_peered":"2024-09-21T03:39:22.087392+0000","last_clean":"2024-09-21T03:39:22.087392+0000","last_became_active":"2024-09-21T03:37:10.832073+0000","last_became_peered":"2024-09-21T03:37:10.832073+0000","last_unstale":"2024-09-21T03:39:22.087392+0000","last_undegraded":"2024-09-21T03:39:22.087392+0000","last_fullsized":"2024-09-21T03:39:22.087392+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_clean_scrub_stamp":"2024-09-21T03:37:09.809805+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:32:19.411097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"82'1","reported_seq":27,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:38:54.096331+0000","last_change":"2024-09-21T03:37:19.905490+0000","last_active":"2024-09-21T03:38:54.096331+0000","last_peered":"2024-09-21T03:38:54.096331+0000","last_clean":"2024-09-21T03:38:54.096331+0000","last_became_active":"2024-09-21T03:37:19.904971+0000","last_became_peered":"2024-09-21T03:37:19.904971+0000","last_unstale":"2024-09-21T03:38:54.096331+0000","last_undegraded":"2024-09-21T03:38:54.096331+0000","last_fullsized":"2024-09-21T03:38:54.096331+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_clean_scrub_stamp":"2024-09-21T03:37:18.876933+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:44:20.655538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"84'9","reported_seq":44,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:39:21.853193+0000","last_change":"2024-09-21T03:37:12.863597+0000","last_active":"2024-09-21T03:39:21.853193+0000","last_peered":"2024-09-21T03:39:21.853193+0000","last_clean":"2024-09-21T03:39:21.853193+0000","last_became_active":"2024-09-21T03:37:12.863223+0000","last_became_peered":"2024-09-21T03:37:12.863223+0000","last_unstale":"2024-09-21T03:39:21.853193+0000","last_undegraded":"2024-09-21T03:39:21.853193+0000","last_fullsized":"2024-09-21T03:39:21.853193+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:29:28.878964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.9","version":"80'3","reported_seq":43,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:38:53.971972+0000","last_change":"2024-09-21T03:37:10.834648+0000","last_active":"2024-09-21T03:38:53.971972+0000","last_peered":"2024-09-21T03:38:53.971972+0000","last_clean":"2024-09-21T03:38:53.971972+0000","last_became_active":"2024-09-21T03:37:10.834293+0000","last_became_peered":"2024-09-21T03:37:10.834293+0000","last_unstale":"2024-09-21T03:38:53.971972+0000","last_undegraded":"2024-09-21T03:38:53.971972+0000","last_fullsized":"2024-09-21T03:38:53.971972+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_clean_scrub_stamp":"2024-09-21T03:37:09.809805+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:10:41.248733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:38:53.672715+0000","last_change":"2024-09-21T03:37:19.912526+0000","last_active":"2024-09-21T03:38:53.672715+0000","last_peered":"2024-09-21T03:38:53.672715+0000","last_clean":"2024-09-21T03:38:53.672715+0000","last_became_active":"2024-09-21T03:37:19.912062+0000","last_became_peered":"2024-09-21T03:37:19.912062+0000","last_unstale":"2024-09-21T03:38:53.672715+0000","last_undegraded":"2024-09-21T03:38:53.672715+0000","last_fullsized":"2024-09-21T03:38:53.672715+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_clean_scrub_stamp":"2024-09-21T03:37:18.876933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:52:40.395834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.9","version":"84'2","reported_seq":37,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:39:21.852191+0000","last_change":"2024-09-21T03:37:12.855672+0000","last_active":"2024-09-21T03:39:21.852191+0000","last_peered":"2024-09-21T03:39:21.852191+0000","last_clean":"2024-09-21T03:39:21.852191+0000","last_became_active":"2024-09-21T03:37:12.855191+0000","last_became_peered":"2024-09-21T03:37:12.855191+0000","last_unstale":"2024-09-21T03:39:21.852191+0000","last_undegraded":"2024-09-21T03:39:21.852191+0000","last_fullsized":"2024-09-21T03:39:21.852191+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:26:56.115697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.8","version":"80'2","reported_seq":42,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:38:54.096441+0000","last_change":"2024-09-21T03:37:11.034752+0000","last_active":"2024-09-21T03:38:54.096441+0000","last_peered":"2024-09-21T03:38:54.096441+0000","last_clean":"2024-09-21T03:38:54.096441+0000","last_became_active":"2024-09-21T03:37:11.034386+0000","last_became_peered":"2024-09-21T03:37:11.034386+0000","last_unstale":"2024-09-21T03:38:54.096441+0000","last_undegraded":"2024-09-21T03:38:54.096441+0000","last_fullsized":"2024-09-21T03:38:54.096441+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_clean_scrub_stamp":"2024-09-21T03:37:09.809805+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:17:23.647411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:38:54.096423+0000","last_change":"2024-09-21T03:37:20.162051+0000","last_active":"2024-09-21T03:38:54.096423+0000","last_peered":"2024-09-21T03:38:54.096423+0000","last_clean":"2024-09-21T03:38:54.096423+0000","last_became_active":"2024-09-21T03:37:20.161452+0000","last_became_peered":"2024-09-21T03:37:20.161452+0000","last_unstale":"2024-09-21T03:38:54.096423+0000","last_undegraded":"2024-09-21T03:38:54.096423+0000","last_fullsized":"2024-09-21T03:38:54.096423+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_clean_scrub_stamp":"2024-09-21T03:37:18.876933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:29:22.544800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"84'7","reported_seq":42,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:39:21.851621+0000","last_change":"2024-09-21T03:37:12.866058+0000","last_active":"2024-09-21T03:39:21.851621+0000","last_peered":"2024-09-21T03:39:21.851621+0000","last_clean":"2024-09-21T03:39:21.851621+0000","last_became_active":"2024-09-21T03:37:12.865694+0000","last_became_peered":"2024-09-21T03:37:12.865694+0000","last_unstale":"2024-09-21T03:39:21.851621+0000","last_undegraded":"2024-09-21T03:39:21.851621+0000","last_fullsized":"2024-09-21T03:39:21.851621+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:56:33.629873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"80'4","reported_seq":44,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:38:54.096939+0000","last_change":"2024-09-21T03:37:11.358619+0000","last_active":"2024-09-21T03:38:54.096939+0000","last_peered":"2024-09-21T03:38:54.096939+0000","last_clean":"2024-09-21T03:38:54.096939+0000","last_became_active":"2024-09-21T03:37:11.358196+0000","last_became_peered":"2024-09-21T03:37:11.358196+0000","last_unstale":"2024-09-21T03:38:54.096939+0000","last_undegraded":"2024-09-21T03:38:54.096939+0000","last_fullsized":"2024-09-21T03:38:54.096939+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_clean_scrub_stamp":"2024-09-21T03:37:09.809805+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:57:13.476641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":27,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:38:53.669752+0000","last_change":"2024-09-21T03:37:19.905711+0000","last_active":"2024-09-21T03:38:53.669752+0000","last_peered":"2024-09-21T03:38:53.669752+0000","last_clean":"2024-09-21T03:38:53.669752+0000","last_became_active":"2024-09-21T03:37:19.905260+0000","last_became_peered":"2024-09-21T03:37:19.905260+0000","last_unstale":"2024-09-21T03:38:53.669752+0000","last_undegraded":"2024-09-21T03:38:53.669752+0000","last_fullsized":"2024-09-21T03:38:53.669752+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_clean_scrub_stamp":"2024-09-21T03:37:18.876933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:21:09.476082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"84'5","reported_seq":40,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:39:21.852928+0000","last_change":"2024-09-21T03:37:12.854376+0000","last_active":"2024-09-21T03:39:21.852928+0000","last_peered":"2024-09-21T03:39:21.852928+0000","last_clean":"2024-09-21T03:39:21.852928+0000","last_became_active":"2024-09-21T03:37:12.852942+0000","last_became_peered":"2024-09-21T03:37:12.852942+0000","last_unstale":"2024-09-21T03:39:21.852928+0000","last_undegraded":"2024-09-21T03:39:21.852928+0000","last_fullsized":"2024-09-21T03:39:21.852928+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:06:49.319002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"84'13","reported_seq":66,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:39:25.878844+0000","last_change":"2024-09-21T03:37:53.056889+0000","last_active":"2024-09-21T03:39:25.878844+0000","last_peered":"2024-09-21T03:39:25.878844+0000","last_clean":"2024-09-21T03:39:25.878844+0000","last_became_active":"2024-09-21T03:37:53.053054+0000","last_became_peered":"2024-09-21T03:37:53.053054+0000","last_unstale":"2024-09-21T03:39:25.878844+0000","last_undegraded":"2024-09-21T03:39:25.878844+0000","last_fullsized":"2024-09-21T03:39:25.878844+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_clean_scrub_stamp":"2024-09-21T03:37:09.809805+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:16:05.539111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,3],"acting":[9,11,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":27,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:38:53.971576+0000","last_change":"2024-09-21T03:37:20.161515+0000","last_active":"2024-09-21T03:38:53.971576+0000","last_peered":"2024-09-21T03:38:53.971576+0000","last_clean":"2024-09-21T03:38:53.971576+0000","last_became_active":"2024-09-21T03:37:20.160548+0000","last_became_peered":"2024-09-21T03:37:20.160548+0000","last_unstale":"2024-09-21T03:38:53.971576+0000","last_undegraded":"2024-09-21T03:38:53.971576+0000","last_fullsized":"2024-09-21T03:38:53.971576+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_clean_scrub_stamp":"2024-09-21T03:37:18.876933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:17:51.144859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"84'9","reported_seq":44,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:39:21.832796+0000","last_change":"2024-09-21T03:37:12.848305+0000","last_active":"2024-09-21T03:39:21.832796+0000","last_peered":"2024-09-21T03:39:21.832796+0000","last_clean":"2024-09-21T03:39:21.832796+0000","last_became_active":"2024-09-21T03:37:12.847916+0000","last_became_peered":"2024-09-21T03:37:12.847916+0000","last_unstale":"2024-09-21T03:39:21.832796+0000","last_undegraded":"2024-09-21T03:39:21.832796+0000","last_fullsized":"2024-09-21T03:39:21.832796+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:48:39.922805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"80'1","reported_seq":42,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:38:54.442955+0000","last_change":"2024-09-21T03:37:11.035964+0000","last_active":"2024-09-21T03:38:54.442955+0000","last_peered":"2024-09-21T03:38:54.442955+0000","last_clean":"2024-09-21T03:38:54.442955+0000","last_became_active":"2024-09-21T03:37:11.035427+0000","last_became_peered":"2024-09-21T03:37:11.035427+0000","last_unstale":"2024-09-21T03:38:54.442955+0000","last_undegraded":"2024-09-21T03:38:54.442955+0000","last_fullsized":"2024-09-21T03:38:54.442955+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_clean_scrub_stamp":"2024-09-21T03:37:09.809805+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:04:51.026753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":27,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:38:54.096205+0000","last_change":"2024-09-21T03:37:19.909428+0000","last_active":"2024-09-21T03:38:54.096205+0000","last_peered":"2024-09-21T03:38:54.096205+0000","last_clean":"2024-09-21T03:38:54.096205+0000","last_became_active":"2024-09-21T03:37:19.908904+0000","last_became_peered":"2024-09-21T03:37:19.908904+0000","last_unstale":"2024-09-21T03:38:54.096205+0000","last_undegraded":"2024-09-21T03:38:54.096205+0000","last_fullsized":"2024-09-21T03:38:54.096205+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_clean_scrub_stamp":"2024-09-21T03:37:18.876933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:12:07.564058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.5","version":"84'4","reported_seq":39,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:39:21.852816+0000","last_change":"2024-09-21T03:37:12.855592+0000","last_active":"2024-09-21T03:39:21.852816+0000","last_peered":"2024-09-21T03:39:21.852816+0000","last_clean":"2024-09-21T03:39:21.852816+0000","last_became_active":"2024-09-21T03:37:12.855244+0000","last_became_peered":"2024-09-21T03:37:12.855244+0000","last_unstale":"2024-09-21T03:39:21.852816+0000","last_undegraded":"2024-09-21T03:39:21.852816+0000","last_fullsized":"2024-09-21T03:39:21.852816+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:21:34.603141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":40,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:38:53.672861+0000","last_change":"2024-09-21T03:37:11.036401+0000","last_active":"2024-09-21T03:38:53.672861+0000","last_peered":"2024-09-21T03:38:53.672861+0000","last_clean":"2024-09-21T03:38:53.672861+0000","last_became_active":"2024-09-21T03:37:11.036003+0000","last_became_peered":"2024-09-21T03:37:11.036003+0000","last_unstale":"2024-09-21T03:38:53.672861+0000","last_undegraded":"2024-09-21T03:38:53.672861+0000","last_fullsized":"2024-09-21T03:38:53.672861+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_clean_scrub_stamp":"2024-09-21T03:37:09.809805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:35:50.549060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":27,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:38:54.442319+0000","last_change":"2024-09-21T03:37:20.160668+0000","last_active":"2024-09-21T03:38:54.442319+0000","last_peered":"2024-09-21T03:38:54.442319+0000","last_clean":"2024-09-21T03:38:54.442319+0000","last_became_active":"2024-09-21T03:37:20.160158+0000","last_became_peered":"2024-09-21T03:37:20.160158+0000","last_unstale":"2024-09-21T03:38:54.442319+0000","last_undegraded":"2024-09-21T03:38:54.442319+0000","last_fullsized":"2024-09-21T03:38:54.442319+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_clean_scrub_stamp":"2024-09-21T03:37:18.876933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:45:25.677667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3","version":"84'7","reported_seq":49,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:39:21.847550+0000","last_change":"2024-09-21T03:37:53.050351+0000","last_active":"2024-09-21T03:39:21.847550+0000","last_peered":"2024-09-21T03:39:21.847550+0000","last_clean":"2024-09-21T03:39:21.847550+0000","last_became_active":"2024-09-21T03:37:53.050003+0000","last_became_peered":"2024-09-21T03:37:53.050003+0000","last_unstale":"2024-09-21T03:39:21.847550+0000","last_undegraded":"2024-09-21T03:39:21.847550+0000","last_fullsized":"2024-09-21T03:39:21.847550+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:49:03.313002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,0],"acting":[10,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"80'1","reported_seq":41,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:38:53.942258+0000","last_change":"2024-09-21T03:37:10.832635+0000","last_active":"2024-09-21T03:38:53.942258+0000","last_peered":"2024-09-21T03:38:53.942258+0000","last_clean":"2024-09-21T03:38:53.942258+0000","last_became_active":"2024-09-21T03:37:10.832280+0000","last_became_peered":"2024-09-21T03:37:10.832280+0000","last_unstale":"2024-09-21T03:38:53.942258+0000","last_undegraded":"2024-09-21T03:38:53.942258+0000","last_fullsized":"2024-09-21T03:38:53.942258+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_clean_scrub_stamp":"2024-09-21T03:37:09.809805+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:54:52.744486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":27,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:38:53.668675+0000","last_change":"2024-09-21T03:37:19.904112+0000","last_active":"2024-09-21T03:38:53.668675+0000","last_peered":"2024-09-21T03:38:53.668675+0000","last_clean":"2024-09-21T03:38:53.668675+0000","last_became_active":"2024-09-21T03:37:19.903677+0000","last_became_peered":"2024-09-21T03:37:19.903677+0000","last_unstale":"2024-09-21T03:38:53.668675+0000","last_undegraded":"2024-09-21T03:38:53.668675+0000","last_fullsized":"2024-09-21T03:38:53.668675+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_clean_scrub_stamp":"2024-09-21T03:37:18.876933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:32:20.096850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"84'8","reported_seq":43,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:39:21.847619+0000","last_change":"2024-09-21T03:37:12.864082+0000","last_active":"2024-09-21T03:39:21.847619+0000","last_peered":"2024-09-21T03:39:21.847619+0000","last_clean":"2024-09-21T03:39:21.847619+0000","last_became_active":"2024-09-21T03:37:12.862745+0000","last_became_peered":"2024-09-21T03:37:12.862745+0000","last_unstale":"2024-09-21T03:39:21.847619+0000","last_undegraded":"2024-09-21T03:39:21.847619+0000","last_fullsized":"2024-09-21T03:39:21.847619+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:25:35.759719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"84'5","reported_seq":146,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:39:28.190435+0000","last_change":"2024-09-21T03:37:11.034463+0000","last_active":"2024-09-21T03:39:28.190435+0000","last_peered":"2024-09-21T03:39:28.190435+0000","last_clean":"2024-09-21T03:39:28.190435+0000","last_became_active":"2024-09-21T03:37:11.033932+0000","last_became_peered":"2024-09-21T03:37:11.033932+0000","last_unstale":"2024-09-21T03:39:28.190435+0000","last_undegraded":"2024-09-21T03:39:28.190435+0000","last_fullsized":"2024-09-21T03:39:28.190435+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_clean_scrub_stamp":"2024-09-21T03:37:09.809805+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:15:18.595442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":101,"num_read_kb":101,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":27,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:38:53.668764+0000","last_change":"2024-09-21T03:37:20.160789+0000","last_active":"2024-09-21T03:38:53.668764+0000","last_peered":"2024-09-21T03:38:53.668764+0000","last_clean":"2024-09-21T03:38:53.668764+0000","last_became_active":"2024-09-21T03:37:20.160231+0000","last_became_peered":"2024-09-21T03:37:20.160231+0000","last_unstale":"2024-09-21T03:38:53.668764+0000","last_undegraded":"2024-09-21T03:38:53.668764+0000","last_fullsized":"2024-09-21T03:38:53.668764+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_clean_scrub_stamp":"2024-09-21T03:37:18.876933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:07:16.762249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"84'5","reported_seq":40,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:39:21.852307+0000","last_change":"2024-09-21T03:37:12.853178+0000","last_active":"2024-09-21T03:39:21.852307+0000","last_peered":"2024-09-21T03:39:21.852307+0000","last_clean":"2024-09-21T03:39:21.852307+0000","last_became_active":"2024-09-21T03:37:12.852822+0000","last_became_peered":"2024-09-21T03:37:12.852822+0000","last_unstale":"2024-09-21T03:39:21.852307+0000","last_undegraded":"2024-09-21T03:39:21.852307+0000","last_fullsized":"2024-09-21T03:39:21.852307+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:48:26.146593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"80'3","reported_seq":43,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:38:53.941666+0000","last_change":"2024-09-21T03:37:10.838939+0000","last_active":"2024-09-21T03:38:53.941666+0000","last_peered":"2024-09-21T03:38:53.941666+0000","last_clean":"2024-09-21T03:38:53.941666+0000","last_became_active":"2024-09-21T03:37:10.838562+0000","last_became_peered":"2024-09-21T03:37:10.838562+0000","last_unstale":"2024-09-21T03:38:53.941666+0000","last_undegraded":"2024-09-21T03:38:53.941666+0000","last_fullsized":"2024-09-21T03:38:53.941666+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_clean_scrub_stamp":"2024-09-21T03:37:09.809805+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:35:32.936824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":34,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:38:54.097042+0000","last_change":"2024-09-21T03:37:53.052940+0000","last_active":"2024-09-21T03:38:54.097042+0000","last_peered":"2024-09-21T03:38:54.097042+0000","last_clean":"2024-09-21T03:38:54.097042+0000","last_became_active":"2024-09-21T03:37:53.052310+0000","last_became_peered":"2024-09-21T03:37:53.052310+0000","last_unstale":"2024-09-21T03:38:54.097042+0000","last_undegraded":"2024-09-21T03:38:54.097042+0000","last_fullsized":"2024-09-21T03:38:54.097042+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_clean_scrub_stamp":"2024-09-21T03:37:18.876933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:31:35.353226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,11,6,10],"acting":[5,11,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"84'8","reported_seq":43,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:39:21.847226+0000","last_change":"2024-09-21T03:37:12.869795+0000","last_active":"2024-09-21T03:39:21.847226+0000","last_peered":"2024-09-21T03:39:21.847226+0000","last_clean":"2024-09-21T03:39:21.847226+0000","last_became_active":"2024-09-21T03:37:12.869381+0000","last_became_peered":"2024-09-21T03:37:12.869381+0000","last_unstale":"2024-09-21T03:39:21.847226+0000","last_undegraded":"2024-09-21T03:39:21.847226+0000","last_fullsized":"2024-09-21T03:39:21.847226+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:39:29.437552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"80'3","reported_seq":45,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:38:54.096584+0000","last_change":"2024-09-21T03:37:10.834853+0000","last_active":"2024-09-21T03:38:54.096584+0000","last_peered":"2024-09-21T03:38:54.096584+0000","last_clean":"2024-09-21T03:38:54.096584+0000","last_became_active":"2024-09-21T03:37:10.834402+0000","last_became_peered":"2024-09-21T03:37:10.834402+0000","last_unstale":"2024-09-21T03:38:54.096584+0000","last_undegraded":"2024-09-21T03:38:54.096584+0000","last_fullsized":"2024-09-21T03:38:54.096584+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_clean_scrub_stamp":"2024-09-21T03:37:09.809805+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:57:29.668570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"20'92","reported_seq":113,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:38:53.941746+0000","last_change":"2024-09-21T03:36:00.521781+0000","last_active":"2024-09-21T03:38:53.941746+0000","last_peered":"2024-09-21T03:38:53.941746+0000","last_clean":"2024-09-21T03:38:53.941746+0000","last_became_active":"2024-09-21T03:36:00.516468+0000","last_became_peered":"2024-09-21T03:36:00.516468+0000","last_unstale":"2024-09-21T03:38:53.941746+0000","last_undegraded":"2024-09-21T03:38:53.941746+0000","last_fullsized":"2024-09-21T03:38:53.941746+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:32:38.215432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:32:38.215432+0000","last_clean_scrub_stamp":"2024-09-21T03:32:38.215432+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:02:16.252544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"82'1","reported_seq":28,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:38:53.941918+0000","last_change":"2024-09-21T03:37:19.911309+0000","last_active":"2024-09-21T03:38:53.941918+0000","last_peered":"2024-09-21T03:38:53.941918+0000","last_clean":"2024-09-21T03:38:53.941918+0000","last_became_active":"2024-09-21T03:37:19.910778+0000","last_became_peered":"2024-09-21T03:37:19.910778+0000","last_unstale":"2024-09-21T03:38:53.941918+0000","last_undegraded":"2024-09-21T03:38:53.941918+0000","last_fullsized":"2024-09-21T03:38:53.941918+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_clean_scrub_stamp":"2024-09-21T03:37:18.876933+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:52:23.307983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"84'7","reported_seq":42,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:39:21.865435+0000","last_change":"2024-09-21T03:37:12.860707+0000","last_active":"2024-09-21T03:39:21.865435+0000","last_peered":"2024-09-21T03:39:21.865435+0000","last_clean":"2024-09-21T03:39:21.865435+0000","last_became_active":"2024-09-21T03:37:12.860341+0000","last_became_peered":"2024-09-21T03:37:12.860341+0000","last_unstale":"2024-09-21T03:39:21.865435+0000","last_undegraded":"2024-09-21T03:39:21.865435+0000","last_fullsized":"2024-09-21T03:39:21.865435+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:55:15.771259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"80'3","reported_seq":22,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:38:54.095929+0000","last_change":"2024-09-21T03:37:53.071125+0000","last_active":"2024-09-21T03:38:54.095929+0000","last_peered":"2024-09-21T03:38:54.095929+0000","last_clean":"2024-09-21T03:38:54.095929+0000","last_became_active":"2024-09-21T03:37:53.052237+0000","last_became_peered":"2024-09-21T03:37:53.052237+0000","last_unstale":"2024-09-21T03:38:54.095929+0000","last_undegraded":"2024-09-21T03:38:54.095929+0000","last_fullsized":"2024-09-21T03:38:54.095929+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_clean_scrub_stamp":"2024-09-21T03:37:09.809805+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:47:17.707045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,0],"acting":[4,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:38:54.096435+0000","last_change":"2024-09-21T03:37:19.909292+0000","last_active":"2024-09-21T03:38:54.096435+0000","last_peered":"2024-09-21T03:38:54.096435+0000","last_clean":"2024-09-21T03:38:54.096435+0000","last_became_active":"2024-09-21T03:37:19.908681+0000","last_became_peered":"2024-09-21T03:37:19.908681+0000","last_unstale":"2024-09-21T03:38:54.096435+0000","last_undegraded":"2024-09-21T03:38:54.096435+0000","last_fullsized":"2024-09-21T03:38:54.096435+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_clean_scrub_stamp":"2024-09-21T03:37:18.876933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:15:49.623109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"84'6","reported_seq":41,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:39:21.852674+0000","last_change":"2024-09-21T03:37:12.857678+0000","last_active":"2024-09-21T03:39:21.852674+0000","last_peered":"2024-09-21T03:39:21.852674+0000","last_clean":"2024-09-21T03:39:21.852674+0000","last_became_active":"2024-09-21T03:37:12.857266+0000","last_became_peered":"2024-09-21T03:37:12.857266+0000","last_unstale":"2024-09-21T03:39:21.852674+0000","last_undegraded":"2024-09-21T03:39:21.852674+0000","last_fullsized":"2024-09-21T03:39:21.852674+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:21:14.135784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"74'1","reported_seq":42,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:38:54.097310+0000","last_change":"2024-09-21T03:37:10.835323+0000","last_active":"2024-09-21T03:38:54.097310+0000","last_peered":"2024-09-21T03:38:54.097310+0000","last_clean":"2024-09-21T03:38:54.097310+0000","last_became_active":"2024-09-21T03:37:10.834950+0000","last_became_peered":"2024-09-21T03:37:10.834950+0000","last_unstale":"2024-09-21T03:38:54.097310+0000","last_undegraded":"2024-09-21T03:38:54.097310+0000","last_fullsized":"2024-09-21T03:38:54.097310+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_clean_scrub_stamp":"2024-09-21T03:37:09.809805+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:40:27.386342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:38:54.095581+0000","last_change":"2024-09-21T03:37:19.906360+0000","last_active":"2024-09-21T03:38:54.095581+0000","last_peered":"2024-09-21T03:38:54.095581+0000","last_clean":"2024-09-21T03:38:54.095581+0000","last_became_active":"2024-09-21T03:37:19.905888+0000","last_became_peered":"2024-09-21T03:37:19.905888+0000","last_unstale":"2024-09-21T03:38:54.095581+0000","last_undegraded":"2024-09-21T03:38:54.095581+0000","last_fullsized":"2024-09-21T03:38:54.095581+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_clean_scrub_stamp":"2024-09-21T03:37:18.876933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:13:55.885538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"84'9","reported_seq":44,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:39:21.854021+0000","last_change":"2024-09-21T03:37:12.849106+0000","last_active":"2024-09-21T03:39:21.854021+0000","last_peered":"2024-09-21T03:39:21.854021+0000","last_clean":"2024-09-21T03:39:21.854021+0000","last_became_active":"2024-09-21T03:37:12.848505+0000","last_became_peered":"2024-09-21T03:37:12.848505+0000","last_unstale":"2024-09-21T03:39:21.854021+0000","last_undegraded":"2024-09-21T03:39:21.854021+0000","last_fullsized":"2024-09-21T03:39:21.854021+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:50:19.569004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"74'2","reported_seq":53,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:38:53.669030+0000","last_change":"2024-09-21T03:37:53.115158+0000","last_active":"2024-09-21T03:38:53.669030+0000","last_peered":"2024-09-21T03:38:53.669030+0000","last_clean":"2024-09-21T03:38:53.669030+0000","last_became_active":"2024-09-21T03:37:53.112603+0000","last_became_peered":"2024-09-21T03:37:53.112603+0000","last_unstale":"2024-09-21T03:38:53.669030+0000","last_undegraded":"2024-09-21T03:38:53.669030+0000","last_fullsized":"2024-09-21T03:38:53.669030+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_clean_scrub_stamp":"2024-09-21T03:37:09.809805+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:23:21.525972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,1],"acting":[6,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:38:54.095667+0000","last_change":"2024-09-21T03:37:19.909040+0000","last_active":"2024-09-21T03:38:54.095667+0000","last_peered":"2024-09-21T03:38:54.095667+0000","last_clean":"2024-09-21T03:38:54.095667+0000","last_became_active":"2024-09-21T03:37:19.908530+0000","last_became_peered":"2024-09-21T03:37:19.908530+0000","last_unstale":"2024-09-21T03:38:54.095667+0000","last_undegraded":"2024-09-21T03:38:54.095667+0000","last_fullsized":"2024-09-21T03:38:54.095667+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_clean_scrub_stamp":"2024-09-21T03:37:18.876933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:35:20.919260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"84'5","reported_seq":40,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:39:21.851946+0000","last_change":"2024-09-21T03:37:12.858984+0000","last_active":"2024-09-21T03:39:21.851946+0000","last_peered":"2024-09-21T03:39:21.851946+0000","last_clean":"2024-09-21T03:39:21.851946+0000","last_became_active":"2024-09-21T03:37:12.858603+0000","last_became_peered":"2024-09-21T03:37:12.858603+0000","last_unstale":"2024-09-21T03:39:21.851946+0000","last_undegraded":"2024-09-21T03:39:21.851946+0000","last_fullsized":"2024-09-21T03:39:21.851946+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:16:45.025304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.f","version":"84'3","reported_seq":44,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:39:27.032126+0000","last_change":"2024-09-21T03:37:10.834844+0000","last_active":"2024-09-21T03:39:27.032126+0000","last_peered":"2024-09-21T03:39:27.032126+0000","last_clean":"2024-09-21T03:39:27.032126+0000","last_became_active":"2024-09-21T03:37:10.833664+0000","last_became_peered":"2024-09-21T03:37:10.833664+0000","last_unstale":"2024-09-21T03:39:27.032126+0000","last_undegraded":"2024-09-21T03:39:27.032126+0000","last_fullsized":"2024-09-21T03:39:27.032126+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_clean_scrub_stamp":"2024-09-21T03:37:09.809805+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:03:49.006414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:38:54.095855+0000","last_change":"2024-09-21T03:37:19.913277+0000","last_active":"2024-09-21T03:38:54.095855+0000","last_peered":"2024-09-21T03:38:54.095855+0000","last_clean":"2024-09-21T03:38:54.095855+0000","last_became_active":"2024-09-21T03:37:19.912781+0000","last_became_peered":"2024-09-21T03:37:19.912781+0000","last_unstale":"2024-09-21T03:38:54.095855+0000","last_undegraded":"2024-09-21T03:38:54.095855+0000","last_fullsized":"2024-09-21T03:38:54.095855+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_clean_scrub_stamp":"2024-09-21T03:37:18.876933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:18:27.980622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"84'6","reported_seq":40,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:39:21.847384+0000","last_change":"2024-09-21T03:37:12.862322+0000","last_active":"2024-09-21T03:39:21.847384+0000","last_peered":"2024-09-21T03:39:21.847384+0000","last_clean":"2024-09-21T03:39:21.847384+0000","last_became_active":"2024-09-21T03:37:12.861938+0000","last_became_peered":"2024-09-21T03:37:12.861938+0000","last_unstale":"2024-09-21T03:39:21.847384+0000","last_undegraded":"2024-09-21T03:39:21.847384+0000","last_fullsized":"2024-09-21T03:39:21.847384+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:01:48.944169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.10","version":"80'15","reported_seq":34,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:38:54.096078+0000","last_change":"2024-09-21T03:37:53.067130+0000","last_active":"2024-09-21T03:38:54.096078+0000","last_peered":"2024-09-21T03:38:54.096078+0000","last_clean":"2024-09-21T03:38:54.096078+0000","last_became_active":"2024-09-21T03:37:53.051596+0000","last_became_peered":"2024-09-21T03:37:53.051596+0000","last_unstale":"2024-09-21T03:38:54.096078+0000","last_undegraded":"2024-09-21T03:38:54.096078+0000","last_fullsized":"2024-09-21T03:38:54.096078+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_clean_scrub_stamp":"2024-09-21T03:37:09.809805+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:02:22.852015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:38:53.668923+0000","last_change":"2024-09-21T03:37:19.913206+0000","last_active":"2024-09-21T03:38:53.668923+0000","last_peered":"2024-09-21T03:38:53.668923+0000","last_clean":"2024-09-21T03:38:53.668923+0000","last_became_active":"2024-09-21T03:37:19.910718+0000","last_became_peered":"2024-09-21T03:37:19.910718+0000","last_unstale":"2024-09-21T03:38:53.668923+0000","last_undegraded":"2024-09-21T03:38:53.668923+0000","last_fullsized":"2024-09-21T03:38:53.668923+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_clean_scrub_stamp":"2024-09-21T03:37:18.876933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:11:31.431237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"84'6","reported_seq":40,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:39:21.854210+0000","last_change":"2024-09-21T03:37:12.857505+0000","last_active":"2024-09-21T03:39:21.854210+0000","last_peered":"2024-09-21T03:39:21.854210+0000","last_clean":"2024-09-21T03:39:21.854210+0000","last_became_active":"2024-09-21T03:37:12.857155+0000","last_became_peered":"2024-09-21T03:37:12.857155+0000","last_unstale":"2024-09-21T03:39:21.854210+0000","last_undegraded":"2024-09-21T03:39:21.854210+0000","last_fullsized":"2024-09-21T03:39:21.854210+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:05:50.387651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"84'5","reported_seq":147,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:39:27.189129+0000","last_change":"2024-09-21T03:37:10.832364+0000","last_active":"2024-09-21T03:39:27.189129+0000","last_peered":"2024-09-21T03:39:27.189129+0000","last_clean":"2024-09-21T03:39:27.189129+0000","last_became_active":"2024-09-21T03:37:10.832004+0000","last_became_peered":"2024-09-21T03:37:10.832004+0000","last_unstale":"2024-09-21T03:39:27.189129+0000","last_undegraded":"2024-09-21T03:39:27.189129+0000","last_fullsized":"2024-09-21T03:39:27.189129+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_clean_scrub_stamp":"2024-09-21T03:37:09.809805+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:21:10.895732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1472,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":104,"num_read_kb":5,"num_write":4,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:38:53.672765+0000","last_change":"2024-09-21T03:37:19.912297+0000","last_active":"2024-09-21T03:38:53.672765+0000","last_peered":"2024-09-21T03:38:53.672765+0000","last_clean":"2024-09-21T03:38:53.672765+0000","last_became_active":"2024-09-21T03:37:19.911903+0000","last_became_peered":"2024-09-21T03:37:19.911903+0000","last_unstale":"2024-09-21T03:38:53.672765+0000","last_undegraded":"2024-09-21T03:38:53.672765+0000","last_fullsized":"2024-09-21T03:38:53.672765+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_clean_scrub_stamp":"2024-09-21T03:37:18.876933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:05:56.747246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.13","version":"84'15","reported_seq":49,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:39:21.853966+0000","last_change":"2024-09-21T03:37:12.856230+0000","last_active":"2024-09-21T03:39:21.853966+0000","last_peered":"2024-09-21T03:39:21.853966+0000","last_clean":"2024-09-21T03:39:21.853966+0000","last_became_active":"2024-09-21T03:37:12.855819+0000","last_became_peered":"2024-09-21T03:37:12.855819+0000","last_unstale":"2024-09-21T03:39:21.853966+0000","last_undegraded":"2024-09-21T03:39:21.853966+0000","last_fullsized":"2024-09-21T03:39:21.853966+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:38:18.618579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"80'3","reported_seq":42,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:38:54.096706+0000","last_change":"2024-09-21T03:37:10.832401+0000","last_active":"2024-09-21T03:38:54.096706+0000","last_peered":"2024-09-21T03:38:54.096706+0000","last_clean":"2024-09-21T03:38:54.096706+0000","last_became_active":"2024-09-21T03:37:10.832033+0000","last_became_peered":"2024-09-21T03:37:10.832033+0000","last_unstale":"2024-09-21T03:38:54.096706+0000","last_undegraded":"2024-09-21T03:38:54.096706+0000","last_fullsized":"2024-09-21T03:38:54.096706+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_clean_scrub_stamp":"2024-09-21T03:37:09.809805+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:25:20.751529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:38:54.096414+0000","last_change":"2024-09-21T03:37:19.905825+0000","last_active":"2024-09-21T03:38:54.096414+0000","last_peered":"2024-09-21T03:38:54.096414+0000","last_clean":"2024-09-21T03:38:54.096414+0000","last_became_active":"2024-09-21T03:37:19.905352+0000","last_became_peered":"2024-09-21T03:37:19.905352+0000","last_unstale":"2024-09-21T03:38:54.096414+0000","last_undegraded":"2024-09-21T03:38:54.096414+0000","last_fullsized":"2024-09-21T03:38:54.096414+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_clean_scrub_stamp":"2024-09-21T03:37:18.876933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:25:05.861981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"84'9","reported_seq":43,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:39:21.847167+0000","last_change":"2024-09-21T03:37:12.852006+0000","last_active":"2024-09-21T03:39:21.847167+0000","last_peered":"2024-09-21T03:39:21.847167+0000","last_clean":"2024-09-21T03:39:21.847167+0000","last_became_active":"2024-09-21T03:37:12.851670+0000","last_became_peered":"2024-09-21T03:37:12.851670+0000","last_unstale":"2024-09-21T03:39:21.847167+0000","last_undegraded":"2024-09-21T03:39:21.847167+0000","last_fullsized":"2024-09-21T03:39:21.847167+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:41:50.177349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"80'6","reported_seq":46,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:38:53.670064+0000","last_change":"2024-09-21T03:37:10.832472+0000","last_active":"2024-09-21T03:38:53.670064+0000","last_peered":"2024-09-21T03:38:53.670064+0000","last_clean":"2024-09-21T03:38:53.670064+0000","last_became_active":"2024-09-21T03:37:10.832085+0000","last_became_peered":"2024-09-21T03:37:10.832085+0000","last_unstale":"2024-09-21T03:38:53.670064+0000","last_undegraded":"2024-09-21T03:38:53.670064+0000","last_fullsized":"2024-09-21T03:38:53.670064+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_clean_scrub_stamp":"2024-09-21T03:37:09.809805+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:06:41.251218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:38:54.096945+0000","last_change":"2024-09-21T03:37:19.914042+0000","last_active":"2024-09-21T03:38:54.096945+0000","last_peered":"2024-09-21T03:38:54.096945+0000","last_clean":"2024-09-21T03:38:54.096945+0000","last_became_active":"2024-09-21T03:37:19.913531+0000","last_became_peered":"2024-09-21T03:37:19.913531+0000","last_unstale":"2024-09-21T03:38:54.096945+0000","last_undegraded":"2024-09-21T03:38:54.096945+0000","last_fullsized":"2024-09-21T03:38:54.096945+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_clean_scrub_stamp":"2024-09-21T03:37:18.876933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:16:40.163359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"84'6","reported_seq":40,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:39:21.852857+0000","last_change":"2024-09-21T03:37:12.852893+0000","last_active":"2024-09-21T03:39:21.852857+0000","last_peered":"2024-09-21T03:39:21.852857+0000","last_clean":"2024-09-21T03:39:21.852857+0000","last_became_active":"2024-09-21T03:37:12.852452+0000","last_became_peered":"2024-09-21T03:37:12.852452+0000","last_unstale":"2024-09-21T03:39:21.852857+0000","last_undegraded":"2024-09-21T03:39:21.852857+0000","last_fullsized":"2024-09-21T03:39:21.852857+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:46:59.393901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.14","version":"84'38","reported_seq":292,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:39:29.194612+0000","last_change":"2024-09-21T03:37:10.837654+0000","last_active":"2024-09-21T03:39:29.194612+0000","last_peered":"2024-09-21T03:39:29.194612+0000","last_clean":"2024-09-21T03:39:29.194612+0000","last_became_active":"2024-09-21T03:37:10.837251+0000","last_became_peered":"2024-09-21T03:37:10.837251+0000","last_unstale":"2024-09-21T03:39:29.194612+0000","last_undegraded":"2024-09-21T03:39:29.194612+0000","last_fullsized":"2024-09-21T03:39:29.194612+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_clean_scrub_stamp":"2024-09-21T03:37:09.809805+0000","objects_scrubbed":0,"log_size":38,"log_dups_size":0,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:43:32.986371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":75140,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":215,"num_read_kb":181,"num_write":39,"num_write_kb":92,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:38:53.669528+0000","last_change":"2024-09-21T03:37:53.053405+0000","last_active":"2024-09-21T03:38:53.669528+0000","last_peered":"2024-09-21T03:38:53.669528+0000","last_clean":"2024-09-21T03:38:53.669528+0000","last_became_active":"2024-09-21T03:37:53.052818+0000","last_became_peered":"2024-09-21T03:37:53.052818+0000","last_unstale":"2024-09-21T03:38:53.669528+0000","last_undegraded":"2024-09-21T03:38:53.669528+0000","last_fullsized":"2024-09-21T03:38:53.669528+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_clean_scrub_stamp":"2024-09-21T03:37:18.876933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:05:09.121036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,11,5],"acting":[10,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.14","version":"84'11","reported_seq":45,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:39:21.857069+0000","last_change":"2024-09-21T03:37:12.866588+0000","last_active":"2024-09-21T03:39:21.857069+0000","last_peered":"2024-09-21T03:39:21.857069+0000","last_clean":"2024-09-21T03:39:21.857069+0000","last_became_active":"2024-09-21T03:37:12.866075+0000","last_became_peered":"2024-09-21T03:37:12.866075+0000","last_unstale":"2024-09-21T03:39:21.857069+0000","last_undegraded":"2024-09-21T03:39:21.857069+0000","last_fullsized":"2024-09-21T03:39:21.857069+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:18:48.817562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"80'4","reported_seq":44,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:38:54.097208+0000","last_change":"2024-09-21T03:37:11.036751+0000","last_active":"2024-09-21T03:38:54.097208+0000","last_peered":"2024-09-21T03:38:54.097208+0000","last_clean":"2024-09-21T03:38:54.097208+0000","last_became_active":"2024-09-21T03:37:11.036125+0000","last_became_peered":"2024-09-21T03:37:11.036125+0000","last_unstale":"2024-09-21T03:38:54.097208+0000","last_undegraded":"2024-09-21T03:38:54.097208+0000","last_fullsized":"2024-09-21T03:38:54.097208+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_clean_scrub_stamp":"2024-09-21T03:37:09.809805+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:46:13.569422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":34,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:38:54.095956+0000","last_change":"2024-09-21T03:37:53.112845+0000","last_active":"2024-09-21T03:38:54.095956+0000","last_peered":"2024-09-21T03:38:54.095956+0000","last_clean":"2024-09-21T03:38:54.095956+0000","last_became_active":"2024-09-21T03:37:53.112286+0000","last_became_peered":"2024-09-21T03:37:53.112286+0000","last_unstale":"2024-09-21T03:38:54.095956+0000","last_undegraded":"2024-09-21T03:38:54.095956+0000","last_fullsized":"2024-09-21T03:38:54.095956+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_clean_scrub_stamp":"2024-09-21T03:37:18.876933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:29:58.062684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,8],"acting":[4,1,2,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"84'9","reported_seq":43,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:39:21.851704+0000","last_change":"2024-09-21T03:37:12.865011+0000","last_active":"2024-09-21T03:39:21.851704+0000","last_peered":"2024-09-21T03:39:21.851704+0000","last_clean":"2024-09-21T03:39:21.851704+0000","last_became_active":"2024-09-21T03:37:12.864645+0000","last_became_peered":"2024-09-21T03:37:12.864645+0000","last_unstale":"2024-09-21T03:39:21.851704+0000","last_undegraded":"2024-09-21T03:39:21.851704+0000","last_fullsized":"2024-09-21T03:39:21.851704+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:27:42.471227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"80'2","reported_seq":41,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:38:53.941930+0000","last_change":"2024-09-21T03:37:10.835171+0000","last_active":"2024-09-21T03:38:53.941930+0000","last_peered":"2024-09-21T03:38:53.941930+0000","last_clean":"2024-09-21T03:38:53.941930+0000","last_became_active":"2024-09-21T03:37:10.834744+0000","last_became_peered":"2024-09-21T03:37:10.834744+0000","last_unstale":"2024-09-21T03:38:53.941930+0000","last_undegraded":"2024-09-21T03:38:53.941930+0000","last_fullsized":"2024-09-21T03:38:53.941930+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_clean_scrub_stamp":"2024-09-21T03:37:09.809805+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:02:37.720457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:38:53.672239+0000","last_change":"2024-09-21T03:37:19.906734+0000","last_active":"2024-09-21T03:38:53.672239+0000","last_peered":"2024-09-21T03:38:53.672239+0000","last_clean":"2024-09-21T03:38:53.672239+0000","last_became_active":"2024-09-21T03:37:19.906327+0000","last_became_peered":"2024-09-21T03:37:19.906327+0000","last_unstale":"2024-09-21T03:38:53.672239+0000","last_undegraded":"2024-09-21T03:38:53.672239+0000","last_fullsized":"2024-09-21T03:38:53.672239+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_clean_scrub_stamp":"2024-09-21T03:37:18.876933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:55:11.603042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.16","version":"84'10","reported_seq":44,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:39:21.866214+0000","last_change":"2024-09-21T03:37:12.866285+0000","last_active":"2024-09-21T03:39:21.866214+0000","last_peered":"2024-09-21T03:39:21.866214+0000","last_clean":"2024-09-21T03:39:21.866214+0000","last_became_active":"2024-09-21T03:37:12.865854+0000","last_became_peered":"2024-09-21T03:37:12.865854+0000","last_unstale":"2024-09-21T03:39:21.866214+0000","last_undegraded":"2024-09-21T03:39:21.866214+0000","last_fullsized":"2024-09-21T03:39:21.866214+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:00:18.075248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"84'4","reported_seq":144,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:39:28.192639+0000","last_change":"2024-09-21T03:37:10.835808+0000","last_active":"2024-09-21T03:39:28.192639+0000","last_peered":"2024-09-21T03:39:28.192639+0000","last_clean":"2024-09-21T03:39:28.192639+0000","last_became_active":"2024-09-21T03:37:10.835434+0000","last_became_peered":"2024-09-21T03:37:10.835434+0000","last_unstale":"2024-09-21T03:39:28.192639+0000","last_undegraded":"2024-09-21T03:39:28.192639+0000","last_fullsized":"2024-09-21T03:39:28.192639+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_clean_scrub_stamp":"2024-09-21T03:37:09.809805+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:35:39.817312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":101,"num_read_kb":101,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:38:53.971629+0000","last_change":"2024-09-21T03:37:19.915986+0000","last_active":"2024-09-21T03:38:53.971629+0000","last_peered":"2024-09-21T03:38:53.971629+0000","last_clean":"2024-09-21T03:38:53.971629+0000","last_became_active":"2024-09-21T03:37:19.915561+0000","last_became_peered":"2024-09-21T03:37:19.915561+0000","last_unstale":"2024-09-21T03:38:53.971629+0000","last_undegraded":"2024-09-21T03:38:53.971629+0000","last_fullsized":"2024-09-21T03:38:53.971629+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_clean_scrub_stamp":"2024-09-21T03:37:18.876933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:21:13.789664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":27,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:38:54.095727+0000","last_change":"2024-09-21T03:37:19.908230+0000","last_active":"2024-09-21T03:38:54.095727+0000","last_peered":"2024-09-21T03:38:54.095727+0000","last_clean":"2024-09-21T03:38:54.095727+0000","last_became_active":"2024-09-21T03:37:19.907843+0000","last_became_peered":"2024-09-21T03:37:19.907843+0000","last_unstale":"2024-09-21T03:38:54.095727+0000","last_undegraded":"2024-09-21T03:38:54.095727+0000","last_fullsized":"2024-09-21T03:38:54.095727+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_clean_scrub_stamp":"2024-09-21T03:37:18.876933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:15:58.142127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"84'11","reported_seq":45,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:39:21.851083+0000","last_change":"2024-09-21T03:37:12.856709+0000","last_active":"2024-09-21T03:39:21.851083+0000","last_peered":"2024-09-21T03:39:21.851083+0000","last_clean":"2024-09-21T03:39:21.851083+0000","last_became_active":"2024-09-21T03:37:12.856278+0000","last_became_peered":"2024-09-21T03:37:12.856278+0000","last_unstale":"2024-09-21T03:39:21.851083+0000","last_undegraded":"2024-09-21T03:39:21.851083+0000","last_fullsized":"2024-09-21T03:39:21.851083+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:16:13.884648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"80'1","reported_seq":40,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:38:54.096239+0000","last_change":"2024-09-21T03:37:10.832456+0000","last_active":"2024-09-21T03:38:54.096239+0000","last_peered":"2024-09-21T03:38:54.096239+0000","last_clean":"2024-09-21T03:38:54.096239+0000","last_became_active":"2024-09-21T03:37:10.832082+0000","last_became_peered":"2024-09-21T03:37:10.832082+0000","last_unstale":"2024-09-21T03:38:54.096239+0000","last_undegraded":"2024-09-21T03:38:54.096239+0000","last_fullsized":"2024-09-21T03:38:54.096239+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_clean_scrub_stamp":"2024-09-21T03:37:09.809805+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:17:12.170616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":27,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:38:54.096158+0000","last_change":"2024-09-21T03:37:19.908084+0000","last_active":"2024-09-21T03:38:54.096158+0000","last_peered":"2024-09-21T03:38:54.096158+0000","last_clean":"2024-09-21T03:38:54.096158+0000","last_became_active":"2024-09-21T03:37:19.907307+0000","last_became_peered":"2024-09-21T03:37:19.907307+0000","last_unstale":"2024-09-21T03:38:54.096158+0000","last_undegraded":"2024-09-21T03:38:54.096158+0000","last_fullsized":"2024-09-21T03:38:54.096158+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_clean_scrub_stamp":"2024-09-21T03:37:18.876933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:45:02.541015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.18","version":"84'8","reported_seq":42,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:39:21.852435+0000","last_change":"2024-09-21T03:37:12.852577+0000","last_active":"2024-09-21T03:39:21.852435+0000","last_peered":"2024-09-21T03:39:21.852435+0000","last_clean":"2024-09-21T03:39:21.852435+0000","last_became_active":"2024-09-21T03:37:12.852186+0000","last_became_peered":"2024-09-21T03:37:12.852186+0000","last_unstale":"2024-09-21T03:39:21.852435+0000","last_undegraded":"2024-09-21T03:39:21.852435+0000","last_fullsized":"2024-09-21T03:39:21.852435+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:51:21.631029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"80'2","reported_seq":41,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:38:53.668330+0000","last_change":"2024-09-21T03:37:10.833587+0000","last_active":"2024-09-21T03:38:53.668330+0000","last_peered":"2024-09-21T03:38:53.668330+0000","last_clean":"2024-09-21T03:38:53.668330+0000","last_became_active":"2024-09-21T03:37:10.833194+0000","last_became_peered":"2024-09-21T03:37:10.833194+0000","last_unstale":"2024-09-21T03:38:53.668330+0000","last_undegraded":"2024-09-21T03:38:53.668330+0000","last_fullsized":"2024-09-21T03:38:53.668330+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_clean_scrub_stamp":"2024-09-21T03:37:09.809805+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:20:20.272108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"80'3","reported_seq":42,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:38:53.669769+0000","last_change":"2024-09-21T03:37:10.839103+0000","last_active":"2024-09-21T03:38:53.669769+0000","last_peered":"2024-09-21T03:38:53.669769+0000","last_clean":"2024-09-21T03:38:53.669769+0000","last_became_active":"2024-09-21T03:37:10.837192+0000","last_became_peered":"2024-09-21T03:37:10.837192+0000","last_unstale":"2024-09-21T03:38:53.669769+0000","last_undegraded":"2024-09-21T03:38:53.669769+0000","last_fullsized":"2024-09-21T03:38:53.669769+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_clean_scrub_stamp":"2024-09-21T03:37:09.809805+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:59:35.128171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"84'8","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:39:21.852475+0000","last_change":"2024-09-21T03:38:54.097890+0000","last_active":"2024-09-21T03:39:21.852475+0000","last_peered":"2024-09-21T03:39:21.852475+0000","last_clean":"2024-09-21T03:39:21.852475+0000","last_became_active":"2024-09-21T03:38:54.097484+0000","last_became_peered":"2024-09-21T03:38:54.097484+0000","last_unstale":"2024-09-21T03:39:21.852475+0000","last_undegraded":"2024-09-21T03:39:21.852475+0000","last_fullsized":"2024-09-21T03:39:21.852475+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:06:16.098400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,3],"acting":[9,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:38:53.669938+0000","last_change":"2024-09-21T03:37:19.909415+0000","last_active":"2024-09-21T03:38:53.669938+0000","last_peered":"2024-09-21T03:38:53.669938+0000","last_clean":"2024-09-21T03:38:53.669938+0000","last_became_active":"2024-09-21T03:37:19.908955+0000","last_became_peered":"2024-09-21T03:37:19.908955+0000","last_unstale":"2024-09-21T03:38:53.669938+0000","last_undegraded":"2024-09-21T03:38:53.669938+0000","last_fullsized":"2024-09-21T03:38:53.669938+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_clean_scrub_stamp":"2024-09-21T03:37:18.876933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:31:18.878725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"84'9","reported_seq":151,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:39:26.189837+0000","last_change":"2024-09-21T03:37:10.836389+0000","last_active":"2024-09-21T03:39:26.189837+0000","last_peered":"2024-09-21T03:39:26.189837+0000","last_clean":"2024-09-21T03:39:26.189837+0000","last_became_active":"2024-09-21T03:37:10.835986+0000","last_became_peered":"2024-09-21T03:37:10.835986+0000","last_unstale":"2024-09-21T03:39:26.189837+0000","last_undegraded":"2024-09-21T03:39:26.189837+0000","last_fullsized":"2024-09-21T03:39:26.189837+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:09.809805+0000","last_clean_scrub_stamp":"2024-09-21T03:37:09.809805+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:05:17.963609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2038,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":103,"num_read_kb":5,"num_write":6,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"84'7","reported_seq":41,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:39:21.834248+0000","last_change":"2024-09-21T03:37:12.856008+0000","last_active":"2024-09-21T03:39:21.834248+0000","last_peered":"2024-09-21T03:39:21.834248+0000","last_clean":"2024-09-21T03:39:21.834248+0000","last_became_active":"2024-09-21T03:37:12.855603+0000","last_became_peered":"2024-09-21T03:37:12.855603+0000","last_unstale":"2024-09-21T03:39:21.834248+0000","last_undegraded":"2024-09-21T03:39:21.834248+0000","last_fullsized":"2024-09-21T03:39:21.834248+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:11:41.920114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:38:53.669941+0000","last_change":"2024-09-21T03:37:19.911507+0000","last_active":"2024-09-21T03:38:53.669941+0000","last_peered":"2024-09-21T03:38:53.669941+0000","last_clean":"2024-09-21T03:38:53.669941+0000","last_became_active":"2024-09-21T03:37:19.911053+0000","last_became_peered":"2024-09-21T03:37:19.911053+0000","last_unstale":"2024-09-21T03:38:53.669941+0000","last_undegraded":"2024-09-21T03:38:53.669941+0000","last_fullsized":"2024-09-21T03:38:53.669941+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:18.876933+0000","last_clean_scrub_stamp":"2024-09-21T03:37:18.876933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:42:11.964829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"84'9","reported_seq":44,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:39:21.853090+0000","last_change":"2024-09-21T03:37:12.862576+0000","last_active":"2024-09-21T03:39:21.853090+0000","last_peered":"2024-09-21T03:39:21.853090+0000","last_clean":"2024-09-21T03:39:21.853090+0000","last_became_active":"2024-09-21T03:37:12.862146+0000","last_became_peered":"2024-09-21T03:37:12.862146+0000","last_unstale":"2024-09-21T03:39:21.853090+0000","last_undegraded":"2024-09-21T03:39:21.853090+0000","last_fullsized":"2024-09-21T03:39:21.853090+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:30:33.272415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"84'12","reported_seq":47,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:39:21.853470+0000","last_change":"2024-09-21T03:37:12.859395+0000","last_active":"2024-09-21T03:39:21.853470+0000","last_peered":"2024-09-21T03:39:21.853470+0000","last_clean":"2024-09-21T03:39:21.853470+0000","last_became_active":"2024-09-21T03:37:12.858914+0000","last_became_peered":"2024-09-21T03:37:12.858914+0000","last_unstale":"2024-09-21T03:39:21.853470+0000","last_undegraded":"2024-09-21T03:39:21.853470+0000","last_fullsized":"2024-09-21T03:39:21.853470+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:13:06.319896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"84'8","reported_seq":43,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:39:21.844893+0000","last_change":"2024-09-21T03:37:12.857467+0000","last_active":"2024-09-21T03:39:21.844893+0000","last_peered":"2024-09-21T03:39:21.844893+0000","last_clean":"2024-09-21T03:39:21.844893+0000","last_became_active":"2024-09-21T03:37:12.856214+0000","last_became_peered":"2024-09-21T03:37:12.856214+0000","last_unstale":"2024-09-21T03:39:21.844893+0000","last_undegraded":"2024-09-21T03:39:21.844893+0000","last_fullsized":"2024-09-21T03:39:21.844893+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:21:58.780018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"84'7","reported_seq":42,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:39:21.842894+0000","last_change":"2024-09-21T03:37:12.858052+0000","last_active":"2024-09-21T03:39:21.842894+0000","last_peered":"2024-09-21T03:39:21.842894+0000","last_clean":"2024-09-21T03:39:21.842894+0000","last_became_active":"2024-09-21T03:37:12.857445+0000","last_became_peered":"2024-09-21T03:37:12.857445+0000","last_unstale":"2024-09-21T03:39:21.842894+0000","last_undegraded":"2024-09-21T03:39:21.842894+0000","last_fullsized":"2024-09-21T03:39:21.842894+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:22:28.058846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"84'4","reported_seq":39,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:39:21.852798+0000","last_change":"2024-09-21T03:37:12.856499+0000","last_active":"2024-09-21T03:39:21.852798+0000","last_peered":"2024-09-21T03:39:21.852798+0000","last_clean":"2024-09-21T03:39:21.852798+0000","last_became_active":"2024-09-21T03:37:12.856146+0000","last_became_peered":"2024-09-21T03:37:12.856146+0000","last_unstale":"2024-09-21T03:39:21.852798+0000","last_undegraded":"2024-09-21T03:39:21.852798+0000","last_fullsized":"2024-09-21T03:39:21.852798+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:03:26.142740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"84'6","reported_seq":41,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:39:21.851873+0000","last_change":"2024-09-21T03:37:12.860830+0000","last_active":"2024-09-21T03:39:21.851873+0000","last_peered":"2024-09-21T03:39:21.851873+0000","last_clean":"2024-09-21T03:39:21.851873+0000","last_became_active":"2024-09-21T03:37:12.860468+0000","last_became_peered":"2024-09-21T03:37:12.860468+0000","last_unstale":"2024-09-21T03:39:21.851873+0000","last_undegraded":"2024-09-21T03:39:21.851873+0000","last_fullsized":"2024-09-21T03:39:21.851873+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:48:15.057622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"84'15","reported_seq":50,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:39:21.849602+0000","last_change":"2024-09-21T03:37:12.864399+0000","last_active":"2024-09-21T03:39:21.849602+0000","last_peered":"2024-09-21T03:39:21.849602+0000","last_clean":"2024-09-21T03:39:21.849602+0000","last_became_active":"2024-09-21T03:37:12.863892+0000","last_became_peered":"2024-09-21T03:37:12.863892+0000","last_unstale":"2024-09-21T03:39:21.849602+0000","last_undegraded":"2024-09-21T03:39:21.849602+0000","last_fullsized":"2024-09-21T03:39:21.849602+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:58:26.647896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"84'2","reported_seq":37,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:39:21.847416+0000","last_change":"2024-09-21T03:37:12.861716+0000","last_active":"2024-09-21T03:39:21.847416+0000","last_peered":"2024-09-21T03:39:21.847416+0000","last_clean":"2024-09-21T03:39:21.847416+0000","last_became_active":"2024-09-21T03:37:12.861392+0000","last_became_peered":"2024-09-21T03:37:12.861392+0000","last_unstale":"2024-09-21T03:39:21.847416+0000","last_undegraded":"2024-09-21T03:39:21.847416+0000","last_fullsized":"2024-09-21T03:39:21.847416+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:28:55.751107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"84'6","reported_seq":41,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:39:21.842051+0000","last_change":"2024-09-21T03:37:12.856167+0000","last_active":"2024-09-21T03:39:21.842051+0000","last_peered":"2024-09-21T03:39:21.842051+0000","last_clean":"2024-09-21T03:39:21.842051+0000","last_became_active":"2024-09-21T03:37:12.855752+0000","last_became_peered":"2024-09-21T03:37:12.855752+0000","last_unstale":"2024-09-21T03:39:21.842051+0000","last_undegraded":"2024-09-21T03:39:21.842051+0000","last_fullsized":"2024-09-21T03:39:21.842051+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:26:30.271342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"84'8","reported_seq":43,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:39:21.852887+0000","last_change":"2024-09-21T03:37:12.862782+0000","last_active":"2024-09-21T03:39:21.852887+0000","last_peered":"2024-09-21T03:39:21.852887+0000","last_clean":"2024-09-21T03:39:21.852887+0000","last_became_active":"2024-09-21T03:37:12.862462+0000","last_became_peered":"2024-09-21T03:37:12.862462+0000","last_unstale":"2024-09-21T03:39:21.852887+0000","last_undegraded":"2024-09-21T03:39:21.852887+0000","last_fullsized":"2024-09-21T03:39:21.852887+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:57:37.744531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"84'10","reported_seq":45,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:39:21.864024+0000","last_change":"2024-09-21T03:37:12.862312+0000","last_active":"2024-09-21T03:39:21.864024+0000","last_peered":"2024-09-21T03:39:21.864024+0000","last_clean":"2024-09-21T03:39:21.864024+0000","last_became_active":"2024-09-21T03:37:12.861940+0000","last_became_peered":"2024-09-21T03:37:12.861940+0000","last_unstale":"2024-09-21T03:39:21.864024+0000","last_undegraded":"2024-09-21T03:39:21.864024+0000","last_fullsized":"2024-09-21T03:39:21.864024+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:08:49.165522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"84'11","reported_seq":46,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:39:21.847982+0000","last_change":"2024-09-21T03:37:12.857012+0000","last_active":"2024-09-21T03:39:21.847982+0000","last_peered":"2024-09-21T03:39:21.847982+0000","last_clean":"2024-09-21T03:39:21.847982+0000","last_became_active":"2024-09-21T03:37:12.856390+0000","last_became_peered":"2024-09-21T03:37:12.856390+0000","last_unstale":"2024-09-21T03:39:21.847982+0000","last_undegraded":"2024-09-21T03:39:21.847982+0000","last_fullsized":"2024-09-21T03:39:21.847982+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:51:00.839709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"84'8","reported_seq":43,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:39:21.851783+0000","last_change":"2024-09-21T03:37:12.857207+0000","last_active":"2024-09-21T03:39:21.851783+0000","last_peered":"2024-09-21T03:39:21.851783+0000","last_clean":"2024-09-21T03:39:21.851783+0000","last_became_active":"2024-09-21T03:37:12.856770+0000","last_became_peered":"2024-09-21T03:37:12.856770+0000","last_unstale":"2024-09-21T03:39:21.851783+0000","last_undegraded":"2024-09-21T03:39:21.851783+0000","last_fullsized":"2024-09-21T03:39:21.851783+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:37:18.272321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2d","version":"84'7","reported_seq":42,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:39:21.845170+0000","last_change":"2024-09-21T03:37:12.858540+0000","last_active":"2024-09-21T03:39:21.845170+0000","last_peered":"2024-09-21T03:39:21.845170+0000","last_clean":"2024-09-21T03:39:21.845170+0000","last_became_active":"2024-09-21T03:37:12.857340+0000","last_became_peered":"2024-09-21T03:37:12.857340+0000","last_unstale":"2024-09-21T03:39:21.845170+0000","last_undegraded":"2024-09-21T03:39:21.845170+0000","last_fullsized":"2024-09-21T03:39:21.845170+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:45:48.982132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"84'10","reported_seq":44,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:39:21.851214+0000","last_change":"2024-09-21T03:37:12.864408+0000","last_active":"2024-09-21T03:39:21.851214+0000","last_peered":"2024-09-21T03:39:21.851214+0000","last_clean":"2024-09-21T03:39:21.851214+0000","last_became_active":"2024-09-21T03:37:12.864082+0000","last_became_peered":"2024-09-21T03:37:12.864082+0000","last_unstale":"2024-09-21T03:39:21.851214+0000","last_undegraded":"2024-09-21T03:39:21.851214+0000","last_fullsized":"2024-09-21T03:39:21.851214+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:19:32.723589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2f","version":"84'8","reported_seq":42,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:39:21.864236+0000","last_change":"2024-09-21T03:37:12.869445+0000","last_active":"2024-09-21T03:39:21.864236+0000","last_peered":"2024-09-21T03:39:21.864236+0000","last_clean":"2024-09-21T03:39:21.864236+0000","last_became_active":"2024-09-21T03:37:12.869058+0000","last_became_peered":"2024-09-21T03:37:12.869058+0000","last_unstale":"2024-09-21T03:39:21.864236+0000","last_undegraded":"2024-09-21T03:39:21.864236+0000","last_fullsized":"2024-09-21T03:39:21.864236+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:37:57.750665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.30","version":"84'9","reported_seq":43,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:39:21.865647+0000","last_change":"2024-09-21T03:37:12.858431+0000","last_active":"2024-09-21T03:39:21.865647+0000","last_peered":"2024-09-21T03:39:21.865647+0000","last_clean":"2024-09-21T03:39:21.865647+0000","last_became_active":"2024-09-21T03:37:12.858047+0000","last_became_peered":"2024-09-21T03:37:12.858047+0000","last_unstale":"2024-09-21T03:39:21.865647+0000","last_undegraded":"2024-09-21T03:39:21.865647+0000","last_fullsized":"2024-09-21T03:39:21.865647+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:14:05.363743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.31","version":"84'7","reported_seq":41,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:39:21.845694+0000","last_change":"2024-09-21T03:37:12.866784+0000","last_active":"2024-09-21T03:39:21.845694+0000","last_peered":"2024-09-21T03:39:21.845694+0000","last_clean":"2024-09-21T03:39:21.845694+0000","last_became_active":"2024-09-21T03:37:12.866435+0000","last_became_peered":"2024-09-21T03:37:12.866435+0000","last_unstale":"2024-09-21T03:39:21.845694+0000","last_undegraded":"2024-09-21T03:39:21.845694+0000","last_fullsized":"2024-09-21T03:39:21.845694+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:09:06.156030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.32","version":"84'7","reported_seq":41,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:39:21.850666+0000","last_change":"2024-09-21T03:37:12.857054+0000","last_active":"2024-09-21T03:39:21.850666+0000","last_peered":"2024-09-21T03:39:21.850666+0000","last_clean":"2024-09-21T03:39:21.850666+0000","last_became_active":"2024-09-21T03:37:12.856671+0000","last_became_peered":"2024-09-21T03:37:12.856671+0000","last_unstale":"2024-09-21T03:39:21.850666+0000","last_undegraded":"2024-09-21T03:39:21.850666+0000","last_fullsized":"2024-09-21T03:39:21.850666+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:38:27.474550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"84'10","reported_seq":44,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:39:21.850693+0000","last_change":"2024-09-21T03:37:12.861501+0000","last_active":"2024-09-21T03:39:21.850693+0000","last_peered":"2024-09-21T03:39:21.850693+0000","last_clean":"2024-09-21T03:39:21.850693+0000","last_became_active":"2024-09-21T03:37:12.861146+0000","last_became_peered":"2024-09-21T03:37:12.861146+0000","last_unstale":"2024-09-21T03:39:21.850693+0000","last_undegraded":"2024-09-21T03:39:21.850693+0000","last_fullsized":"2024-09-21T03:39:21.850693+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:30:55.073854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"84'7","reported_seq":23,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:39:21.850561+0000","last_change":"2024-09-21T03:37:53.051912+0000","last_active":"2024-09-21T03:39:21.850561+0000","last_peered":"2024-09-21T03:39:21.850561+0000","last_clean":"2024-09-21T03:39:21.850561+0000","last_became_active":"2024-09-21T03:37:53.051640+0000","last_became_peered":"2024-09-21T03:37:53.051640+0000","last_unstale":"2024-09-21T03:39:21.850561+0000","last_undegraded":"2024-09-21T03:39:21.850561+0000","last_fullsized":"2024-09-21T03:39:21.850561+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:08:09.046867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,10,8],"acting":[9,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.35","version":"84'6","reported_seq":40,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:39:21.864066+0000","last_change":"2024-09-21T03:37:12.865862+0000","last_active":"2024-09-21T03:39:21.864066+0000","last_peered":"2024-09-21T03:39:21.864066+0000","last_clean":"2024-09-21T03:39:21.864066+0000","last_became_active":"2024-09-21T03:37:12.865468+0000","last_became_peered":"2024-09-21T03:37:12.865468+0000","last_unstale":"2024-09-21T03:39:21.864066+0000","last_undegraded":"2024-09-21T03:39:21.864066+0000","last_fullsized":"2024-09-21T03:39:21.864066+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:18:14.496495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"84'16","reported_seq":50,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:39:21.856994+0000","last_change":"2024-09-21T03:37:12.862688+0000","last_active":"2024-09-21T03:39:21.856994+0000","last_peered":"2024-09-21T03:39:21.856994+0000","last_clean":"2024-09-21T03:39:21.856994+0000","last_became_active":"2024-09-21T03:37:12.862249+0000","last_became_peered":"2024-09-21T03:37:12.862249+0000","last_unstale":"2024-09-21T03:39:21.856994+0000","last_undegraded":"2024-09-21T03:39:21.856994+0000","last_fullsized":"2024-09-21T03:39:21.856994+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:55:54.731640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.37","version":"84'11","reported_seq":45,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:39:21.864822+0000","last_change":"2024-09-21T03:37:12.863595+0000","last_active":"2024-09-21T03:39:21.864822+0000","last_peered":"2024-09-21T03:39:21.864822+0000","last_clean":"2024-09-21T03:39:21.864822+0000","last_became_active":"2024-09-21T03:37:12.863260+0000","last_became_peered":"2024-09-21T03:37:12.863260+0000","last_unstale":"2024-09-21T03:39:21.864822+0000","last_undegraded":"2024-09-21T03:39:21.864822+0000","last_fullsized":"2024-09-21T03:39:21.864822+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:10:07.995690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"84'9","reported_seq":43,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:39:21.847848+0000","last_change":"2024-09-21T03:37:12.865770+0000","last_active":"2024-09-21T03:39:21.847848+0000","last_peered":"2024-09-21T03:39:21.847848+0000","last_clean":"2024-09-21T03:39:21.847848+0000","last_became_active":"2024-09-21T03:37:12.865405+0000","last_became_peered":"2024-09-21T03:37:12.865405+0000","last_unstale":"2024-09-21T03:39:21.847848+0000","last_undegraded":"2024-09-21T03:39:21.847848+0000","last_fullsized":"2024-09-21T03:39:21.847848+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:46:25.255913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"84'5","reported_seq":39,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:39:21.851267+0000","last_change":"2024-09-21T03:37:12.856214+0000","last_active":"2024-09-21T03:39:21.851267+0000","last_peered":"2024-09-21T03:39:21.851267+0000","last_clean":"2024-09-21T03:39:21.851267+0000","last_became_active":"2024-09-21T03:37:12.853924+0000","last_became_peered":"2024-09-21T03:37:12.853924+0000","last_unstale":"2024-09-21T03:39:21.851267+0000","last_undegraded":"2024-09-21T03:39:21.851267+0000","last_fullsized":"2024-09-21T03:39:21.851267+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:10:32.182892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"84'6","reported_seq":40,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:39:21.856643+0000","last_change":"2024-09-21T03:37:12.858851+0000","last_active":"2024-09-21T03:39:21.856643+0000","last_peered":"2024-09-21T03:39:21.856643+0000","last_clean":"2024-09-21T03:39:21.856643+0000","last_became_active":"2024-09-21T03:37:12.858037+0000","last_became_peered":"2024-09-21T03:37:12.858037+0000","last_unstale":"2024-09-21T03:39:21.856643+0000","last_undegraded":"2024-09-21T03:39:21.856643+0000","last_fullsized":"2024-09-21T03:39:21.856643+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:32:23.134789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.3b","version":"84'4","reported_seq":38,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:39:21.837281+0000","last_change":"2024-09-21T03:37:12.852339+0000","last_active":"2024-09-21T03:39:21.837281+0000","last_peered":"2024-09-21T03:39:21.837281+0000","last_clean":"2024-09-21T03:39:21.837281+0000","last_became_active":"2024-09-21T03:37:12.851911+0000","last_became_peered":"2024-09-21T03:37:12.851911+0000","last_unstale":"2024-09-21T03:39:21.837281+0000","last_undegraded":"2024-09-21T03:39:21.837281+0000","last_fullsized":"2024-09-21T03:39:21.837281+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:03:30.739275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3c","version":"84'8","reported_seq":43,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:39:21.851835+0000","last_change":"2024-09-21T03:37:12.861133+0000","last_active":"2024-09-21T03:39:21.851835+0000","last_peered":"2024-09-21T03:39:21.851835+0000","last_clean":"2024-09-21T03:39:21.851835+0000","last_became_active":"2024-09-21T03:37:12.860719+0000","last_became_peered":"2024-09-21T03:37:12.860719+0000","last_unstale":"2024-09-21T03:39:21.851835+0000","last_undegraded":"2024-09-21T03:39:21.851835+0000","last_fullsized":"2024-09-21T03:39:21.851835+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:21:25.241935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3d","version":"84'7","reported_seq":42,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:39:21.858925+0000","last_change":"2024-09-21T03:37:12.865694+0000","last_active":"2024-09-21T03:39:21.858925+0000","last_peered":"2024-09-21T03:39:21.858925+0000","last_clean":"2024-09-21T03:39:21.858925+0000","last_became_active":"2024-09-21T03:37:12.864983+0000","last_became_peered":"2024-09-21T03:37:12.864983+0000","last_unstale":"2024-09-21T03:39:21.858925+0000","last_undegraded":"2024-09-21T03:39:21.858925+0000","last_fullsized":"2024-09-21T03:39:21.858925+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:52:28.583056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"84'4","reported_seq":39,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:39:21.850375+0000","last_change":"2024-09-21T03:37:12.860665+0000","last_active":"2024-09-21T03:39:21.850375+0000","last_peered":"2024-09-21T03:39:21.850375+0000","last_clean":"2024-09-21T03:39:21.850375+0000","last_became_active":"2024-09-21T03:37:12.860316+0000","last_became_peered":"2024-09-21T03:37:12.860316+0000","last_unstale":"2024-09-21T03:39:21.850375+0000","last_undegraded":"2024-09-21T03:39:21.850375+0000","last_fullsized":"2024-09-21T03:39:21.850375+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:13:07.316422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3f","version":"84'8","reported_seq":43,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T03:39:21.864109+0000","last_change":"2024-09-21T03:37:12.869577+0000","last_active":"2024-09-21T03:39:21.864109+0000","last_peered":"2024-09-21T03:39:21.864109+0000","last_clean":"2024-09-21T03:39:21.864109+0000","last_became_active":"2024-09-21T03:37:12.869164+0000","last_became_peered":"2024-09-21T03:37:12.869164+0000","last_unstale":"2024-09-21T03:39:21.864109+0000","last_undegraded":"2024-09-21T03:39:21.864109+0000","last_fullsized":"2024-09-21T03:39:21.864109+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:37:11.828640+0000","last_clean_scrub_stamp":"2024-09-21T03:37:11.828640+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:40:34.167005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":81050,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":635,"num_read_kb":404,"num_write":187,"num_write_kb":142,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":694,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":458752,"data_stored":238110,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":159,"ondisk_log_size":159,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":285501,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":68,"seq":292057776169,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32092,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713060,"statfs":{"total":95995035648,"available":95962173440,"internally_reserved":0,"allocated":929792,"data_stored":480064,"data_compressed":29416,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":63,"seq":270582939694,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36400,"kb_used_data":1056,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708752,"statfs":{"total":95995035648,"available":95957762048,"internally_reserved":0,"allocated":1081344,"data_stored":1143244,"data_compressed":65710,"data_compressed_allocated":86016,"data_compressed_original":671744,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":58,"seq":249108103219,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27944,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717208,"statfs":{"total":95995035648,"available":95966420992,"internally_reserved":0,"allocated":942080,"data_stored":484284,"data_compressed":29416,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":52,"seq":223338299447,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27932,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717220,"statfs":{"total":95995035648,"available":95966433280,"internally_reserved":0,"allocated":929792,"data_stored":479996,"data_compressed":29416,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":46,"seq":197568495677,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36308,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708844,"statfs":{"total":95995035648,"available":95957856256,"internally_reserved":0,"allocated":987136,"data_stored":1077926,"data_compressed":65710,"data_compressed_allocated":86016,"data_compressed_original":671744,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":41,"seq":176093659202,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32080,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713072,"statfs":{"total":95995035648,"available":95962185728,"internally_reserved":0,"allocated":917504,"data_stored":475364,"data_compressed":29416,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":35,"seq":150323855430,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32156,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712996,"statfs":{"total":95995035648,"available":95962107904,"internally_reserved":0,"allocated":995328,"data_stored":1072239,"data_compressed":65710,"data_compressed_allocated":86016,"data_compressed_original":671744,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":30,"seq":128849018955,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27936,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717216,"statfs":{"total":95995035648,"available":95966429184,"internally_reserved":0,"allocated":933888,"data_stored":481480,"data_compressed":29416,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":24,"seq":103079215183,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27932,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717220,"statfs":{"total":95995035648,"available":95966433280,"internally_reserved":0,"allocated":929792,"data_stored":475558,"data_compressed":29416,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":17,"seq":73014444118,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28016,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717136,"statfs":{"total":95995035648,"available":95966347264,"internally_reserved":0,"allocated":1015808,"data_stored":550078,"data_compressed":29416,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574938,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36316,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708836,"statfs":{"total":95995035648,"available":95957848064,"internally_reserved":0,"allocated":995328,"data_stored":547894,"data_compressed":29416,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738464,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36264,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708888,"statfs":{"total":95995035648,"available":95957901312,"internally_reserved":0,"allocated":942080,"data_stored":476752,"data_compressed":29416,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34677,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34677,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34677,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36294,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36294,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36294,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36294,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36294,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":72620,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":74804,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2110,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2892,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":4092,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":626,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":118784,"data_stored":77602,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T03:39:31.388 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 7c374098-77c9-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T03:39:31.462 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:39:31 smithi114 ceph-mon[34140]: pgmap v363: 129 pgs: 129 active+clean; 656 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 1.9 KiB/s wr, 9 op/s 2024-09-21T03:39:31.500 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:39:31 smithi110 ceph-mon[34000]: pgmap v363: 129 pgs: 129 active+clean; 656 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 1.9 KiB/s wr, 9 op/s 2024-09-21T03:39:31.636 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/7c374098-77c9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:39:32.126 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:32.106+0000 7f8d08e67640 1 -- 172.21.15.89:0/3841537204 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8d04108d20 msgr2=0x7f8d0410f550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:32.126 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:32.106+0000 7f8d08e67640 1 --2- 172.21.15.89:0/3841537204 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8d04108d20 0x7f8d0410f550 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f8cfc009ca0 tx=0x7f8cfc02a8c0 comp rx=0 tx=0).stop 2024-09-21T03:39:32.126 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:32.106+0000 7f8d08e67640 1 -- 172.21.15.89:0/3841537204 shutdown_connections 2024-09-21T03:39:32.127 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:32.106+0000 7f8d08e67640 1 --2- 172.21.15.89:0/3841537204 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8d041083d0 0x7f8d041087e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:32.127 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:32.106+0000 7f8d08e67640 1 --2- 172.21.15.89:0/3841537204 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8d04108d20 0x7f8d0410f550 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:32.127 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:32.106+0000 7f8d08e67640 1 --2- 172.21.15.89:0/3841537204 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8d04102ae0 0x7f8d04102ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:32.127 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:32.106+0000 7f8d08e67640 1 -- 172.21.15.89:0/3841537204 >> 172.21.15.89:0/3841537204 conn(0x7f8d04079790 msgr2=0x7f8d04079ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:39:32.127 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:32.106+0000 7f8d08e67640 1 -- 172.21.15.89:0/3841537204 shutdown_connections 2024-09-21T03:39:32.127 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:32.106+0000 7f8d08e67640 1 -- 172.21.15.89:0/3841537204 wait complete. 2024-09-21T03:39:32.127 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:32.106+0000 7f8d08e67640 1 Processor -- start 2024-09-21T03:39:32.127 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:32.107+0000 7f8d08e67640 1 -- start start 2024-09-21T03:39:32.127 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:32.107+0000 7f8d08e67640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8d04102ae0 0x7f8d04199250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:32.127 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:32.107+0000 7f8d08e67640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8d041083d0 0x7f8d041997f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:32.127 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:32.107+0000 7f8d08e67640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8d04108d20 0x7f8d0419ea90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:32.127 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:32.107+0000 7f8d08e67640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f8d041128e0 con 0x7f8d04108d20 2024-09-21T03:39:32.127 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:32.107+0000 7f8d08e67640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f8d04112760 con 0x7f8d041083d0 2024-09-21T03:39:32.127 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:32.107+0000 7f8d08e67640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f8d04112a60 con 0x7f8d04102ae0 2024-09-21T03:39:32.128 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:32.107+0000 7f8d02d9d640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8d04108d20 0x7f8d0419ea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:32.128 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:32.107+0000 7f8d02d9d640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8d04108d20 0x7f8d0419ea90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:35758/0 (socket says 172.21.15.89:35758) 2024-09-21T03:39:32.128 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:32.107+0000 7f8d02d9d640 1 -- 172.21.15.89:0/946972738 learned_addr learned my addr 172.21.15.89:0/946972738 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-21T03:39:32.128 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:32.107+0000 7f8cf9d9b640 1 --2- 172.21.15.89:0/946972738 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8d041083d0 0x7f8d041997f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:32.128 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:32.107+0000 7f8d0259c640 1 --2- 172.21.15.89:0/946972738 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8d04102ae0 0x7f8d04199250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:32.128 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:32.107+0000 7f8d02d9d640 1 -- 172.21.15.89:0/946972738 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8d04102ae0 msgr2=0x7f8d04199250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:32.128 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:32.107+0000 7f8d02d9d640 1 --2- 172.21.15.89:0/946972738 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8d04102ae0 0x7f8d04199250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:32.128 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:32.107+0000 7f8d02d9d640 1 -- 172.21.15.89:0/946972738 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8d041083d0 msgr2=0x7f8d041997f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:32.128 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:32.107+0000 7f8d02d9d640 1 --2- 172.21.15.89:0/946972738 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8d041083d0 0x7f8d041997f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:32.128 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:32.108+0000 7f8d02d9d640 1 -- 172.21.15.89:0/946972738 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d0419f1c0 con 0x7f8d04108d20 2024-09-21T03:39:32.128 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:32.108+0000 7f8d02d9d640 1 --2- 172.21.15.89:0/946972738 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8d04108d20 0x7f8d0419ea90 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f8cfc005e30 tx=0x7f8cfc0093d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:39:32.128 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:32.108+0000 7f8cfbfff640 1 -- 172.21.15.89:0/946972738 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cfc032d10 con 0x7f8d04108d20 2024-09-21T03:39:32.128 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:32.108+0000 7f8d08e67640 1 -- 172.21.15.89:0/946972738 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8d0419f450 con 0x7f8d04108d20 2024-09-21T03:39:32.128 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:32.108+0000 7f8cfbfff640 1 -- 172.21.15.89:0/946972738 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8cfc032eb0 con 0x7f8d04108d20 2024-09-21T03:39:32.128 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:32.108+0000 7f8cfbfff640 1 -- 172.21.15.89:0/946972738 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cfc030b20 con 0x7f8d04108d20 2024-09-21T03:39:32.129 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:32.108+0000 7f8d08e67640 1 -- 172.21.15.89:0/946972738 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8d041a5c10 con 0x7f8d04108d20 2024-09-21T03:39:32.129 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:32.109+0000 7f8d08e67640 1 -- 172.21.15.89:0/946972738 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8d04102ef0 con 0x7f8d04108d20 2024-09-21T03:39:32.129 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:32.110+0000 7f8cfbfff640 1 -- 172.21.15.89:0/946972738 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 20) ==== 154563+0+0 (secure 0 0 0) 0x7f8cfc030cc0 con 0x7f8d04108d20 2024-09-21T03:39:32.129 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:32.111+0000 7f8cfbfff640 1 --2- 172.21.15.89:0/946972738 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f8cf00b65a0 0x7f8cf00b8ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:32.129 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:32.112+0000 7f8d0259c640 1 --2- 172.21.15.89:0/946972738 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f8cf00b65a0 0x7f8cf00b8ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:32.129 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:32.112+0000 7f8cfbfff640 1 -- 172.21.15.89:0/946972738 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 9121+0+0 (secure 0 0 0) 0x7f8cfc108b80 con 0x7f8d04108d20 2024-09-21T03:39:32.129 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:32.112+0000 7f8d0259c640 1 --2- 172.21.15.89:0/946972738 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f8cf00b65a0 0x7f8cf00b8ab0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f8cf4009b60 tx=0x7f8cf4005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:39:32.129 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:32.115+0000 7f8cfbfff640 1 -- 172.21.15.89:0/946972738 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f8cfc0cef30 con 0x7f8d04108d20 2024-09-21T03:39:32.298 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:39:32 smithi089 ceph-mon[30928]: from='client.15459 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T03:39:32.360 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:32.358+0000 7f8d08e67640 1 -- 172.21.15.89:0/946972738 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8d0419a380 con 0x7f8d04108d20 2024-09-21T03:39:32.361 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:32.359+0000 7f8cfbfff640 1 -- 172.21.15.89:0/946972738 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11912 (secure 0 0 0) 0x7f8d0419a380 con 0x7f8d04108d20 2024-09-21T03:39:32.361 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-09-21T03:39:32.361 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":26,"btime":"2024-09-21T03:39:26:058611+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14721,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6835/2341538514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":2341538514},{"type":"v1","addr":"172.21.15.89:6835","nonce":2341538514}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14733,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6837/146708592","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6836","nonce":146708592},{"type":"v1","addr":"172.21.15.89:6837","nonce":146708592}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14781,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6839/2269965499","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6838","nonce":2269965499},{"type":"v1","addr":"172.21.15.89:6839","nonce":2269965499}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14793,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6841/650850383","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6840","nonce":650850383},{"type":"v1","addr":"172.21.15.89:6841","nonce":650850383}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24565,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6833/752336912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6832","nonce":752336912},{"type":"v1","addr":"172.21.15.114:6833","nonce":752336912}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24575,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6833/127607331","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6832","nonce":127607331},{"type":"v1","addr":"172.21.15.110:6833","nonce":127607331}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T03:37:13.570694+0000","modified":"2024-09-21T03:39:25.333954+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24638,"mds_1":24631,"mds_2":24614},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24593":{"gid":24593,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.110:6835/671820654","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6834","nonce":671820654},{"type":"v1","addr":"172.21.15.110:6835","nonce":671820654}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24604":{"gid":24604,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.114:6835/1049714548","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6834","nonce":1049714548},{"type":"v1","addr":"172.21.15.114:6835","nonce":1049714548}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24610":{"gid":24610,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.114:6837/1941213818","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6836","nonce":1941213818},{"type":"v1","addr":"172.21.15.114:6837","nonce":1941213818}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24614":{"gid":24614,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":15,"addr":"172.21.15.110:6837/4047124589","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6836","nonce":4047124589},{"type":"v1","addr":"172.21.15.110:6837","nonce":4047124589}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24631":{"gid":24631,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":15,"addr":"172.21.15.114:6839/3427369944","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6838","nonce":3427369944},{"type":"v1","addr":"172.21.15.114:6839","nonce":3427369944}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24638":{"gid":24638,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.110:6839/132366709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6838","nonce":132366709},{"type":"v1","addr":"172.21.15.110:6839","nonce":132366709}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24638,"qdb_cluster":[24638,24631,24614]},"id":1}]} 2024-09-21T03:39:32.362 INFO:teuthology.orchestra.run.smithi089.stderr:dumped fsmap epoch 26 2024-09-21T03:39:32.363 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:32.361+0000 7f8d08e67640 1 -- 172.21.15.89:0/946972738 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f8cf00b65a0 msgr2=0x7f8cf00b8ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:32.363 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:32.361+0000 7f8d08e67640 1 --2- 172.21.15.89:0/946972738 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f8cf00b65a0 0x7f8cf00b8ab0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f8cf4009b60 tx=0x7f8cf4005f90 comp rx=0 tx=0).stop 2024-09-21T03:39:32.364 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:32.361+0000 7f8d08e67640 1 -- 172.21.15.89:0/946972738 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8d04108d20 msgr2=0x7f8d0419ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:32.364 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:32.361+0000 7f8d08e67640 1 --2- 172.21.15.89:0/946972738 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8d04108d20 0x7f8d0419ea90 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f8cfc005e30 tx=0x7f8cfc0093d0 comp rx=0 tx=0).stop 2024-09-21T03:39:32.364 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:32.361+0000 7f8d08e67640 1 -- 172.21.15.89:0/946972738 shutdown_connections 2024-09-21T03:39:32.364 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:32.361+0000 7f8d08e67640 1 --2- 172.21.15.89:0/946972738 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f8cf00b65a0 0x7f8cf00b8ab0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:32.364 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:32.361+0000 7f8d08e67640 1 --2- 172.21.15.89:0/946972738 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8d041083d0 0x7f8d041997f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:32.364 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:32.361+0000 7f8d08e67640 1 --2- 172.21.15.89:0/946972738 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8d04108d20 0x7f8d0419ea90 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:32.364 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:32.361+0000 7f8d08e67640 1 --2- 172.21.15.89:0/946972738 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8d04102ae0 0x7f8d04199250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:32.364 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:32.361+0000 7f8d08e67640 1 -- 172.21.15.89:0/946972738 >> 172.21.15.89:0/946972738 conn(0x7f8d04079790 msgr2=0x7f8d04106420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:39:32.364 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:32.361+0000 7f8d08e67640 1 -- 172.21.15.89:0/946972738 shutdown_connections 2024-09-21T03:39:32.364 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:32.361+0000 7f8d08e67640 1 -- 172.21.15.89:0/946972738 wait complete. 2024-09-21T03:39:32.460 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:39:32 smithi114 ceph-mon[34140]: from='client.15459 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T03:39:32.500 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:39:32 smithi110 ceph-mon[34000]: from='client.15459 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T03:39:32.968 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 7c374098-77c9-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T03:39:33.216 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/7c374098-77c9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:39:33.311 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:39:33 smithi089 ceph-mon[30928]: pgmap v364: 129 pgs: 129 active+clean; 656 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.9 KiB/s wr, 7 op/s 2024-09-21T03:39:33.311 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:39:33 smithi089 ceph-mon[30928]: from='client.? 172.21.15.89:0/946972738' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:39:33.362 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:39:33 smithi114 ceph-mon[34140]: pgmap v364: 129 pgs: 129 active+clean; 656 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.9 KiB/s wr, 7 op/s 2024-09-21T03:39:33.362 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:39:33 smithi114 ceph-mon[34140]: from='client.? 172.21.15.89:0/946972738' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:39:33.500 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:39:33 smithi110 ceph-mon[34000]: pgmap v364: 129 pgs: 129 active+clean; 656 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.9 KiB/s wr, 7 op/s 2024-09-21T03:39:33.500 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:39:33 smithi110 ceph-mon[34000]: from='client.? 172.21.15.89:0/946972738' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:39:33.702 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:33.699+0000 7fd920a7c640 1 -- 172.21.15.89:0/3992296985 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd91c10de90 msgr2=0x7fd91c110380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:33.702 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:33.699+0000 7fd920a7c640 1 --2- 172.21.15.89:0/3992296985 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd91c10de90 0x7fd91c110380 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fd914009900 tx=0x7fd91402a410 comp rx=0 tx=0).stop 2024-09-21T03:39:33.702 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:33.699+0000 7fd920a7c640 1 -- 172.21.15.89:0/3992296985 shutdown_connections 2024-09-21T03:39:33.702 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:33.699+0000 7fd920a7c640 1 --2- 172.21.15.89:0/3992296985 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd91c105380 0x7fd91c10d8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:33.702 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:33.699+0000 7fd920a7c640 1 --2- 172.21.15.89:0/3992296985 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd91c10de90 0x7fd91c110380 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:33.702 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:33.699+0000 7fd920a7c640 1 --2- 172.21.15.89:0/3992296985 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd91c104a30 0x7fd91c104e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:33.702 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:33.699+0000 7fd920a7c640 1 -- 172.21.15.89:0/3992296985 >> 172.21.15.89:0/3992296985 conn(0x7fd91c079770 msgr2=0x7fd91c079b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:39:33.702 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:33.699+0000 7fd920a7c640 1 -- 172.21.15.89:0/3992296985 shutdown_connections 2024-09-21T03:39:33.702 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:33.699+0000 7fd920a7c640 1 -- 172.21.15.89:0/3992296985 wait complete. 2024-09-21T03:39:33.704 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:33.700+0000 7fd920a7c640 1 Processor -- start 2024-09-21T03:39:33.705 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:33.700+0000 7fd920a7c640 1 -- start start 2024-09-21T03:39:33.705 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:33.700+0000 7fd920a7c640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd91c104a30 0x7fd91c19a450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:33.705 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:33.700+0000 7fd920a7c640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd91c105380 0x7fd91c19a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:33.705 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:33.700+0000 7fd920a7c640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd91c10de90 0x7fd91c19fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:33.705 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:33.700+0000 7fd920a7c640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fd91c113a20 con 0x7fd91c104a30 2024-09-21T03:39:33.705 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:33.700+0000 7fd920a7c640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fd91c1138a0 con 0x7fd91c10de90 2024-09-21T03:39:33.705 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:33.700+0000 7fd920a7c640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fd91c113ba0 con 0x7fd91c105380 2024-09-21T03:39:33.705 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:33.700+0000 7fd91a59c640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd91c104a30 0x7fd91c19a450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:33.705 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:33.700+0000 7fd91a59c640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd91c104a30 0x7fd91c19a450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:35782/0 (socket says 172.21.15.89:35782) 2024-09-21T03:39:33.705 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:33.700+0000 7fd91a59c640 1 -- 172.21.15.89:0/1345964261 learned_addr learned my addr 172.21.15.89:0/1345964261 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-21T03:39:33.705 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:33.701+0000 7fd91ad9d640 1 --2- 172.21.15.89:0/1345964261 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd91c10de90 0x7fd91c19fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:33.705 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:33.701+0000 7fd919d9b640 1 --2- 172.21.15.89:0/1345964261 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd91c105380 0x7fd91c19a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:33.705 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:33.701+0000 7fd91a59c640 1 -- 172.21.15.89:0/1345964261 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd91c105380 msgr2=0x7fd91c19a9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:33.705 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:33.701+0000 7fd91a59c640 1 --2- 172.21.15.89:0/1345964261 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd91c105380 0x7fd91c19a9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:33.705 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:33.701+0000 7fd91a59c640 1 -- 172.21.15.89:0/1345964261 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd91c10de90 msgr2=0x7fd91c19fc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:33.705 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:33.701+0000 7fd91a59c640 1 --2- 172.21.15.89:0/1345964261 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd91c10de90 0x7fd91c19fc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:33.706 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:33.701+0000 7fd91a59c640 1 -- 172.21.15.89:0/1345964261 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd91c1a03c0 con 0x7fd91c104a30 2024-09-21T03:39:33.706 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:33.701+0000 7fd91a59c640 1 --2- 172.21.15.89:0/1345964261 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd91c104a30 0x7fd91c19a450 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fd90c00ba50 tx=0x7fd90c00bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:39:33.706 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:33.701+0000 7fd90b7fe640 1 -- 172.21.15.89:0/1345964261 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd90c019070 con 0x7fd91c104a30 2024-09-21T03:39:33.706 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:33.701+0000 7fd90b7fe640 1 -- 172.21.15.89:0/1345964261 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd90c00cbe0 con 0x7fd91c104a30 2024-09-21T03:39:33.706 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:33.701+0000 7fd920a7c640 1 -- 172.21.15.89:0/1345964261 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd91c1a06b0 con 0x7fd91c104a30 2024-09-21T03:39:33.706 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:33.701+0000 7fd90b7fe640 1 -- 172.21.15.89:0/1345964261 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd90c0075d0 con 0x7fd91c104a30 2024-09-21T03:39:33.706 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:33.701+0000 7fd920a7c640 1 -- 172.21.15.89:0/1345964261 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd91c1a6e10 con 0x7fd91c104a30 2024-09-21T03:39:33.706 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:33.702+0000 7fd920a7c640 1 -- 172.21.15.89:0/1345964261 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd91c06c260 con 0x7fd91c104a30 2024-09-21T03:39:33.707 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:33.704+0000 7fd90b7fe640 1 -- 172.21.15.89:0/1345964261 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 20) ==== 154563+0+0 (secure 0 0 0) 0x7fd90c01c070 con 0x7fd91c104a30 2024-09-21T03:39:33.707 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:33.705+0000 7fd90b7fe640 1 --2- 172.21.15.89:0/1345964261 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7fd9040b65a0 0x7fd9040b8ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:33.707 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:33.705+0000 7fd919d9b640 1 --2- 172.21.15.89:0/1345964261 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7fd9040b65a0 0x7fd9040b8ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:33.707 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:33.705+0000 7fd90b7fe640 1 -- 172.21.15.89:0/1345964261 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 9121+0+0 (secure 0 0 0) 0x7fd90c0e7510 con 0x7fd91c104a30 2024-09-21T03:39:33.707 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:33.705+0000 7fd919d9b640 1 --2- 172.21.15.89:0/1345964261 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7fd9040b65a0 0x7fd9040b8ab0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fd91c10f6f0 tx=0x7fd91000a4c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:39:33.710 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:33.708+0000 7fd90b7fe640 1 -- 172.21.15.89:0/1345964261 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fd90c023070 con 0x7fd91c104a30 2024-09-21T03:39:33.926 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:33.923+0000 7fd920a7c640 1 -- 172.21.15.89:0/1345964261 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd91c104e40 con 0x7fd91c104a30 2024-09-21T03:39:33.926 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:33.924+0000 7fd90b7fe640 1 -- 172.21.15.89:0/1345964261 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11912 (secure 0 0 0) 0x7fd90c0b1ca0 con 0x7fd91c104a30 2024-09-21T03:39:33.927 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-09-21T03:39:33.927 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":26,"btime":"2024-09-21T03:39:26:058611+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14721,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6835/2341538514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":2341538514},{"type":"v1","addr":"172.21.15.89:6835","nonce":2341538514}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14733,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6837/146708592","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6836","nonce":146708592},{"type":"v1","addr":"172.21.15.89:6837","nonce":146708592}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14781,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6839/2269965499","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6838","nonce":2269965499},{"type":"v1","addr":"172.21.15.89:6839","nonce":2269965499}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14793,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6841/650850383","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6840","nonce":650850383},{"type":"v1","addr":"172.21.15.89:6841","nonce":650850383}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24565,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6833/752336912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6832","nonce":752336912},{"type":"v1","addr":"172.21.15.114:6833","nonce":752336912}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24575,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6833/127607331","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6832","nonce":127607331},{"type":"v1","addr":"172.21.15.110:6833","nonce":127607331}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T03:37:13.570694+0000","modified":"2024-09-21T03:39:25.333954+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24638,"mds_1":24631,"mds_2":24614},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24593":{"gid":24593,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.110:6835/671820654","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6834","nonce":671820654},{"type":"v1","addr":"172.21.15.110:6835","nonce":671820654}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24604":{"gid":24604,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.114:6835/1049714548","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6834","nonce":1049714548},{"type":"v1","addr":"172.21.15.114:6835","nonce":1049714548}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24610":{"gid":24610,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.114:6837/1941213818","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6836","nonce":1941213818},{"type":"v1","addr":"172.21.15.114:6837","nonce":1941213818}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24614":{"gid":24614,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":15,"addr":"172.21.15.110:6837/4047124589","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6836","nonce":4047124589},{"type":"v1","addr":"172.21.15.110:6837","nonce":4047124589}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24631":{"gid":24631,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":15,"addr":"172.21.15.114:6839/3427369944","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6838","nonce":3427369944},{"type":"v1","addr":"172.21.15.114:6839","nonce":3427369944}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24638":{"gid":24638,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.110:6839/132366709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6838","nonce":132366709},{"type":"v1","addr":"172.21.15.110:6839","nonce":132366709}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24638,"qdb_cluster":[24638,24631,24614]},"id":1}]} 2024-09-21T03:39:33.927 INFO:teuthology.orchestra.run.smithi089.stderr:dumped fsmap epoch 26 2024-09-21T03:39:33.928 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:33.926+0000 7fd920a7c640 1 -- 172.21.15.89:0/1345964261 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7fd9040b65a0 msgr2=0x7fd9040b8ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:33.929 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:33.926+0000 7fd920a7c640 1 --2- 172.21.15.89:0/1345964261 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7fd9040b65a0 0x7fd9040b8ab0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fd91c10f6f0 tx=0x7fd91000a4c0 comp rx=0 tx=0).stop 2024-09-21T03:39:33.929 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:33.926+0000 7fd920a7c640 1 -- 172.21.15.89:0/1345964261 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd91c104a30 msgr2=0x7fd91c19a450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:33.929 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:33.926+0000 7fd920a7c640 1 --2- 172.21.15.89:0/1345964261 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd91c104a30 0x7fd91c19a450 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fd90c00ba50 tx=0x7fd90c00bf10 comp rx=0 tx=0).stop 2024-09-21T03:39:33.929 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:33.926+0000 7fd920a7c640 1 -- 172.21.15.89:0/1345964261 shutdown_connections 2024-09-21T03:39:33.929 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:33.926+0000 7fd920a7c640 1 --2- 172.21.15.89:0/1345964261 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7fd9040b65a0 0x7fd9040b8ab0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:33.929 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:33.926+0000 7fd920a7c640 1 --2- 172.21.15.89:0/1345964261 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd91c10de90 0x7fd91c19fc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:33.929 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:33.926+0000 7fd920a7c640 1 --2- 172.21.15.89:0/1345964261 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd91c105380 0x7fd91c19a9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:33.929 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:33.926+0000 7fd920a7c640 1 --2- 172.21.15.89:0/1345964261 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd91c104a30 0x7fd91c19a450 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:33.929 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:33.926+0000 7fd920a7c640 1 -- 172.21.15.89:0/1345964261 >> 172.21.15.89:0/1345964261 conn(0x7fd91c079770 msgr2=0x7fd91c100820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:39:33.929 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:33.926+0000 7fd920a7c640 1 -- 172.21.15.89:0/1345964261 shutdown_connections 2024-09-21T03:39:33.929 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:33.926+0000 7fd920a7c640 1 -- 172.21.15.89:0/1345964261 wait complete. 2024-09-21T03:39:34.257 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:39:34 smithi089 ceph-mon[30928]: from='client.? 172.21.15.89:0/1345964261' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:39:34.401 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:39:34 smithi110 ceph-mon[34000]: from='client.? 172.21.15.89:0/1345964261' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:39:34.461 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:39:34 smithi114 ceph-mon[34140]: from='client.? 172.21.15.89:0/1345964261' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:39:34.604 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 7c374098-77c9-11ef-baf5-efdc52797490 -- ceph osd dump --format=json 2024-09-21T03:39:34.852 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/7c374098-77c9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:39:35.296 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:39:35 smithi089 ceph-mon[30928]: pgmap v365: 129 pgs: 129 active+clean; 656 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.9 KiB/s wr, 8 op/s 2024-09-21T03:39:35.395 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:35.382+0000 7f1ba40ea640 1 -- 172.21.15.89:0/3160804090 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1b9c106330 msgr2=0x7f1b9c106740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:35.395 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:35.382+0000 7f1ba40ea640 1 --2- 172.21.15.89:0/3160804090 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1b9c106330 0x7f1b9c106740 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f1b8c009900 tx=0x7f1b8c02a3f0 comp rx=0 tx=0).stop 2024-09-21T03:39:35.396 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:35.382+0000 7f1ba40ea640 1 -- 172.21.15.89:0/3160804090 shutdown_connections 2024-09-21T03:39:35.396 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:35.382+0000 7f1ba40ea640 1 --2- 172.21.15.89:0/3160804090 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1b9c106c80 0x7f1b9c1106d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:35.396 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:35.382+0000 7f1ba40ea640 1 --2- 172.21.15.89:0/3160804090 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1b9c106330 0x7f1b9c106740 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:35.396 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:35.382+0000 7f1ba40ea640 1 --2- 172.21.15.89:0/3160804090 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1b9c102140 0x7f1b9c102550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:35.396 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:35.382+0000 7f1ba40ea640 1 -- 172.21.15.89:0/3160804090 >> 172.21.15.89:0/3160804090 conn(0x7f1b9c079ea0 msgr2=0x7f1b9c07a2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:39:35.396 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:35.382+0000 7f1ba40ea640 1 -- 172.21.15.89:0/3160804090 shutdown_connections 2024-09-21T03:39:35.396 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:35.382+0000 7f1ba40ea640 1 -- 172.21.15.89:0/3160804090 wait complete. 2024-09-21T03:39:35.396 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:35.383+0000 7f1ba40ea640 1 Processor -- start 2024-09-21T03:39:35.396 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:35.383+0000 7f1ba40ea640 1 -- start start 2024-09-21T03:39:35.396 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:35.383+0000 7f1ba40ea640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1b9c102140 0x7f1b9c19a3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:35.396 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:35.383+0000 7f1ba40ea640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1b9c106330 0x7f1b9c19a960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:35.396 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:35.383+0000 7f1ba40ea640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1b9c106c80 0x7f1b9c19fc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:35.396 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:35.383+0000 7f1ba40ea640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f1b9c1139d0 con 0x7f1b9c106330 2024-09-21T03:39:35.397 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:35.383+0000 7f1ba40ea640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f1b9c113850 con 0x7f1b9c102140 2024-09-21T03:39:35.397 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:35.383+0000 7f1ba40ea640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f1b9c113b50 con 0x7f1b9c106c80 2024-09-21T03:39:35.397 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:35.383+0000 7f1ba1685640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1b9c106330 0x7f1b9c19a960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:35.397 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:35.383+0000 7f1ba1685640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1b9c106330 0x7f1b9c19a960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:35810/0 (socket says 172.21.15.89:35810) 2024-09-21T03:39:35.397 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:35.383+0000 7f1ba1685640 1 -- 172.21.15.89:0/1043137971 learned_addr learned my addr 172.21.15.89:0/1043137971 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-21T03:39:35.397 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:35.383+0000 7f1ba1e86640 1 --2- 172.21.15.89:0/1043137971 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1b9c102140 0x7f1b9c19a3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:35.397 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:35.383+0000 7f1ba2687640 1 --2- 172.21.15.89:0/1043137971 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1b9c106c80 0x7f1b9c19fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:35.397 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:35.383+0000 7f1ba1685640 1 -- 172.21.15.89:0/1043137971 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1b9c106c80 msgr2=0x7f1b9c19fc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:35.397 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:35.383+0000 7f1ba1685640 1 --2- 172.21.15.89:0/1043137971 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1b9c106c80 0x7f1b9c19fc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:35.397 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:35.383+0000 7f1ba1685640 1 -- 172.21.15.89:0/1043137971 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1b9c102140 msgr2=0x7f1b9c19a3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:35.397 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:35.383+0000 7f1ba1685640 1 --2- 172.21.15.89:0/1043137971 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1b9c102140 0x7f1b9c19a3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:35.397 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:35.384+0000 7f1ba1685640 1 -- 172.21.15.89:0/1043137971 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b9c1a0270 con 0x7f1b9c106330 2024-09-21T03:39:35.397 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:35.384+0000 7f1ba2687640 1 --2- 172.21.15.89:0/1043137971 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1b9c106c80 0x7f1b9c19fc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:39:35.398 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:35.384+0000 7f1ba1685640 1 --2- 172.21.15.89:0/1043137971 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1b9c106330 0x7f1b9c19a960 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f1b8c005940 tx=0x7f1b8c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:39:35.398 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:35.384+0000 7f1b92ffd640 1 -- 172.21.15.89:0/1043137971 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b8c03f030 con 0x7f1b9c106330 2024-09-21T03:39:35.398 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:35.384+0000 7f1b92ffd640 1 -- 172.21.15.89:0/1043137971 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1b8c003c90 con 0x7f1b9c106330 2024-09-21T03:39:35.398 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:35.384+0000 7f1b92ffd640 1 -- 172.21.15.89:0/1043137971 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b8c035070 con 0x7f1b9c106330 2024-09-21T03:39:35.398 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:35.384+0000 7f1ba40ea640 1 -- 172.21.15.89:0/1043137971 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1b9c1a0500 con 0x7f1b9c106330 2024-09-21T03:39:35.398 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:35.384+0000 7f1ba40ea640 1 -- 172.21.15.89:0/1043137971 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1b9c1a0990 con 0x7f1b9c106330 2024-09-21T03:39:35.398 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:35.385+0000 7f1ba40ea640 1 -- 172.21.15.89:0/1043137971 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1b9c102550 con 0x7f1b9c106330 2024-09-21T03:39:35.398 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:35.387+0000 7f1b92ffd640 1 -- 172.21.15.89:0/1043137971 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 20) ==== 154563+0+0 (secure 0 0 0) 0x7f1b8c042030 con 0x7f1b9c106330 2024-09-21T03:39:35.398 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:35.387+0000 7f1b92ffd640 1 --2- 172.21.15.89:0/1043137971 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f1b880b65d0 0x7f1b880b8ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:35.398 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:35.387+0000 7f1ba1e86640 1 --2- 172.21.15.89:0/1043137971 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f1b880b65d0 0x7f1b880b8ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:35.398 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:35.388+0000 7f1b92ffd640 1 -- 172.21.15.89:0/1043137971 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 9121+0+0 (secure 0 0 0) 0x7f1b8c108280 con 0x7f1b9c106330 2024-09-21T03:39:35.398 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:35.388+0000 7f1ba1e86640 1 --2- 172.21.15.89:0/1043137971 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f1b880b65d0 0x7f1b880b8ae0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f1b9800abe0 tx=0x7f1b9800a2d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:39:35.398 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:35.391+0000 7f1b92ffd640 1 -- 172.21.15.89:0/1043137971 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f1b8c0ce630 con 0x7f1b9c106330 2024-09-21T03:39:35.461 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:39:35 smithi114 ceph-mon[34140]: pgmap v365: 129 pgs: 129 active+clean; 656 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.9 KiB/s wr, 8 op/s 2024-09-21T03:39:35.500 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:39:35 smithi110 ceph-mon[34000]: pgmap v365: 129 pgs: 129 active+clean; 656 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.9 KiB/s wr, 8 op/s 2024-09-21T03:39:35.573 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:35.570+0000 7f1ba40ea640 1 -- 172.21.15.89:0/1043137971 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f1b9c19b4f0 con 0x7f1b9c106330 2024-09-21T03:39:35.574 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:35.572+0000 7f1b92ffd640 1 -- 172.21.15.89:0/1043137971 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v84) ==== 74+0+24810 (secure 0 0 0) 0x7f1b9c19b4f0 con 0x7f1b9c106330 2024-09-21T03:39:35.574 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-09-21T03:39:35.575 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":84,"fsid":"7c374098-77c9-11ef-baf5-efdc52797490","created":"2024-09-21T03:28:56.681458+0000","modified":"2024-09-21T03:38:53.661606+0000","last_up_change":"2024-09-21T03:36:23.368379+0000","last_in_change":"2024-09-21T03:36:00.131331+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T03:32:37.834339+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-21T03:37:09.443341+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"74","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-21T03:37:11.467821+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"74","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-21T03:37:17.884066+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"27ca5ed8-17d1-4f8e-a7ae-084a914721a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6802","nonce":990539042},{"type":"v1","addr":"172.21.15.89:6803","nonce":990539042}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6804","nonce":990539042},{"type":"v1","addr":"172.21.15.89:6805","nonce":990539042}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6808","nonce":990539042},{"type":"v1","addr":"172.21.15.89:6809","nonce":990539042}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6806","nonce":990539042},{"type":"v1","addr":"172.21.15.89:6807","nonce":990539042}]},"public_addr":"172.21.15.89:6803/990539042","cluster_addr":"172.21.15.89:6805/990539042","heartbeat_back_addr":"172.21.15.89:6809/990539042","heartbeat_front_addr":"172.21.15.89:6807/990539042","state":["exists","up"]},{"osd":1,"uuid":"5176a910-5cf9-49d7-aedb-a20cf36ea075","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6800","nonce":3235852628},{"type":"v1","addr":"172.21.15.110:6801","nonce":3235852628}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6802","nonce":3235852628},{"type":"v1","addr":"172.21.15.110:6803","nonce":3235852628}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6806","nonce":3235852628},{"type":"v1","addr":"172.21.15.110:6807","nonce":3235852628}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6804","nonce":3235852628},{"type":"v1","addr":"172.21.15.110:6805","nonce":3235852628}]},"public_addr":"172.21.15.110:6801/3235852628","cluster_addr":"172.21.15.110:6803/3235852628","heartbeat_back_addr":"172.21.15.110:6807/3235852628","heartbeat_front_addr":"172.21.15.110:6805/3235852628","state":["exists","up"]},{"osd":2,"uuid":"d4977df1-b118-4d7e-b9a9-626a8accc812","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6800","nonce":169452122},{"type":"v1","addr":"172.21.15.114:6801","nonce":169452122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6802","nonce":169452122},{"type":"v1","addr":"172.21.15.114:6803","nonce":169452122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6806","nonce":169452122},{"type":"v1","addr":"172.21.15.114:6807","nonce":169452122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6804","nonce":169452122},{"type":"v1","addr":"172.21.15.114:6805","nonce":169452122}]},"public_addr":"172.21.15.114:6801/169452122","cluster_addr":"172.21.15.114:6803/169452122","heartbeat_back_addr":"172.21.15.114:6807/169452122","heartbeat_front_addr":"172.21.15.114:6805/169452122","state":["exists","up"]},{"osd":3,"uuid":"2c7ae1b4-0bf2-4954-81ed-55d03eab82c6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6810","nonce":3016980877},{"type":"v1","addr":"172.21.15.89:6811","nonce":3016980877}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6812","nonce":3016980877},{"type":"v1","addr":"172.21.15.89:6813","nonce":3016980877}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6816","nonce":3016980877},{"type":"v1","addr":"172.21.15.89:6817","nonce":3016980877}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6814","nonce":3016980877},{"type":"v1","addr":"172.21.15.89:6815","nonce":3016980877}]},"public_addr":"172.21.15.89:6811/3016980877","cluster_addr":"172.21.15.89:6813/3016980877","heartbeat_back_addr":"172.21.15.89:6817/3016980877","heartbeat_front_addr":"172.21.15.89:6815/3016980877","state":["exists","up"]},{"osd":4,"uuid":"8e65399d-6e55-4da4-98ba-f805f0c655a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":30,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6808","nonce":3217880476},{"type":"v1","addr":"172.21.15.110:6809","nonce":3217880476}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6810","nonce":3217880476},{"type":"v1","addr":"172.21.15.110:6811","nonce":3217880476}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6814","nonce":3217880476},{"type":"v1","addr":"172.21.15.110:6815","nonce":3217880476}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6812","nonce":3217880476},{"type":"v1","addr":"172.21.15.110:6813","nonce":3217880476}]},"public_addr":"172.21.15.110:6809/3217880476","cluster_addr":"172.21.15.110:6811/3217880476","heartbeat_back_addr":"172.21.15.110:6815/3217880476","heartbeat_front_addr":"172.21.15.110:6813/3217880476","state":["exists","up"]},{"osd":5,"uuid":"b9ec2889-46bd-40e5-8446-73ab28271c67","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6808","nonce":1144140707},{"type":"v1","addr":"172.21.15.114:6809","nonce":1144140707}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6810","nonce":1144140707},{"type":"v1","addr":"172.21.15.114:6811","nonce":1144140707}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6814","nonce":1144140707},{"type":"v1","addr":"172.21.15.114:6815","nonce":1144140707}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6812","nonce":1144140707},{"type":"v1","addr":"172.21.15.114:6813","nonce":1144140707}]},"public_addr":"172.21.15.114:6809/1144140707","cluster_addr":"172.21.15.114:6811/1144140707","heartbeat_back_addr":"172.21.15.114:6815/1144140707","heartbeat_front_addr":"172.21.15.114:6813/1144140707","state":["exists","up"]},{"osd":6,"uuid":"9b79989e-adb0-42b2-b2f6-a2dce56b66e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":41,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6818","nonce":1522586255},{"type":"v1","addr":"172.21.15.89:6819","nonce":1522586255}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6820","nonce":1522586255},{"type":"v1","addr":"172.21.15.89:6821","nonce":1522586255}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6824","nonce":1522586255},{"type":"v1","addr":"172.21.15.89:6825","nonce":1522586255}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6822","nonce":1522586255},{"type":"v1","addr":"172.21.15.89:6823","nonce":1522586255}]},"public_addr":"172.21.15.89:6819/1522586255","cluster_addr":"172.21.15.89:6821/1522586255","heartbeat_back_addr":"172.21.15.89:6825/1522586255","heartbeat_front_addr":"172.21.15.89:6823/1522586255","state":["exists","up"]},{"osd":7,"uuid":"b381821f-141c-47a1-be65-f57ca730391b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6816","nonce":1331818323},{"type":"v1","addr":"172.21.15.110:6817","nonce":1331818323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6818","nonce":1331818323},{"type":"v1","addr":"172.21.15.110:6819","nonce":1331818323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6822","nonce":1331818323},{"type":"v1","addr":"172.21.15.110:6823","nonce":1331818323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6820","nonce":1331818323},{"type":"v1","addr":"172.21.15.110:6821","nonce":1331818323}]},"public_addr":"172.21.15.110:6817/1331818323","cluster_addr":"172.21.15.110:6819/1331818323","heartbeat_back_addr":"172.21.15.110:6823/1331818323","heartbeat_front_addr":"172.21.15.110:6821/1331818323","state":["exists","up"]},{"osd":8,"uuid":"1bc2a5b4-15f3-456f-9d67-e51307a27bcc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6816","nonce":2138556699},{"type":"v1","addr":"172.21.15.114:6817","nonce":2138556699}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6818","nonce":2138556699},{"type":"v1","addr":"172.21.15.114:6819","nonce":2138556699}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6822","nonce":2138556699},{"type":"v1","addr":"172.21.15.114:6823","nonce":2138556699}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6820","nonce":2138556699},{"type":"v1","addr":"172.21.15.114:6821","nonce":2138556699}]},"public_addr":"172.21.15.114:6817/2138556699","cluster_addr":"172.21.15.114:6819/2138556699","heartbeat_back_addr":"172.21.15.114:6823/2138556699","heartbeat_front_addr":"172.21.15.114:6821/2138556699","state":["exists","up"]},{"osd":9,"uuid":"91f5fd29-d4a7-4800-ac7a-e83b13add89e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6826","nonce":1730253068},{"type":"v1","addr":"172.21.15.89:6827","nonce":1730253068}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6828","nonce":1730253068},{"type":"v1","addr":"172.21.15.89:6829","nonce":1730253068}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6832","nonce":1730253068},{"type":"v1","addr":"172.21.15.89:6833","nonce":1730253068}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6830","nonce":1730253068},{"type":"v1","addr":"172.21.15.89:6831","nonce":1730253068}]},"public_addr":"172.21.15.89:6827/1730253068","cluster_addr":"172.21.15.89:6829/1730253068","heartbeat_back_addr":"172.21.15.89:6833/1730253068","heartbeat_front_addr":"172.21.15.89:6831/1730253068","state":["exists","up"]},{"osd":10,"uuid":"01390499-2a4d-4188-b57a-fcb10c668e2c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6824","nonce":2904016117},{"type":"v1","addr":"172.21.15.110:6825","nonce":2904016117}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6826","nonce":2904016117},{"type":"v1","addr":"172.21.15.110:6827","nonce":2904016117}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6830","nonce":2904016117},{"type":"v1","addr":"172.21.15.110:6831","nonce":2904016117}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6828","nonce":2904016117},{"type":"v1","addr":"172.21.15.110:6829","nonce":2904016117}]},"public_addr":"172.21.15.110:6825/2904016117","cluster_addr":"172.21.15.110:6827/2904016117","heartbeat_back_addr":"172.21.15.110:6831/2904016117","heartbeat_front_addr":"172.21.15.110:6829/2904016117","state":["exists","up"]},{"osd":11,"uuid":"c3c0f38a-c3d3-415d-a9f8-875d5b756e35","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6824","nonce":3749854486},{"type":"v1","addr":"172.21.15.114:6825","nonce":3749854486}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6826","nonce":3749854486},{"type":"v1","addr":"172.21.15.114:6827","nonce":3749854486}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6830","nonce":3749854486},{"type":"v1","addr":"172.21.15.114:6831","nonce":3749854486}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6828","nonce":3749854486},{"type":"v1","addr":"172.21.15.114:6829","nonce":3749854486}]},"public_addr":"172.21.15.114:6825/3749854486","cluster_addr":"172.21.15.114:6827/3749854486","heartbeat_back_addr":"172.21.15.114:6831/3749854486","heartbeat_front_addr":"172.21.15.114:6829/3749854486","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:31:46.218908+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:32:10.338230+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:32:34.830596+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:33:00.816102+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:33:25.223838+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:33:50.188230+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:34:16.564255+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:34:41.498230+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:35:05.826322+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:35:31.234450+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:35:56.474373+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:36:21.741254+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.c","mappings":[{"from":2,"to":4}]},{"pgid":"2.e","mappings":[{"from":2,"to":1}]},{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.1b","mappings":[{"from":5,"to":9}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]},{"pgid":"4.6","mappings":[{"from":9,"to":11}]},{"pgid":"4.12","mappings":[{"from":9,"to":10}]},{"pgid":"4.13","mappings":[{"from":9,"to":8}]},{"pgid":"4.1b","mappings":[{"from":5,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.89:6801/655110910":"2024-09-22T03:29:51.519106+0000","172.21.15.89:0/1618544983":"2024-09-22T03:29:51.519106+0000","172.21.15.89:0/1265502758":"2024-09-22T03:29:51.519106+0000","172.21.15.89:0/4017684419":"2024-09-22T03:29:51.519106+0000","172.21.15.89:6801/3312712420":"2024-09-22T03:29:26.004686+0000","172.21.15.89:6800/3312712420":"2024-09-22T03:29:26.004686+0000","172.21.15.89:0/2056576483":"2024-09-22T03:29:26.004686+0000","172.21.15.89:6800/655110910":"2024-09-22T03:29:51.519106+0000","172.21.15.89:0/2382866302":"2024-09-22T03:29:26.004686+0000","172.21.15.89:0/1584351801":"2024-09-22T03:29:26.004686+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T03:39:35.576 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:35.574+0000 7f1ba40ea640 1 -- 172.21.15.89:0/1043137971 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f1b880b65d0 msgr2=0x7f1b880b8ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:35.576 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:35.574+0000 7f1ba40ea640 1 --2- 172.21.15.89:0/1043137971 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f1b880b65d0 0x7f1b880b8ae0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f1b9800abe0 tx=0x7f1b9800a2d0 comp rx=0 tx=0).stop 2024-09-21T03:39:35.577 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:35.574+0000 7f1ba40ea640 1 -- 172.21.15.89:0/1043137971 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1b9c106330 msgr2=0x7f1b9c19a960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:35.577 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:35.574+0000 7f1ba40ea640 1 --2- 172.21.15.89:0/1043137971 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1b9c106330 0x7f1b9c19a960 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f1b8c005940 tx=0x7f1b8c0042f0 comp rx=0 tx=0).stop 2024-09-21T03:39:35.577 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:35.574+0000 7f1ba40ea640 1 -- 172.21.15.89:0/1043137971 shutdown_connections 2024-09-21T03:39:35.577 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:35.574+0000 7f1ba40ea640 1 --2- 172.21.15.89:0/1043137971 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f1b880b65d0 0x7f1b880b8ae0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:35.577 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:35.574+0000 7f1ba40ea640 1 --2- 172.21.15.89:0/1043137971 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1b9c106c80 0x7f1b9c19fc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:35.577 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:35.574+0000 7f1ba40ea640 1 --2- 172.21.15.89:0/1043137971 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1b9c106330 0x7f1b9c19a960 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:35.577 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:35.574+0000 7f1ba40ea640 1 --2- 172.21.15.89:0/1043137971 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1b9c102140 0x7f1b9c19a3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:35.577 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:35.574+0000 7f1ba40ea640 1 -- 172.21.15.89:0/1043137971 >> 172.21.15.89:0/1043137971 conn(0x7f1b9c079ea0 msgr2=0x7f1b9c107640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:39:35.577 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:35.574+0000 7f1ba40ea640 1 -- 172.21.15.89:0/1043137971 shutdown_connections 2024-09-21T03:39:35.577 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:35.574+0000 7f1ba40ea640 1 -- 172.21.15.89:0/1043137971 wait complete. 2024-09-21T03:39:36.221 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 7c374098-77c9-11ef-baf5-efdc52797490 -- ceph daemon mon.a --format=json config get run_dir 2024-09-21T03:39:36.311 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:39:36 smithi089 ceph-mon[30928]: from='client.? 172.21.15.89:0/1043137971' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T03:39:36.461 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:39:36 smithi114 ceph-mon[34140]: from='client.? 172.21.15.89:0/1043137971' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T03:39:36.469 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/7c374098-77c9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:39:36.500 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:39:36 smithi110 ceph-mon[34000]: from='client.? 172.21.15.89:0/1043137971' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T03:39:37.309 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:39:37 smithi089 ceph-mon[30928]: pgmap v366: 129 pgs: 129 active+clean; 656 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 1.8 KiB/s wr, 9 op/s 2024-09-21T03:39:37.461 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:39:37 smithi114 ceph-mon[34140]: pgmap v366: 129 pgs: 129 active+clean; 656 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 1.8 KiB/s wr, 9 op/s 2024-09-21T03:39:37.500 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:39:37 smithi110 ceph-mon[34000]: pgmap v366: 129 pgs: 129 active+clean; 656 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 1.8 KiB/s wr, 9 op/s 2024-09-21T03:39:37.631 INFO:teuthology.orchestra.run.smithi089.stdout:{"run_dir":"/var/run/ceph"} 2024-09-21T03:39:37.631 DEBUG:tasks.cephfs.filesystem:_json_asok output { "run_dir": "/var/run/ceph" } 2024-09-21T03:39:37.632 INFO:tasks.quiescer.fs.[cephfs]:Initializing Quiescer with seed 7864 2024-09-21T03:39:37.632 DEBUG:tasks.quiescer:Yielding 2024-09-21T03:39:37.632 INFO:teuthology.run_tasks:Running task workunit... 2024-09-21T03:39:37.646 INFO:tasks.workunit:Pulling workunits from ref 1843221895adccfb3b692060251dd6a0183bc410 2024-09-21T03:39:37.646 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-09-21T03:39:37.646 INFO:tasks.workunit:timeout=3h 2024-09-21T03:39:37.646 INFO:tasks.workunit:cleanup=True 2024-09-21T03:39:37.646 DEBUG:teuthology.orchestra.run.smithi089:> stat -- /home/ubuntu/cephtest/mnt.0 2024-09-21T03:39:37.647 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 7c374098-77c9-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T03:39:37.663 INFO:teuthology.orchestra.run.smithi089.stdout: File: /home/ubuntu/cephtest/mnt.0 2024-09-21T03:39:37.663 INFO:teuthology.orchestra.run.smithi089.stdout: Size: 0 Blocks: 1 IO Block: 4194304 directory 2024-09-21T03:39:37.663 INFO:teuthology.orchestra.run.smithi089.stdout:Device: 78h/120d Inode: 1 Links: 2 2024-09-21T03:39:37.663 INFO:teuthology.orchestra.run.smithi089.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-21T03:39:37.663 INFO:teuthology.orchestra.run.smithi089.stdout:Context: system_u:object_r:fusefs_t:s0 2024-09-21T03:39:37.664 INFO:teuthology.orchestra.run.smithi089.stdout:Access: 2024-09-21 03:37:54.382393055 +0000 2024-09-21T03:39:37.664 INFO:teuthology.orchestra.run.smithi089.stdout:Modify: 2024-09-21 03:37:54.382393055 +0000 2024-09-21T03:39:37.664 INFO:teuthology.orchestra.run.smithi089.stdout:Change: 2024-09-21 03:39:24.951704959 +0000 2024-09-21T03:39:37.664 INFO:teuthology.orchestra.run.smithi089.stdout: Birth: - 2024-09-21T03:39:37.664 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.0 2024-09-21T03:39:37.664 DEBUG:teuthology.orchestra.run.smithi089:> cd -- /home/ubuntu/cephtest/mnt.0 && sudo install -d -m 0755 --owner=ubuntu -- client.0 2024-09-21T03:39:37.736 DEBUG:teuthology.orchestra.run.smithi089:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://git.ceph.com/ceph-ci.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout 1843221895adccfb3b692060251dd6a0183bc410 2024-09-21T03:39:37.792 INFO:tasks.workunit.client.0.smithi089.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-09-21T03:39:37.906 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/7c374098-77c9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:39:38.441 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:38.424+0000 7f1d4566a640 1 -- 172.21.15.89:0/1074801823 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1d40102ae0 msgr2=0x7f1d40102ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:38.441 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:38.424+0000 7f1d4566a640 1 --2- 172.21.15.89:0/1074801823 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1d40102ae0 0x7f1d40102ef0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f1d30009ca0 tx=0x7f1d3002a8c0 comp rx=0 tx=0).stop 2024-09-21T03:39:38.441 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:38.424+0000 7f1d4566a640 1 -- 172.21.15.89:0/1074801823 shutdown_connections 2024-09-21T03:39:38.441 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:38.424+0000 7f1d4566a640 1 --2- 172.21.15.89:0/1074801823 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1d401083d0 0x7f1d401087e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:38.441 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:38.424+0000 7f1d4566a640 1 --2- 172.21.15.89:0/1074801823 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1d40108d20 0x7f1d4010f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:38.441 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:38.424+0000 7f1d4566a640 1 --2- 172.21.15.89:0/1074801823 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1d40102ae0 0x7f1d40102ef0 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:38.441 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:38.424+0000 7f1d4566a640 1 -- 172.21.15.89:0/1074801823 >> 172.21.15.89:0/1074801823 conn(0x7f1d40079760 msgr2=0x7f1d40079b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:39:38.441 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:38.424+0000 7f1d4566a640 1 -- 172.21.15.89:0/1074801823 shutdown_connections 2024-09-21T03:39:38.441 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:38.424+0000 7f1d4566a640 1 -- 172.21.15.89:0/1074801823 wait complete. 2024-09-21T03:39:38.441 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:38.424+0000 7f1d4566a640 1 Processor -- start 2024-09-21T03:39:38.441 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:38.425+0000 7f1d4566a640 1 -- start start 2024-09-21T03:39:38.441 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:38.425+0000 7f1d4566a640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1d40102ae0 0x7f1d4019b2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:38.442 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:38.425+0000 7f1d4566a640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1d401083d0 0x7f1d4019b820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:38.442 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:38.425+0000 7f1d4566a640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1d40108d20 0x7f1d4019bd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:38.442 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:38.425+0000 7f1d4566a640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f1d401128f0 con 0x7f1d401083d0 2024-09-21T03:39:38.442 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:38.425+0000 7f1d4566a640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f1d40112770 con 0x7f1d40102ae0 2024-09-21T03:39:38.442 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:38.425+0000 7f1d4566a640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f1d40112a70 con 0x7f1d40108d20 2024-09-21T03:39:38.442 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:38.425+0000 7f1d3e7fc640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1d401083d0 0x7f1d4019b820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:38.442 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:38.425+0000 7f1d3e7fc640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1d401083d0 0x7f1d4019b820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:46890/0 (socket says 172.21.15.89:46890) 2024-09-21T03:39:38.442 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:38.425+0000 7f1d3e7fc640 1 -- 172.21.15.89:0/397820422 learned_addr learned my addr 172.21.15.89:0/397820422 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-21T03:39:38.442 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:38.425+0000 7f1d3f7fe640 1 --2- 172.21.15.89:0/397820422 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1d40108d20 0x7f1d4019bd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:38.442 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:38.425+0000 7f1d3e7fc640 1 -- 172.21.15.89:0/397820422 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1d40108d20 msgr2=0x7f1d4019bd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:38.442 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:38.425+0000 7f1d3e7fc640 1 --2- 172.21.15.89:0/397820422 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1d40108d20 0x7f1d4019bd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:38.442 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:38.425+0000 7f1d3effd640 1 --2- 172.21.15.89:0/397820422 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1d40102ae0 0x7f1d4019b2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:38.442 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:38.425+0000 7f1d3e7fc640 1 -- 172.21.15.89:0/397820422 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1d40102ae0 msgr2=0x7f1d4019b2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:38.442 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:38.425+0000 7f1d3e7fc640 1 --2- 172.21.15.89:0/397820422 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1d40102ae0 0x7f1d4019b2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:38.443 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:38.425+0000 7f1d3e7fc640 1 -- 172.21.15.89:0/397820422 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d401a64e0 con 0x7f1d401083d0 2024-09-21T03:39:38.443 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:38.425+0000 7f1d3f7fe640 1 --2- 172.21.15.89:0/397820422 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1d40108d20 0x7f1d4019bd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:39:38.443 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:38.425+0000 7f1d3effd640 1 --2- 172.21.15.89:0/397820422 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1d40102ae0 0x7f1d4019b2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:39:38.443 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:38.425+0000 7f1d3e7fc640 1 --2- 172.21.15.89:0/397820422 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1d401083d0 0x7f1d4019b820 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f1d3400b450 tx=0x7f1d3400b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:39:38.443 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:38.426+0000 7f1d27fff640 1 -- 172.21.15.89:0/397820422 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d34015020 con 0x7f1d401083d0 2024-09-21T03:39:38.443 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:38.426+0000 7f1d27fff640 1 -- 172.21.15.89:0/397820422 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1d34011d20 con 0x7f1d401083d0 2024-09-21T03:39:38.443 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:38.426+0000 7f1d4566a640 1 -- 172.21.15.89:0/397820422 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1d401a67d0 con 0x7f1d401083d0 2024-09-21T03:39:38.443 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:38.426+0000 7f1d27fff640 1 -- 172.21.15.89:0/397820422 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d34010e10 con 0x7f1d401083d0 2024-09-21T03:39:38.443 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:38.426+0000 7f1d4566a640 1 -- 172.21.15.89:0/397820422 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1d401a6d10 con 0x7f1d401083d0 2024-09-21T03:39:38.443 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:38.427+0000 7f1d4566a640 1 -- 172.21.15.89:0/397820422 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1d40102ef0 con 0x7f1d401083d0 2024-09-21T03:39:38.443 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:38.428+0000 7f1d27fff640 1 -- 172.21.15.89:0/397820422 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 20) ==== 154563+0+0 (secure 0 0 0) 0x7f1d34018020 con 0x7f1d401083d0 2024-09-21T03:39:38.443 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:38.429+0000 7f1d27fff640 1 --2- 172.21.15.89:0/397820422 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f1d280b65d0 0x7f1d280b8ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:38.443 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:38.429+0000 7f1d3effd640 1 --2- 172.21.15.89:0/397820422 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f1d280b65d0 0x7f1d280b8ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:38.443 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:38.430+0000 7f1d27fff640 1 -- 172.21.15.89:0/397820422 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 9121+0+0 (secure 0 0 0) 0x7f1d340e76b0 con 0x7f1d401083d0 2024-09-21T03:39:38.444 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:38.430+0000 7f1d3effd640 1 --2- 172.21.15.89:0/397820422 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f1d280b65d0 0x7f1d280b8ae0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f1d30006010 tx=0x7f1d30000f40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:39:38.444 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:38.433+0000 7f1d27fff640 1 -- 172.21.15.89:0/397820422 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f1d3401f020 con 0x7f1d401083d0 2024-09-21T03:39:38.649 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:38.646+0000 7f1d4566a640 1 -- 172.21.15.89:0/397820422 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1d4006bef0 con 0x7f1d401083d0 2024-09-21T03:39:38.649 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:38.647+0000 7f1d27fff640 1 -- 172.21.15.89:0/397820422 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11912 (secure 0 0 0) 0x7f1d3400d040 con 0x7f1d401083d0 2024-09-21T03:39:38.650 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-09-21T03:39:38.650 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":26,"btime":"2024-09-21T03:39:26:058611+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14721,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6835/2341538514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":2341538514},{"type":"v1","addr":"172.21.15.89:6835","nonce":2341538514}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14733,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6837/146708592","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6836","nonce":146708592},{"type":"v1","addr":"172.21.15.89:6837","nonce":146708592}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14781,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6839/2269965499","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6838","nonce":2269965499},{"type":"v1","addr":"172.21.15.89:6839","nonce":2269965499}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14793,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6841/650850383","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6840","nonce":650850383},{"type":"v1","addr":"172.21.15.89:6841","nonce":650850383}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24565,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6833/752336912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6832","nonce":752336912},{"type":"v1","addr":"172.21.15.114:6833","nonce":752336912}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24575,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6833/127607331","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6832","nonce":127607331},{"type":"v1","addr":"172.21.15.110:6833","nonce":127607331}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T03:37:13.570694+0000","modified":"2024-09-21T03:39:25.333954+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24638,"mds_1":24631,"mds_2":24614},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24593":{"gid":24593,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.110:6835/671820654","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6834","nonce":671820654},{"type":"v1","addr":"172.21.15.110:6835","nonce":671820654}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24604":{"gid":24604,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.114:6835/1049714548","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6834","nonce":1049714548},{"type":"v1","addr":"172.21.15.114:6835","nonce":1049714548}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24610":{"gid":24610,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.114:6837/1941213818","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6836","nonce":1941213818},{"type":"v1","addr":"172.21.15.114:6837","nonce":1941213818}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24614":{"gid":24614,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":15,"addr":"172.21.15.110:6837/4047124589","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6836","nonce":4047124589},{"type":"v1","addr":"172.21.15.110:6837","nonce":4047124589}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24631":{"gid":24631,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":15,"addr":"172.21.15.114:6839/3427369944","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6838","nonce":3427369944},{"type":"v1","addr":"172.21.15.114:6839","nonce":3427369944}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24638":{"gid":24638,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.110:6839/132366709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6838","nonce":132366709},{"type":"v1","addr":"172.21.15.110:6839","nonce":132366709}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24638,"qdb_cluster":[24638,24631,24614]},"id":1}]} 2024-09-21T03:39:38.650 INFO:teuthology.orchestra.run.smithi089.stderr:dumped fsmap epoch 26 2024-09-21T03:39:38.652 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:38.650+0000 7f1d4566a640 1 -- 172.21.15.89:0/397820422 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f1d280b65d0 msgr2=0x7f1d280b8ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:38.652 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:38.650+0000 7f1d4566a640 1 --2- 172.21.15.89:0/397820422 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f1d280b65d0 0x7f1d280b8ae0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f1d30006010 tx=0x7f1d30000f40 comp rx=0 tx=0).stop 2024-09-21T03:39:38.652 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:38.650+0000 7f1d4566a640 1 -- 172.21.15.89:0/397820422 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1d401083d0 msgr2=0x7f1d4019b820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:38.652 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:38.650+0000 7f1d4566a640 1 --2- 172.21.15.89:0/397820422 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1d401083d0 0x7f1d4019b820 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f1d3400b450 tx=0x7f1d3400b910 comp rx=0 tx=0).stop 2024-09-21T03:39:38.652 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:38.650+0000 7f1d4566a640 1 -- 172.21.15.89:0/397820422 shutdown_connections 2024-09-21T03:39:38.652 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:38.650+0000 7f1d4566a640 1 --2- 172.21.15.89:0/397820422 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f1d280b65d0 0x7f1d280b8ae0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:38.653 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:38.650+0000 7f1d4566a640 1 --2- 172.21.15.89:0/397820422 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1d40108d20 0x7f1d4019bd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:38.653 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:38.650+0000 7f1d4566a640 1 --2- 172.21.15.89:0/397820422 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1d401083d0 0x7f1d4019b820 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:38.653 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:38.650+0000 7f1d4566a640 1 --2- 172.21.15.89:0/397820422 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1d40102ae0 0x7f1d4019b2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:38.653 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:38.650+0000 7f1d4566a640 1 -- 172.21.15.89:0/397820422 >> 172.21.15.89:0/397820422 conn(0x7f1d40079760 msgr2=0x7f1d40101600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:39:38.653 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:38.650+0000 7f1d4566a640 1 -- 172.21.15.89:0/397820422 shutdown_connections 2024-09-21T03:39:38.653 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:38.650+0000 7f1d4566a640 1 -- 172.21.15.89:0/397820422 wait complete. 2024-09-21T03:39:39.311 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:39:39 smithi089 ceph-mon[30928]: pgmap v367: 129 pgs: 129 active+clean; 656 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 1.3 KiB/s wr, 7 op/s 2024-09-21T03:39:39.311 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:39:39 smithi089 ceph-mon[30928]: from='client.? 172.21.15.89:0/397820422' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:39:39.343 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 26, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-09-21T03:37:13.570694+0000', 'modified': '2024-09-21T03:39:25.333954+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 24638, 'mds_1': 24631, 'mds_2': 24614}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24593': {'gid': 24593, 'name': 'e', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.110:6835/671820654', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.110:6834', 'nonce': 671820654}, {'type': 'v1', 'addr': '172.21.15.110:6835', 'nonce': 671820654}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24604': {'gid': 24604, 'name': 'f', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.114:6835/1049714548', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.114:6834', 'nonce': 1049714548}, {'type': 'v1', 'addr': '172.21.15.114:6835', 'nonce': 1049714548}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24610': {'gid': 24610, 'name': 'c', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.114:6837/1941213818', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.114:6836', 'nonce': 1941213818}, {'type': 'v1', 'addr': '172.21.15.114:6837', 'nonce': 1941213818}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24614': {'gid': 24614, 'name': 'h', 'rank': 2, 'incarnation': 22, 'state': 'up:active', 'state_seq': 15, 'addr': '172.21.15.110:6837/4047124589', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.110:6836', 'nonce': 4047124589}, {'type': 'v1', 'addr': '172.21.15.110:6837', 'nonce': 4047124589}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24631': {'gid': 24631, 'name': 'i', 'rank': 1, 'incarnation': 20, 'state': 'up:active', 'state_seq': 15, 'addr': '172.21.15.114:6839/3427369944', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.114:6838', 'nonce': 3427369944}, {'type': 'v1', 'addr': '172.21.15.114:6839', 'nonce': 3427369944}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24638': {'gid': 24638, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.110:6839/132366709', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.110:6838', 'nonce': 132366709}, {'type': 'v1', 'addr': '172.21.15.110:6839', 'nonce': 132366709}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24638, 'qdb_cluster': [24638, 24631, 24614]} 2024-09-21T03:39:39.343 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2024-09-21T03:39:39.343 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 7c374098-77c9-11ef-baf5-efdc52797490 -- ceph tell mds.h status 2024-09-21T03:39:39.460 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:39:39 smithi114 ceph-mon[34140]: pgmap v367: 129 pgs: 129 active+clean; 656 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 1.3 KiB/s wr, 7 op/s 2024-09-21T03:39:39.461 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:39:39 smithi114 ceph-mon[34140]: from='client.? 172.21.15.89:0/397820422' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:39:39.500 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:39:39 smithi110 ceph-mon[34000]: pgmap v367: 129 pgs: 129 active+clean; 656 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 1.3 KiB/s wr, 7 op/s 2024-09-21T03:39:39.500 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:39:39 smithi110 ceph-mon[34000]: from='client.? 172.21.15.89:0/397820422' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:39:39.636 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/7c374098-77c9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:39:39.686 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:39:39.684+0000 7fc33783f640 1 -- 172.21.15.89:0/1290564455 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc330103fc0 msgr2=0x7fc33010a960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:39.686 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:39:39.684+0000 7fc33783f640 1 --2- 172.21.15.89:0/1290564455 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc330103fc0 0x7fc33010a960 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fc328009c90 tx=0x7fc32802a860 comp rx=0 tx=0).stop 2024-09-21T03:39:39.687 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:39:39.684+0000 7fc33783f640 1 -- 172.21.15.89:0/1290564455 shutdown_connections 2024-09-21T03:39:39.687 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:39:39.684+0000 7fc33783f640 1 --2- 172.21.15.89:0/1290564455 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc330103620 0x7fc330103a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:39.687 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:39:39.684+0000 7fc33783f640 1 --2- 172.21.15.89:0/1290564455 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc330103fc0 0x7fc33010a960 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:39.687 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:39:39.684+0000 7fc33783f640 1 --2- 172.21.15.89:0/1290564455 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc3301025a0 0x7fc330102a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:39.687 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:39:39.684+0000 7fc33783f640 1 -- 172.21.15.89:0/1290564455 >> 172.21.15.89:0/1290564455 conn(0x7fc3300fdee0 msgr2=0x7fc330100390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:39:39.687 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:39:39.684+0000 7fc33783f640 1 -- 172.21.15.89:0/1290564455 shutdown_connections 2024-09-21T03:39:39.687 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:39:39.684+0000 7fc33783f640 1 -- 172.21.15.89:0/1290564455 wait complete. 2024-09-21T03:39:39.689 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:39:39.685+0000 7fc33783f640 1 Processor -- start 2024-09-21T03:39:39.689 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:39:39.685+0000 7fc33783f640 1 -- start start 2024-09-21T03:39:39.689 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:39:39.685+0000 7fc33783f640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc3301025a0 0x7fc330198dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:39.689 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:39:39.685+0000 7fc33783f640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc330103620 0x7fc330199430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:39.689 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:39:39.685+0000 7fc33783f640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc330103fc0 0x7fc330199fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:39.689 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:39:39.685+0000 7fc33783f640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fc33010e0b0 con 0x7fc330103620 2024-09-21T03:39:39.689 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:39:39.685+0000 7fc33783f640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fc33010df30 con 0x7fc3301025a0 2024-09-21T03:39:39.689 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:39:39.685+0000 7fc33783f640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fc33010e230 con 0x7fc330103fc0 2024-09-21T03:39:39.689 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:39:39.685+0000 7fc334dda640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc330103620 0x7fc330199430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:39.690 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:39:39.685+0000 7fc334dda640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc330103620 0x7fc330199430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:46910/0 (socket says 172.21.15.89:46910) 2024-09-21T03:39:39.690 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:39:39.685+0000 7fc334dda640 1 -- 172.21.15.89:0/2840093987 learned_addr learned my addr 172.21.15.89:0/2840093987 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-21T03:39:39.690 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:39:39.685+0000 7fc335ddc640 1 --2- 172.21.15.89:0/2840093987 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc330103fc0 0x7fc330199fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:39.690 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:39:39.685+0000 7fc3355db640 1 --2- 172.21.15.89:0/2840093987 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc3301025a0 0x7fc330198dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:39.690 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:39:39.685+0000 7fc334dda640 1 -- 172.21.15.89:0/2840093987 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc330103fc0 msgr2=0x7fc330199fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:39.690 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:39:39.685+0000 7fc334dda640 1 --2- 172.21.15.89:0/2840093987 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc330103fc0 0x7fc330199fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:39.690 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:39:39.685+0000 7fc334dda640 1 -- 172.21.15.89:0/2840093987 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc3301025a0 msgr2=0x7fc330198dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:39.690 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:39:39.685+0000 7fc334dda640 1 --2- 172.21.15.89:0/2840093987 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc3301025a0 0x7fc330198dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:39.690 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:39:39.685+0000 7fc334dda640 1 -- 172.21.15.89:0/2840093987 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc33019a800 con 0x7fc330103620 2024-09-21T03:39:39.690 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:39:39.686+0000 7fc334dda640 1 --2- 172.21.15.89:0/2840093987 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc330103620 0x7fc330199430 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fc32000c890 tx=0x7fc32000cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:39:39.690 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:39:39.686+0000 7fc3267fc640 1 -- 172.21.15.89:0/2840093987 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc320004220 con 0x7fc330103620 2024-09-21T03:39:39.690 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:39:39.686+0000 7fc33783f640 1 -- 172.21.15.89:0/2840093987 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc33019aaf0 con 0x7fc330103620 2024-09-21T03:39:39.690 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:39:39.686+0000 7fc3267fc640 1 -- 172.21.15.89:0/2840093987 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc320007e40 con 0x7fc330103620 2024-09-21T03:39:39.690 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:39:39.686+0000 7fc3267fc640 1 -- 172.21.15.89:0/2840093987 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc320004a00 con 0x7fc330103620 2024-09-21T03:39:39.691 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:39:39.686+0000 7fc33783f640 1 -- 172.21.15.89:0/2840093987 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3301a11e0 con 0x7fc330103620 2024-09-21T03:39:39.692 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:39:39.689+0000 7fc33e88a740 1 Processor -- start 2024-09-21T03:39:39.693 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:39:39.689+0000 7fc33e88a740 1 -- start start 2024-09-21T03:39:39.693 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:39:39.689+0000 7fc33e88a740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556b264281b0 0x556b267d2060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:39.693 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:39:39.689+0000 7fc33e88a740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556b263ff2f0 0x556b267d26d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:39.693 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:39:39.689+0000 7fc33e88a740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556b264074d0 0x556b267d3220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:39.693 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:39:39.689+0000 7fc33e88a740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x556b267426f0 con 0x556b264281b0 2024-09-21T03:39:39.693 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:39:39.689+0000 7fc33e88a740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x556b26731b20 con 0x556b263ff2f0 2024-09-21T03:39:39.693 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:39:39.689+0000 7fc33e88a740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x556b26735930 con 0x556b264074d0 2024-09-21T03:39:39.693 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:39:39.689+0000 7fc335ddc640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556b264281b0 0x556b267d2060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:39.693 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:39:39.689+0000 7fc335ddc640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556b264281b0 0x556b267d2060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:46914/0 (socket says 172.21.15.89:46914) 2024-09-21T03:39:39.693 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:39:39.689+0000 7fc335ddc640 1 -- 172.21.15.89:0/1344189477 learned_addr learned my addr 172.21.15.89:0/1344189477 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-21T03:39:39.693 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:39:39.689+0000 7fc3355db640 1 --2- 172.21.15.89:0/1344189477 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556b263ff2f0 0x556b267d26d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:39.693 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:39:39.689+0000 7fc3267fc640 1 -- 172.21.15.89:0/2840093987 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 20) ==== 154563+0+0 (secure 0 0 0) 0x7fc320004ba0 con 0x7fc330103620 2024-09-21T03:39:39.693 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:39:39.690+0000 7fc335ddc640 1 -- 172.21.15.89:0/1344189477 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556b264074d0 msgr2=0x556b267d3220 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T03:39:39.694 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:39:39.690+0000 7fc335ddc640 1 --2- 172.21.15.89:0/1344189477 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556b264074d0 0x556b267d3220 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:39.694 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:39:39.690+0000 7fc335ddc640 1 -- 172.21.15.89:0/1344189477 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556b263ff2f0 msgr2=0x556b267d26d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:39:39.694 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:39:39.690+0000 7fc335ddc640 1 --2- 172.21.15.89:0/1344189477 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556b263ff2f0 0x556b267d26d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:39.694 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:39:39.690+0000 7fc335ddc640 1 -- 172.21.15.89:0/1344189477 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556b266f7530 con 0x556b264281b0 2024-09-21T03:39:39.694 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:39:39.690+0000 7fc335ddc640 1 --2- 172.21.15.89:0/1344189477 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556b264281b0 0x556b267d2060 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fc328009c90 tx=0x7fc3280313f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c85dcc89eaec724 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:39:39.694 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:39:39.690+0000 7fc31b7fe640 1 -- 172.21.15.89:0/1344189477 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc328004a90 con 0x556b264281b0 2024-09-21T03:39:39.694 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:39:39.690+0000 7fc31b7fe640 1 -- 172.21.15.89:0/1344189477 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc328036070 con 0x556b264281b0 2024-09-21T03:39:39.694 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:39:39.690+0000 7fc31b7fe640 1 -- 172.21.15.89:0/1344189477 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc328004050 con 0x556b264281b0 2024-09-21T03:39:39.694 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:39:39.690+0000 7fc3267fc640 1 --2- 172.21.15.89:0/2840093987 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7fc31c0b65a0 0x7fc31c0b8ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:39.694 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:39:39.690+0000 7fc3355db640 1 --2- 172.21.15.89:0/2840093987 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7fc31c0b65a0 0x7fc31c0b8ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:39.694 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:39:39.691+0000 7fc3267fc640 1 -- 172.21.15.89:0/2840093987 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 9121+0+0 (secure 0 0 0) 0x7fc3200e7580 con 0x7fc330103620 2024-09-21T03:39:39.699 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:39:39.691+0000 7fc33e88a740 1 -- 172.21.15.89:0/1344189477 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556b264281b0 msgr2=0x556b267d2060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:39.699 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:39:39.691+0000 7fc33e88a740 1 --2- 172.21.15.89:0/1344189477 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556b264281b0 0x556b267d2060 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fc328009c90 tx=0x7fc3280313f0 comp rx=0 tx=0).stop 2024-09-21T03:39:39.699 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:39:39.691+0000 7fc3355db640 1 --2- 172.21.15.89:0/2840093987 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7fc31c0b65a0 0x7fc31c0b8ab0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fc32c009cf0 tx=0x7fc32c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:39:39.699 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:39:39.691+0000 7fc33e88a740 1 -- 172.21.15.89:0/1344189477 shutdown_connections 2024-09-21T03:39:39.699 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:39:39.691+0000 7fc33e88a740 1 --2- 172.21.15.89:0/1344189477 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556b263ff2f0 0x556b267d26d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:39.699 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:39:39.691+0000 7fc33e88a740 1 --2- 172.21.15.89:0/1344189477 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556b264074d0 0x556b267d3220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:39.699 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:39:39.691+0000 7fc33e88a740 1 --2- 172.21.15.89:0/1344189477 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556b264281b0 0x556b267d2060 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:39.699 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:39:39.691+0000 7fc33e88a740 1 -- 172.21.15.89:0/1344189477 >> 172.21.15.89:0/1344189477 conn(0x556b2641c300 msgr2=0x556b267cb8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:39:39.699 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:39:39.691+0000 7fc33e88a740 1 -- 172.21.15.89:0/1344189477 shutdown_connections 2024-09-21T03:39:39.699 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:39:39.691+0000 7fc33e88a740 1 -- 172.21.15.89:0/1344189477 wait complete. 2024-09-21T03:39:39.699 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:39:39.691+0000 7fc33e88a740 1 -- start start 2024-09-21T03:39:39.699 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:39:39.692+0000 7fc33e88a740 1 Processor -- start 2024-09-21T03:39:39.700 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:39:39.692+0000 7fc33e88a740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556b263ff2f0 0x556b267db830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:39.700 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:39:39.692+0000 7fc33e88a740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556b264074d0 0x556b267dbef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:39.700 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:39:39.692+0000 7fc33e88a740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556b264281b0 0x556b267dca60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:39.700 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:39:39.692+0000 7fc33e88a740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x556b26669ab0 con 0x556b264074d0 2024-09-21T03:39:39.700 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:39:39.692+0000 7fc33e88a740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x556b266d7730 con 0x556b263ff2f0 2024-09-21T03:39:39.700 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:39:39.692+0000 7fc33e88a740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x556b267d3ce0 con 0x556b264281b0 2024-09-21T03:39:39.700 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:39:39.692+0000 7fc33e88a740 10 client.0 _ll_register_callbacks cb 0x556b2672b6b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:39:39.700 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:39:39.692+0000 7fc334dda640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556b264074d0 0x556b267dbef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:39.700 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:39:39.692+0000 7fc334dda640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556b264074d0 0x556b267dbef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:46918/0 (socket says 172.21.15.89:46918) 2024-09-21T03:39:39.700 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:39:39.692+0000 7fc334dda640 1 -- 172.21.15.89:0/604052148 learned_addr learned my addr 172.21.15.89:0/604052148 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-21T03:39:39.700 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:39:39.692+0000 7fc335ddc640 1 --2- 172.21.15.89:0/604052148 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556b264281b0 0x556b267dca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:39.700 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:39:39.692+0000 7fc334dda640 1 -- 172.21.15.89:0/604052148 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556b264281b0 msgr2=0x556b267dca60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:39:39.700 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:39:39.692+0000 7fc3355db640 1 --2- 172.21.15.89:0/604052148 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556b263ff2f0 0x556b267db830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:39.700 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:39:39.692+0000 7fc334dda640 1 --2- 172.21.15.89:0/604052148 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556b264281b0 0x556b267dca60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:39.701 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:39:39.692+0000 7fc334dda640 1 -- 172.21.15.89:0/604052148 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556b263ff2f0 msgr2=0x556b267db830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:39:39.701 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:39:39.692+0000 7fc334dda640 1 --2- 172.21.15.89:0/604052148 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556b263ff2f0 0x556b267db830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:39.701 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:39:39.692+0000 7fc334dda640 1 -- 172.21.15.89:0/604052148 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556b26493790 con 0x556b264074d0 2024-09-21T03:39:39.701 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:39:39.692+0000 7fc335ddc640 1 --2- 172.21.15.89:0/604052148 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556b264281b0 0x556b267dca60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:39:39.701 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:39:39.693+0000 7fc334dda640 1 --2- 172.21.15.89:0/604052148 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556b264074d0 0x556b267dbef0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fc3200e5130 tx=0x7fc320007d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d85ccaa806ad5e8 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:39:39.701 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:39:39.693+0000 7fc318ff9640 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2024-09-21T03:39:39.701 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:39:39.693+0000 7fc318ff9640 1 -- 172.21.15.89:0/604052148 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc32001d040 con 0x556b264074d0 2024-09-21T03:39:39.701 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:39:39.693+0000 7fc318ff9640 1 -- 172.21.15.89:0/604052148 <== mon.0 v2:172.21.15.89:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 9121+0+0 (secure 0 0 0) 0x7fc320004e90 con 0x556b264074d0 2024-09-21T03:39:39.701 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:39:39.693+0000 7fc33e88a740 1 -- 172.21.15.89:0/604052148 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc328036070 con 0x556b264074d0 2024-09-21T03:39:39.701 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:39:39.693+0000 7fc318ff9640 1 -- 172.21.15.89:0/604052148 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fc3200ac9a0 con 0x556b264074d0 2024-09-21T03:39:39.701 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:39:39.693+0000 7fc33e88a740 10 client.15495 fetch_fsmap learned FSMap version 26 2024-09-21T03:39:39.701 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:39:39.693+0000 7fc33e88a740 1 -- 172.21.15.89:0/604052148 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x556b2650c620 con 0x556b264074d0 2024-09-21T03:39:39.701 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:39:39.693+0000 7fc318ff9640 1 -- 172.21.15.89:0/604052148 <== mon.0 v2:172.21.15.89:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc328036070 con 0x556b264074d0 2024-09-21T03:39:39.701 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:39:39.693+0000 7fc318ff9640 1 -- 172.21.15.89:0/604052148 <== mon.0 v2:172.21.15.89:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fc3200d0bb0 con 0x556b264074d0 2024-09-21T03:39:39.702 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:39:39.693+0000 7fc33e88a740 10 client.15495 fetch_fsmap finished waiting for FSMap version 26 2024-09-21T03:39:39.702 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:39:39.693+0000 7fc33e88a740 10 client.15495 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T03:39:39.702 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:39:39.693+0000 7fc33e88a740 20 client.15495 populate_metadata read hostname 'smithi089' 2024-09-21T03:39:39.702 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:39:39.693+0000 7fc33e88a740 1 --2- 172.21.15.89:0/604052148 >> [v2:172.21.15.110:6838/132366709,v1:172.21.15.110:6839/132366709] conn(0x556b263e6d80 0x556b267e8f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:39.702 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:39:39.693+0000 7fc33e88a740 4 client.15495 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:39:39.702 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:39:39.693+0000 7fc33e88a740 1 -- 172.21.15.89:0/604052148 --> [v2:172.21.15.110:6838/132366709,v1:172.21.15.110:6839/132366709] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556b26744640 con 0x556b263e6d80 2024-09-21T03:39:39.702 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:39:39.694+0000 7fc3355db640 1 --2- 172.21.15.89:0/604052148 >> [v2:172.21.15.110:6838/132366709,v1:172.21.15.110:6839/132366709] conn(0x556b263e6d80 0x556b267e8f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:39.702 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:39:39.694+0000 7fc318ff9640 10 client.15495 ms_handle_connect on v2:172.21.15.110:6838/132366709 2024-09-21T03:39:39.702 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:39:39.694+0000 7fc3355db640 1 --2- 172.21.15.89:0/604052148 >> [v2:172.21.15.110:6838/132366709,v1:172.21.15.110:6839/132366709] conn(0x556b263e6d80 0x556b267e8f90 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eaf317fbcf8e6455 server_cookie=b67c03dbb5d579f8 in_seq=0 out_seq=0 2024-09-21T03:39:39.702 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:39:39.695+0000 7fc318ff9640 1 -- 172.21.15.89:0/604052148 <== mds.0 v2:172.21.15.110:6838/132366709 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x556b26744640 con 0x556b263e6d80 2024-09-21T03:39:39.702 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:39:39.695+0000 7fc318ff9640 10 client.15495 handle_command_reply: tid=0 2024-09-21T03:39:39.702 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:39:39.695+0000 7fc33e88a740 1 client.15495 shutdown 2024-09-21T03:39:39.703 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:39:39.695+0000 7fc33e88a740 1 -- 172.21.15.89:0/604052148 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556b264074d0 msgr2=0x556b267dbef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:39.703 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:39:39.695+0000 7fc33e88a740 1 --2- 172.21.15.89:0/604052148 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556b264074d0 0x556b267dbef0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fc3200e5130 tx=0x7fc320007d80 comp rx=0 tx=0).stop 2024-09-21T03:39:39.703 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:39:39.695+0000 7fc33e88a740 1 -- 172.21.15.89:0/604052148 shutdown_connections 2024-09-21T03:39:39.703 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:39:39.695+0000 7fc33e88a740 1 --2- 172.21.15.89:0/604052148 >> [v2:172.21.15.110:6838/132366709,v1:172.21.15.110:6839/132366709] conn(0x556b263e6d80 0x556b267e8f90 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:39.703 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:39:39.695+0000 7fc33e88a740 1 --2- 172.21.15.89:0/604052148 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556b264281b0 0x556b267dca60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:39.703 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:39:39.695+0000 7fc318ff9640 0 client.15495 ms_handle_reset on v2:172.21.15.110:6838/132366709 2024-09-21T03:39:39.703 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:39:39.695+0000 7fc33e88a740 1 --2- 172.21.15.89:0/604052148 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556b264074d0 0x556b267dbef0 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:39.703 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:39:39.695+0000 7fc33e88a740 1 --2- 172.21.15.89:0/604052148 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556b263ff2f0 0x556b267db830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:39.703 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:39:39.695+0000 7fc33e88a740 1 -- 172.21.15.89:0/604052148 >> 172.21.15.89:0/604052148 conn(0x556b2641c300 msgr2=0x556b267c9ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:39:39.703 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:39:39.695+0000 7fc33e88a740 1 -- 172.21.15.89:0/604052148 shutdown_connections 2024-09-21T03:39:39.703 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:39:39.695+0000 7fc33e88a740 1 -- 172.21.15.89:0/604052148 wait complete. 2024-09-21T03:39:39.703 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:39:39.695+0000 7fc33e88a740 20 client.15495 trim_cache size 0 max 16384 2024-09-21T03:39:39.713 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:39:39.710+0000 7fc33e88a740 1 Processor -- start 2024-09-21T03:39:39.713 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:39:39.710+0000 7fc33e88a740 1 -- start start 2024-09-21T03:39:39.713 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:39:39.710+0000 7fc33e88a740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556b263ff2f0 0x556b267c9ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:39.713 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:39:39.710+0000 7fc33e88a740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556b264074d0 0x556b267cb290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:39.713 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:39:39.710+0000 7fc33e88a740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556b264281b0 0x556b267e3940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:39.713 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:39:39.710+0000 7fc33e88a740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fc3301acbf0 con 0x556b263ff2f0 2024-09-21T03:39:39.713 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:39:39.710+0000 7fc33e88a740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fc3301aa4c0 con 0x556b264074d0 2024-09-21T03:39:39.713 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:39:39.710+0000 7fc33e88a740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fc3301a8310 con 0x556b264281b0 2024-09-21T03:39:39.713 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:39:39.710+0000 7fc3355db640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556b263ff2f0 0x556b267c9ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:39.714 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:39:39.710+0000 7fc3355db640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556b263ff2f0 0x556b267c9ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:46926/0 (socket says 172.21.15.89:46926) 2024-09-21T03:39:39.714 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:39:39.710+0000 7fc3355db640 1 -- 172.21.15.89:0/2339195340 learned_addr learned my addr 172.21.15.89:0/2339195340 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-21T03:39:39.714 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:39:39.710+0000 7fc3355db640 1 -- 172.21.15.89:0/2339195340 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556b264281b0 msgr2=0x556b267e3940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:39:39.714 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:39:39.710+0000 7fc3355db640 1 --2- 172.21.15.89:0/2339195340 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556b264281b0 0x556b267e3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:39.714 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:39:39.710+0000 7fc3355db640 1 -- 172.21.15.89:0/2339195340 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556b264074d0 msgr2=0x556b267cb290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:39:39.714 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:39:39.710+0000 7fc3355db640 1 --2- 172.21.15.89:0/2339195340 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556b264074d0 0x556b267cb290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:39.714 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:39:39.710+0000 7fc3355db640 1 -- 172.21.15.89:0/2339195340 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556b267e9300 con 0x556b263ff2f0 2024-09-21T03:39:39.714 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:39:39.710+0000 7fc3355db640 1 --2- 172.21.15.89:0/2339195340 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556b263ff2f0 0x556b267c9ea0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x556b267e96e0 tx=0x7fc32c017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=efd35cfb7bb15b56 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:39:39.714 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:39:39.711+0000 7fc31affd640 1 -- 172.21.15.89:0/2339195340 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc32c02a4e0 con 0x556b263ff2f0 2024-09-21T03:39:39.714 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:39:39.711+0000 7fc31affd640 1 -- 172.21.15.89:0/2339195340 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc32c02ab10 con 0x556b263ff2f0 2024-09-21T03:39:39.714 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:39:39.711+0000 7fc31affd640 1 -- 172.21.15.89:0/2339195340 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc32c011b10 con 0x556b263ff2f0 2024-09-21T03:39:39.718 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:39:39.711+0000 7fc33e88a740 1 -- 172.21.15.89:0/2339195340 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556b263ff2f0 msgr2=0x556b267c9ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:39.718 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:39:39.711+0000 7fc33e88a740 1 --2- 172.21.15.89:0/2339195340 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556b263ff2f0 0x556b267c9ea0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x556b267e96e0 tx=0x7fc32c017000 comp rx=0 tx=0).stop 2024-09-21T03:39:39.718 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:39:39.711+0000 7fc33e88a740 1 -- 172.21.15.89:0/2339195340 shutdown_connections 2024-09-21T03:39:39.718 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:39:39.711+0000 7fc33e88a740 1 --2- 172.21.15.89:0/2339195340 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556b264074d0 0x556b267cb290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:39.718 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:39:39.711+0000 7fc33e88a740 1 --2- 172.21.15.89:0/2339195340 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556b264281b0 0x556b267e3940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:39.718 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:39:39.711+0000 7fc33e88a740 1 --2- 172.21.15.89:0/2339195340 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556b263ff2f0 0x556b267c9ea0 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:39.718 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:39:39.711+0000 7fc33e88a740 1 -- 172.21.15.89:0/2339195340 >> 172.21.15.89:0/2339195340 conn(0x556b2641c300 msgr2=0x556b267d25c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:39:39.718 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:39:39.711+0000 7fc33e88a740 1 -- 172.21.15.89:0/2339195340 shutdown_connections 2024-09-21T03:39:39.718 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:39:39.711+0000 7fc33e88a740 1 -- 172.21.15.89:0/2339195340 wait complete. 2024-09-21T03:39:39.718 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:39:39.712+0000 7fc33e88a740 1 -- start start 2024-09-21T03:39:39.719 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:39:39.712+0000 7fc33e88a740 1 Processor -- start 2024-09-21T03:39:39.719 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:39:39.712+0000 7fc33e88a740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556b263ff2f0 0x556b267deda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:39.719 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:39:39.712+0000 7fc33e88a740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556b264074d0 0x556b267df460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:39.719 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:39:39.712+0000 7fc33e88a740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556b264281b0 0x556b267e3050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:39.719 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:39:39.712+0000 7fc33e88a740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fc3301a8980 con 0x556b264074d0 2024-09-21T03:39:39.719 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:39:39.712+0000 7fc33e88a740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fc3301a9df0 con 0x556b263ff2f0 2024-09-21T03:39:39.719 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:39:39.712+0000 7fc33e88a740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fc3301a9050 con 0x556b264281b0 2024-09-21T03:39:39.719 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:39:39.712+0000 7fc33e88a740 10 client.0 _ll_register_callbacks cb 0x7fc3200046c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:39:39.719 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:39:39.712+0000 7fc334dda640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556b264074d0 0x556b267df460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:39.719 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:39:39.712+0000 7fc334dda640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556b264074d0 0x556b267df460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:46930/0 (socket says 172.21.15.89:46930) 2024-09-21T03:39:39.719 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:39:39.712+0000 7fc334dda640 1 -- 172.21.15.89:0/1077182784 learned_addr learned my addr 172.21.15.89:0/1077182784 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-21T03:39:39.719 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:39:39.712+0000 7fc335ddc640 1 --2- 172.21.15.89:0/1077182784 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556b264281b0 0x556b267e3050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:39.719 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:39:39.712+0000 7fc3355db640 1 --2- 172.21.15.89:0/1077182784 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556b263ff2f0 0x556b267deda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:39.719 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:39:39.713+0000 7fc334dda640 1 -- 172.21.15.89:0/1077182784 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556b264281b0 msgr2=0x556b267e3050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:39:39.720 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:39:39.713+0000 7fc334dda640 1 --2- 172.21.15.89:0/1077182784 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556b264281b0 0x556b267e3050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:39.720 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:39:39.713+0000 7fc334dda640 1 -- 172.21.15.89:0/1077182784 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556b263ff2f0 msgr2=0x556b267deda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:39:39.720 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:39:39.713+0000 7fc334dda640 1 --2- 172.21.15.89:0/1077182784 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556b263ff2f0 0x556b267deda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:39.720 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:39:39.713+0000 7fc334dda640 1 -- 172.21.15.89:0/1077182784 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556b267e37b0 con 0x556b264074d0 2024-09-21T03:39:39.720 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:39:39.713+0000 7fc3355db640 1 --2- 172.21.15.89:0/1077182784 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556b263ff2f0 0x556b267deda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:39:39.720 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:39:39.713+0000 7fc334dda640 1 --2- 172.21.15.89:0/1077182784 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556b264074d0 0x556b267df460 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fc32000a630 tx=0x7fc3200ac370 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b765c83498e185fc server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:39:39.720 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:39:39.713+0000 7fc318ff9640 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2024-09-21T03:39:39.720 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:39:39.713+0000 7fc318ff9640 1 -- 172.21.15.89:0/1077182784 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc32001d040 con 0x556b264074d0 2024-09-21T03:39:39.720 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:39:39.713+0000 7fc318ff9640 1 -- 172.21.15.89:0/1077182784 <== mon.0 v2:172.21.15.89:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 9121+0+0 (secure 0 0 0) 0x7fc3200050a0 con 0x556b264074d0 2024-09-21T03:39:39.720 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:39:39.713+0000 7fc33e88a740 1 -- 172.21.15.89:0/1077182784 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc32c02ab10 con 0x556b264074d0 2024-09-21T03:39:39.720 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:39:39.713+0000 7fc318ff9640 1 -- 172.21.15.89:0/1077182784 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fc3200acd40 con 0x556b264074d0 2024-09-21T03:39:39.720 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:39:39.713+0000 7fc33e88a740 10 client.15501 fetch_fsmap learned FSMap version 26 2024-09-21T03:39:39.720 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:39:39.713+0000 7fc33e88a740 1 -- 172.21.15.89:0/1077182784 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x556b267e0b50 con 0x556b264074d0 2024-09-21T03:39:39.720 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:39:39.713+0000 7fc318ff9640 1 -- 172.21.15.89:0/1077182784 <== mon.0 v2:172.21.15.89:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc32c02ab10 con 0x556b264074d0 2024-09-21T03:39:39.720 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:39:39.713+0000 7fc318ff9640 1 -- 172.21.15.89:0/1077182784 <== mon.0 v2:172.21.15.89:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fc3200b2ea0 con 0x556b264074d0 2024-09-21T03:39:39.720 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:39:39.713+0000 7fc33e88a740 10 client.15501 fetch_fsmap finished waiting for FSMap version 26 2024-09-21T03:39:39.721 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:39:39.713+0000 7fc33e88a740 10 client.15501 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T03:39:39.721 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:39:39.713+0000 7fc33e88a740 20 client.15501 populate_metadata read hostname 'smithi089' 2024-09-21T03:39:39.721 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:39:39.713+0000 7fc33e88a740 1 --2- 172.21.15.89:0/1077182784 >> [v2:172.21.15.110:6838/132366709,v1:172.21.15.110:6839/132366709] conn(0x556b263bc1c0 0x556b26855fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:39.721 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:39:39.713+0000 7fc33e88a740 4 client.15501 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T03:39:39.721 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:39:39.713+0000 7fc33e88a740 1 -- 172.21.15.89:0/1077182784 --> [v2:172.21.15.110:6838/132366709,v1:172.21.15.110:6839/132366709] -- command(tid 0: {"prefix": "flush journal"}) -- 0x556b267400f0 con 0x556b263bc1c0 2024-09-21T03:39:39.721 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:39:39.714+0000 7fc3355db640 1 --2- 172.21.15.89:0/1077182784 >> [v2:172.21.15.110:6838/132366709,v1:172.21.15.110:6839/132366709] conn(0x556b263bc1c0 0x556b26855fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:39.721 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:39:39.714+0000 7fc318ff9640 10 client.15501 ms_handle_connect on v2:172.21.15.110:6838/132366709 2024-09-21T03:39:39.721 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:39:39.714+0000 7fc3355db640 1 --2- 172.21.15.89:0/1077182784 >> [v2:172.21.15.110:6838/132366709,v1:172.21.15.110:6839/132366709] conn(0x556b263bc1c0 0x556b26855fa0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5d942ba1df29e96a server_cookie=70d7dae49369ef8b in_seq=0 out_seq=0 2024-09-21T03:39:39.730 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:39:39.727+0000 7fc318ff9640 1 -- 172.21.15.89:0/1077182784 <== mds.0 v2:172.21.15.110:6838/132366709 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x556b267400f0 con 0x556b263bc1c0 2024-09-21T03:39:39.730 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:39:39.727+0000 7fc318ff9640 10 client.15501 handle_command_reply: tid=0 2024-09-21T03:39:39.730 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:39:39.727+0000 7fc33e88a740 1 client.15501 shutdown 2024-09-21T03:39:39.730 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:39:39.728+0000 7fc33e88a740 1 -- 172.21.15.89:0/1077182784 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556b264074d0 msgr2=0x556b267df460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:39.730 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:39:39.728+0000 7fc33e88a740 1 --2- 172.21.15.89:0/1077182784 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556b264074d0 0x556b267df460 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fc32000a630 tx=0x7fc3200ac370 comp rx=0 tx=0).stop 2024-09-21T03:39:39.731 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:39:39.728+0000 7fc33e88a740 1 -- 172.21.15.89:0/1077182784 shutdown_connections 2024-09-21T03:39:39.731 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:39:39.728+0000 7fc33e88a740 1 --2- 172.21.15.89:0/1077182784 >> [v2:172.21.15.110:6838/132366709,v1:172.21.15.110:6839/132366709] conn(0x556b263bc1c0 0x556b26855fa0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:39.731 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:39:39.728+0000 7fc33e88a740 1 --2- 172.21.15.89:0/1077182784 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556b264281b0 0x556b267e3050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:39.731 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:39:39.728+0000 7fc318ff9640 0 client.15501 ms_handle_reset on v2:172.21.15.110:6838/132366709 2024-09-21T03:39:39.731 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:39:39.728+0000 7fc33e88a740 1 --2- 172.21.15.89:0/1077182784 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556b264074d0 0x556b267df460 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:39.731 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:39:39.728+0000 7fc33e88a740 1 --2- 172.21.15.89:0/1077182784 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556b263ff2f0 0x556b267deda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:39.731 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:39:39.728+0000 7fc33e88a740 1 -- 172.21.15.89:0/1077182784 >> 172.21.15.89:0/1077182784 conn(0x556b2641c300 msgr2=0x556b267d25c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:39:39.731 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:39:39.728+0000 7fc33e88a740 1 -- 172.21.15.89:0/1077182784 shutdown_connections 2024-09-21T03:39:39.731 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:39:39.728+0000 7fc33e88a740 1 -- 172.21.15.89:0/1077182784 wait complete. 2024-09-21T03:39:39.731 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:39:39.728+0000 7fc33e88a740 20 client.15501 trim_cache size 0 max 16384 2024-09-21T03:39:39.732 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stdout:{ 2024-09-21T03:39:39.732 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stdout: "message": "", 2024-09-21T03:39:39.732 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stdout: "return_code": 0 2024-09-21T03:39:39.732 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stdout:} 2024-09-21T03:39:39.732 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:39:39.728+0000 7fc33783f640 1 -- 172.21.15.89:0/2840093987 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7fc31c0b65a0 msgr2=0x7fc31c0b8ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:39.733 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:39:39.728+0000 7fc33783f640 1 --2- 172.21.15.89:0/2840093987 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7fc31c0b65a0 0x7fc31c0b8ab0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fc32c009cf0 tx=0x7fc32c005f90 comp rx=0 tx=0).stop 2024-09-21T03:39:39.733 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:39:39.728+0000 7fc33783f640 1 -- 172.21.15.89:0/2840093987 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc330103620 msgr2=0x7fc330199430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:39.733 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:39:39.728+0000 7fc33783f640 1 --2- 172.21.15.89:0/2840093987 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc330103620 0x7fc330199430 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fc32000c890 tx=0x7fc32000cd50 comp rx=0 tx=0).stop 2024-09-21T03:39:39.733 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:39:39.729+0000 7fc33783f640 1 -- 172.21.15.89:0/2840093987 shutdown_connections 2024-09-21T03:39:39.733 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:39:39.729+0000 7fc33783f640 1 --2- 172.21.15.89:0/2840093987 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7fc31c0b65a0 0x7fc31c0b8ab0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:39.733 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:39:39.729+0000 7fc33783f640 1 --2- 172.21.15.89:0/2840093987 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc330103fc0 0x7fc330199fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:39.733 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:39:39.729+0000 7fc33783f640 1 --2- 172.21.15.89:0/2840093987 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc330103620 0x7fc330199430 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:39.733 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:39:39.729+0000 7fc33783f640 1 --2- 172.21.15.89:0/2840093987 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc3301025a0 0x7fc330198dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:39.733 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:39:39.729+0000 7fc33783f640 1 -- 172.21.15.89:0/2840093987 >> 172.21.15.89:0/2840093987 conn(0x7fc3300fdee0 msgr2=0x7fc330106940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:39:39.733 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:39:39.729+0000 7fc33783f640 1 -- 172.21.15.89:0/2840093987 shutdown_connections 2024-09-21T03:39:39.733 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:39:39.729+0000 7fc33783f640 1 -- 172.21.15.89:0/2840093987 wait complete. 2024-09-21T03:39:40.125 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:40.121+0000 7f2a2597f640 1 -- 172.21.15.89:0/1035326504 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2a20105380 msgr2=0x7f2a2010d8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:40.125 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:40.121+0000 7f2a2597f640 1 --2- 172.21.15.89:0/1035326504 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2a20105380 0x7f2a2010d8d0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f2a14009900 tx=0x7f2a1402a410 comp rx=0 tx=0).stop 2024-09-21T03:39:40.126 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:40.121+0000 7f2a2597f640 1 -- 172.21.15.89:0/1035326504 shutdown_connections 2024-09-21T03:39:40.126 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:40.121+0000 7f2a2597f640 1 --2- 172.21.15.89:0/1035326504 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2a2010de60 0x7f2a20110330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:40.126 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:40.121+0000 7f2a2597f640 1 --2- 172.21.15.89:0/1035326504 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2a20105380 0x7f2a2010d8d0 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:40.126 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:40.121+0000 7f2a2597f640 1 --2- 172.21.15.89:0/1035326504 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2a20104a30 0x7f2a20104e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:40.126 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:40.121+0000 7f2a2597f640 1 -- 172.21.15.89:0/1035326504 >> 172.21.15.89:0/1035326504 conn(0x7f2a20079790 msgr2=0x7f2a20079ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:39:40.126 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:40.121+0000 7f2a2597f640 1 -- 172.21.15.89:0/1035326504 shutdown_connections 2024-09-21T03:39:40.126 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:40.121+0000 7f2a2597f640 1 -- 172.21.15.89:0/1035326504 wait complete. 2024-09-21T03:39:40.126 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:40.122+0000 7f2a2597f640 1 Processor -- start 2024-09-21T03:39:40.126 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:40.122+0000 7f2a2597f640 1 -- start start 2024-09-21T03:39:40.126 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:40.122+0000 7f2a2597f640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2a20104a30 0x7f2a2019a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:40.126 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:40.122+0000 7f2a2597f640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2a20105380 0x7f2a2019aa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:40.126 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:40.122+0000 7f2a2597f640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2a2010de60 0x7f2a2019fca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:40.126 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:40.122+0000 7f2a2597f640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f2a20113990 con 0x7f2a20104a30 2024-09-21T03:39:40.126 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:40.122+0000 7f2a2597f640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f2a20113810 con 0x7f2a2010de60 2024-09-21T03:39:40.127 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:40.122+0000 7f2a2597f640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f2a20113b10 con 0x7f2a20105380 2024-09-21T03:39:40.127 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:40.122+0000 7f2a1effd640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2a20104a30 0x7f2a2019a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:40.127 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:40.123+0000 7f2a1effd640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2a20104a30 0x7f2a2019a460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:46942/0 (socket says 172.21.15.89:46942) 2024-09-21T03:39:40.127 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:40.123+0000 7f2a1effd640 1 -- 172.21.15.89:0/736028766 learned_addr learned my addr 172.21.15.89:0/736028766 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-21T03:39:40.127 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:40.123+0000 7f2a1e7fc640 1 --2- 172.21.15.89:0/736028766 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2a20105380 0x7f2a2019aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:40.127 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:40.123+0000 7f2a1f7fe640 1 --2- 172.21.15.89:0/736028766 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2a2010de60 0x7f2a2019fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:40.127 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:40.123+0000 7f2a1effd640 1 -- 172.21.15.89:0/736028766 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2a20105380 msgr2=0x7f2a2019aa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:40.127 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:40.123+0000 7f2a1effd640 1 --2- 172.21.15.89:0/736028766 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2a20105380 0x7f2a2019aa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:40.127 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:40.123+0000 7f2a1effd640 1 -- 172.21.15.89:0/736028766 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2a2010de60 msgr2=0x7f2a2019fca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:40.127 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:40.123+0000 7f2a1effd640 1 --2- 172.21.15.89:0/736028766 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2a2010de60 0x7f2a2019fca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:40.127 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:40.123+0000 7f2a1effd640 1 -- 172.21.15.89:0/736028766 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a201a03d0 con 0x7f2a20104a30 2024-09-21T03:39:40.127 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:40.123+0000 7f2a1effd640 1 --2- 172.21.15.89:0/736028766 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2a20104a30 0x7f2a2019a460 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f2a1000b4a0 tx=0x7f2a1000b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:39:40.130 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:40.124+0000 7f2a2497d640 1 -- 172.21.15.89:0/736028766 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a10014070 con 0x7f2a20104a30 2024-09-21T03:39:40.130 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:40.124+0000 7f2a2597f640 1 -- 172.21.15.89:0/736028766 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2a201a06c0 con 0x7f2a20104a30 2024-09-21T03:39:40.130 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:40.124+0000 7f2a2497d640 1 -- 172.21.15.89:0/736028766 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2a10010aa0 con 0x7f2a20104a30 2024-09-21T03:39:40.130 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:40.124+0000 7f2a2497d640 1 -- 172.21.15.89:0/736028766 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a10017b70 con 0x7f2a20104a30 2024-09-21T03:39:40.130 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:40.124+0000 7f2a2597f640 1 -- 172.21.15.89:0/736028766 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2a20102650 con 0x7f2a20104a30 2024-09-21T03:39:40.130 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:40.126+0000 7f2a2bef3740 1 Processor -- start 2024-09-21T03:39:40.130 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:40.126+0000 7f2a2bef3740 1 -- start start 2024-09-21T03:39:40.130 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:40.126+0000 7f2a2497d640 1 -- 172.21.15.89:0/736028766 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 20) ==== 154563+0+0 (secure 0 0 0) 0x7f2a10017070 con 0x7f2a20104a30 2024-09-21T03:39:40.130 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:40.126+0000 7f2a2bef3740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5649a2057f60 0x5649a23fdf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:40.130 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:40.126+0000 7f2a2bef3740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5649a202f0a0 0x5649a23fe440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:40.130 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:40.126+0000 7f2a2bef3740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5649a2037280 0x5649a23fe980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:40.130 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:40.127+0000 7f2a2bef3740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x5649a2375c10 con 0x5649a2037280 2024-09-21T03:39:40.131 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:40.127+0000 7f2a2bef3740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x5649a2365040 con 0x5649a2057f60 2024-09-21T03:39:40.131 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:40.127+0000 7f2a2bef3740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x5649a2368e50 con 0x5649a202f0a0 2024-09-21T03:39:40.131 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:40.127+0000 7f2a1e7fc640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5649a2037280 0x5649a23fe980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:40.131 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:40.127+0000 7f2a1e7fc640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5649a2037280 0x5649a23fe980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:46950/0 (socket says 172.21.15.89:46950) 2024-09-21T03:39:40.131 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:40.127+0000 7f2a1e7fc640 1 -- 172.21.15.89:0/1744889397 learned_addr learned my addr 172.21.15.89:0/1744889397 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-21T03:39:40.131 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:40.127+0000 7f2a1effd640 1 --2- 172.21.15.89:0/1744889397 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5649a202f0a0 0x5649a23fe440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:40.131 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:40.127+0000 7f2a1f7fe640 1 --2- 172.21.15.89:0/1744889397 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5649a2057f60 0x5649a23fdf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:40.131 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:40.127+0000 7f2a1e7fc640 1 -- 172.21.15.89:0/1744889397 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5649a202f0a0 msgr2=0x5649a23fe440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:39:40.131 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:40.127+0000 7f2a1e7fc640 1 --2- 172.21.15.89:0/1744889397 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5649a202f0a0 0x5649a23fe440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:40.131 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:40.127+0000 7f2a1e7fc640 1 -- 172.21.15.89:0/1744889397 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5649a2057f60 msgr2=0x5649a23fdf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:39:40.131 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:40.127+0000 7f2a1e7fc640 1 --2- 172.21.15.89:0/1744889397 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5649a2057f60 0x5649a23fdf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:40.131 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:40.127+0000 7f2a1e7fc640 1 -- 172.21.15.89:0/1744889397 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5649a20c3540 con 0x5649a2037280 2024-09-21T03:39:40.131 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:40.127+0000 7f2a1e7fc640 1 --2- 172.21.15.89:0/1744889397 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5649a2037280 0x5649a23fe980 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f2a14000c00 tx=0x7f2a14031dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0faad43cf225253 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:39:40.132 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:40.127+0000 7f2a2497d640 1 --2- 172.21.15.89:0/736028766 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f2a080b65a0 0x7f2a080b8ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:40.132 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:40.127+0000 7f2a05ffb640 1 -- 172.21.15.89:0/1744889397 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a14035030 con 0x5649a2037280 2024-09-21T03:39:40.132 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:40.128+0000 7f2a05ffb640 1 -- 172.21.15.89:0/1744889397 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2a140044c0 con 0x5649a2037280 2024-09-21T03:39:40.132 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:40.128+0000 7f2a05ffb640 1 -- 172.21.15.89:0/1744889397 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a14003ae0 con 0x5649a2037280 2024-09-21T03:39:40.132 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:40.128+0000 7f2a1f7fe640 1 --2- 172.21.15.89:0/736028766 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f2a080b65a0 0x7f2a080b8ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:40.132 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:40.128+0000 7f2a2497d640 1 -- 172.21.15.89:0/736028766 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 9121+0+0 (secure 0 0 0) 0x7f2a100e6ee0 con 0x7f2a20104a30 2024-09-21T03:39:40.135 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:40.128+0000 7f2a2bef3740 1 -- 172.21.15.89:0/1744889397 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5649a2037280 msgr2=0x5649a23fe980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:40.135 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:40.128+0000 7f2a1f7fe640 1 --2- 172.21.15.89:0/736028766 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f2a080b65a0 0x7f2a080b8ab0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f2a1800be30 tx=0x7f2a1800b580 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:39:40.135 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:40.128+0000 7f2a2bef3740 1 --2- 172.21.15.89:0/1744889397 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5649a2037280 0x5649a23fe980 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f2a14000c00 tx=0x7f2a14031dc0 comp rx=0 tx=0).stop 2024-09-21T03:39:40.136 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:40.128+0000 7f2a2bef3740 1 -- 172.21.15.89:0/1744889397 shutdown_connections 2024-09-21T03:39:40.136 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:40.128+0000 7f2a2bef3740 1 --2- 172.21.15.89:0/1744889397 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5649a2037280 0x5649a23fe980 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:40.136 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:40.128+0000 7f2a2bef3740 1 --2- 172.21.15.89:0/1744889397 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5649a202f0a0 0x5649a23fe440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:40.136 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:40.128+0000 7f2a2bef3740 1 --2- 172.21.15.89:0/1744889397 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5649a2057f60 0x5649a23fdf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:40.136 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:40.128+0000 7f2a2bef3740 1 -- 172.21.15.89:0/1744889397 >> 172.21.15.89:0/1744889397 conn(0x5649a204c350 msgr2=0x5649a23fbdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:39:40.136 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:40.129+0000 7f2a2bef3740 1 -- 172.21.15.89:0/1744889397 shutdown_connections 2024-09-21T03:39:40.136 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:40.129+0000 7f2a2bef3740 1 -- 172.21.15.89:0/1744889397 wait complete. 2024-09-21T03:39:40.136 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:40.129+0000 7f2a2bef3740 1 -- start start 2024-09-21T03:39:40.136 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:40.129+0000 7f2a2bef3740 1 Processor -- start 2024-09-21T03:39:40.136 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:40.129+0000 7f2a2bef3740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5649a202f0a0 0x5649a2409990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:40.136 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:40.129+0000 7f2a2bef3740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5649a2037280 0x5649a2409ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:40.136 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:40.130+0000 7f2a2bef3740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5649a2057f60 0x5649a24159a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:40.136 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:40.130+0000 7f2a2bef3740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x5649a21c65f0 con 0x5649a2057f60 2024-09-21T03:39:40.137 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:40.130+0000 7f2a2bef3740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x5649a2300010 con 0x5649a2037280 2024-09-21T03:39:40.137 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:40.130+0000 7f2a2bef3740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x5649a240c680 con 0x5649a202f0a0 2024-09-21T03:39:40.137 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:40.130+0000 7f2a2bef3740 10 client.0 _ll_register_callbacks cb 0x5649a235f340 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:39:40.137 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:40.130+0000 7f2a1f7fe640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5649a2057f60 0x5649a24159a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:40.137 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:40.130+0000 7f2a1f7fe640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5649a2057f60 0x5649a24159a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:46962/0 (socket says 172.21.15.89:46962) 2024-09-21T03:39:40.137 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:40.130+0000 7f2a1f7fe640 1 -- 172.21.15.89:0/3705794404 learned_addr learned my addr 172.21.15.89:0/3705794404 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-21T03:39:40.137 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:40.130+0000 7f2a1e7fc640 1 --2- 172.21.15.89:0/3705794404 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5649a2037280 0x5649a2409ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:40.137 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:40.130+0000 7f2a1effd640 1 --2- 172.21.15.89:0/3705794404 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5649a202f0a0 0x5649a2409990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:40.137 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:40.130+0000 7f2a1f7fe640 1 -- 172.21.15.89:0/3705794404 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5649a202f0a0 msgr2=0x5649a2409990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:39:40.137 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:40.130+0000 7f2a1f7fe640 1 --2- 172.21.15.89:0/3705794404 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5649a202f0a0 0x5649a2409990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:40.137 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:40.130+0000 7f2a1f7fe640 1 -- 172.21.15.89:0/3705794404 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5649a2037280 msgr2=0x5649a2409ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:39:40.137 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:40.130+0000 7f2a1f7fe640 1 --2- 172.21.15.89:0/3705794404 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5649a2037280 0x5649a2409ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:40.137 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:40.130+0000 7f2a1f7fe640 1 -- 172.21.15.89:0/3705794404 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5649a232da00 con 0x5649a2057f60 2024-09-21T03:39:40.137 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:40.130+0000 7f2a1f7fe640 1 --2- 172.21.15.89:0/3705794404 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5649a2057f60 0x5649a24159a0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f2a1800eba0 tx=0x7f2a18004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b1b9d0317549b62 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:39:40.138 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:40.130+0000 7f29feffd640 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2024-09-21T03:39:40.138 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:40.130+0000 7f29feffd640 1 -- 172.21.15.89:0/3705794404 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a1800de30 con 0x5649a2057f60 2024-09-21T03:39:40.138 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:40.130+0000 7f29feffd640 1 -- 172.21.15.89:0/3705794404 <== mon.0 v2:172.21.15.89:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 9121+0+0 (secure 0 0 0) 0x7f2a18025400 con 0x5649a2057f60 2024-09-21T03:39:40.138 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:40.130+0000 7f2a2bef3740 1 -- 172.21.15.89:0/3705794404 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5649a2377b60 con 0x5649a2057f60 2024-09-21T03:39:40.138 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:40.131+0000 7f29feffd640 1 -- 172.21.15.89:0/3705794404 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f2a1802e050 con 0x5649a2057f60 2024-09-21T03:39:40.138 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:40.131+0000 7f2a2bef3740 10 client.15513 fetch_fsmap learned FSMap version 26 2024-09-21T03:39:40.138 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:40.131+0000 7f2a2bef3740 1 -- 172.21.15.89:0/3705794404 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5649a2418600 con 0x5649a2057f60 2024-09-21T03:39:40.138 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:40.131+0000 7f29feffd640 1 -- 172.21.15.89:0/3705794404 <== mon.0 v2:172.21.15.89:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x5649a2377b60 con 0x5649a2057f60 2024-09-21T03:39:40.138 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:40.131+0000 7f29feffd640 1 -- 172.21.15.89:0/3705794404 <== mon.0 v2:172.21.15.89:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f2a1801b420 con 0x5649a2057f60 2024-09-21T03:39:40.138 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:40.131+0000 7f2a2bef3740 10 client.15513 fetch_fsmap finished waiting for FSMap version 26 2024-09-21T03:39:40.138 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:40.131+0000 7f2a2bef3740 10 client.15513 resolve_mds: resolved name 'h' to daemon mds.h 2024-09-21T03:39:40.138 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:40.131+0000 7f2a2bef3740 20 client.15513 populate_metadata read hostname 'smithi089' 2024-09-21T03:39:40.138 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:40.131+0000 7f2a2bef3740 1 --2- 172.21.15.89:0/3705794404 >> [v2:172.21.15.110:6836/4047124589,v1:172.21.15.110:6837/4047124589] conn(0x5649a20168e0 0x5649a241c0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:40.138 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:40.131+0000 7f2a2bef3740 4 client.15513 mds_command: new command op to 24614 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:39:40.139 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:40.131+0000 7f2a2bef3740 1 -- 172.21.15.89:0/3705794404 --> [v2:172.21.15.110:6836/4047124589,v1:172.21.15.110:6837/4047124589] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5649a2373610 con 0x5649a20168e0 2024-09-21T03:39:40.139 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:40.131+0000 7f2a1effd640 1 --2- 172.21.15.89:0/3705794404 >> [v2:172.21.15.110:6836/4047124589,v1:172.21.15.110:6837/4047124589] conn(0x5649a20168e0 0x5649a241c0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:40.139 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:40.132+0000 7f29feffd640 10 client.15513 ms_handle_connect on v2:172.21.15.110:6836/4047124589 2024-09-21T03:39:40.139 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:40.132+0000 7f2a1effd640 1 --2- 172.21.15.89:0/3705794404 >> [v2:172.21.15.110:6836/4047124589,v1:172.21.15.110:6837/4047124589] conn(0x5649a20168e0 0x5649a241c0e0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=7a1dfa92a3e56d00 server_cookie=ff3e393f1f4be2cf in_seq=0 out_seq=0 2024-09-21T03:39:40.139 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:40.132+0000 7f29feffd640 1 -- 172.21.15.89:0/3705794404 <== mds.2 v2:172.21.15.110:6836/4047124589 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5649a2373610 con 0x5649a20168e0 2024-09-21T03:39:40.139 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:40.132+0000 7f29feffd640 10 client.15513 handle_command_reply: tid=0 2024-09-21T03:39:40.139 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:40.132+0000 7f2a2bef3740 1 client.15513 shutdown 2024-09-21T03:39:40.139 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:40.133+0000 7f2a2bef3740 1 -- 172.21.15.89:0/3705794404 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5649a2057f60 msgr2=0x5649a24159a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:40.139 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:40.133+0000 7f2a2bef3740 1 --2- 172.21.15.89:0/3705794404 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5649a2057f60 0x5649a24159a0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f2a1800eba0 tx=0x7f2a18004a40 comp rx=0 tx=0).stop 2024-09-21T03:39:40.139 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:40.133+0000 7f2a2bef3740 1 -- 172.21.15.89:0/3705794404 shutdown_connections 2024-09-21T03:39:40.139 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:40.133+0000 7f2a2bef3740 1 --2- 172.21.15.89:0/3705794404 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5649a2037280 0x5649a2409ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:40.139 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:40.133+0000 7f2a2bef3740 1 --2- 172.21.15.89:0/3705794404 >> [v2:172.21.15.110:6836/4047124589,v1:172.21.15.110:6837/4047124589] conn(0x5649a20168e0 0x5649a241c0e0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:40.139 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:40.133+0000 7f2a2bef3740 1 --2- 172.21.15.89:0/3705794404 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5649a2057f60 0x5649a24159a0 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:40.139 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:40.133+0000 7f29feffd640 0 client.15513 ms_handle_reset on v2:172.21.15.110:6836/4047124589 2024-09-21T03:39:40.140 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:40.133+0000 7f2a2bef3740 1 --2- 172.21.15.89:0/3705794404 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5649a202f0a0 0x5649a2409990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:40.140 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:40.133+0000 7f2a2bef3740 1 -- 172.21.15.89:0/3705794404 >> 172.21.15.89:0/3705794404 conn(0x5649a204c350 msgr2=0x5649a2401090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:39:40.140 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:40.133+0000 7f2a2bef3740 1 -- 172.21.15.89:0/3705794404 shutdown_connections 2024-09-21T03:39:40.140 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:40.133+0000 7f2a2bef3740 1 -- 172.21.15.89:0/3705794404 wait complete. 2024-09-21T03:39:40.140 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:40.133+0000 7f2a2bef3740 20 client.15513 trim_cache size 0 max 16384 2024-09-21T03:39:40.154 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:40.151+0000 7f2a2bef3740 1 Processor -- start 2024-09-21T03:39:40.154 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:40.151+0000 7f2a2bef3740 1 -- start start 2024-09-21T03:39:40.154 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:40.151+0000 7f2a2bef3740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5649a20168e0 0x5649a2404020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:40.154 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:40.151+0000 7f2a2bef3740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5649a202f0a0 0x5649a23faac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:40.154 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:40.151+0000 7f2a2bef3740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5649a2037280 0x5649a2415910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:40.154 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:40.151+0000 7f2a2bef3740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x5649a241a420 con 0x5649a2037280 2024-09-21T03:39:40.154 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:40.151+0000 7f2a2bef3740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x5649a241aaf0 con 0x5649a20168e0 2024-09-21T03:39:40.154 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:40.151+0000 7f2a2bef3740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x5649a241b1c0 con 0x5649a202f0a0 2024-09-21T03:39:40.155 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:40.151+0000 7f2a1f7fe640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5649a2037280 0x5649a2415910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:40.155 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:40.151+0000 7f2a1f7fe640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5649a2037280 0x5649a2415910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:46964/0 (socket says 172.21.15.89:46964) 2024-09-21T03:39:40.155 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:40.151+0000 7f2a1f7fe640 1 -- 172.21.15.89:0/143705123 learned_addr learned my addr 172.21.15.89:0/143705123 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-21T03:39:40.155 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:40.151+0000 7f2a1e7fc640 1 --2- 172.21.15.89:0/143705123 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5649a202f0a0 0x5649a23faac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:40.155 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:40.151+0000 7f2a1effd640 1 --2- 172.21.15.89:0/143705123 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5649a20168e0 0x5649a2404020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:40.155 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:40.151+0000 7f2a1f7fe640 1 -- 172.21.15.89:0/143705123 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5649a202f0a0 msgr2=0x5649a23faac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:39:40.155 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:40.151+0000 7f2a1f7fe640 1 --2- 172.21.15.89:0/143705123 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5649a202f0a0 0x5649a23faac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:40.155 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:40.151+0000 7f2a1f7fe640 1 -- 172.21.15.89:0/143705123 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5649a20168e0 msgr2=0x5649a2404020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:39:40.155 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:40.151+0000 7f2a1f7fe640 1 --2- 172.21.15.89:0/143705123 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5649a20168e0 0x5649a2404020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:40.155 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:40.151+0000 7f2a1f7fe640 1 -- 172.21.15.89:0/143705123 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5649a241bd40 con 0x5649a2037280 2024-09-21T03:39:40.155 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:40.152+0000 7f2a1f7fe640 1 --2- 172.21.15.89:0/143705123 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5649a2037280 0x5649a2415910 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f2a18010d30 tx=0x7f2a18010de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58d06c6fd3be0a13 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:39:40.155 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:40.152+0000 7f2a057fa640 1 -- 172.21.15.89:0/143705123 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a18037020 con 0x5649a2037280 2024-09-21T03:39:40.155 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:40.152+0000 7f2a057fa640 1 -- 172.21.15.89:0/143705123 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2a1800b5f0 con 0x5649a2037280 2024-09-21T03:39:40.155 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:40.152+0000 7f2a057fa640 1 -- 172.21.15.89:0/143705123 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a18033870 con 0x5649a2037280 2024-09-21T03:39:40.159 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:40.152+0000 7f2a2bef3740 1 -- 172.21.15.89:0/143705123 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5649a2037280 msgr2=0x5649a2415910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:40.159 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:40.152+0000 7f2a2bef3740 1 --2- 172.21.15.89:0/143705123 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5649a2037280 0x5649a2415910 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f2a18010d30 tx=0x7f2a18010de0 comp rx=0 tx=0).stop 2024-09-21T03:39:40.159 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:40.153+0000 7f2a2bef3740 1 -- 172.21.15.89:0/143705123 shutdown_connections 2024-09-21T03:39:40.160 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:40.153+0000 7f2a2bef3740 1 --2- 172.21.15.89:0/143705123 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5649a2037280 0x5649a2415910 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:40.160 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:40.153+0000 7f2a2bef3740 1 --2- 172.21.15.89:0/143705123 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5649a202f0a0 0x5649a23faac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:40.160 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:40.153+0000 7f2a2bef3740 1 --2- 172.21.15.89:0/143705123 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5649a20168e0 0x5649a2404020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:40.160 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:40.153+0000 7f2a2bef3740 1 -- 172.21.15.89:0/143705123 >> 172.21.15.89:0/143705123 conn(0x5649a204c350 msgr2=0x5649a23fa3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:39:40.160 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:40.153+0000 7f2a2bef3740 1 -- 172.21.15.89:0/143705123 shutdown_connections 2024-09-21T03:39:40.160 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:40.153+0000 7f2a2bef3740 1 -- 172.21.15.89:0/143705123 wait complete. 2024-09-21T03:39:40.160 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:40.153+0000 7f2a2bef3740 1 -- start start 2024-09-21T03:39:40.160 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:40.153+0000 7f2a2bef3740 1 Processor -- start 2024-09-21T03:39:40.160 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:40.154+0000 7f2a2bef3740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5649a20168e0 0x5649a241eb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:40.160 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:40.154+0000 7f2a2bef3740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5649a202f0a0 0x5649a241f130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:40.160 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:40.154+0000 7f2a2bef3740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5649a2037280 0x5649a2424120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:40.160 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:40.154+0000 7f2a2bef3740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x5649a241bf60 con 0x5649a202f0a0 2024-09-21T03:39:40.160 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:40.154+0000 7f2a2bef3740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x5649a241b890 con 0x5649a20168e0 2024-09-21T03:39:40.161 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:40.154+0000 7f2a2bef3740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x5649a241c630 con 0x5649a2037280 2024-09-21T03:39:40.161 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:40.154+0000 7f2a2bef3740 10 client.0 _ll_register_callbacks cb 0x7f2a1800e820 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:39:40.161 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:40.154+0000 7f2a1e7fc640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5649a202f0a0 0x5649a241f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:40.161 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:40.154+0000 7f2a1e7fc640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5649a202f0a0 0x5649a241f130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:46968/0 (socket says 172.21.15.89:46968) 2024-09-21T03:39:40.161 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:40.154+0000 7f2a1e7fc640 1 -- 172.21.15.89:0/2597249742 learned_addr learned my addr 172.21.15.89:0/2597249742 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-21T03:39:40.161 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:40.154+0000 7f2a1f7fe640 1 --2- 172.21.15.89:0/2597249742 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5649a2037280 0x5649a2424120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:40.161 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:40.154+0000 7f2a1effd640 1 --2- 172.21.15.89:0/2597249742 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5649a20168e0 0x5649a241eb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:40.161 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:40.154+0000 7f2a1e7fc640 1 -- 172.21.15.89:0/2597249742 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5649a2037280 msgr2=0x5649a2424120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:39:40.161 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:40.154+0000 7f2a1e7fc640 1 --2- 172.21.15.89:0/2597249742 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5649a2037280 0x5649a2424120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:40.161 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:40.154+0000 7f2a1e7fc640 1 -- 172.21.15.89:0/2597249742 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5649a20168e0 msgr2=0x5649a241eb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:39:40.161 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:40.154+0000 7f2a1e7fc640 1 --2- 172.21.15.89:0/2597249742 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5649a20168e0 0x5649a241eb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:40.161 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:40.154+0000 7f2a1e7fc640 1 -- 172.21.15.89:0/2597249742 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5649a2424760 con 0x5649a202f0a0 2024-09-21T03:39:40.161 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:40.154+0000 7f2a1f7fe640 1 --2- 172.21.15.89:0/2597249742 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5649a2037280 0x5649a2424120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:39:40.161 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:40.154+0000 7f2a1e7fc640 1 --2- 172.21.15.89:0/2597249742 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5649a202f0a0 0x5649a241f130 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f2a14009900 tx=0x7f2a14031630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ec73902b7c52038 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:39:40.161 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:40.154+0000 7f29feffd640 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2024-09-21T03:39:40.162 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:40.154+0000 7f29feffd640 1 -- 172.21.15.89:0/2597249742 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a14035040 con 0x5649a202f0a0 2024-09-21T03:39:40.162 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:40.154+0000 7f29feffd640 1 -- 172.21.15.89:0/2597249742 <== mon.0 v2:172.21.15.89:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 9121+0+0 (secure 0 0 0) 0x7f2a1403c640 con 0x5649a202f0a0 2024-09-21T03:39:40.162 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:40.154+0000 7f2a2bef3740 1 -- 172.21.15.89:0/2597249742 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2a1800b5f0 con 0x5649a202f0a0 2024-09-21T03:39:40.162 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:40.155+0000 7f29feffd640 1 -- 172.21.15.89:0/2597249742 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f2a1403c840 con 0x5649a202f0a0 2024-09-21T03:39:40.162 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:40.155+0000 7f2a2bef3740 10 client.15519 fetch_fsmap learned FSMap version 26 2024-09-21T03:39:40.162 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:40.155+0000 7f2a2bef3740 1 -- 172.21.15.89:0/2597249742 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5649a2487e70 con 0x5649a202f0a0 2024-09-21T03:39:40.162 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:40.155+0000 7f29feffd640 1 -- 172.21.15.89:0/2597249742 <== mon.0 v2:172.21.15.89:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2a1800b5f0 con 0x5649a202f0a0 2024-09-21T03:39:40.162 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:40.155+0000 7f29feffd640 1 -- 172.21.15.89:0/2597249742 <== mon.0 v2:172.21.15.89:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f2a1404fa00 con 0x5649a202f0a0 2024-09-21T03:39:40.162 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:40.155+0000 7f2a2bef3740 10 client.15519 fetch_fsmap finished waiting for FSMap version 26 2024-09-21T03:39:40.162 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:40.155+0000 7f2a2bef3740 10 client.15519 resolve_mds: resolved name 'h' to daemon mds.h 2024-09-21T03:39:40.162 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:40.155+0000 7f2a2bef3740 20 client.15519 populate_metadata read hostname 'smithi089' 2024-09-21T03:39:40.162 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:40.155+0000 7f2a2bef3740 1 --2- 172.21.15.89:0/2597249742 >> [v2:172.21.15.110:6836/4047124589,v1:172.21.15.110:6837/4047124589] conn(0x5649a2057f60 0x5649a248b240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:40.162 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:40.155+0000 7f2a2bef3740 4 client.15519 mds_command: new command op to 24614 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-21T03:39:40.163 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:40.155+0000 7f2a2bef3740 1 -- 172.21.15.89:0/2597249742 --> [v2:172.21.15.110:6836/4047124589,v1:172.21.15.110:6837/4047124589] -- command(tid 0: {"prefix": "status"}) -- 0x5649a2370a40 con 0x5649a2057f60 2024-09-21T03:39:40.163 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:40.155+0000 7f2a1effd640 1 --2- 172.21.15.89:0/2597249742 >> [v2:172.21.15.110:6836/4047124589,v1:172.21.15.110:6837/4047124589] conn(0x5649a2057f60 0x5649a248b240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:40.163 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:40.156+0000 7f29feffd640 10 client.15519 ms_handle_connect on v2:172.21.15.110:6836/4047124589 2024-09-21T03:39:40.163 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:40.156+0000 7f2a1effd640 1 --2- 172.21.15.89:0/2597249742 >> [v2:172.21.15.110:6836/4047124589,v1:172.21.15.110:6837/4047124589] conn(0x5649a2057f60 0x5649a248b240 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=b4ea126893179278 server_cookie=aaba9a3256db266a in_seq=0 out_seq=0 2024-09-21T03:39:40.163 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:40.156+0000 7f29feffd640 1 -- 172.21.15.89:0/2597249742 <== mds.2 v2:172.21.15.110:6836/4047124589 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x5649a2370a40 con 0x5649a2057f60 2024-09-21T03:39:40.163 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:40.156+0000 7f29feffd640 10 client.15519 handle_command_reply: tid=0 2024-09-21T03:39:40.163 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:40.156+0000 7f2a2bef3740 1 client.15519 shutdown 2024-09-21T03:39:40.163 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:40.156+0000 7f2a2bef3740 1 -- 172.21.15.89:0/2597249742 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5649a202f0a0 msgr2=0x5649a241f130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:40.163 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:40.156+0000 7f2a2bef3740 1 --2- 172.21.15.89:0/2597249742 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5649a202f0a0 0x5649a241f130 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f2a14009900 tx=0x7f2a14031630 comp rx=0 tx=0).stop 2024-09-21T03:39:40.163 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:40.156+0000 7f2a2bef3740 1 -- 172.21.15.89:0/2597249742 shutdown_connections 2024-09-21T03:39:40.163 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:40.156+0000 7f2a2bef3740 1 --2- 172.21.15.89:0/2597249742 >> [v2:172.21.15.110:6836/4047124589,v1:172.21.15.110:6837/4047124589] conn(0x5649a2057f60 0x5649a248b240 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:40.163 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:40.156+0000 7f2a2bef3740 1 --2- 172.21.15.89:0/2597249742 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5649a2037280 0x5649a2424120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:40.163 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:40.156+0000 7f2a2bef3740 1 --2- 172.21.15.89:0/2597249742 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5649a202f0a0 0x5649a241f130 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:40.163 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:40.156+0000 7f2a2bef3740 1 --2- 172.21.15.89:0/2597249742 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5649a20168e0 0x5649a241eb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:40.164 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:40.156+0000 7f2a2bef3740 1 -- 172.21.15.89:0/2597249742 >> 172.21.15.89:0/2597249742 conn(0x5649a204c350 msgr2=0x5649a23fede0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:39:40.164 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:40.156+0000 7f29feffd640 0 client.15519 ms_handle_reset on v2:172.21.15.110:6836/4047124589 2024-09-21T03:39:40.164 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:40.156+0000 7f2a2bef3740 1 -- 172.21.15.89:0/2597249742 shutdown_connections 2024-09-21T03:39:40.164 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:40.156+0000 7f2a2bef3740 1 -- 172.21.15.89:0/2597249742 wait complete. 2024-09-21T03:39:40.164 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:40.157+0000 7f2a2bef3740 20 client.15519 trim_cache size 0 max 16384 2024-09-21T03:39:40.164 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-09-21T03:39:40.164 INFO:teuthology.orchestra.run.smithi089.stdout: "cluster_fsid": "7c374098-77c9-11ef-baf5-efdc52797490", 2024-09-21T03:39:40.164 INFO:teuthology.orchestra.run.smithi089.stdout: "whoami": 2, 2024-09-21T03:39:40.164 INFO:teuthology.orchestra.run.smithi089.stdout: "id": 24614, 2024-09-21T03:39:40.164 INFO:teuthology.orchestra.run.smithi089.stdout: "want_state": "up:active", 2024-09-21T03:39:40.164 INFO:teuthology.orchestra.run.smithi089.stdout: "state": "up:active", 2024-09-21T03:39:40.165 INFO:teuthology.orchestra.run.smithi089.stdout: "fs_name": "cephfs", 2024-09-21T03:39:40.165 INFO:teuthology.orchestra.run.smithi089.stdout: "rank_uptime": 112.14467503900001, 2024-09-21T03:39:40.165 INFO:teuthology.orchestra.run.smithi089.stdout: "mdsmap_epoch": 26, 2024-09-21T03:39:40.165 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap_epoch": 84, 2024-09-21T03:39:40.165 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap_epoch_barrier": 84, 2024-09-21T03:39:40.165 INFO:teuthology.orchestra.run.smithi089.stdout: "uptime": 168.134769506 2024-09-21T03:39:40.165 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-09-21T03:39:40.165 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:40.157+0000 7f2a2597f640 1 -- 172.21.15.89:0/736028766 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f2a080b65a0 msgr2=0x7f2a080b8ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:40.166 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:40.157+0000 7f2a2597f640 1 --2- 172.21.15.89:0/736028766 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f2a080b65a0 0x7f2a080b8ab0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f2a1800be30 tx=0x7f2a1800b580 comp rx=0 tx=0).stop 2024-09-21T03:39:40.166 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:40.157+0000 7f2a2597f640 1 -- 172.21.15.89:0/736028766 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2a20104a30 msgr2=0x7f2a2019a460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:40.166 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:40.157+0000 7f2a2597f640 1 --2- 172.21.15.89:0/736028766 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2a20104a30 0x7f2a2019a460 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f2a1000b4a0 tx=0x7f2a1000b960 comp rx=0 tx=0).stop 2024-09-21T03:39:40.166 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:40.157+0000 7f2a2597f640 1 -- 172.21.15.89:0/736028766 shutdown_connections 2024-09-21T03:39:40.166 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:40.157+0000 7f2a2597f640 1 --2- 172.21.15.89:0/736028766 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f2a080b65a0 0x7f2a080b8ab0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:40.166 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:40.157+0000 7f2a2597f640 1 --2- 172.21.15.89:0/736028766 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2a2010de60 0x7f2a2019fca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:40.166 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:40.157+0000 7f2a2597f640 1 --2- 172.21.15.89:0/736028766 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2a20105380 0x7f2a2019aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:40.166 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:40.157+0000 7f2a2597f640 1 --2- 172.21.15.89:0/736028766 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2a20104a30 0x7f2a2019a460 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:40.166 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:40.157+0000 7f2a2597f640 1 -- 172.21.15.89:0/736028766 >> 172.21.15.89:0/736028766 conn(0x7f2a20079790 msgr2=0x7f2a20100930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:39:40.166 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:40.157+0000 7f2a2597f640 1 -- 172.21.15.89:0/736028766 shutdown_connections 2024-09-21T03:39:40.166 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:40.157+0000 7f2a2597f640 1 -- 172.21.15.89:0/736028766 wait complete. 2024-09-21T03:39:40.949 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 7c374098-77c9-11ef-baf5-efdc52797490 -- ceph tell mds.i status 2024-09-21T03:39:41.194 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/7c374098-77c9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:39:41.311 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:39:41 smithi089 ceph-mon[30928]: pgmap v368: 129 pgs: 129 active+clean; 661 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.7 KiB/s wr, 8 op/s 2024-09-21T03:39:41.461 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:39:41 smithi114 ceph-mon[34140]: pgmap v368: 129 pgs: 129 active+clean; 661 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.7 KiB/s wr, 8 op/s 2024-09-21T03:39:41.500 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:39:41 smithi110 ceph-mon[34000]: pgmap v368: 129 pgs: 129 active+clean; 661 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.7 KiB/s wr, 8 op/s 2024-09-21T03:39:41.691 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:41.689+0000 7feaaa08d640 1 -- 172.21.15.89:0/4211157081 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7feaa41083e0 msgr2=0x7feaa41087f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:41.691 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:41.689+0000 7feaaa08d640 1 --2- 172.21.15.89:0/4211157081 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7feaa41083e0 0x7feaa41087f0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7fea8c009900 tx=0x7fea8c02a410 comp rx=0 tx=0).stop 2024-09-21T03:39:41.692 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:41.689+0000 7feaaa08d640 1 -- 172.21.15.89:0/4211157081 shutdown_connections 2024-09-21T03:39:41.692 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:41.689+0000 7feaaa08d640 1 --2- 172.21.15.89:0/4211157081 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7feaa4108d30 0x7feaa410f560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:41.692 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:41.689+0000 7feaaa08d640 1 --2- 172.21.15.89:0/4211157081 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7feaa41083e0 0x7feaa41087f0 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:41.692 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:41.689+0000 7feaaa08d640 1 --2- 172.21.15.89:0/4211157081 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7feaa4102aa0 0x7feaa4102eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:41.692 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:41.689+0000 7feaaa08d640 1 -- 172.21.15.89:0/4211157081 >> 172.21.15.89:0/4211157081 conn(0x7feaa4079750 msgr2=0x7feaa4079b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:39:41.692 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:41.689+0000 7feaaa08d640 1 -- 172.21.15.89:0/4211157081 shutdown_connections 2024-09-21T03:39:41.692 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:41.689+0000 7feaaa08d640 1 -- 172.21.15.89:0/4211157081 wait complete. 2024-09-21T03:39:41.694 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:41.690+0000 7feaaa08d640 1 Processor -- start 2024-09-21T03:39:41.694 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:41.690+0000 7feaaa08d640 1 -- start start 2024-09-21T03:39:41.694 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:41.690+0000 7feaaa08d640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7feaa4102aa0 0x7feaa419a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:41.694 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:41.690+0000 7feaaa08d640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7feaa41083e0 0x7feaa419a990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:41.694 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:41.690+0000 7feaaa08d640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7feaa4108d30 0x7feaa419fc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:41.694 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:41.690+0000 7feaaa08d640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7feaa41139c0 con 0x7feaa41083e0 2024-09-21T03:39:41.694 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:41.690+0000 7feaaa08d640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7feaa4113840 con 0x7feaa4108d30 2024-09-21T03:39:41.694 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:41.690+0000 7feaaa08d640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7feaa4113b40 con 0x7feaa4102aa0 2024-09-21T03:39:41.694 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:41.690+0000 7feaa2ffd640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7feaa41083e0 0x7feaa419a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:41.694 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:41.690+0000 7feaa2ffd640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7feaa41083e0 0x7feaa419a990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:46992/0 (socket says 172.21.15.89:46992) 2024-09-21T03:39:41.694 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:41.690+0000 7feaa2ffd640 1 -- 172.21.15.89:0/4108598965 learned_addr learned my addr 172.21.15.89:0/4108598965 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-21T03:39:41.694 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:41.690+0000 7feaa3fff640 1 --2- 172.21.15.89:0/4108598965 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7feaa4108d30 0x7feaa419fc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:41.695 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:41.690+0000 7feaa37fe640 1 --2- 172.21.15.89:0/4108598965 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7feaa4102aa0 0x7feaa419a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:41.695 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:41.691+0000 7feaa2ffd640 1 -- 172.21.15.89:0/4108598965 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7feaa4102aa0 msgr2=0x7feaa419a3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:41.695 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:41.691+0000 7feaa2ffd640 1 --2- 172.21.15.89:0/4108598965 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7feaa4102aa0 0x7feaa419a3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:41.695 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:41.691+0000 7feaa2ffd640 1 -- 172.21.15.89:0/4108598965 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7feaa4108d30 msgr2=0x7feaa419fc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:41.695 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:41.691+0000 7feaa2ffd640 1 --2- 172.21.15.89:0/4108598965 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7feaa4108d30 0x7feaa419fc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:41.695 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:41.691+0000 7feaa2ffd640 1 -- 172.21.15.89:0/4108598965 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feaa41a0360 con 0x7feaa41083e0 2024-09-21T03:39:41.695 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:41.691+0000 7feaa2ffd640 1 --2- 172.21.15.89:0/4108598965 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7feaa41083e0 0x7feaa419a990 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7fea8c005940 tx=0x7fea8c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:39:41.695 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:41.691+0000 7feaa0ff9640 1 -- 172.21.15.89:0/4108598965 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea8c03f030 con 0x7feaa41083e0 2024-09-21T03:39:41.695 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:41.691+0000 7feaaa08d640 1 -- 172.21.15.89:0/4108598965 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feaa41a05f0 con 0x7feaa41083e0 2024-09-21T03:39:41.695 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:41.691+0000 7feaa0ff9640 1 -- 172.21.15.89:0/4108598965 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fea8c003c90 con 0x7feaa41083e0 2024-09-21T03:39:41.695 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:41.691+0000 7feaa0ff9640 1 -- 172.21.15.89:0/4108598965 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea8c035070 con 0x7feaa41083e0 2024-09-21T03:39:41.695 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:41.691+0000 7feaaa08d640 1 -- 172.21.15.89:0/4108598965 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feaa41a6db0 con 0x7feaa41083e0 2024-09-21T03:39:41.701 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:41.694+0000 7feab0601740 1 Processor -- start 2024-09-21T03:39:41.702 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:41.694+0000 7feab0601740 1 -- start start 2024-09-21T03:39:41.702 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:41.694+0000 7feaa0ff9640 1 -- 172.21.15.89:0/4108598965 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 20) ==== 154563+0+0 (secure 0 0 0) 0x7fea8c042030 con 0x7feaa41083e0 2024-09-21T03:39:41.702 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:41.694+0000 7feab0601740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555b8215bf60 0x555b82501f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:41.702 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:41.694+0000 7feab0601740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555b821330a0 0x555b82502440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:41.702 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:41.694+0000 7feab0601740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555b8213b280 0x555b82502980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:41.702 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:41.694+0000 7feab0601740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x555b82479c10 con 0x555b8215bf60 2024-09-21T03:39:41.702 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:41.694+0000 7feab0601740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x555b82469040 con 0x555b821330a0 2024-09-21T03:39:41.702 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:41.694+0000 7feab0601740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x555b8246ce50 con 0x555b8213b280 2024-09-21T03:39:41.702 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:41.694+0000 7feaa3fff640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555b8215bf60 0x555b82501f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:41.702 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:41.694+0000 7feaa3fff640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555b8215bf60 0x555b82501f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:47002/0 (socket says 172.21.15.89:47002) 2024-09-21T03:39:41.702 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:41.694+0000 7feaa3fff640 1 -- 172.21.15.89:0/2157981287 learned_addr learned my addr 172.21.15.89:0/2157981287 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-21T03:39:41.702 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:41.694+0000 7feaa37fe640 1 --2- 172.21.15.89:0/2157981287 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555b821330a0 0x555b82502440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:41.703 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:41.695+0000 7feaa2ffd640 1 --2- 172.21.15.89:0/2157981287 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555b8213b280 0x555b82502980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:41.703 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:41.695+0000 7feaa3fff640 1 -- 172.21.15.89:0/2157981287 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555b8213b280 msgr2=0x555b82502980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:39:41.703 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:41.695+0000 7feaa3fff640 1 --2- 172.21.15.89:0/2157981287 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555b8213b280 0x555b82502980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:41.703 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:41.695+0000 7feaa3fff640 1 -- 172.21.15.89:0/2157981287 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555b821330a0 msgr2=0x555b82502440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:39:41.703 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:41.695+0000 7feaa3fff640 1 --2- 172.21.15.89:0/2157981287 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555b821330a0 0x555b82502440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:41.703 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:41.695+0000 7feaa3fff640 1 -- 172.21.15.89:0/2157981287 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555b821c7540 con 0x555b8215bf60 2024-09-21T03:39:41.703 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:41.695+0000 7feaa3fff640 1 --2- 172.21.15.89:0/2157981287 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555b8215bf60 0x555b82501f00 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7fea9c00ae90 tx=0x7fea9c00f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4adf8dd78a359a31 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:39:41.703 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:41.695+0000 7fea99ffb640 1 -- 172.21.15.89:0/2157981287 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea9c003b70 con 0x555b8215bf60 2024-09-21T03:39:41.703 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:41.695+0000 7fea99ffb640 1 -- 172.21.15.89:0/2157981287 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fea9c003d10 con 0x555b8215bf60 2024-09-21T03:39:41.703 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:41.695+0000 7fea99ffb640 1 -- 172.21.15.89:0/2157981287 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea9c010b00 con 0x555b8215bf60 2024-09-21T03:39:41.703 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:41.695+0000 7feaa0ff9640 1 --2- 172.21.15.89:0/4108598965 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7fea900b65a0 0x7fea900b8ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:41.703 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:41.695+0000 7feaa37fe640 1 --2- 172.21.15.89:0/4108598965 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7fea900b65a0 0x7fea900b8ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:41.703 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:41.695+0000 7feaa0ff9640 1 -- 172.21.15.89:0/4108598965 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 9121+0+0 (secure 0 0 0) 0x7fea8c1080a0 con 0x7feaa41083e0 2024-09-21T03:39:41.703 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:41.696+0000 7feab0601740 1 -- 172.21.15.89:0/2157981287 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555b8215bf60 msgr2=0x555b82501f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:41.703 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:41.696+0000 7feaa37fe640 1 --2- 172.21.15.89:0/4108598965 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7fea900b65a0 0x7fea900b8ab0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x555b82500b80 tx=0x7fea9400a2d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:39:41.704 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:41.696+0000 7feab0601740 1 --2- 172.21.15.89:0/2157981287 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555b8215bf60 0x555b82501f00 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7fea9c00ae90 tx=0x7fea9c00f790 comp rx=0 tx=0).stop 2024-09-21T03:39:41.704 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:41.696+0000 7feab0601740 1 -- 172.21.15.89:0/2157981287 shutdown_connections 2024-09-21T03:39:41.704 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:41.696+0000 7feab0601740 1 --2- 172.21.15.89:0/2157981287 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555b821330a0 0x555b82502440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:41.704 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:41.696+0000 7feab0601740 1 --2- 172.21.15.89:0/2157981287 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555b8213b280 0x555b82502980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:41.704 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:41.696+0000 7feab0601740 1 --2- 172.21.15.89:0/2157981287 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555b8215bf60 0x555b82501f00 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:41.704 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:41.696+0000 7feab0601740 1 -- 172.21.15.89:0/2157981287 >> 172.21.15.89:0/2157981287 conn(0x555b82150350 msgr2=0x555b824ffdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:39:41.704 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:41.696+0000 7feab0601740 1 -- 172.21.15.89:0/2157981287 shutdown_connections 2024-09-21T03:39:41.704 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:41.696+0000 7feab0601740 1 -- 172.21.15.89:0/2157981287 wait complete. 2024-09-21T03:39:41.704 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:41.697+0000 7feab0601740 1 -- start start 2024-09-21T03:39:41.704 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:41.697+0000 7feab0601740 1 Processor -- start 2024-09-21T03:39:41.704 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:41.697+0000 7feab0601740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555b821330a0 0x555b8250d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:41.704 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:41.697+0000 7feab0601740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555b8213b280 0x555b8250df20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:41.704 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:41.697+0000 7feab0601740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555b8215bf60 0x555b82519a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:41.705 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:41.697+0000 7feab0601740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x555b822ca5f0 con 0x555b8215bf60 2024-09-21T03:39:41.705 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:41.697+0000 7feab0601740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x555b82404010 con 0x555b8213b280 2024-09-21T03:39:41.705 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:41.697+0000 7feab0601740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x555b825106e0 con 0x555b821330a0 2024-09-21T03:39:41.705 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:41.697+0000 7feaa3fff640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555b8215bf60 0x555b82519a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:41.705 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:41.697+0000 7feaa3fff640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555b8215bf60 0x555b82519a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:47012/0 (socket says 172.21.15.89:47012) 2024-09-21T03:39:41.705 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:41.697+0000 7feaa3fff640 1 -- 172.21.15.89:0/1587610178 learned_addr learned my addr 172.21.15.89:0/1587610178 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-21T03:39:41.705 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:41.697+0000 7feab0601740 10 client.0 _ll_register_callbacks cb 0x555b82463340 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:39:41.705 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:41.697+0000 7feaa37fe640 1 --2- 172.21.15.89:0/1587610178 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555b821330a0 0x555b8250d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:41.705 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:41.698+0000 7feaa2ffd640 1 --2- 172.21.15.89:0/1587610178 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555b8213b280 0x555b8250df20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:41.705 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:41.698+0000 7feaa3fff640 1 -- 172.21.15.89:0/1587610178 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555b821330a0 msgr2=0x555b8250d9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:39:41.705 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:41.698+0000 7feaa3fff640 1 --2- 172.21.15.89:0/1587610178 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555b821330a0 0x555b8250d9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:41.705 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:41.698+0000 7feaa3fff640 1 -- 172.21.15.89:0/1587610178 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555b8213b280 msgr2=0x555b8250df20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:39:41.705 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:41.698+0000 7feaa3fff640 1 --2- 172.21.15.89:0/1587610178 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555b8213b280 0x555b8250df20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:41.705 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:41.698+0000 7feaa3fff640 1 -- 172.21.15.89:0/1587610178 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555b82431a00 con 0x555b8215bf60 2024-09-21T03:39:41.706 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:41.698+0000 7feaa3fff640 1 --2- 172.21.15.89:0/1587610178 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555b8215bf60 0x555b82519a00 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7fea9c010680 tx=0x7fea9c010b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=caeff26172675556 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:39:41.706 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:41.698+0000 7fea82ffd640 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2024-09-21T03:39:41.706 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:41.698+0000 7fea82ffd640 1 -- 172.21.15.89:0/1587610178 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea9c01d040 con 0x555b8215bf60 2024-09-21T03:39:41.706 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:41.698+0000 7fea82ffd640 1 -- 172.21.15.89:0/1587610178 <== mon.0 v2:172.21.15.89:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 9121+0+0 (secure 0 0 0) 0x7fea9c010cd0 con 0x555b8215bf60 2024-09-21T03:39:41.706 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:41.698+0000 7feab0601740 1 -- 172.21.15.89:0/1587610178 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fea9c003d10 con 0x555b8215bf60 2024-09-21T03:39:41.706 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:41.698+0000 7fea82ffd640 1 -- 172.21.15.89:0/1587610178 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fea9c030050 con 0x555b8215bf60 2024-09-21T03:39:41.706 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:41.698+0000 7feab0601740 10 client.15531 fetch_fsmap learned FSMap version 26 2024-09-21T03:39:41.706 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:41.698+0000 7feab0601740 1 -- 172.21.15.89:0/1587610178 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x555b8251c610 con 0x555b8215bf60 2024-09-21T03:39:41.706 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:41.699+0000 7fea82ffd640 1 -- 172.21.15.89:0/1587610178 <== mon.0 v2:172.21.15.89:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fea9c003d10 con 0x555b8215bf60 2024-09-21T03:39:41.706 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:41.699+0000 7fea82ffd640 1 -- 172.21.15.89:0/1587610178 <== mon.0 v2:172.21.15.89:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fea9c018700 con 0x555b8215bf60 2024-09-21T03:39:41.706 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:41.699+0000 7feab0601740 10 client.15531 fetch_fsmap finished waiting for FSMap version 26 2024-09-21T03:39:41.706 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:41.699+0000 7feab0601740 10 client.15531 resolve_mds: resolved name 'i' to daemon mds.i 2024-09-21T03:39:41.706 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:41.699+0000 7feab0601740 20 client.15531 populate_metadata read hostname 'smithi089' 2024-09-21T03:39:41.706 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:41.699+0000 7feab0601740 1 --2- 172.21.15.89:0/1587610178 >> [v2:172.21.15.114:6838/3427369944,v1:172.21.15.114:6839/3427369944] conn(0x555b8211a8e0 0x555b8251fce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:41.707 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:41.699+0000 7feab0601740 4 client.15531 mds_command: new command op to 24631 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:39:41.707 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:41.699+0000 7feab0601740 1 -- 172.21.15.89:0/1587610178 --> [v2:172.21.15.114:6838/3427369944,v1:172.21.15.114:6839/3427369944] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555b8247bb60 con 0x555b8211a8e0 2024-09-21T03:39:41.707 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:41.699+0000 7feaa37fe640 1 --2- 172.21.15.89:0/1587610178 >> [v2:172.21.15.114:6838/3427369944,v1:172.21.15.114:6839/3427369944] conn(0x555b8211a8e0 0x555b8251fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:41.707 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:41.700+0000 7fea82ffd640 10 client.15531 ms_handle_connect on v2:172.21.15.114:6838/3427369944 2024-09-21T03:39:41.707 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:41.700+0000 7feaa37fe640 1 --2- 172.21.15.89:0/1587610178 >> [v2:172.21.15.114:6838/3427369944,v1:172.21.15.114:6839/3427369944] conn(0x555b8211a8e0 0x555b8251fce0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=20d59b0f4c182450 server_cookie=45e0790610d25c02 in_seq=0 out_seq=0 2024-09-21T03:39:41.708 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:41.700+0000 7fea82ffd640 1 -- 172.21.15.89:0/1587610178 <== mds.1 v2:172.21.15.114:6838/3427369944 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x555b8247bb60 con 0x555b8211a8e0 2024-09-21T03:39:41.708 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:41.700+0000 7fea82ffd640 10 client.15531 handle_command_reply: tid=0 2024-09-21T03:39:41.709 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:41.700+0000 7feab0601740 1 client.15531 shutdown 2024-09-21T03:39:41.709 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:41.700+0000 7feab0601740 1 -- 172.21.15.89:0/1587610178 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555b8215bf60 msgr2=0x555b82519a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:41.709 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:41.700+0000 7feab0601740 1 --2- 172.21.15.89:0/1587610178 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555b8215bf60 0x555b82519a00 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7fea9c010680 tx=0x7fea9c010b30 comp rx=0 tx=0).stop 2024-09-21T03:39:41.709 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:41.701+0000 7feab0601740 1 -- 172.21.15.89:0/1587610178 shutdown_connections 2024-09-21T03:39:41.709 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:41.701+0000 7feab0601740 1 --2- 172.21.15.89:0/1587610178 >> [v2:172.21.15.114:6838/3427369944,v1:172.21.15.114:6839/3427369944] conn(0x555b8211a8e0 0x555b8251fce0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:41.709 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:41.701+0000 7feab0601740 1 --2- 172.21.15.89:0/1587610178 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555b8213b280 0x555b8250df20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:41.709 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:41.701+0000 7feab0601740 1 --2- 172.21.15.89:0/1587610178 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555b8215bf60 0x555b82519a00 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:41.709 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:41.701+0000 7feab0601740 1 --2- 172.21.15.89:0/1587610178 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555b821330a0 0x555b8250d9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:41.709 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:41.701+0000 7feab0601740 1 -- 172.21.15.89:0/1587610178 >> 172.21.15.89:0/1587610178 conn(0x555b82150350 msgr2=0x555b825050e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:39:41.709 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:41.701+0000 7fea82ffd640 0 client.15531 ms_handle_reset on v2:172.21.15.114:6838/3427369944 2024-09-21T03:39:41.709 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:41.701+0000 7feab0601740 1 -- 172.21.15.89:0/1587610178 shutdown_connections 2024-09-21T03:39:41.709 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:41.701+0000 7feab0601740 1 -- 172.21.15.89:0/1587610178 wait complete. 2024-09-21T03:39:41.709 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:41.701+0000 7feab0601740 20 client.15531 trim_cache size 0 max 16384 2024-09-21T03:39:41.722 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:41.718+0000 7feab0601740 1 Processor -- start 2024-09-21T03:39:41.722 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:41.718+0000 7feab0601740 1 -- start start 2024-09-21T03:39:41.722 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:41.719+0000 7feab0601740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555b821330a0 0x555b82508020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:41.722 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:41.719+0000 7feab0601740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555b8213b280 0x555b82519970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:41.722 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:41.719+0000 7feab0601740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555b8215bf60 0x555b82519eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:41.722 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:41.719+0000 7feab0601740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x555b8251e020 con 0x555b8215bf60 2024-09-21T03:39:41.722 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:41.719+0000 7feab0601740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x555b8251e6f0 con 0x555b8213b280 2024-09-21T03:39:41.722 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:41.719+0000 7feab0601740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x555b8251edc0 con 0x555b821330a0 2024-09-21T03:39:41.722 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:41.719+0000 7feaa3fff640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555b8215bf60 0x555b82519eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:41.722 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:41.719+0000 7feaa3fff640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555b8215bf60 0x555b82519eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:47024/0 (socket says 172.21.15.89:47024) 2024-09-21T03:39:41.723 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:41.719+0000 7feaa3fff640 1 -- 172.21.15.89:0/1098976546 learned_addr learned my addr 172.21.15.89:0/1098976546 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-21T03:39:41.723 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:41.719+0000 7feaa37fe640 1 --2- 172.21.15.89:0/1098976546 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555b821330a0 0x555b82508020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:41.723 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:41.719+0000 7feaa2ffd640 1 --2- 172.21.15.89:0/1098976546 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555b8213b280 0x555b82519970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:41.723 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:41.719+0000 7feaa3fff640 1 -- 172.21.15.89:0/1098976546 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555b821330a0 msgr2=0x555b82508020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:39:41.723 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:41.719+0000 7feaa3fff640 1 --2- 172.21.15.89:0/1098976546 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555b821330a0 0x555b82508020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:41.723 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:41.719+0000 7feaa3fff640 1 -- 172.21.15.89:0/1098976546 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555b8213b280 msgr2=0x555b82519970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:39:41.723 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:41.719+0000 7feaa3fff640 1 --2- 172.21.15.89:0/1098976546 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555b8213b280 0x555b82519970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:41.723 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:41.719+0000 7feaa3fff640 1 -- 172.21.15.89:0/1098976546 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555b8250eaa0 con 0x555b8215bf60 2024-09-21T03:39:41.723 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:41.719+0000 7feaa3fff640 1 --2- 172.21.15.89:0/1098976546 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555b8215bf60 0x555b82519eb0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fea9c010850 tx=0x7fea9c0106b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0b70432bc0fd08d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:39:41.723 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:41.720+0000 7fea997fa640 1 -- 172.21.15.89:0/1098976546 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea9c039070 con 0x555b8215bf60 2024-09-21T03:39:41.723 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:41.720+0000 7fea997fa640 1 -- 172.21.15.89:0/1098976546 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fea9c027070 con 0x555b8215bf60 2024-09-21T03:39:41.723 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:41.720+0000 7fea997fa640 1 -- 172.21.15.89:0/1098976546 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea9c018440 con 0x555b8215bf60 2024-09-21T03:39:41.727 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:41.720+0000 7feab0601740 1 -- 172.21.15.89:0/1098976546 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555b8215bf60 msgr2=0x555b82519eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:41.727 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:41.720+0000 7feab0601740 1 --2- 172.21.15.89:0/1098976546 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555b8215bf60 0x555b82519eb0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fea9c010850 tx=0x7fea9c0106b0 comp rx=0 tx=0).stop 2024-09-21T03:39:41.727 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:41.721+0000 7feab0601740 1 -- 172.21.15.89:0/1098976546 shutdown_connections 2024-09-21T03:39:41.727 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:41.721+0000 7feab0601740 1 --2- 172.21.15.89:0/1098976546 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555b8213b280 0x555b82519970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:41.727 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:41.721+0000 7feab0601740 1 --2- 172.21.15.89:0/1098976546 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555b8215bf60 0x555b82519eb0 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:41.727 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:41.721+0000 7feab0601740 1 --2- 172.21.15.89:0/1098976546 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555b821330a0 0x555b82508020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:41.727 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:41.721+0000 7feab0601740 1 -- 172.21.15.89:0/1098976546 >> 172.21.15.89:0/1098976546 conn(0x555b82150350 msgr2=0x555b825089c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:39:41.727 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:41.721+0000 7feab0601740 1 -- 172.21.15.89:0/1098976546 shutdown_connections 2024-09-21T03:39:41.727 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:41.721+0000 7feab0601740 1 -- 172.21.15.89:0/1098976546 wait complete. 2024-09-21T03:39:41.727 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:41.721+0000 7feab0601740 1 -- start start 2024-09-21T03:39:41.727 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:41.721+0000 7feab0601740 1 Processor -- start 2024-09-21T03:39:41.728 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:41.721+0000 7feab0601740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555b821330a0 0x555b82509df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:41.728 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:41.722+0000 7feab0601740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555b8213b280 0x555b8250a330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:41.728 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:41.722+0000 7feab0601740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555b8215bf60 0x555b8250a870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:41.728 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:41.722+0000 7feab0601740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x555b8251fb60 con 0x555b8215bf60 2024-09-21T03:39:41.728 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:41.722+0000 7feab0601740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x555b8251f490 con 0x555b821330a0 2024-09-21T03:39:41.728 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:41.722+0000 7feab0601740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x555b82520230 con 0x555b8213b280 2024-09-21T03:39:41.728 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:41.722+0000 7feab0601740 10 client.0 _ll_register_callbacks cb 0x7fea9400a590 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:39:41.728 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:41.722+0000 7feaa3fff640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555b8215bf60 0x555b8250a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:41.728 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:41.722+0000 7feaa3fff640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555b8215bf60 0x555b8250a870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:47030/0 (socket says 172.21.15.89:47030) 2024-09-21T03:39:41.728 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:41.722+0000 7feaa3fff640 1 -- 172.21.15.89:0/2582601716 learned_addr learned my addr 172.21.15.89:0/2582601716 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-21T03:39:41.728 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:41.722+0000 7feaa37fe640 1 --2- 172.21.15.89:0/2582601716 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555b821330a0 0x555b82509df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:41.728 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:41.722+0000 7feaa2ffd640 1 --2- 172.21.15.89:0/2582601716 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555b8213b280 0x555b8250a330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:41.728 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:41.722+0000 7feaa3fff640 1 -- 172.21.15.89:0/2582601716 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555b8213b280 msgr2=0x555b8250a330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:39:41.728 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:41.722+0000 7feaa3fff640 1 --2- 172.21.15.89:0/2582601716 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555b8213b280 0x555b8250a330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:41.729 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:41.722+0000 7feaa3fff640 1 -- 172.21.15.89:0/2582601716 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555b821330a0 msgr2=0x555b82509df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:39:41.729 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:41.722+0000 7feaa3fff640 1 --2- 172.21.15.89:0/2582601716 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555b821330a0 0x555b82509df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:41.729 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:41.722+0000 7feaa3fff640 1 -- 172.21.15.89:0/2582601716 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555b82527130 con 0x555b8215bf60 2024-09-21T03:39:41.729 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:41.722+0000 7feaa3fff640 1 --2- 172.21.15.89:0/2582601716 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555b8215bf60 0x555b8250a870 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fea9c00a340 tx=0x7fea9c00ab00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1073c4a0ff158c11 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:39:41.729 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:41.722+0000 7fea82ffd640 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2024-09-21T03:39:41.729 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:41.722+0000 7fea82ffd640 1 -- 172.21.15.89:0/2582601716 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea9c039040 con 0x555b8215bf60 2024-09-21T03:39:41.729 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:41.723+0000 7fea82ffd640 1 -- 172.21.15.89:0/2582601716 <== mon.0 v2:172.21.15.89:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 9121+0+0 (secure 0 0 0) 0x7fea9c004a00 con 0x555b8215bf60 2024-09-21T03:39:41.729 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:41.723+0000 7feab0601740 1 -- 172.21.15.89:0/2582601716 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fea9c027070 con 0x555b8215bf60 2024-09-21T03:39:41.729 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:41.723+0000 7fea82ffd640 1 -- 172.21.15.89:0/2582601716 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fea9c010e10 con 0x555b8215bf60 2024-09-21T03:39:41.729 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:41.723+0000 7feab0601740 10 client.15537 fetch_fsmap learned FSMap version 26 2024-09-21T03:39:41.729 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:41.723+0000 7feab0601740 1 -- 172.21.15.89:0/2582601716 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x555b825283b0 con 0x555b8215bf60 2024-09-21T03:39:41.729 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:41.723+0000 7fea82ffd640 1 -- 172.21.15.89:0/2582601716 <== mon.0 v2:172.21.15.89:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fea9c027070 con 0x555b8215bf60 2024-09-21T03:39:41.729 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:41.723+0000 7fea82ffd640 1 -- 172.21.15.89:0/2582601716 <== mon.0 v2:172.21.15.89:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fea9c0226f0 con 0x555b8215bf60 2024-09-21T03:39:41.729 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:41.723+0000 7feab0601740 10 client.15537 fetch_fsmap finished waiting for FSMap version 26 2024-09-21T03:39:41.729 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:41.723+0000 7feab0601740 10 client.15537 resolve_mds: resolved name 'i' to daemon mds.i 2024-09-21T03:39:41.730 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:41.723+0000 7feab0601740 20 client.15537 populate_metadata read hostname 'smithi089' 2024-09-21T03:39:41.730 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:41.723+0000 7feab0601740 1 --2- 172.21.15.89:0/2582601716 >> [v2:172.21.15.114:6838/3427369944,v1:172.21.15.114:6839/3427369944] conn(0x555b820efc90 0x555b825920b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:41.730 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:41.723+0000 7feab0601740 4 client.15537 mds_command: new command op to 24631 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-21T03:39:41.730 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:41.723+0000 7feab0601740 1 -- 172.21.15.89:0/2582601716 --> [v2:172.21.15.114:6838/3427369944,v1:172.21.15.114:6839/3427369944] -- command(tid 0: {"prefix": "status"}) -- 0x555b82477610 con 0x555b820efc90 2024-09-21T03:39:41.730 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:41.723+0000 7feaa37fe640 1 --2- 172.21.15.89:0/2582601716 >> [v2:172.21.15.114:6838/3427369944,v1:172.21.15.114:6839/3427369944] conn(0x555b820efc90 0x555b825920b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:41.730 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:41.724+0000 7fea82ffd640 10 client.15537 ms_handle_connect on v2:172.21.15.114:6838/3427369944 2024-09-21T03:39:41.730 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:41.724+0000 7feaa37fe640 1 --2- 172.21.15.89:0/2582601716 >> [v2:172.21.15.114:6838/3427369944,v1:172.21.15.114:6839/3427369944] conn(0x555b820efc90 0x555b825920b0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=35dd6b015fc36d7c server_cookie=861c97d365f73119 in_seq=0 out_seq=0 2024-09-21T03:39:41.730 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:41.724+0000 7fea82ffd640 1 -- 172.21.15.89:0/2582601716 <== mds.1 v2:172.21.15.114:6838/3427369944 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x555b82477610 con 0x555b820efc90 2024-09-21T03:39:41.730 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:41.724+0000 7fea82ffd640 10 client.15537 handle_command_reply: tid=0 2024-09-21T03:39:41.730 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:41.724+0000 7feab0601740 1 client.15537 shutdown 2024-09-21T03:39:41.730 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:41.724+0000 7feab0601740 1 -- 172.21.15.89:0/2582601716 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555b8215bf60 msgr2=0x555b8250a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:41.730 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:41.724+0000 7feab0601740 1 --2- 172.21.15.89:0/2582601716 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555b8215bf60 0x555b8250a870 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fea9c00a340 tx=0x7fea9c00ab00 comp rx=0 tx=0).stop 2024-09-21T03:39:41.730 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:41.724+0000 7feab0601740 1 -- 172.21.15.89:0/2582601716 shutdown_connections 2024-09-21T03:39:41.730 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:41.724+0000 7feab0601740 1 --2- 172.21.15.89:0/2582601716 >> [v2:172.21.15.114:6838/3427369944,v1:172.21.15.114:6839/3427369944] conn(0x555b820efc90 0x555b825920b0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:41.731 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:41.724+0000 7feab0601740 1 --2- 172.21.15.89:0/2582601716 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555b8215bf60 0x555b8250a870 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:41.731 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:41.724+0000 7feab0601740 1 --2- 172.21.15.89:0/2582601716 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555b8213b280 0x555b8250a330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:41.731 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:41.724+0000 7fea82ffd640 0 client.15537 ms_handle_reset on v2:172.21.15.114:6838/3427369944 2024-09-21T03:39:41.731 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:41.724+0000 7feab0601740 1 --2- 172.21.15.89:0/2582601716 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555b821330a0 0x555b82509df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:41.731 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:41.724+0000 7feab0601740 1 -- 172.21.15.89:0/2582601716 >> 172.21.15.89:0/2582601716 conn(0x555b82150350 msgr2=0x555b8250d9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:39:41.731 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:41.724+0000 7feab0601740 1 -- 172.21.15.89:0/2582601716 shutdown_connections 2024-09-21T03:39:41.731 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:41.724+0000 7feab0601740 1 -- 172.21.15.89:0/2582601716 wait complete. 2024-09-21T03:39:41.731 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:41.725+0000 7feab0601740 20 client.15537 trim_cache size 0 max 16384 2024-09-21T03:39:41.731 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-09-21T03:39:41.731 INFO:teuthology.orchestra.run.smithi089.stdout: "cluster_fsid": "7c374098-77c9-11ef-baf5-efdc52797490", 2024-09-21T03:39:41.732 INFO:teuthology.orchestra.run.smithi089.stdout: "whoami": 1, 2024-09-21T03:39:41.732 INFO:teuthology.orchestra.run.smithi089.stdout: "id": 24631, 2024-09-21T03:39:41.732 INFO:teuthology.orchestra.run.smithi089.stdout: "want_state": "up:active", 2024-09-21T03:39:41.732 INFO:teuthology.orchestra.run.smithi089.stdout: "state": "up:active", 2024-09-21T03:39:41.732 INFO:teuthology.orchestra.run.smithi089.stdout: "fs_name": "cephfs", 2024-09-21T03:39:41.732 INFO:teuthology.orchestra.run.smithi089.stdout: "rank_uptime": 114.72665424900001, 2024-09-21T03:39:41.732 INFO:teuthology.orchestra.run.smithi089.stdout: "mdsmap_epoch": 26, 2024-09-21T03:39:41.732 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap_epoch": 84, 2024-09-21T03:39:41.732 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap_epoch_barrier": 84, 2024-09-21T03:39:41.732 INFO:teuthology.orchestra.run.smithi089.stdout: "uptime": 167.079725725 2024-09-21T03:39:41.732 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-09-21T03:39:41.733 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:41.725+0000 7feaaa08d640 1 -- 172.21.15.89:0/4108598965 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7fea900b65a0 msgr2=0x7fea900b8ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:41.733 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:41.725+0000 7feaaa08d640 1 --2- 172.21.15.89:0/4108598965 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7fea900b65a0 0x7fea900b8ab0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x555b82500b80 tx=0x7fea9400a2d0 comp rx=0 tx=0).stop 2024-09-21T03:39:41.733 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:41.725+0000 7feaaa08d640 1 -- 172.21.15.89:0/4108598965 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7feaa41083e0 msgr2=0x7feaa419a990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:41.733 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:41.725+0000 7feaaa08d640 1 --2- 172.21.15.89:0/4108598965 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7feaa41083e0 0x7feaa419a990 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7fea8c005940 tx=0x7fea8c0042f0 comp rx=0 tx=0).stop 2024-09-21T03:39:41.733 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:41.725+0000 7feaaa08d640 1 -- 172.21.15.89:0/4108598965 shutdown_connections 2024-09-21T03:39:41.733 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:41.725+0000 7feaaa08d640 1 --2- 172.21.15.89:0/4108598965 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7fea900b65a0 0x7fea900b8ab0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:41.733 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:41.725+0000 7feaaa08d640 1 --2- 172.21.15.89:0/4108598965 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7feaa4108d30 0x7feaa419fc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:41.733 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:41.725+0000 7feaaa08d640 1 --2- 172.21.15.89:0/4108598965 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7feaa41083e0 0x7feaa419a990 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:41.733 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:41.725+0000 7feaaa08d640 1 --2- 172.21.15.89:0/4108598965 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7feaa4102aa0 0x7feaa419a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:41.733 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:41.725+0000 7feaaa08d640 1 -- 172.21.15.89:0/4108598965 >> 172.21.15.89:0/4108598965 conn(0x7feaa4079750 msgr2=0x7feaa4104450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:39:41.734 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:41.725+0000 7feaaa08d640 1 -- 172.21.15.89:0/4108598965 shutdown_connections 2024-09-21T03:39:41.734 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:41.725+0000 7feaaa08d640 1 -- 172.21.15.89:0/4108598965 wait complete. 2024-09-21T03:39:42.477 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 7c374098-77c9-11ef-baf5-efdc52797490 -- ceph tell mds.b status 2024-09-21T03:39:42.724 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/7c374098-77c9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:39:43.220 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:43.200+0000 7f0f2ec72640 1 -- 172.21.15.89:0/2459682808 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0f28102a60 msgr2=0x7f0f28102e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:43.220 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:43.200+0000 7f0f2ec72640 1 --2- 172.21.15.89:0/2459682808 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0f28102a60 0x7f0f28102e70 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f0f18009ca0 tx=0x7f0f1802a8c0 comp rx=0 tx=0).stop 2024-09-21T03:39:43.220 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:43.200+0000 7f0f2ec72640 1 -- 172.21.15.89:0/2459682808 shutdown_connections 2024-09-21T03:39:43.220 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:43.200+0000 7f0f2ec72640 1 --2- 172.21.15.89:0/2459682808 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0f281083a0 0x7f0f281087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:43.220 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:43.200+0000 7f0f2ec72640 1 --2- 172.21.15.89:0/2459682808 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0f28108cf0 0x7f0f2810f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:43.220 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:43.200+0000 7f0f2ec72640 1 --2- 172.21.15.89:0/2459682808 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0f28102a60 0x7f0f28102e70 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:43.220 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:43.200+0000 7f0f2ec72640 1 -- 172.21.15.89:0/2459682808 >> 172.21.15.89:0/2459682808 conn(0x7f0f28079730 msgr2=0x7f0f28079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:39:43.220 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:43.200+0000 7f0f2ec72640 1 -- 172.21.15.89:0/2459682808 shutdown_connections 2024-09-21T03:39:43.220 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:43.200+0000 7f0f2ec72640 1 -- 172.21.15.89:0/2459682808 wait complete. 2024-09-21T03:39:43.221 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:43.200+0000 7f0f2ec72640 1 Processor -- start 2024-09-21T03:39:43.221 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:43.201+0000 7f0f2ec72640 1 -- start start 2024-09-21T03:39:43.221 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:43.201+0000 7f0f2ec72640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0f28102a60 0x7f0f2819a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:43.221 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:43.201+0000 7f0f2ec72640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0f281083a0 0x7f0f2819a990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:43.221 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:43.201+0000 7f0f2ec72640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0f28108cf0 0x7f0f2819fc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:43.221 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:43.201+0000 7f0f2ec72640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f0f28113960 con 0x7f0f28102a60 2024-09-21T03:39:43.221 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:43.201+0000 7f0f2ec72640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f0f281137e0 con 0x7f0f281083a0 2024-09-21T03:39:43.221 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:43.201+0000 7f0f2ec72640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f0f28113ae0 con 0x7f0f28108cf0 2024-09-21T03:39:43.221 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:43.201+0000 7f0f2ca0e640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0f28102a60 0x7f0f2819a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:43.221 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:43.201+0000 7f0f2ca0e640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0f28102a60 0x7f0f2819a3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:47052/0 (socket says 172.21.15.89:47052) 2024-09-21T03:39:43.221 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:43.201+0000 7f0f2ca0e640 1 -- 172.21.15.89:0/220744401 learned_addr learned my addr 172.21.15.89:0/220744401 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-21T03:39:43.221 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:43.201+0000 7f0f27fff640 1 --2- 172.21.15.89:0/220744401 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0f281083a0 0x7f0f2819a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:43.221 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:43.201+0000 7f0f2d20f640 1 --2- 172.21.15.89:0/220744401 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0f28108cf0 0x7f0f2819fc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:43.221 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:43.201+0000 7f0f2ca0e640 1 -- 172.21.15.89:0/220744401 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0f28108cf0 msgr2=0x7f0f2819fc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:43.222 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:43.201+0000 7f0f2ca0e640 1 --2- 172.21.15.89:0/220744401 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0f28108cf0 0x7f0f2819fc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:43.222 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:43.201+0000 7f0f2ca0e640 1 -- 172.21.15.89:0/220744401 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0f281083a0 msgr2=0x7f0f2819a990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:43.222 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:43.201+0000 7f0f2ca0e640 1 --2- 172.21.15.89:0/220744401 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0f281083a0 0x7f0f2819a990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:43.222 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:43.201+0000 7f0f2ca0e640 1 -- 172.21.15.89:0/220744401 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f281a02a0 con 0x7f0f28102a60 2024-09-21T03:39:43.222 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:43.201+0000 7f0f2d20f640 1 --2- 172.21.15.89:0/220744401 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0f28108cf0 0x7f0f2819fc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:39:43.222 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:43.201+0000 7f0f2ca0e640 1 --2- 172.21.15.89:0/220744401 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0f28102a60 0x7f0f2819a3f0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f0f18009dc0 tx=0x7f0f18032bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:39:43.222 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:43.202+0000 7f0f25ffb640 1 -- 172.21.15.89:0/220744401 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f18004040 con 0x7f0f28102a60 2024-09-21T03:39:43.222 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:43.202+0000 7f0f25ffb640 1 -- 172.21.15.89:0/220744401 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0f18004670 con 0x7f0f28102a60 2024-09-21T03:39:43.222 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:43.202+0000 7f0f2ec72640 1 -- 172.21.15.89:0/220744401 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0f281a0530 con 0x7f0f28102a60 2024-09-21T03:39:43.222 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:43.202+0000 7f0f25ffb640 1 -- 172.21.15.89:0/220744401 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f18030c80 con 0x7f0f28102a60 2024-09-21T03:39:43.222 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:43.202+0000 7f0f2ec72640 1 -- 172.21.15.89:0/220744401 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0f281a6db0 con 0x7f0f28102a60 2024-09-21T03:39:43.222 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:43.204+0000 7f0f351e6740 1 Processor -- start 2024-09-21T03:39:43.222 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:43.204+0000 7f0f25ffb640 1 -- 172.21.15.89:0/220744401 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 20) ==== 154563+0+0 (secure 0 0 0) 0x7f0f180041e0 con 0x7f0f28102a60 2024-09-21T03:39:43.222 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:43.204+0000 7f0f351e6740 1 -- start start 2024-09-21T03:39:43.223 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:43.205+0000 7f0f351e6740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559e287c8f60 0x559e28b6ef00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:43.223 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:43.205+0000 7f0f351e6740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559e287a00a0 0x559e28b6f440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:43.223 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:43.205+0000 7f0f351e6740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559e287a8280 0x559e28b6f980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:43.223 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:43.205+0000 7f0f351e6740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x559e28ae6c10 con 0x559e287c8f60 2024-09-21T03:39:43.223 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:43.205+0000 7f0f351e6740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x559e28ad6040 con 0x559e287a00a0 2024-09-21T03:39:43.223 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:43.205+0000 7f0f351e6740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x559e28ad9e50 con 0x559e287a8280 2024-09-21T03:39:43.223 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:43.205+0000 7f0f2d20f640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559e287c8f60 0x559e28b6ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:43.223 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:43.205+0000 7f0f2d20f640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559e287c8f60 0x559e28b6ef00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:47056/0 (socket says 172.21.15.89:47056) 2024-09-21T03:39:43.223 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:43.205+0000 7f0f2d20f640 1 -- 172.21.15.89:0/2432426089 learned_addr learned my addr 172.21.15.89:0/2432426089 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-21T03:39:43.223 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:43.205+0000 7f0f2ca0e640 1 --2- 172.21.15.89:0/2432426089 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559e287a00a0 0x559e28b6f440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:43.223 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:43.205+0000 7f0f27fff640 1 --2- 172.21.15.89:0/2432426089 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559e287a8280 0x559e28b6f980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:43.223 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:43.205+0000 7f0f25ffb640 1 --2- 172.21.15.89:0/220744401 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f0f140b6650 0x7f0f140b8b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:43.223 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:43.205+0000 7f0f2d20f640 1 -- 172.21.15.89:0/2432426089 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559e287a8280 msgr2=0x559e28b6f980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:39:43.223 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:43.205+0000 7f0f2d20f640 1 --2- 172.21.15.89:0/2432426089 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559e287a8280 0x559e28b6f980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:43.224 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:43.205+0000 7f0f2d20f640 1 -- 172.21.15.89:0/2432426089 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559e287a00a0 msgr2=0x559e28b6f440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:39:43.224 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:43.205+0000 7f0f2d20f640 1 --2- 172.21.15.89:0/2432426089 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559e287a00a0 0x559e28b6f440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:43.224 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:43.205+0000 7f0f2d20f640 1 -- 172.21.15.89:0/2432426089 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559e28834540 con 0x559e287c8f60 2024-09-21T03:39:43.224 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:43.205+0000 7f0f25ffb640 1 -- 172.21.15.89:0/220744401 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 9121+0+0 (secure 0 0 0) 0x7f0f18108a40 con 0x7f0f28102a60 2024-09-21T03:39:43.224 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:43.205+0000 7f0f2d20f640 1 --2- 172.21.15.89:0/220744401 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f0f140b6650 0x7f0f140b8b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:43.224 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:43.206+0000 7f0f2d20f640 1 --2- 172.21.15.89:0/2432426089 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559e287c8f60 0x559e28b6ef00 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f0f2000be40 tx=0x7f0f2000f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d507548dd466860d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:39:43.224 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:43.206+0000 7f0f1effd640 1 -- 172.21.15.89:0/2432426089 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f2000fde0 con 0x559e287c8f60 2024-09-21T03:39:43.224 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:43.206+0000 7f0f1effd640 1 -- 172.21.15.89:0/2432426089 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0f2001cda0 con 0x559e287c8f60 2024-09-21T03:39:43.224 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:43.206+0000 7f0f1effd640 1 -- 172.21.15.89:0/2432426089 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f2001bb60 con 0x559e287c8f60 2024-09-21T03:39:43.224 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:43.206+0000 7f0f2d20f640 1 --2- 172.21.15.89:0/220744401 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f0f140b6650 0x7f0f140b8b60 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f0f20020000 tx=0x7f0f2001d000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:39:43.224 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:43.207+0000 7f0f351e6740 1 -- 172.21.15.89:0/2432426089 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559e287c8f60 msgr2=0x559e28b6ef00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:43.224 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:43.207+0000 7f0f351e6740 1 --2- 172.21.15.89:0/2432426089 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559e287c8f60 0x559e28b6ef00 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f0f2000be40 tx=0x7f0f2000f730 comp rx=0 tx=0).stop 2024-09-21T03:39:43.224 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:43.207+0000 7f0f351e6740 1 -- 172.21.15.89:0/2432426089 shutdown_connections 2024-09-21T03:39:43.224 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:43.207+0000 7f0f351e6740 1 --2- 172.21.15.89:0/2432426089 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559e287a00a0 0x559e28b6f440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:43.225 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:43.207+0000 7f0f351e6740 1 --2- 172.21.15.89:0/2432426089 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559e287a8280 0x559e28b6f980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:43.225 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:43.207+0000 7f0f351e6740 1 --2- 172.21.15.89:0/2432426089 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559e287c8f60 0x559e28b6ef00 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:43.225 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:43.207+0000 7f0f351e6740 1 -- 172.21.15.89:0/2432426089 >> 172.21.15.89:0/2432426089 conn(0x559e287bd350 msgr2=0x559e28b6cdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:39:43.225 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:43.207+0000 7f0f351e6740 1 -- 172.21.15.89:0/2432426089 shutdown_connections 2024-09-21T03:39:43.225 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:43.207+0000 7f0f351e6740 1 -- 172.21.15.89:0/2432426089 wait complete. 2024-09-21T03:39:43.225 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:43.207+0000 7f0f351e6740 1 -- start start 2024-09-21T03:39:43.225 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:43.207+0000 7f0f351e6740 1 Processor -- start 2024-09-21T03:39:43.225 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:43.208+0000 7f0f351e6740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559e287a00a0 0x559e28b7a9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:43.225 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:43.208+0000 7f0f351e6740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559e287a8280 0x559e28b7aee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:43.225 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:43.208+0000 7f0f351e6740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559e287c8f60 0x559e28b869d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:43.225 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:43.208+0000 7f0f351e6740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x559e289375f0 con 0x559e287c8f60 2024-09-21T03:39:43.226 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:43.208+0000 7f0f351e6740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x559e28a71010 con 0x559e287a8280 2024-09-21T03:39:43.226 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:43.208+0000 7f0f351e6740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x559e28b7d6e0 con 0x559e287a00a0 2024-09-21T03:39:43.226 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:43.208+0000 7f0f351e6740 10 client.0 _ll_register_callbacks cb 0x559e28ad0340 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:39:43.226 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:43.208+0000 7f0f2d20f640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559e287c8f60 0x559e28b869d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:43.226 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:43.208+0000 7f0f2d20f640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559e287c8f60 0x559e28b869d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:47068/0 (socket says 172.21.15.89:47068) 2024-09-21T03:39:43.226 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:43.208+0000 7f0f2d20f640 1 -- 172.21.15.89:0/2969521123 learned_addr learned my addr 172.21.15.89:0/2969521123 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-21T03:39:43.226 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:43.208+0000 7f0f2ca0e640 1 --2- 172.21.15.89:0/2969521123 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559e287a00a0 0x559e28b7a9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:43.226 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:43.208+0000 7f0f27fff640 1 --2- 172.21.15.89:0/2969521123 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559e287a8280 0x559e28b7aee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:43.226 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:43.208+0000 7f0f2d20f640 1 -- 172.21.15.89:0/2969521123 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559e287a00a0 msgr2=0x559e28b7a9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:39:43.226 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:43.208+0000 7f0f2d20f640 1 --2- 172.21.15.89:0/2969521123 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559e287a00a0 0x559e28b7a9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:43.226 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:43.208+0000 7f0f2d20f640 1 -- 172.21.15.89:0/2969521123 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559e287a8280 msgr2=0x559e28b7aee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:39:43.227 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:43.208+0000 7f0f2d20f640 1 --2- 172.21.15.89:0/2969521123 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559e287a8280 0x559e28b7aee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:43.227 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:43.208+0000 7f0f2d20f640 1 -- 172.21.15.89:0/2969521123 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559e28a9ea00 con 0x559e287c8f60 2024-09-21T03:39:43.227 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:43.208+0000 7f0f2d20f640 1 --2- 172.21.15.89:0/2969521123 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559e287c8f60 0x559e28b869d0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f0f2000b5f0 tx=0x7f0f20004680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f95c3164c1f0c81d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:39:43.227 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:43.208+0000 7f0f07fff640 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2024-09-21T03:39:43.227 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:43.209+0000 7f0f07fff640 1 -- 172.21.15.89:0/2969521123 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f2001b440 con 0x559e287c8f60 2024-09-21T03:39:43.227 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:43.209+0000 7f0f07fff640 1 -- 172.21.15.89:0/2969521123 <== mon.0 v2:172.21.15.89:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 9121+0+0 (secure 0 0 0) 0x7f0f20022070 con 0x559e287c8f60 2024-09-21T03:39:43.227 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:43.209+0000 7f0f351e6740 1 -- 172.21.15.89:0/2969521123 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0f2001cda0 con 0x559e287c8f60 2024-09-21T03:39:43.227 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:43.209+0000 7f0f07fff640 1 -- 172.21.15.89:0/2969521123 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f0f2001b940 con 0x559e287c8f60 2024-09-21T03:39:43.227 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:43.209+0000 7f0f351e6740 10 client.15549 fetch_fsmap learned FSMap version 26 2024-09-21T03:39:43.227 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:43.209+0000 7f0f351e6740 1 -- 172.21.15.89:0/2969521123 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x559e28b895d0 con 0x559e287c8f60 2024-09-21T03:39:43.227 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:43.209+0000 7f0f07fff640 1 -- 172.21.15.89:0/2969521123 <== mon.0 v2:172.21.15.89:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0f2001cda0 con 0x559e287c8f60 2024-09-21T03:39:43.227 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:43.209+0000 7f0f07fff640 1 -- 172.21.15.89:0/2969521123 <== mon.0 v2:172.21.15.89:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f0f20026710 con 0x559e287c8f60 2024-09-21T03:39:43.227 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:43.209+0000 7f0f351e6740 10 client.15549 fetch_fsmap finished waiting for FSMap version 26 2024-09-21T03:39:43.227 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:43.209+0000 7f0f351e6740 10 client.15549 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-21T03:39:43.227 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:43.209+0000 7f0f351e6740 20 client.15549 populate_metadata read hostname 'smithi089' 2024-09-21T03:39:43.228 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:43.209+0000 7f0f351e6740 1 --2- 172.21.15.89:0/2969521123 >> [v2:172.21.15.110:6838/132366709,v1:172.21.15.110:6839/132366709] conn(0x559e287878e0 0x559e28b8ce20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:43.228 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:43.209+0000 7f0f351e6740 4 client.15549 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:39:43.228 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:43.209+0000 7f0f351e6740 1 -- 172.21.15.89:0/2969521123 --> [v2:172.21.15.110:6838/132366709,v1:172.21.15.110:6839/132366709] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559e28ae8b60 con 0x559e287878e0 2024-09-21T03:39:43.228 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:43.210+0000 7f0f2ca0e640 1 --2- 172.21.15.89:0/2969521123 >> [v2:172.21.15.110:6838/132366709,v1:172.21.15.110:6839/132366709] conn(0x559e287878e0 0x559e28b8ce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:43.228 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:43.210+0000 7f0f07fff640 10 client.15549 ms_handle_connect on v2:172.21.15.110:6838/132366709 2024-09-21T03:39:43.228 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:43.210+0000 7f0f2ca0e640 1 --2- 172.21.15.89:0/2969521123 >> [v2:172.21.15.110:6838/132366709,v1:172.21.15.110:6839/132366709] conn(0x559e287878e0 0x559e28b8ce20 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=78a1684577057892 server_cookie=9e7092f467c0063d in_seq=0 out_seq=0 2024-09-21T03:39:43.228 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:43.211+0000 7f0f07fff640 1 -- 172.21.15.89:0/2969521123 <== mds.0 v2:172.21.15.110:6838/132366709 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x559e28ae8b60 con 0x559e287878e0 2024-09-21T03:39:43.228 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:43.211+0000 7f0f07fff640 10 client.15549 handle_command_reply: tid=0 2024-09-21T03:39:43.228 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:43.211+0000 7f0f351e6740 1 client.15549 shutdown 2024-09-21T03:39:43.228 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:43.211+0000 7f0f351e6740 1 -- 172.21.15.89:0/2969521123 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559e287c8f60 msgr2=0x559e28b869d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:43.228 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:43.211+0000 7f0f351e6740 1 --2- 172.21.15.89:0/2969521123 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559e287c8f60 0x559e28b869d0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f0f2000b5f0 tx=0x7f0f20004680 comp rx=0 tx=0).stop 2024-09-21T03:39:43.228 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:43.211+0000 7f0f351e6740 1 -- 172.21.15.89:0/2969521123 shutdown_connections 2024-09-21T03:39:43.228 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:43.211+0000 7f0f351e6740 1 --2- 172.21.15.89:0/2969521123 >> [v2:172.21.15.110:6838/132366709,v1:172.21.15.110:6839/132366709] conn(0x559e287878e0 0x559e28b8ce20 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:43.228 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:43.211+0000 7f0f351e6740 1 --2- 172.21.15.89:0/2969521123 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559e287a8280 0x559e28b7aee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:43.228 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:43.211+0000 7f0f351e6740 1 --2- 172.21.15.89:0/2969521123 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559e287c8f60 0x559e28b869d0 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:43.229 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:43.211+0000 7f0f07fff640 0 client.15549 ms_handle_reset on v2:172.21.15.110:6838/132366709 2024-09-21T03:39:43.229 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:43.211+0000 7f0f351e6740 1 --2- 172.21.15.89:0/2969521123 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559e287a00a0 0x559e28b7a9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:43.229 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:43.211+0000 7f0f351e6740 1 -- 172.21.15.89:0/2969521123 >> 172.21.15.89:0/2969521123 conn(0x559e287bd350 msgr2=0x559e28b720e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:39:43.229 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:43.212+0000 7f0f351e6740 1 -- 172.21.15.89:0/2969521123 shutdown_connections 2024-09-21T03:39:43.229 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:43.212+0000 7f0f351e6740 1 -- 172.21.15.89:0/2969521123 wait complete. 2024-09-21T03:39:43.229 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:43.212+0000 7f0f351e6740 20 client.15549 trim_cache size 0 max 16384 2024-09-21T03:39:43.231 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:43.229+0000 7f0f351e6740 1 Processor -- start 2024-09-21T03:39:43.231 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:43.229+0000 7f0f351e6740 1 -- start start 2024-09-21T03:39:43.233 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:43.230+0000 7f0f351e6740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559e287878e0 0x559e28b8b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:43.233 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:43.230+0000 7f0f351e6740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559e287a00a0 0x559e28b709e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:43.233 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:43.230+0000 7f0f351e6740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559e287a8280 0x559e28b86940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:43.233 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:43.230+0000 7f0f351e6740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x559e28b8b160 con 0x559e287a00a0 2024-09-21T03:39:43.233 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:43.230+0000 7f0f351e6740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x559e28b8b830 con 0x559e287878e0 2024-09-21T03:39:43.233 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:43.230+0000 7f0f351e6740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x559e28b8bf00 con 0x559e287a8280 2024-09-21T03:39:43.233 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:43.230+0000 7f0f27fff640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559e287a00a0 0x559e28b709e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:43.233 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:43.230+0000 7f0f27fff640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559e287a00a0 0x559e28b709e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:47084/0 (socket says 172.21.15.89:47084) 2024-09-21T03:39:43.233 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:43.230+0000 7f0f27fff640 1 -- 172.21.15.89:0/1808413224 learned_addr learned my addr 172.21.15.89:0/1808413224 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-21T03:39:43.233 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:43.230+0000 7f0f2d20f640 1 --2- 172.21.15.89:0/1808413224 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559e287a8280 0x559e28b86940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:43.234 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:43.230+0000 7f0f2ca0e640 1 --2- 172.21.15.89:0/1808413224 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559e287878e0 0x559e28b8b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:43.234 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:43.230+0000 7f0f27fff640 1 -- 172.21.15.89:0/1808413224 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559e287a8280 msgr2=0x559e28b86940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:39:43.234 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:43.230+0000 7f0f27fff640 1 --2- 172.21.15.89:0/1808413224 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559e287a8280 0x559e28b86940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:43.234 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:43.230+0000 7f0f27fff640 1 -- 172.21.15.89:0/1808413224 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559e287878e0 msgr2=0x559e28b8b9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:39:43.234 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:43.230+0000 7f0f27fff640 1 --2- 172.21.15.89:0/1808413224 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559e287878e0 0x559e28b8b9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:43.234 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:43.230+0000 7f0f27fff640 1 -- 172.21.15.89:0/1808413224 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559e28b71d60 con 0x559e287a00a0 2024-09-21T03:39:43.234 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:43.230+0000 7f0f27fff640 1 --2- 172.21.15.89:0/1808413224 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559e287a00a0 0x559e28b709e0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f0f1000add0 tx=0x7f0f1000a2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65517b687bb5d5e1 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:39:43.234 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:43.230+0000 7f0f1e7fc640 1 -- 172.21.15.89:0/1808413224 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f1001e020 con 0x559e287a00a0 2024-09-21T03:39:43.234 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:43.230+0000 7f0f1e7fc640 1 -- 172.21.15.89:0/1808413224 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0f10010e20 con 0x559e287a00a0 2024-09-21T03:39:43.234 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:43.231+0000 7f0f1e7fc640 1 -- 172.21.15.89:0/1808413224 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f1000fc20 con 0x559e287a00a0 2024-09-21T03:39:43.238 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:43.231+0000 7f0f351e6740 1 -- 172.21.15.89:0/1808413224 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559e287a00a0 msgr2=0x559e28b709e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:43.238 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:43.231+0000 7f0f351e6740 1 --2- 172.21.15.89:0/1808413224 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559e287a00a0 0x559e28b709e0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f0f1000add0 tx=0x7f0f1000a2d0 comp rx=0 tx=0).stop 2024-09-21T03:39:43.238 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:43.231+0000 7f0f351e6740 1 -- 172.21.15.89:0/1808413224 shutdown_connections 2024-09-21T03:39:43.238 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:43.231+0000 7f0f351e6740 1 --2- 172.21.15.89:0/1808413224 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559e287a8280 0x559e28b86940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:43.238 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:43.231+0000 7f0f351e6740 1 --2- 172.21.15.89:0/1808413224 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559e287a00a0 0x559e28b709e0 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:43.238 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:43.231+0000 7f0f351e6740 1 --2- 172.21.15.89:0/1808413224 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559e287878e0 0x559e28b8b9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:43.238 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:43.231+0000 7f0f351e6740 1 -- 172.21.15.89:0/1808413224 >> 172.21.15.89:0/1808413224 conn(0x559e287bd350 msgr2=0x559e28b75980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:39:43.238 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:43.231+0000 7f0f351e6740 1 -- 172.21.15.89:0/1808413224 shutdown_connections 2024-09-21T03:39:43.238 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:43.231+0000 7f0f351e6740 1 -- 172.21.15.89:0/1808413224 wait complete. 2024-09-21T03:39:43.238 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:43.232+0000 7f0f351e6740 1 -- start start 2024-09-21T03:39:43.239 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:43.232+0000 7f0f351e6740 1 Processor -- start 2024-09-21T03:39:43.239 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:43.232+0000 7f0f351e6740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559e287878e0 0x559e28b956c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:43.239 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:43.232+0000 7f0f351e6740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559e287a00a0 0x559e28b8eb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:43.239 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:43.232+0000 7f0f351e6740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559e287a8280 0x559e28b8f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:43.239 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:43.232+0000 7f0f351e6740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x559e28b8cca0 con 0x559e287a8280 2024-09-21T03:39:43.239 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:43.232+0000 7f0f351e6740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x559e28b8c5d0 con 0x559e287a00a0 2024-09-21T03:39:43.239 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:43.232+0000 7f0f351e6740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x559e28b8d370 con 0x559e287878e0 2024-09-21T03:39:43.239 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:43.232+0000 7f0f351e6740 10 client.0 _ll_register_callbacks cb 0x7f0f2001f020 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:39:43.239 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:43.232+0000 7f0f2d20f640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559e287a8280 0x559e28b8f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:43.239 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:43.232+0000 7f0f2ca0e640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559e287878e0 0x559e28b956c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:43.239 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:43.232+0000 7f0f2d20f640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559e287a8280 0x559e28b8f0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:47090/0 (socket says 172.21.15.89:47090) 2024-09-21T03:39:43.239 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:43.232+0000 7f0f2d20f640 1 -- 172.21.15.89:0/2681448721 learned_addr learned my addr 172.21.15.89:0/2681448721 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-21T03:39:43.239 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:43.232+0000 7f0f27fff640 1 --2- 172.21.15.89:0/2681448721 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559e287a00a0 0x559e28b8eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:43.239 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:43.233+0000 7f0f2d20f640 1 -- 172.21.15.89:0/2681448721 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559e287878e0 msgr2=0x559e28b956c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:39:43.239 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:43.233+0000 7f0f2d20f640 1 --2- 172.21.15.89:0/2681448721 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559e287878e0 0x559e28b956c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:43.240 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:43.233+0000 7f0f2d20f640 1 -- 172.21.15.89:0/2681448721 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559e287a00a0 msgr2=0x559e28b8eb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:39:43.240 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:43.233+0000 7f0f2d20f640 1 --2- 172.21.15.89:0/2681448721 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559e287a00a0 0x559e28b8eb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:43.240 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:43.233+0000 7f0f2d20f640 1 -- 172.21.15.89:0/2681448721 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559e28b8f670 con 0x559e287a8280 2024-09-21T03:39:43.240 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:43.233+0000 7f0f2d20f640 1 --2- 172.21.15.89:0/2681448721 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559e287a8280 0x559e28b8f0b0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f0f2000be40 tx=0x7f0f2000fc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44dc37b0982b89f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:39:43.240 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:43.233+0000 7f0f07fff640 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2024-09-21T03:39:43.240 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:43.233+0000 7f0f07fff640 1 -- 172.21.15.89:0/2681448721 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f2000b950 con 0x559e287a8280 2024-09-21T03:39:43.240 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:43.233+0000 7f0f07fff640 1 -- 172.21.15.89:0/2681448721 <== mon.0 v2:172.21.15.89:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 9121+0+0 (secure 0 0 0) 0x7f0f2002d040 con 0x559e287a8280 2024-09-21T03:39:43.240 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:43.233+0000 7f0f351e6740 1 -- 172.21.15.89:0/2681448721 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0f10010e20 con 0x559e287a8280 2024-09-21T03:39:43.240 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:43.233+0000 7f0f07fff640 1 -- 172.21.15.89:0/2681448721 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f0f20048050 con 0x559e287a8280 2024-09-21T03:39:43.240 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:43.233+0000 7f0f351e6740 10 client.15555 fetch_fsmap learned FSMap version 26 2024-09-21T03:39:43.240 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:43.233+0000 7f0f351e6740 1 -- 172.21.15.89:0/2681448721 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x559e28b95050 con 0x559e287a8280 2024-09-21T03:39:43.240 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:43.233+0000 7f0f07fff640 1 -- 172.21.15.89:0/2681448721 <== mon.0 v2:172.21.15.89:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0f10010e20 con 0x559e287a8280 2024-09-21T03:39:43.240 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:43.234+0000 7f0f07fff640 1 -- 172.21.15.89:0/2681448721 <== mon.0 v2:172.21.15.89:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f0f2003e030 con 0x559e287a8280 2024-09-21T03:39:43.240 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:43.234+0000 7f0f351e6740 10 client.15555 fetch_fsmap finished waiting for FSMap version 26 2024-09-21T03:39:43.241 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:43.234+0000 7f0f351e6740 10 client.15555 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-21T03:39:43.241 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:43.234+0000 7f0f351e6740 20 client.15555 populate_metadata read hostname 'smithi089' 2024-09-21T03:39:43.241 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:43.234+0000 7f0f351e6740 1 --2- 172.21.15.89:0/2681448721 >> [v2:172.21.15.110:6838/132366709,v1:172.21.15.110:6839/132366709] conn(0x559e287c8f60 0x559e28bfb730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:43.241 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:43.234+0000 7f0f351e6740 4 client.15555 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-21T03:39:43.241 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:43.234+0000 7f0f351e6740 1 -- 172.21.15.89:0/2681448721 --> [v2:172.21.15.110:6838/132366709,v1:172.21.15.110:6839/132366709] -- command(tid 0: {"prefix": "status"}) -- 0x559e28ae4610 con 0x559e287c8f60 2024-09-21T03:39:43.241 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:43.234+0000 7f0f2ca0e640 1 --2- 172.21.15.89:0/2681448721 >> [v2:172.21.15.110:6838/132366709,v1:172.21.15.110:6839/132366709] conn(0x559e287c8f60 0x559e28bfb730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:43.241 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:43.235+0000 7f0f07fff640 10 client.15555 ms_handle_connect on v2:172.21.15.110:6838/132366709 2024-09-21T03:39:43.241 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:43.235+0000 7f0f2ca0e640 1 --2- 172.21.15.89:0/2681448721 >> [v2:172.21.15.110:6838/132366709,v1:172.21.15.110:6839/132366709] conn(0x559e287c8f60 0x559e28bfb730 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=671c4bad48e91904 server_cookie=772afd557211bdb2 in_seq=0 out_seq=0 2024-09-21T03:39:43.241 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:43.235+0000 7f0f07fff640 1 -- 172.21.15.89:0/2681448721 <== mds.0 v2:172.21.15.110:6838/132366709 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x559e28ae4610 con 0x559e287c8f60 2024-09-21T03:39:43.241 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:43.235+0000 7f0f07fff640 10 client.15555 handle_command_reply: tid=0 2024-09-21T03:39:43.241 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:43.235+0000 7f0f351e6740 1 client.15555 shutdown 2024-09-21T03:39:43.241 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:43.235+0000 7f0f351e6740 1 -- 172.21.15.89:0/2681448721 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559e287a8280 msgr2=0x559e28b8f0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:43.241 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:43.235+0000 7f0f351e6740 1 --2- 172.21.15.89:0/2681448721 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559e287a8280 0x559e28b8f0b0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f0f2000be40 tx=0x7f0f2000fc30 comp rx=0 tx=0).stop 2024-09-21T03:39:43.241 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:43.235+0000 7f0f351e6740 1 -- 172.21.15.89:0/2681448721 shutdown_connections 2024-09-21T03:39:43.241 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:43.235+0000 7f0f351e6740 1 --2- 172.21.15.89:0/2681448721 >> [v2:172.21.15.110:6838/132366709,v1:172.21.15.110:6839/132366709] conn(0x559e287c8f60 0x559e28bfb730 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:43.242 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:43.235+0000 7f0f351e6740 1 --2- 172.21.15.89:0/2681448721 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559e287a00a0 0x559e28b8eb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:43.242 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:43.235+0000 7f0f351e6740 1 --2- 172.21.15.89:0/2681448721 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559e287a8280 0x559e28b8f0b0 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:43.242 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:43.235+0000 7f0f351e6740 1 --2- 172.21.15.89:0/2681448721 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559e287878e0 0x559e28b956c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:43.242 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:43.235+0000 7f0f351e6740 1 -- 172.21.15.89:0/2681448721 >> 172.21.15.89:0/2681448721 conn(0x559e287bd350 msgr2=0x559e28b6c770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:39:43.242 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:43.235+0000 7f0f07fff640 0 client.15555 ms_handle_reset on v2:172.21.15.110:6838/132366709 2024-09-21T03:39:43.242 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:43.235+0000 7f0f351e6740 1 -- 172.21.15.89:0/2681448721 shutdown_connections 2024-09-21T03:39:43.242 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:43.235+0000 7f0f351e6740 1 -- 172.21.15.89:0/2681448721 wait complete. 2024-09-21T03:39:43.242 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:43.235+0000 7f0f351e6740 20 client.15555 trim_cache size 0 max 16384 2024-09-21T03:39:43.242 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-09-21T03:39:43.242 INFO:teuthology.orchestra.run.smithi089.stdout: "cluster_fsid": "7c374098-77c9-11ef-baf5-efdc52797490", 2024-09-21T03:39:43.243 INFO:teuthology.orchestra.run.smithi089.stdout: "whoami": 0, 2024-09-21T03:39:43.243 INFO:teuthology.orchestra.run.smithi089.stdout: "id": 24638, 2024-09-21T03:39:43.243 INFO:teuthology.orchestra.run.smithi089.stdout: "want_state": "up:active", 2024-09-21T03:39:43.243 INFO:teuthology.orchestra.run.smithi089.stdout: "state": "up:active", 2024-09-21T03:39:43.243 INFO:teuthology.orchestra.run.smithi089.stdout: "fs_name": "cephfs", 2024-09-21T03:39:43.243 INFO:teuthology.orchestra.run.smithi089.stdout: "rank_uptime": 149.38211384799999, 2024-09-21T03:39:43.243 INFO:teuthology.orchestra.run.smithi089.stdout: "mdsmap_epoch": 26, 2024-09-21T03:39:43.243 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap_epoch": 84, 2024-09-21T03:39:43.243 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap_epoch_barrier": 84, 2024-09-21T03:39:43.243 INFO:teuthology.orchestra.run.smithi089.stdout: "uptime": 159.84989712999999 2024-09-21T03:39:43.243 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-09-21T03:39:43.244 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:43.236+0000 7f0f2ec72640 1 -- 172.21.15.89:0/220744401 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f0f140b6650 msgr2=0x7f0f140b8b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:43.244 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:43.236+0000 7f0f2ec72640 1 --2- 172.21.15.89:0/220744401 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f0f140b6650 0x7f0f140b8b60 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f0f20020000 tx=0x7f0f2001d000 comp rx=0 tx=0).stop 2024-09-21T03:39:43.244 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:43.236+0000 7f0f2ec72640 1 -- 172.21.15.89:0/220744401 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0f28102a60 msgr2=0x7f0f2819a3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:43.244 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:43.236+0000 7f0f2ec72640 1 --2- 172.21.15.89:0/220744401 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0f28102a60 0x7f0f2819a3f0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f0f18009dc0 tx=0x7f0f18032bc0 comp rx=0 tx=0).stop 2024-09-21T03:39:43.244 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:43.236+0000 7f0f2ec72640 1 -- 172.21.15.89:0/220744401 shutdown_connections 2024-09-21T03:39:43.244 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:43.236+0000 7f0f2ec72640 1 --2- 172.21.15.89:0/220744401 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f0f140b6650 0x7f0f140b8b60 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:43.244 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:43.236+0000 7f0f2ec72640 1 --2- 172.21.15.89:0/220744401 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0f281083a0 0x7f0f2819a990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:43.245 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:43.236+0000 7f0f2ec72640 1 --2- 172.21.15.89:0/220744401 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0f28108cf0 0x7f0f2819fc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:43.245 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:43.236+0000 7f0f2ec72640 1 --2- 172.21.15.89:0/220744401 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0f28102a60 0x7f0f2819a3f0 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:43.245 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:43.236+0000 7f0f2ec72640 1 -- 172.21.15.89:0/220744401 >> 172.21.15.89:0/220744401 conn(0x7f0f28079730 msgr2=0x7f0f28101500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:39:43.245 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:43.236+0000 7f0f2ec72640 1 -- 172.21.15.89:0/220744401 shutdown_connections 2024-09-21T03:39:43.245 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:43.236+0000 7f0f2ec72640 1 -- 172.21.15.89:0/220744401 wait complete. 2024-09-21T03:39:43.461 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:39:43 smithi114 ceph-mon[34140]: pgmap v369: 129 pgs: 129 active+clean; 661 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 682 B/s wr, 5 op/s 2024-09-21T03:39:43.500 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:39:43 smithi110 ceph-mon[34000]: pgmap v369: 129 pgs: 129 active+clean; 661 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 682 B/s wr, 5 op/s 2024-09-21T03:39:43.561 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:39:43 smithi089 ceph-mon[30928]: pgmap v369: 129 pgs: 129 active+clean; 661 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 682 B/s wr, 5 op/s 2024-09-21T03:39:44.003 INFO:tasks.quiescer:Ready to start quiesce thrashing; initial delay: 15 sec 2024-09-21T03:39:45.461 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:39:45 smithi114 ceph-mon[34140]: pgmap v370: 129 pgs: 129 active+clean; 661 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 1.5 KiB/s wr, 9 op/s 2024-09-21T03:39:45.500 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:39:45 smithi110 ceph-mon[34000]: pgmap v370: 129 pgs: 129 active+clean; 661 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 1.5 KiB/s wr, 9 op/s 2024-09-21T03:39:45.561 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:39:45 smithi089 ceph-mon[30928]: pgmap v370: 129 pgs: 129 active+clean; 661 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 1.5 KiB/s wr, 9 op/s 2024-09-21T03:39:47.461 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:39:47 smithi114 ceph-mon[34140]: pgmap v371: 129 pgs: 129 active+clean; 673 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 3.0 KiB/s wr, 11 op/s 2024-09-21T03:39:47.500 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:39:47 smithi110 ceph-mon[34000]: pgmap v371: 129 pgs: 129 active+clean; 673 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 3.0 KiB/s wr, 11 op/s 2024-09-21T03:39:47.561 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:39:47 smithi089 ceph-mon[30928]: pgmap v371: 129 pgs: 129 active+clean; 673 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 3.0 KiB/s wr, 11 op/s 2024-09-21T03:39:49.377 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:39:49 smithi089 ceph-mon[30928]: pgmap v372: 129 pgs: 129 active+clean; 673 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 3.0 KiB/s wr, 10 op/s 2024-09-21T03:39:49.461 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:39:49 smithi114 ceph-mon[34140]: pgmap v372: 129 pgs: 129 active+clean; 673 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 3.0 KiB/s wr, 10 op/s 2024-09-21T03:39:49.500 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:39:49 smithi110 ceph-mon[34000]: pgmap v372: 129 pgs: 129 active+clean; 673 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 3.0 KiB/s wr, 10 op/s 2024-09-21T03:39:51.461 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:39:51 smithi114 ceph-mon[34140]: pgmap v373: 129 pgs: 129 active+clean; 673 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 3.1 KiB/s wr, 12 op/s 2024-09-21T03:39:51.500 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:39:51 smithi110 ceph-mon[34000]: pgmap v373: 129 pgs: 129 active+clean; 673 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 3.1 KiB/s wr, 12 op/s 2024-09-21T03:39:51.561 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:39:51 smithi089 ceph-mon[30928]: pgmap v373: 129 pgs: 129 active+clean; 673 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 3.1 KiB/s wr, 12 op/s 2024-09-21T03:39:52.839 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:39:52.838+0000 7fd38d18b640 1 -- 172.21.15.89:0/3996749243 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd38806c030 msgr2=0x7fd388106390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:52.840 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:39:52.838+0000 7fd38d18b640 1 --2- 172.21.15.89:0/3996749243 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd38806c030 0x7fd388106390 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fd378009900 tx=0x7fd37802a3f0 comp rx=0 tx=0).stop 2024-09-21T03:39:52.840 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:39:52.838+0000 7fd38d18b640 1 -- 172.21.15.89:0/3996749243 shutdown_connections 2024-09-21T03:39:52.840 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:39:52.838+0000 7fd38d18b640 1 --2- 172.21.15.89:0/3996749243 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd3881068d0 0x7fd388108cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:52.840 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:39:52.838+0000 7fd38d18b640 1 --2- 172.21.15.89:0/3996749243 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd3881093e0 0x7fd38810b8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:52.840 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:39:52.838+0000 7fd38d18b640 1 --2- 172.21.15.89:0/3996749243 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd38806c030 0x7fd388106390 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:52.840 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:39:52.838+0000 7fd38d18b640 1 -- 172.21.15.89:0/3996749243 >> 172.21.15.89:0/3996749243 conn(0x7fd3880fdf30 msgr2=0x7fd3881003a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:39:52.840 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:39:52.838+0000 7fd38d18b640 1 -- 172.21.15.89:0/3996749243 shutdown_connections 2024-09-21T03:39:52.840 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:39:52.838+0000 7fd38d18b640 1 -- 172.21.15.89:0/3996749243 wait complete. 2024-09-21T03:39:52.843 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:39:52.839+0000 7fd38d18b640 1 Processor -- start 2024-09-21T03:39:52.843 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:39:52.839+0000 7fd38d18b640 1 -- start start 2024-09-21T03:39:52.843 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:39:52.839+0000 7fd38d18b640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd38806c030 0x7fd388199e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:52.843 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:39:52.839+0000 7fd38d18b640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd3881068d0 0x7fd38819a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:52.843 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:39:52.839+0000 7fd38d18b640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd3881093e0 0x7fd38819b080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:52.843 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:39:52.839+0000 7fd38d18b640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fd38810f170 con 0x7fd38806c030 2024-09-21T03:39:52.843 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:39:52.839+0000 7fd38d18b640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fd38810eff0 con 0x7fd3881068d0 2024-09-21T03:39:52.843 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:39:52.839+0000 7fd38d18b640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fd38810f2f0 con 0x7fd3881093e0 2024-09-21T03:39:52.843 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:39:52.839+0000 7fd386d9d640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd38806c030 0x7fd388199e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:52.843 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:39:52.840+0000 7fd386d9d640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd38806c030 0x7fd388199e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:39812/0 (socket says 172.21.15.89:39812) 2024-09-21T03:39:52.843 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:39:52.840+0000 7fd386d9d640 1 -- 172.21.15.89:0/382491769 learned_addr learned my addr 172.21.15.89:0/382491769 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-21T03:39:52.844 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:39:52.840+0000 7fd38659c640 1 --2- 172.21.15.89:0/382491769 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd3881068d0 0x7fd38819a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:52.844 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:39:52.840+0000 7fd38759e640 1 --2- 172.21.15.89:0/382491769 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd3881093e0 0x7fd38819b080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:52.844 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:39:52.840+0000 7fd386d9d640 1 -- 172.21.15.89:0/382491769 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd3881093e0 msgr2=0x7fd38819b080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:52.844 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:39:52.840+0000 7fd386d9d640 1 --2- 172.21.15.89:0/382491769 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd3881093e0 0x7fd38819b080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:52.844 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:39:52.840+0000 7fd386d9d640 1 -- 172.21.15.89:0/382491769 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd3881068d0 msgr2=0x7fd38819a4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:52.844 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:39:52.840+0000 7fd386d9d640 1 --2- 172.21.15.89:0/382491769 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd3881068d0 0x7fd38819a4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:52.844 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:39:52.840+0000 7fd386d9d640 1 -- 172.21.15.89:0/382491769 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd38819b810 con 0x7fd38806c030 2024-09-21T03:39:52.844 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:39:52.840+0000 7fd386d9d640 1 --2- 172.21.15.89:0/382491769 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd38806c030 0x7fd388199e60 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7fd378009230 tx=0x7fd3780042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:39:52.844 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:39:52.840+0000 7fd36ffff640 1 -- 172.21.15.89:0/382491769 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd37803f030 con 0x7fd38806c030 2024-09-21T03:39:52.845 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:39:52.840+0000 7fd36ffff640 1 -- 172.21.15.89:0/382491769 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd378003c90 con 0x7fd38806c030 2024-09-21T03:39:52.845 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:39:52.840+0000 7fd38d18b640 1 -- 172.21.15.89:0/382491769 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd38819baa0 con 0x7fd38806c030 2024-09-21T03:39:52.845 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:39:52.840+0000 7fd36ffff640 1 -- 172.21.15.89:0/382491769 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd378035070 con 0x7fd38806c030 2024-09-21T03:39:52.845 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:39:52.840+0000 7fd38d18b640 1 -- 172.21.15.89:0/382491769 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd3881a20a0 con 0x7fd38806c030 2024-09-21T03:39:52.851 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:39:52.843+0000 7fd3941d8740 1 Processor -- start 2024-09-21T03:39:52.852 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:39:52.843+0000 7fd3941d8740 1 -- start start 2024-09-21T03:39:52.852 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:39:52.843+0000 7fd36ffff640 1 -- 172.21.15.89:0/382491769 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 20) ==== 154563+0+0 (secure 0 0 0) 0x7fd378042030 con 0x7fd38806c030 2024-09-21T03:39:52.852 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:39:52.843+0000 7fd3941d8740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55dfc598c1b0 0x55dfc5d38260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:52.852 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:39:52.843+0000 7fd3941d8740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55dfc59632f0 0x55dfc5d388d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:52.852 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:39:52.843+0000 7fd3941d8740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55dfc596b4d0 0x55dfc5d39420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:52.852 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:39:52.843+0000 7fd3941d8740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55dfc5ca66f0 con 0x55dfc596b4d0 2024-09-21T03:39:52.852 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:39:52.843+0000 7fd3941d8740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55dfc5c95b20 con 0x55dfc59632f0 2024-09-21T03:39:52.852 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:39:52.843+0000 7fd3941d8740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55dfc5c99930 con 0x55dfc598c1b0 2024-09-21T03:39:52.852 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:39:52.843+0000 7fd38659c640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55dfc596b4d0 0x55dfc5d39420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:52.852 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:39:52.843+0000 7fd38659c640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55dfc596b4d0 0x55dfc5d39420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:39826/0 (socket says 172.21.15.89:39826) 2024-09-21T03:39:52.853 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:39:52.843+0000 7fd38659c640 1 -- 172.21.15.89:0/732002582 learned_addr learned my addr 172.21.15.89:0/732002582 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-21T03:39:52.853 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:39:52.843+0000 7fd38759e640 1 --2- 172.21.15.89:0/732002582 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55dfc598c1b0 0x55dfc5d38260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:52.853 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:39:52.843+0000 7fd386d9d640 1 --2- 172.21.15.89:0/732002582 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55dfc59632f0 0x55dfc5d388d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:52.853 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:39:52.844+0000 7fd38659c640 1 -- 172.21.15.89:0/732002582 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55dfc598c1b0 msgr2=0x55dfc5d38260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:39:52.853 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:39:52.844+0000 7fd38659c640 1 --2- 172.21.15.89:0/732002582 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55dfc598c1b0 0x55dfc5d38260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:52.853 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:39:52.844+0000 7fd38659c640 1 -- 172.21.15.89:0/732002582 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55dfc59632f0 msgr2=0x55dfc5d388d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:39:52.853 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:39:52.844+0000 7fd38659c640 1 --2- 172.21.15.89:0/732002582 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55dfc59632f0 0x55dfc5d388d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:52.853 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:39:52.844+0000 7fd38659c640 1 -- 172.21.15.89:0/732002582 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dfc5c5b530 con 0x55dfc596b4d0 2024-09-21T03:39:52.853 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:39:52.844+0000 7fd36ffff640 1 --2- 172.21.15.89:0/382491769 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7fd3700b65a0 0x7fd3700b8ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:52.853 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:39:52.844+0000 7fd38659c640 1 --2- 172.21.15.89:0/732002582 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55dfc596b4d0 0x55dfc5d39420 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7fd37c00cc60 tx=0x7fd37c00e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a605d7cbfb207c75 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:39:52.853 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:39:52.844+0000 7fd38759e640 1 --2- 172.21.15.89:0/382491769 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7fd3700b65a0 0x7fd3700b8ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:52.853 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:39:52.844+0000 7fd36d7fa640 1 -- 172.21.15.89:0/732002582 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd37c014070 con 0x55dfc596b4d0 2024-09-21T03:39:52.854 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:39:52.844+0000 7fd36ffff640 1 -- 172.21.15.89:0/382491769 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 9121+0+0 (secure 0 0 0) 0x7fd378108060 con 0x7fd38806c030 2024-09-21T03:39:52.854 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:39:52.844+0000 7fd36d7fa640 1 -- 172.21.15.89:0/732002582 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd37c0047a0 con 0x55dfc596b4d0 2024-09-21T03:39:52.854 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:39:52.844+0000 7fd36d7fa640 1 -- 172.21.15.89:0/732002582 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd37c010690 con 0x55dfc596b4d0 2024-09-21T03:39:52.854 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:39:52.845+0000 7fd3941d8740 1 -- 172.21.15.89:0/732002582 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55dfc596b4d0 msgr2=0x55dfc5d39420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:52.854 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:39:52.845+0000 7fd38759e640 1 --2- 172.21.15.89:0/382491769 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7fd3700b65a0 0x7fd3700b8ab0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fd380007ed0 tx=0x7fd380007680 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:39:52.854 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:39:52.845+0000 7fd3941d8740 1 --2- 172.21.15.89:0/732002582 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55dfc596b4d0 0x55dfc5d39420 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7fd37c00cc60 tx=0x7fd37c00e720 comp rx=0 tx=0).stop 2024-09-21T03:39:52.854 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:39:52.845+0000 7fd3941d8740 1 -- 172.21.15.89:0/732002582 shutdown_connections 2024-09-21T03:39:52.854 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:39:52.845+0000 7fd3941d8740 1 --2- 172.21.15.89:0/732002582 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55dfc59632f0 0x55dfc5d388d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:52.854 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:39:52.845+0000 7fd3941d8740 1 --2- 172.21.15.89:0/732002582 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55dfc596b4d0 0x55dfc5d39420 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:52.854 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:39:52.845+0000 7fd3941d8740 1 --2- 172.21.15.89:0/732002582 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55dfc598c1b0 0x55dfc5d38260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:52.854 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:39:52.845+0000 7fd3941d8740 1 -- 172.21.15.89:0/732002582 >> 172.21.15.89:0/732002582 conn(0x55dfc5980300 msgr2=0x55dfc5d2f8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:39:52.854 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:39:52.845+0000 7fd3941d8740 1 -- 172.21.15.89:0/732002582 shutdown_connections 2024-09-21T03:39:52.855 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:39:52.845+0000 7fd3941d8740 1 -- 172.21.15.89:0/732002582 wait complete. 2024-09-21T03:39:52.855 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:39:52.845+0000 7fd3941d8740 1 -- start start 2024-09-21T03:39:52.855 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:39:52.846+0000 7fd3941d8740 1 Processor -- start 2024-09-21T03:39:52.855 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:39:52.846+0000 7fd3941d8740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55dfc59632f0 0x55dfc5d41a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:52.855 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:39:52.846+0000 7fd3941d8740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55dfc596b4d0 0x55dfc5d42110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:52.855 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:39:52.846+0000 7fd3941d8740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55dfc598c1b0 0x55dfc5d42c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:52.855 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:39:52.846+0000 7fd3941d8740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55dfc5bcdab0 con 0x55dfc598c1b0 2024-09-21T03:39:52.855 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:39:52.846+0000 7fd3941d8740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55dfc5c3b730 con 0x55dfc596b4d0 2024-09-21T03:39:52.855 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:39:52.846+0000 7fd3941d8740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55dfc5d39f40 con 0x55dfc59632f0 2024-09-21T03:39:52.855 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:39:52.846+0000 7fd3941d8740 10 client.0 _ll_register_callbacks cb 0x55dfc5c8f6b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:39:52.855 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:39:52.846+0000 7fd38759e640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55dfc598c1b0 0x55dfc5d42c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:52.855 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:39:52.846+0000 7fd38759e640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55dfc598c1b0 0x55dfc5d42c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:39836/0 (socket says 172.21.15.89:39836) 2024-09-21T03:39:52.856 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:39:52.846+0000 7fd38759e640 1 -- 172.21.15.89:0/662489231 learned_addr learned my addr 172.21.15.89:0/662489231 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-21T03:39:52.856 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:39:52.846+0000 7fd386d9d640 1 --2- 172.21.15.89:0/662489231 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55dfc59632f0 0x55dfc5d41a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:52.856 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:39:52.846+0000 7fd38659c640 1 --2- 172.21.15.89:0/662489231 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55dfc596b4d0 0x55dfc5d42110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:52.856 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:39:52.846+0000 7fd38759e640 1 -- 172.21.15.89:0/662489231 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55dfc59632f0 msgr2=0x55dfc5d41a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:39:52.856 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:39:52.846+0000 7fd38759e640 1 --2- 172.21.15.89:0/662489231 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55dfc59632f0 0x55dfc5d41a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:52.856 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:39:52.846+0000 7fd38759e640 1 -- 172.21.15.89:0/662489231 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55dfc596b4d0 msgr2=0x55dfc5d42110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:39:52.856 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:39:52.846+0000 7fd38759e640 1 --2- 172.21.15.89:0/662489231 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55dfc596b4d0 0x55dfc5d42110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:52.856 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:39:52.846+0000 7fd38759e640 1 -- 172.21.15.89:0/662489231 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dfc59f7790 con 0x55dfc598c1b0 2024-09-21T03:39:52.856 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:39:52.846+0000 7fd386d9d640 1 --2- 172.21.15.89:0/662489231 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55dfc59632f0 0x55dfc5d41a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:39:52.856 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:39:52.846+0000 7fd38759e640 1 --2- 172.21.15.89:0/662489231 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55dfc598c1b0 0x55dfc5d42c80 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7fd38000edc0 tx=0x7fd380004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19fd5944a7ea7942 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:39:52.856 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:39:52.846+0000 7fd36aff5640 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2024-09-21T03:39:52.856 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:39:52.847+0000 7fd36aff5640 1 -- 172.21.15.89:0/662489231 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd38001cd30 con 0x55dfc598c1b0 2024-09-21T03:39:52.857 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:39:52.847+0000 7fd36aff5640 1 -- 172.21.15.89:0/662489231 <== mon.0 v2:172.21.15.89:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 9121+0+0 (secure 0 0 0) 0x7fd38001d410 con 0x55dfc598c1b0 2024-09-21T03:39:52.857 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:39:52.847+0000 7fd3941d8740 1 -- 172.21.15.89:0/662489231 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd37c0047a0 con 0x55dfc598c1b0 2024-09-21T03:39:52.857 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:39:52.847+0000 7fd36aff5640 1 -- 172.21.15.89:0/662489231 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fd38001d610 con 0x55dfc598c1b0 2024-09-21T03:39:52.857 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:39:52.847+0000 7fd3941d8740 10 client.15567 fetch_fsmap learned FSMap version 26 2024-09-21T03:39:52.857 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:39:52.847+0000 7fd3941d8740 1 -- 172.21.15.89:0/662489231 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55dfc5a70620 con 0x55dfc598c1b0 2024-09-21T03:39:52.857 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:39:52.847+0000 7fd36aff5640 1 -- 172.21.15.89:0/662489231 <== mon.0 v2:172.21.15.89:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd37c0047a0 con 0x55dfc598c1b0 2024-09-21T03:39:52.857 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:39:52.847+0000 7fd36aff5640 1 -- 172.21.15.89:0/662489231 <== mon.0 v2:172.21.15.89:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fd380028820 con 0x55dfc598c1b0 2024-09-21T03:39:52.857 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:39:52.847+0000 7fd3941d8740 10 client.15567 fetch_fsmap finished waiting for FSMap version 26 2024-09-21T03:39:52.857 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:39:52.847+0000 7fd3941d8740 10 client.15567 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T03:39:52.857 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:39:52.847+0000 7fd3941d8740 20 client.15567 populate_metadata read hostname 'smithi089' 2024-09-21T03:39:52.857 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:39:52.847+0000 7fd3941d8740 1 --2- 172.21.15.89:0/662489231 >> [v2:172.21.15.110:6838/132366709,v1:172.21.15.110:6839/132366709] conn(0x55dfc594ad80 0x55dfc5d4f2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:52.857 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:39:52.847+0000 7fd3941d8740 4 client.15567 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:39:52.858 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:39:52.847+0000 7fd3941d8740 1 -- 172.21.15.89:0/662489231 --> [v2:172.21.15.110:6838/132366709,v1:172.21.15.110:6839/132366709] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55dfc5ca8640 con 0x55dfc594ad80 2024-09-21T03:39:52.858 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:39:52.848+0000 7fd386d9d640 1 --2- 172.21.15.89:0/662489231 >> [v2:172.21.15.110:6838/132366709,v1:172.21.15.110:6839/132366709] conn(0x55dfc594ad80 0x55dfc5d4f2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:52.858 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:39:52.848+0000 7fd36aff5640 10 client.15567 ms_handle_connect on v2:172.21.15.110:6838/132366709 2024-09-21T03:39:52.858 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:39:52.848+0000 7fd386d9d640 1 --2- 172.21.15.89:0/662489231 >> [v2:172.21.15.110:6838/132366709,v1:172.21.15.110:6839/132366709] conn(0x55dfc594ad80 0x55dfc5d4f2b0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=86d2aa21c5f6a805 server_cookie=199b1dfbc72efac3 in_seq=0 out_seq=0 2024-09-21T03:39:52.858 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:39:52.849+0000 7fd36aff5640 1 -- 172.21.15.89:0/662489231 <== mds.0 v2:172.21.15.110:6838/132366709 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55dfc5ca8640 con 0x55dfc594ad80 2024-09-21T03:39:52.858 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:39:52.849+0000 7fd36aff5640 10 client.15567 handle_command_reply: tid=0 2024-09-21T03:39:52.858 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:39:52.849+0000 7fd3941d8740 1 client.15567 shutdown 2024-09-21T03:39:52.858 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:39:52.849+0000 7fd3941d8740 1 -- 172.21.15.89:0/662489231 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55dfc598c1b0 msgr2=0x55dfc5d42c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:52.858 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:39:52.849+0000 7fd3941d8740 1 --2- 172.21.15.89:0/662489231 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55dfc598c1b0 0x55dfc5d42c80 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7fd38000edc0 tx=0x7fd380004a40 comp rx=0 tx=0).stop 2024-09-21T03:39:52.858 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:39:52.849+0000 7fd3941d8740 1 -- 172.21.15.89:0/662489231 shutdown_connections 2024-09-21T03:39:52.858 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:39:52.849+0000 7fd3941d8740 1 --2- 172.21.15.89:0/662489231 >> [v2:172.21.15.110:6838/132366709,v1:172.21.15.110:6839/132366709] conn(0x55dfc594ad80 0x55dfc5d4f2b0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:52.858 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:39:52.849+0000 7fd3941d8740 1 --2- 172.21.15.89:0/662489231 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55dfc596b4d0 0x55dfc5d42110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:52.859 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:39:52.849+0000 7fd3941d8740 1 --2- 172.21.15.89:0/662489231 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55dfc598c1b0 0x55dfc5d42c80 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:52.859 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:39:52.849+0000 7fd36aff5640 0 client.15567 ms_handle_reset on v2:172.21.15.110:6838/132366709 2024-09-21T03:39:52.859 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:39:52.849+0000 7fd3941d8740 1 --2- 172.21.15.89:0/662489231 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55dfc59632f0 0x55dfc5d41a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:52.859 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:39:52.849+0000 7fd3941d8740 1 -- 172.21.15.89:0/662489231 >> 172.21.15.89:0/662489231 conn(0x55dfc5980300 msgr2=0x55dfc5d2dea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:39:52.859 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:39:52.849+0000 7fd3941d8740 1 -- 172.21.15.89:0/662489231 shutdown_connections 2024-09-21T03:39:52.859 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:39:52.849+0000 7fd3941d8740 1 -- 172.21.15.89:0/662489231 wait complete. 2024-09-21T03:39:52.859 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:39:52.849+0000 7fd3941d8740 20 client.15567 trim_cache size 0 max 16384 2024-09-21T03:39:52.868 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:39:52.866+0000 7fd3941d8740 1 Processor -- start 2024-09-21T03:39:52.868 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:39:52.866+0000 7fd3941d8740 1 -- start start 2024-09-21T03:39:52.868 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:39:52.866+0000 7fd3941d8740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55dfc59632f0 0x55dfc5d2f220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:52.868 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:39:52.866+0000 7fd3941d8740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55dfc596b4d0 0x55dfc5d2fdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:52.869 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:39:52.866+0000 7fd3941d8740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55dfc598c1b0 0x55dfc5d30300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:52.869 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:39:52.866+0000 7fd3941d8740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fd3881acb70 con 0x55dfc598c1b0 2024-09-21T03:39:52.869 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:39:52.866+0000 7fd3941d8740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fd3881aa440 con 0x55dfc59632f0 2024-09-21T03:39:52.869 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:39:52.866+0000 7fd3941d8740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fd3881a8290 con 0x55dfc596b4d0 2024-09-21T03:39:52.869 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:39:52.866+0000 7fd38759e640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55dfc598c1b0 0x55dfc5d30300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:52.869 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:39:52.866+0000 7fd38759e640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55dfc598c1b0 0x55dfc5d30300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:39844/0 (socket says 172.21.15.89:39844) 2024-09-21T03:39:52.869 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:39:52.866+0000 7fd38759e640 1 -- 172.21.15.89:0/469394814 learned_addr learned my addr 172.21.15.89:0/469394814 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-21T03:39:52.869 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:39:52.866+0000 7fd386d9d640 1 --2- 172.21.15.89:0/469394814 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55dfc59632f0 0x55dfc5d2f220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:52.869 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:39:52.866+0000 7fd38659c640 1 --2- 172.21.15.89:0/469394814 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55dfc596b4d0 0x55dfc5d2fdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:52.869 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:39:52.867+0000 7fd38759e640 1 -- 172.21.15.89:0/469394814 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55dfc596b4d0 msgr2=0x55dfc5d2fdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:39:52.869 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:39:52.867+0000 7fd38759e640 1 --2- 172.21.15.89:0/469394814 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55dfc596b4d0 0x55dfc5d2fdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:52.869 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:39:52.867+0000 7fd38759e640 1 -- 172.21.15.89:0/469394814 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55dfc59632f0 msgr2=0x55dfc5d2f220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:39:52.870 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:39:52.867+0000 7fd38759e640 1 --2- 172.21.15.89:0/469394814 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55dfc59632f0 0x55dfc5d2f220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:52.870 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:39:52.867+0000 7fd38759e640 1 -- 172.21.15.89:0/469394814 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dfc5d49f20 con 0x55dfc598c1b0 2024-09-21T03:39:52.870 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:39:52.867+0000 7fd38759e640 1 --2- 172.21.15.89:0/469394814 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55dfc598c1b0 0x55dfc5d30300 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7fd380016040 tx=0x7fd38000ef90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c344bd191ea5cd03 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:39:52.870 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:39:52.867+0000 7fd36cff9640 1 -- 172.21.15.89:0/469394814 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd380039030 con 0x55dfc598c1b0 2024-09-21T03:39:52.870 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:39:52.867+0000 7fd36cff9640 1 -- 172.21.15.89:0/469394814 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd38001c770 con 0x55dfc598c1b0 2024-09-21T03:39:52.870 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:39:52.867+0000 7fd36cff9640 1 -- 172.21.15.89:0/469394814 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd380028800 con 0x55dfc598c1b0 2024-09-21T03:39:52.873 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:39:52.868+0000 7fd3941d8740 1 -- 172.21.15.89:0/469394814 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55dfc598c1b0 msgr2=0x55dfc5d30300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:52.873 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:39:52.868+0000 7fd3941d8740 1 --2- 172.21.15.89:0/469394814 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55dfc598c1b0 0x55dfc5d30300 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7fd380016040 tx=0x7fd38000ef90 comp rx=0 tx=0).stop 2024-09-21T03:39:52.873 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:39:52.868+0000 7fd3941d8740 1 -- 172.21.15.89:0/469394814 shutdown_connections 2024-09-21T03:39:52.874 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:39:52.868+0000 7fd3941d8740 1 --2- 172.21.15.89:0/469394814 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55dfc598c1b0 0x55dfc5d30300 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:52.874 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:39:52.868+0000 7fd3941d8740 1 --2- 172.21.15.89:0/469394814 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55dfc596b4d0 0x55dfc5d2fdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:52.874 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:39:52.868+0000 7fd3941d8740 1 --2- 172.21.15.89:0/469394814 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55dfc59632f0 0x55dfc5d2f220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:52.874 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:39:52.868+0000 7fd3941d8740 1 -- 172.21.15.89:0/469394814 >> 172.21.15.89:0/469394814 conn(0x55dfc5980300 msgr2=0x55dfc5d2e650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:39:52.874 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:39:52.868+0000 7fd3941d8740 1 -- 172.21.15.89:0/469394814 shutdown_connections 2024-09-21T03:39:52.874 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:39:52.868+0000 7fd3941d8740 1 -- 172.21.15.89:0/469394814 wait complete. 2024-09-21T03:39:52.874 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:39:52.869+0000 7fd3941d8740 1 -- start start 2024-09-21T03:39:52.874 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:39:52.869+0000 7fd3941d8740 1 Processor -- start 2024-09-21T03:39:52.874 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:39:52.869+0000 7fd3941d8740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55dfc59632f0 0x55dfc5d495b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:52.874 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:39:52.869+0000 7fd3941d8740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55dfc596b4d0 0x55dfc5d48270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:52.874 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:39:52.869+0000 7fd3941d8740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55dfc598c1b0 0x55dfc5d487b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:52.874 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:39:52.869+0000 7fd3941d8740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fd3881a8900 con 0x55dfc59632f0 2024-09-21T03:39:52.875 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:39:52.869+0000 7fd3941d8740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fd3881a9d70 con 0x55dfc596b4d0 2024-09-21T03:39:52.875 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:39:52.869+0000 7fd3941d8740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fd3881a8fd0 con 0x55dfc598c1b0 2024-09-21T03:39:52.875 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:39:52.869+0000 7fd3941d8740 10 client.0 _ll_register_callbacks cb 0x7fd380007b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:39:52.875 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:39:52.869+0000 7fd386d9d640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55dfc59632f0 0x55dfc5d495b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:52.875 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:39:52.869+0000 7fd38659c640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55dfc596b4d0 0x55dfc5d48270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:52.875 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:39:52.869+0000 7fd386d9d640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55dfc59632f0 0x55dfc5d495b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:39848/0 (socket says 172.21.15.89:39848) 2024-09-21T03:39:52.875 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:39:52.869+0000 7fd38659c640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55dfc596b4d0 0x55dfc5d48270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.89:34482/0 (socket says 172.21.15.89:34482) 2024-09-21T03:39:52.875 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:39:52.869+0000 7fd386d9d640 1 -- 172.21.15.89:0/2486650880 learned_addr learned my addr 172.21.15.89:0/2486650880 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-21T03:39:52.875 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:39:52.869+0000 7fd38759e640 1 --2- 172.21.15.89:0/2486650880 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55dfc598c1b0 0x55dfc5d487b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:52.875 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:39:52.870+0000 7fd386d9d640 1 -- 172.21.15.89:0/2486650880 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55dfc598c1b0 msgr2=0x55dfc5d487b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:39:52.875 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:39:52.870+0000 7fd386d9d640 1 --2- 172.21.15.89:0/2486650880 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55dfc598c1b0 0x55dfc5d487b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:52.875 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:39:52.870+0000 7fd386d9d640 1 -- 172.21.15.89:0/2486650880 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55dfc596b4d0 msgr2=0x55dfc5d48270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:39:52.876 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:39:52.870+0000 7fd386d9d640 1 --2- 172.21.15.89:0/2486650880 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55dfc596b4d0 0x55dfc5d48270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:52.876 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:39:52.870+0000 7fd386d9d640 1 -- 172.21.15.89:0/2486650880 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dfc5d48ea0 con 0x55dfc59632f0 2024-09-21T03:39:52.876 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:39:52.870+0000 7fd38659c640 1 --2- 172.21.15.89:0/2486650880 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55dfc596b4d0 0x55dfc5d48270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:39:52.876 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:39:52.870+0000 7fd386d9d640 1 --2- 172.21.15.89:0/2486650880 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55dfc59632f0 0x55dfc5d495b0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7fd3780cdca0 tx=0x7fd378009680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9fd700993978b25 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:39:52.876 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:39:52.870+0000 7fd36aff5640 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2024-09-21T03:39:52.876 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:39:52.870+0000 7fd36aff5640 1 -- 172.21.15.89:0/2486650880 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd378049040 con 0x55dfc59632f0 2024-09-21T03:39:52.876 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:39:52.870+0000 7fd36aff5640 1 -- 172.21.15.89:0/2486650880 <== mon.0 v2:172.21.15.89:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 9121+0+0 (secure 0 0 0) 0x7fd3780c9f80 con 0x55dfc59632f0 2024-09-21T03:39:52.876 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:39:52.870+0000 7fd3941d8740 1 -- 172.21.15.89:0/2486650880 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd38001c770 con 0x55dfc59632f0 2024-09-21T03:39:52.876 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:39:52.870+0000 7fd36aff5640 1 -- 172.21.15.89:0/2486650880 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fd3780cd480 con 0x55dfc59632f0 2024-09-21T03:39:52.876 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:39:52.870+0000 7fd3941d8740 10 client.15573 fetch_fsmap learned FSMap version 26 2024-09-21T03:39:52.876 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:39:52.870+0000 7fd3941d8740 1 -- 172.21.15.89:0/2486650880 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55dfc5db9c70 con 0x55dfc59632f0 2024-09-21T03:39:52.876 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:39:52.870+0000 7fd36aff5640 1 -- 172.21.15.89:0/2486650880 <== mon.0 v2:172.21.15.89:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd38001c770 con 0x55dfc59632f0 2024-09-21T03:39:52.876 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:39:52.870+0000 7fd36aff5640 1 -- 172.21.15.89:0/2486650880 <== mon.0 v2:172.21.15.89:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fd37810a110 con 0x55dfc59632f0 2024-09-21T03:39:52.877 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:39:52.871+0000 7fd3941d8740 10 client.15573 fetch_fsmap finished waiting for FSMap version 26 2024-09-21T03:39:52.877 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:39:52.871+0000 7fd3941d8740 10 client.15573 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T03:39:52.877 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:39:52.871+0000 7fd3941d8740 20 client.15573 populate_metadata read hostname 'smithi089' 2024-09-21T03:39:52.877 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:39:52.871+0000 7fd3941d8740 1 --2- 172.21.15.89:0/2486650880 >> [v2:172.21.15.110:6838/132366709,v1:172.21.15.110:6839/132366709] conn(0x55dfc59201c0 0x55dfc5dbc510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:52.877 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:39:52.871+0000 7fd3941d8740 4 client.15573 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T03:39:52.877 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:39:52.871+0000 7fd3941d8740 1 -- 172.21.15.89:0/2486650880 --> [v2:172.21.15.110:6838/132366709,v1:172.21.15.110:6839/132366709] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55dfc5ca40f0 con 0x55dfc59201c0 2024-09-21T03:39:52.877 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:39:52.871+0000 7fd38659c640 1 --2- 172.21.15.89:0/2486650880 >> [v2:172.21.15.110:6838/132366709,v1:172.21.15.110:6839/132366709] conn(0x55dfc59201c0 0x55dfc5dbc510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:52.877 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:39:52.871+0000 7fd38659c640 1 --2- 172.21.15.89:0/2486650880 >> [v2:172.21.15.110:6838/132366709,v1:172.21.15.110:6839/132366709] conn(0x55dfc59201c0 0x55dfc5dbc510 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=315e04d3e1a69738 server_cookie=9f7aa5d9dc54ee25 in_seq=0 out_seq=0 2024-09-21T03:39:52.877 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:39:52.871+0000 7fd36aff5640 10 client.15573 ms_handle_connect on v2:172.21.15.110:6838/132366709 2024-09-21T03:39:52.879 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:39:52.878+0000 7fd36aff5640 1 -- 172.21.15.89:0/2486650880 <== mds.0 v2:172.21.15.110:6838/132366709 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55dfc5ca40f0 con 0x55dfc59201c0 2024-09-21T03:39:52.880 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:39:52.878+0000 7fd36aff5640 10 client.15573 handle_command_reply: tid=0 2024-09-21T03:39:52.880 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:39:52.878+0000 7fd3941d8740 1 client.15573 shutdown 2024-09-21T03:39:52.880 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:39:52.878+0000 7fd3941d8740 1 -- 172.21.15.89:0/2486650880 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55dfc59632f0 msgr2=0x55dfc5d495b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:52.880 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:39:52.878+0000 7fd3941d8740 1 --2- 172.21.15.89:0/2486650880 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55dfc59632f0 0x55dfc5d495b0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7fd3780cdca0 tx=0x7fd378009680 comp rx=0 tx=0).stop 2024-09-21T03:39:52.880 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:39:52.878+0000 7fd3941d8740 1 -- 172.21.15.89:0/2486650880 shutdown_connections 2024-09-21T03:39:52.880 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:39:52.878+0000 7fd3941d8740 1 --2- 172.21.15.89:0/2486650880 >> [v2:172.21.15.110:6838/132366709,v1:172.21.15.110:6839/132366709] conn(0x55dfc59201c0 0x55dfc5dbc510 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:52.880 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:39:52.878+0000 7fd3941d8740 1 --2- 172.21.15.89:0/2486650880 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55dfc596b4d0 0x55dfc5d48270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:52.880 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:39:52.878+0000 7fd3941d8740 1 --2- 172.21.15.89:0/2486650880 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55dfc598c1b0 0x55dfc5d487b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:52.880 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:39:52.878+0000 7fd36aff5640 0 client.15573 ms_handle_reset on v2:172.21.15.110:6838/132366709 2024-09-21T03:39:52.880 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:39:52.878+0000 7fd3941d8740 1 --2- 172.21.15.89:0/2486650880 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55dfc59632f0 0x55dfc5d495b0 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:52.880 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:39:52.878+0000 7fd3941d8740 1 -- 172.21.15.89:0/2486650880 >> 172.21.15.89:0/2486650880 conn(0x55dfc5980300 msgr2=0x55dfc5d2f220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:39:52.880 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:39:52.878+0000 7fd3941d8740 1 -- 172.21.15.89:0/2486650880 shutdown_connections 2024-09-21T03:39:52.880 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:39:52.878+0000 7fd3941d8740 1 -- 172.21.15.89:0/2486650880 wait complete. 2024-09-21T03:39:52.881 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:39:52.878+0000 7fd3941d8740 20 client.15573 trim_cache size 0 max 16384 2024-09-21T03:39:52.881 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stdout:{ 2024-09-21T03:39:52.881 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stdout: "message": "", 2024-09-21T03:39:52.881 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stdout: "return_code": 0 2024-09-21T03:39:52.881 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stdout:} 2024-09-21T03:39:52.882 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:39:52.879+0000 7fd38d18b640 1 -- 172.21.15.89:0/382491769 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7fd3700b65a0 msgr2=0x7fd3700b8ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:52.882 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:39:52.879+0000 7fd38d18b640 1 --2- 172.21.15.89:0/382491769 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7fd3700b65a0 0x7fd3700b8ab0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fd380007ed0 tx=0x7fd380007680 comp rx=0 tx=0).stop 2024-09-21T03:39:52.882 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:39:52.879+0000 7fd38d18b640 1 -- 172.21.15.89:0/382491769 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd38806c030 msgr2=0x7fd388199e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:52.882 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:39:52.879+0000 7fd38d18b640 1 --2- 172.21.15.89:0/382491769 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd38806c030 0x7fd388199e60 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7fd378009230 tx=0x7fd3780042f0 comp rx=0 tx=0).stop 2024-09-21T03:39:52.882 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:39:52.879+0000 7fd38d18b640 1 -- 172.21.15.89:0/382491769 shutdown_connections 2024-09-21T03:39:52.882 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:39:52.879+0000 7fd38d18b640 1 --2- 172.21.15.89:0/382491769 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7fd3700b65a0 0x7fd3700b8ab0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:52.882 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:39:52.879+0000 7fd38d18b640 1 --2- 172.21.15.89:0/382491769 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd3881068d0 0x7fd38819a4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:52.883 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:39:52.879+0000 7fd38d18b640 1 --2- 172.21.15.89:0/382491769 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd3881093e0 0x7fd38819b080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:52.883 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:39:52.879+0000 7fd38d18b640 1 --2- 172.21.15.89:0/382491769 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd38806c030 0x7fd388199e60 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:52.883 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:39:52.879+0000 7fd38d18b640 1 -- 172.21.15.89:0/382491769 >> 172.21.15.89:0/382491769 conn(0x7fd3880fdf30 msgr2=0x7fd388104de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:39:52.883 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:39:52.879+0000 7fd38d18b640 1 -- 172.21.15.89:0/382491769 shutdown_connections 2024-09-21T03:39:52.883 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:39:52.879+0000 7fd38d18b640 1 -- 172.21.15.89:0/382491769 wait complete. 2024-09-21T03:39:53.461 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:39:53 smithi114 ceph-mon[34140]: pgmap v374: 129 pgs: 129 active+clean; 673 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 2.4 KiB/s wr, 9 op/s 2024-09-21T03:39:53.500 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:39:53 smithi110 ceph-mon[34000]: pgmap v374: 129 pgs: 129 active+clean; 673 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 2.4 KiB/s wr, 9 op/s 2024-09-21T03:39:53.561 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:39:53 smithi089 ceph-mon[30928]: pgmap v374: 129 pgs: 129 active+clean; 673 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 2.4 KiB/s wr, 9 op/s 2024-09-21T03:39:55.461 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:39:55 smithi114 ceph-mon[34140]: pgmap v375: 129 pgs: 129 active+clean; 673 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 2.6 KiB/s wr, 11 op/s 2024-09-21T03:39:55.500 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:39:55 smithi110 ceph-mon[34000]: pgmap v375: 129 pgs: 129 active+clean; 673 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 2.6 KiB/s wr, 11 op/s 2024-09-21T03:39:55.561 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:39:55 smithi089 ceph-mon[30928]: pgmap v375: 129 pgs: 129 active+clean; 673 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 2.6 KiB/s wr, 11 op/s 2024-09-21T03:39:57.461 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:39:57 smithi114 ceph-mon[34140]: pgmap v376: 129 pgs: 129 active+clean; 674 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 2.1 KiB/s wr, 9 op/s 2024-09-21T03:39:57.500 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:39:57 smithi110 ceph-mon[34000]: pgmap v376: 129 pgs: 129 active+clean; 674 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 2.1 KiB/s wr, 9 op/s 2024-09-21T03:39:57.561 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:39:57 smithi089 ceph-mon[30928]: pgmap v376: 129 pgs: 129 active+clean; 674 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 2.1 KiB/s wr, 9 op/s 2024-09-21T03:39:59.008 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 14.5 2024-09-21T03:39:59.008 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 7c374098-77c9-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T03:39:59.276 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/7c374098-77c9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:39:59.311 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:39:59 smithi089 ceph-mon[30928]: pgmap v377: 129 pgs: 129 active+clean; 674 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 767 B/s wr, 7 op/s 2024-09-21T03:39:59.461 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:39:59 smithi114 ceph-mon[34140]: pgmap v377: 129 pgs: 129 active+clean; 674 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 767 B/s wr, 7 op/s 2024-09-21T03:39:59.500 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:39:59 smithi110 ceph-mon[34000]: pgmap v377: 129 pgs: 129 active+clean; 674 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 767 B/s wr, 7 op/s 2024-09-21T03:39:59.810 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:59.808+0000 7f29e669b640 1 -- 172.21.15.89:0/767209655 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f29e0102a60 msgr2=0x7f29e0102e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:59.810 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:59.808+0000 7f29e669b640 1 --2- 172.21.15.89:0/767209655 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f29e0102a60 0x7f29e0102e70 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f29d0009900 tx=0x7f29d002a410 comp rx=0 tx=0).stop 2024-09-21T03:39:59.810 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:59.808+0000 7f29e669b640 1 -- 172.21.15.89:0/767209655 shutdown_connections 2024-09-21T03:39:59.811 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:59.808+0000 7f29e669b640 1 --2- 172.21.15.89:0/767209655 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f29e0108cf0 0x7f29e010f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:59.811 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:59.808+0000 7f29e669b640 1 --2- 172.21.15.89:0/767209655 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f29e01083a0 0x7f29e01087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:59.811 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:59.808+0000 7f29e669b640 1 --2- 172.21.15.89:0/767209655 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f29e0102a60 0x7f29e0102e70 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:59.811 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:59.808+0000 7f29e669b640 1 -- 172.21.15.89:0/767209655 >> 172.21.15.89:0/767209655 conn(0x7f29e0079730 msgr2=0x7f29e0079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:39:59.811 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:59.808+0000 7f29e669b640 1 -- 172.21.15.89:0/767209655 shutdown_connections 2024-09-21T03:39:59.811 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:59.808+0000 7f29e669b640 1 -- 172.21.15.89:0/767209655 wait complete. 2024-09-21T03:39:59.812 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:59.808+0000 7f29e669b640 1 Processor -- start 2024-09-21T03:39:59.813 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:59.809+0000 7f29e669b640 1 -- start start 2024-09-21T03:39:59.813 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:59.809+0000 7f29e669b640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f29e0102a60 0x7f29e019b1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:59.813 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:59.809+0000 7f29e669b640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f29e01083a0 0x7f29e019b6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:59.813 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:59.809+0000 7f29e669b640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f29e0108cf0 0x7f29e019bc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:59.813 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:59.809+0000 7f29e669b640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f29e0112860 con 0x7f29e0108cf0 2024-09-21T03:39:59.813 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:59.809+0000 7f29e669b640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f29e01126e0 con 0x7f29e0102a60 2024-09-21T03:39:59.813 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:59.809+0000 7f29e669b640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f29e01129e0 con 0x7f29e01083a0 2024-09-21T03:39:59.813 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:59.809+0000 7f29e4c38640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f29e0108cf0 0x7f29e019bc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:59.813 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:59.809+0000 7f29e4c38640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f29e0108cf0 0x7f29e019bc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:33080/0 (socket says 172.21.15.89:33080) 2024-09-21T03:39:59.813 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:59.809+0000 7f29e4c38640 1 -- 172.21.15.89:0/1603004752 learned_addr learned my addr 172.21.15.89:0/1603004752 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-21T03:39:59.813 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:59.809+0000 7f29dffff640 1 --2- 172.21.15.89:0/1603004752 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f29e0102a60 0x7f29e019b1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:59.813 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:59.809+0000 7f29df7fe640 1 --2- 172.21.15.89:0/1603004752 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f29e01083a0 0x7f29e019b6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:59.813 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:59.810+0000 7f29e4c38640 1 -- 172.21.15.89:0/1603004752 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f29e01083a0 msgr2=0x7f29e019b6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:59.814 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:59.810+0000 7f29e4c38640 1 --2- 172.21.15.89:0/1603004752 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f29e01083a0 0x7f29e019b6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:59.814 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:59.810+0000 7f29e4c38640 1 -- 172.21.15.89:0/1603004752 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f29e0102a60 msgr2=0x7f29e019b1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:59.814 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:59.810+0000 7f29e4c38640 1 --2- 172.21.15.89:0/1603004752 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f29e0102a60 0x7f29e019b1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:59.814 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:59.810+0000 7f29e4c38640 1 -- 172.21.15.89:0/1603004752 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29e01a63e0 con 0x7f29e0108cf0 2024-09-21T03:39:59.814 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:59.810+0000 7f29df7fe640 1 --2- 172.21.15.89:0/1603004752 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f29e01083a0 0x7f29e019b6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:39:59.814 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:59.810+0000 7f29e4c38640 1 --2- 172.21.15.89:0/1603004752 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f29e0108cf0 0x7f29e019bc30 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f29d800cce0 tx=0x7f29d800a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:39:59.814 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:59.810+0000 7f29dd7fa640 1 -- 172.21.15.89:0/1603004752 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29d800f450 con 0x7f29e0108cf0 2024-09-21T03:39:59.814 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:59.810+0000 7f29e669b640 1 -- 172.21.15.89:0/1603004752 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f29e01a66d0 con 0x7f29e0108cf0 2024-09-21T03:39:59.814 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:59.810+0000 7f29dd7fa640 1 -- 172.21.15.89:0/1603004752 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f29d800fa80 con 0x7f29e0108cf0 2024-09-21T03:39:59.814 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:59.810+0000 7f29dd7fa640 1 -- 172.21.15.89:0/1603004752 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29d8018ad0 con 0x7f29e0108cf0 2024-09-21T03:39:59.814 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:59.810+0000 7f29e669b640 1 -- 172.21.15.89:0/1603004752 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29e01a6bc0 con 0x7f29e0108cf0 2024-09-21T03:39:59.815 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:59.811+0000 7f29e669b640 1 -- 172.21.15.89:0/1603004752 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f29e0102e70 con 0x7f29e0108cf0 2024-09-21T03:39:59.816 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:59.814+0000 7f29dd7fa640 1 -- 172.21.15.89:0/1603004752 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 20) ==== 154563+0+0 (secure 0 0 0) 0x7f29d800f5f0 con 0x7f29e0108cf0 2024-09-21T03:39:59.816 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:59.815+0000 7f29dd7fa640 1 --2- 172.21.15.89:0/1603004752 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f29c80b65a0 0x7f29c80b8ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:59.817 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:59.815+0000 7f29dffff640 1 --2- 172.21.15.89:0/1603004752 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f29c80b65a0 0x7f29c80b8ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:59.817 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:59.815+0000 7f29dd7fa640 1 -- 172.21.15.89:0/1603004752 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 9121+0+0 (secure 0 0 0) 0x7f29d80e80e0 con 0x7f29e0108cf0 2024-09-21T03:39:59.817 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:59.815+0000 7f29dffff640 1 --2- 172.21.15.89:0/1603004752 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f29c80b65a0 0x7f29c80b8ab0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f29e006b9e0 tx=0x7f29d0000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:39:59.820 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:39:59.818+0000 7f29dd7fa640 1 -- 172.21.15.89:0/1603004752 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f29d8014030 con 0x7f29e0108cf0 2024-09-21T03:40:00.050 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:00.048+0000 7f29e669b640 1 -- 172.21.15.89:0/1603004752 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f29e006b290 con 0x7f29e0108cf0 2024-09-21T03:40:00.051 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:00.049+0000 7f29dd7fa640 1 -- 172.21.15.89:0/1603004752 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11912 (secure 0 0 0) 0x7f29d80ae520 con 0x7f29e0108cf0 2024-09-21T03:40:00.051 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-09-21T03:40:00.051 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":26,"btime":"2024-09-21T03:39:26:058611+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14721,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6835/2341538514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":2341538514},{"type":"v1","addr":"172.21.15.89:6835","nonce":2341538514}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14733,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6837/146708592","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6836","nonce":146708592},{"type":"v1","addr":"172.21.15.89:6837","nonce":146708592}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14781,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6839/2269965499","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6838","nonce":2269965499},{"type":"v1","addr":"172.21.15.89:6839","nonce":2269965499}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14793,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6841/650850383","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6840","nonce":650850383},{"type":"v1","addr":"172.21.15.89:6841","nonce":650850383}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24565,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6833/752336912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6832","nonce":752336912},{"type":"v1","addr":"172.21.15.114:6833","nonce":752336912}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24575,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6833/127607331","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6832","nonce":127607331},{"type":"v1","addr":"172.21.15.110:6833","nonce":127607331}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T03:37:13.570694+0000","modified":"2024-09-21T03:39:25.333954+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24638,"mds_1":24631,"mds_2":24614},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24593":{"gid":24593,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.110:6835/671820654","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6834","nonce":671820654},{"type":"v1","addr":"172.21.15.110:6835","nonce":671820654}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24604":{"gid":24604,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.114:6835/1049714548","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6834","nonce":1049714548},{"type":"v1","addr":"172.21.15.114:6835","nonce":1049714548}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24610":{"gid":24610,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.114:6837/1941213818","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6836","nonce":1941213818},{"type":"v1","addr":"172.21.15.114:6837","nonce":1941213818}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24614":{"gid":24614,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":15,"addr":"172.21.15.110:6837/4047124589","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6836","nonce":4047124589},{"type":"v1","addr":"172.21.15.110:6837","nonce":4047124589}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24631":{"gid":24631,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":15,"addr":"172.21.15.114:6839/3427369944","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6838","nonce":3427369944},{"type":"v1","addr":"172.21.15.114:6839","nonce":3427369944}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24638":{"gid":24638,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.110:6839/132366709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6838","nonce":132366709},{"type":"v1","addr":"172.21.15.110:6839","nonce":132366709}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24638,"qdb_cluster":[24638,24631,24614]},"id":1}]} 2024-09-21T03:40:00.051 INFO:teuthology.orchestra.run.smithi089.stderr:dumped fsmap epoch 26 2024-09-21T03:40:00.053 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:00.051+0000 7f29e669b640 1 -- 172.21.15.89:0/1603004752 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f29c80b65a0 msgr2=0x7f29c80b8ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:00.053 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:00.051+0000 7f29e669b640 1 --2- 172.21.15.89:0/1603004752 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f29c80b65a0 0x7f29c80b8ab0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f29e006b9e0 tx=0x7f29d0000bc0 comp rx=0 tx=0).stop 2024-09-21T03:40:00.054 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:00.051+0000 7f29e669b640 1 -- 172.21.15.89:0/1603004752 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f29e0108cf0 msgr2=0x7f29e019bc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:00.054 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:00.051+0000 7f29e669b640 1 --2- 172.21.15.89:0/1603004752 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f29e0108cf0 0x7f29e019bc30 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f29d800cce0 tx=0x7f29d800a620 comp rx=0 tx=0).stop 2024-09-21T03:40:00.054 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:00.051+0000 7f29e669b640 1 -- 172.21.15.89:0/1603004752 shutdown_connections 2024-09-21T03:40:00.054 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:00.051+0000 7f29e669b640 1 --2- 172.21.15.89:0/1603004752 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f29c80b65a0 0x7f29c80b8ab0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:00.054 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:00.051+0000 7f29e669b640 1 --2- 172.21.15.89:0/1603004752 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f29e0108cf0 0x7f29e019bc30 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:00.054 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:00.051+0000 7f29e669b640 1 --2- 172.21.15.89:0/1603004752 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f29e01083a0 0x7f29e019b6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:00.054 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:00.051+0000 7f29e669b640 1 --2- 172.21.15.89:0/1603004752 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f29e0102a60 0x7f29e019b1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:00.054 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:00.051+0000 7f29e669b640 1 -- 172.21.15.89:0/1603004752 >> 172.21.15.89:0/1603004752 conn(0x7f29e0079730 msgr2=0x7f29e01014e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:40:00.054 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:00.051+0000 7f29e669b640 1 -- 172.21.15.89:0/1603004752 shutdown_connections 2024-09-21T03:40:00.054 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:00.051+0000 7f29e669b640 1 -- 172.21.15.89:0/1603004752 wait complete. 2024-09-21T03:40:00.311 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:40:00 smithi089 ceph-mon[30928]: overall HEALTH_OK 2024-09-21T03:40:00.311 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:40:00 smithi089 ceph-mon[30928]: from='client.? 172.21.15.89:0/1603004752' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:40:00.461 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:40:00 smithi114 ceph-mon[34140]: overall HEALTH_OK 2024-09-21T03:40:00.461 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:40:00 smithi114 ceph-mon[34140]: from='client.? 172.21.15.89:0/1603004752' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:40:00.500 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:40:00 smithi110 ceph-mon[34000]: overall HEALTH_OK 2024-09-21T03:40:00.500 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:40:00 smithi110 ceph-mon[34000]: from='client.? 172.21.15.89:0/1603004752' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:40:00.732 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24638 quiesce db / --timeout 90 --expiration 134.5 --await-for 45.0' 2024-09-21T03:40:00.732 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 7c374098-77c9-11ef-baf5-efdc52797490 -- timeout 105 ceph tell mds.24638 quiesce db / --timeout 90 --expiration 134.5 --await-for 45.0 2024-09-21T03:40:00.995 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/7c374098-77c9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:40:01.391 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:40:01 smithi089 ceph-mon[30928]: pgmap v378: 129 pgs: 129 active+clean; 674 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 767 B/s wr, 8 op/s 2024-09-21T03:40:01.461 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:40:01 smithi114 ceph-mon[34140]: pgmap v378: 129 pgs: 129 active+clean; 674 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 767 B/s wr, 8 op/s 2024-09-21T03:40:01.499 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:01.497+0000 7f00d1e1d640 1 -- 172.21.15.89:0/755115831 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f00cc102a60 msgr2=0x7f00cc102e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:01.500 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:01.497+0000 7f00d1e1d640 1 --2- 172.21.15.89:0/755115831 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f00cc102a60 0x7f00cc102e70 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f00bc009900 tx=0x7f00bc02a410 comp rx=0 tx=0).stop 2024-09-21T03:40:01.500 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:01.497+0000 7f00d1e1d640 1 -- 172.21.15.89:0/755115831 shutdown_connections 2024-09-21T03:40:01.500 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:01.497+0000 7f00d1e1d640 1 --2- 172.21.15.89:0/755115831 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f00cc108cf0 0x7f00cc10f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:01.500 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:01.497+0000 7f00d1e1d640 1 --2- 172.21.15.89:0/755115831 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f00cc1083a0 0x7f00cc1087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:01.500 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:01.497+0000 7f00d1e1d640 1 --2- 172.21.15.89:0/755115831 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f00cc102a60 0x7f00cc102e70 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:01.500 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:01.497+0000 7f00d1e1d640 1 -- 172.21.15.89:0/755115831 >> 172.21.15.89:0/755115831 conn(0x7f00cc0796d0 msgr2=0x7f00cc079ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:40:01.500 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:01.497+0000 7f00d1e1d640 1 -- 172.21.15.89:0/755115831 shutdown_connections 2024-09-21T03:40:01.500 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:01.497+0000 7f00d1e1d640 1 -- 172.21.15.89:0/755115831 wait complete. 2024-09-21T03:40:01.501 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:40:01 smithi110 ceph-mon[34000]: pgmap v378: 129 pgs: 129 active+clean; 674 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 767 B/s wr, 8 op/s 2024-09-21T03:40:01.502 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:01.498+0000 7f00d1e1d640 1 Processor -- start 2024-09-21T03:40:01.502 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:01.498+0000 7f00d1e1d640 1 -- start start 2024-09-21T03:40:01.502 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:01.498+0000 7f00d1e1d640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f00cc102a60 0x7f00cc19a370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:01.503 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:01.498+0000 7f00d1e1d640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f00cc1083a0 0x7f00cc19a910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:01.503 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:01.498+0000 7f00d1e1d640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f00cc108cf0 0x7f00cc19fbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:01.503 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:01.498+0000 7f00d1e1d640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f00cc113980 con 0x7f00cc102a60 2024-09-21T03:40:01.503 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:01.498+0000 7f00d1e1d640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f00cc113800 con 0x7f00cc1083a0 2024-09-21T03:40:01.503 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:01.498+0000 7f00d1e1d640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f00cc113b00 con 0x7f00cc108cf0 2024-09-21T03:40:01.503 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:01.498+0000 7f00cb7fe640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f00cc102a60 0x7f00cc19a370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:01.503 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:01.498+0000 7f00cb7fe640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f00cc102a60 0x7f00cc19a370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:33096/0 (socket says 172.21.15.89:33096) 2024-09-21T03:40:01.503 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:01.498+0000 7f00cb7fe640 1 -- 172.21.15.89:0/2670811091 learned_addr learned my addr 172.21.15.89:0/2670811091 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-21T03:40:01.503 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:01.499+0000 7f00cbfff640 1 --2- 172.21.15.89:0/2670811091 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f00cc108cf0 0x7f00cc19fbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:01.503 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:01.499+0000 7f00c3fff640 1 --2- 172.21.15.89:0/2670811091 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f00cc1083a0 0x7f00cc19a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:01.503 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:01.499+0000 7f00cb7fe640 1 -- 172.21.15.89:0/2670811091 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f00cc108cf0 msgr2=0x7f00cc19fbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:01.503 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:01.499+0000 7f00cb7fe640 1 --2- 172.21.15.89:0/2670811091 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f00cc108cf0 0x7f00cc19fbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:01.503 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:01.499+0000 7f00cb7fe640 1 -- 172.21.15.89:0/2670811091 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f00cc1083a0 msgr2=0x7f00cc19a910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:01.503 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:01.499+0000 7f00cb7fe640 1 --2- 172.21.15.89:0/2670811091 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f00cc1083a0 0x7f00cc19a910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:01.504 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:01.499+0000 7f00cb7fe640 1 -- 172.21.15.89:0/2670811091 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00cc1a0220 con 0x7f00cc102a60 2024-09-21T03:40:01.504 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:01.499+0000 7f00cb7fe640 1 --2- 172.21.15.89:0/2670811091 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f00cc102a60 0x7f00cc19a370 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f00bc009230 tx=0x7f00bc004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:01.504 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:01.499+0000 7f00c97fa640 1 -- 172.21.15.89:0/2670811091 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00bc03f030 con 0x7f00cc102a60 2024-09-21T03:40:01.504 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:01.499+0000 7f00d1e1d640 1 -- 172.21.15.89:0/2670811091 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f00cc1a04b0 con 0x7f00cc102a60 2024-09-21T03:40:01.504 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:01.499+0000 7f00c97fa640 1 -- 172.21.15.89:0/2670811091 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f00bc003a80 con 0x7f00cc102a60 2024-09-21T03:40:01.504 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:01.499+0000 7f00c97fa640 1 -- 172.21.15.89:0/2670811091 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00bc035070 con 0x7f00cc102a60 2024-09-21T03:40:01.504 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:01.499+0000 7f00d1e1d640 1 -- 172.21.15.89:0/2670811091 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00cc1a0990 con 0x7f00cc102a60 2024-09-21T03:40:01.505 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:01.502+0000 7f00d8391740 1 Processor -- start 2024-09-21T03:40:01.505 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:01.502+0000 7f00d8391740 1 -- start start 2024-09-21T03:40:01.505 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:01.502+0000 7f00d8391740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x565247fcd840 0x56524837ce10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:01.505 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:01.502+0000 7f00d8391740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x565247fa4980 0x56524837d350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:01.505 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:01.502+0000 7f00d8391740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x565247facb60 0x56524837d890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:01.506 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:01.502+0000 7f00d8391740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x5652482eab90 con 0x565247fa4980 2024-09-21T03:40:01.506 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:01.502+0000 7f00d8391740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x5652482d9fc0 con 0x565247fcd840 2024-09-21T03:40:01.506 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:01.502+0000 7f00d8391740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x5652482dddd0 con 0x565247facb60 2024-09-21T03:40:01.506 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:01.503+0000 7f00cbfff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x565247fcd840 0x56524837ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:01.506 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:01.503+0000 7f00cbfff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x565247fcd840 0x56524837ce10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.89:56994/0 (socket says 172.21.15.89:56994) 2024-09-21T03:40:01.506 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:01.503+0000 7f00cbfff640 1 -- 172.21.15.89:0/4257733261 learned_addr learned my addr 172.21.15.89:0/4257733261 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-21T03:40:01.506 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:01.503+0000 7f00c3fff640 1 --2- 172.21.15.89:0/4257733261 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x565247facb60 0x56524837d890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:01.506 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:01.503+0000 7f00cbfff640 1 -- 172.21.15.89:0/4257733261 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x565247facb60 msgr2=0x56524837d890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:40:01.506 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:01.503+0000 7f00cbfff640 1 --2- 172.21.15.89:0/4257733261 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x565247facb60 0x56524837d890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:01.506 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:01.503+0000 7f00cbfff640 1 -- 172.21.15.89:0/4257733261 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x565247fa4980 msgr2=0x56524837d350 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T03:40:01.506 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:01.503+0000 7f00cbfff640 1 --2- 172.21.15.89:0/4257733261 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x565247fa4980 0x56524837d350 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:01.506 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:01.503+0000 7f00cbfff640 1 -- 172.21.15.89:0/4257733261 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x565248261060 con 0x565247fcd840 2024-09-21T03:40:01.506 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:01.503+0000 7f00cbfff640 1 --2- 172.21.15.89:0/4257733261 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x565247fcd840 0x56524837ce10 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f00c4010370 tx=0x7f00c4010830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=725a9d33eca4ca93 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:01.510 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:01.503+0000 7f00c97fa640 1 -- 172.21.15.89:0/2670811091 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 20) ==== 154563+0+0 (secure 0 0 0) 0x7f00bc042030 con 0x7f00cc102a60 2024-09-21T03:40:01.510 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:01.504+0000 7f00c1ffb640 1 -- 172.21.15.89:0/4257733261 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00c4016070 con 0x565247fcd840 2024-09-21T03:40:01.510 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:01.504+0000 7f00c1ffb640 1 -- 172.21.15.89:0/4257733261 <== mon.2 v2:172.21.15.110:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f00c4004d20 con 0x565247fcd840 2024-09-21T03:40:01.510 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:01.504+0000 7f00c1ffb640 1 -- 172.21.15.89:0/4257733261 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00c4005280 con 0x565247fcd840 2024-09-21T03:40:01.510 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:01.504+0000 7f00c97fa640 1 --2- 172.21.15.89:0/2670811091 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f00b80b65d0 0x7f00b80b8ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:01.510 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:01.504+0000 7f00cb7fe640 1 --2- 172.21.15.89:0/2670811091 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f00b80b65d0 0x7f00b80b8ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:01.510 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:01.505+0000 7f00d8391740 1 -- 172.21.15.89:0/4257733261 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x565247fcd840 msgr2=0x56524837ce10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:01.511 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:01.505+0000 7f00d8391740 1 --2- 172.21.15.89:0/4257733261 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x565247fcd840 0x56524837ce10 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f00c4010370 tx=0x7f00c4010830 comp rx=0 tx=0).stop 2024-09-21T03:40:01.511 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:01.505+0000 7f00c97fa640 1 -- 172.21.15.89:0/2670811091 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 9121+0+0 (secure 0 0 0) 0x7f00bc107d90 con 0x7f00cc102a60 2024-09-21T03:40:01.511 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:01.505+0000 7f00d8391740 1 -- 172.21.15.89:0/4257733261 shutdown_connections 2024-09-21T03:40:01.511 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:01.505+0000 7f00d8391740 1 --2- 172.21.15.89:0/4257733261 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x565247facb60 0x56524837d890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:01.511 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:01.505+0000 7f00d8391740 1 --2- 172.21.15.89:0/4257733261 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x565247fa4980 0x56524837d350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:01.511 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:01.505+0000 7f00d8391740 1 --2- 172.21.15.89:0/4257733261 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x565247fcd840 0x56524837ce10 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:01.511 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:01.505+0000 7f00d8391740 1 -- 172.21.15.89:0/4257733261 >> 172.21.15.89:0/4257733261 conn(0x565247fc1990 msgr2=0x56524837ad00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:40:01.511 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:01.505+0000 7f00d8391740 1 -- 172.21.15.89:0/4257733261 shutdown_connections 2024-09-21T03:40:01.511 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:01.505+0000 7f00d8391740 1 -- 172.21.15.89:0/4257733261 wait complete. 2024-09-21T03:40:01.511 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:01.505+0000 7f00cb7fe640 1 --2- 172.21.15.89:0/2670811091 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f00b80b65d0 0x7f00b80b8ae0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x56524837ba90 tx=0x7f00bc0b5040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:01.511 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:01.505+0000 7f00d8391740 1 -- start start 2024-09-21T03:40:01.511 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:01.506+0000 7f00d8391740 1 Processor -- start 2024-09-21T03:40:01.511 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:01.506+0000 7f00d8391740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x565247fa4980 0x565248388a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:01.511 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:01.506+0000 7f00d8391740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x565247facb60 0x565248388f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:01.512 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:01.506+0000 7f00d8391740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x565247fcd840 0x5652483948e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:01.512 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:01.506+0000 7f00d8391740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x56524813b7a0 con 0x565247fcd840 2024-09-21T03:40:01.512 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:01.506+0000 7f00d8391740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x56524827f0a0 con 0x565247fa4980 2024-09-21T03:40:01.512 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:01.506+0000 7f00d8391740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x56524838b5f0 con 0x565247facb60 2024-09-21T03:40:01.512 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:01.506+0000 7f00d8391740 10 client.0 _ll_register_callbacks cb 0x56524825c3c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:40:01.512 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:01.506+0000 7f00cbfff640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x565247fcd840 0x5652483948e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:01.512 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:01.506+0000 7f00cbfff640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x565247fcd840 0x5652483948e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:33102/0 (socket says 172.21.15.89:33102) 2024-09-21T03:40:01.512 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:01.506+0000 7f00cbfff640 1 -- 172.21.15.89:0/3025223243 learned_addr learned my addr 172.21.15.89:0/3025223243 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-21T03:40:01.512 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:01.506+0000 7f00cbfff640 1 --2- 172.21.15.89:0/3025223243 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x565247fa4980 0x565248388a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:01.512 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:01.506+0000 7f00c3fff640 1 --2- 172.21.15.89:0/3025223243 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x565247facb60 0x565248388f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:01.512 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:01.507+0000 7f00cbfff640 1 -- 172.21.15.89:0/3025223243 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x565247facb60 msgr2=0x565248388f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:40:01.512 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:01.507+0000 7f00cbfff640 1 --2- 172.21.15.89:0/3025223243 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x565247facb60 0x565248388f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:01.512 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:01.507+0000 7f00cbfff640 1 -- 172.21.15.89:0/3025223243 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x565247fa4980 msgr2=0x565248388a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:40:01.512 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:01.507+0000 7f00cbfff640 1 --2- 172.21.15.89:0/3025223243 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x565247fa4980 0x565248388a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:01.512 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:01.507+0000 7f00cbfff640 1 -- 172.21.15.89:0/3025223243 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56524826d880 con 0x565247fcd840 2024-09-21T03:40:01.513 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:01.507+0000 7f00cbfff640 1 --2- 172.21.15.89:0/3025223243 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x565247fcd840 0x5652483948e0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x56524837bc40 tx=0x7f00c4013fc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7fdca9eee0ea878 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:01.513 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:01.507+0000 7f00aaffd640 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2024-09-21T03:40:01.513 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:01.507+0000 7f00aaffd640 1 -- 172.21.15.89:0/3025223243 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00c4016070 con 0x565247fcd840 2024-09-21T03:40:01.513 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:01.507+0000 7f00d8391740 1 -- 172.21.15.89:0/3025223243 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f00c4004d20 con 0x565247fcd840 2024-09-21T03:40:01.513 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:01.507+0000 7f00aaffd640 1 -- 172.21.15.89:0/3025223243 <== mon.0 v2:172.21.15.89:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 9121+0+0 (secure 0 0 0) 0x7f00c40059d0 con 0x565247fcd840 2024-09-21T03:40:01.513 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:01.507+0000 7f00aaffd640 1 -- 172.21.15.89:0/3025223243 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f00c4033050 con 0x565247fcd840 2024-09-21T03:40:01.513 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:01.507+0000 7f00d8391740 10 client.15588 fetch_fsmap learned FSMap version 26 2024-09-21T03:40:01.513 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:01.507+0000 7f00d8391740 1 -- 172.21.15.89:0/3025223243 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x565248047b90 con 0x565247fcd840 2024-09-21T03:40:01.513 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:01.507+0000 7f00aaffd640 1 -- 172.21.15.89:0/3025223243 <== mon.0 v2:172.21.15.89:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f00c4004d20 con 0x565247fcd840 2024-09-21T03:40:01.513 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:01.508+0000 7f00aaffd640 1 -- 172.21.15.89:0/3025223243 <== mon.0 v2:172.21.15.89:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f00c400f040 con 0x565247fcd840 2024-09-21T03:40:01.513 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:01.508+0000 7f00d8391740 10 client.15588 fetch_fsmap finished waiting for FSMap version 26 2024-09-21T03:40:01.513 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:01.508+0000 7f00d8391740 10 client.15588 resolve_mds: validated gid 24638 aka daemon mds.b 2024-09-21T03:40:01.513 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:01.508+0000 7f00d8391740 20 client.15588 populate_metadata read hostname 'smithi089' 2024-09-21T03:40:01.513 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:01.508+0000 7f00d8391740 1 --2- 172.21.15.89:0/3025223243 >> [v2:172.21.15.110:6838/132366709,v1:172.21.15.110:6839/132366709] conn(0x565247f8c1c0 0x56524839acb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:01.514 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:01.508+0000 7f00d8391740 4 client.15588 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:40:01.514 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:01.508+0000 7f00d8391740 1 -- 172.21.15.89:0/3025223243 --> [v2:172.21.15.110:6838/132366709,v1:172.21.15.110:6839/132366709] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5652482ecae0 con 0x565247f8c1c0 2024-09-21T03:40:01.514 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:01.508+0000 7f00c3fff640 1 --2- 172.21.15.89:0/3025223243 >> [v2:172.21.15.110:6838/132366709,v1:172.21.15.110:6839/132366709] conn(0x565247f8c1c0 0x56524839acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:01.515 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:01.509+0000 7f00aaffd640 10 client.15588 ms_handle_connect on v2:172.21.15.110:6838/132366709 2024-09-21T03:40:01.515 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:01.509+0000 7f00c3fff640 1 --2- 172.21.15.89:0/3025223243 >> [v2:172.21.15.110:6838/132366709,v1:172.21.15.110:6839/132366709] conn(0x565247f8c1c0 0x56524839acb0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bea8278d28dba829 server_cookie=303ebfdb8e517cbc in_seq=0 out_seq=0 2024-09-21T03:40:01.515 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:01.509+0000 7f00aaffd640 1 -- 172.21.15.89:0/3025223243 <== mds.0 v2:172.21.15.110:6838/132366709 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5652482ecae0 con 0x565247f8c1c0 2024-09-21T03:40:01.515 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:01.509+0000 7f00aaffd640 10 client.15588 handle_command_reply: tid=0 2024-09-21T03:40:01.515 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:01.509+0000 7f00d8391740 1 client.15588 shutdown 2024-09-21T03:40:01.515 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:01.510+0000 7f00d8391740 1 -- 172.21.15.89:0/3025223243 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x565247fcd840 msgr2=0x5652483948e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:01.515 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:01.510+0000 7f00d8391740 1 --2- 172.21.15.89:0/3025223243 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x565247fcd840 0x5652483948e0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x56524837bc40 tx=0x7f00c4013fc0 comp rx=0 tx=0).stop 2024-09-21T03:40:01.516 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:01.510+0000 7f00d8391740 1 -- 172.21.15.89:0/3025223243 shutdown_connections 2024-09-21T03:40:01.516 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:01.510+0000 7f00d8391740 1 --2- 172.21.15.89:0/3025223243 >> [v2:172.21.15.110:6838/132366709,v1:172.21.15.110:6839/132366709] conn(0x565247f8c1c0 0x56524839acb0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:01.516 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:01.510+0000 7f00d8391740 1 --2- 172.21.15.89:0/3025223243 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x565247fcd840 0x5652483948e0 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:01.516 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:01.510+0000 7f00d8391740 1 --2- 172.21.15.89:0/3025223243 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x565247facb60 0x565248388f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:01.516 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:01.510+0000 7f00d8391740 1 --2- 172.21.15.89:0/3025223243 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x565247fa4980 0x565248388a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:01.516 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:01.510+0000 7f00d8391740 1 -- 172.21.15.89:0/3025223243 >> 172.21.15.89:0/3025223243 conn(0x565247fc1990 msgr2=0x56524837fff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:40:01.516 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:01.510+0000 7f00aaffd640 0 client.15588 ms_handle_reset on v2:172.21.15.110:6838/132366709 2024-09-21T03:40:01.516 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:01.510+0000 7f00d8391740 1 -- 172.21.15.89:0/3025223243 shutdown_connections 2024-09-21T03:40:01.516 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:01.510+0000 7f00d8391740 1 -- 172.21.15.89:0/3025223243 wait complete. 2024-09-21T03:40:01.516 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:01.510+0000 7f00d8391740 20 client.15588 trim_cache size 0 max 16384 2024-09-21T03:40:01.529 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:01.527+0000 7f00d8391740 1 Processor -- start 2024-09-21T03:40:01.529 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:01.527+0000 7f00d8391740 1 -- start start 2024-09-21T03:40:01.531 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:01.527+0000 7f00d8391740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x565247fa4980 0x56524837ac80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:01.531 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:01.527+0000 7f00d8391740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x565247facb60 0x5652483947e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:01.531 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:01.528+0000 7f00d8391740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x565247fcd840 0x565248394d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:01.531 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:01.528+0000 7f00d8391740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f00cc1b1010 con 0x565247fa4980 2024-09-21T03:40:01.531 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:01.528+0000 7f00d8391740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f00cc1ae8e0 con 0x565247fcd840 2024-09-21T03:40:01.531 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:01.528+0000 7f00d8391740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f00cc1ac730 con 0x565247facb60 2024-09-21T03:40:01.531 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:01.528+0000 7f00cbfff640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x565247fa4980 0x56524837ac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:01.531 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:01.528+0000 7f00cbfff640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x565247fa4980 0x56524837ac80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:33108/0 (socket says 172.21.15.89:33108) 2024-09-21T03:40:01.531 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:01.528+0000 7f00cbfff640 1 -- 172.21.15.89:0/4138758812 learned_addr learned my addr 172.21.15.89:0/4138758812 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-21T03:40:01.531 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:01.528+0000 7f00c3fff640 1 --2- 172.21.15.89:0/4138758812 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x565247facb60 0x5652483947e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:01.531 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:01.528+0000 7f00cbfff640 1 -- 172.21.15.89:0/4138758812 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x565247facb60 msgr2=0x5652483947e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:40:01.531 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:01.528+0000 7f00cbfff640 1 --2- 172.21.15.89:0/4138758812 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x565247facb60 0x5652483947e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:01.531 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:01.528+0000 7f00cbfff640 1 -- 172.21.15.89:0/4138758812 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x565247fcd840 msgr2=0x565248394d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:40:01.532 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:01.528+0000 7f00cbfff640 1 --2- 172.21.15.89:0/4138758812 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x565247fcd840 0x565248394d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:01.532 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:01.528+0000 7f00cbfff640 1 -- 172.21.15.89:0/4138758812 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x565248389b00 con 0x565247fa4980 2024-09-21T03:40:01.532 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:01.528+0000 7f00cbfff640 1 --2- 172.21.15.89:0/4138758812 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x565247fa4980 0x56524837ac80 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f00c4009db0 tx=0x7f00c402a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d56fa0ad5180a613 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:01.532 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:01.528+0000 7f00c17fa640 1 -- 172.21.15.89:0/4138758812 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00c403d070 con 0x565247fa4980 2024-09-21T03:40:01.532 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:01.529+0000 7f00c17fa640 1 -- 172.21.15.89:0/4138758812 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f00c4010d30 con 0x565247fa4980 2024-09-21T03:40:01.532 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:01.529+0000 7f00c17fa640 1 -- 172.21.15.89:0/4138758812 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00c4016040 con 0x565247fa4980 2024-09-21T03:40:01.535 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:01.529+0000 7f00d8391740 1 -- 172.21.15.89:0/4138758812 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x565247fa4980 msgr2=0x56524837ac80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:01.535 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:01.529+0000 7f00d8391740 1 --2- 172.21.15.89:0/4138758812 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x565247fa4980 0x56524837ac80 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f00c4009db0 tx=0x7f00c402a430 comp rx=0 tx=0).stop 2024-09-21T03:40:01.535 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:01.529+0000 7f00d8391740 1 -- 172.21.15.89:0/4138758812 shutdown_connections 2024-09-21T03:40:01.535 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:01.529+0000 7f00d8391740 1 --2- 172.21.15.89:0/4138758812 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x565247fcd840 0x565248394d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:01.535 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:01.529+0000 7f00d8391740 1 --2- 172.21.15.89:0/4138758812 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x565247facb60 0x5652483947e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:01.535 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:01.529+0000 7f00d8391740 1 --2- 172.21.15.89:0/4138758812 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x565247fa4980 0x56524837ac80 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:01.535 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:01.529+0000 7f00d8391740 1 -- 172.21.15.89:0/4138758812 >> 172.21.15.89:0/4138758812 conn(0x565247fc1990 msgr2=0x56524837a570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:40:01.535 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:01.530+0000 7f00d8391740 1 -- 172.21.15.89:0/4138758812 shutdown_connections 2024-09-21T03:40:01.535 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:01.530+0000 7f00d8391740 1 -- 172.21.15.89:0/4138758812 wait complete. 2024-09-21T03:40:01.535 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:01.530+0000 7f00d8391740 1 -- start start 2024-09-21T03:40:01.535 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:01.530+0000 7f00d8391740 1 Processor -- start 2024-09-21T03:40:01.536 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:01.530+0000 7f00d8391740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x565247fa4980 0x5652483911b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:01.536 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:01.530+0000 7f00d8391740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x565247facb60 0x5652483a0250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:01.536 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:01.530+0000 7f00d8391740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x565247fcd840 0x5652483a0940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:01.536 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:01.530+0000 7f00d8391740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f00cc1acda0 con 0x565247facb60 2024-09-21T03:40:01.536 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:01.530+0000 7f00d8391740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f00cc1ae210 con 0x565247fcd840 2024-09-21T03:40:01.536 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:01.531+0000 7f00d8391740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f00cc1ad470 con 0x565247fa4980 2024-09-21T03:40:01.536 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:01.531+0000 7f00d8391740 10 client.0 _ll_register_callbacks cb 0x7f00c4009e70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:40:01.536 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:01.531+0000 7f00c3fff640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x565247facb60 0x5652483a0250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:01.536 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:01.531+0000 7f00c3fff640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x565247facb60 0x5652483a0250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:33110/0 (socket says 172.21.15.89:33110) 2024-09-21T03:40:01.536 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:01.531+0000 7f00c3fff640 1 -- 172.21.15.89:0/2876673967 learned_addr learned my addr 172.21.15.89:0/2876673967 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-21T03:40:01.536 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:01.531+0000 7f00cbfff640 1 --2- 172.21.15.89:0/2876673967 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x565247fcd840 0x5652483a0940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:01.536 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:01.531+0000 7f00cbfff640 1 --2- 172.21.15.89:0/2876673967 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x565247fa4980 0x5652483911b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:01.536 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:01.531+0000 7f00c3fff640 1 -- 172.21.15.89:0/2876673967 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x565247fa4980 msgr2=0x5652483911b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:40:01.536 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:01.531+0000 7f00c3fff640 1 --2- 172.21.15.89:0/2876673967 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x565247fa4980 0x5652483911b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:01.537 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:01.531+0000 7f00c3fff640 1 -- 172.21.15.89:0/2876673967 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x565247fcd840 msgr2=0x5652483a0940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:40:01.537 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:01.531+0000 7f00c3fff640 1 --2- 172.21.15.89:0/2876673967 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x565247fcd840 0x5652483a0940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:01.537 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:01.531+0000 7f00c3fff640 1 -- 172.21.15.89:0/2876673967 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56524838cc00 con 0x565247facb60 2024-09-21T03:40:01.537 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:01.531+0000 7f00cbfff640 1 --2- 172.21.15.89:0/2876673967 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x565247fcd840 0x5652483a0940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:40:01.537 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:01.531+0000 7f00c3fff640 1 --2- 172.21.15.89:0/2876673967 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x565247facb60 0x5652483a0250 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x56524839b450 tx=0x7f00b40039c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a638cf111e64459b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:01.537 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:01.531+0000 7f00aaffd640 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2024-09-21T03:40:01.537 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:01.531+0000 7f00aaffd640 1 -- 172.21.15.89:0/2876673967 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00b4005050 con 0x565247facb60 2024-09-21T03:40:01.537 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:01.531+0000 7f00aaffd640 1 -- 172.21.15.89:0/2876673967 <== mon.0 v2:172.21.15.89:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 9121+0+0 (secure 0 0 0) 0x7f00b40051f0 con 0x565247facb60 2024-09-21T03:40:01.537 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:01.531+0000 7f00d8391740 1 -- 172.21.15.89:0/2876673967 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f00c4010d30 con 0x565247facb60 2024-09-21T03:40:01.537 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:01.532+0000 7f00aaffd640 1 -- 172.21.15.89:0/2876673967 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f00b40053f0 con 0x565247facb60 2024-09-21T03:40:01.537 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:01.532+0000 7f00d8391740 10 client.15594 fetch_fsmap learned FSMap version 26 2024-09-21T03:40:01.537 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:01.532+0000 7f00d8391740 1 -- 172.21.15.89:0/2876673967 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x565248405830 con 0x565247facb60 2024-09-21T03:40:01.537 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:01.532+0000 7f00aaffd640 1 -- 172.21.15.89:0/2876673967 <== mon.0 v2:172.21.15.89:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f00c4010d30 con 0x565247facb60 2024-09-21T03:40:01.537 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:01.532+0000 7f00aaffd640 1 -- 172.21.15.89:0/2876673967 <== mon.0 v2:172.21.15.89:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f00b401d650 con 0x565247facb60 2024-09-21T03:40:01.537 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:01.532+0000 7f00d8391740 10 client.15594 fetch_fsmap finished waiting for FSMap version 26 2024-09-21T03:40:01.538 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:01.532+0000 7f00d8391740 10 client.15594 resolve_mds: validated gid 24638 aka daemon mds.b 2024-09-21T03:40:01.538 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:01.532+0000 7f00d8391740 20 client.15594 populate_metadata read hostname 'smithi089' 2024-09-21T03:40:01.538 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:01.532+0000 7f00d8391740 1 --2- 172.21.15.89:0/2876673967 >> [v2:172.21.15.110:6838/132366709,v1:172.21.15.110:6839/132366709] conn(0x565247f61570 0x5652484080e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:01.538 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:01.532+0000 7f00d8391740 4 client.15594 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 134.5, "await_for": 45.0}] 2024-09-21T03:40:01.538 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:01.532+0000 7f00d8391740 1 -- 172.21.15.89:0/2876673967 --> [v2:172.21.15.110:6838/132366709,v1:172.21.15.110:6839/132366709] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 134.5, "await_for": 45.0}) -- 0x5652482e8590 con 0x565247f61570 2024-09-21T03:40:01.538 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:01.532+0000 7f00cbfff640 1 --2- 172.21.15.89:0/2876673967 >> [v2:172.21.15.110:6838/132366709,v1:172.21.15.110:6839/132366709] conn(0x565247f61570 0x5652484080e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:01.538 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:01.533+0000 7f00aaffd640 10 client.15594 ms_handle_connect on v2:172.21.15.110:6838/132366709 2024-09-21T03:40:01.538 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:01.533+0000 7f00cbfff640 1 --2- 172.21.15.89:0/2876673967 >> [v2:172.21.15.110:6838/132366709,v1:172.21.15.110:6839/132366709] conn(0x565247f61570 0x5652484080e0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5c76a3a27008d9ea server_cookie=492c92a7f5efae38 in_seq=0 out_seq=0 2024-09-21T03:40:01.543 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:01.541+0000 7f00aaffd640 1 -- 172.21.15.89:0/2876673967 <== mds.0 v2:172.21.15.110:6838/132366709 1 ==== command_reply(tid 0: 0 ) ==== 8+0+589 (crc 0 0 0) 0x5652482e8590 con 0x565247f61570 2024-09-21T03:40:01.543 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:01.541+0000 7f00aaffd640 10 client.15594 handle_command_reply: tid=0 2024-09-21T03:40:01.543 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:01.541+0000 7f00d8391740 1 client.15594 shutdown 2024-09-21T03:40:01.544 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:01.541+0000 7f00d8391740 1 -- 172.21.15.89:0/2876673967 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x565247facb60 msgr2=0x5652483a0250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:01.544 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:01.541+0000 7f00d8391740 1 --2- 172.21.15.89:0/2876673967 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x565247facb60 0x5652483a0250 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x56524839b450 tx=0x7f00b40039c0 comp rx=0 tx=0).stop 2024-09-21T03:40:01.544 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:01.541+0000 7f00d8391740 1 -- 172.21.15.89:0/2876673967 shutdown_connections 2024-09-21T03:40:01.544 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:01.541+0000 7f00d8391740 1 --2- 172.21.15.89:0/2876673967 >> [v2:172.21.15.110:6838/132366709,v1:172.21.15.110:6839/132366709] conn(0x565247f61570 0x5652484080e0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:01.544 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:01.541+0000 7f00d8391740 1 --2- 172.21.15.89:0/2876673967 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x565247fcd840 0x5652483a0940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:01.545 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:01.541+0000 7f00d8391740 1 --2- 172.21.15.89:0/2876673967 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x565247facb60 0x5652483a0250 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:01.545 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:01.541+0000 7f00d8391740 1 --2- 172.21.15.89:0/2876673967 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x565247fa4980 0x5652483911b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:01.545 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:01.541+0000 7f00d8391740 1 -- 172.21.15.89:0/2876673967 >> 172.21.15.89:0/2876673967 conn(0x565247fc1990 msgr2=0x565248389cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:40:01.545 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:01.542+0000 7f00aaffd640 0 client.15594 ms_handle_reset on v2:172.21.15.110:6838/132366709 2024-09-21T03:40:01.545 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:01.542+0000 7f00d8391740 1 -- 172.21.15.89:0/2876673967 shutdown_connections 2024-09-21T03:40:01.545 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:01.542+0000 7f00d8391740 1 -- 172.21.15.89:0/2876673967 wait complete. 2024-09-21T03:40:01.545 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:01.542+0000 7f00d8391740 20 client.15594 trim_cache size 0 max 16384 2024-09-21T03:40:01.545 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-09-21T03:40:01.545 INFO:teuthology.orchestra.run.smithi089.stdout: "epoch": 26, 2024-09-21T03:40:01.545 INFO:teuthology.orchestra.run.smithi089.stdout: "leader": 24638, 2024-09-21T03:40:01.546 INFO:teuthology.orchestra.run.smithi089.stdout: "set_version": 2, 2024-09-21T03:40:01.546 INFO:teuthology.orchestra.run.smithi089.stdout: "sets": { 2024-09-21T03:40:01.546 INFO:teuthology.orchestra.run.smithi089.stdout: "857cf945": { 2024-09-21T03:40:01.546 INFO:teuthology.orchestra.run.smithi089.stdout: "version": 2, 2024-09-21T03:40:01.546 INFO:teuthology.orchestra.run.smithi089.stdout: "age_ref": 0.0, 2024-09-21T03:40:01.546 INFO:teuthology.orchestra.run.smithi089.stdout: "state": { 2024-09-21T03:40:01.546 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "QUIESCED", 2024-09-21T03:40:01.546 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 0.0 2024-09-21T03:40:01.546 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:40:01.546 INFO:teuthology.orchestra.run.smithi089.stdout: "timeout": 90.0, 2024-09-21T03:40:01.546 INFO:teuthology.orchestra.run.smithi089.stdout: "expiration": 134.5, 2024-09-21T03:40:01.546 INFO:teuthology.orchestra.run.smithi089.stdout: "members": { 2024-09-21T03:40:01.546 INFO:teuthology.orchestra.run.smithi089.stdout: "file:/": { 2024-09-21T03:40:01.546 INFO:teuthology.orchestra.run.smithi089.stdout: "excluded": false, 2024-09-21T03:40:01.547 INFO:teuthology.orchestra.run.smithi089.stdout: "state": { 2024-09-21T03:40:01.547 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "QUIESCED", 2024-09-21T03:40:01.547 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 0.0 2024-09-21T03:40:01.547 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-21T03:40:01.547 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-21T03:40:01.547 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-21T03:40:01.547 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-21T03:40:01.547 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-21T03:40:01.547 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-09-21T03:40:01.548 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:01.542+0000 7f00d1e1d640 1 -- 172.21.15.89:0/2670811091 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f00b80b65d0 msgr2=0x7f00b80b8ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:01.548 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:01.542+0000 7f00d1e1d640 1 --2- 172.21.15.89:0/2670811091 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f00b80b65d0 0x7f00b80b8ae0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x56524837ba90 tx=0x7f00bc0b5040 comp rx=0 tx=0).stop 2024-09-21T03:40:01.548 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:01.542+0000 7f00d1e1d640 1 -- 172.21.15.89:0/2670811091 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f00cc102a60 msgr2=0x7f00cc19a370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:01.548 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:01.542+0000 7f00d1e1d640 1 --2- 172.21.15.89:0/2670811091 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f00cc102a60 0x7f00cc19a370 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f00bc009230 tx=0x7f00bc004270 comp rx=0 tx=0).stop 2024-09-21T03:40:01.548 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:01.542+0000 7f00d1e1d640 1 -- 172.21.15.89:0/2670811091 shutdown_connections 2024-09-21T03:40:01.548 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:01.542+0000 7f00d1e1d640 1 --2- 172.21.15.89:0/2670811091 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f00b80b65d0 0x7f00b80b8ae0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:01.548 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:01.542+0000 7f00d1e1d640 1 --2- 172.21.15.89:0/2670811091 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f00cc1083a0 0x7f00cc19a910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:01.548 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:01.542+0000 7f00d1e1d640 1 --2- 172.21.15.89:0/2670811091 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f00cc108cf0 0x7f00cc19fbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:01.548 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:01.542+0000 7f00d1e1d640 1 --2- 172.21.15.89:0/2670811091 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f00cc102a60 0x7f00cc19a370 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:01.548 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:01.542+0000 7f00d1e1d640 1 -- 172.21.15.89:0/2670811091 >> 172.21.15.89:0/2670811091 conn(0x7f00cc0796d0 msgr2=0x7f00cc1064a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:40:01.548 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:01.543+0000 7f00d1e1d640 1 -- 172.21.15.89:0/2670811091 shutdown_connections 2024-09-21T03:40:01.549 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:01.543+0000 7f00d1e1d640 1 -- 172.21.15.89:0/2670811091 wait complete. 2024-09-21T03:40:03.461 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:40:03 smithi114 ceph-mon[34140]: pgmap v379: 129 pgs: 129 active+clean; 674 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 682 B/s wr, 6 op/s 2024-09-21T03:40:03.500 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:40:03 smithi110 ceph-mon[34000]: pgmap v379: 129 pgs: 129 active+clean; 674 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 682 B/s wr, 6 op/s 2024-09-21T03:40:03.561 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:40:03 smithi089 ceph-mon[30928]: pgmap v379: 129 pgs: 129 active+clean; 674 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 682 B/s wr, 6 op/s 2024-09-21T03:40:05.002 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '857cf945', quiesce took 6.0 seconds. Will release after: 8.5 2024-09-21T03:40:05.311 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:40:05 smithi089 ceph-mon[30928]: pgmap v380: 129 pgs: 129 active+clean; 674 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 682 B/s wr, 7 op/s 2024-09-21T03:40:05.461 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:40:05 smithi114 ceph-mon[34140]: pgmap v380: 129 pgs: 129 active+clean; 674 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 682 B/s wr, 7 op/s 2024-09-21T03:40:05.500 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:40:05 smithi110 ceph-mon[34000]: pgmap v380: 129 pgs: 129 active+clean; 674 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 682 B/s wr, 7 op/s 2024-09-21T03:40:05.980 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:05.978+0000 7fc5881ed640 1 -- 172.21.15.89:0/1223495001 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc580103fc0 msgr2=0x7fc58010a960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:05.980 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:05.978+0000 7fc5881ed640 1 --2- 172.21.15.89:0/1223495001 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc580103fc0 0x7fc58010a960 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fc578009c90 tx=0x7fc57802a860 comp rx=0 tx=0).stop 2024-09-21T03:40:05.980 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:05.978+0000 7fc5881ed640 1 -- 172.21.15.89:0/1223495001 shutdown_connections 2024-09-21T03:40:05.980 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:05.978+0000 7fc5881ed640 1 --2- 172.21.15.89:0/1223495001 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc580103fc0 0x7fc58010a960 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:05.981 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:05.978+0000 7fc5881ed640 1 --2- 172.21.15.89:0/1223495001 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc580103620 0x7fc580103a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:05.981 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:05.978+0000 7fc5881ed640 1 --2- 172.21.15.89:0/1223495001 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc5801025a0 0x7fc580102a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:05.981 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:05.978+0000 7fc5881ed640 1 -- 172.21.15.89:0/1223495001 >> 172.21.15.89:0/1223495001 conn(0x7fc5800fdee0 msgr2=0x7fc580100390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:40:05.981 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:05.978+0000 7fc5881ed640 1 -- 172.21.15.89:0/1223495001 shutdown_connections 2024-09-21T03:40:05.981 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:05.978+0000 7fc5881ed640 1 -- 172.21.15.89:0/1223495001 wait complete. 2024-09-21T03:40:05.983 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:05.978+0000 7fc5881ed640 1 Processor -- start 2024-09-21T03:40:05.983 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:05.979+0000 7fc5881ed640 1 -- start start 2024-09-21T03:40:05.983 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:05.979+0000 7fc5881ed640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc5801025a0 0x7fc580199e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:05.983 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:05.979+0000 7fc5881ed640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc580103620 0x7fc58019a490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:05.983 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:05.979+0000 7fc5881ed640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc580103fc0 0x7fc58019b040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:05.983 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:05.979+0000 7fc5881ed640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fc58010e090 con 0x7fc580103fc0 2024-09-21T03:40:05.983 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:05.979+0000 7fc5881ed640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fc58010df10 con 0x7fc580103620 2024-09-21T03:40:05.983 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:05.979+0000 7fc5881ed640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fc58010e210 con 0x7fc5801025a0 2024-09-21T03:40:05.983 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:05.979+0000 7fc58678a640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc580103fc0 0x7fc58019b040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:05.983 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:05.979+0000 7fc58678a640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc580103fc0 0x7fc58019b040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:33120/0 (socket says 172.21.15.89:33120) 2024-09-21T03:40:05.983 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:05.979+0000 7fc58678a640 1 -- 172.21.15.89:0/215902999 learned_addr learned my addr 172.21.15.89:0/215902999 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-21T03:40:05.983 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:05.979+0000 7fc585788640 1 --2- 172.21.15.89:0/215902999 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc580103620 0x7fc58019a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:05.984 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:05.979+0000 7fc585f89640 1 --2- 172.21.15.89:0/215902999 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc5801025a0 0x7fc580199e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:05.984 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:05.979+0000 7fc58678a640 1 -- 172.21.15.89:0/215902999 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc5801025a0 msgr2=0x7fc580199e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:05.984 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:05.979+0000 7fc58678a640 1 --2- 172.21.15.89:0/215902999 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc5801025a0 0x7fc580199e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:05.984 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:05.979+0000 7fc58678a640 1 -- 172.21.15.89:0/215902999 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc580103620 msgr2=0x7fc58019a490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:05.984 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:05.979+0000 7fc58678a640 1 --2- 172.21.15.89:0/215902999 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc580103620 0x7fc58019a490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:05.984 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:05.979+0000 7fc58678a640 1 -- 172.21.15.89:0/215902999 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc58019b860 con 0x7fc580103fc0 2024-09-21T03:40:05.984 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:05.980+0000 7fc58678a640 1 --2- 172.21.15.89:0/215902999 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc580103fc0 0x7fc58019b040 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fc5780011e0 tx=0x7fc578032cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:05.984 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:05.980+0000 7fc576ffd640 1 -- 172.21.15.89:0/215902999 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc578003dd0 con 0x7fc580103fc0 2024-09-21T03:40:05.984 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:05.980+0000 7fc5881ed640 1 -- 172.21.15.89:0/215902999 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc58019baf0 con 0x7fc580103fc0 2024-09-21T03:40:05.984 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:05.980+0000 7fc576ffd640 1 -- 172.21.15.89:0/215902999 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc578040070 con 0x7fc580103fc0 2024-09-21T03:40:05.984 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:05.980+0000 7fc576ffd640 1 -- 172.21.15.89:0/215902999 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5780307c0 con 0x7fc580103fc0 2024-09-21T03:40:05.984 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:05.980+0000 7fc5881ed640 1 -- 172.21.15.89:0/215902999 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5801a2240 con 0x7fc580103fc0 2024-09-21T03:40:05.986 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:05.984+0000 7fc576ffd640 1 -- 172.21.15.89:0/215902999 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 20) ==== 154563+0+0 (secure 0 0 0) 0x7fc578030960 con 0x7fc580103fc0 2024-09-21T03:40:05.986 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:05.984+0000 7fc58f237740 1 Processor -- start 2024-09-21T03:40:05.986 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:05.984+0000 7fc58f237740 1 -- start start 2024-09-21T03:40:05.987 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:05.984+0000 7fc58f237740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559ea7a901b0 0x559ea7e3c260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:05.987 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:05.984+0000 7fc58f237740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559ea7a672f0 0x559ea7e3c8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:05.988 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:05.984+0000 7fc58f237740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559ea7a6f4d0 0x559ea7e3d420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:05.988 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:05.984+0000 7fc58f237740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x559ea7daa6f0 con 0x559ea7a6f4d0 2024-09-21T03:40:05.988 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:05.984+0000 7fc58f237740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x559ea7d99b20 con 0x559ea7a901b0 2024-09-21T03:40:05.988 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:05.984+0000 7fc58f237740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x559ea7d9d930 con 0x559ea7a672f0 2024-09-21T03:40:05.988 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:05.984+0000 7fc585788640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559ea7a6f4d0 0x559ea7e3d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:05.988 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:05.984+0000 7fc585788640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559ea7a901b0 0x559ea7e3c260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:05.988 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:05.984+0000 7fc585788640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559ea7a901b0 0x559ea7e3c260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.89:57052/0 (socket says 172.21.15.89:57052) 2024-09-21T03:40:05.988 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:05.984+0000 7fc585788640 1 -- 172.21.15.89:0/4033840645 learned_addr learned my addr 172.21.15.89:0/4033840645 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-21T03:40:05.988 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:05.984+0000 7fc585f89640 1 --2- 172.21.15.89:0/4033840645 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559ea7a672f0 0x559ea7e3c8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:05.988 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:05.985+0000 7fc585788640 1 -- 172.21.15.89:0/4033840645 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559ea7a672f0 msgr2=0x559ea7e3c8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:40:05.988 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:05.985+0000 7fc585788640 1 --2- 172.21.15.89:0/4033840645 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559ea7a672f0 0x559ea7e3c8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:05.988 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:05.985+0000 7fc585788640 1 -- 172.21.15.89:0/4033840645 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559ea7a6f4d0 msgr2=0x559ea7e3d420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:40:05.988 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:05.985+0000 7fc576ffd640 1 --2- 172.21.15.89:0/215902999 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7fc56c0b65a0 0x7fc56c0b8ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:05.988 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:05.985+0000 7fc585788640 1 --2- 172.21.15.89:0/4033840645 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559ea7a6f4d0 0x559ea7e3d420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:05.989 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:05.985+0000 7fc585788640 1 -- 172.21.15.89:0/4033840645 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559ea7d5f530 con 0x559ea7a901b0 2024-09-21T03:40:05.989 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:05.985+0000 7fc58678a640 1 --2- 172.21.15.89:0/215902999 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7fc56c0b65a0 0x7fc56c0b8ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:05.989 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:05.985+0000 7fc585788640 1 --2- 172.21.15.89:0/4033840645 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559ea7a901b0 0x559ea7e3c260 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fc5700127d0 tx=0x7fc57000b410 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d91ce9170da594d7 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:05.989 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:05.985+0000 7fc576ffd640 1 -- 172.21.15.89:0/215902999 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 9121+0+0 (secure 0 0 0) 0x7fc578108720 con 0x7fc580103fc0 2024-09-21T03:40:05.989 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:05.985+0000 7fc56bfff640 1 -- 172.21.15.89:0/4033840645 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc570004020 con 0x559ea7a901b0 2024-09-21T03:40:05.989 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:05.985+0000 7fc56bfff640 1 -- 172.21.15.89:0/4033840645 <== mon.2 v2:172.21.15.110:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc57000fbf0 con 0x559ea7a901b0 2024-09-21T03:40:05.989 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:05.985+0000 7fc56bfff640 1 -- 172.21.15.89:0/4033840645 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc570003850 con 0x559ea7a901b0 2024-09-21T03:40:05.992 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:05.986+0000 7fc58f237740 1 -- 172.21.15.89:0/4033840645 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559ea7a901b0 msgr2=0x559ea7e3c260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:05.992 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:05.986+0000 7fc58f237740 1 --2- 172.21.15.89:0/4033840645 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559ea7a901b0 0x559ea7e3c260 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fc5700127d0 tx=0x7fc57000b410 comp rx=0 tx=0).stop 2024-09-21T03:40:05.992 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:05.986+0000 7fc58678a640 1 --2- 172.21.15.89:0/215902999 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7fc56c0b65a0 0x7fc56c0b8ab0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fc5780ca9a0 tx=0x7fc5780095a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:05.992 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:05.986+0000 7fc58f237740 1 -- 172.21.15.89:0/4033840645 shutdown_connections 2024-09-21T03:40:05.992 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:05.986+0000 7fc58f237740 1 --2- 172.21.15.89:0/4033840645 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559ea7a6f4d0 0x559ea7e3d420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:05.992 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:05.986+0000 7fc58f237740 1 --2- 172.21.15.89:0/4033840645 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559ea7a672f0 0x559ea7e3c8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:05.992 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:05.986+0000 7fc58f237740 1 --2- 172.21.15.89:0/4033840645 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559ea7a901b0 0x559ea7e3c260 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:05.992 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:05.986+0000 7fc58f237740 1 -- 172.21.15.89:0/4033840645 >> 172.21.15.89:0/4033840645 conn(0x559ea7a84300 msgr2=0x559ea7e338e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:40:05.992 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:05.986+0000 7fc58f237740 1 -- 172.21.15.89:0/4033840645 shutdown_connections 2024-09-21T03:40:05.992 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:05.986+0000 7fc58f237740 1 -- 172.21.15.89:0/4033840645 wait complete. 2024-09-21T03:40:05.992 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:05.987+0000 7fc58f237740 1 -- start start 2024-09-21T03:40:05.993 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:05.987+0000 7fc58f237740 1 Processor -- start 2024-09-21T03:40:05.993 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:05.987+0000 7fc58f237740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559ea7a672f0 0x559ea7e45a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:05.993 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:05.988+0000 7fc58f237740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559ea7a6f4d0 0x559ea7e460e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:05.993 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:05.988+0000 7fc58f237740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559ea7a901b0 0x559ea7e46c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:05.993 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:05.988+0000 7fc585f89640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559ea7a6f4d0 0x559ea7e460e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:05.993 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:05.988+0000 7fc585f89640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559ea7a6f4d0 0x559ea7e460e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:33132/0 (socket says 172.21.15.89:33132) 2024-09-21T03:40:05.993 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:05.988+0000 7fc585f89640 1 -- 172.21.15.89:0/4255946846 learned_addr learned my addr 172.21.15.89:0/4255946846 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-21T03:40:05.993 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:05.988+0000 7fc58f237740 1 -- 172.21.15.89:0/4255946846 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x559ea7cd1ab0 con 0x559ea7a6f4d0 2024-09-21T03:40:05.993 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:05.988+0000 7fc58f237740 1 -- 172.21.15.89:0/4255946846 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x559ea7d3f730 con 0x559ea7a672f0 2024-09-21T03:40:05.993 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:05.988+0000 7fc58f237740 1 -- 172.21.15.89:0/4255946846 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x559ea7e3df40 con 0x559ea7a901b0 2024-09-21T03:40:05.993 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:05.988+0000 7fc585788640 1 --2- 172.21.15.89:0/4255946846 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559ea7a672f0 0x559ea7e45a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:05.993 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:05.988+0000 7fc58f237740 10 client.0 _ll_register_callbacks cb 0x559ea7d936b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:40:05.993 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:05.988+0000 7fc585f89640 1 -- 172.21.15.89:0/4255946846 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559ea7a901b0 msgr2=0x559ea7e46c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:40:05.993 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:05.988+0000 7fc585f89640 1 --2- 172.21.15.89:0/4255946846 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559ea7a901b0 0x559ea7e46c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:05.994 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:05.988+0000 7fc585f89640 1 -- 172.21.15.89:0/4255946846 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559ea7a672f0 msgr2=0x559ea7e45a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:40:05.994 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:05.988+0000 7fc585f89640 1 --2- 172.21.15.89:0/4255946846 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559ea7a672f0 0x559ea7e45a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:05.994 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:05.988+0000 7fc585f89640 1 -- 172.21.15.89:0/4255946846 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559ea7afb790 con 0x559ea7a6f4d0 2024-09-21T03:40:05.994 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:05.988+0000 7fc585f89640 1 --2- 172.21.15.89:0/4255946846 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559ea7a6f4d0 0x559ea7e460e0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fc57c012840 tx=0x7fc57c012d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae59258403fa242f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:05.994 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:05.988+0000 7fc5697fa640 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2024-09-21T03:40:05.994 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:05.989+0000 7fc5697fa640 1 -- 172.21.15.89:0/4255946846 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc57c00f8a0 con 0x559ea7a6f4d0 2024-09-21T03:40:05.994 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:05.989+0000 7fc5697fa640 1 -- 172.21.15.89:0/4255946846 <== mon.0 v2:172.21.15.89:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 9121+0+0 (secure 0 0 0) 0x7fc57c00fa40 con 0x559ea7a6f4d0 2024-09-21T03:40:05.994 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:05.989+0000 7fc58f237740 1 -- 172.21.15.89:0/4255946846 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc57000fbf0 con 0x559ea7a6f4d0 2024-09-21T03:40:05.994 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:05.989+0000 7fc5697fa640 1 -- 172.21.15.89:0/4255946846 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fc57c00fc40 con 0x559ea7a6f4d0 2024-09-21T03:40:05.994 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:05.989+0000 7fc58f237740 10 client.15606 fetch_fsmap learned FSMap version 26 2024-09-21T03:40:05.994 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:05.989+0000 7fc58f237740 1 -- 172.21.15.89:0/4255946846 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x559ea7b74620 con 0x559ea7a6f4d0 2024-09-21T03:40:05.994 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:05.989+0000 7fc5697fa640 1 -- 172.21.15.89:0/4255946846 <== mon.0 v2:172.21.15.89:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc57c00fde0 con 0x559ea7a6f4d0 2024-09-21T03:40:05.994 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:05.989+0000 7fc5697fa640 1 -- 172.21.15.89:0/4255946846 <== mon.0 v2:172.21.15.89:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fc57c01b3c0 con 0x559ea7a6f4d0 2024-09-21T03:40:05.994 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:05.989+0000 7fc58f237740 10 client.15606 fetch_fsmap finished waiting for FSMap version 26 2024-09-21T03:40:05.994 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:05.989+0000 7fc58f237740 10 client.15606 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T03:40:05.995 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:05.989+0000 7fc58f237740 20 client.15606 populate_metadata read hostname 'smithi089' 2024-09-21T03:40:05.995 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:05.989+0000 7fc58f237740 1 --2- 172.21.15.89:0/4255946846 >> [v2:172.21.15.110:6838/132366709,v1:172.21.15.110:6839/132366709] conn(0x559ea7a4ed80 0x559ea7e53290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:05.995 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:05.989+0000 7fc58f237740 4 client.15606 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:40:05.995 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:05.989+0000 7fc58f237740 1 -- 172.21.15.89:0/4255946846 --> [v2:172.21.15.110:6838/132366709,v1:172.21.15.110:6839/132366709] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fc57000fbf0 con 0x559ea7a4ed80 2024-09-21T03:40:05.995 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:05.990+0000 7fc585788640 1 --2- 172.21.15.89:0/4255946846 >> [v2:172.21.15.110:6838/132366709,v1:172.21.15.110:6839/132366709] conn(0x559ea7a4ed80 0x559ea7e53290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:05.996 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:05.990+0000 7fc5697fa640 10 client.15606 ms_handle_connect on v2:172.21.15.110:6838/132366709 2024-09-21T03:40:05.996 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:05.990+0000 7fc585788640 1 --2- 172.21.15.89:0/4255946846 >> [v2:172.21.15.110:6838/132366709,v1:172.21.15.110:6839/132366709] conn(0x559ea7a4ed80 0x559ea7e53290 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f349e94aa5c25ab9 server_cookie=e7001a2fd3719f50 in_seq=0 out_seq=0 2024-09-21T03:40:05.996 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:05.991+0000 7fc5697fa640 1 -- 172.21.15.89:0/4255946846 <== mds.0 v2:172.21.15.110:6838/132366709 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x7fc57000fbf0 con 0x559ea7a4ed80 2024-09-21T03:40:05.996 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:05.991+0000 7fc5697fa640 10 client.15606 handle_command_reply: tid=0 2024-09-21T03:40:05.996 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:05.991+0000 7fc58f237740 1 client.15606 shutdown 2024-09-21T03:40:05.996 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:05.992+0000 7fc58f237740 1 -- 172.21.15.89:0/4255946846 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559ea7a6f4d0 msgr2=0x559ea7e460e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:05.996 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:05.992+0000 7fc58f237740 1 --2- 172.21.15.89:0/4255946846 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559ea7a6f4d0 0x559ea7e460e0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fc57c012840 tx=0x7fc57c012d00 comp rx=0 tx=0).stop 2024-09-21T03:40:05.996 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:05.992+0000 7fc58f237740 1 -- 172.21.15.89:0/4255946846 shutdown_connections 2024-09-21T03:40:05.997 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:05.992+0000 7fc58f237740 1 --2- 172.21.15.89:0/4255946846 >> [v2:172.21.15.110:6838/132366709,v1:172.21.15.110:6839/132366709] conn(0x559ea7a4ed80 0x559ea7e53290 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:05.997 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:05.992+0000 7fc58f237740 1 --2- 172.21.15.89:0/4255946846 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559ea7a901b0 0x559ea7e46c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:05.997 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:05.992+0000 7fc58f237740 1 --2- 172.21.15.89:0/4255946846 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559ea7a6f4d0 0x559ea7e460e0 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:05.997 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:05.992+0000 7fc58f237740 1 --2- 172.21.15.89:0/4255946846 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559ea7a672f0 0x559ea7e45a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:05.997 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:05.992+0000 7fc5697fa640 0 client.15606 ms_handle_reset on v2:172.21.15.110:6838/132366709 2024-09-21T03:40:05.997 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:05.992+0000 7fc58f237740 1 -- 172.21.15.89:0/4255946846 >> 172.21.15.89:0/4255946846 conn(0x559ea7a84300 msgr2=0x559ea7e31ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:40:05.997 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:05.992+0000 7fc58f237740 1 -- 172.21.15.89:0/4255946846 shutdown_connections 2024-09-21T03:40:05.997 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:05.992+0000 7fc58f237740 1 -- 172.21.15.89:0/4255946846 wait complete. 2024-09-21T03:40:05.997 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:05.992+0000 7fc58f237740 20 client.15606 trim_cache size 0 max 16384 2024-09-21T03:40:06.010 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:06.006+0000 7fc58f237740 1 Processor -- start 2024-09-21T03:40:06.010 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:06.006+0000 7fc58f237740 1 -- start start 2024-09-21T03:40:06.010 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:06.006+0000 7fc58f237740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559ea7a84300 0x559ea7e45a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:06.010 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:06.006+0000 7fc58f237740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559ea7a672f0 0x559ea7e45f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:06.010 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:06.006+0000 7fc58f237740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559ea7a6f4d0 0x559ea7e43f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:06.010 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:06.006+0000 7fc58f237740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fc5801acb70 con 0x559ea7a672f0 2024-09-21T03:40:06.010 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:06.006+0000 7fc58f237740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fc5801aa440 con 0x559ea7a6f4d0 2024-09-21T03:40:06.010 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:06.006+0000 7fc58f237740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fc5801a8290 con 0x559ea7a84300 2024-09-21T03:40:06.010 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:06.007+0000 7fc585f89640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559ea7a672f0 0x559ea7e45f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:06.010 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:06.007+0000 7fc585f89640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559ea7a672f0 0x559ea7e45f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:33138/0 (socket says 172.21.15.89:33138) 2024-09-21T03:40:06.010 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:06.007+0000 7fc585f89640 1 -- 172.21.15.89:0/1609542427 learned_addr learned my addr 172.21.15.89:0/1609542427 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-21T03:40:06.010 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:06.007+0000 7fc585788640 1 --2- 172.21.15.89:0/1609542427 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559ea7a84300 0x559ea7e45a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:06.011 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:06.007+0000 7fc585788640 1 --2- 172.21.15.89:0/1609542427 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559ea7a6f4d0 0x559ea7e43f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:06.011 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:06.007+0000 7fc585f89640 1 -- 172.21.15.89:0/1609542427 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559ea7a84300 msgr2=0x559ea7e45a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:40:06.011 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:06.007+0000 7fc585f89640 1 --2- 172.21.15.89:0/1609542427 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559ea7a84300 0x559ea7e45a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:06.011 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:06.007+0000 7fc585f89640 1 -- 172.21.15.89:0/1609542427 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559ea7a6f4d0 msgr2=0x559ea7e43f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:40:06.011 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:06.007+0000 7fc585f89640 1 --2- 172.21.15.89:0/1609542427 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559ea7a6f4d0 0x559ea7e43f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:06.011 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:06.007+0000 7fc585f89640 1 -- 172.21.15.89:0/1609542427 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559ea7e32fd0 con 0x559ea7a672f0 2024-09-21T03:40:06.011 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:06.007+0000 7fc585788640 1 --2- 172.21.15.89:0/1609542427 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559ea7a6f4d0 0x559ea7e43f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:40:06.011 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:06.007+0000 7fc585f89640 1 --2- 172.21.15.89:0/1609542427 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559ea7a672f0 0x559ea7e45f60 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7fc57c0059f0 tx=0x7fc57c00ff90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6f7f89d67e9d29c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:06.011 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:06.007+0000 7fc56b7fe640 1 -- 172.21.15.89:0/1609542427 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc57c01ba70 con 0x559ea7a672f0 2024-09-21T03:40:06.011 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:06.008+0000 7fc56b7fe640 1 -- 172.21.15.89:0/1609542427 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc57c01bc10 con 0x559ea7a672f0 2024-09-21T03:40:06.011 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:06.008+0000 7fc56b7fe640 1 -- 172.21.15.89:0/1609542427 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc57c005580 con 0x559ea7a672f0 2024-09-21T03:40:06.015 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:06.008+0000 7fc58f237740 1 -- 172.21.15.89:0/1609542427 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559ea7a672f0 msgr2=0x559ea7e45f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:06.015 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:06.008+0000 7fc58f237740 1 --2- 172.21.15.89:0/1609542427 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559ea7a672f0 0x559ea7e45f60 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7fc57c0059f0 tx=0x7fc57c00ff90 comp rx=0 tx=0).stop 2024-09-21T03:40:06.015 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:06.008+0000 7fc58f237740 1 -- 172.21.15.89:0/1609542427 shutdown_connections 2024-09-21T03:40:06.015 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:06.008+0000 7fc58f237740 1 --2- 172.21.15.89:0/1609542427 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559ea7a6f4d0 0x559ea7e43f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:06.015 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:06.008+0000 7fc58f237740 1 --2- 172.21.15.89:0/1609542427 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559ea7a672f0 0x559ea7e45f60 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:06.015 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:06.008+0000 7fc58f237740 1 --2- 172.21.15.89:0/1609542427 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559ea7a84300 0x559ea7e45a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:06.015 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:06.009+0000 7fc58f237740 1 -- 172.21.15.89:0/1609542427 >> 172.21.15.89:0/1609542427 conn(0x559ea7e475f0 msgr2=0x559ea7e3d0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:40:06.015 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:06.009+0000 7fc58f237740 1 -- 172.21.15.89:0/1609542427 shutdown_connections 2024-09-21T03:40:06.015 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:06.009+0000 7fc58f237740 1 -- 172.21.15.89:0/1609542427 wait complete. 2024-09-21T03:40:06.015 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:06.009+0000 7fc58f237740 1 -- start start 2024-09-21T03:40:06.015 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:06.009+0000 7fc58f237740 1 Processor -- start 2024-09-21T03:40:06.015 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:06.009+0000 7fc58f237740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559ea7a672f0 0x559ea7e4c950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:06.015 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:06.009+0000 7fc58f237740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559ea7a6f4d0 0x559ea7e4ce90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:06.016 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:06.009+0000 7fc58f237740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559ea7a84300 0x559ea7e4b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:06.016 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:06.009+0000 7fc58f237740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fc5801a8900 con 0x559ea7a6f4d0 2024-09-21T03:40:06.016 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:06.009+0000 7fc58f237740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fc5801a9d70 con 0x559ea7a672f0 2024-09-21T03:40:06.016 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:06.009+0000 7fc58f237740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fc5801a8fd0 con 0x559ea7a84300 2024-09-21T03:40:06.016 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:06.009+0000 7fc58f237740 10 client.0 _ll_register_callbacks cb 0x7fc57c009930 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:40:06.016 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:06.010+0000 7fc585f89640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559ea7a6f4d0 0x559ea7e4ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:06.016 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:06.010+0000 7fc585f89640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559ea7a6f4d0 0x559ea7e4ce90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:33150/0 (socket says 172.21.15.89:33150) 2024-09-21T03:40:06.016 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:06.010+0000 7fc585f89640 1 -- 172.21.15.89:0/3926141713 learned_addr learned my addr 172.21.15.89:0/3926141713 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-21T03:40:06.016 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:06.010+0000 7fc585788640 1 --2- 172.21.15.89:0/3926141713 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559ea7a84300 0x559ea7e4b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:06.016 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:06.010+0000 7fc585788640 1 --2- 172.21.15.89:0/3926141713 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559ea7a672f0 0x559ea7e4c950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:06.016 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:06.010+0000 7fc585f89640 1 -- 172.21.15.89:0/3926141713 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559ea7a84300 msgr2=0x559ea7e4b5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:40:06.016 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:06.010+0000 7fc585f89640 1 --2- 172.21.15.89:0/3926141713 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559ea7a84300 0x559ea7e4b5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:06.016 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:06.010+0000 7fc585f89640 1 -- 172.21.15.89:0/3926141713 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559ea7a672f0 msgr2=0x559ea7e4c950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:40:06.016 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:06.010+0000 7fc585f89640 1 --2- 172.21.15.89:0/3926141713 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559ea7a672f0 0x559ea7e4c950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:06.016 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:06.010+0000 7fc585f89640 1 -- 172.21.15.89:0/3926141713 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559ea7e4d710 con 0x559ea7a6f4d0 2024-09-21T03:40:06.017 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:06.010+0000 7fc585788640 1 --2- 172.21.15.89:0/3926141713 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559ea7a672f0 0x559ea7e4c950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:40:06.017 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:06.010+0000 7fc585f89640 1 --2- 172.21.15.89:0/3926141713 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559ea7a6f4d0 0x559ea7e4ce90 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7fc57c01b9f0 tx=0x7fc57c0054f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a93589ef4f11e6d7 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:06.017 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:06.010+0000 7fc5697fa640 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2024-09-21T03:40:06.017 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:06.010+0000 7fc5697fa640 1 -- 172.21.15.89:0/3926141713 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc57c00fde0 con 0x559ea7a6f4d0 2024-09-21T03:40:06.017 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:06.010+0000 7fc5697fa640 1 -- 172.21.15.89:0/3926141713 <== mon.0 v2:172.21.15.89:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 9121+0+0 (secure 0 0 0) 0x7fc57c01b020 con 0x559ea7a6f4d0 2024-09-21T03:40:06.017 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:06.010+0000 7fc58f237740 1 -- 172.21.15.89:0/3926141713 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x559ea7e4def0 con 0x559ea7a6f4d0 2024-09-21T03:40:06.017 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:06.011+0000 7fc5697fa640 1 -- 172.21.15.89:0/3926141713 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fc57c01b240 con 0x559ea7a6f4d0 2024-09-21T03:40:06.017 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:06.011+0000 7fc58f237740 10 client.15612 fetch_fsmap learned FSMap version 26 2024-09-21T03:40:06.017 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:06.011+0000 7fc58f237740 1 -- 172.21.15.89:0/3926141713 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x559ea7ebe140 con 0x559ea7a6f4d0 2024-09-21T03:40:06.017 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:06.011+0000 7fc5697fa640 1 -- 172.21.15.89:0/3926141713 <== mon.0 v2:172.21.15.89:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x559ea7e4def0 con 0x559ea7a6f4d0 2024-09-21T03:40:06.017 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:06.011+0000 7fc5697fa640 1 -- 172.21.15.89:0/3926141713 <== mon.0 v2:172.21.15.89:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fc57c003b10 con 0x559ea7a6f4d0 2024-09-21T03:40:06.017 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:06.012+0000 7fc58f237740 10 client.15612 fetch_fsmap finished waiting for FSMap version 26 2024-09-21T03:40:06.017 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:06.012+0000 7fc58f237740 10 client.15612 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T03:40:06.017 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:06.012+0000 7fc58f237740 20 client.15612 populate_metadata read hostname 'smithi089' 2024-09-21T03:40:06.018 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:06.012+0000 7fc58f237740 1 --2- 172.21.15.89:0/3926141713 >> [v2:172.21.15.110:6838/132366709,v1:172.21.15.110:6839/132366709] conn(0x559ea7a901b0 0x559ea7ec0b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:06.018 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:06.012+0000 7fc58f237740 4 client.15612 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T03:40:06.018 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:06.012+0000 7fc58f237740 1 -- 172.21.15.89:0/3926141713 --> [v2:172.21.15.110:6838/132366709,v1:172.21.15.110:6839/132366709] -- command(tid 0: {"prefix": "flush journal"}) -- 0x559ea7dac640 con 0x559ea7a901b0 2024-09-21T03:40:06.018 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:06.012+0000 7fc585788640 1 --2- 172.21.15.89:0/3926141713 >> [v2:172.21.15.110:6838/132366709,v1:172.21.15.110:6839/132366709] conn(0x559ea7a901b0 0x559ea7ec0b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:06.018 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:06.012+0000 7fc5697fa640 10 client.15612 ms_handle_connect on v2:172.21.15.110:6838/132366709 2024-09-21T03:40:06.018 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:06.012+0000 7fc585788640 1 --2- 172.21.15.89:0/3926141713 >> [v2:172.21.15.110:6838/132366709,v1:172.21.15.110:6839/132366709] conn(0x559ea7a901b0 0x559ea7ec0b20 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2f9198d5c14c224e server_cookie=6b2060a808d39eb1 in_seq=0 out_seq=0 2024-09-21T03:40:06.020 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:06.018+0000 7fc5697fa640 1 -- 172.21.15.89:0/3926141713 <== mds.0 v2:172.21.15.110:6838/132366709 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x559ea7dac640 con 0x559ea7a901b0 2024-09-21T03:40:06.021 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:06.018+0000 7fc5697fa640 10 client.15612 handle_command_reply: tid=0 2024-09-21T03:40:06.021 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:06.018+0000 7fc58f237740 1 client.15612 shutdown 2024-09-21T03:40:06.021 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:06.018+0000 7fc58f237740 1 -- 172.21.15.89:0/3926141713 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559ea7a6f4d0 msgr2=0x559ea7e4ce90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:06.021 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:06.018+0000 7fc58f237740 1 --2- 172.21.15.89:0/3926141713 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559ea7a6f4d0 0x559ea7e4ce90 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7fc57c01b9f0 tx=0x7fc57c0054f0 comp rx=0 tx=0).stop 2024-09-21T03:40:06.021 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:06.018+0000 7fc58f237740 1 -- 172.21.15.89:0/3926141713 shutdown_connections 2024-09-21T03:40:06.021 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:06.018+0000 7fc58f237740 1 --2- 172.21.15.89:0/3926141713 >> [v2:172.21.15.110:6838/132366709,v1:172.21.15.110:6839/132366709] conn(0x559ea7a901b0 0x559ea7ec0b20 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:06.021 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:06.018+0000 7fc58f237740 1 --2- 172.21.15.89:0/3926141713 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559ea7a84300 0x559ea7e4b5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:06.021 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:06.018+0000 7fc58f237740 1 --2- 172.21.15.89:0/3926141713 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559ea7a6f4d0 0x559ea7e4ce90 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:06.021 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:06.018+0000 7fc58f237740 1 --2- 172.21.15.89:0/3926141713 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559ea7a672f0 0x559ea7e4c950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:06.021 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:06.018+0000 7fc5697fa640 0 client.15612 ms_handle_reset on v2:172.21.15.110:6838/132366709 2024-09-21T03:40:06.021 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:06.018+0000 7fc58f237740 1 -- 172.21.15.89:0/3926141713 >> 172.21.15.89:0/3926141713 conn(0x559ea7e475f0 msgr2=0x559ea7e459c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:40:06.021 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:06.018+0000 7fc58f237740 1 -- 172.21.15.89:0/3926141713 shutdown_connections 2024-09-21T03:40:06.022 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:06.018+0000 7fc58f237740 1 -- 172.21.15.89:0/3926141713 wait complete. 2024-09-21T03:40:06.022 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:06.018+0000 7fc58f237740 20 client.15612 trim_cache size 0 max 16384 2024-09-21T03:40:06.022 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stdout:{ 2024-09-21T03:40:06.022 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stdout: "message": "", 2024-09-21T03:40:06.022 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stdout: "return_code": 0 2024-09-21T03:40:06.022 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stdout:} 2024-09-21T03:40:06.023 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:06.019+0000 7fc5881ed640 1 -- 172.21.15.89:0/215902999 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7fc56c0b65a0 msgr2=0x7fc56c0b8ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:06.023 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:06.019+0000 7fc5881ed640 1 --2- 172.21.15.89:0/215902999 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7fc56c0b65a0 0x7fc56c0b8ab0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fc5780ca9a0 tx=0x7fc5780095a0 comp rx=0 tx=0).stop 2024-09-21T03:40:06.023 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:06.019+0000 7fc5881ed640 1 -- 172.21.15.89:0/215902999 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc580103fc0 msgr2=0x7fc58019b040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:06.023 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:06.019+0000 7fc5881ed640 1 --2- 172.21.15.89:0/215902999 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc580103fc0 0x7fc58019b040 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fc5780011e0 tx=0x7fc578032cc0 comp rx=0 tx=0).stop 2024-09-21T03:40:06.023 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:06.019+0000 7fc5881ed640 1 -- 172.21.15.89:0/215902999 shutdown_connections 2024-09-21T03:40:06.023 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:06.019+0000 7fc5881ed640 1 --2- 172.21.15.89:0/215902999 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7fc56c0b65a0 0x7fc56c0b8ab0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:06.023 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:06.019+0000 7fc5881ed640 1 --2- 172.21.15.89:0/215902999 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc580103620 0x7fc58019a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:06.023 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:06.019+0000 7fc5881ed640 1 --2- 172.21.15.89:0/215902999 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc580103fc0 0x7fc58019b040 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:06.023 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:06.019+0000 7fc5881ed640 1 --2- 172.21.15.89:0/215902999 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc5801025a0 0x7fc580199e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:06.023 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:06.019+0000 7fc5881ed640 1 -- 172.21.15.89:0/215902999 >> 172.21.15.89:0/215902999 conn(0x7fc5800fdee0 msgr2=0x7fc5800ff800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:40:06.024 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:06.019+0000 7fc5881ed640 1 -- 172.21.15.89:0/215902999 shutdown_connections 2024-09-21T03:40:06.024 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:06.019+0000 7fc5881ed640 1 -- 172.21.15.89:0/215902999 wait complete. 2024-09-21T03:40:07.460 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:40:07 smithi114 ceph-mon[34140]: pgmap v381: 129 pgs: 129 active+clean; 674 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 511 B/s wr, 7 op/s 2024-09-21T03:40:07.500 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:40:07 smithi110 ceph-mon[34000]: pgmap v381: 129 pgs: 129 active+clean; 674 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 511 B/s wr, 7 op/s 2024-09-21T03:40:07.561 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:40:07 smithi089 ceph-mon[30928]: pgmap v381: 129 pgs: 129 active+clean; 674 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 511 B/s wr, 7 op/s 2024-09-21T03:40:09.461 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:40:09 smithi114 ceph-mon[34140]: pgmap v382: 129 pgs: 129 active+clean; 674 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 170 B/s wr, 6 op/s 2024-09-21T03:40:09.500 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:40:09 smithi110 ceph-mon[34000]: pgmap v382: 129 pgs: 129 active+clean; 674 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 170 B/s wr, 6 op/s 2024-09-21T03:40:09.561 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:40:09 smithi089 ceph-mon[30928]: pgmap v382: 129 pgs: 129 active+clean; 674 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 170 B/s wr, 6 op/s 2024-09-21T03:40:11.461 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:40:11 smithi114 ceph-mon[34140]: pgmap v383: 129 pgs: 129 active+clean; 675 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 170 B/s wr, 7 op/s 2024-09-21T03:40:11.500 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:40:11 smithi110 ceph-mon[34000]: pgmap v383: 129 pgs: 129 active+clean; 675 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 170 B/s wr, 7 op/s 2024-09-21T03:40:11.561 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:40:11 smithi089 ceph-mon[30928]: pgmap v383: 129 pgs: 129 active+clean; 675 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 170 B/s wr, 7 op/s 2024-09-21T03:40:13.461 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:40:13 smithi114 ceph-mon[34140]: pgmap v384: 129 pgs: 129 active+clean; 675 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 170 B/s wr, 5 op/s 2024-09-21T03:40:13.500 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:40:13 smithi110 ceph-mon[34000]: pgmap v384: 129 pgs: 129 active+clean; 675 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 170 B/s wr, 5 op/s 2024-09-21T03:40:13.503 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 7c374098-77c9-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T03:40:13.527 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:40:13 smithi089 ceph-mon[30928]: pgmap v384: 129 pgs: 129 active+clean; 675 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 170 B/s wr, 5 op/s 2024-09-21T03:40:13.777 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/7c374098-77c9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:40:14.357 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:14.353+0000 7f3840739640 1 -- 172.21.15.89:0/2574052653 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3838102030 msgr2=0x7f3838102440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:14.357 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:14.353+0000 7f3840739640 1 --2- 172.21.15.89:0/2574052653 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3838102030 0x7f3838102440 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f3834009930 tx=0x7f383402a420 comp rx=0 tx=0).stop 2024-09-21T03:40:14.357 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:14.353+0000 7f3840739640 1 -- 172.21.15.89:0/2574052653 shutdown_connections 2024-09-21T03:40:14.357 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:14.353+0000 7f3840739640 1 --2- 172.21.15.89:0/2574052653 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3838106280 0x7f3838106690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:14.357 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:14.353+0000 7f3840739640 1 --2- 172.21.15.89:0/2574052653 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3838106bd0 0x7f3838110660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:14.357 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:14.353+0000 7f3840739640 1 --2- 172.21.15.89:0/2574052653 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3838102030 0x7f3838102440 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:14.357 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:14.353+0000 7f3840739640 1 -- 172.21.15.89:0/2574052653 >> 172.21.15.89:0/2574052653 conn(0x7f3838077c60 msgr2=0x7f38380780a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:40:14.357 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:14.353+0000 7f3840739640 1 -- 172.21.15.89:0/2574052653 shutdown_connections 2024-09-21T03:40:14.357 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:14.353+0000 7f3840739640 1 -- 172.21.15.89:0/2574052653 wait complete. 2024-09-21T03:40:14.357 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:14.354+0000 7f3840739640 1 Processor -- start 2024-09-21T03:40:14.357 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:14.354+0000 7f3840739640 1 -- start start 2024-09-21T03:40:14.358 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:14.355+0000 7f3840739640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3838102030 0x7f383819a500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:14.358 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:14.355+0000 7f3840739640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3838106280 0x7f383819aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:14.358 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:14.355+0000 7f3840739640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3838106bd0 0x7f383819fd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:14.358 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:14.355+0000 7f383e4d5640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3838102030 0x7f383819a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:14.358 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:14.355+0000 7f383e4d5640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3838102030 0x7f383819a500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.89:60582/0 (socket says 172.21.15.89:60582) 2024-09-21T03:40:14.359 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:14.355+0000 7f383dcd4640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3838106280 0x7f383819aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:14.359 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:14.355+0000 7f383dcd4640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3838106280 0x7f383819aaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.89:43996/0 (socket says 172.21.15.89:43996) 2024-09-21T03:40:14.359 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:14.356+0000 7f383ecd6640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3838106bd0 0x7f383819fd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:14.359 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:14.356+0000 7f383ecd6640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3838106bd0 0x7f383819fd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:60784/0 (socket says 172.21.15.89:60784) 2024-09-21T03:40:14.359 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:14.356+0000 7f3840739640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f3838113970 con 0x7f3838106bd0 2024-09-21T03:40:14.359 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:14.356+0000 7f3840739640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f38381137f0 con 0x7f3838106280 2024-09-21T03:40:14.359 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:14.356+0000 7f3840739640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f3838113af0 con 0x7f3838102030 2024-09-21T03:40:14.359 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:14.356+0000 7f383e4d5640 1 -- 172.21.15.89:0/4139935755 learned_addr learned my addr 172.21.15.89:0/4139935755 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-21T03:40:14.361 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:14.356+0000 7f383e4d5640 1 -- 172.21.15.89:0/4139935755 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3838106280 msgr2=0x7f383819aaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:14.361 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:14.356+0000 7f383e4d5640 1 --2- 172.21.15.89:0/4139935755 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3838106280 0x7f383819aaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:14.361 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:14.356+0000 7f383e4d5640 1 -- 172.21.15.89:0/4139935755 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3838106bd0 msgr2=0x7f383819fd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:14.361 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:14.357+0000 7f383e4d5640 1 --2- 172.21.15.89:0/4139935755 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3838106bd0 0x7f383819fd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:14.361 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:14.357+0000 7f383e4d5640 1 -- 172.21.15.89:0/4139935755 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38381a0470 con 0x7f3838102030 2024-09-21T03:40:14.361 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:14.357+0000 7f383ecd6640 1 --2- 172.21.15.89:0/4139935755 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3838106bd0 0x7f383819fd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:40:14.362 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:14.357+0000 7f383e4d5640 1 --2- 172.21.15.89:0/4139935755 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3838102030 0x7f383819a500 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f3834005940 tx=0x7f3834005e10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:14.362 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:14.357+0000 7f382f7fe640 1 -- 172.21.15.89:0/4139935755 <== mon.1 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f383403f030 con 0x7f3838102030 2024-09-21T03:40:14.362 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:14.357+0000 7f382f7fe640 1 -- 172.21.15.89:0/4139935755 <== mon.1 v2:172.21.15.114:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f38340043d0 con 0x7f3838102030 2024-09-21T03:40:14.362 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:14.358+0000 7f382f7fe640 1 -- 172.21.15.89:0/4139935755 <== mon.1 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3834035070 con 0x7f3838102030 2024-09-21T03:40:14.362 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:14.358+0000 7f3840739640 1 -- 172.21.15.89:0/4139935755 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f38381a0700 con 0x7f3838102030 2024-09-21T03:40:14.362 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:14.358+0000 7f3840739640 1 -- 172.21.15.89:0/4139935755 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3838102eb0 con 0x7f3838102030 2024-09-21T03:40:14.363 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:14.359+0000 7f3840739640 1 -- 172.21.15.89:0/4139935755 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f383806ab90 con 0x7f3838102030 2024-09-21T03:40:14.364 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:14.362+0000 7f382f7fe640 1 -- 172.21.15.89:0/4139935755 <== mon.1 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 20) ==== 154563+0+0 (secure 0 0 0) 0x7f3834042030 con 0x7f3838102030 2024-09-21T03:40:14.366 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:14.364+0000 7f382f7fe640 1 --2- 172.21.15.89:0/4139935755 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f38240b65a0 0x7f38240b8ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:14.366 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:14.364+0000 7f382f7fe640 1 -- 172.21.15.89:0/4139935755 <== mon.1 v2:172.21.15.114:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 9121+0+0 (secure 0 0 0) 0x7f3834107f60 con 0x7f3838102030 2024-09-21T03:40:14.366 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:14.364+0000 7f383dcd4640 1 --2- 172.21.15.89:0/4139935755 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f38240b65a0 0x7f38240b8ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:14.366 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:14.365+0000 7f383dcd4640 1 --2- 172.21.15.89:0/4139935755 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f38240b65a0 0x7f38240b8ab0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f3838109210 tx=0x7f3828005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:14.368 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:14.365+0000 7f382f7fe640 1 -- 172.21.15.89:0/4139935755 <== mon.1 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f38340d2690 con 0x7f3838102030 2024-09-21T03:40:14.461 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:40:14 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:40:14.500 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:40:14 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:40:14.564 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:40:14 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:40:14.816 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:14.813+0000 7f3840739640 1 -- 172.21.15.89:0/4139935755 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3838102440 con 0x7f3838102030 2024-09-21T03:40:14.817 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:14.814+0000 7f382f7fe640 1 -- 172.21.15.89:0/4139935755 <== mon.1 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11912 (secure 0 0 0) 0x7f3838102440 con 0x7f3838102030 2024-09-21T03:40:14.817 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-09-21T03:40:14.818 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":26,"btime":"2024-09-21T03:39:26:058611+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14721,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6835/2341538514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":2341538514},{"type":"v1","addr":"172.21.15.89:6835","nonce":2341538514}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14733,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6837/146708592","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6836","nonce":146708592},{"type":"v1","addr":"172.21.15.89:6837","nonce":146708592}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14781,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6839/2269965499","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6838","nonce":2269965499},{"type":"v1","addr":"172.21.15.89:6839","nonce":2269965499}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14793,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6841/650850383","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6840","nonce":650850383},{"type":"v1","addr":"172.21.15.89:6841","nonce":650850383}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24565,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6833/752336912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6832","nonce":752336912},{"type":"v1","addr":"172.21.15.114:6833","nonce":752336912}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24575,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6833/127607331","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6832","nonce":127607331},{"type":"v1","addr":"172.21.15.110:6833","nonce":127607331}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T03:37:13.570694+0000","modified":"2024-09-21T03:39:25.333954+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24638,"mds_1":24631,"mds_2":24614},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24593":{"gid":24593,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.110:6835/671820654","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6834","nonce":671820654},{"type":"v1","addr":"172.21.15.110:6835","nonce":671820654}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24604":{"gid":24604,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.114:6835/1049714548","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6834","nonce":1049714548},{"type":"v1","addr":"172.21.15.114:6835","nonce":1049714548}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24610":{"gid":24610,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.114:6837/1941213818","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6836","nonce":1941213818},{"type":"v1","addr":"172.21.15.114:6837","nonce":1941213818}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24614":{"gid":24614,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":15,"addr":"172.21.15.110:6837/4047124589","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6836","nonce":4047124589},{"type":"v1","addr":"172.21.15.110:6837","nonce":4047124589}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24631":{"gid":24631,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":15,"addr":"172.21.15.114:6839/3427369944","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6838","nonce":3427369944},{"type":"v1","addr":"172.21.15.114:6839","nonce":3427369944}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24638":{"gid":24638,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.110:6839/132366709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6838","nonce":132366709},{"type":"v1","addr":"172.21.15.110:6839","nonce":132366709}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24638,"qdb_cluster":[24638,24631,24614]},"id":1}]} 2024-09-21T03:40:14.818 INFO:teuthology.orchestra.run.smithi089.stderr:dumped fsmap epoch 26 2024-09-21T03:40:14.820 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:14.818+0000 7f3840739640 1 -- 172.21.15.89:0/4139935755 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f38240b65a0 msgr2=0x7f38240b8ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:14.820 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:14.818+0000 7f3840739640 1 --2- 172.21.15.89:0/4139935755 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f38240b65a0 0x7f38240b8ab0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f3838109210 tx=0x7f3828005f90 comp rx=0 tx=0).stop 2024-09-21T03:40:14.820 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:14.818+0000 7f3840739640 1 -- 172.21.15.89:0/4139935755 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3838102030 msgr2=0x7f383819a500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:14.820 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:14.818+0000 7f3840739640 1 --2- 172.21.15.89:0/4139935755 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3838102030 0x7f383819a500 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f3834005940 tx=0x7f3834005e10 comp rx=0 tx=0).stop 2024-09-21T03:40:14.821 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:14.818+0000 7f3840739640 1 -- 172.21.15.89:0/4139935755 shutdown_connections 2024-09-21T03:40:14.821 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:14.818+0000 7f3840739640 1 --2- 172.21.15.89:0/4139935755 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f38240b65a0 0x7f38240b8ab0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:14.821 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:14.818+0000 7f3840739640 1 --2- 172.21.15.89:0/4139935755 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3838106280 0x7f383819aaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:14.821 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:14.818+0000 7f3840739640 1 --2- 172.21.15.89:0/4139935755 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3838106bd0 0x7f383819fd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:14.821 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:14.818+0000 7f3840739640 1 --2- 172.21.15.89:0/4139935755 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3838102030 0x7f383819a500 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:14.821 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:14.818+0000 7f3840739640 1 -- 172.21.15.89:0/4139935755 >> 172.21.15.89:0/4139935755 conn(0x7f3838077c60 msgr2=0x7f3838107600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:40:14.821 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:14.818+0000 7f3840739640 1 -- 172.21.15.89:0/4139935755 shutdown_connections 2024-09-21T03:40:14.821 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:14.818+0000 7f3840739640 1 -- 172.21.15.89:0/4139935755 wait complete. 2024-09-21T03:40:15.641 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24638 quiesce db --set-id 857cf945 --release --await' 2024-09-21T03:40:15.641 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 7c374098-77c9-11ef-baf5-efdc52797490 -- timeout 105 ceph tell mds.24638 quiesce db --set-id 857cf945 --release --await 2024-09-21T03:40:15.704 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:40:15 smithi089 ceph-mon[30928]: pgmap v385: 129 pgs: 129 active+clean; 675 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 170 B/s wr, 6 op/s 2024-09-21T03:40:15.704 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:40:15 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:40:15.704 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:40:15 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:40:15.704 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:40:15 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:40:15.704 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:40:15 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:40:15.705 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:40:15 smithi089 ceph-mon[30928]: from='client.? 172.21.15.89:0/4139935755' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:40:15.705 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:40:15 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:40:15.705 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:40:15 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:40:15.705 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:40:15 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:40:15.711 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:40:15 smithi114 ceph-mon[34140]: pgmap v385: 129 pgs: 129 active+clean; 675 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 170 B/s wr, 6 op/s 2024-09-21T03:40:15.711 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:40:15 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:40:15.711 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:40:15 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:40:15.711 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:40:15 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:40:15.711 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:40:15 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:40:15.711 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:40:15 smithi114 ceph-mon[34140]: from='client.? 172.21.15.89:0/4139935755' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:40:15.711 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:40:15 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:40:15.712 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:40:15 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:40:15.712 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:40:15 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:40:15.750 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:40:15 smithi110 ceph-mon[34000]: pgmap v385: 129 pgs: 129 active+clean; 675 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 170 B/s wr, 6 op/s 2024-09-21T03:40:15.750 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:40:15 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:40:15.750 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:40:15 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:40:15.750 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:40:15 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:40:15.750 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:40:15 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:40:15.750 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:40:15 smithi110 ceph-mon[34000]: from='client.? 172.21.15.89:0/4139935755' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:40:15.751 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:40:15 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:40:15.751 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:40:15 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:40:15.751 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:40:15 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:40:15.959 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/7c374098-77c9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:40:16.536 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:16.533+0000 7f02273c2640 1 -- 172.21.15.89:0/84880363 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0220102a60 msgr2=0x7f0220102e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:16.537 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:16.533+0000 7f02273c2640 1 --2- 172.21.15.89:0/84880363 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0220102a60 0x7f0220102e70 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f021c009900 tx=0x7f021c02a410 comp rx=0 tx=0).stop 2024-09-21T03:40:16.537 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:16.533+0000 7f02273c2640 1 -- 172.21.15.89:0/84880363 shutdown_connections 2024-09-21T03:40:16.537 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:16.533+0000 7f02273c2640 1 --2- 172.21.15.89:0/84880363 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f02201083a0 0x7f02201087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:16.537 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:16.533+0000 7f02273c2640 1 --2- 172.21.15.89:0/84880363 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0220108cf0 0x7f022010f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:16.537 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:16.533+0000 7f02273c2640 1 --2- 172.21.15.89:0/84880363 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0220102a60 0x7f0220102e70 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:16.537 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:16.533+0000 7f02273c2640 1 -- 172.21.15.89:0/84880363 >> 172.21.15.89:0/84880363 conn(0x7f0220079730 msgr2=0x7f0220079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:40:16.537 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:16.533+0000 7f02273c2640 1 -- 172.21.15.89:0/84880363 shutdown_connections 2024-09-21T03:40:16.537 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:16.533+0000 7f02273c2640 1 -- 172.21.15.89:0/84880363 wait complete. 2024-09-21T03:40:16.538 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:16.535+0000 7f02273c2640 1 Processor -- start 2024-09-21T03:40:16.538 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:16.535+0000 7f02273c2640 1 -- start start 2024-09-21T03:40:16.538 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:16.536+0000 7f02273c2640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0220102a60 0x7f0220113140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:16.538 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:16.536+0000 7f02273c2640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f02201083a0 0x7f0220113680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:16.538 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:16.536+0000 7f02273c2640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0220108cf0 0x7f0220113bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:16.539 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:16.536+0000 7f02273c2640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f0220112860 con 0x7f02201083a0 2024-09-21T03:40:16.539 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:16.536+0000 7f02273c2640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f02201126e0 con 0x7f0220108cf0 2024-09-21T03:40:16.539 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:16.536+0000 7f02273c2640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f02201129e0 con 0x7f0220102a60 2024-09-21T03:40:16.540 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:16.536+0000 7f022495d640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f02201083a0 0x7f0220113680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:16.540 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:16.536+0000 7f022515e640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0220102a60 0x7f0220113140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:16.540 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:16.536+0000 7f022595f640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0220108cf0 0x7f0220113bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:16.540 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:16.536+0000 7f022515e640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0220102a60 0x7f0220113140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.89:60592/0 (socket says 172.21.15.89:60592) 2024-09-21T03:40:16.541 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:16.536+0000 7f022515e640 1 -- 172.21.15.89:0/1119774715 learned_addr learned my addr 172.21.15.89:0/1119774715 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-21T03:40:16.541 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:16.536+0000 7f022595f640 1 -- 172.21.15.89:0/1119774715 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0220102a60 msgr2=0x7f0220113140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:16.541 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:16.536+0000 7f022595f640 1 --2- 172.21.15.89:0/1119774715 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0220102a60 0x7f0220113140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:16.541 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:16.536+0000 7f022595f640 1 -- 172.21.15.89:0/1119774715 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f02201083a0 msgr2=0x7f0220113680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:16.541 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:16.537+0000 7f022595f640 1 --2- 172.21.15.89:0/1119774715 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f02201083a0 0x7f0220113680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:16.541 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:16.537+0000 7f022595f640 1 -- 172.21.15.89:0/1119774715 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02201a86c0 con 0x7f0220108cf0 2024-09-21T03:40:16.541 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:16.537+0000 7f022495d640 1 --2- 172.21.15.89:0/1119774715 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f02201083a0 0x7f0220113680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:40:16.541 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:16.537+0000 7f022595f640 1 --2- 172.21.15.89:0/1119774715 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0220108cf0 0x7f0220113bc0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f021800e940 tx=0x7f021800ee00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:16.541 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:16.537+0000 7f02167fc640 1 -- 172.21.15.89:0/1119774715 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0218004d20 con 0x7f0220108cf0 2024-09-21T03:40:16.541 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:16.537+0000 7f02167fc640 1 -- 172.21.15.89:0/1119774715 <== mon.2 v2:172.21.15.110:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0218017e40 con 0x7f0220108cf0 2024-09-21T03:40:16.541 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:16.537+0000 7f02167fc640 1 -- 172.21.15.89:0/1119774715 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02180056c0 con 0x7f0220108cf0 2024-09-21T03:40:16.542 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:16.538+0000 7f02273c2640 1 -- 172.21.15.89:0/1119774715 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f022019a170 con 0x7f0220108cf0 2024-09-21T03:40:16.542 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:16.538+0000 7f02273c2640 1 -- 172.21.15.89:0/1119774715 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f022019a660 con 0x7f0220108cf0 2024-09-21T03:40:16.544 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:16.541+0000 7f022d936740 1 Processor -- start 2024-09-21T03:40:16.544 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:16.542+0000 7f02167fc640 1 -- 172.21.15.89:0/1119774715 <== mon.2 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 20) ==== 154563+0+0 (secure 0 0 0) 0x7f0218005860 con 0x7f0220108cf0 2024-09-21T03:40:16.544 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:16.542+0000 7f022d936740 1 -- start start 2024-09-21T03:40:16.545 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:16.542+0000 7f022d936740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555dcba8b520 0x555dcbe31510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:16.545 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:16.542+0000 7f022d936740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555dcba62660 0x555dcbe31a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:16.545 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:16.542+0000 7f022d936740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555dcba6a840 0x555dcbe31f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:16.545 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:16.543+0000 7f022d936740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x555dcbd1cdc0 con 0x555dcba62660 2024-09-21T03:40:16.546 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:16.543+0000 7f022d936740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x555dcbda91f0 con 0x555dcba6a840 2024-09-21T03:40:16.546 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:16.543+0000 7f022d936740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x555dcbd98620 con 0x555dcba8b520 2024-09-21T03:40:16.546 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:16.543+0000 7f022515e640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555dcba62660 0x555dcbe31a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:16.546 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:16.543+0000 7f022515e640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555dcba62660 0x555dcbe31a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:60812/0 (socket says 172.21.15.89:60812) 2024-09-21T03:40:16.547 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:16.543+0000 7f022515e640 1 -- 172.21.15.89:0/3869953662 learned_addr learned my addr 172.21.15.89:0/3869953662 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-21T03:40:16.547 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:16.543+0000 7f022495d640 1 --2- 172.21.15.89:0/3869953662 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555dcba8b520 0x555dcbe31510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:16.547 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:16.543+0000 7f022495d640 1 --2- 172.21.15.89:0/3869953662 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555dcba6a840 0x555dcbe31f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:16.548 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:16.543+0000 7f022515e640 1 -- 172.21.15.89:0/3869953662 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555dcba8b520 msgr2=0x555dcbe31510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:40:16.548 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:16.543+0000 7f022515e640 1 --2- 172.21.15.89:0/3869953662 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555dcba8b520 0x555dcbe31510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:16.548 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:16.543+0000 7f022515e640 1 -- 172.21.15.89:0/3869953662 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555dcba6a840 msgr2=0x555dcbe31f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:40:16.548 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:16.543+0000 7f022515e640 1 --2- 172.21.15.89:0/3869953662 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555dcba6a840 0x555dcbe31f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:16.548 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:16.543+0000 7f022515e640 1 -- 172.21.15.89:0/3869953662 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555dcbd60fd0 con 0x555dcba62660 2024-09-21T03:40:16.548 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:16.544+0000 7f022515e640 1 --2- 172.21.15.89:0/3869953662 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555dcba62660 0x555dcbe31a50 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f021c0095a0 tx=0x7f021c0095d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a90259118e446e1c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:16.548 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:16.544+0000 7f02167fc640 1 --2- 172.21.15.89:0/1119774715 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f020c0b65a0 0x7f020c0b8ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:16.549 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:16.544+0000 7f020b7fe640 1 -- 172.21.15.89:0/3869953662 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f021c051070 con 0x555dcba62660 2024-09-21T03:40:16.549 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:16.544+0000 7f020b7fe640 1 -- 172.21.15.89:0/3869953662 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f021c0054b0 con 0x555dcba62660 2024-09-21T03:40:16.549 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:16.544+0000 7f020b7fe640 1 -- 172.21.15.89:0/3869953662 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f021c035040 con 0x555dcba62660 2024-09-21T03:40:16.549 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:16.544+0000 7f022495d640 1 --2- 172.21.15.89:0/1119774715 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f020c0b65a0 0x7f020c0b8ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:16.549 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:16.544+0000 7f02167fc640 1 -- 172.21.15.89:0/1119774715 <== mon.2 v2:172.21.15.110:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 9121+0+0 (secure 0 0 0) 0x7f02180efac0 con 0x7f0220108cf0 2024-09-21T03:40:16.549 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:16.545+0000 7f022495d640 1 --2- 172.21.15.89:0/1119774715 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f020c0b65a0 0x7f020c0b8ab0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f021000bfd0 tx=0x7f021000b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:16.550 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:16.545+0000 7f022d936740 1 -- 172.21.15.89:0/3869953662 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555dcba62660 msgr2=0x555dcbe31a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:16.550 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:16.545+0000 7f022d936740 1 --2- 172.21.15.89:0/3869953662 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555dcba62660 0x555dcbe31a50 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f021c0095a0 tx=0x7f021c0095d0 comp rx=0 tx=0).stop 2024-09-21T03:40:16.550 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:16.546+0000 7f022d936740 1 -- 172.21.15.89:0/3869953662 shutdown_connections 2024-09-21T03:40:16.550 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:16.546+0000 7f022d936740 1 --2- 172.21.15.89:0/3869953662 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555dcba6a840 0x555dcbe31f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:16.550 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:16.546+0000 7f022d936740 1 --2- 172.21.15.89:0/3869953662 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555dcba62660 0x555dcbe31a50 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:16.550 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:16.546+0000 7f022d936740 1 --2- 172.21.15.89:0/3869953662 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555dcba8b520 0x555dcbe31510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:16.550 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:16.546+0000 7f022d936740 1 -- 172.21.15.89:0/3869953662 >> 172.21.15.89:0/3869953662 conn(0x555dcba7f910 msgr2=0x555dcbe2f400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:40:16.550 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:16.546+0000 7f022d936740 1 -- 172.21.15.89:0/3869953662 shutdown_connections 2024-09-21T03:40:16.551 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:16.546+0000 7f022d936740 1 -- 172.21.15.89:0/3869953662 wait complete. 2024-09-21T03:40:16.551 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:16.546+0000 7f022d936740 1 -- start start 2024-09-21T03:40:16.551 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:16.547+0000 7f022d936740 1 Processor -- start 2024-09-21T03:40:16.552 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:16.547+0000 7f022d936740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555dcba6a840 0x555dcbe3cc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:16.552 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:16.547+0000 7f022d936740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555dcba8b520 0x555dcbe3d190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:16.552 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:16.548+0000 7f022d936740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555dcba49ea0 0x555dcbe59e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:16.552 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:16.548+0000 7f022d936740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x555dcbd33300 con 0x555dcba6a840 2024-09-21T03:40:16.552 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:16.548+0000 7f022d936740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x555dcbd9c430 con 0x555dcba49ea0 2024-09-21T03:40:16.552 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:16.548+0000 7f022515e640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555dcba6a840 0x555dcbe3cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:16.553 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:16.548+0000 7f022d936740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x555dcbbf9bb0 con 0x555dcba8b520 2024-09-21T03:40:16.553 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:16.548+0000 7f022515e640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555dcba6a840 0x555dcbe3cc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:60824/0 (socket says 172.21.15.89:60824) 2024-09-21T03:40:16.553 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:16.548+0000 7f022515e640 1 -- 172.21.15.89:0/261406288 learned_addr learned my addr 172.21.15.89:0/261406288 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-21T03:40:16.553 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:16.548+0000 7f022d936740 10 client.0 _ll_register_callbacks cb 0x555dcbd4d120 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:40:16.553 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:16.548+0000 7f022495d640 1 --2- 172.21.15.89:0/261406288 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555dcba8b520 0x555dcbe3d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:16.553 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:16.548+0000 7f022595f640 1 --2- 172.21.15.89:0/261406288 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555dcba49ea0 0x555dcbe59e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:16.553 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:16.548+0000 7f022515e640 1 -- 172.21.15.89:0/261406288 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555dcba8b520 msgr2=0x555dcbe3d190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:40:16.553 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:16.548+0000 7f022515e640 1 --2- 172.21.15.89:0/261406288 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555dcba8b520 0x555dcbe3d190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:16.553 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:16.548+0000 7f022515e640 1 -- 172.21.15.89:0/261406288 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555dcba49ea0 msgr2=0x555dcbe59e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:40:16.553 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:16.548+0000 7f022515e640 1 --2- 172.21.15.89:0/261406288 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555dcba49ea0 0x555dcbe59e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:16.554 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:16.548+0000 7f022515e640 1 -- 172.21.15.89:0/261406288 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555dcbd22780 con 0x555dcba6a840 2024-09-21T03:40:16.554 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:16.549+0000 7f022515e640 1 --2- 172.21.15.89:0/261406288 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555dcba6a840 0x555dcbe3cc50 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f021c009850 tx=0x7f021c005250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f3478405eb632fe server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:16.554 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:16.549+0000 7f0208ff9640 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2024-09-21T03:40:16.554 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:16.549+0000 7f0208ff9640 1 -- 172.21.15.89:0/261406288 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f021c035070 con 0x555dcba6a840 2024-09-21T03:40:16.554 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:16.549+0000 7f0208ff9640 1 -- 172.21.15.89:0/261406288 <== mon.0 v2:172.21.15.89:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 9121+0+0 (secure 0 0 0) 0x7f021c0048d0 con 0x555dcba6a840 2024-09-21T03:40:16.554 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:16.549+0000 7f022d936740 1 -- 172.21.15.89:0/261406288 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f021c0054b0 con 0x555dcba6a840 2024-09-21T03:40:16.554 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:16.549+0000 7f0208ff9640 1 -- 172.21.15.89:0/261406288 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f021c051040 con 0x555dcba6a840 2024-09-21T03:40:16.554 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:16.549+0000 7f022d936740 10 client.15630 fetch_fsmap learned FSMap version 26 2024-09-21T03:40:16.554 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:16.549+0000 7f022d936740 1 -- 172.21.15.89:0/261406288 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x555dcbaf6b00 con 0x555dcba6a840 2024-09-21T03:40:16.554 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:16.550+0000 7f0208ff9640 1 -- 172.21.15.89:0/261406288 <== mon.0 v2:172.21.15.89:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f021c0054b0 con 0x555dcba6a840 2024-09-21T03:40:16.556 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:16.551+0000 7f0208ff9640 1 -- 172.21.15.89:0/261406288 <== mon.0 v2:172.21.15.89:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f021c068c10 con 0x555dcba6a840 2024-09-21T03:40:16.556 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:16.551+0000 7f022d936740 10 client.15630 fetch_fsmap finished waiting for FSMap version 26 2024-09-21T03:40:16.556 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:16.551+0000 7f022d936740 10 client.15630 resolve_mds: validated gid 24638 aka daemon mds.b 2024-09-21T03:40:16.556 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:16.551+0000 7f022d936740 20 client.15630 populate_metadata read hostname 'smithi089' 2024-09-21T03:40:16.556 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:16.551+0000 7f022d936740 1 --2- 172.21.15.89:0/261406288 >> [v2:172.21.15.110:6838/132366709,v1:172.21.15.110:6839/132366709] conn(0x555dcba1f250 0x555dcbe5fdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:16.556 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:16.551+0000 7f022d936740 4 client.15630 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:40:16.556 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:16.551+0000 7f022d936740 1 -- 172.21.15.89:0/261406288 --> [v2:172.21.15.110:6838/132366709,v1:172.21.15.110:6839/132366709] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555dcbdab140 con 0x555dcba1f250 2024-09-21T03:40:16.556 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:16.551+0000 7f022495d640 1 --2- 172.21.15.89:0/261406288 >> [v2:172.21.15.110:6838/132366709,v1:172.21.15.110:6839/132366709] conn(0x555dcba1f250 0x555dcbe5fdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:16.556 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:16.552+0000 7f022495d640 1 --2- 172.21.15.89:0/261406288 >> [v2:172.21.15.110:6838/132366709,v1:172.21.15.110:6839/132366709] conn(0x555dcba1f250 0x555dcbe5fdd0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=85daeafdb789776 server_cookie=f4047694d1ff24b in_seq=0 out_seq=0 2024-09-21T03:40:16.557 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:16.552+0000 7f0208ff9640 10 client.15630 ms_handle_connect on v2:172.21.15.110:6838/132366709 2024-09-21T03:40:16.557 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:16.553+0000 7f0208ff9640 1 -- 172.21.15.89:0/261406288 <== mds.0 v2:172.21.15.110:6838/132366709 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x555dcbdab140 con 0x555dcba1f250 2024-09-21T03:40:16.557 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:16.553+0000 7f0208ff9640 10 client.15630 handle_command_reply: tid=0 2024-09-21T03:40:16.557 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:16.553+0000 7f022d936740 1 client.15630 shutdown 2024-09-21T03:40:16.557 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:16.553+0000 7f022d936740 1 -- 172.21.15.89:0/261406288 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555dcba6a840 msgr2=0x555dcbe3cc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:16.557 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:16.553+0000 7f022d936740 1 --2- 172.21.15.89:0/261406288 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555dcba6a840 0x555dcbe3cc50 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f021c009850 tx=0x7f021c005250 comp rx=0 tx=0).stop 2024-09-21T03:40:16.557 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:16.553+0000 7f022d936740 1 -- 172.21.15.89:0/261406288 shutdown_connections 2024-09-21T03:40:16.557 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:16.553+0000 7f022d936740 1 --2- 172.21.15.89:0/261406288 >> [v2:172.21.15.110:6838/132366709,v1:172.21.15.110:6839/132366709] conn(0x555dcba1f250 0x555dcbe5fdd0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:16.557 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:16.553+0000 7f0208ff9640 0 client.15630 ms_handle_reset on v2:172.21.15.110:6838/132366709 2024-09-21T03:40:16.557 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:16.553+0000 7f022d936740 1 --2- 172.21.15.89:0/261406288 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555dcba49ea0 0x555dcbe59e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:16.557 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:16.553+0000 7f022d936740 1 --2- 172.21.15.89:0/261406288 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555dcba8b520 0x555dcbe3d190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:16.557 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:16.553+0000 7f022d936740 1 --2- 172.21.15.89:0/261406288 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555dcba6a840 0x555dcbe3cc50 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:16.557 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:16.553+0000 7f022d936740 1 -- 172.21.15.89:0/261406288 >> 172.21.15.89:0/261406288 conn(0x555dcba7f910 msgr2=0x555dcbe346a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:40:16.557 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:16.553+0000 7f022d936740 1 -- 172.21.15.89:0/261406288 shutdown_connections 2024-09-21T03:40:16.558 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:16.554+0000 7f022d936740 1 -- 172.21.15.89:0/261406288 wait complete. 2024-09-21T03:40:16.558 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:16.554+0000 7f022d936740 20 client.15630 trim_cache size 0 max 16384 2024-09-21T03:40:16.590 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:16.587+0000 7f022d936740 1 Processor -- start 2024-09-21T03:40:16.590 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:16.587+0000 7f022d936740 1 -- start start 2024-09-21T03:40:16.590 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:16.588+0000 7f022d936740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555dcba49ea0 0x555dcbe383f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:16.591 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:16.588+0000 7f022d936740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555dcba6a840 0x555dcbe2f460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:16.591 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:16.588+0000 7f022d936740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555dcba8b520 0x555dcbe32c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:16.591 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:16.588+0000 7f022d936740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f02201a8930 con 0x555dcba8b520 2024-09-21T03:40:16.591 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:16.588+0000 7f022d936740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f0220072f60 con 0x555dcba49ea0 2024-09-21T03:40:16.591 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:16.588+0000 7f022d936740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f0220070db0 con 0x555dcba6a840 2024-09-21T03:40:16.591 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:16.588+0000 7f022595f640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555dcba8b520 0x555dcbe32c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:16.591 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:16.588+0000 7f022595f640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555dcba8b520 0x555dcbe32c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:60838/0 (socket says 172.21.15.89:60838) 2024-09-21T03:40:16.591 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:16.588+0000 7f022595f640 1 -- 172.21.15.89:0/1385510200 learned_addr learned my addr 172.21.15.89:0/1385510200 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-21T03:40:16.591 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:16.588+0000 7f022515e640 1 --2- 172.21.15.89:0/1385510200 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555dcba49ea0 0x555dcbe383f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:16.592 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:16.589+0000 7f022595f640 1 -- 172.21.15.89:0/1385510200 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555dcba6a840 msgr2=0x555dcbe2f460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:40:16.592 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:16.589+0000 7f022595f640 1 --2- 172.21.15.89:0/1385510200 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555dcba6a840 0x555dcbe2f460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:16.592 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:16.589+0000 7f022595f640 1 -- 172.21.15.89:0/1385510200 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555dcba49ea0 msgr2=0x555dcbe383f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:40:16.592 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:16.589+0000 7f022595f640 1 --2- 172.21.15.89:0/1385510200 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555dcba49ea0 0x555dcbe383f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:16.592 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:16.589+0000 7f022595f640 1 -- 172.21.15.89:0/1385510200 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555dcbb6d960 con 0x555dcba8b520 2024-09-21T03:40:16.592 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:16.589+0000 7f022595f640 1 --2- 172.21.15.89:0/1385510200 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555dcba8b520 0x555dcbe32c50 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f02180b57b0 tx=0x7f02180b5170 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15959649c925a72f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:16.592 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:16.589+0000 7f020affd640 1 -- 172.21.15.89:0/1385510200 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02180ba020 con 0x555dcba8b520 2024-09-21T03:40:16.593 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:16.589+0000 7f020affd640 1 -- 172.21.15.89:0/1385510200 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f02180b5330 con 0x555dcba8b520 2024-09-21T03:40:16.593 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:16.589+0000 7f020affd640 1 -- 172.21.15.89:0/1385510200 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02180b13d0 con 0x555dcba8b520 2024-09-21T03:40:16.594 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:16.590+0000 7f022d936740 1 -- 172.21.15.89:0/1385510200 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555dcba8b520 msgr2=0x555dcbe32c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:16.594 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:16.590+0000 7f022d936740 1 --2- 172.21.15.89:0/1385510200 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555dcba8b520 0x555dcbe32c50 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f02180b57b0 tx=0x7f02180b5170 comp rx=0 tx=0).stop 2024-09-21T03:40:16.594 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:16.591+0000 7f022d936740 1 -- 172.21.15.89:0/1385510200 shutdown_connections 2024-09-21T03:40:16.594 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:16.591+0000 7f022d936740 1 --2- 172.21.15.89:0/1385510200 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555dcba8b520 0x555dcbe32c50 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:16.594 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:16.591+0000 7f022d936740 1 --2- 172.21.15.89:0/1385510200 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555dcba6a840 0x555dcbe2f460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:16.594 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:16.591+0000 7f022d936740 1 --2- 172.21.15.89:0/1385510200 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555dcba49ea0 0x555dcbe383f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:16.594 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:16.591+0000 7f022d936740 1 -- 172.21.15.89:0/1385510200 >> 172.21.15.89:0/1385510200 conn(0x555dcba7f910 msgr2=0x555dcbe32540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:40:16.594 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:16.591+0000 7f022d936740 1 -- 172.21.15.89:0/1385510200 shutdown_connections 2024-09-21T03:40:16.594 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:16.591+0000 7f022d936740 1 -- 172.21.15.89:0/1385510200 wait complete. 2024-09-21T03:40:16.594 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:16.592+0000 7f022d936740 1 -- start start 2024-09-21T03:40:16.594 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:16.592+0000 7f022d936740 1 Processor -- start 2024-09-21T03:40:16.597 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:16.592+0000 7f022d936740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555dcba49ea0 0x555dcbe5f910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:16.597 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:16.592+0000 7f022d936740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555dcba6a840 0x555dcbe5fef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:16.597 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:16.593+0000 7f022d936740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555dcba8b520 0x555dcbec3660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:16.597 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:16.593+0000 7f022d936740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f0220071420 con 0x555dcba6a840 2024-09-21T03:40:16.598 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:16.593+0000 7f022d936740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f0220072890 con 0x555dcba8b520 2024-09-21T03:40:16.598 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:16.593+0000 7f022d936740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f0220071af0 con 0x555dcba49ea0 2024-09-21T03:40:16.598 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:16.593+0000 7f022d936740 10 client.0 _ll_register_callbacks cb 0x7f02100125d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:40:16.598 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:16.593+0000 7f022515e640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555dcba49ea0 0x555dcbe5f910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:16.598 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:16.593+0000 7f022595f640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555dcba8b520 0x555dcbec3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:16.598 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:16.593+0000 7f022515e640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555dcba49ea0 0x555dcbe5f910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.89:60612/0 (socket says 172.21.15.89:60612) 2024-09-21T03:40:16.598 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:16.593+0000 7f022515e640 1 -- 172.21.15.89:0/2738164119 learned_addr learned my addr 172.21.15.89:0/2738164119 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-21T03:40:16.598 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:16.593+0000 7f022595f640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555dcba8b520 0x555dcbec3660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.89:44022/0 (socket says 172.21.15.89:44022) 2024-09-21T03:40:16.598 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:16.593+0000 7f022595f640 1 -- 172.21.15.89:0/2738164119 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555dcba49ea0 msgr2=0x555dcbe5f910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:40:16.598 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:16.593+0000 7f022595f640 1 --2- 172.21.15.89:0/2738164119 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555dcba49ea0 0x555dcbe5f910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:16.598 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:16.593+0000 7f022595f640 1 -- 172.21.15.89:0/2738164119 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555dcba6a840 msgr2=0x555dcbe5fef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:40:16.598 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:16.593+0000 7f022595f640 1 --2- 172.21.15.89:0/2738164119 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555dcba6a840 0x555dcbe5fef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:16.599 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:16.593+0000 7f022595f640 1 -- 172.21.15.89:0/2738164119 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555dcbec3cd0 con 0x555dcba8b520 2024-09-21T03:40:16.599 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:16.593+0000 7f022595f640 1 --2- 172.21.15.89:0/2738164119 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555dcba8b520 0x555dcbec3660 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x555dcbe2fcf0 tx=0x7f02180b4d30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b2ff5cadea409918 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:16.599 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:16.593+0000 7f0208ff9640 10 client.0 ms_handle_connect on v2:172.21.15.110:3300/0 2024-09-21T03:40:16.599 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:16.594+0000 7f0208ff9640 1 -- 172.21.15.89:0/2738164119 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02180ba020 con 0x555dcba8b520 2024-09-21T03:40:16.599 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:16.594+0000 7f0208ff9640 1 -- 172.21.15.89:0/2738164119 <== mon.2 v2:172.21.15.110:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 9121+0+0 (secure 0 0 0) 0x7f02180b44f0 con 0x555dcba8b520 2024-09-21T03:40:16.599 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:16.594+0000 7f022d936740 1 -- 172.21.15.89:0/2738164119 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f02180b5330 con 0x555dcba8b520 2024-09-21T03:40:16.599 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:16.594+0000 7f0208ff9640 1 -- 172.21.15.89:0/2738164119 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f02180ba250 con 0x555dcba8b520 2024-09-21T03:40:16.599 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:16.594+0000 7f022d936740 10 client.25199 fetch_fsmap learned FSMap version 26 2024-09-21T03:40:16.599 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:16.594+0000 7f022d936740 1 -- 172.21.15.89:0/2738164119 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x555dcbecb0e0 con 0x555dcba8b520 2024-09-21T03:40:16.599 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:16.594+0000 7f0208ff9640 1 -- 172.21.15.89:0/2738164119 <== mon.2 v2:172.21.15.110:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0218005000 con 0x555dcba8b520 2024-09-21T03:40:16.599 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:16.594+0000 7f0208ff9640 1 -- 172.21.15.89:0/2738164119 <== mon.2 v2:172.21.15.110:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f02180cdce0 con 0x555dcba8b520 2024-09-21T03:40:16.599 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:16.594+0000 7f022d936740 10 client.25199 fetch_fsmap finished waiting for FSMap version 26 2024-09-21T03:40:16.599 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:16.594+0000 7f022d936740 10 client.25199 resolve_mds: validated gid 24638 aka daemon mds.b 2024-09-21T03:40:16.600 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:16.595+0000 7f022d936740 20 client.25199 populate_metadata read hostname 'smithi089' 2024-09-21T03:40:16.600 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:16.595+0000 7f022d936740 1 --2- 172.21.15.89:0/2738164119 >> [v2:172.21.15.110:6838/132366709,v1:172.21.15.110:6839/132366709] conn(0x555dcbecb840 0x555dcbecdc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:16.600 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:16.595+0000 7f022d936740 4 client.25199 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "857cf945", "release": true, "await": true}] 2024-09-21T03:40:16.600 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:16.595+0000 7f022d936740 1 -- 172.21.15.89:0/2738164119 --> [v2:172.21.15.110:6838/132366709,v1:172.21.15.110:6839/132366709] -- command(tid 0: {"prefix": "quiesce db", "set_id": "857cf945", "release": true, "await": true}) -- 0x7f02180b5330 con 0x555dcbecb840 2024-09-21T03:40:16.600 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:16.595+0000 7f022515e640 1 --2- 172.21.15.89:0/2738164119 >> [v2:172.21.15.110:6838/132366709,v1:172.21.15.110:6839/132366709] conn(0x555dcbecb840 0x555dcbecdc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:16.601 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:16.595+0000 7f0208ff9640 10 client.25199 ms_handle_connect on v2:172.21.15.110:6838/132366709 2024-09-21T03:40:16.601 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:16.595+0000 7f022515e640 1 --2- 172.21.15.89:0/2738164119 >> [v2:172.21.15.110:6838/132366709,v1:172.21.15.110:6839/132366709] conn(0x555dcbecb840 0x555dcbecdc60 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f981aef7fc094dc1 server_cookie=35c3d8e6429b6094 in_seq=0 out_seq=0 2024-09-21T03:40:16.601 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:16.597+0000 7f0208ff9640 1 -- 172.21.15.89:0/2738164119 <== mds.0 v2:172.21.15.110:6838/132366709 1 ==== command_reply(tid 0: 0 ) ==== 8+0+589 (crc 0 0 0) 0x7f02180b5330 con 0x555dcbecb840 2024-09-21T03:40:16.601 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:16.598+0000 7f0208ff9640 10 client.25199 handle_command_reply: tid=0 2024-09-21T03:40:16.601 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:16.598+0000 7f022d936740 1 client.25199 shutdown 2024-09-21T03:40:16.602 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:16.598+0000 7f022d936740 1 -- 172.21.15.89:0/2738164119 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555dcba8b520 msgr2=0x555dcbec3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:16.602 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:16.598+0000 7f022d936740 1 --2- 172.21.15.89:0/2738164119 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555dcba8b520 0x555dcbec3660 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x555dcbe2fcf0 tx=0x7f02180b4d30 comp rx=0 tx=0).stop 2024-09-21T03:40:16.602 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:16.598+0000 7f022d936740 1 -- 172.21.15.89:0/2738164119 shutdown_connections 2024-09-21T03:40:16.602 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:16.598+0000 7f022d936740 1 --2- 172.21.15.89:0/2738164119 >> [v2:172.21.15.110:6838/132366709,v1:172.21.15.110:6839/132366709] conn(0x555dcbecb840 0x555dcbecdc60 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:16.602 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:16.598+0000 7f022d936740 1 --2- 172.21.15.89:0/2738164119 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555dcba8b520 0x555dcbec3660 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:16.602 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:16.598+0000 7f022d936740 1 --2- 172.21.15.89:0/2738164119 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555dcba6a840 0x555dcbe5fef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:16.602 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:16.598+0000 7f0208ff9640 0 client.25199 ms_handle_reset on v2:172.21.15.110:6838/132366709 2024-09-21T03:40:16.602 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:16.598+0000 7f022d936740 1 --2- 172.21.15.89:0/2738164119 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555dcba49ea0 0x555dcbe5f910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:16.602 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:16.598+0000 7f022d936740 1 -- 172.21.15.89:0/2738164119 >> 172.21.15.89:0/2738164119 conn(0x555dcba7f910 msgr2=0x555dcbe32f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:40:16.602 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:16.598+0000 7f022d936740 1 -- 172.21.15.89:0/2738164119 shutdown_connections 2024-09-21T03:40:16.603 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:16.598+0000 7f022d936740 1 -- 172.21.15.89:0/2738164119 wait complete. 2024-09-21T03:40:16.603 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:16.598+0000 7f022d936740 20 client.25199 trim_cache size 0 max 16384 2024-09-21T03:40:16.603 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-09-21T03:40:16.603 INFO:teuthology.orchestra.run.smithi089.stdout: "epoch": 26, 2024-09-21T03:40:16.603 INFO:teuthology.orchestra.run.smithi089.stdout: "leader": 24638, 2024-09-21T03:40:16.603 INFO:teuthology.orchestra.run.smithi089.stdout: "set_version": 4, 2024-09-21T03:40:16.603 INFO:teuthology.orchestra.run.smithi089.stdout: "sets": { 2024-09-21T03:40:16.603 INFO:teuthology.orchestra.run.smithi089.stdout: "857cf945": { 2024-09-21T03:40:16.604 INFO:teuthology.orchestra.run.smithi089.stdout: "version": 4, 2024-09-21T03:40:16.604 INFO:teuthology.orchestra.run.smithi089.stdout: "age_ref": 0.0, 2024-09-21T03:40:16.604 INFO:teuthology.orchestra.run.smithi089.stdout: "state": { 2024-09-21T03:40:16.604 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "RELEASED", 2024-09-21T03:40:16.604 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 0.0 2024-09-21T03:40:16.604 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:40:16.604 INFO:teuthology.orchestra.run.smithi089.stdout: "timeout": 90.0, 2024-09-21T03:40:16.604 INFO:teuthology.orchestra.run.smithi089.stdout: "expiration": 134.5, 2024-09-21T03:40:16.604 INFO:teuthology.orchestra.run.smithi089.stdout: "members": { 2024-09-21T03:40:16.604 INFO:teuthology.orchestra.run.smithi089.stdout: "file:/": { 2024-09-21T03:40:16.604 INFO:teuthology.orchestra.run.smithi089.stdout: "excluded": false, 2024-09-21T03:40:16.604 INFO:teuthology.orchestra.run.smithi089.stdout: "state": { 2024-09-21T03:40:16.605 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "RELEASED", 2024-09-21T03:40:16.605 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 0.0 2024-09-21T03:40:16.605 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-21T03:40:16.605 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-21T03:40:16.605 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-21T03:40:16.605 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-21T03:40:16.605 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-21T03:40:16.605 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-09-21T03:40:16.605 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:16.599+0000 7f02273c2640 1 -- 172.21.15.89:0/1119774715 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f020c0b65a0 msgr2=0x7f020c0b8ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:16.606 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:16.599+0000 7f02273c2640 1 --2- 172.21.15.89:0/1119774715 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f020c0b65a0 0x7f020c0b8ab0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f021000bfd0 tx=0x7f021000b390 comp rx=0 tx=0).stop 2024-09-21T03:40:16.606 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:16.599+0000 7f02273c2640 1 -- 172.21.15.89:0/1119774715 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0220108cf0 msgr2=0x7f0220113bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:16.606 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:16.599+0000 7f02273c2640 1 --2- 172.21.15.89:0/1119774715 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0220108cf0 0x7f0220113bc0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f021800e940 tx=0x7f021800ee00 comp rx=0 tx=0).stop 2024-09-21T03:40:16.606 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:16.599+0000 7f02273c2640 1 -- 172.21.15.89:0/1119774715 shutdown_connections 2024-09-21T03:40:16.606 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:16.599+0000 7f02273c2640 1 --2- 172.21.15.89:0/1119774715 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f020c0b65a0 0x7f020c0b8ab0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:16.606 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:16.599+0000 7f02273c2640 1 --2- 172.21.15.89:0/1119774715 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0220108cf0 0x7f0220113bc0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:16.606 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:16.599+0000 7f02273c2640 1 --2- 172.21.15.89:0/1119774715 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f02201083a0 0x7f0220113680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:16.606 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:16.599+0000 7f02273c2640 1 --2- 172.21.15.89:0/1119774715 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0220102a60 0x7f0220113140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:16.606 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:16.599+0000 7f02273c2640 1 -- 172.21.15.89:0/1119774715 >> 172.21.15.89:0/1119774715 conn(0x7f0220079730 msgr2=0x7f0220100620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:40:16.606 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:16.599+0000 7f02273c2640 1 -- 172.21.15.89:0/1119774715 shutdown_connections 2024-09-21T03:40:16.606 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:16.599+0000 7f02273c2640 1 -- 172.21.15.89:0/1119774715 wait complete. 2024-09-21T03:40:17.422 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '857cf945', total seconds elapsed: 18.4 2024-09-21T03:40:17.711 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:40:17 smithi114 ceph-mon[34140]: pgmap v386: 129 pgs: 129 active+clean; 675 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 170 B/s wr, 6 op/s 2024-09-21T03:40:17.750 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:40:17 smithi110 ceph-mon[34000]: pgmap v386: 129 pgs: 129 active+clean; 675 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 170 B/s wr, 6 op/s 2024-09-21T03:40:17.811 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:40:17 smithi089 ceph-mon[30928]: pgmap v386: 129 pgs: 129 active+clean; 675 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 170 B/s wr, 6 op/s 2024-09-21T03:40:19.123 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:19.121+0000 7f39dfb39640 1 -- 172.21.15.89:0/2000999794 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f39d81056e0 msgr2=0x7f39d8107bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:19.124 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:19.121+0000 7f39dfb39640 1 --2- 172.21.15.89:0/2000999794 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f39d81056e0 0x7f39d8107bc0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f39c8009900 tx=0x7f39c802a410 comp rx=0 tx=0).stop 2024-09-21T03:40:19.124 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:19.121+0000 7f39dfb39640 1 -- 172.21.15.89:0/2000999794 shutdown_connections 2024-09-21T03:40:19.124 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:19.121+0000 7f39dfb39640 1 --2- 172.21.15.89:0/2000999794 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f39d8108300 0x7f39d810a810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:19.124 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:19.121+0000 7f39dfb39640 1 --2- 172.21.15.89:0/2000999794 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f39d81056e0 0x7f39d8107bc0 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:19.124 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:19.121+0000 7f39dfb39640 1 --2- 172.21.15.89:0/2000999794 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f39d8102c80 0x7f39d81050a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:19.124 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:19.121+0000 7f39dfb39640 1 -- 172.21.15.89:0/2000999794 >> 172.21.15.89:0/2000999794 conn(0x7f39d80fd690 msgr2=0x7f39d80ffb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:40:19.124 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:19.121+0000 7f39dfb39640 1 -- 172.21.15.89:0/2000999794 shutdown_connections 2024-09-21T03:40:19.124 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:19.121+0000 7f39dfb39640 1 -- 172.21.15.89:0/2000999794 wait complete. 2024-09-21T03:40:19.126 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:19.122+0000 7f39dfb39640 1 Processor -- start 2024-09-21T03:40:19.126 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:19.122+0000 7f39dfb39640 1 -- start start 2024-09-21T03:40:19.126 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:19.122+0000 7f39dfb39640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f39d8102c80 0x7f39d8199ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:19.126 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:19.122+0000 7f39dfb39640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f39d81056e0 0x7f39d819a510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:19.126 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:19.122+0000 7f39dfb39640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f39d8108300 0x7f39d819b0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:19.126 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:19.122+0000 7f39dfb39640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f39d810e0d0 con 0x7f39d8108300 2024-09-21T03:40:19.126 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:19.122+0000 7f39dfb39640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f39d810df50 con 0x7f39d81056e0 2024-09-21T03:40:19.126 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:19.122+0000 7f39dfb39640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f39d810e250 con 0x7f39d8102c80 2024-09-21T03:40:19.126 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:19.123+0000 7f39de0d6640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f39d8108300 0x7f39d819b0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:19.126 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:19.123+0000 7f39de0d6640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f39d8108300 0x7f39d819b0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:34044/0 (socket says 172.21.15.89:34044) 2024-09-21T03:40:19.126 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:19.123+0000 7f39de0d6640 1 -- 172.21.15.89:0/3701156462 learned_addr learned my addr 172.21.15.89:0/3701156462 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-21T03:40:19.127 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:19.123+0000 7f39dd0d4640 1 --2- 172.21.15.89:0/3701156462 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f39d81056e0 0x7f39d819a510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:19.127 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:19.123+0000 7f39dd8d5640 1 --2- 172.21.15.89:0/3701156462 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f39d8102c80 0x7f39d8199ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:19.127 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:19.123+0000 7f39de0d6640 1 -- 172.21.15.89:0/3701156462 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f39d8102c80 msgr2=0x7f39d8199ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:19.127 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:19.123+0000 7f39de0d6640 1 --2- 172.21.15.89:0/3701156462 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f39d8102c80 0x7f39d8199ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:19.127 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:19.123+0000 7f39de0d6640 1 -- 172.21.15.89:0/3701156462 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f39d81056e0 msgr2=0x7f39d819a510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:19.127 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:19.123+0000 7f39de0d6640 1 --2- 172.21.15.89:0/3701156462 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f39d81056e0 0x7f39d819a510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:19.127 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:19.123+0000 7f39de0d6640 1 -- 172.21.15.89:0/3701156462 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39d819b8e0 con 0x7f39d8108300 2024-09-21T03:40:19.127 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:19.123+0000 7f39dd8d5640 1 --2- 172.21.15.89:0/3701156462 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f39d8102c80 0x7f39d8199ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:40:19.127 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:19.123+0000 7f39de0d6640 1 --2- 172.21.15.89:0/3701156462 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f39d8108300 0x7f39d819b0c0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f39d000eb10 tx=0x7f39d000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:19.127 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:19.124+0000 7f39ceffd640 1 -- 172.21.15.89:0/3701156462 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39d0004b20 con 0x7f39d8108300 2024-09-21T03:40:19.127 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:19.124+0000 7f39dfb39640 1 -- 172.21.15.89:0/3701156462 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f39d819bbd0 con 0x7f39d8108300 2024-09-21T03:40:19.128 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:19.124+0000 7f39ceffd640 1 -- 172.21.15.89:0/3701156462 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f39d0004cc0 con 0x7f39d8108300 2024-09-21T03:40:19.128 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:19.124+0000 7f39ceffd640 1 -- 172.21.15.89:0/3701156462 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39d0010690 con 0x7f39d8108300 2024-09-21T03:40:19.128 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:19.124+0000 7f39dfb39640 1 -- 172.21.15.89:0/3701156462 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39d8103270 con 0x7f39d8108300 2024-09-21T03:40:19.129 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:19.127+0000 7f39e6aa8740 1 Processor -- start 2024-09-21T03:40:19.130 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:19.128+0000 7f39ceffd640 1 -- 172.21.15.89:0/3701156462 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 20) ==== 154563+0+0 (secure 0 0 0) 0x7f39d0010830 con 0x7f39d8108300 2024-09-21T03:40:19.130 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:19.128+0000 7f39e6aa8740 1 -- start start 2024-09-21T03:40:19.131 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:19.128+0000 7f39e6aa8740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5591053f41b0 0x55910579be50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:19.131 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:19.128+0000 7f39e6aa8740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5591053cb2f0 0x55910579c4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:19.131 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:19.128+0000 7f39e6aa8740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5591053d34d0 0x55910579d010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:19.131 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:19.129+0000 7f39e6aa8740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55910570e6f0 con 0x5591053f41b0 2024-09-21T03:40:19.131 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:19.129+0000 7f39e6aa8740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x5591056fdb20 con 0x5591053d34d0 2024-09-21T03:40:19.131 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:19.129+0000 7f39e6aa8740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x559105701930 con 0x5591053cb2f0 2024-09-21T03:40:19.132 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:19.129+0000 7f39dd8d5640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5591053cb2f0 0x55910579c4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:19.132 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:19.129+0000 7f39dd8d5640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5591053cb2f0 0x55910579c4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.89:34956/0 (socket says 172.21.15.89:34956) 2024-09-21T03:40:19.132 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:19.129+0000 7f39dd0d4640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5591053f41b0 0x55910579be50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:19.132 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:19.129+0000 7f39dd8d5640 1 -- 172.21.15.89:0/3330593841 learned_addr learned my addr 172.21.15.89:0/3330593841 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-21T03:40:19.132 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:19.129+0000 7f39dd8d5640 1 -- 172.21.15.89:0/3330593841 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5591053d34d0 msgr2=0x55910579d010 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T03:40:19.132 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:19.129+0000 7f39dd8d5640 1 --2- 172.21.15.89:0/3330593841 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5591053d34d0 0x55910579d010 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:19.132 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:19.129+0000 7f39dd8d5640 1 -- 172.21.15.89:0/3330593841 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5591053f41b0 msgr2=0x55910579be50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:40:19.134 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:19.129+0000 7f39dd8d5640 1 --2- 172.21.15.89:0/3330593841 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5591053f41b0 0x55910579be50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:19.134 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:19.129+0000 7f39dd8d5640 1 -- 172.21.15.89:0/3330593841 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5591056c3530 con 0x5591053cb2f0 2024-09-21T03:40:19.134 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:19.129+0000 7f39dd0d4640 1 --2- 172.21.15.89:0/3330593841 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5591053f41b0 0x55910579be50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:40:19.134 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:19.130+0000 7f39ceffd640 1 --2- 172.21.15.89:0/3701156462 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f39c40b6620 0x7f39c40b8b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:19.135 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:19.130+0000 7f39dd8d5640 1 --2- 172.21.15.89:0/3330593841 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5591053cb2f0 0x55910579c4c0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f39d4009d50 tx=0x7f39d40117b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ee9b82920cc83b17 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:19.135 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:19.130+0000 7f39dd0d4640 1 --2- 172.21.15.89:0/3701156462 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f39c40b6620 0x7f39c40b8b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:19.135 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:19.130+0000 7f39c3fff640 1 -- 172.21.15.89:0/3330593841 <== mon.1 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39d4012650 con 0x5591053cb2f0 2024-09-21T03:40:19.135 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:19.130+0000 7f39c3fff640 1 -- 172.21.15.89:0/3330593841 <== mon.1 v2:172.21.15.114:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f39d4009e40 con 0x5591053cb2f0 2024-09-21T03:40:19.135 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:19.130+0000 7f39c3fff640 1 -- 172.21.15.89:0/3330593841 <== mon.1 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39d400fae0 con 0x5591053cb2f0 2024-09-21T03:40:19.135 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:19.131+0000 7f39ceffd640 1 -- 172.21.15.89:0/3701156462 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 9121+0+0 (secure 0 0 0) 0x7f39d00e7e00 con 0x7f39d8108300 2024-09-21T03:40:19.135 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:19.131+0000 7f39e6aa8740 1 -- 172.21.15.89:0/3330593841 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5591053cb2f0 msgr2=0x55910579c4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:19.135 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:19.131+0000 7f39e6aa8740 1 --2- 172.21.15.89:0/3330593841 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5591053cb2f0 0x55910579c4c0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f39d4009d50 tx=0x7f39d40117b0 comp rx=0 tx=0).stop 2024-09-21T03:40:19.135 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:19.131+0000 7f39dd0d4640 1 --2- 172.21.15.89:0/3701156462 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f39c40b6620 0x7f39c40b8b30 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f39c80057c0 tx=0x7f39c8000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:19.135 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:19.131+0000 7f39e6aa8740 1 -- 172.21.15.89:0/3330593841 shutdown_connections 2024-09-21T03:40:19.135 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:19.131+0000 7f39e6aa8740 1 --2- 172.21.15.89:0/3330593841 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5591053d34d0 0x55910579d010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:19.135 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:19.131+0000 7f39e6aa8740 1 --2- 172.21.15.89:0/3330593841 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5591053cb2f0 0x55910579c4c0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:19.135 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:19.131+0000 7f39e6aa8740 1 --2- 172.21.15.89:0/3330593841 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5591053f41b0 0x55910579be50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:19.135 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:19.131+0000 7f39e6aa8740 1 -- 172.21.15.89:0/3330593841 >> 172.21.15.89:0/3330593841 conn(0x5591053e8300 msgr2=0x5591057978e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:40:19.136 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:19.132+0000 7f39e6aa8740 1 -- 172.21.15.89:0/3330593841 shutdown_connections 2024-09-21T03:40:19.136 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:19.132+0000 7f39e6aa8740 1 -- 172.21.15.89:0/3330593841 wait complete. 2024-09-21T03:40:19.136 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:19.132+0000 7f39e6aa8740 1 -- start start 2024-09-21T03:40:19.136 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:19.132+0000 7f39e6aa8740 1 Processor -- start 2024-09-21T03:40:19.138 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:19.133+0000 7f39e6aa8740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5591053cb2f0 0x5591057a55d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:19.138 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:19.133+0000 7f39e6aa8740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5591053d34d0 0x5591057a5c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:19.138 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:19.133+0000 7f39e6aa8740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5591053f41b0 0x5591057a6800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:19.138 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:19.133+0000 7f39e6aa8740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x559105635ab0 con 0x5591053d34d0 2024-09-21T03:40:19.138 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:19.133+0000 7f39e6aa8740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x5591056a3730 con 0x5591053cb2f0 2024-09-21T03:40:19.139 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:19.133+0000 7f39e6aa8740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55910579db30 con 0x5591053f41b0 2024-09-21T03:40:19.139 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:19.133+0000 7f39dd0d4640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5591053d34d0 0x5591057a5c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:19.139 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:19.133+0000 7f39dd0d4640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5591053d34d0 0x5591057a5c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:34054/0 (socket says 172.21.15.89:34054) 2024-09-21T03:40:19.139 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:19.133+0000 7f39dd0d4640 1 -- 172.21.15.89:0/611508198 learned_addr learned my addr 172.21.15.89:0/611508198 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-21T03:40:19.139 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:19.133+0000 7f39dd8d5640 1 --2- 172.21.15.89:0/611508198 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5591053cb2f0 0x5591057a55d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:19.139 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:19.133+0000 7f39de0d6640 1 --2- 172.21.15.89:0/611508198 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5591053f41b0 0x5591057a6800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:19.139 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:19.133+0000 7f39e6aa8740 10 client.0 _ll_register_callbacks cb 0x5591056f76b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:40:19.139 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:19.133+0000 7f39dd0d4640 1 -- 172.21.15.89:0/611508198 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5591053f41b0 msgr2=0x5591057a6800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:40:19.139 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:19.134+0000 7f39dd0d4640 1 --2- 172.21.15.89:0/611508198 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5591053f41b0 0x5591057a6800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:19.139 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:19.134+0000 7f39dd0d4640 1 -- 172.21.15.89:0/611508198 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5591053cb2f0 msgr2=0x5591057a55d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:40:19.139 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:19.134+0000 7f39dd0d4640 1 --2- 172.21.15.89:0/611508198 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5591053cb2f0 0x5591057a55d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:19.139 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:19.134+0000 7f39dd0d4640 1 -- 172.21.15.89:0/611508198 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55910545f790 con 0x5591053d34d0 2024-09-21T03:40:19.139 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:19.134+0000 7f39e6aa8740 1 -- 172.21.15.89:0/611508198 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x559105710640 con 0x5591053d34d0 2024-09-21T03:40:19.139 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:19.134+0000 7f39de0d6640 1 --2- 172.21.15.89:0/611508198 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5591053f41b0 0x5591057a6800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:40:19.140 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:19.134+0000 7f39dd0d4640 1 --2- 172.21.15.89:0/611508198 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5591053d34d0 0x5591057a5c90 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f39c8009900 tx=0x7f39c802a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fcff2c22b55c80c4 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:19.140 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:19.134+0000 7f39c17fa640 10 client.15645 ms_handle_connect on v2:172.21.15.89:3300/0 2024-09-21T03:40:19.140 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:19.134+0000 7f39c17fa640 1 -- 172.21.15.89:0/611508198 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39c8035040 con 0x5591053d34d0 2024-09-21T03:40:19.140 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:19.134+0000 7f39c17fa640 1 -- 172.21.15.89:0/611508198 <== mon.0 v2:172.21.15.89:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 9121+0+0 (secure 0 0 0) 0x7f39c8004240 con 0x5591053d34d0 2024-09-21T03:40:19.140 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:19.134+0000 7f39c17fa640 1 -- 172.21.15.89:0/611508198 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f39c8004440 con 0x5591053d34d0 2024-09-21T03:40:19.140 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:19.134+0000 7f39e6aa8740 10 client.15645 fetch_fsmap learned FSMap version 26 2024-09-21T03:40:19.140 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:19.134+0000 7f39e6aa8740 1 -- 172.21.15.89:0/611508198 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({fsmap=26}) -- 0x5591054d8620 con 0x5591053d34d0 2024-09-21T03:40:19.140 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:19.135+0000 7f39c17fa640 1 -- 172.21.15.89:0/611508198 <== mon.0 v2:172.21.15.89:3300/0 4 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f39c805b060 con 0x5591053d34d0 2024-09-21T03:40:19.140 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:19.135+0000 7f39e6aa8740 10 client.15645 fetch_fsmap finished waiting for FSMap version 26 2024-09-21T03:40:19.140 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:19.135+0000 7f39e6aa8740 10 client.15645 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T03:40:19.140 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:19.135+0000 7f39e6aa8740 20 client.15645 populate_metadata read hostname 'smithi089' 2024-09-21T03:40:19.140 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:19.135+0000 7f39e6aa8740 1 --2- 172.21.15.89:0/611508198 >> [v2:172.21.15.110:6838/132366709,v1:172.21.15.110:6839/132366709] conn(0x5591053b2d80 0x5591057b2bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:19.140 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:19.135+0000 7f39e6aa8740 4 client.15645 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:40:19.140 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:19.135+0000 7f39e6aa8740 1 -- 172.21.15.89:0/611508198 --> [v2:172.21.15.110:6838/132366709,v1:172.21.15.110:6839/132366709] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55910570c0f0 con 0x5591053b2d80 2024-09-21T03:40:19.141 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:19.135+0000 7f39dd8d5640 1 --2- 172.21.15.89:0/611508198 >> [v2:172.21.15.110:6838/132366709,v1:172.21.15.110:6839/132366709] conn(0x5591053b2d80 0x5591057b2bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:19.141 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:19.136+0000 7f39c17fa640 10 client.15645 ms_handle_connect on v2:172.21.15.110:6838/132366709 2024-09-21T03:40:19.141 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:19.136+0000 7f39dd8d5640 1 --2- 172.21.15.89:0/611508198 >> [v2:172.21.15.110:6838/132366709,v1:172.21.15.110:6839/132366709] conn(0x5591053b2d80 0x5591057b2bb0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6c7ed50a43af6a6c server_cookie=6edaef8093d9b699 in_seq=0 out_seq=0 2024-09-21T03:40:19.143 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:19.136+0000 7f39c17fa640 1 -- 172.21.15.89:0/611508198 <== mds.0 v2:172.21.15.110:6838/132366709 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55910570c0f0 con 0x5591053b2d80 2024-09-21T03:40:19.143 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:19.136+0000 7f39c17fa640 10 client.15645 handle_command_reply: tid=0 2024-09-21T03:40:19.143 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:19.136+0000 7f39e6aa8740 1 client.15645 shutdown 2024-09-21T03:40:19.143 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:19.137+0000 7f39e6aa8740 1 -- 172.21.15.89:0/611508198 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5591053d34d0 msgr2=0x5591057a5c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:19.143 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:19.137+0000 7f39e6aa8740 1 --2- 172.21.15.89:0/611508198 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5591053d34d0 0x5591057a5c90 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f39c8009900 tx=0x7f39c802a410 comp rx=0 tx=0).stop 2024-09-21T03:40:19.143 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:19.137+0000 7f39e6aa8740 1 -- 172.21.15.89:0/611508198 shutdown_connections 2024-09-21T03:40:19.143 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:19.137+0000 7f39e6aa8740 1 --2- 172.21.15.89:0/611508198 >> [v2:172.21.15.110:6838/132366709,v1:172.21.15.110:6839/132366709] conn(0x5591053b2d80 0x5591057b2bb0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:19.143 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:19.137+0000 7f39e6aa8740 1 --2- 172.21.15.89:0/611508198 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5591053f41b0 0x5591057a6800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:19.143 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:19.137+0000 7f39c17fa640 0 client.15645 ms_handle_reset on v2:172.21.15.110:6838/132366709 2024-09-21T03:40:19.143 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:19.137+0000 7f39e6aa8740 1 --2- 172.21.15.89:0/611508198 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5591053d34d0 0x5591057a5c90 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:19.143 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:19.137+0000 7f39e6aa8740 1 --2- 172.21.15.89:0/611508198 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5591053cb2f0 0x5591057a55d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:19.143 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:19.137+0000 7f39e6aa8740 1 -- 172.21.15.89:0/611508198 >> 172.21.15.89:0/611508198 conn(0x5591053e8300 msgr2=0x559105795ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:40:19.144 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:19.137+0000 7f39e6aa8740 1 -- 172.21.15.89:0/611508198 shutdown_connections 2024-09-21T03:40:19.144 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:19.137+0000 7f39e6aa8740 1 -- 172.21.15.89:0/611508198 wait complete. 2024-09-21T03:40:19.144 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:19.137+0000 7f39e6aa8740 20 client.15645 trim_cache size 0 max 16384 2024-09-21T03:40:19.168 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:19.166+0000 7f39e6aa8740 1 Processor -- start 2024-09-21T03:40:19.168 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:19.166+0000 7f39e6aa8740 1 -- start start 2024-09-21T03:40:19.169 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:19.166+0000 7f39e6aa8740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5591053cb2f0 0x559105797290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:19.169 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:19.166+0000 7f39e6aa8740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5591053d34d0 0x5591057a47d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:19.169 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:19.166+0000 7f39e6aa8740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5591053f41b0 0x55910579cc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:19.169 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:19.166+0000 7f39e6aa8740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f39d81b0fd0 con 0x5591053f41b0 2024-09-21T03:40:19.169 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:19.166+0000 7f39e6aa8740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f39d81ae8a0 con 0x5591053d34d0 2024-09-21T03:40:19.169 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:19.166+0000 7f39e6aa8740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f39d81ac740 con 0x5591053cb2f0 2024-09-21T03:40:19.169 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:19.166+0000 7f39de0d6640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5591053f41b0 0x55910579cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:19.169 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:19.166+0000 7f39dd8d5640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5591053cb2f0 0x559105797290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:19.169 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:19.166+0000 7f39de0d6640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5591053f41b0 0x55910579cc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:34068/0 (socket says 172.21.15.89:34068) 2024-09-21T03:40:19.169 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:19.166+0000 7f39de0d6640 1 -- 172.21.15.89:0/3098634595 learned_addr learned my addr 172.21.15.89:0/3098634595 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-21T03:40:19.169 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:19.166+0000 7f39dd0d4640 1 --2- 172.21.15.89:0/3098634595 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5591053d34d0 0x5591057a47d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:19.171 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:19.167+0000 7f39dd8d5640 1 -- 172.21.15.89:0/3098634595 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5591053d34d0 msgr2=0x5591057a47d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:40:19.171 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:19.167+0000 7f39dd8d5640 1 --2- 172.21.15.89:0/3098634595 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5591053d34d0 0x5591057a47d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:19.171 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:19.167+0000 7f39dd8d5640 1 -- 172.21.15.89:0/3098634595 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5591053f41b0 msgr2=0x55910579cc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:40:19.171 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:19.167+0000 7f39dd8d5640 1 --2- 172.21.15.89:0/3098634595 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5591053f41b0 0x55910579cc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:19.171 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:19.167+0000 7f39dd8d5640 1 -- 172.21.15.89:0/3098634595 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559105797010 con 0x5591053cb2f0 2024-09-21T03:40:19.171 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:19.167+0000 7f39dd0d4640 1 --2- 172.21.15.89:0/3098634595 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5591053d34d0 0x5591057a47d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:40:19.171 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:19.167+0000 7f39dd8d5640 1 --2- 172.21.15.89:0/3098634595 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5591053cb2f0 0x559105797290 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f39d4009d20 tx=0x7f39d4009530 comp rx=0 tx=0).ready entity=mon.1 client_cookie=788fa4bedad5201f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:19.172 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:19.167+0000 7f39c37fe640 1 -- 172.21.15.89:0/3098634595 <== mon.1 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39d4035070 con 0x5591053cb2f0 2024-09-21T03:40:19.172 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:19.167+0000 7f39c37fe640 1 -- 172.21.15.89:0/3098634595 <== mon.1 v2:172.21.15.114:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f39d40035f0 con 0x5591053cb2f0 2024-09-21T03:40:19.172 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:19.167+0000 7f39c37fe640 1 -- 172.21.15.89:0/3098634595 <== mon.1 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39d40126b0 con 0x5591053cb2f0 2024-09-21T03:40:19.172 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:19.168+0000 7f39e6aa8740 1 -- 172.21.15.89:0/3098634595 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5591053cb2f0 msgr2=0x559105797290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:19.172 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:19.168+0000 7f39e6aa8740 1 --2- 172.21.15.89:0/3098634595 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5591053cb2f0 0x559105797290 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f39d4009d20 tx=0x7f39d4009530 comp rx=0 tx=0).stop 2024-09-21T03:40:19.172 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:19.169+0000 7f39e6aa8740 1 -- 172.21.15.89:0/3098634595 shutdown_connections 2024-09-21T03:40:19.172 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:19.169+0000 7f39e6aa8740 1 --2- 172.21.15.89:0/3098634595 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5591053d34d0 0x5591057a47d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:19.172 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:19.169+0000 7f39e6aa8740 1 --2- 172.21.15.89:0/3098634595 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5591053f41b0 0x55910579cc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:19.172 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:19.169+0000 7f39e6aa8740 1 --2- 172.21.15.89:0/3098634595 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5591053cb2f0 0x559105797290 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:19.172 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:19.169+0000 7f39e6aa8740 1 -- 172.21.15.89:0/3098634595 >> 172.21.15.89:0/3098634595 conn(0x5591053e8300 msgr2=0x5591057a3fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:40:19.172 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:19.169+0000 7f39e6aa8740 1 -- 172.21.15.89:0/3098634595 shutdown_connections 2024-09-21T03:40:19.172 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:19.169+0000 7f39e6aa8740 1 -- 172.21.15.89:0/3098634595 wait complete. 2024-09-21T03:40:19.172 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:19.169+0000 7f39e6aa8740 1 -- start start 2024-09-21T03:40:19.172 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:19.169+0000 7f39e6aa8740 1 Processor -- start 2024-09-21T03:40:19.175 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:19.170+0000 7f39e6aa8740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5591053cb2f0 0x5591057ac320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:19.175 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:19.170+0000 7f39e6aa8740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5591053d34d0 0x5591057ac860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:19.175 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:19.170+0000 7f39e6aa8740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5591053f41b0 0x5591057acda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:19.176 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:19.170+0000 7f39e6aa8740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f39d81acdb0 con 0x5591053f41b0 2024-09-21T03:40:19.176 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:19.170+0000 7f39de0d6640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5591053f41b0 0x5591057acda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:19.176 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:19.170+0000 7f39de0d6640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5591053f41b0 0x5591057acda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:34078/0 (socket says 172.21.15.89:34078) 2024-09-21T03:40:19.176 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:19.170+0000 7f39de0d6640 1 -- 172.21.15.89:0/494810388 learned_addr learned my addr 172.21.15.89:0/494810388 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-21T03:40:19.176 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:19.170+0000 7f39e6aa8740 1 -- 172.21.15.89:0/494810388 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f39d81ae1d0 con 0x5591053d34d0 2024-09-21T03:40:19.176 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:19.170+0000 7f39e6aa8740 1 -- 172.21.15.89:0/494810388 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f39d81ad430 con 0x5591053cb2f0 2024-09-21T03:40:19.176 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:19.170+0000 7f39dd8d5640 1 --2- 172.21.15.89:0/494810388 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5591053cb2f0 0x5591057ac320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:19.176 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:19.170+0000 7f39e6aa8740 10 client.0 _ll_register_callbacks cb 0x7f39c8009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:40:19.176 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:19.170+0000 7f39dd0d4640 1 --2- 172.21.15.89:0/494810388 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5591053d34d0 0x5591057ac860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:19.176 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:19.170+0000 7f39de0d6640 1 -- 172.21.15.89:0/494810388 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5591053cb2f0 msgr2=0x5591057ac320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:40:19.176 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:19.170+0000 7f39de0d6640 1 --2- 172.21.15.89:0/494810388 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5591053cb2f0 0x5591057ac320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:19.176 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:19.170+0000 7f39de0d6640 1 -- 172.21.15.89:0/494810388 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5591053d34d0 msgr2=0x5591057ac860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:40:19.176 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:19.170+0000 7f39de0d6640 1 --2- 172.21.15.89:0/494810388 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5591053d34d0 0x5591057ac860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:19.176 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:19.170+0000 7f39de0d6640 1 -- 172.21.15.89:0/494810388 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5591057aafd0 con 0x5591053f41b0 2024-09-21T03:40:19.176 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:19.171+0000 7f39e6aa8740 1 -- 172.21.15.89:0/494810388 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f39d40035f0 con 0x5591053f41b0 2024-09-21T03:40:19.177 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:19.171+0000 7f39de0d6640 1 --2- 172.21.15.89:0/494810388 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5591053f41b0 0x5591057acda0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f39d00950f0 tx=0x7f39d00b73b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=377dcc2de9d94b84 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:19.177 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:19.171+0000 7f39c17fa640 10 client.15651 ms_handle_connect on v2:172.21.15.89:3300/0 2024-09-21T03:40:19.177 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:19.171+0000 7f39c17fa640 1 -- 172.21.15.89:0/494810388 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39d00a9220 con 0x5591053f41b0 2024-09-21T03:40:19.177 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:19.171+0000 7f39c17fa640 1 -- 172.21.15.89:0/494810388 <== mon.0 v2:172.21.15.89:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 9121+0+0 (secure 0 0 0) 0x7f39d0010b20 con 0x5591053f41b0 2024-09-21T03:40:19.177 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:19.171+0000 7f39c17fa640 1 -- 172.21.15.89:0/494810388 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f39d0010d80 con 0x5591053f41b0 2024-09-21T03:40:19.177 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:19.171+0000 7f39e6aa8740 10 client.15651 fetch_fsmap learned FSMap version 26 2024-09-21T03:40:19.177 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:19.171+0000 7f39e6aa8740 1 -- 172.21.15.89:0/494810388 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({fsmap=26}) -- 0x55910581d890 con 0x5591053f41b0 2024-09-21T03:40:19.177 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:19.171+0000 7f39c17fa640 1 -- 172.21.15.89:0/494810388 <== mon.0 v2:172.21.15.89:3300/0 4 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f39d00a9400 con 0x5591053f41b0 2024-09-21T03:40:19.177 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:19.171+0000 7f39e6aa8740 10 client.15651 fetch_fsmap finished waiting for FSMap version 26 2024-09-21T03:40:19.177 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:19.171+0000 7f39e6aa8740 10 client.15651 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T03:40:19.177 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:19.171+0000 7f39e6aa8740 20 client.15651 populate_metadata read hostname 'smithi089' 2024-09-21T03:40:19.177 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:19.171+0000 7f39e6aa8740 1 --2- 172.21.15.89:0/494810388 >> [v2:172.21.15.110:6838/132366709,v1:172.21.15.110:6839/132366709] conn(0x5591053881c0 0x559105820190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:19.177 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:19.171+0000 7f39e6aa8740 4 client.15651 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T03:40:19.177 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:19.171+0000 7f39e6aa8740 1 -- 172.21.15.89:0/494810388 --> [v2:172.21.15.110:6838/132366709,v1:172.21.15.110:6839/132366709] -- command(tid 0: {"prefix": "flush journal"}) -- 0x559105709520 con 0x5591053881c0 2024-09-21T03:40:19.178 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:19.172+0000 7f39dd8d5640 1 --2- 172.21.15.89:0/494810388 >> [v2:172.21.15.110:6838/132366709,v1:172.21.15.110:6839/132366709] conn(0x5591053881c0 0x559105820190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:19.178 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:19.172+0000 7f39c17fa640 10 client.15651 ms_handle_connect on v2:172.21.15.110:6838/132366709 2024-09-21T03:40:19.178 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:19.172+0000 7f39dd8d5640 1 --2- 172.21.15.89:0/494810388 >> [v2:172.21.15.110:6838/132366709,v1:172.21.15.110:6839/132366709] conn(0x5591053881c0 0x559105820190 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=63745f004c422442 server_cookie=fae9505aeb939ed0 in_seq=0 out_seq=0 2024-09-21T03:40:19.181 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:19.178+0000 7f39c17fa640 1 -- 172.21.15.89:0/494810388 <== mds.0 v2:172.21.15.110:6838/132366709 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x559105709520 con 0x5591053881c0 2024-09-21T03:40:19.181 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:19.178+0000 7f39c17fa640 10 client.15651 handle_command_reply: tid=0 2024-09-21T03:40:19.181 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:19.178+0000 7f39e6aa8740 1 client.15651 shutdown 2024-09-21T03:40:19.181 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:19.178+0000 7f39e6aa8740 1 -- 172.21.15.89:0/494810388 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5591053f41b0 msgr2=0x5591057acda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:19.181 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:19.178+0000 7f39e6aa8740 1 --2- 172.21.15.89:0/494810388 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5591053f41b0 0x5591057acda0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f39d00950f0 tx=0x7f39d00b73b0 comp rx=0 tx=0).stop 2024-09-21T03:40:19.181 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:19.179+0000 7f39e6aa8740 1 -- 172.21.15.89:0/494810388 shutdown_connections 2024-09-21T03:40:19.181 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:19.179+0000 7f39e6aa8740 1 --2- 172.21.15.89:0/494810388 >> [v2:172.21.15.110:6838/132366709,v1:172.21.15.110:6839/132366709] conn(0x5591053881c0 0x559105820190 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:19.181 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:19.179+0000 7f39e6aa8740 1 --2- 172.21.15.89:0/494810388 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5591053d34d0 0x5591057ac860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:19.182 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:19.179+0000 7f39c17fa640 0 client.15651 ms_handle_reset on v2:172.21.15.110:6838/132366709 2024-09-21T03:40:19.182 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:19.179+0000 7f39e6aa8740 1 --2- 172.21.15.89:0/494810388 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5591053f41b0 0x5591057acda0 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:19.182 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:19.179+0000 7f39e6aa8740 1 --2- 172.21.15.89:0/494810388 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5591053cb2f0 0x5591057ac320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:19.182 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:19.179+0000 7f39e6aa8740 1 -- 172.21.15.89:0/494810388 >> 172.21.15.89:0/494810388 conn(0x5591053e8300 msgr2=0x559105797220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:40:19.182 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:19.179+0000 7f39e6aa8740 1 -- 172.21.15.89:0/494810388 shutdown_connections 2024-09-21T03:40:19.182 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:19.179+0000 7f39e6aa8740 1 -- 172.21.15.89:0/494810388 wait complete. 2024-09-21T03:40:19.182 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:19.179+0000 7f39e6aa8740 20 client.15651 trim_cache size 0 max 16384 2024-09-21T03:40:19.182 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stdout:{ 2024-09-21T03:40:19.182 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stdout: "message": "", 2024-09-21T03:40:19.183 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stdout: "return_code": 0 2024-09-21T03:40:19.183 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stdout:} 2024-09-21T03:40:19.183 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:19.180+0000 7f39dfb39640 1 -- 172.21.15.89:0/3701156462 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f39c40b6620 msgr2=0x7f39c40b8b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:19.183 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:19.180+0000 7f39dfb39640 1 --2- 172.21.15.89:0/3701156462 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f39c40b6620 0x7f39c40b8b30 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f39c80057c0 tx=0x7f39c8000bc0 comp rx=0 tx=0).stop 2024-09-21T03:40:19.183 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:19.180+0000 7f39dfb39640 1 -- 172.21.15.89:0/3701156462 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f39d8108300 msgr2=0x7f39d819b0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:19.184 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:19.180+0000 7f39dfb39640 1 --2- 172.21.15.89:0/3701156462 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f39d8108300 0x7f39d819b0c0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f39d000eb10 tx=0x7f39d000c2d0 comp rx=0 tx=0).stop 2024-09-21T03:40:19.184 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:19.180+0000 7f39dfb39640 1 -- 172.21.15.89:0/3701156462 shutdown_connections 2024-09-21T03:40:19.184 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:19.180+0000 7f39dfb39640 1 --2- 172.21.15.89:0/3701156462 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f39c40b6620 0x7f39c40b8b30 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:19.184 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:19.180+0000 7f39dfb39640 1 --2- 172.21.15.89:0/3701156462 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f39d81056e0 0x7f39d819a510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:19.184 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:19.180+0000 7f39dfb39640 1 --2- 172.21.15.89:0/3701156462 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f39d8108300 0x7f39d819b0c0 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:19.184 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:19.180+0000 7f39dfb39640 1 --2- 172.21.15.89:0/3701156462 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f39d8102c80 0x7f39d8199ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:19.184 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:19.180+0000 7f39dfb39640 1 -- 172.21.15.89:0/3701156462 >> 172.21.15.89:0/3701156462 conn(0x7f39d80fd690 msgr2=0x7f39d8107940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:40:19.184 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:19.180+0000 7f39dfb39640 1 -- 172.21.15.89:0/3701156462 shutdown_connections 2024-09-21T03:40:19.184 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:19.180+0000 7f39dfb39640 1 -- 172.21.15.89:0/3701156462 wait complete. 2024-09-21T03:40:19.711 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:40:19 smithi114 ceph-mon[34140]: pgmap v387: 129 pgs: 129 active+clean; 675 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 170 B/s wr, 5 op/s 2024-09-21T03:40:19.750 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:40:19 smithi110 ceph-mon[34000]: pgmap v387: 129 pgs: 129 active+clean; 675 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 170 B/s wr, 5 op/s 2024-09-21T03:40:19.811 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:40:19 smithi089 ceph-mon[30928]: pgmap v387: 129 pgs: 129 active+clean; 675 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 170 B/s wr, 5 op/s 2024-09-21T03:40:21.711 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:40:21 smithi114 ceph-mon[34140]: pgmap v388: 129 pgs: 129 active+clean; 675 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 341 B/s wr, 7 op/s 2024-09-21T03:40:21.750 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:40:21 smithi110 ceph-mon[34000]: pgmap v388: 129 pgs: 129 active+clean; 675 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 341 B/s wr, 7 op/s 2024-09-21T03:40:21.811 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:40:21 smithi089 ceph-mon[30928]: pgmap v388: 129 pgs: 129 active+clean; 675 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 341 B/s wr, 7 op/s 2024-09-21T03:40:23.711 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:40:23 smithi114 ceph-mon[34140]: pgmap v389: 129 pgs: 129 active+clean; 675 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 170 B/s wr, 5 op/s 2024-09-21T03:40:23.750 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:40:23 smithi110 ceph-mon[34000]: pgmap v389: 129 pgs: 129 active+clean; 675 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 170 B/s wr, 5 op/s 2024-09-21T03:40:23.811 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:40:23 smithi089 ceph-mon[30928]: pgmap v389: 129 pgs: 129 active+clean; 675 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 170 B/s wr, 5 op/s 2024-09-21T03:40:25.712 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:40:25 smithi114 ceph-mon[34140]: pgmap v390: 129 pgs: 129 active+clean; 675 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 170 B/s wr, 6 op/s 2024-09-21T03:40:25.750 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:40:25 smithi110 ceph-mon[34000]: pgmap v390: 129 pgs: 129 active+clean; 675 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 170 B/s wr, 6 op/s 2024-09-21T03:40:25.811 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:40:25 smithi089 ceph-mon[30928]: pgmap v390: 129 pgs: 129 active+clean; 675 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 170 B/s wr, 6 op/s 2024-09-21T03:40:27.711 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:40:27 smithi114 ceph-mon[34140]: pgmap v391: 129 pgs: 129 active+clean; 675 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 170 B/s wr, 6 op/s 2024-09-21T03:40:27.750 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:40:27 smithi110 ceph-mon[34000]: pgmap v391: 129 pgs: 129 active+clean; 675 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 170 B/s wr, 6 op/s 2024-09-21T03:40:27.811 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:40:27 smithi089 ceph-mon[30928]: pgmap v391: 129 pgs: 129 active+clean; 675 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 170 B/s wr, 6 op/s 2024-09-21T03:40:29.711 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:40:29 smithi114 ceph-mon[34140]: pgmap v392: 129 pgs: 129 active+clean; 675 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 170 B/s wr, 5 op/s 2024-09-21T03:40:29.750 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:40:29 smithi110 ceph-mon[34000]: pgmap v392: 129 pgs: 129 active+clean; 675 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 170 B/s wr, 5 op/s 2024-09-21T03:40:29.811 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:40:29 smithi089 ceph-mon[30928]: pgmap v392: 129 pgs: 129 active+clean; 675 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 170 B/s wr, 5 op/s 2024-09-21T03:40:31.711 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:40:31 smithi114 ceph-mon[34140]: pgmap v393: 129 pgs: 129 active+clean; 675 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 170 B/s wr, 7 op/s 2024-09-21T03:40:31.750 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:40:31 smithi110 ceph-mon[34000]: pgmap v393: 129 pgs: 129 active+clean; 675 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 170 B/s wr, 7 op/s 2024-09-21T03:40:31.811 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:40:31 smithi089 ceph-mon[30928]: pgmap v393: 129 pgs: 129 active+clean; 675 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 170 B/s wr, 7 op/s 2024-09-21T03:40:32.333 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:32.331+0000 7f4e333a1640 1 -- 172.21.15.89:0/1629696041 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4e2c108300 msgr2=0x7f4e2c10a810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:32.333 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:32.331+0000 7f4e333a1640 1 --2- 172.21.15.89:0/1629696041 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4e2c108300 0x7f4e2c10a810 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f4e24009ca0 tx=0x7f4e2402a8c0 comp rx=0 tx=0).stop 2024-09-21T03:40:32.333 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:32.331+0000 7f4e333a1640 1 -- 172.21.15.89:0/1629696041 shutdown_connections 2024-09-21T03:40:32.333 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:32.331+0000 7f4e333a1640 1 --2- 172.21.15.89:0/1629696041 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4e2c108300 0x7f4e2c10a810 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:32.333 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:32.331+0000 7f4e333a1640 1 --2- 172.21.15.89:0/1629696041 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4e2c1056e0 0x7f4e2c107bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:32.333 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:32.331+0000 7f4e333a1640 1 --2- 172.21.15.89:0/1629696041 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4e2c102c60 0x7f4e2c1050a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:32.333 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:32.331+0000 7f4e333a1640 1 -- 172.21.15.89:0/1629696041 >> 172.21.15.89:0/1629696041 conn(0x7f4e2c0fd690 msgr2=0x7f4e2c0ffb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:40:32.333 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:32.332+0000 7f4e333a1640 1 -- 172.21.15.89:0/1629696041 shutdown_connections 2024-09-21T03:40:32.333 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:32.332+0000 7f4e333a1640 1 -- 172.21.15.89:0/1629696041 wait complete. 2024-09-21T03:40:32.335 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:32.332+0000 7f4e333a1640 1 Processor -- start 2024-09-21T03:40:32.335 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:32.333+0000 7f4e333a1640 1 -- start start 2024-09-21T03:40:32.335 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:32.333+0000 7f4e333a1640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4e2c102c60 0x7f4e2c102100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:32.335 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:32.333+0000 7f4e333a1640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4e2c1056e0 0x7f4e2c19e9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:32.335 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:32.333+0000 7f4e333a1640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4e2c108300 0x7f4e2c1a0f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:32.335 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:32.333+0000 7f4e333a1640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f4e2c10e110 con 0x7f4e2c1056e0 2024-09-21T03:40:32.335 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:32.333+0000 7f4e333a1640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f4e2c10df90 con 0x7f4e2c108300 2024-09-21T03:40:32.336 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:32.333+0000 7f4e333a1640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f4e2c10e290 con 0x7f4e2c102c60 2024-09-21T03:40:32.336 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:32.333+0000 7f4e3093c640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4e2c1056e0 0x7f4e2c19e9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:32.336 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:32.333+0000 7f4e3093c640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4e2c1056e0 0x7f4e2c19e9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:49696/0 (socket says 172.21.15.89:49696) 2024-09-21T03:40:32.336 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:32.333+0000 7f4e3093c640 1 -- 172.21.15.89:0/1205383807 learned_addr learned my addr 172.21.15.89:0/1205383807 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-21T03:40:32.336 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:32.333+0000 7f4e3113d640 1 --2- 172.21.15.89:0/1205383807 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4e2c102c60 0x7f4e2c102100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:32.336 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:32.333+0000 7f4e3193e640 1 --2- 172.21.15.89:0/1205383807 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4e2c108300 0x7f4e2c1a0f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:32.336 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:32.333+0000 7f4e3093c640 1 -- 172.21.15.89:0/1205383807 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4e2c102c60 msgr2=0x7f4e2c102100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:32.336 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:32.333+0000 7f4e3093c640 1 --2- 172.21.15.89:0/1205383807 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4e2c102c60 0x7f4e2c102100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:32.336 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:32.333+0000 7f4e3093c640 1 -- 172.21.15.89:0/1205383807 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4e2c108300 msgr2=0x7f4e2c1a0f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:32.336 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:32.333+0000 7f4e3093c640 1 --2- 172.21.15.89:0/1205383807 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4e2c108300 0x7f4e2c1a0f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:32.336 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:32.333+0000 7f4e3093c640 1 -- 172.21.15.89:0/1205383807 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e2c1a1700 con 0x7f4e2c1056e0 2024-09-21T03:40:32.336 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:32.334+0000 7f4e3093c640 1 --2- 172.21.15.89:0/1205383807 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4e2c1056e0 0x7f4e2c19e9b0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f4e1c00d770 tx=0x7f4e1c00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:32.336 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:32.334+0000 7f4e227fc640 1 -- 172.21.15.89:0/1205383807 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e1c017070 con 0x7f4e2c1056e0 2024-09-21T03:40:32.336 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:32.334+0000 7f4e227fc640 1 -- 172.21.15.89:0/1205383807 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4e1c0054b0 con 0x7f4e2c1056e0 2024-09-21T03:40:32.336 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:32.334+0000 7f4e333a1640 1 -- 172.21.15.89:0/1205383807 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4e2c1a19f0 con 0x7f4e2c1056e0 2024-09-21T03:40:32.336 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:32.334+0000 7f4e227fc640 1 -- 172.21.15.89:0/1205383807 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e1c004c60 con 0x7f4e2c1056e0 2024-09-21T03:40:32.337 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:32.335+0000 7f4e333a1640 1 -- 172.21.15.89:0/1205383807 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4e2c1a1f40 con 0x7f4e2c1056e0 2024-09-21T03:40:32.340 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:32.339+0000 7f4e227fc640 1 -- 172.21.15.89:0/1205383807 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 20) ==== 154563+0+0 (secure 0 0 0) 0x7f4e1c004e00 con 0x7f4e2c1056e0 2024-09-21T03:40:32.340 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:32.339+0000 7f4e3a3ec740 1 Processor -- start 2024-09-21T03:40:32.340 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:32.339+0000 7f4e3a3ec740 1 -- start start 2024-09-21T03:40:32.341 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:32.339+0000 7f4e3a3ec740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ac1d2751b0 0x55ac1d61ce10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:32.341 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:32.339+0000 7f4e3a3ec740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ac1d24c2f0 0x55ac1d61d480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:32.342 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:32.339+0000 7f4e3a3ec740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ac1d2544d0 0x55ac1d61dfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:32.342 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:32.339+0000 7f4e3a3ec740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55ac1d58f6f0 con 0x55ac1d24c2f0 2024-09-21T03:40:32.342 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:32.339+0000 7f4e3a3ec740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55ac1d57eb20 con 0x55ac1d2544d0 2024-09-21T03:40:32.342 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:32.339+0000 7f4e3a3ec740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55ac1d582930 con 0x55ac1d2751b0 2024-09-21T03:40:32.342 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:32.340+0000 7f4e3113d640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ac1d24c2f0 0x55ac1d61d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:32.342 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:32.340+0000 7f4e3113d640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ac1d24c2f0 0x55ac1d61d480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:49704/0 (socket says 172.21.15.89:49704) 2024-09-21T03:40:32.342 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:32.340+0000 7f4e3113d640 1 -- 172.21.15.89:0/435601775 learned_addr learned my addr 172.21.15.89:0/435601775 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-21T03:40:32.342 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:32.340+0000 7f4e3093c640 1 --2- 172.21.15.89:0/435601775 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ac1d2544d0 0x55ac1d61dfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:32.342 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:32.340+0000 7f4e3193e640 1 --2- 172.21.15.89:0/435601775 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ac1d2751b0 0x55ac1d61ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:32.342 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:32.340+0000 7f4e227fc640 1 --2- 172.21.15.89:0/1205383807 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f4e180b6620 0x7f4e180b8b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:32.342 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:32.340+0000 7f4e3113d640 1 -- 172.21.15.89:0/435601775 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ac1d2751b0 msgr2=0x55ac1d61ce10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:40:32.342 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:32.340+0000 7f4e3113d640 1 --2- 172.21.15.89:0/435601775 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ac1d2751b0 0x55ac1d61ce10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:32.342 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:32.340+0000 7f4e3113d640 1 -- 172.21.15.89:0/435601775 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ac1d2544d0 msgr2=0x55ac1d61dfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:40:32.343 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:32.340+0000 7f4e3113d640 1 --2- 172.21.15.89:0/435601775 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ac1d2544d0 0x55ac1d61dfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:32.343 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:32.340+0000 7f4e3113d640 1 -- 172.21.15.89:0/435601775 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ac1d544530 con 0x55ac1d24c2f0 2024-09-21T03:40:32.343 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:32.340+0000 7f4e3193e640 1 --2- 172.21.15.89:0/435601775 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ac1d2751b0 0x55ac1d61ce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:40:32.343 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:32.340+0000 7f4e3093c640 1 --2- 172.21.15.89:0/435601775 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ac1d2544d0 0x55ac1d61dfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:40:32.343 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:32.340+0000 7f4e227fc640 1 -- 172.21.15.89:0/1205383807 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 9121+0+0 (secure 0 0 0) 0x7f4e1c0e7160 con 0x7f4e2c1056e0 2024-09-21T03:40:32.343 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:32.340+0000 7f4e3193e640 1 --2- 172.21.15.89:0/1205383807 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f4e180b6620 0x7f4e180b8b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:32.344 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:32.341+0000 7f4e3193e640 1 --2- 172.21.15.89:0/1205383807 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f4e180b6620 0x7f4e180b8b30 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f4e240011d0 tx=0x7f4e2403f040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:32.344 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:32.341+0000 7f4e3113d640 1 --2- 172.21.15.89:0/435601775 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ac1d24c2f0 0x55ac1d61d480 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f4e28009cf0 tx=0x7f4e28011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f144efbe049f508a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:32.344 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:32.341+0000 7f4e177fe640 1 -- 172.21.15.89:0/435601775 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e28004230 con 0x55ac1d24c2f0 2024-09-21T03:40:32.345 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:32.341+0000 7f4e177fe640 1 -- 172.21.15.89:0/435601775 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4e280043d0 con 0x55ac1d24c2f0 2024-09-21T03:40:32.345 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:32.341+0000 7f4e177fe640 1 -- 172.21.15.89:0/435601775 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e280166b0 con 0x55ac1d24c2f0 2024-09-21T03:40:32.345 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:32.343+0000 7f4e3a3ec740 1 -- 172.21.15.89:0/435601775 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ac1d24c2f0 msgr2=0x55ac1d61d480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:32.345 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:32.343+0000 7f4e3a3ec740 1 --2- 172.21.15.89:0/435601775 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ac1d24c2f0 0x55ac1d61d480 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f4e28009cf0 tx=0x7f4e28011810 comp rx=0 tx=0).stop 2024-09-21T03:40:32.345 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:32.343+0000 7f4e3a3ec740 1 -- 172.21.15.89:0/435601775 shutdown_connections 2024-09-21T03:40:32.345 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:32.343+0000 7f4e3a3ec740 1 --2- 172.21.15.89:0/435601775 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ac1d2544d0 0x55ac1d61dfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:32.345 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:32.343+0000 7f4e3a3ec740 1 --2- 172.21.15.89:0/435601775 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ac1d24c2f0 0x55ac1d61d480 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:32.345 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:32.343+0000 7f4e3a3ec740 1 --2- 172.21.15.89:0/435601775 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ac1d2751b0 0x55ac1d61ce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:32.345 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:32.343+0000 7f4e3a3ec740 1 -- 172.21.15.89:0/435601775 >> 172.21.15.89:0/435601775 conn(0x55ac1d269300 msgr2=0x55ac1d6188a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:40:32.345 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:32.343+0000 7f4e3a3ec740 1 -- 172.21.15.89:0/435601775 shutdown_connections 2024-09-21T03:40:32.345 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:32.343+0000 7f4e3a3ec740 1 -- 172.21.15.89:0/435601775 wait complete. 2024-09-21T03:40:32.347 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:32.344+0000 7f4e3a3ec740 1 -- start start 2024-09-21T03:40:32.347 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:32.344+0000 7f4e3a3ec740 1 Processor -- start 2024-09-21T03:40:32.347 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:32.345+0000 7f4e3a3ec740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ac1d24c2f0 0x55ac1d626580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:32.347 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:32.345+0000 7f4e3a3ec740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ac1d2544d0 0x55ac1d626c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:32.347 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:32.345+0000 7f4e3a3ec740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ac1d2751b0 0x55ac1d6277b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:32.347 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:32.345+0000 7f4e3113d640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ac1d24c2f0 0x55ac1d626580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:32.347 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:32.345+0000 7f4e3093c640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ac1d2544d0 0x55ac1d626c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:32.347 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:32.345+0000 7f4e3113d640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ac1d24c2f0 0x55ac1d626580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.89:53276/0 (socket says 172.21.15.89:53276) 2024-09-21T03:40:32.347 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:32.345+0000 7f4e3093c640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ac1d2544d0 0x55ac1d626c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:49706/0 (socket says 172.21.15.89:49706) 2024-09-21T03:40:32.347 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:32.345+0000 7f4e3113d640 1 -- 172.21.15.89:0/2225636441 learned_addr learned my addr 172.21.15.89:0/2225636441 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-21T03:40:32.348 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:32.345+0000 7f4e3a3ec740 1 -- 172.21.15.89:0/2225636441 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55ac1d4b6ab0 con 0x55ac1d2544d0 2024-09-21T03:40:32.348 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:32.345+0000 7f4e3a3ec740 1 -- 172.21.15.89:0/2225636441 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55ac1d524730 con 0x55ac1d2751b0 2024-09-21T03:40:32.348 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:32.345+0000 7f4e3a3ec740 1 -- 172.21.15.89:0/2225636441 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55ac1d61eaf0 con 0x55ac1d24c2f0 2024-09-21T03:40:32.348 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:32.345+0000 7f4e3193e640 1 --2- 172.21.15.89:0/2225636441 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ac1d2751b0 0x55ac1d6277b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:32.348 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:32.346+0000 7f4e3a3ec740 10 client.0 _ll_register_callbacks cb 0x55ac1d5786b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:40:32.348 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:32.346+0000 7f4e3193e640 1 -- 172.21.15.89:0/2225636441 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ac1d24c2f0 msgr2=0x55ac1d626580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:40:32.348 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:32.346+0000 7f4e3193e640 1 --2- 172.21.15.89:0/2225636441 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ac1d24c2f0 0x55ac1d626580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:32.348 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:32.346+0000 7f4e3193e640 1 -- 172.21.15.89:0/2225636441 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ac1d2544d0 msgr2=0x55ac1d626c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:40:32.348 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:32.346+0000 7f4e3193e640 1 --2- 172.21.15.89:0/2225636441 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ac1d2544d0 0x55ac1d626c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:32.348 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:32.346+0000 7f4e3193e640 1 -- 172.21.15.89:0/2225636441 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ac1d2e0790 con 0x55ac1d2751b0 2024-09-21T03:40:32.348 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:32.346+0000 7f4e3093c640 1 --2- 172.21.15.89:0/2225636441 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ac1d2544d0 0x55ac1d626c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:40:32.348 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:32.346+0000 7f4e3113d640 1 --2- 172.21.15.89:0/2225636441 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ac1d24c2f0 0x55ac1d626580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:40:32.348 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:32.346+0000 7f4e3a3ec740 1 -- 172.21.15.89:0/2225636441 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4e280043d0 con 0x55ac1d2751b0 2024-09-21T03:40:32.349 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:32.346+0000 7f4e3193e640 1 --2- 172.21.15.89:0/2225636441 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ac1d2751b0 0x55ac1d6277b0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f4e2402aeb0 tx=0x7f4e24031000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cf88fe6917ccaab8 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:32.349 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:32.346+0000 7f4e14ff9640 10 client.25226 ms_handle_connect on v2:172.21.15.110:3300/0 2024-09-21T03:40:32.349 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:32.346+0000 7f4e14ff9640 1 -- 172.21.15.89:0/2225636441 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e24030b50 con 0x55ac1d2751b0 2024-09-21T03:40:32.349 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:32.347+0000 7f4e14ff9640 1 -- 172.21.15.89:0/2225636441 <== mon.2 v2:172.21.15.110:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 9121+0+0 (secure 0 0 0) 0x7f4e24030cf0 con 0x55ac1d2751b0 2024-09-21T03:40:32.350 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:32.347+0000 7f4e14ff9640 1 -- 172.21.15.89:0/2225636441 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f4e24051470 con 0x55ac1d2751b0 2024-09-21T03:40:32.350 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:32.347+0000 7f4e3a3ec740 10 client.25226 fetch_fsmap learned FSMap version 26 2024-09-21T03:40:32.350 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:32.347+0000 7f4e3a3ec740 1 -- 172.21.15.89:0/2225636441 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({fsmap=26}) -- 0x55ac1d359620 con 0x55ac1d2751b0 2024-09-21T03:40:32.350 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:32.347+0000 7f4e14ff9640 1 -- 172.21.15.89:0/2225636441 <== mon.2 v2:172.21.15.110:3300/0 4 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f4e24051610 con 0x55ac1d2751b0 2024-09-21T03:40:32.350 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:32.347+0000 7f4e3a3ec740 10 client.25226 fetch_fsmap finished waiting for FSMap version 26 2024-09-21T03:40:32.350 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:32.347+0000 7f4e3a3ec740 10 client.25226 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T03:40:32.350 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:32.347+0000 7f4e3a3ec740 20 client.25226 populate_metadata read hostname 'smithi089' 2024-09-21T03:40:32.350 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:32.347+0000 7f4e3a3ec740 1 --2- 172.21.15.89:0/2225636441 >> [v2:172.21.15.110:6838/132366709,v1:172.21.15.110:6839/132366709] conn(0x55ac1d233d80 0x55ac1d633d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:32.350 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:32.347+0000 7f4e3a3ec740 4 client.25226 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:40:32.350 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:32.347+0000 7f4e3a3ec740 1 -- 172.21.15.89:0/2225636441 --> [v2:172.21.15.110:6838/132366709,v1:172.21.15.110:6839/132366709] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ac1d591640 con 0x55ac1d233d80 2024-09-21T03:40:32.350 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:32.348+0000 7f4e3113d640 1 --2- 172.21.15.89:0/2225636441 >> [v2:172.21.15.110:6838/132366709,v1:172.21.15.110:6839/132366709] conn(0x55ac1d233d80 0x55ac1d633d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:32.350 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:32.348+0000 7f4e14ff9640 10 client.25226 ms_handle_connect on v2:172.21.15.110:6838/132366709 2024-09-21T03:40:32.350 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:32.348+0000 7f4e3113d640 1 --2- 172.21.15.89:0/2225636441 >> [v2:172.21.15.110:6838/132366709,v1:172.21.15.110:6839/132366709] conn(0x55ac1d233d80 0x55ac1d633d00 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a770cbc79ee4b71c server_cookie=d4588073ab53a630 in_seq=0 out_seq=0 2024-09-21T03:40:32.352 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:32.349+0000 7f4e14ff9640 1 -- 172.21.15.89:0/2225636441 <== mds.0 v2:172.21.15.110:6838/132366709 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55ac1d591640 con 0x55ac1d233d80 2024-09-21T03:40:32.352 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:32.349+0000 7f4e14ff9640 10 client.25226 handle_command_reply: tid=0 2024-09-21T03:40:32.352 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:32.349+0000 7f4e3a3ec740 1 client.25226 shutdown 2024-09-21T03:40:32.352 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:32.349+0000 7f4e3a3ec740 1 -- 172.21.15.89:0/2225636441 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ac1d2751b0 msgr2=0x55ac1d6277b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:32.352 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:32.349+0000 7f4e3a3ec740 1 --2- 172.21.15.89:0/2225636441 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ac1d2751b0 0x55ac1d6277b0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f4e2402aeb0 tx=0x7f4e24031000 comp rx=0 tx=0).stop 2024-09-21T03:40:32.352 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:32.349+0000 7f4e3a3ec740 1 -- 172.21.15.89:0/2225636441 shutdown_connections 2024-09-21T03:40:32.352 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:32.349+0000 7f4e3a3ec740 1 --2- 172.21.15.89:0/2225636441 >> [v2:172.21.15.110:6838/132366709,v1:172.21.15.110:6839/132366709] conn(0x55ac1d233d80 0x55ac1d633d00 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:32.352 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:32.349+0000 7f4e3a3ec740 1 --2- 172.21.15.89:0/2225636441 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ac1d2751b0 0x55ac1d6277b0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:32.352 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:32.349+0000 7f4e3a3ec740 1 --2- 172.21.15.89:0/2225636441 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ac1d2544d0 0x55ac1d626c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:32.352 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:32.349+0000 7f4e3a3ec740 1 --2- 172.21.15.89:0/2225636441 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ac1d24c2f0 0x55ac1d626580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:32.352 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:32.349+0000 7f4e3a3ec740 1 -- 172.21.15.89:0/2225636441 >> 172.21.15.89:0/2225636441 conn(0x55ac1d269300 msgr2=0x55ac1d616ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:40:32.352 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:32.349+0000 7f4e14ff9640 0 client.25226 ms_handle_reset on v2:172.21.15.110:6838/132366709 2024-09-21T03:40:32.352 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:32.349+0000 7f4e3a3ec740 1 -- 172.21.15.89:0/2225636441 shutdown_connections 2024-09-21T03:40:32.353 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:32.349+0000 7f4e3a3ec740 1 -- 172.21.15.89:0/2225636441 wait complete. 2024-09-21T03:40:32.353 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:32.350+0000 7f4e3a3ec740 20 client.25226 trim_cache size 0 max 16384 2024-09-21T03:40:32.380 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:32.378+0000 7f4e3a3ec740 1 Processor -- start 2024-09-21T03:40:32.380 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:32.378+0000 7f4e3a3ec740 1 -- start start 2024-09-21T03:40:32.380 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:32.379+0000 7f4e3a3ec740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ac1d24c2f0 0x55ac1d619340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:32.381 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:32.379+0000 7f4e3a3ec740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ac1d2544d0 0x55ac1d619880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:32.381 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:32.379+0000 7f4e3a3ec740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ac1d2751b0 0x55ac1d619dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:32.381 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:32.379+0000 7f4e3a3ec740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f4e2c1acbb0 con 0x55ac1d2544d0 2024-09-21T03:40:32.381 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:32.379+0000 7f4e3a3ec740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f4e2c1aa480 con 0x55ac1d2751b0 2024-09-21T03:40:32.381 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:32.379+0000 7f4e3a3ec740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f4e2c1a82d0 con 0x55ac1d24c2f0 2024-09-21T03:40:32.381 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:32.379+0000 7f4e3093c640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ac1d2544d0 0x55ac1d619880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:32.381 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:32.379+0000 7f4e3093c640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ac1d2544d0 0x55ac1d619880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:49710/0 (socket says 172.21.15.89:49710) 2024-09-21T03:40:32.381 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:32.379+0000 7f4e3093c640 1 -- 172.21.15.89:0/3820703090 learned_addr learned my addr 172.21.15.89:0/3820703090 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-21T03:40:32.381 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:32.379+0000 7f4e3193e640 1 --2- 172.21.15.89:0/3820703090 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ac1d2751b0 0x55ac1d619dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:32.381 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:32.379+0000 7f4e3113d640 1 --2- 172.21.15.89:0/3820703090 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ac1d24c2f0 0x55ac1d619340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:32.381 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:32.379+0000 7f4e3193e640 1 -- 172.21.15.89:0/3820703090 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ac1d24c2f0 msgr2=0x55ac1d619340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:40:32.381 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:32.379+0000 7f4e3193e640 1 --2- 172.21.15.89:0/3820703090 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ac1d24c2f0 0x55ac1d619340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:32.381 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:32.379+0000 7f4e3193e640 1 -- 172.21.15.89:0/3820703090 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ac1d2544d0 msgr2=0x55ac1d619880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:40:32.381 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:32.379+0000 7f4e3193e640 1 --2- 172.21.15.89:0/3820703090 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ac1d2544d0 0x55ac1d619880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:32.381 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:32.379+0000 7f4e3193e640 1 -- 172.21.15.89:0/3820703090 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ac1d626fb0 con 0x55ac1d2751b0 2024-09-21T03:40:32.382 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:32.379+0000 7f4e3093c640 1 --2- 172.21.15.89:0/3820703090 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ac1d2544d0 0x55ac1d619880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:40:32.383 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:32.380+0000 7f4e3193e640 1 --2- 172.21.15.89:0/3820703090 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ac1d2751b0 0x55ac1d619dc0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f4e2402ade0 tx=0x7f4e2404a070 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f7b586b67e6c247a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:32.383 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:32.380+0000 7f4e16ffd640 1 -- 172.21.15.89:0/3820703090 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e24046ab0 con 0x55ac1d2751b0 2024-09-21T03:40:32.383 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:32.380+0000 7f4e16ffd640 1 -- 172.21.15.89:0/3820703090 <== mon.2 v2:172.21.15.110:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4e24061020 con 0x55ac1d2751b0 2024-09-21T03:40:32.383 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:32.380+0000 7f4e16ffd640 1 -- 172.21.15.89:0/3820703090 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e24046eb0 con 0x55ac1d2751b0 2024-09-21T03:40:32.383 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:32.381+0000 7f4e3a3ec740 1 -- 172.21.15.89:0/3820703090 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ac1d2751b0 msgr2=0x55ac1d619dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:32.383 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:32.381+0000 7f4e3a3ec740 1 --2- 172.21.15.89:0/3820703090 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ac1d2751b0 0x55ac1d619dc0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f4e2402ade0 tx=0x7f4e2404a070 comp rx=0 tx=0).stop 2024-09-21T03:40:32.383 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:32.381+0000 7f4e3a3ec740 1 -- 172.21.15.89:0/3820703090 shutdown_connections 2024-09-21T03:40:32.383 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:32.381+0000 7f4e3a3ec740 1 --2- 172.21.15.89:0/3820703090 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ac1d2751b0 0x55ac1d619dc0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:32.384 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:32.381+0000 7f4e3a3ec740 1 --2- 172.21.15.89:0/3820703090 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ac1d2544d0 0x55ac1d619880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:32.384 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:32.381+0000 7f4e3a3ec740 1 --2- 172.21.15.89:0/3820703090 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ac1d24c2f0 0x55ac1d619340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:32.384 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:32.381+0000 7f4e3a3ec740 1 -- 172.21.15.89:0/3820703090 >> 172.21.15.89:0/3820703090 conn(0x55ac1d269300 msgr2=0x55ac1d6181e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:40:32.384 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:32.381+0000 7f4e3a3ec740 1 -- 172.21.15.89:0/3820703090 shutdown_connections 2024-09-21T03:40:32.384 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:32.381+0000 7f4e3a3ec740 1 -- 172.21.15.89:0/3820703090 wait complete. 2024-09-21T03:40:32.384 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:32.382+0000 7f4e3a3ec740 1 -- start start 2024-09-21T03:40:32.384 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:32.382+0000 7f4e3a3ec740 1 Processor -- start 2024-09-21T03:40:32.386 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:32.382+0000 7f4e3a3ec740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ac1d24c2f0 0x55ac1d62b700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:32.387 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:32.382+0000 7f4e3a3ec740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ac1d2544d0 0x55ac1d62bdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:32.387 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:32.382+0000 7f4e3a3ec740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ac1d2751b0 0x55ac1d62c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:32.387 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:32.382+0000 7f4e3a3ec740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f4e2c1a8940 con 0x55ac1d2544d0 2024-09-21T03:40:32.387 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:32.382+0000 7f4e3a3ec740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f4e2c1a9db0 con 0x55ac1d2751b0 2024-09-21T03:40:32.387 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:32.382+0000 7f4e3a3ec740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f4e2c1a9010 con 0x55ac1d24c2f0 2024-09-21T03:40:32.387 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:32.382+0000 7f4e3a3ec740 10 client.0 _ll_register_callbacks cb 0x7f4e24009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:40:32.387 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:32.383+0000 7f4e3093c640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ac1d2544d0 0x55ac1d62bdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:32.387 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:32.383+0000 7f4e3093c640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ac1d2544d0 0x55ac1d62bdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:49720/0 (socket says 172.21.15.89:49720) 2024-09-21T03:40:32.387 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:32.383+0000 7f4e3093c640 1 -- 172.21.15.89:0/2175130608 learned_addr learned my addr 172.21.15.89:0/2175130608 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-21T03:40:32.387 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:32.383+0000 7f4e3193e640 1 --2- 172.21.15.89:0/2175130608 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ac1d2751b0 0x55ac1d62c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:32.387 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:32.383+0000 7f4e3113d640 1 --2- 172.21.15.89:0/2175130608 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ac1d24c2f0 0x55ac1d62b700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:32.387 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:32.383+0000 7f4e3093c640 1 -- 172.21.15.89:0/2175130608 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ac1d24c2f0 msgr2=0x55ac1d62b700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:40:32.387 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:32.383+0000 7f4e3093c640 1 --2- 172.21.15.89:0/2175130608 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ac1d24c2f0 0x55ac1d62b700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:32.388 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:32.383+0000 7f4e3093c640 1 -- 172.21.15.89:0/2175130608 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ac1d2751b0 msgr2=0x55ac1d62c8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:40:32.388 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:32.383+0000 7f4e3093c640 1 --2- 172.21.15.89:0/2175130608 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ac1d2751b0 0x55ac1d62c8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:32.388 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:32.383+0000 7f4e3093c640 1 -- 172.21.15.89:0/2175130608 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ac1d62d040 con 0x55ac1d2544d0 2024-09-21T03:40:32.388 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:32.383+0000 7f4e3193e640 1 --2- 172.21.15.89:0/2175130608 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ac1d2751b0 0x55ac1d62c8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:40:32.388 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:32.383+0000 7f4e3113d640 1 --2- 172.21.15.89:0/2175130608 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ac1d24c2f0 0x55ac1d62b700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:40:32.388 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:32.383+0000 7f4e3093c640 1 --2- 172.21.15.89:0/2175130608 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ac1d2544d0 0x55ac1d62bdc0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f4e1c00df50 tx=0x7f4e1c0e4c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6491f12ec09c3258 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:32.388 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:32.383+0000 7f4e14ff9640 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2024-09-21T03:40:32.388 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:32.384+0000 7f4e14ff9640 1 -- 172.21.15.89:0/2175130608 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e1c020040 con 0x55ac1d2544d0 2024-09-21T03:40:32.388 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:32.384+0000 7f4e3a3ec740 1 -- 172.21.15.89:0/2175130608 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4e24061020 con 0x55ac1d2544d0 2024-09-21T03:40:32.388 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:32.384+0000 7f4e14ff9640 1 -- 172.21.15.89:0/2175130608 <== mon.0 v2:172.21.15.89:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 9121+0+0 (secure 0 0 0) 0x7f4e1c0ac320 con 0x55ac1d2544d0 2024-09-21T03:40:32.388 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:32.384+0000 7f4e14ff9640 1 -- 172.21.15.89:0/2175130608 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f4e1c005300 con 0x55ac1d2544d0 2024-09-21T03:40:32.388 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:32.384+0000 7f4e3a3ec740 10 client.15669 fetch_fsmap learned FSMap version 26 2024-09-21T03:40:32.388 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:32.384+0000 7f4e3a3ec740 1 -- 172.21.15.89:0/2175130608 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55ac1d62e450 con 0x55ac1d2544d0 2024-09-21T03:40:32.388 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:32.384+0000 7f4e14ff9640 1 -- 172.21.15.89:0/2175130608 <== mon.0 v2:172.21.15.89:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4e24061020 con 0x55ac1d2544d0 2024-09-21T03:40:32.388 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:32.384+0000 7f4e14ff9640 1 -- 172.21.15.89:0/2175130608 <== mon.0 v2:172.21.15.89:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f4e1c092ce0 con 0x55ac1d2544d0 2024-09-21T03:40:32.389 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:32.384+0000 7f4e3a3ec740 10 client.15669 fetch_fsmap finished waiting for FSMap version 26 2024-09-21T03:40:32.389 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:32.384+0000 7f4e3a3ec740 10 client.15669 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T03:40:32.389 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:32.384+0000 7f4e3a3ec740 20 client.15669 populate_metadata read hostname 'smithi089' 2024-09-21T03:40:32.389 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:32.384+0000 7f4e3a3ec740 1 --2- 172.21.15.89:0/2175130608 >> [v2:172.21.15.110:6838/132366709,v1:172.21.15.110:6839/132366709] conn(0x55ac1d2091c0 0x55ac1d6a0950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:32.389 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:32.384+0000 7f4e3a3ec740 4 client.15669 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T03:40:32.389 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:32.384+0000 7f4e3a3ec740 1 -- 172.21.15.89:0/2175130608 --> [v2:172.21.15.110:6838/132366709,v1:172.21.15.110:6839/132366709] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55ac1d58d0f0 con 0x55ac1d2091c0 2024-09-21T03:40:32.389 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:32.385+0000 7f4e3113d640 1 --2- 172.21.15.89:0/2175130608 >> [v2:172.21.15.110:6838/132366709,v1:172.21.15.110:6839/132366709] conn(0x55ac1d2091c0 0x55ac1d6a0950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:32.389 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:32.385+0000 7f4e14ff9640 10 client.15669 ms_handle_connect on v2:172.21.15.110:6838/132366709 2024-09-21T03:40:32.389 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:32.385+0000 7f4e3113d640 1 --2- 172.21.15.89:0/2175130608 >> [v2:172.21.15.110:6838/132366709,v1:172.21.15.110:6839/132366709] conn(0x55ac1d2091c0 0x55ac1d6a0950 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4b19260993bd2ee0 server_cookie=22f95f4037d3efac in_seq=0 out_seq=0 2024-09-21T03:40:32.393 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:32.391+0000 7f4e14ff9640 1 -- 172.21.15.89:0/2175130608 <== mds.0 v2:172.21.15.110:6838/132366709 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55ac1d58d0f0 con 0x55ac1d2091c0 2024-09-21T03:40:32.393 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:32.391+0000 7f4e14ff9640 10 client.15669 handle_command_reply: tid=0 2024-09-21T03:40:32.393 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:32.391+0000 7f4e3a3ec740 1 client.15669 shutdown 2024-09-21T03:40:32.393 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:32.392+0000 7f4e3a3ec740 1 -- 172.21.15.89:0/2175130608 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ac1d2544d0 msgr2=0x55ac1d62bdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:32.393 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:32.392+0000 7f4e3a3ec740 1 --2- 172.21.15.89:0/2175130608 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ac1d2544d0 0x55ac1d62bdc0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f4e1c00df50 tx=0x7f4e1c0e4c10 comp rx=0 tx=0).stop 2024-09-21T03:40:32.393 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:32.392+0000 7f4e3a3ec740 1 -- 172.21.15.89:0/2175130608 shutdown_connections 2024-09-21T03:40:32.393 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:32.392+0000 7f4e3a3ec740 1 --2- 172.21.15.89:0/2175130608 >> [v2:172.21.15.110:6838/132366709,v1:172.21.15.110:6839/132366709] conn(0x55ac1d2091c0 0x55ac1d6a0950 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:32.393 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:32.392+0000 7f4e3a3ec740 1 --2- 172.21.15.89:0/2175130608 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ac1d2751b0 0x55ac1d62c8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:32.393 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:32.392+0000 7f4e3a3ec740 1 --2- 172.21.15.89:0/2175130608 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ac1d2544d0 0x55ac1d62bdc0 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:32.393 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:32.392+0000 7f4e3a3ec740 1 --2- 172.21.15.89:0/2175130608 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ac1d24c2f0 0x55ac1d62b700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:32.393 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:32.392+0000 7f4e3a3ec740 1 -- 172.21.15.89:0/2175130608 >> 172.21.15.89:0/2175130608 conn(0x55ac1d269300 msgr2=0x55ac1d619320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:40:32.393 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:32.392+0000 7f4e14ff9640 0 client.15669 ms_handle_reset on v2:172.21.15.110:6838/132366709 2024-09-21T03:40:32.394 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:32.392+0000 7f4e3a3ec740 1 -- 172.21.15.89:0/2175130608 shutdown_connections 2024-09-21T03:40:32.394 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:32.392+0000 7f4e3a3ec740 1 -- 172.21.15.89:0/2175130608 wait complete. 2024-09-21T03:40:32.394 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:32.392+0000 7f4e3a3ec740 20 client.15669 trim_cache size 0 max 16384 2024-09-21T03:40:32.394 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stdout:{ 2024-09-21T03:40:32.395 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stdout: "message": "", 2024-09-21T03:40:32.395 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stdout: "return_code": 0 2024-09-21T03:40:32.395 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stdout:} 2024-09-21T03:40:32.395 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:32.393+0000 7f4e333a1640 1 -- 172.21.15.89:0/1205383807 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f4e180b6620 msgr2=0x7f4e180b8b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:32.395 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:32.393+0000 7f4e333a1640 1 --2- 172.21.15.89:0/1205383807 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f4e180b6620 0x7f4e180b8b30 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f4e240011d0 tx=0x7f4e2403f040 comp rx=0 tx=0).stop 2024-09-21T03:40:32.395 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:32.393+0000 7f4e333a1640 1 -- 172.21.15.89:0/1205383807 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4e2c1056e0 msgr2=0x7f4e2c19e9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:32.396 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:32.393+0000 7f4e333a1640 1 --2- 172.21.15.89:0/1205383807 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4e2c1056e0 0x7f4e2c19e9b0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f4e1c00d770 tx=0x7f4e1c00dc30 comp rx=0 tx=0).stop 2024-09-21T03:40:32.396 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:32.393+0000 7f4e333a1640 1 -- 172.21.15.89:0/1205383807 shutdown_connections 2024-09-21T03:40:32.396 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:32.393+0000 7f4e333a1640 1 --2- 172.21.15.89:0/1205383807 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f4e180b6620 0x7f4e180b8b30 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:32.396 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:32.393+0000 7f4e333a1640 1 --2- 172.21.15.89:0/1205383807 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4e2c108300 0x7f4e2c1a0f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:32.396 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:32.393+0000 7f4e333a1640 1 --2- 172.21.15.89:0/1205383807 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4e2c1056e0 0x7f4e2c19e9b0 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:32.396 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:32.393+0000 7f4e333a1640 1 --2- 172.21.15.89:0/1205383807 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4e2c102c60 0x7f4e2c102100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:32.396 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:32.393+0000 7f4e333a1640 1 -- 172.21.15.89:0/1205383807 >> 172.21.15.89:0/1205383807 conn(0x7f4e2c0fd690 msgr2=0x7f4e2c106540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:40:32.396 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:32.393+0000 7f4e333a1640 1 -- 172.21.15.89:0/1205383807 shutdown_connections 2024-09-21T03:40:32.396 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:32.393+0000 7f4e333a1640 1 -- 172.21.15.89:0/1205383807 wait complete. 2024-09-21T03:40:33.711 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:40:33 smithi114 ceph-mon[34140]: pgmap v394: 129 pgs: 129 active+clean; 675 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 0 B/s wr, 5 op/s 2024-09-21T03:40:33.750 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:40:33 smithi110 ceph-mon[34000]: pgmap v394: 129 pgs: 129 active+clean; 675 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 0 B/s wr, 5 op/s 2024-09-21T03:40:33.811 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:40:33 smithi089 ceph-mon[30928]: pgmap v394: 129 pgs: 129 active+clean; 675 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 0 B/s wr, 5 op/s 2024-09-21T03:40:35.711 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:40:35 smithi114 ceph-mon[34140]: pgmap v395: 129 pgs: 129 active+clean; 675 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 0 B/s wr, 5 op/s 2024-09-21T03:40:35.751 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:40:35 smithi110 ceph-mon[34000]: pgmap v395: 129 pgs: 129 active+clean; 675 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 0 B/s wr, 5 op/s 2024-09-21T03:40:35.811 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:40:35 smithi089 ceph-mon[30928]: pgmap v395: 129 pgs: 129 active+clean; 675 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 0 B/s wr, 5 op/s 2024-09-21T03:40:37.711 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:40:37 smithi114 ceph-mon[34140]: pgmap v396: 129 pgs: 129 active+clean; 676 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 170 B/s wr, 6 op/s 2024-09-21T03:40:37.750 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:40:37 smithi110 ceph-mon[34000]: pgmap v396: 129 pgs: 129 active+clean; 676 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 170 B/s wr, 6 op/s 2024-09-21T03:40:37.811 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:40:37 smithi089 ceph-mon[30928]: pgmap v396: 129 pgs: 129 active+clean; 676 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 170 B/s wr, 6 op/s 2024-09-21T03:40:39.286 INFO:tasks.workunit.client.0.smithi089.stderr:Note: switching to '1843221895adccfb3b692060251dd6a0183bc410'. 2024-09-21T03:40:39.286 INFO:tasks.workunit.client.0.smithi089.stderr: 2024-09-21T03:40:39.286 INFO:tasks.workunit.client.0.smithi089.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-09-21T03:40:39.286 INFO:tasks.workunit.client.0.smithi089.stderr:changes and commit them, and you can discard any commits you make in this 2024-09-21T03:40:39.286 INFO:tasks.workunit.client.0.smithi089.stderr:state without impacting any branches by switching back to a branch. 2024-09-21T03:40:39.287 INFO:tasks.workunit.client.0.smithi089.stderr: 2024-09-21T03:40:39.287 INFO:tasks.workunit.client.0.smithi089.stderr:If you want to create a new branch to retain commits you create, you may 2024-09-21T03:40:39.287 INFO:tasks.workunit.client.0.smithi089.stderr:do so (now or later) by using -c with the switch command. Example: 2024-09-21T03:40:39.287 INFO:tasks.workunit.client.0.smithi089.stderr: 2024-09-21T03:40:39.287 INFO:tasks.workunit.client.0.smithi089.stderr: git switch -c 2024-09-21T03:40:39.287 INFO:tasks.workunit.client.0.smithi089.stderr: 2024-09-21T03:40:39.287 INFO:tasks.workunit.client.0.smithi089.stderr:Or undo this operation with: 2024-09-21T03:40:39.287 INFO:tasks.workunit.client.0.smithi089.stderr: 2024-09-21T03:40:39.287 INFO:tasks.workunit.client.0.smithi089.stderr: git switch - 2024-09-21T03:40:39.287 INFO:tasks.workunit.client.0.smithi089.stderr: 2024-09-21T03:40:39.287 INFO:tasks.workunit.client.0.smithi089.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-09-21T03:40:39.287 INFO:tasks.workunit.client.0.smithi089.stderr: 2024-09-21T03:40:39.287 INFO:tasks.workunit.client.0.smithi089.stderr:HEAD is now at 1843221895a Merge PR #56776 into wip-pdonnell-testing-20240920.212106-debug 2024-09-21T03:40:39.292 DEBUG:teuthology.orchestra.run.smithi089:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-09-21T03:40:39.356 INFO:tasks.workunit.client.0.smithi089.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-09-21T03:40:39.358 INFO:tasks.workunit.client.0.smithi089.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-09-21T03:40:39.358 INFO:tasks.workunit.client.0.smithi089.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-09-21T03:40:39.711 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:40:39 smithi114 ceph-mon[34140]: pgmap v397: 129 pgs: 129 active+clean; 676 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 170 B/s wr, 5 op/s 2024-09-21T03:40:39.750 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:40:39 smithi110 ceph-mon[34000]: pgmap v397: 129 pgs: 129 active+clean; 676 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 170 B/s wr, 5 op/s 2024-09-21T03:40:39.811 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:40:39 smithi089 ceph-mon[30928]: pgmap v397: 129 pgs: 129 active+clean; 676 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 170 B/s wr, 5 op/s 2024-09-21T03:40:40.044 INFO:tasks.workunit.client.0.smithi089.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-09-21T03:40:40.115 INFO:tasks.workunit.client.0.smithi089.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-09-21T03:40:40.156 INFO:tasks.workunit.client.0.smithi089.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-09-21T03:40:40.158 INFO:tasks.workunit.client.0.smithi089.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-09-21T03:40:40.158 INFO:tasks.workunit.client.0.smithi089.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-09-21T03:40:40.197 INFO:tasks.workunit.client.0.smithi089.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-09-21T03:40:40.201 DEBUG:teuthology.orchestra.run.smithi089:> set -ex 2024-09-21T03:40:40.201 DEBUG:teuthology.orchestra.run.smithi089:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-09-21T03:40:40.257 INFO:tasks.workunit:Running workunits matching suites/fsync-tester.sh on client.0... 2024-09-21T03:40:40.258 INFO:tasks.workunit:Running workunit suites/fsync-tester.sh... 2024-09-21T03:40:40.259 DEBUG:teuthology.orchestra.run.smithi089:workunit test suites/fsync-tester.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=1843221895adccfb3b692060251dd6a0183bc410 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/fsync-tester.sh 2024-09-21T03:40:40.326 INFO:tasks.workunit.client.0.smithi089.stderr:+ wget http://download.ceph.com/qa/fsync-tester.c 2024-09-21T03:40:40.347 INFO:tasks.workunit.client.0.smithi089.stderr:--2024-09-21 03:40:40-- http://download.ceph.com/qa/fsync-tester.c 2024-09-21T03:40:40.348 INFO:tasks.workunit.client.0.smithi089.stderr:Resolving download.ceph.com (download.ceph.com)... 158.69.68.124, 2607:5300:201:2000::3:58a1 2024-09-21T03:40:40.369 INFO:tasks.workunit.client.0.smithi089.stderr:Connecting to download.ceph.com (download.ceph.com)|158.69.68.124|:80... connected. 2024-09-21T03:40:40.390 INFO:tasks.workunit.client.0.smithi089.stderr:HTTP request sent, awaiting response... 200 OK 2024-09-21T03:40:40.390 INFO:tasks.workunit.client.0.smithi089.stderr:Length: 3024 (3.0K) [application/octet-stream] 2024-09-21T03:40:40.392 INFO:tasks.workunit.client.0.smithi089.stderr:Saving to: ‘fsync-tester.c’ 2024-09-21T03:40:40.396 INFO:tasks.workunit.client.0.smithi089.stderr: 2024-09-21T03:40:40.396 INFO:tasks.workunit.client.0.smithi089.stderr: 0K .. 100% 562M=0s 2024-09-21T03:40:40.396 INFO:tasks.workunit.client.0.smithi089.stderr: 2024-09-21T03:40:40.396 INFO:tasks.workunit.client.0.smithi089.stderr:2024-09-21 03:40:40 (562 MB/s) - ‘fsync-tester.c’ saved [3024/3024] 2024-09-21T03:40:40.396 INFO:tasks.workunit.client.0.smithi089.stderr: 2024-09-21T03:40:40.396 INFO:tasks.workunit.client.0.smithi089.stderr:+ gcc -D_GNU_SOURCE fsync-tester.c -o fsync-tester 2024-09-21T03:40:40.482 INFO:tasks.workunit.client.0.smithi089.stderr:+ ./fsync-tester 2024-09-21T03:40:41.711 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:40:41 smithi114 ceph-mon[34140]: pgmap v398: 129 pgs: 129 active+clean; 676 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 170 B/s wr, 7 op/s 2024-09-21T03:40:41.750 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:40:41 smithi110 ceph-mon[34000]: pgmap v398: 129 pgs: 129 active+clean; 676 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 170 B/s wr, 7 op/s 2024-09-21T03:40:41.811 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:40:41 smithi089 ceph-mon[30928]: pgmap v398: 129 pgs: 129 active+clean; 676 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 170 B/s wr, 7 op/s 2024-09-21T03:40:43.711 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:40:43 smithi114 ceph-mon[34140]: pgmap v399: 129 pgs: 129 active+clean; 676 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 170 B/s wr, 5 op/s 2024-09-21T03:40:43.750 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:40:43 smithi110 ceph-mon[34000]: pgmap v399: 129 pgs: 129 active+clean; 676 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 170 B/s wr, 5 op/s 2024-09-21T03:40:43.811 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:40:43 smithi089 ceph-mon[30928]: pgmap v399: 129 pgs: 129 active+clean; 676 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 170 B/s wr, 5 op/s 2024-09-21T03:40:44.351 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 17.0 2024-09-21T03:40:44.352 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 7c374098-77c9-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T03:40:44.625 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/7c374098-77c9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:40:45.157 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:45.150+0000 7f67aa02b640 1 -- 172.21.15.89:0/1408402181 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f67a410de10 msgr2=0x7f67a4110300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:45.157 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:45.150+0000 7f67aa02b640 1 --2- 172.21.15.89:0/1408402181 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f67a410de10 0x7f67a4110300 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f679c009ca0 tx=0x7f679c02a8c0 comp rx=0 tx=0).stop 2024-09-21T03:40:45.157 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:45.150+0000 7f67aa02b640 1 -- 172.21.15.89:0/1408402181 shutdown_connections 2024-09-21T03:40:45.157 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:45.150+0000 7f67aa02b640 1 --2- 172.21.15.89:0/1408402181 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f67a4105300 0x7f67a410d850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:45.157 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:45.150+0000 7f67aa02b640 1 --2- 172.21.15.89:0/1408402181 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f67a410de10 0x7f67a4110300 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:45.157 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:45.150+0000 7f67aa02b640 1 --2- 172.21.15.89:0/1408402181 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f67a41049b0 0x7f67a4104dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:45.158 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:45.150+0000 7f67aa02b640 1 -- 172.21.15.89:0/1408402181 >> 172.21.15.89:0/1408402181 conn(0x7f67a4079730 msgr2=0x7f67a4079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:40:45.158 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:45.150+0000 7f67aa02b640 1 -- 172.21.15.89:0/1408402181 shutdown_connections 2024-09-21T03:40:45.158 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:45.150+0000 7f67aa02b640 1 -- 172.21.15.89:0/1408402181 wait complete. 2024-09-21T03:40:45.158 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:45.151+0000 7f67aa02b640 1 Processor -- start 2024-09-21T03:40:45.158 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:45.151+0000 7f67aa02b640 1 -- start start 2024-09-21T03:40:45.158 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:45.152+0000 7f67aa02b640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f67a41049b0 0x7f67a419a420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:45.158 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:45.152+0000 7f67aa02b640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f67a4105300 0x7f67a419a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:45.158 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:45.152+0000 7f67aa02b640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f67a410de10 0x7f67a419fc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:45.158 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:45.152+0000 7f67aa02b640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f67a4113970 con 0x7f67a410de10 2024-09-21T03:40:45.158 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:45.152+0000 7f67aa02b640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f67a41137f0 con 0x7f67a4105300 2024-09-21T03:40:45.158 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:45.152+0000 7f67aa02b640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f67a4113af0 con 0x7f67a41049b0 2024-09-21T03:40:45.158 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:45.152+0000 7f67a3fff640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f67a410de10 0x7f67a419fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:45.158 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:45.152+0000 7f67a2ffd640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f67a4105300 0x7f67a419a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:45.158 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:45.152+0000 7f67a3fff640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f67a410de10 0x7f67a419fc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:60606/0 (socket says 172.21.15.89:60606) 2024-09-21T03:40:45.159 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:45.152+0000 7f67a3fff640 1 -- 172.21.15.89:0/3303564755 learned_addr learned my addr 172.21.15.89:0/3303564755 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-21T03:40:45.159 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:45.152+0000 7f67a37fe640 1 --2- 172.21.15.89:0/3303564755 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f67a41049b0 0x7f67a419a420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:45.159 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:45.152+0000 7f67a3fff640 1 -- 172.21.15.89:0/3303564755 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f67a41049b0 msgr2=0x7f67a419a420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:45.159 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:45.152+0000 7f67a3fff640 1 --2- 172.21.15.89:0/3303564755 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f67a41049b0 0x7f67a419a420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:45.159 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:45.152+0000 7f67a3fff640 1 -- 172.21.15.89:0/3303564755 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f67a4105300 msgr2=0x7f67a419a9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:45.159 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:45.152+0000 7f67a3fff640 1 --2- 172.21.15.89:0/3303564755 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f67a4105300 0x7f67a419a9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:45.159 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:45.152+0000 7f67a3fff640 1 -- 172.21.15.89:0/3303564755 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67a41a0390 con 0x7f67a410de10 2024-09-21T03:40:45.159 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:45.152+0000 7f67a3fff640 1 --2- 172.21.15.89:0/3303564755 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f67a410de10 0x7f67a419fc60 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f679c005e30 tx=0x7f679c0093d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:45.159 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:45.153+0000 7f67a0ff9640 1 -- 172.21.15.89:0/3303564755 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f679c032d10 con 0x7f67a410de10 2024-09-21T03:40:45.159 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:45.153+0000 7f67aa02b640 1 -- 172.21.15.89:0/3303564755 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f67a41a0620 con 0x7f67a410de10 2024-09-21T03:40:45.159 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:45.153+0000 7f67aa02b640 1 -- 172.21.15.89:0/3303564755 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67a41a6de0 con 0x7f67a410de10 2024-09-21T03:40:45.159 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:45.153+0000 7f67a0ff9640 1 -- 172.21.15.89:0/3303564755 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f679c032eb0 con 0x7f67a410de10 2024-09-21T03:40:45.159 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:45.153+0000 7f67a0ff9640 1 -- 172.21.15.89:0/3303564755 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f679c030b20 con 0x7f67a410de10 2024-09-21T03:40:45.159 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:45.154+0000 7f67aa02b640 1 -- 172.21.15.89:0/3303564755 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6790005290 con 0x7f67a410de10 2024-09-21T03:40:45.160 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:45.157+0000 7f67a0ff9640 1 -- 172.21.15.89:0/3303564755 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 20) ==== 154563+0+0 (secure 0 0 0) 0x7f679c030cc0 con 0x7f67a410de10 2024-09-21T03:40:45.160 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:45.158+0000 7f67a0ff9640 1 --2- 172.21.15.89:0/3303564755 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f678c0b6620 0x7f678c0b8b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:45.160 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:45.158+0000 7f67a37fe640 1 --2- 172.21.15.89:0/3303564755 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f678c0b6620 0x7f678c0b8b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:45.160 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:45.158+0000 7f67a0ff9640 1 -- 172.21.15.89:0/3303564755 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 9121+0+0 (secure 0 0 0) 0x7f679c108ab0 con 0x7f67a410de10 2024-09-21T03:40:45.161 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:45.158+0000 7f67a37fe640 1 --2- 172.21.15.89:0/3303564755 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f678c0b6620 0x7f678c0b8b30 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f67a406bc60 tx=0x7f679400a2d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:45.163 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:45.161+0000 7f67a0ff9640 1 -- 172.21.15.89:0/3303564755 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f679c0cee60 con 0x7f67a410de10 2024-09-21T03:40:45.420 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:45.418+0000 7f67aa02b640 1 -- 172.21.15.89:0/3303564755 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6790005580 con 0x7f67a410de10 2024-09-21T03:40:45.421 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:45.419+0000 7f67a0ff9640 1 -- 172.21.15.89:0/3303564755 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11912 (secure 0 0 0) 0x7f679c0d3240 con 0x7f67a410de10 2024-09-21T03:40:45.421 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-09-21T03:40:45.421 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":26,"btime":"2024-09-21T03:39:26:058611+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14721,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6835/2341538514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":2341538514},{"type":"v1","addr":"172.21.15.89:6835","nonce":2341538514}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14733,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6837/146708592","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6836","nonce":146708592},{"type":"v1","addr":"172.21.15.89:6837","nonce":146708592}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14781,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6839/2269965499","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6838","nonce":2269965499},{"type":"v1","addr":"172.21.15.89:6839","nonce":2269965499}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14793,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6841/650850383","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6840","nonce":650850383},{"type":"v1","addr":"172.21.15.89:6841","nonce":650850383}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24565,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6833/752336912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6832","nonce":752336912},{"type":"v1","addr":"172.21.15.114:6833","nonce":752336912}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24575,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6833/127607331","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6832","nonce":127607331},{"type":"v1","addr":"172.21.15.110:6833","nonce":127607331}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T03:37:13.570694+0000","modified":"2024-09-21T03:39:25.333954+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24638,"mds_1":24631,"mds_2":24614},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24593":{"gid":24593,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.110:6835/671820654","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6834","nonce":671820654},{"type":"v1","addr":"172.21.15.110:6835","nonce":671820654}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24604":{"gid":24604,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.114:6835/1049714548","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6834","nonce":1049714548},{"type":"v1","addr":"172.21.15.114:6835","nonce":1049714548}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24610":{"gid":24610,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.114:6837/1941213818","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6836","nonce":1941213818},{"type":"v1","addr":"172.21.15.114:6837","nonce":1941213818}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24614":{"gid":24614,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":15,"addr":"172.21.15.110:6837/4047124589","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6836","nonce":4047124589},{"type":"v1","addr":"172.21.15.110:6837","nonce":4047124589}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24631":{"gid":24631,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":15,"addr":"172.21.15.114:6839/3427369944","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6838","nonce":3427369944},{"type":"v1","addr":"172.21.15.114:6839","nonce":3427369944}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24638":{"gid":24638,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.110:6839/132366709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6838","nonce":132366709},{"type":"v1","addr":"172.21.15.110:6839","nonce":132366709}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24638,"qdb_cluster":[24638,24631,24614]},"id":1}]} 2024-09-21T03:40:45.422 INFO:teuthology.orchestra.run.smithi089.stderr:dumped fsmap epoch 26 2024-09-21T03:40:45.424 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:45.422+0000 7f67aa02b640 1 -- 172.21.15.89:0/3303564755 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f678c0b6620 msgr2=0x7f678c0b8b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:45.424 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:45.422+0000 7f67aa02b640 1 --2- 172.21.15.89:0/3303564755 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f678c0b6620 0x7f678c0b8b30 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f67a406bc60 tx=0x7f679400a2d0 comp rx=0 tx=0).stop 2024-09-21T03:40:45.424 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:45.422+0000 7f67aa02b640 1 -- 172.21.15.89:0/3303564755 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f67a410de10 msgr2=0x7f67a419fc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:45.424 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:45.422+0000 7f67aa02b640 1 --2- 172.21.15.89:0/3303564755 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f67a410de10 0x7f67a419fc60 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f679c005e30 tx=0x7f679c0093d0 comp rx=0 tx=0).stop 2024-09-21T03:40:45.425 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:45.422+0000 7f67aa02b640 1 -- 172.21.15.89:0/3303564755 shutdown_connections 2024-09-21T03:40:45.425 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:45.422+0000 7f67aa02b640 1 --2- 172.21.15.89:0/3303564755 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f678c0b6620 0x7f678c0b8b30 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:45.425 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:45.422+0000 7f67aa02b640 1 --2- 172.21.15.89:0/3303564755 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f67a4105300 0x7f67a419a9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:45.425 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:45.422+0000 7f67aa02b640 1 --2- 172.21.15.89:0/3303564755 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f67a410de10 0x7f67a419fc60 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:45.425 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:45.422+0000 7f67aa02b640 1 --2- 172.21.15.89:0/3303564755 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f67a41049b0 0x7f67a419a420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:45.425 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:45.422+0000 7f67aa02b640 1 -- 172.21.15.89:0/3303564755 >> 172.21.15.89:0/3303564755 conn(0x7f67a4079730 msgr2=0x7f67a40ff550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:40:45.425 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:45.422+0000 7f67aa02b640 1 -- 172.21.15.89:0/3303564755 shutdown_connections 2024-09-21T03:40:45.425 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:45.422+0000 7f67aa02b640 1 -- 172.21.15.89:0/3303564755 wait complete. 2024-09-21T03:40:45.506 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:45.504+0000 7efe1f0cc640 1 -- 172.21.15.89:0/3302469052 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efe18103650 msgr2=0x7efe18103a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:45.507 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:45.504+0000 7efe1f0cc640 1 --2- 172.21.15.89:0/3302469052 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efe18103650 0x7efe18103a60 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7efe04009900 tx=0x7efe0402a410 comp rx=0 tx=0).stop 2024-09-21T03:40:45.507 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:45.504+0000 7efe1f0cc640 1 -- 172.21.15.89:0/3302469052 shutdown_connections 2024-09-21T03:40:45.507 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:45.504+0000 7efe1f0cc640 1 --2- 172.21.15.89:0/3302469052 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7efe18103ff0 0x7efe1810a990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:45.507 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:45.504+0000 7efe1f0cc640 1 --2- 172.21.15.89:0/3302469052 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efe18103650 0x7efe18103a60 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:45.507 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:45.504+0000 7efe1f0cc640 1 --2- 172.21.15.89:0/3302469052 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7efe181025d0 0x7efe18102a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:45.507 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:45.504+0000 7efe1f0cc640 1 -- 172.21.15.89:0/3302469052 >> 172.21.15.89:0/3302469052 conn(0x7efe180fdf30 msgr2=0x7efe181003c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:40:45.507 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:45.504+0000 7efe1f0cc640 1 -- 172.21.15.89:0/3302469052 shutdown_connections 2024-09-21T03:40:45.507 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:45.504+0000 7efe1f0cc640 1 -- 172.21.15.89:0/3302469052 wait complete. 2024-09-21T03:40:45.509 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:45.505+0000 7efe1f0cc640 1 Processor -- start 2024-09-21T03:40:45.509 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:45.505+0000 7efe1f0cc640 1 -- start start 2024-09-21T03:40:45.509 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:45.505+0000 7efe1f0cc640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7efe181025d0 0x7efe1806be10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:45.510 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:45.505+0000 7efe1f0cc640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efe18103650 0x7efe1819ec20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:45.510 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:45.505+0000 7efe1f0cc640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7efe18103ff0 0x7efe181a1170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:45.510 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:45.505+0000 7efe1f0cc640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7efe1810e0d0 con 0x7efe18103650 2024-09-21T03:40:45.510 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:45.505+0000 7efe1f0cc640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7efe1810df50 con 0x7efe181025d0 2024-09-21T03:40:45.510 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:45.505+0000 7efe1f0cc640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7efe1810e250 con 0x7efe18103ff0 2024-09-21T03:40:45.510 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:45.506+0000 7efe17fff640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efe18103650 0x7efe1819ec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:45.510 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:45.506+0000 7efe17fff640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efe18103650 0x7efe1819ec20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:60634/0 (socket says 172.21.15.89:60634) 2024-09-21T03:40:45.510 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:45.506+0000 7efe17fff640 1 -- 172.21.15.89:0/3884303944 learned_addr learned my addr 172.21.15.89:0/3884303944 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-21T03:40:45.510 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:45.506+0000 7efe1ce68640 1 --2- 172.21.15.89:0/3884303944 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7efe181025d0 0x7efe1806be10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:45.510 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:45.506+0000 7efe1d669640 1 --2- 172.21.15.89:0/3884303944 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7efe18103ff0 0x7efe181a1170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:45.510 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:45.506+0000 7efe17fff640 1 -- 172.21.15.89:0/3884303944 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7efe18103ff0 msgr2=0x7efe181a1170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:45.510 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:45.506+0000 7efe17fff640 1 --2- 172.21.15.89:0/3884303944 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7efe18103ff0 0x7efe181a1170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:45.510 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:45.506+0000 7efe17fff640 1 -- 172.21.15.89:0/3884303944 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7efe181025d0 msgr2=0x7efe1806be10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:45.510 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:45.506+0000 7efe17fff640 1 --2- 172.21.15.89:0/3884303944 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7efe181025d0 0x7efe1806be10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:45.511 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:45.506+0000 7efe17fff640 1 -- 172.21.15.89:0/3884303944 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe181a1970 con 0x7efe18103650 2024-09-21T03:40:45.511 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:45.506+0000 7efe1d669640 1 --2- 172.21.15.89:0/3884303944 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7efe18103ff0 0x7efe181a1170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:40:45.511 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:45.506+0000 7efe1ce68640 1 --2- 172.21.15.89:0/3884303944 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7efe181025d0 0x7efe1806be10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:40:45.511 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:45.506+0000 7efe17fff640 1 --2- 172.21.15.89:0/3884303944 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efe18103650 0x7efe1819ec20 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7efe04009230 tx=0x7efe040042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:45.511 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:45.506+0000 7efe15ffb640 1 -- 172.21.15.89:0/3884303944 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe0403f030 con 0x7efe18103650 2024-09-21T03:40:45.511 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:45.507+0000 7efe1f0cc640 1 -- 172.21.15.89:0/3884303944 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efe181a1c00 con 0x7efe18103650 2024-09-21T03:40:45.511 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:45.507+0000 7efe15ffb640 1 -- 172.21.15.89:0/3884303944 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7efe04032e20 con 0x7efe18103650 2024-09-21T03:40:45.511 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:45.507+0000 7efe15ffb640 1 -- 172.21.15.89:0/3884303944 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe04035070 con 0x7efe18103650 2024-09-21T03:40:45.511 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:45.507+0000 7efe1f0cc640 1 -- 172.21.15.89:0/3884303944 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efe181a2140 con 0x7efe18103650 2024-09-21T03:40:45.513 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:45.510+0000 7efe2603b740 1 Processor -- start 2024-09-21T03:40:45.513 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:45.510+0000 7efe2603b740 1 -- start start 2024-09-21T03:40:45.513 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:45.510+0000 7efe15ffb640 1 -- 172.21.15.89:0/3884303944 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 20) ==== 154563+0+0 (secure 0 0 0) 0x7efe04042030 con 0x7efe18103650 2024-09-21T03:40:45.513 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:45.510+0000 7efe2603b740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563ccbba31b0 0x563ccbf4f260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:45.513 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:45.510+0000 7efe2603b740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563ccbb7a2f0 0x563ccbf4f8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:45.513 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:45.510+0000 7efe2603b740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x563ccbb824d0 0x563ccbf50420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:45.514 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:45.510+0000 7efe2603b740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x563ccbebd6f0 con 0x563ccbb7a2f0 2024-09-21T03:40:45.514 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:45.510+0000 7efe2603b740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x563ccbeacb20 con 0x563ccbba31b0 2024-09-21T03:40:45.514 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:45.510+0000 7efe2603b740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x563ccbeb0930 con 0x563ccbb824d0 2024-09-21T03:40:45.514 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:45.510+0000 7efe1ce68640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563ccbb7a2f0 0x563ccbf4f8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:45.514 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:45.510+0000 7efe1ce68640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563ccbb7a2f0 0x563ccbf4f8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:60640/0 (socket says 172.21.15.89:60640) 2024-09-21T03:40:45.514 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:45.510+0000 7efe1ce68640 1 -- 172.21.15.89:0/3543187291 learned_addr learned my addr 172.21.15.89:0/3543187291 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-21T03:40:45.514 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:45.510+0000 7efe1d669640 1 --2- 172.21.15.89:0/3543187291 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563ccbba31b0 0x563ccbf4f260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:45.514 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:45.510+0000 7efe17fff640 1 --2- 172.21.15.89:0/3543187291 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x563ccbb824d0 0x563ccbf50420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:45.514 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:45.511+0000 7efe1ce68640 1 -- 172.21.15.89:0/3543187291 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x563ccbb824d0 msgr2=0x563ccbf50420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:40:45.514 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:45.511+0000 7efe1ce68640 1 --2- 172.21.15.89:0/3543187291 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x563ccbb824d0 0x563ccbf50420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:45.514 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:45.511+0000 7efe1ce68640 1 -- 172.21.15.89:0/3543187291 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563ccbba31b0 msgr2=0x563ccbf4f260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:40:45.514 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:45.511+0000 7efe1ce68640 1 --2- 172.21.15.89:0/3543187291 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563ccbba31b0 0x563ccbf4f260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:45.514 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:45.511+0000 7efe1ce68640 1 -- 172.21.15.89:0/3543187291 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563ccbe72530 con 0x563ccbb7a2f0 2024-09-21T03:40:45.514 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:45.511+0000 7efe1d669640 1 --2- 172.21.15.89:0/3543187291 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563ccbba31b0 0x563ccbf4f260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:40:45.515 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:45.511+0000 7efe1ce68640 1 --2- 172.21.15.89:0/3543187291 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563ccbb7a2f0 0x563ccbf4f8d0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7efe0c0104d0 tx=0x7efe0c010990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=255b1575684fbc6f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:45.515 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:45.511+0000 7efe02ffd640 1 -- 172.21.15.89:0/3543187291 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe0c004220 con 0x563ccbb7a2f0 2024-09-21T03:40:45.517 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:45.511+0000 7efe15ffb640 1 --2- 172.21.15.89:0/3884303944 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7efe080b6620 0x7efe080b8b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:45.517 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:45.511+0000 7efe02ffd640 1 -- 172.21.15.89:0/3543187291 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7efe0c0043c0 con 0x563ccbb7a2f0 2024-09-21T03:40:45.517 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:45.511+0000 7efe02ffd640 1 -- 172.21.15.89:0/3543187291 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe0c011940 con 0x563ccbb7a2f0 2024-09-21T03:40:45.517 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:45.512+0000 7efe1d669640 1 --2- 172.21.15.89:0/3884303944 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7efe080b6620 0x7efe080b8b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:45.517 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:45.512+0000 7efe15ffb640 1 -- 172.21.15.89:0/3884303944 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 9121+0+0 (secure 0 0 0) 0x7efe04107d20 con 0x7efe18103650 2024-09-21T03:40:45.517 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:45.512+0000 7efe2603b740 1 -- 172.21.15.89:0/3543187291 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563ccbb7a2f0 msgr2=0x563ccbf4f8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:45.517 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:45.512+0000 7efe1d669640 1 --2- 172.21.15.89:0/3884303944 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7efe080b6620 0x7efe080b8b30 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7efe10005fd0 tx=0x7efe10009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:45.517 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:45.512+0000 7efe2603b740 1 --2- 172.21.15.89:0/3543187291 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563ccbb7a2f0 0x563ccbf4f8d0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7efe0c0104d0 tx=0x7efe0c010990 comp rx=0 tx=0).stop 2024-09-21T03:40:45.517 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:45.513+0000 7efe2603b740 1 -- 172.21.15.89:0/3543187291 shutdown_connections 2024-09-21T03:40:45.517 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:45.513+0000 7efe2603b740 1 --2- 172.21.15.89:0/3543187291 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x563ccbb824d0 0x563ccbf50420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:45.517 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:45.513+0000 7efe2603b740 1 --2- 172.21.15.89:0/3543187291 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563ccbb7a2f0 0x563ccbf4f8d0 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:45.518 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:45.513+0000 7efe2603b740 1 --2- 172.21.15.89:0/3543187291 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563ccbba31b0 0x563ccbf4f260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:45.518 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:45.513+0000 7efe2603b740 1 -- 172.21.15.89:0/3543187291 >> 172.21.15.89:0/3543187291 conn(0x563ccbb97300 msgr2=0x563ccbf468e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:40:45.518 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:45.513+0000 7efe2603b740 1 -- 172.21.15.89:0/3543187291 shutdown_connections 2024-09-21T03:40:45.518 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:45.513+0000 7efe2603b740 1 -- 172.21.15.89:0/3543187291 wait complete. 2024-09-21T03:40:45.518 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:45.513+0000 7efe2603b740 1 -- start start 2024-09-21T03:40:45.518 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:45.513+0000 7efe2603b740 1 Processor -- start 2024-09-21T03:40:45.518 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:45.514+0000 7efe2603b740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563ccbb824d0 0x563ccbf58b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:45.518 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:45.514+0000 7efe2603b740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563ccbba31b0 0x563ccbf591d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:45.518 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:45.514+0000 7efe2603b740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x563ccbb61d80 0x563ccbf59d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:45.518 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:45.514+0000 7efe2603b740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x563ccbde4ab0 con 0x563ccbb824d0 2024-09-21T03:40:45.518 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:45.514+0000 7efe2603b740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x563ccbe52730 con 0x563ccbba31b0 2024-09-21T03:40:45.518 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:45.514+0000 7efe2603b740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x563ccbf50f40 con 0x563ccbb61d80 2024-09-21T03:40:45.518 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:45.514+0000 7efe2603b740 10 client.0 _ll_register_callbacks cb 0x563ccbea66b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:40:45.518 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:45.514+0000 7efe1ce68640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563ccbb824d0 0x563ccbf58b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:45.519 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:45.514+0000 7efe1ce68640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563ccbb824d0 0x563ccbf58b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:60654/0 (socket says 172.21.15.89:60654) 2024-09-21T03:40:45.519 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:45.514+0000 7efe1ce68640 1 -- 172.21.15.89:0/854354491 learned_addr learned my addr 172.21.15.89:0/854354491 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-21T03:40:45.519 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:45.514+0000 7efe1d669640 1 --2- 172.21.15.89:0/854354491 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x563ccbb61d80 0x563ccbf59d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:45.519 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:45.514+0000 7efe17fff640 1 --2- 172.21.15.89:0/854354491 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563ccbba31b0 0x563ccbf591d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:45.519 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:45.514+0000 7efe1ce68640 1 -- 172.21.15.89:0/854354491 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x563ccbb61d80 msgr2=0x563ccbf59d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:40:45.519 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:45.514+0000 7efe1ce68640 1 --2- 172.21.15.89:0/854354491 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x563ccbb61d80 0x563ccbf59d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:45.519 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:45.514+0000 7efe1ce68640 1 -- 172.21.15.89:0/854354491 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563ccbba31b0 msgr2=0x563ccbf591d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:40:45.519 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:45.514+0000 7efe1ce68640 1 --2- 172.21.15.89:0/854354491 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563ccbba31b0 0x563ccbf591d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:45.519 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:45.514+0000 7efe1ce68640 1 -- 172.21.15.89:0/854354491 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563ccbc0e790 con 0x563ccbb824d0 2024-09-21T03:40:45.519 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:45.514+0000 7efe17fff640 1 --2- 172.21.15.89:0/854354491 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563ccbba31b0 0x563ccbf591d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:40:45.519 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:45.514+0000 7efe1ce68640 1 --2- 172.21.15.89:0/854354491 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563ccbb824d0 0x563ccbf58b10 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7efe0c0104a0 tx=0x7efe0c00be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0612cfb361e33aa server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:45.519 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:45.514+0000 7efdf7fff640 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2024-09-21T03:40:45.519 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:45.515+0000 7efdf7fff640 1 -- 172.21.15.89:0/854354491 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe0c005550 con 0x563ccbb824d0 2024-09-21T03:40:45.519 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:45.515+0000 7efdf7fff640 1 -- 172.21.15.89:0/854354491 <== mon.0 v2:172.21.15.89:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 9121+0+0 (secure 0 0 0) 0x7efe0c0268d0 con 0x563ccbb824d0 2024-09-21T03:40:45.520 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:45.515+0000 7efe2603b740 1 -- 172.21.15.89:0/854354491 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efe0c0043c0 con 0x563ccbb824d0 2024-09-21T03:40:45.521 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:45.515+0000 7efdf7fff640 1 -- 172.21.15.89:0/854354491 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7efe0c038050 con 0x563ccbb824d0 2024-09-21T03:40:45.522 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:45.515+0000 7efe2603b740 10 client.15687 fetch_fsmap learned FSMap version 26 2024-09-21T03:40:45.522 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:45.515+0000 7efe2603b740 1 -- 172.21.15.89:0/854354491 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x563ccbc87620 con 0x563ccbb824d0 2024-09-21T03:40:45.522 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:45.515+0000 7efdf7fff640 1 -- 172.21.15.89:0/854354491 <== mon.0 v2:172.21.15.89:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7efe0c0043c0 con 0x563ccbb824d0 2024-09-21T03:40:45.522 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:45.515+0000 7efdf7fff640 1 -- 172.21.15.89:0/854354491 <== mon.0 v2:172.21.15.89:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7efe0c0046c0 con 0x563ccbb824d0 2024-09-21T03:40:45.522 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:45.515+0000 7efe2603b740 10 client.15687 fetch_fsmap finished waiting for FSMap version 26 2024-09-21T03:40:45.522 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:45.515+0000 7efe2603b740 10 client.15687 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T03:40:45.522 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:45.515+0000 7efe2603b740 20 client.15687 populate_metadata read hostname 'smithi089' 2024-09-21T03:40:45.522 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:45.515+0000 7efe2603b740 1 --2- 172.21.15.89:0/854354491 >> [v2:172.21.15.110:6838/132366709,v1:172.21.15.110:6839/132366709] conn(0x563ccbb371c0 0x563ccbf4dd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:45.522 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:45.515+0000 7efe2603b740 4 client.15687 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:40:45.522 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:45.515+0000 7efe2603b740 1 -- 172.21.15.89:0/854354491 --> [v2:172.21.15.110:6838/132366709,v1:172.21.15.110:6839/132366709] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563ccbebf640 con 0x563ccbb371c0 2024-09-21T03:40:45.522 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:45.516+0000 7efe17fff640 1 --2- 172.21.15.89:0/854354491 >> [v2:172.21.15.110:6838/132366709,v1:172.21.15.110:6839/132366709] conn(0x563ccbb371c0 0x563ccbf4dd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:45.522 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:45.516+0000 7efe17fff640 1 --2- 172.21.15.89:0/854354491 >> [v2:172.21.15.110:6838/132366709,v1:172.21.15.110:6839/132366709] conn(0x563ccbb371c0 0x563ccbf4dd60 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5ea90f415fae5521 server_cookie=8345a14613a9c26e in_seq=0 out_seq=0 2024-09-21T03:40:45.522 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:45.516+0000 7efdf7fff640 10 client.15687 ms_handle_connect on v2:172.21.15.110:6838/132366709 2024-09-21T03:40:45.523 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:45.517+0000 7efdf7fff640 1 -- 172.21.15.89:0/854354491 <== mds.0 v2:172.21.15.110:6838/132366709 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x563ccbebf640 con 0x563ccbb371c0 2024-09-21T03:40:45.523 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:45.517+0000 7efdf7fff640 10 client.15687 handle_command_reply: tid=0 2024-09-21T03:40:45.523 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:45.517+0000 7efe2603b740 1 client.15687 shutdown 2024-09-21T03:40:45.523 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:45.517+0000 7efe2603b740 1 -- 172.21.15.89:0/854354491 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563ccbb824d0 msgr2=0x563ccbf58b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:45.523 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:45.517+0000 7efe2603b740 1 --2- 172.21.15.89:0/854354491 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563ccbb824d0 0x563ccbf58b10 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7efe0c0104a0 tx=0x7efe0c00be30 comp rx=0 tx=0).stop 2024-09-21T03:40:45.523 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:45.517+0000 7efe2603b740 1 -- 172.21.15.89:0/854354491 shutdown_connections 2024-09-21T03:40:45.523 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:45.517+0000 7efe2603b740 1 --2- 172.21.15.89:0/854354491 >> [v2:172.21.15.110:6838/132366709,v1:172.21.15.110:6839/132366709] conn(0x563ccbb371c0 0x563ccbf4dd60 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:45.523 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:45.517+0000 7efe2603b740 1 --2- 172.21.15.89:0/854354491 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563ccbba31b0 0x563ccbf591d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:45.523 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:45.517+0000 7efdf7fff640 0 client.15687 ms_handle_reset on v2:172.21.15.110:6838/132366709 2024-09-21T03:40:45.523 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:45.517+0000 7efe2603b740 1 --2- 172.21.15.89:0/854354491 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x563ccbb61d80 0x563ccbf59d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:45.523 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:45.517+0000 7efe2603b740 1 --2- 172.21.15.89:0/854354491 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563ccbb824d0 0x563ccbf58b10 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:45.523 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:45.517+0000 7efe2603b740 1 -- 172.21.15.89:0/854354491 >> 172.21.15.89:0/854354491 conn(0x563ccbb97300 msgr2=0x563ccbf44ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:40:45.523 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:45.517+0000 7efe2603b740 1 -- 172.21.15.89:0/854354491 shutdown_connections 2024-09-21T03:40:45.523 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:45.517+0000 7efe2603b740 1 -- 172.21.15.89:0/854354491 wait complete. 2024-09-21T03:40:45.524 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:45.518+0000 7efe2603b740 20 client.15687 trim_cache size 0 max 16384 2024-09-21T03:40:45.540 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:45.537+0000 7efe2603b740 1 Processor -- start 2024-09-21T03:40:45.541 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:45.537+0000 7efe2603b740 1 -- start start 2024-09-21T03:40:45.541 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:45.538+0000 7efe2603b740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563ccbb371c0 0x563ccbf44ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:45.541 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:45.538+0000 7efe2603b740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563ccbb61d80 0x563ccbf500f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:45.541 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:45.538+0000 7efe2603b740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x563ccbb824d0 0x563ccbf415c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:45.541 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:45.538+0000 7efe2603b740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7efe181acba0 con 0x563ccbb371c0 2024-09-21T03:40:45.541 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:45.538+0000 7efe2603b740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7efe181aa470 con 0x563ccbb61d80 2024-09-21T03:40:45.541 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:45.538+0000 7efe2603b740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7efe181a82c0 con 0x563ccbb824d0 2024-09-21T03:40:45.541 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:45.538+0000 7efe1ce68640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563ccbb371c0 0x563ccbf44ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:45.541 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:45.538+0000 7efe1ce68640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563ccbb371c0 0x563ccbf44ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:60662/0 (socket says 172.21.15.89:60662) 2024-09-21T03:40:45.541 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:45.538+0000 7efe1ce68640 1 -- 172.21.15.89:0/3618989479 learned_addr learned my addr 172.21.15.89:0/3618989479 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-21T03:40:45.541 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:45.538+0000 7efe1d669640 1 --2- 172.21.15.89:0/3618989479 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x563ccbb824d0 0x563ccbf415c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:45.541 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:45.538+0000 7efe17fff640 1 --2- 172.21.15.89:0/3618989479 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563ccbb61d80 0x563ccbf500f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:45.541 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:45.538+0000 7efe1ce68640 1 -- 172.21.15.89:0/3618989479 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x563ccbb824d0 msgr2=0x563ccbf415c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:40:45.541 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:45.538+0000 7efe1ce68640 1 --2- 172.21.15.89:0/3618989479 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x563ccbb824d0 0x563ccbf415c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:45.541 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:45.538+0000 7efe1ce68640 1 -- 172.21.15.89:0/3618989479 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563ccbb61d80 msgr2=0x563ccbf500f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:40:45.542 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:45.538+0000 7efe1ce68640 1 --2- 172.21.15.89:0/3618989479 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563ccbb61d80 0x563ccbf500f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:45.542 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:45.538+0000 7efe1ce68640 1 -- 172.21.15.89:0/3618989479 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563ccbf5a0b0 con 0x563ccbb371c0 2024-09-21T03:40:45.542 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:45.539+0000 7efe1ce68640 1 --2- 172.21.15.89:0/3618989479 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563ccbb371c0 0x563ccbf44ea0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7efe0c0103f0 tx=0x7efe0c004000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8cd781d8fdbfefbd server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:45.545 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:45.539+0000 7efe027fc640 1 -- 172.21.15.89:0/3618989479 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe0c041070 con 0x563ccbb371c0 2024-09-21T03:40:45.545 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:45.539+0000 7efe027fc640 1 -- 172.21.15.89:0/3618989479 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7efe0c02a070 con 0x563ccbb371c0 2024-09-21T03:40:45.545 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:45.539+0000 7efe027fc640 1 -- 172.21.15.89:0/3618989479 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe0c0268d0 con 0x563ccbb371c0 2024-09-21T03:40:45.545 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:45.540+0000 7efe2603b740 1 -- 172.21.15.89:0/3618989479 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563ccbb371c0 msgr2=0x563ccbf44ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:45.545 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:45.540+0000 7efe2603b740 1 --2- 172.21.15.89:0/3618989479 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563ccbb371c0 0x563ccbf44ea0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7efe0c0103f0 tx=0x7efe0c004000 comp rx=0 tx=0).stop 2024-09-21T03:40:45.545 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:45.540+0000 7efe2603b740 1 -- 172.21.15.89:0/3618989479 shutdown_connections 2024-09-21T03:40:45.546 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:45.540+0000 7efe2603b740 1 --2- 172.21.15.89:0/3618989479 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563ccbb61d80 0x563ccbf500f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:45.546 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:45.540+0000 7efe2603b740 1 --2- 172.21.15.89:0/3618989479 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x563ccbb824d0 0x563ccbf415c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:45.546 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:45.540+0000 7efe2603b740 1 --2- 172.21.15.89:0/3618989479 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563ccbb371c0 0x563ccbf44ea0 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:45.546 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:45.540+0000 7efe2603b740 1 -- 172.21.15.89:0/3618989479 >> 172.21.15.89:0/3618989479 conn(0x563ccbb97300 msgr2=0x563ccbf57d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:40:45.546 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:45.540+0000 7efe2603b740 1 -- 172.21.15.89:0/3618989479 shutdown_connections 2024-09-21T03:40:45.546 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:45.540+0000 7efe2603b740 1 -- 172.21.15.89:0/3618989479 wait complete. 2024-09-21T03:40:45.546 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:45.540+0000 7efe2603b740 1 -- start start 2024-09-21T03:40:45.546 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:45.540+0000 7efe2603b740 1 Processor -- start 2024-09-21T03:40:45.546 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:45.541+0000 7efe2603b740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x563ccbb61d80 0x563ccbf68370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:45.546 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:45.541+0000 7efe2603b740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563ccbb824d0 0x563ccbf66c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:45.546 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:45.541+0000 7efe2603b740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563ccbba31b0 0x563ccbf67180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:45.546 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:45.541+0000 7efe2603b740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7efe181a8930 con 0x563ccbba31b0 2024-09-21T03:40:45.546 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:45.541+0000 7efe2603b740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7efe181a9da0 con 0x563ccbb824d0 2024-09-21T03:40:45.546 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:45.541+0000 7efe2603b740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7efe181a9000 con 0x563ccbb61d80 2024-09-21T03:40:45.546 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:45.541+0000 7efe2603b740 10 client.0 _ll_register_callbacks cb 0x7efe0c00bf50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:40:45.547 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:45.541+0000 7efe1d669640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563ccbba31b0 0x563ccbf67180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:45.547 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:45.541+0000 7efe1d669640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563ccbba31b0 0x563ccbf67180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:60668/0 (socket says 172.21.15.89:60668) 2024-09-21T03:40:45.547 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:45.541+0000 7efe1d669640 1 -- 172.21.15.89:0/2074792629 learned_addr learned my addr 172.21.15.89:0/2074792629 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-21T03:40:45.547 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:45.541+0000 7efe1ce68640 1 --2- 172.21.15.89:0/2074792629 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x563ccbb61d80 0x563ccbf68370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:45.547 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:45.541+0000 7efe17fff640 1 --2- 172.21.15.89:0/2074792629 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563ccbb824d0 0x563ccbf66c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:45.547 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:45.541+0000 7efe1d669640 1 -- 172.21.15.89:0/2074792629 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x563ccbb61d80 msgr2=0x563ccbf68370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:40:45.547 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:45.541+0000 7efe1d669640 1 --2- 172.21.15.89:0/2074792629 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x563ccbb61d80 0x563ccbf68370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:45.547 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:45.541+0000 7efe1d669640 1 -- 172.21.15.89:0/2074792629 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563ccbb824d0 msgr2=0x563ccbf66c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:40:45.547 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:45.541+0000 7efe1d669640 1 --2- 172.21.15.89:0/2074792629 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563ccbb824d0 0x563ccbf66c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:45.547 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:45.541+0000 7efe1d669640 1 -- 172.21.15.89:0/2074792629 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563ccbf67910 con 0x563ccbba31b0 2024-09-21T03:40:45.547 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:45.541+0000 7efe17fff640 1 --2- 172.21.15.89:0/2074792629 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563ccbb824d0 0x563ccbf66c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:40:45.547 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:45.541+0000 7efe1ce68640 1 --2- 172.21.15.89:0/2074792629 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x563ccbb61d80 0x563ccbf68370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:40:45.547 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:45.542+0000 7efe1d669640 1 --2- 172.21.15.89:0/2074792629 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563ccbba31b0 0x563ccbf67180 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7efe10003b00 tx=0x7efe10004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22f968b93caf97a6 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:45.547 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:45.542+0000 7efdf7fff640 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2024-09-21T03:40:45.548 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:45.542+0000 7efdf7fff640 1 -- 172.21.15.89:0/2074792629 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe1002f060 con 0x563ccbba31b0 2024-09-21T03:40:45.548 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:45.542+0000 7efdf7fff640 1 -- 172.21.15.89:0/2074792629 <== mon.0 v2:172.21.15.89:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 9121+0+0 (secure 0 0 0) 0x7efe1001f430 con 0x563ccbba31b0 2024-09-21T03:40:45.548 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:45.542+0000 7efe2603b740 1 -- 172.21.15.89:0/2074792629 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efe0c02a070 con 0x563ccbba31b0 2024-09-21T03:40:45.548 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:45.542+0000 7efdf7fff640 1 -- 172.21.15.89:0/2074792629 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7efe10036050 con 0x563ccbba31b0 2024-09-21T03:40:45.548 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:45.542+0000 7efe2603b740 10 client.15693 fetch_fsmap learned FSMap version 26 2024-09-21T03:40:45.548 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:45.542+0000 7efe2603b740 1 -- 172.21.15.89:0/2074792629 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x563ccbfcbf40 con 0x563ccbba31b0 2024-09-21T03:40:45.548 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:45.542+0000 7efdf7fff640 1 -- 172.21.15.89:0/2074792629 <== mon.0 v2:172.21.15.89:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7efe0c02a070 con 0x563ccbba31b0 2024-09-21T03:40:45.548 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:45.543+0000 7efdf7fff640 1 -- 172.21.15.89:0/2074792629 <== mon.0 v2:172.21.15.89:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7efe1001e5d0 con 0x563ccbba31b0 2024-09-21T03:40:45.548 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:45.543+0000 7efe2603b740 10 client.15693 fetch_fsmap finished waiting for FSMap version 26 2024-09-21T03:40:45.548 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:45.543+0000 7efe2603b740 10 client.15693 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T03:40:45.548 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:45.543+0000 7efe2603b740 20 client.15693 populate_metadata read hostname 'smithi089' 2024-09-21T03:40:45.548 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:45.543+0000 7efe2603b740 1 --2- 172.21.15.89:0/2074792629 >> [v2:172.21.15.110:6838/132366709,v1:172.21.15.110:6839/132366709] conn(0x563ccbfcc900 0x563ccbfced20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:45.548 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:45.543+0000 7efe2603b740 4 client.15693 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T03:40:45.549 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:45.543+0000 7efe2603b740 1 -- 172.21.15.89:0/2074792629 --> [v2:172.21.15.110:6838/132366709,v1:172.21.15.110:6839/132366709] -- command(tid 0: {"prefix": "flush journal"}) -- 0x563ccbebb0f0 con 0x563ccbfcc900 2024-09-21T03:40:45.549 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:45.543+0000 7efe1ce68640 1 --2- 172.21.15.89:0/2074792629 >> [v2:172.21.15.110:6838/132366709,v1:172.21.15.110:6839/132366709] conn(0x563ccbfcc900 0x563ccbfced20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:45.549 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:45.543+0000 7efdf7fff640 10 client.15693 ms_handle_connect on v2:172.21.15.110:6838/132366709 2024-09-21T03:40:45.549 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:45.543+0000 7efe1ce68640 1 --2- 172.21.15.89:0/2074792629 >> [v2:172.21.15.110:6838/132366709,v1:172.21.15.110:6839/132366709] conn(0x563ccbfcc900 0x563ccbfced20 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=14f4e880e1c58f70 server_cookie=4d1e3113bdd87fd6 in_seq=0 out_seq=0 2024-09-21T03:40:45.557 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:45.554+0000 7efdf7fff640 1 -- 172.21.15.89:0/2074792629 <== mds.0 v2:172.21.15.110:6838/132366709 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x563ccbebb0f0 con 0x563ccbfcc900 2024-09-21T03:40:45.557 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:45.554+0000 7efdf7fff640 10 client.15693 handle_command_reply: tid=0 2024-09-21T03:40:45.557 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:45.554+0000 7efe2603b740 1 client.15693 shutdown 2024-09-21T03:40:45.557 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:45.555+0000 7efe2603b740 1 -- 172.21.15.89:0/2074792629 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563ccbba31b0 msgr2=0x563ccbf67180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:45.557 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:45.555+0000 7efe2603b740 1 --2- 172.21.15.89:0/2074792629 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563ccbba31b0 0x563ccbf67180 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7efe10003b00 tx=0x7efe10004a40 comp rx=0 tx=0).stop 2024-09-21T03:40:45.557 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:45.555+0000 7efe2603b740 1 -- 172.21.15.89:0/2074792629 shutdown_connections 2024-09-21T03:40:45.558 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:45.555+0000 7efe2603b740 1 --2- 172.21.15.89:0/2074792629 >> [v2:172.21.15.110:6838/132366709,v1:172.21.15.110:6839/132366709] conn(0x563ccbfcc900 0x563ccbfced20 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:45.558 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:45.555+0000 7efe2603b740 1 --2- 172.21.15.89:0/2074792629 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563ccbb824d0 0x563ccbf66c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:45.558 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:45.555+0000 7efe2603b740 1 --2- 172.21.15.89:0/2074792629 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563ccbba31b0 0x563ccbf67180 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:45.558 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:45.555+0000 7efe2603b740 1 --2- 172.21.15.89:0/2074792629 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x563ccbb61d80 0x563ccbf68370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:45.558 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:45.555+0000 7efe2603b740 1 -- 172.21.15.89:0/2074792629 >> 172.21.15.89:0/2074792629 conn(0x563ccbb97300 msgr2=0x563ccbf500f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:40:45.558 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:45.555+0000 7efdf7fff640 0 client.15693 ms_handle_reset on v2:172.21.15.110:6838/132366709 2024-09-21T03:40:45.558 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:45.555+0000 7efe2603b740 1 -- 172.21.15.89:0/2074792629 shutdown_connections 2024-09-21T03:40:45.558 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:45.555+0000 7efe2603b740 1 -- 172.21.15.89:0/2074792629 wait complete. 2024-09-21T03:40:45.558 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:45.555+0000 7efe2603b740 20 client.15693 trim_cache size 0 max 16384 2024-09-21T03:40:45.558 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stdout:{ 2024-09-21T03:40:45.558 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stdout: "message": "", 2024-09-21T03:40:45.559 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stdout: "return_code": 0 2024-09-21T03:40:45.559 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stdout:} 2024-09-21T03:40:45.559 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:45.556+0000 7efe1f0cc640 1 -- 172.21.15.89:0/3884303944 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7efe080b6620 msgr2=0x7efe080b8b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:45.559 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:45.556+0000 7efe1f0cc640 1 --2- 172.21.15.89:0/3884303944 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7efe080b6620 0x7efe080b8b30 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7efe10005fd0 tx=0x7efe10009500 comp rx=0 tx=0).stop 2024-09-21T03:40:45.559 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:45.556+0000 7efe1f0cc640 1 -- 172.21.15.89:0/3884303944 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efe18103650 msgr2=0x7efe1819ec20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:45.559 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:45.556+0000 7efe1f0cc640 1 --2- 172.21.15.89:0/3884303944 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efe18103650 0x7efe1819ec20 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7efe04009230 tx=0x7efe040042f0 comp rx=0 tx=0).stop 2024-09-21T03:40:45.560 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:45.556+0000 7efe1f0cc640 1 -- 172.21.15.89:0/3884303944 shutdown_connections 2024-09-21T03:40:45.560 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:45.556+0000 7efe1f0cc640 1 --2- 172.21.15.89:0/3884303944 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7efe080b6620 0x7efe080b8b30 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:45.560 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:45.556+0000 7efe1f0cc640 1 --2- 172.21.15.89:0/3884303944 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7efe18103ff0 0x7efe181a1170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:45.560 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:45.556+0000 7efe1f0cc640 1 --2- 172.21.15.89:0/3884303944 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efe18103650 0x7efe1819ec20 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:45.560 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:45.556+0000 7efe1f0cc640 1 --2- 172.21.15.89:0/3884303944 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7efe181025d0 0x7efe1806be10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:45.560 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:45.556+0000 7efe1f0cc640 1 -- 172.21.15.89:0/3884303944 >> 172.21.15.89:0/3884303944 conn(0x7efe180fdf30 msgr2=0x7efe18106950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:40:45.560 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:45.556+0000 7efe1f0cc640 1 -- 172.21.15.89:0/3884303944 shutdown_connections 2024-09-21T03:40:45.560 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:45.556+0000 7efe1f0cc640 1 -- 172.21.15.89:0/3884303944 wait complete. 2024-09-21T03:40:45.711 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:40:45 smithi114 ceph-mon[34140]: pgmap v400: 129 pgs: 129 active+clean; 57 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 4.7 MiB/s wr, 8 op/s 2024-09-21T03:40:45.711 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:40:45 smithi114 ceph-mon[34140]: from='client.? 172.21.15.89:0/3303564755' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:40:45.750 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:40:45 smithi110 ceph-mon[34000]: pgmap v400: 129 pgs: 129 active+clean; 57 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 4.7 MiB/s wr, 8 op/s 2024-09-21T03:40:45.750 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:40:45 smithi110 ceph-mon[34000]: from='client.? 172.21.15.89:0/3303564755' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:40:45.771 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:40:45 smithi089 ceph-mon[30928]: pgmap v400: 129 pgs: 129 active+clean; 57 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 4.7 MiB/s wr, 8 op/s 2024-09-21T03:40:45.771 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:40:45 smithi089 ceph-mon[30928]: from='client.? 172.21.15.89:0/3303564755' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:40:46.207 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24638 quiesce db / --timeout 90 --expiration 137.0 --await-for 45.0' 2024-09-21T03:40:46.207 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 7c374098-77c9-11ef-baf5-efdc52797490 -- timeout 105 ceph tell mds.24638 quiesce db / --timeout 90 --expiration 137.0 --await-for 45.0 2024-09-21T03:40:46.506 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/7c374098-77c9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:40:47.057 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:47.055+0000 7fa49d165640 1 -- 172.21.15.89:0/2324677039 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa498108d10 msgr2=0x7fa49810f540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:47.058 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:47.055+0000 7fa49d165640 1 --2- 172.21.15.89:0/2324677039 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa498108d10 0x7fa49810f540 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7fa490009ca0 tx=0x7fa49002a8c0 comp rx=0 tx=0).stop 2024-09-21T03:40:47.058 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:47.055+0000 7fa49d165640 1 -- 172.21.15.89:0/2324677039 shutdown_connections 2024-09-21T03:40:47.058 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:47.055+0000 7fa49d165640 1 --2- 172.21.15.89:0/2324677039 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa4981083c0 0x7fa4981087d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:47.058 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:47.055+0000 7fa49d165640 1 --2- 172.21.15.89:0/2324677039 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa498108d10 0x7fa49810f540 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:47.058 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:47.055+0000 7fa49d165640 1 --2- 172.21.15.89:0/2324677039 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa498102a80 0x7fa498102e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:47.058 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:47.055+0000 7fa49d165640 1 -- 172.21.15.89:0/2324677039 >> 172.21.15.89:0/2324677039 conn(0x7fa498079760 msgr2=0x7fa498079b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:40:47.058 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:47.055+0000 7fa49d165640 1 -- 172.21.15.89:0/2324677039 shutdown_connections 2024-09-21T03:40:47.058 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:47.055+0000 7fa49d165640 1 -- 172.21.15.89:0/2324677039 wait complete. 2024-09-21T03:40:47.060 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:47.056+0000 7fa49d165640 1 Processor -- start 2024-09-21T03:40:47.060 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:47.056+0000 7fa49d165640 1 -- start start 2024-09-21T03:40:47.060 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:47.056+0000 7fa49d165640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa498102a80 0x7fa49819b150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:47.060 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:47.056+0000 7fa49d165640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa4981083c0 0x7fa49819b690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:47.060 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:47.056+0000 7fa49d165640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa498108d10 0x7fa49819bbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:47.060 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:47.056+0000 7fa49d165640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fa498112880 con 0x7fa4981083c0 2024-09-21T03:40:47.060 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:47.056+0000 7fa49d165640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fa498112700 con 0x7fa498102a80 2024-09-21T03:40:47.060 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:47.056+0000 7fa49d165640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fa498112a00 con 0x7fa498108d10 2024-09-21T03:40:47.060 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:47.056+0000 7fa48ffff640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa4981083c0 0x7fa49819b690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:47.060 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:47.056+0000 7fa49759e640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa498108d10 0x7fa49819bbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:47.060 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:47.056+0000 7fa48ffff640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa4981083c0 0x7fa49819b690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:60682/0 (socket says 172.21.15.89:60682) 2024-09-21T03:40:47.060 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:47.056+0000 7fa48ffff640 1 -- 172.21.15.89:0/3262798382 learned_addr learned my addr 172.21.15.89:0/3262798382 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-21T03:40:47.061 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:47.056+0000 7fa496d9d640 1 --2- 172.21.15.89:0/3262798382 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa498102a80 0x7fa49819b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:47.061 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:47.057+0000 7fa48ffff640 1 -- 172.21.15.89:0/3262798382 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa498108d10 msgr2=0x7fa49819bbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:47.061 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:47.057+0000 7fa48ffff640 1 --2- 172.21.15.89:0/3262798382 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa498108d10 0x7fa49819bbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:47.061 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:47.057+0000 7fa48ffff640 1 -- 172.21.15.89:0/3262798382 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa498102a80 msgr2=0x7fa49819b150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:47.061 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:47.057+0000 7fa48ffff640 1 --2- 172.21.15.89:0/3262798382 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa498102a80 0x7fa49819b150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:47.061 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:47.057+0000 7fa48ffff640 1 -- 172.21.15.89:0/3262798382 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4981a6350 con 0x7fa4981083c0 2024-09-21T03:40:47.061 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:47.057+0000 7fa49759e640 1 --2- 172.21.15.89:0/3262798382 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa498108d10 0x7fa49819bbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:40:47.061 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:47.057+0000 7fa48ffff640 1 --2- 172.21.15.89:0/3262798382 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa4981083c0 0x7fa49819b690 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7fa48000c970 tx=0x7fa48000cd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:47.061 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:47.057+0000 7fa494d99640 1 -- 172.21.15.89:0/3262798382 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa480014070 con 0x7fa4981083c0 2024-09-21T03:40:47.061 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:47.057+0000 7fa49d165640 1 -- 172.21.15.89:0/3262798382 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa4981a6640 con 0x7fa4981083c0 2024-09-21T03:40:47.061 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:47.057+0000 7fa494d99640 1 -- 172.21.15.89:0/3262798382 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa480004440 con 0x7fa4981083c0 2024-09-21T03:40:47.061 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:47.057+0000 7fa494d99640 1 -- 172.21.15.89:0/3262798382 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa480003cd0 con 0x7fa4981083c0 2024-09-21T03:40:47.061 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:47.057+0000 7fa49d165640 1 -- 172.21.15.89:0/3262798382 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4981a6bd0 con 0x7fa4981083c0 2024-09-21T03:40:47.062 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:47.060+0000 7fa4a36d9740 1 Processor -- start 2024-09-21T03:40:47.062 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:47.060+0000 7fa4a36d9740 1 -- start start 2024-09-21T03:40:47.063 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:47.060+0000 7fa4a36d9740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ff5f855840 0x55ff5fbfc600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:47.063 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:47.060+0000 7fa4a36d9740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ff5f82c980 0x55ff5fbfcb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:47.063 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:47.060+0000 7fa4a36d9740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ff5f834b60 0x55ff5fbfd080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:47.063 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:47.060+0000 7fa4a36d9740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55ff5fb72b90 con 0x55ff5f82c980 2024-09-21T03:40:47.063 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:47.060+0000 7fa4a36d9740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55ff5fb61fc0 con 0x55ff5f855840 2024-09-21T03:40:47.064 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:47.060+0000 7fa4a36d9740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55ff5fb65dd0 con 0x55ff5f834b60 2024-09-21T03:40:47.064 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:47.060+0000 7fa494d99640 1 -- 172.21.15.89:0/3262798382 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 20) ==== 154563+0+0 (secure 0 0 0) 0x7fa480017070 con 0x7fa4981083c0 2024-09-21T03:40:47.064 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:47.061+0000 7fa496d9d640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ff5f82c980 0x55ff5fbfcb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:47.064 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:47.061+0000 7fa496d9d640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ff5f82c980 0x55ff5fbfcb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:60692/0 (socket says 172.21.15.89:60692) 2024-09-21T03:40:47.064 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:47.061+0000 7fa496d9d640 1 -- 172.21.15.89:0/829936330 learned_addr learned my addr 172.21.15.89:0/829936330 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-21T03:40:47.064 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:47.061+0000 7fa49759e640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ff5f855840 0x55ff5fbfc600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:47.064 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:47.061+0000 7fa48ffff640 1 --2- 172.21.15.89:0/829936330 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ff5f834b60 0x55ff5fbfd080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:47.064 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:47.061+0000 7fa49759e640 1 -- 172.21.15.89:0/829936330 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ff5f834b60 msgr2=0x55ff5fbfd080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:40:47.064 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:47.061+0000 7fa49759e640 1 --2- 172.21.15.89:0/829936330 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ff5f834b60 0x55ff5fbfd080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:47.064 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:47.061+0000 7fa49759e640 1 -- 172.21.15.89:0/829936330 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ff5f82c980 msgr2=0x55ff5fbfcb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:40:47.064 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:47.061+0000 7fa49759e640 1 --2- 172.21.15.89:0/829936330 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ff5f82c980 0x55ff5fbfcb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:47.064 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:47.061+0000 7fa49759e640 1 -- 172.21.15.89:0/829936330 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ff5fae9060 con 0x55ff5f855840 2024-09-21T03:40:47.064 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:47.061+0000 7fa49759e640 1 --2- 172.21.15.89:0/829936330 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ff5f855840 0x55ff5fbfc600 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fa490031040 tx=0x7fa490031aa0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d79837fe36aad8b4 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:47.066 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:47.062+0000 7fa48d7fa640 1 -- 172.21.15.89:0/829936330 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa490003970 con 0x55ff5f855840 2024-09-21T03:40:47.066 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:47.062+0000 7fa48d7fa640 1 -- 172.21.15.89:0/829936330 <== mon.2 v2:172.21.15.110:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa490036030 con 0x55ff5f855840 2024-09-21T03:40:47.066 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:47.062+0000 7fa48d7fa640 1 -- 172.21.15.89:0/829936330 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4900305c0 con 0x55ff5f855840 2024-09-21T03:40:47.066 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:47.062+0000 7fa494d99640 1 --2- 172.21.15.89:0/3262798382 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7fa4840b65d0 0x7fa4840b8ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:47.066 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:47.062+0000 7fa496d9d640 1 --2- 172.21.15.89:0/3262798382 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7fa4840b65d0 0x7fa4840b8ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:47.066 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:47.062+0000 7fa494d99640 1 -- 172.21.15.89:0/3262798382 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 9121+0+0 (secure 0 0 0) 0x7fa4800e71e0 con 0x7fa4981083c0 2024-09-21T03:40:47.066 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:47.063+0000 7fa4a36d9740 1 -- 172.21.15.89:0/829936330 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ff5f855840 msgr2=0x55ff5fbfc600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:47.066 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:47.063+0000 7fa496d9d640 1 --2- 172.21.15.89:0/3262798382 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7fa4840b65d0 0x7fa4840b8ae0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x55ff5fbfb280 tx=0x7fa48800a2d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:47.066 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:47.063+0000 7fa4a36d9740 1 --2- 172.21.15.89:0/829936330 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ff5f855840 0x55ff5fbfc600 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fa490031040 tx=0x7fa490031aa0 comp rx=0 tx=0).stop 2024-09-21T03:40:47.066 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:47.063+0000 7fa4a36d9740 1 -- 172.21.15.89:0/829936330 shutdown_connections 2024-09-21T03:40:47.066 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:47.063+0000 7fa4a36d9740 1 --2- 172.21.15.89:0/829936330 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ff5f834b60 0x55ff5fbfd080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:47.067 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:47.063+0000 7fa4a36d9740 1 --2- 172.21.15.89:0/829936330 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ff5f82c980 0x55ff5fbfcb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:47.067 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:47.063+0000 7fa4a36d9740 1 --2- 172.21.15.89:0/829936330 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ff5f855840 0x55ff5fbfc600 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:47.067 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:47.063+0000 7fa4a36d9740 1 -- 172.21.15.89:0/829936330 >> 172.21.15.89:0/829936330 conn(0x55ff5f849990 msgr2=0x55ff5fbfa4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:40:47.067 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:47.063+0000 7fa4a36d9740 1 -- 172.21.15.89:0/829936330 shutdown_connections 2024-09-21T03:40:47.067 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:47.063+0000 7fa4a36d9740 1 -- 172.21.15.89:0/829936330 wait complete. 2024-09-21T03:40:47.067 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:47.063+0000 7fa4a36d9740 1 -- start start 2024-09-21T03:40:47.067 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:47.064+0000 7fa4a36d9740 1 Processor -- start 2024-09-21T03:40:47.069 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:47.064+0000 7fa4a36d9740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ff5f82c980 0x55ff5fc08240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:47.069 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:47.064+0000 7fa4a36d9740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ff5f834b60 0x55ff5fc08780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:47.069 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:47.064+0000 7fa4a36d9740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ff5f855840 0x55ff5fc13ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:47.069 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:47.064+0000 7fa4a36d9740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55ff5f9c37a0 con 0x55ff5f82c980 2024-09-21T03:40:47.069 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:47.064+0000 7fa4a36d9740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55ff5fb070a0 con 0x55ff5f855840 2024-09-21T03:40:47.069 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:47.064+0000 7fa4a36d9740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55ff5fc0ac00 con 0x55ff5f834b60 2024-09-21T03:40:47.069 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:47.064+0000 7fa496d9d640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ff5f82c980 0x55ff5fc08240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:47.070 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:47.064+0000 7fa496d9d640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ff5f82c980 0x55ff5fc08240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:60708/0 (socket says 172.21.15.89:60708) 2024-09-21T03:40:47.070 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:47.064+0000 7fa496d9d640 1 -- 172.21.15.89:0/2594469997 learned_addr learned my addr 172.21.15.89:0/2594469997 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-21T03:40:47.070 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:47.064+0000 7fa4a36d9740 10 client.0 _ll_register_callbacks cb 0x55ff5fae43c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:40:47.070 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:47.064+0000 7fa48ffff640 1 --2- 172.21.15.89:0/2594469997 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ff5f834b60 0x55ff5fc08780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:47.070 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:47.064+0000 7fa49759e640 1 --2- 172.21.15.89:0/2594469997 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ff5f855840 0x55ff5fc13ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:47.070 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:47.064+0000 7fa496d9d640 1 -- 172.21.15.89:0/2594469997 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ff5f834b60 msgr2=0x55ff5fc08780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:40:47.070 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:47.064+0000 7fa496d9d640 1 --2- 172.21.15.89:0/2594469997 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ff5f834b60 0x55ff5fc08780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:47.070 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:47.064+0000 7fa496d9d640 1 -- 172.21.15.89:0/2594469997 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ff5f855840 msgr2=0x55ff5fc13ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:40:47.070 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:47.065+0000 7fa496d9d640 1 --2- 172.21.15.89:0/2594469997 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ff5f855840 0x55ff5fc13ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:47.070 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:47.065+0000 7fa496d9d640 1 -- 172.21.15.89:0/2594469997 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ff5faf5880 con 0x55ff5f82c980 2024-09-21T03:40:47.070 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:47.065+0000 7fa496d9d640 1 --2- 172.21.15.89:0/2594469997 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ff5f82c980 0x55ff5fc08240 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7fa48801cc00 tx=0x7fa488004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5592f5436ffdce63 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:47.070 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:47.065+0000 7fa47e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2024-09-21T03:40:47.070 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:47.065+0000 7fa47e7fc640 1 -- 172.21.15.89:0/2594469997 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa48802c060 con 0x55ff5f82c980 2024-09-21T03:40:47.071 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:47.065+0000 7fa47e7fc640 1 -- 172.21.15.89:0/2594469997 <== mon.0 v2:172.21.15.89:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 9121+0+0 (secure 0 0 0) 0x7fa488016030 con 0x55ff5f82c980 2024-09-21T03:40:47.071 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:47.065+0000 7fa4a36d9740 1 -- 172.21.15.89:0/2594469997 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa490036030 con 0x55ff5f82c980 2024-09-21T03:40:47.071 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:47.066+0000 7fa47e7fc640 1 -- 172.21.15.89:0/2594469997 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fa488033050 con 0x55ff5f82c980 2024-09-21T03:40:47.071 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:47.066+0000 7fa4a36d9740 10 client.15705 fetch_fsmap learned FSMap version 26 2024-09-21T03:40:47.071 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:47.066+0000 7fa4a36d9740 1 -- 172.21.15.89:0/2594469997 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55ff5f8cfb90 con 0x55ff5f82c980 2024-09-21T03:40:47.071 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:47.066+0000 7fa47e7fc640 1 -- 172.21.15.89:0/2594469997 <== mon.0 v2:172.21.15.89:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa490036030 con 0x55ff5f82c980 2024-09-21T03:40:47.071 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:47.066+0000 7fa47e7fc640 1 -- 172.21.15.89:0/2594469997 <== mon.0 v2:172.21.15.89:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fa48801a3a0 con 0x55ff5f82c980 2024-09-21T03:40:47.071 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:47.066+0000 7fa4a36d9740 10 client.15705 fetch_fsmap finished waiting for FSMap version 26 2024-09-21T03:40:47.071 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:47.066+0000 7fa4a36d9740 10 client.15705 resolve_mds: validated gid 24638 aka daemon mds.b 2024-09-21T03:40:47.071 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:47.066+0000 7fa4a36d9740 20 client.15705 populate_metadata read hostname 'smithi089' 2024-09-21T03:40:47.071 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:47.066+0000 7fa4a36d9740 1 --2- 172.21.15.89:0/2594469997 >> [v2:172.21.15.110:6838/132366709,v1:172.21.15.110:6839/132366709] conn(0x55ff5f8141c0 0x55ff5fc19fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:47.071 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:47.066+0000 7fa4a36d9740 4 client.15705 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:40:47.071 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:47.066+0000 7fa4a36d9740 1 -- 172.21.15.89:0/2594469997 --> [v2:172.21.15.110:6838/132366709,v1:172.21.15.110:6839/132366709] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ff5fb74ae0 con 0x55ff5f8141c0 2024-09-21T03:40:47.071 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:47.067+0000 7fa48ffff640 1 --2- 172.21.15.89:0/2594469997 >> [v2:172.21.15.110:6838/132366709,v1:172.21.15.110:6839/132366709] conn(0x55ff5f8141c0 0x55ff5fc19fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:47.071 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:47.067+0000 7fa48ffff640 1 --2- 172.21.15.89:0/2594469997 >> [v2:172.21.15.110:6838/132366709,v1:172.21.15.110:6839/132366709] conn(0x55ff5f8141c0 0x55ff5fc19fb0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d0d4d013c401c87b server_cookie=8835db52eecf3641 in_seq=0 out_seq=0 2024-09-21T03:40:47.071 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:47.067+0000 7fa47e7fc640 10 client.15705 ms_handle_connect on v2:172.21.15.110:6838/132366709 2024-09-21T03:40:47.072 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:47.068+0000 7fa47e7fc640 1 -- 172.21.15.89:0/2594469997 <== mds.0 v2:172.21.15.110:6838/132366709 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x7fa480086e50 con 0x55ff5f8141c0 2024-09-21T03:40:47.073 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:47.068+0000 7fa47e7fc640 10 client.15705 handle_command_reply: tid=0 2024-09-21T03:40:47.073 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:47.068+0000 7fa4a36d9740 1 client.15705 shutdown 2024-09-21T03:40:47.073 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:47.068+0000 7fa4a36d9740 1 -- 172.21.15.89:0/2594469997 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ff5f82c980 msgr2=0x55ff5fc08240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:47.073 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:47.068+0000 7fa4a36d9740 1 --2- 172.21.15.89:0/2594469997 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ff5f82c980 0x55ff5fc08240 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7fa48801cc00 tx=0x7fa488004730 comp rx=0 tx=0).stop 2024-09-21T03:40:47.073 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:47.068+0000 7fa4a36d9740 1 -- 172.21.15.89:0/2594469997 shutdown_connections 2024-09-21T03:40:47.073 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:47.068+0000 7fa4a36d9740 1 --2- 172.21.15.89:0/2594469997 >> [v2:172.21.15.110:6838/132366709,v1:172.21.15.110:6839/132366709] conn(0x55ff5f8141c0 0x55ff5fc19fb0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:47.073 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:47.068+0000 7fa4a36d9740 1 --2- 172.21.15.89:0/2594469997 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ff5f855840 0x55ff5fc13ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:47.073 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:47.068+0000 7fa4a36d9740 1 --2- 172.21.15.89:0/2594469997 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ff5f834b60 0x55ff5fc08780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:47.073 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:47.068+0000 7fa4a36d9740 1 --2- 172.21.15.89:0/2594469997 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ff5f82c980 0x55ff5fc08240 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:47.073 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:47.068+0000 7fa4a36d9740 1 -- 172.21.15.89:0/2594469997 >> 172.21.15.89:0/2594469997 conn(0x55ff5f849990 msgr2=0x55ff5fbff7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:40:47.073 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:47.068+0000 7fa47e7fc640 0 client.15705 ms_handle_reset on v2:172.21.15.110:6838/132366709 2024-09-21T03:40:47.073 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:47.068+0000 7fa4a36d9740 1 -- 172.21.15.89:0/2594469997 shutdown_connections 2024-09-21T03:40:47.073 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:47.068+0000 7fa4a36d9740 1 -- 172.21.15.89:0/2594469997 wait complete. 2024-09-21T03:40:47.073 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:47.068+0000 7fa4a36d9740 20 client.15705 trim_cache size 0 max 16384 2024-09-21T03:40:47.087 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:47.085+0000 7fa4a36d9740 1 Processor -- start 2024-09-21T03:40:47.087 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:47.085+0000 7fa4a36d9740 1 -- start start 2024-09-21T03:40:47.089 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:47.086+0000 7fa4a36d9740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ff5f834b60 0x55ff5fbfdd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:47.089 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:47.086+0000 7fa4a36d9740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ff5f855840 0x55ff5fc05230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:47.089 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:47.086+0000 7fa4a36d9740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ff5f7e9570 0x55ff5fc05770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:47.089 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:47.086+0000 7fa4a36d9740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fa4981b3180 con 0x55ff5f7e9570 2024-09-21T03:40:47.089 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:47.086+0000 7fa4a36d9740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fa4981b0a50 con 0x55ff5f834b60 2024-09-21T03:40:47.089 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:47.086+0000 7fa4a36d9740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fa4981ae8a0 con 0x55ff5f855840 2024-09-21T03:40:47.089 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:47.086+0000 7fa49759e640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ff5f7e9570 0x55ff5fc05770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:47.090 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:47.086+0000 7fa49759e640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ff5f7e9570 0x55ff5fc05770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:60724/0 (socket says 172.21.15.89:60724) 2024-09-21T03:40:47.090 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:47.086+0000 7fa49759e640 1 -- 172.21.15.89:0/3595858418 learned_addr learned my addr 172.21.15.89:0/3595858418 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-21T03:40:47.090 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:47.086+0000 7fa48ffff640 1 --2- 172.21.15.89:0/3595858418 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ff5f855840 0x55ff5fc05230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:47.090 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:47.086+0000 7fa496d9d640 1 --2- 172.21.15.89:0/3595858418 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ff5f834b60 0x55ff5fbfdd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:47.090 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:47.086+0000 7fa49759e640 1 -- 172.21.15.89:0/3595858418 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ff5f855840 msgr2=0x55ff5fc05230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:40:47.090 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:47.086+0000 7fa49759e640 1 --2- 172.21.15.89:0/3595858418 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ff5f855840 0x55ff5fc05230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:47.090 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:47.086+0000 7fa49759e640 1 -- 172.21.15.89:0/3595858418 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ff5f834b60 msgr2=0x55ff5fbfdd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:40:47.090 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:47.086+0000 7fa49759e640 1 --2- 172.21.15.89:0/3595858418 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ff5f834b60 0x55ff5fbfdd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:47.090 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:47.086+0000 7fa49759e640 1 -- 172.21.15.89:0/3595858418 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ff5fbf5ad0 con 0x55ff5f7e9570 2024-09-21T03:40:47.090 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:47.086+0000 7fa48ffff640 1 --2- 172.21.15.89:0/3595858418 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ff5f855840 0x55ff5fc05230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:40:47.090 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:47.087+0000 7fa49759e640 1 --2- 172.21.15.89:0/3595858418 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ff5f7e9570 0x55ff5fc05770 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7fa490009ca0 tx=0x7fa49002a8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89e5ad9e643ea44f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:47.090 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:47.087+0000 7fa48cff9640 1 -- 172.21.15.89:0/3595858418 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa49004a040 con 0x55ff5f7e9570 2024-09-21T03:40:47.090 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:47.087+0000 7fa48cff9640 1 -- 172.21.15.89:0/3595858418 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa490056070 con 0x55ff5f7e9570 2024-09-21T03:40:47.090 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:47.087+0000 7fa48cff9640 1 -- 172.21.15.89:0/3595858418 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4900305c0 con 0x55ff5f7e9570 2024-09-21T03:40:47.096 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:47.088+0000 7fa4a36d9740 1 -- 172.21.15.89:0/3595858418 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ff5f7e9570 msgr2=0x55ff5fc05770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:47.096 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:47.088+0000 7fa4a36d9740 1 --2- 172.21.15.89:0/3595858418 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ff5f7e9570 0x55ff5fc05770 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7fa490009ca0 tx=0x7fa49002a8c0 comp rx=0 tx=0).stop 2024-09-21T03:40:47.096 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:47.088+0000 7fa4a36d9740 1 -- 172.21.15.89:0/3595858418 shutdown_connections 2024-09-21T03:40:47.096 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:47.088+0000 7fa4a36d9740 1 --2- 172.21.15.89:0/3595858418 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ff5f7e9570 0x55ff5fc05770 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:47.096 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:47.088+0000 7fa4a36d9740 1 --2- 172.21.15.89:0/3595858418 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ff5f855840 0x55ff5fc05230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:47.096 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:47.088+0000 7fa4a36d9740 1 --2- 172.21.15.89:0/3595858418 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ff5f834b60 0x55ff5fbfdd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:47.096 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:47.088+0000 7fa4a36d9740 1 -- 172.21.15.89:0/3595858418 >> 172.21.15.89:0/3595858418 conn(0x55ff5f849990 msgr2=0x55ff5fbfd630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:40:47.096 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:47.088+0000 7fa4a36d9740 1 -- 172.21.15.89:0/3595858418 shutdown_connections 2024-09-21T03:40:47.096 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:47.088+0000 7fa4a36d9740 1 -- 172.21.15.89:0/3595858418 wait complete. 2024-09-21T03:40:47.096 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:47.088+0000 7fa4a36d9740 1 -- start start 2024-09-21T03:40:47.097 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:47.088+0000 7fa4a36d9740 1 Processor -- start 2024-09-21T03:40:47.097 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:47.089+0000 7fa4a36d9740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ff5f7e9570 0x55ff5fc81da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:47.097 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:47.089+0000 7fa4a36d9740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ff5f834b60 0x55ff5fc82360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:47.097 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:47.089+0000 7fa4a36d9740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ff5f855840 0x55ff5fc0d750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:47.097 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:47.089+0000 7fa4a36d9740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fa4981aef10 con 0x55ff5f7e9570 2024-09-21T03:40:47.097 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:47.089+0000 7fa4a36d9740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fa4981b0380 con 0x55ff5f834b60 2024-09-21T03:40:47.097 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:47.089+0000 7fa4a36d9740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fa4981af5e0 con 0x55ff5f855840 2024-09-21T03:40:47.097 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:47.089+0000 7fa4a36d9740 10 client.0 _ll_register_callbacks cb 0x7fa48800ab20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:40:47.097 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:47.089+0000 7fa496d9d640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ff5f7e9570 0x55ff5fc81da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:47.097 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:47.089+0000 7fa496d9d640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ff5f7e9570 0x55ff5fc81da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:60740/0 (socket says 172.21.15.89:60740) 2024-09-21T03:40:47.097 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:47.089+0000 7fa496d9d640 1 -- 172.21.15.89:0/2601408470 learned_addr learned my addr 172.21.15.89:0/2601408470 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-21T03:40:47.097 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:47.089+0000 7fa49759e640 1 --2- 172.21.15.89:0/2601408470 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ff5f855840 0x55ff5fc0d750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:47.097 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:47.089+0000 7fa48ffff640 1 --2- 172.21.15.89:0/2601408470 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ff5f834b60 0x55ff5fc82360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:47.098 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:47.089+0000 7fa496d9d640 1 -- 172.21.15.89:0/2601408470 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ff5f855840 msgr2=0x55ff5fc0d750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:40:47.098 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:47.089+0000 7fa496d9d640 1 --2- 172.21.15.89:0/2601408470 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ff5f855840 0x55ff5fc0d750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:47.098 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:47.089+0000 7fa496d9d640 1 -- 172.21.15.89:0/2601408470 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ff5f834b60 msgr2=0x55ff5fc82360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:40:47.098 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:47.089+0000 7fa496d9d640 1 --2- 172.21.15.89:0/2601408470 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ff5f834b60 0x55ff5fc82360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:47.098 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:47.089+0000 7fa496d9d640 1 -- 172.21.15.89:0/2601408470 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ff5fc0dee0 con 0x55ff5f7e9570 2024-09-21T03:40:47.098 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:47.089+0000 7fa49759e640 1 --2- 172.21.15.89:0/2601408470 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ff5f855840 0x55ff5fc0d750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:40:47.098 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:47.089+0000 7fa48ffff640 1 --2- 172.21.15.89:0/2601408470 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ff5f834b60 0x55ff5fc82360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:40:47.098 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:47.090+0000 7fa496d9d640 1 --2- 172.21.15.89:0/2601408470 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ff5f7e9570 0x55ff5fc81da0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7fa48800abf0 tx=0x7fa48801a630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=497c5117db731ec server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:47.098 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:47.090+0000 7fa47e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2024-09-21T03:40:47.098 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:47.090+0000 7fa47e7fc640 1 -- 172.21.15.89:0/2601408470 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa48801ae40 con 0x55ff5f7e9570 2024-09-21T03:40:47.098 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:47.090+0000 7fa47e7fc640 1 -- 172.21.15.89:0/2601408470 <== mon.0 v2:172.21.15.89:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 9121+0+0 (secure 0 0 0) 0x7fa488005020 con 0x55ff5f7e9570 2024-09-21T03:40:47.098 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:47.090+0000 7fa4a36d9740 1 -- 172.21.15.89:0/2601408470 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa490056070 con 0x55ff5f7e9570 2024-09-21T03:40:47.098 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:47.090+0000 7fa47e7fc640 1 -- 172.21.15.89:0/2601408470 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fa488048050 con 0x55ff5f7e9570 2024-09-21T03:40:47.099 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:47.090+0000 7fa4a36d9740 10 client.15711 fetch_fsmap learned FSMap version 26 2024-09-21T03:40:47.099 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:47.090+0000 7fa4a36d9740 1 -- 172.21.15.89:0/2601408470 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55ff5fc7f480 con 0x55ff5f7e9570 2024-09-21T03:40:47.099 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:47.090+0000 7fa47e7fc640 1 -- 172.21.15.89:0/2601408470 <== mon.0 v2:172.21.15.89:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa490056070 con 0x55ff5f7e9570 2024-09-21T03:40:47.099 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:47.090+0000 7fa47e7fc640 1 -- 172.21.15.89:0/2601408470 <== mon.0 v2:172.21.15.89:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fa488040020 con 0x55ff5f7e9570 2024-09-21T03:40:47.099 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:47.091+0000 7fa4a36d9740 10 client.15711 fetch_fsmap finished waiting for FSMap version 26 2024-09-21T03:40:47.099 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:47.091+0000 7fa4a36d9740 10 client.15711 resolve_mds: validated gid 24638 aka daemon mds.b 2024-09-21T03:40:47.099 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:47.091+0000 7fa4a36d9740 20 client.15711 populate_metadata read hostname 'smithi089' 2024-09-21T03:40:47.099 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:47.091+0000 7fa4a36d9740 1 --2- 172.21.15.89:0/2601408470 >> [v2:172.21.15.110:6838/132366709,v1:172.21.15.110:6839/132366709] conn(0x55ff5fc8b060 0x55ff5fc8d480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:47.099 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:47.091+0000 7fa4a36d9740 4 client.15711 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 137.0, "await_for": 45.0}] 2024-09-21T03:40:47.099 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:47.091+0000 7fa4a36d9740 1 -- 172.21.15.89:0/2601408470 --> [v2:172.21.15.110:6838/132366709,v1:172.21.15.110:6839/132366709] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 137.0, "await_for": 45.0}) -- 0x55ff5fb70590 con 0x55ff5fc8b060 2024-09-21T03:40:47.099 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:47.091+0000 7fa48ffff640 1 --2- 172.21.15.89:0/2601408470 >> [v2:172.21.15.110:6838/132366709,v1:172.21.15.110:6839/132366709] conn(0x55ff5fc8b060 0x55ff5fc8d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:47.099 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:47.091+0000 7fa47e7fc640 10 client.15711 ms_handle_connect on v2:172.21.15.110:6838/132366709 2024-09-21T03:40:47.099 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:47.091+0000 7fa48ffff640 1 --2- 172.21.15.89:0/2601408470 >> [v2:172.21.15.110:6838/132366709,v1:172.21.15.110:6839/132366709] conn(0x55ff5fc8b060 0x55ff5fc8d480 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=69c799f5377c9491 server_cookie=6c663c27a65a653 in_seq=0 out_seq=0 2024-09-21T03:40:47.106 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:47.103+0000 7fa47e7fc640 1 -- 172.21.15.89:0/2601408470 <== mds.0 v2:172.21.15.110:6838/132366709 1 ==== command_reply(tid 0: 0 ) ==== 8+0+589 (crc 0 0 0) 0x55ff5fb70590 con 0x55ff5fc8b060 2024-09-21T03:40:47.106 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:47.103+0000 7fa47e7fc640 10 client.15711 handle_command_reply: tid=0 2024-09-21T03:40:47.106 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:47.103+0000 7fa4a36d9740 1 client.15711 shutdown 2024-09-21T03:40:47.106 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:47.103+0000 7fa4a36d9740 1 -- 172.21.15.89:0/2601408470 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ff5f7e9570 msgr2=0x55ff5fc81da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:47.106 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:47.103+0000 7fa4a36d9740 1 --2- 172.21.15.89:0/2601408470 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ff5f7e9570 0x55ff5fc81da0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7fa48800abf0 tx=0x7fa48801a630 comp rx=0 tx=0).stop 2024-09-21T03:40:47.106 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:47.103+0000 7fa4a36d9740 1 -- 172.21.15.89:0/2601408470 shutdown_connections 2024-09-21T03:40:47.106 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:47.103+0000 7fa4a36d9740 1 --2- 172.21.15.89:0/2601408470 >> [v2:172.21.15.110:6838/132366709,v1:172.21.15.110:6839/132366709] conn(0x55ff5fc8b060 0x55ff5fc8d480 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:47.106 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:47.103+0000 7fa4a36d9740 1 --2- 172.21.15.89:0/2601408470 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ff5f834b60 0x55ff5fc82360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:47.106 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:47.103+0000 7fa4a36d9740 1 --2- 172.21.15.89:0/2601408470 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ff5f855840 0x55ff5fc0d750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:47.107 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:47.103+0000 7fa47e7fc640 0 client.15711 ms_handle_reset on v2:172.21.15.110:6838/132366709 2024-09-21T03:40:47.107 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:47.103+0000 7fa4a36d9740 1 --2- 172.21.15.89:0/2601408470 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ff5f7e9570 0x55ff5fc81da0 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:47.107 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:47.103+0000 7fa4a36d9740 1 -- 172.21.15.89:0/2601408470 >> 172.21.15.89:0/2601408470 conn(0x55ff5f849990 msgr2=0x55ff5fc094c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:40:47.107 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:47.103+0000 7fa4a36d9740 1 -- 172.21.15.89:0/2601408470 shutdown_connections 2024-09-21T03:40:47.107 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:47.103+0000 7fa4a36d9740 1 -- 172.21.15.89:0/2601408470 wait complete. 2024-09-21T03:40:47.107 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:47.103+0000 7fa4a36d9740 20 client.15711 trim_cache size 0 max 16384 2024-09-21T03:40:47.107 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-09-21T03:40:47.107 INFO:teuthology.orchestra.run.smithi089.stdout: "epoch": 26, 2024-09-21T03:40:47.107 INFO:teuthology.orchestra.run.smithi089.stdout: "leader": 24638, 2024-09-21T03:40:47.107 INFO:teuthology.orchestra.run.smithi089.stdout: "set_version": 6, 2024-09-21T03:40:47.108 INFO:teuthology.orchestra.run.smithi089.stdout: "sets": { 2024-09-21T03:40:47.108 INFO:teuthology.orchestra.run.smithi089.stdout: "1beabdc5": { 2024-09-21T03:40:47.108 INFO:teuthology.orchestra.run.smithi089.stdout: "version": 6, 2024-09-21T03:40:47.108 INFO:teuthology.orchestra.run.smithi089.stdout: "age_ref": 0.0, 2024-09-21T03:40:47.108 INFO:teuthology.orchestra.run.smithi089.stdout: "state": { 2024-09-21T03:40:47.108 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "QUIESCED", 2024-09-21T03:40:47.108 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 0.0 2024-09-21T03:40:47.108 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:40:47.108 INFO:teuthology.orchestra.run.smithi089.stdout: "timeout": 90.0, 2024-09-21T03:40:47.108 INFO:teuthology.orchestra.run.smithi089.stdout: "expiration": 137.0, 2024-09-21T03:40:47.108 INFO:teuthology.orchestra.run.smithi089.stdout: "members": { 2024-09-21T03:40:47.108 INFO:teuthology.orchestra.run.smithi089.stdout: "file:/": { 2024-09-21T03:40:47.108 INFO:teuthology.orchestra.run.smithi089.stdout: "excluded": false, 2024-09-21T03:40:47.108 INFO:teuthology.orchestra.run.smithi089.stdout: "state": { 2024-09-21T03:40:47.108 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "QUIESCED", 2024-09-21T03:40:47.109 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 0.0 2024-09-21T03:40:47.109 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-21T03:40:47.109 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-21T03:40:47.109 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-21T03:40:47.109 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-21T03:40:47.109 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-21T03:40:47.109 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-09-21T03:40:47.109 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:47.104+0000 7fa49d165640 1 -- 172.21.15.89:0/3262798382 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7fa4840b65d0 msgr2=0x7fa4840b8ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:47.110 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:47.104+0000 7fa49d165640 1 --2- 172.21.15.89:0/3262798382 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7fa4840b65d0 0x7fa4840b8ae0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x55ff5fbfb280 tx=0x7fa48800a2d0 comp rx=0 tx=0).stop 2024-09-21T03:40:47.110 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:47.104+0000 7fa49d165640 1 -- 172.21.15.89:0/3262798382 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa4981083c0 msgr2=0x7fa49819b690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:47.110 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:47.104+0000 7fa49d165640 1 --2- 172.21.15.89:0/3262798382 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa4981083c0 0x7fa49819b690 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7fa48000c970 tx=0x7fa48000cd70 comp rx=0 tx=0).stop 2024-09-21T03:40:47.110 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:47.104+0000 7fa49d165640 1 -- 172.21.15.89:0/3262798382 shutdown_connections 2024-09-21T03:40:47.110 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:47.104+0000 7fa49d165640 1 --2- 172.21.15.89:0/3262798382 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7fa4840b65d0 0x7fa4840b8ae0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:47.110 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:47.104+0000 7fa49d165640 1 --2- 172.21.15.89:0/3262798382 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa498108d10 0x7fa49819bbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:47.110 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:47.104+0000 7fa49d165640 1 --2- 172.21.15.89:0/3262798382 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa4981083c0 0x7fa49819b690 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:47.110 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:47.104+0000 7fa49d165640 1 --2- 172.21.15.89:0/3262798382 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa498102a80 0x7fa49819b150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:47.110 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:47.104+0000 7fa49d165640 1 -- 172.21.15.89:0/3262798382 >> 172.21.15.89:0/3262798382 conn(0x7fa498079760 msgr2=0x7fa498104390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:40:47.110 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:47.104+0000 7fa49d165640 1 -- 172.21.15.89:0/3262798382 shutdown_connections 2024-09-21T03:40:47.110 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:40:47.104+0000 7fa49d165640 1 -- 172.21.15.89:0/3262798382 wait complete. 2024-09-21T03:40:47.750 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:40:47 smithi110 ceph-mon[34000]: pgmap v401: 129 pgs: 129 active+clean; 69 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 5.7 MiB/s wr, 10 op/s 2024-09-21T03:40:47.757 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:40:47 smithi089 ceph-mon[30928]: pgmap v401: 129 pgs: 129 active+clean; 69 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 5.7 MiB/s wr, 10 op/s 2024-09-21T03:40:47.785 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '1beabdc5', quiesce took 3.4 seconds. Will release after: 13.6 2024-09-21T03:40:47.961 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:40:47 smithi114 ceph-mon[34140]: pgmap v401: 129 pgs: 129 active+clean; 69 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 5.7 MiB/s wr, 10 op/s 2024-09-21T03:40:49.750 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:40:49 smithi110 ceph-mon[34000]: pgmap v402: 129 pgs: 129 active+clean; 129 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 MiB/s wr, 12 op/s 2024-09-21T03:40:49.811 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:40:49 smithi089 ceph-mon[30928]: pgmap v402: 129 pgs: 129 active+clean; 129 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 MiB/s wr, 12 op/s 2024-09-21T03:40:49.961 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:40:49 smithi114 ceph-mon[34140]: pgmap v402: 129 pgs: 129 active+clean; 129 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 MiB/s wr, 12 op/s 2024-09-21T03:40:51.750 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:40:51 smithi110 ceph-mon[34000]: pgmap v403: 129 pgs: 129 active+clean; 185 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 9.6 KiB/s rd, 15 MiB/s wr, 16 op/s 2024-09-21T03:40:51.811 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:40:51 smithi089 ceph-mon[30928]: pgmap v403: 129 pgs: 129 active+clean; 185 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 9.6 KiB/s rd, 15 MiB/s wr, 16 op/s 2024-09-21T03:40:51.960 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:40:51 smithi114 ceph-mon[34140]: pgmap v403: 129 pgs: 129 active+clean; 185 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 9.6 KiB/s rd, 15 MiB/s wr, 16 op/s 2024-09-21T03:40:53.750 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:40:53 smithi110 ceph-mon[34000]: pgmap v404: 129 pgs: 129 active+clean; 185 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 15 MiB/s wr, 14 op/s 2024-09-21T03:40:53.811 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:40:53 smithi089 ceph-mon[30928]: pgmap v404: 129 pgs: 129 active+clean; 185 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 15 MiB/s wr, 14 op/s 2024-09-21T03:40:53.961 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:40:53 smithi114 ceph-mon[34140]: pgmap v404: 129 pgs: 129 active+clean; 185 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 15 MiB/s wr, 14 op/s 2024-09-21T03:40:55.750 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:40:55 smithi110 ceph-mon[34000]: pgmap v405: 129 pgs: 129 active+clean; 189 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 16 MiB/s wr, 16 op/s 2024-09-21T03:40:55.750 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:40:55 smithi110 ceph-mon[34000]: mgrmap e21: x(active, since 11m), standbys: z, y 2024-09-21T03:40:55.811 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:40:55 smithi089 ceph-mon[30928]: pgmap v405: 129 pgs: 129 active+clean; 189 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 16 MiB/s wr, 16 op/s 2024-09-21T03:40:55.811 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:40:55 smithi089 ceph-mon[30928]: mgrmap e21: x(active, since 11m), standbys: z, y 2024-09-21T03:40:55.961 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:40:55 smithi114 ceph-mon[34140]: pgmap v405: 129 pgs: 129 active+clean; 189 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 16 MiB/s wr, 16 op/s 2024-09-21T03:40:55.961 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:40:55 smithi114 ceph-mon[34140]: mgrmap e21: x(active, since 11m), standbys: z, y 2024-09-21T03:40:57.750 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:40:57 smithi110 ceph-mon[34000]: pgmap v406: 129 pgs: 129 active+clean; 192 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 11 MiB/s wr, 15 op/s 2024-09-21T03:40:57.811 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:40:57 smithi089 ceph-mon[30928]: pgmap v406: 129 pgs: 129 active+clean; 192 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 11 MiB/s wr, 15 op/s 2024-09-21T03:40:57.961 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:40:57 smithi114 ceph-mon[34140]: pgmap v406: 129 pgs: 129 active+clean; 192 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 11 MiB/s wr, 15 op/s 2024-09-21T03:40:58.668 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:58.666+0000 7f4654be4640 1 -- 172.21.15.89:0/505675186 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4650103640 msgr2=0x7f4650103a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:58.668 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:58.666+0000 7f4654be4640 1 --2- 172.21.15.89:0/505675186 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4650103640 0x7f4650103a50 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f4644009900 tx=0x7f464402a410 comp rx=0 tx=0).stop 2024-09-21T03:40:58.669 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:58.666+0000 7f4654be4640 1 -- 172.21.15.89:0/505675186 shutdown_connections 2024-09-21T03:40:58.669 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:58.666+0000 7f4654be4640 1 --2- 172.21.15.89:0/505675186 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4650103fe0 0x7f465010a980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:58.669 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:58.666+0000 7f4654be4640 1 --2- 172.21.15.89:0/505675186 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4650103640 0x7f4650103a50 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:58.669 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:58.666+0000 7f4654be4640 1 --2- 172.21.15.89:0/505675186 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f46501025c0 0x7f4650102a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:58.669 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:58.666+0000 7f4654be4640 1 -- 172.21.15.89:0/505675186 >> 172.21.15.89:0/505675186 conn(0x7f46500fdf00 msgr2=0x7f46501003b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:40:58.669 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:58.666+0000 7f4654be4640 1 -- 172.21.15.89:0/505675186 shutdown_connections 2024-09-21T03:40:58.669 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:58.666+0000 7f4654be4640 1 -- 172.21.15.89:0/505675186 wait complete. 2024-09-21T03:40:58.671 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:58.667+0000 7f4654be4640 1 Processor -- start 2024-09-21T03:40:58.671 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:58.667+0000 7f4654be4640 1 -- start start 2024-09-21T03:40:58.671 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:58.667+0000 7f4654be4640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f46501025c0 0x7f465006c210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:58.671 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:58.667+0000 7f4654be4640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4650103640 0x7f465019ec00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:58.671 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:58.667+0000 7f4654be4640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4650103fe0 0x7f46501a1150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:58.671 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:58.667+0000 7f4654be4640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f465010e0a0 con 0x7f4650103fe0 2024-09-21T03:40:58.671 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:58.667+0000 7f4654be4640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f465010df20 con 0x7f4650103640 2024-09-21T03:40:58.671 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:58.667+0000 7f4654be4640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f465010e220 con 0x7f46501025c0 2024-09-21T03:40:58.671 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:58.667+0000 7f464ed9d640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4650103fe0 0x7f46501a1150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:58.671 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:58.667+0000 7f464ed9d640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4650103fe0 0x7f46501a1150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:40390/0 (socket says 172.21.15.89:40390) 2024-09-21T03:40:58.672 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:58.667+0000 7f464ed9d640 1 -- 172.21.15.89:0/2191924046 learned_addr learned my addr 172.21.15.89:0/2191924046 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-21T03:40:58.672 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:58.667+0000 7f464dd9b640 1 --2- 172.21.15.89:0/2191924046 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4650103640 0x7f465019ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:58.672 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:58.667+0000 7f464e59c640 1 --2- 172.21.15.89:0/2191924046 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f46501025c0 0x7f465006c210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:58.672 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:58.667+0000 7f464ed9d640 1 -- 172.21.15.89:0/2191924046 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f46501025c0 msgr2=0x7f465006c210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:58.672 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:58.668+0000 7f464ed9d640 1 --2- 172.21.15.89:0/2191924046 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f46501025c0 0x7f465006c210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:58.672 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:58.668+0000 7f464ed9d640 1 -- 172.21.15.89:0/2191924046 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4650103640 msgr2=0x7f465019ec00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:58.672 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:58.668+0000 7f464ed9d640 1 --2- 172.21.15.89:0/2191924046 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4650103640 0x7f465019ec00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:58.672 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:58.668+0000 7f464ed9d640 1 -- 172.21.15.89:0/2191924046 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46501a18e0 con 0x7f4650103fe0 2024-09-21T03:40:58.672 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:58.668+0000 7f464ed9d640 1 --2- 172.21.15.89:0/2191924046 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4650103fe0 0x7f46501a1150 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f464800d950 tx=0x7f464800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:58.672 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:58.668+0000 7f463f7fe640 1 -- 172.21.15.89:0/2191924046 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4648017030 con 0x7f4650103fe0 2024-09-21T03:40:58.672 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:58.668+0000 7f463f7fe640 1 -- 172.21.15.89:0/2191924046 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4648004020 con 0x7f4650103fe0 2024-09-21T03:40:58.672 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:58.668+0000 7f4654be4640 1 -- 172.21.15.89:0/2191924046 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46501a1bd0 con 0x7f4650103fe0 2024-09-21T03:40:58.672 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:58.668+0000 7f463f7fe640 1 -- 172.21.15.89:0/2191924046 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4648003ab0 con 0x7f4650103fe0 2024-09-21T03:40:58.672 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:58.668+0000 7f4654be4640 1 -- 172.21.15.89:0/2191924046 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46501a20d0 con 0x7f4650103fe0 2024-09-21T03:40:58.674 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:58.671+0000 7f465bc32740 1 Processor -- start 2024-09-21T03:40:58.674 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:58.671+0000 7f463f7fe640 1 -- 172.21.15.89:0/2191924046 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 21) ==== 154512+0+0 (secure 0 0 0) 0x7f464801a070 con 0x7f4650103fe0 2024-09-21T03:40:58.675 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:58.671+0000 7f465bc32740 1 -- start start 2024-09-21T03:40:58.675 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:58.671+0000 7f465bc32740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557c89f331b0 0x557c8a2dae50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:58.675 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:58.671+0000 7f465bc32740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x557c89f0a2f0 0x557c8a2db4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:58.675 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:58.671+0000 7f465bc32740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557c89f124d0 0x557c8a2dc010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:58.675 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:58.671+0000 7f465bc32740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x557c8a24d6f0 con 0x557c89f124d0 2024-09-21T03:40:58.675 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:58.671+0000 7f465bc32740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x557c8a23cb20 con 0x557c89f331b0 2024-09-21T03:40:58.675 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:58.671+0000 7f465bc32740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x557c8a240930 con 0x557c89f0a2f0 2024-09-21T03:40:58.675 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:58.671+0000 7f464dd9b640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557c89f124d0 0x557c8a2dc010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:58.675 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:58.671+0000 7f464dd9b640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557c89f124d0 0x557c8a2dc010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:40406/0 (socket says 172.21.15.89:40406) 2024-09-21T03:40:58.675 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:58.671+0000 7f464dd9b640 1 -- 172.21.15.89:0/73950126 learned_addr learned my addr 172.21.15.89:0/73950126 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-21T03:40:58.675 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:58.671+0000 7f464e59c640 1 --2- 172.21.15.89:0/73950126 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x557c89f0a2f0 0x557c8a2db4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:58.675 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:58.671+0000 7f464dd9b640 1 --2- 172.21.15.89:0/73950126 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557c89f331b0 0x557c8a2dae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:58.675 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:58.672+0000 7f464dd9b640 1 -- 172.21.15.89:0/73950126 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x557c89f0a2f0 msgr2=0x557c8a2db4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:40:58.675 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:58.672+0000 7f464dd9b640 1 --2- 172.21.15.89:0/73950126 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x557c89f0a2f0 0x557c8a2db4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:58.676 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:58.672+0000 7f464dd9b640 1 -- 172.21.15.89:0/73950126 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557c89f331b0 msgr2=0x557c8a2dae50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:40:58.676 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:58.672+0000 7f464dd9b640 1 --2- 172.21.15.89:0/73950126 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557c89f331b0 0x557c8a2dae50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:58.676 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:58.672+0000 7f464dd9b640 1 -- 172.21.15.89:0/73950126 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557c8a202530 con 0x557c89f124d0 2024-09-21T03:40:58.676 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:58.672+0000 7f464e59c640 1 --2- 172.21.15.89:0/73950126 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x557c89f0a2f0 0x557c8a2db4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:40:58.676 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:58.672+0000 7f463f7fe640 1 --2- 172.21.15.89:0/2191924046 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f46380b6510 0x7f46380b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:58.676 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:58.672+0000 7f464dd9b640 1 --2- 172.21.15.89:0/73950126 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557c89f124d0 0x557c8a2dc010 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f4644031460 tx=0x7f4644005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c46a7fce4632b582 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:58.676 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:58.672+0000 7f464e59c640 1 --2- 172.21.15.89:0/2191924046 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f46380b6510 0x7f46380b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:58.676 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:58.672+0000 7f463f7fe640 1 -- 172.21.15.89:0/2191924046 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 9121+0+0 (secure 0 0 0) 0x7f46480e6e90 con 0x7f4650103fe0 2024-09-21T03:40:58.676 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:58.672+0000 7f463cff9640 1 -- 172.21.15.89:0/73950126 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4644035040 con 0x557c89f124d0 2024-09-21T03:40:58.676 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:58.672+0000 7f463cff9640 1 -- 172.21.15.89:0/73950126 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f46440044c0 con 0x557c89f124d0 2024-09-21T03:40:58.676 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:58.672+0000 7f463cff9640 1 -- 172.21.15.89:0/73950126 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4644003aa0 con 0x557c89f124d0 2024-09-21T03:40:58.681 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:58.673+0000 7f465bc32740 1 -- 172.21.15.89:0/73950126 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557c89f124d0 msgr2=0x557c8a2dc010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:58.681 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:58.673+0000 7f464e59c640 1 --2- 172.21.15.89:0/2191924046 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f46380b6510 0x7f46380b8a20 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f4640011630 tx=0x7f4640007700 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:58.681 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:58.673+0000 7f465bc32740 1 --2- 172.21.15.89:0/73950126 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557c89f124d0 0x557c8a2dc010 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f4644031460 tx=0x7f4644005fd0 comp rx=0 tx=0).stop 2024-09-21T03:40:58.681 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:58.673+0000 7f465bc32740 1 -- 172.21.15.89:0/73950126 shutdown_connections 2024-09-21T03:40:58.681 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:58.673+0000 7f465bc32740 1 --2- 172.21.15.89:0/73950126 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557c89f124d0 0x557c8a2dc010 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:58.681 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:58.673+0000 7f465bc32740 1 --2- 172.21.15.89:0/73950126 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x557c89f0a2f0 0x557c8a2db4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:58.681 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:58.673+0000 7f465bc32740 1 --2- 172.21.15.89:0/73950126 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557c89f331b0 0x557c8a2dae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:58.681 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:58.673+0000 7f465bc32740 1 -- 172.21.15.89:0/73950126 >> 172.21.15.89:0/73950126 conn(0x557c89f27300 msgr2=0x557c8a2d68e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:40:58.681 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:58.673+0000 7f465bc32740 1 -- 172.21.15.89:0/73950126 shutdown_connections 2024-09-21T03:40:58.681 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:58.673+0000 7f465bc32740 1 -- 172.21.15.89:0/73950126 wait complete. 2024-09-21T03:40:58.682 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:58.674+0000 7f465bc32740 1 -- start start 2024-09-21T03:40:58.682 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:58.674+0000 7f465bc32740 1 Processor -- start 2024-09-21T03:40:58.682 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:58.674+0000 7f465bc32740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557c89f0a2f0 0x557c8a2e4620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:58.682 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:58.674+0000 7f465bc32740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x557c89f124d0 0x557c8a2e4ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:58.682 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:58.674+0000 7f465bc32740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557c89f331b0 0x557c8a2e5850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:58.682 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:58.674+0000 7f465bc32740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x557c8a174ab0 con 0x557c89f0a2f0 2024-09-21T03:40:58.682 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:58.674+0000 7f465bc32740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x557c8a1e2730 con 0x557c89f331b0 2024-09-21T03:40:58.682 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:58.674+0000 7f465bc32740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x557c8a2dcad0 con 0x557c89f124d0 2024-09-21T03:40:58.682 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:58.674+0000 7f465bc32740 10 client.0 _ll_register_callbacks cb 0x557c8a2366b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:40:58.682 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:58.674+0000 7f464e59c640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557c89f0a2f0 0x557c8a2e4620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:58.682 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:58.674+0000 7f464e59c640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557c89f0a2f0 0x557c8a2e4620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:40420/0 (socket says 172.21.15.89:40420) 2024-09-21T03:40:58.682 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:58.674+0000 7f464e59c640 1 -- 172.21.15.89:0/2060991850 learned_addr learned my addr 172.21.15.89:0/2060991850 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-21T03:40:58.682 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:58.674+0000 7f464ed9d640 1 --2- 172.21.15.89:0/2060991850 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557c89f331b0 0x557c8a2e5850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:58.682 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:58.674+0000 7f464dd9b640 1 --2- 172.21.15.89:0/2060991850 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x557c89f124d0 0x557c8a2e4ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:58.682 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:58.674+0000 7f464e59c640 1 -- 172.21.15.89:0/2060991850 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x557c89f124d0 msgr2=0x557c8a2e4ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:40:58.683 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:58.674+0000 7f464e59c640 1 --2- 172.21.15.89:0/2060991850 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x557c89f124d0 0x557c8a2e4ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:58.683 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:58.674+0000 7f464e59c640 1 -- 172.21.15.89:0/2060991850 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557c89f331b0 msgr2=0x557c8a2e5850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:40:58.683 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:58.674+0000 7f464e59c640 1 --2- 172.21.15.89:0/2060991850 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557c89f331b0 0x557c8a2e5850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:58.683 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:58.674+0000 7f464e59c640 1 -- 172.21.15.89:0/2060991850 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557c89f9e790 con 0x557c89f0a2f0 2024-09-21T03:40:58.683 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:58.674+0000 7f464dd9b640 1 --2- 172.21.15.89:0/2060991850 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x557c89f124d0 0x557c8a2e4ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:40:58.683 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:58.675+0000 7f464e59c640 1 --2- 172.21.15.89:0/2060991850 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557c89f0a2f0 0x557c8a2e4620 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f464000ffe0 tx=0x7f4640004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c68bb472e1dd317 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:58.683 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:58.675+0000 7f4635ffb640 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2024-09-21T03:40:58.683 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:58.675+0000 7f4635ffb640 1 -- 172.21.15.89:0/2060991850 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f464001e620 con 0x557c89f0a2f0 2024-09-21T03:40:58.683 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:58.675+0000 7f4635ffb640 1 -- 172.21.15.89:0/2060991850 <== mon.0 v2:172.21.15.89:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 9121+0+0 (secure 0 0 0) 0x7f464001e7c0 con 0x557c89f0a2f0 2024-09-21T03:40:58.683 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:58.675+0000 7f465bc32740 1 -- 172.21.15.89:0/2060991850 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x557c8a24f640 con 0x557c89f0a2f0 2024-09-21T03:40:58.683 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:58.675+0000 7f4635ffb640 1 -- 172.21.15.89:0/2060991850 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f4640031050 con 0x557c89f0a2f0 2024-09-21T03:40:58.683 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:58.675+0000 7f465bc32740 10 client.15723 fetch_fsmap learned FSMap version 26 2024-09-21T03:40:58.683 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:58.675+0000 7f465bc32740 1 -- 172.21.15.89:0/2060991850 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x557c8a017620 con 0x557c89f0a2f0 2024-09-21T03:40:58.683 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:58.675+0000 7f4635ffb640 1 -- 172.21.15.89:0/2060991850 <== mon.0 v2:172.21.15.89:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x557c8a24f640 con 0x557c89f0a2f0 2024-09-21T03:40:58.683 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:58.675+0000 7f4635ffb640 1 -- 172.21.15.89:0/2060991850 <== mon.0 v2:172.21.15.89:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f464001b990 con 0x557c89f0a2f0 2024-09-21T03:40:58.684 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:58.676+0000 7f465bc32740 10 client.15723 fetch_fsmap finished waiting for FSMap version 26 2024-09-21T03:40:58.684 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:58.676+0000 7f465bc32740 10 client.15723 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T03:40:58.684 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:58.676+0000 7f465bc32740 20 client.15723 populate_metadata read hostname 'smithi089' 2024-09-21T03:40:58.684 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:58.676+0000 7f465bc32740 1 --2- 172.21.15.89:0/2060991850 >> [v2:172.21.15.110:6838/132366709,v1:172.21.15.110:6839/132366709] conn(0x557c89ef1d80 0x557c8a2f1ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:58.684 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:58.676+0000 7f465bc32740 4 client.15723 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:40:58.684 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:58.676+0000 7f465bc32740 1 -- 172.21.15.89:0/2060991850 --> [v2:172.21.15.110:6838/132366709,v1:172.21.15.110:6839/132366709] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557c8a24b0f0 con 0x557c89ef1d80 2024-09-21T03:40:58.684 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:58.676+0000 7f464dd9b640 1 --2- 172.21.15.89:0/2060991850 >> [v2:172.21.15.110:6838/132366709,v1:172.21.15.110:6839/132366709] conn(0x557c89ef1d80 0x557c8a2f1ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:58.684 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:58.676+0000 7f4635ffb640 10 client.15723 ms_handle_connect on v2:172.21.15.110:6838/132366709 2024-09-21T03:40:58.684 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:58.676+0000 7f464dd9b640 1 --2- 172.21.15.89:0/2060991850 >> [v2:172.21.15.110:6838/132366709,v1:172.21.15.110:6839/132366709] conn(0x557c89ef1d80 0x557c8a2f1ea0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e066d7327ddc82c4 server_cookie=795b32e1bf68200e in_seq=0 out_seq=0 2024-09-21T03:40:58.684 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:58.677+0000 7f4635ffb640 1 -- 172.21.15.89:0/2060991850 <== mds.0 v2:172.21.15.110:6838/132366709 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x557c8a24b0f0 con 0x557c89ef1d80 2024-09-21T03:40:58.684 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:58.677+0000 7f4635ffb640 10 client.15723 handle_command_reply: tid=0 2024-09-21T03:40:58.684 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:58.677+0000 7f465bc32740 1 client.15723 shutdown 2024-09-21T03:40:58.684 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:58.677+0000 7f465bc32740 1 -- 172.21.15.89:0/2060991850 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557c89f0a2f0 msgr2=0x557c8a2e4620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:58.684 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:58.677+0000 7f465bc32740 1 --2- 172.21.15.89:0/2060991850 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557c89f0a2f0 0x557c8a2e4620 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f464000ffe0 tx=0x7f4640004a40 comp rx=0 tx=0).stop 2024-09-21T03:40:58.684 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:58.677+0000 7f465bc32740 1 -- 172.21.15.89:0/2060991850 shutdown_connections 2024-09-21T03:40:58.684 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:58.677+0000 7f465bc32740 1 --2- 172.21.15.89:0/2060991850 >> [v2:172.21.15.110:6838/132366709,v1:172.21.15.110:6839/132366709] conn(0x557c89ef1d80 0x557c8a2f1ea0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:58.685 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:58.677+0000 7f465bc32740 1 --2- 172.21.15.89:0/2060991850 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557c89f331b0 0x557c8a2e5850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:58.685 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:58.677+0000 7f465bc32740 1 --2- 172.21.15.89:0/2060991850 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x557c89f124d0 0x557c8a2e4ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:58.685 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:58.677+0000 7f4635ffb640 0 client.15723 ms_handle_reset on v2:172.21.15.110:6838/132366709 2024-09-21T03:40:58.685 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:58.677+0000 7f465bc32740 1 --2- 172.21.15.89:0/2060991850 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557c89f0a2f0 0x557c8a2e4620 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:58.685 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:58.677+0000 7f465bc32740 1 -- 172.21.15.89:0/2060991850 >> 172.21.15.89:0/2060991850 conn(0x557c89f27300 msgr2=0x557c8a2d4ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:40:58.685 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:58.677+0000 7f465bc32740 1 -- 172.21.15.89:0/2060991850 shutdown_connections 2024-09-21T03:40:58.685 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:58.677+0000 7f465bc32740 1 -- 172.21.15.89:0/2060991850 wait complete. 2024-09-21T03:40:58.685 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:58.677+0000 7f465bc32740 20 client.15723 trim_cache size 0 max 16384 2024-09-21T03:40:58.695 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:58.692+0000 7f465bc32740 1 Processor -- start 2024-09-21T03:40:58.695 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:58.692+0000 7f465bc32740 1 -- start start 2024-09-21T03:40:58.695 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:58.692+0000 7f465bc32740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557c89f0a2f0 0x557c8a2e2b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:58.696 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:58.692+0000 7f465bc32740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x557c89f124d0 0x557c8a2e30b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:58.696 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:58.692+0000 7f465bc32740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557c89f331b0 0x557c8a2dadf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:58.696 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:58.692+0000 7f465bc32740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f46501aedf0 con 0x557c89f331b0 2024-09-21T03:40:58.696 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:58.692+0000 7f465bc32740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f46501ac6c0 con 0x557c89f0a2f0 2024-09-21T03:40:58.696 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:58.692+0000 7f465bc32740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f46501aa510 con 0x557c89f124d0 2024-09-21T03:40:58.696 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:58.692+0000 7f464ed9d640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557c89f331b0 0x557c8a2dadf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:58.696 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:58.692+0000 7f464ed9d640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557c89f331b0 0x557c8a2dadf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:40426/0 (socket says 172.21.15.89:40426) 2024-09-21T03:40:58.696 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:58.692+0000 7f464ed9d640 1 -- 172.21.15.89:0/3374449154 learned_addr learned my addr 172.21.15.89:0/3374449154 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-21T03:40:58.696 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:58.692+0000 7f464e59c640 1 --2- 172.21.15.89:0/3374449154 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557c89f0a2f0 0x557c8a2e2b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:58.696 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:58.692+0000 7f464dd9b640 1 --2- 172.21.15.89:0/3374449154 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x557c89f124d0 0x557c8a2e30b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:58.696 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:58.693+0000 7f464ed9d640 1 -- 172.21.15.89:0/3374449154 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x557c89f124d0 msgr2=0x557c8a2e30b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:40:58.696 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:58.693+0000 7f464ed9d640 1 --2- 172.21.15.89:0/3374449154 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x557c89f124d0 0x557c8a2e30b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:58.696 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:58.693+0000 7f464ed9d640 1 -- 172.21.15.89:0/3374449154 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557c89f0a2f0 msgr2=0x557c8a2e2b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:40:58.696 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:58.693+0000 7f464ed9d640 1 --2- 172.21.15.89:0/3374449154 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557c89f0a2f0 0x557c8a2e2b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:58.696 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:58.693+0000 7f464ed9d640 1 -- 172.21.15.89:0/3374449154 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557c8a2d7010 con 0x557c89f331b0 2024-09-21T03:40:58.696 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:58.693+0000 7f464ed9d640 1 --2- 172.21.15.89:0/3374449154 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557c89f331b0 0x557c8a2dadf0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f46480ac9a0 tx=0x7f46480ac7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2145cfe4d790aafa server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:58.697 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:58.693+0000 7f4637fff640 1 -- 172.21.15.89:0/3374449154 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46480b1780 con 0x557c89f331b0 2024-09-21T03:40:58.697 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:58.693+0000 7f4637fff640 1 -- 172.21.15.89:0/3374449154 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f46480b66c0 con 0x557c89f331b0 2024-09-21T03:40:58.697 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:58.693+0000 7f4637fff640 1 -- 172.21.15.89:0/3374449154 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46480d0300 con 0x557c89f331b0 2024-09-21T03:40:58.699 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:58.694+0000 7f465bc32740 1 -- 172.21.15.89:0/3374449154 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557c89f331b0 msgr2=0x557c8a2dadf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:58.700 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:58.694+0000 7f465bc32740 1 --2- 172.21.15.89:0/3374449154 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557c89f331b0 0x557c8a2dadf0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f46480ac9a0 tx=0x7f46480ac7b0 comp rx=0 tx=0).stop 2024-09-21T03:40:58.700 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:58.694+0000 7f465bc32740 1 -- 172.21.15.89:0/3374449154 shutdown_connections 2024-09-21T03:40:58.700 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:58.694+0000 7f465bc32740 1 --2- 172.21.15.89:0/3374449154 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557c89f331b0 0x557c8a2dadf0 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:58.700 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:58.694+0000 7f465bc32740 1 --2- 172.21.15.89:0/3374449154 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x557c89f124d0 0x557c8a2e30b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:58.700 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:58.694+0000 7f465bc32740 1 --2- 172.21.15.89:0/3374449154 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557c89f0a2f0 0x557c8a2e2b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:58.700 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:58.694+0000 7f465bc32740 1 -- 172.21.15.89:0/3374449154 >> 172.21.15.89:0/3374449154 conn(0x557c89f27300 msgr2=0x557c8a2d5610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:40:58.700 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:58.694+0000 7f465bc32740 1 -- 172.21.15.89:0/3374449154 shutdown_connections 2024-09-21T03:40:58.700 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:58.694+0000 7f465bc32740 1 -- 172.21.15.89:0/3374449154 wait complete. 2024-09-21T03:40:58.700 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:58.694+0000 7f465bc32740 1 -- start start 2024-09-21T03:40:58.700 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:58.694+0000 7f465bc32740 1 Processor -- start 2024-09-21T03:40:58.700 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:58.694+0000 7f465bc32740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557c89f0a2f0 0x557c8a2ec3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:58.700 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:58.694+0000 7f465bc32740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557c89f124d0 0x557c8a2eb080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:58.700 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:58.694+0000 7f465bc32740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x557c89f331b0 0x557c8a2eb5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:58.700 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:58.694+0000 7f465bc32740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f46501aab80 con 0x557c89f124d0 2024-09-21T03:40:58.700 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:58.694+0000 7f465bc32740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f46501abff0 con 0x557c89f0a2f0 2024-09-21T03:40:58.701 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:58.694+0000 7f465bc32740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f46501ab250 con 0x557c89f331b0 2024-09-21T03:40:58.701 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:58.694+0000 7f465bc32740 10 client.0 _ll_register_callbacks cb 0x7f464000fc60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:40:58.701 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:58.695+0000 7f464dd9b640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557c89f124d0 0x557c8a2eb080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:58.701 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:58.695+0000 7f464dd9b640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557c89f124d0 0x557c8a2eb080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:40430/0 (socket says 172.21.15.89:40430) 2024-09-21T03:40:58.701 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:58.695+0000 7f464dd9b640 1 -- 172.21.15.89:0/2074705148 learned_addr learned my addr 172.21.15.89:0/2074705148 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-21T03:40:58.701 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:58.695+0000 7f464e59c640 1 --2- 172.21.15.89:0/2074705148 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557c89f0a2f0 0x557c8a2ec3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:58.701 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:58.695+0000 7f464dd9b640 1 -- 172.21.15.89:0/2074705148 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x557c89f331b0 msgr2=0x557c8a2eb5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:40:58.701 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:58.695+0000 7f464dd9b640 1 --2- 172.21.15.89:0/2074705148 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x557c89f331b0 0x557c8a2eb5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:58.701 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:58.695+0000 7f464dd9b640 1 -- 172.21.15.89:0/2074705148 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557c89f0a2f0 msgr2=0x557c8a2ec3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:40:58.701 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:58.695+0000 7f464dd9b640 1 --2- 172.21.15.89:0/2074705148 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557c89f0a2f0 0x557c8a2ec3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:58.701 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:58.695+0000 7f464dd9b640 1 -- 172.21.15.89:0/2074705148 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557c8a2ebd50 con 0x557c89f124d0 2024-09-21T03:40:58.701 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:58.695+0000 7f464dd9b640 1 --2- 172.21.15.89:0/2074705148 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557c89f124d0 0x557c8a2eb080 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f4644009900 tx=0x7f464402a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b20ce9cb13613981 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:58.701 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:58.695+0000 7f4635ffb640 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2024-09-21T03:40:58.701 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:58.695+0000 7f4635ffb640 1 -- 172.21.15.89:0/2074705148 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4644035040 con 0x557c89f124d0 2024-09-21T03:40:58.701 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:58.695+0000 7f4635ffb640 1 -- 172.21.15.89:0/2074705148 <== mon.0 v2:172.21.15.89:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 9121+0+0 (secure 0 0 0) 0x7f4644031c50 con 0x557c89f124d0 2024-09-21T03:40:58.701 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:58.695+0000 7f465bc32740 1 -- 172.21.15.89:0/2074705148 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f46480b66c0 con 0x557c89f124d0 2024-09-21T03:40:58.702 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:58.695+0000 7f4635ffb640 1 -- 172.21.15.89:0/2074705148 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f464405a050 con 0x557c89f124d0 2024-09-21T03:40:58.702 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:58.695+0000 7f465bc32740 10 client.15729 fetch_fsmap learned FSMap version 26 2024-09-21T03:40:58.702 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:58.695+0000 7f465bc32740 1 -- 172.21.15.89:0/2074705148 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x557c8a360cb0 con 0x557c89f124d0 2024-09-21T03:40:58.702 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:58.696+0000 7f4635ffb640 1 -- 172.21.15.89:0/2074705148 <== mon.0 v2:172.21.15.89:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f46480b66c0 con 0x557c89f124d0 2024-09-21T03:40:58.702 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:58.696+0000 7f4635ffb640 1 -- 172.21.15.89:0/2074705148 <== mon.0 v2:172.21.15.89:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f464403d450 con 0x557c89f124d0 2024-09-21T03:40:58.702 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:58.696+0000 7f465bc32740 10 client.15729 fetch_fsmap finished waiting for FSMap version 26 2024-09-21T03:40:58.702 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:58.696+0000 7f465bc32740 10 client.15729 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T03:40:58.702 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:58.696+0000 7f465bc32740 20 client.15729 populate_metadata read hostname 'smithi089' 2024-09-21T03:40:58.702 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:58.696+0000 7f465bc32740 1 --2- 172.21.15.89:0/2074705148 >> [v2:172.21.15.110:6838/132366709,v1:172.21.15.110:6839/132366709] conn(0x557c89ec71c0 0x557c8a363160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:58.702 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:58.696+0000 7f465bc32740 4 client.15729 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T03:40:58.702 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:58.696+0000 7f465bc32740 1 -- 172.21.15.89:0/2074705148 --> [v2:172.21.15.110:6838/132366709,v1:172.21.15.110:6839/132366709] -- command(tid 0: {"prefix": "flush journal"}) -- 0x557c8a248520 con 0x557c89ec71c0 2024-09-21T03:40:58.702 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:58.696+0000 7f464e59c640 1 --2- 172.21.15.89:0/2074705148 >> [v2:172.21.15.110:6838/132366709,v1:172.21.15.110:6839/132366709] conn(0x557c89ec71c0 0x557c8a363160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:58.702 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:58.696+0000 7f4635ffb640 10 client.15729 ms_handle_connect on v2:172.21.15.110:6838/132366709 2024-09-21T03:40:58.702 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:58.696+0000 7f464e59c640 1 --2- 172.21.15.89:0/2074705148 >> [v2:172.21.15.110:6838/132366709,v1:172.21.15.110:6839/132366709] conn(0x557c89ec71c0 0x557c8a363160 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=be2efd4385319048 server_cookie=1189b41f08fa419a in_seq=0 out_seq=0 2024-09-21T03:40:58.707 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:58.704+0000 7f4635ffb640 1 -- 172.21.15.89:0/2074705148 <== mds.0 v2:172.21.15.110:6838/132366709 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x557c8a248520 con 0x557c89ec71c0 2024-09-21T03:40:58.708 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:58.704+0000 7f4635ffb640 10 client.15729 handle_command_reply: tid=0 2024-09-21T03:40:58.708 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:58.704+0000 7f465bc32740 1 client.15729 shutdown 2024-09-21T03:40:58.708 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:58.705+0000 7f465bc32740 1 -- 172.21.15.89:0/2074705148 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557c89f124d0 msgr2=0x557c8a2eb080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:58.708 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:58.705+0000 7f465bc32740 1 --2- 172.21.15.89:0/2074705148 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557c89f124d0 0x557c8a2eb080 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f4644009900 tx=0x7f464402a410 comp rx=0 tx=0).stop 2024-09-21T03:40:58.708 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:58.705+0000 7f465bc32740 1 -- 172.21.15.89:0/2074705148 shutdown_connections 2024-09-21T03:40:58.708 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:58.705+0000 7f465bc32740 1 --2- 172.21.15.89:0/2074705148 >> [v2:172.21.15.110:6838/132366709,v1:172.21.15.110:6839/132366709] conn(0x557c89ec71c0 0x557c8a363160 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:58.708 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:58.705+0000 7f465bc32740 1 --2- 172.21.15.89:0/2074705148 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x557c89f331b0 0x557c8a2eb5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:58.708 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:58.705+0000 7f465bc32740 1 --2- 172.21.15.89:0/2074705148 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557c89f124d0 0x557c8a2eb080 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:58.708 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:58.705+0000 7f4635ffb640 0 client.15729 ms_handle_reset on v2:172.21.15.110:6838/132366709 2024-09-21T03:40:58.708 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:58.705+0000 7f465bc32740 1 --2- 172.21.15.89:0/2074705148 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557c89f0a2f0 0x557c8a2ec3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:58.708 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:58.705+0000 7f465bc32740 1 -- 172.21.15.89:0/2074705148 >> 172.21.15.89:0/2074705148 conn(0x557c89f27300 msgr2=0x557c8a2e3e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:40:58.708 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:58.705+0000 7f465bc32740 1 -- 172.21.15.89:0/2074705148 shutdown_connections 2024-09-21T03:40:58.708 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:58.705+0000 7f465bc32740 1 -- 172.21.15.89:0/2074705148 wait complete. 2024-09-21T03:40:58.708 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:58.705+0000 7f465bc32740 20 client.15729 trim_cache size 0 max 16384 2024-09-21T03:40:58.709 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stdout:{ 2024-09-21T03:40:58.709 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stdout: "message": "", 2024-09-21T03:40:58.709 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stdout: "return_code": 0 2024-09-21T03:40:58.709 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stdout:} 2024-09-21T03:40:58.709 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:58.706+0000 7f4654be4640 1 -- 172.21.15.89:0/2191924046 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f46380b6510 msgr2=0x7f46380b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:58.710 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:58.706+0000 7f4654be4640 1 --2- 172.21.15.89:0/2191924046 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f46380b6510 0x7f46380b8a20 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f4640011630 tx=0x7f4640007700 comp rx=0 tx=0).stop 2024-09-21T03:40:58.710 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:58.706+0000 7f4654be4640 1 -- 172.21.15.89:0/2191924046 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4650103fe0 msgr2=0x7f46501a1150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:58.710 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:58.706+0000 7f4654be4640 1 --2- 172.21.15.89:0/2191924046 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4650103fe0 0x7f46501a1150 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f464800d950 tx=0x7f464800de10 comp rx=0 tx=0).stop 2024-09-21T03:40:58.710 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:58.706+0000 7f4654be4640 1 -- 172.21.15.89:0/2191924046 shutdown_connections 2024-09-21T03:40:58.710 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:58.706+0000 7f4654be4640 1 --2- 172.21.15.89:0/2191924046 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f46380b6510 0x7f46380b8a20 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:58.710 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:58.706+0000 7f4654be4640 1 --2- 172.21.15.89:0/2191924046 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4650103640 0x7f465019ec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:58.710 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:58.706+0000 7f4654be4640 1 --2- 172.21.15.89:0/2191924046 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4650103fe0 0x7f46501a1150 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:58.710 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:58.706+0000 7f4654be4640 1 --2- 172.21.15.89:0/2191924046 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f46501025c0 0x7f465006c210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:58.710 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:58.706+0000 7f4654be4640 1 -- 172.21.15.89:0/2191924046 >> 172.21.15.89:0/2191924046 conn(0x7f46500fdf00 msgr2=0x7f4650106940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:40:58.710 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:58.706+0000 7f4654be4640 1 -- 172.21.15.89:0/2191924046 shutdown_connections 2024-09-21T03:40:58.710 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:40:58.706+0000 7f4654be4640 1 -- 172.21.15.89:0/2191924046 wait complete. 2024-09-21T03:40:59.750 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:40:59 smithi110 ceph-mon[34000]: pgmap v407: 129 pgs: 129 active+clean; 192 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 MiB/s wr, 13 op/s 2024-09-21T03:40:59.811 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:40:59 smithi089 ceph-mon[30928]: pgmap v407: 129 pgs: 129 active+clean; 192 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 MiB/s wr, 13 op/s 2024-09-21T03:40:59.961 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:40:59 smithi114 ceph-mon[34140]: pgmap v407: 129 pgs: 129 active+clean; 192 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 MiB/s wr, 13 op/s 2024-09-21T03:41:01.386 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 7c374098-77c9-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T03:41:01.649 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/7c374098-77c9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:41:01.750 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:41:01 smithi110 ceph-mon[34000]: pgmap v408: 129 pgs: 129 active+clean; 192 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 5.3 MiB/s wr, 11 op/s 2024-09-21T03:41:01.770 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:41:01 smithi089 ceph-mon[30928]: pgmap v408: 129 pgs: 129 active+clean; 192 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 5.3 MiB/s wr, 11 op/s 2024-09-21T03:41:01.961 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:41:01 smithi114 ceph-mon[34140]: pgmap v408: 129 pgs: 129 active+clean; 192 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 5.3 MiB/s wr, 11 op/s 2024-09-21T03:41:02.952 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:41:02 smithi089 ceph-mon[30928]: pgmap v409: 129 pgs: 129 active+clean; 192 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 660 KiB/s wr, 6 op/s 2024-09-21T03:41:02.960 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:41:02 smithi114 ceph-mon[34140]: pgmap v409: 129 pgs: 129 active+clean; 192 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 660 KiB/s wr, 6 op/s 2024-09-21T03:41:03.000 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:41:02 smithi110 ceph-mon[34000]: pgmap v409: 129 pgs: 129 active+clean; 192 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 660 KiB/s wr, 6 op/s 2024-09-21T03:41:03.981 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:03.978+0000 7fb415f23640 1 -- 172.21.15.89:0/2257346690 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb4101083c0 msgr2=0x7fb4101087d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:03.981 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:03.978+0000 7fb415f23640 1 --2- 172.21.15.89:0/2257346690 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb4101083c0 0x7fb4101087d0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7fb3f8009900 tx=0x7fb3f802a410 comp rx=0 tx=0).stop 2024-09-21T03:41:03.981 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:03.978+0000 7fb415f23640 1 -- 172.21.15.89:0/2257346690 shutdown_connections 2024-09-21T03:41:03.981 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:03.978+0000 7fb415f23640 1 --2- 172.21.15.89:0/2257346690 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb410108d10 0x7fb41010f540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:03.981 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:03.978+0000 7fb415f23640 1 --2- 172.21.15.89:0/2257346690 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb4101083c0 0x7fb4101087d0 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:03.981 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:03.978+0000 7fb415f23640 1 --2- 172.21.15.89:0/2257346690 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb410102a80 0x7fb410102e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:03.981 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:03.978+0000 7fb415f23640 1 -- 172.21.15.89:0/2257346690 >> 172.21.15.89:0/2257346690 conn(0x7fb410079740 msgr2=0x7fb410079b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:41:03.982 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:03.978+0000 7fb415f23640 1 -- 172.21.15.89:0/2257346690 shutdown_connections 2024-09-21T03:41:03.982 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:03.978+0000 7fb415f23640 1 -- 172.21.15.89:0/2257346690 wait complete. 2024-09-21T03:41:03.982 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:03.979+0000 7fb415f23640 1 Processor -- start 2024-09-21T03:41:03.982 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:03.979+0000 7fb415f23640 1 -- start start 2024-09-21T03:41:03.984 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:03.979+0000 7fb415f23640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb410102a80 0x7fb41019a3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:03.984 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:03.979+0000 7fb415f23640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb4101083c0 0x7fb41019a940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:03.984 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:03.979+0000 7fb415f23640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb410108d10 0x7fb41019fbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:03.984 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:03.979+0000 7fb415f23640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fb410113950 con 0x7fb410102a80 2024-09-21T03:41:03.984 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:03.979+0000 7fb415f23640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fb4101137d0 con 0x7fb4101083c0 2024-09-21T03:41:03.984 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:03.979+0000 7fb415f23640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fb410113ad0 con 0x7fb410108d10 2024-09-21T03:41:03.984 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:03.979+0000 7fb40f7fe640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb410102a80 0x7fb41019a3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:03.984 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:03.979+0000 7fb40f7fe640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb410102a80 0x7fb41019a3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:40458/0 (socket says 172.21.15.89:40458) 2024-09-21T03:41:03.984 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:03.979+0000 7fb40f7fe640 1 -- 172.21.15.89:0/2122618678 learned_addr learned my addr 172.21.15.89:0/2122618678 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-21T03:41:03.984 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:03.979+0000 7fb407fff640 1 --2- 172.21.15.89:0/2122618678 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb4101083c0 0x7fb41019a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:03.984 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:03.980+0000 7fb40ffff640 1 --2- 172.21.15.89:0/2122618678 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb410108d10 0x7fb41019fbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:03.984 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:03.980+0000 7fb40f7fe640 1 -- 172.21.15.89:0/2122618678 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb410108d10 msgr2=0x7fb41019fbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:03.985 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:03.980+0000 7fb40f7fe640 1 --2- 172.21.15.89:0/2122618678 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb410108d10 0x7fb41019fbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:03.985 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:03.980+0000 7fb40f7fe640 1 -- 172.21.15.89:0/2122618678 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb4101083c0 msgr2=0x7fb41019a940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:03.985 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:03.980+0000 7fb40f7fe640 1 --2- 172.21.15.89:0/2122618678 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb4101083c0 0x7fb41019a940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:03.985 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:03.980+0000 7fb40f7fe640 1 -- 172.21.15.89:0/2122618678 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4101a0250 con 0x7fb410102a80 2024-09-21T03:41:03.985 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:03.980+0000 7fb40ffff640 1 --2- 172.21.15.89:0/2122618678 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb410108d10 0x7fb41019fbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:41:03.985 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:03.980+0000 7fb407fff640 1 --2- 172.21.15.89:0/2122618678 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb4101083c0 0x7fb41019a940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:41:03.985 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:03.980+0000 7fb40f7fe640 1 --2- 172.21.15.89:0/2122618678 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb410102a80 0x7fb41019a3a0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7fb40000c920 tx=0x7fb40000cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:41:03.985 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:03.980+0000 7fb40d7fa640 1 -- 172.21.15.89:0/2122618678 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb400007c70 con 0x7fb410102a80 2024-09-21T03:41:03.985 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:03.980+0000 7fb415f23640 1 -- 172.21.15.89:0/2122618678 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb4101a0540 con 0x7fb410102a80 2024-09-21T03:41:03.985 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:03.980+0000 7fb40d7fa640 1 -- 172.21.15.89:0/2122618678 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb400007e10 con 0x7fb410102a80 2024-09-21T03:41:03.985 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:03.980+0000 7fb40d7fa640 1 -- 172.21.15.89:0/2122618678 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb400018710 con 0x7fb410102a80 2024-09-21T03:41:03.985 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:03.980+0000 7fb415f23640 1 -- 172.21.15.89:0/2122618678 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb41010feb0 con 0x7fb410102a80 2024-09-21T03:41:03.985 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:03.981+0000 7fb415f23640 1 -- 172.21.15.89:0/2122618678 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb410103d00 con 0x7fb410102a80 2024-09-21T03:41:03.986 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:03.983+0000 7fb40d7fa640 1 -- 172.21.15.89:0/2122618678 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 21) ==== 154512+0+0 (secure 0 0 0) 0x7fb40000f450 con 0x7fb410102a80 2024-09-21T03:41:03.986 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:03.984+0000 7fb40d7fa640 1 --2- 172.21.15.89:0/2122618678 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7fb3fc0b6540 0x7fb3fc0b8a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:03.986 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:03.984+0000 7fb40d7fa640 1 -- 172.21.15.89:0/2122618678 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 9121+0+0 (secure 0 0 0) 0x7fb4000e7b10 con 0x7fb410102a80 2024-09-21T03:41:03.986 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:03.984+0000 7fb407fff640 1 --2- 172.21.15.89:0/2122618678 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7fb3fc0b6540 0x7fb3fc0b8a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:03.987 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:03.984+0000 7fb407fff640 1 --2- 172.21.15.89:0/2122618678 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7fb3fc0b6540 0x7fb3fc0b8a50 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fb3f8006010 tx=0x7fb3f8005de0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:41:03.989 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:03.987+0000 7fb40d7fa640 1 -- 172.21.15.89:0/2122618678 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fb400014030 con 0x7fb410102a80 2024-09-21T03:41:04.206 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:04.203+0000 7fb415f23640 1 -- 172.21.15.89:0/2122618678 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb410102e90 con 0x7fb410102a80 2024-09-21T03:41:04.206 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:04.204+0000 7fb40d7fa640 1 -- 172.21.15.89:0/2122618678 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11912 (secure 0 0 0) 0x7fb4000ae0b0 con 0x7fb410102a80 2024-09-21T03:41:04.207 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-09-21T03:41:04.207 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":26,"btime":"2024-09-21T03:39:26:058611+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14721,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6835/2341538514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":2341538514},{"type":"v1","addr":"172.21.15.89:6835","nonce":2341538514}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14733,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6837/146708592","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6836","nonce":146708592},{"type":"v1","addr":"172.21.15.89:6837","nonce":146708592}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14781,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6839/2269965499","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6838","nonce":2269965499},{"type":"v1","addr":"172.21.15.89:6839","nonce":2269965499}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14793,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6841/650850383","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6840","nonce":650850383},{"type":"v1","addr":"172.21.15.89:6841","nonce":650850383}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24565,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6833/752336912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6832","nonce":752336912},{"type":"v1","addr":"172.21.15.114:6833","nonce":752336912}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24575,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6833/127607331","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6832","nonce":127607331},{"type":"v1","addr":"172.21.15.110:6833","nonce":127607331}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T03:37:13.570694+0000","modified":"2024-09-21T03:39:25.333954+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24638,"mds_1":24631,"mds_2":24614},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24593":{"gid":24593,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.110:6835/671820654","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6834","nonce":671820654},{"type":"v1","addr":"172.21.15.110:6835","nonce":671820654}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24604":{"gid":24604,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.114:6835/1049714548","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6834","nonce":1049714548},{"type":"v1","addr":"172.21.15.114:6835","nonce":1049714548}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24610":{"gid":24610,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.114:6837/1941213818","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6836","nonce":1941213818},{"type":"v1","addr":"172.21.15.114:6837","nonce":1941213818}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24614":{"gid":24614,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":15,"addr":"172.21.15.110:6837/4047124589","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6836","nonce":4047124589},{"type":"v1","addr":"172.21.15.110:6837","nonce":4047124589}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24631":{"gid":24631,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":15,"addr":"172.21.15.114:6839/3427369944","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6838","nonce":3427369944},{"type":"v1","addr":"172.21.15.114:6839","nonce":3427369944}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24638":{"gid":24638,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.110:6839/132366709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6838","nonce":132366709},{"type":"v1","addr":"172.21.15.110:6839","nonce":132366709}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24638,"qdb_cluster":[24638,24631,24614]},"id":1}]} 2024-09-21T03:41:04.207 INFO:teuthology.orchestra.run.smithi089.stderr:dumped fsmap epoch 26 2024-09-21T03:41:04.209 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:04.206+0000 7fb415f23640 1 -- 172.21.15.89:0/2122618678 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7fb3fc0b6540 msgr2=0x7fb3fc0b8a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:04.209 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:04.206+0000 7fb415f23640 1 --2- 172.21.15.89:0/2122618678 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7fb3fc0b6540 0x7fb3fc0b8a50 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fb3f8006010 tx=0x7fb3f8005de0 comp rx=0 tx=0).stop 2024-09-21T03:41:04.209 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:04.206+0000 7fb415f23640 1 -- 172.21.15.89:0/2122618678 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb410102a80 msgr2=0x7fb41019a3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:04.209 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:04.206+0000 7fb415f23640 1 --2- 172.21.15.89:0/2122618678 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb410102a80 0x7fb41019a3a0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7fb40000c920 tx=0x7fb40000cde0 comp rx=0 tx=0).stop 2024-09-21T03:41:04.209 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:04.206+0000 7fb415f23640 1 -- 172.21.15.89:0/2122618678 shutdown_connections 2024-09-21T03:41:04.209 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:04.206+0000 7fb415f23640 1 --2- 172.21.15.89:0/2122618678 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7fb3fc0b6540 0x7fb3fc0b8a50 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:04.209 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:04.206+0000 7fb415f23640 1 --2- 172.21.15.89:0/2122618678 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb4101083c0 0x7fb41019a940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:04.209 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:04.206+0000 7fb415f23640 1 --2- 172.21.15.89:0/2122618678 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb410108d10 0x7fb41019fbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:04.209 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:04.206+0000 7fb415f23640 1 --2- 172.21.15.89:0/2122618678 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb410102a80 0x7fb41019a3a0 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:04.210 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:04.206+0000 7fb415f23640 1 -- 172.21.15.89:0/2122618678 >> 172.21.15.89:0/2122618678 conn(0x7fb410079740 msgr2=0x7fb410105310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:41:04.210 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:04.207+0000 7fb415f23640 1 -- 172.21.15.89:0/2122618678 shutdown_connections 2024-09-21T03:41:04.210 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:04.207+0000 7fb415f23640 1 -- 172.21.15.89:0/2122618678 wait complete. 2024-09-21T03:41:04.811 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:41:04 smithi089 ceph-mon[30928]: pgmap v410: 129 pgs: 129 active+clean; 192 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 661 KiB/s wr, 9 op/s 2024-09-21T03:41:04.811 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:41:04 smithi089 ceph-mon[30928]: from='client.? 172.21.15.89:0/2122618678' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:41:04.894 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24638 quiesce db --set-id 1beabdc5 --release --await' 2024-09-21T03:41:04.894 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 7c374098-77c9-11ef-baf5-efdc52797490 -- timeout 105 ceph tell mds.24638 quiesce db --set-id 1beabdc5 --release --await 2024-09-21T03:41:05.141 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/7c374098-77c9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:41:05.211 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:41:04 smithi114 ceph-mon[34140]: pgmap v410: 129 pgs: 129 active+clean; 192 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 661 KiB/s wr, 9 op/s 2024-09-21T03:41:05.211 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:41:04 smithi114 ceph-mon[34140]: from='client.? 172.21.15.89:0/2122618678' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:41:05.250 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:41:04 smithi110 ceph-mon[34000]: pgmap v410: 129 pgs: 129 active+clean; 192 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 661 KiB/s wr, 9 op/s 2024-09-21T03:41:05.250 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:41:04 smithi110 ceph-mon[34000]: from='client.? 172.21.15.89:0/2122618678' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:41:05.624 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:05.618+0000 7fdafab4d640 1 -- 172.21.15.89:0/2244508180 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdaf4108cf0 msgr2=0x7fdaf410f520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:05.624 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:05.618+0000 7fdafab4d640 1 --2- 172.21.15.89:0/2244508180 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdaf4108cf0 0x7fdaf410f520 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7fdaec009ca0 tx=0x7fdaec02a8c0 comp rx=0 tx=0).stop 2024-09-21T03:41:05.624 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:05.618+0000 7fdafab4d640 1 -- 172.21.15.89:0/2244508180 shutdown_connections 2024-09-21T03:41:05.624 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:05.618+0000 7fdafab4d640 1 --2- 172.21.15.89:0/2244508180 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdaf4108cf0 0x7fdaf410f520 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:05.624 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:05.618+0000 7fdafab4d640 1 --2- 172.21.15.89:0/2244508180 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fdaf41083a0 0x7fdaf41087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:05.624 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:05.618+0000 7fdafab4d640 1 --2- 172.21.15.89:0/2244508180 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdaf4102a60 0x7fdaf4102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:05.625 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:05.618+0000 7fdafab4d640 1 -- 172.21.15.89:0/2244508180 >> 172.21.15.89:0/2244508180 conn(0x7fdaf4079720 msgr2=0x7fdaf4079b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:41:05.625 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:05.618+0000 7fdafab4d640 1 -- 172.21.15.89:0/2244508180 shutdown_connections 2024-09-21T03:41:05.625 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:05.618+0000 7fdafab4d640 1 -- 172.21.15.89:0/2244508180 wait complete. 2024-09-21T03:41:05.625 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:05.619+0000 7fdafab4d640 1 Processor -- start 2024-09-21T03:41:05.625 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:05.619+0000 7fdafab4d640 1 -- start start 2024-09-21T03:41:05.625 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:05.619+0000 7fdafab4d640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdaf4102a60 0x7fdaf419a390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:05.625 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:05.619+0000 7fdafab4d640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fdaf41083a0 0x7fdaf419a930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:05.625 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:05.619+0000 7fdafab4d640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdaf4108cf0 0x7fdaf419fbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:05.625 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:05.619+0000 7fdafab4d640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fdaf41139e0 con 0x7fdaf4108cf0 2024-09-21T03:41:05.625 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:05.619+0000 7fdafab4d640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fdaf4113860 con 0x7fdaf4102a60 2024-09-21T03:41:05.625 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:05.619+0000 7fdafab4d640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fdaf4113b60 con 0x7fdaf41083a0 2024-09-21T03:41:05.625 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:05.619+0000 7fdaf90ea640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdaf4108cf0 0x7fdaf419fbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:05.625 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:05.619+0000 7fdaf90ea640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdaf4108cf0 0x7fdaf419fbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:40486/0 (socket says 172.21.15.89:40486) 2024-09-21T03:41:05.625 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:05.619+0000 7fdaf90ea640 1 -- 172.21.15.89:0/2018233316 learned_addr learned my addr 172.21.15.89:0/2018233316 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-21T03:41:05.626 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:05.620+0000 7fdaf88e9640 1 --2- 172.21.15.89:0/2018233316 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdaf4102a60 0x7fdaf419a390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:05.626 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:05.620+0000 7fdaf3fff640 1 --2- 172.21.15.89:0/2018233316 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fdaf41083a0 0x7fdaf419a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:05.626 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:05.620+0000 7fdaf90ea640 1 -- 172.21.15.89:0/2018233316 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fdaf41083a0 msgr2=0x7fdaf419a930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:05.626 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:05.620+0000 7fdaf90ea640 1 --2- 172.21.15.89:0/2018233316 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fdaf41083a0 0x7fdaf419a930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:05.626 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:05.620+0000 7fdaf90ea640 1 -- 172.21.15.89:0/2018233316 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdaf4102a60 msgr2=0x7fdaf419a390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:05.626 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:05.620+0000 7fdaf90ea640 1 --2- 172.21.15.89:0/2018233316 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdaf4102a60 0x7fdaf419a390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:05.626 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:05.620+0000 7fdaf90ea640 1 -- 172.21.15.89:0/2018233316 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdaf41a0300 con 0x7fdaf4108cf0 2024-09-21T03:41:05.626 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:05.620+0000 7fdaf88e9640 1 --2- 172.21.15.89:0/2018233316 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdaf4102a60 0x7fdaf419a390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:41:05.626 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:05.620+0000 7fdaf90ea640 1 --2- 172.21.15.89:0/2018233316 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdaf4108cf0 0x7fdaf419fbd0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fdaec005cc0 tx=0x7fdaec032c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:41:05.626 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:05.620+0000 7fdaf1ffb640 1 -- 172.21.15.89:0/2018233316 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdaec004040 con 0x7fdaf4108cf0 2024-09-21T03:41:05.626 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:05.620+0000 7fdaf1ffb640 1 -- 172.21.15.89:0/2018233316 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fdaec004670 con 0x7fdaf4108cf0 2024-09-21T03:41:05.626 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:05.620+0000 7fdafab4d640 1 -- 172.21.15.89:0/2018233316 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdaf41a0590 con 0x7fdaf4108cf0 2024-09-21T03:41:05.626 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:05.620+0000 7fdaf1ffb640 1 -- 172.21.15.89:0/2018233316 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdaec030cc0 con 0x7fdaf4108cf0 2024-09-21T03:41:05.626 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:05.620+0000 7fdafab4d640 1 -- 172.21.15.89:0/2018233316 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdaf41a6d50 con 0x7fdaf4108cf0 2024-09-21T03:41:05.631 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:05.623+0000 7fdb010c1740 1 Processor -- start 2024-09-21T03:41:05.631 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:05.623+0000 7fdaf1ffb640 1 -- 172.21.15.89:0/2018233316 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 21) ==== 154512+0+0 (secure 0 0 0) 0x7fdaec04e440 con 0x7fdaf4108cf0 2024-09-21T03:41:05.631 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:05.623+0000 7fdb010c1740 1 -- start start 2024-09-21T03:41:05.631 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:05.624+0000 7fdb010c1740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558331b72520 0x558331f18510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:05.631 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:05.624+0000 7fdb010c1740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558331b49660 0x558331f18a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:05.632 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:05.624+0000 7fdb010c1740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558331b51840 0x558331f18f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:05.632 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:05.624+0000 7fdb010c1740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x558331e03dc0 con 0x558331b72520 2024-09-21T03:41:05.632 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:05.624+0000 7fdb010c1740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x558331e901f0 con 0x558331b49660 2024-09-21T03:41:05.632 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:05.624+0000 7fdb010c1740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x558331e7f620 con 0x558331b51840 2024-09-21T03:41:05.632 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:05.624+0000 7fdaf3fff640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558331b72520 0x558331f18510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:05.632 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:05.624+0000 7fdaf3fff640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558331b72520 0x558331f18510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:40502/0 (socket says 172.21.15.89:40502) 2024-09-21T03:41:05.632 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:05.624+0000 7fdaf3fff640 1 -- 172.21.15.89:0/2822467454 learned_addr learned my addr 172.21.15.89:0/2822467454 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-21T03:41:05.632 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:05.624+0000 7fdaf88e9640 1 --2- 172.21.15.89:0/2822467454 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558331b49660 0x558331f18a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:05.632 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:05.624+0000 7fdaf3fff640 1 --2- 172.21.15.89:0/2822467454 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558331b51840 0x558331f18f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:05.632 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:05.624+0000 7fdaf3fff640 1 -- 172.21.15.89:0/2822467454 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558331b51840 msgr2=0x558331f18f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:41:05.632 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:05.624+0000 7fdaf3fff640 1 --2- 172.21.15.89:0/2822467454 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558331b51840 0x558331f18f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:05.632 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:05.624+0000 7fdaf3fff640 1 -- 172.21.15.89:0/2822467454 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558331b49660 msgr2=0x558331f18a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:41:05.632 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:05.624+0000 7fdaf3fff640 1 --2- 172.21.15.89:0/2822467454 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558331b49660 0x558331f18a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:05.632 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:05.624+0000 7fdaf3fff640 1 -- 172.21.15.89:0/2822467454 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558331e47fd0 con 0x558331b72520 2024-09-21T03:41:05.632 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:05.624+0000 7fdaf1ffb640 1 --2- 172.21.15.89:0/2018233316 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7fdadc0b6510 0x7fdadc0b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:05.633 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:05.624+0000 7fdaf88e9640 1 --2- 172.21.15.89:0/2018233316 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7fdadc0b6510 0x7fdadc0b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:05.633 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:05.624+0000 7fdaf3fff640 1 --2- 172.21.15.89:0/2822467454 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558331b72520 0x558331f18510 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7fdae800cfc0 tx=0x7fdae800c5a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9d350e3b97c87a6 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:41:05.633 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:05.625+0000 7fdaf1ffb640 1 -- 172.21.15.89:0/2018233316 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 9121+0+0 (secure 0 0 0) 0x7fdaec108c80 con 0x7fdaf4108cf0 2024-09-21T03:41:05.633 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:05.625+0000 7fdadaffd640 1 -- 172.21.15.89:0/2822467454 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdae8018030 con 0x558331b72520 2024-09-21T03:41:05.633 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:05.625+0000 7fdadaffd640 1 -- 172.21.15.89:0/2822467454 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fdae80054b0 con 0x558331b72520 2024-09-21T03:41:05.633 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:05.625+0000 7fdadaffd640 1 -- 172.21.15.89:0/2822467454 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdae800d550 con 0x558331b72520 2024-09-21T03:41:05.633 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:05.626+0000 7fdb010c1740 1 -- 172.21.15.89:0/2822467454 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558331b72520 msgr2=0x558331f18510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:05.633 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:05.626+0000 7fdaf88e9640 1 --2- 172.21.15.89:0/2018233316 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7fdadc0b6510 0x7fdadc0b8a20 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x558331f17190 tx=0x7fdae4008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:41:05.633 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:05.626+0000 7fdb010c1740 1 --2- 172.21.15.89:0/2822467454 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558331b72520 0x558331f18510 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7fdae800cfc0 tx=0x7fdae800c5a0 comp rx=0 tx=0).stop 2024-09-21T03:41:05.633 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:05.626+0000 7fdb010c1740 1 -- 172.21.15.89:0/2822467454 shutdown_connections 2024-09-21T03:41:05.633 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:05.626+0000 7fdb010c1740 1 --2- 172.21.15.89:0/2822467454 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558331b49660 0x558331f18a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:05.633 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:05.626+0000 7fdb010c1740 1 --2- 172.21.15.89:0/2822467454 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558331b51840 0x558331f18f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:05.633 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:05.626+0000 7fdb010c1740 1 --2- 172.21.15.89:0/2822467454 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558331b72520 0x558331f18510 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:05.634 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:05.626+0000 7fdb010c1740 1 -- 172.21.15.89:0/2822467454 >> 172.21.15.89:0/2822467454 conn(0x558331b66910 msgr2=0x558331f16400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:41:05.634 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:05.626+0000 7fdb010c1740 1 -- 172.21.15.89:0/2822467454 shutdown_connections 2024-09-21T03:41:05.634 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:05.626+0000 7fdb010c1740 1 -- 172.21.15.89:0/2822467454 wait complete. 2024-09-21T03:41:05.634 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:05.626+0000 7fdb010c1740 1 -- start start 2024-09-21T03:41:05.634 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:05.626+0000 7fdb010c1740 1 Processor -- start 2024-09-21T03:41:05.634 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:05.627+0000 7fdb010c1740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558331b49660 0x558331f23c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:05.634 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:05.627+0000 7fdb010c1740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558331b51840 0x558331f241d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:05.634 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:05.627+0000 7fdb010c1740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558331b72520 0x558331f342a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:05.634 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:05.627+0000 7fdb010c1740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x558331e1a300 con 0x558331b72520 2024-09-21T03:41:05.634 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:05.627+0000 7fdb010c1740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x558331e83430 con 0x558331b51840 2024-09-21T03:41:05.634 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:05.627+0000 7fdb010c1740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x558331ce0bb0 con 0x558331b49660 2024-09-21T03:41:05.634 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:05.627+0000 7fdb010c1740 10 client.0 _ll_register_callbacks cb 0x558331e34120 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:41:05.634 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:05.627+0000 7fdaf90ea640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558331b72520 0x558331f342a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:05.634 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:05.627+0000 7fdaf90ea640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558331b72520 0x558331f342a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:40510/0 (socket says 172.21.15.89:40510) 2024-09-21T03:41:05.634 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:05.627+0000 7fdaf90ea640 1 -- 172.21.15.89:0/4202057245 learned_addr learned my addr 172.21.15.89:0/4202057245 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-21T03:41:05.635 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:05.627+0000 7fdaf3fff640 1 --2- 172.21.15.89:0/4202057245 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558331b51840 0x558331f241d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:05.635 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:05.627+0000 7fdaf88e9640 1 --2- 172.21.15.89:0/4202057245 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558331b49660 0x558331f23c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:05.635 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:05.627+0000 7fdaf90ea640 1 -- 172.21.15.89:0/4202057245 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558331b49660 msgr2=0x558331f23c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:41:05.635 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:05.627+0000 7fdaf90ea640 1 --2- 172.21.15.89:0/4202057245 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558331b49660 0x558331f23c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:05.635 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:05.627+0000 7fdaf90ea640 1 -- 172.21.15.89:0/4202057245 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558331b51840 msgr2=0x558331f241d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:41:05.635 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:05.627+0000 7fdaf90ea640 1 --2- 172.21.15.89:0/4202057245 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558331b51840 0x558331f241d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:05.635 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:05.627+0000 7fdaf90ea640 1 -- 172.21.15.89:0/4202057245 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558331e09780 con 0x558331b72520 2024-09-21T03:41:05.635 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:05.627+0000 7fdaf90ea640 1 --2- 172.21.15.89:0/4202057245 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558331b72520 0x558331f342a0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7fdaec0cb060 tx=0x7fdaec0ce920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ce67bb7ebef9c8b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:41:05.635 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:05.627+0000 7fdad3fff640 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2024-09-21T03:41:05.635 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:05.627+0000 7fdad3fff640 1 -- 172.21.15.89:0/4202057245 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdaec004820 con 0x558331b72520 2024-09-21T03:41:05.635 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:05.627+0000 7fdad3fff640 1 -- 172.21.15.89:0/4202057245 <== mon.0 v2:172.21.15.89:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 9121+0+0 (secure 0 0 0) 0x7fdaec0d8020 con 0x558331b72520 2024-09-21T03:41:05.635 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:05.627+0000 7fdb010c1740 1 -- 172.21.15.89:0/4202057245 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdae80054b0 con 0x558331b72520 2024-09-21T03:41:05.635 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:05.628+0000 7fdad3fff640 1 -- 172.21.15.89:0/4202057245 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fdaec0098d0 con 0x558331b72520 2024-09-21T03:41:05.635 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:05.628+0000 7fdb010c1740 10 client.15747 fetch_fsmap learned FSMap version 26 2024-09-21T03:41:05.636 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:05.628+0000 7fdb010c1740 1 -- 172.21.15.89:0/4202057245 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x558331bddb00 con 0x558331b72520 2024-09-21T03:41:05.636 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:05.628+0000 7fdad3fff640 1 -- 172.21.15.89:0/4202057245 <== mon.0 v2:172.21.15.89:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fdae80054b0 con 0x558331b72520 2024-09-21T03:41:05.636 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:05.628+0000 7fdad3fff640 1 -- 172.21.15.89:0/4202057245 <== mon.0 v2:172.21.15.89:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fdaec0cdcf0 con 0x558331b72520 2024-09-21T03:41:05.636 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:05.628+0000 7fdb010c1740 10 client.15747 fetch_fsmap finished waiting for FSMap version 26 2024-09-21T03:41:05.636 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:05.628+0000 7fdb010c1740 10 client.15747 resolve_mds: validated gid 24638 aka daemon mds.b 2024-09-21T03:41:05.636 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:05.628+0000 7fdb010c1740 20 client.15747 populate_metadata read hostname 'smithi089' 2024-09-21T03:41:05.636 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:05.628+0000 7fdb010c1740 1 --2- 172.21.15.89:0/4202057245 >> [v2:172.21.15.110:6838/132366709,v1:172.21.15.110:6839/132366709] conn(0x558331b30ea0 0x558331f3a540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:05.636 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:05.628+0000 7fdb010c1740 4 client.15747 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:41:05.636 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:05.628+0000 7fdb010c1740 1 -- 172.21.15.89:0/4202057245 --> [v2:172.21.15.110:6838/132366709,v1:172.21.15.110:6839/132366709] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558331e92140 con 0x558331b30ea0 2024-09-21T03:41:05.636 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:05.628+0000 7fdaf88e9640 1 --2- 172.21.15.89:0/4202057245 >> [v2:172.21.15.110:6838/132366709,v1:172.21.15.110:6839/132366709] conn(0x558331b30ea0 0x558331f3a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:05.636 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:05.629+0000 7fdad3fff640 10 client.15747 ms_handle_connect on v2:172.21.15.110:6838/132366709 2024-09-21T03:41:05.636 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:05.629+0000 7fdaf88e9640 1 --2- 172.21.15.89:0/4202057245 >> [v2:172.21.15.110:6838/132366709,v1:172.21.15.110:6839/132366709] conn(0x558331b30ea0 0x558331f3a540 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c4ccd595b4849812 server_cookie=283ca592aa44cf5 in_seq=0 out_seq=0 2024-09-21T03:41:05.637 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:05.630+0000 7fdad3fff640 1 -- 172.21.15.89:0/4202057245 <== mds.0 v2:172.21.15.110:6838/132366709 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x558331e92140 con 0x558331b30ea0 2024-09-21T03:41:05.637 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:05.630+0000 7fdad3fff640 10 client.15747 handle_command_reply: tid=0 2024-09-21T03:41:05.637 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:05.630+0000 7fdb010c1740 1 client.15747 shutdown 2024-09-21T03:41:05.637 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:05.630+0000 7fdb010c1740 1 -- 172.21.15.89:0/4202057245 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558331b72520 msgr2=0x558331f342a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:05.638 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:05.630+0000 7fdb010c1740 1 --2- 172.21.15.89:0/4202057245 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558331b72520 0x558331f342a0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7fdaec0cb060 tx=0x7fdaec0ce920 comp rx=0 tx=0).stop 2024-09-21T03:41:05.638 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:05.630+0000 7fdb010c1740 1 -- 172.21.15.89:0/4202057245 shutdown_connections 2024-09-21T03:41:05.638 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:05.630+0000 7fdb010c1740 1 --2- 172.21.15.89:0/4202057245 >> [v2:172.21.15.110:6838/132366709,v1:172.21.15.110:6839/132366709] conn(0x558331b30ea0 0x558331f3a540 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:05.638 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:05.630+0000 7fdad3fff640 0 client.15747 ms_handle_reset on v2:172.21.15.110:6838/132366709 2024-09-21T03:41:05.638 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:05.630+0000 7fdb010c1740 1 --2- 172.21.15.89:0/4202057245 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558331b51840 0x558331f241d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:05.638 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:05.630+0000 7fdb010c1740 1 --2- 172.21.15.89:0/4202057245 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558331b72520 0x558331f342a0 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:05.638 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:05.630+0000 7fdb010c1740 1 --2- 172.21.15.89:0/4202057245 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558331b49660 0x558331f23c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:05.638 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:05.630+0000 7fdb010c1740 1 -- 172.21.15.89:0/4202057245 >> 172.21.15.89:0/4202057245 conn(0x558331b66910 msgr2=0x558331f1b6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:41:05.638 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:05.630+0000 7fdb010c1740 1 -- 172.21.15.89:0/4202057245 shutdown_connections 2024-09-21T03:41:05.638 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:05.630+0000 7fdb010c1740 1 -- 172.21.15.89:0/4202057245 wait complete. 2024-09-21T03:41:05.638 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:05.630+0000 7fdb010c1740 20 client.15747 trim_cache size 0 max 16384 2024-09-21T03:41:05.650 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:05.647+0000 7fdb010c1740 1 Processor -- start 2024-09-21T03:41:05.650 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:05.647+0000 7fdb010c1740 1 -- start start 2024-09-21T03:41:05.650 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:05.647+0000 7fdb010c1740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558331b30ea0 0x558331f19c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:05.650 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:05.647+0000 7fdb010c1740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558331b49660 0x558331f1d5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:05.650 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:05.647+0000 7fdb010c1740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558331b51840 0x558331f1db30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:05.650 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:05.647+0000 7fdb010c1740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fdaf41b0ff0 con 0x558331b30ea0 2024-09-21T03:41:05.651 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:05.647+0000 7fdb010c1740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fdaf41ae8c0 con 0x558331b49660 2024-09-21T03:41:05.651 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:05.647+0000 7fdb010c1740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fdaf41ac710 con 0x558331b51840 2024-09-21T03:41:05.651 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:05.647+0000 7fdaf88e9640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558331b30ea0 0x558331f19c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:05.651 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:05.647+0000 7fdaf88e9640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558331b30ea0 0x558331f19c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:40524/0 (socket says 172.21.15.89:40524) 2024-09-21T03:41:05.651 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:05.647+0000 7fdaf88e9640 1 -- 172.21.15.89:0/1438464649 learned_addr learned my addr 172.21.15.89:0/1438464649 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-21T03:41:05.651 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:05.647+0000 7fdaf90ea640 1 --2- 172.21.15.89:0/1438464649 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558331b51840 0x558331f1db30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:05.651 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:05.648+0000 7fdaf3fff640 1 --2- 172.21.15.89:0/1438464649 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558331b49660 0x558331f1d5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:05.651 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:05.648+0000 7fdaf88e9640 1 -- 172.21.15.89:0/1438464649 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558331b51840 msgr2=0x558331f1db30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:41:05.651 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:05.648+0000 7fdaf88e9640 1 --2- 172.21.15.89:0/1438464649 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558331b51840 0x558331f1db30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:05.651 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:05.648+0000 7fdaf88e9640 1 -- 172.21.15.89:0/1438464649 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558331b49660 msgr2=0x558331f1d5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:41:05.651 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:05.648+0000 7fdaf88e9640 1 --2- 172.21.15.89:0/1438464649 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558331b49660 0x558331f1d5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:05.651 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:05.648+0000 7fdaf88e9640 1 -- 172.21.15.89:0/1438464649 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558331c54960 con 0x558331b30ea0 2024-09-21T03:41:05.651 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:05.648+0000 7fdaf88e9640 1 --2- 172.21.15.89:0/1438464649 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558331b30ea0 0x558331f19c50 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fdae400fb90 tx=0x7fdae4012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55f451dcde85a30 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:41:05.651 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:05.648+0000 7fdada7fc640 1 -- 172.21.15.89:0/1438464649 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdae401b040 con 0x558331b30ea0 2024-09-21T03:41:05.652 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:05.648+0000 7fdada7fc640 1 -- 172.21.15.89:0/1438464649 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fdae402e070 con 0x558331b30ea0 2024-09-21T03:41:05.652 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:05.648+0000 7fdada7fc640 1 -- 172.21.15.89:0/1438464649 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdae4015670 con 0x558331b30ea0 2024-09-21T03:41:05.654 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:05.649+0000 7fdb010c1740 1 -- 172.21.15.89:0/1438464649 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558331b30ea0 msgr2=0x558331f19c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:05.654 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:05.649+0000 7fdb010c1740 1 --2- 172.21.15.89:0/1438464649 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558331b30ea0 0x558331f19c50 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fdae400fb90 tx=0x7fdae4012000 comp rx=0 tx=0).stop 2024-09-21T03:41:05.655 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:05.649+0000 7fdb010c1740 1 -- 172.21.15.89:0/1438464649 shutdown_connections 2024-09-21T03:41:05.655 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:05.649+0000 7fdb010c1740 1 --2- 172.21.15.89:0/1438464649 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558331b49660 0x558331f1d5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:05.655 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:05.649+0000 7fdb010c1740 1 --2- 172.21.15.89:0/1438464649 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558331b51840 0x558331f1db30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:05.655 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:05.649+0000 7fdb010c1740 1 --2- 172.21.15.89:0/1438464649 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558331b30ea0 0x558331f19c50 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:05.655 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:05.649+0000 7fdb010c1740 1 -- 172.21.15.89:0/1438464649 >> 172.21.15.89:0/1438464649 conn(0x558331b66910 msgr2=0x558331f19540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:41:05.655 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:05.649+0000 7fdb010c1740 1 -- 172.21.15.89:0/1438464649 shutdown_connections 2024-09-21T03:41:05.655 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:05.649+0000 7fdb010c1740 1 -- 172.21.15.89:0/1438464649 wait complete. 2024-09-21T03:41:05.655 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:05.649+0000 7fdb010c1740 1 -- start start 2024-09-21T03:41:05.655 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:05.649+0000 7fdb010c1740 1 Processor -- start 2024-09-21T03:41:05.655 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:05.650+0000 7fdb010c1740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558331b30ea0 0x558331f2c640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:05.656 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:05.650+0000 7fdb010c1740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558331b49660 0x558331f2cb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:05.656 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:05.650+0000 7fdb010c1740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558331b51840 0x558331fa3f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:05.656 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:05.650+0000 7fdb010c1740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fdaf41acd80 con 0x558331b49660 2024-09-21T03:41:05.656 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:05.650+0000 7fdb010c1740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fdaf41ae1f0 con 0x558331b51840 2024-09-21T03:41:05.656 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:05.650+0000 7fdb010c1740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fdaf41ad450 con 0x558331b30ea0 2024-09-21T03:41:05.656 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:05.650+0000 7fdb010c1740 10 client.0 _ll_register_callbacks cb 0x7fdaec009460 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:41:05.656 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:05.650+0000 7fdaf3fff640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558331b49660 0x558331f2cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:05.656 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:05.650+0000 7fdaf3fff640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558331b49660 0x558331f2cb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:40536/0 (socket says 172.21.15.89:40536) 2024-09-21T03:41:05.656 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:05.650+0000 7fdaf3fff640 1 -- 172.21.15.89:0/3904565645 learned_addr learned my addr 172.21.15.89:0/3904565645 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-21T03:41:05.656 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:05.650+0000 7fdaf88e9640 1 --2- 172.21.15.89:0/3904565645 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558331b30ea0 0x558331f2c640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:05.656 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:05.650+0000 7fdaf3fff640 1 -- 172.21.15.89:0/3904565645 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558331b30ea0 msgr2=0x558331f2c640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:41:05.656 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:05.650+0000 7fdaf3fff640 1 --2- 172.21.15.89:0/3904565645 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558331b30ea0 0x558331f2c640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:05.656 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:05.650+0000 7fdaf3fff640 1 -- 172.21.15.89:0/3904565645 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558331b51840 msgr2=0x558331fa3f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:41:05.656 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:05.650+0000 7fdaf90ea640 1 --2- 172.21.15.89:0/3904565645 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558331b51840 0x558331fa3f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:05.657 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:05.650+0000 7fdaf3fff640 1 --2- 172.21.15.89:0/3904565645 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558331b51840 0x558331fa3f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:05.657 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:05.650+0000 7fdaf3fff640 1 -- 172.21.15.89:0/3904565645 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558331fa4480 con 0x558331b49660 2024-09-21T03:41:05.657 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:05.650+0000 7fdaf90ea640 1 --2- 172.21.15.89:0/3904565645 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558331b51840 0x558331fa3f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:41:05.657 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:05.651+0000 7fdaf3fff640 1 --2- 172.21.15.89:0/3904565645 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558331b49660 0x558331f2cb80 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7fdae8012410 tx=0x7fdae8004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=243cd52d9aad552 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:41:05.657 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:05.651+0000 7fdad3fff640 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2024-09-21T03:41:05.657 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:05.651+0000 7fdad3fff640 1 -- 172.21.15.89:0/3904565645 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdae8018040 con 0x558331b49660 2024-09-21T03:41:05.657 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:05.651+0000 7fdad3fff640 1 -- 172.21.15.89:0/3904565645 <== mon.0 v2:172.21.15.89:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 9121+0+0 (secure 0 0 0) 0x7fdae8004220 con 0x558331b49660 2024-09-21T03:41:05.657 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:05.651+0000 7fdb010c1740 1 -- 172.21.15.89:0/3904565645 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdae402e070 con 0x558331b49660 2024-09-21T03:41:05.657 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:05.651+0000 7fdad3fff640 1 -- 172.21.15.89:0/3904565645 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fdae8004420 con 0x558331b49660 2024-09-21T03:41:05.657 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:05.651+0000 7fdb010c1740 10 client.15753 fetch_fsmap learned FSMap version 26 2024-09-21T03:41:05.657 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:05.651+0000 7fdb010c1740 1 -- 172.21.15.89:0/3904565645 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x558331fa55e0 con 0x558331b49660 2024-09-21T03:41:05.657 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:05.651+0000 7fdad3fff640 1 -- 172.21.15.89:0/3904565645 <== mon.0 v2:172.21.15.89:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fdae402e070 con 0x558331b49660 2024-09-21T03:41:05.657 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:05.651+0000 7fdad3fff640 1 -- 172.21.15.89:0/3904565645 <== mon.0 v2:172.21.15.89:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fdae8026070 con 0x558331b49660 2024-09-21T03:41:05.657 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:05.651+0000 7fdb010c1740 10 client.15753 fetch_fsmap finished waiting for FSMap version 26 2024-09-21T03:41:05.657 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:05.651+0000 7fdb010c1740 10 client.15753 resolve_mds: validated gid 24638 aka daemon mds.b 2024-09-21T03:41:05.658 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:05.651+0000 7fdb010c1740 20 client.15753 populate_metadata read hostname 'smithi089' 2024-09-21T03:41:05.658 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:05.651+0000 7fdb010c1740 1 --2- 172.21.15.89:0/3904565645 >> [v2:172.21.15.110:6838/132366709,v1:172.21.15.110:6839/132366709] conn(0x558331b72520 0x558331fa7c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:05.658 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:05.651+0000 7fdb010c1740 4 client.15753 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "1beabdc5", "release": true, "await": true}] 2024-09-21T03:41:05.658 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:05.651+0000 7fdb010c1740 1 -- 172.21.15.89:0/3904565645 --> [v2:172.21.15.110:6838/132366709,v1:172.21.15.110:6839/132366709] -- command(tid 0: {"prefix": "quiesce db", "set_id": "1beabdc5", "release": true, "await": true}) -- 0x558331e8dbf0 con 0x558331b72520 2024-09-21T03:41:05.658 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:05.652+0000 7fdaf88e9640 1 --2- 172.21.15.89:0/3904565645 >> [v2:172.21.15.110:6838/132366709,v1:172.21.15.110:6839/132366709] conn(0x558331b72520 0x558331fa7c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:05.658 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:05.652+0000 7fdad3fff640 10 client.15753 ms_handle_connect on v2:172.21.15.110:6838/132366709 2024-09-21T03:41:05.658 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:05.652+0000 7fdaf88e9640 1 --2- 172.21.15.89:0/3904565645 >> [v2:172.21.15.110:6838/132366709,v1:172.21.15.110:6839/132366709] conn(0x558331b72520 0x558331fa7c30 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3cb2e7b8984b680a server_cookie=ff0b397977aba15e in_seq=0 out_seq=0 2024-09-21T03:41:05.659 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:05.654+0000 7fdad3fff640 1 -- 172.21.15.89:0/3904565645 <== mds.0 v2:172.21.15.110:6838/132366709 1 ==== command_reply(tid 0: 0 ) ==== 8+0+589 (crc 0 0 0) 0x558331e8dbf0 con 0x558331b72520 2024-09-21T03:41:05.659 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:05.654+0000 7fdad3fff640 10 client.15753 handle_command_reply: tid=0 2024-09-21T03:41:05.659 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:05.654+0000 7fdb010c1740 1 client.15753 shutdown 2024-09-21T03:41:05.659 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:05.654+0000 7fdb010c1740 1 -- 172.21.15.89:0/3904565645 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558331b49660 msgr2=0x558331f2cb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:05.660 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:05.654+0000 7fdb010c1740 1 --2- 172.21.15.89:0/3904565645 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558331b49660 0x558331f2cb80 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7fdae8012410 tx=0x7fdae8004ad0 comp rx=0 tx=0).stop 2024-09-21T03:41:05.660 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:05.654+0000 7fdb010c1740 1 -- 172.21.15.89:0/3904565645 shutdown_connections 2024-09-21T03:41:05.660 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:05.654+0000 7fdb010c1740 1 --2- 172.21.15.89:0/3904565645 >> [v2:172.21.15.110:6838/132366709,v1:172.21.15.110:6839/132366709] conn(0x558331b72520 0x558331fa7c30 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:05.660 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:05.654+0000 7fdb010c1740 1 --2- 172.21.15.89:0/3904565645 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558331b51840 0x558331fa3f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:05.660 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:05.654+0000 7fdad3fff640 0 client.15753 ms_handle_reset on v2:172.21.15.110:6838/132366709 2024-09-21T03:41:05.660 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:05.654+0000 7fdb010c1740 1 --2- 172.21.15.89:0/3904565645 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558331b49660 0x558331f2cb80 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:05.660 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:05.654+0000 7fdb010c1740 1 --2- 172.21.15.89:0/3904565645 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558331b30ea0 0x558331f2c640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:05.660 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:05.654+0000 7fdb010c1740 1 -- 172.21.15.89:0/3904565645 >> 172.21.15.89:0/3904565645 conn(0x558331b66910 msgr2=0x558331f24f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:41:05.660 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:05.654+0000 7fdb010c1740 1 -- 172.21.15.89:0/3904565645 shutdown_connections 2024-09-21T03:41:05.660 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:05.654+0000 7fdb010c1740 1 -- 172.21.15.89:0/3904565645 wait complete. 2024-09-21T03:41:05.660 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:05.655+0000 7fdb010c1740 20 client.15753 trim_cache size 0 max 16384 2024-09-21T03:41:05.661 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-09-21T03:41:05.661 INFO:teuthology.orchestra.run.smithi089.stdout: "epoch": 26, 2024-09-21T03:41:05.661 INFO:teuthology.orchestra.run.smithi089.stdout: "leader": 24638, 2024-09-21T03:41:05.661 INFO:teuthology.orchestra.run.smithi089.stdout: "set_version": 8, 2024-09-21T03:41:05.661 INFO:teuthology.orchestra.run.smithi089.stdout: "sets": { 2024-09-21T03:41:05.661 INFO:teuthology.orchestra.run.smithi089.stdout: "1beabdc5": { 2024-09-21T03:41:05.661 INFO:teuthology.orchestra.run.smithi089.stdout: "version": 8, 2024-09-21T03:41:05.661 INFO:teuthology.orchestra.run.smithi089.stdout: "age_ref": 0.0, 2024-09-21T03:41:05.661 INFO:teuthology.orchestra.run.smithi089.stdout: "state": { 2024-09-21T03:41:05.661 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "RELEASED", 2024-09-21T03:41:05.661 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 0.0 2024-09-21T03:41:05.661 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:41:05.661 INFO:teuthology.orchestra.run.smithi089.stdout: "timeout": 90.0, 2024-09-21T03:41:05.661 INFO:teuthology.orchestra.run.smithi089.stdout: "expiration": 137.0, 2024-09-21T03:41:05.662 INFO:teuthology.orchestra.run.smithi089.stdout: "members": { 2024-09-21T03:41:05.662 INFO:teuthology.orchestra.run.smithi089.stdout: "file:/": { 2024-09-21T03:41:05.662 INFO:teuthology.orchestra.run.smithi089.stdout: "excluded": false, 2024-09-21T03:41:05.662 INFO:teuthology.orchestra.run.smithi089.stdout: "state": { 2024-09-21T03:41:05.662 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "RELEASED", 2024-09-21T03:41:05.662 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 0.0 2024-09-21T03:41:05.662 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-21T03:41:05.662 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-21T03:41:05.662 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-21T03:41:05.662 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-21T03:41:05.662 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-21T03:41:05.662 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-09-21T03:41:05.663 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:05.655+0000 7fdafab4d640 1 -- 172.21.15.89:0/2018233316 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7fdadc0b6510 msgr2=0x7fdadc0b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:05.663 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:05.655+0000 7fdafab4d640 1 --2- 172.21.15.89:0/2018233316 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7fdadc0b6510 0x7fdadc0b8a20 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x558331f17190 tx=0x7fdae4008040 comp rx=0 tx=0).stop 2024-09-21T03:41:05.663 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:05.655+0000 7fdafab4d640 1 -- 172.21.15.89:0/2018233316 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdaf4108cf0 msgr2=0x7fdaf419fbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:05.663 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:05.655+0000 7fdafab4d640 1 --2- 172.21.15.89:0/2018233316 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdaf4108cf0 0x7fdaf419fbd0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fdaec005cc0 tx=0x7fdaec032c40 comp rx=0 tx=0).stop 2024-09-21T03:41:05.663 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:05.655+0000 7fdafab4d640 1 -- 172.21.15.89:0/2018233316 shutdown_connections 2024-09-21T03:41:05.663 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:05.655+0000 7fdafab4d640 1 --2- 172.21.15.89:0/2018233316 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7fdadc0b6510 0x7fdadc0b8a20 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:05.663 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:05.655+0000 7fdafab4d640 1 --2- 172.21.15.89:0/2018233316 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdaf4108cf0 0x7fdaf419fbd0 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:05.663 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:05.655+0000 7fdafab4d640 1 --2- 172.21.15.89:0/2018233316 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fdaf41083a0 0x7fdaf419a930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:05.664 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:05.655+0000 7fdafab4d640 1 --2- 172.21.15.89:0/2018233316 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdaf4102a60 0x7fdaf419a390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:05.664 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:05.655+0000 7fdafab4d640 1 -- 172.21.15.89:0/2018233316 >> 172.21.15.89:0/2018233316 conn(0x7fdaf4079720 msgr2=0x7fdaf4105200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:41:05.664 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:05.656+0000 7fdafab4d640 1 -- 172.21.15.89:0/2018233316 shutdown_connections 2024-09-21T03:41:05.664 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:05.656+0000 7fdafab4d640 1 -- 172.21.15.89:0/2018233316 wait complete. 2024-09-21T03:41:06.355 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '1beabdc5', total seconds elapsed: 22.0 2024-09-21T03:41:07.052 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:41:06 smithi110 ceph-mon[34000]: pgmap v411: 129 pgs: 129 active+clean; 192 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 319 KiB/s wr, 9 op/s 2024-09-21T03:41:07.061 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:41:06 smithi089 ceph-mon[30928]: pgmap v411: 129 pgs: 129 active+clean; 192 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 319 KiB/s wr, 9 op/s 2024-09-21T03:41:07.211 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:41:06 smithi114 ceph-mon[34140]: pgmap v411: 129 pgs: 129 active+clean; 192 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 319 KiB/s wr, 9 op/s 2024-09-21T03:41:09.061 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:41:08 smithi089 ceph-mon[30928]: pgmap v412: 129 pgs: 129 active+clean; 212 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 1.7 MiB/s wr, 8 op/s 2024-09-21T03:41:09.211 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:41:08 smithi114 ceph-mon[34140]: pgmap v412: 129 pgs: 129 active+clean; 212 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 1.7 MiB/s wr, 8 op/s 2024-09-21T03:41:09.250 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:41:08 smithi110 ceph-mon[34000]: pgmap v412: 129 pgs: 129 active+clean; 212 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 1.7 MiB/s wr, 8 op/s 2024-09-21T03:41:11.061 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:41:10 smithi089 ceph-mon[30928]: pgmap v413: 129 pgs: 129 active+clean; 254 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 5.4 MiB/s wr, 83 op/s 2024-09-21T03:41:11.211 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:41:10 smithi114 ceph-mon[34140]: pgmap v413: 129 pgs: 129 active+clean; 254 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 5.4 MiB/s wr, 83 op/s 2024-09-21T03:41:11.250 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:41:10 smithi110 ceph-mon[34000]: pgmap v413: 129 pgs: 129 active+clean; 254 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 5.4 MiB/s wr, 83 op/s 2024-09-21T03:41:11.875 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:11.873+0000 7f7b5aa32640 1 -- 172.21.15.89:0/1221553107 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7b54101b80 msgr2=0x7f7b54101ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:11.875 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:11.873+0000 7f7b5aa32640 1 --2- 172.21.15.89:0/1221553107 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7b54101b80 0x7f7b54101ff0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f7b5000a970 tx=0x7f7b5002b690 comp rx=0 tx=0).stop 2024-09-21T03:41:11.876 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:11.874+0000 7f7b5aa32640 1 -- 172.21.15.89:0/1221553107 shutdown_connections 2024-09-21T03:41:11.876 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:11.874+0000 7f7b5aa32640 1 --2- 172.21.15.89:0/1221553107 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7b541035a0 0x7f7b54109f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:11.876 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:11.874+0000 7f7b5aa32640 1 --2- 172.21.15.89:0/1221553107 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7b54102c00 0x7f7b54103010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:11.876 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:11.874+0000 7f7b5aa32640 1 --2- 172.21.15.89:0/1221553107 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7b54101b80 0x7f7b54101ff0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:11.876 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:11.874+0000 7f7b5aa32640 1 -- 172.21.15.89:0/1221553107 >> 172.21.15.89:0/1221553107 conn(0x7f7b540fd4c0 msgr2=0x7f7b540ff970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:41:11.877 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:11.875+0000 7f7b5aa32640 1 -- 172.21.15.89:0/1221553107 shutdown_connections 2024-09-21T03:41:11.877 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:11.875+0000 7f7b5aa32640 1 -- 172.21.15.89:0/1221553107 wait complete. 2024-09-21T03:41:11.877 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:11.876+0000 7f7b5aa32640 1 Processor -- start 2024-09-21T03:41:11.877 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:11.876+0000 7f7b5aa32640 1 -- start start 2024-09-21T03:41:11.878 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:11.877+0000 7f7b5aa32640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7b54101b80 0x7f7b541a0310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:11.878 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:11.877+0000 7f7b5aa32640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7b54102c00 0x7f7b541a0980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:11.878 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:11.877+0000 7f7b5aa32640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7b541035a0 0x7f7b541a1530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:11.878 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:11.877+0000 7f7b5aa32640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f7b5410c500 con 0x7f7b54102c00 2024-09-21T03:41:11.878 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:11.877+0000 7f7b5aa32640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f7b5410c380 con 0x7f7b54101b80 2024-09-21T03:41:11.878 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:11.877+0000 7f7b5aa32640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f7b5410c680 con 0x7f7b541035a0 2024-09-21T03:41:11.879 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:11.877+0000 7f7b59a30640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7b54101b80 0x7f7b541a0310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:11.879 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:11.878+0000 7f7b59a30640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7b54101b80 0x7f7b541a0310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.89:35970/0 (socket says 172.21.15.89:35970) 2024-09-21T03:41:11.879 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:11.878+0000 7f7b59a30640 1 -- 172.21.15.89:0/3513014199 learned_addr learned my addr 172.21.15.89:0/3513014199 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-21T03:41:11.879 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:11.878+0000 7f7b5a231640 1 --2- 172.21.15.89:0/3513014199 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7b541035a0 0x7f7b541a1530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:11.880 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:11.879+0000 7f7b59a30640 1 -- 172.21.15.89:0/3513014199 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7b541035a0 msgr2=0x7f7b541a1530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:11.880 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:11.879+0000 7f7b59a30640 1 --2- 172.21.15.89:0/3513014199 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7b541035a0 0x7f7b541a1530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:11.881 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:11.879+0000 7f7b59a30640 1 -- 172.21.15.89:0/3513014199 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7b54102c00 msgr2=0x7f7b541a0980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:11.881 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:11.879+0000 7f7b59a30640 1 --2- 172.21.15.89:0/3513014199 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7b54102c00 0x7f7b541a0980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:11.881 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:11.879+0000 7f7b59a30640 1 -- 172.21.15.89:0/3513014199 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b541a1d50 con 0x7f7b54101b80 2024-09-21T03:41:11.881 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:11.880+0000 7f7b59a30640 1 --2- 172.21.15.89:0/3513014199 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7b54101b80 0x7f7b541a0310 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f7b5002bbb0 tx=0x7f7b500042f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:41:11.882 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:11.880+0000 7f7b4affd640 1 -- 172.21.15.89:0/3513014199 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b50040070 con 0x7f7b54101b80 2024-09-21T03:41:11.882 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:11.881+0000 7f7b4affd640 1 -- 172.21.15.89:0/3513014199 <== mon.2 v2:172.21.15.110:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7b50030920 con 0x7f7b54101b80 2024-09-21T03:41:11.882 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:11.881+0000 7f7b5aa32640 1 -- 172.21.15.89:0/3513014199 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7b541a1fe0 con 0x7f7b54101b80 2024-09-21T03:41:11.882 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:11.881+0000 7f7b4affd640 1 -- 172.21.15.89:0/3513014199 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b5003c410 con 0x7f7b54101b80 2024-09-21T03:41:11.882 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:11.881+0000 7f7b5aa32640 1 -- 172.21.15.89:0/3513014199 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7b541a8730 con 0x7f7b54101b80 2024-09-21T03:41:11.886 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:11.885+0000 7f7b62c07740 1 Processor -- start 2024-09-21T03:41:11.891 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:11.887+0000 7f7b4affd640 1 -- 172.21.15.89:0/3513014199 <== mon.2 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 21) ==== 154512+0+0 (secure 0 0 0) 0x7f7b500035f0 con 0x7f7b54101b80 2024-09-21T03:41:11.891 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:11.888+0000 7f7b4affd640 1 --2- 172.21.15.89:0/3513014199 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f7b400b6600 0x7f7b400b8b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:11.891 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:11.889+0000 7f7b4affd640 1 -- 172.21.15.89:0/3513014199 <== mon.2 v2:172.21.15.110:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 9121+0+0 (secure 0 0 0) 0x7f7b50108d50 con 0x7f7b54101b80 2024-09-21T03:41:11.891 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:11.889+0000 7f7b5a231640 1 --2- 172.21.15.89:0/3513014199 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f7b400b6600 0x7f7b400b8b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:11.891 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:11.889+0000 7f7b62c07740 1 -- start start 2024-09-21T03:41:11.892 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:11.889+0000 7f7b62c07740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558c0ecf01b0 0x558c0f098040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:11.892 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:11.889+0000 7f7b62c07740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558c0ecc72f0 0x558c0f0986b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:11.892 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:11.889+0000 7f7b62c07740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558c0eccf4d0 0x558c0f099200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:11.892 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:11.890+0000 7f7b62c07740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x558c0f00a6f0 con 0x558c0ecf01b0 2024-09-21T03:41:11.892 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:11.890+0000 7f7b62c07740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x558c0eff9b20 con 0x558c0ecc72f0 2024-09-21T03:41:11.892 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:11.890+0000 7f7b62c07740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x558c0effd930 con 0x558c0eccf4d0 2024-09-21T03:41:11.892 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:11.890+0000 7f7b59a30640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558c0ecf01b0 0x558c0f098040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:11.892 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:11.890+0000 7f7b59a30640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558c0ecf01b0 0x558c0f098040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:34594/0 (socket says 172.21.15.89:34594) 2024-09-21T03:41:11.893 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:11.890+0000 7f7b59a30640 1 -- 172.21.15.89:0/1295732417 learned_addr learned my addr 172.21.15.89:0/1295732417 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-21T03:41:11.893 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:11.890+0000 7f7b5922f640 1 --2- 172.21.15.89:0/1295732417 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558c0ecc72f0 0x558c0f0986b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:11.894 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:11.890+0000 7f7b5a231640 1 --2- 172.21.15.89:0/1295732417 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558c0eccf4d0 0x558c0f099200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:11.894 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:11.890+0000 7f7b5a231640 1 --2- 172.21.15.89:0/3513014199 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f7b400b6600 0x7f7b400b8b10 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f7b44005fd0 tx=0x7f7b44009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:41:11.894 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:11.891+0000 7f7b59a30640 1 -- 172.21.15.89:0/1295732417 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558c0eccf4d0 msgr2=0x558c0f099200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:41:11.894 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:11.891+0000 7f7b59a30640 1 --2- 172.21.15.89:0/1295732417 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558c0eccf4d0 0x558c0f099200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:11.894 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:11.891+0000 7f7b59a30640 1 -- 172.21.15.89:0/1295732417 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558c0ecc72f0 msgr2=0x558c0f0986b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:41:11.894 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:11.891+0000 7f7b59a30640 1 --2- 172.21.15.89:0/1295732417 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558c0ecc72f0 0x558c0f0986b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:11.895 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:11.891+0000 7f7b59a30640 1 -- 172.21.15.89:0/1295732417 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558c0efbf530 con 0x558c0ecf01b0 2024-09-21T03:41:11.895 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:11.891+0000 7f7b5a231640 1 --2- 172.21.15.89:0/1295732417 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558c0eccf4d0 0x558c0f099200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:41:11.895 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:11.892+0000 7f7b59a30640 1 --2- 172.21.15.89:0/1295732417 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558c0ecf01b0 0x558c0f098040 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f7b500cbd70 tx=0x7f7b50005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=823db5a039c9b163 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:41:11.895 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:11.892+0000 7f7b3ffff640 1 -- 172.21.15.89:0/1295732417 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b500038e0 con 0x558c0ecf01b0 2024-09-21T03:41:11.895 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:11.892+0000 7f7b3ffff640 1 -- 172.21.15.89:0/1295732417 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7b50030d40 con 0x558c0ecf01b0 2024-09-21T03:41:11.895 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:11.893+0000 7f7b3ffff640 1 -- 172.21.15.89:0/1295732417 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b500044e0 con 0x558c0ecf01b0 2024-09-21T03:41:11.895 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:11.894+0000 7f7b62c07740 1 -- 172.21.15.89:0/1295732417 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558c0ecf01b0 msgr2=0x558c0f098040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:11.895 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:11.894+0000 7f7b62c07740 1 --2- 172.21.15.89:0/1295732417 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558c0ecf01b0 0x558c0f098040 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f7b500cbd70 tx=0x7f7b50005f90 comp rx=0 tx=0).stop 2024-09-21T03:41:11.898 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:11.897+0000 7f7b62c07740 1 -- 172.21.15.89:0/1295732417 shutdown_connections 2024-09-21T03:41:11.899 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:11.897+0000 7f7b62c07740 1 --2- 172.21.15.89:0/1295732417 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558c0ecc72f0 0x558c0f0986b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:11.899 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:11.897+0000 7f7b62c07740 1 --2- 172.21.15.89:0/1295732417 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558c0eccf4d0 0x558c0f099200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:11.899 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:11.897+0000 7f7b62c07740 1 --2- 172.21.15.89:0/1295732417 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558c0ecf01b0 0x558c0f098040 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:11.899 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:11.897+0000 7f7b62c07740 1 -- 172.21.15.89:0/1295732417 >> 172.21.15.89:0/1295732417 conn(0x558c0ece4300 msgr2=0x558c0f093ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:41:11.902 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:11.901+0000 7f7b62c07740 1 -- 172.21.15.89:0/1295732417 shutdown_connections 2024-09-21T03:41:11.903 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:11.901+0000 7f7b62c07740 1 -- 172.21.15.89:0/1295732417 wait complete. 2024-09-21T03:41:11.903 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:11.902+0000 7f7b62c07740 1 -- start start 2024-09-21T03:41:11.903 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:11.902+0000 7f7b62c07740 1 Processor -- start 2024-09-21T03:41:11.904 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:11.903+0000 7f7b62c07740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558c0ecc72f0 0x558c0f0b24c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:11.904 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:11.903+0000 7f7b62c07740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558c0eccf4d0 0x558c0f0b2b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:11.905 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:11.903+0000 7f7b62c07740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558c0ecf01b0 0x558c0f0b36f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:11.906 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:11.903+0000 7f7b5922f640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558c0eccf4d0 0x558c0f0b2b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:11.906 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:11.903+0000 7f7b5922f640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558c0eccf4d0 0x558c0f0b2b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.89:34816/0 (socket says 172.21.15.89:34816) 2024-09-21T03:41:11.906 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:11.903+0000 7f7b5922f640 1 -- 172.21.15.89:0/4015323979 learned_addr learned my addr 172.21.15.89:0/4015323979 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-21T03:41:11.906 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:11.904+0000 7f7b62c07740 1 -- 172.21.15.89:0/4015323979 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x558c0ef31ab0 con 0x558c0ecc72f0 2024-09-21T03:41:11.906 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:11.904+0000 7f7b62c07740 1 -- 172.21.15.89:0/4015323979 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x558c0ef9f730 con 0x558c0ecf01b0 2024-09-21T03:41:11.907 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:11.904+0000 7f7b62c07740 1 -- 172.21.15.89:0/4015323979 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x558c0f00a6f0 con 0x558c0eccf4d0 2024-09-21T03:41:11.907 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:11.904+0000 7f7b62c07740 10 client.0 _ll_register_callbacks cb 0x558c0eff36b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:41:11.907 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:11.904+0000 7f7b59a30640 1 --2- 172.21.15.89:0/4015323979 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558c0ecc72f0 0x558c0f0b24c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:11.907 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:11.904+0000 7f7b5a231640 1 --2- 172.21.15.89:0/4015323979 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558c0ecf01b0 0x558c0f0b36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:11.907 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:11.904+0000 7f7b5922f640 1 -- 172.21.15.89:0/4015323979 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558c0ecf01b0 msgr2=0x558c0f0b36f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:41:11.907 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:11.904+0000 7f7b5922f640 1 --2- 172.21.15.89:0/4015323979 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558c0ecf01b0 0x558c0f0b36f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:11.907 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:11.904+0000 7f7b5922f640 1 -- 172.21.15.89:0/4015323979 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558c0ecc72f0 msgr2=0x558c0f0b24c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:41:11.907 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:11.904+0000 7f7b5922f640 1 --2- 172.21.15.89:0/4015323979 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558c0ecc72f0 0x558c0f0b24c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:11.907 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:11.904+0000 7f7b5922f640 1 -- 172.21.15.89:0/4015323979 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558c0ed5b790 con 0x558c0eccf4d0 2024-09-21T03:41:11.907 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:11.904+0000 7f7b5922f640 1 --2- 172.21.15.89:0/4015323979 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558c0eccf4d0 0x558c0f0b2b80 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f7b4c00aad0 tx=0x7f7b4c00af90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5b4fcf003c007be8 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:41:11.907 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:11.904+0000 7f7b3d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.114:3300/0 2024-09-21T03:41:11.907 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:11.905+0000 7f7b3d7fa640 1 -- 172.21.15.89:0/4015323979 <== mon.1 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b4c018020 con 0x558c0eccf4d0 2024-09-21T03:41:11.907 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:11.905+0000 7f7b3d7fa640 1 -- 172.21.15.89:0/4015323979 <== mon.1 v2:172.21.15.114:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 9121+0+0 (secure 0 0 0) 0x7f7b4c00ec30 con 0x558c0eccf4d0 2024-09-21T03:41:11.907 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:11.905+0000 7f7b62c07740 1 -- 172.21.15.89:0/4015323979 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7b50030d40 con 0x558c0eccf4d0 2024-09-21T03:41:11.908 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:11.905+0000 7f7b3d7fa640 1 -- 172.21.15.89:0/4015323979 <== mon.1 v2:172.21.15.114:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f7b4c00ee30 con 0x558c0eccf4d0 2024-09-21T03:41:11.909 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:11.906+0000 7f7b62c07740 10 client.25354 fetch_fsmap learned FSMap version 26 2024-09-21T03:41:11.909 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:11.906+0000 7f7b62c07740 1 -- 172.21.15.89:0/4015323979 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x558c0edd4620 con 0x558c0eccf4d0 2024-09-21T03:41:11.909 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:11.907+0000 7f7b3d7fa640 1 -- 172.21.15.89:0/4015323979 <== mon.1 v2:172.21.15.114:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7b4c014890 con 0x558c0eccf4d0 2024-09-21T03:41:11.909 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:11.907+0000 7f7b3d7fa640 1 -- 172.21.15.89:0/4015323979 <== mon.1 v2:172.21.15.114:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f7b4c01c710 con 0x558c0eccf4d0 2024-09-21T03:41:11.909 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:11.907+0000 7f7b62c07740 10 client.25354 fetch_fsmap finished waiting for FSMap version 26 2024-09-21T03:41:11.909 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:11.907+0000 7f7b62c07740 10 client.25354 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T03:41:11.909 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:11.907+0000 7f7b62c07740 20 client.25354 populate_metadata read hostname 'smithi089' 2024-09-21T03:41:11.909 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:11.907+0000 7f7b62c07740 1 --2- 172.21.15.89:0/4015323979 >> [v2:172.21.15.110:6838/132366709,v1:172.21.15.110:6839/132366709] conn(0x558c0ecaed80 0x558c0f0bfbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:11.909 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:11.907+0000 7f7b62c07740 4 client.25354 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:41:11.909 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:11.908+0000 7f7b62c07740 1 -- 172.21.15.89:0/4015323979 --> [v2:172.21.15.110:6838/132366709,v1:172.21.15.110:6839/132366709] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f7b50030d40 con 0x558c0ecaed80 2024-09-21T03:41:11.909 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:11.908+0000 7f7b59a30640 1 --2- 172.21.15.89:0/4015323979 >> [v2:172.21.15.110:6838/132366709,v1:172.21.15.110:6839/132366709] conn(0x558c0ecaed80 0x558c0f0bfbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:11.910 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:11.908+0000 7f7b3d7fa640 10 client.25354 ms_handle_connect on v2:172.21.15.110:6838/132366709 2024-09-21T03:41:11.910 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:11.908+0000 7f7b59a30640 1 --2- 172.21.15.89:0/4015323979 >> [v2:172.21.15.110:6838/132366709,v1:172.21.15.110:6839/132366709] conn(0x558c0ecaed80 0x558c0f0bfbe0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4623ff36b201f7cc server_cookie=35a47639aa0a9517 in_seq=0 out_seq=0 2024-09-21T03:41:11.911 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:11.909+0000 7f7b3d7fa640 1 -- 172.21.15.89:0/4015323979 <== mds.0 v2:172.21.15.110:6838/132366709 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x7f7b50030d40 con 0x558c0ecaed80 2024-09-21T03:41:11.911 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:11.909+0000 7f7b3d7fa640 10 client.25354 handle_command_reply: tid=0 2024-09-21T03:41:11.911 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:11.910+0000 7f7b62c07740 1 client.25354 shutdown 2024-09-21T03:41:11.912 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:11.911+0000 7f7b62c07740 1 -- 172.21.15.89:0/4015323979 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558c0eccf4d0 msgr2=0x558c0f0b2b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:11.912 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:11.911+0000 7f7b62c07740 1 --2- 172.21.15.89:0/4015323979 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558c0eccf4d0 0x558c0f0b2b80 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f7b4c00aad0 tx=0x7f7b4c00af90 comp rx=0 tx=0).stop 2024-09-21T03:41:11.913 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:11.911+0000 7f7b62c07740 1 -- 172.21.15.89:0/4015323979 shutdown_connections 2024-09-21T03:41:11.913 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:11.911+0000 7f7b62c07740 1 --2- 172.21.15.89:0/4015323979 >> [v2:172.21.15.110:6838/132366709,v1:172.21.15.110:6839/132366709] conn(0x558c0ecaed80 0x558c0f0bfbe0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:11.913 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:11.911+0000 7f7b3d7fa640 0 client.25354 ms_handle_reset on v2:172.21.15.110:6838/132366709 2024-09-21T03:41:11.913 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:11.911+0000 7f7b62c07740 1 --2- 172.21.15.89:0/4015323979 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558c0ecf01b0 0x558c0f0b36f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:11.913 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:11.911+0000 7f7b62c07740 1 --2- 172.21.15.89:0/4015323979 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558c0eccf4d0 0x558c0f0b2b80 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:11.913 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:11.911+0000 7f7b62c07740 1 --2- 172.21.15.89:0/4015323979 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558c0ecc72f0 0x558c0f0b24c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:11.913 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:11.911+0000 7f7b62c07740 1 -- 172.21.15.89:0/4015323979 >> 172.21.15.89:0/4015323979 conn(0x558c0ece4300 msgr2=0x558c0f092150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:41:11.913 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:11.911+0000 7f7b62c07740 1 -- 172.21.15.89:0/4015323979 shutdown_connections 2024-09-21T03:41:11.913 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:11.911+0000 7f7b62c07740 1 -- 172.21.15.89:0/4015323979 wait complete. 2024-09-21T03:41:11.913 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:11.911+0000 7f7b62c07740 20 client.25354 trim_cache size 0 max 16384 2024-09-21T03:41:11.930 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:11.928+0000 7f7b62c07740 1 Processor -- start 2024-09-21T03:41:11.930 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:11.928+0000 7f7b62c07740 1 -- start start 2024-09-21T03:41:11.930 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:11.928+0000 7f7b62c07740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558c0ecaed80 0x558c0f0b16c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:11.930 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:11.928+0000 7f7b62c07740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558c0ecc72f0 0x558c0f0b0a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:11.930 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:11.928+0000 7f7b62c07740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558c0eccf4d0 0x558c0f0b0f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:11.930 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:11.928+0000 7f7b62c07740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f7b541acbc0 con 0x558c0ecaed80 2024-09-21T03:41:11.930 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:11.928+0000 7f7b62c07740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f7b541aa490 con 0x558c0eccf4d0 2024-09-21T03:41:11.930 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:11.928+0000 7f7b62c07740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f7b54074fd0 con 0x558c0ecc72f0 2024-09-21T03:41:11.930 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:11.928+0000 7f7b59a30640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558c0ecaed80 0x558c0f0b16c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:11.931 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:11.929+0000 7f7b5922f640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558c0ecc72f0 0x558c0f0b0a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:11.931 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:11.929+0000 7f7b5a231640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558c0eccf4d0 0x558c0f0b0f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:11.931 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:11.929+0000 7f7b5922f640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558c0ecc72f0 0x558c0f0b0a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.89:34826/0 (socket says 172.21.15.89:34826) 2024-09-21T03:41:11.931 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:11.929+0000 7f7b5922f640 1 -- 172.21.15.89:0/3297800450 learned_addr learned my addr 172.21.15.89:0/3297800450 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-21T03:41:11.931 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:11.929+0000 7f7b5922f640 1 -- 172.21.15.89:0/3297800450 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558c0eccf4d0 msgr2=0x558c0f0b0f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:41:11.932 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:11.929+0000 7f7b5922f640 1 --2- 172.21.15.89:0/3297800450 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558c0eccf4d0 0x558c0f0b0f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:11.932 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:11.929+0000 7f7b5922f640 1 -- 172.21.15.89:0/3297800450 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558c0ecaed80 msgr2=0x558c0f0b16c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:41:11.932 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:11.929+0000 7f7b5922f640 1 --2- 172.21.15.89:0/3297800450 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558c0ecaed80 0x558c0f0b16c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:11.932 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:11.929+0000 7f7b5922f640 1 -- 172.21.15.89:0/3297800450 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558c0f094450 con 0x558c0ecc72f0 2024-09-21T03:41:11.932 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:11.929+0000 7f7b5922f640 1 --2- 172.21.15.89:0/3297800450 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558c0ecc72f0 0x558c0f0b0a50 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f7b4c004c00 tx=0x7f7b4c00a740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=80af3408da47b37f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:41:11.932 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:11.930+0000 7f7b3f7fe640 1 -- 172.21.15.89:0/3297800450 <== mon.1 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b4c02b020 con 0x558c0ecc72f0 2024-09-21T03:41:11.932 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:11.930+0000 7f7b3f7fe640 1 -- 172.21.15.89:0/3297800450 <== mon.1 v2:172.21.15.114:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7b4c00ec30 con 0x558c0ecc72f0 2024-09-21T03:41:11.932 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:11.930+0000 7f7b3f7fe640 1 -- 172.21.15.89:0/3297800450 <== mon.1 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b4c013b00 con 0x558c0ecc72f0 2024-09-21T03:41:11.934 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:11.931+0000 7f7b62c07740 1 -- 172.21.15.89:0/3297800450 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558c0ecc72f0 msgr2=0x558c0f0b0a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:11.934 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:11.931+0000 7f7b62c07740 1 --2- 172.21.15.89:0/3297800450 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558c0ecc72f0 0x558c0f0b0a50 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f7b4c004c00 tx=0x7f7b4c00a740 comp rx=0 tx=0).stop 2024-09-21T03:41:11.934 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:11.932+0000 7f7b62c07740 1 -- 172.21.15.89:0/3297800450 shutdown_connections 2024-09-21T03:41:11.934 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:11.932+0000 7f7b62c07740 1 --2- 172.21.15.89:0/3297800450 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558c0eccf4d0 0x558c0f0b0f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:11.934 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:11.932+0000 7f7b62c07740 1 --2- 172.21.15.89:0/3297800450 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558c0ecc72f0 0x558c0f0b0a50 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:11.934 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:11.932+0000 7f7b62c07740 1 --2- 172.21.15.89:0/3297800450 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558c0ecaed80 0x558c0f0b16c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:11.934 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:11.932+0000 7f7b62c07740 1 -- 172.21.15.89:0/3297800450 >> 172.21.15.89:0/3297800450 conn(0x558c0ece4300 msgr2=0x558c0f0b24c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:41:11.936 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:11.934+0000 7f7b62c07740 1 -- 172.21.15.89:0/3297800450 shutdown_connections 2024-09-21T03:41:11.936 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:11.935+0000 7f7b62c07740 1 -- 172.21.15.89:0/3297800450 wait complete. 2024-09-21T03:41:11.936 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:11.935+0000 7f7b62c07740 1 -- start start 2024-09-21T03:41:11.937 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:11.935+0000 7f7b62c07740 1 Processor -- start 2024-09-21T03:41:11.937 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:11.936+0000 7f7b62c07740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558c0ecaed80 0x558c0f0ba080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:11.938 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:11.936+0000 7f7b62c07740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558c0ecc72f0 0x558c0f0b8d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:11.938 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:11.936+0000 7f7b62c07740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558c0eccf4d0 0x558c0f0b9280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:11.939 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:11.937+0000 7f7b5922f640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558c0ecc72f0 0x558c0f0b8d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:11.939 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:11.937+0000 7f7b5922f640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558c0ecc72f0 0x558c0f0b8d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.89:34836/0 (socket says 172.21.15.89:34836) 2024-09-21T03:41:11.939 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:11.937+0000 7f7b5922f640 1 -- 172.21.15.89:0/2149893865 learned_addr learned my addr 172.21.15.89:0/2149893865 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-21T03:41:11.939 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:11.937+0000 7f7b62c07740 1 -- 172.21.15.89:0/2149893865 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f7b541a8950 con 0x558c0eccf4d0 2024-09-21T03:41:11.939 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:11.937+0000 7f7b62c07740 1 -- 172.21.15.89:0/2149893865 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f7b541a9dc0 con 0x558c0ecaed80 2024-09-21T03:41:11.939 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:11.937+0000 7f7b62c07740 1 -- 172.21.15.89:0/2149893865 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f7b541a9020 con 0x558c0ecc72f0 2024-09-21T03:41:11.939 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:11.937+0000 7f7b59a30640 1 --2- 172.21.15.89:0/2149893865 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558c0ecaed80 0x558c0f0ba080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:11.939 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:11.937+0000 7f7b62c07740 10 client.0 _ll_register_callbacks cb 0x7f7b4c0099f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:41:11.939 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:11.938+0000 7f7b5922f640 1 -- 172.21.15.89:0/2149893865 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558c0ecaed80 msgr2=0x558c0f0ba080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:41:11.940 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:11.938+0000 7f7b5922f640 1 --2- 172.21.15.89:0/2149893865 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558c0ecaed80 0x558c0f0ba080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:11.940 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:11.938+0000 7f7b5922f640 1 -- 172.21.15.89:0/2149893865 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558c0eccf4d0 msgr2=0x558c0f0b9280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:41:11.940 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:11.938+0000 7f7b5922f640 1 --2- 172.21.15.89:0/2149893865 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558c0eccf4d0 0x558c0f0b9280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:11.940 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:11.938+0000 7f7b5922f640 1 -- 172.21.15.89:0/2149893865 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558c0f0b97c0 con 0x558c0ecc72f0 2024-09-21T03:41:11.941 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:11.938+0000 7f7b5922f640 1 --2- 172.21.15.89:0/2149893865 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558c0ecc72f0 0x558c0f0b8d40 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f7b4c01cbf0 tx=0x7f7b4c01cf80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1fa818ebab585a58 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:41:11.941 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:11.939+0000 7f7b3d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.114:3300/0 2024-09-21T03:41:11.941 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:11.939+0000 7f7b3d7fa640 1 -- 172.21.15.89:0/2149893865 <== mon.1 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b4c035060 con 0x558c0ecc72f0 2024-09-21T03:41:11.941 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:11.939+0000 7f7b3d7fa640 1 -- 172.21.15.89:0/2149893865 <== mon.1 v2:172.21.15.114:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 9121+0+0 (secure 0 0 0) 0x7f7b4c00ea30 con 0x558c0ecc72f0 2024-09-21T03:41:11.941 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:11.939+0000 7f7b62c07740 1 -- 172.21.15.89:0/2149893865 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7b4c00ec30 con 0x558c0ecc72f0 2024-09-21T03:41:11.941 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:11.940+0000 7f7b3d7fa640 1 -- 172.21.15.89:0/2149893865 <== mon.1 v2:172.21.15.114:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f7b4c03c050 con 0x558c0ecc72f0 2024-09-21T03:41:11.943 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:11.942+0000 7f7b62c07740 10 client.25360 fetch_fsmap learned FSMap version 26 2024-09-21T03:41:11.943 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:11.942+0000 7f7b62c07740 1 -- 172.21.15.89:0/2149893865 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x558c0f0b8600 con 0x558c0ecc72f0 2024-09-21T03:41:11.944 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:11.943+0000 7f7b3d7fa640 1 -- 172.21.15.89:0/2149893865 <== mon.1 v2:172.21.15.114:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7b4c00ec30 con 0x558c0ecc72f0 2024-09-21T03:41:11.944 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:11.943+0000 7f7b3d7fa640 1 -- 172.21.15.89:0/2149893865 <== mon.1 v2:172.21.15.114:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f7b4c030960 con 0x558c0ecc72f0 2024-09-21T03:41:11.944 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:11.943+0000 7f7b62c07740 10 client.25360 fetch_fsmap finished waiting for FSMap version 26 2024-09-21T03:41:11.944 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:11.943+0000 7f7b62c07740 10 client.25360 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T03:41:11.945 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:11.943+0000 7f7b62c07740 20 client.25360 populate_metadata read hostname 'smithi089' 2024-09-21T03:41:11.945 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:11.943+0000 7f7b62c07740 1 --2- 172.21.15.89:0/2149893865 >> [v2:172.21.15.110:6838/132366709,v1:172.21.15.110:6839/132366709] conn(0x558c0ecf01b0 0x558c0f12cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:11.945 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:11.943+0000 7f7b62c07740 4 client.25360 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T03:41:11.945 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:11.943+0000 7f7b62c07740 1 -- 172.21.15.89:0/2149893865 --> [v2:172.21.15.110:6838/132366709,v1:172.21.15.110:6839/132366709] -- command(tid 0: {"prefix": "flush journal"}) -- 0x558c0f00c640 con 0x558c0ecf01b0 2024-09-21T03:41:11.945 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:11.943+0000 7f7b59a30640 1 --2- 172.21.15.89:0/2149893865 >> [v2:172.21.15.110:6838/132366709,v1:172.21.15.110:6839/132366709] conn(0x558c0ecf01b0 0x558c0f12cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:11.946 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:11.945+0000 7f7b59a30640 1 --2- 172.21.15.89:0/2149893865 >> [v2:172.21.15.110:6838/132366709,v1:172.21.15.110:6839/132366709] conn(0x558c0ecf01b0 0x558c0f12cbf0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=648e1cfd49eb8785 server_cookie=448d2e3e0f7a64d6 in_seq=0 out_seq=0 2024-09-21T03:41:11.946 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:11.945+0000 7f7b3d7fa640 10 client.25360 ms_handle_connect on v2:172.21.15.110:6838/132366709 2024-09-21T03:41:11.958 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:11.957+0000 7f7b3d7fa640 1 -- 172.21.15.89:0/2149893865 <== mds.0 v2:172.21.15.110:6838/132366709 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x558c0f00c640 con 0x558c0ecf01b0 2024-09-21T03:41:11.958 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:11.957+0000 7f7b3d7fa640 10 client.25360 handle_command_reply: tid=0 2024-09-21T03:41:11.958 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:11.957+0000 7f7b62c07740 1 client.25360 shutdown 2024-09-21T03:41:11.959 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:11.958+0000 7f7b62c07740 1 -- 172.21.15.89:0/2149893865 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558c0ecc72f0 msgr2=0x558c0f0b8d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:11.959 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:11.958+0000 7f7b62c07740 1 --2- 172.21.15.89:0/2149893865 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558c0ecc72f0 0x558c0f0b8d40 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f7b4c01cbf0 tx=0x7f7b4c01cf80 comp rx=0 tx=0).stop 2024-09-21T03:41:11.963 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:11.961+0000 7f7b62c07740 1 -- 172.21.15.89:0/2149893865 shutdown_connections 2024-09-21T03:41:11.963 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:11.961+0000 7f7b62c07740 1 --2- 172.21.15.89:0/2149893865 >> [v2:172.21.15.110:6838/132366709,v1:172.21.15.110:6839/132366709] conn(0x558c0ecf01b0 0x558c0f12cbf0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:11.963 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:11.961+0000 7f7b62c07740 1 --2- 172.21.15.89:0/2149893865 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558c0eccf4d0 0x558c0f0b9280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:11.963 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:11.961+0000 7f7b62c07740 1 --2- 172.21.15.89:0/2149893865 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558c0ecc72f0 0x558c0f0b8d40 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:11.963 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:11.961+0000 7f7b62c07740 1 --2- 172.21.15.89:0/2149893865 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558c0ecaed80 0x558c0f0ba080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:11.963 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:11.961+0000 7f7b62c07740 1 -- 172.21.15.89:0/2149893865 >> 172.21.15.89:0/2149893865 conn(0x558c0ece4300 msgr2=0x558c0f093410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:41:11.963 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:11.961+0000 7f7b3d7fa640 0 client.25360 ms_handle_reset on v2:172.21.15.110:6838/132366709 2024-09-21T03:41:11.963 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:11.961+0000 7f7b62c07740 1 -- 172.21.15.89:0/2149893865 shutdown_connections 2024-09-21T03:41:11.963 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:11.962+0000 7f7b62c07740 1 -- 172.21.15.89:0/2149893865 wait complete. 2024-09-21T03:41:11.964 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:11.962+0000 7f7b62c07740 20 client.25360 trim_cache size 0 max 16384 2024-09-21T03:41:11.965 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stdout:{ 2024-09-21T03:41:11.965 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stdout: "message": "", 2024-09-21T03:41:11.965 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stdout: "return_code": 0 2024-09-21T03:41:11.965 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stdout:} 2024-09-21T03:41:11.966 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:11.965+0000 7f7b5aa32640 1 -- 172.21.15.89:0/3513014199 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f7b400b6600 msgr2=0x7f7b400b8b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:11.966 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:11.965+0000 7f7b5aa32640 1 --2- 172.21.15.89:0/3513014199 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f7b400b6600 0x7f7b400b8b10 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f7b44005fd0 tx=0x7f7b44009260 comp rx=0 tx=0).stop 2024-09-21T03:41:11.966 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:11.965+0000 7f7b5aa32640 1 -- 172.21.15.89:0/3513014199 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7b54101b80 msgr2=0x7f7b541a0310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:11.966 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:11.965+0000 7f7b5aa32640 1 --2- 172.21.15.89:0/3513014199 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7b54101b80 0x7f7b541a0310 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f7b5002bbb0 tx=0x7f7b500042f0 comp rx=0 tx=0).stop 2024-09-21T03:41:11.967 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:11.965+0000 7f7b5aa32640 1 -- 172.21.15.89:0/3513014199 shutdown_connections 2024-09-21T03:41:11.967 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:11.965+0000 7f7b5aa32640 1 --2- 172.21.15.89:0/3513014199 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f7b400b6600 0x7f7b400b8b10 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:11.967 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:11.965+0000 7f7b5aa32640 1 --2- 172.21.15.89:0/3513014199 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7b541035a0 0x7f7b541a1530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:11.967 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:11.965+0000 7f7b5aa32640 1 --2- 172.21.15.89:0/3513014199 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7b54102c00 0x7f7b541a0980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:11.967 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:11.965+0000 7f7b5aa32640 1 --2- 172.21.15.89:0/3513014199 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7b54101b80 0x7f7b541a0310 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:11.967 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:11.965+0000 7f7b5aa32640 1 -- 172.21.15.89:0/3513014199 >> 172.21.15.89:0/3513014199 conn(0x7f7b540fd4c0 msgr2=0x7f7b5410eee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:41:11.967 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:11.965+0000 7f7b5aa32640 1 -- 172.21.15.89:0/3513014199 shutdown_connections 2024-09-21T03:41:11.967 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:11.966+0000 7f7b5aa32640 1 -- 172.21.15.89:0/3513014199 wait complete. 2024-09-21T03:41:13.061 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:41:12 smithi089 ceph-mon[30928]: pgmap v414: 129 pgs: 129 active+clean; 254 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 5.4 MiB/s wr, 81 op/s 2024-09-21T03:41:13.211 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:41:12 smithi114 ceph-mon[34140]: pgmap v414: 129 pgs: 129 active+clean; 254 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 5.4 MiB/s wr, 81 op/s 2024-09-21T03:41:13.250 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:41:12 smithi110 ceph-mon[34000]: pgmap v414: 129 pgs: 129 active+clean; 254 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 5.4 MiB/s wr, 81 op/s 2024-09-21T03:41:15.061 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:41:14 smithi089 ceph-mon[30928]: pgmap v415: 129 pgs: 129 active+clean; 258 MiB data, 599 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 10 MiB/s wr, 1.09k op/s 2024-09-21T03:41:15.210 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:41:14 smithi114 ceph-mon[34140]: pgmap v415: 129 pgs: 129 active+clean; 258 MiB data, 599 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 10 MiB/s wr, 1.09k op/s 2024-09-21T03:41:15.250 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:41:14 smithi110 ceph-mon[34000]: pgmap v415: 129 pgs: 129 active+clean; 258 MiB data, 599 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 10 MiB/s wr, 1.09k op/s 2024-09-21T03:41:16.061 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:41:15 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:41:16.061 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:41:15 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:41:16.061 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:41:15 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:41:16.211 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:41:15 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:41:16.211 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:41:15 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:41:16.211 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:41:15 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:41:16.250 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:41:15 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:41:16.250 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:41:15 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:41:16.250 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:41:15 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:41:17.061 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:41:16 smithi089 ceph-mon[30928]: pgmap v416: 129 pgs: 129 active+clean; 258 MiB data, 613 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 11 MiB/s wr, 1.35k op/s 2024-09-21T03:41:17.061 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:41:16 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:41:17.061 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:41:16 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:41:17.061 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:41:16 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:41:17.211 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:41:16 smithi114 ceph-mon[34140]: pgmap v416: 129 pgs: 129 active+clean; 258 MiB data, 613 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 11 MiB/s wr, 1.35k op/s 2024-09-21T03:41:17.211 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:41:16 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:41:17.211 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:41:16 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:41:17.211 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:41:16 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:41:17.250 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:41:16 smithi110 ceph-mon[34000]: pgmap v416: 129 pgs: 129 active+clean; 258 MiB data, 613 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 11 MiB/s wr, 1.35k op/s 2024-09-21T03:41:17.250 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:41:16 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:41:17.250 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:41:16 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:41:17.250 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:41:16 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:41:19.061 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:41:18 smithi089 ceph-mon[30928]: pgmap v417: 129 pgs: 129 active+clean; 258 MiB data, 706 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 14 MiB/s wr, 1.95k op/s 2024-09-21T03:41:19.210 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:41:18 smithi114 ceph-mon[34140]: pgmap v417: 129 pgs: 129 active+clean; 258 MiB data, 706 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 14 MiB/s wr, 1.95k op/s 2024-09-21T03:41:19.250 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:41:18 smithi110 ceph-mon[34000]: pgmap v417: 129 pgs: 129 active+clean; 258 MiB data, 706 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 14 MiB/s wr, 1.95k op/s 2024-09-21T03:41:21.061 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:41:20 smithi089 ceph-mon[30928]: pgmap v418: 129 pgs: 129 active+clean; 258 MiB data, 944 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 16 MiB/s wr, 2.70k op/s 2024-09-21T03:41:21.210 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:41:20 smithi114 ceph-mon[34140]: pgmap v418: 129 pgs: 129 active+clean; 258 MiB data, 944 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 16 MiB/s wr, 2.70k op/s 2024-09-21T03:41:21.250 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:41:20 smithi110 ceph-mon[34000]: pgmap v418: 129 pgs: 129 active+clean; 258 MiB data, 944 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 16 MiB/s wr, 2.70k op/s 2024-09-21T03:41:23.061 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:41:22 smithi089 ceph-mon[30928]: pgmap v419: 129 pgs: 129 active+clean; 258 MiB data, 944 MiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 12 MiB/s wr, 2.63k op/s 2024-09-21T03:41:23.210 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:41:22 smithi114 ceph-mon[34140]: pgmap v419: 129 pgs: 129 active+clean; 258 MiB data, 944 MiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 12 MiB/s wr, 2.63k op/s 2024-09-21T03:41:23.249 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:41:22 smithi110 ceph-mon[34000]: pgmap v419: 129 pgs: 129 active+clean; 258 MiB data, 944 MiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 12 MiB/s wr, 2.63k op/s 2024-09-21T03:41:25.061 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:41:24 smithi089 ceph-mon[30928]: pgmap v420: 129 pgs: 129 active+clean; 258 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 17 MiB/s wr, 3.70k op/s 2024-09-21T03:41:25.151 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:25.148+0000 7efca759e640 1 -- 172.21.15.89:0/3014001039 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7efca8101720 msgr2=0x7efca8101b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:25.151 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:25.148+0000 7efca759e640 1 --2- 172.21.15.89:0/3014001039 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7efca8101720 0x7efca8101b90 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7efc9c00baf0 tx=0x7efc9c02acc0 comp rx=0 tx=0).stop 2024-09-21T03:41:25.151 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:25.148+0000 7efca759e640 1 -- 172.21.15.89:0/3014001039 shutdown_connections 2024-09-21T03:41:25.151 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:25.148+0000 7efca759e640 1 --2- 172.21.15.89:0/3014001039 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efca8103140 0x7efca8109ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:25.151 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:25.148+0000 7efca759e640 1 --2- 172.21.15.89:0/3014001039 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7efca81027a0 0x7efca8102bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:25.151 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:25.148+0000 7efca759e640 1 --2- 172.21.15.89:0/3014001039 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7efca8101720 0x7efca8101b90 secure :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7efc9c00baf0 tx=0x7efc9c02acc0 comp rx=0 tx=0).stop 2024-09-21T03:41:25.151 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:25.149+0000 7efca759e640 1 -- 172.21.15.89:0/3014001039 >> 172.21.15.89:0/3014001039 conn(0x7efca80fd060 msgr2=0x7efca80ff510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:41:25.153 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:25.150+0000 7efca759e640 1 -- 172.21.15.89:0/3014001039 shutdown_connections 2024-09-21T03:41:25.153 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:25.150+0000 7efca759e640 1 -- 172.21.15.89:0/3014001039 wait complete. 2024-09-21T03:41:25.154 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:25.152+0000 7efca759e640 1 Processor -- start 2024-09-21T03:41:25.154 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:25.152+0000 7efca759e640 1 -- start start 2024-09-21T03:41:25.155 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:25.152+0000 7efca759e640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7efca81027a0 0x7efca819c2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:25.155 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:25.152+0000 7efca759e640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efca8103140 0x7efca819c960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:25.155 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:25.152+0000 7efca759e640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7efca819d420 0x7efca819d920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:25.155 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:25.152+0000 7efca759e640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7efca810cf40 con 0x7efca8103140 2024-09-21T03:41:25.155 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:25.152+0000 7efca759e640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7efca810cdc0 con 0x7efca819d420 2024-09-21T03:41:25.155 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:25.152+0000 7efca759e640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7efca810d0c0 con 0x7efca81027a0 2024-09-21T03:41:25.156 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:25.153+0000 7efca6d9d640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7efca819d420 0x7efca819d920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:25.156 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:25.153+0000 7efca6d9d640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7efca819d420 0x7efca819d920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.89:41610/0 (socket says 172.21.15.89:41610) 2024-09-21T03:41:25.156 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:25.153+0000 7efca6d9d640 1 -- 172.21.15.89:0/3561707424 learned_addr learned my addr 172.21.15.89:0/3561707424 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-21T03:41:25.156 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:25.154+0000 7efca659c640 1 --2- 172.21.15.89:0/3561707424 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7efca81027a0 0x7efca819c2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:25.157 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:25.154+0000 7efca6d9d640 1 -- 172.21.15.89:0/3561707424 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7efca81027a0 msgr2=0x7efca819c2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:25.157 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:25.154+0000 7efca6d9d640 1 --2- 172.21.15.89:0/3561707424 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7efca81027a0 0x7efca819c2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:25.157 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:25.154+0000 7efca6d9d640 1 -- 172.21.15.89:0/3561707424 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efca8103140 msgr2=0x7efca819c960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:25.157 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:25.154+0000 7efca6d9d640 1 --2- 172.21.15.89:0/3561707424 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efca8103140 0x7efca819c960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:25.157 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:25.154+0000 7efca6d9d640 1 -- 172.21.15.89:0/3561707424 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efca819e140 con 0x7efca819d420 2024-09-21T03:41:25.157 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:25.154+0000 7efca6d9d640 1 --2- 172.21.15.89:0/3561707424 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7efca819d420 0x7efca819d920 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7efca000b970 tx=0x7efca000be30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:41:25.157 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:25.154+0000 7efc977fe640 1 -- 172.21.15.89:0/3561707424 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efca0019070 con 0x7efca819d420 2024-09-21T03:41:25.157 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:25.154+0000 7efc977fe640 1 -- 172.21.15.89:0/3561707424 <== mon.2 v2:172.21.15.110:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7efca0014440 con 0x7efca819d420 2024-09-21T03:41:25.158 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:25.156+0000 7efc977fe640 1 -- 172.21.15.89:0/3561707424 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efca00075b0 con 0x7efca819d420 2024-09-21T03:41:25.158 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:25.156+0000 7efca759e640 1 -- 172.21.15.89:0/3561707424 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efca81a44d0 con 0x7efca819d420 2024-09-21T03:41:25.158 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:25.156+0000 7efca759e640 1 -- 172.21.15.89:0/3561707424 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efca81a49a0 con 0x7efca819d420 2024-09-21T03:41:25.166 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:25.161+0000 7efcb37c0740 1 Processor -- start 2024-09-21T03:41:25.166 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:25.163+0000 7efc977fe640 1 -- 172.21.15.89:0/3561707424 <== mon.2 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 21) ==== 154512+0+0 (secure 0 0 0) 0x7efca000cb40 con 0x7efca819d420 2024-09-21T03:41:25.166 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:25.164+0000 7efc977fe640 1 --2- 172.21.15.89:0/3561707424 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7efc900b64b0 0x7efc900b89c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:25.167 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:25.165+0000 7efca5d9b640 1 --2- 172.21.15.89:0/3561707424 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7efc900b64b0 0x7efc900b89c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:25.167 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:25.165+0000 7efc977fe640 1 -- 172.21.15.89:0/3561707424 <== mon.2 v2:172.21.15.110:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 9121+0+0 (secure 0 0 0) 0x7efca00e4440 con 0x7efca819d420 2024-09-21T03:41:25.168 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:25.166+0000 7efca5d9b640 1 --2- 172.21.15.89:0/3561707424 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7efc900b64b0 0x7efc900b89c0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7efc980096f0 tx=0x7efc98006c40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:41:25.169 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:25.167+0000 7efcb37c0740 1 -- start start 2024-09-21T03:41:25.170 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:25.167+0000 7efcb37c0740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5630b6f0a1b0 0x5630b72ba850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:25.170 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:25.167+0000 7efcb37c0740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5630b6ee12f0 0x5630b72baec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:25.170 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:25.167+0000 7efcb37c0740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5630b6ee94d0 0x5630b72bba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:25.170 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:25.167+0000 7efcb37c0740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x5630b72246f0 con 0x5630b6ee94d0 2024-09-21T03:41:25.170 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:25.167+0000 7efcb37c0740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x5630b7213b20 con 0x5630b6ee12f0 2024-09-21T03:41:25.170 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:25.167+0000 7efcb37c0740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x5630b7217930 con 0x5630b6f0a1b0 2024-09-21T03:41:25.171 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:25.169+0000 7efca5d9b640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5630b6ee12f0 0x5630b72baec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:25.171 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:25.169+0000 7efca5d9b640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5630b6ee12f0 0x5630b72baec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.89:41624/0 (socket says 172.21.15.89:41624) 2024-09-21T03:41:25.171 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:25.169+0000 7efca5d9b640 1 -- 172.21.15.89:0/1215474707 learned_addr learned my addr 172.21.15.89:0/1215474707 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-21T03:41:25.172 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:25.170+0000 7efca5d9b640 1 -- 172.21.15.89:0/1215474707 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5630b6f0a1b0 msgr2=0x5630b72ba850 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T03:41:25.172 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:25.170+0000 7efca5d9b640 1 --2- 172.21.15.89:0/1215474707 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5630b6f0a1b0 0x5630b72ba850 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:25.172 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:25.170+0000 7efca5d9b640 1 -- 172.21.15.89:0/1215474707 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5630b6ee94d0 msgr2=0x5630b72bba10 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T03:41:25.172 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:25.170+0000 7efca5d9b640 1 --2- 172.21.15.89:0/1215474707 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5630b6ee94d0 0x5630b72bba10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:25.172 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:25.170+0000 7efca5d9b640 1 -- 172.21.15.89:0/1215474707 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5630b71d9530 con 0x5630b6ee12f0 2024-09-21T03:41:25.173 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:25.170+0000 7efca5d9b640 1 --2- 172.21.15.89:0/1215474707 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5630b6ee12f0 0x5630b72baec0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7efc9800b880 tx=0x7efc9800e000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a5bb09669de2e251 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:41:25.173 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:25.171+0000 7efc94ff9640 1 -- 172.21.15.89:0/1215474707 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc98023070 con 0x5630b6ee12f0 2024-09-21T03:41:25.174 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:25.172+0000 7efc94ff9640 1 -- 172.21.15.89:0/1215474707 <== mon.2 v2:172.21.15.110:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7efc9800be00 con 0x5630b6ee12f0 2024-09-21T03:41:25.174 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:25.172+0000 7efc94ff9640 1 -- 172.21.15.89:0/1215474707 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc9801ec40 con 0x5630b6ee12f0 2024-09-21T03:41:25.176 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:25.174+0000 7efcb37c0740 1 -- 172.21.15.89:0/1215474707 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5630b6ee12f0 msgr2=0x5630b72baec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:25.177 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:25.174+0000 7efcb37c0740 1 --2- 172.21.15.89:0/1215474707 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5630b6ee12f0 0x5630b72baec0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7efc9800b880 tx=0x7efc9800e000 comp rx=0 tx=0).stop 2024-09-21T03:41:25.179 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:25.177+0000 7efcb37c0740 1 -- 172.21.15.89:0/1215474707 shutdown_connections 2024-09-21T03:41:25.179 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:25.177+0000 7efcb37c0740 1 --2- 172.21.15.89:0/1215474707 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5630b6ee12f0 0x5630b72baec0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:25.179 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:25.177+0000 7efcb37c0740 1 --2- 172.21.15.89:0/1215474707 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5630b6ee94d0 0x5630b72bba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:25.179 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:25.177+0000 7efcb37c0740 1 --2- 172.21.15.89:0/1215474707 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5630b6f0a1b0 0x5630b72ba850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:25.180 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:25.177+0000 7efcb37c0740 1 -- 172.21.15.89:0/1215474707 >> 172.21.15.89:0/1215474707 conn(0x5630b6efe300 msgr2=0x5630b72adad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:41:25.189 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:25.187+0000 7efcb37c0740 1 -- 172.21.15.89:0/1215474707 shutdown_connections 2024-09-21T03:41:25.189 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:25.187+0000 7efcb37c0740 1 -- 172.21.15.89:0/1215474707 wait complete. 2024-09-21T03:41:25.190 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:25.188+0000 7efcb37c0740 1 -- start start 2024-09-21T03:41:25.190 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:25.188+0000 7efcb37c0740 1 Processor -- start 2024-09-21T03:41:25.195 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:25.189+0000 7efcb37c0740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5630b6ee12f0 0x5630b72b3250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:25.195 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:25.190+0000 7efca659c640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5630b6ee12f0 0x5630b72b3250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:25.195 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:25.190+0000 7efca659c640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5630b6ee12f0 0x5630b72b3250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.89:36662/0 (socket says 172.21.15.89:36662) 2024-09-21T03:41:25.195 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:25.189+0000 7efcb37c0740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5630b6ee94d0 0x5630b72b3910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:25.195 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:25.190+0000 7efcb37c0740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5630b6f0a1b0 0x5630b72b4480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:25.195 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:25.190+0000 7efcb37c0740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x5630b714bab0 con 0x5630b6f0a1b0 2024-09-21T03:41:25.195 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:25.190+0000 7efcb37c0740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x5630b71b9730 con 0x5630b6ee94d0 2024-09-21T03:41:25.195 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:25.190+0000 7efcb37c0740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x5630b72bc530 con 0x5630b6ee12f0 2024-09-21T03:41:25.195 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:25.190+0000 7efca659c640 1 -- 172.21.15.89:0/2864774334 learned_addr learned my addr 172.21.15.89:0/2864774334 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-21T03:41:25.195 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:25.190+0000 7efcb37c0740 10 client.0 _ll_register_callbacks cb 0x5630b720d6b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:41:25.196 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:25.190+0000 7efca5d9b640 1 --2- 172.21.15.89:0/2864774334 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5630b6ee94d0 0x5630b72b3910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:25.196 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:25.190+0000 7efca6d9d640 1 --2- 172.21.15.89:0/2864774334 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5630b6f0a1b0 0x5630b72b4480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:25.196 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:25.191+0000 7efca659c640 1 -- 172.21.15.89:0/2864774334 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5630b6ee94d0 msgr2=0x5630b72b3910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:41:25.196 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:25.191+0000 7efca659c640 1 --2- 172.21.15.89:0/2864774334 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5630b6ee94d0 0x5630b72b3910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:25.196 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:25.191+0000 7efca659c640 1 -- 172.21.15.89:0/2864774334 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5630b6f0a1b0 msgr2=0x5630b72b4480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:41:25.196 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:25.191+0000 7efca659c640 1 --2- 172.21.15.89:0/2864774334 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5630b6f0a1b0 0x5630b72b4480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:25.196 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:25.191+0000 7efca659c640 1 -- 172.21.15.89:0/2864774334 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5630b6f75790 con 0x5630b6ee12f0 2024-09-21T03:41:25.196 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:25.191+0000 7efca659c640 1 --2- 172.21.15.89:0/2864774334 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5630b6ee12f0 0x5630b72b3250 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7efc9c009230 tx=0x7efc9c02b940 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c231476659cb2d2a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:41:25.196 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:25.191+0000 7efc8dffb640 10 client.0 ms_handle_connect on v2:172.21.15.114:3300/0 2024-09-21T03:41:25.196 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:25.192+0000 7efc8dffb640 1 -- 172.21.15.89:0/2864774334 <== mon.1 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc9c041020 con 0x5630b6ee12f0 2024-09-21T03:41:25.196 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:25.192+0000 7efcb37c0740 1 -- 172.21.15.89:0/2864774334 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efc9800be00 con 0x5630b6ee12f0 2024-09-21T03:41:25.196 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:25.192+0000 7efc8dffb640 1 -- 172.21.15.89:0/2864774334 <== mon.1 v2:172.21.15.114:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 9121+0+0 (secure 0 0 0) 0x7efc9c005040 con 0x5630b6ee12f0 2024-09-21T03:41:25.196 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:25.192+0000 7efc8dffb640 1 -- 172.21.15.89:0/2864774334 <== mon.1 v2:172.21.15.114:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7efc9c04c050 con 0x5630b6ee12f0 2024-09-21T03:41:25.197 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:25.194+0000 7efcb37c0740 10 client.25363 fetch_fsmap learned FSMap version 26 2024-09-21T03:41:25.197 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:25.194+0000 7efcb37c0740 1 -- 172.21.15.89:0/2864774334 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5630b6fee620 con 0x5630b6ee12f0 2024-09-21T03:41:25.200 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:25.198+0000 7efc8dffb640 1 -- 172.21.15.89:0/2864774334 <== mon.1 v2:172.21.15.114:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7efc9800be00 con 0x5630b6ee12f0 2024-09-21T03:41:25.202 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:25.200+0000 7efc8dffb640 1 -- 172.21.15.89:0/2864774334 <== mon.1 v2:172.21.15.114:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7efc9c0046c0 con 0x5630b6ee12f0 2024-09-21T03:41:25.202 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:25.200+0000 7efcb37c0740 10 client.25363 fetch_fsmap finished waiting for FSMap version 26 2024-09-21T03:41:25.202 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:25.200+0000 7efcb37c0740 10 client.25363 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T03:41:25.202 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:25.200+0000 7efcb37c0740 20 client.25363 populate_metadata read hostname 'smithi089' 2024-09-21T03:41:25.203 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:25.200+0000 7efcb37c0740 1 --2- 172.21.15.89:0/2864774334 >> [v2:172.21.15.110:6838/132366709,v1:172.21.15.110:6839/132366709] conn(0x5630b6ec8d80 0x5630b72da2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:25.203 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:25.200+0000 7efcb37c0740 4 client.25363 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:41:25.203 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:25.200+0000 7efcb37c0740 1 -- 172.21.15.89:0/2864774334 --> [v2:172.21.15.110:6838/132366709,v1:172.21.15.110:6839/132366709] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5630b7226640 con 0x5630b6ec8d80 2024-09-21T03:41:25.204 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:25.201+0000 7efca5d9b640 1 --2- 172.21.15.89:0/2864774334 >> [v2:172.21.15.110:6838/132366709,v1:172.21.15.110:6839/132366709] conn(0x5630b6ec8d80 0x5630b72da2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:25.206 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:25.204+0000 7efca5d9b640 1 --2- 172.21.15.89:0/2864774334 >> [v2:172.21.15.110:6838/132366709,v1:172.21.15.110:6839/132366709] conn(0x5630b6ec8d80 0x5630b72da2b0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e3a218a5bf7e1c8 server_cookie=fa86dd5dee46ea49 in_seq=0 out_seq=0 2024-09-21T03:41:25.206 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:25.204+0000 7efc8dffb640 10 client.25363 ms_handle_connect on v2:172.21.15.110:6838/132366709 2024-09-21T03:41:25.208 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:25.205+0000 7efc8dffb640 1 -- 172.21.15.89:0/2864774334 <== mds.0 v2:172.21.15.110:6838/132366709 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5630b7226640 con 0x5630b6ec8d80 2024-09-21T03:41:25.208 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:25.206+0000 7efc8dffb640 10 client.25363 handle_command_reply: tid=0 2024-09-21T03:41:25.208 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:25.206+0000 7efcb37c0740 1 client.25363 shutdown 2024-09-21T03:41:25.210 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:25.208+0000 7efcb37c0740 1 -- 172.21.15.89:0/2864774334 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5630b6ee12f0 msgr2=0x5630b72b3250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:25.210 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:25.208+0000 7efcb37c0740 1 --2- 172.21.15.89:0/2864774334 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5630b6ee12f0 0x5630b72b3250 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7efc9c009230 tx=0x7efc9c02b940 comp rx=0 tx=0).stop 2024-09-21T03:41:25.210 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:25.208+0000 7efcb37c0740 1 -- 172.21.15.89:0/2864774334 shutdown_connections 2024-09-21T03:41:25.210 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:25.208+0000 7efcb37c0740 1 --2- 172.21.15.89:0/2864774334 >> [v2:172.21.15.110:6838/132366709,v1:172.21.15.110:6839/132366709] conn(0x5630b6ec8d80 0x5630b72da2b0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:25.210 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:25.208+0000 7efcb37c0740 1 --2- 172.21.15.89:0/2864774334 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5630b6ee94d0 0x5630b72b3910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:25.210 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:25.208+0000 7efcb37c0740 1 --2- 172.21.15.89:0/2864774334 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5630b6f0a1b0 0x5630b72b4480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:25.211 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:25.208+0000 7efcb37c0740 1 --2- 172.21.15.89:0/2864774334 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5630b6ee12f0 0x5630b72b3250 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:25.211 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:25.208+0000 7efcb37c0740 1 -- 172.21.15.89:0/2864774334 >> 172.21.15.89:0/2864774334 conn(0x5630b6efe300 msgr2=0x5630b72ac150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:41:25.211 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:41:24 smithi114 ceph-mon[34140]: pgmap v420: 129 pgs: 129 active+clean; 258 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 17 MiB/s wr, 3.70k op/s 2024-09-21T03:41:25.211 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:25.208+0000 7efc8dffb640 0 client.25363 ms_handle_reset on v2:172.21.15.110:6838/132366709 2024-09-21T03:41:25.212 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:25.210+0000 7efcb37c0740 1 -- 172.21.15.89:0/2864774334 shutdown_connections 2024-09-21T03:41:25.212 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:25.210+0000 7efcb37c0740 1 -- 172.21.15.89:0/2864774334 wait complete. 2024-09-21T03:41:25.213 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:25.211+0000 7efcb37c0740 20 client.25363 trim_cache size 0 max 16384 2024-09-21T03:41:25.242 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:25.240+0000 7efcb37c0740 1 Processor -- start 2024-09-21T03:41:25.243 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:25.241+0000 7efcb37c0740 1 -- start start 2024-09-21T03:41:25.243 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:25.241+0000 7efcb37c0740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5630b6ee12f0 0x5630b72b1c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:25.243 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:25.241+0000 7efcb37c0740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5630b6ee94d0 0x5630b72a8870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:25.243 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:25.241+0000 7efcb37c0740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5630b6f0a1b0 0x5630b72ad410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:25.243 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:25.241+0000 7efcb37c0740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7efca81ad070 con 0x5630b6ee12f0 2024-09-21T03:41:25.243 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:25.241+0000 7efcb37c0740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7efca81aa940 con 0x5630b6f0a1b0 2024-09-21T03:41:25.243 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:25.241+0000 7efcb37c0740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7efca81a8840 con 0x5630b6ee94d0 2024-09-21T03:41:25.244 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:25.241+0000 7efca6d9d640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5630b6f0a1b0 0x5630b72ad410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:25.244 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:25.241+0000 7efca6d9d640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5630b6f0a1b0 0x5630b72ad410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.89:41638/0 (socket says 172.21.15.89:41638) 2024-09-21T03:41:25.244 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:25.241+0000 7efca6d9d640 1 -- 172.21.15.89:0/695896155 learned_addr learned my addr 172.21.15.89:0/695896155 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-21T03:41:25.244 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:25.242+0000 7efca659c640 1 --2- 172.21.15.89:0/695896155 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5630b6ee12f0 0x5630b72b1c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:25.244 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:25.242+0000 7efca6d9d640 1 -- 172.21.15.89:0/695896155 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5630b6ee94d0 msgr2=0x5630b72a8870 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T03:41:25.244 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:25.242+0000 7efca6d9d640 1 --2- 172.21.15.89:0/695896155 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5630b6ee94d0 0x5630b72a8870 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:25.245 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:25.242+0000 7efca6d9d640 1 -- 172.21.15.89:0/695896155 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5630b6ee12f0 msgr2=0x5630b72b1c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:41:25.245 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:25.242+0000 7efca6d9d640 1 --2- 172.21.15.89:0/695896155 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5630b6ee12f0 0x5630b72b1c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:25.245 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:25.242+0000 7efca6d9d640 1 -- 172.21.15.89:0/695896155 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5630b72ba150 con 0x5630b6f0a1b0 2024-09-21T03:41:25.245 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:25.242+0000 7efca6d9d640 1 --2- 172.21.15.89:0/695896155 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5630b6f0a1b0 0x5630b72ad410 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7efca00a9860 tx=0x7efca00b47b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=786e23fb2c41dcfa server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:41:25.245 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:25.243+0000 7efc8ffff640 1 -- 172.21.15.89:0/695896155 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efca00a91b0 con 0x5630b6f0a1b0 2024-09-21T03:41:25.245 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:25.243+0000 7efc8ffff640 1 -- 172.21.15.89:0/695896155 <== mon.2 v2:172.21.15.110:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7efca0020e50 con 0x5630b6f0a1b0 2024-09-21T03:41:25.245 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:25.243+0000 7efc8ffff640 1 -- 172.21.15.89:0/695896155 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efca00a9350 con 0x5630b6f0a1b0 2024-09-21T03:41:25.247 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:25.245+0000 7efcb37c0740 1 -- 172.21.15.89:0/695896155 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5630b6f0a1b0 msgr2=0x5630b72ad410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:25.247 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:25.245+0000 7efcb37c0740 1 --2- 172.21.15.89:0/695896155 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5630b6f0a1b0 0x5630b72ad410 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7efca00a9860 tx=0x7efca00b47b0 comp rx=0 tx=0).stop 2024-09-21T03:41:25.247 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:25.245+0000 7efcb37c0740 1 -- 172.21.15.89:0/695896155 shutdown_connections 2024-09-21T03:41:25.248 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:25.245+0000 7efcb37c0740 1 --2- 172.21.15.89:0/695896155 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5630b6f0a1b0 0x5630b72ad410 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:25.248 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:25.245+0000 7efcb37c0740 1 --2- 172.21.15.89:0/695896155 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5630b6ee94d0 0x5630b72a8870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:25.248 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:25.245+0000 7efcb37c0740 1 --2- 172.21.15.89:0/695896155 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5630b6ee12f0 0x5630b72b1c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:25.248 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:25.245+0000 7efcb37c0740 1 -- 172.21.15.89:0/695896155 >> 172.21.15.89:0/695896155 conn(0x5630b6efe300 msgr2=0x5630b72da240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:41:25.248 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:25.245+0000 7efcb37c0740 1 -- 172.21.15.89:0/695896155 shutdown_connections 2024-09-21T03:41:25.248 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:25.246+0000 7efcb37c0740 1 -- 172.21.15.89:0/695896155 wait complete. 2024-09-21T03:41:25.248 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:25.246+0000 7efcb37c0740 1 -- start start 2024-09-21T03:41:25.249 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:25.247+0000 7efcb37c0740 1 Processor -- start 2024-09-21T03:41:25.249 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:25.247+0000 7efcb37c0740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5630b6ee12f0 0x5630b72e9220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:25.250 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:41:24 smithi110 ceph-mon[34000]: pgmap v420: 129 pgs: 129 active+clean; 258 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 17 MiB/s wr, 3.70k op/s 2024-09-21T03:41:25.250 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:25.248+0000 7efcb37c0740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5630b6ee94d0 0x5630b72e98e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:25.251 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:25.248+0000 7efcb37c0740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5630b6f0a1b0 0x5630b72ea450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:25.251 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:25.249+0000 7efcb37c0740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7efca81a8e00 con 0x5630b6f0a1b0 2024-09-21T03:41:25.251 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:25.249+0000 7efcb37c0740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7efca81aa270 con 0x5630b6ee94d0 2024-09-21T03:41:25.251 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:25.249+0000 7efcb37c0740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7efca81a94d0 con 0x5630b6ee12f0 2024-09-21T03:41:25.251 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:25.249+0000 7efcb37c0740 10 client.0 _ll_register_callbacks cb 0x7efc9c0044b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:41:25.252 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:25.249+0000 7efca6d9d640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5630b6f0a1b0 0x5630b72ea450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:25.252 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:25.249+0000 7efca6d9d640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5630b6f0a1b0 0x5630b72ea450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:49692/0 (socket says 172.21.15.89:49692) 2024-09-21T03:41:25.252 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:25.249+0000 7efca6d9d640 1 -- 172.21.15.89:0/4208136455 learned_addr learned my addr 172.21.15.89:0/4208136455 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-21T03:41:25.253 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:25.251+0000 7efca6d9d640 1 -- 172.21.15.89:0/4208136455 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5630b6ee12f0 msgr2=0x5630b72e9220 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T03:41:25.253 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:25.251+0000 7efca6d9d640 1 --2- 172.21.15.89:0/4208136455 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5630b6ee12f0 0x5630b72e9220 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:25.253 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:25.251+0000 7efca6d9d640 1 -- 172.21.15.89:0/4208136455 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5630b6ee94d0 msgr2=0x5630b72e98e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:41:25.253 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:25.251+0000 7efca5d9b640 1 --2- 172.21.15.89:0/4208136455 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5630b6ee94d0 0x5630b72e98e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:25.253 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:25.251+0000 7efca6d9d640 1 --2- 172.21.15.89:0/4208136455 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5630b6ee94d0 0x5630b72e98e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:25.253 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:25.251+0000 7efca6d9d640 1 -- 172.21.15.89:0/4208136455 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5630b72eabb0 con 0x5630b6f0a1b0 2024-09-21T03:41:25.255 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:25.251+0000 7efca6d9d640 1 --2- 172.21.15.89:0/4208136455 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5630b6f0a1b0 0x5630b72ea450 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7efca00b13f0 tx=0x7efca00a8160 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b48322a0c8a32be4 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:41:25.255 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:25.252+0000 7efc8dffb640 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2024-09-21T03:41:25.255 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:25.252+0000 7efc8dffb640 1 -- 172.21.15.89:0/4208136455 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efca00a9520 con 0x5630b6f0a1b0 2024-09-21T03:41:25.255 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:25.252+0000 7efc8dffb640 1 -- 172.21.15.89:0/4208136455 <== mon.0 v2:172.21.15.89:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 9121+0+0 (secure 0 0 0) 0x7efca00abc20 con 0x5630b6f0a1b0 2024-09-21T03:41:25.255 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:25.253+0000 7efcb37c0740 1 -- 172.21.15.89:0/4208136455 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efca0020e50 con 0x5630b6f0a1b0 2024-09-21T03:41:25.259 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:25.253+0000 7efc8dffb640 1 -- 172.21.15.89:0/4208136455 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7efca00a96c0 con 0x5630b6f0a1b0 2024-09-21T03:41:25.259 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:25.253+0000 7efcb37c0740 10 client.15771 fetch_fsmap learned FSMap version 26 2024-09-21T03:41:25.259 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:25.253+0000 7efcb37c0740 1 -- 172.21.15.89:0/4208136455 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5630b72ebed0 con 0x5630b6f0a1b0 2024-09-21T03:41:25.259 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:25.254+0000 7efc8dffb640 1 -- 172.21.15.89:0/4208136455 <== mon.0 v2:172.21.15.89:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7efca0020e50 con 0x5630b6f0a1b0 2024-09-21T03:41:25.259 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:25.254+0000 7efc8dffb640 1 -- 172.21.15.89:0/4208136455 <== mon.0 v2:172.21.15.89:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7efca00ce9d0 con 0x5630b6f0a1b0 2024-09-21T03:41:25.259 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:25.254+0000 7efcb37c0740 10 client.15771 fetch_fsmap finished waiting for FSMap version 26 2024-09-21T03:41:25.259 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:25.254+0000 7efcb37c0740 10 client.15771 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T03:41:25.259 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:25.254+0000 7efcb37c0740 20 client.15771 populate_metadata read hostname 'smithi089' 2024-09-21T03:41:25.259 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:25.254+0000 7efcb37c0740 1 --2- 172.21.15.89:0/4208136455 >> [v2:172.21.15.110:6838/132366709,v1:172.21.15.110:6839/132366709] conn(0x5630b6e9e1c0 0x5630b7346c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:25.259 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:25.254+0000 7efcb37c0740 4 client.15771 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T03:41:25.259 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:25.254+0000 7efcb37c0740 1 -- 172.21.15.89:0/4208136455 --> [v2:172.21.15.110:6838/132366709,v1:172.21.15.110:6839/132366709] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5630b72220f0 con 0x5630b6e9e1c0 2024-09-21T03:41:25.260 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:25.258+0000 7efca659c640 1 --2- 172.21.15.89:0/4208136455 >> [v2:172.21.15.110:6838/132366709,v1:172.21.15.110:6839/132366709] conn(0x5630b6e9e1c0 0x5630b7346c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:25.261 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:25.259+0000 7efca659c640 1 --2- 172.21.15.89:0/4208136455 >> [v2:172.21.15.110:6838/132366709,v1:172.21.15.110:6839/132366709] conn(0x5630b6e9e1c0 0x5630b7346c50 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=80f336753b2deca2 server_cookie=487da87fc18d444f in_seq=0 out_seq=0 2024-09-21T03:41:25.262 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:25.259+0000 7efc8dffb640 10 client.15771 ms_handle_connect on v2:172.21.15.110:6838/132366709 2024-09-21T03:41:25.290 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:25.288+0000 7efc8dffb640 1 -- 172.21.15.89:0/4208136455 <== mds.0 v2:172.21.15.110:6838/132366709 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5630b72220f0 con 0x5630b6e9e1c0 2024-09-21T03:41:25.290 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:25.288+0000 7efc8dffb640 10 client.15771 handle_command_reply: tid=0 2024-09-21T03:41:25.290 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:25.288+0000 7efcb37c0740 1 client.15771 shutdown 2024-09-21T03:41:25.291 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:25.289+0000 7efcb37c0740 1 -- 172.21.15.89:0/4208136455 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5630b6f0a1b0 msgr2=0x5630b72ea450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:25.291 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:25.289+0000 7efcb37c0740 1 --2- 172.21.15.89:0/4208136455 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5630b6f0a1b0 0x5630b72ea450 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7efca00b13f0 tx=0x7efca00a8160 comp rx=0 tx=0).stop 2024-09-21T03:41:25.293 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:25.290+0000 7efcb37c0740 1 -- 172.21.15.89:0/4208136455 shutdown_connections 2024-09-21T03:41:25.293 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:25.290+0000 7efcb37c0740 1 --2- 172.21.15.89:0/4208136455 >> [v2:172.21.15.110:6838/132366709,v1:172.21.15.110:6839/132366709] conn(0x5630b6e9e1c0 0x5630b7346c50 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:25.293 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:25.290+0000 7efcb37c0740 1 --2- 172.21.15.89:0/4208136455 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5630b6ee94d0 0x5630b72e98e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:25.293 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:25.290+0000 7efcb37c0740 1 --2- 172.21.15.89:0/4208136455 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5630b6f0a1b0 0x5630b72ea450 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:25.293 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:25.290+0000 7efcb37c0740 1 --2- 172.21.15.89:0/4208136455 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5630b6ee12f0 0x5630b72e9220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:25.293 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:25.290+0000 7efcb37c0740 1 -- 172.21.15.89:0/4208136455 >> 172.21.15.89:0/4208136455 conn(0x5630b6efe300 msgr2=0x5630b72a8870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:41:25.293 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:25.291+0000 7efc8dffb640 0 client.15771 ms_handle_reset on v2:172.21.15.110:6838/132366709 2024-09-21T03:41:25.294 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:25.292+0000 7efcb37c0740 1 -- 172.21.15.89:0/4208136455 shutdown_connections 2024-09-21T03:41:25.294 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:25.292+0000 7efcb37c0740 1 -- 172.21.15.89:0/4208136455 wait complete. 2024-09-21T03:41:25.294 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:25.292+0000 7efcb37c0740 20 client.15771 trim_cache size 0 max 16384 2024-09-21T03:41:25.294 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stdout:{ 2024-09-21T03:41:25.295 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stdout: "message": "", 2024-09-21T03:41:25.295 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stdout: "return_code": 0 2024-09-21T03:41:25.295 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stdout:} 2024-09-21T03:41:25.295 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:25.293+0000 7efca759e640 1 -- 172.21.15.89:0/3561707424 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7efc900b64b0 msgr2=0x7efc900b89c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:25.295 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:25.293+0000 7efca759e640 1 --2- 172.21.15.89:0/3561707424 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7efc900b64b0 0x7efc900b89c0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7efc980096f0 tx=0x7efc98006c40 comp rx=0 tx=0).stop 2024-09-21T03:41:25.295 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:25.293+0000 7efca759e640 1 -- 172.21.15.89:0/3561707424 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7efca819d420 msgr2=0x7efca819d920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:25.295 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:25.293+0000 7efca759e640 1 --2- 172.21.15.89:0/3561707424 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7efca819d420 0x7efca819d920 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7efca000b970 tx=0x7efca000be30 comp rx=0 tx=0).stop 2024-09-21T03:41:25.296 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:25.294+0000 7efca759e640 1 -- 172.21.15.89:0/3561707424 shutdown_connections 2024-09-21T03:41:25.296 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:25.294+0000 7efca759e640 1 --2- 172.21.15.89:0/3561707424 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7efc900b64b0 0x7efc900b89c0 secure :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7efc980096f0 tx=0x7efc98006c40 comp rx=0 tx=0).stop 2024-09-21T03:41:25.296 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:25.294+0000 7efca759e640 1 --2- 172.21.15.89:0/3561707424 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7efca819d420 0x7efca819d920 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:25.297 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:25.294+0000 7efca759e640 1 --2- 172.21.15.89:0/3561707424 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efca8103140 0x7efca819c960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:25.297 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:25.294+0000 7efca759e640 1 --2- 172.21.15.89:0/3561707424 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7efca81027a0 0x7efca819c2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:25.297 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:25.294+0000 7efca759e640 1 -- 172.21.15.89:0/3561707424 >> 172.21.15.89:0/3561707424 conn(0x7efca80fd060 msgr2=0x7efca80fde30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:41:25.298 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:25.296+0000 7efca759e640 1 -- 172.21.15.89:0/3561707424 shutdown_connections 2024-09-21T03:41:25.299 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:25.297+0000 7efca759e640 1 -- 172.21.15.89:0/3561707424 wait complete. 2024-09-21T03:41:27.061 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:41:26 smithi089 ceph-mon[30928]: pgmap v421: 129 pgs: 129 active+clean; 258 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 13 MiB/s wr, 2.94k op/s 2024-09-21T03:41:27.210 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:41:26 smithi114 ceph-mon[34140]: pgmap v421: 129 pgs: 129 active+clean; 258 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 13 MiB/s wr, 2.94k op/s 2024-09-21T03:41:27.250 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:41:26 smithi110 ceph-mon[34000]: pgmap v421: 129 pgs: 129 active+clean; 258 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 13 MiB/s wr, 2.94k op/s 2024-09-21T03:41:29.061 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:41:28 smithi089 ceph-mon[30928]: pgmap v422: 129 pgs: 129 active+clean; 258 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 14 MiB/s wr, 3.25k op/s 2024-09-21T03:41:29.211 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:41:28 smithi114 ceph-mon[34140]: pgmap v422: 129 pgs: 129 active+clean; 258 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 14 MiB/s wr, 3.25k op/s 2024-09-21T03:41:29.250 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:41:28 smithi110 ceph-mon[34000]: pgmap v422: 129 pgs: 129 active+clean; 258 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 14 MiB/s wr, 3.25k op/s 2024-09-21T03:41:31.211 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:41:30 smithi114 ceph-mon[34140]: pgmap v423: 129 pgs: 129 active+clean; 258 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 15 MiB/s wr, 3.40k op/s 2024-09-21T03:41:31.250 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:41:30 smithi110 ceph-mon[34000]: pgmap v423: 129 pgs: 129 active+clean; 258 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 15 MiB/s wr, 3.40k op/s 2024-09-21T03:41:31.311 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:41:30 smithi089 ceph-mon[30928]: pgmap v423: 129 pgs: 129 active+clean; 258 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 15 MiB/s wr, 3.40k op/s 2024-09-21T03:41:33.165 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:41:32 smithi089 ceph-mon[30928]: pgmap v424: 129 pgs: 129 active+clean; 258 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 12 MiB/s wr, 2.65k op/s 2024-09-21T03:41:33.210 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:41:32 smithi114 ceph-mon[34140]: pgmap v424: 129 pgs: 129 active+clean; 258 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 12 MiB/s wr, 2.65k op/s 2024-09-21T03:41:33.250 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:41:32 smithi110 ceph-mon[34000]: pgmap v424: 129 pgs: 129 active+clean; 258 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 12 MiB/s wr, 2.65k op/s 2024-09-21T03:41:35.214 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:41:34 smithi114 ceph-mon[34140]: pgmap v425: 129 pgs: 129 active+clean; 258 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 17 MiB/s wr, 3.72k op/s 2024-09-21T03:41:35.250 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:41:34 smithi110 ceph-mon[34000]: pgmap v425: 129 pgs: 129 active+clean; 258 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 17 MiB/s wr, 3.72k op/s 2024-09-21T03:41:35.311 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:41:34 smithi089 ceph-mon[30928]: pgmap v425: 129 pgs: 129 active+clean; 258 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 17 MiB/s wr, 3.72k op/s 2024-09-21T03:41:37.211 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:41:36 smithi114 ceph-mon[34140]: pgmap v426: 129 pgs: 129 active+clean; 258 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 13 MiB/s wr, 2.89k op/s 2024-09-21T03:41:37.250 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:41:36 smithi110 ceph-mon[34000]: pgmap v426: 129 pgs: 129 active+clean; 258 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 13 MiB/s wr, 2.89k op/s 2024-09-21T03:41:37.311 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:41:36 smithi089 ceph-mon[30928]: pgmap v426: 129 pgs: 129 active+clean; 258 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 13 MiB/s wr, 2.89k op/s 2024-09-21T03:41:37.927 INFO:tasks.quiescer.fs.[cephfs]:Total duration (55.6) is longer than `split_if_longer` (35.0), will split into two consecutive quiesces 2024-09-21T03:41:37.928 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 27.8 2024-09-21T03:41:37.928 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 7c374098-77c9-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T03:41:38.468 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/7c374098-77c9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:41:38.501 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:38.499+0000 7fa306032640 1 -- 172.21.15.89:0/3601082928 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa30010a3c0 msgr2=0x7fa30010c7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:38.501 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:38.499+0000 7fa306032640 1 --2- 172.21.15.89:0/3601082928 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa30010a3c0 0x7fa30010c7e0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fa2fc009900 tx=0x7fa2fc02a420 comp rx=0 tx=0).stop 2024-09-21T03:41:38.502 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:38.499+0000 7fa306032640 1 -- 172.21.15.89:0/3601082928 shutdown_connections 2024-09-21T03:41:38.502 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:38.499+0000 7fa306032640 1 --2- 172.21.15.89:0/3601082928 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa30010f830 0x7fa30010fd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:38.502 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:38.499+0000 7fa306032640 1 --2- 172.21.15.89:0/3601082928 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa30010cd20 0x7fa30010f140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:38.502 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:38.499+0000 7fa306032640 1 --2- 172.21.15.89:0/3601082928 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa30010a3c0 0x7fa30010c7e0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:38.502 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:38.499+0000 7fa306032640 1 -- 172.21.15.89:0/3601082928 >> 172.21.15.89:0/3601082928 conn(0x7fa300070640 msgr2=0x7fa300072af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:41:38.506 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:38.503+0000 7fa306032640 1 -- 172.21.15.89:0/3601082928 shutdown_connections 2024-09-21T03:41:38.507 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:38.503+0000 7fa306032640 1 -- 172.21.15.89:0/3601082928 wait complete. 2024-09-21T03:41:38.507 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:38.504+0000 7fa306032640 1 Processor -- start 2024-09-21T03:41:38.507 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:38.504+0000 7fa306032640 1 -- start start 2024-09-21T03:41:38.508 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:38.505+0000 7fa306032640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa30010a3c0 0x7fa3001a8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:38.508 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:38.505+0000 7fa306032640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa30010cd20 0x7fa3001a90e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:38.508 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:38.505+0000 7fa306032640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa30010f830 0x7fa3001a9c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:38.508 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:38.505+0000 7fa306032640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fa30011de00 con 0x7fa30010a3c0 2024-09-21T03:41:38.508 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:38.505+0000 7fa306032640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fa30011dc80 con 0x7fa30010f830 2024-09-21T03:41:38.508 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:38.505+0000 7fa306032640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fa30011df80 con 0x7fa30010cd20 2024-09-21T03:41:38.508 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:38.505+0000 7fa305831640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa30010f830 0x7fa3001a9c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:38.508 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:38.505+0000 7fa305831640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa30010f830 0x7fa3001a9c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.89:43914/0 (socket says 172.21.15.89:43914) 2024-09-21T03:41:38.509 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:38.505+0000 7fa305831640 1 -- 172.21.15.89:0/3856145499 learned_addr learned my addr 172.21.15.89:0/3856145499 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-21T03:41:38.509 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:38.505+0000 7fa30482f640 1 --2- 172.21.15.89:0/3856145499 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa30010cd20 0x7fa3001a90e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:38.509 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:38.506+0000 7fa305831640 1 -- 172.21.15.89:0/3856145499 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa30010cd20 msgr2=0x7fa3001a90e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:38.509 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:38.506+0000 7fa305831640 1 --2- 172.21.15.89:0/3856145499 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa30010cd20 0x7fa3001a90e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:38.509 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:38.506+0000 7fa305831640 1 -- 172.21.15.89:0/3856145499 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa30010a3c0 msgr2=0x7fa3001a8a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:38.509 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:38.506+0000 7fa305831640 1 --2- 172.21.15.89:0/3856145499 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa30010a3c0 0x7fa3001a8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:38.509 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:38.506+0000 7fa305831640 1 -- 172.21.15.89:0/3856145499 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3001aa420 con 0x7fa30010f830 2024-09-21T03:41:38.509 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:38.507+0000 7fa305831640 1 --2- 172.21.15.89:0/3856145499 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa30010f830 0x7fa3001a9c90 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fa2f000c970 tx=0x7fa2f000ce30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:41:38.511 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:38.508+0000 7fa2f67fc640 1 -- 172.21.15.89:0/3856145499 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2f0007bb0 con 0x7fa30010f830 2024-09-21T03:41:38.511 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:38.508+0000 7fa2f67fc640 1 -- 172.21.15.89:0/3856145499 <== mon.2 v2:172.21.15.110:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa2f0007d50 con 0x7fa30010f830 2024-09-21T03:41:38.511 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:38.508+0000 7fa2f67fc640 1 -- 172.21.15.89:0/3856145499 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2f0018680 con 0x7fa30010f830 2024-09-21T03:41:38.512 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:38.509+0000 7fa306032640 1 -- 172.21.15.89:0/3856145499 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa3001aa710 con 0x7fa30010f830 2024-09-21T03:41:38.512 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:38.509+0000 7fa306032640 1 -- 172.21.15.89:0/3856145499 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3001b0cb0 con 0x7fa30010f830 2024-09-21T03:41:38.515 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:38.513+0000 7fa30e200740 1 Processor -- start 2024-09-21T03:41:38.518 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:38.516+0000 7fa30e200740 1 -- start start 2024-09-21T03:41:38.518 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:38.516+0000 7fa30e200740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c55c3c11b0 0x55c55c769040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:38.519 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:38.516+0000 7fa30e200740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c55c3982f0 0x55c55c7696b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:38.519 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:38.516+0000 7fa30e200740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c55c3a04d0 0x55c55c76a200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:38.519 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:38.516+0000 7fa2f67fc640 1 -- 172.21.15.89:0/3856145499 <== mon.2 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 21) ==== 154512+0+0 (secure 0 0 0) 0x7fa2f000fa00 con 0x7fa30010f830 2024-09-21T03:41:38.519 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:38.516+0000 7fa30e200740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55c55c6db6f0 con 0x55c55c3c11b0 2024-09-21T03:41:38.519 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:38.516+0000 7fa30e200740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55c55c6cab20 con 0x55c55c3a04d0 2024-09-21T03:41:38.519 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:38.516+0000 7fa30e200740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55c55c6ce930 con 0x55c55c3982f0 2024-09-21T03:41:38.520 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:38.517+0000 7fa30482f640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c55c3a04d0 0x55c55c76a200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:38.520 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:38.517+0000 7fa2f67fc640 1 --2- 172.21.15.89:0/3856145499 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7fa2ec0b66e0 0x7fa2ec0b8bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:38.520 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:38.518+0000 7fa30482f640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c55c3a04d0 0x55c55c76a200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.89:43930/0 (socket says 172.21.15.89:43930) 2024-09-21T03:41:38.520 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:38.518+0000 7fa30482f640 1 -- 172.21.15.89:0/1030224338 learned_addr learned my addr 172.21.15.89:0/1030224338 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-21T03:41:38.525 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:38.518+0000 7fa2f67fc640 1 -- 172.21.15.89:0/3856145499 <== mon.2 v2:172.21.15.110:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 9121+0+0 (secure 0 0 0) 0x7fa2f00e7680 con 0x7fa30010f830 2024-09-21T03:41:38.525 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:38.518+0000 7fa30482f640 1 --2- 172.21.15.89:0/1030224338 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c55c3c11b0 0x55c55c769040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:38.525 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:38.520+0000 7fa30482f640 1 -- 172.21.15.89:0/1030224338 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c55c3982f0 msgr2=0x55c55c7696b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:41:38.525 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:38.522+0000 7fa305831640 1 --2- 172.21.15.89:0/3856145499 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7fa2ec0b66e0 0x7fa2ec0b8bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:38.526 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:38.523+0000 7fa305831640 1 --2- 172.21.15.89:0/3856145499 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7fa2ec0b66e0 0x7fa2ec0b8bf0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fa2f00ad3c0 tx=0x7fa2f0005e20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:41:38.526 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:38.518+0000 7fa305030640 1 --2- 172.21.15.89:0/1030224338 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c55c3982f0 0x55c55c7696b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:38.527 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:38.524+0000 7fa30482f640 1 --2- 172.21.15.89:0/1030224338 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c55c3982f0 0x55c55c7696b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:38.527 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:38.524+0000 7fa30482f640 1 -- 172.21.15.89:0/1030224338 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c55c3c11b0 msgr2=0x55c55c769040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:41:38.527 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:38.524+0000 7fa30482f640 1 --2- 172.21.15.89:0/1030224338 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c55c3c11b0 0x55c55c769040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:38.527 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:38.524+0000 7fa30482f640 1 -- 172.21.15.89:0/1030224338 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c55c690530 con 0x55c55c3a04d0 2024-09-21T03:41:38.527 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:38.524+0000 7fa305030640 1 --2- 172.21.15.89:0/1030224338 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c55c3982f0 0x55c55c7696b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:41:38.527 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:38.525+0000 7fa30482f640 1 --2- 172.21.15.89:0/1030224338 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c55c3a04d0 0x55c55c76a200 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fa2f8011840 tx=0x7fa2f8009920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8285dd003acd825 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:41:38.532 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:38.530+0000 7fa2eb7fe640 1 -- 172.21.15.89:0/1030224338 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2f8003970 con 0x55c55c3a04d0 2024-09-21T03:41:38.532 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:38.530+0000 7fa2eb7fe640 1 -- 172.21.15.89:0/1030224338 <== mon.2 v2:172.21.15.110:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa2f800f8c0 con 0x55c55c3a04d0 2024-09-21T03:41:38.532 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:38.530+0000 7fa2eb7fe640 1 -- 172.21.15.89:0/1030224338 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2f8004550 con 0x55c55c3a04d0 2024-09-21T03:41:38.535 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:38.533+0000 7fa30e200740 1 -- 172.21.15.89:0/1030224338 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c55c3a04d0 msgr2=0x55c55c76a200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:38.535 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:38.533+0000 7fa30e200740 1 --2- 172.21.15.89:0/1030224338 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c55c3a04d0 0x55c55c76a200 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fa2f8011840 tx=0x7fa2f8009920 comp rx=0 tx=0).stop 2024-09-21T03:41:38.536 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:38.533+0000 7fa30e200740 1 -- 172.21.15.89:0/1030224338 shutdown_connections 2024-09-21T03:41:38.536 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:38.533+0000 7fa30e200740 1 --2- 172.21.15.89:0/1030224338 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c55c3a04d0 0x55c55c76a200 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:38.536 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:38.533+0000 7fa30e200740 1 --2- 172.21.15.89:0/1030224338 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c55c3982f0 0x55c55c7696b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:38.536 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:38.533+0000 7fa30e200740 1 --2- 172.21.15.89:0/1030224338 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c55c3c11b0 0x55c55c769040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:38.536 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:38.533+0000 7fa30e200740 1 -- 172.21.15.89:0/1030224338 >> 172.21.15.89:0/1030224338 conn(0x55c55c3b5300 msgr2=0x55c55c764ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:41:38.536 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:38.534+0000 7fa30e200740 1 -- 172.21.15.89:0/1030224338 shutdown_connections 2024-09-21T03:41:38.536 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:38.534+0000 7fa30e200740 1 -- 172.21.15.89:0/1030224338 wait complete. 2024-09-21T03:41:38.536 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:38.534+0000 7fa30e200740 1 -- start start 2024-09-21T03:41:38.537 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:38.534+0000 7fa30e200740 1 Processor -- start 2024-09-21T03:41:38.537 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:38.535+0000 7fa30e200740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c55c3982f0 0x55c55c7728c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:38.537 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:38.535+0000 7fa30e200740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c55c3a04d0 0x55c55c772f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:38.537 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:38.535+0000 7fa30e200740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c55c3c11b0 0x55c55c773af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:38.537 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:38.535+0000 7fa30e200740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55c55c602ab0 con 0x55c55c3a04d0 2024-09-21T03:41:38.537 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:38.535+0000 7fa30e200740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55c55c670730 con 0x55c55c3982f0 2024-09-21T03:41:38.538 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:38.535+0000 7fa30e200740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55c55c76ad20 con 0x55c55c3c11b0 2024-09-21T03:41:38.538 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:38.535+0000 7fa30e200740 10 client.0 _ll_register_callbacks cb 0x55c55c6c46b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:41:38.538 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:38.536+0000 7fa305030640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c55c3a04d0 0x55c55c772f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:38.538 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:38.536+0000 7fa305030640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c55c3a04d0 0x55c55c772f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:33674/0 (socket says 172.21.15.89:33674) 2024-09-21T03:41:38.538 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:38.536+0000 7fa305030640 1 -- 172.21.15.89:0/1102893882 learned_addr learned my addr 172.21.15.89:0/1102893882 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-21T03:41:38.539 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:38.537+0000 7fa305030640 1 -- 172.21.15.89:0/1102893882 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c55c3c11b0 msgr2=0x55c55c773af0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T03:41:38.539 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:38.537+0000 7fa305030640 1 --2- 172.21.15.89:0/1102893882 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c55c3c11b0 0x55c55c773af0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:38.539 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:38.537+0000 7fa305030640 1 -- 172.21.15.89:0/1102893882 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c55c3982f0 msgr2=0x55c55c7728c0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T03:41:38.539 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:38.537+0000 7fa305030640 1 --2- 172.21.15.89:0/1102893882 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c55c3982f0 0x55c55c7728c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:38.539 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:38.537+0000 7fa305030640 1 -- 172.21.15.89:0/1102893882 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c55c42c790 con 0x55c55c3a04d0 2024-09-21T03:41:38.541 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:38.538+0000 7fa305030640 1 --2- 172.21.15.89:0/1102893882 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c55c3a04d0 0x55c55c772f80 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7fa2fc02a940 tx=0x7fa2fc004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7401e8b782a4eac8 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:41:38.542 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:38.540+0000 7fa2e8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2024-09-21T03:41:38.543 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:38.540+0000 7fa2e8ff9640 1 -- 172.21.15.89:0/1102893882 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2fc041070 con 0x55c55c3a04d0 2024-09-21T03:41:38.543 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:38.540+0000 7fa2e8ff9640 1 -- 172.21.15.89:0/1102893882 <== mon.0 v2:172.21.15.89:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 9121+0+0 (secure 0 0 0) 0x7fa2fc004440 con 0x55c55c3a04d0 2024-09-21T03:41:38.543 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:38.540+0000 7fa30e200740 1 -- 172.21.15.89:0/1102893882 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa2f800f8c0 con 0x55c55c3a04d0 2024-09-21T03:41:38.544 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:38.542+0000 7fa2e8ff9640 1 -- 172.21.15.89:0/1102893882 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fa2fc04b050 con 0x55c55c3a04d0 2024-09-21T03:41:38.544 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:38.542+0000 7fa30e200740 10 client.15780 fetch_fsmap learned FSMap version 26 2024-09-21T03:41:38.544 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:38.542+0000 7fa30e200740 1 -- 172.21.15.89:0/1102893882 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55c55c4a5620 con 0x55c55c3a04d0 2024-09-21T03:41:38.546 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:38.543+0000 7fa2e8ff9640 1 -- 172.21.15.89:0/1102893882 <== mon.0 v2:172.21.15.89:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa2f800f8c0 con 0x55c55c3a04d0 2024-09-21T03:41:38.546 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:38.543+0000 7fa2e8ff9640 1 -- 172.21.15.89:0/1102893882 <== mon.0 v2:172.21.15.89:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fa2fc03a050 con 0x55c55c3a04d0 2024-09-21T03:41:38.546 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:38.543+0000 7fa30e200740 10 client.15780 fetch_fsmap finished waiting for FSMap version 26 2024-09-21T03:41:38.546 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:38.543+0000 7fa30e200740 10 client.15780 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T03:41:38.546 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:38.543+0000 7fa30e200740 20 client.15780 populate_metadata read hostname 'smithi089' 2024-09-21T03:41:38.546 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:38.543+0000 7fa30e200740 1 --2- 172.21.15.89:0/1102893882 >> [v2:172.21.15.110:6838/132366709,v1:172.21.15.110:6839/132366709] conn(0x55c55c37fd80 0x55c55c780430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:38.546 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:38.543+0000 7fa30e200740 4 client.15780 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:41:38.546 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:38.543+0000 7fa30e200740 1 -- 172.21.15.89:0/1102893882 --> [v2:172.21.15.110:6838/132366709,v1:172.21.15.110:6839/132366709] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c55c6dd640 con 0x55c55c37fd80 2024-09-21T03:41:38.553 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:38.550+0000 7fa30482f640 1 --2- 172.21.15.89:0/1102893882 >> [v2:172.21.15.110:6838/132366709,v1:172.21.15.110:6839/132366709] conn(0x55c55c37fd80 0x55c55c780430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:38.558 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:38.555+0000 7fa30482f640 1 --2- 172.21.15.89:0/1102893882 >> [v2:172.21.15.110:6838/132366709,v1:172.21.15.110:6839/132366709] conn(0x55c55c37fd80 0x55c55c780430 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f7516d26286a1db server_cookie=b0ac8dca5e725e73 in_seq=0 out_seq=0 2024-09-21T03:41:38.558 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:38.555+0000 7fa2e8ff9640 10 client.15780 ms_handle_connect on v2:172.21.15.110:6838/132366709 2024-09-21T03:41:38.561 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:38.557+0000 7fa2e8ff9640 1 -- 172.21.15.89:0/1102893882 <== mds.0 v2:172.21.15.110:6838/132366709 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55c55c6dd640 con 0x55c55c37fd80 2024-09-21T03:41:38.561 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:38.557+0000 7fa2e8ff9640 10 client.15780 handle_command_reply: tid=0 2024-09-21T03:41:38.561 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:38.557+0000 7fa30e200740 1 client.15780 shutdown 2024-09-21T03:41:38.561 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:38.558+0000 7fa30e200740 1 -- 172.21.15.89:0/1102893882 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c55c3a04d0 msgr2=0x55c55c772f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:38.561 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:38.558+0000 7fa30e200740 1 --2- 172.21.15.89:0/1102893882 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c55c3a04d0 0x55c55c772f80 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7fa2fc02a940 tx=0x7fa2fc004040 comp rx=0 tx=0).stop 2024-09-21T03:41:38.561 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:38.559+0000 7fa30e200740 1 -- 172.21.15.89:0/1102893882 shutdown_connections 2024-09-21T03:41:38.561 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:38.559+0000 7fa30e200740 1 --2- 172.21.15.89:0/1102893882 >> [v2:172.21.15.110:6838/132366709,v1:172.21.15.110:6839/132366709] conn(0x55c55c37fd80 0x55c55c780430 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:38.561 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:38.559+0000 7fa30e200740 1 --2- 172.21.15.89:0/1102893882 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c55c3c11b0 0x55c55c773af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:38.561 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:38.559+0000 7fa30e200740 1 --2- 172.21.15.89:0/1102893882 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c55c3a04d0 0x55c55c772f80 secure :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7fa2fc02a940 tx=0x7fa2fc004040 comp rx=0 tx=0).stop 2024-09-21T03:41:38.561 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:38.559+0000 7fa30e200740 1 --2- 172.21.15.89:0/1102893882 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c55c3982f0 0x55c55c7728c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:38.561 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:38.559+0000 7fa30e200740 1 -- 172.21.15.89:0/1102893882 >> 172.21.15.89:0/1102893882 conn(0x55c55c3b5300 msgr2=0x55c55c763150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:41:38.562 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:38.559+0000 7fa2e8ff9640 0 client.15780 ms_handle_reset on v2:172.21.15.110:6838/132366709 2024-09-21T03:41:38.562 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:38.559+0000 7fa30e200740 1 -- 172.21.15.89:0/1102893882 shutdown_connections 2024-09-21T03:41:38.562 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:38.560+0000 7fa30e200740 1 -- 172.21.15.89:0/1102893882 wait complete. 2024-09-21T03:41:38.563 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:38.561+0000 7fa30e200740 20 client.15780 trim_cache size 0 max 16384 2024-09-21T03:41:38.594 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:38.592+0000 7fa30e200740 1 Processor -- start 2024-09-21T03:41:38.595 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:38.592+0000 7fa30e200740 1 -- start start 2024-09-21T03:41:38.595 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:38.592+0000 7fa30e200740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c55c3982f0 0x55c55c7728c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:38.595 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:38.592+0000 7fa30e200740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c55c3c11b0 0x55c55c772e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:38.595 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:38.592+0000 7fa30e200740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c55c3551c0 0x55c55c768fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:38.596 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:38.592+0000 7fa30e200740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fa3001bd9f0 con 0x55c55c3551c0 2024-09-21T03:41:38.596 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:38.592+0000 7fa30e200740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fa3001bb2c0 con 0x55c55c3c11b0 2024-09-21T03:41:38.596 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:38.592+0000 7fa30e200740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fa3001b9110 con 0x55c55c3982f0 2024-09-21T03:41:38.596 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:38.593+0000 7fa305030640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c55c3c11b0 0x55c55c772e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:38.596 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:38.593+0000 7fa30482f640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c55c3551c0 0x55c55c768fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:38.596 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:38.593+0000 7fa30482f640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c55c3551c0 0x55c55c768fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:33680/0 (socket says 172.21.15.89:33680) 2024-09-21T03:41:38.596 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:38.593+0000 7fa30482f640 1 -- 172.21.15.89:0/3220125401 learned_addr learned my addr 172.21.15.89:0/3220125401 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-21T03:41:38.596 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:38.593+0000 7fa30482f640 1 --2- 172.21.15.89:0/3220125401 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c55c3982f0 0x55c55c7728c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:38.596 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:38.593+0000 7fa30482f640 1 -- 172.21.15.89:0/3220125401 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c55c3982f0 msgr2=0x55c55c7728c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:41:38.596 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:38.593+0000 7fa30482f640 1 --2- 172.21.15.89:0/3220125401 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c55c3982f0 0x55c55c7728c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:38.596 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:38.593+0000 7fa30482f640 1 -- 172.21.15.89:0/3220125401 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c55c3c11b0 msgr2=0x55c55c772e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:41:38.596 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:38.593+0000 7fa30482f640 1 --2- 172.21.15.89:0/3220125401 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c55c3c11b0 0x55c55c772e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:38.596 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:38.593+0000 7fa30482f640 1 -- 172.21.15.89:0/3220125401 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c55c77c8c0 con 0x55c55c3551c0 2024-09-21T03:41:38.597 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:38.593+0000 7fa305030640 1 --2- 172.21.15.89:0/3220125401 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c55c3c11b0 0x55c55c772e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:41:38.602 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:38.598+0000 7fa30482f640 1 --2- 172.21.15.89:0/3220125401 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c55c3551c0 0x55c55c768fe0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7fa2f8002230 tx=0x7fa2f8005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65228597e658d446 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:41:38.602 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:38.600+0000 7fa2eaffd640 1 -- 172.21.15.89:0/3220125401 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2f800fd70 con 0x55c55c3551c0 2024-09-21T03:41:38.603 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:38.600+0000 7fa2eaffd640 1 -- 172.21.15.89:0/3220125401 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa2f8003e00 con 0x55c55c3551c0 2024-09-21T03:41:38.604 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:38.600+0000 7fa2eaffd640 1 -- 172.21.15.89:0/3220125401 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2f80056a0 con 0x55c55c3551c0 2024-09-21T03:41:38.604 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:38.601+0000 7fa30e200740 1 -- 172.21.15.89:0/3220125401 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c55c3551c0 msgr2=0x55c55c768fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:38.604 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:38.601+0000 7fa30e200740 1 --2- 172.21.15.89:0/3220125401 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c55c3551c0 0x55c55c768fe0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7fa2f8002230 tx=0x7fa2f8005fd0 comp rx=0 tx=0).stop 2024-09-21T03:41:38.604 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:38.601+0000 7fa30e200740 1 -- 172.21.15.89:0/3220125401 shutdown_connections 2024-09-21T03:41:38.604 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:38.601+0000 7fa30e200740 1 --2- 172.21.15.89:0/3220125401 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c55c3c11b0 0x55c55c772e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:38.605 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:38.601+0000 7fa30e200740 1 --2- 172.21.15.89:0/3220125401 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c55c3551c0 0x55c55c768fe0 secure :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7fa2f8002230 tx=0x7fa2f8005fd0 comp rx=0 tx=0).stop 2024-09-21T03:41:38.605 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:38.601+0000 7fa30e200740 1 --2- 172.21.15.89:0/3220125401 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c55c3982f0 0x55c55c7728c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:38.605 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:38.601+0000 7fa30e200740 1 -- 172.21.15.89:0/3220125401 >> 172.21.15.89:0/3220125401 conn(0x55c55c3b5300 msgr2=0x55c55c764480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:41:38.605 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:38.603+0000 7fa30e200740 1 -- 172.21.15.89:0/3220125401 shutdown_connections 2024-09-21T03:41:38.605 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:38.603+0000 7fa30e200740 1 -- 172.21.15.89:0/3220125401 wait complete. 2024-09-21T03:41:38.606 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:38.603+0000 7fa30e200740 1 -- start start 2024-09-21T03:41:38.606 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:38.603+0000 7fa30e200740 1 Processor -- start 2024-09-21T03:41:38.607 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:38.604+0000 7fa30e200740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c55c3982f0 0x55c55c77a650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:38.607 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:38.604+0000 7fa30e200740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c55c3c11b0 0x55c55c779310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:38.607 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:38.604+0000 7fa30e200740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c55c7798a0 0x55c55c779d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:38.607 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:38.604+0000 7fa30e200740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fa3001b9780 con 0x55c55c7798a0 2024-09-21T03:41:38.607 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:38.604+0000 7fa30e200740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fa3001babf0 con 0x55c55c3c11b0 2024-09-21T03:41:38.607 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:38.604+0000 7fa30e200740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fa3001b9e50 con 0x55c55c3982f0 2024-09-21T03:41:38.607 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:38.604+0000 7fa30e200740 10 client.0 _ll_register_callbacks cb 0x7fa2f80093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:41:38.607 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:38.605+0000 7fa305030640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c55c3c11b0 0x55c55c779310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:38.607 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:38.605+0000 7fa30482f640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c55c7798a0 0x55c55c779d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:38.608 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:38.606+0000 7fa30482f640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c55c7798a0 0x55c55c779d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:33686/0 (socket says 172.21.15.89:33686) 2024-09-21T03:41:38.608 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:38.606+0000 7fa30482f640 1 -- 172.21.15.89:0/3907039479 learned_addr learned my addr 172.21.15.89:0/3907039479 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-21T03:41:38.608 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:38.606+0000 7fa30482f640 1 --2- 172.21.15.89:0/3907039479 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c55c3982f0 0x55c55c77a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:38.608 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:38.606+0000 7fa30482f640 1 -- 172.21.15.89:0/3907039479 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c55c3c11b0 msgr2=0x55c55c779310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:41:38.609 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:38.606+0000 7fa30482f640 1 --2- 172.21.15.89:0/3907039479 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c55c3c11b0 0x55c55c779310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:38.609 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:38.606+0000 7fa30482f640 1 -- 172.21.15.89:0/3907039479 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c55c7798a0 msgr2=0x55c55c779d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:41:38.609 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:38.606+0000 7fa30482f640 1 --2- 172.21.15.89:0/3907039479 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c55c7798a0 0x55c55c779d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:38.609 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:38.606+0000 7fa30482f640 1 -- 172.21.15.89:0/3907039479 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c55c778190 con 0x55c55c3982f0 2024-09-21T03:41:38.613 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:38.610+0000 7fa30482f640 1 --2- 172.21.15.89:0/3907039479 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c55c3982f0 0x55c55c77a650 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fa2f8009690 tx=0x7fa2f8005fd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1d420497bd6f7b9a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:41:38.613 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:38.610+0000 7fa2e8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.114:3300/0 2024-09-21T03:41:38.613 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:38.610+0000 7fa2e8ff9640 1 -- 172.21.15.89:0/3907039479 <== mon.1 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2f8003b70 con 0x55c55c3982f0 2024-09-21T03:41:38.613 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:38.611+0000 7fa30e200740 1 -- 172.21.15.89:0/3907039479 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa2f8003e00 con 0x55c55c3982f0 2024-09-21T03:41:38.613 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:38.611+0000 7fa2e8ff9640 1 -- 172.21.15.89:0/3907039479 <== mon.1 v2:172.21.15.114:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 9121+0+0 (secure 0 0 0) 0x7fa2f8005420 con 0x55c55c3982f0 2024-09-21T03:41:38.613 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:38.611+0000 7fa2e8ff9640 1 -- 172.21.15.89:0/3907039479 <== mon.1 v2:172.21.15.114:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fa2f8005620 con 0x55c55c3982f0 2024-09-21T03:41:38.617 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:38.613+0000 7fa30e200740 10 client.25372 fetch_fsmap learned FSMap version 26 2024-09-21T03:41:38.617 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:38.613+0000 7fa30e200740 1 -- 172.21.15.89:0/3907039479 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55c55c7eb2a0 con 0x55c55c3982f0 2024-09-21T03:41:38.622 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:38.619+0000 7fa2e8ff9640 1 -- 172.21.15.89:0/3907039479 <== mon.1 v2:172.21.15.114:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa2f8003e00 con 0x55c55c3982f0 2024-09-21T03:41:38.623 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:38.620+0000 7fa2e8ff9640 1 -- 172.21.15.89:0/3907039479 <== mon.1 v2:172.21.15.114:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fa2f80378c0 con 0x55c55c3982f0 2024-09-21T03:41:38.623 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:38.621+0000 7fa30e200740 10 client.25372 fetch_fsmap finished waiting for FSMap version 26 2024-09-21T03:41:38.623 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:38.621+0000 7fa30e200740 10 client.25372 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T03:41:38.623 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:38.621+0000 7fa30e200740 20 client.25372 populate_metadata read hostname 'smithi089' 2024-09-21T03:41:38.623 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:38.621+0000 7fa30e200740 1 --2- 172.21.15.89:0/3907039479 >> [v2:172.21.15.110:6838/132366709,v1:172.21.15.110:6839/132366709] conn(0x55c55c7eb7c0 0x55c55c7edbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:38.624 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:38.621+0000 7fa30e200740 4 client.25372 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T03:41:38.624 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:38.621+0000 7fa30e200740 1 -- 172.21.15.89:0/3907039479 --> [v2:172.21.15.110:6838/132366709,v1:172.21.15.110:6839/132366709] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55c55c6d90f0 con 0x55c55c7eb7c0 2024-09-21T03:41:38.626 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:38.623+0000 7fa305030640 1 --2- 172.21.15.89:0/3907039479 >> [v2:172.21.15.110:6838/132366709,v1:172.21.15.110:6839/132366709] conn(0x55c55c7eb7c0 0x55c55c7edbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:38.628 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:38.625+0000 7fa305030640 1 --2- 172.21.15.89:0/3907039479 >> [v2:172.21.15.110:6838/132366709,v1:172.21.15.110:6839/132366709] conn(0x55c55c7eb7c0 0x55c55c7edbe0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=929923933d7b250e server_cookie=eda56b04579802b4 in_seq=0 out_seq=0 2024-09-21T03:41:38.630 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:38.626+0000 7fa2e8ff9640 10 client.25372 ms_handle_connect on v2:172.21.15.110:6838/132366709 2024-09-21T03:41:38.670 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:38.668+0000 7fa2e8ff9640 1 -- 172.21.15.89:0/3907039479 <== mds.0 v2:172.21.15.110:6838/132366709 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55c55c6d90f0 con 0x55c55c7eb7c0 2024-09-21T03:41:38.670 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:38.668+0000 7fa2e8ff9640 10 client.25372 handle_command_reply: tid=0 2024-09-21T03:41:38.671 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:38.668+0000 7fa30e200740 1 client.25372 shutdown 2024-09-21T03:41:38.671 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:38.668+0000 7fa30e200740 1 -- 172.21.15.89:0/3907039479 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c55c3982f0 msgr2=0x55c55c77a650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:38.671 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:38.668+0000 7fa30e200740 1 --2- 172.21.15.89:0/3907039479 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c55c3982f0 0x55c55c77a650 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fa2f8009690 tx=0x7fa2f8005fd0 comp rx=0 tx=0).stop 2024-09-21T03:41:38.672 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:38.669+0000 7fa30e200740 1 -- 172.21.15.89:0/3907039479 shutdown_connections 2024-09-21T03:41:38.672 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:38.669+0000 7fa30e200740 1 --2- 172.21.15.89:0/3907039479 >> [v2:172.21.15.110:6838/132366709,v1:172.21.15.110:6839/132366709] conn(0x55c55c7eb7c0 0x55c55c7edbe0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:38.672 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:38.669+0000 7fa30e200740 1 --2- 172.21.15.89:0/3907039479 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c55c3c11b0 0x55c55c779310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:38.672 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:38.669+0000 7fa30e200740 1 --2- 172.21.15.89:0/3907039479 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c55c7798a0 0x55c55c779d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:38.672 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:38.669+0000 7fa30e200740 1 --2- 172.21.15.89:0/3907039479 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c55c3982f0 0x55c55c77a650 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:38.672 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:38.669+0000 7fa30e200740 1 -- 172.21.15.89:0/3907039479 >> 172.21.15.89:0/3907039479 conn(0x55c55c3b5300 msgr2=0x55c55c7728a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:41:38.672 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:38.669+0000 7fa2e8ff9640 0 client.25372 ms_handle_reset on v2:172.21.15.110:6838/132366709 2024-09-21T03:41:38.673 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:38.670+0000 7fa30e200740 1 -- 172.21.15.89:0/3907039479 shutdown_connections 2024-09-21T03:41:38.673 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:38.671+0000 7fa30e200740 1 -- 172.21.15.89:0/3907039479 wait complete. 2024-09-21T03:41:38.677 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stdout:{ 2024-09-21T03:41:38.677 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stdout: "message": "", 2024-09-21T03:41:38.677 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stdout: "return_code": 0 2024-09-21T03:41:38.677 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stdout:} 2024-09-21T03:41:38.677 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:38.673+0000 7fa30e200740 20 client.25372 trim_cache size 0 max 16384 2024-09-21T03:41:38.677 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:38.675+0000 7fa306032640 1 -- 172.21.15.89:0/3856145499 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7fa2ec0b66e0 msgr2=0x7fa2ec0b8bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:38.678 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:38.675+0000 7fa306032640 1 --2- 172.21.15.89:0/3856145499 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7fa2ec0b66e0 0x7fa2ec0b8bf0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fa2f00ad3c0 tx=0x7fa2f0005e20 comp rx=0 tx=0).stop 2024-09-21T03:41:38.678 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:38.675+0000 7fa306032640 1 -- 172.21.15.89:0/3856145499 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa30010f830 msgr2=0x7fa3001a9c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:38.678 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:38.675+0000 7fa306032640 1 --2- 172.21.15.89:0/3856145499 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa30010f830 0x7fa3001a9c90 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fa2f000c970 tx=0x7fa2f000ce30 comp rx=0 tx=0).stop 2024-09-21T03:41:38.678 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:38.676+0000 7fa306032640 1 -- 172.21.15.89:0/3856145499 shutdown_connections 2024-09-21T03:41:38.678 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:38.676+0000 7fa306032640 1 --2- 172.21.15.89:0/3856145499 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7fa2ec0b66e0 0x7fa2ec0b8bf0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:38.678 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:38.676+0000 7fa306032640 1 --2- 172.21.15.89:0/3856145499 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa30010f830 0x7fa3001a9c90 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:38.678 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:38.676+0000 7fa306032640 1 --2- 172.21.15.89:0/3856145499 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa30010cd20 0x7fa3001a90e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:38.678 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:38.676+0000 7fa306032640 1 --2- 172.21.15.89:0/3856145499 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa30010a3c0 0x7fa3001a8a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:38.678 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:38.676+0000 7fa306032640 1 -- 172.21.15.89:0/3856145499 >> 172.21.15.89:0/3856145499 conn(0x7fa300070640 msgr2=0x7fa30010b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:41:38.679 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:38.677+0000 7fa306032640 1 -- 172.21.15.89:0/3856145499 shutdown_connections 2024-09-21T03:41:38.683 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:38.680+0000 7fa306032640 1 -- 172.21.15.89:0/3856145499 wait complete. 2024-09-21T03:41:39.061 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:41:38 smithi089 ceph-mon[30928]: pgmap v427: 129 pgs: 129 active+clean; 258 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 14 MiB/s wr, 3.21k op/s 2024-09-21T03:41:39.211 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:41:38 smithi114 ceph-mon[34140]: pgmap v427: 129 pgs: 129 active+clean; 258 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 14 MiB/s wr, 3.21k op/s 2024-09-21T03:41:39.250 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:41:38 smithi110 ceph-mon[34000]: pgmap v427: 129 pgs: 129 active+clean; 258 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 14 MiB/s wr, 3.21k op/s 2024-09-21T03:41:41.211 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:41:40 smithi114 ceph-mon[34140]: pgmap v428: 129 pgs: 129 active+clean; 258 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 15 MiB/s wr, 3.37k op/s 2024-09-21T03:41:41.250 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:41:40 smithi110 ceph-mon[34000]: pgmap v428: 129 pgs: 129 active+clean; 258 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 15 MiB/s wr, 3.37k op/s 2024-09-21T03:41:41.311 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:41:40 smithi089 ceph-mon[30928]: pgmap v428: 129 pgs: 129 active+clean; 258 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 15 MiB/s wr, 3.37k op/s 2024-09-21T03:41:43.211 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:41:42 smithi114 ceph-mon[34140]: pgmap v429: 129 pgs: 129 active+clean; 258 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 12 MiB/s wr, 2.61k op/s 2024-09-21T03:41:43.250 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:41:42 smithi110 ceph-mon[34000]: pgmap v429: 129 pgs: 129 active+clean; 258 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 12 MiB/s wr, 2.61k op/s 2024-09-21T03:41:43.302 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:41:42 smithi089 ceph-mon[30928]: pgmap v429: 129 pgs: 129 active+clean; 258 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 12 MiB/s wr, 2.61k op/s 2024-09-21T03:41:45.122 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-09-21T03:41:45.122 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":26,"btime":"2024-09-21T03:39:26:058611+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14721,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6835/2341538514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":2341538514},{"type":"v1","addr":"172.21.15.89:6835","nonce":2341538514}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14733,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6837/146708592","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6836","nonce":146708592},{"type":"v1","addr":"172.21.15.89:6837","nonce":146708592}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14781,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6839/2269965499","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6838","nonce":2269965499},{"type":"v1","addr":"172.21.15.89:6839","nonce":2269965499}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14793,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6841/650850383","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6840","nonce":650850383},{"type":"v1","addr":"172.21.15.89:6841","nonce":650850383}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24565,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6833/752336912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6832","nonce":752336912},{"type":"v1","addr":"172.21.15.114:6833","nonce":752336912}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24575,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6833/127607331","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6832","nonce":127607331},{"type":"v1","addr":"172.21.15.110:6833","nonce":127607331}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T03:37:13.570694+0000","modified":"2024-09-21T03:39:25.333954+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24638,"mds_1":24631,"mds_2":24614},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24593":{"gid":24593,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.110:6835/671820654","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6834","nonce":671820654},{"type":"v1","addr":"172.21.15.110:6835","nonce":671820654}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24604":{"gid":24604,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.114:6835/1049714548","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6834","nonce":1049714548},{"type":"v1","addr":"172.21.15.114:6835","nonce":1049714548}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24610":{"gid":24610,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.114:6837/1941213818","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6836","nonce":1941213818},{"type":"v1","addr":"172.21.15.114:6837","nonce":1941213818}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24614":{"gid":24614,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":15,"addr":"172.21.15.110:6837/4047124589","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6836","nonce":4047124589},{"type":"v1","addr":"172.21.15.110:6837","nonce":4047124589}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24631":{"gid":24631,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":15,"addr":"172.21.15.114:6839/3427369944","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6838","nonce":3427369944},{"type":"v1","addr":"172.21.15.114:6839","nonce":3427369944}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24638":{"gid":24638,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.110:6839/132366709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6838","nonce":132366709},{"type":"v1","addr":"172.21.15.110:6839","nonce":132366709}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24638,"qdb_cluster":[24638,24631,24614]},"id":1}]} 2024-09-21T03:41:45.123 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:43.764+0000 7f0ea0e30640 1 -- 172.21.15.89:0/3519530354 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0e9c1049d0 msgr2=0x7f0e9c104de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:45.123 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:43.764+0000 7f0ea0e30640 1 --2- 172.21.15.89:0/3519530354 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0e9c1049d0 0x7f0e9c104de0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f0e8c009900 tx=0x7f0e8c02a410 comp rx=0 tx=0).stop 2024-09-21T03:41:45.123 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:43.764+0000 7f0ea0e30640 1 -- 172.21.15.89:0/3519530354 shutdown_connections 2024-09-21T03:41:45.123 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:43.764+0000 7f0ea0e30640 1 --2- 172.21.15.89:0/3519530354 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0e9c10de30 0x7f0e9c110320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:45.123 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:43.764+0000 7f0ea0e30640 1 --2- 172.21.15.89:0/3519530354 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0e9c105320 0x7f0e9c10d870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:45.123 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:43.764+0000 7f0ea0e30640 1 --2- 172.21.15.89:0/3519530354 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0e9c1049d0 0x7f0e9c104de0 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:45.123 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:43.764+0000 7f0ea0e30640 1 -- 172.21.15.89:0/3519530354 >> 172.21.15.89:0/3519530354 conn(0x7f0e9c079740 msgr2=0x7f0e9c079b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:41:45.123 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:43.764+0000 7f0ea0e30640 1 -- 172.21.15.89:0/3519530354 shutdown_connections 2024-09-21T03:41:45.123 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:43.764+0000 7f0ea0e30640 1 -- 172.21.15.89:0/3519530354 wait complete. 2024-09-21T03:41:45.123 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:43.765+0000 7f0ea0e30640 1 Processor -- start 2024-09-21T03:41:45.123 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:43.765+0000 7f0ea0e30640 1 -- start start 2024-09-21T03:41:45.124 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:43.765+0000 7f0ea0e30640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0e9c1049d0 0x7f0e9c19a410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:45.124 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:43.765+0000 7f0ea0e30640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0e9c105320 0x7f0e9c19a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:45.124 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:43.765+0000 7f0ea0e30640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0e9c10de30 0x7f0e9c19fc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:45.124 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:43.765+0000 7f0ea0e30640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f0e9c113940 con 0x7f0e9c105320 2024-09-21T03:41:45.124 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:43.765+0000 7f0ea0e30640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f0e9c1137c0 con 0x7f0e9c1049d0 2024-09-21T03:41:45.124 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:43.765+0000 7f0ea0e30640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f0e9c113ac0 con 0x7f0e9c10de30 2024-09-21T03:41:45.124 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:43.765+0000 7f0e99d9b640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0e9c105320 0x7f0e9c19a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:45.124 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:43.765+0000 7f0e9a59c640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0e9c1049d0 0x7f0e9c19a410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:45.124 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:43.765+0000 7f0e99d9b640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0e9c105320 0x7f0e9c19a9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:33694/0 (socket says 172.21.15.89:33694) 2024-09-21T03:41:45.124 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:43.765+0000 7f0e99d9b640 1 -- 172.21.15.89:0/923100544 learned_addr learned my addr 172.21.15.89:0/923100544 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-21T03:41:45.124 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:43.765+0000 7f0e9ad9d640 1 --2- 172.21.15.89:0/923100544 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0e9c10de30 0x7f0e9c19fc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:45.124 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:43.766+0000 7f0e9a59c640 1 -- 172.21.15.89:0/923100544 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0e9c10de30 msgr2=0x7f0e9c19fc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:45.124 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:43.766+0000 7f0e9a59c640 1 --2- 172.21.15.89:0/923100544 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0e9c10de30 0x7f0e9c19fc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:45.125 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:43.766+0000 7f0e9a59c640 1 -- 172.21.15.89:0/923100544 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0e9c105320 msgr2=0x7f0e9c19a9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:45.125 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:43.766+0000 7f0e9a59c640 1 --2- 172.21.15.89:0/923100544 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0e9c105320 0x7f0e9c19a9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:45.125 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:43.766+0000 7f0e9a59c640 1 -- 172.21.15.89:0/923100544 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e9c1a02c0 con 0x7f0e9c1049d0 2024-09-21T03:41:45.125 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:43.766+0000 7f0e99d9b640 1 --2- 172.21.15.89:0/923100544 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0e9c105320 0x7f0e9c19a9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:41:45.125 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:43.766+0000 7f0e9a59c640 1 --2- 172.21.15.89:0/923100544 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0e9c1049d0 0x7f0e9c19a410 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f0e8c009230 tx=0x7f0e8c0042f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:41:45.125 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:43.766+0000 7f0e8b7fe640 1 -- 172.21.15.89:0/923100544 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e8c03f070 con 0x7f0e9c1049d0 2024-09-21T03:41:45.125 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:43.766+0000 7f0e8b7fe640 1 -- 172.21.15.89:0/923100544 <== mon.2 v2:172.21.15.110:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0e8c03b460 con 0x7f0e9c1049d0 2024-09-21T03:41:45.125 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:43.766+0000 7f0ea0e30640 1 -- 172.21.15.89:0/923100544 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0e9c1a0550 con 0x7f0e9c1049d0 2024-09-21T03:41:45.125 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:43.766+0000 7f0e8b7fe640 1 -- 172.21.15.89:0/923100544 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e8c035070 con 0x7f0e9c1049d0 2024-09-21T03:41:45.125 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:43.766+0000 7f0ea0e30640 1 -- 172.21.15.89:0/923100544 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0e9c1a0a00 con 0x7f0e9c1049d0 2024-09-21T03:41:45.125 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:43.767+0000 7f0ea0e30640 1 -- 172.21.15.89:0/923100544 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0e9c06c240 con 0x7f0e9c1049d0 2024-09-21T03:41:45.125 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:43.769+0000 7f0e8b7fe640 1 -- 172.21.15.89:0/923100544 <== mon.2 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 21) ==== 154512+0+0 (secure 0 0 0) 0x7f0e8c03b910 con 0x7f0e9c1049d0 2024-09-21T03:41:45.125 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:43.769+0000 7f0e8b7fe640 1 --2- 172.21.15.89:0/923100544 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f0e840b6540 0x7f0e840b8a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:45.125 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:43.769+0000 7f0e99d9b640 1 --2- 172.21.15.89:0/923100544 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f0e840b6540 0x7f0e840b8a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:45.125 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:43.769+0000 7f0e8b7fe640 1 -- 172.21.15.89:0/923100544 <== mon.2 v2:172.21.15.110:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 9121+0+0 (secure 0 0 0) 0x7f0e8c107b10 con 0x7f0e9c1049d0 2024-09-21T03:41:45.126 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:43.770+0000 7f0e99d9b640 1 --2- 172.21.15.89:0/923100544 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f0e840b6540 0x7f0e840b8a50 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f0e9c07a060 tx=0x7f0e9000a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:41:45.126 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:43.773+0000 7f0e8b7fe640 1 -- 172.21.15.89:0/923100544 <== mon.2 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f0e8c0d23a0 con 0x7f0e9c1049d0 2024-09-21T03:41:45.126 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:44.182+0000 7f0ea0e30640 1 -- 172.21.15.89:0/923100544 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0e9c104de0 con 0x7f0e9c1049d0 2024-09-21T03:41:45.126 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:44.183+0000 7f0e8b7fe640 1 -- 172.21.15.89:0/923100544 <== mon.2 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11912 (secure 0 0 0) 0x7f0e8c0d72e0 con 0x7f0e9c1049d0 2024-09-21T03:41:45.126 INFO:teuthology.orchestra.run.smithi089.stderr:dumped fsmap epoch 26 2024-09-21T03:41:45.126 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:44.187+0000 7f0ea0e30640 1 -- 172.21.15.89:0/923100544 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f0e840b6540 msgr2=0x7f0e840b8a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:45.126 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:44.187+0000 7f0ea0e30640 1 --2- 172.21.15.89:0/923100544 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f0e840b6540 0x7f0e840b8a50 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f0e9c07a060 tx=0x7f0e9000a380 comp rx=0 tx=0).stop 2024-09-21T03:41:45.126 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:44.187+0000 7f0ea0e30640 1 -- 172.21.15.89:0/923100544 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0e9c1049d0 msgr2=0x7f0e9c19a410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:45.126 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:44.187+0000 7f0ea0e30640 1 --2- 172.21.15.89:0/923100544 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0e9c1049d0 0x7f0e9c19a410 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f0e8c009230 tx=0x7f0e8c0042f0 comp rx=0 tx=0).stop 2024-09-21T03:41:45.126 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:44.187+0000 7f0ea0e30640 1 -- 172.21.15.89:0/923100544 shutdown_connections 2024-09-21T03:41:45.126 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:44.187+0000 7f0ea0e30640 1 --2- 172.21.15.89:0/923100544 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f0e840b6540 0x7f0e840b8a50 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:45.126 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:44.187+0000 7f0ea0e30640 1 --2- 172.21.15.89:0/923100544 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0e9c10de30 0x7f0e9c19fc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:45.126 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:44.187+0000 7f0ea0e30640 1 --2- 172.21.15.89:0/923100544 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0e9c105320 0x7f0e9c19a9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:45.126 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:44.187+0000 7f0ea0e30640 1 --2- 172.21.15.89:0/923100544 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0e9c1049d0 0x7f0e9c19a410 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:45.126 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:44.187+0000 7f0ea0e30640 1 -- 172.21.15.89:0/923100544 >> 172.21.15.89:0/923100544 conn(0x7f0e9c079740 msgr2=0x7f0e9c0ff7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:41:45.127 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:44.188+0000 7f0ea0e30640 1 -- 172.21.15.89:0/923100544 shutdown_connections 2024-09-21T03:41:45.127 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:44.188+0000 7f0ea0e30640 1 -- 172.21.15.89:0/923100544 wait complete. 2024-09-21T03:41:45.127 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:41:44 smithi089 ceph-mon[30928]: pgmap v430: 129 pgs: 129 active+clean; 258 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 15 MiB/s wr, 3.31k op/s 2024-09-21T03:41:45.127 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:41:44 smithi089 ceph-mon[30928]: from='client.? 172.21.15.89:0/923100544' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:41:45.211 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:41:44 smithi114 ceph-mon[34140]: pgmap v430: 129 pgs: 129 active+clean; 258 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 15 MiB/s wr, 3.31k op/s 2024-09-21T03:41:45.211 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:41:44 smithi114 ceph-mon[34140]: from='client.? 172.21.15.89:0/923100544' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:41:45.250 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:41:44 smithi110 ceph-mon[34000]: pgmap v430: 129 pgs: 129 active+clean; 258 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 15 MiB/s wr, 3.31k op/s 2024-09-21T03:41:45.250 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:41:44 smithi110 ceph-mon[34000]: from='client.? 172.21.15.89:0/923100544' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:41:47.211 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:41:46 smithi114 ceph-mon[34140]: pgmap v431: 129 pgs: 129 active+clean; 258 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 11 MiB/s wr, 2.54k op/s 2024-09-21T03:41:47.250 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:41:46 smithi110 ceph-mon[34000]: pgmap v431: 129 pgs: 129 active+clean; 258 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 11 MiB/s wr, 2.54k op/s 2024-09-21T03:41:47.311 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:41:46 smithi089 ceph-mon[30928]: pgmap v431: 129 pgs: 129 active+clean; 258 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 11 MiB/s wr, 2.54k op/s 2024-09-21T03:41:49.211 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:41:48 smithi114 ceph-mon[34140]: pgmap v432: 129 pgs: 129 active+clean; 258 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 12 MiB/s wr, 2.63k op/s 2024-09-21T03:41:49.250 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:41:48 smithi110 ceph-mon[34000]: pgmap v432: 129 pgs: 129 active+clean; 258 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 12 MiB/s wr, 2.63k op/s 2024-09-21T03:41:49.311 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:41:48 smithi089 ceph-mon[30928]: pgmap v432: 129 pgs: 129 active+clean; 258 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 12 MiB/s wr, 2.63k op/s 2024-09-21T03:41:51.210 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:41:50 smithi114 ceph-mon[34140]: pgmap v433: 129 pgs: 129 active+clean; 258 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 13 MiB/s wr, 2.78k op/s 2024-09-21T03:41:51.249 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:41:50 smithi110 ceph-mon[34000]: pgmap v433: 129 pgs: 129 active+clean; 258 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 13 MiB/s wr, 2.78k op/s 2024-09-21T03:41:51.311 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:41:50 smithi089 ceph-mon[30928]: pgmap v433: 129 pgs: 129 active+clean; 258 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 13 MiB/s wr, 2.78k op/s 2024-09-21T03:41:51.870 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:51.867+0000 7fb31eb8f640 1 -- 172.21.15.89:0/2904015102 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb31810f830 msgr2=0x7fb31810fd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:51.870 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:51.867+0000 7fb31eb8f640 1 --2- 172.21.15.89:0/2904015102 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb31810f830 0x7fb31810fd10 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fb310009be0 tx=0x7fb31002a710 comp rx=0 tx=0).stop 2024-09-21T03:41:51.871 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:51.868+0000 7fb31eb8f640 1 -- 172.21.15.89:0/2904015102 shutdown_connections 2024-09-21T03:41:51.871 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:51.868+0000 7fb31eb8f640 1 --2- 172.21.15.89:0/2904015102 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb31810f830 0x7fb31810fd10 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:51.872 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:51.868+0000 7fb31eb8f640 1 --2- 172.21.15.89:0/2904015102 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb31810cd20 0x7fb31810f140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:51.872 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:51.868+0000 7fb31eb8f640 1 --2- 172.21.15.89:0/2904015102 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb31810a3c0 0x7fb31810c7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:51.872 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:51.868+0000 7fb31eb8f640 1 -- 172.21.15.89:0/2904015102 >> 172.21.15.89:0/2904015102 conn(0x7fb318070640 msgr2=0x7fb318072af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:41:51.872 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:51.869+0000 7fb31eb8f640 1 -- 172.21.15.89:0/2904015102 shutdown_connections 2024-09-21T03:41:51.876 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:51.873+0000 7fb31eb8f640 1 -- 172.21.15.89:0/2904015102 wait complete. 2024-09-21T03:41:51.876 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:51.874+0000 7fb31eb8f640 1 Processor -- start 2024-09-21T03:41:51.883 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:51.877+0000 7fb31eb8f640 1 -- start start 2024-09-21T03:41:51.884 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:51.877+0000 7fb31eb8f640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb31810a3c0 0x7fb3181a8a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:51.884 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:51.877+0000 7fb31eb8f640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb31810cd20 0x7fb3181a90b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:51.884 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:51.877+0000 7fb31eb8f640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb31810f830 0x7fb3181a9c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:51.884 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:51.877+0000 7fb31eb8f640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fb31811de00 con 0x7fb31810a3c0 2024-09-21T03:41:51.884 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:51.877+0000 7fb31eb8f640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fb31811dc80 con 0x7fb31810f830 2024-09-21T03:41:51.884 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:51.877+0000 7fb31eb8f640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fb31811df80 con 0x7fb31810cd20 2024-09-21T03:41:51.884 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:51.878+0000 7fb31db8d640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb31810a3c0 0x7fb3181a8a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:51.884 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:51.878+0000 7fb31db8d640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb31810a3c0 0x7fb3181a8a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:45166/0 (socket says 172.21.15.89:45166) 2024-09-21T03:41:51.884 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:51.878+0000 7fb31db8d640 1 -- 172.21.15.89:0/2800856099 learned_addr learned my addr 172.21.15.89:0/2800856099 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-21T03:41:51.884 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:51.878+0000 7fb31d38c640 1 --2- 172.21.15.89:0/2800856099 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb31810cd20 0x7fb3181a90b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:51.884 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:51.878+0000 7fb31e38e640 1 --2- 172.21.15.89:0/2800856099 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb31810f830 0x7fb3181a9c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:51.884 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:51.879+0000 7fb31db8d640 1 -- 172.21.15.89:0/2800856099 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb31810cd20 msgr2=0x7fb3181a90b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:51.884 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:51.879+0000 7fb31db8d640 1 --2- 172.21.15.89:0/2800856099 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb31810cd20 0x7fb3181a90b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:51.884 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:51.879+0000 7fb31db8d640 1 -- 172.21.15.89:0/2800856099 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb31810f830 msgr2=0x7fb3181a9c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:51.885 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:51.879+0000 7fb31db8d640 1 --2- 172.21.15.89:0/2800856099 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb31810f830 0x7fb3181a9c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:51.885 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:51.879+0000 7fb31db8d640 1 -- 172.21.15.89:0/2800856099 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3181aa480 con 0x7fb31810a3c0 2024-09-21T03:41:51.885 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:51.879+0000 7fb31db8d640 1 --2- 172.21.15.89:0/2800856099 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb31810a3c0 0x7fb3181a8a40 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7fb30800bea0 tx=0x7fb30800bf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:41:51.885 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:51.879+0000 7fb30effd640 1 -- 172.21.15.89:0/2800856099 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb308039070 con 0x7fb31810a3c0 2024-09-21T03:41:51.885 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:51.880+0000 7fb30effd640 1 -- 172.21.15.89:0/2800856099 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb308009c30 con 0x7fb31810a3c0 2024-09-21T03:41:51.885 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:51.880+0000 7fb30effd640 1 -- 172.21.15.89:0/2800856099 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb308031630 con 0x7fb31810a3c0 2024-09-21T03:41:51.885 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:51.881+0000 7fb31eb8f640 1 -- 172.21.15.89:0/2800856099 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb3181aa710 con 0x7fb31810a3c0 2024-09-21T03:41:51.885 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:51.881+0000 7fb31eb8f640 1 -- 172.21.15.89:0/2800856099 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb3180757d0 con 0x7fb31810a3c0 2024-09-21T03:41:51.887 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:51.885+0000 7fb326e41740 1 Processor -- start 2024-09-21T03:41:51.892 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:51.889+0000 7fb326e41740 1 -- start start 2024-09-21T03:41:51.892 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:51.890+0000 7fb30effd640 1 -- 172.21.15.89:0/2800856099 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 21) ==== 154512+0+0 (secure 0 0 0) 0x7fb308032470 con 0x7fb31810a3c0 2024-09-21T03:41:51.892 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:51.890+0000 7fb326e41740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a95806f1b0 0x55a958417040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:51.892 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:51.890+0000 7fb326e41740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a9580462f0 0x55a9584176b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:51.895 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:51.890+0000 7fb326e41740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a95804e4d0 0x55a958418200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:51.895 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:51.890+0000 7fb326e41740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55a9583896f0 con 0x55a95806f1b0 2024-09-21T03:41:51.895 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:51.890+0000 7fb326e41740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55a958378b20 con 0x55a95804e4d0 2024-09-21T03:41:51.895 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:51.890+0000 7fb326e41740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55a95837c930 con 0x55a9580462f0 2024-09-21T03:41:51.895 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:51.890+0000 7fb31e38e640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a95806f1b0 0x55a958417040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:51.895 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:51.890+0000 7fb31e38e640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a95806f1b0 0x55a958417040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:45182/0 (socket says 172.21.15.89:45182) 2024-09-21T03:41:51.895 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:51.890+0000 7fb31e38e640 1 -- 172.21.15.89:0/833347515 learned_addr learned my addr 172.21.15.89:0/833347515 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-21T03:41:51.895 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:51.890+0000 7fb31d38c640 1 --2- 172.21.15.89:0/833347515 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a95804e4d0 0x55a958418200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:51.895 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:51.893+0000 7fb31db8d640 1 --2- 172.21.15.89:0/833347515 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a9580462f0 0x55a9584176b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:51.895 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:51.893+0000 7fb31d38c640 1 -- 172.21.15.89:0/833347515 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a9580462f0 msgr2=0x55a9584176b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:41:51.896 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:51.893+0000 7fb31d38c640 1 --2- 172.21.15.89:0/833347515 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a9580462f0 0x55a9584176b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:51.896 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:51.893+0000 7fb31d38c640 1 -- 172.21.15.89:0/833347515 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a95806f1b0 msgr2=0x55a958417040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:41:51.896 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:51.893+0000 7fb31d38c640 1 --2- 172.21.15.89:0/833347515 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a95806f1b0 0x55a958417040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:51.896 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:51.893+0000 7fb31d38c640 1 -- 172.21.15.89:0/833347515 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a95833e530 con 0x55a95804e4d0 2024-09-21T03:41:51.896 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:51.893+0000 7fb31d38c640 1 --2- 172.21.15.89:0/833347515 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a95804e4d0 0x55a958418200 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fb314005fd0 tx=0x7fb3140107a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bc7b1283f60fb4ae server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:41:51.897 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:51.894+0000 7fb303fff640 1 -- 172.21.15.89:0/833347515 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb31401c480 con 0x55a95804e4d0 2024-09-21T03:41:51.897 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:51.894+0000 7fb303fff640 1 -- 172.21.15.89:0/833347515 <== mon.2 v2:172.21.15.110:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb31401cab0 con 0x55a95804e4d0 2024-09-21T03:41:51.897 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:51.894+0000 7fb303fff640 1 -- 172.21.15.89:0/833347515 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb314011af0 con 0x55a95804e4d0 2024-09-21T03:41:51.898 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:51.895+0000 7fb30effd640 1 --2- 172.21.15.89:0/2800856099 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7fb3040b6720 0x7fb3040b8c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:51.898 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:51.895+0000 7fb30effd640 1 -- 172.21.15.89:0/2800856099 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 9121+0+0 (secure 0 0 0) 0x7fb308108fb0 con 0x7fb31810a3c0 2024-09-21T03:41:51.898 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:51.895+0000 7fb326e41740 1 -- 172.21.15.89:0/833347515 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a95804e4d0 msgr2=0x55a958418200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:51.899 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:51.895+0000 7fb326e41740 1 --2- 172.21.15.89:0/833347515 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a95804e4d0 0x55a958418200 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fb314005fd0 tx=0x7fb3140107a0 comp rx=0 tx=0).stop 2024-09-21T03:41:51.901 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:51.898+0000 7fb31e38e640 1 --2- 172.21.15.89:0/2800856099 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7fb3040b6720 0x7fb3040b8c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:51.901 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:51.899+0000 7fb326e41740 1 -- 172.21.15.89:0/833347515 shutdown_connections 2024-09-21T03:41:51.901 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:51.899+0000 7fb326e41740 1 --2- 172.21.15.89:0/833347515 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a95804e4d0 0x55a958418200 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:51.901 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:51.899+0000 7fb326e41740 1 --2- 172.21.15.89:0/833347515 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a9580462f0 0x55a9584176b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:51.901 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:51.899+0000 7fb326e41740 1 --2- 172.21.15.89:0/833347515 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a95806f1b0 0x55a958417040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:51.901 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:51.899+0000 7fb326e41740 1 -- 172.21.15.89:0/833347515 >> 172.21.15.89:0/833347515 conn(0x55a958063300 msgr2=0x55a958412ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:41:51.902 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:51.900+0000 7fb31e38e640 1 --2- 172.21.15.89:0/2800856099 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7fb3040b6720 0x7fb3040b8c30 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fb3100099a0 tx=0x7fb310030740 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:41:51.902 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:51.900+0000 7fb326e41740 1 -- 172.21.15.89:0/833347515 shutdown_connections 2024-09-21T03:41:51.903 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:51.901+0000 7fb326e41740 1 -- 172.21.15.89:0/833347515 wait complete. 2024-09-21T03:41:51.904 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:51.902+0000 7fb326e41740 1 -- start start 2024-09-21T03:41:51.904 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:51.902+0000 7fb326e41740 1 Processor -- start 2024-09-21T03:41:51.905 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:51.902+0000 7fb326e41740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a9580462f0 0x55a958420820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:51.905 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:51.903+0000 7fb326e41740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a95804e4d0 0x55a958420ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:51.905 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:51.903+0000 7fb326e41740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a95806f1b0 0x55a958421a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:51.905 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:51.903+0000 7fb326e41740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55a9582b0ab0 con 0x55a95804e4d0 2024-09-21T03:41:51.905 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:51.903+0000 7fb326e41740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55a95831e730 con 0x55a9580462f0 2024-09-21T03:41:51.906 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:51.903+0000 7fb326e41740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55a958418d20 con 0x55a95806f1b0 2024-09-21T03:41:51.906 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:51.903+0000 7fb326e41740 10 client.0 _ll_register_callbacks cb 0x55a9583726b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:41:51.906 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:51.903+0000 7fb31d38c640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a95804e4d0 0x55a958420ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:51.906 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:51.903+0000 7fb31d38c640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a95804e4d0 0x55a958420ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:45194/0 (socket says 172.21.15.89:45194) 2024-09-21T03:41:51.906 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:51.903+0000 7fb31d38c640 1 -- 172.21.15.89:0/2133591093 learned_addr learned my addr 172.21.15.89:0/2133591093 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-21T03:41:51.906 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:51.903+0000 7fb31e38e640 1 --2- 172.21.15.89:0/2133591093 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a95806f1b0 0x55a958421a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:51.907 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:51.903+0000 7fb31db8d640 1 --2- 172.21.15.89:0/2133591093 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a9580462f0 0x55a958420820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:51.907 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:51.903+0000 7fb31d38c640 1 -- 172.21.15.89:0/2133591093 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a95806f1b0 msgr2=0x55a958421a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:41:51.907 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:51.903+0000 7fb31d38c640 1 --2- 172.21.15.89:0/2133591093 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a95806f1b0 0x55a958421a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:51.907 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:51.903+0000 7fb31d38c640 1 -- 172.21.15.89:0/2133591093 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a9580462f0 msgr2=0x55a958420820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:41:51.907 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:51.903+0000 7fb31d38c640 1 --2- 172.21.15.89:0/2133591093 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a9580462f0 0x55a958420820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:51.907 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:51.903+0000 7fb31d38c640 1 -- 172.21.15.89:0/2133591093 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a9580da790 con 0x55a95804e4d0 2024-09-21T03:41:51.907 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:51.904+0000 7fb31d38c640 1 --2- 172.21.15.89:0/2133591093 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a95804e4d0 0x55a958420ee0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7fb314005bc0 tx=0x7fb31401ce90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6634e3aa9bcd49d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:41:51.907 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:51.904+0000 7fb3017fa640 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2024-09-21T03:41:51.908 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:51.906+0000 7fb3017fa640 1 -- 172.21.15.89:0/2133591093 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb31402b060 con 0x55a95804e4d0 2024-09-21T03:41:51.908 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:51.906+0000 7fb3017fa640 1 -- 172.21.15.89:0/2133591093 <== mon.0 v2:172.21.15.89:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 9121+0+0 (secure 0 0 0) 0x7fb31401c890 con 0x55a95804e4d0 2024-09-21T03:41:51.909 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:51.906+0000 7fb326e41740 1 -- 172.21.15.89:0/2133591093 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb31401cab0 con 0x55a95804e4d0 2024-09-21T03:41:51.911 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:51.909+0000 7fb3017fa640 1 -- 172.21.15.89:0/2133591093 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fb314005040 con 0x55a95804e4d0 2024-09-21T03:41:51.915 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:51.912+0000 7fb326e41740 10 client.15804 fetch_fsmap learned FSMap version 26 2024-09-21T03:41:51.915 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:51.912+0000 7fb326e41740 1 -- 172.21.15.89:0/2133591093 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55a958153620 con 0x55a95804e4d0 2024-09-21T03:41:51.921 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:51.917+0000 7fb3017fa640 1 -- 172.21.15.89:0/2133591093 <== mon.0 v2:172.21.15.89:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb31401cab0 con 0x55a95804e4d0 2024-09-21T03:41:51.928 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:51.917+0000 7fb3017fa640 1 -- 172.21.15.89:0/2133591093 <== mon.0 v2:172.21.15.89:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fb314025070 con 0x55a95804e4d0 2024-09-21T03:41:51.929 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:51.925+0000 7fb326e41740 10 client.15804 fetch_fsmap finished waiting for FSMap version 26 2024-09-21T03:41:51.929 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:51.925+0000 7fb326e41740 10 client.15804 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T03:41:51.929 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:51.925+0000 7fb326e41740 20 client.15804 populate_metadata read hostname 'smithi089' 2024-09-21T03:41:51.929 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:51.925+0000 7fb326e41740 1 --2- 172.21.15.89:0/2133591093 >> [v2:172.21.15.110:6838/132366709,v1:172.21.15.110:6839/132366709] conn(0x55a95802dd80 0x55a95842e2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:51.929 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:51.925+0000 7fb326e41740 4 client.15804 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:41:51.929 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:51.925+0000 7fb326e41740 1 -- 172.21.15.89:0/2133591093 --> [v2:172.21.15.110:6838/132366709,v1:172.21.15.110:6839/132366709] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a95838b640 con 0x55a95802dd80 2024-09-21T03:41:51.930 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:51.928+0000 7fb31db8d640 1 --2- 172.21.15.89:0/2133591093 >> [v2:172.21.15.110:6838/132366709,v1:172.21.15.110:6839/132366709] conn(0x55a95802dd80 0x55a95842e2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:51.934 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:51.931+0000 7fb3017fa640 10 client.15804 ms_handle_connect on v2:172.21.15.110:6838/132366709 2024-09-21T03:41:51.934 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:51.931+0000 7fb31db8d640 1 --2- 172.21.15.89:0/2133591093 >> [v2:172.21.15.110:6838/132366709,v1:172.21.15.110:6839/132366709] conn(0x55a95802dd80 0x55a95842e2b0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8c03b111dcafc05e server_cookie=ccd98724870c20aa in_seq=0 out_seq=0 2024-09-21T03:41:51.934 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:51.932+0000 7fb3017fa640 1 -- 172.21.15.89:0/2133591093 <== mds.0 v2:172.21.15.110:6838/132366709 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55a95838b640 con 0x55a95802dd80 2024-09-21T03:41:51.934 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:51.932+0000 7fb3017fa640 10 client.15804 handle_command_reply: tid=0 2024-09-21T03:41:51.934 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:51.932+0000 7fb326e41740 1 client.15804 shutdown 2024-09-21T03:41:51.935 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:51.932+0000 7fb326e41740 1 -- 172.21.15.89:0/2133591093 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a95804e4d0 msgr2=0x55a958420ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:51.935 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:51.932+0000 7fb326e41740 1 --2- 172.21.15.89:0/2133591093 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a95804e4d0 0x55a958420ee0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7fb314005bc0 tx=0x7fb31401ce90 comp rx=0 tx=0).stop 2024-09-21T03:41:51.937 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:51.934+0000 7fb326e41740 1 -- 172.21.15.89:0/2133591093 shutdown_connections 2024-09-21T03:41:51.937 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:51.934+0000 7fb326e41740 1 --2- 172.21.15.89:0/2133591093 >> [v2:172.21.15.110:6838/132366709,v1:172.21.15.110:6839/132366709] conn(0x55a95802dd80 0x55a95842e2b0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:51.937 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:51.935+0000 7fb326e41740 1 --2- 172.21.15.89:0/2133591093 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a95806f1b0 0x55a958421a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:51.937 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:51.935+0000 7fb326e41740 1 --2- 172.21.15.89:0/2133591093 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a95804e4d0 0x55a958420ee0 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:51.937 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:51.935+0000 7fb326e41740 1 --2- 172.21.15.89:0/2133591093 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a9580462f0 0x55a958420820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:51.937 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:51.935+0000 7fb326e41740 1 -- 172.21.15.89:0/2133591093 >> 172.21.15.89:0/2133591093 conn(0x55a958063300 msgr2=0x55a958411150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:41:51.938 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:51.935+0000 7fb3017fa640 0 client.15804 ms_handle_reset on v2:172.21.15.110:6838/132366709 2024-09-21T03:41:51.941 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:51.939+0000 7fb326e41740 1 -- 172.21.15.89:0/2133591093 shutdown_connections 2024-09-21T03:41:51.941 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:51.939+0000 7fb326e41740 1 -- 172.21.15.89:0/2133591093 wait complete. 2024-09-21T03:41:51.941 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:51.939+0000 7fb326e41740 20 client.15804 trim_cache size 0 max 16384 2024-09-21T03:41:51.978 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:51.975+0000 7fb326e41740 1 Processor -- start 2024-09-21T03:41:51.978 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:51.976+0000 7fb326e41740 1 -- start start 2024-09-21T03:41:51.978 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:51.976+0000 7fb326e41740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a9580462f0 0x55a95840d870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:51.978 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:51.976+0000 7fb326e41740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a95804e4d0 0x55a958428930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:51.979 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:51.976+0000 7fb326e41740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a95806f1b0 0x55a95841ed30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:51.979 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:51.977+0000 7fb326e41740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fb3180824d0 con 0x55a95806f1b0 2024-09-21T03:41:51.979 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:51.977+0000 7fb326e41740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fb31807fda0 con 0x55a9580462f0 2024-09-21T03:41:51.979 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:51.977+0000 7fb326e41740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fb31807dbf0 con 0x55a95804e4d0 2024-09-21T03:41:51.980 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:51.977+0000 7fb31db8d640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a9580462f0 0x55a95840d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:51.980 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:51.977+0000 7fb31db8d640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a9580462f0 0x55a95840d870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.89:33544/0 (socket says 172.21.15.89:33544) 2024-09-21T03:41:51.980 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:51.977+0000 7fb31db8d640 1 -- 172.21.15.89:0/1605260044 learned_addr learned my addr 172.21.15.89:0/1605260044 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-21T03:41:51.981 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:51.978+0000 7fb31d38c640 1 --2- 172.21.15.89:0/1605260044 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a95804e4d0 0x55a958428930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:51.981 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:51.978+0000 7fb31db8d640 1 -- 172.21.15.89:0/1605260044 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a95804e4d0 msgr2=0x55a958428930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:41:51.981 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:51.978+0000 7fb31e38e640 1 --2- 172.21.15.89:0/1605260044 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a95806f1b0 0x55a95841ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:51.981 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:51.978+0000 7fb31db8d640 1 --2- 172.21.15.89:0/1605260044 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a95804e4d0 0x55a958428930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:51.981 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:51.978+0000 7fb31db8d640 1 -- 172.21.15.89:0/1605260044 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a95806f1b0 msgr2=0x55a95841ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:41:51.981 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:51.978+0000 7fb31db8d640 1 --2- 172.21.15.89:0/1605260044 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a95806f1b0 0x55a95841ed30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:51.981 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:51.978+0000 7fb31db8d640 1 -- 172.21.15.89:0/1605260044 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a95840cdf0 con 0x55a9580462f0 2024-09-21T03:41:51.981 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:51.979+0000 7fb31d38c640 1 --2- 172.21.15.89:0/1605260044 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a95804e4d0 0x55a958428930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:41:51.981 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:51.979+0000 7fb31e38e640 1 --2- 172.21.15.89:0/1605260044 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a95806f1b0 0x55a95841ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:41:51.981 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:51.979+0000 7fb31db8d640 1 --2- 172.21.15.89:0/1605260044 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a9580462f0 0x55a95840d870 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fb308106670 tx=0x7fb308000bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a6d7f6013d52579e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:41:51.982 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:51.981+0000 7fb3037fe640 1 -- 172.21.15.89:0/1605260044 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3080d2070 con 0x55a9580462f0 2024-09-21T03:41:51.982 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:51.981+0000 7fb3037fe640 1 -- 172.21.15.89:0/1605260044 <== mon.2 v2:172.21.15.110:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb3080ce8f0 con 0x55a9580462f0 2024-09-21T03:41:51.982 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:51.981+0000 7fb3037fe640 1 -- 172.21.15.89:0/1605260044 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3080ca730 con 0x55a9580462f0 2024-09-21T03:41:51.986 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:51.984+0000 7fb326e41740 1 -- 172.21.15.89:0/1605260044 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a9580462f0 msgr2=0x55a95840d870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:51.986 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:51.984+0000 7fb326e41740 1 --2- 172.21.15.89:0/1605260044 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a9580462f0 0x55a95840d870 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fb308106670 tx=0x7fb308000bc0 comp rx=0 tx=0).stop 2024-09-21T03:41:51.991 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:51.988+0000 7fb326e41740 1 -- 172.21.15.89:0/1605260044 shutdown_connections 2024-09-21T03:41:51.991 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:51.988+0000 7fb326e41740 1 --2- 172.21.15.89:0/1605260044 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a95806f1b0 0x55a95841ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:51.991 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:51.988+0000 7fb326e41740 1 --2- 172.21.15.89:0/1605260044 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a95804e4d0 0x55a958428930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:51.991 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:51.989+0000 7fb326e41740 1 --2- 172.21.15.89:0/1605260044 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a9580462f0 0x55a95840d870 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:51.991 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:51.989+0000 7fb326e41740 1 -- 172.21.15.89:0/1605260044 >> 172.21.15.89:0/1605260044 conn(0x55a958063300 msgr2=0x55a958412410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:41:51.995 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:51.993+0000 7fb326e41740 1 -- 172.21.15.89:0/1605260044 shutdown_connections 2024-09-21T03:41:51.996 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:51.993+0000 7fb326e41740 1 -- 172.21.15.89:0/1605260044 wait complete. 2024-09-21T03:41:51.996 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:51.994+0000 7fb326e41740 1 -- start start 2024-09-21T03:41:51.996 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:51.994+0000 7fb326e41740 1 Processor -- start 2024-09-21T03:41:51.997 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:51.995+0000 7fb326e41740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a9580462f0 0x55a9584251d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:51.997 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:51.995+0000 7fb326e41740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a95804e4d0 0x55a958425890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:51.999 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:51.996+0000 7fb326e41740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a95806f1b0 0x55a9584263b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:51.999 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:51.996+0000 7fb31db8d640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a9580462f0 0x55a9584251d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:51.999 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:51.996+0000 7fb31db8d640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a9580462f0 0x55a9584251d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.89:34622/0 (socket says 172.21.15.89:34622) 2024-09-21T03:41:51.999 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:51.996+0000 7fb31e38e640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a95806f1b0 0x55a9584263b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:51.999 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:51.996+0000 7fb31d38c640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a95804e4d0 0x55a958425890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:51.999 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:51.996+0000 7fb31d38c640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a95804e4d0 0x55a958425890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.89:33548/0 (socket says 172.21.15.89:33548) 2024-09-21T03:41:51.999 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:51.996+0000 7fb31e38e640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a95806f1b0 0x55a9584263b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:45206/0 (socket says 172.21.15.89:45206) 2024-09-21T03:41:52.000 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:51.996+0000 7fb31db8d640 1 -- 172.21.15.89:0/520169010 learned_addr learned my addr 172.21.15.89:0/520169010 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-21T03:41:52.000 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:51.996+0000 7fb326e41740 1 -- 172.21.15.89:0/520169010 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fb31807e260 con 0x55a95806f1b0 2024-09-21T03:41:52.000 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:51.997+0000 7fb326e41740 1 -- 172.21.15.89:0/520169010 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fb31807f6d0 con 0x55a95804e4d0 2024-09-21T03:41:52.000 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:51.997+0000 7fb326e41740 1 -- 172.21.15.89:0/520169010 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fb31807e930 con 0x55a9580462f0 2024-09-21T03:41:52.000 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:51.997+0000 7fb326e41740 10 client.0 _ll_register_callbacks cb 0x7fb31400bae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:41:52.003 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:51.999+0000 7fb31db8d640 1 -- 172.21.15.89:0/520169010 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a95804e4d0 msgr2=0x55a958425890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:41:52.003 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:51.999+0000 7fb31db8d640 1 --2- 172.21.15.89:0/520169010 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a95804e4d0 0x55a958425890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:52.003 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:51.999+0000 7fb31db8d640 1 -- 172.21.15.89:0/520169010 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a95806f1b0 msgr2=0x55a9584263b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:41:52.003 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:51.999+0000 7fb31db8d640 1 --2- 172.21.15.89:0/520169010 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a95806f1b0 0x55a9584263b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:52.004 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:51.999+0000 7fb31db8d640 1 -- 172.21.15.89:0/520169010 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a958426b10 con 0x55a9580462f0 2024-09-21T03:41:52.004 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:52.001+0000 7fb31db8d640 1 --2- 172.21.15.89:0/520169010 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a9580462f0 0x55a9584251d0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fb3080cadf0 tx=0x7fb3080cb280 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b4dea2ffb5c6e4b8 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:41:52.004 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:52.002+0000 7fb3017fa640 10 client.0 ms_handle_connect on v2:172.21.15.114:3300/0 2024-09-21T03:41:52.004 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:52.002+0000 7fb3017fa640 1 -- 172.21.15.89:0/520169010 <== mon.1 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3080d2040 con 0x55a9580462f0 2024-09-21T03:41:52.004 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:52.002+0000 7fb3017fa640 1 -- 172.21.15.89:0/520169010 <== mon.1 v2:172.21.15.114:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 9121+0+0 (secure 0 0 0) 0x7fb3080ca730 con 0x55a9580462f0 2024-09-21T03:41:52.004 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:52.002+0000 7fb326e41740 1 -- 172.21.15.89:0/520169010 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb3080ce8f0 con 0x55a9580462f0 2024-09-21T03:41:52.008 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:52.006+0000 7fb3017fa640 1 -- 172.21.15.89:0/520169010 <== mon.1 v2:172.21.15.114:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fb3080ca930 con 0x55a9580462f0 2024-09-21T03:41:52.014 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:52.012+0000 7fb326e41740 10 client.25387 fetch_fsmap learned FSMap version 26 2024-09-21T03:41:52.014 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:52.012+0000 7fb326e41740 1 -- 172.21.15.89:0/520169010 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55a958427cf0 con 0x55a9580462f0 2024-09-21T03:41:52.017 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:52.015+0000 7fb3017fa640 1 -- 172.21.15.89:0/520169010 <== mon.1 v2:172.21.15.114:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb3080ce8f0 con 0x55a9580462f0 2024-09-21T03:41:52.018 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:52.015+0000 7fb3017fa640 1 -- 172.21.15.89:0/520169010 <== mon.1 v2:172.21.15.114:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fb3080d4a50 con 0x55a9580462f0 2024-09-21T03:41:52.018 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:52.015+0000 7fb326e41740 10 client.25387 fetch_fsmap finished waiting for FSMap version 26 2024-09-21T03:41:52.018 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:52.015+0000 7fb326e41740 10 client.25387 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T03:41:52.018 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:52.015+0000 7fb326e41740 20 client.25387 populate_metadata read hostname 'smithi089' 2024-09-21T03:41:52.018 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:52.015+0000 7fb326e41740 1 --2- 172.21.15.89:0/520169010 >> [v2:172.21.15.110:6838/132366709,v1:172.21.15.110:6839/132366709] conn(0x55a9580031c0 0x55a958428650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:52.018 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:52.015+0000 7fb326e41740 4 client.25387 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T03:41:52.018 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:52.015+0000 7fb326e41740 1 -- 172.21.15.89:0/520169010 --> [v2:172.21.15.110:6838/132366709,v1:172.21.15.110:6839/132366709] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55a9583870f0 con 0x55a9580031c0 2024-09-21T03:41:52.022 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:52.020+0000 7fb31d38c640 1 --2- 172.21.15.89:0/520169010 >> [v2:172.21.15.110:6838/132366709,v1:172.21.15.110:6839/132366709] conn(0x55a9580031c0 0x55a958428650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:52.023 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:52.021+0000 7fb31d38c640 1 --2- 172.21.15.89:0/520169010 >> [v2:172.21.15.110:6838/132366709,v1:172.21.15.110:6839/132366709] conn(0x55a9580031c0 0x55a958428650 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=98fb6cc2e157a1d5 server_cookie=54a70d29f407a992 in_seq=0 out_seq=0 2024-09-21T03:41:52.024 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:52.022+0000 7fb3017fa640 10 client.25387 ms_handle_connect on v2:172.21.15.110:6838/132366709 2024-09-21T03:41:52.069 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:52.067+0000 7fb3017fa640 1 -- 172.21.15.89:0/520169010 <== mds.0 v2:172.21.15.110:6838/132366709 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55a9583870f0 con 0x55a9580031c0 2024-09-21T03:41:52.069 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:52.067+0000 7fb3017fa640 10 client.25387 handle_command_reply: tid=0 2024-09-21T03:41:52.069 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:52.067+0000 7fb326e41740 1 client.25387 shutdown 2024-09-21T03:41:52.071 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:52.069+0000 7fb326e41740 1 -- 172.21.15.89:0/520169010 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a9580462f0 msgr2=0x55a9584251d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:52.071 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:52.069+0000 7fb326e41740 1 --2- 172.21.15.89:0/520169010 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a9580462f0 0x55a9584251d0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fb3080cadf0 tx=0x7fb3080cb280 comp rx=0 tx=0).stop 2024-09-21T03:41:52.072 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:52.070+0000 7fb326e41740 1 -- 172.21.15.89:0/520169010 shutdown_connections 2024-09-21T03:41:52.072 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:52.070+0000 7fb326e41740 1 --2- 172.21.15.89:0/520169010 >> [v2:172.21.15.110:6838/132366709,v1:172.21.15.110:6839/132366709] conn(0x55a9580031c0 0x55a958428650 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:52.072 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:52.070+0000 7fb326e41740 1 --2- 172.21.15.89:0/520169010 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a95804e4d0 0x55a958425890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:52.072 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:52.070+0000 7fb326e41740 1 --2- 172.21.15.89:0/520169010 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a95806f1b0 0x55a9584263b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:52.072 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:52.070+0000 7fb326e41740 1 --2- 172.21.15.89:0/520169010 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a9580462f0 0x55a9584251d0 secure :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fb3080cadf0 tx=0x7fb3080cb280 comp rx=0 tx=0).stop 2024-09-21T03:41:52.072 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:52.070+0000 7fb326e41740 1 -- 172.21.15.89:0/520169010 >> 172.21.15.89:0/520169010 conn(0x55a958063300 msgr2=0x55a95841fa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:41:52.073 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:52.070+0000 7fb3017fa640 0 client.25387 ms_handle_reset on v2:172.21.15.110:6838/132366709 2024-09-21T03:41:52.074 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:52.072+0000 7fb326e41740 1 -- 172.21.15.89:0/520169010 shutdown_connections 2024-09-21T03:41:52.075 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:52.072+0000 7fb326e41740 1 -- 172.21.15.89:0/520169010 wait complete. 2024-09-21T03:41:52.075 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:52.072+0000 7fb326e41740 20 client.25387 trim_cache size 0 max 16384 2024-09-21T03:41:52.075 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stdout:{ 2024-09-21T03:41:52.075 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stdout: "message": "", 2024-09-21T03:41:52.075 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stdout: "return_code": 0 2024-09-21T03:41:52.075 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stdout:} 2024-09-21T03:41:52.076 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:52.073+0000 7fb31eb8f640 1 -- 172.21.15.89:0/2800856099 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7fb3040b6720 msgr2=0x7fb3040b8c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:52.076 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:52.073+0000 7fb31eb8f640 1 --2- 172.21.15.89:0/2800856099 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7fb3040b6720 0x7fb3040b8c30 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fb3100099a0 tx=0x7fb310030740 comp rx=0 tx=0).stop 2024-09-21T03:41:52.076 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:52.073+0000 7fb31eb8f640 1 -- 172.21.15.89:0/2800856099 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb31810a3c0 msgr2=0x7fb3181a8a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:52.077 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:52.073+0000 7fb31eb8f640 1 --2- 172.21.15.89:0/2800856099 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb31810a3c0 0x7fb3181a8a40 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7fb30800bea0 tx=0x7fb30800bf90 comp rx=0 tx=0).stop 2024-09-21T03:41:52.077 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:52.073+0000 7fb31eb8f640 1 -- 172.21.15.89:0/2800856099 shutdown_connections 2024-09-21T03:41:52.077 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:52.073+0000 7fb31eb8f640 1 --2- 172.21.15.89:0/2800856099 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7fb3040b6720 0x7fb3040b8c30 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:52.077 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:52.073+0000 7fb31eb8f640 1 --2- 172.21.15.89:0/2800856099 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb31810f830 0x7fb3181a9c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:52.077 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:52.074+0000 7fb31eb8f640 1 --2- 172.21.15.89:0/2800856099 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb31810cd20 0x7fb3181a90b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:52.077 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:52.074+0000 7fb31eb8f640 1 --2- 172.21.15.89:0/2800856099 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb31810a3c0 0x7fb3181a8a40 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:52.077 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:52.074+0000 7fb31eb8f640 1 -- 172.21.15.89:0/2800856099 >> 172.21.15.89:0/2800856099 conn(0x7fb318070640 msgr2=0x7fb31810b290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:41:52.077 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:52.075+0000 7fb31eb8f640 1 -- 172.21.15.89:0/2800856099 shutdown_connections 2024-09-21T03:41:52.077 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:41:52.075+0000 7fb31eb8f640 1 -- 172.21.15.89:0/2800856099 wait complete. 2024-09-21T03:41:53.210 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:41:52 smithi114 ceph-mon[34140]: pgmap v434: 129 pgs: 129 active+clean; 258 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 9.3 MiB/s wr, 2.05k op/s 2024-09-21T03:41:53.250 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:41:52 smithi110 ceph-mon[34000]: pgmap v434: 129 pgs: 129 active+clean; 258 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 9.3 MiB/s wr, 2.05k op/s 2024-09-21T03:41:53.311 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:41:52 smithi089 ceph-mon[30928]: pgmap v434: 129 pgs: 129 active+clean; 258 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 9.3 MiB/s wr, 2.05k op/s 2024-09-21T03:41:54.995 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24638 quiesce db / --timeout 90 --expiration 147.8 --await-for 45.0' 2024-09-21T03:41:54.996 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 7c374098-77c9-11ef-baf5-efdc52797490 -- timeout 105 ceph tell mds.24638 quiesce db / --timeout 90 --expiration 147.8 --await-for 45.0 2024-09-21T03:41:55.211 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:41:54 smithi114 ceph-mon[34140]: pgmap v435: 129 pgs: 129 active+clean; 258 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 12 MiB/s wr, 2.79k op/s 2024-09-21T03:41:55.250 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:41:54 smithi110 ceph-mon[34000]: pgmap v435: 129 pgs: 129 active+clean; 258 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 12 MiB/s wr, 2.79k op/s 2024-09-21T03:41:55.311 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:41:54 smithi089 ceph-mon[30928]: pgmap v435: 129 pgs: 129 active+clean; 258 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 12 MiB/s wr, 2.79k op/s 2024-09-21T03:41:55.564 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/7c374098-77c9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:41:57.211 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:41:56 smithi114 ceph-mon[34140]: pgmap v436: 129 pgs: 129 active+clean; 258 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 11 MiB/s wr, 2.51k op/s 2024-09-21T03:41:57.250 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:41:56 smithi110 ceph-mon[34000]: pgmap v436: 129 pgs: 129 active+clean; 258 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 11 MiB/s wr, 2.51k op/s 2024-09-21T03:41:57.311 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:41:56 smithi089 ceph-mon[30928]: pgmap v436: 129 pgs: 129 active+clean; 258 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 11 MiB/s wr, 2.51k op/s 2024-09-21T03:41:59.211 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:41:58 smithi114 ceph-mon[34140]: pgmap v437: 129 pgs: 129 active+clean; 258 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 11 MiB/s wr, 2.52k op/s 2024-09-21T03:41:59.250 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:41:58 smithi110 ceph-mon[34000]: pgmap v437: 129 pgs: 129 active+clean; 258 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 11 MiB/s wr, 2.52k op/s 2024-09-21T03:41:59.311 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:41:58 smithi089 ceph-mon[30928]: pgmap v437: 129 pgs: 129 active+clean; 258 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 11 MiB/s wr, 2.52k op/s 2024-09-21T03:42:00.800 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:59.702+0000 7f6911325640 1 -- 172.21.15.89:0/1428874340 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f690c111d70 msgr2=0x7f690c10a3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:00.800 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:59.702+0000 7f6911325640 1 --2- 172.21.15.89:0/1428874340 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f690c111d70 0x7f690c10a3c0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f6904009be0 tx=0x7f690402a710 comp rx=0 tx=0).stop 2024-09-21T03:42:00.800 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:59.702+0000 7f6911325640 1 -- 172.21.15.89:0/1428874340 shutdown_connections 2024-09-21T03:42:00.801 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:59.702+0000 7f6911325640 1 --2- 172.21.15.89:0/1428874340 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f690c11cba0 0x7f690c11efe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:00.801 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:59.702+0000 7f6911325640 1 --2- 172.21.15.89:0/1428874340 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f690c10a900 0x7f690c10ad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:00.801 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:59.702+0000 7f6911325640 1 --2- 172.21.15.89:0/1428874340 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f690c111d70 0x7f690c10a3c0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:00.801 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:59.703+0000 7f6911325640 1 -- 172.21.15.89:0/1428874340 >> 172.21.15.89:0/1428874340 conn(0x7f690c06ea10 msgr2=0x7f690c06ee50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:42:00.801 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:59.703+0000 7f6911325640 1 -- 172.21.15.89:0/1428874340 shutdown_connections 2024-09-21T03:42:00.801 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:59.704+0000 7f6911325640 1 -- 172.21.15.89:0/1428874340 wait complete. 2024-09-21T03:42:00.801 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:59.706+0000 7f6911325640 1 Processor -- start 2024-09-21T03:42:00.801 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:59.706+0000 7f6911325640 1 -- start start 2024-09-21T03:42:00.801 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:59.707+0000 7f6911325640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f690c10a900 0x7f690c1a90b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:00.801 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:59.707+0000 7f6911325640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f690c111d70 0x7f690c1a9650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:00.801 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:59.707+0000 7f6911325640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f690c11cba0 0x7f690c1ae8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:00.801 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:59.707+0000 7f6911325640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f690c122660 con 0x7f690c111d70 2024-09-21T03:42:00.801 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:59.707+0000 7f6911325640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f690c1224e0 con 0x7f690c11cba0 2024-09-21T03:42:00.802 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:59.707+0000 7f6911325640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f690c1227e0 con 0x7f690c10a900 2024-09-21T03:42:00.802 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:59.709+0000 7f6910b24640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f690c11cba0 0x7f690c1ae8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:00.802 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:59.709+0000 7f690bfff640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f690c10a900 0x7f690c1a90b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:00.802 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:59.710+0000 7f690bfff640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f690c10a900 0x7f690c1a90b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.89:38548/0 (socket says 172.21.15.89:38548) 2024-09-21T03:42:00.802 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:59.710+0000 7f690bfff640 1 -- 172.21.15.89:0/1685988324 learned_addr learned my addr 172.21.15.89:0/1685988324 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-21T03:42:00.802 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:59.710+0000 7f690b7fe640 1 --2- 172.21.15.89:0/1685988324 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f690c111d70 0x7f690c1a9650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:00.802 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:59.710+0000 7f690bfff640 1 -- 172.21.15.89:0/1685988324 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f690c11cba0 msgr2=0x7f690c1ae8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:00.802 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:59.710+0000 7f690bfff640 1 --2- 172.21.15.89:0/1685988324 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f690c11cba0 0x7f690c1ae8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:00.802 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:59.710+0000 7f690bfff640 1 -- 172.21.15.89:0/1685988324 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f690c111d70 msgr2=0x7f690c1a9650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:00.802 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:59.710+0000 7f690bfff640 1 --2- 172.21.15.89:0/1685988324 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f690c111d70 0x7f690c1a9650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:00.802 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:59.710+0000 7f690bfff640 1 -- 172.21.15.89:0/1685988324 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f690c1af020 con 0x7f690c10a900 2024-09-21T03:42:00.802 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:59.711+0000 7f690bfff640 1 --2- 172.21.15.89:0/1685988324 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f690c10a900 0x7f690c1a90b0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f6904005c40 tx=0x7f69040316f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:00.802 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:59.711+0000 7f69097fa640 1 -- 172.21.15.89:0/1685988324 <== mon.1 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6904041020 con 0x7f690c10a900 2024-09-21T03:42:00.803 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:59.711+0000 7f69097fa640 1 -- 172.21.15.89:0/1685988324 <== mon.1 v2:172.21.15.114:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f69040318e0 con 0x7f690c10a900 2024-09-21T03:42:00.803 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:59.711+0000 7f69097fa640 1 -- 172.21.15.89:0/1685988324 <== mon.1 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6904036040 con 0x7f690c10a900 2024-09-21T03:42:00.803 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:59.712+0000 7f6911325640 1 -- 172.21.15.89:0/1685988324 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f690c1af2b0 con 0x7f690c10a900 2024-09-21T03:42:00.803 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:59.713+0000 7f6911325640 1 -- 172.21.15.89:0/1685988324 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f690c1af680 con 0x7f690c10a900 2024-09-21T03:42:00.803 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:59.718+0000 7f6918afb740 1 Processor -- start 2024-09-21T03:42:00.803 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:59.720+0000 7f6918afb740 1 -- start start 2024-09-21T03:42:00.803 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:59.721+0000 7f6918afb740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fbfd61e840 0x55fbfd9c57c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:00.803 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:59.721+0000 7f6918afb740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55fbfd5f5980 0x55fbfd9c5d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:00.803 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:59.721+0000 7f6918afb740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55fbfd5fdb60 0x55fbfd9c6240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:00.803 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:59.721+0000 7f6918afb740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55fbfd93bb90 con 0x55fbfd5f5980 2024-09-21T03:42:00.803 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:59.721+0000 7f6918afb740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55fbfd92afc0 con 0x55fbfd5fdb60 2024-09-21T03:42:00.803 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:59.721+0000 7f6918afb740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55fbfd92edd0 con 0x55fbfd61e840 2024-09-21T03:42:00.803 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:59.722+0000 7f690b7fe640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55fbfd5fdb60 0x55fbfd9c6240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:00.803 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:59.722+0000 7f690b7fe640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55fbfd5fdb60 0x55fbfd9c6240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.89:47948/0 (socket says 172.21.15.89:47948) 2024-09-21T03:42:00.803 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:59.722+0000 7f690b7fe640 1 -- 172.21.15.89:0/695075650 learned_addr learned my addr 172.21.15.89:0/695075650 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-21T03:42:00.804 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:59.722+0000 7f69097fa640 1 -- 172.21.15.89:0/1685988324 <== mon.1 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 21) ==== 154512+0+0 (secure 0 0 0) 0x7f6904044020 con 0x7f690c10a900 2024-09-21T03:42:00.804 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:59.722+0000 7f690bfff640 1 --2- 172.21.15.89:0/695075650 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55fbfd5f5980 0x55fbfd9c5d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:00.804 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:59.723+0000 7f690bfff640 1 -- 172.21.15.89:0/695075650 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fbfd61e840 msgr2=0x55fbfd9c57c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:42:00.804 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:59.723+0000 7f690bfff640 1 --2- 172.21.15.89:0/695075650 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fbfd61e840 0x55fbfd9c57c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:00.804 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:59.723+0000 7f690bfff640 1 -- 172.21.15.89:0/695075650 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55fbfd5fdb60 msgr2=0x55fbfd9c6240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:42:00.804 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:59.723+0000 7f690bfff640 1 --2- 172.21.15.89:0/695075650 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55fbfd5fdb60 0x55fbfd9c6240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:00.804 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:59.723+0000 7f690bfff640 1 -- 172.21.15.89:0/695075650 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fbfd8b2060 con 0x55fbfd5f5980 2024-09-21T03:42:00.804 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:59.724+0000 7f69097fa640 1 --2- 172.21.15.89:0/1685988324 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f68f40b6640 0x7f68f40b8b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:00.804 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:59.724+0000 7f690bfff640 1 --2- 172.21.15.89:0/695075650 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55fbfd5f5980 0x55fbfd9c5d00 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f6904105ce0 tx=0x7f690410e710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91281180768c3115 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:00.804 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:59.724+0000 7f69097fa640 1 -- 172.21.15.89:0/1685988324 <== mon.1 v2:172.21.15.114:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 9121+0+0 (secure 0 0 0) 0x7f69041085f0 con 0x7f690c10a900 2024-09-21T03:42:00.804 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:59.724+0000 7f68f27fc640 1 -- 172.21.15.89:0/695075650 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69040ca9f0 con 0x55fbfd5f5980 2024-09-21T03:42:00.805 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:59.725+0000 7f68f27fc640 1 -- 172.21.15.89:0/695075650 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f69040cdeb0 con 0x55fbfd5f5980 2024-09-21T03:42:00.805 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:59.725+0000 7f68f27fc640 1 -- 172.21.15.89:0/695075650 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69040cdb10 con 0x55fbfd5f5980 2024-09-21T03:42:00.805 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:59.725+0000 7f6910b24640 1 --2- 172.21.15.89:0/1685988324 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f68f40b6640 0x7f68f40b8b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:00.805 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:59.726+0000 7f6918afb740 1 -- 172.21.15.89:0/695075650 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55fbfd5f5980 msgr2=0x55fbfd9c5d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:00.805 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:59.726+0000 7f6918afb740 1 --2- 172.21.15.89:0/695075650 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55fbfd5f5980 0x55fbfd9c5d00 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f6904105ce0 tx=0x7f690410e710 comp rx=0 tx=0).stop 2024-09-21T03:42:00.805 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:59.726+0000 7f6918afb740 1 -- 172.21.15.89:0/695075650 shutdown_connections 2024-09-21T03:42:00.805 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:59.726+0000 7f6918afb740 1 --2- 172.21.15.89:0/695075650 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55fbfd5fdb60 0x55fbfd9c6240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:00.805 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:59.726+0000 7f6918afb740 1 --2- 172.21.15.89:0/695075650 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55fbfd5f5980 0x55fbfd9c5d00 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:00.805 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:59.726+0000 7f6918afb740 1 --2- 172.21.15.89:0/695075650 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fbfd61e840 0x55fbfd9c57c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:00.805 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:59.726+0000 7f6918afb740 1 -- 172.21.15.89:0/695075650 >> 172.21.15.89:0/695075650 conn(0x55fbfd612990 msgr2=0x55fbfd9c36b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:42:00.805 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:59.727+0000 7f6918afb740 1 -- 172.21.15.89:0/695075650 shutdown_connections 2024-09-21T03:42:00.805 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:59.727+0000 7f6910b24640 1 --2- 172.21.15.89:0/1685988324 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f68f40b6640 0x7f68f40b8b50 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f690000b590 tx=0x7f690000b520 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:00.805 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:59.727+0000 7f6918afb740 1 -- 172.21.15.89:0/695075650 wait complete. 2024-09-21T03:42:00.806 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:59.728+0000 7f6918afb740 1 -- start start 2024-09-21T03:42:00.806 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:59.728+0000 7f6918afb740 1 Processor -- start 2024-09-21T03:42:00.806 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:59.729+0000 7f6918afb740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55fbfd5f5980 0x55fbfd9d1360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:00.806 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:59.729+0000 7f6918afb740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fbfd5fdb60 0x55fbfd9d18a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:00.806 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:59.729+0000 7f6918afb740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55fbfd61e840 0x55fbfd9ede50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:00.806 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:59.729+0000 7f6918afb740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55fbfd78c7a0 con 0x55fbfd5f5980 2024-09-21T03:42:00.806 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:59.729+0000 7f6918afb740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55fbfd8d00a0 con 0x55fbfd61e840 2024-09-21T03:42:00.806 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:59.729+0000 7f6918afb740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55fbfd93bb90 con 0x55fbfd5fdb60 2024-09-21T03:42:00.806 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:59.729+0000 7f6918afb740 10 client.0 _ll_register_callbacks cb 0x55fbfd8ad3c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:42:00.806 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:59.730+0000 7f690b7fe640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fbfd5fdb60 0x55fbfd9d18a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:00.806 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:59.730+0000 7f6910b24640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55fbfd61e840 0x55fbfd9ede50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:00.806 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:59.730+0000 7f690bfff640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55fbfd5f5980 0x55fbfd9d1360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:00.806 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:59.730+0000 7f690bfff640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55fbfd5f5980 0x55fbfd9d1360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:48618/0 (socket says 172.21.15.89:48618) 2024-09-21T03:42:00.807 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:59.730+0000 7f690bfff640 1 -- 172.21.15.89:0/4106244134 learned_addr learned my addr 172.21.15.89:0/4106244134 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-21T03:42:00.807 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:59.731+0000 7f690bfff640 1 -- 172.21.15.89:0/4106244134 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fbfd5fdb60 msgr2=0x55fbfd9d18a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:42:00.807 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:59.731+0000 7f690bfff640 1 --2- 172.21.15.89:0/4106244134 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fbfd5fdb60 0x55fbfd9d18a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:00.807 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:59.731+0000 7f690bfff640 1 -- 172.21.15.89:0/4106244134 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55fbfd61e840 msgr2=0x55fbfd9ede50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:42:00.807 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:59.731+0000 7f690bfff640 1 --2- 172.21.15.89:0/4106244134 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55fbfd61e840 0x55fbfd9ede50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:00.807 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:59.731+0000 7f690bfff640 1 -- 172.21.15.89:0/4106244134 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fbfd8be880 con 0x55fbfd5f5980 2024-09-21T03:42:00.807 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:59.731+0000 7f690bfff640 1 --2- 172.21.15.89:0/4106244134 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55fbfd5f5980 0x55fbfd9d1360 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f69040ce330 tx=0x7f6904009500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f18a0ed5779d7c4 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:00.807 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:59.731+0000 7f68eb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2024-09-21T03:42:00.807 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:59.732+0000 7f68eb7fe640 1 -- 172.21.15.89:0/4106244134 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69040cdcf0 con 0x55fbfd5f5980 2024-09-21T03:42:00.807 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:59.732+0000 7f6918afb740 1 -- 172.21.15.89:0/4106244134 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f69040cdeb0 con 0x55fbfd5f5980 2024-09-21T03:42:00.807 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:59.732+0000 7f68eb7fe640 1 -- 172.21.15.89:0/4106244134 <== mon.0 v2:172.21.15.89:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 9121+0+0 (secure 0 0 0) 0x7f690403cb90 con 0x55fbfd5f5980 2024-09-21T03:42:00.807 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:59.733+0000 7f68eb7fe640 1 -- 172.21.15.89:0/4106244134 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f69040ce170 con 0x55fbfd5f5980 2024-09-21T03:42:00.807 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:59.734+0000 7f6918afb740 10 client.15819 fetch_fsmap learned FSMap version 26 2024-09-21T03:42:00.807 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:59.734+0000 7f6918afb740 1 -- 172.21.15.89:0/4106244134 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55fbfd698b90 con 0x55fbfd5f5980 2024-09-21T03:42:00.808 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:59.734+0000 7f68eb7fe640 1 -- 172.21.15.89:0/4106244134 <== mon.0 v2:172.21.15.89:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f69040cdeb0 con 0x55fbfd5f5980 2024-09-21T03:42:00.808 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:59.734+0000 7f68eb7fe640 1 -- 172.21.15.89:0/4106244134 <== mon.0 v2:172.21.15.89:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f690404e060 con 0x55fbfd5f5980 2024-09-21T03:42:00.808 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:59.735+0000 7f6918afb740 10 client.15819 fetch_fsmap finished waiting for FSMap version 26 2024-09-21T03:42:00.808 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:59.735+0000 7f6918afb740 10 client.15819 resolve_mds: validated gid 24638 aka daemon mds.b 2024-09-21T03:42:00.808 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:59.735+0000 7f6918afb740 20 client.15819 populate_metadata read hostname 'smithi089' 2024-09-21T03:42:00.808 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:59.735+0000 7f6918afb740 1 --2- 172.21.15.89:0/4106244134 >> [v2:172.21.15.110:6838/132366709,v1:172.21.15.110:6839/132366709] conn(0x55fbfd5dd1c0 0x55fbfd9f3d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:00.808 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:59.735+0000 7f6918afb740 4 client.15819 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:42:00.808 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:59.735+0000 7f6918afb740 1 -- 172.21.15.89:0/4106244134 --> [v2:172.21.15.110:6838/132366709,v1:172.21.15.110:6839/132366709] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fbfd93dae0 con 0x55fbfd5dd1c0 2024-09-21T03:42:00.808 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:59.737+0000 7f690b7fe640 1 --2- 172.21.15.89:0/4106244134 >> [v2:172.21.15.110:6838/132366709,v1:172.21.15.110:6839/132366709] conn(0x55fbfd5dd1c0 0x55fbfd9f3d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:00.808 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:59.740+0000 7f690b7fe640 1 --2- 172.21.15.89:0/4106244134 >> [v2:172.21.15.110:6838/132366709,v1:172.21.15.110:6839/132366709] conn(0x55fbfd5dd1c0 0x55fbfd9f3d00 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=41379aea4e59cf22 server_cookie=68de7d4f2a7159a6 in_seq=0 out_seq=0 2024-09-21T03:42:00.808 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:59.740+0000 7f68eb7fe640 10 client.15819 ms_handle_connect on v2:172.21.15.110:6838/132366709 2024-09-21T03:42:00.808 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:59.742+0000 7f68eb7fe640 1 -- 172.21.15.89:0/4106244134 <== mds.0 v2:172.21.15.110:6838/132366709 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55fbfd93dae0 con 0x55fbfd5dd1c0 2024-09-21T03:42:00.808 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:59.742+0000 7f68eb7fe640 10 client.15819 handle_command_reply: tid=0 2024-09-21T03:42:00.808 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:59.744+0000 7f6918afb740 1 client.15819 shutdown 2024-09-21T03:42:00.809 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:59.744+0000 7f6918afb740 1 -- 172.21.15.89:0/4106244134 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55fbfd5f5980 msgr2=0x55fbfd9d1360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:00.809 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:59.744+0000 7f6918afb740 1 --2- 172.21.15.89:0/4106244134 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55fbfd5f5980 0x55fbfd9d1360 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f69040ce330 tx=0x7f6904009500 comp rx=0 tx=0).stop 2024-09-21T03:42:00.809 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:59.746+0000 7f6918afb740 1 -- 172.21.15.89:0/4106244134 shutdown_connections 2024-09-21T03:42:00.809 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:59.746+0000 7f6918afb740 1 --2- 172.21.15.89:0/4106244134 >> [v2:172.21.15.110:6838/132366709,v1:172.21.15.110:6839/132366709] conn(0x55fbfd5dd1c0 0x55fbfd9f3d00 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:00.809 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:59.746+0000 7f6918afb740 1 --2- 172.21.15.89:0/4106244134 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55fbfd61e840 0x55fbfd9ede50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:00.809 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:59.746+0000 7f6918afb740 1 --2- 172.21.15.89:0/4106244134 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fbfd5fdb60 0x55fbfd9d18a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:00.809 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:59.746+0000 7f6918afb740 1 --2- 172.21.15.89:0/4106244134 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55fbfd5f5980 0x55fbfd9d1360 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:00.809 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:59.746+0000 7f6918afb740 1 -- 172.21.15.89:0/4106244134 >> 172.21.15.89:0/4106244134 conn(0x55fbfd612990 msgr2=0x55fbfd9c67f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:42:00.809 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:59.746+0000 7f68eb7fe640 0 client.15819 ms_handle_reset on v2:172.21.15.110:6838/132366709 2024-09-21T03:42:00.809 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:59.746+0000 7f6918afb740 1 -- 172.21.15.89:0/4106244134 shutdown_connections 2024-09-21T03:42:00.809 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:59.747+0000 7f6918afb740 1 -- 172.21.15.89:0/4106244134 wait complete. 2024-09-21T03:42:00.809 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:59.747+0000 7f6918afb740 20 client.15819 trim_cache size 0 max 16384 2024-09-21T03:42:00.809 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:59.781+0000 7f6918afb740 1 Processor -- start 2024-09-21T03:42:00.809 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:59.782+0000 7f6918afb740 1 -- start start 2024-09-21T03:42:00.809 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:59.782+0000 7f6918afb740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fbfd5f5980 0x55fbfd9c26f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:00.810 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:59.782+0000 7f6918afb740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55fbfd5fdb60 0x55fbfd9eddd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:00.810 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:59.782+0000 7f6918afb740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55fbfd61e840 0x55fbfd9ee310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:00.810 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:59.783+0000 7f6918afb740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f690c1b9050 con 0x55fbfd61e840 2024-09-21T03:42:00.810 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:59.783+0000 7f6918afb740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f690c1b9720 con 0x55fbfd5fdb60 2024-09-21T03:42:00.810 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:59.783+0000 7f6918afb740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f690c1b6ff0 con 0x55fbfd5f5980 2024-09-21T03:42:00.810 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:59.783+0000 7f6910b24640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55fbfd61e840 0x55fbfd9ee310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:00.810 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:59.783+0000 7f6910b24640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55fbfd61e840 0x55fbfd9ee310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:48630/0 (socket says 172.21.15.89:48630) 2024-09-21T03:42:00.810 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:59.783+0000 7f6910b24640 1 -- 172.21.15.89:0/3373431919 learned_addr learned my addr 172.21.15.89:0/3373431919 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-21T03:42:00.810 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:59.784+0000 7f6910b24640 1 -- 172.21.15.89:0/3373431919 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fbfd5f5980 msgr2=0x55fbfd9c26f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T03:42:00.810 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:59.784+0000 7f6910b24640 1 --2- 172.21.15.89:0/3373431919 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fbfd5f5980 0x55fbfd9c26f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:00.810 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:59.784+0000 7f6910b24640 1 -- 172.21.15.89:0/3373431919 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55fbfd5fdb60 msgr2=0x55fbfd9eddd0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T03:42:00.810 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:59.784+0000 7f6910b24640 1 --2- 172.21.15.89:0/3373431919 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55fbfd5fdb60 0x55fbfd9eddd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:00.810 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:59.784+0000 7f6910b24640 1 -- 172.21.15.89:0/3373431919 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fbfd9d20b0 con 0x55fbfd61e840 2024-09-21T03:42:00.811 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:59.784+0000 7f6910b24640 1 --2- 172.21.15.89:0/3373431919 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55fbfd61e840 0x55fbfd9ee310 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f6900003c10 tx=0x7f6900013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fbcbde6a9bc2e059 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:00.811 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:59.789+0000 7f68f1ffb640 1 -- 172.21.15.89:0/3373431919 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6900011da0 con 0x55fbfd61e840 2024-09-21T03:42:00.811 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:59.789+0000 7f68f1ffb640 1 -- 172.21.15.89:0/3373431919 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6900022070 con 0x55fbfd61e840 2024-09-21T03:42:00.811 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:59.790+0000 7f6918afb740 1 -- 172.21.15.89:0/3373431919 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55fbfd61e840 msgr2=0x55fbfd9ee310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:00.811 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:59.790+0000 7f6918afb740 1 --2- 172.21.15.89:0/3373431919 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55fbfd61e840 0x55fbfd9ee310 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f6900003c10 tx=0x7f6900013000 comp rx=0 tx=0).stop 2024-09-21T03:42:00.811 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:59.796+0000 7f6918afb740 1 -- 172.21.15.89:0/3373431919 shutdown_connections 2024-09-21T03:42:00.811 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:59.796+0000 7f6918afb740 1 --2- 172.21.15.89:0/3373431919 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55fbfd5fdb60 0x55fbfd9eddd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:00.811 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:59.796+0000 7f6918afb740 1 --2- 172.21.15.89:0/3373431919 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55fbfd61e840 0x55fbfd9ee310 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:00.811 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:59.796+0000 7f6918afb740 1 --2- 172.21.15.89:0/3373431919 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fbfd5f5980 0x55fbfd9c26f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:00.811 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:59.796+0000 7f6918afb740 1 -- 172.21.15.89:0/3373431919 >> 172.21.15.89:0/3373431919 conn(0x55fbfd612990 msgr2=0x55fbfd9c35f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:42:00.811 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:59.802+0000 7f6918afb740 1 -- 172.21.15.89:0/3373431919 shutdown_connections 2024-09-21T03:42:00.811 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:59.803+0000 7f6918afb740 1 -- 172.21.15.89:0/3373431919 wait complete. 2024-09-21T03:42:00.811 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:59.803+0000 7f6918afb740 1 -- start start 2024-09-21T03:42:00.811 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:59.804+0000 7f6918afb740 1 Processor -- start 2024-09-21T03:42:00.812 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:59.804+0000 7f6918afb740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55fbfd5f5980 0x55fbfd9e5220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:00.812 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:59.805+0000 7f6918afb740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fbfd5fdb60 0x55fbfd9e5760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:00.812 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:59.805+0000 7f6918afb740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55fbfd61e840 0x55fbfda5e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:00.812 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:59.805+0000 7f6918afb740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f690c11a530 con 0x55fbfd5f5980 2024-09-21T03:42:00.812 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:59.805+0000 7f6918afb740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f690c1b6920 con 0x55fbfd61e840 2024-09-21T03:42:00.812 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:59.805+0000 7f6918afb740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f690c1b5b80 con 0x55fbfd5fdb60 2024-09-21T03:42:00.812 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:59.805+0000 7f6918afb740 10 client.0 _ll_register_callbacks cb 0x7f68fc00d3e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:42:00.812 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:59.806+0000 7f690bfff640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55fbfd5f5980 0x55fbfd9e5220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:00.812 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:59.806+0000 7f690b7fe640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fbfd5fdb60 0x55fbfd9e5760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:00.812 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:59.806+0000 7f690bfff640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55fbfd5f5980 0x55fbfd9e5220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:48640/0 (socket says 172.21.15.89:48640) 2024-09-21T03:42:00.812 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:59.806+0000 7f690bfff640 1 -- 172.21.15.89:0/2045679317 learned_addr learned my addr 172.21.15.89:0/2045679317 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-21T03:42:00.813 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:59.806+0000 7f6910b24640 1 --2- 172.21.15.89:0/2045679317 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55fbfd61e840 0x55fbfda5e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:00.813 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:59.806+0000 7f690bfff640 1 -- 172.21.15.89:0/2045679317 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fbfd5fdb60 msgr2=0x55fbfd9e5760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:42:00.813 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:59.806+0000 7f690bfff640 1 --2- 172.21.15.89:0/2045679317 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fbfd5fdb60 0x55fbfd9e5760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:00.813 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:59.806+0000 7f690bfff640 1 -- 172.21.15.89:0/2045679317 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55fbfd61e840 msgr2=0x55fbfda5e5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:42:00.813 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:59.806+0000 7f690bfff640 1 --2- 172.21.15.89:0/2045679317 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55fbfd61e840 0x55fbfda5e5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:00.813 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:59.806+0000 7f690bfff640 1 -- 172.21.15.89:0/2045679317 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fbfd698b90 con 0x55fbfd5f5980 2024-09-21T03:42:00.813 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:59.807+0000 7f690bfff640 1 --2- 172.21.15.89:0/2045679317 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55fbfd5f5980 0x55fbfd9e5220 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f69040cb610 tx=0x7f69040cd900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e510163672080c5a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:00.813 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:59.807+0000 7f68eb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2024-09-21T03:42:00.813 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:59.807+0000 7f68eb7fe640 1 -- 172.21.15.89:0/2045679317 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69040ca240 con 0x55fbfd5f5980 2024-09-21T03:42:00.813 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:59.808+0000 7f6918afb740 1 -- 172.21.15.89:0/2045679317 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6900022070 con 0x55fbfd5f5980 2024-09-21T03:42:00.813 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:59.808+0000 7f68eb7fe640 1 -- 172.21.15.89:0/2045679317 <== mon.0 v2:172.21.15.89:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 9121+0+0 (secure 0 0 0) 0x7f6904004d50 con 0x55fbfd5f5980 2024-09-21T03:42:00.813 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:59.808+0000 7f68eb7fe640 1 -- 172.21.15.89:0/2045679317 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f69040cdcf0 con 0x55fbfd5f5980 2024-09-21T03:42:00.814 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:59.808+0000 7f6918afb740 10 client.15825 fetch_fsmap learned FSMap version 26 2024-09-21T03:42:00.814 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:59.808+0000 7f6918afb740 1 -- 172.21.15.89:0/2045679317 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55fbfd698b90 con 0x55fbfd5f5980 2024-09-21T03:42:00.814 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:59.809+0000 7f68eb7fe640 1 -- 172.21.15.89:0/2045679317 <== mon.0 v2:172.21.15.89:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6900022070 con 0x55fbfd5f5980 2024-09-21T03:42:00.814 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:59.809+0000 7f68eb7fe640 1 -- 172.21.15.89:0/2045679317 <== mon.0 v2:172.21.15.89:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f69040c9cf0 con 0x55fbfd5f5980 2024-09-21T03:42:00.814 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:59.809+0000 7f6918afb740 10 client.15825 fetch_fsmap finished waiting for FSMap version 26 2024-09-21T03:42:00.814 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:59.809+0000 7f6918afb740 10 client.15825 resolve_mds: validated gid 24638 aka daemon mds.b 2024-09-21T03:42:00.814 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:59.809+0000 7f6918afb740 20 client.15825 populate_metadata read hostname 'smithi089' 2024-09-21T03:42:00.814 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:59.809+0000 7f6918afb740 1 --2- 172.21.15.89:0/2045679317 >> [v2:172.21.15.110:6838/132366709,v1:172.21.15.110:6839/132366709] conn(0x55fbfd5b2570 0x55fbfda61de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:00.814 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:59.809+0000 7f6918afb740 4 client.15825 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 147.8, "await_for": 45.0}] 2024-09-21T03:42:00.814 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:59.809+0000 7f6918afb740 1 -- 172.21.15.89:0/2045679317 --> [v2:172.21.15.110:6838/132366709,v1:172.21.15.110:6839/132366709] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 147.8, "await_for": 45.0}) -- 0x55fbfd939590 con 0x55fbfd5b2570 2024-09-21T03:42:00.814 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:59.811+0000 7f690b7fe640 1 --2- 172.21.15.89:0/2045679317 >> [v2:172.21.15.110:6838/132366709,v1:172.21.15.110:6839/132366709] conn(0x55fbfd5b2570 0x55fbfda61de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:00.814 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:59.811+0000 7f690b7fe640 1 --2- 172.21.15.89:0/2045679317 >> [v2:172.21.15.110:6838/132366709,v1:172.21.15.110:6839/132366709] conn(0x55fbfd5b2570 0x55fbfda61de0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=14342bcd897f2847 server_cookie=154ad5c67306b65f in_seq=0 out_seq=0 2024-09-21T03:42:00.814 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:59.811+0000 7f68eb7fe640 10 client.15825 ms_handle_connect on v2:172.21.15.110:6838/132366709 2024-09-21T03:42:00.814 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:59.859+0000 7f68eb7fe640 1 -- 172.21.15.89:0/2045679317 <== mds.0 v2:172.21.15.110:6838/132366709 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55fbfd939590 con 0x55fbfd5b2570 2024-09-21T03:42:00.815 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:59.860+0000 7f68eb7fe640 10 client.15825 handle_command_reply: tid=0 2024-09-21T03:42:00.815 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:59.860+0000 7f6918afb740 1 client.15825 shutdown 2024-09-21T03:42:00.815 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:59.860+0000 7f6918afb740 1 -- 172.21.15.89:0/2045679317 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55fbfd5f5980 msgr2=0x55fbfd9e5220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:00.815 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:59.860+0000 7f6918afb740 1 --2- 172.21.15.89:0/2045679317 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55fbfd5f5980 0x55fbfd9e5220 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f69040cb610 tx=0x7f69040cd900 comp rx=0 tx=0).stop 2024-09-21T03:42:00.815 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:59.860+0000 7f6918afb740 1 -- 172.21.15.89:0/2045679317 shutdown_connections 2024-09-21T03:42:00.815 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:59.860+0000 7f6918afb740 1 --2- 172.21.15.89:0/2045679317 >> [v2:172.21.15.110:6838/132366709,v1:172.21.15.110:6839/132366709] conn(0x55fbfd5b2570 0x55fbfda61de0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:00.815 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:59.860+0000 7f6918afb740 1 --2- 172.21.15.89:0/2045679317 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55fbfd61e840 0x55fbfda5e5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:00.815 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:59.860+0000 7f6918afb740 1 --2- 172.21.15.89:0/2045679317 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fbfd5fdb60 0x55fbfd9e5760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:00.815 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:59.860+0000 7f6918afb740 1 --2- 172.21.15.89:0/2045679317 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55fbfd5f5980 0x55fbfd9e5220 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:00.815 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:59.860+0000 7f6918afb740 1 -- 172.21.15.89:0/2045679317 >> 172.21.15.89:0/2045679317 conn(0x55fbfd612990 msgr2=0x55fbfd9c67f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:42:00.815 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:59.860+0000 7f68eb7fe640 0 client.15825 ms_handle_reset on v2:172.21.15.110:6838/132366709 2024-09-21T03:42:00.815 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:59.861+0000 7f6918afb740 1 -- 172.21.15.89:0/2045679317 shutdown_connections 2024-09-21T03:42:00.816 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-09-21T03:42:00.816 INFO:teuthology.orchestra.run.smithi089.stdout: "epoch": 26, 2024-09-21T03:42:00.816 INFO:teuthology.orchestra.run.smithi089.stdout: "leader": 24638, 2024-09-21T03:42:00.816 INFO:teuthology.orchestra.run.smithi089.stdout: "set_version": 10, 2024-09-21T03:42:00.816 INFO:teuthology.orchestra.run.smithi089.stdout: "sets": { 2024-09-21T03:42:00.816 INFO:teuthology.orchestra.run.smithi089.stdout: "6404c199": { 2024-09-21T03:42:00.816 INFO:teuthology.orchestra.run.smithi089.stdout: "version": 10, 2024-09-21T03:42:00.816 INFO:teuthology.orchestra.run.smithi089.stdout: "age_ref": 0.0, 2024-09-21T03:42:00.816 INFO:teuthology.orchestra.run.smithi089.stdout: "state": { 2024-09-21T03:42:00.816 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "QUIESCED", 2024-09-21T03:42:00.817 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 0.0 2024-09-21T03:42:00.817 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:00.817 INFO:teuthology.orchestra.run.smithi089.stdout: "timeout": 90.0, 2024-09-21T03:42:00.817 INFO:teuthology.orchestra.run.smithi089.stdout: "expiration": 147.8, 2024-09-21T03:42:00.817 INFO:teuthology.orchestra.run.smithi089.stdout: "members": { 2024-09-21T03:42:00.817 INFO:teuthology.orchestra.run.smithi089.stdout: "file:/": { 2024-09-21T03:42:00.817 INFO:teuthology.orchestra.run.smithi089.stdout: "excluded": false, 2024-09-21T03:42:00.817 INFO:teuthology.orchestra.run.smithi089.stdout: "state": { 2024-09-21T03:42:00.817 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "QUIESCED", 2024-09-21T03:42:00.817 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 0.0 2024-09-21T03:42:00.817 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-21T03:42:00.817 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-21T03:42:00.818 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-21T03:42:00.818 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-21T03:42:00.818 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-21T03:42:00.818 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-09-21T03:42:00.818 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:59.861+0000 7f6918afb740 1 -- 172.21.15.89:0/2045679317 wait complete. 2024-09-21T03:42:00.818 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:59.861+0000 7f6918afb740 20 client.15825 trim_cache size 0 max 16384 2024-09-21T03:42:00.818 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:59.864+0000 7f6911325640 1 -- 172.21.15.89:0/1685988324 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f68f40b6640 msgr2=0x7f68f40b8b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:00.818 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:59.864+0000 7f6911325640 1 --2- 172.21.15.89:0/1685988324 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f68f40b6640 0x7f68f40b8b50 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f690000b590 tx=0x7f690000b520 comp rx=0 tx=0).stop 2024-09-21T03:42:00.818 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:59.864+0000 7f6911325640 1 -- 172.21.15.89:0/1685988324 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f690c10a900 msgr2=0x7f690c1a90b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:00.819 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:59.864+0000 7f6911325640 1 --2- 172.21.15.89:0/1685988324 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f690c10a900 0x7f690c1a90b0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f6904005c40 tx=0x7f69040316f0 comp rx=0 tx=0).stop 2024-09-21T03:42:00.819 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:59.864+0000 7f6911325640 1 -- 172.21.15.89:0/1685988324 shutdown_connections 2024-09-21T03:42:00.819 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:59.864+0000 7f6911325640 1 --2- 172.21.15.89:0/1685988324 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f68f40b6640 0x7f68f40b8b50 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:00.819 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:59.864+0000 7f6911325640 1 --2- 172.21.15.89:0/1685988324 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f690c11cba0 0x7f690c1ae8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:00.819 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:59.864+0000 7f6911325640 1 --2- 172.21.15.89:0/1685988324 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f690c111d70 0x7f690c1a9650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:00.819 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:59.864+0000 7f6911325640 1 --2- 172.21.15.89:0/1685988324 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f690c10a900 0x7f690c1a90b0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:00.819 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:59.864+0000 7f6911325640 1 -- 172.21.15.89:0/1685988324 >> 172.21.15.89:0/1685988324 conn(0x7f690c06ea10 msgr2=0x7f690c11ab90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:42:00.819 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:59.865+0000 7f6911325640 1 -- 172.21.15.89:0/1685988324 shutdown_connections 2024-09-21T03:42:00.819 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:41:59.865+0000 7f6911325640 1 -- 172.21.15.89:0/1685988324 wait complete. 2024-09-21T03:42:01.061 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:42:00 smithi089 ceph-mon[30928]: pgmap v438: 129 pgs: 129 active+clean; 258 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 14 MiB/s wr, 3.04k op/s 2024-09-21T03:42:01.211 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:42:00 smithi114 ceph-mon[34140]: pgmap v438: 129 pgs: 129 active+clean; 258 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 14 MiB/s wr, 3.04k op/s 2024-09-21T03:42:01.250 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:42:00 smithi110 ceph-mon[34000]: pgmap v438: 129 pgs: 129 active+clean; 258 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 14 MiB/s wr, 3.04k op/s 2024-09-21T03:42:03.211 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:42:02 smithi114 ceph-mon[34140]: pgmap v439: 129 pgs: 129 active+clean; 258 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 10 MiB/s wr, 2.32k op/s 2024-09-21T03:42:03.250 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:42:02 smithi110 ceph-mon[34000]: pgmap v439: 129 pgs: 129 active+clean; 258 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 10 MiB/s wr, 2.32k op/s 2024-09-21T03:42:03.383 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:42:02 smithi089 ceph-mon[30928]: pgmap v439: 129 pgs: 129 active+clean; 258 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 10 MiB/s wr, 2.32k op/s 2024-09-21T03:42:05.196 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:42:05.194+0000 7ff691919640 1 -- 172.21.15.89:0/1085628837 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff68c102c60 msgr2=0x7ff68c1050a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:05.197 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:42:05.194+0000 7ff691919640 1 --2- 172.21.15.89:0/1085628837 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff68c102c60 0x7ff68c1050a0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7ff684009900 tx=0x7ff68402a410 comp rx=0 tx=0).stop 2024-09-21T03:42:05.197 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:42:05.194+0000 7ff691919640 1 -- 172.21.15.89:0/1085628837 shutdown_connections 2024-09-21T03:42:05.197 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:42:05.194+0000 7ff691919640 1 --2- 172.21.15.89:0/1085628837 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff68c108300 0x7ff68c10a810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:05.197 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:42:05.194+0000 7ff691919640 1 --2- 172.21.15.89:0/1085628837 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff68c1056e0 0x7ff68c107bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:05.197 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:42:05.194+0000 7ff691919640 1 --2- 172.21.15.89:0/1085628837 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff68c102c60 0x7ff68c1050a0 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:05.197 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:42:05.194+0000 7ff691919640 1 -- 172.21.15.89:0/1085628837 >> 172.21.15.89:0/1085628837 conn(0x7ff68c0fd690 msgr2=0x7ff68c0ffb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:42:05.197 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:42:05.194+0000 7ff691919640 1 -- 172.21.15.89:0/1085628837 shutdown_connections 2024-09-21T03:42:05.197 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:42:05.194+0000 7ff691919640 1 -- 172.21.15.89:0/1085628837 wait complete. 2024-09-21T03:42:05.197 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:42:05.194+0000 7ff691919640 1 Processor -- start 2024-09-21T03:42:05.199 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:42:05.195+0000 7ff691919640 1 -- start start 2024-09-21T03:42:05.199 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:42:05.195+0000 7ff691919640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff68c102c60 0x7ff68c199e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:05.200 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:42:05.195+0000 7ff691919640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff68c1056e0 0x7ff68c19a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:05.200 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:42:05.195+0000 7ff691919640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff68c108300 0x7ff68c19b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:05.200 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:42:05.195+0000 7ff691919640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7ff68c10e0a0 con 0x7ff68c108300 2024-09-21T03:42:05.200 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:42:05.195+0000 7ff691919640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7ff68c10df20 con 0x7ff68c102c60 2024-09-21T03:42:05.200 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:42:05.195+0000 7ff691919640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7ff68c10e220 con 0x7ff68c1056e0 2024-09-21T03:42:05.200 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:42:05.195+0000 7ff68b7fe640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff68c108300 0x7ff68c19b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:05.200 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:42:05.195+0000 7ff68b7fe640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff68c108300 0x7ff68c19b0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:48656/0 (socket says 172.21.15.89:48656) 2024-09-21T03:42:05.200 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:42:05.195+0000 7ff68b7fe640 1 -- 172.21.15.89:0/3888467809 learned_addr learned my addr 172.21.15.89:0/3888467809 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-21T03:42:05.200 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:42:05.195+0000 7ff68a7fc640 1 --2- 172.21.15.89:0/3888467809 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff68c1056e0 0x7ff68c19a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:05.200 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:42:05.195+0000 7ff68affd640 1 --2- 172.21.15.89:0/3888467809 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff68c102c60 0x7ff68c199e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:05.200 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:42:05.195+0000 7ff68b7fe640 1 -- 172.21.15.89:0/3888467809 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff68c1056e0 msgr2=0x7ff68c19a4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:05.200 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:42:05.195+0000 7ff68b7fe640 1 --2- 172.21.15.89:0/3888467809 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff68c1056e0 0x7ff68c19a4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:05.200 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:42:05.195+0000 7ff68b7fe640 1 -- 172.21.15.89:0/3888467809 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff68c102c60 msgr2=0x7ff68c199e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:05.200 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:42:05.195+0000 7ff68b7fe640 1 --2- 172.21.15.89:0/3888467809 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff68c102c60 0x7ff68c199e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:05.200 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:42:05.195+0000 7ff68b7fe640 1 -- 172.21.15.89:0/3888467809 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff68c19b8c0 con 0x7ff68c108300 2024-09-21T03:42:05.201 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:42:05.196+0000 7ff68b7fe640 1 --2- 172.21.15.89:0/3888467809 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff68c108300 0x7ff68c19b0a0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7ff67c00ead0 tx=0x7ff67c00ef90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:05.201 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:42:05.196+0000 7ff690917640 1 -- 172.21.15.89:0/3888467809 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff67c00cea0 con 0x7ff68c108300 2024-09-21T03:42:05.201 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:42:05.196+0000 7ff691919640 1 -- 172.21.15.89:0/3888467809 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff68c19bbb0 con 0x7ff68c108300 2024-09-21T03:42:05.201 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:42:05.196+0000 7ff690917640 1 -- 172.21.15.89:0/3888467809 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff67c004e40 con 0x7ff68c108300 2024-09-21T03:42:05.201 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:42:05.196+0000 7ff690917640 1 -- 172.21.15.89:0/3888467809 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff67c010690 con 0x7ff68c108300 2024-09-21T03:42:05.201 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:42:05.196+0000 7ff691919640 1 -- 172.21.15.89:0/3888467809 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff68c1a22a0 con 0x7ff68c108300 2024-09-21T03:42:05.206 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:42:05.198+0000 7ff698884740 1 Processor -- start 2024-09-21T03:42:05.206 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:42:05.198+0000 7ff690917640 1 -- 172.21.15.89:0/3888467809 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 21) ==== 154512+0+0 (secure 0 0 0) 0x7ff67c0048d0 con 0x7ff68c108300 2024-09-21T03:42:05.207 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:42:05.198+0000 7ff698884740 1 -- start start 2024-09-21T03:42:05.207 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:42:05.199+0000 7ff698884740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55bb4f7df1b0 0x55bb4fb86e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:05.207 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:42:05.199+0000 7ff698884740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55bb4f7b62f0 0x55bb4fb874c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:05.207 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:42:05.199+0000 7ff698884740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55bb4f7be4d0 0x55bb4fb88010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:05.207 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:42:05.199+0000 7ff698884740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55bb4faf96f0 con 0x55bb4f7b62f0 2024-09-21T03:42:05.207 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:42:05.199+0000 7ff698884740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55bb4fae8b20 con 0x55bb4f7be4d0 2024-09-21T03:42:05.207 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:42:05.199+0000 7ff698884740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55bb4faec930 con 0x55bb4f7df1b0 2024-09-21T03:42:05.207 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:42:05.199+0000 7ff68affd640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55bb4f7b62f0 0x55bb4fb874c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:05.207 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:42:05.199+0000 7ff68affd640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55bb4f7b62f0 0x55bb4fb874c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:48660/0 (socket says 172.21.15.89:48660) 2024-09-21T03:42:05.207 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:42:05.199+0000 7ff68affd640 1 -- 172.21.15.89:0/1972671533 learned_addr learned my addr 172.21.15.89:0/1972671533 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-21T03:42:05.207 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:42:05.199+0000 7ff68a7fc640 1 --2- 172.21.15.89:0/1972671533 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55bb4f7df1b0 0x55bb4fb86e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:05.207 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:42:05.199+0000 7ff68a7fc640 1 --2- 172.21.15.89:0/1972671533 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55bb4f7be4d0 0x55bb4fb88010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:05.207 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:42:05.199+0000 7ff690917640 1 --2- 172.21.15.89:0/3888467809 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7ff6740b6510 0x7ff6740b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:05.207 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:42:05.199+0000 7ff68a7fc640 1 -- 172.21.15.89:0/1972671533 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55bb4f7be4d0 msgr2=0x55bb4fb88010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:42:05.208 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:42:05.199+0000 7ff68a7fc640 1 --2- 172.21.15.89:0/1972671533 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55bb4f7be4d0 0x55bb4fb88010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:05.208 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:42:05.199+0000 7ff68a7fc640 1 -- 172.21.15.89:0/1972671533 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55bb4f7b62f0 msgr2=0x55bb4fb874c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:42:05.208 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:42:05.199+0000 7ff68a7fc640 1 --2- 172.21.15.89:0/1972671533 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55bb4f7b62f0 0x55bb4fb874c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:05.208 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:42:05.199+0000 7ff68a7fc640 1 -- 172.21.15.89:0/1972671533 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bb4faae530 con 0x55bb4f7df1b0 2024-09-21T03:42:05.208 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:42:05.199+0000 7ff68affd640 1 --2- 172.21.15.89:0/1972671533 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55bb4f7b62f0 0x55bb4fb874c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:42:05.208 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:42:05.199+0000 7ff68b7fe640 1 --2- 172.21.15.89:0/3888467809 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7ff6740b6510 0x7ff6740b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:05.208 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:42:05.200+0000 7ff68a7fc640 1 --2- 172.21.15.89:0/1972671533 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55bb4f7df1b0 0x55bb4fb86e50 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7ff680009a30 tx=0x7ff6800099c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b5c5dc564af4bc26 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:05.208 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:42:05.200+0000 7ff690917640 1 -- 172.21.15.89:0/3888467809 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 9121+0+0 (secure 0 0 0) 0x7ff67c0e7c20 con 0x7ff68c108300 2024-09-21T03:42:05.208 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:42:05.200+0000 7ff671ffb640 1 -- 172.21.15.89:0/1972671533 <== mon.1 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff680004d20 con 0x55bb4f7df1b0 2024-09-21T03:42:05.208 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:42:05.200+0000 7ff671ffb640 1 -- 172.21.15.89:0/1972671533 <== mon.1 v2:172.21.15.114:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff680003e00 con 0x55bb4f7df1b0 2024-09-21T03:42:05.208 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:42:05.200+0000 7ff671ffb640 1 -- 172.21.15.89:0/1972671533 <== mon.1 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff680004250 con 0x55bb4f7df1b0 2024-09-21T03:42:05.208 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:42:05.201+0000 7ff698884740 1 -- 172.21.15.89:0/1972671533 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55bb4f7df1b0 msgr2=0x55bb4fb86e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:05.208 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:42:05.201+0000 7ff68b7fe640 1 --2- 172.21.15.89:0/3888467809 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7ff6740b6510 0x7ff6740b8a20 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7ff67c00eaa0 tx=0x7ff67c079040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:05.208 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:42:05.201+0000 7ff698884740 1 --2- 172.21.15.89:0/1972671533 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55bb4f7df1b0 0x55bb4fb86e50 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7ff680009a30 tx=0x7ff6800099c0 comp rx=0 tx=0).stop 2024-09-21T03:42:05.208 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:42:05.201+0000 7ff698884740 1 -- 172.21.15.89:0/1972671533 shutdown_connections 2024-09-21T03:42:05.208 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:42:05.201+0000 7ff698884740 1 --2- 172.21.15.89:0/1972671533 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55bb4f7be4d0 0x55bb4fb88010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:05.209 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:42:05.201+0000 7ff698884740 1 --2- 172.21.15.89:0/1972671533 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55bb4f7b62f0 0x55bb4fb874c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:05.209 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:42:05.201+0000 7ff698884740 1 --2- 172.21.15.89:0/1972671533 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55bb4f7df1b0 0x55bb4fb86e50 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:05.209 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:42:05.201+0000 7ff698884740 1 -- 172.21.15.89:0/1972671533 >> 172.21.15.89:0/1972671533 conn(0x55bb4f7d3300 msgr2=0x55bb4fb828e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:42:05.209 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:42:05.201+0000 7ff698884740 1 -- 172.21.15.89:0/1972671533 shutdown_connections 2024-09-21T03:42:05.209 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:42:05.201+0000 7ff698884740 1 -- 172.21.15.89:0/1972671533 wait complete. 2024-09-21T03:42:05.209 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:42:05.201+0000 7ff698884740 1 -- start start 2024-09-21T03:42:05.209 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:42:05.202+0000 7ff698884740 1 Processor -- start 2024-09-21T03:42:05.209 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:42:05.202+0000 7ff698884740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55bb4f7b62f0 0x55bb4fb90610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:05.209 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:42:05.202+0000 7ff698884740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55bb4f7be4d0 0x55bb4fb90cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:05.209 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:42:05.202+0000 7ff698884740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55bb4f7df1b0 0x55bb4fb91840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:05.209 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:42:05.202+0000 7ff698884740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55bb4fa20ab0 con 0x55bb4f7df1b0 2024-09-21T03:42:05.209 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:42:05.202+0000 7ff698884740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55bb4fa8e730 con 0x55bb4f7b62f0 2024-09-21T03:42:05.209 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:42:05.202+0000 7ff698884740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55bb4fb88b30 con 0x55bb4f7be4d0 2024-09-21T03:42:05.209 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:42:05.202+0000 7ff698884740 10 client.0 _ll_register_callbacks cb 0x55bb4fae26b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:42:05.210 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:42:05.202+0000 7ff68a7fc640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55bb4f7df1b0 0x55bb4fb91840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:05.210 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:42:05.202+0000 7ff68a7fc640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55bb4f7df1b0 0x55bb4fb91840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:48666/0 (socket says 172.21.15.89:48666) 2024-09-21T03:42:05.210 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:42:05.202+0000 7ff68a7fc640 1 -- 172.21.15.89:0/2920013186 learned_addr learned my addr 172.21.15.89:0/2920013186 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-21T03:42:05.210 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:42:05.202+0000 7ff68affd640 1 --2- 172.21.15.89:0/2920013186 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55bb4f7be4d0 0x55bb4fb90cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:05.210 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:42:05.202+0000 7ff68a7fc640 1 --2- 172.21.15.89:0/2920013186 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55bb4f7b62f0 0x55bb4fb90610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:05.210 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:42:05.202+0000 7ff68a7fc640 1 -- 172.21.15.89:0/2920013186 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55bb4f7be4d0 msgr2=0x55bb4fb90cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:42:05.210 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:42:05.202+0000 7ff68a7fc640 1 --2- 172.21.15.89:0/2920013186 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55bb4f7be4d0 0x55bb4fb90cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:05.210 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:42:05.202+0000 7ff68a7fc640 1 -- 172.21.15.89:0/2920013186 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55bb4f7b62f0 msgr2=0x55bb4fb90610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:42:05.210 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:42:05.202+0000 7ff68a7fc640 1 --2- 172.21.15.89:0/2920013186 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55bb4f7b62f0 0x55bb4fb90610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:05.210 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:42:05.202+0000 7ff68a7fc640 1 -- 172.21.15.89:0/2920013186 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bb4f84a790 con 0x55bb4f7df1b0 2024-09-21T03:42:05.210 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:42:05.203+0000 7ff68a7fc640 1 --2- 172.21.15.89:0/2920013186 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55bb4f7df1b0 0x55bb4fb91840 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7ff680009fd0 tx=0x7ff680009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78253bdbc866a514 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:05.210 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:42:05.203+0000 7ff66affd640 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2024-09-21T03:42:05.210 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:42:05.203+0000 7ff66affd640 1 -- 172.21.15.89:0/2920013186 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff680023060 con 0x55bb4f7df1b0 2024-09-21T03:42:05.211 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:42:05.203+0000 7ff66affd640 1 -- 172.21.15.89:0/2920013186 <== mon.0 v2:172.21.15.89:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 9121+0+0 (secure 0 0 0) 0x7ff680003b80 con 0x55bb4f7df1b0 2024-09-21T03:42:05.211 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:42:05.203+0000 7ff698884740 1 -- 172.21.15.89:0/2920013186 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff680003e00 con 0x55bb4f7df1b0 2024-09-21T03:42:05.211 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:42:05.203+0000 7ff66affd640 1 -- 172.21.15.89:0/2920013186 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7ff680004a40 con 0x55bb4f7df1b0 2024-09-21T03:42:05.211 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:42:05.203+0000 7ff698884740 10 client.15837 fetch_fsmap learned FSMap version 26 2024-09-21T03:42:05.211 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:42:05.203+0000 7ff698884740 1 -- 172.21.15.89:0/2920013186 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55bb4f8c3620 con 0x55bb4f7df1b0 2024-09-21T03:42:05.211 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:42:05.203+0000 7ff66affd640 1 -- 172.21.15.89:0/2920013186 <== mon.0 v2:172.21.15.89:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff680003e00 con 0x55bb4f7df1b0 2024-09-21T03:42:05.211 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:42:05.203+0000 7ff66affd640 1 -- 172.21.15.89:0/2920013186 <== mon.0 v2:172.21.15.89:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7ff68001c020 con 0x55bb4f7df1b0 2024-09-21T03:42:05.211 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:42:05.203+0000 7ff698884740 10 client.15837 fetch_fsmap finished waiting for FSMap version 26 2024-09-21T03:42:05.211 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:42:05.203+0000 7ff698884740 10 client.15837 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T03:42:05.211 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:42:05.203+0000 7ff698884740 20 client.15837 populate_metadata read hostname 'smithi089' 2024-09-21T03:42:05.211 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:42:05.203+0000 7ff698884740 1 --2- 172.21.15.89:0/2920013186 >> [v2:172.21.15.110:6838/132366709,v1:172.21.15.110:6839/132366709] conn(0x55bb4f79dd80 0x55bb4fb9df20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:05.211 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:42:05.203+0000 7ff698884740 4 client.15837 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:42:05.211 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:42:05.203+0000 7ff698884740 1 -- 172.21.15.89:0/2920013186 --> [v2:172.21.15.110:6838/132366709,v1:172.21.15.110:6839/132366709] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bb4fafb640 con 0x55bb4f79dd80 2024-09-21T03:42:05.211 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:42:05.204+0000 7ff68affd640 1 --2- 172.21.15.89:0/2920013186 >> [v2:172.21.15.110:6838/132366709,v1:172.21.15.110:6839/132366709] conn(0x55bb4f79dd80 0x55bb4fb9df20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:05.211 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:42:05.204+0000 7ff68affd640 1 --2- 172.21.15.89:0/2920013186 >> [v2:172.21.15.110:6838/132366709,v1:172.21.15.110:6839/132366709] conn(0x55bb4f79dd80 0x55bb4fb9df20 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d0ccdee5b14b4581 server_cookie=d56897bbb5e47bbd in_seq=0 out_seq=0 2024-09-21T03:42:05.212 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:42:05.204+0000 7ff66affd640 10 client.15837 ms_handle_connect on v2:172.21.15.110:6838/132366709 2024-09-21T03:42:05.212 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:42:04 smithi114 ceph-mon[34140]: pgmap v440: 129 pgs: 129 active+clean; 258 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 12 MiB/s wr, 2.71k op/s 2024-09-21T03:42:05.213 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:42:05.205+0000 7ff66affd640 1 -- 172.21.15.89:0/2920013186 <== mds.0 v2:172.21.15.110:6838/132366709 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55bb4fafb640 con 0x55bb4f79dd80 2024-09-21T03:42:05.213 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:42:05.205+0000 7ff66affd640 10 client.15837 handle_command_reply: tid=0 2024-09-21T03:42:05.213 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:42:05.205+0000 7ff698884740 1 client.15837 shutdown 2024-09-21T03:42:05.213 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:42:05.205+0000 7ff698884740 1 -- 172.21.15.89:0/2920013186 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55bb4f7df1b0 msgr2=0x55bb4fb91840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:05.213 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:42:05.205+0000 7ff698884740 1 --2- 172.21.15.89:0/2920013186 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55bb4f7df1b0 0x55bb4fb91840 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7ff680009fd0 tx=0x7ff680009780 comp rx=0 tx=0).stop 2024-09-21T03:42:05.213 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:42:05.205+0000 7ff698884740 1 -- 172.21.15.89:0/2920013186 shutdown_connections 2024-09-21T03:42:05.214 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:42:05.205+0000 7ff698884740 1 --2- 172.21.15.89:0/2920013186 >> [v2:172.21.15.110:6838/132366709,v1:172.21.15.110:6839/132366709] conn(0x55bb4f79dd80 0x55bb4fb9df20 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:05.214 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:42:05.205+0000 7ff698884740 1 --2- 172.21.15.89:0/2920013186 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55bb4f7df1b0 0x55bb4fb91840 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:05.214 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:42:05.205+0000 7ff66affd640 0 client.15837 ms_handle_reset on v2:172.21.15.110:6838/132366709 2024-09-21T03:42:05.214 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:42:05.205+0000 7ff698884740 1 --2- 172.21.15.89:0/2920013186 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55bb4f7be4d0 0x55bb4fb90cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:05.214 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:42:05.205+0000 7ff698884740 1 --2- 172.21.15.89:0/2920013186 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55bb4f7b62f0 0x55bb4fb90610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:05.214 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:42:05.205+0000 7ff698884740 1 -- 172.21.15.89:0/2920013186 >> 172.21.15.89:0/2920013186 conn(0x55bb4f7d3300 msgr2=0x55bb4fb80ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:42:05.214 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:42:05.205+0000 7ff698884740 1 -- 172.21.15.89:0/2920013186 shutdown_connections 2024-09-21T03:42:05.214 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:42:05.205+0000 7ff698884740 1 -- 172.21.15.89:0/2920013186 wait complete. 2024-09-21T03:42:05.214 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:42:05.205+0000 7ff698884740 20 client.15837 trim_cache size 0 max 16384 2024-09-21T03:42:05.223 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:42:05.220+0000 7ff698884740 1 Processor -- start 2024-09-21T03:42:05.223 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:42:05.220+0000 7ff698884740 1 -- start start 2024-09-21T03:42:05.223 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:42:05.220+0000 7ff698884740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55bb4f7b62f0 0x55bb4fb905f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:05.223 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:42:05.220+0000 7ff698884740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55bb4f7be4d0 0x55bb4fb91da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:05.223 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:42:05.220+0000 7ff698884740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55bb4f7df1b0 0x55bb4fb8f830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:05.223 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:42:05.220+0000 7ff698884740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7ff68c1aeee0 con 0x55bb4f7b62f0 2024-09-21T03:42:05.223 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:42:05.220+0000 7ff698884740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7ff68c1ac7b0 con 0x55bb4f7be4d0 2024-09-21T03:42:05.224 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:42:05.220+0000 7ff698884740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7ff68c1aa600 con 0x55bb4f7df1b0 2024-09-21T03:42:05.224 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:42:05.220+0000 7ff68a7fc640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55bb4f7b62f0 0x55bb4fb905f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:05.224 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:42:05.220+0000 7ff68a7fc640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55bb4f7b62f0 0x55bb4fb905f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:48674/0 (socket says 172.21.15.89:48674) 2024-09-21T03:42:05.224 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:42:05.220+0000 7ff68a7fc640 1 -- 172.21.15.89:0/248513724 learned_addr learned my addr 172.21.15.89:0/248513724 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-21T03:42:05.224 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:42:05.220+0000 7ff68a7fc640 1 -- 172.21.15.89:0/248513724 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55bb4f7df1b0 msgr2=0x55bb4fb8f830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:42:05.224 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:42:05.220+0000 7ff68a7fc640 1 --2- 172.21.15.89:0/248513724 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55bb4f7df1b0 0x55bb4fb8f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:05.224 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:42:05.220+0000 7ff68a7fc640 1 -- 172.21.15.89:0/248513724 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55bb4f7be4d0 msgr2=0x55bb4fb91da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:42:05.224 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:42:05.220+0000 7ff68a7fc640 1 --2- 172.21.15.89:0/248513724 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55bb4f7be4d0 0x55bb4fb91da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:05.224 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:42:05.220+0000 7ff68a7fc640 1 -- 172.21.15.89:0/248513724 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bb4fb8e7a0 con 0x55bb4f7b62f0 2024-09-21T03:42:05.224 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:42:05.221+0000 7ff68a7fc640 1 --2- 172.21.15.89:0/248513724 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55bb4f7b62f0 0x55bb4fb905f0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7ff680006010 tx=0x7ff6800099c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e997d2ffb6df7e19 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:05.224 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:42:05.221+0000 7ff6717fa640 1 -- 172.21.15.89:0/248513724 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff680023070 con 0x55bb4f7b62f0 2024-09-21T03:42:05.224 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:42:05.221+0000 7ff6717fa640 1 -- 172.21.15.89:0/248513724 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff6800126f0 con 0x55bb4f7b62f0 2024-09-21T03:42:05.224 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:42:05.221+0000 7ff6717fa640 1 -- 172.21.15.89:0/248513724 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff680004be0 con 0x55bb4f7b62f0 2024-09-21T03:42:05.228 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:42:05.222+0000 7ff698884740 1 -- 172.21.15.89:0/248513724 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55bb4f7b62f0 msgr2=0x55bb4fb905f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:05.228 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:42:05.222+0000 7ff698884740 1 --2- 172.21.15.89:0/248513724 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55bb4f7b62f0 0x55bb4fb905f0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7ff680006010 tx=0x7ff6800099c0 comp rx=0 tx=0).stop 2024-09-21T03:42:05.228 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:42:05.222+0000 7ff698884740 1 -- 172.21.15.89:0/248513724 shutdown_connections 2024-09-21T03:42:05.228 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:42:05.222+0000 7ff698884740 1 --2- 172.21.15.89:0/248513724 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55bb4f7be4d0 0x55bb4fb91da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:05.228 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:42:05.222+0000 7ff698884740 1 --2- 172.21.15.89:0/248513724 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55bb4f7df1b0 0x55bb4fb8f830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:05.228 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:42:05.222+0000 7ff698884740 1 --2- 172.21.15.89:0/248513724 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55bb4f7b62f0 0x55bb4fb905f0 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:05.228 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:42:05.222+0000 7ff698884740 1 -- 172.21.15.89:0/248513724 >> 172.21.15.89:0/248513724 conn(0x55bb4f7d3300 msgr2=0x55bb4fb87e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:42:05.228 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:42:05.222+0000 7ff698884740 1 -- 172.21.15.89:0/248513724 shutdown_connections 2024-09-21T03:42:05.228 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:42:05.222+0000 7ff698884740 1 -- 172.21.15.89:0/248513724 wait complete. 2024-09-21T03:42:05.229 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:42:05.222+0000 7ff698884740 1 -- start start 2024-09-21T03:42:05.229 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:42:05.222+0000 7ff698884740 1 Processor -- start 2024-09-21T03:42:05.229 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:42:05.223+0000 7ff698884740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55bb4f7b62f0 0x55bb4fb94fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:05.229 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:42:05.223+0000 7ff698884740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55bb4f7be4d0 0x55bb4fb95690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:05.229 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:42:05.223+0000 7ff698884740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55bb4f7df1b0 0x55bb4fb96200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:05.229 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:42:05.223+0000 7ff698884740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7ff68c1aac70 con 0x55bb4f7df1b0 2024-09-21T03:42:05.229 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:42:05.223+0000 7ff698884740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7ff68c1ac0e0 con 0x55bb4f7be4d0 2024-09-21T03:42:05.229 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:42:05.223+0000 7ff698884740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7ff68c1ab340 con 0x55bb4f7b62f0 2024-09-21T03:42:05.229 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:42:05.223+0000 7ff698884740 10 client.0 _ll_register_callbacks cb 0x7ff67c009260 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:42:05.229 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:42:05.223+0000 7ff68a7fc640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55bb4f7df1b0 0x55bb4fb96200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:05.229 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:42:05.223+0000 7ff68a7fc640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55bb4f7df1b0 0x55bb4fb96200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:48684/0 (socket says 172.21.15.89:48684) 2024-09-21T03:42:05.229 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:42:05.223+0000 7ff68a7fc640 1 -- 172.21.15.89:0/1487045810 learned_addr learned my addr 172.21.15.89:0/1487045810 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-21T03:42:05.229 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:42:05.223+0000 7ff68a7fc640 1 --2- 172.21.15.89:0/1487045810 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55bb4f7b62f0 0x55bb4fb94fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:05.229 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:42:05.223+0000 7ff68affd640 1 --2- 172.21.15.89:0/1487045810 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55bb4f7be4d0 0x55bb4fb95690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:05.229 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:42:05.223+0000 7ff68a7fc640 1 -- 172.21.15.89:0/1487045810 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55bb4f7b62f0 msgr2=0x55bb4fb94fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:42:05.229 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:42:05.223+0000 7ff68a7fc640 1 --2- 172.21.15.89:0/1487045810 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55bb4f7b62f0 0x55bb4fb94fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:05.230 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:42:05.223+0000 7ff68a7fc640 1 -- 172.21.15.89:0/1487045810 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55bb4f7be4d0 msgr2=0x55bb4fb95690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:42:05.230 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:42:05.223+0000 7ff68a7fc640 1 --2- 172.21.15.89:0/1487045810 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55bb4f7be4d0 0x55bb4fb95690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:05.230 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:42:05.223+0000 7ff68a7fc640 1 -- 172.21.15.89:0/1487045810 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bb4fb96960 con 0x55bb4f7df1b0 2024-09-21T03:42:05.230 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:42:05.223+0000 7ff68affd640 1 --2- 172.21.15.89:0/1487045810 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55bb4f7be4d0 0x55bb4fb95690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:42:05.230 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:42:05.223+0000 7ff68a7fc640 1 --2- 172.21.15.89:0/1487045810 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55bb4f7df1b0 0x55bb4fb96200 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7ff6800097c0 tx=0x7ff680020040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dce6f38b89663be7 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:05.230 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:42:05.223+0000 7ff66affd640 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2024-09-21T03:42:05.230 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:42:05.223+0000 7ff66affd640 1 -- 172.21.15.89:0/1487045810 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff680023050 con 0x55bb4f7df1b0 2024-09-21T03:42:05.230 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:42:05.223+0000 7ff66affd640 1 -- 172.21.15.89:0/1487045810 <== mon.0 v2:172.21.15.89:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 9121+0+0 (secure 0 0 0) 0x7ff680004ca0 con 0x55bb4f7df1b0 2024-09-21T03:42:05.230 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:42:05.224+0000 7ff698884740 1 -- 172.21.15.89:0/1487045810 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff6800126f0 con 0x55bb4f7df1b0 2024-09-21T03:42:05.230 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:42:05.224+0000 7ff66affd640 1 -- 172.21.15.89:0/1487045810 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7ff680005740 con 0x55bb4f7df1b0 2024-09-21T03:42:05.230 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:42:05.224+0000 7ff698884740 10 client.15843 fetch_fsmap learned FSMap version 26 2024-09-21T03:42:05.230 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:42:05.224+0000 7ff698884740 1 -- 172.21.15.89:0/1487045810 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55bb4fb97c50 con 0x55bb4f7df1b0 2024-09-21T03:42:05.230 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:42:05.224+0000 7ff66affd640 1 -- 172.21.15.89:0/1487045810 <== mon.0 v2:172.21.15.89:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff6800126f0 con 0x55bb4f7df1b0 2024-09-21T03:42:05.230 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:42:05.224+0000 7ff66affd640 1 -- 172.21.15.89:0/1487045810 <== mon.0 v2:172.21.15.89:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7ff68001c030 con 0x55bb4f7df1b0 2024-09-21T03:42:05.231 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:42:05.224+0000 7ff698884740 10 client.15843 fetch_fsmap finished waiting for FSMap version 26 2024-09-21T03:42:05.231 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:42:05.224+0000 7ff698884740 10 client.15843 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T03:42:05.231 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:42:05.224+0000 7ff698884740 20 client.15843 populate_metadata read hostname 'smithi089' 2024-09-21T03:42:05.231 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:42:05.224+0000 7ff698884740 1 --2- 172.21.15.89:0/1487045810 >> [v2:172.21.15.110:6838/132366709,v1:172.21.15.110:6839/132366709] conn(0x55bb4f7731c0 0x55bb4fc0a910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:05.231 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:42:05.224+0000 7ff698884740 4 client.15843 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T03:42:05.231 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:42:05.224+0000 7ff698884740 1 -- 172.21.15.89:0/1487045810 --> [v2:172.21.15.110:6838/132366709,v1:172.21.15.110:6839/132366709] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55bb4faf70f0 con 0x55bb4f7731c0 2024-09-21T03:42:05.231 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:42:05.224+0000 7ff68affd640 1 --2- 172.21.15.89:0/1487045810 >> [v2:172.21.15.110:6838/132366709,v1:172.21.15.110:6839/132366709] conn(0x55bb4f7731c0 0x55bb4fc0a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:05.231 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:42:05.225+0000 7ff68affd640 1 --2- 172.21.15.89:0/1487045810 >> [v2:172.21.15.110:6838/132366709,v1:172.21.15.110:6839/132366709] conn(0x55bb4f7731c0 0x55bb4fc0a910 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4ad548d145eb52c5 server_cookie=a7d5fc1d03e070bb in_seq=0 out_seq=0 2024-09-21T03:42:05.231 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:42:05.225+0000 7ff66affd640 10 client.15843 ms_handle_connect on v2:172.21.15.110:6838/132366709 2024-09-21T03:42:05.237 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:42:05.234+0000 7ff66affd640 1 -- 172.21.15.89:0/1487045810 <== mds.0 v2:172.21.15.110:6838/132366709 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55bb4faf70f0 con 0x55bb4f7731c0 2024-09-21T03:42:05.237 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:42:05.234+0000 7ff66affd640 10 client.15843 handle_command_reply: tid=0 2024-09-21T03:42:05.237 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:42:05.234+0000 7ff698884740 1 client.15843 shutdown 2024-09-21T03:42:05.237 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:42:05.234+0000 7ff698884740 1 -- 172.21.15.89:0/1487045810 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55bb4f7df1b0 msgr2=0x55bb4fb96200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:05.237 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:42:05.234+0000 7ff698884740 1 --2- 172.21.15.89:0/1487045810 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55bb4f7df1b0 0x55bb4fb96200 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7ff6800097c0 tx=0x7ff680020040 comp rx=0 tx=0).stop 2024-09-21T03:42:05.238 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:42:05.234+0000 7ff698884740 1 -- 172.21.15.89:0/1487045810 shutdown_connections 2024-09-21T03:42:05.238 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:42:05.234+0000 7ff698884740 1 --2- 172.21.15.89:0/1487045810 >> [v2:172.21.15.110:6838/132366709,v1:172.21.15.110:6839/132366709] conn(0x55bb4f7731c0 0x55bb4fc0a910 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:05.238 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:42:05.234+0000 7ff698884740 1 --2- 172.21.15.89:0/1487045810 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55bb4f7be4d0 0x55bb4fb95690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:05.238 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:42:05.234+0000 7ff698884740 1 --2- 172.21.15.89:0/1487045810 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55bb4f7df1b0 0x55bb4fb96200 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:05.238 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:42:05.234+0000 7ff66affd640 0 client.15843 ms_handle_reset on v2:172.21.15.110:6838/132366709 2024-09-21T03:42:05.238 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:42:05.234+0000 7ff698884740 1 --2- 172.21.15.89:0/1487045810 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55bb4f7b62f0 0x55bb4fb94fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:05.238 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:42:05.234+0000 7ff698884740 1 -- 172.21.15.89:0/1487045810 >> 172.21.15.89:0/1487045810 conn(0x55bb4f7d3300 msgr2=0x55bb4fb8f830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:42:05.239 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:42:05.234+0000 7ff698884740 1 -- 172.21.15.89:0/1487045810 shutdown_connections 2024-09-21T03:42:05.239 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:42:05.234+0000 7ff698884740 1 -- 172.21.15.89:0/1487045810 wait complete. 2024-09-21T03:42:05.239 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:42:05.234+0000 7ff698884740 20 client.15843 trim_cache size 0 max 16384 2024-09-21T03:42:05.239 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stdout:{ 2024-09-21T03:42:05.239 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stdout: "message": "", 2024-09-21T03:42:05.239 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stdout: "return_code": 0 2024-09-21T03:42:05.239 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stdout:} 2024-09-21T03:42:05.240 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:42:05.235+0000 7ff691919640 1 -- 172.21.15.89:0/3888467809 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7ff6740b6510 msgr2=0x7ff6740b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:05.240 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:42:05.235+0000 7ff691919640 1 --2- 172.21.15.89:0/3888467809 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7ff6740b6510 0x7ff6740b8a20 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7ff67c00eaa0 tx=0x7ff67c079040 comp rx=0 tx=0).stop 2024-09-21T03:42:05.240 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:42:05.235+0000 7ff691919640 1 -- 172.21.15.89:0/3888467809 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff68c108300 msgr2=0x7ff68c19b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:05.240 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:42:05.235+0000 7ff691919640 1 --2- 172.21.15.89:0/3888467809 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff68c108300 0x7ff68c19b0a0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7ff67c00ead0 tx=0x7ff67c00ef90 comp rx=0 tx=0).stop 2024-09-21T03:42:05.240 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:42:05.235+0000 7ff691919640 1 -- 172.21.15.89:0/3888467809 shutdown_connections 2024-09-21T03:42:05.240 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:42:05.235+0000 7ff691919640 1 --2- 172.21.15.89:0/3888467809 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7ff6740b6510 0x7ff6740b8a20 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:05.240 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:42:05.235+0000 7ff691919640 1 --2- 172.21.15.89:0/3888467809 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff68c108300 0x7ff68c19b0a0 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:05.241 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:42:05.235+0000 7ff691919640 1 --2- 172.21.15.89:0/3888467809 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff68c1056e0 0x7ff68c19a4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:05.241 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:42:05.235+0000 7ff691919640 1 --2- 172.21.15.89:0/3888467809 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff68c102c60 0x7ff68c199e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:05.241 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:42:05.235+0000 7ff691919640 1 -- 172.21.15.89:0/3888467809 >> 172.21.15.89:0/3888467809 conn(0x7ff68c0fd690 msgr2=0x7ff68c1079d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:42:05.241 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:42:05.235+0000 7ff691919640 1 -- 172.21.15.89:0/3888467809 shutdown_connections 2024-09-21T03:42:05.241 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:42:05.235+0000 7ff691919640 1 -- 172.21.15.89:0/3888467809 wait complete. 2024-09-21T03:42:05.250 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:42:04 smithi110 ceph-mon[34000]: pgmap v440: 129 pgs: 129 active+clean; 258 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 12 MiB/s wr, 2.71k op/s 2024-09-21T03:42:05.311 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:42:04 smithi089 ceph-mon[30928]: pgmap v440: 129 pgs: 129 active+clean; 258 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 12 MiB/s wr, 2.71k op/s 2024-09-21T03:42:07.211 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:42:06 smithi114 ceph-mon[34140]: pgmap v441: 129 pgs: 129 active+clean; 258 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 9.8 MiB/s wr, 2.14k op/s 2024-09-21T03:42:07.250 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:42:06 smithi110 ceph-mon[34000]: pgmap v441: 129 pgs: 129 active+clean; 258 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 9.8 MiB/s wr, 2.14k op/s 2024-09-21T03:42:07.311 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:42:06 smithi089 ceph-mon[30928]: pgmap v441: 129 pgs: 129 active+clean; 258 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 9.8 MiB/s wr, 2.14k op/s 2024-09-21T03:42:09.211 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:42:08 smithi114 ceph-mon[34140]: pgmap v442: 129 pgs: 129 active+clean; 258 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 7.6 MiB/s wr, 1.72k op/s 2024-09-21T03:42:09.250 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:42:08 smithi110 ceph-mon[34000]: pgmap v442: 129 pgs: 129 active+clean; 258 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 7.6 MiB/s wr, 1.72k op/s 2024-09-21T03:42:09.311 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:42:08 smithi089 ceph-mon[30928]: pgmap v442: 129 pgs: 129 active+clean; 258 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 7.6 MiB/s wr, 1.72k op/s 2024-09-21T03:42:09.371 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '6404c199', quiesce took 31.4 seconds. Will release after: -3.599999999999998 2024-09-21T03:42:09.371 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 7c374098-77c9-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T03:42:09.623 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/7c374098-77c9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:42:11.091 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:42:10 smithi089 ceph-mon[30928]: pgmap v443: 129 pgs: 129 active+clean; 258 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 6.3 MiB/s wr, 1.41k op/s 2024-09-21T03:42:11.191 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:11.180+0000 7f7c0d701640 1 -- 172.21.15.89:0/2190321472 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7c08103ba0 msgr2=0x7f7c08103fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:11.191 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:11.180+0000 7f7c0d701640 1 --2- 172.21.15.89:0/2190321472 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7c08103ba0 0x7f7c08103fb0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f7bf8009ca0 tx=0x7f7bf802a8c0 comp rx=0 tx=0).stop 2024-09-21T03:42:11.191 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:11.180+0000 7f7c0d701640 1 -- 172.21.15.89:0/2190321472 shutdown_connections 2024-09-21T03:42:11.191 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:11.180+0000 7f7c0d701640 1 --2- 172.21.15.89:0/2190321472 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7c081094e0 0x7f7c081098f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:11.191 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:11.180+0000 7f7c0d701640 1 --2- 172.21.15.89:0/2190321472 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7c08109e30 0x7f7c08110660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:11.191 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:11.180+0000 7f7c0d701640 1 --2- 172.21.15.89:0/2190321472 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7c08103ba0 0x7f7c08103fb0 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:11.192 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:11.180+0000 7f7c0d701640 1 -- 172.21.15.89:0/2190321472 >> 172.21.15.89:0/2190321472 conn(0x7f7c080ff630 msgr2=0x7f7c08101a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:42:11.192 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:11.180+0000 7f7c0d701640 1 -- 172.21.15.89:0/2190321472 shutdown_connections 2024-09-21T03:42:11.192 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:11.180+0000 7f7c0d701640 1 -- 172.21.15.89:0/2190321472 wait complete. 2024-09-21T03:42:11.192 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:11.180+0000 7f7c0d701640 1 Processor -- start 2024-09-21T03:42:11.192 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:11.181+0000 7f7c0d701640 1 -- start start 2024-09-21T03:42:11.192 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:11.181+0000 7f7c0d701640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7c08103ba0 0x7f7c0819a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:11.192 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:11.181+0000 7f7c0d701640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7c081094e0 0x7f7c0819a980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:11.192 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:11.181+0000 7f7c0d701640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7c08109e30 0x7f7c0819fc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:11.192 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:11.181+0000 7f7c0d701640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f7c081139f0 con 0x7f7c081094e0 2024-09-21T03:42:11.192 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:11.181+0000 7f7c0d701640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f7c08113870 con 0x7f7c08103ba0 2024-09-21T03:42:11.192 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:11.181+0000 7f7c0d701640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f7c08113b70 con 0x7f7c08109e30 2024-09-21T03:42:11.192 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:11.181+0000 7f7c067fc640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7c081094e0 0x7f7c0819a980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:11.192 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:11.181+0000 7f7c067fc640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7c081094e0 0x7f7c0819a980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:49762/0 (socket says 172.21.15.89:49762) 2024-09-21T03:42:11.192 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:11.181+0000 7f7c067fc640 1 -- 172.21.15.89:0/2206813293 learned_addr learned my addr 172.21.15.89:0/2206813293 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-21T03:42:11.193 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:11.181+0000 7f7c077fe640 1 --2- 172.21.15.89:0/2206813293 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7c08109e30 0x7f7c0819fc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:11.193 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:11.181+0000 7f7c06ffd640 1 --2- 172.21.15.89:0/2206813293 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7c08103ba0 0x7f7c0819a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:11.193 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:11.181+0000 7f7c067fc640 1 -- 172.21.15.89:0/2206813293 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7c08109e30 msgr2=0x7f7c0819fc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:11.193 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:11.181+0000 7f7c067fc640 1 --2- 172.21.15.89:0/2206813293 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7c08109e30 0x7f7c0819fc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:11.193 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:11.181+0000 7f7c067fc640 1 -- 172.21.15.89:0/2206813293 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7c08103ba0 msgr2=0x7f7c0819a3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:11.193 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:11.181+0000 7f7c067fc640 1 --2- 172.21.15.89:0/2206813293 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7c08103ba0 0x7f7c0819a3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:11.193 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:11.181+0000 7f7c067fc640 1 -- 172.21.15.89:0/2206813293 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c081a0350 con 0x7f7c081094e0 2024-09-21T03:42:11.193 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:11.182+0000 7f7c067fc640 1 --2- 172.21.15.89:0/2206813293 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7c081094e0 0x7f7c0819a980 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f7bf000d740 tx=0x7f7bf000dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:11.193 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:11.182+0000 7f7bfffff640 1 -- 172.21.15.89:0/2206813293 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bf0017070 con 0x7f7c081094e0 2024-09-21T03:42:11.193 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:11.182+0000 7f7bfffff640 1 -- 172.21.15.89:0/2206813293 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7bf000de50 con 0x7f7c081094e0 2024-09-21T03:42:11.193 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:11.182+0000 7f7bfffff640 1 -- 172.21.15.89:0/2206813293 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bf000f620 con 0x7f7c081094e0 2024-09-21T03:42:11.193 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:11.182+0000 7f7c0d701640 1 -- 172.21.15.89:0/2206813293 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7c081a0640 con 0x7f7c081094e0 2024-09-21T03:42:11.193 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:11.182+0000 7f7c0d701640 1 -- 172.21.15.89:0/2206813293 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7c081a6da0 con 0x7f7c081094e0 2024-09-21T03:42:11.193 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:11.183+0000 7f7c0d701640 1 -- 172.21.15.89:0/2206813293 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7c08103fb0 con 0x7f7c081094e0 2024-09-21T03:42:11.193 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:11.185+0000 7f7bfffff640 1 -- 172.21.15.89:0/2206813293 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 21) ==== 154512+0+0 (secure 0 0 0) 0x7f7bf001a070 con 0x7f7c081094e0 2024-09-21T03:42:11.194 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:11.186+0000 7f7bfffff640 1 --2- 172.21.15.89:0/2206813293 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f7bf40b6590 0x7f7bf40b8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:11.194 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:11.186+0000 7f7c06ffd640 1 --2- 172.21.15.89:0/2206813293 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f7bf40b6590 0x7f7bf40b8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:11.194 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:11.186+0000 7f7bfffff640 1 -- 172.21.15.89:0/2206813293 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 9121+0+0 (secure 0 0 0) 0x7f7bf00e6ef0 con 0x7f7c081094e0 2024-09-21T03:42:11.194 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:11.187+0000 7f7c06ffd640 1 --2- 172.21.15.89:0/2206813293 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f7bf40b6590 0x7f7bf40b8aa0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f7bf8000f80 tx=0x7f7bf8031000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:11.194 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:11.190+0000 7f7bfffff640 1 -- 172.21.15.89:0/2206813293 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f7bf00b1860 con 0x7f7c081094e0 2024-09-21T03:42:11.211 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:42:10 smithi114 ceph-mon[34140]: pgmap v443: 129 pgs: 129 active+clean; 258 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 6.3 MiB/s wr, 1.41k op/s 2024-09-21T03:42:11.250 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:42:10 smithi110 ceph-mon[34000]: pgmap v443: 129 pgs: 129 active+clean; 258 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 6.3 MiB/s wr, 1.41k op/s 2024-09-21T03:42:11.406 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:11.403+0000 7f7c0d701640 1 -- 172.21.15.89:0/2206813293 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7c0819b510 con 0x7f7c081094e0 2024-09-21T03:42:11.406 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:11.404+0000 7f7bfffff640 1 -- 172.21.15.89:0/2206813293 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11912 (secure 0 0 0) 0x7f7bf00b67a0 con 0x7f7c081094e0 2024-09-21T03:42:11.407 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-09-21T03:42:11.407 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":26,"btime":"2024-09-21T03:39:26:058611+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14721,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6835/2341538514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":2341538514},{"type":"v1","addr":"172.21.15.89:6835","nonce":2341538514}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14733,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6837/146708592","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6836","nonce":146708592},{"type":"v1","addr":"172.21.15.89:6837","nonce":146708592}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14781,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6839/2269965499","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6838","nonce":2269965499},{"type":"v1","addr":"172.21.15.89:6839","nonce":2269965499}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14793,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6841/650850383","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6840","nonce":650850383},{"type":"v1","addr":"172.21.15.89:6841","nonce":650850383}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24565,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6833/752336912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6832","nonce":752336912},{"type":"v1","addr":"172.21.15.114:6833","nonce":752336912}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24575,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6833/127607331","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6832","nonce":127607331},{"type":"v1","addr":"172.21.15.110:6833","nonce":127607331}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T03:37:13.570694+0000","modified":"2024-09-21T03:39:25.333954+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24638,"mds_1":24631,"mds_2":24614},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24593":{"gid":24593,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.110:6835/671820654","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6834","nonce":671820654},{"type":"v1","addr":"172.21.15.110:6835","nonce":671820654}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24604":{"gid":24604,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.114:6835/1049714548","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6834","nonce":1049714548},{"type":"v1","addr":"172.21.15.114:6835","nonce":1049714548}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24610":{"gid":24610,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.114:6837/1941213818","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6836","nonce":1941213818},{"type":"v1","addr":"172.21.15.114:6837","nonce":1941213818}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24614":{"gid":24614,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":15,"addr":"172.21.15.110:6837/4047124589","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6836","nonce":4047124589},{"type":"v1","addr":"172.21.15.110:6837","nonce":4047124589}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24631":{"gid":24631,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":15,"addr":"172.21.15.114:6839/3427369944","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6838","nonce":3427369944},{"type":"v1","addr":"172.21.15.114:6839","nonce":3427369944}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24638":{"gid":24638,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.110:6839/132366709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6838","nonce":132366709},{"type":"v1","addr":"172.21.15.110:6839","nonce":132366709}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24638,"qdb_cluster":[24638,24631,24614]},"id":1}]} 2024-09-21T03:42:11.409 INFO:teuthology.orchestra.run.smithi089.stderr:dumped fsmap epoch 26 2024-09-21T03:42:11.410 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:11.406+0000 7f7c0d701640 1 -- 172.21.15.89:0/2206813293 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f7bf40b6590 msgr2=0x7f7bf40b8aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:11.410 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:11.406+0000 7f7c0d701640 1 --2- 172.21.15.89:0/2206813293 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f7bf40b6590 0x7f7bf40b8aa0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f7bf8000f80 tx=0x7f7bf8031000 comp rx=0 tx=0).stop 2024-09-21T03:42:11.410 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:11.406+0000 7f7c0d701640 1 -- 172.21.15.89:0/2206813293 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7c081094e0 msgr2=0x7f7c0819a980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:11.410 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:11.406+0000 7f7c0d701640 1 --2- 172.21.15.89:0/2206813293 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7c081094e0 0x7f7c0819a980 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f7bf000d740 tx=0x7f7bf000dc00 comp rx=0 tx=0).stop 2024-09-21T03:42:11.410 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:11.406+0000 7f7c0d701640 1 -- 172.21.15.89:0/2206813293 shutdown_connections 2024-09-21T03:42:11.410 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:11.406+0000 7f7c0d701640 1 --2- 172.21.15.89:0/2206813293 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f7bf40b6590 0x7f7bf40b8aa0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:11.410 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:11.406+0000 7f7c0d701640 1 --2- 172.21.15.89:0/2206813293 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7c08109e30 0x7f7c0819fc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:11.410 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:11.406+0000 7f7c0d701640 1 --2- 172.21.15.89:0/2206813293 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7c081094e0 0x7f7c0819a980 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:11.410 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:11.406+0000 7f7c0d701640 1 --2- 172.21.15.89:0/2206813293 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7c08103ba0 0x7f7c0819a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:11.410 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:11.406+0000 7f7c0d701640 1 -- 172.21.15.89:0/2206813293 >> 172.21.15.89:0/2206813293 conn(0x7f7c080ff630 msgr2=0x7f7c08101a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:42:11.410 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:11.406+0000 7f7c0d701640 1 -- 172.21.15.89:0/2206813293 shutdown_connections 2024-09-21T03:42:11.410 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:11.406+0000 7f7c0d701640 1 -- 172.21.15.89:0/2206813293 wait complete. 2024-09-21T03:42:12.173 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:42:11 smithi089 ceph-mon[30928]: from='client.? 172.21.15.89:0/2206813293' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:42:12.202 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24638 quiesce db --set-id 6404c199 --release --await' 2024-09-21T03:42:12.202 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 7c374098-77c9-11ef-baf5-efdc52797490 -- timeout 105 ceph tell mds.24638 quiesce db --set-id 6404c199 --release --await 2024-09-21T03:42:12.210 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:42:11 smithi114 ceph-mon[34140]: from='client.? 172.21.15.89:0/2206813293' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:42:12.250 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:42:11 smithi110 ceph-mon[34000]: from='client.? 172.21.15.89:0/2206813293' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:42:12.450 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/7c374098-77c9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:42:12.952 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:12.947+0000 7f82d6e25640 1 -- 172.21.15.89:0/921772978 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f82d0102a80 msgr2=0x7f82d0102e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:12.953 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:12.947+0000 7f82d6e25640 1 --2- 172.21.15.89:0/921772978 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f82d0102a80 0x7f82d0102e90 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f82cc009900 tx=0x7f82cc02a410 comp rx=0 tx=0).stop 2024-09-21T03:42:12.953 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:12.947+0000 7f82d6e25640 1 -- 172.21.15.89:0/921772978 shutdown_connections 2024-09-21T03:42:12.953 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:12.947+0000 7f82d6e25640 1 --2- 172.21.15.89:0/921772978 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f82d01083c0 0x7f82d01087d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:12.953 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:12.947+0000 7f82d6e25640 1 --2- 172.21.15.89:0/921772978 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f82d0108d10 0x7f82d010f540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:12.953 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:12.947+0000 7f82d6e25640 1 --2- 172.21.15.89:0/921772978 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f82d0102a80 0x7f82d0102e90 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:12.953 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:12.947+0000 7f82d6e25640 1 -- 172.21.15.89:0/921772978 >> 172.21.15.89:0/921772978 conn(0x7f82d0079740 msgr2=0x7f82d0079b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:42:12.953 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:12.947+0000 7f82d6e25640 1 -- 172.21.15.89:0/921772978 shutdown_connections 2024-09-21T03:42:12.953 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:12.947+0000 7f82d6e25640 1 -- 172.21.15.89:0/921772978 wait complete. 2024-09-21T03:42:12.953 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:12.948+0000 7f82d6e25640 1 Processor -- start 2024-09-21T03:42:12.953 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:12.948+0000 7f82d6e25640 1 -- start start 2024-09-21T03:42:12.953 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:12.948+0000 7f82d6e25640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f82d0102a80 0x7f82d019a450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:12.953 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:12.948+0000 7f82d6e25640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f82d01083c0 0x7f82d019a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:12.953 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:12.948+0000 7f82d6e25640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f82d0108d10 0x7f82d019fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:12.954 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:12.948+0000 7f82d6e25640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f82d01128b0 con 0x7f82d0102a80 2024-09-21T03:42:12.954 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:12.948+0000 7f82d6e25640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f82d0112730 con 0x7f82d0108d10 2024-09-21T03:42:12.954 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:12.948+0000 7f82d6e25640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f82d0112a30 con 0x7f82d01083c0 2024-09-21T03:42:12.954 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:12.948+0000 7f82d4bc1640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f82d0102a80 0x7f82d019a450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:12.954 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:12.948+0000 7f82d4bc1640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f82d0102a80 0x7f82d019a450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:49780/0 (socket says 172.21.15.89:49780) 2024-09-21T03:42:12.954 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:12.948+0000 7f82d4bc1640 1 -- 172.21.15.89:0/3266654704 learned_addr learned my addr 172.21.15.89:0/3266654704 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-21T03:42:12.954 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:12.949+0000 7f82d53c2640 1 --2- 172.21.15.89:0/3266654704 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f82d0108d10 0x7f82d019fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:12.954 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:12.949+0000 7f82c7fff640 1 --2- 172.21.15.89:0/3266654704 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f82d01083c0 0x7f82d019a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:12.954 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:12.949+0000 7f82d4bc1640 1 -- 172.21.15.89:0/3266654704 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f82d01083c0 msgr2=0x7f82d019a9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:12.954 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:12.949+0000 7f82d4bc1640 1 --2- 172.21.15.89:0/3266654704 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f82d01083c0 0x7f82d019a9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:12.954 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:12.949+0000 7f82d4bc1640 1 -- 172.21.15.89:0/3266654704 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f82d0108d10 msgr2=0x7f82d019fc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:12.954 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:12.949+0000 7f82d4bc1640 1 --2- 172.21.15.89:0/3266654704 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f82d0108d10 0x7f82d019fc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:12.954 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:12.949+0000 7f82d4bc1640 1 -- 172.21.15.89:0/3266654704 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82d01a03c0 con 0x7f82d0102a80 2024-09-21T03:42:12.954 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:12.949+0000 7f82d53c2640 1 --2- 172.21.15.89:0/3266654704 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f82d0108d10 0x7f82d019fc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:42:12.954 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:12.949+0000 7f82d4bc1640 1 --2- 172.21.15.89:0/3266654704 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f82d0102a80 0x7f82d019a450 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f82cc009a20 tx=0x7f82cc004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:12.955 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:12.949+0000 7f82c5ffb640 1 -- 172.21.15.89:0/3266654704 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82cc03f030 con 0x7f82d0102a80 2024-09-21T03:42:12.955 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:12.949+0000 7f82c5ffb640 1 -- 172.21.15.89:0/3266654704 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f82cc0035f0 con 0x7f82d0102a80 2024-09-21T03:42:12.955 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:12.949+0000 7f82d6e25640 1 -- 172.21.15.89:0/3266654704 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f82d01a0650 con 0x7f82d0102a80 2024-09-21T03:42:12.955 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:12.949+0000 7f82c5ffb640 1 -- 172.21.15.89:0/3266654704 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82cc035070 con 0x7f82d0102a80 2024-09-21T03:42:12.955 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:12.949+0000 7f82d6e25640 1 -- 172.21.15.89:0/3266654704 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82d01a6e10 con 0x7f82d0102a80 2024-09-21T03:42:12.959 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:12.952+0000 7f82dd399740 1 Processor -- start 2024-09-21T03:42:12.959 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:12.952+0000 7f82dd399740 1 -- start start 2024-09-21T03:42:12.959 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:12.952+0000 7f82c5ffb640 1 -- 172.21.15.89:0/3266654704 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 21) ==== 154512+0+0 (secure 0 0 0) 0x7f82cc003aa0 con 0x7f82d0102a80 2024-09-21T03:42:12.959 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:12.952+0000 7f82dd399740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56517a9c8520 0x56517ad6e510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:12.959 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:12.952+0000 7f82dd399740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56517a99f660 0x56517ad6ea50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:12.959 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:12.952+0000 7f82dd399740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56517a9a7840 0x56517ad6ef90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:12.960 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:12.952+0000 7f82dd399740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x56517ac59dc0 con 0x56517a99f660 2024-09-21T03:42:12.960 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:12.952+0000 7f82dd399740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x56517ace61f0 con 0x56517a9a7840 2024-09-21T03:42:12.960 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:12.952+0000 7f82dd399740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x56517acd5620 con 0x56517a9c8520 2024-09-21T03:42:12.960 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:12.952+0000 7f82d4bc1640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56517a99f660 0x56517ad6ea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:12.960 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:12.952+0000 7f82d4bc1640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56517a99f660 0x56517ad6ea50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:49790/0 (socket says 172.21.15.89:49790) 2024-09-21T03:42:12.960 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:12.952+0000 7f82d53c2640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56517a9c8520 0x56517ad6e510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:12.960 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:12.952+0000 7f82d4bc1640 1 -- 172.21.15.89:0/724593800 learned_addr learned my addr 172.21.15.89:0/724593800 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-21T03:42:12.960 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:12.953+0000 7f82c7fff640 1 --2- 172.21.15.89:0/724593800 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56517a9a7840 0x56517ad6ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:12.960 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:12.953+0000 7f82d4bc1640 1 -- 172.21.15.89:0/724593800 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56517a9c8520 msgr2=0x56517ad6e510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:42:12.960 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:12.953+0000 7f82d4bc1640 1 --2- 172.21.15.89:0/724593800 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56517a9c8520 0x56517ad6e510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:12.960 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:12.953+0000 7f82d4bc1640 1 -- 172.21.15.89:0/724593800 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56517a9a7840 msgr2=0x56517ad6ef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:42:12.960 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:12.953+0000 7f82d4bc1640 1 --2- 172.21.15.89:0/724593800 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56517a9a7840 0x56517ad6ef90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:12.960 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:12.953+0000 7f82d4bc1640 1 -- 172.21.15.89:0/724593800 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56517ac9dfd0 con 0x56517a99f660 2024-09-21T03:42:12.960 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:12.953+0000 7f82c5ffb640 1 --2- 172.21.15.89:0/3266654704 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f82b80b6590 0x7f82b80b8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:12.961 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:12.953+0000 7f82d4bc1640 1 --2- 172.21.15.89:0/724593800 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56517a99f660 0x56517ad6ea50 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f82cc1104b0 tx=0x7f82cc1105a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6ccd726c9524109 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:12.961 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:12.953+0000 7f82d53c2640 1 --2- 172.21.15.89:0/3266654704 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f82b80b6590 0x7f82b80b8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:12.961 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:12.953+0000 7f82b6ffd640 1 -- 172.21.15.89:0/724593800 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82cc0ca4a0 con 0x56517a99f660 2024-09-21T03:42:12.961 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:12.953+0000 7f82c5ffb640 1 -- 172.21.15.89:0/3266654704 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 9121+0+0 (secure 0 0 0) 0x7f82cc108040 con 0x7f82d0102a80 2024-09-21T03:42:12.961 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:12.953+0000 7f82b6ffd640 1 -- 172.21.15.89:0/724593800 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f82cc115070 con 0x56517a99f660 2024-09-21T03:42:12.961 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:12.953+0000 7f82b6ffd640 1 -- 172.21.15.89:0/724593800 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82cc0cf5e0 con 0x56517a99f660 2024-09-21T03:42:12.961 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:12.954+0000 7f82dd399740 1 -- 172.21.15.89:0/724593800 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56517a99f660 msgr2=0x56517ad6ea50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:12.961 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:12.954+0000 7f82d53c2640 1 --2- 172.21.15.89:0/3266654704 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f82b80b6590 0x7f82b80b8aa0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f82c8005fd0 tx=0x7f82c8008630 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:12.961 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:12.954+0000 7f82dd399740 1 --2- 172.21.15.89:0/724593800 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56517a99f660 0x56517ad6ea50 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f82cc1104b0 tx=0x7f82cc1105a0 comp rx=0 tx=0).stop 2024-09-21T03:42:12.961 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:12.954+0000 7f82dd399740 1 -- 172.21.15.89:0/724593800 shutdown_connections 2024-09-21T03:42:12.961 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:12.954+0000 7f82dd399740 1 --2- 172.21.15.89:0/724593800 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56517a9a7840 0x56517ad6ef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:12.961 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:12.954+0000 7f82dd399740 1 --2- 172.21.15.89:0/724593800 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56517a99f660 0x56517ad6ea50 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:12.961 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:12.954+0000 7f82dd399740 1 --2- 172.21.15.89:0/724593800 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56517a9c8520 0x56517ad6e510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:12.961 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:12.954+0000 7f82dd399740 1 -- 172.21.15.89:0/724593800 >> 172.21.15.89:0/724593800 conn(0x56517a9bc910 msgr2=0x56517ad6c400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:42:12.961 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:12.954+0000 7f82dd399740 1 -- 172.21.15.89:0/724593800 shutdown_connections 2024-09-21T03:42:12.961 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:12.954+0000 7f82dd399740 1 -- 172.21.15.89:0/724593800 wait complete. 2024-09-21T03:42:12.962 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:12.955+0000 7f82dd399740 1 -- start start 2024-09-21T03:42:12.962 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:12.955+0000 7f82dd399740 1 Processor -- start 2024-09-21T03:42:12.962 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:12.955+0000 7f82dd399740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56517a99f660 0x56517ad7df10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:12.962 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:12.955+0000 7f82dd399740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56517a9a7840 0x56517ad7e450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:12.962 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:12.955+0000 7f82dd399740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56517a9c8520 0x56517ad7e990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:12.962 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:12.955+0000 7f82dd399740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x56517ac70300 con 0x56517a9a7840 2024-09-21T03:42:12.962 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:12.955+0000 7f82dd399740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x56517acd9430 con 0x56517a9c8520 2024-09-21T03:42:12.962 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:12.955+0000 7f82dd399740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x56517ab36bb0 con 0x56517a99f660 2024-09-21T03:42:12.962 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:12.955+0000 7f82dd399740 10 client.0 _ll_register_callbacks cb 0x56517ac8a120 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:42:12.962 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:12.955+0000 7f82c7fff640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56517a9a7840 0x56517ad7e450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:12.962 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:12.956+0000 7f82c7fff640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56517a9a7840 0x56517ad7e450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:49792/0 (socket says 172.21.15.89:49792) 2024-09-21T03:42:12.962 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:12.956+0000 7f82c7fff640 1 -- 172.21.15.89:0/2573357197 learned_addr learned my addr 172.21.15.89:0/2573357197 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-21T03:42:12.962 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:12.956+0000 7f82d4bc1640 1 --2- 172.21.15.89:0/2573357197 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56517a99f660 0x56517ad7df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:12.962 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:12.956+0000 7f82d53c2640 1 --2- 172.21.15.89:0/2573357197 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56517a9c8520 0x56517ad7e990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:12.963 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:12.956+0000 7f82c7fff640 1 -- 172.21.15.89:0/2573357197 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56517a99f660 msgr2=0x56517ad7df10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:42:12.963 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:12.956+0000 7f82c7fff640 1 --2- 172.21.15.89:0/2573357197 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56517a99f660 0x56517ad7df10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:12.963 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:12.956+0000 7f82c7fff640 1 -- 172.21.15.89:0/2573357197 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56517a9c8520 msgr2=0x56517ad7e990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:42:12.963 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:12.956+0000 7f82c7fff640 1 --2- 172.21.15.89:0/2573357197 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56517a9c8520 0x56517ad7e990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:12.963 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:12.956+0000 7f82c7fff640 1 -- 172.21.15.89:0/2573357197 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56517ac5f780 con 0x56517a9a7840 2024-09-21T03:42:12.963 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:12.956+0000 7f82c7fff640 1 --2- 172.21.15.89:0/2573357197 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56517a9a7840 0x56517ad7e450 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f82c000bf10 tx=0x7f82c000f5b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ac5b5d54933e1dd server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:12.963 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:12.956+0000 7f82affff640 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2024-09-21T03:42:12.963 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:12.956+0000 7f82affff640 1 -- 172.21.15.89:0/2573357197 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82c0018020 con 0x56517a9a7840 2024-09-21T03:42:12.963 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:12.956+0000 7f82affff640 1 -- 172.21.15.89:0/2573357197 <== mon.0 v2:172.21.15.89:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 9121+0+0 (secure 0 0 0) 0x7f82c0014660 con 0x56517a9a7840 2024-09-21T03:42:12.963 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:12.956+0000 7f82dd399740 1 -- 172.21.15.89:0/2573357197 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f82cc115070 con 0x56517a9a7840 2024-09-21T03:42:12.963 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:12.956+0000 7f82affff640 1 -- 172.21.15.89:0/2573357197 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f82c0023050 con 0x56517a9a7840 2024-09-21T03:42:12.963 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:12.956+0000 7f82dd399740 10 client.15861 fetch_fsmap learned FSMap version 26 2024-09-21T03:42:12.963 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:12.956+0000 7f82dd399740 1 -- 172.21.15.89:0/2573357197 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x56517aa33b00 con 0x56517a9a7840 2024-09-21T03:42:12.963 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:12.957+0000 7f82affff640 1 -- 172.21.15.89:0/2573357197 <== mon.0 v2:172.21.15.89:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f82cc115070 con 0x56517a9a7840 2024-09-21T03:42:12.963 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:12.957+0000 7f82affff640 1 -- 172.21.15.89:0/2573357197 <== mon.0 v2:172.21.15.89:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f82c0013970 con 0x56517a9a7840 2024-09-21T03:42:12.964 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:12.957+0000 7f82dd399740 10 client.15861 fetch_fsmap finished waiting for FSMap version 26 2024-09-21T03:42:12.964 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:12.957+0000 7f82dd399740 10 client.15861 resolve_mds: validated gid 24638 aka daemon mds.b 2024-09-21T03:42:12.964 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:12.957+0000 7f82dd399740 20 client.15861 populate_metadata read hostname 'smithi089' 2024-09-21T03:42:12.964 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:12.957+0000 7f82dd399740 1 --2- 172.21.15.89:0/2573357197 >> [v2:172.21.15.110:6838/132366709,v1:172.21.15.110:6839/132366709] conn(0x56517a986ea0 0x56517ad90050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:12.964 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:12.957+0000 7f82dd399740 4 client.15861 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:42:12.964 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:12.957+0000 7f82dd399740 1 -- 172.21.15.89:0/2573357197 --> [v2:172.21.15.110:6838/132366709,v1:172.21.15.110:6839/132366709] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56517ace8140 con 0x56517a986ea0 2024-09-21T03:42:12.965 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:12.957+0000 7f82d4bc1640 1 --2- 172.21.15.89:0/2573357197 >> [v2:172.21.15.110:6838/132366709,v1:172.21.15.110:6839/132366709] conn(0x56517a986ea0 0x56517ad90050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:12.965 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:12.958+0000 7f82affff640 10 client.15861 ms_handle_connect on v2:172.21.15.110:6838/132366709 2024-09-21T03:42:12.965 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:12.958+0000 7f82d4bc1640 1 --2- 172.21.15.89:0/2573357197 >> [v2:172.21.15.110:6838/132366709,v1:172.21.15.110:6839/132366709] conn(0x56517a986ea0 0x56517ad90050 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=37cf9971dd886d80 server_cookie=a43137a025b3a9dc in_seq=0 out_seq=0 2024-09-21T03:42:12.965 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:12.958+0000 7f82affff640 1 -- 172.21.15.89:0/2573357197 <== mds.0 v2:172.21.15.110:6838/132366709 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x56517ace8140 con 0x56517a986ea0 2024-09-21T03:42:12.965 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:12.958+0000 7f82affff640 10 client.15861 handle_command_reply: tid=0 2024-09-21T03:42:12.965 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:12.959+0000 7f82dd399740 1 client.15861 shutdown 2024-09-21T03:42:12.965 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:12.959+0000 7f82dd399740 1 -- 172.21.15.89:0/2573357197 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56517a9a7840 msgr2=0x56517ad7e450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:12.966 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:12.959+0000 7f82dd399740 1 --2- 172.21.15.89:0/2573357197 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56517a9a7840 0x56517ad7e450 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f82c000bf10 tx=0x7f82c000f5b0 comp rx=0 tx=0).stop 2024-09-21T03:42:12.966 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:12.959+0000 7f82dd399740 1 -- 172.21.15.89:0/2573357197 shutdown_connections 2024-09-21T03:42:12.966 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:12.959+0000 7f82dd399740 1 --2- 172.21.15.89:0/2573357197 >> [v2:172.21.15.110:6838/132366709,v1:172.21.15.110:6839/132366709] conn(0x56517a986ea0 0x56517ad90050 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:12.966 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:12.959+0000 7f82dd399740 1 --2- 172.21.15.89:0/2573357197 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56517a9c8520 0x56517ad7e990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:12.966 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:12.959+0000 7f82dd399740 1 --2- 172.21.15.89:0/2573357197 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56517a9a7840 0x56517ad7e450 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:12.966 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:12.959+0000 7f82dd399740 1 --2- 172.21.15.89:0/2573357197 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56517a99f660 0x56517ad7df10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:12.966 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:12.959+0000 7f82dd399740 1 -- 172.21.15.89:0/2573357197 >> 172.21.15.89:0/2573357197 conn(0x56517a9bc910 msgr2=0x56517ad71660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:42:12.966 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:12.959+0000 7f82affff640 0 client.15861 ms_handle_reset on v2:172.21.15.110:6838/132366709 2024-09-21T03:42:12.966 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:12.959+0000 7f82dd399740 1 -- 172.21.15.89:0/2573357197 shutdown_connections 2024-09-21T03:42:12.966 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:12.959+0000 7f82dd399740 1 -- 172.21.15.89:0/2573357197 wait complete. 2024-09-21T03:42:12.966 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:12.959+0000 7f82dd399740 20 client.15861 trim_cache size 0 max 16384 2024-09-21T03:42:12.979 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:12.976+0000 7f82dd399740 1 Processor -- start 2024-09-21T03:42:12.979 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:12.976+0000 7f82dd399740 1 -- start start 2024-09-21T03:42:12.979 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:12.976+0000 7f82dd399740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56517a99f660 0x56517ad6afa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:12.979 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:12.976+0000 7f82dd399740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56517a9a7840 0x56517ad79610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:12.979 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:12.976+0000 7f82dd399740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56517a9c8520 0x56517ad7fb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:12.979 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:12.976+0000 7f82dd399740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f82d01b1150 con 0x56517a99f660 2024-09-21T03:42:12.979 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:12.976+0000 7f82dd399740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f82d01aea20 con 0x56517a9a7840 2024-09-21T03:42:12.979 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:12.976+0000 7f82dd399740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f82d01ac870 con 0x56517a9c8520 2024-09-21T03:42:12.980 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:12.976+0000 7f82d4bc1640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56517a99f660 0x56517ad6afa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:12.980 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:12.976+0000 7f82d4bc1640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56517a99f660 0x56517ad6afa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:49800/0 (socket says 172.21.15.89:49800) 2024-09-21T03:42:12.980 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:12.976+0000 7f82d4bc1640 1 -- 172.21.15.89:0/920112266 learned_addr learned my addr 172.21.15.89:0/920112266 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-21T03:42:12.980 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:12.976+0000 7f82c7fff640 1 --2- 172.21.15.89:0/920112266 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56517a9a7840 0x56517ad79610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:12.980 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:12.976+0000 7f82d4bc1640 1 -- 172.21.15.89:0/920112266 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56517a9c8520 msgr2=0x56517ad7fb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:42:12.980 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:12.976+0000 7f82d4bc1640 1 --2- 172.21.15.89:0/920112266 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56517a9c8520 0x56517ad7fb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:12.980 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:12.976+0000 7f82d4bc1640 1 -- 172.21.15.89:0/920112266 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56517a9a7840 msgr2=0x56517ad79610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:42:12.980 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:12.976+0000 7f82d4bc1640 1 --2- 172.21.15.89:0/920112266 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56517a9a7840 0x56517ad79610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:12.980 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:12.976+0000 7f82d4bc1640 1 -- 172.21.15.89:0/920112266 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56517aaaa960 con 0x56517a99f660 2024-09-21T03:42:12.980 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:12.977+0000 7f82d4bc1640 1 --2- 172.21.15.89:0/920112266 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56517a99f660 0x56517ad6afa0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f82cc105810 tx=0x7f82cc02a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1bc636887533fb19 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:12.980 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:12.977+0000 7f82b67fc640 1 -- 172.21.15.89:0/920112266 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82cc0efe60 con 0x56517a99f660 2024-09-21T03:42:12.980 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:12.977+0000 7f82b67fc640 1 -- 172.21.15.89:0/920112266 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f82cc115040 con 0x56517a99f660 2024-09-21T03:42:12.980 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:12.977+0000 7f82b67fc640 1 -- 172.21.15.89:0/920112266 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82cc0cf840 con 0x56517a99f660 2024-09-21T03:42:12.983 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:12.978+0000 7f82dd399740 1 -- 172.21.15.89:0/920112266 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56517a99f660 msgr2=0x56517ad6afa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:12.983 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:12.978+0000 7f82dd399740 1 --2- 172.21.15.89:0/920112266 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56517a99f660 0x56517ad6afa0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f82cc105810 tx=0x7f82cc02a410 comp rx=0 tx=0).stop 2024-09-21T03:42:12.983 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:12.978+0000 7f82dd399740 1 -- 172.21.15.89:0/920112266 shutdown_connections 2024-09-21T03:42:12.983 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:12.978+0000 7f82dd399740 1 --2- 172.21.15.89:0/920112266 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56517a9a7840 0x56517ad79610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:12.983 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:12.978+0000 7f82dd399740 1 --2- 172.21.15.89:0/920112266 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56517a9c8520 0x56517ad7fb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:12.984 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:12.978+0000 7f82dd399740 1 --2- 172.21.15.89:0/920112266 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56517a99f660 0x56517ad6afa0 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:12.984 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:12.978+0000 7f82dd399740 1 -- 172.21.15.89:0/920112266 >> 172.21.15.89:0/920112266 conn(0x56517a9bc910 msgr2=0x56517ad7f430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:42:12.984 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:12.978+0000 7f82dd399740 1 -- 172.21.15.89:0/920112266 shutdown_connections 2024-09-21T03:42:12.984 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:12.978+0000 7f82dd399740 1 -- 172.21.15.89:0/920112266 wait complete. 2024-09-21T03:42:12.984 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:12.978+0000 7f82dd399740 1 -- start start 2024-09-21T03:42:12.984 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:12.978+0000 7f82dd399740 1 Processor -- start 2024-09-21T03:42:12.984 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:12.979+0000 7f82dd399740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56517a99f660 0x56517ad86150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:12.984 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:12.979+0000 7f82dd399740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56517a9a7840 0x56517ad86730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:12.984 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:12.979+0000 7f82dd399740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56517a9c8520 0x56517ad80e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:12.984 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:12.979+0000 7f82dd399740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f82d01acee0 con 0x56517a9c8520 2024-09-21T03:42:12.984 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:12.979+0000 7f82dd399740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f82d01ae350 con 0x56517a9a7840 2024-09-21T03:42:12.984 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:12.979+0000 7f82dd399740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f82d01ad5b0 con 0x56517a99f660 2024-09-21T03:42:12.984 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:12.979+0000 7f82dd399740 10 client.0 _ll_register_callbacks cb 0x7f82c000b2a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:42:12.984 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:12.979+0000 7f82d53c2640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56517a9c8520 0x56517ad80e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:12.985 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:12.979+0000 7f82d53c2640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56517a9c8520 0x56517ad80e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:49810/0 (socket says 172.21.15.89:49810) 2024-09-21T03:42:12.985 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:12.979+0000 7f82d53c2640 1 -- 172.21.15.89:0/1866737905 learned_addr learned my addr 172.21.15.89:0/1866737905 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-21T03:42:12.985 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:12.979+0000 7f82c7fff640 1 --2- 172.21.15.89:0/1866737905 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56517a9a7840 0x56517ad86730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:12.985 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:12.979+0000 7f82d4bc1640 1 --2- 172.21.15.89:0/1866737905 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56517a99f660 0x56517ad86150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:12.985 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:12.979+0000 7f82d53c2640 1 -- 172.21.15.89:0/1866737905 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56517a99f660 msgr2=0x56517ad86150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:42:12.985 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:12.979+0000 7f82d53c2640 1 --2- 172.21.15.89:0/1866737905 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56517a99f660 0x56517ad86150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:12.985 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:12.979+0000 7f82d53c2640 1 -- 172.21.15.89:0/1866737905 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56517a9a7840 msgr2=0x56517ad86730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:42:12.985 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:12.979+0000 7f82d53c2640 1 --2- 172.21.15.89:0/1866737905 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56517a9a7840 0x56517ad86730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:12.985 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:12.979+0000 7f82d53c2640 1 -- 172.21.15.89:0/1866737905 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56517ad81620 con 0x56517a9c8520 2024-09-21T03:42:12.985 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:12.979+0000 7f82d53c2640 1 --2- 172.21.15.89:0/1866737905 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56517a9c8520 0x56517ad80e90 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f82c801dd40 tx=0x7f82c8004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d9f6c182466289f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:12.985 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:12.979+0000 7f82affff640 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2024-09-21T03:42:12.985 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:12.979+0000 7f82affff640 1 -- 172.21.15.89:0/1866737905 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82c800ed40 con 0x56517a9c8520 2024-09-21T03:42:12.985 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:12.979+0000 7f82affff640 1 -- 172.21.15.89:0/1866737905 <== mon.0 v2:172.21.15.89:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 9121+0+0 (secure 0 0 0) 0x7f82c801e400 con 0x56517a9c8520 2024-09-21T03:42:12.985 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:12.979+0000 7f82dd399740 1 -- 172.21.15.89:0/1866737905 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f82cc115040 con 0x56517a9c8520 2024-09-21T03:42:12.986 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:12.980+0000 7f82affff640 1 -- 172.21.15.89:0/1866737905 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f82c801e600 con 0x56517a9c8520 2024-09-21T03:42:12.986 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:12.980+0000 7f82dd399740 10 client.15867 fetch_fsmap learned FSMap version 26 2024-09-21T03:42:12.986 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:12.980+0000 7f82dd399740 1 -- 172.21.15.89:0/1866737905 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x56517adfb540 con 0x56517a9c8520 2024-09-21T03:42:12.986 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:12.980+0000 7f82affff640 1 -- 172.21.15.89:0/1866737905 <== mon.0 v2:172.21.15.89:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f82cc115040 con 0x56517a9c8520 2024-09-21T03:42:12.986 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:12.980+0000 7f82affff640 1 -- 172.21.15.89:0/1866737905 <== mon.0 v2:172.21.15.89:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f82c80277d0 con 0x56517a9c8520 2024-09-21T03:42:12.986 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:12.980+0000 7f82dd399740 10 client.15867 fetch_fsmap finished waiting for FSMap version 26 2024-09-21T03:42:12.986 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:12.980+0000 7f82dd399740 10 client.15867 resolve_mds: validated gid 24638 aka daemon mds.b 2024-09-21T03:42:12.986 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:12.980+0000 7f82dd399740 20 client.15867 populate_metadata read hostname 'smithi089' 2024-09-21T03:42:12.986 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:12.980+0000 7f82dd399740 1 --2- 172.21.15.89:0/1866737905 >> [v2:172.21.15.110:6838/132366709,v1:172.21.15.110:6839/132366709] conn(0x56517a95c250 0x56517adfddb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:12.986 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:12.980+0000 7f82dd399740 4 client.15867 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "6404c199", "release": true, "await": true}] 2024-09-21T03:42:12.986 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:12.980+0000 7f82dd399740 1 -- 172.21.15.89:0/1866737905 --> [v2:172.21.15.110:6838/132366709,v1:172.21.15.110:6839/132366709] -- command(tid 0: {"prefix": "quiesce db", "set_id": "6404c199", "release": true, "await": true}) -- 0x56517ace3bf0 con 0x56517a95c250 2024-09-21T03:42:12.986 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:12.980+0000 7f82d4bc1640 1 --2- 172.21.15.89:0/1866737905 >> [v2:172.21.15.110:6838/132366709,v1:172.21.15.110:6839/132366709] conn(0x56517a95c250 0x56517adfddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:12.986 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:12.981+0000 7f82affff640 10 client.15867 ms_handle_connect on v2:172.21.15.110:6838/132366709 2024-09-21T03:42:12.986 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:12.981+0000 7f82d4bc1640 1 --2- 172.21.15.89:0/1866737905 >> [v2:172.21.15.110:6838/132366709,v1:172.21.15.110:6839/132366709] conn(0x56517a95c250 0x56517adfddb0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=297980e22a13952b server_cookie=4166dfd38c473752 in_seq=0 out_seq=0 2024-09-21T03:42:12.988 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:12.983+0000 7f82affff640 1 -- 172.21.15.89:0/1866737905 <== mds.0 v2:172.21.15.110:6838/132366709 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x56517ace3bf0 con 0x56517a95c250 2024-09-21T03:42:12.988 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:12.983+0000 7f82affff640 10 client.15867 handle_command_reply: tid=0 2024-09-21T03:42:12.988 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:12.983+0000 7f82dd399740 1 client.15867 shutdown 2024-09-21T03:42:12.988 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:12.983+0000 7f82dd399740 1 -- 172.21.15.89:0/1866737905 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56517a9c8520 msgr2=0x56517ad80e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:12.988 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:12.983+0000 7f82dd399740 1 --2- 172.21.15.89:0/1866737905 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56517a9c8520 0x56517ad80e90 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f82c801dd40 tx=0x7f82c8004ab0 comp rx=0 tx=0).stop 2024-09-21T03:42:12.988 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:12.983+0000 7f82dd399740 1 -- 172.21.15.89:0/1866737905 shutdown_connections 2024-09-21T03:42:12.988 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:12.983+0000 7f82dd399740 1 --2- 172.21.15.89:0/1866737905 >> [v2:172.21.15.110:6838/132366709,v1:172.21.15.110:6839/132366709] conn(0x56517a95c250 0x56517adfddb0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:12.988 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:12.983+0000 7f82dd399740 1 --2- 172.21.15.89:0/1866737905 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56517a9a7840 0x56517ad86730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:12.988 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:12.983+0000 7f82dd399740 1 --2- 172.21.15.89:0/1866737905 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56517a9c8520 0x56517ad80e90 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:12.988 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:12.983+0000 7f82affff640 0 client.15867 ms_handle_reset on v2:172.21.15.110:6838/132366709 2024-09-21T03:42:12.988 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:12.983+0000 7f82dd399740 1 --2- 172.21.15.89:0/1866737905 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56517a99f660 0x56517ad86150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:12.988 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:12.983+0000 7f82dd399740 1 -- 172.21.15.89:0/1866737905 >> 172.21.15.89:0/1866737905 conn(0x56517a9bc910 msgr2=0x56517ad793e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:42:12.988 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:12.983+0000 7f82dd399740 1 -- 172.21.15.89:0/1866737905 shutdown_connections 2024-09-21T03:42:12.988 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:12.983+0000 7f82dd399740 1 -- 172.21.15.89:0/1866737905 wait complete. 2024-09-21T03:42:12.988 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:12.983+0000 7f82dd399740 20 client.15867 trim_cache size 0 max 16384 2024-09-21T03:42:12.989 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-09-21T03:42:12.989 INFO:teuthology.orchestra.run.smithi089.stdout: "epoch": 26, 2024-09-21T03:42:12.989 INFO:teuthology.orchestra.run.smithi089.stdout: "leader": 24638, 2024-09-21T03:42:12.989 INFO:teuthology.orchestra.run.smithi089.stdout: "set_version": 12, 2024-09-21T03:42:12.989 INFO:teuthology.orchestra.run.smithi089.stdout: "sets": { 2024-09-21T03:42:12.989 INFO:teuthology.orchestra.run.smithi089.stdout: "6404c199": { 2024-09-21T03:42:12.989 INFO:teuthology.orchestra.run.smithi089.stdout: "version": 12, 2024-09-21T03:42:12.989 INFO:teuthology.orchestra.run.smithi089.stdout: "age_ref": 0.0, 2024-09-21T03:42:12.989 INFO:teuthology.orchestra.run.smithi089.stdout: "state": { 2024-09-21T03:42:12.989 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "RELEASED", 2024-09-21T03:42:12.990 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 0.0 2024-09-21T03:42:12.990 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:12.990 INFO:teuthology.orchestra.run.smithi089.stdout: "timeout": 90.0, 2024-09-21T03:42:12.990 INFO:teuthology.orchestra.run.smithi089.stdout: "expiration": 147.8, 2024-09-21T03:42:12.990 INFO:teuthology.orchestra.run.smithi089.stdout: "members": { 2024-09-21T03:42:12.990 INFO:teuthology.orchestra.run.smithi089.stdout: "file:/": { 2024-09-21T03:42:12.990 INFO:teuthology.orchestra.run.smithi089.stdout: "excluded": false, 2024-09-21T03:42:12.990 INFO:teuthology.orchestra.run.smithi089.stdout: "state": { 2024-09-21T03:42:12.990 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "RELEASED", 2024-09-21T03:42:12.990 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 0.0 2024-09-21T03:42:12.990 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-21T03:42:12.990 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-21T03:42:12.990 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-21T03:42:12.990 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-21T03:42:12.990 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-21T03:42:12.991 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-09-21T03:42:12.991 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:12.984+0000 7f82d6e25640 1 -- 172.21.15.89:0/3266654704 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f82b80b6590 msgr2=0x7f82b80b8aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:12.991 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:12.984+0000 7f82d6e25640 1 --2- 172.21.15.89:0/3266654704 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f82b80b6590 0x7f82b80b8aa0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f82c8005fd0 tx=0x7f82c8008630 comp rx=0 tx=0).stop 2024-09-21T03:42:12.991 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:12.984+0000 7f82d6e25640 1 -- 172.21.15.89:0/3266654704 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f82d0102a80 msgr2=0x7f82d019a450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:12.991 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:12.984+0000 7f82d6e25640 1 --2- 172.21.15.89:0/3266654704 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f82d0102a80 0x7f82d019a450 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f82cc009a20 tx=0x7f82cc004270 comp rx=0 tx=0).stop 2024-09-21T03:42:12.992 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:12.984+0000 7f82d6e25640 1 -- 172.21.15.89:0/3266654704 shutdown_connections 2024-09-21T03:42:12.992 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:12.984+0000 7f82d6e25640 1 --2- 172.21.15.89:0/3266654704 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f82b80b6590 0x7f82b80b8aa0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:12.992 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:12.984+0000 7f82d6e25640 1 --2- 172.21.15.89:0/3266654704 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f82d0108d10 0x7f82d019fc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:12.992 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:12.984+0000 7f82d6e25640 1 --2- 172.21.15.89:0/3266654704 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f82d01083c0 0x7f82d019a9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:12.992 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:12.984+0000 7f82d6e25640 1 --2- 172.21.15.89:0/3266654704 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f82d0102a80 0x7f82d019a450 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:12.992 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:12.984+0000 7f82d6e25640 1 -- 172.21.15.89:0/3266654704 >> 172.21.15.89:0/3266654704 conn(0x7f82d0079740 msgr2=0x7f82d0101ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:42:12.992 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:12.984+0000 7f82d6e25640 1 -- 172.21.15.89:0/3266654704 shutdown_connections 2024-09-21T03:42:12.992 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:12.984+0000 7f82d6e25640 1 -- 172.21.15.89:0/3266654704 wait complete. 2024-09-21T03:42:12.996 INFO:tasks.workunit.client.0.smithi089.stdout:setting up random write file 2024-09-21T03:42:12.997 INFO:tasks.workunit.client.0.smithi089.stdout:done setting up random write file 2024-09-21T03:42:12.997 INFO:tasks.workunit.client.0.smithi089.stdout:starting fsync run 2024-09-21T03:42:12.997 INFO:tasks.workunit.client.0.smithi089.stdout:write time: 0.0045s fsync time: 0.0059s 2024-09-21T03:42:12.997 INFO:tasks.workunit.client.0.smithi089.stdout:write time: 0.0028s fsync time: 0.0413s 2024-09-21T03:42:12.997 INFO:tasks.workunit.client.0.smithi089.stdout:write time: 0.0033s fsync time: 0.0146s 2024-09-21T03:42:12.997 INFO:tasks.workunit.client.0.smithi089.stdout:write time: 0.0024s fsync time: 0.0100s 2024-09-21T03:42:12.997 INFO:tasks.workunit.client.0.smithi089.stdout:write time: 0.0045s fsync time: 0.0098s 2024-09-21T03:42:12.997 INFO:tasks.workunit.client.0.smithi089.stdout:write time: 0.0027s fsync time: 0.0112s 2024-09-21T03:42:12.997 INFO:tasks.workunit.client.0.smithi089.stdout:write time: 0.0030s fsync time: 0.0182s 2024-09-21T03:42:12.997 INFO:tasks.workunit.client.0.smithi089.stdout:write time: 0.0033s fsync time: 0.0128s 2024-09-21T03:42:12.997 INFO:tasks.workunit.client.0.smithi089.stdout:write time: 0.0030s fsync time: 0.0097s 2024-09-21T03:42:12.997 INFO:tasks.workunit.client.0.smithi089.stdout:write time: 0.0030s fsync time: 0.0279s 2024-09-21T03:42:12.997 INFO:tasks.workunit.client.0.smithi089.stdout:write time: 0.0032s fsync time: 0.0140s 2024-09-21T03:42:12.997 INFO:tasks.workunit.client.0.smithi089.stdout:write time: 0.0031s fsync time: 0.0070s 2024-09-21T03:42:12.997 INFO:tasks.workunit.client.0.smithi089.stdout:write time: 0.0027s fsync time: 0.0272s 2024-09-21T03:42:12.997 INFO:tasks.workunit.client.0.smithi089.stdout:write time: 0.0031s fsync time: 0.0821s 2024-09-21T03:42:12.998 INFO:tasks.workunit.client.0.smithi089.stdout:write time: 0.0028s fsync time: 0.0614s 2024-09-21T03:42:12.998 INFO:tasks.workunit.client.0.smithi089.stdout:write time: 0.0031s fsync time: 0.0116s 2024-09-21T03:42:12.998 INFO:tasks.workunit.client.0.smithi089.stdout:write time: 0.0041s fsync time: 0.0280s 2024-09-21T03:42:12.998 INFO:tasks.workunit.client.0.smithi089.stdout:write time: 0.0053s fsync time: 0.0477s 2024-09-21T03:42:12.998 INFO:tasks.workunit.client.0.smithi089.stdout:write time: 0.0062s fsync time: 0.0674s 2024-09-21T03:42:12.998 INFO:tasks.workunit.client.0.smithi089.stdout:write time: 0.0049s fsync time: 0.0173s 2024-09-21T03:42:12.998 INFO:tasks.workunit.client.0.smithi089.stdout:write time: 0.0041s fsync time: 0.0153s 2024-09-21T03:42:12.998 INFO:tasks.workunit.client.0.smithi089.stdout:write time: 0.0029s fsync time: 0.0539s 2024-09-21T03:42:12.998 INFO:tasks.workunit.client.0.smithi089.stdout:write time: 0.0028s fsync time: 0.0323s 2024-09-21T03:42:12.998 INFO:tasks.workunit.client.0.smithi089.stdout:write time: 0.0100s fsync time: 0.0195s 2024-09-21T03:42:12.998 INFO:tasks.workunit.client.0.smithi089.stdout:write time: 0.0034s fsync time: 0.0118s 2024-09-21T03:42:12.998 INFO:tasks.workunit.client.0.smithi089.stdout:write time: 0.0048s fsync time: 0.0570s 2024-09-21T03:42:12.998 INFO:tasks.workunit.client.0.smithi089.stdout:write time: 0.0034s fsync time: 0.1526s 2024-09-21T03:42:12.998 INFO:tasks.workunit.client.0.smithi089.stdout:write time: 0.0051s fsync time: 0.0108s 2024-09-21T03:42:12.999 INFO:tasks.workunit.client.0.smithi089.stdout:write time: 0.0052s fsync time: 0.0385s 2024-09-21T03:42:12.999 INFO:tasks.workunit.client.0.smithi089.stdout:write time: 0.0131s fsync time: 0.0335s 2024-09-21T03:42:12.999 INFO:tasks.workunit.client.0.smithi089.stdout:write time: 0.0041s fsync time: 0.0100s 2024-09-21T03:42:12.999 INFO:tasks.workunit.client.0.smithi089.stdout:write time: 0.0033s fsync time: 0.0604s 2024-09-21T03:42:12.999 INFO:tasks.workunit.client.0.smithi089.stdout:write time: 0.0034s fsync time: 0.0216s 2024-09-21T03:42:12.999 INFO:tasks.workunit.client.0.smithi089.stdout:write time: 0.0791s fsync time: 0.0278s 2024-09-21T03:42:12.999 INFO:tasks.workunit.client.0.smithi089.stdout:write time: 0.0028s fsync time: 0.0341s 2024-09-21T03:42:12.999 INFO:tasks.workunit.client.0.smithi089.stdout:write time: 0.0029s fsync time: 0.2811s 2024-09-21T03:42:12.999 INFO:tasks.workunit.client.0.smithi089.stdout:write time: 0.0054s fsync time: 0.0175s 2024-09-21T03:42:12.999 INFO:tasks.workunit.client.0.smithi089.stdout:write time: 0.0037s fsync time: 0.0612s 2024-09-21T03:42:12.999 INFO:tasks.workunit.client.0.smithi089.stdout:write time: 0.1470s fsync time: 0.0741s 2024-09-21T03:42:12.999 INFO:tasks.workunit.client.0.smithi089.stdout:write time: 0.0182s fsync time: 0.0278s 2024-09-21T03:42:12.999 INFO:tasks.workunit.client.0.smithi089.stdout:write time: 0.0031s fsync time: 0.0880s 2024-09-21T03:42:12.999 INFO:tasks.workunit.client.0.smithi089.stdout:write time: 0.0027s fsync time: 0.1540s 2024-09-21T03:42:13.000 INFO:tasks.workunit.client.0.smithi089.stdout:write time: 0.0050s fsync time: 0.0377s 2024-09-21T03:42:13.000 INFO:tasks.workunit.client.0.smithi089.stdout:write time: 0.0085s fsync time: 0.0317s 2024-09-21T03:42:13.000 INFO:tasks.workunit.client.0.smithi089.stdout:write time: 0.0026s fsync time: 0.0110s 2024-09-21T03:42:13.000 INFO:tasks.workunit.client.0.smithi089.stdout:write time: 0.0043s fsync time: 0.0299s 2024-09-21T03:42:13.000 INFO:tasks.workunit.client.0.smithi089.stdout:write time: 0.0179s fsync time: 0.0368s 2024-09-21T03:42:13.000 INFO:tasks.workunit.client.0.smithi089.stdout:write time: 0.0047s fsync time: 0.0383s 2024-09-21T03:42:13.000 INFO:tasks.workunit.client.0.smithi089.stdout:write time: 0.0108s fsync time: 0.0558s 2024-09-21T03:42:13.000 INFO:tasks.workunit.client.0.smithi089.stdout:write time: 0.0087s fsync time: 0.0297s 2024-09-21T03:42:13.000 INFO:tasks.workunit.client.0.smithi089.stdout:write time: 12.6362s fsync time: 0.0082s 2024-09-21T03:42:13.000 INFO:tasks.workunit.client.0.smithi089.stdout:run done 51 fsyncs total, killing random writer 2024-09-21T03:42:13.003 INFO:tasks.workunit.client.0.smithi089.stdout:/home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin 2024-09-21T03:42:13.003 INFO:tasks.workunit.client.0.smithi089.stderr:+ echo /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin 2024-09-21T03:42:13.003 INFO:tasks.workunit.client.0.smithi089.stderr:+ whereis lsof 2024-09-21T03:42:13.134 INFO:tasks.workunit.client.0.smithi089.stdout:lsof: /usr/bin/lsof /usr/share/man/man1/lsof.1.gz 2024-09-21T03:42:13.134 INFO:tasks.workunit.client.0.smithi089.stderr:+ lsof 2024-09-21T03:42:13.212 INFO:tasks.workunit.client.0.smithi089.stderr:lsof: WARNING: can't stat() overlay file system /var/lib/containers/storage/overlay/0b1266360c1e947b91c951aa5a770f203724c9783f3817320d7b3f833fdd3623/merged 2024-09-21T03:42:13.212 INFO:tasks.workunit.client.0.smithi089.stderr: Output information may be incomplete. 2024-09-21T03:42:13.212 INFO:tasks.workunit.client.0.smithi089.stderr:lsof: WARNING: can't stat() overlay file system /var/lib/containers/storage/overlay/74f483b2de23aa3db5ef9fda5e7c9ec9d9869ed0e09e152a27d160b31080f0d7/merged 2024-09-21T03:42:13.212 INFO:tasks.workunit.client.0.smithi089.stderr: Output information may be incomplete. 2024-09-21T03:42:13.212 INFO:tasks.workunit.client.0.smithi089.stderr:lsof: WARNING: can't stat() overlay file system /var/lib/containers/storage/overlay/0c33d786533fa7c6c9b4ece0dd8c3bfc33ee457df01a35b7a874175fb0d24091/merged 2024-09-21T03:42:13.212 INFO:tasks.workunit.client.0.smithi089.stderr: Output information may be incomplete. 2024-09-21T03:42:13.212 INFO:tasks.workunit.client.0.smithi089.stderr:lsof: WARNING: can't stat() overlay file system /var/lib/containers/storage/overlay/1e263b0a542f58eaf76d73e377dd5959e47135a86a85f6eb5eae463c0822e8c2/merged 2024-09-21T03:42:13.212 INFO:tasks.workunit.client.0.smithi089.stderr: Output information may be incomplete. 2024-09-21T03:42:13.212 INFO:tasks.workunit.client.0.smithi089.stderr:lsof: WARNING: can't stat() overlay file system /var/lib/containers/storage/overlay/2f136aa314ce51b9e309ae462848aa731dc41b3b9ee884ab006901dd9af81ad8/merged 2024-09-21T03:42:13.212 INFO:tasks.workunit.client.0.smithi089.stderr: Output information may be incomplete. 2024-09-21T03:42:13.212 INFO:tasks.workunit.client.0.smithi089.stderr:lsof: WARNING: can't stat() overlay file system /var/lib/containers/storage/overlay/64f7a59823e11e80a119f6f784180bf56bca738c73a5be4e7e05673bb3400150/merged 2024-09-21T03:42:13.212 INFO:tasks.workunit.client.0.smithi089.stderr: Output information may be incomplete. 2024-09-21T03:42:13.212 INFO:tasks.workunit.client.0.smithi089.stderr:lsof: WARNING: can't stat() overlay file system /var/lib/containers/storage/overlay/cc0f955263cae7210080df6836f384f710eb6af21a08203f2d154f023c022d18/merged 2024-09-21T03:42:13.213 INFO:tasks.workunit.client.0.smithi089.stderr: Output information may be incomplete. 2024-09-21T03:42:13.213 INFO:tasks.workunit.client.0.smithi089.stderr:lsof: WARNING: can't stat() overlay file system /var/lib/containers/storage/overlay/9a935757ca5b5da6d9a52aaebf493ea1b50d289d444a1fc297330740dbd3169d/merged 2024-09-21T03:42:13.213 INFO:tasks.workunit.client.0.smithi089.stderr: Output information may be incomplete. 2024-09-21T03:42:13.213 INFO:tasks.workunit.client.0.smithi089.stderr:lsof: WARNING: can't stat() overlay file system /var/lib/containers/storage/overlay/050ed91c932a05cf2dc3c502aa3bf5b8822fb5b205b8284f46e11cf4638c07a2/merged 2024-09-21T03:42:13.213 INFO:tasks.workunit.client.0.smithi089.stderr: Output information may be incomplete. 2024-09-21T03:42:13.213 INFO:tasks.workunit.client.0.smithi089.stderr:lsof: WARNING: can't stat() overlay file system /var/lib/containers/storage/overlay/34ca3c8ad8cdf365681f5fde2e944c6107cead8c5aea4405ffe1ac85eb695e9c/merged 2024-09-21T03:42:13.213 INFO:tasks.workunit.client.0.smithi089.stderr: Output information may be incomplete. 2024-09-21T03:42:13.213 INFO:tasks.workunit.client.0.smithi089.stderr:lsof: WARNING: can't stat() overlay file system /var/lib/containers/storage/overlay/cf106208cc3b85bdd2560a4ba838d689b53b71ef165f349ef52b54ff452dc9a0/merged 2024-09-21T03:42:13.213 INFO:tasks.workunit.client.0.smithi089.stderr: Output information may be incomplete. 2024-09-21T03:42:13.213 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:42:12 smithi114 ceph-mon[34140]: pgmap v444: 129 pgs: 129 active+clean; 258 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 2.4 MiB/s wr, 560 op/s 2024-09-21T03:42:13.250 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:42:12 smithi110 ceph-mon[34000]: pgmap v444: 129 pgs: 129 active+clean; 258 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 2.4 MiB/s wr, 560 op/s 2024-09-21T03:42:13.297 INFO:tasks.workunit.client.0.smithi089.stdout:COMMAND PID TID TASKCMD USER FD TYPE DEVICE SIZE/OFF NODE NAME 2024-09-21T03:42:13.298 INFO:tasks.workunit.client.0.smithi089.stdout:systemd 1 root cwd unknown /proc/1/cwd (readlink: Permission denied) 2024-09-21T03:42:13.298 INFO:tasks.workunit.client.0.smithi089.stdout:systemd 1 root rtd unknown /proc/1/root (readlink: Permission denied) 2024-09-21T03:42:13.298 INFO:tasks.workunit.client.0.smithi089.stdout:systemd 1 root txt unknown /proc/1/exe (readlink: Permission denied) 2024-09-21T03:42:13.298 INFO:tasks.workunit.client.0.smithi089.stdout:systemd 1 root NOFD /proc/1/fd (opendir: Permission denied) 2024-09-21T03:42:13.298 INFO:tasks.workunit.client.0.smithi089.stdout:kthreadd 2 root cwd unknown /proc/2/cwd (readlink: Permission denied) 2024-09-21T03:42:13.298 INFO:tasks.workunit.client.0.smithi089.stdout:kthreadd 2 root rtd unknown /proc/2/root (readlink: Permission denied) 2024-09-21T03:42:13.298 INFO:tasks.workunit.client.0.smithi089.stdout:kthreadd 2 root txt unknown /proc/2/exe (readlink: Permission denied) 2024-09-21T03:42:13.298 INFO:tasks.workunit.client.0.smithi089.stdout:kthreadd 2 root NOFD /proc/2/fd (opendir: Permission denied) 2024-09-21T03:42:13.298 INFO:tasks.workunit.client.0.smithi089.stdout:pool_work 3 root cwd unknown /proc/3/cwd (readlink: Permission denied) 2024-09-21T03:42:13.298 INFO:tasks.workunit.client.0.smithi089.stdout:pool_work 3 root rtd unknown /proc/3/root (readlink: Permission denied) 2024-09-21T03:42:13.298 INFO:tasks.workunit.client.0.smithi089.stdout:pool_work 3 root txt unknown /proc/3/exe (readlink: Permission denied) 2024-09-21T03:42:13.298 INFO:tasks.workunit.client.0.smithi089.stdout:pool_work 3 root NOFD /proc/3/fd (opendir: Permission denied) 2024-09-21T03:42:13.299 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/R 4 root cwd unknown /proc/4/cwd (readlink: Permission denied) 2024-09-21T03:42:13.299 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/R 4 root rtd unknown /proc/4/root (readlink: Permission denied) 2024-09-21T03:42:13.299 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/R 4 root txt unknown /proc/4/exe (readlink: Permission denied) 2024-09-21T03:42:13.299 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/R 4 root NOFD /proc/4/fd (opendir: Permission denied) 2024-09-21T03:42:13.299 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/R 5 root cwd unknown /proc/5/cwd (readlink: Permission denied) 2024-09-21T03:42:13.299 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/R 5 root rtd unknown /proc/5/root (readlink: Permission denied) 2024-09-21T03:42:13.299 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/R 5 root txt unknown /proc/5/exe (readlink: Permission denied) 2024-09-21T03:42:13.299 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/R 5 root NOFD /proc/5/fd (opendir: Permission denied) 2024-09-21T03:42:13.299 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/R 6 root cwd unknown /proc/6/cwd (readlink: Permission denied) 2024-09-21T03:42:13.299 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/R 6 root rtd unknown /proc/6/root (readlink: Permission denied) 2024-09-21T03:42:13.299 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/R 6 root txt unknown /proc/6/exe (readlink: Permission denied) 2024-09-21T03:42:13.299 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/R 6 root NOFD /proc/6/fd (opendir: Permission denied) 2024-09-21T03:42:13.299 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/R 7 root cwd unknown /proc/7/cwd (readlink: Permission denied) 2024-09-21T03:42:13.299 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/R 7 root rtd unknown /proc/7/root (readlink: Permission denied) 2024-09-21T03:42:13.300 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/R 7 root txt unknown /proc/7/exe (readlink: Permission denied) 2024-09-21T03:42:13.300 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/R 7 root NOFD /proc/7/fd (opendir: Permission denied) 2024-09-21T03:42:13.300 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/0 9 root cwd unknown /proc/9/cwd (readlink: Permission denied) 2024-09-21T03:42:13.300 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/0 9 root rtd unknown /proc/9/root (readlink: Permission denied) 2024-09-21T03:42:13.300 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/0 9 root txt unknown /proc/9/exe (readlink: Permission denied) 2024-09-21T03:42:13.300 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/0 9 root NOFD /proc/9/fd (opendir: Permission denied) 2024-09-21T03:42:13.300 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/R 11 root cwd unknown /proc/11/cwd (readlink: Permission denied) 2024-09-21T03:42:13.300 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/R 11 root rtd unknown /proc/11/root (readlink: Permission denied) 2024-09-21T03:42:13.300 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/R 11 root txt unknown /proc/11/exe (readlink: Permission denied) 2024-09-21T03:42:13.300 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/R 11 root NOFD /proc/11/fd (opendir: Permission denied) 2024-09-21T03:42:13.300 INFO:tasks.workunit.client.0.smithi089.stdout:rcu_tasks 13 root cwd unknown /proc/13/cwd (readlink: Permission denied) 2024-09-21T03:42:13.300 INFO:tasks.workunit.client.0.smithi089.stdout:rcu_tasks 13 root rtd unknown /proc/13/root (readlink: Permission denied) 2024-09-21T03:42:13.300 INFO:tasks.workunit.client.0.smithi089.stdout:rcu_tasks 13 root txt unknown /proc/13/exe (readlink: Permission denied) 2024-09-21T03:42:13.300 INFO:tasks.workunit.client.0.smithi089.stdout:rcu_tasks 13 root NOFD /proc/13/fd (opendir: Permission denied) 2024-09-21T03:42:13.301 INFO:tasks.workunit.client.0.smithi089.stdout:rcu_tasks 14 root cwd unknown /proc/14/cwd (readlink: Permission denied) 2024-09-21T03:42:13.301 INFO:tasks.workunit.client.0.smithi089.stdout:rcu_tasks 14 root rtd unknown /proc/14/root (readlink: Permission denied) 2024-09-21T03:42:13.301 INFO:tasks.workunit.client.0.smithi089.stdout:rcu_tasks 14 root txt unknown /proc/14/exe (readlink: Permission denied) 2024-09-21T03:42:13.301 INFO:tasks.workunit.client.0.smithi089.stdout:rcu_tasks 14 root NOFD /proc/14/fd (opendir: Permission denied) 2024-09-21T03:42:13.301 INFO:tasks.workunit.client.0.smithi089.stdout:rcu_tasks 15 root cwd unknown /proc/15/cwd (readlink: Permission denied) 2024-09-21T03:42:13.301 INFO:tasks.workunit.client.0.smithi089.stdout:rcu_tasks 15 root rtd unknown /proc/15/root (readlink: Permission denied) 2024-09-21T03:42:13.301 INFO:tasks.workunit.client.0.smithi089.stdout:rcu_tasks 15 root txt unknown /proc/15/exe (readlink: Permission denied) 2024-09-21T03:42:13.301 INFO:tasks.workunit.client.0.smithi089.stdout:rcu_tasks 15 root NOFD /proc/15/fd (opendir: Permission denied) 2024-09-21T03:42:13.301 INFO:tasks.workunit.client.0.smithi089.stdout:ksoftirqd 16 root cwd unknown /proc/16/cwd (readlink: Permission denied) 2024-09-21T03:42:13.301 INFO:tasks.workunit.client.0.smithi089.stdout:ksoftirqd 16 root rtd unknown /proc/16/root (readlink: Permission denied) 2024-09-21T03:42:13.301 INFO:tasks.workunit.client.0.smithi089.stdout:ksoftirqd 16 root txt unknown /proc/16/exe (readlink: Permission denied) 2024-09-21T03:42:13.301 INFO:tasks.workunit.client.0.smithi089.stdout:ksoftirqd 16 root NOFD /proc/16/fd (opendir: Permission denied) 2024-09-21T03:42:13.301 INFO:tasks.workunit.client.0.smithi089.stdout:pr/ttyS1 17 root cwd unknown /proc/17/cwd (readlink: Permission denied) 2024-09-21T03:42:13.301 INFO:tasks.workunit.client.0.smithi089.stdout:pr/ttyS1 17 root rtd unknown /proc/17/root (readlink: Permission denied) 2024-09-21T03:42:13.302 INFO:tasks.workunit.client.0.smithi089.stdout:pr/ttyS1 17 root txt unknown /proc/17/exe (readlink: Permission denied) 2024-09-21T03:42:13.302 INFO:tasks.workunit.client.0.smithi089.stdout:pr/ttyS1 17 root NOFD /proc/17/fd (opendir: Permission denied) 2024-09-21T03:42:13.302 INFO:tasks.workunit.client.0.smithi089.stdout:rcu_preem 18 root cwd unknown /proc/18/cwd (readlink: Permission denied) 2024-09-21T03:42:13.302 INFO:tasks.workunit.client.0.smithi089.stdout:rcu_preem 18 root rtd unknown /proc/18/root (readlink: Permission denied) 2024-09-21T03:42:13.302 INFO:tasks.workunit.client.0.smithi089.stdout:rcu_preem 18 root txt unknown /proc/18/exe (readlink: Permission denied) 2024-09-21T03:42:13.302 INFO:tasks.workunit.client.0.smithi089.stdout:rcu_preem 18 root NOFD /proc/18/fd (opendir: Permission denied) 2024-09-21T03:42:13.302 INFO:tasks.workunit.client.0.smithi089.stdout:migration 19 root cwd unknown /proc/19/cwd (readlink: Permission denied) 2024-09-21T03:42:13.302 INFO:tasks.workunit.client.0.smithi089.stdout:migration 19 root rtd unknown /proc/19/root (readlink: Permission denied) 2024-09-21T03:42:13.302 INFO:tasks.workunit.client.0.smithi089.stdout:migration 19 root txt unknown /proc/19/exe (readlink: Permission denied) 2024-09-21T03:42:13.302 INFO:tasks.workunit.client.0.smithi089.stdout:migration 19 root NOFD /proc/19/fd (opendir: Permission denied) 2024-09-21T03:42:13.302 INFO:tasks.workunit.client.0.smithi089.stdout:idle_inje 20 root cwd unknown /proc/20/cwd (readlink: Permission denied) 2024-09-21T03:42:13.302 INFO:tasks.workunit.client.0.smithi089.stdout:idle_inje 20 root rtd unknown /proc/20/root (readlink: Permission denied) 2024-09-21T03:42:13.302 INFO:tasks.workunit.client.0.smithi089.stdout:idle_inje 20 root txt unknown /proc/20/exe (readlink: Permission denied) 2024-09-21T03:42:13.303 INFO:tasks.workunit.client.0.smithi089.stdout:idle_inje 20 root NOFD /proc/20/fd (opendir: Permission denied) 2024-09-21T03:42:13.303 INFO:tasks.workunit.client.0.smithi089.stdout:cpuhp/0 22 root cwd unknown /proc/22/cwd (readlink: Permission denied) 2024-09-21T03:42:13.303 INFO:tasks.workunit.client.0.smithi089.stdout:cpuhp/0 22 root rtd unknown /proc/22/root (readlink: Permission denied) 2024-09-21T03:42:13.303 INFO:tasks.workunit.client.0.smithi089.stdout:cpuhp/0 22 root txt unknown /proc/22/exe (readlink: Permission denied) 2024-09-21T03:42:13.303 INFO:tasks.workunit.client.0.smithi089.stdout:cpuhp/0 22 root NOFD /proc/22/fd (opendir: Permission denied) 2024-09-21T03:42:13.303 INFO:tasks.workunit.client.0.smithi089.stdout:cpuhp/1 23 root cwd unknown /proc/23/cwd (readlink: Permission denied) 2024-09-21T03:42:13.303 INFO:tasks.workunit.client.0.smithi089.stdout:cpuhp/1 23 root rtd unknown /proc/23/root (readlink: Permission denied) 2024-09-21T03:42:13.303 INFO:tasks.workunit.client.0.smithi089.stdout:cpuhp/1 23 root txt unknown /proc/23/exe (readlink: Permission denied) 2024-09-21T03:42:13.303 INFO:tasks.workunit.client.0.smithi089.stdout:cpuhp/1 23 root NOFD /proc/23/fd (opendir: Permission denied) 2024-09-21T03:42:13.303 INFO:tasks.workunit.client.0.smithi089.stdout:idle_inje 24 root cwd unknown /proc/24/cwd (readlink: Permission denied) 2024-09-21T03:42:13.303 INFO:tasks.workunit.client.0.smithi089.stdout:idle_inje 24 root rtd unknown /proc/24/root (readlink: Permission denied) 2024-09-21T03:42:13.303 INFO:tasks.workunit.client.0.smithi089.stdout:idle_inje 24 root txt unknown /proc/24/exe (readlink: Permission denied) 2024-09-21T03:42:13.303 INFO:tasks.workunit.client.0.smithi089.stdout:idle_inje 24 root NOFD /proc/24/fd (opendir: Permission denied) 2024-09-21T03:42:13.303 INFO:tasks.workunit.client.0.smithi089.stdout:migration 25 root cwd unknown /proc/25/cwd (readlink: Permission denied) 2024-09-21T03:42:13.304 INFO:tasks.workunit.client.0.smithi089.stdout:migration 25 root rtd unknown /proc/25/root (readlink: Permission denied) 2024-09-21T03:42:13.304 INFO:tasks.workunit.client.0.smithi089.stdout:migration 25 root txt unknown /proc/25/exe (readlink: Permission denied) 2024-09-21T03:42:13.304 INFO:tasks.workunit.client.0.smithi089.stdout:migration 25 root NOFD /proc/25/fd (opendir: Permission denied) 2024-09-21T03:42:13.304 INFO:tasks.workunit.client.0.smithi089.stdout:ksoftirqd 26 root cwd unknown /proc/26/cwd (readlink: Permission denied) 2024-09-21T03:42:13.304 INFO:tasks.workunit.client.0.smithi089.stdout:ksoftirqd 26 root rtd unknown /proc/26/root (readlink: Permission denied) 2024-09-21T03:42:13.304 INFO:tasks.workunit.client.0.smithi089.stdout:ksoftirqd 26 root txt unknown /proc/26/exe (readlink: Permission denied) 2024-09-21T03:42:13.304 INFO:tasks.workunit.client.0.smithi089.stdout:ksoftirqd 26 root NOFD /proc/26/fd (opendir: Permission denied) 2024-09-21T03:42:13.304 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/1 28 root cwd unknown /proc/28/cwd (readlink: Permission denied) 2024-09-21T03:42:13.304 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/1 28 root rtd unknown /proc/28/root (readlink: Permission denied) 2024-09-21T03:42:13.304 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/1 28 root txt unknown /proc/28/exe (readlink: Permission denied) 2024-09-21T03:42:13.304 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/1 28 root NOFD /proc/28/fd (opendir: Permission denied) 2024-09-21T03:42:13.304 INFO:tasks.workunit.client.0.smithi089.stdout:cpuhp/2 29 root cwd unknown /proc/29/cwd (readlink: Permission denied) 2024-09-21T03:42:13.304 INFO:tasks.workunit.client.0.smithi089.stdout:cpuhp/2 29 root rtd unknown /proc/29/root (readlink: Permission denied) 2024-09-21T03:42:13.305 INFO:tasks.workunit.client.0.smithi089.stdout:cpuhp/2 29 root txt unknown /proc/29/exe (readlink: Permission denied) 2024-09-21T03:42:13.305 INFO:tasks.workunit.client.0.smithi089.stdout:cpuhp/2 29 root NOFD /proc/29/fd (opendir: Permission denied) 2024-09-21T03:42:13.305 INFO:tasks.workunit.client.0.smithi089.stdout:idle_inje 30 root cwd unknown /proc/30/cwd (readlink: Permission denied) 2024-09-21T03:42:13.305 INFO:tasks.workunit.client.0.smithi089.stdout:idle_inje 30 root rtd unknown /proc/30/root (readlink: Permission denied) 2024-09-21T03:42:13.305 INFO:tasks.workunit.client.0.smithi089.stdout:idle_inje 30 root txt unknown /proc/30/exe (readlink: Permission denied) 2024-09-21T03:42:13.305 INFO:tasks.workunit.client.0.smithi089.stdout:idle_inje 30 root NOFD /proc/30/fd (opendir: Permission denied) 2024-09-21T03:42:13.305 INFO:tasks.workunit.client.0.smithi089.stdout:migration 31 root cwd unknown /proc/31/cwd (readlink: Permission denied) 2024-09-21T03:42:13.305 INFO:tasks.workunit.client.0.smithi089.stdout:migration 31 root rtd unknown /proc/31/root (readlink: Permission denied) 2024-09-21T03:42:13.305 INFO:tasks.workunit.client.0.smithi089.stdout:migration 31 root txt unknown /proc/31/exe (readlink: Permission denied) 2024-09-21T03:42:13.305 INFO:tasks.workunit.client.0.smithi089.stdout:migration 31 root NOFD /proc/31/fd (opendir: Permission denied) 2024-09-21T03:42:13.305 INFO:tasks.workunit.client.0.smithi089.stdout:ksoftirqd 32 root cwd unknown /proc/32/cwd (readlink: Permission denied) 2024-09-21T03:42:13.305 INFO:tasks.workunit.client.0.smithi089.stdout:ksoftirqd 32 root rtd unknown /proc/32/root (readlink: Permission denied) 2024-09-21T03:42:13.305 INFO:tasks.workunit.client.0.smithi089.stdout:ksoftirqd 32 root txt unknown /proc/32/exe (readlink: Permission denied) 2024-09-21T03:42:13.305 INFO:tasks.workunit.client.0.smithi089.stdout:ksoftirqd 32 root NOFD /proc/32/fd (opendir: Permission denied) 2024-09-21T03:42:13.306 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/2 34 root cwd unknown /proc/34/cwd (readlink: Permission denied) 2024-09-21T03:42:13.306 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/2 34 root rtd unknown /proc/34/root (readlink: Permission denied) 2024-09-21T03:42:13.306 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/2 34 root txt unknown /proc/34/exe (readlink: Permission denied) 2024-09-21T03:42:13.306 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/2 34 root NOFD /proc/34/fd (opendir: Permission denied) 2024-09-21T03:42:13.306 INFO:tasks.workunit.client.0.smithi089.stdout:cpuhp/3 35 root cwd unknown /proc/35/cwd (readlink: Permission denied) 2024-09-21T03:42:13.306 INFO:tasks.workunit.client.0.smithi089.stdout:cpuhp/3 35 root rtd unknown /proc/35/root (readlink: Permission denied) 2024-09-21T03:42:13.306 INFO:tasks.workunit.client.0.smithi089.stdout:cpuhp/3 35 root txt unknown /proc/35/exe (readlink: Permission denied) 2024-09-21T03:42:13.306 INFO:tasks.workunit.client.0.smithi089.stdout:cpuhp/3 35 root NOFD /proc/35/fd (opendir: Permission denied) 2024-09-21T03:42:13.306 INFO:tasks.workunit.client.0.smithi089.stdout:idle_inje 36 root cwd unknown /proc/36/cwd (readlink: Permission denied) 2024-09-21T03:42:13.306 INFO:tasks.workunit.client.0.smithi089.stdout:idle_inje 36 root rtd unknown /proc/36/root (readlink: Permission denied) 2024-09-21T03:42:13.306 INFO:tasks.workunit.client.0.smithi089.stdout:idle_inje 36 root txt unknown /proc/36/exe (readlink: Permission denied) 2024-09-21T03:42:13.306 INFO:tasks.workunit.client.0.smithi089.stdout:idle_inje 36 root NOFD /proc/36/fd (opendir: Permission denied) 2024-09-21T03:42:13.306 INFO:tasks.workunit.client.0.smithi089.stdout:migration 37 root cwd unknown /proc/37/cwd (readlink: Permission denied) 2024-09-21T03:42:13.307 INFO:tasks.workunit.client.0.smithi089.stdout:migration 37 root rtd unknown /proc/37/root (readlink: Permission denied) 2024-09-21T03:42:13.307 INFO:tasks.workunit.client.0.smithi089.stdout:migration 37 root txt unknown /proc/37/exe (readlink: Permission denied) 2024-09-21T03:42:13.307 INFO:tasks.workunit.client.0.smithi089.stdout:migration 37 root NOFD /proc/37/fd (opendir: Permission denied) 2024-09-21T03:42:13.307 INFO:tasks.workunit.client.0.smithi089.stdout:ksoftirqd 38 root cwd unknown /proc/38/cwd (readlink: Permission denied) 2024-09-21T03:42:13.307 INFO:tasks.workunit.client.0.smithi089.stdout:ksoftirqd 38 root rtd unknown /proc/38/root (readlink: Permission denied) 2024-09-21T03:42:13.307 INFO:tasks.workunit.client.0.smithi089.stdout:ksoftirqd 38 root txt unknown /proc/38/exe (readlink: Permission denied) 2024-09-21T03:42:13.307 INFO:tasks.workunit.client.0.smithi089.stdout:ksoftirqd 38 root NOFD /proc/38/fd (opendir: Permission denied) 2024-09-21T03:42:13.307 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/3 40 root cwd unknown /proc/40/cwd (readlink: Permission denied) 2024-09-21T03:42:13.307 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/3 40 root rtd unknown /proc/40/root (readlink: Permission denied) 2024-09-21T03:42:13.307 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/3 40 root txt unknown /proc/40/exe (readlink: Permission denied) 2024-09-21T03:42:13.307 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/3 40 root NOFD /proc/40/fd (opendir: Permission denied) 2024-09-21T03:42:13.307 INFO:tasks.workunit.client.0.smithi089.stdout:cpuhp/4 41 root cwd unknown /proc/41/cwd (readlink: Permission denied) 2024-09-21T03:42:13.307 INFO:tasks.workunit.client.0.smithi089.stdout:cpuhp/4 41 root rtd unknown /proc/41/root (readlink: Permission denied) 2024-09-21T03:42:13.307 INFO:tasks.workunit.client.0.smithi089.stdout:cpuhp/4 41 root txt unknown /proc/41/exe (readlink: Permission denied) 2024-09-21T03:42:13.307 INFO:tasks.workunit.client.0.smithi089.stdout:cpuhp/4 41 root NOFD /proc/41/fd (opendir: Permission denied) 2024-09-21T03:42:13.308 INFO:tasks.workunit.client.0.smithi089.stdout:idle_inje 42 root cwd unknown /proc/42/cwd (readlink: Permission denied) 2024-09-21T03:42:13.308 INFO:tasks.workunit.client.0.smithi089.stdout:idle_inje 42 root rtd unknown /proc/42/root (readlink: Permission denied) 2024-09-21T03:42:13.308 INFO:tasks.workunit.client.0.smithi089.stdout:idle_inje 42 root txt unknown /proc/42/exe (readlink: Permission denied) 2024-09-21T03:42:13.308 INFO:tasks.workunit.client.0.smithi089.stdout:idle_inje 42 root NOFD /proc/42/fd (opendir: Permission denied) 2024-09-21T03:42:13.308 INFO:tasks.workunit.client.0.smithi089.stdout:migration 43 root cwd unknown /proc/43/cwd (readlink: Permission denied) 2024-09-21T03:42:13.308 INFO:tasks.workunit.client.0.smithi089.stdout:migration 43 root rtd unknown /proc/43/root (readlink: Permission denied) 2024-09-21T03:42:13.308 INFO:tasks.workunit.client.0.smithi089.stdout:migration 43 root txt unknown /proc/43/exe (readlink: Permission denied) 2024-09-21T03:42:13.308 INFO:tasks.workunit.client.0.smithi089.stdout:migration 43 root NOFD /proc/43/fd (opendir: Permission denied) 2024-09-21T03:42:13.308 INFO:tasks.workunit.client.0.smithi089.stdout:ksoftirqd 44 root cwd unknown /proc/44/cwd (readlink: Permission denied) 2024-09-21T03:42:13.308 INFO:tasks.workunit.client.0.smithi089.stdout:ksoftirqd 44 root rtd unknown /proc/44/root (readlink: Permission denied) 2024-09-21T03:42:13.308 INFO:tasks.workunit.client.0.smithi089.stdout:ksoftirqd 44 root txt unknown /proc/44/exe (readlink: Permission denied) 2024-09-21T03:42:13.308 INFO:tasks.workunit.client.0.smithi089.stdout:ksoftirqd 44 root NOFD /proc/44/fd (opendir: Permission denied) 2024-09-21T03:42:13.308 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/4 46 root cwd unknown /proc/46/cwd (readlink: Permission denied) 2024-09-21T03:42:13.308 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/4 46 root rtd unknown /proc/46/root (readlink: Permission denied) 2024-09-21T03:42:13.308 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/4 46 root txt unknown /proc/46/exe (readlink: Permission denied) 2024-09-21T03:42:13.309 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/4 46 root NOFD /proc/46/fd (opendir: Permission denied) 2024-09-21T03:42:13.309 INFO:tasks.workunit.client.0.smithi089.stdout:cpuhp/5 47 root cwd unknown /proc/47/cwd (readlink: Permission denied) 2024-09-21T03:42:13.309 INFO:tasks.workunit.client.0.smithi089.stdout:cpuhp/5 47 root rtd unknown /proc/47/root (readlink: Permission denied) 2024-09-21T03:42:13.309 INFO:tasks.workunit.client.0.smithi089.stdout:cpuhp/5 47 root txt unknown /proc/47/exe (readlink: Permission denied) 2024-09-21T03:42:13.309 INFO:tasks.workunit.client.0.smithi089.stdout:cpuhp/5 47 root NOFD /proc/47/fd (opendir: Permission denied) 2024-09-21T03:42:13.309 INFO:tasks.workunit.client.0.smithi089.stdout:idle_inje 48 root cwd unknown /proc/48/cwd (readlink: Permission denied) 2024-09-21T03:42:13.309 INFO:tasks.workunit.client.0.smithi089.stdout:idle_inje 48 root rtd unknown /proc/48/root (readlink: Permission denied) 2024-09-21T03:42:13.309 INFO:tasks.workunit.client.0.smithi089.stdout:idle_inje 48 root txt unknown /proc/48/exe (readlink: Permission denied) 2024-09-21T03:42:13.309 INFO:tasks.workunit.client.0.smithi089.stdout:idle_inje 48 root NOFD /proc/48/fd (opendir: Permission denied) 2024-09-21T03:42:13.309 INFO:tasks.workunit.client.0.smithi089.stdout:migration 49 root cwd unknown /proc/49/cwd (readlink: Permission denied) 2024-09-21T03:42:13.309 INFO:tasks.workunit.client.0.smithi089.stdout:migration 49 root rtd unknown /proc/49/root (readlink: Permission denied) 2024-09-21T03:42:13.309 INFO:tasks.workunit.client.0.smithi089.stdout:migration 49 root txt unknown /proc/49/exe (readlink: Permission denied) 2024-09-21T03:42:13.309 INFO:tasks.workunit.client.0.smithi089.stdout:migration 49 root NOFD /proc/49/fd (opendir: Permission denied) 2024-09-21T03:42:13.309 INFO:tasks.workunit.client.0.smithi089.stdout:ksoftirqd 50 root cwd unknown /proc/50/cwd (readlink: Permission denied) 2024-09-21T03:42:13.310 INFO:tasks.workunit.client.0.smithi089.stdout:ksoftirqd 50 root rtd unknown /proc/50/root (readlink: Permission denied) 2024-09-21T03:42:13.310 INFO:tasks.workunit.client.0.smithi089.stdout:ksoftirqd 50 root txt unknown /proc/50/exe (readlink: Permission denied) 2024-09-21T03:42:13.310 INFO:tasks.workunit.client.0.smithi089.stdout:ksoftirqd 50 root NOFD /proc/50/fd (opendir: Permission denied) 2024-09-21T03:42:13.310 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/5 52 root cwd unknown /proc/52/cwd (readlink: Permission denied) 2024-09-21T03:42:13.310 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/5 52 root rtd unknown /proc/52/root (readlink: Permission denied) 2024-09-21T03:42:13.310 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/5 52 root txt unknown /proc/52/exe (readlink: Permission denied) 2024-09-21T03:42:13.310 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/5 52 root NOFD /proc/52/fd (opendir: Permission denied) 2024-09-21T03:42:13.310 INFO:tasks.workunit.client.0.smithi089.stdout:cpuhp/6 53 root cwd unknown /proc/53/cwd (readlink: Permission denied) 2024-09-21T03:42:13.310 INFO:tasks.workunit.client.0.smithi089.stdout:cpuhp/6 53 root rtd unknown /proc/53/root (readlink: Permission denied) 2024-09-21T03:42:13.310 INFO:tasks.workunit.client.0.smithi089.stdout:cpuhp/6 53 root txt unknown /proc/53/exe (readlink: Permission denied) 2024-09-21T03:42:13.310 INFO:tasks.workunit.client.0.smithi089.stdout:cpuhp/6 53 root NOFD /proc/53/fd (opendir: Permission denied) 2024-09-21T03:42:13.310 INFO:tasks.workunit.client.0.smithi089.stdout:idle_inje 54 root cwd unknown /proc/54/cwd (readlink: Permission denied) 2024-09-21T03:42:13.310 INFO:tasks.workunit.client.0.smithi089.stdout:idle_inje 54 root rtd unknown /proc/54/root (readlink: Permission denied) 2024-09-21T03:42:13.310 INFO:tasks.workunit.client.0.smithi089.stdout:idle_inje 54 root txt unknown /proc/54/exe (readlink: Permission denied) 2024-09-21T03:42:13.311 INFO:tasks.workunit.client.0.smithi089.stdout:idle_inje 54 root NOFD /proc/54/fd (opendir: Permission denied) 2024-09-21T03:42:13.311 INFO:tasks.workunit.client.0.smithi089.stdout:migration 55 root cwd unknown /proc/55/cwd (readlink: Permission denied) 2024-09-21T03:42:13.311 INFO:tasks.workunit.client.0.smithi089.stdout:migration 55 root rtd unknown /proc/55/root (readlink: Permission denied) 2024-09-21T03:42:13.311 INFO:tasks.workunit.client.0.smithi089.stdout:migration 55 root txt unknown /proc/55/exe (readlink: Permission denied) 2024-09-21T03:42:13.311 INFO:tasks.workunit.client.0.smithi089.stdout:migration 55 root NOFD /proc/55/fd (opendir: Permission denied) 2024-09-21T03:42:13.311 INFO:tasks.workunit.client.0.smithi089.stdout:ksoftirqd 56 root cwd unknown /proc/56/cwd (readlink: Permission denied) 2024-09-21T03:42:13.311 INFO:tasks.workunit.client.0.smithi089.stdout:ksoftirqd 56 root rtd unknown /proc/56/root (readlink: Permission denied) 2024-09-21T03:42:13.311 INFO:tasks.workunit.client.0.smithi089.stdout:ksoftirqd 56 root txt unknown /proc/56/exe (readlink: Permission denied) 2024-09-21T03:42:13.311 INFO:tasks.workunit.client.0.smithi089.stdout:ksoftirqd 56 root NOFD /proc/56/fd (opendir: Permission denied) 2024-09-21T03:42:13.311 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/6 58 root cwd unknown /proc/58/cwd (readlink: Permission denied) 2024-09-21T03:42:13.311 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/6 58 root rtd unknown /proc/58/root (readlink: Permission denied) 2024-09-21T03:42:13.311 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/6 58 root txt unknown /proc/58/exe (readlink: Permission denied) 2024-09-21T03:42:13.311 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/6 58 root NOFD /proc/58/fd (opendir: Permission denied) 2024-09-21T03:42:13.311 INFO:tasks.workunit.client.0.smithi089.stdout:cpuhp/7 59 root cwd unknown /proc/59/cwd (readlink: Permission denied) 2024-09-21T03:42:13.311 INFO:tasks.workunit.client.0.smithi089.stdout:cpuhp/7 59 root rtd unknown /proc/59/root (readlink: Permission denied) 2024-09-21T03:42:13.312 INFO:tasks.workunit.client.0.smithi089.stdout:cpuhp/7 59 root txt unknown /proc/59/exe (readlink: Permission denied) 2024-09-21T03:42:13.312 INFO:tasks.workunit.client.0.smithi089.stdout:cpuhp/7 59 root NOFD /proc/59/fd (opendir: Permission denied) 2024-09-21T03:42:13.312 INFO:tasks.workunit.client.0.smithi089.stdout:idle_inje 60 root cwd unknown /proc/60/cwd (readlink: Permission denied) 2024-09-21T03:42:13.312 INFO:tasks.workunit.client.0.smithi089.stdout:idle_inje 60 root rtd unknown /proc/60/root (readlink: Permission denied) 2024-09-21T03:42:13.312 INFO:tasks.workunit.client.0.smithi089.stdout:idle_inje 60 root txt unknown /proc/60/exe (readlink: Permission denied) 2024-09-21T03:42:13.312 INFO:tasks.workunit.client.0.smithi089.stdout:idle_inje 60 root NOFD /proc/60/fd (opendir: Permission denied) 2024-09-21T03:42:13.312 INFO:tasks.workunit.client.0.smithi089.stdout:migration 61 root cwd unknown /proc/61/cwd (readlink: Permission denied) 2024-09-21T03:42:13.312 INFO:tasks.workunit.client.0.smithi089.stdout:migration 61 root rtd unknown /proc/61/root (readlink: Permission denied) 2024-09-21T03:42:13.312 INFO:tasks.workunit.client.0.smithi089.stdout:migration 61 root txt unknown /proc/61/exe (readlink: Permission denied) 2024-09-21T03:42:13.312 INFO:tasks.workunit.client.0.smithi089.stdout:migration 61 root NOFD /proc/61/fd (opendir: Permission denied) 2024-09-21T03:42:13.312 INFO:tasks.workunit.client.0.smithi089.stdout:ksoftirqd 62 root cwd unknown /proc/62/cwd (readlink: Permission denied) 2024-09-21T03:42:13.312 INFO:tasks.workunit.client.0.smithi089.stdout:ksoftirqd 62 root rtd unknown /proc/62/root (readlink: Permission denied) 2024-09-21T03:42:13.312 INFO:tasks.workunit.client.0.smithi089.stdout:ksoftirqd 62 root txt unknown /proc/62/exe (readlink: Permission denied) 2024-09-21T03:42:13.312 INFO:tasks.workunit.client.0.smithi089.stdout:ksoftirqd 62 root NOFD /proc/62/fd (opendir: Permission denied) 2024-09-21T03:42:13.313 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/7 64 root cwd unknown /proc/64/cwd (readlink: Permission denied) 2024-09-21T03:42:13.313 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/7 64 root rtd unknown /proc/64/root (readlink: Permission denied) 2024-09-21T03:42:13.313 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/7 64 root txt unknown /proc/64/exe (readlink: Permission denied) 2024-09-21T03:42:13.313 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/7 64 root NOFD /proc/64/fd (opendir: Permission denied) 2024-09-21T03:42:13.313 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/u 66 root cwd unknown /proc/66/cwd (readlink: Permission denied) 2024-09-21T03:42:13.313 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/u 66 root rtd unknown /proc/66/root (readlink: Permission denied) 2024-09-21T03:42:13.313 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/u 66 root txt unknown /proc/66/exe (readlink: Permission denied) 2024-09-21T03:42:13.313 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/u 66 root NOFD /proc/66/fd (opendir: Permission denied) 2024-09-21T03:42:13.313 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/u 67 root cwd unknown /proc/67/cwd (readlink: Permission denied) 2024-09-21T03:42:13.313 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/u 67 root rtd unknown /proc/67/root (readlink: Permission denied) 2024-09-21T03:42:13.313 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/u 67 root txt unknown /proc/67/exe (readlink: Permission denied) 2024-09-21T03:42:13.313 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/u 67 root NOFD /proc/67/fd (opendir: Permission denied) 2024-09-21T03:42:13.313 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/u 70 root cwd unknown /proc/70/cwd (readlink: Permission denied) 2024-09-21T03:42:13.313 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/u 70 root rtd unknown /proc/70/root (readlink: Permission denied) 2024-09-21T03:42:13.314 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/u 70 root txt unknown /proc/70/exe (readlink: Permission denied) 2024-09-21T03:42:13.314 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/u 70 root NOFD /proc/70/fd (opendir: Permission denied) 2024-09-21T03:42:13.314 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/u 71 root cwd unknown /proc/71/cwd (readlink: Permission denied) 2024-09-21T03:42:13.314 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/u 71 root rtd unknown /proc/71/root (readlink: Permission denied) 2024-09-21T03:42:13.314 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/u 71 root txt unknown /proc/71/exe (readlink: Permission denied) 2024-09-21T03:42:13.314 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/u 71 root NOFD /proc/71/fd (opendir: Permission denied) 2024-09-21T03:42:13.314 INFO:tasks.workunit.client.0.smithi089.stdout:kdevtmpfs 72 root cwd unknown /proc/72/cwd (readlink: Permission denied) 2024-09-21T03:42:13.314 INFO:tasks.workunit.client.0.smithi089.stdout:kdevtmpfs 72 root rtd unknown /proc/72/root (readlink: Permission denied) 2024-09-21T03:42:13.314 INFO:tasks.workunit.client.0.smithi089.stdout:kdevtmpfs 72 root txt unknown /proc/72/exe (readlink: Permission denied) 2024-09-21T03:42:13.314 INFO:tasks.workunit.client.0.smithi089.stdout:kdevtmpfs 72 root NOFD /proc/72/fd (opendir: Permission denied) 2024-09-21T03:42:13.314 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/R 73 root cwd unknown /proc/73/cwd (readlink: Permission denied) 2024-09-21T03:42:13.314 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/R 73 root rtd unknown /proc/73/root (readlink: Permission denied) 2024-09-21T03:42:13.314 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/R 73 root txt unknown /proc/73/exe (readlink: Permission denied) 2024-09-21T03:42:13.314 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/R 73 root NOFD /proc/73/fd (opendir: Permission denied) 2024-09-21T03:42:13.315 INFO:tasks.workunit.client.0.smithi089.stdout:kauditd 74 root cwd unknown /proc/74/cwd (readlink: Permission denied) 2024-09-21T03:42:13.315 INFO:tasks.workunit.client.0.smithi089.stdout:kauditd 74 root rtd unknown /proc/74/root (readlink: Permission denied) 2024-09-21T03:42:13.315 INFO:tasks.workunit.client.0.smithi089.stdout:kauditd 74 root txt unknown /proc/74/exe (readlink: Permission denied) 2024-09-21T03:42:13.315 INFO:tasks.workunit.client.0.smithi089.stdout:kauditd 74 root NOFD /proc/74/fd (opendir: Permission denied) 2024-09-21T03:42:13.315 INFO:tasks.workunit.client.0.smithi089.stdout:khungtask 76 root cwd unknown /proc/76/cwd (readlink: Permission denied) 2024-09-21T03:42:13.315 INFO:tasks.workunit.client.0.smithi089.stdout:khungtask 76 root rtd unknown /proc/76/root (readlink: Permission denied) 2024-09-21T03:42:13.315 INFO:tasks.workunit.client.0.smithi089.stdout:khungtask 76 root txt unknown /proc/76/exe (readlink: Permission denied) 2024-09-21T03:42:13.315 INFO:tasks.workunit.client.0.smithi089.stdout:khungtask 76 root NOFD /proc/76/fd (opendir: Permission denied) 2024-09-21T03:42:13.315 INFO:tasks.workunit.client.0.smithi089.stdout:oom_reape 77 root cwd unknown /proc/77/cwd (readlink: Permission denied) 2024-09-21T03:42:13.315 INFO:tasks.workunit.client.0.smithi089.stdout:oom_reape 77 root rtd unknown /proc/77/root (readlink: Permission denied) 2024-09-21T03:42:13.315 INFO:tasks.workunit.client.0.smithi089.stdout:oom_reape 77 root txt unknown /proc/77/exe (readlink: Permission denied) 2024-09-21T03:42:13.315 INFO:tasks.workunit.client.0.smithi089.stdout:oom_reape 77 root NOFD /proc/77/fd (opendir: Permission denied) 2024-09-21T03:42:13.315 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/R 78 root cwd unknown /proc/78/cwd (readlink: Permission denied) 2024-09-21T03:42:13.315 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/R 78 root rtd unknown /proc/78/root (readlink: Permission denied) 2024-09-21T03:42:13.316 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/R 78 root txt unknown /proc/78/exe (readlink: Permission denied) 2024-09-21T03:42:13.316 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/R 78 root NOFD /proc/78/fd (opendir: Permission denied) 2024-09-21T03:42:13.316 INFO:tasks.workunit.client.0.smithi089.stdout:kcompactd 79 root cwd unknown /proc/79/cwd (readlink: Permission denied) 2024-09-21T03:42:13.316 INFO:tasks.workunit.client.0.smithi089.stdout:kcompactd 79 root rtd unknown /proc/79/root (readlink: Permission denied) 2024-09-21T03:42:13.316 INFO:tasks.workunit.client.0.smithi089.stdout:kcompactd 79 root txt unknown /proc/79/exe (readlink: Permission denied) 2024-09-21T03:42:13.316 INFO:tasks.workunit.client.0.smithi089.stdout:kcompactd 79 root NOFD /proc/79/fd (opendir: Permission denied) 2024-09-21T03:42:13.316 INFO:tasks.workunit.client.0.smithi089.stdout:ksmd 80 root cwd unknown /proc/80/cwd (readlink: Permission denied) 2024-09-21T03:42:13.316 INFO:tasks.workunit.client.0.smithi089.stdout:ksmd 80 root rtd unknown /proc/80/root (readlink: Permission denied) 2024-09-21T03:42:13.316 INFO:tasks.workunit.client.0.smithi089.stdout:ksmd 80 root txt unknown /proc/80/exe (readlink: Permission denied) 2024-09-21T03:42:13.316 INFO:tasks.workunit.client.0.smithi089.stdout:ksmd 80 root NOFD /proc/80/fd (opendir: Permission denied) 2024-09-21T03:42:13.316 INFO:tasks.workunit.client.0.smithi089.stdout:khugepage 82 root cwd unknown /proc/82/cwd (readlink: Permission denied) 2024-09-21T03:42:13.316 INFO:tasks.workunit.client.0.smithi089.stdout:khugepage 82 root rtd unknown /proc/82/root (readlink: Permission denied) 2024-09-21T03:42:13.316 INFO:tasks.workunit.client.0.smithi089.stdout:khugepage 82 root txt unknown /proc/82/exe (readlink: Permission denied) 2024-09-21T03:42:13.316 INFO:tasks.workunit.client.0.smithi089.stdout:khugepage 82 root NOFD /proc/82/fd (opendir: Permission denied) 2024-09-21T03:42:13.316 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/R 83 root cwd unknown /proc/83/cwd (readlink: Permission denied) 2024-09-21T03:42:13.317 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/R 83 root rtd unknown /proc/83/root (readlink: Permission denied) 2024-09-21T03:42:13.317 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/R 83 root txt unknown /proc/83/exe (readlink: Permission denied) 2024-09-21T03:42:13.317 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/R 83 root NOFD /proc/83/fd (opendir: Permission denied) 2024-09-21T03:42:13.317 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/R 84 root cwd unknown /proc/84/cwd (readlink: Permission denied) 2024-09-21T03:42:13.317 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/R 84 root rtd unknown /proc/84/root (readlink: Permission denied) 2024-09-21T03:42:13.317 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/R 84 root txt unknown /proc/84/exe (readlink: Permission denied) 2024-09-21T03:42:13.317 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/R 84 root NOFD /proc/84/fd (opendir: Permission denied) 2024-09-21T03:42:13.317 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/R 85 root cwd unknown /proc/85/cwd (readlink: Permission denied) 2024-09-21T03:42:13.317 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/R 85 root rtd unknown /proc/85/root (readlink: Permission denied) 2024-09-21T03:42:13.317 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/R 85 root txt unknown /proc/85/exe (readlink: Permission denied) 2024-09-21T03:42:13.317 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/R 85 root NOFD /proc/85/fd (opendir: Permission denied) 2024-09-21T03:42:13.317 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/R 86 root cwd unknown /proc/86/cwd (readlink: Permission denied) 2024-09-21T03:42:13.317 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/R 86 root rtd unknown /proc/86/root (readlink: Permission denied) 2024-09-21T03:42:13.317 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/R 86 root txt unknown /proc/86/exe (readlink: Permission denied) 2024-09-21T03:42:13.318 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/R 86 root NOFD /proc/86/fd (opendir: Permission denied) 2024-09-21T03:42:13.318 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/R 87 root cwd unknown /proc/87/cwd (readlink: Permission denied) 2024-09-21T03:42:13.318 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/R 87 root rtd unknown /proc/87/root (readlink: Permission denied) 2024-09-21T03:42:13.318 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/R 87 root txt unknown /proc/87/exe (readlink: Permission denied) 2024-09-21T03:42:13.318 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/R 87 root NOFD /proc/87/fd (opendir: Permission denied) 2024-09-21T03:42:13.318 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/R 88 root cwd unknown /proc/88/cwd (readlink: Permission denied) 2024-09-21T03:42:13.318 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/R 88 root rtd unknown /proc/88/root (readlink: Permission denied) 2024-09-21T03:42:13.318 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/R 88 root txt unknown /proc/88/exe (readlink: Permission denied) 2024-09-21T03:42:13.318 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/R 88 root NOFD /proc/88/fd (opendir: Permission denied) 2024-09-21T03:42:13.318 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/R 89 root cwd unknown /proc/89/cwd (readlink: Permission denied) 2024-09-21T03:42:13.318 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/R 89 root rtd unknown /proc/89/root (readlink: Permission denied) 2024-09-21T03:42:13.318 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/R 89 root txt unknown /proc/89/exe (readlink: Permission denied) 2024-09-21T03:42:13.318 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/R 89 root NOFD /proc/89/fd (opendir: Permission denied) 2024-09-21T03:42:13.318 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/R 90 root cwd unknown /proc/90/cwd (readlink: Permission denied) 2024-09-21T03:42:13.318 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/R 90 root rtd unknown /proc/90/root (readlink: Permission denied) 2024-09-21T03:42:13.319 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/R 90 root txt unknown /proc/90/exe (readlink: Permission denied) 2024-09-21T03:42:13.319 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/R 90 root NOFD /proc/90/fd (opendir: Permission denied) 2024-09-21T03:42:13.319 INFO:tasks.workunit.client.0.smithi089.stdout:watchdogd 91 root cwd unknown /proc/91/cwd (readlink: Permission denied) 2024-09-21T03:42:13.319 INFO:tasks.workunit.client.0.smithi089.stdout:watchdogd 91 root rtd unknown /proc/91/root (readlink: Permission denied) 2024-09-21T03:42:13.319 INFO:tasks.workunit.client.0.smithi089.stdout:watchdogd 91 root txt unknown /proc/91/exe (readlink: Permission denied) 2024-09-21T03:42:13.319 INFO:tasks.workunit.client.0.smithi089.stdout:watchdogd 91 root NOFD /proc/91/fd (opendir: Permission denied) 2024-09-21T03:42:13.319 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/6 93 root cwd unknown /proc/93/cwd (readlink: Permission denied) 2024-09-21T03:42:13.319 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/6 93 root rtd unknown /proc/93/root (readlink: Permission denied) 2024-09-21T03:42:13.319 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/6 93 root txt unknown /proc/93/exe (readlink: Permission denied) 2024-09-21T03:42:13.319 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/6 93 root NOFD /proc/93/fd (opendir: Permission denied) 2024-09-21T03:42:13.319 INFO:tasks.workunit.client.0.smithi089.stdout:kswapd0 95 root cwd unknown /proc/95/cwd (readlink: Permission denied) 2024-09-21T03:42:13.319 INFO:tasks.workunit.client.0.smithi089.stdout:kswapd0 95 root rtd unknown /proc/95/root (readlink: Permission denied) 2024-09-21T03:42:13.319 INFO:tasks.workunit.client.0.smithi089.stdout:kswapd0 95 root txt unknown /proc/95/exe (readlink: Permission denied) 2024-09-21T03:42:13.319 INFO:tasks.workunit.client.0.smithi089.stdout:kswapd0 95 root NOFD /proc/95/fd (opendir: Permission denied) 2024-09-21T03:42:13.320 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/4 100 root cwd unknown /proc/100/cwd (readlink: Permission denied) 2024-09-21T03:42:13.320 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/4 100 root rtd unknown /proc/100/root (readlink: Permission denied) 2024-09-21T03:42:13.320 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/4 100 root txt unknown /proc/100/exe (readlink: Permission denied) 2024-09-21T03:42:13.320 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/4 100 root NOFD /proc/100/fd (opendir: Permission denied) 2024-09-21T03:42:13.320 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/R 101 root cwd unknown /proc/101/cwd (readlink: Permission denied) 2024-09-21T03:42:13.320 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/R 101 root rtd unknown /proc/101/root (readlink: Permission denied) 2024-09-21T03:42:13.320 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/R 101 root txt unknown /proc/101/exe (readlink: Permission denied) 2024-09-21T03:42:13.320 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/R 101 root NOFD /proc/101/fd (opendir: Permission denied) 2024-09-21T03:42:13.320 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/5 108 root cwd unknown /proc/108/cwd (readlink: Permission denied) 2024-09-21T03:42:13.320 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/5 108 root rtd unknown /proc/108/root (readlink: Permission denied) 2024-09-21T03:42:13.320 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/5 108 root txt unknown /proc/108/exe (readlink: Permission denied) 2024-09-21T03:42:13.320 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/5 108 root NOFD /proc/108/fd (opendir: Permission denied) 2024-09-21T03:42:13.320 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/R 110 root cwd unknown /proc/110/cwd (readlink: Permission denied) 2024-09-21T03:42:13.320 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/R 110 root rtd unknown /proc/110/root (readlink: Permission denied) 2024-09-21T03:42:13.321 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/R 110 root txt unknown /proc/110/exe (readlink: Permission denied) 2024-09-21T03:42:13.321 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/R 110 root NOFD /proc/110/fd (opendir: Permission denied) 2024-09-21T03:42:13.321 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/R 111 root cwd unknown /proc/111/cwd (readlink: Permission denied) 2024-09-21T03:42:13.321 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/R 111 root rtd unknown /proc/111/root (readlink: Permission denied) 2024-09-21T03:42:13.321 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/R 111 root txt unknown /proc/111/exe (readlink: Permission denied) 2024-09-21T03:42:13.321 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/R 111 root NOFD /proc/111/fd (opendir: Permission denied) 2024-09-21T03:42:13.321 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/R 112 root cwd unknown /proc/112/cwd (readlink: Permission denied) 2024-09-21T03:42:13.321 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/R 112 root rtd unknown /proc/112/root (readlink: Permission denied) 2024-09-21T03:42:13.321 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/R 112 root txt unknown /proc/112/exe (readlink: Permission denied) 2024-09-21T03:42:13.321 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/R 112 root NOFD /proc/112/fd (opendir: Permission denied) 2024-09-21T03:42:13.321 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/R 115 root cwd unknown /proc/115/cwd (readlink: Permission denied) 2024-09-21T03:42:13.321 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/R 115 root rtd unknown /proc/115/root (readlink: Permission denied) 2024-09-21T03:42:13.321 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/R 115 root txt unknown /proc/115/exe (readlink: Permission denied) 2024-09-21T03:42:13.322 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/R 115 root NOFD /proc/115/fd (opendir: Permission denied) 2024-09-21T03:42:13.322 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/4 116 root cwd unknown /proc/116/cwd (readlink: Permission denied) 2024-09-21T03:42:13.322 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/4 116 root rtd unknown /proc/116/root (readlink: Permission denied) 2024-09-21T03:42:13.322 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/4 116 root txt unknown /proc/116/exe (readlink: Permission denied) 2024-09-21T03:42:13.322 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/4 116 root NOFD /proc/116/fd (opendir: Permission denied) 2024-09-21T03:42:13.322 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/R 117 root cwd unknown /proc/117/cwd (readlink: Permission denied) 2024-09-21T03:42:13.322 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/R 117 root rtd unknown /proc/117/root (readlink: Permission denied) 2024-09-21T03:42:13.322 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/R 117 root txt unknown /proc/117/exe (readlink: Permission denied) 2024-09-21T03:42:13.322 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/R 117 root NOFD /proc/117/fd (opendir: Permission denied) 2024-09-21T03:42:13.322 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/R 126 root cwd unknown /proc/126/cwd (readlink: Permission denied) 2024-09-21T03:42:13.322 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/R 126 root rtd unknown /proc/126/root (readlink: Permission denied) 2024-09-21T03:42:13.322 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/R 126 root txt unknown /proc/126/exe (readlink: Permission denied) 2024-09-21T03:42:13.322 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/R 126 root NOFD /proc/126/fd (opendir: Permission denied) 2024-09-21T03:42:13.323 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/u 238 root cwd unknown /proc/238/cwd (readlink: Permission denied) 2024-09-21T03:42:13.323 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/u 238 root rtd unknown /proc/238/root (readlink: Permission denied) 2024-09-21T03:42:13.323 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/u 238 root txt unknown /proc/238/exe (readlink: Permission denied) 2024-09-21T03:42:13.323 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/u 238 root NOFD /proc/238/fd (opendir: Permission denied) 2024-09-21T03:42:13.323 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/1 278 root cwd unknown /proc/278/cwd (readlink: Permission denied) 2024-09-21T03:42:13.323 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/1 278 root rtd unknown /proc/278/root (readlink: Permission denied) 2024-09-21T03:42:13.323 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/1 278 root txt unknown /proc/278/exe (readlink: Permission denied) 2024-09-21T03:42:13.323 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/1 278 root NOFD /proc/278/fd (opendir: Permission denied) 2024-09-21T03:42:13.323 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/3 282 root cwd unknown /proc/282/cwd (readlink: Permission denied) 2024-09-21T03:42:13.323 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/3 282 root rtd unknown /proc/282/root (readlink: Permission denied) 2024-09-21T03:42:13.323 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/3 282 root txt unknown /proc/282/exe (readlink: Permission denied) 2024-09-21T03:42:13.323 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/3 282 root NOFD /proc/282/fd (opendir: Permission denied) 2024-09-21T03:42:13.323 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/2 284 root cwd unknown /proc/284/cwd (readlink: Permission denied) 2024-09-21T03:42:13.323 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/2 284 root rtd unknown /proc/284/root (readlink: Permission denied) 2024-09-21T03:42:13.323 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/2 284 root txt unknown /proc/284/exe (readlink: Permission denied) 2024-09-21T03:42:13.324 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/2 284 root NOFD /proc/284/fd (opendir: Permission denied) 2024-09-21T03:42:13.324 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/7 286 root cwd unknown /proc/286/cwd (readlink: Permission denied) 2024-09-21T03:42:13.324 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/7 286 root rtd unknown /proc/286/root (readlink: Permission denied) 2024-09-21T03:42:13.324 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/7 286 root txt unknown /proc/286/exe (readlink: Permission denied) 2024-09-21T03:42:13.324 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/7 286 root NOFD /proc/286/fd (opendir: Permission denied) 2024-09-21T03:42:13.324 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/2 349 root cwd unknown /proc/349/cwd (readlink: Permission denied) 2024-09-21T03:42:13.324 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/2 349 root rtd unknown /proc/349/root (readlink: Permission denied) 2024-09-21T03:42:13.324 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/2 349 root txt unknown /proc/349/exe (readlink: Permission denied) 2024-09-21T03:42:13.324 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/2 349 root NOFD /proc/349/fd (opendir: Permission denied) 2024-09-21T03:42:13.324 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/6 361 root cwd unknown /proc/361/cwd (readlink: Permission denied) 2024-09-21T03:42:13.324 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/6 361 root rtd unknown /proc/361/root (readlink: Permission denied) 2024-09-21T03:42:13.324 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/6 361 root txt unknown /proc/361/exe (readlink: Permission denied) 2024-09-21T03:42:13.324 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/6 361 root NOFD /proc/361/fd (opendir: Permission denied) 2024-09-21T03:42:13.324 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/5 366 root cwd unknown /proc/366/cwd (readlink: Permission denied) 2024-09-21T03:42:13.325 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/5 366 root rtd unknown /proc/366/root (readlink: Permission denied) 2024-09-21T03:42:13.325 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/5 366 root txt unknown /proc/366/exe (readlink: Permission denied) 2024-09-21T03:42:13.325 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/5 366 root NOFD /proc/366/fd (opendir: Permission denied) 2024-09-21T03:42:13.325 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/0 383 root cwd unknown /proc/383/cwd (readlink: Permission denied) 2024-09-21T03:42:13.325 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/0 383 root rtd unknown /proc/383/root (readlink: Permission denied) 2024-09-21T03:42:13.325 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/0 383 root txt unknown /proc/383/exe (readlink: Permission denied) 2024-09-21T03:42:13.325 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/0 383 root NOFD /proc/383/fd (opendir: Permission denied) 2024-09-21T03:42:13.325 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/R 436 root cwd unknown /proc/436/cwd (readlink: Permission denied) 2024-09-21T03:42:13.325 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/R 436 root rtd unknown /proc/436/root (readlink: Permission denied) 2024-09-21T03:42:13.325 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/R 436 root txt unknown /proc/436/exe (readlink: Permission denied) 2024-09-21T03:42:13.325 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/R 436 root NOFD /proc/436/fd (opendir: Permission denied) 2024-09-21T03:42:13.325 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/R 445 root cwd unknown /proc/445/cwd (readlink: Permission denied) 2024-09-21T03:42:13.325 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/R 445 root rtd unknown /proc/445/root (readlink: Permission denied) 2024-09-21T03:42:13.326 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/R 445 root txt unknown /proc/445/exe (readlink: Permission denied) 2024-09-21T03:42:13.326 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/R 445 root NOFD /proc/445/fd (opendir: Permission denied) 2024-09-21T03:42:13.326 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/R 446 root cwd unknown /proc/446/cwd (readlink: Permission denied) 2024-09-21T03:42:13.326 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/R 446 root rtd unknown /proc/446/root (readlink: Permission denied) 2024-09-21T03:42:13.326 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/R 446 root txt unknown /proc/446/exe (readlink: Permission denied) 2024-09-21T03:42:13.326 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/R 446 root NOFD /proc/446/fd (opendir: Permission denied) 2024-09-21T03:42:13.326 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/R 447 root cwd unknown /proc/447/cwd (readlink: Permission denied) 2024-09-21T03:42:13.326 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/R 447 root rtd unknown /proc/447/root (readlink: Permission denied) 2024-09-21T03:42:13.326 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/R 447 root txt unknown /proc/447/exe (readlink: Permission denied) 2024-09-21T03:42:13.326 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/R 447 root NOFD /proc/447/fd (opendir: Permission denied) 2024-09-21T03:42:13.326 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/R 448 root cwd unknown /proc/448/cwd (readlink: Permission denied) 2024-09-21T03:42:13.326 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/R 448 root rtd unknown /proc/448/root (readlink: Permission denied) 2024-09-21T03:42:13.326 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/R 448 root txt unknown /proc/448/exe (readlink: Permission denied) 2024-09-21T03:42:13.326 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/R 448 root NOFD /proc/448/fd (opendir: Permission denied) 2024-09-21T03:42:13.327 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/R 450 root cwd unknown /proc/450/cwd (readlink: Permission denied) 2024-09-21T03:42:13.327 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/R 450 root rtd unknown /proc/450/root (readlink: Permission denied) 2024-09-21T03:42:13.327 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/R 450 root txt unknown /proc/450/exe (readlink: Permission denied) 2024-09-21T03:42:13.327 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/R 450 root NOFD /proc/450/fd (opendir: Permission denied) 2024-09-21T03:42:13.327 INFO:tasks.workunit.client.0.smithi089.stdout:scsi_eh_0 451 root cwd unknown /proc/451/cwd (readlink: Permission denied) 2024-09-21T03:42:13.327 INFO:tasks.workunit.client.0.smithi089.stdout:scsi_eh_0 451 root rtd unknown /proc/451/root (readlink: Permission denied) 2024-09-21T03:42:13.327 INFO:tasks.workunit.client.0.smithi089.stdout:scsi_eh_0 451 root txt unknown /proc/451/exe (readlink: Permission denied) 2024-09-21T03:42:13.327 INFO:tasks.workunit.client.0.smithi089.stdout:scsi_eh_0 451 root NOFD /proc/451/fd (opendir: Permission denied) 2024-09-21T03:42:13.327 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/R 452 root cwd unknown /proc/452/cwd (readlink: Permission denied) 2024-09-21T03:42:13.327 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/R 452 root rtd unknown /proc/452/root (readlink: Permission denied) 2024-09-21T03:42:13.327 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/R 452 root txt unknown /proc/452/exe (readlink: Permission denied) 2024-09-21T03:42:13.327 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/R 452 root NOFD /proc/452/fd (opendir: Permission denied) 2024-09-21T03:42:13.327 INFO:tasks.workunit.client.0.smithi089.stdout:scsi_eh_1 453 root cwd unknown /proc/453/cwd (readlink: Permission denied) 2024-09-21T03:42:13.327 INFO:tasks.workunit.client.0.smithi089.stdout:scsi_eh_1 453 root rtd unknown /proc/453/root (readlink: Permission denied) 2024-09-21T03:42:13.328 INFO:tasks.workunit.client.0.smithi089.stdout:scsi_eh_1 453 root txt unknown /proc/453/exe (readlink: Permission denied) 2024-09-21T03:42:13.328 INFO:tasks.workunit.client.0.smithi089.stdout:scsi_eh_1 453 root NOFD /proc/453/fd (opendir: Permission denied) 2024-09-21T03:42:13.328 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/R 454 root cwd unknown /proc/454/cwd (readlink: Permission denied) 2024-09-21T03:42:13.328 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/R 454 root rtd unknown /proc/454/root (readlink: Permission denied) 2024-09-21T03:42:13.328 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/R 454 root txt unknown /proc/454/exe (readlink: Permission denied) 2024-09-21T03:42:13.328 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/R 454 root NOFD /proc/454/fd (opendir: Permission denied) 2024-09-21T03:42:13.328 INFO:tasks.workunit.client.0.smithi089.stdout:scsi_eh_2 455 root cwd unknown /proc/455/cwd (readlink: Permission denied) 2024-09-21T03:42:13.328 INFO:tasks.workunit.client.0.smithi089.stdout:scsi_eh_2 455 root rtd unknown /proc/455/root (readlink: Permission denied) 2024-09-21T03:42:13.328 INFO:tasks.workunit.client.0.smithi089.stdout:scsi_eh_2 455 root txt unknown /proc/455/exe (readlink: Permission denied) 2024-09-21T03:42:13.328 INFO:tasks.workunit.client.0.smithi089.stdout:scsi_eh_2 455 root NOFD /proc/455/fd (opendir: Permission denied) 2024-09-21T03:42:13.328 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/R 456 root cwd unknown /proc/456/cwd (readlink: Permission denied) 2024-09-21T03:42:13.328 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/R 456 root rtd unknown /proc/456/root (readlink: Permission denied) 2024-09-21T03:42:13.328 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/R 456 root txt unknown /proc/456/exe (readlink: Permission denied) 2024-09-21T03:42:13.328 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/R 456 root NOFD /proc/456/fd (opendir: Permission denied) 2024-09-21T03:42:13.328 INFO:tasks.workunit.client.0.smithi089.stdout:scsi_eh_3 457 root cwd unknown /proc/457/cwd (readlink: Permission denied) 2024-09-21T03:42:13.329 INFO:tasks.workunit.client.0.smithi089.stdout:scsi_eh_3 457 root rtd unknown /proc/457/root (readlink: Permission denied) 2024-09-21T03:42:13.329 INFO:tasks.workunit.client.0.smithi089.stdout:scsi_eh_3 457 root txt unknown /proc/457/exe (readlink: Permission denied) 2024-09-21T03:42:13.329 INFO:tasks.workunit.client.0.smithi089.stdout:scsi_eh_3 457 root NOFD /proc/457/fd (opendir: Permission denied) 2024-09-21T03:42:13.329 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/R 458 root cwd unknown /proc/458/cwd (readlink: Permission denied) 2024-09-21T03:42:13.329 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/R 458 root rtd unknown /proc/458/root (readlink: Permission denied) 2024-09-21T03:42:13.329 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/R 458 root txt unknown /proc/458/exe (readlink: Permission denied) 2024-09-21T03:42:13.329 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/R 458 root NOFD /proc/458/fd (opendir: Permission denied) 2024-09-21T03:42:13.329 INFO:tasks.workunit.client.0.smithi089.stdout:scsi_eh_4 459 root cwd unknown /proc/459/cwd (readlink: Permission denied) 2024-09-21T03:42:13.329 INFO:tasks.workunit.client.0.smithi089.stdout:scsi_eh_4 459 root rtd unknown /proc/459/root (readlink: Permission denied) 2024-09-21T03:42:13.329 INFO:tasks.workunit.client.0.smithi089.stdout:scsi_eh_4 459 root txt unknown /proc/459/exe (readlink: Permission denied) 2024-09-21T03:42:13.329 INFO:tasks.workunit.client.0.smithi089.stdout:scsi_eh_4 459 root NOFD /proc/459/fd (opendir: Permission denied) 2024-09-21T03:42:13.329 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/R 460 root cwd unknown /proc/460/cwd (readlink: Permission denied) 2024-09-21T03:42:13.329 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/R 460 root rtd unknown /proc/460/root (readlink: Permission denied) 2024-09-21T03:42:13.329 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/R 460 root txt unknown /proc/460/exe (readlink: Permission denied) 2024-09-21T03:42:13.330 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/R 460 root NOFD /proc/460/fd (opendir: Permission denied) 2024-09-21T03:42:13.330 INFO:tasks.workunit.client.0.smithi089.stdout:scsi_eh_5 461 root cwd unknown /proc/461/cwd (readlink: Permission denied) 2024-09-21T03:42:13.330 INFO:tasks.workunit.client.0.smithi089.stdout:scsi_eh_5 461 root rtd unknown /proc/461/root (readlink: Permission denied) 2024-09-21T03:42:13.330 INFO:tasks.workunit.client.0.smithi089.stdout:scsi_eh_5 461 root txt unknown /proc/461/exe (readlink: Permission denied) 2024-09-21T03:42:13.330 INFO:tasks.workunit.client.0.smithi089.stdout:scsi_eh_5 461 root NOFD /proc/461/fd (opendir: Permission denied) 2024-09-21T03:42:13.330 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/R 462 root cwd unknown /proc/462/cwd (readlink: Permission denied) 2024-09-21T03:42:13.330 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/R 462 root rtd unknown /proc/462/root (readlink: Permission denied) 2024-09-21T03:42:13.330 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/R 462 root txt unknown /proc/462/exe (readlink: Permission denied) 2024-09-21T03:42:13.330 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/R 462 root NOFD /proc/462/fd (opendir: Permission denied) 2024-09-21T03:42:13.330 INFO:tasks.workunit.client.0.smithi089.stdout:scsi_eh_6 463 root cwd unknown /proc/463/cwd (readlink: Permission denied) 2024-09-21T03:42:13.330 INFO:tasks.workunit.client.0.smithi089.stdout:scsi_eh_6 463 root rtd unknown /proc/463/root (readlink: Permission denied) 2024-09-21T03:42:13.330 INFO:tasks.workunit.client.0.smithi089.stdout:scsi_eh_6 463 root txt unknown /proc/463/exe (readlink: Permission denied) 2024-09-21T03:42:13.330 INFO:tasks.workunit.client.0.smithi089.stdout:scsi_eh_6 463 root NOFD /proc/463/fd (opendir: Permission denied) 2024-09-21T03:42:13.330 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/R 464 root cwd unknown /proc/464/cwd (readlink: Permission denied) 2024-09-21T03:42:13.330 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/R 464 root rtd unknown /proc/464/root (readlink: Permission denied) 2024-09-21T03:42:13.331 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/R 464 root txt unknown /proc/464/exe (readlink: Permission denied) 2024-09-21T03:42:13.331 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/R 464 root NOFD /proc/464/fd (opendir: Permission denied) 2024-09-21T03:42:13.331 INFO:tasks.workunit.client.0.smithi089.stdout:scsi_eh_7 465 root cwd unknown /proc/465/cwd (readlink: Permission denied) 2024-09-21T03:42:13.331 INFO:tasks.workunit.client.0.smithi089.stdout:scsi_eh_7 465 root rtd unknown /proc/465/root (readlink: Permission denied) 2024-09-21T03:42:13.331 INFO:tasks.workunit.client.0.smithi089.stdout:scsi_eh_7 465 root txt unknown /proc/465/exe (readlink: Permission denied) 2024-09-21T03:42:13.331 INFO:tasks.workunit.client.0.smithi089.stdout:scsi_eh_7 465 root NOFD /proc/465/fd (opendir: Permission denied) 2024-09-21T03:42:13.331 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/R 466 root cwd unknown /proc/466/cwd (readlink: Permission denied) 2024-09-21T03:42:13.331 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/R 466 root rtd unknown /proc/466/root (readlink: Permission denied) 2024-09-21T03:42:13.331 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/R 466 root txt unknown /proc/466/exe (readlink: Permission denied) 2024-09-21T03:42:13.331 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/R 466 root NOFD /proc/466/fd (opendir: Permission denied) 2024-09-21T03:42:13.331 INFO:tasks.workunit.client.0.smithi089.stdout:scsi_eh_8 467 root cwd unknown /proc/467/cwd (readlink: Permission denied) 2024-09-21T03:42:13.331 INFO:tasks.workunit.client.0.smithi089.stdout:scsi_eh_8 467 root rtd unknown /proc/467/root (readlink: Permission denied) 2024-09-21T03:42:13.331 INFO:tasks.workunit.client.0.smithi089.stdout:scsi_eh_8 467 root txt unknown /proc/467/exe (readlink: Permission denied) 2024-09-21T03:42:13.331 INFO:tasks.workunit.client.0.smithi089.stdout:scsi_eh_8 467 root NOFD /proc/467/fd (opendir: Permission denied) 2024-09-21T03:42:13.332 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/R 468 root cwd unknown /proc/468/cwd (readlink: Permission denied) 2024-09-21T03:42:13.332 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/R 468 root rtd unknown /proc/468/root (readlink: Permission denied) 2024-09-21T03:42:13.332 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/R 468 root txt unknown /proc/468/exe (readlink: Permission denied) 2024-09-21T03:42:13.332 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/R 468 root NOFD /proc/468/fd (opendir: Permission denied) 2024-09-21T03:42:13.332 INFO:tasks.workunit.client.0.smithi089.stdout:scsi_eh_9 469 root cwd unknown /proc/469/cwd (readlink: Permission denied) 2024-09-21T03:42:13.332 INFO:tasks.workunit.client.0.smithi089.stdout:scsi_eh_9 469 root rtd unknown /proc/469/root (readlink: Permission denied) 2024-09-21T03:42:13.332 INFO:tasks.workunit.client.0.smithi089.stdout:scsi_eh_9 469 root txt unknown /proc/469/exe (readlink: Permission denied) 2024-09-21T03:42:13.332 INFO:tasks.workunit.client.0.smithi089.stdout:scsi_eh_9 469 root NOFD /proc/469/fd (opendir: Permission denied) 2024-09-21T03:42:13.332 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/R 470 root cwd unknown /proc/470/cwd (readlink: Permission denied) 2024-09-21T03:42:13.332 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/R 470 root rtd unknown /proc/470/root (readlink: Permission denied) 2024-09-21T03:42:13.332 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/R 470 root txt unknown /proc/470/exe (readlink: Permission denied) 2024-09-21T03:42:13.332 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/R 470 root NOFD /proc/470/fd (opendir: Permission denied) 2024-09-21T03:42:13.332 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/u 472 root cwd unknown /proc/472/cwd (readlink: Permission denied) 2024-09-21T03:42:13.332 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/u 472 root rtd unknown /proc/472/root (readlink: Permission denied) 2024-09-21T03:42:13.333 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/u 472 root txt unknown /proc/472/exe (readlink: Permission denied) 2024-09-21T03:42:13.333 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/u 472 root NOFD /proc/472/fd (opendir: Permission denied) 2024-09-21T03:42:13.333 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/1 498 root cwd unknown /proc/498/cwd (readlink: Permission denied) 2024-09-21T03:42:13.333 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/1 498 root rtd unknown /proc/498/root (readlink: Permission denied) 2024-09-21T03:42:13.333 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/1 498 root txt unknown /proc/498/exe (readlink: Permission denied) 2024-09-21T03:42:13.333 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/1 498 root NOFD /proc/498/fd (opendir: Permission denied) 2024-09-21T03:42:13.333 INFO:tasks.workunit.client.0.smithi089.stdout:jbd2/sda1 502 root cwd unknown /proc/502/cwd (readlink: Permission denied) 2024-09-21T03:42:13.333 INFO:tasks.workunit.client.0.smithi089.stdout:jbd2/sda1 502 root rtd unknown /proc/502/root (readlink: Permission denied) 2024-09-21T03:42:13.333 INFO:tasks.workunit.client.0.smithi089.stdout:jbd2/sda1 502 root txt unknown /proc/502/exe (readlink: Permission denied) 2024-09-21T03:42:13.333 INFO:tasks.workunit.client.0.smithi089.stdout:jbd2/sda1 502 root NOFD /proc/502/fd (opendir: Permission denied) 2024-09-21T03:42:13.333 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/R 503 root cwd unknown /proc/503/cwd (readlink: Permission denied) 2024-09-21T03:42:13.333 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/R 503 root rtd unknown /proc/503/root (readlink: Permission denied) 2024-09-21T03:42:13.333 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/R 503 root txt unknown /proc/503/exe (readlink: Permission denied) 2024-09-21T03:42:13.333 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/R 503 root NOFD /proc/503/fd (opendir: Permission denied) 2024-09-21T03:42:13.333 INFO:tasks.workunit.client.0.smithi089.stdout:systemd-j 579 root cwd unknown /proc/579/cwd (readlink: Permission denied) 2024-09-21T03:42:13.334 INFO:tasks.workunit.client.0.smithi089.stdout:systemd-j 579 root rtd unknown /proc/579/root (readlink: Permission denied) 2024-09-21T03:42:13.334 INFO:tasks.workunit.client.0.smithi089.stdout:systemd-j 579 root txt unknown /proc/579/exe (readlink: Permission denied) 2024-09-21T03:42:13.334 INFO:tasks.workunit.client.0.smithi089.stdout:systemd-j 579 root NOFD /proc/579/fd (opendir: Permission denied) 2024-09-21T03:42:13.334 INFO:tasks.workunit.client.0.smithi089.stdout:ext4lazyi 585 root cwd unknown /proc/585/cwd (readlink: Permission denied) 2024-09-21T03:42:13.334 INFO:tasks.workunit.client.0.smithi089.stdout:ext4lazyi 585 root rtd unknown /proc/585/root (readlink: Permission denied) 2024-09-21T03:42:13.334 INFO:tasks.workunit.client.0.smithi089.stdout:ext4lazyi 585 root txt unknown /proc/585/exe (readlink: Permission denied) 2024-09-21T03:42:13.334 INFO:tasks.workunit.client.0.smithi089.stdout:ext4lazyi 585 root NOFD /proc/585/fd (opendir: Permission denied) 2024-09-21T03:42:13.334 INFO:tasks.workunit.client.0.smithi089.stdout:systemd-u 595 root cwd unknown /proc/595/cwd (readlink: Permission denied) 2024-09-21T03:42:13.334 INFO:tasks.workunit.client.0.smithi089.stdout:systemd-u 595 root rtd unknown /proc/595/root (readlink: Permission denied) 2024-09-21T03:42:13.334 INFO:tasks.workunit.client.0.smithi089.stdout:systemd-u 595 root txt unknown /proc/595/exe (readlink: Permission denied) 2024-09-21T03:42:13.334 INFO:tasks.workunit.client.0.smithi089.stdout:systemd-u 595 root NOFD /proc/595/fd (opendir: Permission denied) 2024-09-21T03:42:13.334 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/R 637 root cwd unknown /proc/637/cwd (readlink: Permission denied) 2024-09-21T03:42:13.334 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/R 637 root rtd unknown /proc/637/root (readlink: Permission denied) 2024-09-21T03:42:13.335 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/R 637 root txt unknown /proc/637/exe (readlink: Permission denied) 2024-09-21T03:42:13.335 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/R 637 root NOFD /proc/637/fd (opendir: Permission denied) 2024-09-21T03:42:13.335 INFO:tasks.workunit.client.0.smithi089.stdout:kipmi0 652 root cwd unknown /proc/652/cwd (readlink: Permission denied) 2024-09-21T03:42:13.335 INFO:tasks.workunit.client.0.smithi089.stdout:kipmi0 652 root rtd unknown /proc/652/root (readlink: Permission denied) 2024-09-21T03:42:13.335 INFO:tasks.workunit.client.0.smithi089.stdout:kipmi0 652 root txt unknown /proc/652/exe (readlink: Permission denied) 2024-09-21T03:42:13.335 INFO:tasks.workunit.client.0.smithi089.stdout:kipmi0 652 root NOFD /proc/652/fd (opendir: Permission denied) 2024-09-21T03:42:13.335 INFO:tasks.workunit.client.0.smithi089.stdout:rpcbind 663 rpc cwd unknown /proc/663/cwd (readlink: Permission denied) 2024-09-21T03:42:13.335 INFO:tasks.workunit.client.0.smithi089.stdout:rpcbind 663 rpc rtd unknown /proc/663/root (readlink: Permission denied) 2024-09-21T03:42:13.335 INFO:tasks.workunit.client.0.smithi089.stdout:rpcbind 663 rpc txt unknown /proc/663/exe (readlink: Permission denied) 2024-09-21T03:42:13.335 INFO:tasks.workunit.client.0.smithi089.stdout:rpcbind 663 rpc NOFD /proc/663/fd (opendir: Permission denied) 2024-09-21T03:42:13.335 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/R 671 root cwd unknown /proc/671/cwd (readlink: Permission denied) 2024-09-21T03:42:13.335 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/R 671 root rtd unknown /proc/671/root (readlink: Permission denied) 2024-09-21T03:42:13.335 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/R 671 root txt unknown /proc/671/exe (readlink: Permission denied) 2024-09-21T03:42:13.335 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/R 671 root NOFD /proc/671/fd (opendir: Permission denied) 2024-09-21T03:42:13.336 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/R 673 root cwd unknown /proc/673/cwd (readlink: Permission denied) 2024-09-21T03:42:13.336 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/R 673 root rtd unknown /proc/673/root (readlink: Permission denied) 2024-09-21T03:42:13.336 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/R 673 root txt unknown /proc/673/exe (readlink: Permission denied) 2024-09-21T03:42:13.336 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/R 673 root NOFD /proc/673/fd (opendir: Permission denied) 2024-09-21T03:42:13.336 INFO:tasks.workunit.client.0.smithi089.stdout:auditd 676 root cwd unknown /proc/676/cwd (readlink: Permission denied) 2024-09-21T03:42:13.336 INFO:tasks.workunit.client.0.smithi089.stdout:auditd 676 root rtd unknown /proc/676/root (readlink: Permission denied) 2024-09-21T03:42:13.336 INFO:tasks.workunit.client.0.smithi089.stdout:auditd 676 root txt unknown /proc/676/exe (readlink: Permission denied) 2024-09-21T03:42:13.336 INFO:tasks.workunit.client.0.smithi089.stdout:auditd 676 root NOFD /proc/676/fd (opendir: Permission denied) 2024-09-21T03:42:13.336 INFO:tasks.workunit.client.0.smithi089.stdout:auditd 676 677 auditd root cwd unknown /proc/676/task/677/cwd (readlink: Permission denied) 2024-09-21T03:42:13.336 INFO:tasks.workunit.client.0.smithi089.stdout:auditd 676 677 auditd root rtd unknown /proc/676/task/677/root (readlink: Permission denied) 2024-09-21T03:42:13.336 INFO:tasks.workunit.client.0.smithi089.stdout:auditd 676 677 auditd root txt unknown /proc/676/task/677/exe (readlink: Permission denied) 2024-09-21T03:42:13.336 INFO:tasks.workunit.client.0.smithi089.stdout:auditd 676 677 auditd root NOFD /proc/676/task/677/fd (opendir: Permission denied) 2024-09-21T03:42:13.336 INFO:tasks.workunit.client.0.smithi089.stdout:auditd 676 679 auditd root cwd unknown /proc/676/task/679/cwd (readlink: Permission denied) 2024-09-21T03:42:13.336 INFO:tasks.workunit.client.0.smithi089.stdout:auditd 676 679 auditd root rtd unknown /proc/676/task/679/root (readlink: Permission denied) 2024-09-21T03:42:13.337 INFO:tasks.workunit.client.0.smithi089.stdout:auditd 676 679 auditd root txt unknown /proc/676/task/679/exe (readlink: Permission denied) 2024-09-21T03:42:13.337 INFO:tasks.workunit.client.0.smithi089.stdout:auditd 676 679 auditd root NOFD /proc/676/task/679/fd (opendir: Permission denied) 2024-09-21T03:42:13.337 INFO:tasks.workunit.client.0.smithi089.stdout:sedispatc 678 root cwd unknown /proc/678/cwd (readlink: Permission denied) 2024-09-21T03:42:13.337 INFO:tasks.workunit.client.0.smithi089.stdout:sedispatc 678 root rtd unknown /proc/678/root (readlink: Permission denied) 2024-09-21T03:42:13.337 INFO:tasks.workunit.client.0.smithi089.stdout:sedispatc 678 root txt unknown /proc/678/exe (readlink: Permission denied) 2024-09-21T03:42:13.337 INFO:tasks.workunit.client.0.smithi089.stdout:sedispatc 678 root NOFD /proc/678/fd (opendir: Permission denied) 2024-09-21T03:42:13.337 INFO:tasks.workunit.client.0.smithi089.stdout:dbus-brok 720 dbus cwd unknown /proc/720/cwd (readlink: Permission denied) 2024-09-21T03:42:13.337 INFO:tasks.workunit.client.0.smithi089.stdout:dbus-brok 720 dbus rtd unknown /proc/720/root (readlink: Permission denied) 2024-09-21T03:42:13.337 INFO:tasks.workunit.client.0.smithi089.stdout:dbus-brok 720 dbus txt unknown /proc/720/exe (readlink: Permission denied) 2024-09-21T03:42:13.337 INFO:tasks.workunit.client.0.smithi089.stdout:dbus-brok 720 dbus NOFD /proc/720/fd (opendir: Permission denied) 2024-09-21T03:42:13.337 INFO:tasks.workunit.client.0.smithi089.stdout:dbus-brok 721 dbus cwd unknown /proc/721/cwd (readlink: Permission denied) 2024-09-21T03:42:13.337 INFO:tasks.workunit.client.0.smithi089.stdout:dbus-brok 721 dbus rtd unknown /proc/721/root (readlink: Permission denied) 2024-09-21T03:42:13.337 INFO:tasks.workunit.client.0.smithi089.stdout:dbus-brok 721 dbus txt unknown /proc/721/exe (readlink: Permission denied) 2024-09-21T03:42:13.337 INFO:tasks.workunit.client.0.smithi089.stdout:dbus-brok 721 dbus NOFD /proc/721/fd (opendir: Permission denied) 2024-09-21T03:42:13.338 INFO:tasks.workunit.client.0.smithi089.stdout:NetworkMa 722 root cwd unknown /proc/722/cwd (readlink: Permission denied) 2024-09-21T03:42:13.338 INFO:tasks.workunit.client.0.smithi089.stdout:NetworkMa 722 root rtd unknown /proc/722/root (readlink: Permission denied) 2024-09-21T03:42:13.338 INFO:tasks.workunit.client.0.smithi089.stdout:NetworkMa 722 root txt unknown /proc/722/exe (readlink: Permission denied) 2024-09-21T03:42:13.338 INFO:tasks.workunit.client.0.smithi089.stdout:NetworkMa 722 root NOFD /proc/722/fd (opendir: Permission denied) 2024-09-21T03:42:13.338 INFO:tasks.workunit.client.0.smithi089.stdout:NetworkMa 722 739 gmain root cwd unknown /proc/722/task/739/cwd (readlink: Permission denied) 2024-09-21T03:42:13.338 INFO:tasks.workunit.client.0.smithi089.stdout:NetworkMa 722 739 gmain root rtd unknown /proc/722/task/739/root (readlink: Permission denied) 2024-09-21T03:42:13.338 INFO:tasks.workunit.client.0.smithi089.stdout:NetworkMa 722 739 gmain root txt unknown /proc/722/task/739/exe (readlink: Permission denied) 2024-09-21T03:42:13.338 INFO:tasks.workunit.client.0.smithi089.stdout:NetworkMa 722 739 gmain root NOFD /proc/722/task/739/fd (opendir: Permission denied) 2024-09-21T03:42:13.338 INFO:tasks.workunit.client.0.smithi089.stdout:NetworkMa 722 740 gdbus root cwd unknown /proc/722/task/740/cwd (readlink: Permission denied) 2024-09-21T03:42:13.338 INFO:tasks.workunit.client.0.smithi089.stdout:NetworkMa 722 740 gdbus root rtd unknown /proc/722/task/740/root (readlink: Permission denied) 2024-09-21T03:42:13.338 INFO:tasks.workunit.client.0.smithi089.stdout:NetworkMa 722 740 gdbus root txt unknown /proc/722/task/740/exe (readlink: Permission denied) 2024-09-21T03:42:13.338 INFO:tasks.workunit.client.0.smithi089.stdout:NetworkMa 722 740 gdbus root NOFD /proc/722/task/740/fd (opendir: Permission denied) 2024-09-21T03:42:13.338 INFO:tasks.workunit.client.0.smithi089.stdout:irqbalanc 725 root cwd unknown /proc/725/cwd (readlink: Permission denied) 2024-09-21T03:42:13.339 INFO:tasks.workunit.client.0.smithi089.stdout:irqbalanc 725 root rtd unknown /proc/725/root (readlink: Permission denied) 2024-09-21T03:42:13.339 INFO:tasks.workunit.client.0.smithi089.stdout:irqbalanc 725 root txt unknown /proc/725/exe (readlink: Permission denied) 2024-09-21T03:42:13.339 INFO:tasks.workunit.client.0.smithi089.stdout:irqbalanc 725 root NOFD /proc/725/fd (opendir: Permission denied) 2024-09-21T03:42:13.339 INFO:tasks.workunit.client.0.smithi089.stdout:irqbalanc 725 732 gmain root cwd unknown /proc/725/task/732/cwd (readlink: Permission denied) 2024-09-21T03:42:13.339 INFO:tasks.workunit.client.0.smithi089.stdout:irqbalanc 725 732 gmain root rtd unknown /proc/725/task/732/root (readlink: Permission denied) 2024-09-21T03:42:13.339 INFO:tasks.workunit.client.0.smithi089.stdout:irqbalanc 725 732 gmain root txt unknown /proc/725/task/732/exe (readlink: Permission denied) 2024-09-21T03:42:13.339 INFO:tasks.workunit.client.0.smithi089.stdout:irqbalanc 725 732 gmain root NOFD /proc/725/task/732/fd (opendir: Permission denied) 2024-09-21T03:42:13.339 INFO:tasks.workunit.client.0.smithi089.stdout:lsmd 726 libstoragemgmt cwd unknown /proc/726/cwd (readlink: Permission denied) 2024-09-21T03:42:13.339 INFO:tasks.workunit.client.0.smithi089.stdout:lsmd 726 libstoragemgmt rtd unknown /proc/726/root (readlink: Permission denied) 2024-09-21T03:42:13.339 INFO:tasks.workunit.client.0.smithi089.stdout:lsmd 726 libstoragemgmt txt unknown /proc/726/exe (readlink: Permission denied) 2024-09-21T03:42:13.339 INFO:tasks.workunit.client.0.smithi089.stdout:lsmd 726 libstoragemgmt NOFD /proc/726/fd (opendir: Permission denied) 2024-09-21T03:42:13.339 INFO:tasks.workunit.client.0.smithi089.stdout:mcelog 727 root cwd unknown /proc/727/cwd (readlink: Permission denied) 2024-09-21T03:42:13.339 INFO:tasks.workunit.client.0.smithi089.stdout:mcelog 727 root rtd unknown /proc/727/root (readlink: Permission denied) 2024-09-21T03:42:13.339 INFO:tasks.workunit.client.0.smithi089.stdout:mcelog 727 root txt unknown /proc/727/exe (readlink: Permission denied) 2024-09-21T03:42:13.340 INFO:tasks.workunit.client.0.smithi089.stdout:mcelog 727 root NOFD /proc/727/fd (opendir: Permission denied) 2024-09-21T03:42:13.340 INFO:tasks.workunit.client.0.smithi089.stdout:smartd 728 root cwd unknown /proc/728/cwd (readlink: Permission denied) 2024-09-21T03:42:13.340 INFO:tasks.workunit.client.0.smithi089.stdout:smartd 728 root rtd unknown /proc/728/root (readlink: Permission denied) 2024-09-21T03:42:13.340 INFO:tasks.workunit.client.0.smithi089.stdout:smartd 728 root txt unknown /proc/728/exe (readlink: Permission denied) 2024-09-21T03:42:13.340 INFO:tasks.workunit.client.0.smithi089.stdout:smartd 728 root NOFD /proc/728/fd (opendir: Permission denied) 2024-09-21T03:42:13.340 INFO:tasks.workunit.client.0.smithi089.stdout:systemd-l 730 root cwd unknown /proc/730/cwd (readlink: Permission denied) 2024-09-21T03:42:13.340 INFO:tasks.workunit.client.0.smithi089.stdout:systemd-l 730 root rtd unknown /proc/730/root (readlink: Permission denied) 2024-09-21T03:42:13.340 INFO:tasks.workunit.client.0.smithi089.stdout:systemd-l 730 root txt unknown /proc/730/exe (readlink: Permission denied) 2024-09-21T03:42:13.340 INFO:tasks.workunit.client.0.smithi089.stdout:systemd-l 730 root NOFD /proc/730/fd (opendir: Permission denied) 2024-09-21T03:42:13.340 INFO:tasks.workunit.client.0.smithi089.stdout:sshd 774 root cwd unknown /proc/774/cwd (readlink: Permission denied) 2024-09-21T03:42:13.340 INFO:tasks.workunit.client.0.smithi089.stdout:sshd 774 root rtd unknown /proc/774/root (readlink: Permission denied) 2024-09-21T03:42:13.340 INFO:tasks.workunit.client.0.smithi089.stdout:sshd 774 root txt unknown /proc/774/exe (readlink: Permission denied) 2024-09-21T03:42:13.340 INFO:tasks.workunit.client.0.smithi089.stdout:sshd 774 root NOFD /proc/774/fd (opendir: Permission denied) 2024-09-21T03:42:13.340 INFO:tasks.workunit.client.0.smithi089.stdout:gssproxy 778 root cwd unknown /proc/778/cwd (readlink: Permission denied) 2024-09-21T03:42:13.341 INFO:tasks.workunit.client.0.smithi089.stdout:gssproxy 778 root rtd unknown /proc/778/root (readlink: Permission denied) 2024-09-21T03:42:13.341 INFO:tasks.workunit.client.0.smithi089.stdout:gssproxy 778 root txt unknown /proc/778/exe (readlink: Permission denied) 2024-09-21T03:42:13.341 INFO:tasks.workunit.client.0.smithi089.stdout:gssproxy 778 root NOFD /proc/778/fd (opendir: Permission denied) 2024-09-21T03:42:13.341 INFO:tasks.workunit.client.0.smithi089.stdout:gssproxy 778 779 gssproxy root cwd unknown /proc/778/task/779/cwd (readlink: Permission denied) 2024-09-21T03:42:13.341 INFO:tasks.workunit.client.0.smithi089.stdout:gssproxy 778 779 gssproxy root rtd unknown /proc/778/task/779/root (readlink: Permission denied) 2024-09-21T03:42:13.341 INFO:tasks.workunit.client.0.smithi089.stdout:gssproxy 778 779 gssproxy root txt unknown /proc/778/task/779/exe (readlink: Permission denied) 2024-09-21T03:42:13.341 INFO:tasks.workunit.client.0.smithi089.stdout:gssproxy 778 779 gssproxy root NOFD /proc/778/task/779/fd (opendir: Permission denied) 2024-09-21T03:42:13.341 INFO:tasks.workunit.client.0.smithi089.stdout:gssproxy 778 780 gssproxy root cwd unknown /proc/778/task/780/cwd (readlink: Permission denied) 2024-09-21T03:42:13.341 INFO:tasks.workunit.client.0.smithi089.stdout:gssproxy 778 780 gssproxy root rtd unknown /proc/778/task/780/root (readlink: Permission denied) 2024-09-21T03:42:13.341 INFO:tasks.workunit.client.0.smithi089.stdout:gssproxy 778 780 gssproxy root txt unknown /proc/778/task/780/exe (readlink: Permission denied) 2024-09-21T03:42:13.341 INFO:tasks.workunit.client.0.smithi089.stdout:gssproxy 778 780 gssproxy root NOFD /proc/778/task/780/fd (opendir: Permission denied) 2024-09-21T03:42:13.341 INFO:tasks.workunit.client.0.smithi089.stdout:gssproxy 778 781 gssproxy root cwd unknown /proc/778/task/781/cwd (readlink: Permission denied) 2024-09-21T03:42:13.341 INFO:tasks.workunit.client.0.smithi089.stdout:gssproxy 778 781 gssproxy root rtd unknown /proc/778/task/781/root (readlink: Permission denied) 2024-09-21T03:42:13.341 INFO:tasks.workunit.client.0.smithi089.stdout:gssproxy 778 781 gssproxy root txt unknown /proc/778/task/781/exe (readlink: Permission denied) 2024-09-21T03:42:13.342 INFO:tasks.workunit.client.0.smithi089.stdout:gssproxy 778 781 gssproxy root NOFD /proc/778/task/781/fd (opendir: Permission denied) 2024-09-21T03:42:13.342 INFO:tasks.workunit.client.0.smithi089.stdout:gssproxy 778 782 gssproxy root cwd unknown /proc/778/task/782/cwd (readlink: Permission denied) 2024-09-21T03:42:13.342 INFO:tasks.workunit.client.0.smithi089.stdout:gssproxy 778 782 gssproxy root rtd unknown /proc/778/task/782/root (readlink: Permission denied) 2024-09-21T03:42:13.342 INFO:tasks.workunit.client.0.smithi089.stdout:gssproxy 778 782 gssproxy root txt unknown /proc/778/task/782/exe (readlink: Permission denied) 2024-09-21T03:42:13.342 INFO:tasks.workunit.client.0.smithi089.stdout:gssproxy 778 782 gssproxy root NOFD /proc/778/task/782/fd (opendir: Permission denied) 2024-09-21T03:42:13.342 INFO:tasks.workunit.client.0.smithi089.stdout:gssproxy 778 783 gssproxy root cwd unknown /proc/778/task/783/cwd (readlink: Permission denied) 2024-09-21T03:42:13.342 INFO:tasks.workunit.client.0.smithi089.stdout:gssproxy 778 783 gssproxy root rtd unknown /proc/778/task/783/root (readlink: Permission denied) 2024-09-21T03:42:13.342 INFO:tasks.workunit.client.0.smithi089.stdout:gssproxy 778 783 gssproxy root txt unknown /proc/778/task/783/exe (readlink: Permission denied) 2024-09-21T03:42:13.342 INFO:tasks.workunit.client.0.smithi089.stdout:gssproxy 778 783 gssproxy root NOFD /proc/778/task/783/fd (opendir: Permission denied) 2024-09-21T03:42:13.342 INFO:tasks.workunit.client.0.smithi089.stdout:atd 785 root cwd unknown /proc/785/cwd (readlink: Permission denied) 2024-09-21T03:42:13.342 INFO:tasks.workunit.client.0.smithi089.stdout:atd 785 root rtd unknown /proc/785/root (readlink: Permission denied) 2024-09-21T03:42:13.342 INFO:tasks.workunit.client.0.smithi089.stdout:atd 785 root txt unknown /proc/785/exe (readlink: Permission denied) 2024-09-21T03:42:13.342 INFO:tasks.workunit.client.0.smithi089.stdout:atd 785 root NOFD /proc/785/fd (opendir: Permission denied) 2024-09-21T03:42:13.342 INFO:tasks.workunit.client.0.smithi089.stdout:crond 786 root cwd unknown /proc/786/cwd (readlink: Permission denied) 2024-09-21T03:42:13.343 INFO:tasks.workunit.client.0.smithi089.stdout:crond 786 root rtd unknown /proc/786/root (readlink: Permission denied) 2024-09-21T03:42:13.343 INFO:tasks.workunit.client.0.smithi089.stdout:crond 786 root txt unknown /proc/786/exe (readlink: Permission denied) 2024-09-21T03:42:13.343 INFO:tasks.workunit.client.0.smithi089.stdout:crond 786 root NOFD /proc/786/fd (opendir: Permission denied) 2024-09-21T03:42:13.343 INFO:tasks.workunit.client.0.smithi089.stdout:nrpe 812 nrpe cwd unknown /proc/812/cwd (readlink: Permission denied) 2024-09-21T03:42:13.343 INFO:tasks.workunit.client.0.smithi089.stdout:nrpe 812 nrpe rtd unknown /proc/812/root (readlink: Permission denied) 2024-09-21T03:42:13.343 INFO:tasks.workunit.client.0.smithi089.stdout:nrpe 812 nrpe txt unknown /proc/812/exe (readlink: Permission denied) 2024-09-21T03:42:13.343 INFO:tasks.workunit.client.0.smithi089.stdout:nrpe 812 nrpe NOFD /proc/812/fd (opendir: Permission denied) 2024-09-21T03:42:13.343 INFO:tasks.workunit.client.0.smithi089.stdout:agetty 1206 root cwd unknown /proc/1206/cwd (readlink: Permission denied) 2024-09-21T03:42:13.343 INFO:tasks.workunit.client.0.smithi089.stdout:agetty 1206 root rtd unknown /proc/1206/root (readlink: Permission denied) 2024-09-21T03:42:13.343 INFO:tasks.workunit.client.0.smithi089.stdout:agetty 1206 root txt unknown /proc/1206/exe (readlink: Permission denied) 2024-09-21T03:42:13.343 INFO:tasks.workunit.client.0.smithi089.stdout:agetty 1206 root NOFD /proc/1206/fd (opendir: Permission denied) 2024-09-21T03:42:13.343 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/1 1210 root cwd unknown /proc/1210/cwd (readlink: Permission denied) 2024-09-21T03:42:13.343 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/1 1210 root rtd unknown /proc/1210/root (readlink: Permission denied) 2024-09-21T03:42:13.343 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/1 1210 root txt unknown /proc/1210/exe (readlink: Permission denied) 2024-09-21T03:42:13.344 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/1 1210 root NOFD /proc/1210/fd (opendir: Permission denied) 2024-09-21T03:42:13.344 INFO:tasks.workunit.client.0.smithi089.stdout:agetty 1211 root cwd unknown /proc/1211/cwd (readlink: Permission denied) 2024-09-21T03:42:13.344 INFO:tasks.workunit.client.0.smithi089.stdout:agetty 1211 root rtd unknown /proc/1211/root (readlink: Permission denied) 2024-09-21T03:42:13.344 INFO:tasks.workunit.client.0.smithi089.stdout:agetty 1211 root txt unknown /proc/1211/exe (readlink: Permission denied) 2024-09-21T03:42:13.344 INFO:tasks.workunit.client.0.smithi089.stdout:agetty 1211 root NOFD /proc/1211/fd (opendir: Permission denied) 2024-09-21T03:42:13.344 INFO:tasks.workunit.client.0.smithi089.stdout:systemd 4589 ubuntu cwd DIR 8,1 4096 2 / 2024-09-21T03:42:13.344 INFO:tasks.workunit.client.0.smithi089.stdout:systemd 4589 ubuntu rtd DIR 8,1 4096 2 / 2024-09-21T03:42:13.344 INFO:tasks.workunit.client.0.smithi089.stdout:systemd 4589 ubuntu txt REG 8,1 98208 22394 /usr/lib/systemd/systemd 2024-09-21T03:42:13.344 INFO:tasks.workunit.client.0.smithi089.stdout:systemd 4589 ubuntu mem REG 8,1 21219 2231 /etc/selinux/targeted/contexts/files/file_contexts.homedirs.bin 2024-09-21T03:42:13.344 INFO:tasks.workunit.client.0.smithi089.stdout:systemd 4589 ubuntu mem REG 8,1 600667 2132 /etc/selinux/targeted/contexts/files/file_contexts.bin 2024-09-21T03:42:13.344 INFO:tasks.workunit.client.0.smithi089.stdout:systemd 4589 ubuntu mem REG 8,1 636848 11327 /usr/lib64/libpcre2-8.so.0.11.0 2024-09-21T03:42:13.344 INFO:tasks.workunit.client.0.smithi089.stdout:systemd 4589 ubuntu mem REG 8,1 912440 11215 /usr/lib64/libm.so.6 2024-09-21T03:42:13.344 INFO:tasks.workunit.client.0.smithi089.stdout:systemd 4589 ubuntu mem REG 8,1 178728 11282 /usr/lib64/liblzma.so.5.2.5 2024-09-21T03:42:13.344 INFO:tasks.workunit.client.0.smithi089.stdout:systemd 4589 ubuntu mem REG 8,1 1714208 11376 /usr/lib64/libp11-kit.so.0.3.1 2024-09-21T03:42:13.345 INFO:tasks.workunit.client.0.smithi089.stdout:systemd 4589 ubuntu DEL REG 8,1 11595 /usr/lib64/libcrypto.so.3.2.2 2024-09-21T03:42:13.345 INFO:tasks.workunit.client.0.smithi089.stdout:systemd 4589 ubuntu mem REG 8,1 44784 11373 /usr/lib64/libffi.so.8.1.0 2024-09-21T03:42:13.345 INFO:tasks.workunit.client.0.smithi089.stdout:systemd 4589 ubuntu mem REG 8,1 882376 11287 /usr/lib64/libzstd.so.1.5.1 2024-09-21T03:42:13.345 INFO:tasks.workunit.client.0.smithi089.stdout:systemd 4589 ubuntu mem REG 8,1 2544296 11212 /usr/lib64/libc.so.6 2024-09-21T03:42:13.345 INFO:tasks.workunit.client.0.smithi089.stdout:systemd 4589 ubuntu mem REG 8,1 153600 11397 /usr/lib64/libgpg-error.so.0.32.0 2024-09-21T03:42:13.345 INFO:tasks.workunit.client.0.smithi089.stdout:systemd 4589 ubuntu mem REG 8,1 144120 11455 /usr/lib64/liblz4.so.1.9.3 2024-09-21T03:42:13.345 INFO:tasks.workunit.client.0.smithi089.stdout:systemd 4589 ubuntu mem REG 8,1 1304864 11471 /usr/lib64/libgcrypt.so.20.4.0 2024-09-21T03:42:13.345 INFO:tasks.workunit.client.0.smithi089.stdout:systemd 4589 ubuntu mem REG 8,1 201808 11284 /usr/lib64/libcrypt.so.2.0.0 2024-09-21T03:42:13.345 INFO:tasks.workunit.client.0.smithi089.stdout:systemd 4589 ubuntu mem REG 8,1 229424 11714 /usr/lib64/libblkid.so.1.1.0 2024-09-21T03:42:13.345 INFO:tasks.workunit.client.0.smithi089.stdout:systemd 4589 ubuntu mem REG 8,1 3826840 50305 /usr/lib64/systemd/libsystemd-shared-252.so 2024-09-21T03:42:13.345 INFO:tasks.workunit.client.0.smithi089.stdout:systemd 4589 ubuntu mem REG 8,1 28552 11392 /usr/lib64/libattr.so.1.1.2501 2024-09-21T03:42:13.345 INFO:tasks.workunit.client.0.smithi089.stdout:systemd 4589 ubuntu mem REG 8,1 102672 11274 /usr/lib64/libz.so.1.2.11 2024-09-21T03:42:13.345 INFO:tasks.workunit.client.0.smithi089.stdout:systemd 4589 ubuntu mem REG 8,1 291496 11744 /usr/lib64/libmount.so.1.1.0 2024-09-21T03:42:13.345 INFO:tasks.workunit.client.0.smithi089.stdout:systemd 4589 ubuntu mem REG 8,1 2162040 50302 /usr/lib64/systemd/libsystemd-core-252.so 2024-09-21T03:42:13.345 INFO:tasks.workunit.client.0.smithi089.stdout:systemd 4589 ubuntu mem REG 8,1 32528 11311 /usr/lib64/libcap-ng.so.0.0.0 2024-09-21T03:42:13.346 INFO:tasks.workunit.client.0.smithi089.stdout:systemd 4589 ubuntu mem REG 8,1 40520 12544 /usr/lib64/libeconf.so.0.4.1 2024-09-21T03:42:13.346 INFO:tasks.workunit.client.0.smithi089.stdout:systemd 4589 ubuntu mem REG 8,1 36296 11290 /usr/lib64/libcap.so.2.48 2024-09-21T03:42:13.346 INFO:tasks.workunit.client.0.smithi089.stdout:systemd 4589 ubuntu mem REG 8,1 40496 11393 /usr/lib64/libacl.so.1.1.2301 2024-09-21T03:42:13.346 INFO:tasks.workunit.client.0.smithi089.stdout:systemd 4589 ubuntu mem REG 8,1 175744 11412 /usr/lib64/libselinux.so.1 2024-09-21T03:42:13.346 INFO:tasks.workunit.client.0.smithi089.stdout:systemd 4589 ubuntu mem REG 8,1 107528 11768 /usr/lib64/libkmod.so.2.3.6 2024-09-21T03:42:13.346 INFO:tasks.workunit.client.0.smithi089.stdout:systemd 4589 ubuntu mem REG 8,1 163464 11315 /usr/lib64/libaudit.so.1.0.0 2024-09-21T03:42:13.346 INFO:tasks.workunit.client.0.smithi089.stdout:systemd 4589 ubuntu mem REG 8,1 69592 12553 /usr/lib64/libpam.so.0.85.1 2024-09-21T03:42:13.346 INFO:tasks.workunit.client.0.smithi089.stdout:systemd 4589 ubuntu mem REG 8,1 127680 11560 /usr/lib64/libseccomp.so.2.5.2 2024-09-21T03:42:13.346 INFO:tasks.workunit.client.0.smithi089.stdout:systemd 4589 ubuntu mem REG 8,1 108136 14698 /usr/lib64/libgcc_s-11-20240719.so.1 2024-09-21T03:42:13.346 INFO:tasks.workunit.client.0.smithi089.stdout:systemd 4589 ubuntu mem REG 0,19 0 19 /sys/fs/selinux/status 2024-09-21T03:42:13.346 INFO:tasks.workunit.client.0.smithi089.stdout:systemd 4589 ubuntu mem REG 8,1 877424 15688 /usr/lib64/ld-linux-x86-64.so.2 2024-09-21T03:42:13.346 INFO:tasks.workunit.client.0.smithi089.stdout:systemd 4589 ubuntu 0r CHR 1,3 0t0 4 /dev/null 2024-09-21T03:42:13.346 INFO:tasks.workunit.client.0.smithi089.stdout:systemd 4589 ubuntu 1u unix 0x0000000000000000 0t0 22327 type=STREAM (CONNECTED) 2024-09-21T03:42:13.346 INFO:tasks.workunit.client.0.smithi089.stdout:systemd 4589 ubuntu 2u unix 0x0000000000000000 0t0 22327 type=STREAM (CONNECTED) 2024-09-21T03:42:13.347 INFO:tasks.workunit.client.0.smithi089.stdout:systemd 4589 ubuntu 3u unix 0x0000000000000000 0t0 23419 type=DGRAM (CONNECTED) 2024-09-21T03:42:13.347 INFO:tasks.workunit.client.0.smithi089.stdout:systemd 4589 ubuntu 4u a_inode 0,14 0 2081 [eventpoll:5,6,8,9,11,12,14,15,16,17,19,20,21] 2024-09-21T03:42:13.347 INFO:tasks.workunit.client.0.smithi089.stdout:systemd 4589 ubuntu 5u a_inode 0,14 0 2081 [signalfd] 2024-09-21T03:42:13.347 INFO:tasks.workunit.client.0.smithi089.stdout:systemd 4589 ubuntu 6r a_inode 0,14 0 2081 inotify 2024-09-21T03:42:13.347 INFO:tasks.workunit.client.0.smithi089.stdout:systemd 4589 ubuntu 7r DIR 0,25 0 3417 /sys/fs/cgroup/user.slice/user-1000.slice/user@1000.service 2024-09-21T03:42:13.347 INFO:tasks.workunit.client.0.smithi089.stdout:systemd 4589 ubuntu 8u a_inode 0,14 0 2081 [timerfd] 2024-09-21T03:42:13.347 INFO:tasks.workunit.client.0.smithi089.stdout:systemd 4589 ubuntu 9u a_inode 0,14 0 2081 [eventpoll:10,13] 2024-09-21T03:42:13.347 INFO:tasks.workunit.client.0.smithi089.stdout:systemd 4589 ubuntu 10r REG 0,20 0 23408 /proc/4589/mountinfo 2024-09-21T03:42:13.347 INFO:tasks.workunit.client.0.smithi089.stdout:systemd 4589 ubuntu 11r a_inode 0,14 0 2081 inotify 2024-09-21T03:42:13.347 INFO:tasks.workunit.client.0.smithi089.stdout:systemd 4589 ubuntu 12u unix 0x0000000000000000 0t0 22360 /run/user/1000/bus type=STREAM (LISTEN) 2024-09-21T03:42:13.347 INFO:tasks.workunit.client.0.smithi089.stdout:systemd 4589 ubuntu 13r a_inode 0,14 0 2081 inotify 2024-09-21T03:42:13.347 INFO:tasks.workunit.client.0.smithi089.stdout:systemd 4589 ubuntu 14r REG 0,20 0 4026532068 /proc/swaps 2024-09-21T03:42:13.347 INFO:tasks.workunit.client.0.smithi089.stdout:systemd 4589 ubuntu 15u netlink 0t0 23431 KOBJECT_UEVENT 2024-09-21T03:42:13.347 INFO:tasks.workunit.client.0.smithi089.stdout:systemd 4589 ubuntu 16u unix 0x0000000000000000 0t0 23436 /run/user/1000/systemd/notify type=DGRAM (UNCONNECTED) 2024-09-21T03:42:13.348 INFO:tasks.workunit.client.0.smithi089.stdout:systemd 4589 ubuntu 17u unix 0x0000000000000000 0t0 23437 type=DGRAM (CONNECTED) 2024-09-21T03:42:13.348 INFO:tasks.workunit.client.0.smithi089.stdout:systemd 4589 ubuntu 18u unix 0x0000000000000000 0t0 23438 type=DGRAM (CONNECTED) 2024-09-21T03:42:13.348 INFO:tasks.workunit.client.0.smithi089.stdout:systemd 4589 ubuntu 19u unix 0x0000000000000000 0t0 23439 /run/user/1000/systemd/private type=STREAM (LISTEN) 2024-09-21T03:42:13.348 INFO:tasks.workunit.client.0.smithi089.stdout:systemd 4589 ubuntu 20u unix 0x0000000000000000 0t0 23441 type=STREAM (CONNECTED) 2024-09-21T03:42:13.348 INFO:tasks.workunit.client.0.smithi089.stdout:systemd 4589 ubuntu 21u a_inode 0,14 0 2081 [timerfd] 2024-09-21T03:42:13.348 INFO:tasks.workunit.client.0.smithi089.stdout:(sd-pam) 4603 ubuntu cwd unknown /proc/4603/cwd (readlink: Permission denied) 2024-09-21T03:42:13.348 INFO:tasks.workunit.client.0.smithi089.stdout:(sd-pam) 4603 ubuntu rtd unknown /proc/4603/root (readlink: Permission denied) 2024-09-21T03:42:13.348 INFO:tasks.workunit.client.0.smithi089.stdout:(sd-pam) 4603 ubuntu txt unknown /proc/4603/exe (readlink: Permission denied) 2024-09-21T03:42:13.348 INFO:tasks.workunit.client.0.smithi089.stdout:(sd-pam) 4603 ubuntu NOFD /proc/4603/fd (opendir: Permission denied) 2024-09-21T03:42:13.348 INFO:tasks.workunit.client.0.smithi089.stdout:rsyslogd 5073 root cwd unknown /proc/5073/cwd (readlink: Permission denied) 2024-09-21T03:42:13.348 INFO:tasks.workunit.client.0.smithi089.stdout:rsyslogd 5073 root rtd unknown /proc/5073/root (readlink: Permission denied) 2024-09-21T03:42:13.348 INFO:tasks.workunit.client.0.smithi089.stdout:rsyslogd 5073 root txt unknown /proc/5073/exe (readlink: Permission denied) 2024-09-21T03:42:13.348 INFO:tasks.workunit.client.0.smithi089.stdout:rsyslogd 5073 root NOFD /proc/5073/fd (opendir: Permission denied) 2024-09-21T03:42:13.348 INFO:tasks.workunit.client.0.smithi089.stdout:rsyslogd 5073 5075 in:imjour root cwd unknown /proc/5073/task/5075/cwd (readlink: Permission denied) 2024-09-21T03:42:13.348 INFO:tasks.workunit.client.0.smithi089.stdout:rsyslogd 5073 5075 in:imjour root rtd unknown /proc/5073/task/5075/root (readlink: Permission denied) 2024-09-21T03:42:13.349 INFO:tasks.workunit.client.0.smithi089.stdout:rsyslogd 5073 5075 in:imjour root txt unknown /proc/5073/task/5075/exe (readlink: Permission denied) 2024-09-21T03:42:13.349 INFO:tasks.workunit.client.0.smithi089.stdout:rsyslogd 5073 5075 in:imjour root NOFD /proc/5073/task/5075/fd (opendir: Permission denied) 2024-09-21T03:42:13.349 INFO:tasks.workunit.client.0.smithi089.stdout:rsyslogd 5073 5076 rs:main root cwd unknown /proc/5073/task/5076/cwd (readlink: Permission denied) 2024-09-21T03:42:13.349 INFO:tasks.workunit.client.0.smithi089.stdout:rsyslogd 5073 5076 rs:main root rtd unknown /proc/5073/task/5076/root (readlink: Permission denied) 2024-09-21T03:42:13.349 INFO:tasks.workunit.client.0.smithi089.stdout:rsyslogd 5073 5076 rs:main root txt unknown /proc/5073/task/5076/exe (readlink: Permission denied) 2024-09-21T03:42:13.349 INFO:tasks.workunit.client.0.smithi089.stdout:rsyslogd 5073 5076 rs:main root NOFD /proc/5073/task/5076/fd (opendir: Permission denied) 2024-09-21T03:42:13.349 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/R 18647 root cwd unknown /proc/18647/cwd (readlink: Permission denied) 2024-09-21T03:42:13.349 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/R 18647 root rtd unknown /proc/18647/root (readlink: Permission denied) 2024-09-21T03:42:13.349 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/R 18647 root txt unknown /proc/18647/exe (readlink: Permission denied) 2024-09-21T03:42:13.349 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/R 18647 root NOFD /proc/18647/fd (opendir: Permission denied) 2024-09-21T03:42:13.349 INFO:tasks.workunit.client.0.smithi089.stdout:polkitd 19198 polkitd cwd unknown /proc/19198/cwd (readlink: Permission denied) 2024-09-21T03:42:13.349 INFO:tasks.workunit.client.0.smithi089.stdout:polkitd 19198 polkitd rtd unknown /proc/19198/root (readlink: Permission denied) 2024-09-21T03:42:13.349 INFO:tasks.workunit.client.0.smithi089.stdout:polkitd 19198 polkitd txt unknown /proc/19198/exe (readlink: Permission denied) 2024-09-21T03:42:13.349 INFO:tasks.workunit.client.0.smithi089.stdout:polkitd 19198 polkitd NOFD /proc/19198/fd (opendir: Permission denied) 2024-09-21T03:42:13.349 INFO:tasks.workunit.client.0.smithi089.stdout:polkitd 19198 19200 gmain polkitd cwd unknown /proc/19198/task/19200/cwd (readlink: Permission denied) 2024-09-21T03:42:13.350 INFO:tasks.workunit.client.0.smithi089.stdout:polkitd 19198 19200 gmain polkitd rtd unknown /proc/19198/task/19200/root (readlink: Permission denied) 2024-09-21T03:42:13.350 INFO:tasks.workunit.client.0.smithi089.stdout:polkitd 19198 19200 gmain polkitd txt unknown /proc/19198/task/19200/exe (readlink: Permission denied) 2024-09-21T03:42:13.350 INFO:tasks.workunit.client.0.smithi089.stdout:polkitd 19198 19200 gmain polkitd NOFD /proc/19198/task/19200/fd (opendir: Permission denied) 2024-09-21T03:42:13.350 INFO:tasks.workunit.client.0.smithi089.stdout:polkitd 19198 19201 gdbus polkitd cwd unknown /proc/19198/task/19201/cwd (readlink: Permission denied) 2024-09-21T03:42:13.350 INFO:tasks.workunit.client.0.smithi089.stdout:polkitd 19198 19201 gdbus polkitd rtd unknown /proc/19198/task/19201/root (readlink: Permission denied) 2024-09-21T03:42:13.350 INFO:tasks.workunit.client.0.smithi089.stdout:polkitd 19198 19201 gdbus polkitd txt unknown /proc/19198/task/19201/exe (readlink: Permission denied) 2024-09-21T03:42:13.350 INFO:tasks.workunit.client.0.smithi089.stdout:polkitd 19198 19201 gdbus polkitd NOFD /proc/19198/task/19201/fd (opendir: Permission denied) 2024-09-21T03:42:13.350 INFO:tasks.workunit.client.0.smithi089.stdout:polkitd 19198 19202 JS\x20Hel polkitd cwd unknown /proc/19198/task/19202/cwd (readlink: Permission denied) 2024-09-21T03:42:13.350 INFO:tasks.workunit.client.0.smithi089.stdout:polkitd 19198 19202 JS\x20Hel polkitd rtd unknown /proc/19198/task/19202/root (readlink: Permission denied) 2024-09-21T03:42:13.350 INFO:tasks.workunit.client.0.smithi089.stdout:polkitd 19198 19202 JS\x20Hel polkitd txt unknown /proc/19198/task/19202/exe (readlink: Permission denied) 2024-09-21T03:42:13.350 INFO:tasks.workunit.client.0.smithi089.stdout:polkitd 19198 19202 JS\x20Hel polkitd NOFD /proc/19198/task/19202/fd (opendir: Permission denied) 2024-09-21T03:42:13.350 INFO:tasks.workunit.client.0.smithi089.stdout:polkitd 19198 19203 JS\x20Hel polkitd cwd unknown /proc/19198/task/19203/cwd (readlink: Permission denied) 2024-09-21T03:42:13.350 INFO:tasks.workunit.client.0.smithi089.stdout:polkitd 19198 19203 JS\x20Hel polkitd rtd unknown /proc/19198/task/19203/root (readlink: Permission denied) 2024-09-21T03:42:13.350 INFO:tasks.workunit.client.0.smithi089.stdout:polkitd 19198 19203 JS\x20Hel polkitd txt unknown /proc/19198/task/19203/exe (readlink: Permission denied) 2024-09-21T03:42:13.350 INFO:tasks.workunit.client.0.smithi089.stdout:polkitd 19198 19203 JS\x20Hel polkitd NOFD /proc/19198/task/19203/fd (opendir: Permission denied) 2024-09-21T03:42:13.351 INFO:tasks.workunit.client.0.smithi089.stdout:polkitd 19198 19204 JS\x20Hel polkitd cwd unknown /proc/19198/task/19204/cwd (readlink: Permission denied) 2024-09-21T03:42:13.351 INFO:tasks.workunit.client.0.smithi089.stdout:polkitd 19198 19204 JS\x20Hel polkitd rtd unknown /proc/19198/task/19204/root (readlink: Permission denied) 2024-09-21T03:42:13.351 INFO:tasks.workunit.client.0.smithi089.stdout:polkitd 19198 19204 JS\x20Hel polkitd txt unknown /proc/19198/task/19204/exe (readlink: Permission denied) 2024-09-21T03:42:13.351 INFO:tasks.workunit.client.0.smithi089.stdout:polkitd 19198 19204 JS\x20Hel polkitd NOFD /proc/19198/task/19204/fd (opendir: Permission denied) 2024-09-21T03:42:13.351 INFO:tasks.workunit.client.0.smithi089.stdout:polkitd 19198 19205 JS\x20Hel polkitd cwd unknown /proc/19198/task/19205/cwd (readlink: Permission denied) 2024-09-21T03:42:13.351 INFO:tasks.workunit.client.0.smithi089.stdout:polkitd 19198 19205 JS\x20Hel polkitd rtd unknown /proc/19198/task/19205/root (readlink: Permission denied) 2024-09-21T03:42:13.351 INFO:tasks.workunit.client.0.smithi089.stdout:polkitd 19198 19205 JS\x20Hel polkitd txt unknown /proc/19198/task/19205/exe (readlink: Permission denied) 2024-09-21T03:42:13.351 INFO:tasks.workunit.client.0.smithi089.stdout:polkitd 19198 19205 JS\x20Hel polkitd NOFD /proc/19198/task/19205/fd (opendir: Permission denied) 2024-09-21T03:42:13.351 INFO:tasks.workunit.client.0.smithi089.stdout:polkitd 19198 19206 JS\x20Hel polkitd cwd unknown /proc/19198/task/19206/cwd (readlink: Permission denied) 2024-09-21T03:42:13.351 INFO:tasks.workunit.client.0.smithi089.stdout:polkitd 19198 19206 JS\x20Hel polkitd rtd unknown /proc/19198/task/19206/root (readlink: Permission denied) 2024-09-21T03:42:13.351 INFO:tasks.workunit.client.0.smithi089.stdout:polkitd 19198 19206 JS\x20Hel polkitd txt unknown /proc/19198/task/19206/exe (readlink: Permission denied) 2024-09-21T03:42:13.351 INFO:tasks.workunit.client.0.smithi089.stdout:polkitd 19198 19206 JS\x20Hel polkitd NOFD /proc/19198/task/19206/fd (opendir: Permission denied) 2024-09-21T03:42:13.351 INFO:tasks.workunit.client.0.smithi089.stdout:polkitd 19198 19207 JS\x20Hel polkitd cwd unknown /proc/19198/task/19207/cwd (readlink: Permission denied) 2024-09-21T03:42:13.351 INFO:tasks.workunit.client.0.smithi089.stdout:polkitd 19198 19207 JS\x20Hel polkitd rtd unknown /proc/19198/task/19207/root (readlink: Permission denied) 2024-09-21T03:42:13.352 INFO:tasks.workunit.client.0.smithi089.stdout:polkitd 19198 19207 JS\x20Hel polkitd txt unknown /proc/19198/task/19207/exe (readlink: Permission denied) 2024-09-21T03:42:13.352 INFO:tasks.workunit.client.0.smithi089.stdout:polkitd 19198 19207 JS\x20Hel polkitd NOFD /proc/19198/task/19207/fd (opendir: Permission denied) 2024-09-21T03:42:13.352 INFO:tasks.workunit.client.0.smithi089.stdout:polkitd 19198 19208 JS\x20Hel polkitd cwd unknown /proc/19198/task/19208/cwd (readlink: Permission denied) 2024-09-21T03:42:13.352 INFO:tasks.workunit.client.0.smithi089.stdout:polkitd 19198 19208 JS\x20Hel polkitd rtd unknown /proc/19198/task/19208/root (readlink: Permission denied) 2024-09-21T03:42:13.352 INFO:tasks.workunit.client.0.smithi089.stdout:polkitd 19198 19208 JS\x20Hel polkitd txt unknown /proc/19198/task/19208/exe (readlink: Permission denied) 2024-09-21T03:42:13.352 INFO:tasks.workunit.client.0.smithi089.stdout:polkitd 19198 19208 JS\x20Hel polkitd NOFD /proc/19198/task/19208/fd (opendir: Permission denied) 2024-09-21T03:42:13.352 INFO:tasks.workunit.client.0.smithi089.stdout:polkitd 19198 19209 JS\x20Hel polkitd cwd unknown /proc/19198/task/19209/cwd (readlink: Permission denied) 2024-09-21T03:42:13.352 INFO:tasks.workunit.client.0.smithi089.stdout:polkitd 19198 19209 JS\x20Hel polkitd rtd unknown /proc/19198/task/19209/root (readlink: Permission denied) 2024-09-21T03:42:13.352 INFO:tasks.workunit.client.0.smithi089.stdout:polkitd 19198 19209 JS\x20Hel polkitd txt unknown /proc/19198/task/19209/exe (readlink: Permission denied) 2024-09-21T03:42:13.352 INFO:tasks.workunit.client.0.smithi089.stdout:polkitd 19198 19209 JS\x20Hel polkitd NOFD /proc/19198/task/19209/fd (opendir: Permission denied) 2024-09-21T03:42:13.352 INFO:tasks.workunit.client.0.smithi089.stdout:polkitd 19198 19210 polkitd polkitd cwd unknown /proc/19198/task/19210/cwd (readlink: Permission denied) 2024-09-21T03:42:13.352 INFO:tasks.workunit.client.0.smithi089.stdout:polkitd 19198 19210 polkitd polkitd rtd unknown /proc/19198/task/19210/root (readlink: Permission denied) 2024-09-21T03:42:13.352 INFO:tasks.workunit.client.0.smithi089.stdout:polkitd 19198 19210 polkitd polkitd txt unknown /proc/19198/task/19210/exe (readlink: Permission denied) 2024-09-21T03:42:13.352 INFO:tasks.workunit.client.0.smithi089.stdout:polkitd 19198 19210 polkitd polkitd NOFD /proc/19198/task/19210/fd (opendir: Permission denied) 2024-09-21T03:42:13.353 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/R 21117 root cwd unknown /proc/21117/cwd (readlink: Permission denied) 2024-09-21T03:42:13.353 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/R 21117 root rtd unknown /proc/21117/root (readlink: Permission denied) 2024-09-21T03:42:13.353 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/R 21117 root txt unknown /proc/21117/exe (readlink: Permission denied) 2024-09-21T03:42:13.353 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/R 21117 root NOFD /proc/21117/fd (opendir: Permission denied) 2024-09-21T03:42:13.353 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/R 21146 root cwd unknown /proc/21146/cwd (readlink: Permission denied) 2024-09-21T03:42:13.353 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/R 21146 root rtd unknown /proc/21146/root (readlink: Permission denied) 2024-09-21T03:42:13.353 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/R 21146 root txt unknown /proc/21146/exe (readlink: Permission denied) 2024-09-21T03:42:13.353 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/R 21146 root NOFD /proc/21146/fd (opendir: Permission denied) 2024-09-21T03:42:13.353 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/R 21175 root cwd unknown /proc/21175/cwd (readlink: Permission denied) 2024-09-21T03:42:13.353 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/R 21175 root rtd unknown /proc/21175/root (readlink: Permission denied) 2024-09-21T03:42:13.353 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/R 21175 root txt unknown /proc/21175/exe (readlink: Permission denied) 2024-09-21T03:42:13.353 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/R 21175 root NOFD /proc/21175/fd (opendir: Permission denied) 2024-09-21T03:42:13.353 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/R 21204 root cwd unknown /proc/21204/cwd (readlink: Permission denied) 2024-09-21T03:42:13.353 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/R 21204 root rtd unknown /proc/21204/root (readlink: Permission denied) 2024-09-21T03:42:13.353 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/R 21204 root txt unknown /proc/21204/exe (readlink: Permission denied) 2024-09-21T03:42:13.354 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/R 21204 root NOFD /proc/21204/fd (opendir: Permission denied) 2024-09-21T03:42:13.354 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/R 21233 root cwd unknown /proc/21233/cwd (readlink: Permission denied) 2024-09-21T03:42:13.354 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/R 21233 root rtd unknown /proc/21233/root (readlink: Permission denied) 2024-09-21T03:42:13.354 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/R 21233 root txt unknown /proc/21233/exe (readlink: Permission denied) 2024-09-21T03:42:13.354 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/R 21233 root NOFD /proc/21233/fd (opendir: Permission denied) 2024-09-21T03:42:13.354 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/R 21397 root cwd unknown /proc/21397/cwd (readlink: Permission denied) 2024-09-21T03:42:13.354 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/R 21397 root rtd unknown /proc/21397/root (readlink: Permission denied) 2024-09-21T03:42:13.354 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/R 21397 root txt unknown /proc/21397/exe (readlink: Permission denied) 2024-09-21T03:42:13.354 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/R 21397 root NOFD /proc/21397/fd (opendir: Permission denied) 2024-09-21T03:42:13.354 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/R 21398 root cwd unknown /proc/21398/cwd (readlink: Permission denied) 2024-09-21T03:42:13.354 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/R 21398 root rtd unknown /proc/21398/root (readlink: Permission denied) 2024-09-21T03:42:13.354 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/R 21398 root txt unknown /proc/21398/exe (readlink: Permission denied) 2024-09-21T03:42:13.354 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/R 21398 root NOFD /proc/21398/fd (opendir: Permission denied) 2024-09-21T03:42:13.354 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/R 21399 root cwd unknown /proc/21399/cwd (readlink: Permission denied) 2024-09-21T03:42:13.355 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/R 21399 root rtd unknown /proc/21399/root (readlink: Permission denied) 2024-09-21T03:42:13.355 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/R 21399 root txt unknown /proc/21399/exe (readlink: Permission denied) 2024-09-21T03:42:13.355 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/R 21399 root NOFD /proc/21399/fd (opendir: Permission denied) 2024-09-21T03:42:13.355 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/R 21400 root cwd unknown /proc/21400/cwd (readlink: Permission denied) 2024-09-21T03:42:13.355 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/R 21400 root rtd unknown /proc/21400/root (readlink: Permission denied) 2024-09-21T03:42:13.355 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/R 21400 root txt unknown /proc/21400/exe (readlink: Permission denied) 2024-09-21T03:42:13.355 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/R 21400 root NOFD /proc/21400/fd (opendir: Permission denied) 2024-09-21T03:42:13.355 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/R 21401 root cwd unknown /proc/21401/cwd (readlink: Permission denied) 2024-09-21T03:42:13.355 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/R 21401 root rtd unknown /proc/21401/root (readlink: Permission denied) 2024-09-21T03:42:13.355 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/R 21401 root txt unknown /proc/21401/exe (readlink: Permission denied) 2024-09-21T03:42:13.355 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/R 21401 root NOFD /proc/21401/fd (opendir: Permission denied) 2024-09-21T03:42:13.355 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/R 21402 root cwd unknown /proc/21402/cwd (readlink: Permission denied) 2024-09-21T03:42:13.355 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/R 21402 root rtd unknown /proc/21402/root (readlink: Permission denied) 2024-09-21T03:42:13.355 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/R 21402 root txt unknown /proc/21402/exe (readlink: Permission denied) 2024-09-21T03:42:13.356 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/R 21402 root NOFD /proc/21402/fd (opendir: Permission denied) 2024-09-21T03:42:13.356 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/R 21403 root cwd unknown /proc/21403/cwd (readlink: Permission denied) 2024-09-21T03:42:13.356 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/R 21403 root rtd unknown /proc/21403/root (readlink: Permission denied) 2024-09-21T03:42:13.356 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/R 21403 root txt unknown /proc/21403/exe (readlink: Permission denied) 2024-09-21T03:42:13.356 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/R 21403 root NOFD /proc/21403/fd (opendir: Permission denied) 2024-09-21T03:42:13.356 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/R 21404 root cwd unknown /proc/21404/cwd (readlink: Permission denied) 2024-09-21T03:42:13.356 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/R 21404 root rtd unknown /proc/21404/root (readlink: Permission denied) 2024-09-21T03:42:13.356 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/R 21404 root txt unknown /proc/21404/exe (readlink: Permission denied) 2024-09-21T03:42:13.356 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/R 21404 root NOFD /proc/21404/fd (opendir: Permission denied) 2024-09-21T03:42:13.356 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/R 21405 root cwd unknown /proc/21405/cwd (readlink: Permission denied) 2024-09-21T03:42:13.356 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/R 21405 root rtd unknown /proc/21405/root (readlink: Permission denied) 2024-09-21T03:42:13.356 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/R 21405 root txt unknown /proc/21405/exe (readlink: Permission denied) 2024-09-21T03:42:13.356 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/R 21405 root NOFD /proc/21405/fd (opendir: Permission denied) 2024-09-21T03:42:13.356 INFO:tasks.workunit.client.0.smithi089.stdout:xfsaild/d 21406 root cwd unknown /proc/21406/cwd (readlink: Permission denied) 2024-09-21T03:42:13.356 INFO:tasks.workunit.client.0.smithi089.stdout:xfsaild/d 21406 root rtd unknown /proc/21406/root (readlink: Permission denied) 2024-09-21T03:42:13.357 INFO:tasks.workunit.client.0.smithi089.stdout:xfsaild/d 21406 root txt unknown /proc/21406/exe (readlink: Permission denied) 2024-09-21T03:42:13.357 INFO:tasks.workunit.client.0.smithi089.stdout:xfsaild/d 21406 root NOFD /proc/21406/fd (opendir: Permission denied) 2024-09-21T03:42:13.357 INFO:tasks.workunit.client.0.smithi089.stdout:sshd 22326 root cwd unknown /proc/22326/cwd (readlink: Permission denied) 2024-09-21T03:42:13.357 INFO:tasks.workunit.client.0.smithi089.stdout:sshd 22326 root rtd unknown /proc/22326/root (readlink: Permission denied) 2024-09-21T03:42:13.357 INFO:tasks.workunit.client.0.smithi089.stdout:sshd 22326 root txt unknown /proc/22326/exe (readlink: Permission denied) 2024-09-21T03:42:13.357 INFO:tasks.workunit.client.0.smithi089.stdout:sshd 22326 root NOFD /proc/22326/fd (opendir: Permission denied) 2024-09-21T03:42:13.357 INFO:tasks.workunit.client.0.smithi089.stdout:sshd 22329 ubuntu cwd unknown /proc/22329/cwd (readlink: Permission denied) 2024-09-21T03:42:13.357 INFO:tasks.workunit.client.0.smithi089.stdout:sshd 22329 ubuntu rtd unknown /proc/22329/root (readlink: Permission denied) 2024-09-21T03:42:13.357 INFO:tasks.workunit.client.0.smithi089.stdout:sshd 22329 ubuntu txt unknown /proc/22329/exe (readlink: Permission denied) 2024-09-21T03:42:13.357 INFO:tasks.workunit.client.0.smithi089.stdout:sshd 22329 ubuntu NOFD /proc/22329/fd (opendir: Permission denied) 2024-09-21T03:42:13.357 INFO:tasks.workunit.client.0.smithi089.stdout:chronyd 22389 chrony cwd unknown /proc/22389/cwd (readlink: Permission denied) 2024-09-21T03:42:13.357 INFO:tasks.workunit.client.0.smithi089.stdout:chronyd 22389 chrony rtd unknown /proc/22389/root (readlink: Permission denied) 2024-09-21T03:42:13.357 INFO:tasks.workunit.client.0.smithi089.stdout:chronyd 22389 chrony txt unknown /proc/22389/exe (readlink: Permission denied) 2024-09-21T03:42:13.358 INFO:tasks.workunit.client.0.smithi089.stdout:chronyd 22389 chrony NOFD /proc/22389/fd (opendir: Permission denied) 2024-09-21T03:42:13.358 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-cras 22695 ceph cwd unknown /proc/22695/cwd (readlink: Permission denied) 2024-09-21T03:42:13.358 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-cras 22695 ceph rtd unknown /proc/22695/root (readlink: Permission denied) 2024-09-21T03:42:13.358 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-cras 22695 ceph txt unknown /proc/22695/exe (readlink: Permission denied) 2024-09-21T03:42:13.358 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-cras 22695 ceph NOFD /proc/22695/fd (opendir: Permission denied) 2024-09-21T03:42:13.358 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/6 22706 root cwd unknown /proc/22706/cwd (readlink: Permission denied) 2024-09-21T03:42:13.358 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/6 22706 root rtd unknown /proc/22706/root (readlink: Permission denied) 2024-09-21T03:42:13.358 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/6 22706 root txt unknown /proc/22706/exe (readlink: Permission denied) 2024-09-21T03:42:13.358 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/6 22706 root NOFD /proc/22706/fd (opendir: Permission denied) 2024-09-21T03:42:13.358 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/3 23894 root cwd unknown /proc/23894/cwd (readlink: Permission denied) 2024-09-21T03:42:13.358 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/3 23894 root rtd unknown /proc/23894/root (readlink: Permission denied) 2024-09-21T03:42:13.358 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/3 23894 root txt unknown /proc/23894/exe (readlink: Permission denied) 2024-09-21T03:42:13.358 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/3 23894 root NOFD /proc/23894/fd (opendir: Permission denied) 2024-09-21T03:42:13.358 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/4 23964 root cwd unknown /proc/23964/cwd (readlink: Permission denied) 2024-09-21T03:42:13.359 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/4 23964 root rtd unknown /proc/23964/root (readlink: Permission denied) 2024-09-21T03:42:13.359 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/4 23964 root txt unknown /proc/23964/exe (readlink: Permission denied) 2024-09-21T03:42:13.359 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/4 23964 root NOFD /proc/23964/fd (opendir: Permission denied) 2024-09-21T03:42:13.359 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/7 24308 root cwd unknown /proc/24308/cwd (readlink: Permission denied) 2024-09-21T03:42:13.359 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/7 24308 root rtd unknown /proc/24308/root (readlink: Permission denied) 2024-09-21T03:42:13.359 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/7 24308 root txt unknown /proc/24308/exe (readlink: Permission denied) 2024-09-21T03:42:13.359 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/7 24308 root NOFD /proc/24308/fd (opendir: Permission denied) 2024-09-21T03:42:13.359 INFO:tasks.workunit.client.0.smithi089.stdout:sudo 29925 root cwd unknown /proc/29925/cwd (readlink: Permission denied) 2024-09-21T03:42:13.359 INFO:tasks.workunit.client.0.smithi089.stdout:sudo 29925 root rtd unknown /proc/29925/root (readlink: Permission denied) 2024-09-21T03:42:13.359 INFO:tasks.workunit.client.0.smithi089.stdout:sudo 29925 root txt unknown /proc/29925/exe (readlink: Permission denied) 2024-09-21T03:42:13.359 INFO:tasks.workunit.client.0.smithi089.stdout:sudo 29925 root NOFD /proc/29925/fd (opendir: Permission denied) 2024-09-21T03:42:13.359 INFO:tasks.workunit.client.0.smithi089.stdout:journalct 29942 root cwd unknown /proc/29942/cwd (readlink: Permission denied) 2024-09-21T03:42:13.359 INFO:tasks.workunit.client.0.smithi089.stdout:journalct 29942 root rtd unknown /proc/29942/root (readlink: Permission denied) 2024-09-21T03:42:13.359 INFO:tasks.workunit.client.0.smithi089.stdout:journalct 29942 root txt unknown /proc/29942/exe (readlink: Permission denied) 2024-09-21T03:42:13.359 INFO:tasks.workunit.client.0.smithi089.stdout:journalct 29942 root NOFD /proc/29942/fd (opendir: Permission denied) 2024-09-21T03:42:13.360 INFO:tasks.workunit.client.0.smithi089.stdout:sudo 29943 root cwd unknown /proc/29943/cwd (readlink: Permission denied) 2024-09-21T03:42:13.360 INFO:tasks.workunit.client.0.smithi089.stdout:sudo 29943 root rtd unknown /proc/29943/root (readlink: Permission denied) 2024-09-21T03:42:13.360 INFO:tasks.workunit.client.0.smithi089.stdout:sudo 29943 root txt unknown /proc/29943/exe (readlink: Permission denied) 2024-09-21T03:42:13.360 INFO:tasks.workunit.client.0.smithi089.stdout:sudo 29943 root NOFD /proc/29943/fd (opendir: Permission denied) 2024-09-21T03:42:13.360 INFO:tasks.workunit.client.0.smithi089.stdout:journalct 29960 root cwd unknown /proc/29960/cwd (readlink: Permission denied) 2024-09-21T03:42:13.360 INFO:tasks.workunit.client.0.smithi089.stdout:journalct 29960 root rtd unknown /proc/29960/root (readlink: Permission denied) 2024-09-21T03:42:13.360 INFO:tasks.workunit.client.0.smithi089.stdout:journalct 29960 root txt unknown /proc/29960/exe (readlink: Permission denied) 2024-09-21T03:42:13.360 INFO:tasks.workunit.client.0.smithi089.stdout:journalct 29960 root NOFD /proc/29960/fd (opendir: Permission denied) 2024-09-21T03:42:13.360 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/0 30256 root cwd unknown /proc/30256/cwd (readlink: Permission denied) 2024-09-21T03:42:13.360 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/0 30256 root rtd unknown /proc/30256/root (readlink: Permission denied) 2024-09-21T03:42:13.360 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/0 30256 root txt unknown /proc/30256/exe (readlink: Permission denied) 2024-09-21T03:42:13.360 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/0 30256 root NOFD /proc/30256/fd (opendir: Permission denied) 2024-09-21T03:42:13.360 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/3 30716 root cwd unknown /proc/30716/cwd (readlink: Permission denied) 2024-09-21T03:42:13.360 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/3 30716 root rtd unknown /proc/30716/root (readlink: Permission denied) 2024-09-21T03:42:13.361 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/3 30716 root txt unknown /proc/30716/exe (readlink: Permission denied) 2024-09-21T03:42:13.361 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/3 30716 root NOFD /proc/30716/fd (opendir: Permission denied) 2024-09-21T03:42:13.361 INFO:tasks.workunit.client.0.smithi089.stdout:conmon 30924 root cwd unknown /proc/30924/cwd (readlink: Permission denied) 2024-09-21T03:42:13.361 INFO:tasks.workunit.client.0.smithi089.stdout:conmon 30924 root rtd unknown /proc/30924/root (readlink: Permission denied) 2024-09-21T03:42:13.361 INFO:tasks.workunit.client.0.smithi089.stdout:conmon 30924 root txt unknown /proc/30924/exe (readlink: Permission denied) 2024-09-21T03:42:13.361 INFO:tasks.workunit.client.0.smithi089.stdout:conmon 30924 root NOFD /proc/30924/fd (opendir: Permission denied) 2024-09-21T03:42:13.361 INFO:tasks.workunit.client.0.smithi089.stdout:podman-in 30926 root cwd unknown /proc/30926/cwd (readlink: Permission denied) 2024-09-21T03:42:13.361 INFO:tasks.workunit.client.0.smithi089.stdout:podman-in 30926 root rtd unknown /proc/30926/root (readlink: Permission denied) 2024-09-21T03:42:13.361 INFO:tasks.workunit.client.0.smithi089.stdout:podman-in 30926 root txt unknown /proc/30926/exe (readlink: Permission denied) 2024-09-21T03:42:13.361 INFO:tasks.workunit.client.0.smithi089.stdout:podman-in 30926 root NOFD /proc/30926/fd (opendir: Permission denied) 2024-09-21T03:42:13.361 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mon 30928 ceph cwd unknown /proc/30928/cwd (readlink: Permission denied) 2024-09-21T03:42:13.361 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mon 30928 ceph rtd unknown /proc/30928/root (readlink: Permission denied) 2024-09-21T03:42:13.361 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mon 30928 ceph txt unknown /proc/30928/exe (readlink: Permission denied) 2024-09-21T03:42:13.361 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mon 30928 ceph NOFD /proc/30928/fd (opendir: Permission denied) 2024-09-21T03:42:13.362 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mon 30928 30929 log ceph cwd unknown /proc/30928/task/30929/cwd (readlink: Permission denied) 2024-09-21T03:42:13.362 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mon 30928 30929 log ceph rtd unknown /proc/30928/task/30929/root (readlink: Permission denied) 2024-09-21T03:42:13.362 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mon 30928 30929 log ceph txt unknown /proc/30928/task/30929/exe (readlink: Permission denied) 2024-09-21T03:42:13.362 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mon 30928 30929 log ceph NOFD /proc/30928/task/30929/fd (opendir: Permission denied) 2024-09-21T03:42:13.362 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mon 30928 30930 service ceph cwd unknown /proc/30928/task/30930/cwd (readlink: Permission denied) 2024-09-21T03:42:13.362 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mon 30928 30930 service ceph rtd unknown /proc/30928/task/30930/root (readlink: Permission denied) 2024-09-21T03:42:13.362 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mon 30928 30930 service ceph txt unknown /proc/30928/task/30930/exe (readlink: Permission denied) 2024-09-21T03:42:13.362 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mon 30928 30930 service ceph NOFD /proc/30928/task/30930/fd (opendir: Permission denied) 2024-09-21T03:42:13.362 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mon 30928 30931 admin_soc ceph cwd unknown /proc/30928/task/30931/cwd (readlink: Permission denied) 2024-09-21T03:42:13.362 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mon 30928 30931 admin_soc ceph rtd unknown /proc/30928/task/30931/root (readlink: Permission denied) 2024-09-21T03:42:13.362 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mon 30928 30931 admin_soc ceph txt unknown /proc/30928/task/30931/exe (readlink: Permission denied) 2024-09-21T03:42:13.362 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mon 30928 30931 admin_soc ceph NOFD /proc/30928/task/30931/fd (opendir: Permission denied) 2024-09-21T03:42:13.362 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mon 30928 30932 signal_ha ceph cwd unknown /proc/30928/task/30932/cwd (readlink: Permission denied) 2024-09-21T03:42:13.362 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mon 30928 30932 signal_ha ceph rtd unknown /proc/30928/task/30932/root (readlink: Permission denied) 2024-09-21T03:42:13.363 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mon 30928 30932 signal_ha ceph txt unknown /proc/30928/task/30932/exe (readlink: Permission denied) 2024-09-21T03:42:13.363 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mon 30928 30932 signal_ha ceph NOFD /proc/30928/task/30932/fd (opendir: Permission denied) 2024-09-21T03:42:13.363 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mon 30928 30933 rocksdb:l ceph cwd unknown /proc/30928/task/30933/cwd (readlink: Permission denied) 2024-09-21T03:42:13.363 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mon 30928 30933 rocksdb:l ceph rtd unknown /proc/30928/task/30933/root (readlink: Permission denied) 2024-09-21T03:42:13.363 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mon 30928 30933 rocksdb:l ceph txt unknown /proc/30928/task/30933/exe (readlink: Permission denied) 2024-09-21T03:42:13.363 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mon 30928 30933 rocksdb:l ceph NOFD /proc/30928/task/30933/fd (opendir: Permission denied) 2024-09-21T03:42:13.363 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mon 30928 30934 rocksdb:h ceph cwd unknown /proc/30928/task/30934/cwd (readlink: Permission denied) 2024-09-21T03:42:13.363 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mon 30928 30934 rocksdb:h ceph rtd unknown /proc/30928/task/30934/root (readlink: Permission denied) 2024-09-21T03:42:13.363 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mon 30928 30934 rocksdb:h ceph txt unknown /proc/30928/task/30934/exe (readlink: Permission denied) 2024-09-21T03:42:13.363 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mon 30928 30934 rocksdb:h ceph NOFD /proc/30928/task/30934/fd (opendir: Permission denied) 2024-09-21T03:42:13.363 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mon 30928 30950 ceph-mon ceph cwd unknown /proc/30928/task/30950/cwd (readlink: Permission denied) 2024-09-21T03:42:13.363 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mon 30928 30950 ceph-mon ceph rtd unknown /proc/30928/task/30950/root (readlink: Permission denied) 2024-09-21T03:42:13.363 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mon 30928 30950 ceph-mon ceph txt unknown /proc/30928/task/30950/exe (readlink: Permission denied) 2024-09-21T03:42:13.363 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mon 30928 30950 ceph-mon ceph NOFD /proc/30928/task/30950/fd (opendir: Permission denied) 2024-09-21T03:42:13.363 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mon 30928 30951 fn_monsto ceph cwd unknown /proc/30928/task/30951/cwd (readlink: Permission denied) 2024-09-21T03:42:13.364 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mon 30928 30951 fn_monsto ceph rtd unknown /proc/30928/task/30951/root (readlink: Permission denied) 2024-09-21T03:42:13.364 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mon 30928 30951 fn_monsto ceph txt unknown /proc/30928/task/30951/exe (readlink: Permission denied) 2024-09-21T03:42:13.364 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mon 30928 30951 fn_monsto ceph NOFD /proc/30928/task/30951/fd (opendir: Permission denied) 2024-09-21T03:42:13.364 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mon 30928 30952 msgr-work ceph cwd unknown /proc/30928/task/30952/cwd (readlink: Permission denied) 2024-09-21T03:42:13.364 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mon 30928 30952 msgr-work ceph rtd unknown /proc/30928/task/30952/root (readlink: Permission denied) 2024-09-21T03:42:13.364 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mon 30928 30952 msgr-work ceph txt unknown /proc/30928/task/30952/exe (readlink: Permission denied) 2024-09-21T03:42:13.364 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mon 30928 30952 msgr-work ceph NOFD /proc/30928/task/30952/fd (opendir: Permission denied) 2024-09-21T03:42:13.364 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mon 30928 30953 msgr-work ceph cwd unknown /proc/30928/task/30953/cwd (readlink: Permission denied) 2024-09-21T03:42:13.364 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mon 30928 30953 msgr-work ceph rtd unknown /proc/30928/task/30953/root (readlink: Permission denied) 2024-09-21T03:42:13.364 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mon 30928 30953 msgr-work ceph txt unknown /proc/30928/task/30953/exe (readlink: Permission denied) 2024-09-21T03:42:13.364 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mon 30928 30953 msgr-work ceph NOFD /proc/30928/task/30953/fd (opendir: Permission denied) 2024-09-21T03:42:13.364 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mon 30928 30954 msgr-work ceph cwd unknown /proc/30928/task/30954/cwd (readlink: Permission denied) 2024-09-21T03:42:13.364 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mon 30928 30954 msgr-work ceph rtd unknown /proc/30928/task/30954/root (readlink: Permission denied) 2024-09-21T03:42:13.365 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mon 30928 30954 msgr-work ceph txt unknown /proc/30928/task/30954/exe (readlink: Permission denied) 2024-09-21T03:42:13.365 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mon 30928 30954 msgr-work ceph NOFD /proc/30928/task/30954/fd (opendir: Permission denied) 2024-09-21T03:42:13.365 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mon 30928 30955 OpHistory ceph cwd unknown /proc/30928/task/30955/cwd (readlink: Permission denied) 2024-09-21T03:42:13.365 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mon 30928 30955 OpHistory ceph rtd unknown /proc/30928/task/30955/root (readlink: Permission denied) 2024-09-21T03:42:13.365 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mon 30928 30955 OpHistory ceph txt unknown /proc/30928/task/30955/exe (readlink: Permission denied) 2024-09-21T03:42:13.365 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mon 30928 30955 OpHistory ceph NOFD /proc/30928/task/30955/fd (opendir: Permission denied) 2024-09-21T03:42:13.365 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mon 30928 30956 fin ceph cwd unknown /proc/30928/task/30956/cwd (readlink: Permission denied) 2024-09-21T03:42:13.365 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mon 30928 30956 fin ceph rtd unknown /proc/30928/task/30956/root (readlink: Permission denied) 2024-09-21T03:42:13.365 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mon 30928 30956 fin ceph txt unknown /proc/30928/task/30956/exe (readlink: Permission denied) 2024-09-21T03:42:13.365 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mon 30928 30956 fin ceph NOFD /proc/30928/task/30956/fd (opendir: Permission denied) 2024-09-21T03:42:13.365 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mon 30928 30957 safe_time ceph cwd unknown /proc/30928/task/30957/cwd (readlink: Permission denied) 2024-09-21T03:42:13.365 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mon 30928 30957 safe_time ceph rtd unknown /proc/30928/task/30957/root (readlink: Permission denied) 2024-09-21T03:42:13.365 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mon 30928 30957 safe_time ceph txt unknown /proc/30928/task/30957/exe (readlink: Permission denied) 2024-09-21T03:42:13.365 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mon 30928 30957 safe_time ceph NOFD /proc/30928/task/30957/fd (opendir: Permission denied) 2024-09-21T03:42:13.365 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mon 30928 30958 cpu_tp ceph cwd unknown /proc/30928/task/30958/cwd (readlink: Permission denied) 2024-09-21T03:42:13.366 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mon 30928 30958 cpu_tp ceph rtd unknown /proc/30928/task/30958/root (readlink: Permission denied) 2024-09-21T03:42:13.366 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mon 30928 30958 cpu_tp ceph txt unknown /proc/30928/task/30958/exe (readlink: Permission denied) 2024-09-21T03:42:13.366 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mon 30928 30958 cpu_tp ceph NOFD /proc/30928/task/30958/fd (opendir: Permission denied) 2024-09-21T03:42:13.366 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mon 30928 30959 cpu_tp ceph cwd unknown /proc/30928/task/30959/cwd (readlink: Permission denied) 2024-09-21T03:42:13.366 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mon 30928 30959 cpu_tp ceph rtd unknown /proc/30928/task/30959/root (readlink: Permission denied) 2024-09-21T03:42:13.366 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mon 30928 30959 cpu_tp ceph txt unknown /proc/30928/task/30959/exe (readlink: Permission denied) 2024-09-21T03:42:13.366 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mon 30928 30959 cpu_tp ceph NOFD /proc/30928/task/30959/fd (opendir: Permission denied) 2024-09-21T03:42:13.366 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mon 30928 30960 cpu_tp ceph cwd unknown /proc/30928/task/30960/cwd (readlink: Permission denied) 2024-09-21T03:42:13.366 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mon 30928 30960 cpu_tp ceph rtd unknown /proc/30928/task/30960/root (readlink: Permission denied) 2024-09-21T03:42:13.366 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mon 30928 30960 cpu_tp ceph txt unknown /proc/30928/task/30960/exe (readlink: Permission denied) 2024-09-21T03:42:13.366 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mon 30928 30960 cpu_tp ceph NOFD /proc/30928/task/30960/fd (opendir: Permission denied) 2024-09-21T03:42:13.366 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mon 30928 30961 cpu_tp ceph cwd unknown /proc/30928/task/30961/cwd (readlink: Permission denied) 2024-09-21T03:42:13.366 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mon 30928 30961 cpu_tp ceph rtd unknown /proc/30928/task/30961/root (readlink: Permission denied) 2024-09-21T03:42:13.367 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mon 30928 30961 cpu_tp ceph txt unknown /proc/30928/task/30961/exe (readlink: Permission denied) 2024-09-21T03:42:13.367 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mon 30928 30961 cpu_tp ceph NOFD /proc/30928/task/30961/fd (opendir: Permission denied) 2024-09-21T03:42:13.367 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mon 30928 30962 ms_dispat ceph cwd unknown /proc/30928/task/30962/cwd (readlink: Permission denied) 2024-09-21T03:42:13.367 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mon 30928 30962 ms_dispat ceph rtd unknown /proc/30928/task/30962/root (readlink: Permission denied) 2024-09-21T03:42:13.367 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mon 30928 30962 ms_dispat ceph txt unknown /proc/30928/task/30962/exe (readlink: Permission denied) 2024-09-21T03:42:13.367 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mon 30928 30962 ms_dispat ceph NOFD /proc/30928/task/30962/fd (opendir: Permission denied) 2024-09-21T03:42:13.367 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mon 30928 30963 ms_local ceph cwd unknown /proc/30928/task/30963/cwd (readlink: Permission denied) 2024-09-21T03:42:13.367 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mon 30928 30963 ms_local ceph rtd unknown /proc/30928/task/30963/root (readlink: Permission denied) 2024-09-21T03:42:13.367 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mon 30928 30963 ms_local ceph txt unknown /proc/30928/task/30963/exe (readlink: Permission denied) 2024-09-21T03:42:13.367 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mon 30928 30963 ms_local ceph NOFD /proc/30928/task/30963/fd (opendir: Permission denied) 2024-09-21T03:42:13.367 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mon 30928 30964 safe_time ceph cwd unknown /proc/30928/task/30964/cwd (readlink: Permission denied) 2024-09-21T03:42:13.367 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mon 30928 30964 safe_time ceph rtd unknown /proc/30928/task/30964/root (readlink: Permission denied) 2024-09-21T03:42:13.368 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mon 30928 30964 safe_time ceph txt unknown /proc/30928/task/30964/exe (readlink: Permission denied) 2024-09-21T03:42:13.368 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mon 30928 30964 safe_time ceph NOFD /proc/30928/task/30964/fd (opendir: Permission denied) 2024-09-21T03:42:13.368 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mon 30928 30965 ms_dispat ceph cwd unknown /proc/30928/task/30965/cwd (readlink: Permission denied) 2024-09-21T03:42:13.368 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mon 30928 30965 ms_dispat ceph rtd unknown /proc/30928/task/30965/root (readlink: Permission denied) 2024-09-21T03:42:13.368 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mon 30928 30965 ms_dispat ceph txt unknown /proc/30928/task/30965/exe (readlink: Permission denied) 2024-09-21T03:42:13.368 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mon 30928 30965 ms_dispat ceph NOFD /proc/30928/task/30965/fd (opendir: Permission denied) 2024-09-21T03:42:13.368 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mon 30928 30966 ms_local ceph cwd unknown /proc/30928/task/30966/cwd (readlink: Permission denied) 2024-09-21T03:42:13.368 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mon 30928 30966 ms_local ceph rtd unknown /proc/30928/task/30966/root (readlink: Permission denied) 2024-09-21T03:42:13.368 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mon 30928 30966 ms_local ceph txt unknown /proc/30928/task/30966/exe (readlink: Permission denied) 2024-09-21T03:42:13.368 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mon 30928 30966 ms_local ceph NOFD /proc/30928/task/30966/fd (opendir: Permission denied) 2024-09-21T03:42:13.368 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mon 30928 69143 rstore_co ceph cwd unknown /proc/30928/task/69143/cwd (readlink: Permission denied) 2024-09-21T03:42:13.368 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mon 30928 69143 rstore_co ceph rtd unknown /proc/30928/task/69143/root (readlink: Permission denied) 2024-09-21T03:42:13.368 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mon 30928 69143 rstore_co ceph txt unknown /proc/30928/task/69143/exe (readlink: Permission denied) 2024-09-21T03:42:13.369 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mon 30928 69143 rstore_co ceph NOFD /proc/30928/task/69143/fd (opendir: Permission denied) 2024-09-21T03:42:13.369 INFO:tasks.workunit.client.0.smithi089.stdout:conmon 31154 root cwd unknown /proc/31154/cwd (readlink: Permission denied) 2024-09-21T03:42:13.369 INFO:tasks.workunit.client.0.smithi089.stdout:conmon 31154 root rtd unknown /proc/31154/root (readlink: Permission denied) 2024-09-21T03:42:13.369 INFO:tasks.workunit.client.0.smithi089.stdout:conmon 31154 root txt unknown /proc/31154/exe (readlink: Permission denied) 2024-09-21T03:42:13.369 INFO:tasks.workunit.client.0.smithi089.stdout:conmon 31154 root NOFD /proc/31154/fd (opendir: Permission denied) 2024-09-21T03:42:13.369 INFO:tasks.workunit.client.0.smithi089.stdout:podman-in 31156 root cwd unknown /proc/31156/cwd (readlink: Permission denied) 2024-09-21T03:42:13.369 INFO:tasks.workunit.client.0.smithi089.stdout:podman-in 31156 root rtd unknown /proc/31156/root (readlink: Permission denied) 2024-09-21T03:42:13.369 INFO:tasks.workunit.client.0.smithi089.stdout:podman-in 31156 root txt unknown /proc/31156/exe (readlink: Permission denied) 2024-09-21T03:42:13.369 INFO:tasks.workunit.client.0.smithi089.stdout:podman-in 31156 root NOFD /proc/31156/fd (opendir: Permission denied) 2024-09-21T03:42:13.369 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 ceph cwd unknown /proc/31158/cwd (readlink: Permission denied) 2024-09-21T03:42:13.369 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 ceph rtd unknown /proc/31158/root (readlink: Permission denied) 2024-09-21T03:42:13.369 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 ceph txt unknown /proc/31158/exe (readlink: Permission denied) 2024-09-21T03:42:13.369 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 ceph NOFD /proc/31158/fd (opendir: Permission denied) 2024-09-21T03:42:13.369 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33233 log ceph cwd unknown /proc/31158/task/33233/cwd (readlink: Permission denied) 2024-09-21T03:42:13.369 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33233 log ceph rtd unknown /proc/31158/task/33233/root (readlink: Permission denied) 2024-09-21T03:42:13.369 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33233 log ceph txt unknown /proc/31158/task/33233/exe (readlink: Permission denied) 2024-09-21T03:42:13.370 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33233 log ceph NOFD /proc/31158/task/33233/fd (opendir: Permission denied) 2024-09-21T03:42:13.370 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33235 msgr-work ceph cwd unknown /proc/31158/task/33235/cwd (readlink: Permission denied) 2024-09-21T03:42:13.370 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33235 msgr-work ceph rtd unknown /proc/31158/task/33235/root (readlink: Permission denied) 2024-09-21T03:42:13.370 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33235 msgr-work ceph txt unknown /proc/31158/task/33235/exe (readlink: Permission denied) 2024-09-21T03:42:13.370 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33235 msgr-work ceph NOFD /proc/31158/task/33235/fd (opendir: Permission denied) 2024-09-21T03:42:13.370 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33236 msgr-work ceph cwd unknown /proc/31158/task/33236/cwd (readlink: Permission denied) 2024-09-21T03:42:13.370 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33236 msgr-work ceph rtd unknown /proc/31158/task/33236/root (readlink: Permission denied) 2024-09-21T03:42:13.370 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33236 msgr-work ceph txt unknown /proc/31158/task/33236/exe (readlink: Permission denied) 2024-09-21T03:42:13.370 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33236 msgr-work ceph NOFD /proc/31158/task/33236/fd (opendir: Permission denied) 2024-09-21T03:42:13.370 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33237 msgr-work ceph cwd unknown /proc/31158/task/33237/cwd (readlink: Permission denied) 2024-09-21T03:42:13.370 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33237 msgr-work ceph rtd unknown /proc/31158/task/33237/root (readlink: Permission denied) 2024-09-21T03:42:13.370 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33237 msgr-work ceph txt unknown /proc/31158/task/33237/exe (readlink: Permission denied) 2024-09-21T03:42:13.370 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33237 msgr-work ceph NOFD /proc/31158/task/33237/fd (opendir: Permission denied) 2024-09-21T03:42:13.370 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33241 service ceph cwd unknown /proc/31158/task/33241/cwd (readlink: Permission denied) 2024-09-21T03:42:13.370 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33241 service ceph rtd unknown /proc/31158/task/33241/root (readlink: Permission denied) 2024-09-21T03:42:13.371 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33241 service ceph txt unknown /proc/31158/task/33241/exe (readlink: Permission denied) 2024-09-21T03:42:13.371 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33241 service ceph NOFD /proc/31158/task/33241/fd (opendir: Permission denied) 2024-09-21T03:42:13.371 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33242 admin_soc ceph cwd unknown /proc/31158/task/33242/cwd (readlink: Permission denied) 2024-09-21T03:42:13.371 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33242 admin_soc ceph rtd unknown /proc/31158/task/33242/root (readlink: Permission denied) 2024-09-21T03:42:13.371 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33242 admin_soc ceph txt unknown /proc/31158/task/33242/exe (readlink: Permission denied) 2024-09-21T03:42:13.371 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33242 admin_soc ceph NOFD /proc/31158/task/33242/fd (opendir: Permission denied) 2024-09-21T03:42:13.371 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33243 ceph_time ceph cwd unknown /proc/31158/task/33243/cwd (readlink: Permission denied) 2024-09-21T03:42:13.371 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33243 ceph_time ceph rtd unknown /proc/31158/task/33243/root (readlink: Permission denied) 2024-09-21T03:42:13.371 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33243 ceph_time ceph txt unknown /proc/31158/task/33243/exe (readlink: Permission denied) 2024-09-21T03:42:13.371 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33243 ceph_time ceph NOFD /proc/31158/task/33243/fd (opendir: Permission denied) 2024-09-21T03:42:13.371 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33244 fn_anonym ceph cwd unknown /proc/31158/task/33244/cwd (readlink: Permission denied) 2024-09-21T03:42:13.371 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33244 fn_anonym ceph rtd unknown /proc/31158/task/33244/root (readlink: Permission denied) 2024-09-21T03:42:13.371 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33244 fn_anonym ceph txt unknown /proc/31158/task/33244/exe (readlink: Permission denied) 2024-09-21T03:42:13.371 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33244 fn_anonym ceph NOFD /proc/31158/task/33244/fd (opendir: Permission denied) 2024-09-21T03:42:13.372 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33245 signal_ha ceph cwd unknown /proc/31158/task/33245/cwd (readlink: Permission denied) 2024-09-21T03:42:13.372 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33245 signal_ha ceph rtd unknown /proc/31158/task/33245/root (readlink: Permission denied) 2024-09-21T03:42:13.372 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33245 signal_ha ceph txt unknown /proc/31158/task/33245/exe (readlink: Permission denied) 2024-09-21T03:42:13.372 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33245 signal_ha ceph NOFD /proc/31158/task/33245/fd (opendir: Permission denied) 2024-09-21T03:42:13.372 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33246 mgrsb-fin ceph cwd unknown /proc/31158/task/33246/cwd (readlink: Permission denied) 2024-09-21T03:42:13.372 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33246 mgrsb-fin ceph rtd unknown /proc/31158/task/33246/root (readlink: Permission denied) 2024-09-21T03:42:13.372 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33246 mgrsb-fin ceph txt unknown /proc/31158/task/33246/exe (readlink: Permission denied) 2024-09-21T03:42:13.372 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33246 mgrsb-fin ceph NOFD /proc/31158/task/33246/fd (opendir: Permission denied) 2024-09-21T03:42:13.372 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33247 ms_dispat ceph cwd unknown /proc/31158/task/33247/cwd (readlink: Permission denied) 2024-09-21T03:42:13.372 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33247 ms_dispat ceph rtd unknown /proc/31158/task/33247/root (readlink: Permission denied) 2024-09-21T03:42:13.372 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33247 ms_dispat ceph txt unknown /proc/31158/task/33247/exe (readlink: Permission denied) 2024-09-21T03:42:13.372 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33247 ms_dispat ceph NOFD /proc/31158/task/33247/fd (opendir: Permission denied) 2024-09-21T03:42:13.372 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33248 ms_local ceph cwd unknown /proc/31158/task/33248/cwd (readlink: Permission denied) 2024-09-21T03:42:13.372 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33248 ms_local ceph rtd unknown /proc/31158/task/33248/root (readlink: Permission denied) 2024-09-21T03:42:13.373 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33248 ms_local ceph txt unknown /proc/31158/task/33248/exe (readlink: Permission denied) 2024-09-21T03:42:13.373 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33248 ms_local ceph NOFD /proc/31158/task/33248/fd (opendir: Permission denied) 2024-09-21T03:42:13.373 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33249 io_contex ceph cwd unknown /proc/31158/task/33249/cwd (readlink: Permission denied) 2024-09-21T03:42:13.373 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33249 io_contex ceph rtd unknown /proc/31158/task/33249/root (readlink: Permission denied) 2024-09-21T03:42:13.373 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33249 io_contex ceph txt unknown /proc/31158/task/33249/exe (readlink: Permission denied) 2024-09-21T03:42:13.373 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33249 io_contex ceph NOFD /proc/31158/task/33249/fd (opendir: Permission denied) 2024-09-21T03:42:13.373 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33250 io_contex ceph cwd unknown /proc/31158/task/33250/cwd (readlink: Permission denied) 2024-09-21T03:42:13.373 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33250 io_contex ceph rtd unknown /proc/31158/task/33250/root (readlink: Permission denied) 2024-09-21T03:42:13.373 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33250 io_contex ceph txt unknown /proc/31158/task/33250/exe (readlink: Permission denied) 2024-09-21T03:42:13.373 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33250 io_contex ceph NOFD /proc/31158/task/33250/fd (opendir: Permission denied) 2024-09-21T03:42:13.373 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33251 safe_time ceph cwd unknown /proc/31158/task/33251/cwd (readlink: Permission denied) 2024-09-21T03:42:13.373 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33251 safe_time ceph rtd unknown /proc/31158/task/33251/root (readlink: Permission denied) 2024-09-21T03:42:13.373 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33251 safe_time ceph txt unknown /proc/31158/task/33251/exe (readlink: Permission denied) 2024-09-21T03:42:13.373 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33251 safe_time ceph NOFD /proc/31158/task/33251/fd (opendir: Permission denied) 2024-09-21T03:42:13.373 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33252 safe_time ceph cwd unknown /proc/31158/task/33252/cwd (readlink: Permission denied) 2024-09-21T03:42:13.374 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33252 safe_time ceph rtd unknown /proc/31158/task/33252/root (readlink: Permission denied) 2024-09-21T03:42:13.374 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33252 safe_time ceph txt unknown /proc/31158/task/33252/exe (readlink: Permission denied) 2024-09-21T03:42:13.374 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33252 safe_time ceph NOFD /proc/31158/task/33252/fd (opendir: Permission denied) 2024-09-21T03:42:13.374 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33253 safe_time ceph cwd unknown /proc/31158/task/33253/cwd (readlink: Permission denied) 2024-09-21T03:42:13.374 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33253 safe_time ceph rtd unknown /proc/31158/task/33253/root (readlink: Permission denied) 2024-09-21T03:42:13.374 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33253 safe_time ceph txt unknown /proc/31158/task/33253/exe (readlink: Permission denied) 2024-09-21T03:42:13.374 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33253 safe_time ceph NOFD /proc/31158/task/33253/fd (opendir: Permission denied) 2024-09-21T03:42:13.374 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33254 fn_anonym ceph cwd unknown /proc/31158/task/33254/cwd (readlink: Permission denied) 2024-09-21T03:42:13.374 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33254 fn_anonym ceph rtd unknown /proc/31158/task/33254/root (readlink: Permission denied) 2024-09-21T03:42:13.374 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33254 fn_anonym ceph txt unknown /proc/31158/task/33254/exe (readlink: Permission denied) 2024-09-21T03:42:13.374 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33254 fn_anonym ceph NOFD /proc/31158/task/33254/fd (opendir: Permission denied) 2024-09-21T03:42:13.374 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33255 flusher ceph cwd unknown /proc/31158/task/33255/cwd (readlink: Permission denied) 2024-09-21T03:42:13.374 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33255 flusher ceph rtd unknown /proc/31158/task/33255/root (readlink: Permission denied) 2024-09-21T03:42:13.374 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33255 flusher ceph txt unknown /proc/31158/task/33255/exe (readlink: Permission denied) 2024-09-21T03:42:13.375 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33255 flusher ceph NOFD /proc/31158/task/33255/fd (opendir: Permission denied) 2024-09-21T03:42:13.375 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33256 safe_time ceph cwd unknown /proc/31158/task/33256/cwd (readlink: Permission denied) 2024-09-21T03:42:13.375 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33256 safe_time ceph rtd unknown /proc/31158/task/33256/root (readlink: Permission denied) 2024-09-21T03:42:13.375 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33256 safe_time ceph txt unknown /proc/31158/task/33256/exe (readlink: Permission denied) 2024-09-21T03:42:13.375 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33256 safe_time ceph NOFD /proc/31158/task/33256/fd (opendir: Permission denied) 2024-09-21T03:42:13.375 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33380 OpHistory ceph cwd unknown /proc/31158/task/33380/cwd (readlink: Permission denied) 2024-09-21T03:42:13.375 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33380 OpHistory ceph rtd unknown /proc/31158/task/33380/root (readlink: Permission denied) 2024-09-21T03:42:13.375 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33380 OpHistory ceph txt unknown /proc/31158/task/33380/exe (readlink: Permission denied) 2024-09-21T03:42:13.375 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33380 OpHistory ceph NOFD /proc/31158/task/33380/fd (opendir: Permission denied) 2024-09-21T03:42:13.375 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33381 mgr-fin ceph cwd unknown /proc/31158/task/33381/cwd (readlink: Permission denied) 2024-09-21T03:42:13.375 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33381 mgr-fin ceph rtd unknown /proc/31158/task/33381/root (readlink: Permission denied) 2024-09-21T03:42:13.375 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33381 mgr-fin ceph txt unknown /proc/31158/task/33381/exe (readlink: Permission denied) 2024-09-21T03:42:13.375 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33381 mgr-fin ceph NOFD /proc/31158/task/33381/fd (opendir: Permission denied) 2024-09-21T03:42:13.376 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33382 ms_dispat ceph cwd unknown /proc/31158/task/33382/cwd (readlink: Permission denied) 2024-09-21T03:42:13.376 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33382 ms_dispat ceph rtd unknown /proc/31158/task/33382/root (readlink: Permission denied) 2024-09-21T03:42:13.376 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33382 ms_dispat ceph txt unknown /proc/31158/task/33382/exe (readlink: Permission denied) 2024-09-21T03:42:13.376 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33382 ms_dispat ceph NOFD /proc/31158/task/33382/fd (opendir: Permission denied) 2024-09-21T03:42:13.376 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33383 ms_local ceph cwd unknown /proc/31158/task/33383/cwd (readlink: Permission denied) 2024-09-21T03:42:13.376 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33383 ms_local ceph rtd unknown /proc/31158/task/33383/root (readlink: Permission denied) 2024-09-21T03:42:13.376 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33383 ms_local ceph txt unknown /proc/31158/task/33383/exe (readlink: Permission denied) 2024-09-21T03:42:13.376 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33383 ms_local ceph NOFD /proc/31158/task/33383/fd (opendir: Permission denied) 2024-09-21T03:42:13.376 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33384 safe_time ceph cwd unknown /proc/31158/task/33384/cwd (readlink: Permission denied) 2024-09-21T03:42:13.376 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33384 safe_time ceph rtd unknown /proc/31158/task/33384/root (readlink: Permission denied) 2024-09-21T03:42:13.376 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33384 safe_time ceph txt unknown /proc/31158/task/33384/exe (readlink: Permission denied) 2024-09-21T03:42:13.376 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33384 safe_time ceph NOFD /proc/31158/task/33384/fd (opendir: Permission denied) 2024-09-21T03:42:13.376 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33385 cmdfin ceph cwd unknown /proc/31158/task/33385/cwd (readlink: Permission denied) 2024-09-21T03:42:13.376 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33385 cmdfin ceph rtd unknown /proc/31158/task/33385/root (readlink: Permission denied) 2024-09-21T03:42:13.377 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33385 cmdfin ceph txt unknown /proc/31158/task/33385/exe (readlink: Permission denied) 2024-09-21T03:42:13.377 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33385 cmdfin ceph NOFD /proc/31158/task/33385/fd (opendir: Permission denied) 2024-09-21T03:42:13.377 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33389 io_contex ceph cwd unknown /proc/31158/task/33389/cwd (readlink: Permission denied) 2024-09-21T03:42:13.377 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33389 io_contex ceph rtd unknown /proc/31158/task/33389/root (readlink: Permission denied) 2024-09-21T03:42:13.377 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33389 io_contex ceph txt unknown /proc/31158/task/33389/exe (readlink: Permission denied) 2024-09-21T03:42:13.377 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33389 io_contex ceph NOFD /proc/31158/task/33389/fd (opendir: Permission denied) 2024-09-21T03:42:13.377 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33390 io_contex ceph cwd unknown /proc/31158/task/33390/cwd (readlink: Permission denied) 2024-09-21T03:42:13.377 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33390 io_contex ceph rtd unknown /proc/31158/task/33390/root (readlink: Permission denied) 2024-09-21T03:42:13.377 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33390 io_contex ceph txt unknown /proc/31158/task/33390/exe (readlink: Permission denied) 2024-09-21T03:42:13.377 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33390 io_contex ceph NOFD /proc/31158/task/33390/fd (opendir: Permission denied) 2024-09-21T03:42:13.377 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33391 ceph_time ceph cwd unknown /proc/31158/task/33391/cwd (readlink: Permission denied) 2024-09-21T03:42:13.377 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33391 ceph_time ceph rtd unknown /proc/31158/task/33391/root (readlink: Permission denied) 2024-09-21T03:42:13.377 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33391 ceph_time ceph txt unknown /proc/31158/task/33391/exe (readlink: Permission denied) 2024-09-21T03:42:13.377 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33391 ceph_time ceph NOFD /proc/31158/task/33391/fd (opendir: Permission denied) 2024-09-21T03:42:13.378 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33392 ms_dispat ceph cwd unknown /proc/31158/task/33392/cwd (readlink: Permission denied) 2024-09-21T03:42:13.378 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33392 ms_dispat ceph rtd unknown /proc/31158/task/33392/root (readlink: Permission denied) 2024-09-21T03:42:13.378 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33392 ms_dispat ceph txt unknown /proc/31158/task/33392/exe (readlink: Permission denied) 2024-09-21T03:42:13.378 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33392 ms_dispat ceph NOFD /proc/31158/task/33392/fd (opendir: Permission denied) 2024-09-21T03:42:13.378 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33393 ms_local ceph cwd unknown /proc/31158/task/33393/cwd (readlink: Permission denied) 2024-09-21T03:42:13.378 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33393 ms_local ceph rtd unknown /proc/31158/task/33393/root (readlink: Permission denied) 2024-09-21T03:42:13.378 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33393 ms_local ceph txt unknown /proc/31158/task/33393/exe (readlink: Permission denied) 2024-09-21T03:42:13.378 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33393 ms_local ceph NOFD /proc/31158/task/33393/fd (opendir: Permission denied) 2024-09-21T03:42:13.378 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33394 safe_time ceph cwd unknown /proc/31158/task/33394/cwd (readlink: Permission denied) 2024-09-21T03:42:13.378 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33394 safe_time ceph rtd unknown /proc/31158/task/33394/root (readlink: Permission denied) 2024-09-21T03:42:13.378 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33394 safe_time ceph txt unknown /proc/31158/task/33394/exe (readlink: Permission denied) 2024-09-21T03:42:13.378 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33394 safe_time ceph NOFD /proc/31158/task/33394/fd (opendir: Permission denied) 2024-09-21T03:42:13.378 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33395 safe_time ceph cwd unknown /proc/31158/task/33395/cwd (readlink: Permission denied) 2024-09-21T03:42:13.378 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33395 safe_time ceph rtd unknown /proc/31158/task/33395/root (readlink: Permission denied) 2024-09-21T03:42:13.379 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33395 safe_time ceph txt unknown /proc/31158/task/33395/exe (readlink: Permission denied) 2024-09-21T03:42:13.379 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33395 safe_time ceph NOFD /proc/31158/task/33395/fd (opendir: Permission denied) 2024-09-21T03:42:13.379 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33396 balancer ceph cwd unknown /proc/31158/task/33396/cwd (readlink: Permission denied) 2024-09-21T03:42:13.379 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33396 balancer ceph rtd unknown /proc/31158/task/33396/root (readlink: Permission denied) 2024-09-21T03:42:13.379 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33396 balancer ceph txt unknown /proc/31158/task/33396/exe (readlink: Permission denied) 2024-09-21T03:42:13.379 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33396 balancer ceph NOFD /proc/31158/task/33396/fd (opendir: Permission denied) 2024-09-21T03:42:13.379 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33397 m-fin-bal ceph cwd unknown /proc/31158/task/33397/cwd (readlink: Permission denied) 2024-09-21T03:42:13.379 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33397 m-fin-bal ceph rtd unknown /proc/31158/task/33397/root (readlink: Permission denied) 2024-09-21T03:42:13.379 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33397 m-fin-bal ceph txt unknown /proc/31158/task/33397/exe (readlink: Permission denied) 2024-09-21T03:42:13.379 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33397 m-fin-bal ceph NOFD /proc/31158/task/33397/fd (opendir: Permission denied) 2024-09-21T03:42:13.379 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33398 mgr-fin ceph cwd unknown /proc/31158/task/33398/cwd (readlink: Permission denied) 2024-09-21T03:42:13.379 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33398 mgr-fin ceph rtd unknown /proc/31158/task/33398/root (readlink: Permission denied) 2024-09-21T03:42:13.379 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33398 mgr-fin ceph txt unknown /proc/31158/task/33398/exe (readlink: Permission denied) 2024-09-21T03:42:13.379 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33398 mgr-fin ceph NOFD /proc/31158/task/33398/fd (opendir: Permission denied) 2024-09-21T03:42:13.380 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33399 mgr-fin ceph cwd unknown /proc/31158/task/33399/cwd (readlink: Permission denied) 2024-09-21T03:42:13.380 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33399 mgr-fin ceph rtd unknown /proc/31158/task/33399/root (readlink: Permission denied) 2024-09-21T03:42:13.380 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33399 mgr-fin ceph txt unknown /proc/31158/task/33399/exe (readlink: Permission denied) 2024-09-21T03:42:13.380 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33399 mgr-fin ceph NOFD /proc/31158/task/33399/fd (opendir: Permission denied) 2024-09-21T03:42:13.380 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33400 mgr-fin ceph cwd unknown /proc/31158/task/33400/cwd (readlink: Permission denied) 2024-09-21T03:42:13.380 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33400 mgr-fin ceph rtd unknown /proc/31158/task/33400/root (readlink: Permission denied) 2024-09-21T03:42:13.380 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33400 mgr-fin ceph txt unknown /proc/31158/task/33400/exe (readlink: Permission denied) 2024-09-21T03:42:13.380 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33400 mgr-fin ceph NOFD /proc/31158/task/33400/fd (opendir: Permission denied) 2024-09-21T03:42:13.380 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33401 mgr-fin ceph cwd unknown /proc/31158/task/33401/cwd (readlink: Permission denied) 2024-09-21T03:42:13.380 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33401 mgr-fin ceph rtd unknown /proc/31158/task/33401/root (readlink: Permission denied) 2024-09-21T03:42:13.380 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33401 mgr-fin ceph txt unknown /proc/31158/task/33401/exe (readlink: Permission denied) 2024-09-21T03:42:13.380 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33401 mgr-fin ceph NOFD /proc/31158/task/33401/fd (opendir: Permission denied) 2024-09-21T03:42:13.380 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33402 mgr-fin ceph cwd unknown /proc/31158/task/33402/cwd (readlink: Permission denied) 2024-09-21T03:42:13.380 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33402 mgr-fin ceph rtd unknown /proc/31158/task/33402/root (readlink: Permission denied) 2024-09-21T03:42:13.380 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33402 mgr-fin ceph txt unknown /proc/31158/task/33402/exe (readlink: Permission denied) 2024-09-21T03:42:13.381 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33402 mgr-fin ceph NOFD /proc/31158/task/33402/fd (opendir: Permission denied) 2024-09-21T03:42:13.381 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33403 mgr-fin ceph cwd unknown /proc/31158/task/33403/cwd (readlink: Permission denied) 2024-09-21T03:42:13.381 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33403 mgr-fin ceph rtd unknown /proc/31158/task/33403/root (readlink: Permission denied) 2024-09-21T03:42:13.381 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33403 mgr-fin ceph txt unknown /proc/31158/task/33403/exe (readlink: Permission denied) 2024-09-21T03:42:13.381 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33403 mgr-fin ceph NOFD /proc/31158/task/33403/fd (opendir: Permission denied) 2024-09-21T03:42:13.381 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33404 mgr-fin ceph cwd unknown /proc/31158/task/33404/cwd (readlink: Permission denied) 2024-09-21T03:42:13.381 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33404 mgr-fin ceph rtd unknown /proc/31158/task/33404/root (readlink: Permission denied) 2024-09-21T03:42:13.381 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33404 mgr-fin ceph txt unknown /proc/31158/task/33404/exe (readlink: Permission denied) 2024-09-21T03:42:13.381 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33404 mgr-fin ceph NOFD /proc/31158/task/33404/fd (opendir: Permission denied) 2024-09-21T03:42:13.381 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33405 mgr-fin ceph cwd unknown /proc/31158/task/33405/cwd (readlink: Permission denied) 2024-09-21T03:42:13.381 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33405 mgr-fin ceph rtd unknown /proc/31158/task/33405/root (readlink: Permission denied) 2024-09-21T03:42:13.381 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33405 mgr-fin ceph txt unknown /proc/31158/task/33405/exe (readlink: Permission denied) 2024-09-21T03:42:13.381 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33405 mgr-fin ceph NOFD /proc/31158/task/33405/fd (opendir: Permission denied) 2024-09-21T03:42:13.381 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33406 mgr-fin ceph cwd unknown /proc/31158/task/33406/cwd (readlink: Permission denied) 2024-09-21T03:42:13.381 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33406 mgr-fin ceph rtd unknown /proc/31158/task/33406/root (readlink: Permission denied) 2024-09-21T03:42:13.382 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33406 mgr-fin ceph txt unknown /proc/31158/task/33406/exe (readlink: Permission denied) 2024-09-21T03:42:13.382 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33406 mgr-fin ceph NOFD /proc/31158/task/33406/fd (opendir: Permission denied) 2024-09-21T03:42:13.382 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33407 mgr-fin ceph cwd unknown /proc/31158/task/33407/cwd (readlink: Permission denied) 2024-09-21T03:42:13.382 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33407 mgr-fin ceph rtd unknown /proc/31158/task/33407/root (readlink: Permission denied) 2024-09-21T03:42:13.382 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33407 mgr-fin ceph txt unknown /proc/31158/task/33407/exe (readlink: Permission denied) 2024-09-21T03:42:13.382 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33407 mgr-fin ceph NOFD /proc/31158/task/33407/fd (opendir: Permission denied) 2024-09-21T03:42:13.382 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33408 mgr-fin ceph cwd unknown /proc/31158/task/33408/cwd (readlink: Permission denied) 2024-09-21T03:42:13.382 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33408 mgr-fin ceph rtd unknown /proc/31158/task/33408/root (readlink: Permission denied) 2024-09-21T03:42:13.382 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33408 mgr-fin ceph txt unknown /proc/31158/task/33408/exe (readlink: Permission denied) 2024-09-21T03:42:13.382 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33408 mgr-fin ceph NOFD /proc/31158/task/33408/fd (opendir: Permission denied) 2024-09-21T03:42:13.382 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33409 mgr-fin ceph cwd unknown /proc/31158/task/33409/cwd (readlink: Permission denied) 2024-09-21T03:42:13.382 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33409 mgr-fin ceph rtd unknown /proc/31158/task/33409/root (readlink: Permission denied) 2024-09-21T03:42:13.382 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33409 mgr-fin ceph txt unknown /proc/31158/task/33409/exe (readlink: Permission denied) 2024-09-21T03:42:13.382 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33409 mgr-fin ceph NOFD /proc/31158/task/33409/fd (opendir: Permission denied) 2024-09-21T03:42:13.383 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33410 mgr-fin ceph cwd unknown /proc/31158/task/33410/cwd (readlink: Permission denied) 2024-09-21T03:42:13.383 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33410 mgr-fin ceph rtd unknown /proc/31158/task/33410/root (readlink: Permission denied) 2024-09-21T03:42:13.383 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33410 mgr-fin ceph txt unknown /proc/31158/task/33410/exe (readlink: Permission denied) 2024-09-21T03:42:13.383 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33410 mgr-fin ceph NOFD /proc/31158/task/33410/fd (opendir: Permission denied) 2024-09-21T03:42:13.383 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33411 mgr-fin ceph cwd unknown /proc/31158/task/33411/cwd (readlink: Permission denied) 2024-09-21T03:42:13.383 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33411 mgr-fin ceph rtd unknown /proc/31158/task/33411/root (readlink: Permission denied) 2024-09-21T03:42:13.383 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33411 mgr-fin ceph txt unknown /proc/31158/task/33411/exe (readlink: Permission denied) 2024-09-21T03:42:13.383 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33411 mgr-fin ceph NOFD /proc/31158/task/33411/fd (opendir: Permission denied) 2024-09-21T03:42:13.383 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33412 mgr-fin ceph cwd unknown /proc/31158/task/33412/cwd (readlink: Permission denied) 2024-09-21T03:42:13.383 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33412 mgr-fin ceph rtd unknown /proc/31158/task/33412/root (readlink: Permission denied) 2024-09-21T03:42:13.383 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33412 mgr-fin ceph txt unknown /proc/31158/task/33412/exe (readlink: Permission denied) 2024-09-21T03:42:13.383 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33412 mgr-fin ceph NOFD /proc/31158/task/33412/fd (opendir: Permission denied) 2024-09-21T03:42:13.383 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33413 cephadm ceph cwd unknown /proc/31158/task/33413/cwd (readlink: Permission denied) 2024-09-21T03:42:13.383 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33413 cephadm ceph rtd unknown /proc/31158/task/33413/root (readlink: Permission denied) 2024-09-21T03:42:13.383 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33413 cephadm ceph txt unknown /proc/31158/task/33413/exe (readlink: Permission denied) 2024-09-21T03:42:13.384 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33413 cephadm ceph NOFD /proc/31158/task/33413/fd (opendir: Permission denied) 2024-09-21T03:42:13.384 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33414 m-fin-cep ceph cwd unknown /proc/31158/task/33414/cwd (readlink: Permission denied) 2024-09-21T03:42:13.384 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33414 m-fin-cep ceph rtd unknown /proc/31158/task/33414/root (readlink: Permission denied) 2024-09-21T03:42:13.384 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33414 m-fin-cep ceph txt unknown /proc/31158/task/33414/exe (readlink: Permission denied) 2024-09-21T03:42:13.384 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33414 m-fin-cep ceph NOFD /proc/31158/task/33414/fd (opendir: Permission denied) 2024-09-21T03:42:13.384 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33415 mgr-fin ceph cwd unknown /proc/31158/task/33415/cwd (readlink: Permission denied) 2024-09-21T03:42:13.384 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33415 mgr-fin ceph rtd unknown /proc/31158/task/33415/root (readlink: Permission denied) 2024-09-21T03:42:13.384 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33415 mgr-fin ceph txt unknown /proc/31158/task/33415/exe (readlink: Permission denied) 2024-09-21T03:42:13.384 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33415 mgr-fin ceph NOFD /proc/31158/task/33415/fd (opendir: Permission denied) 2024-09-21T03:42:13.384 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33416 mgr-fin ceph cwd unknown /proc/31158/task/33416/cwd (readlink: Permission denied) 2024-09-21T03:42:13.384 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33416 mgr-fin ceph rtd unknown /proc/31158/task/33416/root (readlink: Permission denied) 2024-09-21T03:42:13.384 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33416 mgr-fin ceph txt unknown /proc/31158/task/33416/exe (readlink: Permission denied) 2024-09-21T03:42:13.384 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33416 mgr-fin ceph NOFD /proc/31158/task/33416/fd (opendir: Permission denied) 2024-09-21T03:42:13.384 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33417 mgr-fin ceph cwd unknown /proc/31158/task/33417/cwd (readlink: Permission denied) 2024-09-21T03:42:13.385 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33417 mgr-fin ceph rtd unknown /proc/31158/task/33417/root (readlink: Permission denied) 2024-09-21T03:42:13.385 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33417 mgr-fin ceph txt unknown /proc/31158/task/33417/exe (readlink: Permission denied) 2024-09-21T03:42:13.385 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33417 mgr-fin ceph NOFD /proc/31158/task/33417/fd (opendir: Permission denied) 2024-09-21T03:42:13.385 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33418 mgr-fin ceph cwd unknown /proc/31158/task/33418/cwd (readlink: Permission denied) 2024-09-21T03:42:13.385 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33418 mgr-fin ceph rtd unknown /proc/31158/task/33418/root (readlink: Permission denied) 2024-09-21T03:42:13.385 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33418 mgr-fin ceph txt unknown /proc/31158/task/33418/exe (readlink: Permission denied) 2024-09-21T03:42:13.385 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33418 mgr-fin ceph NOFD /proc/31158/task/33418/fd (opendir: Permission denied) 2024-09-21T03:42:13.385 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33419 crash ceph cwd unknown /proc/31158/task/33419/cwd (readlink: Permission denied) 2024-09-21T03:42:13.385 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33419 crash ceph rtd unknown /proc/31158/task/33419/root (readlink: Permission denied) 2024-09-21T03:42:13.385 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33419 crash ceph txt unknown /proc/31158/task/33419/exe (readlink: Permission denied) 2024-09-21T03:42:13.385 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33419 crash ceph NOFD /proc/31158/task/33419/fd (opendir: Permission denied) 2024-09-21T03:42:13.385 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33420 m-fin-cra ceph cwd unknown /proc/31158/task/33420/cwd (readlink: Permission denied) 2024-09-21T03:42:13.385 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33420 m-fin-cra ceph rtd unknown /proc/31158/task/33420/root (readlink: Permission denied) 2024-09-21T03:42:13.385 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33420 m-fin-cra ceph txt unknown /proc/31158/task/33420/exe (readlink: Permission denied) 2024-09-21T03:42:13.386 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33420 m-fin-cra ceph NOFD /proc/31158/task/33420/fd (opendir: Permission denied) 2024-09-21T03:42:13.386 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33421 cephadm ceph cwd unknown /proc/31158/task/33421/cwd (readlink: Permission denied) 2024-09-21T03:42:13.386 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33421 cephadm ceph rtd unknown /proc/31158/task/33421/root (readlink: Permission denied) 2024-09-21T03:42:13.386 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33421 cephadm ceph txt unknown /proc/31158/task/33421/exe (readlink: Permission denied) 2024-09-21T03:42:13.386 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33421 cephadm ceph NOFD /proc/31158/task/33421/fd (opendir: Permission denied) 2024-09-21T03:42:13.386 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33422 mgr-fin ceph cwd unknown /proc/31158/task/33422/cwd (readlink: Permission denied) 2024-09-21T03:42:13.386 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33422 mgr-fin ceph rtd unknown /proc/31158/task/33422/root (readlink: Permission denied) 2024-09-21T03:42:13.386 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33422 mgr-fin ceph txt unknown /proc/31158/task/33422/exe (readlink: Permission denied) 2024-09-21T03:42:13.386 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33422 mgr-fin ceph NOFD /proc/31158/task/33422/fd (opendir: Permission denied) 2024-09-21T03:42:13.386 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33423 mgr-fin ceph cwd unknown /proc/31158/task/33423/cwd (readlink: Permission denied) 2024-09-21T03:42:13.386 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33423 mgr-fin ceph rtd unknown /proc/31158/task/33423/root (readlink: Permission denied) 2024-09-21T03:42:13.386 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33423 mgr-fin ceph txt unknown /proc/31158/task/33423/exe (readlink: Permission denied) 2024-09-21T03:42:13.386 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33423 mgr-fin ceph NOFD /proc/31158/task/33423/fd (opendir: Permission denied) 2024-09-21T03:42:13.386 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33424 dashboard ceph cwd unknown /proc/31158/task/33424/cwd (readlink: Permission denied) 2024-09-21T03:42:13.387 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33424 dashboard ceph rtd unknown /proc/31158/task/33424/root (readlink: Permission denied) 2024-09-21T03:42:13.387 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33424 dashboard ceph txt unknown /proc/31158/task/33424/exe (readlink: Permission denied) 2024-09-21T03:42:13.387 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33424 dashboard ceph NOFD /proc/31158/task/33424/fd (opendir: Permission denied) 2024-09-21T03:42:13.387 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33425 mgr-fin ceph cwd unknown /proc/31158/task/33425/cwd (readlink: Permission denied) 2024-09-21T03:42:13.387 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33425 mgr-fin ceph rtd unknown /proc/31158/task/33425/root (readlink: Permission denied) 2024-09-21T03:42:13.387 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33425 mgr-fin ceph txt unknown /proc/31158/task/33425/exe (readlink: Permission denied) 2024-09-21T03:42:13.387 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33425 mgr-fin ceph NOFD /proc/31158/task/33425/fd (opendir: Permission denied) 2024-09-21T03:42:13.387 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33426 m-fin-das ceph cwd unknown /proc/31158/task/33426/cwd (readlink: Permission denied) 2024-09-21T03:42:13.387 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33426 m-fin-das ceph rtd unknown /proc/31158/task/33426/root (readlink: Permission denied) 2024-09-21T03:42:13.387 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33426 m-fin-das ceph txt unknown /proc/31158/task/33426/exe (readlink: Permission denied) 2024-09-21T03:42:13.387 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33426 m-fin-das ceph NOFD /proc/31158/task/33426/fd (opendir: Permission denied) 2024-09-21T03:42:13.387 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33427 mgr-fin ceph cwd unknown /proc/31158/task/33427/cwd (readlink: Permission denied) 2024-09-21T03:42:13.387 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33427 mgr-fin ceph rtd unknown /proc/31158/task/33427/root (readlink: Permission denied) 2024-09-21T03:42:13.387 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33427 mgr-fin ceph txt unknown /proc/31158/task/33427/exe (readlink: Permission denied) 2024-09-21T03:42:13.387 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33427 mgr-fin ceph NOFD /proc/31158/task/33427/fd (opendir: Permission denied) 2024-09-21T03:42:13.387 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33428 mgr-fin ceph cwd unknown /proc/31158/task/33428/cwd (readlink: Permission denied) 2024-09-21T03:42:13.388 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33428 mgr-fin ceph rtd unknown /proc/31158/task/33428/root (readlink: Permission denied) 2024-09-21T03:42:13.388 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33428 mgr-fin ceph txt unknown /proc/31158/task/33428/exe (readlink: Permission denied) 2024-09-21T03:42:13.388 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33428 mgr-fin ceph NOFD /proc/31158/task/33428/fd (opendir: Permission denied) 2024-09-21T03:42:13.388 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33429 devicehea ceph cwd unknown /proc/31158/task/33429/cwd (readlink: Permission denied) 2024-09-21T03:42:13.388 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33429 devicehea ceph rtd unknown /proc/31158/task/33429/root (readlink: Permission denied) 2024-09-21T03:42:13.388 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33429 devicehea ceph txt unknown /proc/31158/task/33429/exe (readlink: Permission denied) 2024-09-21T03:42:13.388 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33429 devicehea ceph NOFD /proc/31158/task/33429/fd (opendir: Permission denied) 2024-09-21T03:42:13.388 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33430 m-fin-dev ceph cwd unknown /proc/31158/task/33430/cwd (readlink: Permission denied) 2024-09-21T03:42:13.388 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33430 m-fin-dev ceph rtd unknown /proc/31158/task/33430/root (readlink: Permission denied) 2024-09-21T03:42:13.388 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33430 m-fin-dev ceph txt unknown /proc/31158/task/33430/exe (readlink: Permission denied) 2024-09-21T03:42:13.388 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33430 m-fin-dev ceph NOFD /proc/31158/task/33430/fd (opendir: Permission denied) 2024-09-21T03:42:13.388 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33431 mgr-fin ceph cwd unknown /proc/31158/task/33431/cwd (readlink: Permission denied) 2024-09-21T03:42:13.388 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33431 mgr-fin ceph rtd unknown /proc/31158/task/33431/root (readlink: Permission denied) 2024-09-21T03:42:13.388 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33431 mgr-fin ceph txt unknown /proc/31158/task/33431/exe (readlink: Permission denied) 2024-09-21T03:42:13.389 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33431 mgr-fin ceph NOFD /proc/31158/task/33431/fd (opendir: Permission denied) 2024-09-21T03:42:13.389 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33432 mgr-fin ceph cwd unknown /proc/31158/task/33432/cwd (readlink: Permission denied) 2024-09-21T03:42:13.389 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33432 mgr-fin ceph rtd unknown /proc/31158/task/33432/root (readlink: Permission denied) 2024-09-21T03:42:13.389 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33432 mgr-fin ceph txt unknown /proc/31158/task/33432/exe (readlink: Permission denied) 2024-09-21T03:42:13.389 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33432 mgr-fin ceph NOFD /proc/31158/task/33432/fd (opendir: Permission denied) 2024-09-21T03:42:13.389 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33434 m-fin-ios ceph cwd unknown /proc/31158/task/33434/cwd (readlink: Permission denied) 2024-09-21T03:42:13.389 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33434 m-fin-ios ceph rtd unknown /proc/31158/task/33434/root (readlink: Permission denied) 2024-09-21T03:42:13.389 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33434 m-fin-ios ceph txt unknown /proc/31158/task/33434/exe (readlink: Permission denied) 2024-09-21T03:42:13.389 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33434 m-fin-ios ceph NOFD /proc/31158/task/33434/fd (opendir: Permission denied) 2024-09-21T03:42:13.389 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33435 mgr-fin ceph cwd unknown /proc/31158/task/33435/cwd (readlink: Permission denied) 2024-09-21T03:42:13.389 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33435 mgr-fin ceph rtd unknown /proc/31158/task/33435/root (readlink: Permission denied) 2024-09-21T03:42:13.390 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33435 mgr-fin ceph txt unknown /proc/31158/task/33435/exe (readlink: Permission denied) 2024-09-21T03:42:13.390 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33435 mgr-fin ceph NOFD /proc/31158/task/33435/fd (opendir: Permission denied) 2024-09-21T03:42:13.390 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33437 mgr-fin ceph cwd unknown /proc/31158/task/33437/cwd (readlink: Permission denied) 2024-09-21T03:42:13.390 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33437 mgr-fin ceph rtd unknown /proc/31158/task/33437/root (readlink: Permission denied) 2024-09-21T03:42:13.390 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33437 mgr-fin ceph txt unknown /proc/31158/task/33437/exe (readlink: Permission denied) 2024-09-21T03:42:13.390 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33437 mgr-fin ceph NOFD /proc/31158/task/33437/fd (opendir: Permission denied) 2024-09-21T03:42:13.390 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33438 m-fin-nfs ceph cwd unknown /proc/31158/task/33438/cwd (readlink: Permission denied) 2024-09-21T03:42:13.390 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33438 m-fin-nfs ceph rtd unknown /proc/31158/task/33438/root (readlink: Permission denied) 2024-09-21T03:42:13.390 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33438 m-fin-nfs ceph txt unknown /proc/31158/task/33438/exe (readlink: Permission denied) 2024-09-21T03:42:13.390 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33438 m-fin-nfs ceph NOFD /proc/31158/task/33438/fd (opendir: Permission denied) 2024-09-21T03:42:13.390 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33440 m-fin-orc ceph cwd unknown /proc/31158/task/33440/cwd (readlink: Permission denied) 2024-09-21T03:42:13.390 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33440 m-fin-orc ceph rtd unknown /proc/31158/task/33440/root (readlink: Permission denied) 2024-09-21T03:42:13.391 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33440 m-fin-orc ceph txt unknown /proc/31158/task/33440/exe (readlink: Permission denied) 2024-09-21T03:42:13.391 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33440 m-fin-orc ceph NOFD /proc/31158/task/33440/fd (opendir: Permission denied) 2024-09-21T03:42:13.391 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33441 cephadm ceph cwd unknown /proc/31158/task/33441/cwd (readlink: Permission denied) 2024-09-21T03:42:13.391 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33441 cephadm ceph rtd unknown /proc/31158/task/33441/root (readlink: Permission denied) 2024-09-21T03:42:13.391 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33441 cephadm ceph txt unknown /proc/31158/task/33441/exe (readlink: Permission denied) 2024-09-21T03:42:13.391 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33441 cephadm ceph NOFD /proc/31158/task/33441/fd (opendir: Permission denied) 2024-09-21T03:42:13.391 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33442 pg_autosc ceph cwd unknown /proc/31158/task/33442/cwd (readlink: Permission denied) 2024-09-21T03:42:13.391 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33442 pg_autosc ceph rtd unknown /proc/31158/task/33442/root (readlink: Permission denied) 2024-09-21T03:42:13.391 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33442 pg_autosc ceph txt unknown /proc/31158/task/33442/exe (readlink: Permission denied) 2024-09-21T03:42:13.391 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33442 pg_autosc ceph NOFD /proc/31158/task/33442/fd (opendir: Permission denied) 2024-09-21T03:42:13.391 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33443 m-fin-pg_ ceph cwd unknown /proc/31158/task/33443/cwd (readlink: Permission denied) 2024-09-21T03:42:13.391 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33443 m-fin-pg_ ceph rtd unknown /proc/31158/task/33443/root (readlink: Permission denied) 2024-09-21T03:42:13.391 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33443 m-fin-pg_ ceph txt unknown /proc/31158/task/33443/exe (readlink: Permission denied) 2024-09-21T03:42:13.391 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33443 m-fin-pg_ ceph NOFD /proc/31158/task/33443/fd (opendir: Permission denied) 2024-09-21T03:42:13.391 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33444 progress ceph cwd unknown /proc/31158/task/33444/cwd (readlink: Permission denied) 2024-09-21T03:42:13.392 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33444 progress ceph rtd unknown /proc/31158/task/33444/root (readlink: Permission denied) 2024-09-21T03:42:13.392 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33444 progress ceph txt unknown /proc/31158/task/33444/exe (readlink: Permission denied) 2024-09-21T03:42:13.392 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33444 progress ceph NOFD /proc/31158/task/33444/fd (opendir: Permission denied) 2024-09-21T03:42:13.392 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33445 m-fin-pro ceph cwd unknown /proc/31158/task/33445/cwd (readlink: Permission denied) 2024-09-21T03:42:13.392 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33445 m-fin-pro ceph rtd unknown /proc/31158/task/33445/root (readlink: Permission denied) 2024-09-21T03:42:13.392 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33445 m-fin-pro ceph txt unknown /proc/31158/task/33445/exe (readlink: Permission denied) 2024-09-21T03:42:13.392 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33445 m-fin-pro ceph NOFD /proc/31158/task/33445/fd (opendir: Permission denied) 2024-09-21T03:42:13.392 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33449 io_contex ceph cwd unknown /proc/31158/task/33449/cwd (readlink: Permission denied) 2024-09-21T03:42:13.392 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33449 io_contex ceph rtd unknown /proc/31158/task/33449/root (readlink: Permission denied) 2024-09-21T03:42:13.392 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33449 io_contex ceph txt unknown /proc/31158/task/33449/exe (readlink: Permission denied) 2024-09-21T03:42:13.392 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33449 io_contex ceph NOFD /proc/31158/task/33449/fd (opendir: Permission denied) 2024-09-21T03:42:13.392 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33450 io_contex ceph cwd unknown /proc/31158/task/33450/cwd (readlink: Permission denied) 2024-09-21T03:42:13.392 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33450 io_contex ceph rtd unknown /proc/31158/task/33450/root (readlink: Permission denied) 2024-09-21T03:42:13.392 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33450 io_contex ceph txt unknown /proc/31158/task/33450/exe (readlink: Permission denied) 2024-09-21T03:42:13.393 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33450 io_contex ceph NOFD /proc/31158/task/33450/fd (opendir: Permission denied) 2024-09-21T03:42:13.393 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33451 ceph_time ceph cwd unknown /proc/31158/task/33451/cwd (readlink: Permission denied) 2024-09-21T03:42:13.393 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33451 ceph_time ceph rtd unknown /proc/31158/task/33451/root (readlink: Permission denied) 2024-09-21T03:42:13.393 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33451 ceph_time ceph txt unknown /proc/31158/task/33451/exe (readlink: Permission denied) 2024-09-21T03:42:13.393 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33451 ceph_time ceph NOFD /proc/31158/task/33451/fd (opendir: Permission denied) 2024-09-21T03:42:13.393 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33452 ms_dispat ceph cwd unknown /proc/31158/task/33452/cwd (readlink: Permission denied) 2024-09-21T03:42:13.393 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33452 ms_dispat ceph rtd unknown /proc/31158/task/33452/root (readlink: Permission denied) 2024-09-21T03:42:13.393 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33452 ms_dispat ceph txt unknown /proc/31158/task/33452/exe (readlink: Permission denied) 2024-09-21T03:42:13.393 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33452 ms_dispat ceph NOFD /proc/31158/task/33452/fd (opendir: Permission denied) 2024-09-21T03:42:13.393 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33453 ms_local ceph cwd unknown /proc/31158/task/33453/cwd (readlink: Permission denied) 2024-09-21T03:42:13.393 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33453 ms_local ceph rtd unknown /proc/31158/task/33453/root (readlink: Permission denied) 2024-09-21T03:42:13.393 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33453 ms_local ceph txt unknown /proc/31158/task/33453/exe (readlink: Permission denied) 2024-09-21T03:42:13.393 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33453 ms_local ceph NOFD /proc/31158/task/33453/fd (opendir: Permission denied) 2024-09-21T03:42:13.393 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33454 safe_time ceph cwd unknown /proc/31158/task/33454/cwd (readlink: Permission denied) 2024-09-21T03:42:13.393 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33454 safe_time ceph rtd unknown /proc/31158/task/33454/root (readlink: Permission denied) 2024-09-21T03:42:13.393 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33454 safe_time ceph txt unknown /proc/31158/task/33454/exe (readlink: Permission denied) 2024-09-21T03:42:13.394 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33454 safe_time ceph NOFD /proc/31158/task/33454/fd (opendir: Permission denied) 2024-09-21T03:42:13.394 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33455 safe_time ceph cwd unknown /proc/31158/task/33455/cwd (readlink: Permission denied) 2024-09-21T03:42:13.394 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33455 safe_time ceph rtd unknown /proc/31158/task/33455/root (readlink: Permission denied) 2024-09-21T03:42:13.394 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33455 safe_time ceph txt unknown /proc/31158/task/33455/exe (readlink: Permission denied) 2024-09-21T03:42:13.394 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33455 safe_time ceph NOFD /proc/31158/task/33455/fd (opendir: Permission denied) 2024-09-21T03:42:13.394 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33456 mgr-fin ceph cwd unknown /proc/31158/task/33456/cwd (readlink: Permission denied) 2024-09-21T03:42:13.394 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33456 mgr-fin ceph rtd unknown /proc/31158/task/33456/root (readlink: Permission denied) 2024-09-21T03:42:13.394 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33456 mgr-fin ceph txt unknown /proc/31158/task/33456/exe (readlink: Permission denied) 2024-09-21T03:42:13.394 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33456 mgr-fin ceph NOFD /proc/31158/task/33456/fd (opendir: Permission denied) 2024-09-21T03:42:13.394 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33458 m-fin-rbd ceph cwd unknown /proc/31158/task/33458/cwd (readlink: Permission denied) 2024-09-21T03:42:13.394 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33458 m-fin-rbd ceph rtd unknown /proc/31158/task/33458/root (readlink: Permission denied) 2024-09-21T03:42:13.394 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33458 m-fin-rbd ceph txt unknown /proc/31158/task/33458/exe (readlink: Permission denied) 2024-09-21T03:42:13.394 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33458 m-fin-rbd ceph NOFD /proc/31158/task/33458/fd (opendir: Permission denied) 2024-09-21T03:42:13.394 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33459 restful ceph cwd unknown /proc/31158/task/33459/cwd (readlink: Permission denied) 2024-09-21T03:42:13.395 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33459 restful ceph rtd unknown /proc/31158/task/33459/root (readlink: Permission denied) 2024-09-21T03:42:13.395 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33459 restful ceph txt unknown /proc/31158/task/33459/exe (readlink: Permission denied) 2024-09-21T03:42:13.395 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33459 restful ceph NOFD /proc/31158/task/33459/fd (opendir: Permission denied) 2024-09-21T03:42:13.395 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33460 m-fin-res ceph cwd unknown /proc/31158/task/33460/cwd (readlink: Permission denied) 2024-09-21T03:42:13.395 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33460 m-fin-res ceph rtd unknown /proc/31158/task/33460/root (readlink: Permission denied) 2024-09-21T03:42:13.395 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33460 m-fin-res ceph txt unknown /proc/31158/task/33460/exe (readlink: Permission denied) 2024-09-21T03:42:13.395 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33460 m-fin-res ceph NOFD /proc/31158/task/33460/fd (opendir: Permission denied) 2024-09-21T03:42:13.395 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33462 m-fin-sta ceph cwd unknown /proc/31158/task/33462/cwd (readlink: Permission denied) 2024-09-21T03:42:13.395 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33462 m-fin-sta ceph rtd unknown /proc/31158/task/33462/root (readlink: Permission denied) 2024-09-21T03:42:13.395 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33462 m-fin-sta ceph txt unknown /proc/31158/task/33462/exe (readlink: Permission denied) 2024-09-21T03:42:13.395 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33462 m-fin-sta ceph NOFD /proc/31158/task/33462/fd (opendir: Permission denied) 2024-09-21T03:42:13.395 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33463 mgr-fin ceph cwd unknown /proc/31158/task/33463/cwd (readlink: Permission denied) 2024-09-21T03:42:13.395 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33463 mgr-fin ceph rtd unknown /proc/31158/task/33463/root (readlink: Permission denied) 2024-09-21T03:42:13.395 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33463 mgr-fin ceph txt unknown /proc/31158/task/33463/exe (readlink: Permission denied) 2024-09-21T03:42:13.395 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33463 mgr-fin ceph NOFD /proc/31158/task/33463/fd (opendir: Permission denied) 2024-09-21T03:42:13.396 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33464 telemetry ceph cwd unknown /proc/31158/task/33464/cwd (readlink: Permission denied) 2024-09-21T03:42:13.396 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33464 telemetry ceph rtd unknown /proc/31158/task/33464/root (readlink: Permission denied) 2024-09-21T03:42:13.396 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33464 telemetry ceph txt unknown /proc/31158/task/33464/exe (readlink: Permission denied) 2024-09-21T03:42:13.396 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33464 telemetry ceph NOFD /proc/31158/task/33464/fd (opendir: Permission denied) 2024-09-21T03:42:13.396 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33465 m-fin-tel ceph cwd unknown /proc/31158/task/33465/cwd (readlink: Permission denied) 2024-09-21T03:42:13.396 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33465 m-fin-tel ceph rtd unknown /proc/31158/task/33465/root (readlink: Permission denied) 2024-09-21T03:42:13.396 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33465 m-fin-tel ceph txt unknown /proc/31158/task/33465/exe (readlink: Permission denied) 2024-09-21T03:42:13.396 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33465 m-fin-tel ceph NOFD /proc/31158/task/33465/fd (opendir: Permission denied) 2024-09-21T03:42:13.396 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33466 mgr-fin ceph cwd unknown /proc/31158/task/33466/cwd (readlink: Permission denied) 2024-09-21T03:42:13.396 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33466 mgr-fin ceph rtd unknown /proc/31158/task/33466/root (readlink: Permission denied) 2024-09-21T03:42:13.396 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33466 mgr-fin ceph txt unknown /proc/31158/task/33466/exe (readlink: Permission denied) 2024-09-21T03:42:13.396 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33466 mgr-fin ceph NOFD /proc/31158/task/33466/fd (opendir: Permission denied) 2024-09-21T03:42:13.396 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33467 mgr-fin ceph cwd unknown /proc/31158/task/33467/cwd (readlink: Permission denied) 2024-09-21T03:42:13.397 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33467 mgr-fin ceph rtd unknown /proc/31158/task/33467/root (readlink: Permission denied) 2024-09-21T03:42:13.397 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33467 mgr-fin ceph txt unknown /proc/31158/task/33467/exe (readlink: Permission denied) 2024-09-21T03:42:13.397 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33467 mgr-fin ceph NOFD /proc/31158/task/33467/fd (opendir: Permission denied) 2024-09-21T03:42:13.397 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33468 mgr-fin ceph cwd unknown /proc/31158/task/33468/cwd (readlink: Permission denied) 2024-09-21T03:42:13.397 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33468 mgr-fin ceph rtd unknown /proc/31158/task/33468/root (readlink: Permission denied) 2024-09-21T03:42:13.397 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33468 mgr-fin ceph txt unknown /proc/31158/task/33468/exe (readlink: Permission denied) 2024-09-21T03:42:13.397 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33468 mgr-fin ceph NOFD /proc/31158/task/33468/fd (opendir: Permission denied) 2024-09-21T03:42:13.397 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33472 mgr-fin ceph cwd unknown /proc/31158/task/33472/cwd (readlink: Permission denied) 2024-09-21T03:42:13.397 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33472 mgr-fin ceph rtd unknown /proc/31158/task/33472/root (readlink: Permission denied) 2024-09-21T03:42:13.397 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33472 mgr-fin ceph txt unknown /proc/31158/task/33472/exe (readlink: Permission denied) 2024-09-21T03:42:13.397 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33472 mgr-fin ceph NOFD /proc/31158/task/33472/fd (opendir: Permission denied) 2024-09-21T03:42:13.397 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33473 io_contex ceph cwd unknown /proc/31158/task/33473/cwd (readlink: Permission denied) 2024-09-21T03:42:13.397 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33473 io_contex ceph rtd unknown /proc/31158/task/33473/root (readlink: Permission denied) 2024-09-21T03:42:13.398 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33473 io_contex ceph txt unknown /proc/31158/task/33473/exe (readlink: Permission denied) 2024-09-21T03:42:13.398 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33473 io_contex ceph NOFD /proc/31158/task/33473/fd (opendir: Permission denied) 2024-09-21T03:42:13.398 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33474 io_contex ceph cwd unknown /proc/31158/task/33474/cwd (readlink: Permission denied) 2024-09-21T03:42:13.398 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33474 io_contex ceph rtd unknown /proc/31158/task/33474/root (readlink: Permission denied) 2024-09-21T03:42:13.398 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33474 io_contex ceph txt unknown /proc/31158/task/33474/exe (readlink: Permission denied) 2024-09-21T03:42:13.398 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33474 io_contex ceph NOFD /proc/31158/task/33474/fd (opendir: Permission denied) 2024-09-21T03:42:13.398 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33475 ceph_time ceph cwd unknown /proc/31158/task/33475/cwd (readlink: Permission denied) 2024-09-21T03:42:13.398 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33475 ceph_time ceph rtd unknown /proc/31158/task/33475/root (readlink: Permission denied) 2024-09-21T03:42:13.398 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33475 ceph_time ceph txt unknown /proc/31158/task/33475/exe (readlink: Permission denied) 2024-09-21T03:42:13.398 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33475 ceph_time ceph NOFD /proc/31158/task/33475/fd (opendir: Permission denied) 2024-09-21T03:42:13.398 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33476 ms_dispat ceph cwd unknown /proc/31158/task/33476/cwd (readlink: Permission denied) 2024-09-21T03:42:13.398 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33476 ms_dispat ceph rtd unknown /proc/31158/task/33476/root (readlink: Permission denied) 2024-09-21T03:42:13.398 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33476 ms_dispat ceph txt unknown /proc/31158/task/33476/exe (readlink: Permission denied) 2024-09-21T03:42:13.398 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33476 ms_dispat ceph NOFD /proc/31158/task/33476/fd (opendir: Permission denied) 2024-09-21T03:42:13.399 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33477 ms_local ceph cwd unknown /proc/31158/task/33477/cwd (readlink: Permission denied) 2024-09-21T03:42:13.399 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33477 ms_local ceph rtd unknown /proc/31158/task/33477/root (readlink: Permission denied) 2024-09-21T03:42:13.399 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33477 ms_local ceph txt unknown /proc/31158/task/33477/exe (readlink: Permission denied) 2024-09-21T03:42:13.399 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33477 ms_local ceph NOFD /proc/31158/task/33477/fd (opendir: Permission denied) 2024-09-21T03:42:13.399 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33478 safe_time ceph cwd unknown /proc/31158/task/33478/cwd (readlink: Permission denied) 2024-09-21T03:42:13.399 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33478 safe_time ceph rtd unknown /proc/31158/task/33478/root (readlink: Permission denied) 2024-09-21T03:42:13.399 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33478 safe_time ceph txt unknown /proc/31158/task/33478/exe (readlink: Permission denied) 2024-09-21T03:42:13.399 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33478 safe_time ceph NOFD /proc/31158/task/33478/fd (opendir: Permission denied) 2024-09-21T03:42:13.399 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33479 safe_time ceph cwd unknown /proc/31158/task/33479/cwd (readlink: Permission denied) 2024-09-21T03:42:13.399 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33479 safe_time ceph rtd unknown /proc/31158/task/33479/root (readlink: Permission denied) 2024-09-21T03:42:13.399 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33479 safe_time ceph txt unknown /proc/31158/task/33479/exe (readlink: Permission denied) 2024-09-21T03:42:13.399 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33479 safe_time ceph NOFD /proc/31158/task/33479/fd (opendir: Permission denied) 2024-09-21T03:42:13.399 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33480 mgr-fin ceph cwd unknown /proc/31158/task/33480/cwd (readlink: Permission denied) 2024-09-21T03:42:13.399 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33480 mgr-fin ceph rtd unknown /proc/31158/task/33480/root (readlink: Permission denied) 2024-09-21T03:42:13.399 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33480 mgr-fin ceph txt unknown /proc/31158/task/33480/exe (readlink: Permission denied) 2024-09-21T03:42:13.399 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33480 mgr-fin ceph NOFD /proc/31158/task/33480/fd (opendir: Permission denied) 2024-09-21T03:42:13.400 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33481 mgr-fin ceph cwd unknown /proc/31158/task/33481/cwd (readlink: Permission denied) 2024-09-21T03:42:13.400 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33481 mgr-fin ceph rtd unknown /proc/31158/task/33481/root (readlink: Permission denied) 2024-09-21T03:42:13.400 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33481 mgr-fin ceph txt unknown /proc/31158/task/33481/exe (readlink: Permission denied) 2024-09-21T03:42:13.400 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33481 mgr-fin ceph NOFD /proc/31158/task/33481/fd (opendir: Permission denied) 2024-09-21T03:42:13.400 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33482 mgr-fin ceph cwd unknown /proc/31158/task/33482/cwd (readlink: Permission denied) 2024-09-21T03:42:13.400 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33482 mgr-fin ceph rtd unknown /proc/31158/task/33482/root (readlink: Permission denied) 2024-09-21T03:42:13.400 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33482 mgr-fin ceph txt unknown /proc/31158/task/33482/exe (readlink: Permission denied) 2024-09-21T03:42:13.400 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33482 mgr-fin ceph NOFD /proc/31158/task/33482/fd (opendir: Permission denied) 2024-09-21T03:42:13.400 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33483 mgr-fin ceph cwd unknown /proc/31158/task/33483/cwd (readlink: Permission denied) 2024-09-21T03:42:13.400 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33483 mgr-fin ceph rtd unknown /proc/31158/task/33483/root (readlink: Permission denied) 2024-09-21T03:42:13.400 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33483 mgr-fin ceph txt unknown /proc/31158/task/33483/exe (readlink: Permission denied) 2024-09-21T03:42:13.400 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33483 mgr-fin ceph NOFD /proc/31158/task/33483/fd (opendir: Permission denied) 2024-09-21T03:42:13.400 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33484 mgr-fin ceph cwd unknown /proc/31158/task/33484/cwd (readlink: Permission denied) 2024-09-21T03:42:13.400 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33484 mgr-fin ceph rtd unknown /proc/31158/task/33484/root (readlink: Permission denied) 2024-09-21T03:42:13.401 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33484 mgr-fin ceph txt unknown /proc/31158/task/33484/exe (readlink: Permission denied) 2024-09-21T03:42:13.401 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33484 mgr-fin ceph NOFD /proc/31158/task/33484/fd (opendir: Permission denied) 2024-09-21T03:42:13.401 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33485 mgr-fin ceph cwd unknown /proc/31158/task/33485/cwd (readlink: Permission denied) 2024-09-21T03:42:13.401 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33485 mgr-fin ceph rtd unknown /proc/31158/task/33485/root (readlink: Permission denied) 2024-09-21T03:42:13.401 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33485 mgr-fin ceph txt unknown /proc/31158/task/33485/exe (readlink: Permission denied) 2024-09-21T03:42:13.401 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33485 mgr-fin ceph NOFD /proc/31158/task/33485/fd (opendir: Permission denied) 2024-09-21T03:42:13.401 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33486 mgr-fin ceph cwd unknown /proc/31158/task/33486/cwd (readlink: Permission denied) 2024-09-21T03:42:13.401 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33486 mgr-fin ceph rtd unknown /proc/31158/task/33486/root (readlink: Permission denied) 2024-09-21T03:42:13.401 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33486 mgr-fin ceph txt unknown /proc/31158/task/33486/exe (readlink: Permission denied) 2024-09-21T03:42:13.401 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33486 mgr-fin ceph NOFD /proc/31158/task/33486/fd (opendir: Permission denied) 2024-09-21T03:42:13.401 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33487 mgr-fin ceph cwd unknown /proc/31158/task/33487/cwd (readlink: Permission denied) 2024-09-21T03:42:13.401 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33487 mgr-fin ceph rtd unknown /proc/31158/task/33487/root (readlink: Permission denied) 2024-09-21T03:42:13.401 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33487 mgr-fin ceph txt unknown /proc/31158/task/33487/exe (readlink: Permission denied) 2024-09-21T03:42:13.401 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33487 mgr-fin ceph NOFD /proc/31158/task/33487/fd (opendir: Permission denied) 2024-09-21T03:42:13.401 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33488 mgr-fin ceph cwd unknown /proc/31158/task/33488/cwd (readlink: Permission denied) 2024-09-21T03:42:13.402 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33488 mgr-fin ceph rtd unknown /proc/31158/task/33488/root (readlink: Permission denied) 2024-09-21T03:42:13.402 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33488 mgr-fin ceph txt unknown /proc/31158/task/33488/exe (readlink: Permission denied) 2024-09-21T03:42:13.402 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33488 mgr-fin ceph NOFD /proc/31158/task/33488/fd (opendir: Permission denied) 2024-09-21T03:42:13.402 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33489 mgr-fin ceph cwd unknown /proc/31158/task/33489/cwd (readlink: Permission denied) 2024-09-21T03:42:13.402 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33489 mgr-fin ceph rtd unknown /proc/31158/task/33489/root (readlink: Permission denied) 2024-09-21T03:42:13.402 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33489 mgr-fin ceph txt unknown /proc/31158/task/33489/exe (readlink: Permission denied) 2024-09-21T03:42:13.402 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33489 mgr-fin ceph NOFD /proc/31158/task/33489/fd (opendir: Permission denied) 2024-09-21T03:42:13.402 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33490 mgr-fin ceph cwd unknown /proc/31158/task/33490/cwd (readlink: Permission denied) 2024-09-21T03:42:13.402 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33490 mgr-fin ceph rtd unknown /proc/31158/task/33490/root (readlink: Permission denied) 2024-09-21T03:42:13.402 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33490 mgr-fin ceph txt unknown /proc/31158/task/33490/exe (readlink: Permission denied) 2024-09-21T03:42:13.402 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33490 mgr-fin ceph NOFD /proc/31158/task/33490/fd (opendir: Permission denied) 2024-09-21T03:42:13.402 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33491 mgr-fin ceph cwd unknown /proc/31158/task/33491/cwd (readlink: Permission denied) 2024-09-21T03:42:13.402 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33491 mgr-fin ceph rtd unknown /proc/31158/task/33491/root (readlink: Permission denied) 2024-09-21T03:42:13.402 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33491 mgr-fin ceph txt unknown /proc/31158/task/33491/exe (readlink: Permission denied) 2024-09-21T03:42:13.402 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33491 mgr-fin ceph NOFD /proc/31158/task/33491/fd (opendir: Permission denied) 2024-09-21T03:42:13.403 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33493 m-fin-vol ceph cwd unknown /proc/31158/task/33493/cwd (readlink: Permission denied) 2024-09-21T03:42:13.403 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33493 m-fin-vol ceph rtd unknown /proc/31158/task/33493/root (readlink: Permission denied) 2024-09-21T03:42:13.403 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33493 m-fin-vol ceph txt unknown /proc/31158/task/33493/exe (readlink: Permission denied) 2024-09-21T03:42:13.403 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33493 m-fin-vol ceph NOFD /proc/31158/task/33493/fd (opendir: Permission denied) 2024-09-21T03:42:13.403 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33662 mgr-fin ceph cwd unknown /proc/31158/task/33662/cwd (readlink: Permission denied) 2024-09-21T03:42:13.403 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33662 mgr-fin ceph rtd unknown /proc/31158/task/33662/root (readlink: Permission denied) 2024-09-21T03:42:13.403 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33662 mgr-fin ceph txt unknown /proc/31158/task/33662/exe (readlink: Permission denied) 2024-09-21T03:42:13.403 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33662 mgr-fin ceph NOFD /proc/31158/task/33662/fd (opendir: Permission denied) 2024-09-21T03:42:13.403 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33663 mgr-fin ceph cwd unknown /proc/31158/task/33663/cwd (readlink: Permission denied) 2024-09-21T03:42:13.403 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33663 mgr-fin ceph rtd unknown /proc/31158/task/33663/root (readlink: Permission denied) 2024-09-21T03:42:13.403 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33663 mgr-fin ceph txt unknown /proc/31158/task/33663/exe (readlink: Permission denied) 2024-09-21T03:42:13.403 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33663 mgr-fin ceph NOFD /proc/31158/task/33663/fd (opendir: Permission denied) 2024-09-21T03:42:13.403 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33664 mgr-fin ceph cwd unknown /proc/31158/task/33664/cwd (readlink: Permission denied) 2024-09-21T03:42:13.403 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33664 mgr-fin ceph rtd unknown /proc/31158/task/33664/root (readlink: Permission denied) 2024-09-21T03:42:13.404 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33664 mgr-fin ceph txt unknown /proc/31158/task/33664/exe (readlink: Permission denied) 2024-09-21T03:42:13.404 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33664 mgr-fin ceph NOFD /proc/31158/task/33664/fd (opendir: Permission denied) 2024-09-21T03:42:13.404 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33665 mgr-fin ceph cwd unknown /proc/31158/task/33665/cwd (readlink: Permission denied) 2024-09-21T03:42:13.404 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33665 mgr-fin ceph rtd unknown /proc/31158/task/33665/root (readlink: Permission denied) 2024-09-21T03:42:13.404 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33665 mgr-fin ceph txt unknown /proc/31158/task/33665/exe (readlink: Permission denied) 2024-09-21T03:42:13.404 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33665 mgr-fin ceph NOFD /proc/31158/task/33665/fd (opendir: Permission denied) 2024-09-21T03:42:13.404 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33666 mgr-fin ceph cwd unknown /proc/31158/task/33666/cwd (readlink: Permission denied) 2024-09-21T03:42:13.404 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33666 mgr-fin ceph rtd unknown /proc/31158/task/33666/root (readlink: Permission denied) 2024-09-21T03:42:13.404 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33666 mgr-fin ceph txt unknown /proc/31158/task/33666/exe (readlink: Permission denied) 2024-09-21T03:42:13.404 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33666 mgr-fin ceph NOFD /proc/31158/task/33666/fd (opendir: Permission denied) 2024-09-21T03:42:13.404 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33667 mgr-fin ceph cwd unknown /proc/31158/task/33667/cwd (readlink: Permission denied) 2024-09-21T03:42:13.404 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33667 mgr-fin ceph rtd unknown /proc/31158/task/33667/root (readlink: Permission denied) 2024-09-21T03:42:13.404 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33667 mgr-fin ceph txt unknown /proc/31158/task/33667/exe (readlink: Permission denied) 2024-09-21T03:42:13.404 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33667 mgr-fin ceph NOFD /proc/31158/task/33667/fd (opendir: Permission denied) 2024-09-21T03:42:13.404 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33668 mgr-fin ceph cwd unknown /proc/31158/task/33668/cwd (readlink: Permission denied) 2024-09-21T03:42:13.405 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33668 mgr-fin ceph rtd unknown /proc/31158/task/33668/root (readlink: Permission denied) 2024-09-21T03:42:13.405 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33668 mgr-fin ceph txt unknown /proc/31158/task/33668/exe (readlink: Permission denied) 2024-09-21T03:42:13.405 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33668 mgr-fin ceph NOFD /proc/31158/task/33668/fd (opendir: Permission denied) 2024-09-21T03:42:13.405 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33669 mgr-fin ceph cwd unknown /proc/31158/task/33669/cwd (readlink: Permission denied) 2024-09-21T03:42:13.405 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33669 mgr-fin ceph rtd unknown /proc/31158/task/33669/root (readlink: Permission denied) 2024-09-21T03:42:13.405 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33669 mgr-fin ceph txt unknown /proc/31158/task/33669/exe (readlink: Permission denied) 2024-09-21T03:42:13.405 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33669 mgr-fin ceph NOFD /proc/31158/task/33669/fd (opendir: Permission denied) 2024-09-21T03:42:13.405 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33670 mgr-fin ceph cwd unknown /proc/31158/task/33670/cwd (readlink: Permission denied) 2024-09-21T03:42:13.405 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33670 mgr-fin ceph rtd unknown /proc/31158/task/33670/root (readlink: Permission denied) 2024-09-21T03:42:13.405 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33670 mgr-fin ceph txt unknown /proc/31158/task/33670/exe (readlink: Permission denied) 2024-09-21T03:42:13.405 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33670 mgr-fin ceph NOFD /proc/31158/task/33670/fd (opendir: Permission denied) 2024-09-21T03:42:13.405 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33671 mgr-fin ceph cwd unknown /proc/31158/task/33671/cwd (readlink: Permission denied) 2024-09-21T03:42:13.405 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33671 mgr-fin ceph rtd unknown /proc/31158/task/33671/root (readlink: Permission denied) 2024-09-21T03:42:13.405 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33671 mgr-fin ceph txt unknown /proc/31158/task/33671/exe (readlink: Permission denied) 2024-09-21T03:42:13.405 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33671 mgr-fin ceph NOFD /proc/31158/task/33671/fd (opendir: Permission denied) 2024-09-21T03:42:13.405 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33672 mgr-fin ceph cwd unknown /proc/31158/task/33672/cwd (readlink: Permission denied) 2024-09-21T03:42:13.405 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33672 mgr-fin ceph rtd unknown /proc/31158/task/33672/root (readlink: Permission denied) 2024-09-21T03:42:13.406 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33672 mgr-fin ceph txt unknown /proc/31158/task/33672/exe (readlink: Permission denied) 2024-09-21T03:42:13.406 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33672 mgr-fin ceph NOFD /proc/31158/task/33672/fd (opendir: Permission denied) 2024-09-21T03:42:13.406 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33674 mgr-fin ceph cwd unknown /proc/31158/task/33674/cwd (readlink: Permission denied) 2024-09-21T03:42:13.406 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33674 mgr-fin ceph rtd unknown /proc/31158/task/33674/root (readlink: Permission denied) 2024-09-21T03:42:13.406 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33674 mgr-fin ceph txt unknown /proc/31158/task/33674/exe (readlink: Permission denied) 2024-09-21T03:42:13.406 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33674 mgr-fin ceph NOFD /proc/31158/task/33674/fd (opendir: Permission denied) 2024-09-21T03:42:13.406 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33675 mgr-fin ceph cwd unknown /proc/31158/task/33675/cwd (readlink: Permission denied) 2024-09-21T03:42:13.406 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33675 mgr-fin ceph rtd unknown /proc/31158/task/33675/root (readlink: Permission denied) 2024-09-21T03:42:13.406 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33675 mgr-fin ceph txt unknown /proc/31158/task/33675/exe (readlink: Permission denied) 2024-09-21T03:42:13.406 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33675 mgr-fin ceph NOFD /proc/31158/task/33675/fd (opendir: Permission denied) 2024-09-21T03:42:13.406 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33676 mgr-fin ceph cwd unknown /proc/31158/task/33676/cwd (readlink: Permission denied) 2024-09-21T03:42:13.406 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33676 mgr-fin ceph rtd unknown /proc/31158/task/33676/root (readlink: Permission denied) 2024-09-21T03:42:13.406 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33676 mgr-fin ceph txt unknown /proc/31158/task/33676/exe (readlink: Permission denied) 2024-09-21T03:42:13.406 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33676 mgr-fin ceph NOFD /proc/31158/task/33676/fd (opendir: Permission denied) 2024-09-21T03:42:13.406 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33677 mgr-fin ceph cwd unknown /proc/31158/task/33677/cwd (readlink: Permission denied) 2024-09-21T03:42:13.407 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33677 mgr-fin ceph rtd unknown /proc/31158/task/33677/root (readlink: Permission denied) 2024-09-21T03:42:13.407 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33677 mgr-fin ceph txt unknown /proc/31158/task/33677/exe (readlink: Permission denied) 2024-09-21T03:42:13.407 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33677 mgr-fin ceph NOFD /proc/31158/task/33677/fd (opendir: Permission denied) 2024-09-21T03:42:13.407 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33678 mgr-fin ceph cwd unknown /proc/31158/task/33678/cwd (readlink: Permission denied) 2024-09-21T03:42:13.407 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33678 mgr-fin ceph rtd unknown /proc/31158/task/33678/root (readlink: Permission denied) 2024-09-21T03:42:13.407 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33678 mgr-fin ceph txt unknown /proc/31158/task/33678/exe (readlink: Permission denied) 2024-09-21T03:42:13.407 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33678 mgr-fin ceph NOFD /proc/31158/task/33678/fd (opendir: Permission denied) 2024-09-21T03:42:13.407 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33679 mgr-fin ceph cwd unknown /proc/31158/task/33679/cwd (readlink: Permission denied) 2024-09-21T03:42:13.407 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33679 mgr-fin ceph rtd unknown /proc/31158/task/33679/root (readlink: Permission denied) 2024-09-21T03:42:13.407 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33679 mgr-fin ceph txt unknown /proc/31158/task/33679/exe (readlink: Permission denied) 2024-09-21T03:42:13.407 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33679 mgr-fin ceph NOFD /proc/31158/task/33679/fd (opendir: Permission denied) 2024-09-21T03:42:13.407 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33680 mgr-fin ceph cwd unknown /proc/31158/task/33680/cwd (readlink: Permission denied) 2024-09-21T03:42:13.407 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33680 mgr-fin ceph rtd unknown /proc/31158/task/33680/root (readlink: Permission denied) 2024-09-21T03:42:13.407 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33680 mgr-fin ceph txt unknown /proc/31158/task/33680/exe (readlink: Permission denied) 2024-09-21T03:42:13.408 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33680 mgr-fin ceph NOFD /proc/31158/task/33680/fd (opendir: Permission denied) 2024-09-21T03:42:13.408 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33681 mgr-fin ceph cwd unknown /proc/31158/task/33681/cwd (readlink: Permission denied) 2024-09-21T03:42:13.408 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33681 mgr-fin ceph rtd unknown /proc/31158/task/33681/root (readlink: Permission denied) 2024-09-21T03:42:13.408 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33681 mgr-fin ceph txt unknown /proc/31158/task/33681/exe (readlink: Permission denied) 2024-09-21T03:42:13.408 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33681 mgr-fin ceph NOFD /proc/31158/task/33681/fd (opendir: Permission denied) 2024-09-21T03:42:13.408 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33682 mgr-fin ceph cwd unknown /proc/31158/task/33682/cwd (readlink: Permission denied) 2024-09-21T03:42:13.408 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33682 mgr-fin ceph rtd unknown /proc/31158/task/33682/root (readlink: Permission denied) 2024-09-21T03:42:13.408 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33682 mgr-fin ceph txt unknown /proc/31158/task/33682/exe (readlink: Permission denied) 2024-09-21T03:42:13.408 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33682 mgr-fin ceph NOFD /proc/31158/task/33682/fd (opendir: Permission denied) 2024-09-21T03:42:13.408 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33683 mgr-fin ceph cwd unknown /proc/31158/task/33683/cwd (readlink: Permission denied) 2024-09-21T03:42:13.408 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33683 mgr-fin ceph rtd unknown /proc/31158/task/33683/root (readlink: Permission denied) 2024-09-21T03:42:13.408 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33683 mgr-fin ceph txt unknown /proc/31158/task/33683/exe (readlink: Permission denied) 2024-09-21T03:42:13.408 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33683 mgr-fin ceph NOFD /proc/31158/task/33683/fd (opendir: Permission denied) 2024-09-21T03:42:13.408 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33684 mgr-fin ceph cwd unknown /proc/31158/task/33684/cwd (readlink: Permission denied) 2024-09-21T03:42:13.408 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33684 mgr-fin ceph rtd unknown /proc/31158/task/33684/root (readlink: Permission denied) 2024-09-21T03:42:13.409 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33684 mgr-fin ceph txt unknown /proc/31158/task/33684/exe (readlink: Permission denied) 2024-09-21T03:42:13.409 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 33684 mgr-fin ceph NOFD /proc/31158/task/33684/fd (opendir: Permission denied) 2024-09-21T03:42:13.409 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 34258 dashboard ceph cwd unknown /proc/31158/task/34258/cwd (readlink: Permission denied) 2024-09-21T03:42:13.409 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 34258 dashboard ceph rtd unknown /proc/31158/task/34258/root (readlink: Permission denied) 2024-09-21T03:42:13.409 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 34258 dashboard ceph txt unknown /proc/31158/task/34258/exe (readlink: Permission denied) 2024-09-21T03:42:13.409 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 34258 dashboard ceph NOFD /proc/31158/task/34258/fd (opendir: Permission denied) 2024-09-21T03:42:13.409 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 34259 dashboard ceph cwd unknown /proc/31158/task/34259/cwd (readlink: Permission denied) 2024-09-21T03:42:13.409 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 34259 dashboard ceph rtd unknown /proc/31158/task/34259/root (readlink: Permission denied) 2024-09-21T03:42:13.409 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 34259 dashboard ceph txt unknown /proc/31158/task/34259/exe (readlink: Permission denied) 2024-09-21T03:42:13.409 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 34259 dashboard ceph NOFD /proc/31158/task/34259/fd (opendir: Permission denied) 2024-09-21T03:42:13.409 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 34260 dashboard ceph cwd unknown /proc/31158/task/34260/cwd (readlink: Permission denied) 2024-09-21T03:42:13.409 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 34260 dashboard ceph rtd unknown /proc/31158/task/34260/root (readlink: Permission denied) 2024-09-21T03:42:13.409 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 34260 dashboard ceph txt unknown /proc/31158/task/34260/exe (readlink: Permission denied) 2024-09-21T03:42:13.409 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 34260 dashboard ceph NOFD /proc/31158/task/34260/fd (opendir: Permission denied) 2024-09-21T03:42:13.410 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 34261 dashboard ceph cwd unknown /proc/31158/task/34261/cwd (readlink: Permission denied) 2024-09-21T03:42:13.410 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 34261 dashboard ceph rtd unknown /proc/31158/task/34261/root (readlink: Permission denied) 2024-09-21T03:42:13.410 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 34261 dashboard ceph txt unknown /proc/31158/task/34261/exe (readlink: Permission denied) 2024-09-21T03:42:13.410 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 34261 dashboard ceph NOFD /proc/31158/task/34261/fd (opendir: Permission denied) 2024-09-21T03:42:13.410 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 34262 dashboard ceph cwd unknown /proc/31158/task/34262/cwd (readlink: Permission denied) 2024-09-21T03:42:13.410 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 34262 dashboard ceph rtd unknown /proc/31158/task/34262/root (readlink: Permission denied) 2024-09-21T03:42:13.410 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 34262 dashboard ceph txt unknown /proc/31158/task/34262/exe (readlink: Permission denied) 2024-09-21T03:42:13.410 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 34262 dashboard ceph NOFD /proc/31158/task/34262/fd (opendir: Permission denied) 2024-09-21T03:42:13.410 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 34263 dashboard ceph cwd unknown /proc/31158/task/34263/cwd (readlink: Permission denied) 2024-09-21T03:42:13.410 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 34263 dashboard ceph rtd unknown /proc/31158/task/34263/root (readlink: Permission denied) 2024-09-21T03:42:13.410 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 34263 dashboard ceph txt unknown /proc/31158/task/34263/exe (readlink: Permission denied) 2024-09-21T03:42:13.410 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 34263 dashboard ceph NOFD /proc/31158/task/34263/fd (opendir: Permission denied) 2024-09-21T03:42:13.410 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 34264 dashboard ceph cwd unknown /proc/31158/task/34264/cwd (readlink: Permission denied) 2024-09-21T03:42:13.411 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 34264 dashboard ceph rtd unknown /proc/31158/task/34264/root (readlink: Permission denied) 2024-09-21T03:42:13.411 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 34264 dashboard ceph txt unknown /proc/31158/task/34264/exe (readlink: Permission denied) 2024-09-21T03:42:13.411 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 34264 dashboard ceph NOFD /proc/31158/task/34264/fd (opendir: Permission denied) 2024-09-21T03:42:13.411 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 34265 dashboard ceph cwd unknown /proc/31158/task/34265/cwd (readlink: Permission denied) 2024-09-21T03:42:13.411 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 34265 dashboard ceph rtd unknown /proc/31158/task/34265/root (readlink: Permission denied) 2024-09-21T03:42:13.411 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 34265 dashboard ceph txt unknown /proc/31158/task/34265/exe (readlink: Permission denied) 2024-09-21T03:42:13.411 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 34265 dashboard ceph NOFD /proc/31158/task/34265/fd (opendir: Permission denied) 2024-09-21T03:42:13.411 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 34267 dashboard ceph cwd unknown /proc/31158/task/34267/cwd (readlink: Permission denied) 2024-09-21T03:42:13.411 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 34267 dashboard ceph rtd unknown /proc/31158/task/34267/root (readlink: Permission denied) 2024-09-21T03:42:13.411 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 34267 dashboard ceph txt unknown /proc/31158/task/34267/exe (readlink: Permission denied) 2024-09-21T03:42:13.411 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 34267 dashboard ceph NOFD /proc/31158/task/34267/fd (opendir: Permission denied) 2024-09-21T03:42:13.411 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 34268 dashboard ceph cwd unknown /proc/31158/task/34268/cwd (readlink: Permission denied) 2024-09-21T03:42:13.411 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 34268 dashboard ceph rtd unknown /proc/31158/task/34268/root (readlink: Permission denied) 2024-09-21T03:42:13.411 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 34268 dashboard ceph txt unknown /proc/31158/task/34268/exe (readlink: Permission denied) 2024-09-21T03:42:13.411 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 34268 dashboard ceph NOFD /proc/31158/task/34268/fd (opendir: Permission denied) 2024-09-21T03:42:13.412 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 34269 dashboard ceph cwd unknown /proc/31158/task/34269/cwd (readlink: Permission denied) 2024-09-21T03:42:13.412 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 34269 dashboard ceph rtd unknown /proc/31158/task/34269/root (readlink: Permission denied) 2024-09-21T03:42:13.412 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 34269 dashboard ceph txt unknown /proc/31158/task/34269/exe (readlink: Permission denied) 2024-09-21T03:42:13.412 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 34269 dashboard ceph NOFD /proc/31158/task/34269/fd (opendir: Permission denied) 2024-09-21T03:42:13.412 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 34323 dashboard ceph cwd unknown /proc/31158/task/34323/cwd (readlink: Permission denied) 2024-09-21T03:42:13.412 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 34323 dashboard ceph rtd unknown /proc/31158/task/34323/root (readlink: Permission denied) 2024-09-21T03:42:13.412 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 34323 dashboard ceph txt unknown /proc/31158/task/34323/exe (readlink: Permission denied) 2024-09-21T03:42:13.412 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 34323 dashboard ceph NOFD /proc/31158/task/34323/fd (opendir: Permission denied) 2024-09-21T03:42:13.412 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 50762 devicehea ceph cwd unknown /proc/31158/task/50762/cwd (readlink: Permission denied) 2024-09-21T03:42:13.412 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 50762 devicehea ceph rtd unknown /proc/31158/task/50762/root (readlink: Permission denied) 2024-09-21T03:42:13.412 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 50762 devicehea ceph txt unknown /proc/31158/task/50762/exe (readlink: Permission denied) 2024-09-21T03:42:13.412 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 50762 devicehea ceph NOFD /proc/31158/task/50762/fd (opendir: Permission denied) 2024-09-21T03:42:13.412 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 50768 io_contex ceph cwd unknown /proc/31158/task/50768/cwd (readlink: Permission denied) 2024-09-21T03:42:13.412 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 50768 io_contex ceph rtd unknown /proc/31158/task/50768/root (readlink: Permission denied) 2024-09-21T03:42:13.412 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 50768 io_contex ceph txt unknown /proc/31158/task/50768/exe (readlink: Permission denied) 2024-09-21T03:42:13.413 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 50768 io_contex ceph NOFD /proc/31158/task/50768/fd (opendir: Permission denied) 2024-09-21T03:42:13.413 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 50769 io_contex ceph cwd unknown /proc/31158/task/50769/cwd (readlink: Permission denied) 2024-09-21T03:42:13.413 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 50769 io_contex ceph rtd unknown /proc/31158/task/50769/root (readlink: Permission denied) 2024-09-21T03:42:13.413 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 50769 io_contex ceph txt unknown /proc/31158/task/50769/exe (readlink: Permission denied) 2024-09-21T03:42:13.413 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 50769 io_contex ceph NOFD /proc/31158/task/50769/fd (opendir: Permission denied) 2024-09-21T03:42:13.413 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 50770 ceph_time ceph cwd unknown /proc/31158/task/50770/cwd (readlink: Permission denied) 2024-09-21T03:42:13.413 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 50770 ceph_time ceph rtd unknown /proc/31158/task/50770/root (readlink: Permission denied) 2024-09-21T03:42:13.413 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 50770 ceph_time ceph txt unknown /proc/31158/task/50770/exe (readlink: Permission denied) 2024-09-21T03:42:13.413 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 50770 ceph_time ceph NOFD /proc/31158/task/50770/fd (opendir: Permission denied) 2024-09-21T03:42:13.413 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 50771 ms_dispat ceph cwd unknown /proc/31158/task/50771/cwd (readlink: Permission denied) 2024-09-21T03:42:13.413 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 50771 ms_dispat ceph rtd unknown /proc/31158/task/50771/root (readlink: Permission denied) 2024-09-21T03:42:13.413 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 50771 ms_dispat ceph txt unknown /proc/31158/task/50771/exe (readlink: Permission denied) 2024-09-21T03:42:13.413 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 50771 ms_dispat ceph NOFD /proc/31158/task/50771/fd (opendir: Permission denied) 2024-09-21T03:42:13.413 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 50772 ms_local ceph cwd unknown /proc/31158/task/50772/cwd (readlink: Permission denied) 2024-09-21T03:42:13.413 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 50772 ms_local ceph rtd unknown /proc/31158/task/50772/root (readlink: Permission denied) 2024-09-21T03:42:13.414 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 50772 ms_local ceph txt unknown /proc/31158/task/50772/exe (readlink: Permission denied) 2024-09-21T03:42:13.414 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 50772 ms_local ceph NOFD /proc/31158/task/50772/fd (opendir: Permission denied) 2024-09-21T03:42:13.414 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 50773 safe_time ceph cwd unknown /proc/31158/task/50773/cwd (readlink: Permission denied) 2024-09-21T03:42:13.414 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 50773 safe_time ceph rtd unknown /proc/31158/task/50773/root (readlink: Permission denied) 2024-09-21T03:42:13.414 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 50773 safe_time ceph txt unknown /proc/31158/task/50773/exe (readlink: Permission denied) 2024-09-21T03:42:13.414 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 50773 safe_time ceph NOFD /proc/31158/task/50773/fd (opendir: Permission denied) 2024-09-21T03:42:13.414 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 50774 safe_time ceph cwd unknown /proc/31158/task/50774/cwd (readlink: Permission denied) 2024-09-21T03:42:13.414 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 50774 safe_time ceph rtd unknown /proc/31158/task/50774/root (readlink: Permission denied) 2024-09-21T03:42:13.414 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 50774 safe_time ceph txt unknown /proc/31158/task/50774/exe (readlink: Permission denied) 2024-09-21T03:42:13.414 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mgr 31158 50774 safe_time ceph NOFD /proc/31158/task/50774/fd (opendir: Permission denied) 2024-09-21T03:42:13.414 INFO:tasks.workunit.client.0.smithi089.stdout:systemd 32181 root cwd unknown /proc/32181/cwd (readlink: Permission denied) 2024-09-21T03:42:13.414 INFO:tasks.workunit.client.0.smithi089.stdout:systemd 32181 root rtd unknown /proc/32181/root (readlink: Permission denied) 2024-09-21T03:42:13.414 INFO:tasks.workunit.client.0.smithi089.stdout:systemd 32181 root txt unknown /proc/32181/exe (readlink: Permission denied) 2024-09-21T03:42:13.415 INFO:tasks.workunit.client.0.smithi089.stdout:systemd 32181 root NOFD /proc/32181/fd (opendir: Permission denied) 2024-09-21T03:42:13.415 INFO:tasks.workunit.client.0.smithi089.stdout:(sd-pam) 32183 root cwd unknown /proc/32183/cwd (readlink: Permission denied) 2024-09-21T03:42:13.415 INFO:tasks.workunit.client.0.smithi089.stdout:(sd-pam) 32183 root rtd unknown /proc/32183/root (readlink: Permission denied) 2024-09-21T03:42:13.415 INFO:tasks.workunit.client.0.smithi089.stdout:(sd-pam) 32183 root txt unknown /proc/32183/exe (readlink: Permission denied) 2024-09-21T03:42:13.415 INFO:tasks.workunit.client.0.smithi089.stdout:(sd-pam) 32183 root NOFD /proc/32183/fd (opendir: Permission denied) 2024-09-21T03:42:13.415 INFO:tasks.workunit.client.0.smithi089.stdout:sshd 33494 root cwd unknown /proc/33494/cwd (readlink: Permission denied) 2024-09-21T03:42:13.415 INFO:tasks.workunit.client.0.smithi089.stdout:sshd 33494 root rtd unknown /proc/33494/root (readlink: Permission denied) 2024-09-21T03:42:13.415 INFO:tasks.workunit.client.0.smithi089.stdout:sshd 33494 root txt unknown /proc/33494/exe (readlink: Permission denied) 2024-09-21T03:42:13.415 INFO:tasks.workunit.client.0.smithi089.stdout:sshd 33494 root NOFD /proc/33494/fd (opendir: Permission denied) 2024-09-21T03:42:13.415 INFO:tasks.workunit.client.0.smithi089.stdout:sshd 33497 root cwd unknown /proc/33497/cwd (readlink: Permission denied) 2024-09-21T03:42:13.415 INFO:tasks.workunit.client.0.smithi089.stdout:sshd 33497 root rtd unknown /proc/33497/root (readlink: Permission denied) 2024-09-21T03:42:13.415 INFO:tasks.workunit.client.0.smithi089.stdout:sshd 33497 root txt unknown /proc/33497/exe (readlink: Permission denied) 2024-09-21T03:42:13.415 INFO:tasks.workunit.client.0.smithi089.stdout:sshd 33497 root NOFD /proc/33497/fd (opendir: Permission denied) 2024-09-21T03:42:13.415 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/1 39956 root cwd unknown /proc/39956/cwd (readlink: Permission denied) 2024-09-21T03:42:13.416 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/1 39956 root rtd unknown /proc/39956/root (readlink: Permission denied) 2024-09-21T03:42:13.416 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/1 39956 root txt unknown /proc/39956/exe (readlink: Permission denied) 2024-09-21T03:42:13.416 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/1 39956 root NOFD /proc/39956/fd (opendir: Permission denied) 2024-09-21T03:42:13.416 INFO:tasks.workunit.client.0.smithi089.stdout:conmon 46605 root cwd unknown /proc/46605/cwd (readlink: Permission denied) 2024-09-21T03:42:13.416 INFO:tasks.workunit.client.0.smithi089.stdout:conmon 46605 root rtd unknown /proc/46605/root (readlink: Permission denied) 2024-09-21T03:42:13.416 INFO:tasks.workunit.client.0.smithi089.stdout:conmon 46605 root txt unknown /proc/46605/exe (readlink: Permission denied) 2024-09-21T03:42:13.416 INFO:tasks.workunit.client.0.smithi089.stdout:conmon 46605 root NOFD /proc/46605/fd (opendir: Permission denied) 2024-09-21T03:42:13.416 INFO:tasks.workunit.client.0.smithi089.stdout:podman-in 46607 root cwd unknown /proc/46607/cwd (readlink: Permission denied) 2024-09-21T03:42:13.416 INFO:tasks.workunit.client.0.smithi089.stdout:podman-in 46607 root rtd unknown /proc/46607/root (readlink: Permission denied) 2024-09-21T03:42:13.416 INFO:tasks.workunit.client.0.smithi089.stdout:podman-in 46607 root txt unknown /proc/46607/exe (readlink: Permission denied) 2024-09-21T03:42:13.416 INFO:tasks.workunit.client.0.smithi089.stdout:podman-in 46607 root NOFD /proc/46607/fd (opendir: Permission denied) 2024-09-21T03:42:13.416 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 46609 ceph cwd unknown /proc/46609/cwd (readlink: Permission denied) 2024-09-21T03:42:13.416 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 46609 ceph rtd unknown /proc/46609/root (readlink: Permission denied) 2024-09-21T03:42:13.416 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 46609 ceph txt unknown /proc/46609/exe (readlink: Permission denied) 2024-09-21T03:42:13.416 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 46609 ceph NOFD /proc/46609/fd (opendir: Permission denied) 2024-09-21T03:42:13.417 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 46609 46610 log ceph cwd unknown /proc/46609/task/46610/cwd (readlink: Permission denied) 2024-09-21T03:42:13.417 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 46609 46610 log ceph rtd unknown /proc/46609/task/46610/root (readlink: Permission denied) 2024-09-21T03:42:13.417 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 46609 46610 log ceph txt unknown /proc/46609/task/46610/exe (readlink: Permission denied) 2024-09-21T03:42:13.417 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 46609 46610 log ceph NOFD /proc/46609/task/46610/fd (opendir: Permission denied) 2024-09-21T03:42:13.417 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 46609 46612 msgr-work ceph cwd unknown /proc/46609/task/46612/cwd (readlink: Permission denied) 2024-09-21T03:42:13.417 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 46609 46612 msgr-work ceph rtd unknown /proc/46609/task/46612/root (readlink: Permission denied) 2024-09-21T03:42:13.417 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 46609 46612 msgr-work ceph txt unknown /proc/46609/task/46612/exe (readlink: Permission denied) 2024-09-21T03:42:13.417 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 46609 46612 msgr-work ceph NOFD /proc/46609/task/46612/fd (opendir: Permission denied) 2024-09-21T03:42:13.417 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 46609 46613 msgr-work ceph cwd unknown /proc/46609/task/46613/cwd (readlink: Permission denied) 2024-09-21T03:42:13.417 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 46609 46613 msgr-work ceph rtd unknown /proc/46609/task/46613/root (readlink: Permission denied) 2024-09-21T03:42:13.417 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 46609 46613 msgr-work ceph txt unknown /proc/46609/task/46613/exe (readlink: Permission denied) 2024-09-21T03:42:13.417 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 46609 46613 msgr-work ceph NOFD /proc/46609/task/46613/fd (opendir: Permission denied) 2024-09-21T03:42:13.417 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 46609 46614 msgr-work ceph cwd unknown /proc/46609/task/46614/cwd (readlink: Permission denied) 2024-09-21T03:42:13.417 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 46609 46614 msgr-work ceph rtd unknown /proc/46609/task/46614/root (readlink: Permission denied) 2024-09-21T03:42:13.418 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 46609 46614 msgr-work ceph txt unknown /proc/46609/task/46614/exe (readlink: Permission denied) 2024-09-21T03:42:13.418 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 46609 46614 msgr-work ceph NOFD /proc/46609/task/46614/fd (opendir: Permission denied) 2024-09-21T03:42:13.418 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 46609 46618 service ceph cwd unknown /proc/46609/task/46618/cwd (readlink: Permission denied) 2024-09-21T03:42:13.418 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 46609 46618 service ceph rtd unknown /proc/46609/task/46618/root (readlink: Permission denied) 2024-09-21T03:42:13.418 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 46609 46618 service ceph txt unknown /proc/46609/task/46618/exe (readlink: Permission denied) 2024-09-21T03:42:13.418 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 46609 46618 service ceph NOFD /proc/46609/task/46618/fd (opendir: Permission denied) 2024-09-21T03:42:13.418 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 46609 46619 admin_soc ceph cwd unknown /proc/46609/task/46619/cwd (readlink: Permission denied) 2024-09-21T03:42:13.418 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 46609 46619 admin_soc ceph rtd unknown /proc/46609/task/46619/root (readlink: Permission denied) 2024-09-21T03:42:13.418 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 46609 46619 admin_soc ceph txt unknown /proc/46609/task/46619/exe (readlink: Permission denied) 2024-09-21T03:42:13.418 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 46609 46619 admin_soc ceph NOFD /proc/46609/task/46619/fd (opendir: Permission denied) 2024-09-21T03:42:13.418 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 46609 46886 signal_ha ceph cwd unknown /proc/46609/task/46886/cwd (readlink: Permission denied) 2024-09-21T03:42:13.418 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 46609 46886 signal_ha ceph rtd unknown /proc/46609/task/46886/root (readlink: Permission denied) 2024-09-21T03:42:13.418 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 46609 46886 signal_ha ceph txt unknown /proc/46609/task/46886/exe (readlink: Permission denied) 2024-09-21T03:42:13.418 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 46609 46886 signal_ha ceph NOFD /proc/46609/task/46886/fd (opendir: Permission denied) 2024-09-21T03:42:13.419 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 46609 46887 io_contex ceph cwd unknown /proc/46609/task/46887/cwd (readlink: Permission denied) 2024-09-21T03:42:13.419 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 46609 46887 io_contex ceph rtd unknown /proc/46609/task/46887/root (readlink: Permission denied) 2024-09-21T03:42:13.419 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 46609 46887 io_contex ceph txt unknown /proc/46609/task/46887/exe (readlink: Permission denied) 2024-09-21T03:42:13.419 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 46609 46887 io_contex ceph NOFD /proc/46609/task/46887/fd (opendir: Permission denied) 2024-09-21T03:42:13.419 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 46609 46888 io_contex ceph cwd unknown /proc/46609/task/46888/cwd (readlink: Permission denied) 2024-09-21T03:42:13.419 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 46609 46888 io_contex ceph rtd unknown /proc/46609/task/46888/root (readlink: Permission denied) 2024-09-21T03:42:13.419 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 46609 46888 io_contex ceph txt unknown /proc/46609/task/46888/exe (readlink: Permission denied) 2024-09-21T03:42:13.419 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 46609 46888 io_contex ceph NOFD /proc/46609/task/46888/fd (opendir: Permission denied) 2024-09-21T03:42:13.419 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 46609 46907 OpHistory ceph cwd unknown /proc/46609/task/46907/cwd (readlink: Permission denied) 2024-09-21T03:42:13.419 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 46609 46907 OpHistory ceph rtd unknown /proc/46609/task/46907/root (readlink: Permission denied) 2024-09-21T03:42:13.419 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 46609 46907 OpHistory ceph txt unknown /proc/46609/task/46907/exe (readlink: Permission denied) 2024-09-21T03:42:13.419 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 46609 46907 OpHistory ceph NOFD /proc/46609/task/46907/fd (opendir: Permission denied) 2024-09-21T03:42:13.419 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 46609 46908 ceph_time ceph cwd unknown /proc/46609/task/46908/cwd (readlink: Permission denied) 2024-09-21T03:42:13.419 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 46609 46908 ceph_time ceph rtd unknown /proc/46609/task/46908/root (readlink: Permission denied) 2024-09-21T03:42:13.420 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 46609 46908 ceph_time ceph txt unknown /proc/46609/task/46908/exe (readlink: Permission denied) 2024-09-21T03:42:13.420 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 46609 46908 ceph_time ceph NOFD /proc/46609/task/46908/fd (opendir: Permission denied) 2024-09-21T03:42:13.420 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 46609 46928 ceph-osd ceph cwd unknown /proc/46609/task/46928/cwd (readlink: Permission denied) 2024-09-21T03:42:13.420 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 46609 46928 ceph-osd ceph rtd unknown /proc/46609/task/46928/root (readlink: Permission denied) 2024-09-21T03:42:13.420 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 46609 46928 ceph-osd ceph txt unknown /proc/46609/task/46928/exe (readlink: Permission denied) 2024-09-21T03:42:13.420 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 46609 46928 ceph-osd ceph NOFD /proc/46609/task/46928/fd (opendir: Permission denied) 2024-09-21T03:42:13.420 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 46609 46961 ceph-osd ceph cwd unknown /proc/46609/task/46961/cwd (readlink: Permission denied) 2024-09-21T03:42:13.420 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 46609 46961 ceph-osd ceph rtd unknown /proc/46609/task/46961/root (readlink: Permission denied) 2024-09-21T03:42:13.420 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 46609 46961 ceph-osd ceph txt unknown /proc/46609/task/46961/exe (readlink: Permission denied) 2024-09-21T03:42:13.420 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 46609 46961 ceph-osd ceph NOFD /proc/46609/task/46961/fd (opendir: Permission denied) 2024-09-21T03:42:13.420 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 46609 46988 ceph-osd ceph cwd unknown /proc/46609/task/46988/cwd (readlink: Permission denied) 2024-09-21T03:42:13.420 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 46609 46988 ceph-osd ceph rtd unknown /proc/46609/task/46988/root (readlink: Permission denied) 2024-09-21T03:42:13.420 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 46609 46988 ceph-osd ceph txt unknown /proc/46609/task/46988/exe (readlink: Permission denied) 2024-09-21T03:42:13.420 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 46609 46988 ceph-osd ceph NOFD /proc/46609/task/46988/fd (opendir: Permission denied) 2024-09-21T03:42:13.420 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 46609 46996 ceph-osd ceph cwd unknown /proc/46609/task/46996/cwd (readlink: Permission denied) 2024-09-21T03:42:13.421 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 46609 46996 ceph-osd ceph rtd unknown /proc/46609/task/46996/root (readlink: Permission denied) 2024-09-21T03:42:13.421 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 46609 46996 ceph-osd ceph txt unknown /proc/46609/task/46996/exe (readlink: Permission denied) 2024-09-21T03:42:13.421 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 46609 46996 ceph-osd ceph NOFD /proc/46609/task/46996/fd (opendir: Permission denied) 2024-09-21T03:42:13.421 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 46609 47004 ceph-osd ceph cwd unknown /proc/46609/task/47004/cwd (readlink: Permission denied) 2024-09-21T03:42:13.421 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 46609 47004 ceph-osd ceph rtd unknown /proc/46609/task/47004/root (readlink: Permission denied) 2024-09-21T03:42:13.421 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 46609 47004 ceph-osd ceph txt unknown /proc/46609/task/47004/exe (readlink: Permission denied) 2024-09-21T03:42:13.421 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 46609 47004 ceph-osd ceph NOFD /proc/46609/task/47004/fd (opendir: Permission denied) 2024-09-21T03:42:13.421 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 46609 47025 ceph-osd ceph cwd unknown /proc/46609/task/47025/cwd (readlink: Permission denied) 2024-09-21T03:42:13.421 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 46609 47025 ceph-osd ceph rtd unknown /proc/46609/task/47025/root (readlink: Permission denied) 2024-09-21T03:42:13.421 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 46609 47025 ceph-osd ceph txt unknown /proc/46609/task/47025/exe (readlink: Permission denied) 2024-09-21T03:42:13.421 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 46609 47025 ceph-osd ceph NOFD /proc/46609/task/47025/fd (opendir: Permission denied) 2024-09-21T03:42:13.421 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 46609 47029 ceph-osd ceph cwd unknown /proc/46609/task/47029/cwd (readlink: Permission denied) 2024-09-21T03:42:13.421 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 46609 47029 ceph-osd ceph rtd unknown /proc/46609/task/47029/root (readlink: Permission denied) 2024-09-21T03:42:13.421 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 46609 47029 ceph-osd ceph txt unknown /proc/46609/task/47029/exe (readlink: Permission denied) 2024-09-21T03:42:13.422 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 46609 47029 ceph-osd ceph NOFD /proc/46609/task/47029/fd (opendir: Permission denied) 2024-09-21T03:42:13.422 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 46609 47081 ceph-osd ceph cwd unknown /proc/46609/task/47081/cwd (readlink: Permission denied) 2024-09-21T03:42:13.422 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 46609 47081 ceph-osd ceph rtd unknown /proc/46609/task/47081/root (readlink: Permission denied) 2024-09-21T03:42:13.422 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 46609 47081 ceph-osd ceph txt unknown /proc/46609/task/47081/exe (readlink: Permission denied) 2024-09-21T03:42:13.422 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 46609 47081 ceph-osd ceph NOFD /proc/46609/task/47081/fd (opendir: Permission denied) 2024-09-21T03:42:13.422 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 46609 47082 ceph-osd ceph cwd unknown /proc/46609/task/47082/cwd (readlink: Permission denied) 2024-09-21T03:42:13.422 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 46609 47082 ceph-osd ceph rtd unknown /proc/46609/task/47082/root (readlink: Permission denied) 2024-09-21T03:42:13.422 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 46609 47082 ceph-osd ceph txt unknown /proc/46609/task/47082/exe (readlink: Permission denied) 2024-09-21T03:42:13.422 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 46609 47082 ceph-osd ceph NOFD /proc/46609/task/47082/fd (opendir: Permission denied) 2024-09-21T03:42:13.422 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 46609 47083 safe_time ceph cwd unknown /proc/46609/task/47083/cwd (readlink: Permission denied) 2024-09-21T03:42:13.422 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 46609 47083 safe_time ceph rtd unknown /proc/46609/task/47083/root (readlink: Permission denied) 2024-09-21T03:42:13.422 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 46609 47083 safe_time ceph txt unknown /proc/46609/task/47083/exe (readlink: Permission denied) 2024-09-21T03:42:13.422 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 46609 47083 safe_time ceph NOFD /proc/46609/task/47083/fd (opendir: Permission denied) 2024-09-21T03:42:13.423 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 46609 47084 safe_time ceph cwd unknown /proc/46609/task/47084/cwd (readlink: Permission denied) 2024-09-21T03:42:13.423 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 46609 47084 safe_time ceph rtd unknown /proc/46609/task/47084/root (readlink: Permission denied) 2024-09-21T03:42:13.423 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 46609 47084 safe_time ceph txt unknown /proc/46609/task/47084/exe (readlink: Permission denied) 2024-09-21T03:42:13.423 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 46609 47084 safe_time ceph NOFD /proc/46609/task/47084/fd (opendir: Permission denied) 2024-09-21T03:42:13.423 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 46609 47085 safe_time ceph cwd unknown /proc/46609/task/47085/cwd (readlink: Permission denied) 2024-09-21T03:42:13.423 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 46609 47085 safe_time ceph rtd unknown /proc/46609/task/47085/root (readlink: Permission denied) 2024-09-21T03:42:13.423 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 46609 47085 safe_time ceph txt unknown /proc/46609/task/47085/exe (readlink: Permission denied) 2024-09-21T03:42:13.423 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 46609 47085 safe_time ceph NOFD /proc/46609/task/47085/fd (opendir: Permission denied) 2024-09-21T03:42:13.423 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 46609 47086 safe_time ceph cwd unknown /proc/46609/task/47086/cwd (readlink: Permission denied) 2024-09-21T03:42:13.423 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 46609 47086 safe_time ceph rtd unknown /proc/46609/task/47086/root (readlink: Permission denied) 2024-09-21T03:42:13.423 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 46609 47086 safe_time ceph txt unknown /proc/46609/task/47086/exe (readlink: Permission denied) 2024-09-21T03:42:13.423 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 46609 47086 safe_time ceph NOFD /proc/46609/task/47086/fd (opendir: Permission denied) 2024-09-21T03:42:13.423 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 46609 47087 fn_anonym ceph cwd unknown /proc/46609/task/47087/cwd (readlink: Permission denied) 2024-09-21T03:42:13.423 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 46609 47087 fn_anonym ceph rtd unknown /proc/46609/task/47087/root (readlink: Permission denied) 2024-09-21T03:42:13.423 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 46609 47087 fn_anonym ceph txt unknown /proc/46609/task/47087/exe (readlink: Permission denied) 2024-09-21T03:42:13.424 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 46609 47087 fn_anonym ceph NOFD /proc/46609/task/47087/fd (opendir: Permission denied) 2024-09-21T03:42:13.424 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 46609 47168 rocksdb:l ceph cwd unknown /proc/46609/task/47168/cwd (readlink: Permission denied) 2024-09-21T03:42:13.424 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 46609 47168 rocksdb:l ceph rtd unknown /proc/46609/task/47168/root (readlink: Permission denied) 2024-09-21T03:42:13.424 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 46609 47168 rocksdb:l ceph txt unknown /proc/46609/task/47168/exe (readlink: Permission denied) 2024-09-21T03:42:13.424 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 46609 47168 rocksdb:l ceph NOFD /proc/46609/task/47168/fd (opendir: Permission denied) 2024-09-21T03:42:13.424 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 46609 47169 rocksdb:l ceph cwd unknown /proc/46609/task/47169/cwd (readlink: Permission denied) 2024-09-21T03:42:13.424 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 46609 47169 rocksdb:l ceph rtd unknown /proc/46609/task/47169/root (readlink: Permission denied) 2024-09-21T03:42:13.424 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 46609 47169 rocksdb:l ceph txt unknown /proc/46609/task/47169/exe (readlink: Permission denied) 2024-09-21T03:42:13.424 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 46609 47169 rocksdb:l ceph NOFD /proc/46609/task/47169/fd (opendir: Permission denied) 2024-09-21T03:42:13.424 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 46609 47170 rocksdb:l ceph cwd unknown /proc/46609/task/47170/cwd (readlink: Permission denied) 2024-09-21T03:42:13.424 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 46609 47170 rocksdb:l ceph rtd unknown /proc/46609/task/47170/root (readlink: Permission denied) 2024-09-21T03:42:13.424 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 46609 47170 rocksdb:l ceph txt unknown /proc/46609/task/47170/exe (readlink: Permission denied) 2024-09-21T03:42:13.424 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 46609 47170 rocksdb:l ceph NOFD /proc/46609/task/47170/fd (opendir: Permission denied) 2024-09-21T03:42:13.424 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 46609 47171 rocksdb:h ceph cwd unknown /proc/46609/task/47171/cwd (readlink: Permission denied) 2024-09-21T03:42:13.425 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 46609 47171 rocksdb:h ceph rtd unknown /proc/46609/task/47171/root (readlink: Permission denied) 2024-09-21T03:42:13.425 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 46609 47171 rocksdb:h ceph txt unknown /proc/46609/task/47171/exe (readlink: Permission denied) 2024-09-21T03:42:13.425 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 46609 47171 rocksdb:h ceph NOFD /proc/46609/task/47171/fd (opendir: Permission denied) 2024-09-21T03:42:13.425 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 46609 47581 bstore_ai ceph cwd unknown /proc/46609/task/47581/cwd (readlink: Permission denied) 2024-09-21T03:42:13.425 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 46609 47581 bstore_ai ceph rtd unknown /proc/46609/task/47581/root (readlink: Permission denied) 2024-09-21T03:42:13.425 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 46609 47581 bstore_ai ceph txt unknown /proc/46609/task/47581/exe (readlink: Permission denied) 2024-09-21T03:42:13.425 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 46609 47581 bstore_ai ceph NOFD /proc/46609/task/47581/fd (opendir: Permission denied) 2024-09-21T03:42:13.425 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 46609 47771 bstore_ai ceph cwd unknown /proc/46609/task/47771/cwd (readlink: Permission denied) 2024-09-21T03:42:13.425 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 46609 47771 bstore_ai ceph rtd unknown /proc/46609/task/47771/root (readlink: Permission denied) 2024-09-21T03:42:13.425 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 46609 47771 bstore_ai ceph txt unknown /proc/46609/task/47771/exe (readlink: Permission denied) 2024-09-21T03:42:13.425 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 46609 47771 bstore_ai ceph NOFD /proc/46609/task/47771/fd (opendir: Permission denied) 2024-09-21T03:42:13.425 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 46609 47953 ceph-osd ceph cwd unknown /proc/46609/task/47953/cwd (readlink: Permission denied) 2024-09-21T03:42:13.425 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 46609 47953 ceph-osd ceph rtd unknown /proc/46609/task/47953/root (readlink: Permission denied) 2024-09-21T03:42:13.425 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 46609 47953 ceph-osd ceph txt unknown /proc/46609/task/47953/exe (readlink: Permission denied) 2024-09-21T03:42:13.426 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 46609 47953 ceph-osd ceph NOFD /proc/46609/task/47953/fd (opendir: Permission denied) 2024-09-21T03:42:13.426 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 46609 47954 cfin ceph cwd unknown /proc/46609/task/47954/cwd (readlink: Permission denied) 2024-09-21T03:42:13.426 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 46609 47954 cfin ceph rtd unknown /proc/46609/task/47954/root (readlink: Permission denied) 2024-09-21T03:42:13.426 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 46609 47954 cfin ceph txt unknown /proc/46609/task/47954/exe (readlink: Permission denied) 2024-09-21T03:42:13.426 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 46609 47954 cfin ceph NOFD /proc/46609/task/47954/fd (opendir: Permission denied) 2024-09-21T03:42:13.426 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 46609 47955 bstore_kv ceph cwd unknown /proc/46609/task/47955/cwd (readlink: Permission denied) 2024-09-21T03:42:13.426 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 46609 47955 bstore_kv ceph rtd unknown /proc/46609/task/47955/root (readlink: Permission denied) 2024-09-21T03:42:13.426 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 46609 47955 bstore_kv ceph txt unknown /proc/46609/task/47955/exe (readlink: Permission denied) 2024-09-21T03:42:13.426 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 46609 47955 bstore_kv ceph NOFD /proc/46609/task/47955/fd (opendir: Permission denied) 2024-09-21T03:42:13.426 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 46609 47956 bstore_kv ceph cwd unknown /proc/46609/task/47956/cwd (readlink: Permission denied) 2024-09-21T03:42:13.426 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 46609 47956 bstore_kv ceph rtd unknown /proc/46609/task/47956/root (readlink: Permission denied) 2024-09-21T03:42:13.426 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 46609 47956 bstore_kv ceph txt unknown /proc/46609/task/47956/exe (readlink: Permission denied) 2024-09-21T03:42:13.426 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 46609 47956 bstore_kv ceph NOFD /proc/46609/task/47956/fd (opendir: Permission denied) 2024-09-21T03:42:13.426 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 46609 47957 bstore_me ceph cwd unknown /proc/46609/task/47957/cwd (readlink: Permission denied) 2024-09-21T03:42:13.427 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 46609 47957 bstore_me ceph rtd unknown /proc/46609/task/47957/root (readlink: Permission denied) 2024-09-21T03:42:13.427 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 46609 47957 bstore_me ceph txt unknown /proc/46609/task/47957/exe (readlink: Permission denied) 2024-09-21T03:42:13.427 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 46609 47957 bstore_me ceph NOFD /proc/46609/task/47957/fd (opendir: Permission denied) 2024-09-21T03:42:13.427 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 46609 47958 ms_dispat ceph cwd unknown /proc/46609/task/47958/cwd (readlink: Permission denied) 2024-09-21T03:42:13.427 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 46609 47958 ms_dispat ceph rtd unknown /proc/46609/task/47958/root (readlink: Permission denied) 2024-09-21T03:42:13.427 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 46609 47958 ms_dispat ceph txt unknown /proc/46609/task/47958/exe (readlink: Permission denied) 2024-09-21T03:42:13.427 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 46609 47958 ms_dispat ceph NOFD /proc/46609/task/47958/fd (opendir: Permission denied) 2024-09-21T03:42:13.427 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 46609 47959 ms_local ceph cwd unknown /proc/46609/task/47959/cwd (readlink: Permission denied) 2024-09-21T03:42:13.427 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 46609 47959 ms_local ceph rtd unknown /proc/46609/task/47959/root (readlink: Permission denied) 2024-09-21T03:42:13.427 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 46609 47959 ms_local ceph txt unknown /proc/46609/task/47959/exe (readlink: Permission denied) 2024-09-21T03:42:13.427 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 46609 47959 ms_local ceph NOFD /proc/46609/task/47959/fd (opendir: Permission denied) 2024-09-21T03:42:13.427 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 46609 47960 safe_time ceph cwd unknown /proc/46609/task/47960/cwd (readlink: Permission denied) 2024-09-21T03:42:13.427 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 46609 47960 safe_time ceph rtd unknown /proc/46609/task/47960/root (readlink: Permission denied) 2024-09-21T03:42:13.427 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 46609 47960 safe_time ceph txt unknown /proc/46609/task/47960/exe (readlink: Permission denied) 2024-09-21T03:42:13.428 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 46609 47960 safe_time ceph NOFD /proc/46609/task/47960/fd (opendir: Permission denied) 2024-09-21T03:42:13.428 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 46609 47961 safe_time ceph cwd unknown /proc/46609/task/47961/cwd (readlink: Permission denied) 2024-09-21T03:42:13.428 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 46609 47961 safe_time ceph rtd unknown /proc/46609/task/47961/root (readlink: Permission denied) 2024-09-21T03:42:13.428 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 46609 47961 safe_time ceph txt unknown /proc/46609/task/47961/exe (readlink: Permission denied) 2024-09-21T03:42:13.428 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 46609 47961 safe_time ceph NOFD /proc/46609/task/47961/fd (opendir: Permission denied) 2024-09-21T03:42:13.428 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 46609 47962 ms_dispat ceph cwd unknown /proc/46609/task/47962/cwd (readlink: Permission denied) 2024-09-21T03:42:13.428 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 46609 47962 ms_dispat ceph rtd unknown /proc/46609/task/47962/root (readlink: Permission denied) 2024-09-21T03:42:13.428 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 46609 47962 ms_dispat ceph txt unknown /proc/46609/task/47962/exe (readlink: Permission denied) 2024-09-21T03:42:13.428 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 46609 47962 ms_dispat ceph NOFD /proc/46609/task/47962/fd (opendir: Permission denied) 2024-09-21T03:42:13.428 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 46609 47963 ms_local ceph cwd unknown /proc/46609/task/47963/cwd (readlink: Permission denied) 2024-09-21T03:42:13.428 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 46609 47963 ms_local ceph rtd unknown /proc/46609/task/47963/root (readlink: Permission denied) 2024-09-21T03:42:13.428 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 46609 47963 ms_local ceph txt unknown /proc/46609/task/47963/exe (readlink: Permission denied) 2024-09-21T03:42:13.428 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 46609 47963 ms_local ceph NOFD /proc/46609/task/47963/fd (opendir: Permission denied) 2024-09-21T03:42:13.428 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 46609 47964 ms_dispat ceph cwd unknown /proc/46609/task/47964/cwd (readlink: Permission denied) 2024-09-21T03:42:13.429 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 46609 47964 ms_dispat ceph rtd unknown /proc/46609/task/47964/root (readlink: Permission denied) 2024-09-21T03:42:13.429 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 46609 47964 ms_dispat ceph txt unknown /proc/46609/task/47964/exe (readlink: Permission denied) 2024-09-21T03:42:13.429 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 46609 47964 ms_dispat ceph NOFD /proc/46609/task/47964/fd (opendir: Permission denied) 2024-09-21T03:42:13.429 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 46609 47965 ms_local ceph cwd unknown /proc/46609/task/47965/cwd (readlink: Permission denied) 2024-09-21T03:42:13.429 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 46609 47965 ms_local ceph rtd unknown /proc/46609/task/47965/root (readlink: Permission denied) 2024-09-21T03:42:13.429 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 46609 47965 ms_local ceph txt unknown /proc/46609/task/47965/exe (readlink: Permission denied) 2024-09-21T03:42:13.429 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 46609 47965 ms_local ceph NOFD /proc/46609/task/47965/fd (opendir: Permission denied) 2024-09-21T03:42:13.429 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 46609 47966 ms_dispat ceph cwd unknown /proc/46609/task/47966/cwd (readlink: Permission denied) 2024-09-21T03:42:13.429 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 46609 47966 ms_dispat ceph rtd unknown /proc/46609/task/47966/root (readlink: Permission denied) 2024-09-21T03:42:13.429 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 46609 47966 ms_dispat ceph txt unknown /proc/46609/task/47966/exe (readlink: Permission denied) 2024-09-21T03:42:13.429 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 46609 47966 ms_dispat ceph NOFD /proc/46609/task/47966/fd (opendir: Permission denied) 2024-09-21T03:42:13.429 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 46609 47967 ms_local ceph cwd unknown /proc/46609/task/47967/cwd (readlink: Permission denied) 2024-09-21T03:42:13.429 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 46609 47967 ms_local ceph rtd unknown /proc/46609/task/47967/root (readlink: Permission denied) 2024-09-21T03:42:13.429 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 46609 47967 ms_local ceph txt unknown /proc/46609/task/47967/exe (readlink: Permission denied) 2024-09-21T03:42:13.429 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 46609 47967 ms_local ceph NOFD /proc/46609/task/47967/fd (opendir: Permission denied) 2024-09-21T03:42:13.430 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 46609 47968 ms_dispat ceph cwd unknown /proc/46609/task/47968/cwd (readlink: Permission denied) 2024-09-21T03:42:13.430 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 46609 47968 ms_dispat ceph rtd unknown /proc/46609/task/47968/root (readlink: Permission denied) 2024-09-21T03:42:13.430 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 46609 47968 ms_dispat ceph txt unknown /proc/46609/task/47968/exe (readlink: Permission denied) 2024-09-21T03:42:13.430 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 46609 47968 ms_dispat ceph NOFD /proc/46609/task/47968/fd (opendir: Permission denied) 2024-09-21T03:42:13.430 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 46609 47969 ms_local ceph cwd unknown /proc/46609/task/47969/cwd (readlink: Permission denied) 2024-09-21T03:42:13.430 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 46609 47969 ms_local ceph rtd unknown /proc/46609/task/47969/root (readlink: Permission denied) 2024-09-21T03:42:13.430 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 46609 47969 ms_local ceph txt unknown /proc/46609/task/47969/exe (readlink: Permission denied) 2024-09-21T03:42:13.430 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 46609 47969 ms_local ceph NOFD /proc/46609/task/47969/fd (opendir: Permission denied) 2024-09-21T03:42:13.430 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 46609 47970 ms_dispat ceph cwd unknown /proc/46609/task/47970/cwd (readlink: Permission denied) 2024-09-21T03:42:13.430 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 46609 47970 ms_dispat ceph rtd unknown /proc/46609/task/47970/root (readlink: Permission denied) 2024-09-21T03:42:13.430 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 46609 47970 ms_dispat ceph txt unknown /proc/46609/task/47970/exe (readlink: Permission denied) 2024-09-21T03:42:13.430 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 46609 47970 ms_dispat ceph NOFD /proc/46609/task/47970/fd (opendir: Permission denied) 2024-09-21T03:42:13.430 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 46609 47971 ms_local ceph cwd unknown /proc/46609/task/47971/cwd (readlink: Permission denied) 2024-09-21T03:42:13.430 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 46609 47971 ms_local ceph rtd unknown /proc/46609/task/47971/root (readlink: Permission denied) 2024-09-21T03:42:13.431 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 46609 47971 ms_local ceph txt unknown /proc/46609/task/47971/exe (readlink: Permission denied) 2024-09-21T03:42:13.431 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 46609 47971 ms_local ceph NOFD /proc/46609/task/47971/fd (opendir: Permission denied) 2024-09-21T03:42:13.431 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 46609 47972 ms_dispat ceph cwd unknown /proc/46609/task/47972/cwd (readlink: Permission denied) 2024-09-21T03:42:13.431 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 46609 47972 ms_dispat ceph rtd unknown /proc/46609/task/47972/root (readlink: Permission denied) 2024-09-21T03:42:13.431 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 46609 47972 ms_dispat ceph txt unknown /proc/46609/task/47972/exe (readlink: Permission denied) 2024-09-21T03:42:13.431 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 46609 47972 ms_dispat ceph NOFD /proc/46609/task/47972/fd (opendir: Permission denied) 2024-09-21T03:42:13.431 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 46609 47973 ms_local ceph cwd unknown /proc/46609/task/47973/cwd (readlink: Permission denied) 2024-09-21T03:42:13.431 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 46609 47973 ms_local ceph rtd unknown /proc/46609/task/47973/root (readlink: Permission denied) 2024-09-21T03:42:13.431 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 46609 47973 ms_local ceph txt unknown /proc/46609/task/47973/exe (readlink: Permission denied) 2024-09-21T03:42:13.431 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 46609 47973 ms_local ceph NOFD /proc/46609/task/47973/fd (opendir: Permission denied) 2024-09-21T03:42:13.431 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 46609 47974 fn_anonym ceph cwd unknown /proc/46609/task/47974/cwd (readlink: Permission denied) 2024-09-21T03:42:13.431 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 46609 47974 fn_anonym ceph rtd unknown /proc/46609/task/47974/root (readlink: Permission denied) 2024-09-21T03:42:13.431 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 46609 47974 fn_anonym ceph txt unknown /proc/46609/task/47974/exe (readlink: Permission denied) 2024-09-21T03:42:13.431 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 46609 47974 fn_anonym ceph NOFD /proc/46609/task/47974/fd (opendir: Permission denied) 2024-09-21T03:42:13.431 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 46609 47975 finisher ceph cwd unknown /proc/46609/task/47975/cwd (readlink: Permission denied) 2024-09-21T03:42:13.432 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 46609 47975 finisher ceph rtd unknown /proc/46609/task/47975/root (readlink: Permission denied) 2024-09-21T03:42:13.432 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 46609 47975 finisher ceph txt unknown /proc/46609/task/47975/exe (readlink: Permission denied) 2024-09-21T03:42:13.432 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 46609 47975 finisher ceph NOFD /proc/46609/task/47975/fd (opendir: Permission denied) 2024-09-21T03:42:13.432 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 46609 47976 safe_time ceph cwd unknown /proc/46609/task/47976/cwd (readlink: Permission denied) 2024-09-21T03:42:13.432 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 46609 47976 safe_time ceph rtd unknown /proc/46609/task/47976/root (readlink: Permission denied) 2024-09-21T03:42:13.432 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 46609 47976 safe_time ceph txt unknown /proc/46609/task/47976/exe (readlink: Permission denied) 2024-09-21T03:42:13.432 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 46609 47976 safe_time ceph NOFD /proc/46609/task/47976/fd (opendir: Permission denied) 2024-09-21T03:42:13.432 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 46609 47977 safe_time ceph cwd unknown /proc/46609/task/47977/cwd (readlink: Permission denied) 2024-09-21T03:42:13.432 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 46609 47977 safe_time ceph rtd unknown /proc/46609/task/47977/root (readlink: Permission denied) 2024-09-21T03:42:13.432 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 46609 47977 safe_time ceph txt unknown /proc/46609/task/47977/exe (readlink: Permission denied) 2024-09-21T03:42:13.432 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 46609 47977 safe_time ceph NOFD /proc/46609/task/47977/fd (opendir: Permission denied) 2024-09-21T03:42:13.432 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 46609 47978 ceph-osd ceph cwd unknown /proc/46609/task/47978/cwd (readlink: Permission denied) 2024-09-21T03:42:13.432 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 46609 47978 ceph-osd ceph rtd unknown /proc/46609/task/47978/root (readlink: Permission denied) 2024-09-21T03:42:13.433 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 46609 47978 ceph-osd ceph txt unknown /proc/46609/task/47978/exe (readlink: Permission denied) 2024-09-21T03:42:13.433 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 46609 47978 ceph-osd ceph NOFD /proc/46609/task/47978/fd (opendir: Permission denied) 2024-09-21T03:42:13.433 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 46609 47979 osd_srv_a ceph cwd unknown /proc/46609/task/47979/cwd (readlink: Permission denied) 2024-09-21T03:42:13.433 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 46609 47979 osd_srv_a ceph rtd unknown /proc/46609/task/47979/root (readlink: Permission denied) 2024-09-21T03:42:13.433 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 46609 47979 osd_srv_a ceph txt unknown /proc/46609/task/47979/exe (readlink: Permission denied) 2024-09-21T03:42:13.433 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 46609 47979 osd_srv_a ceph NOFD /proc/46609/task/47979/fd (opendir: Permission denied) 2024-09-21T03:42:13.433 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 46609 47980 tp_osd_tp ceph cwd unknown /proc/46609/task/47980/cwd (readlink: Permission denied) 2024-09-21T03:42:13.433 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 46609 47980 tp_osd_tp ceph rtd unknown /proc/46609/task/47980/root (readlink: Permission denied) 2024-09-21T03:42:13.433 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 46609 47980 tp_osd_tp ceph txt unknown /proc/46609/task/47980/exe (readlink: Permission denied) 2024-09-21T03:42:13.433 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 46609 47980 tp_osd_tp ceph NOFD /proc/46609/task/47980/fd (opendir: Permission denied) 2024-09-21T03:42:13.433 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 46609 47981 tp_osd_tp ceph cwd unknown /proc/46609/task/47981/cwd (readlink: Permission denied) 2024-09-21T03:42:13.433 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 46609 47981 tp_osd_tp ceph rtd unknown /proc/46609/task/47981/root (readlink: Permission denied) 2024-09-21T03:42:13.433 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 46609 47981 tp_osd_tp ceph txt unknown /proc/46609/task/47981/exe (readlink: Permission denied) 2024-09-21T03:42:13.433 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 46609 47981 tp_osd_tp ceph NOFD /proc/46609/task/47981/fd (opendir: Permission denied) 2024-09-21T03:42:13.433 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 46609 47982 tp_osd_tp ceph cwd unknown /proc/46609/task/47982/cwd (readlink: Permission denied) 2024-09-21T03:42:13.434 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 46609 47982 tp_osd_tp ceph rtd unknown /proc/46609/task/47982/root (readlink: Permission denied) 2024-09-21T03:42:13.434 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 46609 47982 tp_osd_tp ceph txt unknown /proc/46609/task/47982/exe (readlink: Permission denied) 2024-09-21T03:42:13.434 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 46609 47982 tp_osd_tp ceph NOFD /proc/46609/task/47982/fd (opendir: Permission denied) 2024-09-21T03:42:13.434 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 46609 47983 tp_osd_tp ceph cwd unknown /proc/46609/task/47983/cwd (readlink: Permission denied) 2024-09-21T03:42:13.434 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 46609 47983 tp_osd_tp ceph rtd unknown /proc/46609/task/47983/root (readlink: Permission denied) 2024-09-21T03:42:13.434 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 46609 47983 tp_osd_tp ceph txt unknown /proc/46609/task/47983/exe (readlink: Permission denied) 2024-09-21T03:42:13.434 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 46609 47983 tp_osd_tp ceph NOFD /proc/46609/task/47983/fd (opendir: Permission denied) 2024-09-21T03:42:13.434 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 46609 47984 tp_osd_tp ceph cwd unknown /proc/46609/task/47984/cwd (readlink: Permission denied) 2024-09-21T03:42:13.434 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 46609 47984 tp_osd_tp ceph rtd unknown /proc/46609/task/47984/root (readlink: Permission denied) 2024-09-21T03:42:13.434 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 46609 47984 tp_osd_tp ceph txt unknown /proc/46609/task/47984/exe (readlink: Permission denied) 2024-09-21T03:42:13.434 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 46609 47984 tp_osd_tp ceph NOFD /proc/46609/task/47984/fd (opendir: Permission denied) 2024-09-21T03:42:13.434 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 46609 47985 tp_osd_tp ceph cwd unknown /proc/46609/task/47985/cwd (readlink: Permission denied) 2024-09-21T03:42:13.434 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 46609 47985 tp_osd_tp ceph rtd unknown /proc/46609/task/47985/root (readlink: Permission denied) 2024-09-21T03:42:13.434 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 46609 47985 tp_osd_tp ceph txt unknown /proc/46609/task/47985/exe (readlink: Permission denied) 2024-09-21T03:42:13.434 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 46609 47985 tp_osd_tp ceph NOFD /proc/46609/task/47985/fd (opendir: Permission denied) 2024-09-21T03:42:13.435 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 46609 47986 tp_osd_tp ceph cwd unknown /proc/46609/task/47986/cwd (readlink: Permission denied) 2024-09-21T03:42:13.435 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 46609 47986 tp_osd_tp ceph rtd unknown /proc/46609/task/47986/root (readlink: Permission denied) 2024-09-21T03:42:13.435 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 46609 47986 tp_osd_tp ceph txt unknown /proc/46609/task/47986/exe (readlink: Permission denied) 2024-09-21T03:42:13.435 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 46609 47986 tp_osd_tp ceph NOFD /proc/46609/task/47986/fd (opendir: Permission denied) 2024-09-21T03:42:13.435 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 46609 47987 tp_osd_tp ceph cwd unknown /proc/46609/task/47987/cwd (readlink: Permission denied) 2024-09-21T03:42:13.435 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 46609 47987 tp_osd_tp ceph rtd unknown /proc/46609/task/47987/root (readlink: Permission denied) 2024-09-21T03:42:13.435 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 46609 47987 tp_osd_tp ceph txt unknown /proc/46609/task/47987/exe (readlink: Permission denied) 2024-09-21T03:42:13.435 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 46609 47987 tp_osd_tp ceph NOFD /proc/46609/task/47987/fd (opendir: Permission denied) 2024-09-21T03:42:13.435 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 46609 47988 tp_osd_tp ceph cwd unknown /proc/46609/task/47988/cwd (readlink: Permission denied) 2024-09-21T03:42:13.435 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 46609 47988 tp_osd_tp ceph rtd unknown /proc/46609/task/47988/root (readlink: Permission denied) 2024-09-21T03:42:13.435 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 46609 47988 tp_osd_tp ceph txt unknown /proc/46609/task/47988/exe (readlink: Permission denied) 2024-09-21T03:42:13.435 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 46609 47988 tp_osd_tp ceph NOFD /proc/46609/task/47988/fd (opendir: Permission denied) 2024-09-21T03:42:13.435 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 46609 47989 tp_osd_tp ceph cwd unknown /proc/46609/task/47989/cwd (readlink: Permission denied) 2024-09-21T03:42:13.435 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 46609 47989 tp_osd_tp ceph rtd unknown /proc/46609/task/47989/root (readlink: Permission denied) 2024-09-21T03:42:13.436 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 46609 47989 tp_osd_tp ceph txt unknown /proc/46609/task/47989/exe (readlink: Permission denied) 2024-09-21T03:42:13.436 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 46609 47989 tp_osd_tp ceph NOFD /proc/46609/task/47989/fd (opendir: Permission denied) 2024-09-21T03:42:13.436 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 46609 47990 tp_osd_tp ceph cwd unknown /proc/46609/task/47990/cwd (readlink: Permission denied) 2024-09-21T03:42:13.436 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 46609 47990 tp_osd_tp ceph rtd unknown /proc/46609/task/47990/root (readlink: Permission denied) 2024-09-21T03:42:13.436 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 46609 47990 tp_osd_tp ceph txt unknown /proc/46609/task/47990/exe (readlink: Permission denied) 2024-09-21T03:42:13.436 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 46609 47990 tp_osd_tp ceph NOFD /proc/46609/task/47990/fd (opendir: Permission denied) 2024-09-21T03:42:13.436 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 46609 47991 tp_osd_tp ceph cwd unknown /proc/46609/task/47991/cwd (readlink: Permission denied) 2024-09-21T03:42:13.436 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 46609 47991 tp_osd_tp ceph rtd unknown /proc/46609/task/47991/root (readlink: Permission denied) 2024-09-21T03:42:13.436 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 46609 47991 tp_osd_tp ceph txt unknown /proc/46609/task/47991/exe (readlink: Permission denied) 2024-09-21T03:42:13.436 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 46609 47991 tp_osd_tp ceph NOFD /proc/46609/task/47991/fd (opendir: Permission denied) 2024-09-21T03:42:13.436 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 46609 47992 tp_osd_tp ceph cwd unknown /proc/46609/task/47992/cwd (readlink: Permission denied) 2024-09-21T03:42:13.436 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 46609 47992 tp_osd_tp ceph rtd unknown /proc/46609/task/47992/root (readlink: Permission denied) 2024-09-21T03:42:13.436 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 46609 47992 tp_osd_tp ceph txt unknown /proc/46609/task/47992/exe (readlink: Permission denied) 2024-09-21T03:42:13.436 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 46609 47992 tp_osd_tp ceph NOFD /proc/46609/task/47992/fd (opendir: Permission denied) 2024-09-21T03:42:13.436 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 46609 47993 tp_osd_tp ceph cwd unknown /proc/46609/task/47993/cwd (readlink: Permission denied) 2024-09-21T03:42:13.437 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 46609 47993 tp_osd_tp ceph rtd unknown /proc/46609/task/47993/root (readlink: Permission denied) 2024-09-21T03:42:13.437 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 46609 47993 tp_osd_tp ceph txt unknown /proc/46609/task/47993/exe (readlink: Permission denied) 2024-09-21T03:42:13.437 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 46609 47993 tp_osd_tp ceph NOFD /proc/46609/task/47993/fd (opendir: Permission denied) 2024-09-21T03:42:13.437 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 46609 47994 tp_osd_tp ceph cwd unknown /proc/46609/task/47994/cwd (readlink: Permission denied) 2024-09-21T03:42:13.437 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 46609 47994 tp_osd_tp ceph rtd unknown /proc/46609/task/47994/root (readlink: Permission denied) 2024-09-21T03:42:13.437 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 46609 47994 tp_osd_tp ceph txt unknown /proc/46609/task/47994/exe (readlink: Permission denied) 2024-09-21T03:42:13.437 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 46609 47994 tp_osd_tp ceph NOFD /proc/46609/task/47994/fd (opendir: Permission denied) 2024-09-21T03:42:13.437 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 46609 47995 tp_osd_tp ceph cwd unknown /proc/46609/task/47995/cwd (readlink: Permission denied) 2024-09-21T03:42:13.437 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 46609 47995 tp_osd_tp ceph rtd unknown /proc/46609/task/47995/root (readlink: Permission denied) 2024-09-21T03:42:13.437 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 46609 47995 tp_osd_tp ceph txt unknown /proc/46609/task/47995/exe (readlink: Permission denied) 2024-09-21T03:42:13.437 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 46609 47995 tp_osd_tp ceph NOFD /proc/46609/task/47995/fd (opendir: Permission denied) 2024-09-21T03:42:13.437 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 46609 47996 osd_srv_h ceph cwd unknown /proc/46609/task/47996/cwd (readlink: Permission denied) 2024-09-21T03:42:13.437 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 46609 47996 osd_srv_h ceph rtd unknown /proc/46609/task/47996/root (readlink: Permission denied) 2024-09-21T03:42:13.437 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 46609 47996 osd_srv_h ceph txt unknown /proc/46609/task/47996/exe (readlink: Permission denied) 2024-09-21T03:42:13.437 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 46609 47996 osd_srv_h ceph NOFD /proc/46609/task/47996/fd (opendir: Permission denied) 2024-09-21T03:42:13.438 INFO:tasks.workunit.client.0.smithi089.stdout:sudo 47106 root cwd unknown /proc/47106/cwd (readlink: Permission denied) 2024-09-21T03:42:13.438 INFO:tasks.workunit.client.0.smithi089.stdout:sudo 47106 root rtd unknown /proc/47106/root (readlink: Permission denied) 2024-09-21T03:42:13.438 INFO:tasks.workunit.client.0.smithi089.stdout:sudo 47106 root txt unknown /proc/47106/exe (readlink: Permission denied) 2024-09-21T03:42:13.438 INFO:tasks.workunit.client.0.smithi089.stdout:sudo 47106 root NOFD /proc/47106/fd (opendir: Permission denied) 2024-09-21T03:42:13.438 INFO:tasks.workunit.client.0.smithi089.stdout:journalct 47123 root cwd unknown /proc/47123/cwd (readlink: Permission denied) 2024-09-21T03:42:13.438 INFO:tasks.workunit.client.0.smithi089.stdout:journalct 47123 root rtd unknown /proc/47123/root (readlink: Permission denied) 2024-09-21T03:42:13.438 INFO:tasks.workunit.client.0.smithi089.stdout:journalct 47123 root txt unknown /proc/47123/exe (readlink: Permission denied) 2024-09-21T03:42:13.438 INFO:tasks.workunit.client.0.smithi089.stdout:journalct 47123 root NOFD /proc/47123/fd (opendir: Permission denied) 2024-09-21T03:42:13.438 INFO:tasks.workunit.client.0.smithi089.stdout:conmon 52765 root cwd unknown /proc/52765/cwd (readlink: Permission denied) 2024-09-21T03:42:13.438 INFO:tasks.workunit.client.0.smithi089.stdout:conmon 52765 root rtd unknown /proc/52765/root (readlink: Permission denied) 2024-09-21T03:42:13.438 INFO:tasks.workunit.client.0.smithi089.stdout:conmon 52765 root txt unknown /proc/52765/exe (readlink: Permission denied) 2024-09-21T03:42:13.438 INFO:tasks.workunit.client.0.smithi089.stdout:conmon 52765 root NOFD /proc/52765/fd (opendir: Permission denied) 2024-09-21T03:42:13.438 INFO:tasks.workunit.client.0.smithi089.stdout:podman-in 52767 root cwd unknown /proc/52767/cwd (readlink: Permission denied) 2024-09-21T03:42:13.438 INFO:tasks.workunit.client.0.smithi089.stdout:podman-in 52767 root rtd unknown /proc/52767/root (readlink: Permission denied) 2024-09-21T03:42:13.439 INFO:tasks.workunit.client.0.smithi089.stdout:podman-in 52767 root txt unknown /proc/52767/exe (readlink: Permission denied) 2024-09-21T03:42:13.439 INFO:tasks.workunit.client.0.smithi089.stdout:podman-in 52767 root NOFD /proc/52767/fd (opendir: Permission denied) 2024-09-21T03:42:13.439 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 52769 ceph cwd unknown /proc/52769/cwd (readlink: Permission denied) 2024-09-21T03:42:13.439 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 52769 ceph rtd unknown /proc/52769/root (readlink: Permission denied) 2024-09-21T03:42:13.439 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 52769 ceph txt unknown /proc/52769/exe (readlink: Permission denied) 2024-09-21T03:42:13.439 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 52769 ceph NOFD /proc/52769/fd (opendir: Permission denied) 2024-09-21T03:42:13.439 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 52769 52770 log ceph cwd unknown /proc/52769/task/52770/cwd (readlink: Permission denied) 2024-09-21T03:42:13.439 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 52769 52770 log ceph rtd unknown /proc/52769/task/52770/root (readlink: Permission denied) 2024-09-21T03:42:13.439 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 52769 52770 log ceph txt unknown /proc/52769/task/52770/exe (readlink: Permission denied) 2024-09-21T03:42:13.439 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 52769 52770 log ceph NOFD /proc/52769/task/52770/fd (opendir: Permission denied) 2024-09-21T03:42:13.439 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 52769 52772 msgr-work ceph cwd unknown /proc/52769/task/52772/cwd (readlink: Permission denied) 2024-09-21T03:42:13.439 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 52769 52772 msgr-work ceph rtd unknown /proc/52769/task/52772/root (readlink: Permission denied) 2024-09-21T03:42:13.439 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 52769 52772 msgr-work ceph txt unknown /proc/52769/task/52772/exe (readlink: Permission denied) 2024-09-21T03:42:13.439 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 52769 52772 msgr-work ceph NOFD /proc/52769/task/52772/fd (opendir: Permission denied) 2024-09-21T03:42:13.440 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 52769 52773 msgr-work ceph cwd unknown /proc/52769/task/52773/cwd (readlink: Permission denied) 2024-09-21T03:42:13.440 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 52769 52773 msgr-work ceph rtd unknown /proc/52769/task/52773/root (readlink: Permission denied) 2024-09-21T03:42:13.440 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 52769 52773 msgr-work ceph txt unknown /proc/52769/task/52773/exe (readlink: Permission denied) 2024-09-21T03:42:13.440 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 52769 52773 msgr-work ceph NOFD /proc/52769/task/52773/fd (opendir: Permission denied) 2024-09-21T03:42:13.440 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 52769 52774 msgr-work ceph cwd unknown /proc/52769/task/52774/cwd (readlink: Permission denied) 2024-09-21T03:42:13.440 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 52769 52774 msgr-work ceph rtd unknown /proc/52769/task/52774/root (readlink: Permission denied) 2024-09-21T03:42:13.440 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 52769 52774 msgr-work ceph txt unknown /proc/52769/task/52774/exe (readlink: Permission denied) 2024-09-21T03:42:13.440 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 52769 52774 msgr-work ceph NOFD /proc/52769/task/52774/fd (opendir: Permission denied) 2024-09-21T03:42:13.440 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 52769 52778 service ceph cwd unknown /proc/52769/task/52778/cwd (readlink: Permission denied) 2024-09-21T03:42:13.440 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 52769 52778 service ceph rtd unknown /proc/52769/task/52778/root (readlink: Permission denied) 2024-09-21T03:42:13.440 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 52769 52778 service ceph txt unknown /proc/52769/task/52778/exe (readlink: Permission denied) 2024-09-21T03:42:13.440 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 52769 52778 service ceph NOFD /proc/52769/task/52778/fd (opendir: Permission denied) 2024-09-21T03:42:13.440 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 52769 52779 admin_soc ceph cwd unknown /proc/52769/task/52779/cwd (readlink: Permission denied) 2024-09-21T03:42:13.440 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 52769 52779 admin_soc ceph rtd unknown /proc/52769/task/52779/root (readlink: Permission denied) 2024-09-21T03:42:13.441 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 52769 52779 admin_soc ceph txt unknown /proc/52769/task/52779/exe (readlink: Permission denied) 2024-09-21T03:42:13.441 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 52769 52779 admin_soc ceph NOFD /proc/52769/task/52779/fd (opendir: Permission denied) 2024-09-21T03:42:13.441 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 52769 53050 signal_ha ceph cwd unknown /proc/52769/task/53050/cwd (readlink: Permission denied) 2024-09-21T03:42:13.441 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 52769 53050 signal_ha ceph rtd unknown /proc/52769/task/53050/root (readlink: Permission denied) 2024-09-21T03:42:13.441 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 52769 53050 signal_ha ceph txt unknown /proc/52769/task/53050/exe (readlink: Permission denied) 2024-09-21T03:42:13.441 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 52769 53050 signal_ha ceph NOFD /proc/52769/task/53050/fd (opendir: Permission denied) 2024-09-21T03:42:13.441 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 52769 53051 io_contex ceph cwd unknown /proc/52769/task/53051/cwd (readlink: Permission denied) 2024-09-21T03:42:13.441 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 52769 53051 io_contex ceph rtd unknown /proc/52769/task/53051/root (readlink: Permission denied) 2024-09-21T03:42:13.441 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 52769 53051 io_contex ceph txt unknown /proc/52769/task/53051/exe (readlink: Permission denied) 2024-09-21T03:42:13.441 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 52769 53051 io_contex ceph NOFD /proc/52769/task/53051/fd (opendir: Permission denied) 2024-09-21T03:42:13.441 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 52769 53052 io_contex ceph cwd unknown /proc/52769/task/53052/cwd (readlink: Permission denied) 2024-09-21T03:42:13.441 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 52769 53052 io_contex ceph rtd unknown /proc/52769/task/53052/root (readlink: Permission denied) 2024-09-21T03:42:13.441 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 52769 53052 io_contex ceph txt unknown /proc/52769/task/53052/exe (readlink: Permission denied) 2024-09-21T03:42:13.441 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 52769 53052 io_contex ceph NOFD /proc/52769/task/53052/fd (opendir: Permission denied) 2024-09-21T03:42:13.442 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 52769 53071 OpHistory ceph cwd unknown /proc/52769/task/53071/cwd (readlink: Permission denied) 2024-09-21T03:42:13.442 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 52769 53071 OpHistory ceph rtd unknown /proc/52769/task/53071/root (readlink: Permission denied) 2024-09-21T03:42:13.442 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 52769 53071 OpHistory ceph txt unknown /proc/52769/task/53071/exe (readlink: Permission denied) 2024-09-21T03:42:13.442 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 52769 53071 OpHistory ceph NOFD /proc/52769/task/53071/fd (opendir: Permission denied) 2024-09-21T03:42:13.442 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 52769 53072 ceph_time ceph cwd unknown /proc/52769/task/53072/cwd (readlink: Permission denied) 2024-09-21T03:42:13.442 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 52769 53072 ceph_time ceph rtd unknown /proc/52769/task/53072/root (readlink: Permission denied) 2024-09-21T03:42:13.442 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 52769 53072 ceph_time ceph txt unknown /proc/52769/task/53072/exe (readlink: Permission denied) 2024-09-21T03:42:13.442 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 52769 53072 ceph_time ceph NOFD /proc/52769/task/53072/fd (opendir: Permission denied) 2024-09-21T03:42:13.442 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 52769 53256 ceph-osd ceph cwd unknown /proc/52769/task/53256/cwd (readlink: Permission denied) 2024-09-21T03:42:13.442 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 52769 53256 ceph-osd ceph rtd unknown /proc/52769/task/53256/root (readlink: Permission denied) 2024-09-21T03:42:13.442 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 52769 53256 ceph-osd ceph txt unknown /proc/52769/task/53256/exe (readlink: Permission denied) 2024-09-21T03:42:13.442 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 52769 53256 ceph-osd ceph NOFD /proc/52769/task/53256/fd (opendir: Permission denied) 2024-09-21T03:42:13.442 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 52769 53257 safe_time ceph cwd unknown /proc/52769/task/53257/cwd (readlink: Permission denied) 2024-09-21T03:42:13.442 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 52769 53257 safe_time ceph rtd unknown /proc/52769/task/53257/root (readlink: Permission denied) 2024-09-21T03:42:13.443 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 52769 53257 safe_time ceph txt unknown /proc/52769/task/53257/exe (readlink: Permission denied) 2024-09-21T03:42:13.443 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 52769 53257 safe_time ceph NOFD /proc/52769/task/53257/fd (opendir: Permission denied) 2024-09-21T03:42:13.443 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 52769 53258 safe_time ceph cwd unknown /proc/52769/task/53258/cwd (readlink: Permission denied) 2024-09-21T03:42:13.443 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 52769 53258 safe_time ceph rtd unknown /proc/52769/task/53258/root (readlink: Permission denied) 2024-09-21T03:42:13.443 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 52769 53258 safe_time ceph txt unknown /proc/52769/task/53258/exe (readlink: Permission denied) 2024-09-21T03:42:13.443 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 52769 53258 safe_time ceph NOFD /proc/52769/task/53258/fd (opendir: Permission denied) 2024-09-21T03:42:13.443 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 52769 53259 safe_time ceph cwd unknown /proc/52769/task/53259/cwd (readlink: Permission denied) 2024-09-21T03:42:13.443 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 52769 53259 safe_time ceph rtd unknown /proc/52769/task/53259/root (readlink: Permission denied) 2024-09-21T03:42:13.443 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 52769 53259 safe_time ceph txt unknown /proc/52769/task/53259/exe (readlink: Permission denied) 2024-09-21T03:42:13.443 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 52769 53259 safe_time ceph NOFD /proc/52769/task/53259/fd (opendir: Permission denied) 2024-09-21T03:42:13.443 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 52769 53260 safe_time ceph cwd unknown /proc/52769/task/53260/cwd (readlink: Permission denied) 2024-09-21T03:42:13.443 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 52769 53260 safe_time ceph rtd unknown /proc/52769/task/53260/root (readlink: Permission denied) 2024-09-21T03:42:13.443 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 52769 53260 safe_time ceph txt unknown /proc/52769/task/53260/exe (readlink: Permission denied) 2024-09-21T03:42:13.443 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 52769 53260 safe_time ceph NOFD /proc/52769/task/53260/fd (opendir: Permission denied) 2024-09-21T03:42:13.443 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 52769 53261 fn_anonym ceph cwd unknown /proc/52769/task/53261/cwd (readlink: Permission denied) 2024-09-21T03:42:13.444 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 52769 53261 fn_anonym ceph rtd unknown /proc/52769/task/53261/root (readlink: Permission denied) 2024-09-21T03:42:13.444 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 52769 53261 fn_anonym ceph txt unknown /proc/52769/task/53261/exe (readlink: Permission denied) 2024-09-21T03:42:13.444 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 52769 53261 fn_anonym ceph NOFD /proc/52769/task/53261/fd (opendir: Permission denied) 2024-09-21T03:42:13.444 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 52769 53351 rocksdb:l ceph cwd unknown /proc/52769/task/53351/cwd (readlink: Permission denied) 2024-09-21T03:42:13.444 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 52769 53351 rocksdb:l ceph rtd unknown /proc/52769/task/53351/root (readlink: Permission denied) 2024-09-21T03:42:13.444 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 52769 53351 rocksdb:l ceph txt unknown /proc/52769/task/53351/exe (readlink: Permission denied) 2024-09-21T03:42:13.444 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 52769 53351 rocksdb:l ceph NOFD /proc/52769/task/53351/fd (opendir: Permission denied) 2024-09-21T03:42:13.444 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 52769 53352 rocksdb:l ceph cwd unknown /proc/52769/task/53352/cwd (readlink: Permission denied) 2024-09-21T03:42:13.444 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 52769 53352 rocksdb:l ceph rtd unknown /proc/52769/task/53352/root (readlink: Permission denied) 2024-09-21T03:42:13.444 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 52769 53352 rocksdb:l ceph txt unknown /proc/52769/task/53352/exe (readlink: Permission denied) 2024-09-21T03:42:13.444 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 52769 53352 rocksdb:l ceph NOFD /proc/52769/task/53352/fd (opendir: Permission denied) 2024-09-21T03:42:13.444 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 52769 53353 rocksdb:l ceph cwd unknown /proc/52769/task/53353/cwd (readlink: Permission denied) 2024-09-21T03:42:13.444 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 52769 53353 rocksdb:l ceph rtd unknown /proc/52769/task/53353/root (readlink: Permission denied) 2024-09-21T03:42:13.445 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 52769 53353 rocksdb:l ceph txt unknown /proc/52769/task/53353/exe (readlink: Permission denied) 2024-09-21T03:42:13.445 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 52769 53353 rocksdb:l ceph NOFD /proc/52769/task/53353/fd (opendir: Permission denied) 2024-09-21T03:42:13.445 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 52769 53354 rocksdb:h ceph cwd unknown /proc/52769/task/53354/cwd (readlink: Permission denied) 2024-09-21T03:42:13.445 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 52769 53354 rocksdb:h ceph rtd unknown /proc/52769/task/53354/root (readlink: Permission denied) 2024-09-21T03:42:13.445 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 52769 53354 rocksdb:h ceph txt unknown /proc/52769/task/53354/exe (readlink: Permission denied) 2024-09-21T03:42:13.445 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 52769 53354 rocksdb:h ceph NOFD /proc/52769/task/53354/fd (opendir: Permission denied) 2024-09-21T03:42:13.445 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 52769 53762 bstore_ai ceph cwd unknown /proc/52769/task/53762/cwd (readlink: Permission denied) 2024-09-21T03:42:13.445 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 52769 53762 bstore_ai ceph rtd unknown /proc/52769/task/53762/root (readlink: Permission denied) 2024-09-21T03:42:13.445 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 52769 53762 bstore_ai ceph txt unknown /proc/52769/task/53762/exe (readlink: Permission denied) 2024-09-21T03:42:13.445 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 52769 53762 bstore_ai ceph NOFD /proc/52769/task/53762/fd (opendir: Permission denied) 2024-09-21T03:42:13.445 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 52769 53952 bstore_ai ceph cwd unknown /proc/52769/task/53952/cwd (readlink: Permission denied) 2024-09-21T03:42:13.445 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 52769 53952 bstore_ai ceph rtd unknown /proc/52769/task/53952/root (readlink: Permission denied) 2024-09-21T03:42:13.445 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 52769 53952 bstore_ai ceph txt unknown /proc/52769/task/53952/exe (readlink: Permission denied) 2024-09-21T03:42:13.445 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 52769 53952 bstore_ai ceph NOFD /proc/52769/task/53952/fd (opendir: Permission denied) 2024-09-21T03:42:13.446 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 52769 54134 ceph-osd ceph cwd unknown /proc/52769/task/54134/cwd (readlink: Permission denied) 2024-09-21T03:42:13.446 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 52769 54134 ceph-osd ceph rtd unknown /proc/52769/task/54134/root (readlink: Permission denied) 2024-09-21T03:42:13.446 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 52769 54134 ceph-osd ceph txt unknown /proc/52769/task/54134/exe (readlink: Permission denied) 2024-09-21T03:42:13.446 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 52769 54134 ceph-osd ceph NOFD /proc/52769/task/54134/fd (opendir: Permission denied) 2024-09-21T03:42:13.446 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 52769 54135 cfin ceph cwd unknown /proc/52769/task/54135/cwd (readlink: Permission denied) 2024-09-21T03:42:13.446 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 52769 54135 cfin ceph rtd unknown /proc/52769/task/54135/root (readlink: Permission denied) 2024-09-21T03:42:13.446 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 52769 54135 cfin ceph txt unknown /proc/52769/task/54135/exe (readlink: Permission denied) 2024-09-21T03:42:13.446 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 52769 54135 cfin ceph NOFD /proc/52769/task/54135/fd (opendir: Permission denied) 2024-09-21T03:42:13.446 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 52769 54136 bstore_kv ceph cwd unknown /proc/52769/task/54136/cwd (readlink: Permission denied) 2024-09-21T03:42:13.446 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 52769 54136 bstore_kv ceph rtd unknown /proc/52769/task/54136/root (readlink: Permission denied) 2024-09-21T03:42:13.446 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 52769 54136 bstore_kv ceph txt unknown /proc/52769/task/54136/exe (readlink: Permission denied) 2024-09-21T03:42:13.446 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 52769 54136 bstore_kv ceph NOFD /proc/52769/task/54136/fd (opendir: Permission denied) 2024-09-21T03:42:13.446 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 52769 54137 bstore_kv ceph cwd unknown /proc/52769/task/54137/cwd (readlink: Permission denied) 2024-09-21T03:42:13.447 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 52769 54137 bstore_kv ceph rtd unknown /proc/52769/task/54137/root (readlink: Permission denied) 2024-09-21T03:42:13.447 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 52769 54137 bstore_kv ceph txt unknown /proc/52769/task/54137/exe (readlink: Permission denied) 2024-09-21T03:42:13.447 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 52769 54137 bstore_kv ceph NOFD /proc/52769/task/54137/fd (opendir: Permission denied) 2024-09-21T03:42:13.447 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 52769 54138 bstore_me ceph cwd unknown /proc/52769/task/54138/cwd (readlink: Permission denied) 2024-09-21T03:42:13.447 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 52769 54138 bstore_me ceph rtd unknown /proc/52769/task/54138/root (readlink: Permission denied) 2024-09-21T03:42:13.447 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 52769 54138 bstore_me ceph txt unknown /proc/52769/task/54138/exe (readlink: Permission denied) 2024-09-21T03:42:13.447 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 52769 54138 bstore_me ceph NOFD /proc/52769/task/54138/fd (opendir: Permission denied) 2024-09-21T03:42:13.447 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 52769 54139 ms_dispat ceph cwd unknown /proc/52769/task/54139/cwd (readlink: Permission denied) 2024-09-21T03:42:13.447 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 52769 54139 ms_dispat ceph rtd unknown /proc/52769/task/54139/root (readlink: Permission denied) 2024-09-21T03:42:13.447 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 52769 54139 ms_dispat ceph txt unknown /proc/52769/task/54139/exe (readlink: Permission denied) 2024-09-21T03:42:13.447 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 52769 54139 ms_dispat ceph NOFD /proc/52769/task/54139/fd (opendir: Permission denied) 2024-09-21T03:42:13.447 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 52769 54140 ms_local ceph cwd unknown /proc/52769/task/54140/cwd (readlink: Permission denied) 2024-09-21T03:42:13.447 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 52769 54140 ms_local ceph rtd unknown /proc/52769/task/54140/root (readlink: Permission denied) 2024-09-21T03:42:13.447 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 52769 54140 ms_local ceph txt unknown /proc/52769/task/54140/exe (readlink: Permission denied) 2024-09-21T03:42:13.448 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 52769 54140 ms_local ceph NOFD /proc/52769/task/54140/fd (opendir: Permission denied) 2024-09-21T03:42:13.448 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 52769 54141 safe_time ceph cwd unknown /proc/52769/task/54141/cwd (readlink: Permission denied) 2024-09-21T03:42:13.448 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 52769 54141 safe_time ceph rtd unknown /proc/52769/task/54141/root (readlink: Permission denied) 2024-09-21T03:42:13.448 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 52769 54141 safe_time ceph txt unknown /proc/52769/task/54141/exe (readlink: Permission denied) 2024-09-21T03:42:13.448 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 52769 54141 safe_time ceph NOFD /proc/52769/task/54141/fd (opendir: Permission denied) 2024-09-21T03:42:13.448 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 52769 54142 safe_time ceph cwd unknown /proc/52769/task/54142/cwd (readlink: Permission denied) 2024-09-21T03:42:13.448 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 52769 54142 safe_time ceph rtd unknown /proc/52769/task/54142/root (readlink: Permission denied) 2024-09-21T03:42:13.448 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 52769 54142 safe_time ceph txt unknown /proc/52769/task/54142/exe (readlink: Permission denied) 2024-09-21T03:42:13.448 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 52769 54142 safe_time ceph NOFD /proc/52769/task/54142/fd (opendir: Permission denied) 2024-09-21T03:42:13.448 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 52769 54143 ms_dispat ceph cwd unknown /proc/52769/task/54143/cwd (readlink: Permission denied) 2024-09-21T03:42:13.448 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 52769 54143 ms_dispat ceph rtd unknown /proc/52769/task/54143/root (readlink: Permission denied) 2024-09-21T03:42:13.448 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 52769 54143 ms_dispat ceph txt unknown /proc/52769/task/54143/exe (readlink: Permission denied) 2024-09-21T03:42:13.448 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 52769 54143 ms_dispat ceph NOFD /proc/52769/task/54143/fd (opendir: Permission denied) 2024-09-21T03:42:13.448 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 52769 54144 ms_local ceph cwd unknown /proc/52769/task/54144/cwd (readlink: Permission denied) 2024-09-21T03:42:13.448 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 52769 54144 ms_local ceph rtd unknown /proc/52769/task/54144/root (readlink: Permission denied) 2024-09-21T03:42:13.449 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 52769 54144 ms_local ceph txt unknown /proc/52769/task/54144/exe (readlink: Permission denied) 2024-09-21T03:42:13.449 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 52769 54144 ms_local ceph NOFD /proc/52769/task/54144/fd (opendir: Permission denied) 2024-09-21T03:42:13.449 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 52769 54145 ms_dispat ceph cwd unknown /proc/52769/task/54145/cwd (readlink: Permission denied) 2024-09-21T03:42:13.449 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 52769 54145 ms_dispat ceph rtd unknown /proc/52769/task/54145/root (readlink: Permission denied) 2024-09-21T03:42:13.449 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 52769 54145 ms_dispat ceph txt unknown /proc/52769/task/54145/exe (readlink: Permission denied) 2024-09-21T03:42:13.449 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 52769 54145 ms_dispat ceph NOFD /proc/52769/task/54145/fd (opendir: Permission denied) 2024-09-21T03:42:13.449 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 52769 54146 ms_local ceph cwd unknown /proc/52769/task/54146/cwd (readlink: Permission denied) 2024-09-21T03:42:13.449 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 52769 54146 ms_local ceph rtd unknown /proc/52769/task/54146/root (readlink: Permission denied) 2024-09-21T03:42:13.449 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 52769 54146 ms_local ceph txt unknown /proc/52769/task/54146/exe (readlink: Permission denied) 2024-09-21T03:42:13.449 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 52769 54146 ms_local ceph NOFD /proc/52769/task/54146/fd (opendir: Permission denied) 2024-09-21T03:42:13.449 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 52769 54147 ms_dispat ceph cwd unknown /proc/52769/task/54147/cwd (readlink: Permission denied) 2024-09-21T03:42:13.449 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 52769 54147 ms_dispat ceph rtd unknown /proc/52769/task/54147/root (readlink: Permission denied) 2024-09-21T03:42:13.449 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 52769 54147 ms_dispat ceph txt unknown /proc/52769/task/54147/exe (readlink: Permission denied) 2024-09-21T03:42:13.449 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 52769 54147 ms_dispat ceph NOFD /proc/52769/task/54147/fd (opendir: Permission denied) 2024-09-21T03:42:13.449 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 52769 54148 ms_local ceph cwd unknown /proc/52769/task/54148/cwd (readlink: Permission denied) 2024-09-21T03:42:13.450 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 52769 54148 ms_local ceph rtd unknown /proc/52769/task/54148/root (readlink: Permission denied) 2024-09-21T03:42:13.450 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 52769 54148 ms_local ceph txt unknown /proc/52769/task/54148/exe (readlink: Permission denied) 2024-09-21T03:42:13.450 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 52769 54148 ms_local ceph NOFD /proc/52769/task/54148/fd (opendir: Permission denied) 2024-09-21T03:42:13.450 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 52769 54149 ms_dispat ceph cwd unknown /proc/52769/task/54149/cwd (readlink: Permission denied) 2024-09-21T03:42:13.450 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 52769 54149 ms_dispat ceph rtd unknown /proc/52769/task/54149/root (readlink: Permission denied) 2024-09-21T03:42:13.450 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 52769 54149 ms_dispat ceph txt unknown /proc/52769/task/54149/exe (readlink: Permission denied) 2024-09-21T03:42:13.450 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 52769 54149 ms_dispat ceph NOFD /proc/52769/task/54149/fd (opendir: Permission denied) 2024-09-21T03:42:13.450 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 52769 54150 ms_local ceph cwd unknown /proc/52769/task/54150/cwd (readlink: Permission denied) 2024-09-21T03:42:13.450 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 52769 54150 ms_local ceph rtd unknown /proc/52769/task/54150/root (readlink: Permission denied) 2024-09-21T03:42:13.450 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 52769 54150 ms_local ceph txt unknown /proc/52769/task/54150/exe (readlink: Permission denied) 2024-09-21T03:42:13.450 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 52769 54150 ms_local ceph NOFD /proc/52769/task/54150/fd (opendir: Permission denied) 2024-09-21T03:42:13.450 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 52769 54151 ms_dispat ceph cwd unknown /proc/52769/task/54151/cwd (readlink: Permission denied) 2024-09-21T03:42:13.450 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 52769 54151 ms_dispat ceph rtd unknown /proc/52769/task/54151/root (readlink: Permission denied) 2024-09-21T03:42:13.450 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 52769 54151 ms_dispat ceph txt unknown /proc/52769/task/54151/exe (readlink: Permission denied) 2024-09-21T03:42:13.451 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 52769 54151 ms_dispat ceph NOFD /proc/52769/task/54151/fd (opendir: Permission denied) 2024-09-21T03:42:13.451 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 52769 54152 ms_local ceph cwd unknown /proc/52769/task/54152/cwd (readlink: Permission denied) 2024-09-21T03:42:13.451 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 52769 54152 ms_local ceph rtd unknown /proc/52769/task/54152/root (readlink: Permission denied) 2024-09-21T03:42:13.451 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 52769 54152 ms_local ceph txt unknown /proc/52769/task/54152/exe (readlink: Permission denied) 2024-09-21T03:42:13.451 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 52769 54152 ms_local ceph NOFD /proc/52769/task/54152/fd (opendir: Permission denied) 2024-09-21T03:42:13.451 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 52769 54153 ms_dispat ceph cwd unknown /proc/52769/task/54153/cwd (readlink: Permission denied) 2024-09-21T03:42:13.451 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 52769 54153 ms_dispat ceph rtd unknown /proc/52769/task/54153/root (readlink: Permission denied) 2024-09-21T03:42:13.451 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 52769 54153 ms_dispat ceph txt unknown /proc/52769/task/54153/exe (readlink: Permission denied) 2024-09-21T03:42:13.451 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 52769 54153 ms_dispat ceph NOFD /proc/52769/task/54153/fd (opendir: Permission denied) 2024-09-21T03:42:13.451 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 52769 54154 ms_local ceph cwd unknown /proc/52769/task/54154/cwd (readlink: Permission denied) 2024-09-21T03:42:13.451 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 52769 54154 ms_local ceph rtd unknown /proc/52769/task/54154/root (readlink: Permission denied) 2024-09-21T03:42:13.451 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 52769 54154 ms_local ceph txt unknown /proc/52769/task/54154/exe (readlink: Permission denied) 2024-09-21T03:42:13.451 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 52769 54154 ms_local ceph NOFD /proc/52769/task/54154/fd (opendir: Permission denied) 2024-09-21T03:42:13.451 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 52769 54155 fn_anonym ceph cwd unknown /proc/52769/task/54155/cwd (readlink: Permission denied) 2024-09-21T03:42:13.452 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 52769 54155 fn_anonym ceph rtd unknown /proc/52769/task/54155/root (readlink: Permission denied) 2024-09-21T03:42:13.452 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 52769 54155 fn_anonym ceph txt unknown /proc/52769/task/54155/exe (readlink: Permission denied) 2024-09-21T03:42:13.452 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 52769 54155 fn_anonym ceph NOFD /proc/52769/task/54155/fd (opendir: Permission denied) 2024-09-21T03:42:13.452 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 52769 54156 finisher ceph cwd unknown /proc/52769/task/54156/cwd (readlink: Permission denied) 2024-09-21T03:42:13.452 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 52769 54156 finisher ceph rtd unknown /proc/52769/task/54156/root (readlink: Permission denied) 2024-09-21T03:42:13.452 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 52769 54156 finisher ceph txt unknown /proc/52769/task/54156/exe (readlink: Permission denied) 2024-09-21T03:42:13.452 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 52769 54156 finisher ceph NOFD /proc/52769/task/54156/fd (opendir: Permission denied) 2024-09-21T03:42:13.452 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 52769 54157 safe_time ceph cwd unknown /proc/52769/task/54157/cwd (readlink: Permission denied) 2024-09-21T03:42:13.452 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 52769 54157 safe_time ceph rtd unknown /proc/52769/task/54157/root (readlink: Permission denied) 2024-09-21T03:42:13.452 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 52769 54157 safe_time ceph txt unknown /proc/52769/task/54157/exe (readlink: Permission denied) 2024-09-21T03:42:13.452 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 52769 54157 safe_time ceph NOFD /proc/52769/task/54157/fd (opendir: Permission denied) 2024-09-21T03:42:13.452 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 52769 54158 safe_time ceph cwd unknown /proc/52769/task/54158/cwd (readlink: Permission denied) 2024-09-21T03:42:13.452 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 52769 54158 safe_time ceph rtd unknown /proc/52769/task/54158/root (readlink: Permission denied) 2024-09-21T03:42:13.452 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 52769 54158 safe_time ceph txt unknown /proc/52769/task/54158/exe (readlink: Permission denied) 2024-09-21T03:42:13.452 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 52769 54158 safe_time ceph NOFD /proc/52769/task/54158/fd (opendir: Permission denied) 2024-09-21T03:42:13.453 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 52769 54159 ceph-osd ceph cwd unknown /proc/52769/task/54159/cwd (readlink: Permission denied) 2024-09-21T03:42:13.453 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 52769 54159 ceph-osd ceph rtd unknown /proc/52769/task/54159/root (readlink: Permission denied) 2024-09-21T03:42:13.453 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 52769 54159 ceph-osd ceph txt unknown /proc/52769/task/54159/exe (readlink: Permission denied) 2024-09-21T03:42:13.453 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 52769 54159 ceph-osd ceph NOFD /proc/52769/task/54159/fd (opendir: Permission denied) 2024-09-21T03:42:13.453 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 52769 54160 osd_srv_a ceph cwd unknown /proc/52769/task/54160/cwd (readlink: Permission denied) 2024-09-21T03:42:13.453 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 52769 54160 osd_srv_a ceph rtd unknown /proc/52769/task/54160/root (readlink: Permission denied) 2024-09-21T03:42:13.453 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 52769 54160 osd_srv_a ceph txt unknown /proc/52769/task/54160/exe (readlink: Permission denied) 2024-09-21T03:42:13.453 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 52769 54160 osd_srv_a ceph NOFD /proc/52769/task/54160/fd (opendir: Permission denied) 2024-09-21T03:42:13.453 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 52769 54161 tp_osd_tp ceph cwd unknown /proc/52769/task/54161/cwd (readlink: Permission denied) 2024-09-21T03:42:13.453 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 52769 54161 tp_osd_tp ceph rtd unknown /proc/52769/task/54161/root (readlink: Permission denied) 2024-09-21T03:42:13.453 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 52769 54161 tp_osd_tp ceph txt unknown /proc/52769/task/54161/exe (readlink: Permission denied) 2024-09-21T03:42:13.453 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 52769 54161 tp_osd_tp ceph NOFD /proc/52769/task/54161/fd (opendir: Permission denied) 2024-09-21T03:42:13.453 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 52769 54162 tp_osd_tp ceph cwd unknown /proc/52769/task/54162/cwd (readlink: Permission denied) 2024-09-21T03:42:13.453 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 52769 54162 tp_osd_tp ceph rtd unknown /proc/52769/task/54162/root (readlink: Permission denied) 2024-09-21T03:42:13.454 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 52769 54162 tp_osd_tp ceph txt unknown /proc/52769/task/54162/exe (readlink: Permission denied) 2024-09-21T03:42:13.454 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 52769 54162 tp_osd_tp ceph NOFD /proc/52769/task/54162/fd (opendir: Permission denied) 2024-09-21T03:42:13.454 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 52769 54163 tp_osd_tp ceph cwd unknown /proc/52769/task/54163/cwd (readlink: Permission denied) 2024-09-21T03:42:13.454 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 52769 54163 tp_osd_tp ceph rtd unknown /proc/52769/task/54163/root (readlink: Permission denied) 2024-09-21T03:42:13.454 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 52769 54163 tp_osd_tp ceph txt unknown /proc/52769/task/54163/exe (readlink: Permission denied) 2024-09-21T03:42:13.454 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 52769 54163 tp_osd_tp ceph NOFD /proc/52769/task/54163/fd (opendir: Permission denied) 2024-09-21T03:42:13.454 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 52769 54164 tp_osd_tp ceph cwd unknown /proc/52769/task/54164/cwd (readlink: Permission denied) 2024-09-21T03:42:13.454 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 52769 54164 tp_osd_tp ceph rtd unknown /proc/52769/task/54164/root (readlink: Permission denied) 2024-09-21T03:42:13.454 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 52769 54164 tp_osd_tp ceph txt unknown /proc/52769/task/54164/exe (readlink: Permission denied) 2024-09-21T03:42:13.454 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 52769 54164 tp_osd_tp ceph NOFD /proc/52769/task/54164/fd (opendir: Permission denied) 2024-09-21T03:42:13.454 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 52769 54165 tp_osd_tp ceph cwd unknown /proc/52769/task/54165/cwd (readlink: Permission denied) 2024-09-21T03:42:13.454 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 52769 54165 tp_osd_tp ceph rtd unknown /proc/52769/task/54165/root (readlink: Permission denied) 2024-09-21T03:42:13.454 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 52769 54165 tp_osd_tp ceph txt unknown /proc/52769/task/54165/exe (readlink: Permission denied) 2024-09-21T03:42:13.454 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 52769 54165 tp_osd_tp ceph NOFD /proc/52769/task/54165/fd (opendir: Permission denied) 2024-09-21T03:42:13.455 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 52769 54166 tp_osd_tp ceph cwd unknown /proc/52769/task/54166/cwd (readlink: Permission denied) 2024-09-21T03:42:13.455 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 52769 54166 tp_osd_tp ceph rtd unknown /proc/52769/task/54166/root (readlink: Permission denied) 2024-09-21T03:42:13.455 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 52769 54166 tp_osd_tp ceph txt unknown /proc/52769/task/54166/exe (readlink: Permission denied) 2024-09-21T03:42:13.455 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 52769 54166 tp_osd_tp ceph NOFD /proc/52769/task/54166/fd (opendir: Permission denied) 2024-09-21T03:42:13.455 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 52769 54167 tp_osd_tp ceph cwd unknown /proc/52769/task/54167/cwd (readlink: Permission denied) 2024-09-21T03:42:13.455 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 52769 54167 tp_osd_tp ceph rtd unknown /proc/52769/task/54167/root (readlink: Permission denied) 2024-09-21T03:42:13.455 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 52769 54167 tp_osd_tp ceph txt unknown /proc/52769/task/54167/exe (readlink: Permission denied) 2024-09-21T03:42:13.455 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 52769 54167 tp_osd_tp ceph NOFD /proc/52769/task/54167/fd (opendir: Permission denied) 2024-09-21T03:42:13.455 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 52769 54168 tp_osd_tp ceph cwd unknown /proc/52769/task/54168/cwd (readlink: Permission denied) 2024-09-21T03:42:13.455 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 52769 54168 tp_osd_tp ceph rtd unknown /proc/52769/task/54168/root (readlink: Permission denied) 2024-09-21T03:42:13.455 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 52769 54168 tp_osd_tp ceph txt unknown /proc/52769/task/54168/exe (readlink: Permission denied) 2024-09-21T03:42:13.455 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 52769 54168 tp_osd_tp ceph NOFD /proc/52769/task/54168/fd (opendir: Permission denied) 2024-09-21T03:42:13.455 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 52769 54169 tp_osd_tp ceph cwd unknown /proc/52769/task/54169/cwd (readlink: Permission denied) 2024-09-21T03:42:13.455 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 52769 54169 tp_osd_tp ceph rtd unknown /proc/52769/task/54169/root (readlink: Permission denied) 2024-09-21T03:42:13.455 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 52769 54169 tp_osd_tp ceph txt unknown /proc/52769/task/54169/exe (readlink: Permission denied) 2024-09-21T03:42:13.456 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 52769 54169 tp_osd_tp ceph NOFD /proc/52769/task/54169/fd (opendir: Permission denied) 2024-09-21T03:42:13.456 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 52769 54170 tp_osd_tp ceph cwd unknown /proc/52769/task/54170/cwd (readlink: Permission denied) 2024-09-21T03:42:13.456 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 52769 54170 tp_osd_tp ceph rtd unknown /proc/52769/task/54170/root (readlink: Permission denied) 2024-09-21T03:42:13.456 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 52769 54170 tp_osd_tp ceph txt unknown /proc/52769/task/54170/exe (readlink: Permission denied) 2024-09-21T03:42:13.456 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 52769 54170 tp_osd_tp ceph NOFD /proc/52769/task/54170/fd (opendir: Permission denied) 2024-09-21T03:42:13.456 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 52769 54171 tp_osd_tp ceph cwd unknown /proc/52769/task/54171/cwd (readlink: Permission denied) 2024-09-21T03:42:13.456 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 52769 54171 tp_osd_tp ceph rtd unknown /proc/52769/task/54171/root (readlink: Permission denied) 2024-09-21T03:42:13.456 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 52769 54171 tp_osd_tp ceph txt unknown /proc/52769/task/54171/exe (readlink: Permission denied) 2024-09-21T03:42:13.456 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 52769 54171 tp_osd_tp ceph NOFD /proc/52769/task/54171/fd (opendir: Permission denied) 2024-09-21T03:42:13.456 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 52769 54172 tp_osd_tp ceph cwd unknown /proc/52769/task/54172/cwd (readlink: Permission denied) 2024-09-21T03:42:13.456 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 52769 54172 tp_osd_tp ceph rtd unknown /proc/52769/task/54172/root (readlink: Permission denied) 2024-09-21T03:42:13.456 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 52769 54172 tp_osd_tp ceph txt unknown /proc/52769/task/54172/exe (readlink: Permission denied) 2024-09-21T03:42:13.456 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 52769 54172 tp_osd_tp ceph NOFD /proc/52769/task/54172/fd (opendir: Permission denied) 2024-09-21T03:42:13.456 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 52769 54173 tp_osd_tp ceph cwd unknown /proc/52769/task/54173/cwd (readlink: Permission denied) 2024-09-21T03:42:13.456 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 52769 54173 tp_osd_tp ceph rtd unknown /proc/52769/task/54173/root (readlink: Permission denied) 2024-09-21T03:42:13.457 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 52769 54173 tp_osd_tp ceph txt unknown /proc/52769/task/54173/exe (readlink: Permission denied) 2024-09-21T03:42:13.457 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 52769 54173 tp_osd_tp ceph NOFD /proc/52769/task/54173/fd (opendir: Permission denied) 2024-09-21T03:42:13.457 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 52769 54174 tp_osd_tp ceph cwd unknown /proc/52769/task/54174/cwd (readlink: Permission denied) 2024-09-21T03:42:13.457 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 52769 54174 tp_osd_tp ceph rtd unknown /proc/52769/task/54174/root (readlink: Permission denied) 2024-09-21T03:42:13.457 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 52769 54174 tp_osd_tp ceph txt unknown /proc/52769/task/54174/exe (readlink: Permission denied) 2024-09-21T03:42:13.457 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 52769 54174 tp_osd_tp ceph NOFD /proc/52769/task/54174/fd (opendir: Permission denied) 2024-09-21T03:42:13.457 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 52769 54175 tp_osd_tp ceph cwd unknown /proc/52769/task/54175/cwd (readlink: Permission denied) 2024-09-21T03:42:13.457 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 52769 54175 tp_osd_tp ceph rtd unknown /proc/52769/task/54175/root (readlink: Permission denied) 2024-09-21T03:42:13.457 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 52769 54175 tp_osd_tp ceph txt unknown /proc/52769/task/54175/exe (readlink: Permission denied) 2024-09-21T03:42:13.457 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 52769 54175 tp_osd_tp ceph NOFD /proc/52769/task/54175/fd (opendir: Permission denied) 2024-09-21T03:42:13.457 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 52769 54176 tp_osd_tp ceph cwd unknown /proc/52769/task/54176/cwd (readlink: Permission denied) 2024-09-21T03:42:13.457 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 52769 54176 tp_osd_tp ceph rtd unknown /proc/52769/task/54176/root (readlink: Permission denied) 2024-09-21T03:42:13.457 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 52769 54176 tp_osd_tp ceph txt unknown /proc/52769/task/54176/exe (readlink: Permission denied) 2024-09-21T03:42:13.457 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 52769 54176 tp_osd_tp ceph NOFD /proc/52769/task/54176/fd (opendir: Permission denied) 2024-09-21T03:42:13.457 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 52769 54177 osd_srv_h ceph cwd unknown /proc/52769/task/54177/cwd (readlink: Permission denied) 2024-09-21T03:42:13.458 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 52769 54177 osd_srv_h ceph rtd unknown /proc/52769/task/54177/root (readlink: Permission denied) 2024-09-21T03:42:13.458 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 52769 54177 osd_srv_h ceph txt unknown /proc/52769/task/54177/exe (readlink: Permission denied) 2024-09-21T03:42:13.458 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 52769 54177 osd_srv_h ceph NOFD /proc/52769/task/54177/fd (opendir: Permission denied) 2024-09-21T03:42:13.458 INFO:tasks.workunit.client.0.smithi089.stdout:sudo 53308 root cwd unknown /proc/53308/cwd (readlink: Permission denied) 2024-09-21T03:42:13.458 INFO:tasks.workunit.client.0.smithi089.stdout:sudo 53308 root rtd unknown /proc/53308/root (readlink: Permission denied) 2024-09-21T03:42:13.458 INFO:tasks.workunit.client.0.smithi089.stdout:sudo 53308 root txt unknown /proc/53308/exe (readlink: Permission denied) 2024-09-21T03:42:13.458 INFO:tasks.workunit.client.0.smithi089.stdout:sudo 53308 root NOFD /proc/53308/fd (opendir: Permission denied) 2024-09-21T03:42:13.458 INFO:tasks.workunit.client.0.smithi089.stdout:journalct 53329 root cwd unknown /proc/53329/cwd (readlink: Permission denied) 2024-09-21T03:42:13.458 INFO:tasks.workunit.client.0.smithi089.stdout:journalct 53329 root rtd unknown /proc/53329/root (readlink: Permission denied) 2024-09-21T03:42:13.458 INFO:tasks.workunit.client.0.smithi089.stdout:journalct 53329 root txt unknown /proc/53329/exe (readlink: Permission denied) 2024-09-21T03:42:13.458 INFO:tasks.workunit.client.0.smithi089.stdout:journalct 53329 root NOFD /proc/53329/fd (opendir: Permission denied) 2024-09-21T03:42:13.458 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/6 58628 root cwd unknown /proc/58628/cwd (readlink: Permission denied) 2024-09-21T03:42:13.458 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/6 58628 root rtd unknown /proc/58628/root (readlink: Permission denied) 2024-09-21T03:42:13.458 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/6 58628 root txt unknown /proc/58628/exe (readlink: Permission denied) 2024-09-21T03:42:13.459 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/6 58628 root NOFD /proc/58628/fd (opendir: Permission denied) 2024-09-21T03:42:13.459 INFO:tasks.workunit.client.0.smithi089.stdout:conmon 59191 root cwd unknown /proc/59191/cwd (readlink: Permission denied) 2024-09-21T03:42:13.459 INFO:tasks.workunit.client.0.smithi089.stdout:conmon 59191 root rtd unknown /proc/59191/root (readlink: Permission denied) 2024-09-21T03:42:13.459 INFO:tasks.workunit.client.0.smithi089.stdout:conmon 59191 root txt unknown /proc/59191/exe (readlink: Permission denied) 2024-09-21T03:42:13.459 INFO:tasks.workunit.client.0.smithi089.stdout:conmon 59191 root NOFD /proc/59191/fd (opendir: Permission denied) 2024-09-21T03:42:13.459 INFO:tasks.workunit.client.0.smithi089.stdout:podman-in 59193 root cwd unknown /proc/59193/cwd (readlink: Permission denied) 2024-09-21T03:42:13.459 INFO:tasks.workunit.client.0.smithi089.stdout:podman-in 59193 root rtd unknown /proc/59193/root (readlink: Permission denied) 2024-09-21T03:42:13.459 INFO:tasks.workunit.client.0.smithi089.stdout:podman-in 59193 root txt unknown /proc/59193/exe (readlink: Permission denied) 2024-09-21T03:42:13.459 INFO:tasks.workunit.client.0.smithi089.stdout:podman-in 59193 root NOFD /proc/59193/fd (opendir: Permission denied) 2024-09-21T03:42:13.459 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 59195 ceph cwd unknown /proc/59195/cwd (readlink: Permission denied) 2024-09-21T03:42:13.459 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 59195 ceph rtd unknown /proc/59195/root (readlink: Permission denied) 2024-09-21T03:42:13.459 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 59195 ceph txt unknown /proc/59195/exe (readlink: Permission denied) 2024-09-21T03:42:13.460 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 59195 ceph NOFD /proc/59195/fd (opendir: Permission denied) 2024-09-21T03:42:13.460 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 59195 59196 log ceph cwd unknown /proc/59195/task/59196/cwd (readlink: Permission denied) 2024-09-21T03:42:13.460 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 59195 59196 log ceph rtd unknown /proc/59195/task/59196/root (readlink: Permission denied) 2024-09-21T03:42:13.460 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 59195 59196 log ceph txt unknown /proc/59195/task/59196/exe (readlink: Permission denied) 2024-09-21T03:42:13.460 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 59195 59196 log ceph NOFD /proc/59195/task/59196/fd (opendir: Permission denied) 2024-09-21T03:42:13.460 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 59195 59198 msgr-work ceph cwd unknown /proc/59195/task/59198/cwd (readlink: Permission denied) 2024-09-21T03:42:13.460 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 59195 59198 msgr-work ceph rtd unknown /proc/59195/task/59198/root (readlink: Permission denied) 2024-09-21T03:42:13.460 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 59195 59198 msgr-work ceph txt unknown /proc/59195/task/59198/exe (readlink: Permission denied) 2024-09-21T03:42:13.460 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 59195 59198 msgr-work ceph NOFD /proc/59195/task/59198/fd (opendir: Permission denied) 2024-09-21T03:42:13.460 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 59195 59199 msgr-work ceph cwd unknown /proc/59195/task/59199/cwd (readlink: Permission denied) 2024-09-21T03:42:13.460 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 59195 59199 msgr-work ceph rtd unknown /proc/59195/task/59199/root (readlink: Permission denied) 2024-09-21T03:42:13.460 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 59195 59199 msgr-work ceph txt unknown /proc/59195/task/59199/exe (readlink: Permission denied) 2024-09-21T03:42:13.460 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 59195 59199 msgr-work ceph NOFD /proc/59195/task/59199/fd (opendir: Permission denied) 2024-09-21T03:42:13.460 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 59195 59200 msgr-work ceph cwd unknown /proc/59195/task/59200/cwd (readlink: Permission denied) 2024-09-21T03:42:13.461 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 59195 59200 msgr-work ceph rtd unknown /proc/59195/task/59200/root (readlink: Permission denied) 2024-09-21T03:42:13.461 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 59195 59200 msgr-work ceph txt unknown /proc/59195/task/59200/exe (readlink: Permission denied) 2024-09-21T03:42:13.461 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 59195 59200 msgr-work ceph NOFD /proc/59195/task/59200/fd (opendir: Permission denied) 2024-09-21T03:42:13.461 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 59195 59204 service ceph cwd unknown /proc/59195/task/59204/cwd (readlink: Permission denied) 2024-09-21T03:42:13.461 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 59195 59204 service ceph rtd unknown /proc/59195/task/59204/root (readlink: Permission denied) 2024-09-21T03:42:13.461 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 59195 59204 service ceph txt unknown /proc/59195/task/59204/exe (readlink: Permission denied) 2024-09-21T03:42:13.461 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 59195 59204 service ceph NOFD /proc/59195/task/59204/fd (opendir: Permission denied) 2024-09-21T03:42:13.461 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 59195 59205 admin_soc ceph cwd unknown /proc/59195/task/59205/cwd (readlink: Permission denied) 2024-09-21T03:42:13.461 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 59195 59205 admin_soc ceph rtd unknown /proc/59195/task/59205/root (readlink: Permission denied) 2024-09-21T03:42:13.461 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 59195 59205 admin_soc ceph txt unknown /proc/59195/task/59205/exe (readlink: Permission denied) 2024-09-21T03:42:13.461 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 59195 59205 admin_soc ceph NOFD /proc/59195/task/59205/fd (opendir: Permission denied) 2024-09-21T03:42:13.461 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 59195 59474 signal_ha ceph cwd unknown /proc/59195/task/59474/cwd (readlink: Permission denied) 2024-09-21T03:42:13.461 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 59195 59474 signal_ha ceph rtd unknown /proc/59195/task/59474/root (readlink: Permission denied) 2024-09-21T03:42:13.462 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 59195 59474 signal_ha ceph txt unknown /proc/59195/task/59474/exe (readlink: Permission denied) 2024-09-21T03:42:13.462 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 59195 59474 signal_ha ceph NOFD /proc/59195/task/59474/fd (opendir: Permission denied) 2024-09-21T03:42:13.462 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 59195 59475 io_contex ceph cwd unknown /proc/59195/task/59475/cwd (readlink: Permission denied) 2024-09-21T03:42:13.462 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 59195 59475 io_contex ceph rtd unknown /proc/59195/task/59475/root (readlink: Permission denied) 2024-09-21T03:42:13.462 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 59195 59475 io_contex ceph txt unknown /proc/59195/task/59475/exe (readlink: Permission denied) 2024-09-21T03:42:13.462 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 59195 59475 io_contex ceph NOFD /proc/59195/task/59475/fd (opendir: Permission denied) 2024-09-21T03:42:13.462 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 59195 59476 io_contex ceph cwd unknown /proc/59195/task/59476/cwd (readlink: Permission denied) 2024-09-21T03:42:13.462 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 59195 59476 io_contex ceph rtd unknown /proc/59195/task/59476/root (readlink: Permission denied) 2024-09-21T03:42:13.462 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 59195 59476 io_contex ceph txt unknown /proc/59195/task/59476/exe (readlink: Permission denied) 2024-09-21T03:42:13.462 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 59195 59476 io_contex ceph NOFD /proc/59195/task/59476/fd (opendir: Permission denied) 2024-09-21T03:42:13.462 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 59195 59490 OpHistory ceph cwd unknown /proc/59195/task/59490/cwd (readlink: Permission denied) 2024-09-21T03:42:13.462 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 59195 59490 OpHistory ceph rtd unknown /proc/59195/task/59490/root (readlink: Permission denied) 2024-09-21T03:42:13.462 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 59195 59490 OpHistory ceph txt unknown /proc/59195/task/59490/exe (readlink: Permission denied) 2024-09-21T03:42:13.463 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 59195 59490 OpHistory ceph NOFD /proc/59195/task/59490/fd (opendir: Permission denied) 2024-09-21T03:42:13.463 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 59195 59491 ceph_time ceph cwd unknown /proc/59195/task/59491/cwd (readlink: Permission denied) 2024-09-21T03:42:13.463 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 59195 59491 ceph_time ceph rtd unknown /proc/59195/task/59491/root (readlink: Permission denied) 2024-09-21T03:42:13.463 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 59195 59491 ceph_time ceph txt unknown /proc/59195/task/59491/exe (readlink: Permission denied) 2024-09-21T03:42:13.463 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 59195 59491 ceph_time ceph NOFD /proc/59195/task/59491/fd (opendir: Permission denied) 2024-09-21T03:42:13.463 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 59195 59550 ceph-osd ceph cwd unknown /proc/59195/task/59550/cwd (readlink: Permission denied) 2024-09-21T03:42:13.463 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 59195 59550 ceph-osd ceph rtd unknown /proc/59195/task/59550/root (readlink: Permission denied) 2024-09-21T03:42:13.463 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 59195 59550 ceph-osd ceph txt unknown /proc/59195/task/59550/exe (readlink: Permission denied) 2024-09-21T03:42:13.463 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 59195 59550 ceph-osd ceph NOFD /proc/59195/task/59550/fd (opendir: Permission denied) 2024-09-21T03:42:13.463 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 59195 59578 ceph-osd ceph cwd unknown /proc/59195/task/59578/cwd (readlink: Permission denied) 2024-09-21T03:42:13.463 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 59195 59578 ceph-osd ceph rtd unknown /proc/59195/task/59578/root (readlink: Permission denied) 2024-09-21T03:42:13.463 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 59195 59578 ceph-osd ceph txt unknown /proc/59195/task/59578/exe (readlink: Permission denied) 2024-09-21T03:42:13.463 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 59195 59578 ceph-osd ceph NOFD /proc/59195/task/59578/fd (opendir: Permission denied) 2024-09-21T03:42:13.463 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 59195 59613 ceph-osd ceph cwd unknown /proc/59195/task/59613/cwd (readlink: Permission denied) 2024-09-21T03:42:13.463 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 59195 59613 ceph-osd ceph rtd unknown /proc/59195/task/59613/root (readlink: Permission denied) 2024-09-21T03:42:13.464 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 59195 59613 ceph-osd ceph txt unknown /proc/59195/task/59613/exe (readlink: Permission denied) 2024-09-21T03:42:13.464 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 59195 59613 ceph-osd ceph NOFD /proc/59195/task/59613/fd (opendir: Permission denied) 2024-09-21T03:42:13.464 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 59195 59621 ceph-osd ceph cwd unknown /proc/59195/task/59621/cwd (readlink: Permission denied) 2024-09-21T03:42:13.464 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 59195 59621 ceph-osd ceph rtd unknown /proc/59195/task/59621/root (readlink: Permission denied) 2024-09-21T03:42:13.464 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 59195 59621 ceph-osd ceph txt unknown /proc/59195/task/59621/exe (readlink: Permission denied) 2024-09-21T03:42:13.464 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 59195 59621 ceph-osd ceph NOFD /proc/59195/task/59621/fd (opendir: Permission denied) 2024-09-21T03:42:13.464 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 59195 59643 ceph-osd ceph cwd unknown /proc/59195/task/59643/cwd (readlink: Permission denied) 2024-09-21T03:42:13.464 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 59195 59643 ceph-osd ceph rtd unknown /proc/59195/task/59643/root (readlink: Permission denied) 2024-09-21T03:42:13.464 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 59195 59643 ceph-osd ceph txt unknown /proc/59195/task/59643/exe (readlink: Permission denied) 2024-09-21T03:42:13.464 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 59195 59643 ceph-osd ceph NOFD /proc/59195/task/59643/fd (opendir: Permission denied) 2024-09-21T03:42:13.464 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 59195 59647 ceph-osd ceph cwd unknown /proc/59195/task/59647/cwd (readlink: Permission denied) 2024-09-21T03:42:13.464 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 59195 59647 ceph-osd ceph rtd unknown /proc/59195/task/59647/root (readlink: Permission denied) 2024-09-21T03:42:13.464 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 59195 59647 ceph-osd ceph txt unknown /proc/59195/task/59647/exe (readlink: Permission denied) 2024-09-21T03:42:13.464 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 59195 59647 ceph-osd ceph NOFD /proc/59195/task/59647/fd (opendir: Permission denied) 2024-09-21T03:42:13.465 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 59195 59652 ceph-osd ceph cwd unknown /proc/59195/task/59652/cwd (readlink: Permission denied) 2024-09-21T03:42:13.465 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 59195 59652 ceph-osd ceph rtd unknown /proc/59195/task/59652/root (readlink: Permission denied) 2024-09-21T03:42:13.465 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 59195 59652 ceph-osd ceph txt unknown /proc/59195/task/59652/exe (readlink: Permission denied) 2024-09-21T03:42:13.465 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 59195 59652 ceph-osd ceph NOFD /proc/59195/task/59652/fd (opendir: Permission denied) 2024-09-21T03:42:13.465 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 59195 59716 ceph-osd ceph cwd unknown /proc/59195/task/59716/cwd (readlink: Permission denied) 2024-09-21T03:42:13.465 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 59195 59716 ceph-osd ceph rtd unknown /proc/59195/task/59716/root (readlink: Permission denied) 2024-09-21T03:42:13.465 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 59195 59716 ceph-osd ceph txt unknown /proc/59195/task/59716/exe (readlink: Permission denied) 2024-09-21T03:42:13.465 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 59195 59716 ceph-osd ceph NOFD /proc/59195/task/59716/fd (opendir: Permission denied) 2024-09-21T03:42:13.465 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 59195 59717 ceph-osd ceph cwd unknown /proc/59195/task/59717/cwd (readlink: Permission denied) 2024-09-21T03:42:13.465 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 59195 59717 ceph-osd ceph rtd unknown /proc/59195/task/59717/root (readlink: Permission denied) 2024-09-21T03:42:13.465 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 59195 59717 ceph-osd ceph txt unknown /proc/59195/task/59717/exe (readlink: Permission denied) 2024-09-21T03:42:13.465 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 59195 59717 ceph-osd ceph NOFD /proc/59195/task/59717/fd (opendir: Permission denied) 2024-09-21T03:42:13.465 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 59195 59718 safe_time ceph cwd unknown /proc/59195/task/59718/cwd (readlink: Permission denied) 2024-09-21T03:42:13.465 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 59195 59718 safe_time ceph rtd unknown /proc/59195/task/59718/root (readlink: Permission denied) 2024-09-21T03:42:13.465 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 59195 59718 safe_time ceph txt unknown /proc/59195/task/59718/exe (readlink: Permission denied) 2024-09-21T03:42:13.466 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 59195 59718 safe_time ceph NOFD /proc/59195/task/59718/fd (opendir: Permission denied) 2024-09-21T03:42:13.466 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 59195 59719 safe_time ceph cwd unknown /proc/59195/task/59719/cwd (readlink: Permission denied) 2024-09-21T03:42:13.466 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 59195 59719 safe_time ceph rtd unknown /proc/59195/task/59719/root (readlink: Permission denied) 2024-09-21T03:42:13.466 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 59195 59719 safe_time ceph txt unknown /proc/59195/task/59719/exe (readlink: Permission denied) 2024-09-21T03:42:13.466 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 59195 59719 safe_time ceph NOFD /proc/59195/task/59719/fd (opendir: Permission denied) 2024-09-21T03:42:13.466 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 59195 59720 safe_time ceph cwd unknown /proc/59195/task/59720/cwd (readlink: Permission denied) 2024-09-21T03:42:13.466 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 59195 59720 safe_time ceph rtd unknown /proc/59195/task/59720/root (readlink: Permission denied) 2024-09-21T03:42:13.466 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 59195 59720 safe_time ceph txt unknown /proc/59195/task/59720/exe (readlink: Permission denied) 2024-09-21T03:42:13.466 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 59195 59720 safe_time ceph NOFD /proc/59195/task/59720/fd (opendir: Permission denied) 2024-09-21T03:42:13.466 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 59195 59721 safe_time ceph cwd unknown /proc/59195/task/59721/cwd (readlink: Permission denied) 2024-09-21T03:42:13.466 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 59195 59721 safe_time ceph rtd unknown /proc/59195/task/59721/root (readlink: Permission denied) 2024-09-21T03:42:13.466 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 59195 59721 safe_time ceph txt unknown /proc/59195/task/59721/exe (readlink: Permission denied) 2024-09-21T03:42:13.466 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 59195 59721 safe_time ceph NOFD /proc/59195/task/59721/fd (opendir: Permission denied) 2024-09-21T03:42:13.466 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 59195 59722 fn_anonym ceph cwd unknown /proc/59195/task/59722/cwd (readlink: Permission denied) 2024-09-21T03:42:13.466 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 59195 59722 fn_anonym ceph rtd unknown /proc/59195/task/59722/root (readlink: Permission denied) 2024-09-21T03:42:13.467 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 59195 59722 fn_anonym ceph txt unknown /proc/59195/task/59722/exe (readlink: Permission denied) 2024-09-21T03:42:13.467 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 59195 59722 fn_anonym ceph NOFD /proc/59195/task/59722/fd (opendir: Permission denied) 2024-09-21T03:42:13.467 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 59195 59793 rocksdb:l ceph cwd unknown /proc/59195/task/59793/cwd (readlink: Permission denied) 2024-09-21T03:42:13.467 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 59195 59793 rocksdb:l ceph rtd unknown /proc/59195/task/59793/root (readlink: Permission denied) 2024-09-21T03:42:13.467 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 59195 59793 rocksdb:l ceph txt unknown /proc/59195/task/59793/exe (readlink: Permission denied) 2024-09-21T03:42:13.467 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 59195 59793 rocksdb:l ceph NOFD /proc/59195/task/59793/fd (opendir: Permission denied) 2024-09-21T03:42:13.467 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 59195 59794 rocksdb:l ceph cwd unknown /proc/59195/task/59794/cwd (readlink: Permission denied) 2024-09-21T03:42:13.467 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 59195 59794 rocksdb:l ceph rtd unknown /proc/59195/task/59794/root (readlink: Permission denied) 2024-09-21T03:42:13.467 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 59195 59794 rocksdb:l ceph txt unknown /proc/59195/task/59794/exe (readlink: Permission denied) 2024-09-21T03:42:13.467 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 59195 59794 rocksdb:l ceph NOFD /proc/59195/task/59794/fd (opendir: Permission denied) 2024-09-21T03:42:13.467 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 59195 59795 rocksdb:l ceph cwd unknown /proc/59195/task/59795/cwd (readlink: Permission denied) 2024-09-21T03:42:13.467 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 59195 59795 rocksdb:l ceph rtd unknown /proc/59195/task/59795/root (readlink: Permission denied) 2024-09-21T03:42:13.467 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 59195 59795 rocksdb:l ceph txt unknown /proc/59195/task/59795/exe (readlink: Permission denied) 2024-09-21T03:42:13.468 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 59195 59795 rocksdb:l ceph NOFD /proc/59195/task/59795/fd (opendir: Permission denied) 2024-09-21T03:42:13.468 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 59195 59796 rocksdb:h ceph cwd unknown /proc/59195/task/59796/cwd (readlink: Permission denied) 2024-09-21T03:42:13.468 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 59195 59796 rocksdb:h ceph rtd unknown /proc/59195/task/59796/root (readlink: Permission denied) 2024-09-21T03:42:13.468 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 59195 59796 rocksdb:h ceph txt unknown /proc/59195/task/59796/exe (readlink: Permission denied) 2024-09-21T03:42:13.468 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 59195 59796 rocksdb:h ceph NOFD /proc/59195/task/59796/fd (opendir: Permission denied) 2024-09-21T03:42:13.468 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 59195 60182 bstore_ai ceph cwd unknown /proc/59195/task/60182/cwd (readlink: Permission denied) 2024-09-21T03:42:13.468 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 59195 60182 bstore_ai ceph rtd unknown /proc/59195/task/60182/root (readlink: Permission denied) 2024-09-21T03:42:13.468 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 59195 60182 bstore_ai ceph txt unknown /proc/59195/task/60182/exe (readlink: Permission denied) 2024-09-21T03:42:13.468 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 59195 60182 bstore_ai ceph NOFD /proc/59195/task/60182/fd (opendir: Permission denied) 2024-09-21T03:42:13.468 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 59195 60389 bstore_ai ceph cwd unknown /proc/59195/task/60389/cwd (readlink: Permission denied) 2024-09-21T03:42:13.468 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 59195 60389 bstore_ai ceph rtd unknown /proc/59195/task/60389/root (readlink: Permission denied) 2024-09-21T03:42:13.468 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 59195 60389 bstore_ai ceph txt unknown /proc/59195/task/60389/exe (readlink: Permission denied) 2024-09-21T03:42:13.469 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 59195 60389 bstore_ai ceph NOFD /proc/59195/task/60389/fd (opendir: Permission denied) 2024-09-21T03:42:13.469 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 59195 60571 ceph-osd ceph cwd unknown /proc/59195/task/60571/cwd (readlink: Permission denied) 2024-09-21T03:42:13.469 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 59195 60571 ceph-osd ceph rtd unknown /proc/59195/task/60571/root (readlink: Permission denied) 2024-09-21T03:42:13.469 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 59195 60571 ceph-osd ceph txt unknown /proc/59195/task/60571/exe (readlink: Permission denied) 2024-09-21T03:42:13.469 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 59195 60571 ceph-osd ceph NOFD /proc/59195/task/60571/fd (opendir: Permission denied) 2024-09-21T03:42:13.469 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 59195 60572 cfin ceph cwd unknown /proc/59195/task/60572/cwd (readlink: Permission denied) 2024-09-21T03:42:13.469 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 59195 60572 cfin ceph rtd unknown /proc/59195/task/60572/root (readlink: Permission denied) 2024-09-21T03:42:13.469 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 59195 60572 cfin ceph txt unknown /proc/59195/task/60572/exe (readlink: Permission denied) 2024-09-21T03:42:13.469 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 59195 60572 cfin ceph NOFD /proc/59195/task/60572/fd (opendir: Permission denied) 2024-09-21T03:42:13.469 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 59195 60573 bstore_kv ceph cwd unknown /proc/59195/task/60573/cwd (readlink: Permission denied) 2024-09-21T03:42:13.469 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 59195 60573 bstore_kv ceph rtd unknown /proc/59195/task/60573/root (readlink: Permission denied) 2024-09-21T03:42:13.469 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 59195 60573 bstore_kv ceph txt unknown /proc/59195/task/60573/exe (readlink: Permission denied) 2024-09-21T03:42:13.469 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 59195 60573 bstore_kv ceph NOFD /proc/59195/task/60573/fd (opendir: Permission denied) 2024-09-21T03:42:13.469 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 59195 60574 bstore_kv ceph cwd unknown /proc/59195/task/60574/cwd (readlink: Permission denied) 2024-09-21T03:42:13.470 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 59195 60574 bstore_kv ceph rtd unknown /proc/59195/task/60574/root (readlink: Permission denied) 2024-09-21T03:42:13.470 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 59195 60574 bstore_kv ceph txt unknown /proc/59195/task/60574/exe (readlink: Permission denied) 2024-09-21T03:42:13.470 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 59195 60574 bstore_kv ceph NOFD /proc/59195/task/60574/fd (opendir: Permission denied) 2024-09-21T03:42:13.470 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 59195 60575 bstore_me ceph cwd unknown /proc/59195/task/60575/cwd (readlink: Permission denied) 2024-09-21T03:42:13.470 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 59195 60575 bstore_me ceph rtd unknown /proc/59195/task/60575/root (readlink: Permission denied) 2024-09-21T03:42:13.470 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 59195 60575 bstore_me ceph txt unknown /proc/59195/task/60575/exe (readlink: Permission denied) 2024-09-21T03:42:13.470 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 59195 60575 bstore_me ceph NOFD /proc/59195/task/60575/fd (opendir: Permission denied) 2024-09-21T03:42:13.470 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 59195 60576 ms_dispat ceph cwd unknown /proc/59195/task/60576/cwd (readlink: Permission denied) 2024-09-21T03:42:13.470 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 59195 60576 ms_dispat ceph rtd unknown /proc/59195/task/60576/root (readlink: Permission denied) 2024-09-21T03:42:13.470 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 59195 60576 ms_dispat ceph txt unknown /proc/59195/task/60576/exe (readlink: Permission denied) 2024-09-21T03:42:13.470 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 59195 60576 ms_dispat ceph NOFD /proc/59195/task/60576/fd (opendir: Permission denied) 2024-09-21T03:42:13.470 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 59195 60577 ms_local ceph cwd unknown /proc/59195/task/60577/cwd (readlink: Permission denied) 2024-09-21T03:42:13.470 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 59195 60577 ms_local ceph rtd unknown /proc/59195/task/60577/root (readlink: Permission denied) 2024-09-21T03:42:13.470 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 59195 60577 ms_local ceph txt unknown /proc/59195/task/60577/exe (readlink: Permission denied) 2024-09-21T03:42:13.471 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 59195 60577 ms_local ceph NOFD /proc/59195/task/60577/fd (opendir: Permission denied) 2024-09-21T03:42:13.471 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 59195 60578 safe_time ceph cwd unknown /proc/59195/task/60578/cwd (readlink: Permission denied) 2024-09-21T03:42:13.471 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 59195 60578 safe_time ceph rtd unknown /proc/59195/task/60578/root (readlink: Permission denied) 2024-09-21T03:42:13.471 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 59195 60578 safe_time ceph txt unknown /proc/59195/task/60578/exe (readlink: Permission denied) 2024-09-21T03:42:13.471 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 59195 60578 safe_time ceph NOFD /proc/59195/task/60578/fd (opendir: Permission denied) 2024-09-21T03:42:13.471 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 59195 60579 safe_time ceph cwd unknown /proc/59195/task/60579/cwd (readlink: Permission denied) 2024-09-21T03:42:13.471 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 59195 60579 safe_time ceph rtd unknown /proc/59195/task/60579/root (readlink: Permission denied) 2024-09-21T03:42:13.471 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 59195 60579 safe_time ceph txt unknown /proc/59195/task/60579/exe (readlink: Permission denied) 2024-09-21T03:42:13.471 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 59195 60579 safe_time ceph NOFD /proc/59195/task/60579/fd (opendir: Permission denied) 2024-09-21T03:42:13.471 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 59195 60580 ms_dispat ceph cwd unknown /proc/59195/task/60580/cwd (readlink: Permission denied) 2024-09-21T03:42:13.471 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 59195 60580 ms_dispat ceph rtd unknown /proc/59195/task/60580/root (readlink: Permission denied) 2024-09-21T03:42:13.471 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 59195 60580 ms_dispat ceph txt unknown /proc/59195/task/60580/exe (readlink: Permission denied) 2024-09-21T03:42:13.471 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 59195 60580 ms_dispat ceph NOFD /proc/59195/task/60580/fd (opendir: Permission denied) 2024-09-21T03:42:13.471 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 59195 60581 ms_local ceph cwd unknown /proc/59195/task/60581/cwd (readlink: Permission denied) 2024-09-21T03:42:13.471 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 59195 60581 ms_local ceph rtd unknown /proc/59195/task/60581/root (readlink: Permission denied) 2024-09-21T03:42:13.472 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 59195 60581 ms_local ceph txt unknown /proc/59195/task/60581/exe (readlink: Permission denied) 2024-09-21T03:42:13.472 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 59195 60581 ms_local ceph NOFD /proc/59195/task/60581/fd (opendir: Permission denied) 2024-09-21T03:42:13.472 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 59195 60582 ms_dispat ceph cwd unknown /proc/59195/task/60582/cwd (readlink: Permission denied) 2024-09-21T03:42:13.472 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 59195 60582 ms_dispat ceph rtd unknown /proc/59195/task/60582/root (readlink: Permission denied) 2024-09-21T03:42:13.472 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 59195 60582 ms_dispat ceph txt unknown /proc/59195/task/60582/exe (readlink: Permission denied) 2024-09-21T03:42:13.472 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 59195 60582 ms_dispat ceph NOFD /proc/59195/task/60582/fd (opendir: Permission denied) 2024-09-21T03:42:13.472 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 59195 60583 ms_local ceph cwd unknown /proc/59195/task/60583/cwd (readlink: Permission denied) 2024-09-21T03:42:13.472 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 59195 60583 ms_local ceph rtd unknown /proc/59195/task/60583/root (readlink: Permission denied) 2024-09-21T03:42:13.472 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 59195 60583 ms_local ceph txt unknown /proc/59195/task/60583/exe (readlink: Permission denied) 2024-09-21T03:42:13.472 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 59195 60583 ms_local ceph NOFD /proc/59195/task/60583/fd (opendir: Permission denied) 2024-09-21T03:42:13.472 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 59195 60584 ms_dispat ceph cwd unknown /proc/59195/task/60584/cwd (readlink: Permission denied) 2024-09-21T03:42:13.472 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 59195 60584 ms_dispat ceph rtd unknown /proc/59195/task/60584/root (readlink: Permission denied) 2024-09-21T03:42:13.472 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 59195 60584 ms_dispat ceph txt unknown /proc/59195/task/60584/exe (readlink: Permission denied) 2024-09-21T03:42:13.472 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 59195 60584 ms_dispat ceph NOFD /proc/59195/task/60584/fd (opendir: Permission denied) 2024-09-21T03:42:13.472 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 59195 60585 ms_local ceph cwd unknown /proc/59195/task/60585/cwd (readlink: Permission denied) 2024-09-21T03:42:13.472 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 59195 60585 ms_local ceph rtd unknown /proc/59195/task/60585/root (readlink: Permission denied) 2024-09-21T03:42:13.473 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 59195 60585 ms_local ceph txt unknown /proc/59195/task/60585/exe (readlink: Permission denied) 2024-09-21T03:42:13.473 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 59195 60585 ms_local ceph NOFD /proc/59195/task/60585/fd (opendir: Permission denied) 2024-09-21T03:42:13.473 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 59195 60586 ms_dispat ceph cwd unknown /proc/59195/task/60586/cwd (readlink: Permission denied) 2024-09-21T03:42:13.473 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 59195 60586 ms_dispat ceph rtd unknown /proc/59195/task/60586/root (readlink: Permission denied) 2024-09-21T03:42:13.473 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 59195 60586 ms_dispat ceph txt unknown /proc/59195/task/60586/exe (readlink: Permission denied) 2024-09-21T03:42:13.473 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 59195 60586 ms_dispat ceph NOFD /proc/59195/task/60586/fd (opendir: Permission denied) 2024-09-21T03:42:13.473 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 59195 60587 ms_local ceph cwd unknown /proc/59195/task/60587/cwd (readlink: Permission denied) 2024-09-21T03:42:13.473 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 59195 60587 ms_local ceph rtd unknown /proc/59195/task/60587/root (readlink: Permission denied) 2024-09-21T03:42:13.473 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 59195 60587 ms_local ceph txt unknown /proc/59195/task/60587/exe (readlink: Permission denied) 2024-09-21T03:42:13.473 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 59195 60587 ms_local ceph NOFD /proc/59195/task/60587/fd (opendir: Permission denied) 2024-09-21T03:42:13.473 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 59195 60588 ms_dispat ceph cwd unknown /proc/59195/task/60588/cwd (readlink: Permission denied) 2024-09-21T03:42:13.473 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 59195 60588 ms_dispat ceph rtd unknown /proc/59195/task/60588/root (readlink: Permission denied) 2024-09-21T03:42:13.473 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 59195 60588 ms_dispat ceph txt unknown /proc/59195/task/60588/exe (readlink: Permission denied) 2024-09-21T03:42:13.473 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 59195 60588 ms_dispat ceph NOFD /proc/59195/task/60588/fd (opendir: Permission denied) 2024-09-21T03:42:13.474 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 59195 60589 ms_local ceph cwd unknown /proc/59195/task/60589/cwd (readlink: Permission denied) 2024-09-21T03:42:13.474 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 59195 60589 ms_local ceph rtd unknown /proc/59195/task/60589/root (readlink: Permission denied) 2024-09-21T03:42:13.474 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 59195 60589 ms_local ceph txt unknown /proc/59195/task/60589/exe (readlink: Permission denied) 2024-09-21T03:42:13.474 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 59195 60589 ms_local ceph NOFD /proc/59195/task/60589/fd (opendir: Permission denied) 2024-09-21T03:42:13.474 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 59195 60590 ms_dispat ceph cwd unknown /proc/59195/task/60590/cwd (readlink: Permission denied) 2024-09-21T03:42:13.474 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 59195 60590 ms_dispat ceph rtd unknown /proc/59195/task/60590/root (readlink: Permission denied) 2024-09-21T03:42:13.474 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 59195 60590 ms_dispat ceph txt unknown /proc/59195/task/60590/exe (readlink: Permission denied) 2024-09-21T03:42:13.474 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 59195 60590 ms_dispat ceph NOFD /proc/59195/task/60590/fd (opendir: Permission denied) 2024-09-21T03:42:13.474 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 59195 60591 ms_local ceph cwd unknown /proc/59195/task/60591/cwd (readlink: Permission denied) 2024-09-21T03:42:13.474 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 59195 60591 ms_local ceph rtd unknown /proc/59195/task/60591/root (readlink: Permission denied) 2024-09-21T03:42:13.474 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 59195 60591 ms_local ceph txt unknown /proc/59195/task/60591/exe (readlink: Permission denied) 2024-09-21T03:42:13.474 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 59195 60591 ms_local ceph NOFD /proc/59195/task/60591/fd (opendir: Permission denied) 2024-09-21T03:42:13.474 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 59195 60592 fn_anonym ceph cwd unknown /proc/59195/task/60592/cwd (readlink: Permission denied) 2024-09-21T03:42:13.475 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 59195 60592 fn_anonym ceph rtd unknown /proc/59195/task/60592/root (readlink: Permission denied) 2024-09-21T03:42:13.475 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 59195 60592 fn_anonym ceph txt unknown /proc/59195/task/60592/exe (readlink: Permission denied) 2024-09-21T03:42:13.475 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 59195 60592 fn_anonym ceph NOFD /proc/59195/task/60592/fd (opendir: Permission denied) 2024-09-21T03:42:13.475 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 59195 60593 finisher ceph cwd unknown /proc/59195/task/60593/cwd (readlink: Permission denied) 2024-09-21T03:42:13.475 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 59195 60593 finisher ceph rtd unknown /proc/59195/task/60593/root (readlink: Permission denied) 2024-09-21T03:42:13.475 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 59195 60593 finisher ceph txt unknown /proc/59195/task/60593/exe (readlink: Permission denied) 2024-09-21T03:42:13.475 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 59195 60593 finisher ceph NOFD /proc/59195/task/60593/fd (opendir: Permission denied) 2024-09-21T03:42:13.475 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 59195 60594 safe_time ceph cwd unknown /proc/59195/task/60594/cwd (readlink: Permission denied) 2024-09-21T03:42:13.475 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 59195 60594 safe_time ceph rtd unknown /proc/59195/task/60594/root (readlink: Permission denied) 2024-09-21T03:42:13.475 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 59195 60594 safe_time ceph txt unknown /proc/59195/task/60594/exe (readlink: Permission denied) 2024-09-21T03:42:13.475 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 59195 60594 safe_time ceph NOFD /proc/59195/task/60594/fd (opendir: Permission denied) 2024-09-21T03:42:13.475 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 59195 60595 safe_time ceph cwd unknown /proc/59195/task/60595/cwd (readlink: Permission denied) 2024-09-21T03:42:13.476 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 59195 60595 safe_time ceph rtd unknown /proc/59195/task/60595/root (readlink: Permission denied) 2024-09-21T03:42:13.476 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 59195 60595 safe_time ceph txt unknown /proc/59195/task/60595/exe (readlink: Permission denied) 2024-09-21T03:42:13.476 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 59195 60595 safe_time ceph NOFD /proc/59195/task/60595/fd (opendir: Permission denied) 2024-09-21T03:42:13.476 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 59195 60596 ceph-osd ceph cwd unknown /proc/59195/task/60596/cwd (readlink: Permission denied) 2024-09-21T03:42:13.476 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 59195 60596 ceph-osd ceph rtd unknown /proc/59195/task/60596/root (readlink: Permission denied) 2024-09-21T03:42:13.476 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 59195 60596 ceph-osd ceph txt unknown /proc/59195/task/60596/exe (readlink: Permission denied) 2024-09-21T03:42:13.476 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 59195 60596 ceph-osd ceph NOFD /proc/59195/task/60596/fd (opendir: Permission denied) 2024-09-21T03:42:13.476 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 59195 60597 osd_srv_a ceph cwd unknown /proc/59195/task/60597/cwd (readlink: Permission denied) 2024-09-21T03:42:13.476 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 59195 60597 osd_srv_a ceph rtd unknown /proc/59195/task/60597/root (readlink: Permission denied) 2024-09-21T03:42:13.476 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 59195 60597 osd_srv_a ceph txt unknown /proc/59195/task/60597/exe (readlink: Permission denied) 2024-09-21T03:42:13.476 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 59195 60597 osd_srv_a ceph NOFD /proc/59195/task/60597/fd (opendir: Permission denied) 2024-09-21T03:42:13.476 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 59195 60598 tp_osd_tp ceph cwd unknown /proc/59195/task/60598/cwd (readlink: Permission denied) 2024-09-21T03:42:13.476 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 59195 60598 tp_osd_tp ceph rtd unknown /proc/59195/task/60598/root (readlink: Permission denied) 2024-09-21T03:42:13.476 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 59195 60598 tp_osd_tp ceph txt unknown /proc/59195/task/60598/exe (readlink: Permission denied) 2024-09-21T03:42:13.477 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 59195 60598 tp_osd_tp ceph NOFD /proc/59195/task/60598/fd (opendir: Permission denied) 2024-09-21T03:42:13.477 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 59195 60599 tp_osd_tp ceph cwd unknown /proc/59195/task/60599/cwd (readlink: Permission denied) 2024-09-21T03:42:13.477 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 59195 60599 tp_osd_tp ceph rtd unknown /proc/59195/task/60599/root (readlink: Permission denied) 2024-09-21T03:42:13.477 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 59195 60599 tp_osd_tp ceph txt unknown /proc/59195/task/60599/exe (readlink: Permission denied) 2024-09-21T03:42:13.477 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 59195 60599 tp_osd_tp ceph NOFD /proc/59195/task/60599/fd (opendir: Permission denied) 2024-09-21T03:42:13.477 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 59195 60600 tp_osd_tp ceph cwd unknown /proc/59195/task/60600/cwd (readlink: Permission denied) 2024-09-21T03:42:13.477 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 59195 60600 tp_osd_tp ceph rtd unknown /proc/59195/task/60600/root (readlink: Permission denied) 2024-09-21T03:42:13.477 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 59195 60600 tp_osd_tp ceph txt unknown /proc/59195/task/60600/exe (readlink: Permission denied) 2024-09-21T03:42:13.477 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 59195 60600 tp_osd_tp ceph NOFD /proc/59195/task/60600/fd (opendir: Permission denied) 2024-09-21T03:42:13.477 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 59195 60601 tp_osd_tp ceph cwd unknown /proc/59195/task/60601/cwd (readlink: Permission denied) 2024-09-21T03:42:13.477 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 59195 60601 tp_osd_tp ceph rtd unknown /proc/59195/task/60601/root (readlink: Permission denied) 2024-09-21T03:42:13.477 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 59195 60601 tp_osd_tp ceph txt unknown /proc/59195/task/60601/exe (readlink: Permission denied) 2024-09-21T03:42:13.477 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 59195 60601 tp_osd_tp ceph NOFD /proc/59195/task/60601/fd (opendir: Permission denied) 2024-09-21T03:42:13.477 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 59195 60602 tp_osd_tp ceph cwd unknown /proc/59195/task/60602/cwd (readlink: Permission denied) 2024-09-21T03:42:13.478 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 59195 60602 tp_osd_tp ceph rtd unknown /proc/59195/task/60602/root (readlink: Permission denied) 2024-09-21T03:42:13.478 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 59195 60602 tp_osd_tp ceph txt unknown /proc/59195/task/60602/exe (readlink: Permission denied) 2024-09-21T03:42:13.478 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 59195 60602 tp_osd_tp ceph NOFD /proc/59195/task/60602/fd (opendir: Permission denied) 2024-09-21T03:42:13.478 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 59195 60603 tp_osd_tp ceph cwd unknown /proc/59195/task/60603/cwd (readlink: Permission denied) 2024-09-21T03:42:13.478 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 59195 60603 tp_osd_tp ceph rtd unknown /proc/59195/task/60603/root (readlink: Permission denied) 2024-09-21T03:42:13.478 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 59195 60603 tp_osd_tp ceph txt unknown /proc/59195/task/60603/exe (readlink: Permission denied) 2024-09-21T03:42:13.478 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 59195 60603 tp_osd_tp ceph NOFD /proc/59195/task/60603/fd (opendir: Permission denied) 2024-09-21T03:42:13.478 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 59195 60604 tp_osd_tp ceph cwd unknown /proc/59195/task/60604/cwd (readlink: Permission denied) 2024-09-21T03:42:13.478 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 59195 60604 tp_osd_tp ceph rtd unknown /proc/59195/task/60604/root (readlink: Permission denied) 2024-09-21T03:42:13.478 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 59195 60604 tp_osd_tp ceph txt unknown /proc/59195/task/60604/exe (readlink: Permission denied) 2024-09-21T03:42:13.478 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 59195 60604 tp_osd_tp ceph NOFD /proc/59195/task/60604/fd (opendir: Permission denied) 2024-09-21T03:42:13.479 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 59195 60605 tp_osd_tp ceph cwd unknown /proc/59195/task/60605/cwd (readlink: Permission denied) 2024-09-21T03:42:13.479 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 59195 60605 tp_osd_tp ceph rtd unknown /proc/59195/task/60605/root (readlink: Permission denied) 2024-09-21T03:42:13.479 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 59195 60605 tp_osd_tp ceph txt unknown /proc/59195/task/60605/exe (readlink: Permission denied) 2024-09-21T03:42:13.479 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 59195 60605 tp_osd_tp ceph NOFD /proc/59195/task/60605/fd (opendir: Permission denied) 2024-09-21T03:42:13.479 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 59195 60606 tp_osd_tp ceph cwd unknown /proc/59195/task/60606/cwd (readlink: Permission denied) 2024-09-21T03:42:13.479 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 59195 60606 tp_osd_tp ceph rtd unknown /proc/59195/task/60606/root (readlink: Permission denied) 2024-09-21T03:42:13.479 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 59195 60606 tp_osd_tp ceph txt unknown /proc/59195/task/60606/exe (readlink: Permission denied) 2024-09-21T03:42:13.479 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 59195 60606 tp_osd_tp ceph NOFD /proc/59195/task/60606/fd (opendir: Permission denied) 2024-09-21T03:42:13.479 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 59195 60607 tp_osd_tp ceph cwd unknown /proc/59195/task/60607/cwd (readlink: Permission denied) 2024-09-21T03:42:13.479 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 59195 60607 tp_osd_tp ceph rtd unknown /proc/59195/task/60607/root (readlink: Permission denied) 2024-09-21T03:42:13.479 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 59195 60607 tp_osd_tp ceph txt unknown /proc/59195/task/60607/exe (readlink: Permission denied) 2024-09-21T03:42:13.479 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 59195 60607 tp_osd_tp ceph NOFD /proc/59195/task/60607/fd (opendir: Permission denied) 2024-09-21T03:42:13.480 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 59195 60608 tp_osd_tp ceph cwd unknown /proc/59195/task/60608/cwd (readlink: Permission denied) 2024-09-21T03:42:13.480 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 59195 60608 tp_osd_tp ceph rtd unknown /proc/59195/task/60608/root (readlink: Permission denied) 2024-09-21T03:42:13.480 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 59195 60608 tp_osd_tp ceph txt unknown /proc/59195/task/60608/exe (readlink: Permission denied) 2024-09-21T03:42:13.480 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 59195 60608 tp_osd_tp ceph NOFD /proc/59195/task/60608/fd (opendir: Permission denied) 2024-09-21T03:42:13.480 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 59195 60609 tp_osd_tp ceph cwd unknown /proc/59195/task/60609/cwd (readlink: Permission denied) 2024-09-21T03:42:13.480 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 59195 60609 tp_osd_tp ceph rtd unknown /proc/59195/task/60609/root (readlink: Permission denied) 2024-09-21T03:42:13.480 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 59195 60609 tp_osd_tp ceph txt unknown /proc/59195/task/60609/exe (readlink: Permission denied) 2024-09-21T03:42:13.480 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 59195 60609 tp_osd_tp ceph NOFD /proc/59195/task/60609/fd (opendir: Permission denied) 2024-09-21T03:42:13.480 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 59195 60610 tp_osd_tp ceph cwd unknown /proc/59195/task/60610/cwd (readlink: Permission denied) 2024-09-21T03:42:13.480 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 59195 60610 tp_osd_tp ceph rtd unknown /proc/59195/task/60610/root (readlink: Permission denied) 2024-09-21T03:42:13.480 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 59195 60610 tp_osd_tp ceph txt unknown /proc/59195/task/60610/exe (readlink: Permission denied) 2024-09-21T03:42:13.480 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 59195 60610 tp_osd_tp ceph NOFD /proc/59195/task/60610/fd (opendir: Permission denied) 2024-09-21T03:42:13.481 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 59195 60611 tp_osd_tp ceph cwd unknown /proc/59195/task/60611/cwd (readlink: Permission denied) 2024-09-21T03:42:13.481 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 59195 60611 tp_osd_tp ceph rtd unknown /proc/59195/task/60611/root (readlink: Permission denied) 2024-09-21T03:42:13.481 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 59195 60611 tp_osd_tp ceph txt unknown /proc/59195/task/60611/exe (readlink: Permission denied) 2024-09-21T03:42:13.481 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 59195 60611 tp_osd_tp ceph NOFD /proc/59195/task/60611/fd (opendir: Permission denied) 2024-09-21T03:42:13.481 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 59195 60612 tp_osd_tp ceph cwd unknown /proc/59195/task/60612/cwd (readlink: Permission denied) 2024-09-21T03:42:13.481 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 59195 60612 tp_osd_tp ceph rtd unknown /proc/59195/task/60612/root (readlink: Permission denied) 2024-09-21T03:42:13.481 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 59195 60612 tp_osd_tp ceph txt unknown /proc/59195/task/60612/exe (readlink: Permission denied) 2024-09-21T03:42:13.481 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 59195 60612 tp_osd_tp ceph NOFD /proc/59195/task/60612/fd (opendir: Permission denied) 2024-09-21T03:42:13.481 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 59195 60613 tp_osd_tp ceph cwd unknown /proc/59195/task/60613/cwd (readlink: Permission denied) 2024-09-21T03:42:13.481 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 59195 60613 tp_osd_tp ceph rtd unknown /proc/59195/task/60613/root (readlink: Permission denied) 2024-09-21T03:42:13.481 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 59195 60613 tp_osd_tp ceph txt unknown /proc/59195/task/60613/exe (readlink: Permission denied) 2024-09-21T03:42:13.481 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 59195 60613 tp_osd_tp ceph NOFD /proc/59195/task/60613/fd (opendir: Permission denied) 2024-09-21T03:42:13.481 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 59195 60614 osd_srv_h ceph cwd unknown /proc/59195/task/60614/cwd (readlink: Permission denied) 2024-09-21T03:42:13.481 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 59195 60614 osd_srv_h ceph rtd unknown /proc/59195/task/60614/root (readlink: Permission denied) 2024-09-21T03:42:13.482 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 59195 60614 osd_srv_h ceph txt unknown /proc/59195/task/60614/exe (readlink: Permission denied) 2024-09-21T03:42:13.482 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 59195 60614 osd_srv_h ceph NOFD /proc/59195/task/60614/fd (opendir: Permission denied) 2024-09-21T03:42:13.482 INFO:tasks.workunit.client.0.smithi089.stdout:sudo 59740 root cwd unknown /proc/59740/cwd (readlink: Permission denied) 2024-09-21T03:42:13.482 INFO:tasks.workunit.client.0.smithi089.stdout:sudo 59740 root rtd unknown /proc/59740/root (readlink: Permission denied) 2024-09-21T03:42:13.482 INFO:tasks.workunit.client.0.smithi089.stdout:sudo 59740 root txt unknown /proc/59740/exe (readlink: Permission denied) 2024-09-21T03:42:13.482 INFO:tasks.workunit.client.0.smithi089.stdout:sudo 59740 root NOFD /proc/59740/fd (opendir: Permission denied) 2024-09-21T03:42:13.482 INFO:tasks.workunit.client.0.smithi089.stdout:journalct 59771 root cwd unknown /proc/59771/cwd (readlink: Permission denied) 2024-09-21T03:42:13.482 INFO:tasks.workunit.client.0.smithi089.stdout:journalct 59771 root rtd unknown /proc/59771/root (readlink: Permission denied) 2024-09-21T03:42:13.482 INFO:tasks.workunit.client.0.smithi089.stdout:journalct 59771 root txt unknown /proc/59771/exe (readlink: Permission denied) 2024-09-21T03:42:13.482 INFO:tasks.workunit.client.0.smithi089.stdout:journalct 59771 root NOFD /proc/59771/fd (opendir: Permission denied) 2024-09-21T03:42:13.482 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/u 62755 root cwd unknown /proc/62755/cwd (readlink: Permission denied) 2024-09-21T03:42:13.482 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/u 62755 root rtd unknown /proc/62755/root (readlink: Permission denied) 2024-09-21T03:42:13.482 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/u 62755 root txt unknown /proc/62755/exe (readlink: Permission denied) 2024-09-21T03:42:13.482 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/u 62755 root NOFD /proc/62755/fd (opendir: Permission denied) 2024-09-21T03:42:13.482 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/0 62866 root cwd unknown /proc/62866/cwd (readlink: Permission denied) 2024-09-21T03:42:13.483 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/0 62866 root rtd unknown /proc/62866/root (readlink: Permission denied) 2024-09-21T03:42:13.483 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/0 62866 root txt unknown /proc/62866/exe (readlink: Permission denied) 2024-09-21T03:42:13.483 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/0 62866 root NOFD /proc/62866/fd (opendir: Permission denied) 2024-09-21T03:42:13.483 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/7 62867 root cwd unknown /proc/62867/cwd (readlink: Permission denied) 2024-09-21T03:42:13.483 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/7 62867 root rtd unknown /proc/62867/root (readlink: Permission denied) 2024-09-21T03:42:13.483 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/7 62867 root txt unknown /proc/62867/exe (readlink: Permission denied) 2024-09-21T03:42:13.483 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/7 62867 root NOFD /proc/62867/fd (opendir: Permission denied) 2024-09-21T03:42:13.483 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/2 64827 root cwd unknown /proc/64827/cwd (readlink: Permission denied) 2024-09-21T03:42:13.483 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/2 64827 root rtd unknown /proc/64827/root (readlink: Permission denied) 2024-09-21T03:42:13.483 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/2 64827 root txt unknown /proc/64827/exe (readlink: Permission denied) 2024-09-21T03:42:13.483 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/2 64827 root NOFD /proc/64827/fd (opendir: Permission denied) 2024-09-21T03:42:13.483 INFO:tasks.workunit.client.0.smithi089.stdout:conmon 65404 root cwd unknown /proc/65404/cwd (readlink: Permission denied) 2024-09-21T03:42:13.484 INFO:tasks.workunit.client.0.smithi089.stdout:conmon 65404 root rtd unknown /proc/65404/root (readlink: Permission denied) 2024-09-21T03:42:13.484 INFO:tasks.workunit.client.0.smithi089.stdout:conmon 65404 root txt unknown /proc/65404/exe (readlink: Permission denied) 2024-09-21T03:42:13.484 INFO:tasks.workunit.client.0.smithi089.stdout:conmon 65404 root NOFD /proc/65404/fd (opendir: Permission denied) 2024-09-21T03:42:13.484 INFO:tasks.workunit.client.0.smithi089.stdout:podman-in 65406 root cwd unknown /proc/65406/cwd (readlink: Permission denied) 2024-09-21T03:42:13.484 INFO:tasks.workunit.client.0.smithi089.stdout:podman-in 65406 root rtd unknown /proc/65406/root (readlink: Permission denied) 2024-09-21T03:42:13.484 INFO:tasks.workunit.client.0.smithi089.stdout:podman-in 65406 root txt unknown /proc/65406/exe (readlink: Permission denied) 2024-09-21T03:42:13.484 INFO:tasks.workunit.client.0.smithi089.stdout:podman-in 65406 root NOFD /proc/65406/fd (opendir: Permission denied) 2024-09-21T03:42:13.484 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 65408 ceph cwd unknown /proc/65408/cwd (readlink: Permission denied) 2024-09-21T03:42:13.484 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 65408 ceph rtd unknown /proc/65408/root (readlink: Permission denied) 2024-09-21T03:42:13.484 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 65408 ceph txt unknown /proc/65408/exe (readlink: Permission denied) 2024-09-21T03:42:13.484 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 65408 ceph NOFD /proc/65408/fd (opendir: Permission denied) 2024-09-21T03:42:13.484 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 65408 65409 log ceph cwd unknown /proc/65408/task/65409/cwd (readlink: Permission denied) 2024-09-21T03:42:13.485 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 65408 65409 log ceph rtd unknown /proc/65408/task/65409/root (readlink: Permission denied) 2024-09-21T03:42:13.485 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 65408 65409 log ceph txt unknown /proc/65408/task/65409/exe (readlink: Permission denied) 2024-09-21T03:42:13.485 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 65408 65409 log ceph NOFD /proc/65408/task/65409/fd (opendir: Permission denied) 2024-09-21T03:42:13.485 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 65408 65411 msgr-work ceph cwd unknown /proc/65408/task/65411/cwd (readlink: Permission denied) 2024-09-21T03:42:13.485 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 65408 65411 msgr-work ceph rtd unknown /proc/65408/task/65411/root (readlink: Permission denied) 2024-09-21T03:42:13.485 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 65408 65411 msgr-work ceph txt unknown /proc/65408/task/65411/exe (readlink: Permission denied) 2024-09-21T03:42:13.485 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 65408 65411 msgr-work ceph NOFD /proc/65408/task/65411/fd (opendir: Permission denied) 2024-09-21T03:42:13.485 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 65408 65412 msgr-work ceph cwd unknown /proc/65408/task/65412/cwd (readlink: Permission denied) 2024-09-21T03:42:13.485 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 65408 65412 msgr-work ceph rtd unknown /proc/65408/task/65412/root (readlink: Permission denied) 2024-09-21T03:42:13.485 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 65408 65412 msgr-work ceph txt unknown /proc/65408/task/65412/exe (readlink: Permission denied) 2024-09-21T03:42:13.485 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 65408 65412 msgr-work ceph NOFD /proc/65408/task/65412/fd (opendir: Permission denied) 2024-09-21T03:42:13.485 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 65408 65413 msgr-work ceph cwd unknown /proc/65408/task/65413/cwd (readlink: Permission denied) 2024-09-21T03:42:13.485 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 65408 65413 msgr-work ceph rtd unknown /proc/65408/task/65413/root (readlink: Permission denied) 2024-09-21T03:42:13.485 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 65408 65413 msgr-work ceph txt unknown /proc/65408/task/65413/exe (readlink: Permission denied) 2024-09-21T03:42:13.485 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 65408 65413 msgr-work ceph NOFD /proc/65408/task/65413/fd (opendir: Permission denied) 2024-09-21T03:42:13.486 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 65408 65417 service ceph cwd unknown /proc/65408/task/65417/cwd (readlink: Permission denied) 2024-09-21T03:42:13.486 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 65408 65417 service ceph rtd unknown /proc/65408/task/65417/root (readlink: Permission denied) 2024-09-21T03:42:13.486 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 65408 65417 service ceph txt unknown /proc/65408/task/65417/exe (readlink: Permission denied) 2024-09-21T03:42:13.486 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 65408 65417 service ceph NOFD /proc/65408/task/65417/fd (opendir: Permission denied) 2024-09-21T03:42:13.486 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 65408 65418 admin_soc ceph cwd unknown /proc/65408/task/65418/cwd (readlink: Permission denied) 2024-09-21T03:42:13.486 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 65408 65418 admin_soc ceph rtd unknown /proc/65408/task/65418/root (readlink: Permission denied) 2024-09-21T03:42:13.486 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 65408 65418 admin_soc ceph txt unknown /proc/65408/task/65418/exe (readlink: Permission denied) 2024-09-21T03:42:13.486 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 65408 65418 admin_soc ceph NOFD /proc/65408/task/65418/fd (opendir: Permission denied) 2024-09-21T03:42:13.486 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 65408 65704 signal_ha ceph cwd unknown /proc/65408/task/65704/cwd (readlink: Permission denied) 2024-09-21T03:42:13.486 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 65408 65704 signal_ha ceph rtd unknown /proc/65408/task/65704/root (readlink: Permission denied) 2024-09-21T03:42:13.486 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 65408 65704 signal_ha ceph txt unknown /proc/65408/task/65704/exe (readlink: Permission denied) 2024-09-21T03:42:13.486 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 65408 65704 signal_ha ceph NOFD /proc/65408/task/65704/fd (opendir: Permission denied) 2024-09-21T03:42:13.486 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 65408 65705 io_contex ceph cwd unknown /proc/65408/task/65705/cwd (readlink: Permission denied) 2024-09-21T03:42:13.487 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 65408 65705 io_contex ceph rtd unknown /proc/65408/task/65705/root (readlink: Permission denied) 2024-09-21T03:42:13.487 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 65408 65705 io_contex ceph txt unknown /proc/65408/task/65705/exe (readlink: Permission denied) 2024-09-21T03:42:13.487 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 65408 65705 io_contex ceph NOFD /proc/65408/task/65705/fd (opendir: Permission denied) 2024-09-21T03:42:13.487 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 65408 65706 io_contex ceph cwd unknown /proc/65408/task/65706/cwd (readlink: Permission denied) 2024-09-21T03:42:13.487 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 65408 65706 io_contex ceph rtd unknown /proc/65408/task/65706/root (readlink: Permission denied) 2024-09-21T03:42:13.487 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 65408 65706 io_contex ceph txt unknown /proc/65408/task/65706/exe (readlink: Permission denied) 2024-09-21T03:42:13.487 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 65408 65706 io_contex ceph NOFD /proc/65408/task/65706/fd (opendir: Permission denied) 2024-09-21T03:42:13.487 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 65408 65708 OpHistory ceph cwd unknown /proc/65408/task/65708/cwd (readlink: Permission denied) 2024-09-21T03:42:13.487 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 65408 65708 OpHistory ceph rtd unknown /proc/65408/task/65708/root (readlink: Permission denied) 2024-09-21T03:42:13.487 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 65408 65708 OpHistory ceph txt unknown /proc/65408/task/65708/exe (readlink: Permission denied) 2024-09-21T03:42:13.487 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 65408 65708 OpHistory ceph NOFD /proc/65408/task/65708/fd (opendir: Permission denied) 2024-09-21T03:42:13.487 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 65408 65709 ceph_time ceph cwd unknown /proc/65408/task/65709/cwd (readlink: Permission denied) 2024-09-21T03:42:13.487 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 65408 65709 ceph_time ceph rtd unknown /proc/65408/task/65709/root (readlink: Permission denied) 2024-09-21T03:42:13.487 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 65408 65709 ceph_time ceph txt unknown /proc/65408/task/65709/exe (readlink: Permission denied) 2024-09-21T03:42:13.487 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 65408 65709 ceph_time ceph NOFD /proc/65408/task/65709/fd (opendir: Permission denied) 2024-09-21T03:42:13.488 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 65408 65971 ceph-osd ceph cwd unknown /proc/65408/task/65971/cwd (readlink: Permission denied) 2024-09-21T03:42:13.488 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 65408 65971 ceph-osd ceph rtd unknown /proc/65408/task/65971/root (readlink: Permission denied) 2024-09-21T03:42:13.488 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 65408 65971 ceph-osd ceph txt unknown /proc/65408/task/65971/exe (readlink: Permission denied) 2024-09-21T03:42:13.488 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 65408 65971 ceph-osd ceph NOFD /proc/65408/task/65971/fd (opendir: Permission denied) 2024-09-21T03:42:13.488 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 65408 65972 safe_time ceph cwd unknown /proc/65408/task/65972/cwd (readlink: Permission denied) 2024-09-21T03:42:13.488 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 65408 65972 safe_time ceph rtd unknown /proc/65408/task/65972/root (readlink: Permission denied) 2024-09-21T03:42:13.488 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 65408 65972 safe_time ceph txt unknown /proc/65408/task/65972/exe (readlink: Permission denied) 2024-09-21T03:42:13.488 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 65408 65972 safe_time ceph NOFD /proc/65408/task/65972/fd (opendir: Permission denied) 2024-09-21T03:42:13.488 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 65408 65973 safe_time ceph cwd unknown /proc/65408/task/65973/cwd (readlink: Permission denied) 2024-09-21T03:42:13.488 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 65408 65973 safe_time ceph rtd unknown /proc/65408/task/65973/root (readlink: Permission denied) 2024-09-21T03:42:13.488 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 65408 65973 safe_time ceph txt unknown /proc/65408/task/65973/exe (readlink: Permission denied) 2024-09-21T03:42:13.488 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 65408 65973 safe_time ceph NOFD /proc/65408/task/65973/fd (opendir: Permission denied) 2024-09-21T03:42:13.488 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 65408 65974 safe_time ceph cwd unknown /proc/65408/task/65974/cwd (readlink: Permission denied) 2024-09-21T03:42:13.488 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 65408 65974 safe_time ceph rtd unknown /proc/65408/task/65974/root (readlink: Permission denied) 2024-09-21T03:42:13.489 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 65408 65974 safe_time ceph txt unknown /proc/65408/task/65974/exe (readlink: Permission denied) 2024-09-21T03:42:13.489 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 65408 65974 safe_time ceph NOFD /proc/65408/task/65974/fd (opendir: Permission denied) 2024-09-21T03:42:13.489 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 65408 65975 safe_time ceph cwd unknown /proc/65408/task/65975/cwd (readlink: Permission denied) 2024-09-21T03:42:13.489 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 65408 65975 safe_time ceph rtd unknown /proc/65408/task/65975/root (readlink: Permission denied) 2024-09-21T03:42:13.489 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 65408 65975 safe_time ceph txt unknown /proc/65408/task/65975/exe (readlink: Permission denied) 2024-09-21T03:42:13.489 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 65408 65975 safe_time ceph NOFD /proc/65408/task/65975/fd (opendir: Permission denied) 2024-09-21T03:42:13.489 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 65408 65976 fn_anonym ceph cwd unknown /proc/65408/task/65976/cwd (readlink: Permission denied) 2024-09-21T03:42:13.489 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 65408 65976 fn_anonym ceph rtd unknown /proc/65408/task/65976/root (readlink: Permission denied) 2024-09-21T03:42:13.489 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 65408 65976 fn_anonym ceph txt unknown /proc/65408/task/65976/exe (readlink: Permission denied) 2024-09-21T03:42:13.489 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 65408 65976 fn_anonym ceph NOFD /proc/65408/task/65976/fd (opendir: Permission denied) 2024-09-21T03:42:13.489 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 65408 66040 rocksdb:l ceph cwd unknown /proc/65408/task/66040/cwd (readlink: Permission denied) 2024-09-21T03:42:13.489 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 65408 66040 rocksdb:l ceph rtd unknown /proc/65408/task/66040/root (readlink: Permission denied) 2024-09-21T03:42:13.489 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 65408 66040 rocksdb:l ceph txt unknown /proc/65408/task/66040/exe (readlink: Permission denied) 2024-09-21T03:42:13.489 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 65408 66040 rocksdb:l ceph NOFD /proc/65408/task/66040/fd (opendir: Permission denied) 2024-09-21T03:42:13.489 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 65408 66041 rocksdb:l ceph cwd unknown /proc/65408/task/66041/cwd (readlink: Permission denied) 2024-09-21T03:42:13.490 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 65408 66041 rocksdb:l ceph rtd unknown /proc/65408/task/66041/root (readlink: Permission denied) 2024-09-21T03:42:13.490 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 65408 66041 rocksdb:l ceph txt unknown /proc/65408/task/66041/exe (readlink: Permission denied) 2024-09-21T03:42:13.490 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 65408 66041 rocksdb:l ceph NOFD /proc/65408/task/66041/fd (opendir: Permission denied) 2024-09-21T03:42:13.490 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 65408 66042 rocksdb:l ceph cwd unknown /proc/65408/task/66042/cwd (readlink: Permission denied) 2024-09-21T03:42:13.490 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 65408 66042 rocksdb:l ceph rtd unknown /proc/65408/task/66042/root (readlink: Permission denied) 2024-09-21T03:42:13.490 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 65408 66042 rocksdb:l ceph txt unknown /proc/65408/task/66042/exe (readlink: Permission denied) 2024-09-21T03:42:13.490 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 65408 66042 rocksdb:l ceph NOFD /proc/65408/task/66042/fd (opendir: Permission denied) 2024-09-21T03:42:13.490 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 65408 66043 rocksdb:h ceph cwd unknown /proc/65408/task/66043/cwd (readlink: Permission denied) 2024-09-21T03:42:13.490 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 65408 66043 rocksdb:h ceph rtd unknown /proc/65408/task/66043/root (readlink: Permission denied) 2024-09-21T03:42:13.490 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 65408 66043 rocksdb:h ceph txt unknown /proc/65408/task/66043/exe (readlink: Permission denied) 2024-09-21T03:42:13.490 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 65408 66043 rocksdb:h ceph NOFD /proc/65408/task/66043/fd (opendir: Permission denied) 2024-09-21T03:42:13.490 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 65408 66443 bstore_ai ceph cwd unknown /proc/65408/task/66443/cwd (readlink: Permission denied) 2024-09-21T03:42:13.490 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 65408 66443 bstore_ai ceph rtd unknown /proc/65408/task/66443/root (readlink: Permission denied) 2024-09-21T03:42:13.490 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 65408 66443 bstore_ai ceph txt unknown /proc/65408/task/66443/exe (readlink: Permission denied) 2024-09-21T03:42:13.491 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 65408 66443 bstore_ai ceph NOFD /proc/65408/task/66443/fd (opendir: Permission denied) 2024-09-21T03:42:13.491 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 65408 66642 bstore_ai ceph cwd unknown /proc/65408/task/66642/cwd (readlink: Permission denied) 2024-09-21T03:42:13.491 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 65408 66642 bstore_ai ceph rtd unknown /proc/65408/task/66642/root (readlink: Permission denied) 2024-09-21T03:42:13.491 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 65408 66642 bstore_ai ceph txt unknown /proc/65408/task/66642/exe (readlink: Permission denied) 2024-09-21T03:42:13.491 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 65408 66642 bstore_ai ceph NOFD /proc/65408/task/66642/fd (opendir: Permission denied) 2024-09-21T03:42:13.491 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 65408 66827 ceph-osd ceph cwd unknown /proc/65408/task/66827/cwd (readlink: Permission denied) 2024-09-21T03:42:13.491 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 65408 66827 ceph-osd ceph rtd unknown /proc/65408/task/66827/root (readlink: Permission denied) 2024-09-21T03:42:13.491 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 65408 66827 ceph-osd ceph txt unknown /proc/65408/task/66827/exe (readlink: Permission denied) 2024-09-21T03:42:13.491 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 65408 66827 ceph-osd ceph NOFD /proc/65408/task/66827/fd (opendir: Permission denied) 2024-09-21T03:42:13.491 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 65408 66828 cfin ceph cwd unknown /proc/65408/task/66828/cwd (readlink: Permission denied) 2024-09-21T03:42:13.491 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 65408 66828 cfin ceph rtd unknown /proc/65408/task/66828/root (readlink: Permission denied) 2024-09-21T03:42:13.491 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 65408 66828 cfin ceph txt unknown /proc/65408/task/66828/exe (readlink: Permission denied) 2024-09-21T03:42:13.491 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 65408 66828 cfin ceph NOFD /proc/65408/task/66828/fd (opendir: Permission denied) 2024-09-21T03:42:13.492 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 65408 66829 bstore_kv ceph cwd unknown /proc/65408/task/66829/cwd (readlink: Permission denied) 2024-09-21T03:42:13.492 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 65408 66829 bstore_kv ceph rtd unknown /proc/65408/task/66829/root (readlink: Permission denied) 2024-09-21T03:42:13.492 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 65408 66829 bstore_kv ceph txt unknown /proc/65408/task/66829/exe (readlink: Permission denied) 2024-09-21T03:42:13.492 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 65408 66829 bstore_kv ceph NOFD /proc/65408/task/66829/fd (opendir: Permission denied) 2024-09-21T03:42:13.492 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 65408 66830 bstore_kv ceph cwd unknown /proc/65408/task/66830/cwd (readlink: Permission denied) 2024-09-21T03:42:13.493 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 65408 66830 bstore_kv ceph rtd unknown /proc/65408/task/66830/root (readlink: Permission denied) 2024-09-21T03:42:13.494 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 65408 66830 bstore_kv ceph txt unknown /proc/65408/task/66830/exe (readlink: Permission denied) 2024-09-21T03:42:13.494 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 65408 66830 bstore_kv ceph NOFD /proc/65408/task/66830/fd (opendir: Permission denied) 2024-09-21T03:42:13.494 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 65408 66831 bstore_me ceph cwd unknown /proc/65408/task/66831/cwd (readlink: Permission denied) 2024-09-21T03:42:13.494 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 65408 66831 bstore_me ceph rtd unknown /proc/65408/task/66831/root (readlink: Permission denied) 2024-09-21T03:42:13.494 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 65408 66831 bstore_me ceph txt unknown /proc/65408/task/66831/exe (readlink: Permission denied) 2024-09-21T03:42:13.494 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 65408 66831 bstore_me ceph NOFD /proc/65408/task/66831/fd (opendir: Permission denied) 2024-09-21T03:42:13.494 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 65408 66834 ms_dispat ceph cwd unknown /proc/65408/task/66834/cwd (readlink: Permission denied) 2024-09-21T03:42:13.494 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 65408 66834 ms_dispat ceph rtd unknown /proc/65408/task/66834/root (readlink: Permission denied) 2024-09-21T03:42:13.495 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 65408 66834 ms_dispat ceph txt unknown /proc/65408/task/66834/exe (readlink: Permission denied) 2024-09-21T03:42:13.495 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 65408 66834 ms_dispat ceph NOFD /proc/65408/task/66834/fd (opendir: Permission denied) 2024-09-21T03:42:13.495 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 65408 66835 ms_local ceph cwd unknown /proc/65408/task/66835/cwd (readlink: Permission denied) 2024-09-21T03:42:13.495 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 65408 66835 ms_local ceph rtd unknown /proc/65408/task/66835/root (readlink: Permission denied) 2024-09-21T03:42:13.495 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 65408 66835 ms_local ceph txt unknown /proc/65408/task/66835/exe (readlink: Permission denied) 2024-09-21T03:42:13.495 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 65408 66835 ms_local ceph NOFD /proc/65408/task/66835/fd (opendir: Permission denied) 2024-09-21T03:42:13.495 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 65408 66836 safe_time ceph cwd unknown /proc/65408/task/66836/cwd (readlink: Permission denied) 2024-09-21T03:42:13.495 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 65408 66836 safe_time ceph rtd unknown /proc/65408/task/66836/root (readlink: Permission denied) 2024-09-21T03:42:13.495 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 65408 66836 safe_time ceph txt unknown /proc/65408/task/66836/exe (readlink: Permission denied) 2024-09-21T03:42:13.495 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 65408 66836 safe_time ceph NOFD /proc/65408/task/66836/fd (opendir: Permission denied) 2024-09-21T03:42:13.496 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 65408 66837 safe_time ceph cwd unknown /proc/65408/task/66837/cwd (readlink: Permission denied) 2024-09-21T03:42:13.496 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 65408 66837 safe_time ceph rtd unknown /proc/65408/task/66837/root (readlink: Permission denied) 2024-09-21T03:42:13.496 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 65408 66837 safe_time ceph txt unknown /proc/65408/task/66837/exe (readlink: Permission denied) 2024-09-21T03:42:13.496 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 65408 66837 safe_time ceph NOFD /proc/65408/task/66837/fd (opendir: Permission denied) 2024-09-21T03:42:13.496 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 65408 66838 ms_dispat ceph cwd unknown /proc/65408/task/66838/cwd (readlink: Permission denied) 2024-09-21T03:42:13.496 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 65408 66838 ms_dispat ceph rtd unknown /proc/65408/task/66838/root (readlink: Permission denied) 2024-09-21T03:42:13.496 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 65408 66838 ms_dispat ceph txt unknown /proc/65408/task/66838/exe (readlink: Permission denied) 2024-09-21T03:42:13.496 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 65408 66838 ms_dispat ceph NOFD /proc/65408/task/66838/fd (opendir: Permission denied) 2024-09-21T03:42:13.496 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 65408 66839 ms_local ceph cwd unknown /proc/65408/task/66839/cwd (readlink: Permission denied) 2024-09-21T03:42:13.496 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 65408 66839 ms_local ceph rtd unknown /proc/65408/task/66839/root (readlink: Permission denied) 2024-09-21T03:42:13.497 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 65408 66839 ms_local ceph txt unknown /proc/65408/task/66839/exe (readlink: Permission denied) 2024-09-21T03:42:13.497 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 65408 66839 ms_local ceph NOFD /proc/65408/task/66839/fd (opendir: Permission denied) 2024-09-21T03:42:13.497 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 65408 66840 ms_dispat ceph cwd unknown /proc/65408/task/66840/cwd (readlink: Permission denied) 2024-09-21T03:42:13.497 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 65408 66840 ms_dispat ceph rtd unknown /proc/65408/task/66840/root (readlink: Permission denied) 2024-09-21T03:42:13.497 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 65408 66840 ms_dispat ceph txt unknown /proc/65408/task/66840/exe (readlink: Permission denied) 2024-09-21T03:42:13.497 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 65408 66840 ms_dispat ceph NOFD /proc/65408/task/66840/fd (opendir: Permission denied) 2024-09-21T03:42:13.497 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 65408 66841 ms_local ceph cwd unknown /proc/65408/task/66841/cwd (readlink: Permission denied) 2024-09-21T03:42:13.497 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 65408 66841 ms_local ceph rtd unknown /proc/65408/task/66841/root (readlink: Permission denied) 2024-09-21T03:42:13.497 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 65408 66841 ms_local ceph txt unknown /proc/65408/task/66841/exe (readlink: Permission denied) 2024-09-21T03:42:13.497 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 65408 66841 ms_local ceph NOFD /proc/65408/task/66841/fd (opendir: Permission denied) 2024-09-21T03:42:13.497 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 65408 66842 ms_dispat ceph cwd unknown /proc/65408/task/66842/cwd (readlink: Permission denied) 2024-09-21T03:42:13.497 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 65408 66842 ms_dispat ceph rtd unknown /proc/65408/task/66842/root (readlink: Permission denied) 2024-09-21T03:42:13.497 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 65408 66842 ms_dispat ceph txt unknown /proc/65408/task/66842/exe (readlink: Permission denied) 2024-09-21T03:42:13.497 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 65408 66842 ms_dispat ceph NOFD /proc/65408/task/66842/fd (opendir: Permission denied) 2024-09-21T03:42:13.498 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 65408 66843 ms_local ceph cwd unknown /proc/65408/task/66843/cwd (readlink: Permission denied) 2024-09-21T03:42:13.498 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 65408 66843 ms_local ceph rtd unknown /proc/65408/task/66843/root (readlink: Permission denied) 2024-09-21T03:42:13.498 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 65408 66843 ms_local ceph txt unknown /proc/65408/task/66843/exe (readlink: Permission denied) 2024-09-21T03:42:13.498 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 65408 66843 ms_local ceph NOFD /proc/65408/task/66843/fd (opendir: Permission denied) 2024-09-21T03:42:13.498 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 65408 66844 ms_dispat ceph cwd unknown /proc/65408/task/66844/cwd (readlink: Permission denied) 2024-09-21T03:42:13.498 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 65408 66844 ms_dispat ceph rtd unknown /proc/65408/task/66844/root (readlink: Permission denied) 2024-09-21T03:42:13.498 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 65408 66844 ms_dispat ceph txt unknown /proc/65408/task/66844/exe (readlink: Permission denied) 2024-09-21T03:42:13.498 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 65408 66844 ms_dispat ceph NOFD /proc/65408/task/66844/fd (opendir: Permission denied) 2024-09-21T03:42:13.498 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 65408 66845 ms_local ceph cwd unknown /proc/65408/task/66845/cwd (readlink: Permission denied) 2024-09-21T03:42:13.498 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 65408 66845 ms_local ceph rtd unknown /proc/65408/task/66845/root (readlink: Permission denied) 2024-09-21T03:42:13.498 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 65408 66845 ms_local ceph txt unknown /proc/65408/task/66845/exe (readlink: Permission denied) 2024-09-21T03:42:13.498 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 65408 66845 ms_local ceph NOFD /proc/65408/task/66845/fd (opendir: Permission denied) 2024-09-21T03:42:13.498 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 65408 66846 ms_dispat ceph cwd unknown /proc/65408/task/66846/cwd (readlink: Permission denied) 2024-09-21T03:42:13.499 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 65408 66846 ms_dispat ceph rtd unknown /proc/65408/task/66846/root (readlink: Permission denied) 2024-09-21T03:42:13.499 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 65408 66846 ms_dispat ceph txt unknown /proc/65408/task/66846/exe (readlink: Permission denied) 2024-09-21T03:42:13.499 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 65408 66846 ms_dispat ceph NOFD /proc/65408/task/66846/fd (opendir: Permission denied) 2024-09-21T03:42:13.499 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 65408 66847 ms_local ceph cwd unknown /proc/65408/task/66847/cwd (readlink: Permission denied) 2024-09-21T03:42:13.499 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 65408 66847 ms_local ceph rtd unknown /proc/65408/task/66847/root (readlink: Permission denied) 2024-09-21T03:42:13.499 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 65408 66847 ms_local ceph txt unknown /proc/65408/task/66847/exe (readlink: Permission denied) 2024-09-21T03:42:13.499 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 65408 66847 ms_local ceph NOFD /proc/65408/task/66847/fd (opendir: Permission denied) 2024-09-21T03:42:13.499 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 65408 66848 ms_dispat ceph cwd unknown /proc/65408/task/66848/cwd (readlink: Permission denied) 2024-09-21T03:42:13.499 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 65408 66848 ms_dispat ceph rtd unknown /proc/65408/task/66848/root (readlink: Permission denied) 2024-09-21T03:42:13.499 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 65408 66848 ms_dispat ceph txt unknown /proc/65408/task/66848/exe (readlink: Permission denied) 2024-09-21T03:42:13.499 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 65408 66848 ms_dispat ceph NOFD /proc/65408/task/66848/fd (opendir: Permission denied) 2024-09-21T03:42:13.499 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 65408 66849 ms_local ceph cwd unknown /proc/65408/task/66849/cwd (readlink: Permission denied) 2024-09-21T03:42:13.499 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 65408 66849 ms_local ceph rtd unknown /proc/65408/task/66849/root (readlink: Permission denied) 2024-09-21T03:42:13.500 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 65408 66849 ms_local ceph txt unknown /proc/65408/task/66849/exe (readlink: Permission denied) 2024-09-21T03:42:13.500 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 65408 66849 ms_local ceph NOFD /proc/65408/task/66849/fd (opendir: Permission denied) 2024-09-21T03:42:13.500 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 65408 66850 fn_anonym ceph cwd unknown /proc/65408/task/66850/cwd (readlink: Permission denied) 2024-09-21T03:42:13.500 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 65408 66850 fn_anonym ceph rtd unknown /proc/65408/task/66850/root (readlink: Permission denied) 2024-09-21T03:42:13.500 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 65408 66850 fn_anonym ceph txt unknown /proc/65408/task/66850/exe (readlink: Permission denied) 2024-09-21T03:42:13.500 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 65408 66850 fn_anonym ceph NOFD /proc/65408/task/66850/fd (opendir: Permission denied) 2024-09-21T03:42:13.500 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 65408 66851 finisher ceph cwd unknown /proc/65408/task/66851/cwd (readlink: Permission denied) 2024-09-21T03:42:13.500 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 65408 66851 finisher ceph rtd unknown /proc/65408/task/66851/root (readlink: Permission denied) 2024-09-21T03:42:13.500 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 65408 66851 finisher ceph txt unknown /proc/65408/task/66851/exe (readlink: Permission denied) 2024-09-21T03:42:13.500 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 65408 66851 finisher ceph NOFD /proc/65408/task/66851/fd (opendir: Permission denied) 2024-09-21T03:42:13.500 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 65408 66852 safe_time ceph cwd unknown /proc/65408/task/66852/cwd (readlink: Permission denied) 2024-09-21T03:42:13.500 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 65408 66852 safe_time ceph rtd unknown /proc/65408/task/66852/root (readlink: Permission denied) 2024-09-21T03:42:13.500 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 65408 66852 safe_time ceph txt unknown /proc/65408/task/66852/exe (readlink: Permission denied) 2024-09-21T03:42:13.500 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 65408 66852 safe_time ceph NOFD /proc/65408/task/66852/fd (opendir: Permission denied) 2024-09-21T03:42:13.501 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 65408 66853 safe_time ceph cwd unknown /proc/65408/task/66853/cwd (readlink: Permission denied) 2024-09-21T03:42:13.501 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 65408 66853 safe_time ceph rtd unknown /proc/65408/task/66853/root (readlink: Permission denied) 2024-09-21T03:42:13.501 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 65408 66853 safe_time ceph txt unknown /proc/65408/task/66853/exe (readlink: Permission denied) 2024-09-21T03:42:13.501 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 65408 66853 safe_time ceph NOFD /proc/65408/task/66853/fd (opendir: Permission denied) 2024-09-21T03:42:13.501 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 65408 66854 ceph-osd ceph cwd unknown /proc/65408/task/66854/cwd (readlink: Permission denied) 2024-09-21T03:42:13.501 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 65408 66854 ceph-osd ceph rtd unknown /proc/65408/task/66854/root (readlink: Permission denied) 2024-09-21T03:42:13.501 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 65408 66854 ceph-osd ceph txt unknown /proc/65408/task/66854/exe (readlink: Permission denied) 2024-09-21T03:42:13.501 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 65408 66854 ceph-osd ceph NOFD /proc/65408/task/66854/fd (opendir: Permission denied) 2024-09-21T03:42:13.501 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 65408 66855 osd_srv_a ceph cwd unknown /proc/65408/task/66855/cwd (readlink: Permission denied) 2024-09-21T03:42:13.501 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 65408 66855 osd_srv_a ceph rtd unknown /proc/65408/task/66855/root (readlink: Permission denied) 2024-09-21T03:42:13.501 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 65408 66855 osd_srv_a ceph txt unknown /proc/65408/task/66855/exe (readlink: Permission denied) 2024-09-21T03:42:13.501 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 65408 66855 osd_srv_a ceph NOFD /proc/65408/task/66855/fd (opendir: Permission denied) 2024-09-21T03:42:13.501 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 65408 66856 tp_osd_tp ceph cwd unknown /proc/65408/task/66856/cwd (readlink: Permission denied) 2024-09-21T03:42:13.502 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 65408 66856 tp_osd_tp ceph rtd unknown /proc/65408/task/66856/root (readlink: Permission denied) 2024-09-21T03:42:13.502 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 65408 66856 tp_osd_tp ceph txt unknown /proc/65408/task/66856/exe (readlink: Permission denied) 2024-09-21T03:42:13.502 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 65408 66856 tp_osd_tp ceph NOFD /proc/65408/task/66856/fd (opendir: Permission denied) 2024-09-21T03:42:13.502 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 65408 66857 tp_osd_tp ceph cwd unknown /proc/65408/task/66857/cwd (readlink: Permission denied) 2024-09-21T03:42:13.502 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 65408 66857 tp_osd_tp ceph rtd unknown /proc/65408/task/66857/root (readlink: Permission denied) 2024-09-21T03:42:13.502 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 65408 66857 tp_osd_tp ceph txt unknown /proc/65408/task/66857/exe (readlink: Permission denied) 2024-09-21T03:42:13.502 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 65408 66857 tp_osd_tp ceph NOFD /proc/65408/task/66857/fd (opendir: Permission denied) 2024-09-21T03:42:13.502 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 65408 66858 tp_osd_tp ceph cwd unknown /proc/65408/task/66858/cwd (readlink: Permission denied) 2024-09-21T03:42:13.502 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 65408 66858 tp_osd_tp ceph rtd unknown /proc/65408/task/66858/root (readlink: Permission denied) 2024-09-21T03:42:13.502 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 65408 66858 tp_osd_tp ceph txt unknown /proc/65408/task/66858/exe (readlink: Permission denied) 2024-09-21T03:42:13.502 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 65408 66858 tp_osd_tp ceph NOFD /proc/65408/task/66858/fd (opendir: Permission denied) 2024-09-21T03:42:13.502 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 65408 66859 tp_osd_tp ceph cwd unknown /proc/65408/task/66859/cwd (readlink: Permission denied) 2024-09-21T03:42:13.502 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 65408 66859 tp_osd_tp ceph rtd unknown /proc/65408/task/66859/root (readlink: Permission denied) 2024-09-21T03:42:13.503 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 65408 66859 tp_osd_tp ceph txt unknown /proc/65408/task/66859/exe (readlink: Permission denied) 2024-09-21T03:42:13.503 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 65408 66859 tp_osd_tp ceph NOFD /proc/65408/task/66859/fd (opendir: Permission denied) 2024-09-21T03:42:13.503 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 65408 66860 tp_osd_tp ceph cwd unknown /proc/65408/task/66860/cwd (readlink: Permission denied) 2024-09-21T03:42:13.503 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 65408 66860 tp_osd_tp ceph rtd unknown /proc/65408/task/66860/root (readlink: Permission denied) 2024-09-21T03:42:13.503 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 65408 66860 tp_osd_tp ceph txt unknown /proc/65408/task/66860/exe (readlink: Permission denied) 2024-09-21T03:42:13.503 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 65408 66860 tp_osd_tp ceph NOFD /proc/65408/task/66860/fd (opendir: Permission denied) 2024-09-21T03:42:13.503 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 65408 66861 tp_osd_tp ceph cwd unknown /proc/65408/task/66861/cwd (readlink: Permission denied) 2024-09-21T03:42:13.503 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 65408 66861 tp_osd_tp ceph rtd unknown /proc/65408/task/66861/root (readlink: Permission denied) 2024-09-21T03:42:13.503 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 65408 66861 tp_osd_tp ceph txt unknown /proc/65408/task/66861/exe (readlink: Permission denied) 2024-09-21T03:42:13.503 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 65408 66861 tp_osd_tp ceph NOFD /proc/65408/task/66861/fd (opendir: Permission denied) 2024-09-21T03:42:13.503 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 65408 66862 tp_osd_tp ceph cwd unknown /proc/65408/task/66862/cwd (readlink: Permission denied) 2024-09-21T03:42:13.503 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 65408 66862 tp_osd_tp ceph rtd unknown /proc/65408/task/66862/root (readlink: Permission denied) 2024-09-21T03:42:13.503 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 65408 66862 tp_osd_tp ceph txt unknown /proc/65408/task/66862/exe (readlink: Permission denied) 2024-09-21T03:42:13.503 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 65408 66862 tp_osd_tp ceph NOFD /proc/65408/task/66862/fd (opendir: Permission denied) 2024-09-21T03:42:13.504 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 65408 66863 tp_osd_tp ceph cwd unknown /proc/65408/task/66863/cwd (readlink: Permission denied) 2024-09-21T03:42:13.504 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 65408 66863 tp_osd_tp ceph rtd unknown /proc/65408/task/66863/root (readlink: Permission denied) 2024-09-21T03:42:13.504 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 65408 66863 tp_osd_tp ceph txt unknown /proc/65408/task/66863/exe (readlink: Permission denied) 2024-09-21T03:42:13.504 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 65408 66863 tp_osd_tp ceph NOFD /proc/65408/task/66863/fd (opendir: Permission denied) 2024-09-21T03:42:13.504 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 65408 66865 tp_osd_tp ceph cwd unknown /proc/65408/task/66865/cwd (readlink: Permission denied) 2024-09-21T03:42:13.504 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 65408 66865 tp_osd_tp ceph rtd unknown /proc/65408/task/66865/root (readlink: Permission denied) 2024-09-21T03:42:13.504 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 65408 66865 tp_osd_tp ceph txt unknown /proc/65408/task/66865/exe (readlink: Permission denied) 2024-09-21T03:42:13.504 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 65408 66865 tp_osd_tp ceph NOFD /proc/65408/task/66865/fd (opendir: Permission denied) 2024-09-21T03:42:13.504 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 65408 66866 tp_osd_tp ceph cwd unknown /proc/65408/task/66866/cwd (readlink: Permission denied) 2024-09-21T03:42:13.504 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 65408 66866 tp_osd_tp ceph rtd unknown /proc/65408/task/66866/root (readlink: Permission denied) 2024-09-21T03:42:13.504 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 65408 66866 tp_osd_tp ceph txt unknown /proc/65408/task/66866/exe (readlink: Permission denied) 2024-09-21T03:42:13.504 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 65408 66866 tp_osd_tp ceph NOFD /proc/65408/task/66866/fd (opendir: Permission denied) 2024-09-21T03:42:13.504 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 65408 66867 tp_osd_tp ceph cwd unknown /proc/65408/task/66867/cwd (readlink: Permission denied) 2024-09-21T03:42:13.504 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 65408 66867 tp_osd_tp ceph rtd unknown /proc/65408/task/66867/root (readlink: Permission denied) 2024-09-21T03:42:13.504 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 65408 66867 tp_osd_tp ceph txt unknown /proc/65408/task/66867/exe (readlink: Permission denied) 2024-09-21T03:42:13.505 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 65408 66867 tp_osd_tp ceph NOFD /proc/65408/task/66867/fd (opendir: Permission denied) 2024-09-21T03:42:13.505 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 65408 66868 tp_osd_tp ceph cwd unknown /proc/65408/task/66868/cwd (readlink: Permission denied) 2024-09-21T03:42:13.505 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 65408 66868 tp_osd_tp ceph rtd unknown /proc/65408/task/66868/root (readlink: Permission denied) 2024-09-21T03:42:13.505 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 65408 66868 tp_osd_tp ceph txt unknown /proc/65408/task/66868/exe (readlink: Permission denied) 2024-09-21T03:42:13.505 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 65408 66868 tp_osd_tp ceph NOFD /proc/65408/task/66868/fd (opendir: Permission denied) 2024-09-21T03:42:13.505 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 65408 66869 tp_osd_tp ceph cwd unknown /proc/65408/task/66869/cwd (readlink: Permission denied) 2024-09-21T03:42:13.505 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 65408 66869 tp_osd_tp ceph rtd unknown /proc/65408/task/66869/root (readlink: Permission denied) 2024-09-21T03:42:13.505 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 65408 66869 tp_osd_tp ceph txt unknown /proc/65408/task/66869/exe (readlink: Permission denied) 2024-09-21T03:42:13.505 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 65408 66869 tp_osd_tp ceph NOFD /proc/65408/task/66869/fd (opendir: Permission denied) 2024-09-21T03:42:13.505 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 65408 66870 tp_osd_tp ceph cwd unknown /proc/65408/task/66870/cwd (readlink: Permission denied) 2024-09-21T03:42:13.505 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 65408 66870 tp_osd_tp ceph rtd unknown /proc/65408/task/66870/root (readlink: Permission denied) 2024-09-21T03:42:13.505 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 65408 66870 tp_osd_tp ceph txt unknown /proc/65408/task/66870/exe (readlink: Permission denied) 2024-09-21T03:42:13.505 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 65408 66870 tp_osd_tp ceph NOFD /proc/65408/task/66870/fd (opendir: Permission denied) 2024-09-21T03:42:13.505 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 65408 66871 tp_osd_tp ceph cwd unknown /proc/65408/task/66871/cwd (readlink: Permission denied) 2024-09-21T03:42:13.505 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 65408 66871 tp_osd_tp ceph rtd unknown /proc/65408/task/66871/root (readlink: Permission denied) 2024-09-21T03:42:13.506 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 65408 66871 tp_osd_tp ceph txt unknown /proc/65408/task/66871/exe (readlink: Permission denied) 2024-09-21T03:42:13.506 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 65408 66871 tp_osd_tp ceph NOFD /proc/65408/task/66871/fd (opendir: Permission denied) 2024-09-21T03:42:13.506 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 65408 66872 tp_osd_tp ceph cwd unknown /proc/65408/task/66872/cwd (readlink: Permission denied) 2024-09-21T03:42:13.506 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 65408 66872 tp_osd_tp ceph rtd unknown /proc/65408/task/66872/root (readlink: Permission denied) 2024-09-21T03:42:13.506 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 65408 66872 tp_osd_tp ceph txt unknown /proc/65408/task/66872/exe (readlink: Permission denied) 2024-09-21T03:42:13.506 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 65408 66872 tp_osd_tp ceph NOFD /proc/65408/task/66872/fd (opendir: Permission denied) 2024-09-21T03:42:13.506 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 65408 66873 osd_srv_h ceph cwd unknown /proc/65408/task/66873/cwd (readlink: Permission denied) 2024-09-21T03:42:13.506 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 65408 66873 osd_srv_h ceph rtd unknown /proc/65408/task/66873/root (readlink: Permission denied) 2024-09-21T03:42:13.506 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 65408 66873 osd_srv_h ceph txt unknown /proc/65408/task/66873/exe (readlink: Permission denied) 2024-09-21T03:42:13.506 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-osd 65408 66873 osd_srv_h ceph NOFD /proc/65408/task/66873/fd (opendir: Permission denied) 2024-09-21T03:42:13.506 INFO:tasks.workunit.client.0.smithi089.stdout:sudo 65995 root cwd unknown /proc/65995/cwd (readlink: Permission denied) 2024-09-21T03:42:13.506 INFO:tasks.workunit.client.0.smithi089.stdout:sudo 65995 root rtd unknown /proc/65995/root (readlink: Permission denied) 2024-09-21T03:42:13.506 INFO:tasks.workunit.client.0.smithi089.stdout:sudo 65995 root txt unknown /proc/65995/exe (readlink: Permission denied) 2024-09-21T03:42:13.506 INFO:tasks.workunit.client.0.smithi089.stdout:sudo 65995 root NOFD /proc/65995/fd (opendir: Permission denied) 2024-09-21T03:42:13.507 INFO:tasks.workunit.client.0.smithi089.stdout:journalct 66012 root cwd unknown /proc/66012/cwd (readlink: Permission denied) 2024-09-21T03:42:13.507 INFO:tasks.workunit.client.0.smithi089.stdout:journalct 66012 root rtd unknown /proc/66012/root (readlink: Permission denied) 2024-09-21T03:42:13.507 INFO:tasks.workunit.client.0.smithi089.stdout:journalct 66012 root txt unknown /proc/66012/exe (readlink: Permission denied) 2024-09-21T03:42:13.507 INFO:tasks.workunit.client.0.smithi089.stdout:journalct 66012 root NOFD /proc/66012/fd (opendir: Permission denied) 2024-09-21T03:42:13.507 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/5 68755 root cwd unknown /proc/68755/cwd (readlink: Permission denied) 2024-09-21T03:42:13.507 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/5 68755 root rtd unknown /proc/68755/root (readlink: Permission denied) 2024-09-21T03:42:13.507 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/5 68755 root txt unknown /proc/68755/exe (readlink: Permission denied) 2024-09-21T03:42:13.507 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/5 68755 root NOFD /proc/68755/fd (opendir: Permission denied) 2024-09-21T03:42:13.507 INFO:tasks.workunit.client.0.smithi089.stdout:sudo 69867 root cwd unknown /proc/69867/cwd (readlink: Permission denied) 2024-09-21T03:42:13.507 INFO:tasks.workunit.client.0.smithi089.stdout:sudo 69867 root rtd unknown /proc/69867/root (readlink: Permission denied) 2024-09-21T03:42:13.507 INFO:tasks.workunit.client.0.smithi089.stdout:sudo 69867 root txt unknown /proc/69867/exe (readlink: Permission denied) 2024-09-21T03:42:13.507 INFO:tasks.workunit.client.0.smithi089.stdout:sudo 69867 root NOFD /proc/69867/fd (opendir: Permission denied) 2024-09-21T03:42:13.507 INFO:tasks.workunit.client.0.smithi089.stdout:journalct 69884 root cwd unknown /proc/69884/cwd (readlink: Permission denied) 2024-09-21T03:42:13.507 INFO:tasks.workunit.client.0.smithi089.stdout:journalct 69884 root rtd unknown /proc/69884/root (readlink: Permission denied) 2024-09-21T03:42:13.507 INFO:tasks.workunit.client.0.smithi089.stdout:journalct 69884 root txt unknown /proc/69884/exe (readlink: Permission denied) 2024-09-21T03:42:13.508 INFO:tasks.workunit.client.0.smithi089.stdout:journalct 69884 root NOFD /proc/69884/fd (opendir: Permission denied) 2024-09-21T03:42:13.508 INFO:tasks.workunit.client.0.smithi089.stdout:sudo 69885 root cwd unknown /proc/69885/cwd (readlink: Permission denied) 2024-09-21T03:42:13.508 INFO:tasks.workunit.client.0.smithi089.stdout:sudo 69885 root rtd unknown /proc/69885/root (readlink: Permission denied) 2024-09-21T03:42:13.508 INFO:tasks.workunit.client.0.smithi089.stdout:sudo 69885 root txt unknown /proc/69885/exe (readlink: Permission denied) 2024-09-21T03:42:13.508 INFO:tasks.workunit.client.0.smithi089.stdout:sudo 69885 root NOFD /proc/69885/fd (opendir: Permission denied) 2024-09-21T03:42:13.508 INFO:tasks.workunit.client.0.smithi089.stdout:journalct 69902 root cwd unknown /proc/69902/cwd (readlink: Permission denied) 2024-09-21T03:42:13.508 INFO:tasks.workunit.client.0.smithi089.stdout:journalct 69902 root rtd unknown /proc/69902/root (readlink: Permission denied) 2024-09-21T03:42:13.508 INFO:tasks.workunit.client.0.smithi089.stdout:journalct 69902 root txt unknown /proc/69902/exe (readlink: Permission denied) 2024-09-21T03:42:13.508 INFO:tasks.workunit.client.0.smithi089.stdout:journalct 69902 root NOFD /proc/69902/fd (opendir: Permission denied) 2024-09-21T03:42:13.508 INFO:tasks.workunit.client.0.smithi089.stdout:sudo 69903 root cwd unknown /proc/69903/cwd (readlink: Permission denied) 2024-09-21T03:42:13.508 INFO:tasks.workunit.client.0.smithi089.stdout:sudo 69903 root rtd unknown /proc/69903/root (readlink: Permission denied) 2024-09-21T03:42:13.508 INFO:tasks.workunit.client.0.smithi089.stdout:sudo 69903 root txt unknown /proc/69903/exe (readlink: Permission denied) 2024-09-21T03:42:13.508 INFO:tasks.workunit.client.0.smithi089.stdout:sudo 69903 root NOFD /proc/69903/fd (opendir: Permission denied) 2024-09-21T03:42:13.508 INFO:tasks.workunit.client.0.smithi089.stdout:journalct 69920 root cwd unknown /proc/69920/cwd (readlink: Permission denied) 2024-09-21T03:42:13.508 INFO:tasks.workunit.client.0.smithi089.stdout:journalct 69920 root rtd unknown /proc/69920/root (readlink: Permission denied) 2024-09-21T03:42:13.509 INFO:tasks.workunit.client.0.smithi089.stdout:journalct 69920 root txt unknown /proc/69920/exe (readlink: Permission denied) 2024-09-21T03:42:13.509 INFO:tasks.workunit.client.0.smithi089.stdout:journalct 69920 root NOFD /proc/69920/fd (opendir: Permission denied) 2024-09-21T03:42:13.509 INFO:tasks.workunit.client.0.smithi089.stdout:sudo 69921 root cwd unknown /proc/69921/cwd (readlink: Permission denied) 2024-09-21T03:42:13.509 INFO:tasks.workunit.client.0.smithi089.stdout:sudo 69921 root rtd unknown /proc/69921/root (readlink: Permission denied) 2024-09-21T03:42:13.509 INFO:tasks.workunit.client.0.smithi089.stdout:sudo 69921 root txt unknown /proc/69921/exe (readlink: Permission denied) 2024-09-21T03:42:13.509 INFO:tasks.workunit.client.0.smithi089.stdout:sudo 69921 root NOFD /proc/69921/fd (opendir: Permission denied) 2024-09-21T03:42:13.509 INFO:tasks.workunit.client.0.smithi089.stdout:journalct 69938 root cwd unknown /proc/69938/cwd (readlink: Permission denied) 2024-09-21T03:42:13.509 INFO:tasks.workunit.client.0.smithi089.stdout:journalct 69938 root rtd unknown /proc/69938/root (readlink: Permission denied) 2024-09-21T03:42:13.509 INFO:tasks.workunit.client.0.smithi089.stdout:journalct 69938 root txt unknown /proc/69938/exe (readlink: Permission denied) 2024-09-21T03:42:13.509 INFO:tasks.workunit.client.0.smithi089.stdout:journalct 69938 root NOFD /proc/69938/fd (opendir: Permission denied) 2024-09-21T03:42:13.509 INFO:tasks.workunit.client.0.smithi089.stdout:conmon 70422 root cwd unknown /proc/70422/cwd (readlink: Permission denied) 2024-09-21T03:42:13.509 INFO:tasks.workunit.client.0.smithi089.stdout:conmon 70422 root rtd unknown /proc/70422/root (readlink: Permission denied) 2024-09-21T03:42:13.509 INFO:tasks.workunit.client.0.smithi089.stdout:conmon 70422 root txt unknown /proc/70422/exe (readlink: Permission denied) 2024-09-21T03:42:13.509 INFO:tasks.workunit.client.0.smithi089.stdout:conmon 70422 root NOFD /proc/70422/fd (opendir: Permission denied) 2024-09-21T03:42:13.509 INFO:tasks.workunit.client.0.smithi089.stdout:podman-in 70424 root cwd unknown /proc/70424/cwd (readlink: Permission denied) 2024-09-21T03:42:13.510 INFO:tasks.workunit.client.0.smithi089.stdout:podman-in 70424 root rtd unknown /proc/70424/root (readlink: Permission denied) 2024-09-21T03:42:13.510 INFO:tasks.workunit.client.0.smithi089.stdout:podman-in 70424 root txt unknown /proc/70424/exe (readlink: Permission denied) 2024-09-21T03:42:13.510 INFO:tasks.workunit.client.0.smithi089.stdout:podman-in 70424 root NOFD /proc/70424/fd (opendir: Permission denied) 2024-09-21T03:42:13.510 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mds 70437 ceph cwd unknown /proc/70437/cwd (readlink: Permission denied) 2024-09-21T03:42:13.510 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mds 70437 ceph rtd unknown /proc/70437/root (readlink: Permission denied) 2024-09-21T03:42:13.510 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mds 70437 ceph txt unknown /proc/70437/exe (readlink: Permission denied) 2024-09-21T03:42:13.510 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mds 70437 ceph NOFD /proc/70437/fd (opendir: Permission denied) 2024-09-21T03:42:13.510 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mds 70437 70438 log ceph cwd unknown /proc/70437/task/70438/cwd (readlink: Permission denied) 2024-09-21T03:42:13.510 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mds 70437 70438 log ceph rtd unknown /proc/70437/task/70438/root (readlink: Permission denied) 2024-09-21T03:42:13.510 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mds 70437 70438 log ceph txt unknown /proc/70437/task/70438/exe (readlink: Permission denied) 2024-09-21T03:42:13.510 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mds 70437 70438 log ceph NOFD /proc/70437/task/70438/fd (opendir: Permission denied) 2024-09-21T03:42:13.510 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mds 70437 70440 msgr-work ceph cwd unknown /proc/70437/task/70440/cwd (readlink: Permission denied) 2024-09-21T03:42:13.510 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mds 70437 70440 msgr-work ceph rtd unknown /proc/70437/task/70440/root (readlink: Permission denied) 2024-09-21T03:42:13.510 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mds 70437 70440 msgr-work ceph txt unknown /proc/70437/task/70440/exe (readlink: Permission denied) 2024-09-21T03:42:13.511 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mds 70437 70440 msgr-work ceph NOFD /proc/70437/task/70440/fd (opendir: Permission denied) 2024-09-21T03:42:13.511 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mds 70437 70441 msgr-work ceph cwd unknown /proc/70437/task/70441/cwd (readlink: Permission denied) 2024-09-21T03:42:13.511 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mds 70437 70441 msgr-work ceph rtd unknown /proc/70437/task/70441/root (readlink: Permission denied) 2024-09-21T03:42:13.511 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mds 70437 70441 msgr-work ceph txt unknown /proc/70437/task/70441/exe (readlink: Permission denied) 2024-09-21T03:42:13.511 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mds 70437 70441 msgr-work ceph NOFD /proc/70437/task/70441/fd (opendir: Permission denied) 2024-09-21T03:42:13.511 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mds 70437 70442 msgr-work ceph cwd unknown /proc/70437/task/70442/cwd (readlink: Permission denied) 2024-09-21T03:42:13.511 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mds 70437 70442 msgr-work ceph rtd unknown /proc/70437/task/70442/root (readlink: Permission denied) 2024-09-21T03:42:13.511 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mds 70437 70442 msgr-work ceph txt unknown /proc/70437/task/70442/exe (readlink: Permission denied) 2024-09-21T03:42:13.511 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mds 70437 70442 msgr-work ceph NOFD /proc/70437/task/70442/fd (opendir: Permission denied) 2024-09-21T03:42:13.511 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mds 70437 70446 service ceph cwd unknown /proc/70437/task/70446/cwd (readlink: Permission denied) 2024-09-21T03:42:13.511 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mds 70437 70446 service ceph rtd unknown /proc/70437/task/70446/root (readlink: Permission denied) 2024-09-21T03:42:13.511 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mds 70437 70446 service ceph txt unknown /proc/70437/task/70446/exe (readlink: Permission denied) 2024-09-21T03:42:13.511 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mds 70437 70446 service ceph NOFD /proc/70437/task/70446/fd (opendir: Permission denied) 2024-09-21T03:42:13.511 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mds 70437 70447 admin_soc ceph cwd unknown /proc/70437/task/70447/cwd (readlink: Permission denied) 2024-09-21T03:42:13.512 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mds 70437 70447 admin_soc ceph rtd unknown /proc/70437/task/70447/root (readlink: Permission denied) 2024-09-21T03:42:13.512 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mds 70437 70447 admin_soc ceph txt unknown /proc/70437/task/70447/exe (readlink: Permission denied) 2024-09-21T03:42:13.512 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mds 70437 70447 admin_soc ceph NOFD /proc/70437/task/70447/fd (opendir: Permission denied) 2024-09-21T03:42:13.512 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mds 70437 70448 signal_ha ceph cwd unknown /proc/70437/task/70448/cwd (readlink: Permission denied) 2024-09-21T03:42:13.512 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mds 70437 70448 signal_ha ceph rtd unknown /proc/70437/task/70448/root (readlink: Permission denied) 2024-09-21T03:42:13.512 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mds 70437 70448 signal_ha ceph txt unknown /proc/70437/task/70448/exe (readlink: Permission denied) 2024-09-21T03:42:13.512 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mds 70437 70448 signal_ha ceph NOFD /proc/70437/task/70448/fd (opendir: Permission denied) 2024-09-21T03:42:13.512 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mds 70437 70449 io_contex ceph cwd unknown /proc/70437/task/70449/cwd (readlink: Permission denied) 2024-09-21T03:42:13.512 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mds 70437 70449 io_contex ceph rtd unknown /proc/70437/task/70449/root (readlink: Permission denied) 2024-09-21T03:42:13.512 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mds 70437 70449 io_contex ceph txt unknown /proc/70437/task/70449/exe (readlink: Permission denied) 2024-09-21T03:42:13.512 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mds 70437 70449 io_contex ceph NOFD /proc/70437/task/70449/fd (opendir: Permission denied) 2024-09-21T03:42:13.512 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mds 70437 70450 io_contex ceph cwd unknown /proc/70437/task/70450/cwd (readlink: Permission denied) 2024-09-21T03:42:13.512 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mds 70437 70450 io_contex ceph rtd unknown /proc/70437/task/70450/root (readlink: Permission denied) 2024-09-21T03:42:13.512 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mds 70437 70450 io_contex ceph txt unknown /proc/70437/task/70450/exe (readlink: Permission denied) 2024-09-21T03:42:13.512 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mds 70437 70450 io_contex ceph NOFD /proc/70437/task/70450/fd (opendir: Permission denied) 2024-09-21T03:42:13.513 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mds 70437 70451 ms_dispat ceph cwd unknown /proc/70437/task/70451/cwd (readlink: Permission denied) 2024-09-21T03:42:13.513 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mds 70437 70451 ms_dispat ceph rtd unknown /proc/70437/task/70451/root (readlink: Permission denied) 2024-09-21T03:42:13.513 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mds 70437 70451 ms_dispat ceph txt unknown /proc/70437/task/70451/exe (readlink: Permission denied) 2024-09-21T03:42:13.513 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mds 70437 70451 ms_dispat ceph NOFD /proc/70437/task/70451/fd (opendir: Permission denied) 2024-09-21T03:42:13.513 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mds 70437 70452 ms_local ceph cwd unknown /proc/70437/task/70452/cwd (readlink: Permission denied) 2024-09-21T03:42:13.513 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mds 70437 70452 ms_local ceph rtd unknown /proc/70437/task/70452/root (readlink: Permission denied) 2024-09-21T03:42:13.513 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mds 70437 70452 ms_local ceph txt unknown /proc/70437/task/70452/exe (readlink: Permission denied) 2024-09-21T03:42:13.513 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mds 70437 70452 ms_local ceph NOFD /proc/70437/task/70452/fd (opendir: Permission denied) 2024-09-21T03:42:13.513 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mds 70437 70453 safe_time ceph cwd unknown /proc/70437/task/70453/cwd (readlink: Permission denied) 2024-09-21T03:42:13.513 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mds 70437 70453 safe_time ceph rtd unknown /proc/70437/task/70453/root (readlink: Permission denied) 2024-09-21T03:42:13.513 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mds 70437 70453 safe_time ceph txt unknown /proc/70437/task/70453/exe (readlink: Permission denied) 2024-09-21T03:42:13.513 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mds 70437 70453 safe_time ceph NOFD /proc/70437/task/70453/fd (opendir: Permission denied) 2024-09-21T03:42:13.513 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mds 70437 70454 safe_time ceph cwd unknown /proc/70437/task/70454/cwd (readlink: Permission denied) 2024-09-21T03:42:13.513 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mds 70437 70454 safe_time ceph rtd unknown /proc/70437/task/70454/root (readlink: Permission denied) 2024-09-21T03:42:13.513 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mds 70437 70454 safe_time ceph txt unknown /proc/70437/task/70454/exe (readlink: Permission denied) 2024-09-21T03:42:13.514 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mds 70437 70454 safe_time ceph NOFD /proc/70437/task/70454/fd (opendir: Permission denied) 2024-09-21T03:42:13.514 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mds 70437 70455 ceph-mds ceph cwd unknown /proc/70437/task/70455/cwd (readlink: Permission denied) 2024-09-21T03:42:13.514 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mds 70437 70455 ceph-mds ceph rtd unknown /proc/70437/task/70455/root (readlink: Permission denied) 2024-09-21T03:42:13.514 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mds 70437 70455 ceph-mds ceph txt unknown /proc/70437/task/70455/exe (readlink: Permission denied) 2024-09-21T03:42:13.514 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mds 70437 70455 ceph-mds ceph NOFD /proc/70437/task/70455/fd (opendir: Permission denied) 2024-09-21T03:42:13.514 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mds 70437 70636 safe_time ceph cwd unknown /proc/70437/task/70636/cwd (readlink: Permission denied) 2024-09-21T03:42:13.514 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mds 70437 70636 safe_time ceph rtd unknown /proc/70437/task/70636/root (readlink: Permission denied) 2024-09-21T03:42:13.514 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mds 70437 70636 safe_time ceph txt unknown /proc/70437/task/70636/exe (readlink: Permission denied) 2024-09-21T03:42:13.514 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mds 70437 70636 safe_time ceph NOFD /proc/70437/task/70636/fd (opendir: Permission denied) 2024-09-21T03:42:13.514 INFO:tasks.workunit.client.0.smithi089.stdout:conmon 70792 root cwd unknown /proc/70792/cwd (readlink: Permission denied) 2024-09-21T03:42:13.514 INFO:tasks.workunit.client.0.smithi089.stdout:conmon 70792 root rtd unknown /proc/70792/root (readlink: Permission denied) 2024-09-21T03:42:13.514 INFO:tasks.workunit.client.0.smithi089.stdout:conmon 70792 root txt unknown /proc/70792/exe (readlink: Permission denied) 2024-09-21T03:42:13.514 INFO:tasks.workunit.client.0.smithi089.stdout:conmon 70792 root NOFD /proc/70792/fd (opendir: Permission denied) 2024-09-21T03:42:13.515 INFO:tasks.workunit.client.0.smithi089.stdout:podman-in 70794 root cwd unknown /proc/70794/cwd (readlink: Permission denied) 2024-09-21T03:42:13.515 INFO:tasks.workunit.client.0.smithi089.stdout:podman-in 70794 root rtd unknown /proc/70794/root (readlink: Permission denied) 2024-09-21T03:42:13.515 INFO:tasks.workunit.client.0.smithi089.stdout:podman-in 70794 root txt unknown /proc/70794/exe (readlink: Permission denied) 2024-09-21T03:42:13.515 INFO:tasks.workunit.client.0.smithi089.stdout:podman-in 70794 root NOFD /proc/70794/fd (opendir: Permission denied) 2024-09-21T03:42:13.515 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mds 70796 ceph cwd unknown /proc/70796/cwd (readlink: Permission denied) 2024-09-21T03:42:13.515 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mds 70796 ceph rtd unknown /proc/70796/root (readlink: Permission denied) 2024-09-21T03:42:13.515 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mds 70796 ceph txt unknown /proc/70796/exe (readlink: Permission denied) 2024-09-21T03:42:13.515 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mds 70796 ceph NOFD /proc/70796/fd (opendir: Permission denied) 2024-09-21T03:42:13.515 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mds 70796 70797 log ceph cwd unknown /proc/70796/task/70797/cwd (readlink: Permission denied) 2024-09-21T03:42:13.515 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mds 70796 70797 log ceph rtd unknown /proc/70796/task/70797/root (readlink: Permission denied) 2024-09-21T03:42:13.515 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mds 70796 70797 log ceph txt unknown /proc/70796/task/70797/exe (readlink: Permission denied) 2024-09-21T03:42:13.515 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mds 70796 70797 log ceph NOFD /proc/70796/task/70797/fd (opendir: Permission denied) 2024-09-21T03:42:13.516 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mds 70796 70799 msgr-work ceph cwd unknown /proc/70796/task/70799/cwd (readlink: Permission denied) 2024-09-21T03:42:13.516 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mds 70796 70799 msgr-work ceph rtd unknown /proc/70796/task/70799/root (readlink: Permission denied) 2024-09-21T03:42:13.516 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mds 70796 70799 msgr-work ceph txt unknown /proc/70796/task/70799/exe (readlink: Permission denied) 2024-09-21T03:42:13.516 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mds 70796 70799 msgr-work ceph NOFD /proc/70796/task/70799/fd (opendir: Permission denied) 2024-09-21T03:42:13.516 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mds 70796 70800 msgr-work ceph cwd unknown /proc/70796/task/70800/cwd (readlink: Permission denied) 2024-09-21T03:42:13.516 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mds 70796 70800 msgr-work ceph rtd unknown /proc/70796/task/70800/root (readlink: Permission denied) 2024-09-21T03:42:13.516 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mds 70796 70800 msgr-work ceph txt unknown /proc/70796/task/70800/exe (readlink: Permission denied) 2024-09-21T03:42:13.516 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mds 70796 70800 msgr-work ceph NOFD /proc/70796/task/70800/fd (opendir: Permission denied) 2024-09-21T03:42:13.516 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mds 70796 70801 msgr-work ceph cwd unknown /proc/70796/task/70801/cwd (readlink: Permission denied) 2024-09-21T03:42:13.516 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mds 70796 70801 msgr-work ceph rtd unknown /proc/70796/task/70801/root (readlink: Permission denied) 2024-09-21T03:42:13.516 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mds 70796 70801 msgr-work ceph txt unknown /proc/70796/task/70801/exe (readlink: Permission denied) 2024-09-21T03:42:13.517 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mds 70796 70801 msgr-work ceph NOFD /proc/70796/task/70801/fd (opendir: Permission denied) 2024-09-21T03:42:13.517 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mds 70796 70805 service ceph cwd unknown /proc/70796/task/70805/cwd (readlink: Permission denied) 2024-09-21T03:42:13.517 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mds 70796 70805 service ceph rtd unknown /proc/70796/task/70805/root (readlink: Permission denied) 2024-09-21T03:42:13.517 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mds 70796 70805 service ceph txt unknown /proc/70796/task/70805/exe (readlink: Permission denied) 2024-09-21T03:42:13.517 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mds 70796 70805 service ceph NOFD /proc/70796/task/70805/fd (opendir: Permission denied) 2024-09-21T03:42:13.517 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mds 70796 70806 admin_soc ceph cwd unknown /proc/70796/task/70806/cwd (readlink: Permission denied) 2024-09-21T03:42:13.517 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mds 70796 70806 admin_soc ceph rtd unknown /proc/70796/task/70806/root (readlink: Permission denied) 2024-09-21T03:42:13.517 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mds 70796 70806 admin_soc ceph txt unknown /proc/70796/task/70806/exe (readlink: Permission denied) 2024-09-21T03:42:13.517 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mds 70796 70806 admin_soc ceph NOFD /proc/70796/task/70806/fd (opendir: Permission denied) 2024-09-21T03:42:13.517 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mds 70796 70807 signal_ha ceph cwd unknown /proc/70796/task/70807/cwd (readlink: Permission denied) 2024-09-21T03:42:13.517 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mds 70796 70807 signal_ha ceph rtd unknown /proc/70796/task/70807/root (readlink: Permission denied) 2024-09-21T03:42:13.517 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mds 70796 70807 signal_ha ceph txt unknown /proc/70796/task/70807/exe (readlink: Permission denied) 2024-09-21T03:42:13.517 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mds 70796 70807 signal_ha ceph NOFD /proc/70796/task/70807/fd (opendir: Permission denied) 2024-09-21T03:42:13.518 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mds 70796 70808 io_contex ceph cwd unknown /proc/70796/task/70808/cwd (readlink: Permission denied) 2024-09-21T03:42:13.518 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mds 70796 70808 io_contex ceph rtd unknown /proc/70796/task/70808/root (readlink: Permission denied) 2024-09-21T03:42:13.518 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mds 70796 70808 io_contex ceph txt unknown /proc/70796/task/70808/exe (readlink: Permission denied) 2024-09-21T03:42:13.518 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mds 70796 70808 io_contex ceph NOFD /proc/70796/task/70808/fd (opendir: Permission denied) 2024-09-21T03:42:13.518 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mds 70796 70809 io_contex ceph cwd unknown /proc/70796/task/70809/cwd (readlink: Permission denied) 2024-09-21T03:42:13.518 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mds 70796 70809 io_contex ceph rtd unknown /proc/70796/task/70809/root (readlink: Permission denied) 2024-09-21T03:42:13.518 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mds 70796 70809 io_contex ceph txt unknown /proc/70796/task/70809/exe (readlink: Permission denied) 2024-09-21T03:42:13.518 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mds 70796 70809 io_contex ceph NOFD /proc/70796/task/70809/fd (opendir: Permission denied) 2024-09-21T03:42:13.518 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mds 70796 70810 ms_dispat ceph cwd unknown /proc/70796/task/70810/cwd (readlink: Permission denied) 2024-09-21T03:42:13.518 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mds 70796 70810 ms_dispat ceph rtd unknown /proc/70796/task/70810/root (readlink: Permission denied) 2024-09-21T03:42:13.518 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mds 70796 70810 ms_dispat ceph txt unknown /proc/70796/task/70810/exe (readlink: Permission denied) 2024-09-21T03:42:13.518 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mds 70796 70810 ms_dispat ceph NOFD /proc/70796/task/70810/fd (opendir: Permission denied) 2024-09-21T03:42:13.518 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mds 70796 70811 ms_local ceph cwd unknown /proc/70796/task/70811/cwd (readlink: Permission denied) 2024-09-21T03:42:13.518 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mds 70796 70811 ms_local ceph rtd unknown /proc/70796/task/70811/root (readlink: Permission denied) 2024-09-21T03:42:13.518 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mds 70796 70811 ms_local ceph txt unknown /proc/70796/task/70811/exe (readlink: Permission denied) 2024-09-21T03:42:13.519 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mds 70796 70811 ms_local ceph NOFD /proc/70796/task/70811/fd (opendir: Permission denied) 2024-09-21T03:42:13.519 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mds 70796 70812 safe_time ceph cwd unknown /proc/70796/task/70812/cwd (readlink: Permission denied) 2024-09-21T03:42:13.519 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mds 70796 70812 safe_time ceph rtd unknown /proc/70796/task/70812/root (readlink: Permission denied) 2024-09-21T03:42:13.519 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mds 70796 70812 safe_time ceph txt unknown /proc/70796/task/70812/exe (readlink: Permission denied) 2024-09-21T03:42:13.519 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mds 70796 70812 safe_time ceph NOFD /proc/70796/task/70812/fd (opendir: Permission denied) 2024-09-21T03:42:13.519 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mds 70796 70813 safe_time ceph cwd unknown /proc/70796/task/70813/cwd (readlink: Permission denied) 2024-09-21T03:42:13.519 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mds 70796 70813 safe_time ceph rtd unknown /proc/70796/task/70813/root (readlink: Permission denied) 2024-09-21T03:42:13.519 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mds 70796 70813 safe_time ceph txt unknown /proc/70796/task/70813/exe (readlink: Permission denied) 2024-09-21T03:42:13.519 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mds 70796 70813 safe_time ceph NOFD /proc/70796/task/70813/fd (opendir: Permission denied) 2024-09-21T03:42:13.519 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mds 70796 70814 ceph-mds ceph cwd unknown /proc/70796/task/70814/cwd (readlink: Permission denied) 2024-09-21T03:42:13.519 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mds 70796 70814 ceph-mds ceph rtd unknown /proc/70796/task/70814/root (readlink: Permission denied) 2024-09-21T03:42:13.519 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mds 70796 70814 ceph-mds ceph txt unknown /proc/70796/task/70814/exe (readlink: Permission denied) 2024-09-21T03:42:13.520 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mds 70796 70814 ceph-mds ceph NOFD /proc/70796/task/70814/fd (opendir: Permission denied) 2024-09-21T03:42:13.520 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mds 70796 70890 safe_time ceph cwd unknown /proc/70796/task/70890/cwd (readlink: Permission denied) 2024-09-21T03:42:13.520 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mds 70796 70890 safe_time ceph rtd unknown /proc/70796/task/70890/root (readlink: Permission denied) 2024-09-21T03:42:13.520 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mds 70796 70890 safe_time ceph txt unknown /proc/70796/task/70890/exe (readlink: Permission denied) 2024-09-21T03:42:13.520 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mds 70796 70890 safe_time ceph NOFD /proc/70796/task/70890/fd (opendir: Permission denied) 2024-09-21T03:42:13.520 INFO:tasks.workunit.client.0.smithi089.stdout:conmon 71550 root cwd unknown /proc/71550/cwd (readlink: Permission denied) 2024-09-21T03:42:13.520 INFO:tasks.workunit.client.0.smithi089.stdout:conmon 71550 root rtd unknown /proc/71550/root (readlink: Permission denied) 2024-09-21T03:42:13.520 INFO:tasks.workunit.client.0.smithi089.stdout:conmon 71550 root txt unknown /proc/71550/exe (readlink: Permission denied) 2024-09-21T03:42:13.520 INFO:tasks.workunit.client.0.smithi089.stdout:conmon 71550 root NOFD /proc/71550/fd (opendir: Permission denied) 2024-09-21T03:42:13.520 INFO:tasks.workunit.client.0.smithi089.stdout:podman-in 71552 root cwd unknown /proc/71552/cwd (readlink: Permission denied) 2024-09-21T03:42:13.520 INFO:tasks.workunit.client.0.smithi089.stdout:podman-in 71552 root rtd unknown /proc/71552/root (readlink: Permission denied) 2024-09-21T03:42:13.520 INFO:tasks.workunit.client.0.smithi089.stdout:podman-in 71552 root txt unknown /proc/71552/exe (readlink: Permission denied) 2024-09-21T03:42:13.520 INFO:tasks.workunit.client.0.smithi089.stdout:podman-in 71552 root NOFD /proc/71552/fd (opendir: Permission denied) 2024-09-21T03:42:13.521 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mds 71554 ceph cwd unknown /proc/71554/cwd (readlink: Permission denied) 2024-09-21T03:42:13.521 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mds 71554 ceph rtd unknown /proc/71554/root (readlink: Permission denied) 2024-09-21T03:42:13.521 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mds 71554 ceph txt unknown /proc/71554/exe (readlink: Permission denied) 2024-09-21T03:42:13.521 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mds 71554 ceph NOFD /proc/71554/fd (opendir: Permission denied) 2024-09-21T03:42:13.521 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mds 71554 71567 log ceph cwd unknown /proc/71554/task/71567/cwd (readlink: Permission denied) 2024-09-21T03:42:13.521 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mds 71554 71567 log ceph rtd unknown /proc/71554/task/71567/root (readlink: Permission denied) 2024-09-21T03:42:13.521 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mds 71554 71567 log ceph txt unknown /proc/71554/task/71567/exe (readlink: Permission denied) 2024-09-21T03:42:13.521 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mds 71554 71567 log ceph NOFD /proc/71554/task/71567/fd (opendir: Permission denied) 2024-09-21T03:42:13.521 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mds 71554 71569 msgr-work ceph cwd unknown /proc/71554/task/71569/cwd (readlink: Permission denied) 2024-09-21T03:42:13.521 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mds 71554 71569 msgr-work ceph rtd unknown /proc/71554/task/71569/root (readlink: Permission denied) 2024-09-21T03:42:13.521 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mds 71554 71569 msgr-work ceph txt unknown /proc/71554/task/71569/exe (readlink: Permission denied) 2024-09-21T03:42:13.521 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mds 71554 71569 msgr-work ceph NOFD /proc/71554/task/71569/fd (opendir: Permission denied) 2024-09-21T03:42:13.521 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mds 71554 71570 msgr-work ceph cwd unknown /proc/71554/task/71570/cwd (readlink: Permission denied) 2024-09-21T03:42:13.522 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mds 71554 71570 msgr-work ceph rtd unknown /proc/71554/task/71570/root (readlink: Permission denied) 2024-09-21T03:42:13.522 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mds 71554 71570 msgr-work ceph txt unknown /proc/71554/task/71570/exe (readlink: Permission denied) 2024-09-21T03:42:13.522 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mds 71554 71570 msgr-work ceph NOFD /proc/71554/task/71570/fd (opendir: Permission denied) 2024-09-21T03:42:13.522 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mds 71554 71571 msgr-work ceph cwd unknown /proc/71554/task/71571/cwd (readlink: Permission denied) 2024-09-21T03:42:13.522 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mds 71554 71571 msgr-work ceph rtd unknown /proc/71554/task/71571/root (readlink: Permission denied) 2024-09-21T03:42:13.522 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mds 71554 71571 msgr-work ceph txt unknown /proc/71554/task/71571/exe (readlink: Permission denied) 2024-09-21T03:42:13.522 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mds 71554 71571 msgr-work ceph NOFD /proc/71554/task/71571/fd (opendir: Permission denied) 2024-09-21T03:42:13.522 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mds 71554 71575 service ceph cwd unknown /proc/71554/task/71575/cwd (readlink: Permission denied) 2024-09-21T03:42:13.522 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mds 71554 71575 service ceph rtd unknown /proc/71554/task/71575/root (readlink: Permission denied) 2024-09-21T03:42:13.522 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mds 71554 71575 service ceph txt unknown /proc/71554/task/71575/exe (readlink: Permission denied) 2024-09-21T03:42:13.522 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mds 71554 71575 service ceph NOFD /proc/71554/task/71575/fd (opendir: Permission denied) 2024-09-21T03:42:13.523 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mds 71554 71576 admin_soc ceph cwd unknown /proc/71554/task/71576/cwd (readlink: Permission denied) 2024-09-21T03:42:13.523 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mds 71554 71576 admin_soc ceph rtd unknown /proc/71554/task/71576/root (readlink: Permission denied) 2024-09-21T03:42:13.523 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mds 71554 71576 admin_soc ceph txt unknown /proc/71554/task/71576/exe (readlink: Permission denied) 2024-09-21T03:42:13.523 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mds 71554 71576 admin_soc ceph NOFD /proc/71554/task/71576/fd (opendir: Permission denied) 2024-09-21T03:42:13.523 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mds 71554 71577 signal_ha ceph cwd unknown /proc/71554/task/71577/cwd (readlink: Permission denied) 2024-09-21T03:42:13.523 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mds 71554 71577 signal_ha ceph rtd unknown /proc/71554/task/71577/root (readlink: Permission denied) 2024-09-21T03:42:13.523 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mds 71554 71577 signal_ha ceph txt unknown /proc/71554/task/71577/exe (readlink: Permission denied) 2024-09-21T03:42:13.523 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mds 71554 71577 signal_ha ceph NOFD /proc/71554/task/71577/fd (opendir: Permission denied) 2024-09-21T03:42:13.523 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mds 71554 71578 io_contex ceph cwd unknown /proc/71554/task/71578/cwd (readlink: Permission denied) 2024-09-21T03:42:13.523 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mds 71554 71578 io_contex ceph rtd unknown /proc/71554/task/71578/root (readlink: Permission denied) 2024-09-21T03:42:13.523 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mds 71554 71578 io_contex ceph txt unknown /proc/71554/task/71578/exe (readlink: Permission denied) 2024-09-21T03:42:13.524 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mds 71554 71578 io_contex ceph NOFD /proc/71554/task/71578/fd (opendir: Permission denied) 2024-09-21T03:42:13.524 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mds 71554 71579 io_contex ceph cwd unknown /proc/71554/task/71579/cwd (readlink: Permission denied) 2024-09-21T03:42:13.524 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mds 71554 71579 io_contex ceph rtd unknown /proc/71554/task/71579/root (readlink: Permission denied) 2024-09-21T03:42:13.524 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mds 71554 71579 io_contex ceph txt unknown /proc/71554/task/71579/exe (readlink: Permission denied) 2024-09-21T03:42:13.524 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mds 71554 71579 io_contex ceph NOFD /proc/71554/task/71579/fd (opendir: Permission denied) 2024-09-21T03:42:13.524 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mds 71554 71580 ms_dispat ceph cwd unknown /proc/71554/task/71580/cwd (readlink: Permission denied) 2024-09-21T03:42:13.524 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mds 71554 71580 ms_dispat ceph rtd unknown /proc/71554/task/71580/root (readlink: Permission denied) 2024-09-21T03:42:13.524 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mds 71554 71580 ms_dispat ceph txt unknown /proc/71554/task/71580/exe (readlink: Permission denied) 2024-09-21T03:42:13.524 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mds 71554 71580 ms_dispat ceph NOFD /proc/71554/task/71580/fd (opendir: Permission denied) 2024-09-21T03:42:13.524 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mds 71554 71581 ms_local ceph cwd unknown /proc/71554/task/71581/cwd (readlink: Permission denied) 2024-09-21T03:42:13.524 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mds 71554 71581 ms_local ceph rtd unknown /proc/71554/task/71581/root (readlink: Permission denied) 2024-09-21T03:42:13.524 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mds 71554 71581 ms_local ceph txt unknown /proc/71554/task/71581/exe (readlink: Permission denied) 2024-09-21T03:42:13.525 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mds 71554 71581 ms_local ceph NOFD /proc/71554/task/71581/fd (opendir: Permission denied) 2024-09-21T03:42:13.525 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mds 71554 71582 safe_time ceph cwd unknown /proc/71554/task/71582/cwd (readlink: Permission denied) 2024-09-21T03:42:13.525 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mds 71554 71582 safe_time ceph rtd unknown /proc/71554/task/71582/root (readlink: Permission denied) 2024-09-21T03:42:13.525 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mds 71554 71582 safe_time ceph txt unknown /proc/71554/task/71582/exe (readlink: Permission denied) 2024-09-21T03:42:13.525 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mds 71554 71582 safe_time ceph NOFD /proc/71554/task/71582/fd (opendir: Permission denied) 2024-09-21T03:42:13.525 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mds 71554 71583 safe_time ceph cwd unknown /proc/71554/task/71583/cwd (readlink: Permission denied) 2024-09-21T03:42:13.525 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mds 71554 71583 safe_time ceph rtd unknown /proc/71554/task/71583/root (readlink: Permission denied) 2024-09-21T03:42:13.525 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mds 71554 71583 safe_time ceph txt unknown /proc/71554/task/71583/exe (readlink: Permission denied) 2024-09-21T03:42:13.525 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mds 71554 71583 safe_time ceph NOFD /proc/71554/task/71583/fd (opendir: Permission denied) 2024-09-21T03:42:13.525 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mds 71554 71584 ceph-mds ceph cwd unknown /proc/71554/task/71584/cwd (readlink: Permission denied) 2024-09-21T03:42:13.525 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mds 71554 71584 ceph-mds ceph rtd unknown /proc/71554/task/71584/root (readlink: Permission denied) 2024-09-21T03:42:13.525 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mds 71554 71584 ceph-mds ceph txt unknown /proc/71554/task/71584/exe (readlink: Permission denied) 2024-09-21T03:42:13.525 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mds 71554 71584 ceph-mds ceph NOFD /proc/71554/task/71584/fd (opendir: Permission denied) 2024-09-21T03:42:13.525 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mds 71554 71747 safe_time ceph cwd unknown /proc/71554/task/71747/cwd (readlink: Permission denied) 2024-09-21T03:42:13.526 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mds 71554 71747 safe_time ceph rtd unknown /proc/71554/task/71747/root (readlink: Permission denied) 2024-09-21T03:42:13.526 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mds 71554 71747 safe_time ceph txt unknown /proc/71554/task/71747/exe (readlink: Permission denied) 2024-09-21T03:42:13.526 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mds 71554 71747 safe_time ceph NOFD /proc/71554/task/71747/fd (opendir: Permission denied) 2024-09-21T03:42:13.526 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/u 71768 root cwd unknown /proc/71768/cwd (readlink: Permission denied) 2024-09-21T03:42:13.526 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/u 71768 root rtd unknown /proc/71768/root (readlink: Permission denied) 2024-09-21T03:42:13.526 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/u 71768 root txt unknown /proc/71768/exe (readlink: Permission denied) 2024-09-21T03:42:13.526 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/u 71768 root NOFD /proc/71768/fd (opendir: Permission denied) 2024-09-21T03:42:13.526 INFO:tasks.workunit.client.0.smithi089.stdout:conmon 71915 root cwd unknown /proc/71915/cwd (readlink: Permission denied) 2024-09-21T03:42:13.526 INFO:tasks.workunit.client.0.smithi089.stdout:conmon 71915 root rtd unknown /proc/71915/root (readlink: Permission denied) 2024-09-21T03:42:13.526 INFO:tasks.workunit.client.0.smithi089.stdout:conmon 71915 root txt unknown /proc/71915/exe (readlink: Permission denied) 2024-09-21T03:42:13.526 INFO:tasks.workunit.client.0.smithi089.stdout:conmon 71915 root NOFD /proc/71915/fd (opendir: Permission denied) 2024-09-21T03:42:13.526 INFO:tasks.workunit.client.0.smithi089.stdout:podman-in 71917 root cwd unknown /proc/71917/cwd (readlink: Permission denied) 2024-09-21T03:42:13.527 INFO:tasks.workunit.client.0.smithi089.stdout:podman-in 71917 root rtd unknown /proc/71917/root (readlink: Permission denied) 2024-09-21T03:42:13.527 INFO:tasks.workunit.client.0.smithi089.stdout:podman-in 71917 root txt unknown /proc/71917/exe (readlink: Permission denied) 2024-09-21T03:42:13.527 INFO:tasks.workunit.client.0.smithi089.stdout:podman-in 71917 root NOFD /proc/71917/fd (opendir: Permission denied) 2024-09-21T03:42:13.527 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mds 71919 ceph cwd unknown /proc/71919/cwd (readlink: Permission denied) 2024-09-21T03:42:13.527 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mds 71919 ceph rtd unknown /proc/71919/root (readlink: Permission denied) 2024-09-21T03:42:13.527 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mds 71919 ceph txt unknown /proc/71919/exe (readlink: Permission denied) 2024-09-21T03:42:13.527 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mds 71919 ceph NOFD /proc/71919/fd (opendir: Permission denied) 2024-09-21T03:42:13.527 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mds 71919 71920 log ceph cwd unknown /proc/71919/task/71920/cwd (readlink: Permission denied) 2024-09-21T03:42:13.527 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mds 71919 71920 log ceph rtd unknown /proc/71919/task/71920/root (readlink: Permission denied) 2024-09-21T03:42:13.527 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mds 71919 71920 log ceph txt unknown /proc/71919/task/71920/exe (readlink: Permission denied) 2024-09-21T03:42:13.527 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mds 71919 71920 log ceph NOFD /proc/71919/task/71920/fd (opendir: Permission denied) 2024-09-21T03:42:13.527 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mds 71919 71922 msgr-work ceph cwd unknown /proc/71919/task/71922/cwd (readlink: Permission denied) 2024-09-21T03:42:13.527 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mds 71919 71922 msgr-work ceph rtd unknown /proc/71919/task/71922/root (readlink: Permission denied) 2024-09-21T03:42:13.528 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mds 71919 71922 msgr-work ceph txt unknown /proc/71919/task/71922/exe (readlink: Permission denied) 2024-09-21T03:42:13.528 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mds 71919 71922 msgr-work ceph NOFD /proc/71919/task/71922/fd (opendir: Permission denied) 2024-09-21T03:42:13.528 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mds 71919 71923 msgr-work ceph cwd unknown /proc/71919/task/71923/cwd (readlink: Permission denied) 2024-09-21T03:42:13.528 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mds 71919 71923 msgr-work ceph rtd unknown /proc/71919/task/71923/root (readlink: Permission denied) 2024-09-21T03:42:13.528 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mds 71919 71923 msgr-work ceph txt unknown /proc/71919/task/71923/exe (readlink: Permission denied) 2024-09-21T03:42:13.528 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mds 71919 71923 msgr-work ceph NOFD /proc/71919/task/71923/fd (opendir: Permission denied) 2024-09-21T03:42:13.528 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mds 71919 71924 msgr-work ceph cwd unknown /proc/71919/task/71924/cwd (readlink: Permission denied) 2024-09-21T03:42:13.528 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mds 71919 71924 msgr-work ceph rtd unknown /proc/71919/task/71924/root (readlink: Permission denied) 2024-09-21T03:42:13.528 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mds 71919 71924 msgr-work ceph txt unknown /proc/71919/task/71924/exe (readlink: Permission denied) 2024-09-21T03:42:13.528 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mds 71919 71924 msgr-work ceph NOFD /proc/71919/task/71924/fd (opendir: Permission denied) 2024-09-21T03:42:13.528 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mds 71919 71928 service ceph cwd unknown /proc/71919/task/71928/cwd (readlink: Permission denied) 2024-09-21T03:42:13.528 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mds 71919 71928 service ceph rtd unknown /proc/71919/task/71928/root (readlink: Permission denied) 2024-09-21T03:42:13.528 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mds 71919 71928 service ceph txt unknown /proc/71919/task/71928/exe (readlink: Permission denied) 2024-09-21T03:42:13.528 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mds 71919 71928 service ceph NOFD /proc/71919/task/71928/fd (opendir: Permission denied) 2024-09-21T03:42:13.529 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mds 71919 71929 admin_soc ceph cwd unknown /proc/71919/task/71929/cwd (readlink: Permission denied) 2024-09-21T03:42:13.529 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mds 71919 71929 admin_soc ceph rtd unknown /proc/71919/task/71929/root (readlink: Permission denied) 2024-09-21T03:42:13.529 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mds 71919 71929 admin_soc ceph txt unknown /proc/71919/task/71929/exe (readlink: Permission denied) 2024-09-21T03:42:13.529 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mds 71919 71929 admin_soc ceph NOFD /proc/71919/task/71929/fd (opendir: Permission denied) 2024-09-21T03:42:13.529 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mds 71919 71930 signal_ha ceph cwd unknown /proc/71919/task/71930/cwd (readlink: Permission denied) 2024-09-21T03:42:13.529 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mds 71919 71930 signal_ha ceph rtd unknown /proc/71919/task/71930/root (readlink: Permission denied) 2024-09-21T03:42:13.529 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mds 71919 71930 signal_ha ceph txt unknown /proc/71919/task/71930/exe (readlink: Permission denied) 2024-09-21T03:42:13.529 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mds 71919 71930 signal_ha ceph NOFD /proc/71919/task/71930/fd (opendir: Permission denied) 2024-09-21T03:42:13.529 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mds 71919 71931 io_contex ceph cwd unknown /proc/71919/task/71931/cwd (readlink: Permission denied) 2024-09-21T03:42:13.529 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mds 71919 71931 io_contex ceph rtd unknown /proc/71919/task/71931/root (readlink: Permission denied) 2024-09-21T03:42:13.529 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mds 71919 71931 io_contex ceph txt unknown /proc/71919/task/71931/exe (readlink: Permission denied) 2024-09-21T03:42:13.529 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mds 71919 71931 io_contex ceph NOFD /proc/71919/task/71931/fd (opendir: Permission denied) 2024-09-21T03:42:13.529 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mds 71919 71932 io_contex ceph cwd unknown /proc/71919/task/71932/cwd (readlink: Permission denied) 2024-09-21T03:42:13.529 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mds 71919 71932 io_contex ceph rtd unknown /proc/71919/task/71932/root (readlink: Permission denied) 2024-09-21T03:42:13.529 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mds 71919 71932 io_contex ceph txt unknown /proc/71919/task/71932/exe (readlink: Permission denied) 2024-09-21T03:42:13.530 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mds 71919 71932 io_contex ceph NOFD /proc/71919/task/71932/fd (opendir: Permission denied) 2024-09-21T03:42:13.530 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mds 71919 71933 ms_dispat ceph cwd unknown /proc/71919/task/71933/cwd (readlink: Permission denied) 2024-09-21T03:42:13.530 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mds 71919 71933 ms_dispat ceph rtd unknown /proc/71919/task/71933/root (readlink: Permission denied) 2024-09-21T03:42:13.530 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mds 71919 71933 ms_dispat ceph txt unknown /proc/71919/task/71933/exe (readlink: Permission denied) 2024-09-21T03:42:13.530 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mds 71919 71933 ms_dispat ceph NOFD /proc/71919/task/71933/fd (opendir: Permission denied) 2024-09-21T03:42:13.530 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mds 71919 71934 ms_local ceph cwd unknown /proc/71919/task/71934/cwd (readlink: Permission denied) 2024-09-21T03:42:13.530 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mds 71919 71934 ms_local ceph rtd unknown /proc/71919/task/71934/root (readlink: Permission denied) 2024-09-21T03:42:13.530 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mds 71919 71934 ms_local ceph txt unknown /proc/71919/task/71934/exe (readlink: Permission denied) 2024-09-21T03:42:13.530 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mds 71919 71934 ms_local ceph NOFD /proc/71919/task/71934/fd (opendir: Permission denied) 2024-09-21T03:42:13.530 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mds 71919 71935 safe_time ceph cwd unknown /proc/71919/task/71935/cwd (readlink: Permission denied) 2024-09-21T03:42:13.530 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mds 71919 71935 safe_time ceph rtd unknown /proc/71919/task/71935/root (readlink: Permission denied) 2024-09-21T03:42:13.530 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mds 71919 71935 safe_time ceph txt unknown /proc/71919/task/71935/exe (readlink: Permission denied) 2024-09-21T03:42:13.530 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mds 71919 71935 safe_time ceph NOFD /proc/71919/task/71935/fd (opendir: Permission denied) 2024-09-21T03:42:13.530 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mds 71919 71936 safe_time ceph cwd unknown /proc/71919/task/71936/cwd (readlink: Permission denied) 2024-09-21T03:42:13.531 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mds 71919 71936 safe_time ceph rtd unknown /proc/71919/task/71936/root (readlink: Permission denied) 2024-09-21T03:42:13.531 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mds 71919 71936 safe_time ceph txt unknown /proc/71919/task/71936/exe (readlink: Permission denied) 2024-09-21T03:42:13.531 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mds 71919 71936 safe_time ceph NOFD /proc/71919/task/71936/fd (opendir: Permission denied) 2024-09-21T03:42:13.531 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mds 71919 71937 ceph-mds ceph cwd unknown /proc/71919/task/71937/cwd (readlink: Permission denied) 2024-09-21T03:42:13.531 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mds 71919 71937 ceph-mds ceph rtd unknown /proc/71919/task/71937/root (readlink: Permission denied) 2024-09-21T03:42:13.531 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mds 71919 71937 ceph-mds ceph txt unknown /proc/71919/task/71937/exe (readlink: Permission denied) 2024-09-21T03:42:13.531 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mds 71919 71937 ceph-mds ceph NOFD /proc/71919/task/71937/fd (opendir: Permission denied) 2024-09-21T03:42:13.531 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mds 71919 72020 safe_time ceph cwd unknown /proc/71919/task/72020/cwd (readlink: Permission denied) 2024-09-21T03:42:13.531 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mds 71919 72020 safe_time ceph rtd unknown /proc/71919/task/72020/root (readlink: Permission denied) 2024-09-21T03:42:13.531 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mds 71919 72020 safe_time ceph txt unknown /proc/71919/task/72020/exe (readlink: Permission denied) 2024-09-21T03:42:13.531 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-mds 71919 72020 safe_time ceph NOFD /proc/71919/task/72020/fd (opendir: Permission denied) 2024-09-21T03:42:13.531 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/1 74379 root cwd unknown /proc/74379/cwd (readlink: Permission denied) 2024-09-21T03:42:13.531 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/1 74379 root rtd unknown /proc/74379/root (readlink: Permission denied) 2024-09-21T03:42:13.531 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/1 74379 root txt unknown /proc/74379/exe (readlink: Permission denied) 2024-09-21T03:42:13.532 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/1 74379 root NOFD /proc/74379/fd (opendir: Permission denied) 2024-09-21T03:42:13.532 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/3 79026 root cwd unknown /proc/79026/cwd (readlink: Permission denied) 2024-09-21T03:42:13.532 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/3 79026 root rtd unknown /proc/79026/root (readlink: Permission denied) 2024-09-21T03:42:13.532 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/3 79026 root txt unknown /proc/79026/exe (readlink: Permission denied) 2024-09-21T03:42:13.532 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/3 79026 root NOFD /proc/79026/fd (opendir: Permission denied) 2024-09-21T03:42:13.532 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/4 79618 root cwd unknown /proc/79618/cwd (readlink: Permission denied) 2024-09-21T03:42:13.532 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/4 79618 root rtd unknown /proc/79618/root (readlink: Permission denied) 2024-09-21T03:42:13.532 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/4 79618 root txt unknown /proc/79618/exe (readlink: Permission denied) 2024-09-21T03:42:13.532 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/4 79618 root NOFD /proc/79618/fd (opendir: Permission denied) 2024-09-21T03:42:13.532 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/6 84023 root cwd unknown /proc/84023/cwd (readlink: Permission denied) 2024-09-21T03:42:13.532 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/6 84023 root rtd unknown /proc/84023/root (readlink: Permission denied) 2024-09-21T03:42:13.532 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/6 84023 root txt unknown /proc/84023/exe (readlink: Permission denied) 2024-09-21T03:42:13.532 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/6 84023 root NOFD /proc/84023/fd (opendir: Permission denied) 2024-09-21T03:42:13.532 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/6 84024 root cwd unknown /proc/84024/cwd (readlink: Permission denied) 2024-09-21T03:42:13.533 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/6 84024 root rtd unknown /proc/84024/root (readlink: Permission denied) 2024-09-21T03:42:13.533 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/6 84024 root txt unknown /proc/84024/exe (readlink: Permission denied) 2024-09-21T03:42:13.533 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/6 84024 root NOFD /proc/84024/fd (opendir: Permission denied) 2024-09-21T03:42:13.533 INFO:tasks.workunit.client.0.smithi089.stdout:bash 84508 ubuntu cwd DIR 8,1 4096 5553 /home/ubuntu 2024-09-21T03:42:13.533 INFO:tasks.workunit.client.0.smithi089.stdout:bash 84508 ubuntu rtd DIR 8,1 4096 2 / 2024-09-21T03:42:13.533 INFO:tasks.workunit.client.0.smithi089.stdout:bash 84508 ubuntu txt REG 8,1 1389072 11254 /usr/bin/bash 2024-09-21T03:42:13.533 INFO:tasks.workunit.client.0.smithi089.stdout:bash 84508 ubuntu mem REG 8,1 2586930 19889 /usr/lib/locale/en_US.utf8/LC_COLLATE 2024-09-21T03:42:13.533 INFO:tasks.workunit.client.0.smithi089.stdout:bash 84508 ubuntu mem REG 8,1 2544296 11212 /usr/lib64/libc.so.6 2024-09-21T03:42:13.533 INFO:tasks.workunit.client.0.smithi089.stdout:bash 84508 ubuntu mem REG 8,1 346132 32690 /usr/lib/locale/C.utf8/LC_CTYPE 2024-09-21T03:42:13.533 INFO:tasks.workunit.client.0.smithi089.stdout:bash 84508 ubuntu mem REG 8,1 54 19892 /usr/lib/locale/en_US.utf8/LC_NUMERIC 2024-09-21T03:42:13.533 INFO:tasks.workunit.client.0.smithi089.stdout:bash 84508 ubuntu mem REG 8,1 3284 32609 /usr/lib/locale/en_US.utf8/LC_TIME 2024-09-21T03:42:13.533 INFO:tasks.workunit.client.0.smithi089.stdout:bash 84508 ubuntu mem REG 8,1 286 32606 /usr/lib/locale/en_US.utf8/LC_MONETARY 2024-09-21T03:42:13.533 INFO:tasks.workunit.client.0.smithi089.stdout:bash 84508 ubuntu mem REG 8,1 57 30370 /usr/lib/locale/en_US.utf8/LC_MESSAGES/SYS_LC_MESSAGES 2024-09-21T03:42:13.533 INFO:tasks.workunit.client.0.smithi089.stdout:bash 84508 ubuntu mem REG 8,1 34 32659 /usr/lib/locale/en_US.utf8/LC_PAPER 2024-09-21T03:42:13.533 INFO:tasks.workunit.client.0.smithi089.stdout:bash 84508 ubuntu mem REG 8,1 77 19891 /usr/lib/locale/en_US.utf8/LC_NAME 2024-09-21T03:42:13.533 INFO:tasks.workunit.client.0.smithi089.stdout:bash 84508 ubuntu mem REG 8,1 167 32601 /usr/lib/locale/en_US.utf8/LC_ADDRESS 2024-09-21T03:42:13.534 INFO:tasks.workunit.client.0.smithi089.stdout:bash 84508 ubuntu mem REG 8,1 59 32608 /usr/lib/locale/en_US.utf8/LC_TELEPHONE 2024-09-21T03:42:13.534 INFO:tasks.workunit.client.0.smithi089.stdout:bash 84508 ubuntu mem REG 8,1 23 32605 /usr/lib/locale/en_US.utf8/LC_MEASUREMENT 2024-09-21T03:42:13.534 INFO:tasks.workunit.client.0.smithi089.stdout:bash 84508 ubuntu mem REG 8,1 195080 11252 /usr/lib64/libtinfo.so.6.2 2024-09-21T03:42:13.534 INFO:tasks.workunit.client.0.smithi089.stdout:bash 84508 ubuntu mem REG 8,1 26988 31782 /usr/lib64/gconv/gconv-modules.cache 2024-09-21T03:42:13.534 INFO:tasks.workunit.client.0.smithi089.stdout:bash 84508 ubuntu mem REG 8,1 369 32604 /usr/lib/locale/en_US.utf8/LC_IDENTIFICATION 2024-09-21T03:42:13.534 INFO:tasks.workunit.client.0.smithi089.stdout:bash 84508 ubuntu mem REG 8,1 877424 15688 /usr/lib64/ld-linux-x86-64.so.2 2024-09-21T03:42:13.534 INFO:tasks.workunit.client.0.smithi089.stdout:bash 84508 ubuntu 0r FIFO 0,13 0t0 227256 pipe 2024-09-21T03:42:13.534 INFO:tasks.workunit.client.0.smithi089.stdout:bash 84508 ubuntu 1w FIFO 0,13 0t0 227257 pipe 2024-09-21T03:42:13.534 INFO:tasks.workunit.client.0.smithi089.stdout:bash 84508 ubuntu 2w FIFO 0,13 0t0 227258 pipe 2024-09-21T03:42:13.534 INFO:tasks.workunit.client.0.smithi089.stdout:sudo 84540 root cwd unknown /proc/84540/cwd (readlink: Permission denied) 2024-09-21T03:42:13.534 INFO:tasks.workunit.client.0.smithi089.stdout:sudo 84540 root rtd unknown /proc/84540/root (readlink: Permission denied) 2024-09-21T03:42:13.534 INFO:tasks.workunit.client.0.smithi089.stdout:sudo 84540 root txt unknown /proc/84540/exe (readlink: Permission denied) 2024-09-21T03:42:13.534 INFO:tasks.workunit.client.0.smithi089.stdout:sudo 84540 root NOFD /proc/84540/fd (opendir: Permission denied) 2024-09-21T03:42:13.534 INFO:tasks.workunit.client.0.smithi089.stdout:sudo 84543 root cwd unknown /proc/84543/cwd (readlink: Permission denied) 2024-09-21T03:42:13.535 INFO:tasks.workunit.client.0.smithi089.stdout:sudo 84543 root rtd unknown /proc/84543/root (readlink: Permission denied) 2024-09-21T03:42:13.535 INFO:tasks.workunit.client.0.smithi089.stdout:sudo 84543 root txt unknown /proc/84543/exe (readlink: Permission denied) 2024-09-21T03:42:13.535 INFO:tasks.workunit.client.0.smithi089.stdout:sudo 84543 root NOFD /proc/84543/fd (opendir: Permission denied) 2024-09-21T03:42:13.535 INFO:tasks.workunit.client.0.smithi089.stdout:daemon-he 84546 root cwd unknown /proc/84546/cwd (readlink: Permission denied) 2024-09-21T03:42:13.535 INFO:tasks.workunit.client.0.smithi089.stdout:daemon-he 84546 root rtd unknown /proc/84546/root (readlink: Permission denied) 2024-09-21T03:42:13.535 INFO:tasks.workunit.client.0.smithi089.stdout:daemon-he 84546 root txt unknown /proc/84546/exe (readlink: Permission denied) 2024-09-21T03:42:13.535 INFO:tasks.workunit.client.0.smithi089.stdout:daemon-he 84546 root NOFD /proc/84546/fd (opendir: Permission denied) 2024-09-21T03:42:13.535 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-fuse 84548 root cwd unknown /proc/84548/cwd (readlink: Permission denied) 2024-09-21T03:42:13.535 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-fuse 84548 root rtd unknown /proc/84548/root (readlink: Permission denied) 2024-09-21T03:42:13.535 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-fuse 84548 root txt unknown /proc/84548/exe (readlink: Permission denied) 2024-09-21T03:42:13.535 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-fuse 84548 root NOFD /proc/84548/fd (opendir: Permission denied) 2024-09-21T03:42:13.535 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-fuse 84548 84567 log root cwd unknown /proc/84548/task/84567/cwd (readlink: Permission denied) 2024-09-21T03:42:13.535 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-fuse 84548 84567 log root rtd unknown /proc/84548/task/84567/root (readlink: Permission denied) 2024-09-21T03:42:13.535 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-fuse 84548 84567 log root txt unknown /proc/84548/task/84567/exe (readlink: Permission denied) 2024-09-21T03:42:13.535 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-fuse 84548 84567 log root NOFD /proc/84548/task/84567/fd (opendir: Permission denied) 2024-09-21T03:42:13.536 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-fuse 84548 84569 msgr-work root cwd unknown /proc/84548/task/84569/cwd (readlink: Permission denied) 2024-09-21T03:42:13.536 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-fuse 84548 84569 msgr-work root rtd unknown /proc/84548/task/84569/root (readlink: Permission denied) 2024-09-21T03:42:13.536 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-fuse 84548 84569 msgr-work root txt unknown /proc/84548/task/84569/exe (readlink: Permission denied) 2024-09-21T03:42:13.536 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-fuse 84548 84569 msgr-work root NOFD /proc/84548/task/84569/fd (opendir: Permission denied) 2024-09-21T03:42:13.536 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-fuse 84548 84570 msgr-work root cwd unknown /proc/84548/task/84570/cwd (readlink: Permission denied) 2024-09-21T03:42:13.536 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-fuse 84548 84570 msgr-work root rtd unknown /proc/84548/task/84570/root (readlink: Permission denied) 2024-09-21T03:42:13.536 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-fuse 84548 84570 msgr-work root txt unknown /proc/84548/task/84570/exe (readlink: Permission denied) 2024-09-21T03:42:13.536 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-fuse 84548 84570 msgr-work root NOFD /proc/84548/task/84570/fd (opendir: Permission denied) 2024-09-21T03:42:13.536 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-fuse 84548 84571 msgr-work root cwd unknown /proc/84548/task/84571/cwd (readlink: Permission denied) 2024-09-21T03:42:13.536 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-fuse 84548 84571 msgr-work root rtd unknown /proc/84548/task/84571/root (readlink: Permission denied) 2024-09-21T03:42:13.536 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-fuse 84548 84571 msgr-work root txt unknown /proc/84548/task/84571/exe (readlink: Permission denied) 2024-09-21T03:42:13.536 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-fuse 84548 84571 msgr-work root NOFD /proc/84548/task/84571/fd (opendir: Permission denied) 2024-09-21T03:42:13.536 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-fuse 84548 84575 service root cwd unknown /proc/84548/task/84575/cwd (readlink: Permission denied) 2024-09-21T03:42:13.536 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-fuse 84548 84575 service root rtd unknown /proc/84548/task/84575/root (readlink: Permission denied) 2024-09-21T03:42:13.536 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-fuse 84548 84575 service root txt unknown /proc/84548/task/84575/exe (readlink: Permission denied) 2024-09-21T03:42:13.537 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-fuse 84548 84575 service root NOFD /proc/84548/task/84575/fd (opendir: Permission denied) 2024-09-21T03:42:13.537 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-fuse 84548 84576 admin_soc root cwd unknown /proc/84548/task/84576/cwd (readlink: Permission denied) 2024-09-21T03:42:13.537 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-fuse 84548 84576 admin_soc root rtd unknown /proc/84548/task/84576/root (readlink: Permission denied) 2024-09-21T03:42:13.537 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-fuse 84548 84576 admin_soc root txt unknown /proc/84548/task/84576/exe (readlink: Permission denied) 2024-09-21T03:42:13.537 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-fuse 84548 84576 admin_soc root NOFD /proc/84548/task/84576/fd (opendir: Permission denied) 2024-09-21T03:42:13.537 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-fuse 84548 84577 signal_ha root cwd unknown /proc/84548/task/84577/cwd (readlink: Permission denied) 2024-09-21T03:42:13.537 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-fuse 84548 84577 signal_ha root rtd unknown /proc/84548/task/84577/root (readlink: Permission denied) 2024-09-21T03:42:13.537 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-fuse 84548 84577 signal_ha root txt unknown /proc/84548/task/84577/exe (readlink: Permission denied) 2024-09-21T03:42:13.537 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-fuse 84548 84577 signal_ha root NOFD /proc/84548/task/84577/fd (opendir: Permission denied) 2024-09-21T03:42:13.537 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-fuse 84548 84578 io_contex root cwd unknown /proc/84548/task/84578/cwd (readlink: Permission denied) 2024-09-21T03:42:13.537 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-fuse 84548 84578 io_contex root rtd unknown /proc/84548/task/84578/root (readlink: Permission denied) 2024-09-21T03:42:13.537 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-fuse 84548 84578 io_contex root txt unknown /proc/84548/task/84578/exe (readlink: Permission denied) 2024-09-21T03:42:13.537 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-fuse 84548 84578 io_contex root NOFD /proc/84548/task/84578/fd (opendir: Permission denied) 2024-09-21T03:42:13.537 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-fuse 84548 84579 io_contex root cwd unknown /proc/84548/task/84579/cwd (readlink: Permission denied) 2024-09-21T03:42:13.537 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-fuse 84548 84579 io_contex root rtd unknown /proc/84548/task/84579/root (readlink: Permission denied) 2024-09-21T03:42:13.537 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-fuse 84548 84579 io_contex root txt unknown /proc/84548/task/84579/exe (readlink: Permission denied) 2024-09-21T03:42:13.538 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-fuse 84548 84579 io_contex root NOFD /proc/84548/task/84579/fd (opendir: Permission denied) 2024-09-21T03:42:13.538 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-fuse 84548 84580 ceph_time root cwd unknown /proc/84548/task/84580/cwd (readlink: Permission denied) 2024-09-21T03:42:13.538 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-fuse 84548 84580 ceph_time root rtd unknown /proc/84548/task/84580/root (readlink: Permission denied) 2024-09-21T03:42:13.538 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-fuse 84548 84580 ceph_time root txt unknown /proc/84548/task/84580/exe (readlink: Permission denied) 2024-09-21T03:42:13.538 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-fuse 84548 84580 ceph_time root NOFD /proc/84548/task/84580/fd (opendir: Permission denied) 2024-09-21T03:42:13.538 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-fuse 84548 84581 fn_anonym root cwd unknown /proc/84548/task/84581/cwd (readlink: Permission denied) 2024-09-21T03:42:13.538 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-fuse 84548 84581 fn_anonym root rtd unknown /proc/84548/task/84581/root (readlink: Permission denied) 2024-09-21T03:42:13.538 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-fuse 84548 84581 fn_anonym root txt unknown /proc/84548/task/84581/exe (readlink: Permission denied) 2024-09-21T03:42:13.538 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-fuse 84548 84581 fn_anonym root NOFD /proc/84548/task/84581/fd (opendir: Permission denied) 2024-09-21T03:42:13.538 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-fuse 84548 84582 fn_anonym root cwd unknown /proc/84548/task/84582/cwd (readlink: Permission denied) 2024-09-21T03:42:13.538 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-fuse 84548 84582 fn_anonym root rtd unknown /proc/84548/task/84582/root (readlink: Permission denied) 2024-09-21T03:42:13.538 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-fuse 84548 84582 fn_anonym root txt unknown /proc/84548/task/84582/exe (readlink: Permission denied) 2024-09-21T03:42:13.538 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-fuse 84548 84582 fn_anonym root NOFD /proc/84548/task/84582/fd (opendir: Permission denied) 2024-09-21T03:42:13.538 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-fuse 84548 84583 fn_anonym root cwd unknown /proc/84548/task/84583/cwd (readlink: Permission denied) 2024-09-21T03:42:13.538 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-fuse 84548 84583 fn_anonym root rtd unknown /proc/84548/task/84583/root (readlink: Permission denied) 2024-09-21T03:42:13.539 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-fuse 84548 84583 fn_anonym root txt unknown /proc/84548/task/84583/exe (readlink: Permission denied) 2024-09-21T03:42:13.539 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-fuse 84548 84583 fn_anonym root NOFD /proc/84548/task/84583/fd (opendir: Permission denied) 2024-09-21T03:42:13.539 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-fuse 84548 84584 fn_anonym root cwd unknown /proc/84548/task/84584/cwd (readlink: Permission denied) 2024-09-21T03:42:13.539 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-fuse 84548 84584 fn_anonym root rtd unknown /proc/84548/task/84584/root (readlink: Permission denied) 2024-09-21T03:42:13.539 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-fuse 84548 84584 fn_anonym root txt unknown /proc/84548/task/84584/exe (readlink: Permission denied) 2024-09-21T03:42:13.539 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-fuse 84548 84584 fn_anonym root NOFD /proc/84548/task/84584/fd (opendir: Permission denied) 2024-09-21T03:42:13.539 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-fuse 84548 84585 fn_anonym root cwd unknown /proc/84548/task/84585/cwd (readlink: Permission denied) 2024-09-21T03:42:13.539 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-fuse 84548 84585 fn_anonym root rtd unknown /proc/84548/task/84585/root (readlink: Permission denied) 2024-09-21T03:42:13.539 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-fuse 84548 84585 fn_anonym root txt unknown /proc/84548/task/84585/exe (readlink: Permission denied) 2024-09-21T03:42:13.539 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-fuse 84548 84585 fn_anonym root NOFD /proc/84548/task/84585/fd (opendir: Permission denied) 2024-09-21T03:42:13.539 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-fuse 84548 84586 safe_time root cwd unknown /proc/84548/task/84586/cwd (readlink: Permission denied) 2024-09-21T03:42:13.539 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-fuse 84548 84586 safe_time root rtd unknown /proc/84548/task/84586/root (readlink: Permission denied) 2024-09-21T03:42:13.539 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-fuse 84548 84586 safe_time root txt unknown /proc/84548/task/84586/exe (readlink: Permission denied) 2024-09-21T03:42:13.539 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-fuse 84548 84586 safe_time root NOFD /proc/84548/task/84586/fd (opendir: Permission denied) 2024-09-21T03:42:13.540 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-fuse 84548 84587 fn_anonym root cwd unknown /proc/84548/task/84587/cwd (readlink: Permission denied) 2024-09-21T03:42:13.540 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-fuse 84548 84587 fn_anonym root rtd unknown /proc/84548/task/84587/root (readlink: Permission denied) 2024-09-21T03:42:13.540 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-fuse 84548 84587 fn_anonym root txt unknown /proc/84548/task/84587/exe (readlink: Permission denied) 2024-09-21T03:42:13.540 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-fuse 84548 84587 fn_anonym root NOFD /proc/84548/task/84587/fd (opendir: Permission denied) 2024-09-21T03:42:13.540 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-fuse 84548 84588 flusher root cwd unknown /proc/84548/task/84588/cwd (readlink: Permission denied) 2024-09-21T03:42:13.540 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-fuse 84548 84588 flusher root rtd unknown /proc/84548/task/84588/root (readlink: Permission denied) 2024-09-21T03:42:13.540 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-fuse 84548 84588 flusher root txt unknown /proc/84548/task/84588/exe (readlink: Permission denied) 2024-09-21T03:42:13.540 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-fuse 84548 84588 flusher root NOFD /proc/84548/task/84588/fd (opendir: Permission denied) 2024-09-21T03:42:13.540 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-fuse 84548 84589 ms_dispat root cwd unknown /proc/84548/task/84589/cwd (readlink: Permission denied) 2024-09-21T03:42:13.540 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-fuse 84548 84589 ms_dispat root rtd unknown /proc/84548/task/84589/root (readlink: Permission denied) 2024-09-21T03:42:13.540 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-fuse 84548 84589 ms_dispat root txt unknown /proc/84548/task/84589/exe (readlink: Permission denied) 2024-09-21T03:42:13.540 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-fuse 84548 84589 ms_dispat root NOFD /proc/84548/task/84589/fd (opendir: Permission denied) 2024-09-21T03:42:13.540 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-fuse 84548 84590 ms_local root cwd unknown /proc/84548/task/84590/cwd (readlink: Permission denied) 2024-09-21T03:42:13.540 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-fuse 84548 84590 ms_local root rtd unknown /proc/84548/task/84590/root (readlink: Permission denied) 2024-09-21T03:42:13.540 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-fuse 84548 84590 ms_local root txt unknown /proc/84548/task/84590/exe (readlink: Permission denied) 2024-09-21T03:42:13.541 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-fuse 84548 84590 ms_local root NOFD /proc/84548/task/84590/fd (opendir: Permission denied) 2024-09-21T03:42:13.541 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-fuse 84548 84591 safe_time root cwd unknown /proc/84548/task/84591/cwd (readlink: Permission denied) 2024-09-21T03:42:13.541 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-fuse 84548 84591 safe_time root rtd unknown /proc/84548/task/84591/root (readlink: Permission denied) 2024-09-21T03:42:13.541 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-fuse 84548 84591 safe_time root txt unknown /proc/84548/task/84591/exe (readlink: Permission denied) 2024-09-21T03:42:13.541 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-fuse 84548 84591 safe_time root NOFD /proc/84548/task/84591/fd (opendir: Permission denied) 2024-09-21T03:42:13.541 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-fuse 84548 84593 ceph-fuse root cwd unknown /proc/84548/task/84593/cwd (readlink: Permission denied) 2024-09-21T03:42:13.541 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-fuse 84548 84593 ceph-fuse root rtd unknown /proc/84548/task/84593/root (readlink: Permission denied) 2024-09-21T03:42:13.541 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-fuse 84548 84593 ceph-fuse root txt unknown /proc/84548/task/84593/exe (readlink: Permission denied) 2024-09-21T03:42:13.541 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-fuse 84548 84593 ceph-fuse root NOFD /proc/84548/task/84593/fd (opendir: Permission denied) 2024-09-21T03:42:13.541 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-fuse 84548 84610 ceph-fuse root cwd unknown /proc/84548/task/84610/cwd (readlink: Permission denied) 2024-09-21T03:42:13.541 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-fuse 84548 84610 ceph-fuse root rtd unknown /proc/84548/task/84610/root (readlink: Permission denied) 2024-09-21T03:42:13.541 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-fuse 84548 84610 ceph-fuse root txt unknown /proc/84548/task/84610/exe (readlink: Permission denied) 2024-09-21T03:42:13.541 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-fuse 84548 84610 ceph-fuse root NOFD /proc/84548/task/84610/fd (opendir: Permission denied) 2024-09-21T03:42:13.541 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-fuse 84548 84612 ceph-fuse root cwd unknown /proc/84548/task/84612/cwd (readlink: Permission denied) 2024-09-21T03:42:13.542 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-fuse 84548 84612 ceph-fuse root rtd unknown /proc/84548/task/84612/root (readlink: Permission denied) 2024-09-21T03:42:13.542 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-fuse 84548 84612 ceph-fuse root txt unknown /proc/84548/task/84612/exe (readlink: Permission denied) 2024-09-21T03:42:13.542 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-fuse 84548 84612 ceph-fuse root NOFD /proc/84548/task/84612/fd (opendir: Permission denied) 2024-09-21T03:42:13.542 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-fuse 84548 87218 ceph-fuse root cwd unknown /proc/84548/task/87218/cwd (readlink: Permission denied) 2024-09-21T03:42:13.542 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-fuse 84548 87218 ceph-fuse root rtd unknown /proc/84548/task/87218/root (readlink: Permission denied) 2024-09-21T03:42:13.542 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-fuse 84548 87218 ceph-fuse root txt unknown /proc/84548/task/87218/exe (readlink: Permission denied) 2024-09-21T03:42:13.542 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-fuse 84548 87218 ceph-fuse root NOFD /proc/84548/task/87218/fd (opendir: Permission denied) 2024-09-21T03:42:13.542 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-fuse 84548 87282 ceph-fuse root cwd unknown /proc/84548/task/87282/cwd (readlink: Permission denied) 2024-09-21T03:42:13.542 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-fuse 84548 87282 ceph-fuse root rtd unknown /proc/84548/task/87282/root (readlink: Permission denied) 2024-09-21T03:42:13.542 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-fuse 84548 87282 ceph-fuse root txt unknown /proc/84548/task/87282/exe (readlink: Permission denied) 2024-09-21T03:42:13.542 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-fuse 84548 87282 ceph-fuse root NOFD /proc/84548/task/87282/fd (opendir: Permission denied) 2024-09-21T03:42:13.542 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-fuse 84548 87844 ceph-fuse root cwd unknown /proc/84548/task/87844/cwd (readlink: Permission denied) 2024-09-21T03:42:13.542 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-fuse 84548 87844 ceph-fuse root rtd unknown /proc/84548/task/87844/root (readlink: Permission denied) 2024-09-21T03:42:13.542 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-fuse 84548 87844 ceph-fuse root txt unknown /proc/84548/task/87844/exe (readlink: Permission denied) 2024-09-21T03:42:13.543 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-fuse 84548 87844 ceph-fuse root NOFD /proc/84548/task/87844/fd (opendir: Permission denied) 2024-09-21T03:42:13.543 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-fuse 84548 87845 ceph-fuse root cwd unknown /proc/84548/task/87845/cwd (readlink: Permission denied) 2024-09-21T03:42:13.543 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-fuse 84548 87845 ceph-fuse root rtd unknown /proc/84548/task/87845/root (readlink: Permission denied) 2024-09-21T03:42:13.543 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-fuse 84548 87845 ceph-fuse root txt unknown /proc/84548/task/87845/exe (readlink: Permission denied) 2024-09-21T03:42:13.543 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-fuse 84548 87845 ceph-fuse root NOFD /proc/84548/task/87845/fd (opendir: Permission denied) 2024-09-21T03:42:13.543 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-fuse 84548 87846 ceph-fuse root cwd unknown /proc/84548/task/87846/cwd (readlink: Permission denied) 2024-09-21T03:42:13.543 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-fuse 84548 87846 ceph-fuse root rtd unknown /proc/84548/task/87846/root (readlink: Permission denied) 2024-09-21T03:42:13.543 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-fuse 84548 87846 ceph-fuse root txt unknown /proc/84548/task/87846/exe (readlink: Permission denied) 2024-09-21T03:42:13.543 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-fuse 84548 87846 ceph-fuse root NOFD /proc/84548/task/87846/fd (opendir: Permission denied) 2024-09-21T03:42:13.543 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-fuse 84548 87996 ceph-fuse root cwd unknown /proc/84548/task/87996/cwd (readlink: Permission denied) 2024-09-21T03:42:13.543 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-fuse 84548 87996 ceph-fuse root rtd unknown /proc/84548/task/87996/root (readlink: Permission denied) 2024-09-21T03:42:13.543 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-fuse 84548 87996 ceph-fuse root txt unknown /proc/84548/task/87996/exe (readlink: Permission denied) 2024-09-21T03:42:13.543 INFO:tasks.workunit.client.0.smithi089.stdout:ceph-fuse 84548 87996 ceph-fuse root NOFD /proc/84548/task/87996/fd (opendir: Permission denied) 2024-09-21T03:42:13.543 INFO:tasks.workunit.client.0.smithi089.stdout:sudo 84829 root cwd unknown /proc/84829/cwd (readlink: Permission denied) 2024-09-21T03:42:13.543 INFO:tasks.workunit.client.0.smithi089.stdout:sudo 84829 root rtd unknown /proc/84829/root (readlink: Permission denied) 2024-09-21T03:42:13.544 INFO:tasks.workunit.client.0.smithi089.stdout:sudo 84829 root txt unknown /proc/84829/exe (readlink: Permission denied) 2024-09-21T03:42:13.544 INFO:tasks.workunit.client.0.smithi089.stdout:sudo 84829 root NOFD /proc/84829/fd (opendir: Permission denied) 2024-09-21T03:42:13.544 INFO:tasks.workunit.client.0.smithi089.stdout:daemon-he 84846 root cwd unknown /proc/84846/cwd (readlink: Permission denied) 2024-09-21T03:42:13.544 INFO:tasks.workunit.client.0.smithi089.stdout:daemon-he 84846 root rtd unknown /proc/84846/root (readlink: Permission denied) 2024-09-21T03:42:13.544 INFO:tasks.workunit.client.0.smithi089.stdout:daemon-he 84846 root txt unknown /proc/84846/exe (readlink: Permission denied) 2024-09-21T03:42:13.544 INFO:tasks.workunit.client.0.smithi089.stdout:daemon-he 84846 root NOFD /proc/84846/fd (opendir: Permission denied) 2024-09-21T03:42:13.544 INFO:tasks.workunit.client.0.smithi089.stdout:bash 84863 root cwd unknown /proc/84863/cwd (readlink: Permission denied) 2024-09-21T03:42:13.544 INFO:tasks.workunit.client.0.smithi089.stdout:bash 84863 root rtd unknown /proc/84863/root (readlink: Permission denied) 2024-09-21T03:42:13.544 INFO:tasks.workunit.client.0.smithi089.stdout:bash 84863 root txt unknown /proc/84863/exe (readlink: Permission denied) 2024-09-21T03:42:13.544 INFO:tasks.workunit.client.0.smithi089.stdout:bash 84863 root NOFD /proc/84863/fd (opendir: Permission denied) 2024-09-21T03:42:13.544 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/3 85289 root cwd unknown /proc/85289/cwd (readlink: Permission denied) 2024-09-21T03:42:13.544 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/3 85289 root rtd unknown /proc/85289/root (readlink: Permission denied) 2024-09-21T03:42:13.544 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/3 85289 root txt unknown /proc/85289/exe (readlink: Permission denied) 2024-09-21T03:42:13.544 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/3 85289 root NOFD /proc/85289/fd (opendir: Permission denied) 2024-09-21T03:42:13.545 INFO:tasks.workunit.client.0.smithi089.stdout:timeout 87201 ubuntu cwd DIR 0,120 269505656 1099511627782 /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-09-21T03:42:13.545 INFO:tasks.workunit.client.0.smithi089.stdout:timeout 87201 ubuntu rtd DIR 8,1 4096 2 / 2024-09-21T03:42:13.545 INFO:tasks.workunit.client.0.smithi089.stdout:timeout 87201 ubuntu txt REG 8,1 41128 11685 /usr/bin/timeout 2024-09-21T03:42:13.545 INFO:tasks.workunit.client.0.smithi089.stdout:timeout 87201 ubuntu mem REG 8,1 2586930 19889 /usr/lib/locale/en_US.utf8/LC_COLLATE 2024-09-21T03:42:13.545 INFO:tasks.workunit.client.0.smithi089.stdout:timeout 87201 ubuntu mem REG 8,1 2544296 11212 /usr/lib64/libc.so.6 2024-09-21T03:42:13.545 INFO:tasks.workunit.client.0.smithi089.stdout:timeout 87201 ubuntu mem REG 8,1 346132 32690 /usr/lib/locale/C.utf8/LC_CTYPE 2024-09-21T03:42:13.545 INFO:tasks.workunit.client.0.smithi089.stdout:timeout 87201 ubuntu mem REG 8,1 54 19892 /usr/lib/locale/en_US.utf8/LC_NUMERIC 2024-09-21T03:42:13.545 INFO:tasks.workunit.client.0.smithi089.stdout:timeout 87201 ubuntu mem REG 8,1 3284 32609 /usr/lib/locale/en_US.utf8/LC_TIME 2024-09-21T03:42:13.545 INFO:tasks.workunit.client.0.smithi089.stdout:timeout 87201 ubuntu mem REG 8,1 286 32606 /usr/lib/locale/en_US.utf8/LC_MONETARY 2024-09-21T03:42:13.545 INFO:tasks.workunit.client.0.smithi089.stdout:timeout 87201 ubuntu mem REG 8,1 57 30370 /usr/lib/locale/en_US.utf8/LC_MESSAGES/SYS_LC_MESSAGES 2024-09-21T03:42:13.545 INFO:tasks.workunit.client.0.smithi089.stdout:timeout 87201 ubuntu mem REG 8,1 34 32659 /usr/lib/locale/en_US.utf8/LC_PAPER 2024-09-21T03:42:13.545 INFO:tasks.workunit.client.0.smithi089.stdout:timeout 87201 ubuntu mem REG 8,1 77 19891 /usr/lib/locale/en_US.utf8/LC_NAME 2024-09-21T03:42:13.545 INFO:tasks.workunit.client.0.smithi089.stdout:timeout 87201 ubuntu mem REG 8,1 167 32601 /usr/lib/locale/en_US.utf8/LC_ADDRESS 2024-09-21T03:42:13.545 INFO:tasks.workunit.client.0.smithi089.stdout:timeout 87201 ubuntu mem REG 8,1 59 32608 /usr/lib/locale/en_US.utf8/LC_TELEPHONE 2024-09-21T03:42:13.545 INFO:tasks.workunit.client.0.smithi089.stdout:timeout 87201 ubuntu mem REG 8,1 23 32605 /usr/lib/locale/en_US.utf8/LC_MEASUREMENT 2024-09-21T03:42:13.546 INFO:tasks.workunit.client.0.smithi089.stdout:timeout 87201 ubuntu mem REG 8,1 26988 31782 /usr/lib64/gconv/gconv-modules.cache 2024-09-21T03:42:13.546 INFO:tasks.workunit.client.0.smithi089.stdout:timeout 87201 ubuntu mem REG 8,1 369 32604 /usr/lib/locale/en_US.utf8/LC_IDENTIFICATION 2024-09-21T03:42:13.546 INFO:tasks.workunit.client.0.smithi089.stdout:timeout 87201 ubuntu mem REG 8,1 877424 15688 /usr/lib64/ld-linux-x86-64.so.2 2024-09-21T03:42:13.546 INFO:tasks.workunit.client.0.smithi089.stdout:timeout 87201 ubuntu 0r FIFO 0,13 0t0 237902 pipe 2024-09-21T03:42:13.546 INFO:tasks.workunit.client.0.smithi089.stdout:timeout 87201 ubuntu 1w FIFO 0,13 0t0 237903 pipe 2024-09-21T03:42:13.546 INFO:tasks.workunit.client.0.smithi089.stdout:timeout 87201 ubuntu 2w FIFO 0,13 0t0 237904 pipe 2024-09-21T03:42:13.546 INFO:tasks.workunit.client.0.smithi089.stdout:fsync-tes 87220 ubuntu cwd DIR 0,120 269505656 1099511627782 /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-09-21T03:42:13.546 INFO:tasks.workunit.client.0.smithi089.stdout:fsync-tes 87220 ubuntu rtd DIR 8,1 4096 2 / 2024-09-21T03:42:13.546 INFO:tasks.workunit.client.0.smithi089.stdout:fsync-tes 87220 ubuntu txt REG 8,1 1389072 11254 /usr/bin/bash 2024-09-21T03:42:13.546 INFO:tasks.workunit.client.0.smithi089.stdout:fsync-tes 87220 ubuntu mem REG 8,1 2586930 19889 /usr/lib/locale/en_US.utf8/LC_COLLATE 2024-09-21T03:42:13.546 INFO:tasks.workunit.client.0.smithi089.stdout:fsync-tes 87220 ubuntu mem REG 8,1 2544296 11212 /usr/lib64/libc.so.6 2024-09-21T03:42:13.546 INFO:tasks.workunit.client.0.smithi089.stdout:fsync-tes 87220 ubuntu mem REG 8,1 346132 32690 /usr/lib/locale/C.utf8/LC_CTYPE 2024-09-21T03:42:13.546 INFO:tasks.workunit.client.0.smithi089.stdout:fsync-tes 87220 ubuntu mem REG 8,1 54 19892 /usr/lib/locale/en_US.utf8/LC_NUMERIC 2024-09-21T03:42:13.546 INFO:tasks.workunit.client.0.smithi089.stdout:fsync-tes 87220 ubuntu mem REG 8,1 3284 32609 /usr/lib/locale/en_US.utf8/LC_TIME 2024-09-21T03:42:13.547 INFO:tasks.workunit.client.0.smithi089.stdout:fsync-tes 87220 ubuntu mem REG 8,1 286 32606 /usr/lib/locale/en_US.utf8/LC_MONETARY 2024-09-21T03:42:13.547 INFO:tasks.workunit.client.0.smithi089.stdout:fsync-tes 87220 ubuntu mem REG 8,1 57 30370 /usr/lib/locale/en_US.utf8/LC_MESSAGES/SYS_LC_MESSAGES 2024-09-21T03:42:13.547 INFO:tasks.workunit.client.0.smithi089.stdout:fsync-tes 87220 ubuntu mem REG 8,1 34 32659 /usr/lib/locale/en_US.utf8/LC_PAPER 2024-09-21T03:42:13.547 INFO:tasks.workunit.client.0.smithi089.stdout:fsync-tes 87220 ubuntu mem REG 8,1 77 19891 /usr/lib/locale/en_US.utf8/LC_NAME 2024-09-21T03:42:13.547 INFO:tasks.workunit.client.0.smithi089.stdout:fsync-tes 87220 ubuntu mem REG 8,1 167 32601 /usr/lib/locale/en_US.utf8/LC_ADDRESS 2024-09-21T03:42:13.547 INFO:tasks.workunit.client.0.smithi089.stdout:fsync-tes 87220 ubuntu mem REG 8,1 59 32608 /usr/lib/locale/en_US.utf8/LC_TELEPHONE 2024-09-21T03:42:13.547 INFO:tasks.workunit.client.0.smithi089.stdout:fsync-tes 87220 ubuntu mem REG 8,1 23 32605 /usr/lib/locale/en_US.utf8/LC_MEASUREMENT 2024-09-21T03:42:13.547 INFO:tasks.workunit.client.0.smithi089.stdout:fsync-tes 87220 ubuntu mem REG 8,1 195080 11252 /usr/lib64/libtinfo.so.6.2 2024-09-21T03:42:13.547 INFO:tasks.workunit.client.0.smithi089.stdout:fsync-tes 87220 ubuntu mem REG 8,1 26988 31782 /usr/lib64/gconv/gconv-modules.cache 2024-09-21T03:42:13.547 INFO:tasks.workunit.client.0.smithi089.stdout:fsync-tes 87220 ubuntu mem REG 8,1 369 32604 /usr/lib/locale/en_US.utf8/LC_IDENTIFICATION 2024-09-21T03:42:13.547 INFO:tasks.workunit.client.0.smithi089.stdout:fsync-tes 87220 ubuntu mem REG 8,1 877424 15688 /usr/lib64/ld-linux-x86-64.so.2 2024-09-21T03:42:13.547 INFO:tasks.workunit.client.0.smithi089.stdout:fsync-tes 87220 ubuntu 0r FIFO 0,13 0t0 237902 pipe 2024-09-21T03:42:13.547 INFO:tasks.workunit.client.0.smithi089.stdout:fsync-tes 87220 ubuntu 1w FIFO 0,13 0t0 237903 pipe 2024-09-21T03:42:13.547 INFO:tasks.workunit.client.0.smithi089.stdout:fsync-tes 87220 ubuntu 2w FIFO 0,13 0t0 237904 pipe 2024-09-21T03:42:13.547 INFO:tasks.workunit.client.0.smithi089.stdout:fsync-tes 87220 ubuntu 255r REG 8,1 207 547345 /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/fsync-tester.sh 2024-09-21T03:42:13.548 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/2 87586 root cwd unknown /proc/87586/cwd (readlink: Permission denied) 2024-09-21T03:42:13.548 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/2 87586 root rtd unknown /proc/87586/root (readlink: Permission denied) 2024-09-21T03:42:13.548 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/2 87586 root txt unknown /proc/87586/exe (readlink: Permission denied) 2024-09-21T03:42:13.548 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/2 87586 root NOFD /proc/87586/fd (opendir: Permission denied) 2024-09-21T03:42:13.548 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/7 87587 root cwd unknown /proc/87587/cwd (readlink: Permission denied) 2024-09-21T03:42:13.548 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/7 87587 root rtd unknown /proc/87587/root (readlink: Permission denied) 2024-09-21T03:42:13.548 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/7 87587 root txt unknown /proc/87587/exe (readlink: Permission denied) 2024-09-21T03:42:13.548 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/7 87587 root NOFD /proc/87587/fd (opendir: Permission denied) 2024-09-21T03:42:13.548 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/0 87699 root cwd unknown /proc/87699/cwd (readlink: Permission denied) 2024-09-21T03:42:13.548 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/0 87699 root rtd unknown /proc/87699/root (readlink: Permission denied) 2024-09-21T03:42:13.548 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/0 87699 root txt unknown /proc/87699/exe (readlink: Permission denied) 2024-09-21T03:42:13.548 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/0 87699 root NOFD /proc/87699/fd (opendir: Permission denied) 2024-09-21T03:42:13.548 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/u 88033 root cwd unknown /proc/88033/cwd (readlink: Permission denied) 2024-09-21T03:42:13.548 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/u 88033 root rtd unknown /proc/88033/root (readlink: Permission denied) 2024-09-21T03:42:13.548 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/u 88033 root txt unknown /proc/88033/exe (readlink: Permission denied) 2024-09-21T03:42:13.549 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/u 88033 root NOFD /proc/88033/fd (opendir: Permission denied) 2024-09-21T03:42:13.549 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/5 88034 root cwd unknown /proc/88034/cwd (readlink: Permission denied) 2024-09-21T03:42:13.549 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/5 88034 root rtd unknown /proc/88034/root (readlink: Permission denied) 2024-09-21T03:42:13.549 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/5 88034 root txt unknown /proc/88034/exe (readlink: Permission denied) 2024-09-21T03:42:13.549 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/5 88034 root NOFD /proc/88034/fd (opendir: Permission denied) 2024-09-21T03:42:13.549 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/5 88035 root cwd unknown /proc/88035/cwd (readlink: Permission denied) 2024-09-21T03:42:13.549 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/5 88035 root rtd unknown /proc/88035/root (readlink: Permission denied) 2024-09-21T03:42:13.549 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/5 88035 root txt unknown /proc/88035/exe (readlink: Permission denied) 2024-09-21T03:42:13.549 INFO:tasks.workunit.client.0.smithi089.stdout:kworker/5 88035 root NOFD /proc/88035/fd (opendir: Permission denied) 2024-09-21T03:42:13.549 INFO:tasks.workunit.client.0.smithi089.stdout:sleep 88409 root cwd unknown /proc/88409/cwd (readlink: Permission denied) 2024-09-21T03:42:13.549 INFO:tasks.workunit.client.0.smithi089.stdout:sleep 88409 root rtd unknown /proc/88409/root (readlink: Permission denied) 2024-09-21T03:42:13.549 INFO:tasks.workunit.client.0.smithi089.stdout:sleep 88409 root txt unknown /proc/88409/exe (readlink: Permission denied) 2024-09-21T03:42:13.549 INFO:tasks.workunit.client.0.smithi089.stdout:sleep 88409 root NOFD /proc/88409/fd (opendir: Permission denied) 2024-09-21T03:42:13.549 INFO:tasks.workunit.client.0.smithi089.stdout:sudo 88529 root cwd unknown /proc/88529/cwd (readlink: Permission denied) 2024-09-21T03:42:13.549 INFO:tasks.workunit.client.0.smithi089.stdout:sudo 88529 root rtd unknown /proc/88529/root (readlink: Permission denied) 2024-09-21T03:42:13.550 INFO:tasks.workunit.client.0.smithi089.stdout:sudo 88529 root txt unknown /proc/88529/exe (readlink: Permission denied) 2024-09-21T03:42:13.550 INFO:tasks.workunit.client.0.smithi089.stdout:sudo 88529 root NOFD /proc/88529/fd (opendir: Permission denied) 2024-09-21T03:42:13.550 INFO:tasks.workunit.client.0.smithi089.stdout:cephadm 88546 root cwd unknown /proc/88546/cwd (readlink: Permission denied) 2024-09-21T03:42:13.550 INFO:tasks.workunit.client.0.smithi089.stdout:cephadm 88546 root rtd unknown /proc/88546/root (readlink: Permission denied) 2024-09-21T03:42:13.550 INFO:tasks.workunit.client.0.smithi089.stdout:cephadm 88546 root txt unknown /proc/88546/exe (readlink: Permission denied) 2024-09-21T03:42:13.550 INFO:tasks.workunit.client.0.smithi089.stdout:cephadm 88546 root NOFD /proc/88546/fd (opendir: Permission denied) 2024-09-21T03:42:13.550 INFO:tasks.workunit.client.0.smithi089.stdout:podman 88587 root cwd unknown /proc/88587/cwd (readlink: Permission denied) 2024-09-21T03:42:13.550 INFO:tasks.workunit.client.0.smithi089.stdout:podman 88587 root rtd unknown /proc/88587/root (readlink: Permission denied) 2024-09-21T03:42:13.550 INFO:tasks.workunit.client.0.smithi089.stdout:podman 88587 root txt unknown /proc/88587/exe (readlink: Permission denied) 2024-09-21T03:42:13.550 INFO:tasks.workunit.client.0.smithi089.stdout:podman 88587 root NOFD /proc/88587/fd (opendir: Permission denied) 2024-09-21T03:42:13.550 INFO:tasks.workunit.client.0.smithi089.stdout:podman 88587 88588 podman root cwd unknown /proc/88587/task/88588/cwd (readlink: Permission denied) 2024-09-21T03:42:13.550 INFO:tasks.workunit.client.0.smithi089.stdout:podman 88587 88588 podman root rtd unknown /proc/88587/task/88588/root (readlink: Permission denied) 2024-09-21T03:42:13.550 INFO:tasks.workunit.client.0.smithi089.stdout:podman 88587 88588 podman root txt unknown /proc/88587/task/88588/exe (readlink: Permission denied) 2024-09-21T03:42:13.550 INFO:tasks.workunit.client.0.smithi089.stdout:podman 88587 88588 podman root NOFD /proc/88587/task/88588/fd (opendir: Permission denied) 2024-09-21T03:42:13.551 INFO:tasks.workunit.client.0.smithi089.stdout:podman 88587 88589 podman root cwd unknown /proc/88587/task/88589/cwd (readlink: Permission denied) 2024-09-21T03:42:13.551 INFO:tasks.workunit.client.0.smithi089.stdout:podman 88587 88589 podman root rtd unknown /proc/88587/task/88589/root (readlink: Permission denied) 2024-09-21T03:42:13.551 INFO:tasks.workunit.client.0.smithi089.stdout:podman 88587 88589 podman root txt unknown /proc/88587/task/88589/exe (readlink: Permission denied) 2024-09-21T03:42:13.551 INFO:tasks.workunit.client.0.smithi089.stdout:podman 88587 88589 podman root NOFD /proc/88587/task/88589/fd (opendir: Permission denied) 2024-09-21T03:42:13.551 INFO:tasks.workunit.client.0.smithi089.stdout:podman 88587 88590 podman root cwd unknown /proc/88587/task/88590/cwd (readlink: Permission denied) 2024-09-21T03:42:13.551 INFO:tasks.workunit.client.0.smithi089.stdout:podman 88587 88590 podman root rtd unknown /proc/88587/task/88590/root (readlink: Permission denied) 2024-09-21T03:42:13.551 INFO:tasks.workunit.client.0.smithi089.stdout:podman 88587 88590 podman root txt unknown /proc/88587/task/88590/exe (readlink: Permission denied) 2024-09-21T03:42:13.551 INFO:tasks.workunit.client.0.smithi089.stdout:podman 88587 88590 podman root NOFD /proc/88587/task/88590/fd (opendir: Permission denied) 2024-09-21T03:42:13.551 INFO:tasks.workunit.client.0.smithi089.stdout:podman 88587 88591 podman root cwd unknown /proc/88587/task/88591/cwd (readlink: Permission denied) 2024-09-21T03:42:13.551 INFO:tasks.workunit.client.0.smithi089.stdout:podman 88587 88591 podman root rtd unknown /proc/88587/task/88591/root (readlink: Permission denied) 2024-09-21T03:42:13.551 INFO:tasks.workunit.client.0.smithi089.stdout:podman 88587 88591 podman root txt unknown /proc/88587/task/88591/exe (readlink: Permission denied) 2024-09-21T03:42:13.551 INFO:tasks.workunit.client.0.smithi089.stdout:podman 88587 88591 podman root NOFD /proc/88587/task/88591/fd (opendir: Permission denied) 2024-09-21T03:42:13.551 INFO:tasks.workunit.client.0.smithi089.stdout:podman 88587 88592 podman root cwd unknown /proc/88587/task/88592/cwd (readlink: Permission denied) 2024-09-21T03:42:13.551 INFO:tasks.workunit.client.0.smithi089.stdout:podman 88587 88592 podman root rtd unknown /proc/88587/task/88592/root (readlink: Permission denied) 2024-09-21T03:42:13.551 INFO:tasks.workunit.client.0.smithi089.stdout:podman 88587 88592 podman root txt unknown /proc/88587/task/88592/exe (readlink: Permission denied) 2024-09-21T03:42:13.552 INFO:tasks.workunit.client.0.smithi089.stdout:podman 88587 88592 podman root NOFD /proc/88587/task/88592/fd (opendir: Permission denied) 2024-09-21T03:42:13.552 INFO:tasks.workunit.client.0.smithi089.stdout:podman 88587 88593 podman root cwd unknown /proc/88587/task/88593/cwd (readlink: Permission denied) 2024-09-21T03:42:13.552 INFO:tasks.workunit.client.0.smithi089.stdout:podman 88587 88593 podman root rtd unknown /proc/88587/task/88593/root (readlink: Permission denied) 2024-09-21T03:42:13.552 INFO:tasks.workunit.client.0.smithi089.stdout:podman 88587 88593 podman root txt unknown /proc/88587/task/88593/exe (readlink: Permission denied) 2024-09-21T03:42:13.552 INFO:tasks.workunit.client.0.smithi089.stdout:podman 88587 88593 podman root NOFD /proc/88587/task/88593/fd (opendir: Permission denied) 2024-09-21T03:42:13.552 INFO:tasks.workunit.client.0.smithi089.stdout:podman 88587 88594 podman root cwd unknown /proc/88587/task/88594/cwd (readlink: Permission denied) 2024-09-21T03:42:13.552 INFO:tasks.workunit.client.0.smithi089.stdout:podman 88587 88594 podman root rtd unknown /proc/88587/task/88594/root (readlink: Permission denied) 2024-09-21T03:42:13.552 INFO:tasks.workunit.client.0.smithi089.stdout:podman 88587 88594 podman root txt unknown /proc/88587/task/88594/exe (readlink: Permission denied) 2024-09-21T03:42:13.552 INFO:tasks.workunit.client.0.smithi089.stdout:podman 88587 88594 podman root NOFD /proc/88587/task/88594/fd (opendir: Permission denied) 2024-09-21T03:42:13.552 INFO:tasks.workunit.client.0.smithi089.stdout:podman 88587 88595 podman root cwd unknown /proc/88587/task/88595/cwd (readlink: Permission denied) 2024-09-21T03:42:13.552 INFO:tasks.workunit.client.0.smithi089.stdout:podman 88587 88595 podman root rtd unknown /proc/88587/task/88595/root (readlink: Permission denied) 2024-09-21T03:42:13.552 INFO:tasks.workunit.client.0.smithi089.stdout:podman 88587 88595 podman root txt unknown /proc/88587/task/88595/exe (readlink: Permission denied) 2024-09-21T03:42:13.552 INFO:tasks.workunit.client.0.smithi089.stdout:podman 88587 88595 podman root NOFD /proc/88587/task/88595/fd (opendir: Permission denied) 2024-09-21T03:42:13.552 INFO:tasks.workunit.client.0.smithi089.stdout:podman 88587 88596 podman root cwd unknown /proc/88587/task/88596/cwd (readlink: Permission denied) 2024-09-21T03:42:13.552 INFO:tasks.workunit.client.0.smithi089.stdout:podman 88587 88596 podman root rtd unknown /proc/88587/task/88596/root (readlink: Permission denied) 2024-09-21T03:42:13.553 INFO:tasks.workunit.client.0.smithi089.stdout:podman 88587 88596 podman root txt unknown /proc/88587/task/88596/exe (readlink: Permission denied) 2024-09-21T03:42:13.553 INFO:tasks.workunit.client.0.smithi089.stdout:podman 88587 88596 podman root NOFD /proc/88587/task/88596/fd (opendir: Permission denied) 2024-09-21T03:42:13.553 INFO:tasks.workunit.client.0.smithi089.stdout:podman 88587 88597 podman root cwd unknown /proc/88587/task/88597/cwd (readlink: Permission denied) 2024-09-21T03:42:13.553 INFO:tasks.workunit.client.0.smithi089.stdout:podman 88587 88597 podman root rtd unknown /proc/88587/task/88597/root (readlink: Permission denied) 2024-09-21T03:42:13.553 INFO:tasks.workunit.client.0.smithi089.stdout:podman 88587 88597 podman root txt unknown /proc/88587/task/88597/exe (readlink: Permission denied) 2024-09-21T03:42:13.553 INFO:tasks.workunit.client.0.smithi089.stdout:podman 88587 88597 podman root NOFD /proc/88587/task/88597/fd (opendir: Permission denied) 2024-09-21T03:42:13.553 INFO:tasks.workunit.client.0.smithi089.stdout:podman 88587 88598 podman root cwd unknown /proc/88587/task/88598/cwd (readlink: Permission denied) 2024-09-21T03:42:13.553 INFO:tasks.workunit.client.0.smithi089.stdout:podman 88587 88598 podman root rtd unknown /proc/88587/task/88598/root (readlink: Permission denied) 2024-09-21T03:42:13.553 INFO:tasks.workunit.client.0.smithi089.stdout:podman 88587 88598 podman root txt unknown /proc/88587/task/88598/exe (readlink: Permission denied) 2024-09-21T03:42:13.553 INFO:tasks.workunit.client.0.smithi089.stdout:podman 88587 88598 podman root NOFD /proc/88587/task/88598/fd (opendir: Permission denied) 2024-09-21T03:42:13.553 INFO:tasks.workunit.client.0.smithi089.stdout:podman 88587 88599 podman root cwd unknown /proc/88587/task/88599/cwd (readlink: Permission denied) 2024-09-21T03:42:13.553 INFO:tasks.workunit.client.0.smithi089.stdout:podman 88587 88599 podman root rtd unknown /proc/88587/task/88599/root (readlink: Permission denied) 2024-09-21T03:42:13.553 INFO:tasks.workunit.client.0.smithi089.stdout:podman 88587 88599 podman root txt unknown /proc/88587/task/88599/exe (readlink: Permission denied) 2024-09-21T03:42:13.554 INFO:tasks.workunit.client.0.smithi089.stdout:podman 88587 88599 podman root NOFD /proc/88587/task/88599/fd (opendir: Permission denied) 2024-09-21T03:42:13.554 INFO:tasks.workunit.client.0.smithi089.stdout:podman 88587 88600 podman root cwd unknown /proc/88587/task/88600/cwd (readlink: Permission denied) 2024-09-21T03:42:13.554 INFO:tasks.workunit.client.0.smithi089.stdout:podman 88587 88600 podman root rtd unknown /proc/88587/task/88600/root (readlink: Permission denied) 2024-09-21T03:42:13.554 INFO:tasks.workunit.client.0.smithi089.stdout:podman 88587 88600 podman root txt unknown /proc/88587/task/88600/exe (readlink: Permission denied) 2024-09-21T03:42:13.554 INFO:tasks.workunit.client.0.smithi089.stdout:podman 88587 88600 podman root NOFD /proc/88587/task/88600/fd (opendir: Permission denied) 2024-09-21T03:42:13.554 INFO:tasks.workunit.client.0.smithi089.stdout:conmon 88602 root cwd unknown /proc/88602/cwd (readlink: Permission denied) 2024-09-21T03:42:13.554 INFO:tasks.workunit.client.0.smithi089.stdout:conmon 88602 root rtd unknown /proc/88602/root (readlink: Permission denied) 2024-09-21T03:42:13.554 INFO:tasks.workunit.client.0.smithi089.stdout:conmon 88602 root txt unknown /proc/88602/exe (readlink: Permission denied) 2024-09-21T03:42:13.554 INFO:tasks.workunit.client.0.smithi089.stdout:conmon 88602 root NOFD /proc/88602/fd (opendir: Permission denied) 2024-09-21T03:42:13.554 INFO:tasks.workunit.client.0.smithi089.stdout:podman 88654 root cwd unknown /proc/88654/cwd (readlink: Permission denied) 2024-09-21T03:42:13.554 INFO:tasks.workunit.client.0.smithi089.stdout:podman 88654 root rtd unknown /proc/88654/root (readlink: Permission denied) 2024-09-21T03:42:13.554 INFO:tasks.workunit.client.0.smithi089.stdout:podman 88654 root txt unknown /proc/88654/exe (readlink: Permission denied) 2024-09-21T03:42:13.554 INFO:tasks.workunit.client.0.smithi089.stdout:podman 88654 root NOFD /proc/88654/fd (opendir: Permission denied) 2024-09-21T03:42:13.554 INFO:tasks.workunit.client.0.smithi089.stdout:podman 88654 88655 podman root cwd unknown /proc/88654/task/88655/cwd (readlink: Permission denied) 2024-09-21T03:42:13.555 INFO:tasks.workunit.client.0.smithi089.stdout:podman 88654 88655 podman root rtd unknown /proc/88654/task/88655/root (readlink: Permission denied) 2024-09-21T03:42:13.555 INFO:tasks.workunit.client.0.smithi089.stdout:podman 88654 88655 podman root txt unknown /proc/88654/task/88655/exe (readlink: Permission denied) 2024-09-21T03:42:13.555 INFO:tasks.workunit.client.0.smithi089.stdout:podman 88654 88655 podman root NOFD /proc/88654/task/88655/fd (opendir: Permission denied) 2024-09-21T03:42:13.555 INFO:tasks.workunit.client.0.smithi089.stdout:podman 88654 88656 podman root cwd unknown /proc/88654/task/88656/cwd (readlink: Permission denied) 2024-09-21T03:42:13.555 INFO:tasks.workunit.client.0.smithi089.stdout:podman 88654 88656 podman root rtd unknown /proc/88654/task/88656/root (readlink: Permission denied) 2024-09-21T03:42:13.555 INFO:tasks.workunit.client.0.smithi089.stdout:podman 88654 88656 podman root txt unknown /proc/88654/task/88656/exe (readlink: Permission denied) 2024-09-21T03:42:13.555 INFO:tasks.workunit.client.0.smithi089.stdout:podman 88654 88656 podman root NOFD /proc/88654/task/88656/fd (opendir: Permission denied) 2024-09-21T03:42:13.555 INFO:tasks.workunit.client.0.smithi089.stdout:podman 88654 88657 podman root cwd unknown /proc/88654/task/88657/cwd (readlink: Permission denied) 2024-09-21T03:42:13.555 INFO:tasks.workunit.client.0.smithi089.stdout:podman 88654 88657 podman root rtd unknown /proc/88654/task/88657/root (readlink: Permission denied) 2024-09-21T03:42:13.555 INFO:tasks.workunit.client.0.smithi089.stdout:podman 88654 88657 podman root txt unknown /proc/88654/task/88657/exe (readlink: Permission denied) 2024-09-21T03:42:13.555 INFO:tasks.workunit.client.0.smithi089.stdout:podman 88654 88657 podman root NOFD /proc/88654/task/88657/fd (opendir: Permission denied) 2024-09-21T03:42:13.555 INFO:tasks.workunit.client.0.smithi089.stdout:podman 88654 88658 podman root cwd unknown /proc/88654/task/88658/cwd (readlink: Permission denied) 2024-09-21T03:42:13.555 INFO:tasks.workunit.client.0.smithi089.stdout:podman 88654 88658 podman root rtd unknown /proc/88654/task/88658/root (readlink: Permission denied) 2024-09-21T03:42:13.555 INFO:tasks.workunit.client.0.smithi089.stdout:podman 88654 88658 podman root txt unknown /proc/88654/task/88658/exe (readlink: Permission denied) 2024-09-21T03:42:13.555 INFO:tasks.workunit.client.0.smithi089.stdout:podman 88654 88658 podman root NOFD /proc/88654/task/88658/fd (opendir: Permission denied) 2024-09-21T03:42:13.555 INFO:tasks.workunit.client.0.smithi089.stdout:podman 88654 88659 podman root cwd unknown /proc/88654/task/88659/cwd (readlink: Permission denied) 2024-09-21T03:42:13.556 INFO:tasks.workunit.client.0.smithi089.stdout:podman 88654 88659 podman root rtd unknown /proc/88654/task/88659/root (readlink: Permission denied) 2024-09-21T03:42:13.556 INFO:tasks.workunit.client.0.smithi089.stdout:podman 88654 88659 podman root txt unknown /proc/88654/task/88659/exe (readlink: Permission denied) 2024-09-21T03:42:13.556 INFO:tasks.workunit.client.0.smithi089.stdout:podman 88654 88659 podman root NOFD /proc/88654/task/88659/fd (opendir: Permission denied) 2024-09-21T03:42:13.556 INFO:tasks.workunit.client.0.smithi089.stdout:podman 88654 88661 podman root cwd unknown /proc/88654/task/88661/cwd (readlink: Permission denied) 2024-09-21T03:42:13.556 INFO:tasks.workunit.client.0.smithi089.stdout:podman 88654 88661 podman root rtd unknown /proc/88654/task/88661/root (readlink: Permission denied) 2024-09-21T03:42:13.556 INFO:tasks.workunit.client.0.smithi089.stdout:podman 88654 88661 podman root txt unknown /proc/88654/task/88661/exe (readlink: Permission denied) 2024-09-21T03:42:13.556 INFO:tasks.workunit.client.0.smithi089.stdout:podman 88654 88661 podman root NOFD /proc/88654/task/88661/fd (opendir: Permission denied) 2024-09-21T03:42:13.556 INFO:tasks.workunit.client.0.smithi089.stdout:podman 88654 88662 podman root cwd unknown /proc/88654/task/88662/cwd (readlink: Permission denied) 2024-09-21T03:42:13.556 INFO:tasks.workunit.client.0.smithi089.stdout:podman 88654 88662 podman root rtd unknown /proc/88654/task/88662/root (readlink: Permission denied) 2024-09-21T03:42:13.556 INFO:tasks.workunit.client.0.smithi089.stdout:podman 88654 88662 podman root txt unknown /proc/88654/task/88662/exe (readlink: Permission denied) 2024-09-21T03:42:13.556 INFO:tasks.workunit.client.0.smithi089.stdout:podman 88654 88662 podman root NOFD /proc/88654/task/88662/fd (opendir: Permission denied) 2024-09-21T03:42:13.556 INFO:tasks.workunit.client.0.smithi089.stdout:setrouble 88660 setroubleshoot cwd unknown /proc/88660/cwd (readlink: Permission denied) 2024-09-21T03:42:13.556 INFO:tasks.workunit.client.0.smithi089.stdout:setrouble 88660 setroubleshoot rtd unknown /proc/88660/root (readlink: Permission denied) 2024-09-21T03:42:13.557 INFO:tasks.workunit.client.0.smithi089.stdout:setrouble 88660 setroubleshoot txt unknown /proc/88660/exe (readlink: Permission denied) 2024-09-21T03:42:13.557 INFO:tasks.workunit.client.0.smithi089.stdout:setrouble 88660 setroubleshoot NOFD /proc/88660/fd (opendir: Permission denied) 2024-09-21T03:42:13.557 INFO:tasks.workunit.client.0.smithi089.stdout:lsof 88663 ubuntu cwd DIR 0,120 269505656 1099511627782 /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-09-21T03:42:13.557 INFO:tasks.workunit.client.0.smithi089.stdout:lsof 88663 ubuntu rtd DIR 8,1 4096 2 / 2024-09-21T03:42:13.557 INFO:tasks.workunit.client.0.smithi089.stdout:lsof 88663 ubuntu txt REG 8,1 179232 13934 /usr/bin/lsof 2024-09-21T03:42:13.557 INFO:tasks.workunit.client.0.smithi089.stdout:lsof 88663 ubuntu mem REG 8,1 5436544 2392 /usr/lib64/libcrypto.so.3.2.2 2024-09-21T03:42:13.557 INFO:tasks.workunit.client.0.smithi089.stdout:lsof 88663 ubuntu mem REG 8,1 346132 32690 /usr/lib/locale/C.utf8/LC_CTYPE 2024-09-21T03:42:13.557 INFO:tasks.workunit.client.0.smithi089.stdout:lsof 88663 ubuntu mem REG 8,1 102672 11274 /usr/lib64/libz.so.1.2.11 2024-09-21T03:42:13.557 INFO:tasks.workunit.client.0.smithi089.stdout:lsof 88663 ubuntu mem REG 8,1 70880 11221 /usr/lib64/libresolv.so.2 2024-09-21T03:42:13.557 INFO:tasks.workunit.client.0.smithi089.stdout:lsof 88663 ubuntu mem REG 8,1 906368 12996 /usr/lib64/libkrb5.so.3.3 2024-09-21T03:42:13.557 INFO:tasks.workunit.client.0.smithi089.stdout:lsof 88663 ubuntu mem REG 8,1 358560 12986 /usr/lib64/libgssapi_krb5.so.2.2 2024-09-21T03:42:13.557 INFO:tasks.workunit.client.0.smithi089.stdout:lsof 88663 ubuntu mem REG 8,1 2544296 11212 /usr/lib64/libc.so.6 2024-09-21T03:42:13.557 INFO:tasks.workunit.client.0.smithi089.stdout:lsof 88663 ubuntu mem REG 8,1 23992 11348 /usr/lib64/libkeyutils.so.1.10 2024-09-21T03:42:13.557 INFO:tasks.workunit.client.0.smithi089.stdout:lsof 88663 ubuntu mem REG 8,1 66320 12998 /usr/lib64/libkrb5support.so.0.1 2024-09-21T03:42:13.558 INFO:tasks.workunit.client.0.smithi089.stdout:lsof 88663 ubuntu mem REG 8,1 23872 11298 /usr/lib64/libcom_err.so.2.1 2024-09-21T03:42:13.558 INFO:tasks.workunit.client.0.smithi089.stdout:lsof 88663 ubuntu mem REG 8,1 99112 12990 /usr/lib64/libk5crypto.so.3.1 2024-09-21T03:42:13.558 INFO:tasks.workunit.client.0.smithi089.stdout:lsof 88663 ubuntu mem REG 8,1 636848 11327 /usr/lib64/libpcre2-8.so.0.11.0 2024-09-21T03:42:13.558 INFO:tasks.workunit.client.0.smithi089.stdout:lsof 88663 ubuntu mem REG 8,1 196224 13024 /usr/lib64/libtirpc.so.3.0.0 2024-09-21T03:42:13.558 INFO:tasks.workunit.client.0.smithi089.stdout:lsof 88663 ubuntu mem REG 8,1 175744 11412 /usr/lib64/libselinux.so.1 2024-09-21T03:42:13.558 INFO:tasks.workunit.client.0.smithi089.stdout:lsof 88663 ubuntu mem REG 8,1 26988 31782 /usr/lib64/gconv/gconv-modules.cache 2024-09-21T03:42:13.558 INFO:tasks.workunit.client.0.smithi089.stdout:lsof 88663 ubuntu mem REG 8,1 877424 15688 /usr/lib64/ld-linux-x86-64.so.2 2024-09-21T03:42:13.558 INFO:tasks.workunit.client.0.smithi089.stdout:lsof 88663 ubuntu 0r FIFO 0,13 0t0 237902 pipe 2024-09-21T03:42:13.558 INFO:tasks.workunit.client.0.smithi089.stdout:lsof 88663 ubuntu 1w FIFO 0,13 0t0 237903 pipe 2024-09-21T03:42:13.558 INFO:tasks.workunit.client.0.smithi089.stdout:lsof 88663 ubuntu 2w FIFO 0,13 0t0 237904 pipe 2024-09-21T03:42:13.558 INFO:tasks.workunit.client.0.smithi089.stdout:lsof 88663 ubuntu 3r DIR 0,20 0 1 /proc 2024-09-21T03:42:13.558 INFO:tasks.workunit.client.0.smithi089.stdout:lsof 88663 ubuntu 4r DIR 0,20 0 239381 /proc/88663/fd 2024-09-21T03:42:13.558 INFO:tasks.workunit.client.0.smithi089.stdout:lsof 88663 ubuntu 5w FIFO 0,13 0t0 239386 pipe 2024-09-21T03:42:13.558 INFO:tasks.workunit.client.0.smithi089.stdout:lsof 88663 ubuntu 6r FIFO 0,13 0t0 239387 pipe 2024-09-21T03:42:13.559 INFO:tasks.workunit.client.0.smithi089.stdout:lsof 88664 ubuntu cwd DIR 0,120 269505656 1099511627782 /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-09-21T03:42:13.559 INFO:tasks.workunit.client.0.smithi089.stdout:lsof 88664 ubuntu rtd DIR 8,1 4096 2 / 2024-09-21T03:42:13.559 INFO:tasks.workunit.client.0.smithi089.stdout:lsof 88664 ubuntu txt REG 8,1 179232 13934 /usr/bin/lsof 2024-09-21T03:42:13.559 INFO:tasks.workunit.client.0.smithi089.stdout:lsof 88664 ubuntu mem REG 8,1 5436544 2392 /usr/lib64/libcrypto.so.3.2.2 2024-09-21T03:42:13.559 INFO:tasks.workunit.client.0.smithi089.stdout:lsof 88664 ubuntu mem REG 8,1 346132 32690 /usr/lib/locale/C.utf8/LC_CTYPE 2024-09-21T03:42:13.559 INFO:tasks.workunit.client.0.smithi089.stdout:lsof 88664 ubuntu mem REG 8,1 102672 11274 /usr/lib64/libz.so.1.2.11 2024-09-21T03:42:13.559 INFO:tasks.workunit.client.0.smithi089.stdout:lsof 88664 ubuntu mem REG 8,1 70880 11221 /usr/lib64/libresolv.so.2 2024-09-21T03:42:13.559 INFO:tasks.workunit.client.0.smithi089.stdout:lsof 88664 ubuntu mem REG 8,1 906368 12996 /usr/lib64/libkrb5.so.3.3 2024-09-21T03:42:13.559 INFO:tasks.workunit.client.0.smithi089.stdout:lsof 88664 ubuntu mem REG 8,1 358560 12986 /usr/lib64/libgssapi_krb5.so.2.2 2024-09-21T03:42:13.559 INFO:tasks.workunit.client.0.smithi089.stdout:lsof 88664 ubuntu mem REG 8,1 2544296 11212 /usr/lib64/libc.so.6 2024-09-21T03:42:13.559 INFO:tasks.workunit.client.0.smithi089.stdout:lsof 88664 ubuntu mem REG 8,1 23992 11348 /usr/lib64/libkeyutils.so.1.10 2024-09-21T03:42:13.559 INFO:tasks.workunit.client.0.smithi089.stdout:lsof 88664 ubuntu mem REG 8,1 66320 12998 /usr/lib64/libkrb5support.so.0.1 2024-09-21T03:42:13.559 INFO:tasks.workunit.client.0.smithi089.stdout:lsof 88664 ubuntu mem REG 8,1 23872 11298 /usr/lib64/libcom_err.so.2.1 2024-09-21T03:42:13.559 INFO:tasks.workunit.client.0.smithi089.stdout:lsof 88664 ubuntu mem REG 8,1 99112 12990 /usr/lib64/libk5crypto.so.3.1 2024-09-21T03:42:13.559 INFO:tasks.workunit.client.0.smithi089.stdout:lsof 88664 ubuntu mem REG 8,1 636848 11327 /usr/lib64/libpcre2-8.so.0.11.0 2024-09-21T03:42:13.560 INFO:tasks.workunit.client.0.smithi089.stdout:lsof 88664 ubuntu mem REG 8,1 196224 13024 /usr/lib64/libtirpc.so.3.0.0 2024-09-21T03:42:13.560 INFO:tasks.workunit.client.0.smithi089.stdout:lsof 88664 ubuntu mem REG 8,1 175744 11412 /usr/lib64/libselinux.so.1 2024-09-21T03:42:13.560 INFO:tasks.workunit.client.0.smithi089.stdout:lsof 88664 ubuntu mem REG 8,1 26988 31782 /usr/lib64/gconv/gconv-modules.cache 2024-09-21T03:42:13.560 INFO:tasks.workunit.client.0.smithi089.stdout:lsof 88664 ubuntu mem REG 8,1 877424 15688 /usr/lib64/ld-linux-x86-64.so.2 2024-09-21T03:42:13.560 INFO:tasks.workunit.client.0.smithi089.stdout:lsof 88664 ubuntu 4r FIFO 0,13 0t0 239386 pipe 2024-09-21T03:42:13.560 INFO:tasks.workunit.client.0.smithi089.stdout:lsof 88664 ubuntu 7w FIFO 0,13 0t0 239387 pipe 2024-09-21T03:42:13.560 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-09-21T03:42:13.560 DEBUG:teuthology.orchestra.run.smithi089:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-09-21T03:42:13.561 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:42:12 smithi089 ceph-mon[30928]: pgmap v444: 129 pgs: 129 active+clean; 258 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 2.4 MiB/s wr, 560 op/s 2024-09-21T03:42:13.623 INFO:tasks.workunit:Stopping ['suites/fsync-tester.sh'] on client.0... 2024-09-21T03:42:13.624 DEBUG:teuthology.orchestra.run.smithi089:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-09-21T03:42:13.929 DEBUG:teuthology.parallel:result is None 2024-09-21T03:42:13.929 DEBUG:teuthology.orchestra.run.smithi089:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2024-09-21T03:42:13.955 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '6404c199', total seconds elapsed: 36.0 2024-09-21T03:42:13.955 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 27.8 2024-09-21T03:42:13.955 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 7c374098-77c9-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T03:42:13.963 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2024-09-21T03:42:13.963 DEBUG:teuthology.run_tasks:Unwinding manager quiescer 2024-09-21T03:42:13.974 INFO:tasks.quiescer:joining Quiescers 2024-09-21T03:42:13.974 WARNING:tasks.quiescer:The quiescer is requested to stop, running cancel all 2024-09-21T03:42:13.974 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 7c374098-77c9-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T03:42:14.275 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/7c374098-77c9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:42:14.279 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/7c374098-77c9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:42:15.001 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:42:14 smithi089 ceph-mon[30928]: pgmap v445: 129 pgs: 129 active+clean; 258 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 2.6 MiB/s wr, 605 op/s 2024-09-21T03:42:15.071 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:14.998+0000 7fc4624c9640 1 -- 172.21.15.89:0/3515180806 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc45c106c20 msgr2=0x7fc45c110670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:15.071 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:14.998+0000 7fc4624c9640 1 --2- 172.21.15.89:0/3515180806 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc45c106c20 0x7fc45c110670 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7fc454009ca0 tx=0x7fc45402a8c0 comp rx=0 tx=0).stop 2024-09-21T03:42:15.071 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:14.998+0000 7fc4624c9640 1 -- 172.21.15.89:0/3515180806 shutdown_connections 2024-09-21T03:42:15.072 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:14.998+0000 7fc4624c9640 1 --2- 172.21.15.89:0/3515180806 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc45c106c20 0x7fc45c110670 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:15.072 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:14.998+0000 7fc4624c9640 1 --2- 172.21.15.89:0/3515180806 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc45c1062d0 0x7fc45c1066e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:15.072 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:14.998+0000 7fc4624c9640 1 --2- 172.21.15.89:0/3515180806 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc45c101ff0 0x7fc45c102400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:15.072 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:14.998+0000 7fc4624c9640 1 -- 172.21.15.89:0/3515180806 >> 172.21.15.89:0/3515180806 conn(0x7fc45c079e30 msgr2=0x7fc45c07a240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:42:15.072 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:14.998+0000 7fc4624c9640 1 -- 172.21.15.89:0/3515180806 shutdown_connections 2024-09-21T03:42:15.072 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:14.998+0000 7fc4624c9640 1 -- 172.21.15.89:0/3515180806 wait complete. 2024-09-21T03:42:15.072 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:14.999+0000 7fc4624c9640 1 Processor -- start 2024-09-21T03:42:15.072 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:14.999+0000 7fc4624c9640 1 -- start start 2024-09-21T03:42:15.072 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:14.999+0000 7fc4624c9640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc45c101ff0 0x7fc45c19c550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:15.072 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:14.999+0000 7fc4624c9640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc45c1062d0 0x7fc45c19caf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:15.072 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:14.999+0000 7fc4624c9640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc45c106c20 0x7fc45c1a1d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:15.072 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:14.999+0000 7fc4624c9640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fc45c1139c0 con 0x7fc45c106c20 2024-09-21T03:42:15.072 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:14.999+0000 7fc4624c9640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fc45c113840 con 0x7fc45c101ff0 2024-09-21T03:42:15.072 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:14.999+0000 7fc4624c9640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fc45c113b40 con 0x7fc45c1062d0 2024-09-21T03:42:15.073 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:14.999+0000 7fc460a66640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc45c106c20 0x7fc45c1a1d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:15.073 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:14.999+0000 7fc45bfff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc45c101ff0 0x7fc45c19c550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:15.073 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:14.999+0000 7fc460a66640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc45c106c20 0x7fc45c1a1d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:49834/0 (socket says 172.21.15.89:49834) 2024-09-21T03:42:15.073 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:14.999+0000 7fc45bfff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc45c101ff0 0x7fc45c19c550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.89:41434/0 (socket says 172.21.15.89:41434) 2024-09-21T03:42:15.073 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:14.999+0000 7fc460a66640 1 -- 172.21.15.89:0/3106515515 learned_addr learned my addr 172.21.15.89:0/3106515515 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-21T03:42:15.073 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:15.000+0000 7fc453fff640 1 --2- 172.21.15.89:0/3106515515 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc45c1062d0 0x7fc45c19caf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:15.073 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:15.000+0000 7fc460a66640 1 -- 172.21.15.89:0/3106515515 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc45c1062d0 msgr2=0x7fc45c19caf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:15.073 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:15.000+0000 7fc460a66640 1 --2- 172.21.15.89:0/3106515515 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc45c1062d0 0x7fc45c19caf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:15.073 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:15.000+0000 7fc460a66640 1 -- 172.21.15.89:0/3106515515 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc45c101ff0 msgr2=0x7fc45c19c550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:15.073 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:15.000+0000 7fc460a66640 1 --2- 172.21.15.89:0/3106515515 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc45c101ff0 0x7fc45c19c550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:15.074 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:15.000+0000 7fc460a66640 1 -- 172.21.15.89:0/3106515515 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc45c1a24c0 con 0x7fc45c106c20 2024-09-21T03:42:15.074 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:15.000+0000 7fc460a66640 1 --2- 172.21.15.89:0/3106515515 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc45c106c20 0x7fc45c1a1d90 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7fc454005cc0 tx=0x7fc454032a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:15.074 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:15.000+0000 7fc459ffb640 1 -- 172.21.15.89:0/3106515515 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc454032e00 con 0x7fc45c106c20 2024-09-21T03:42:15.074 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:15.000+0000 7fc4624c9640 1 -- 172.21.15.89:0/3106515515 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc45c1a2750 con 0x7fc45c106c20 2024-09-21T03:42:15.074 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:15.000+0000 7fc459ffb640 1 -- 172.21.15.89:0/3106515515 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc4540044d0 con 0x7fc45c106c20 2024-09-21T03:42:15.074 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:15.000+0000 7fc459ffb640 1 -- 172.21.15.89:0/3106515515 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc454030b70 con 0x7fc45c106c20 2024-09-21T03:42:15.075 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:15.000+0000 7fc4624c9640 1 -- 172.21.15.89:0/3106515515 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc45c1a8f10 con 0x7fc45c106c20 2024-09-21T03:42:15.075 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:15.001+0000 7fc4624c9640 1 -- 172.21.15.89:0/3106515515 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc45c06b950 con 0x7fc45c106c20 2024-09-21T03:42:15.075 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:15.004+0000 7fc459ffb640 1 -- 172.21.15.89:0/3106515515 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 21) ==== 154512+0+0 (secure 0 0 0) 0x7fc454030d10 con 0x7fc45c106c20 2024-09-21T03:42:15.075 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:15.004+0000 7fc459ffb640 1 --2- 172.21.15.89:0/3106515515 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7fc4480b6510 0x7fc4480b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:15.075 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:15.005+0000 7fc45bfff640 1 --2- 172.21.15.89:0/3106515515 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7fc4480b6510 0x7fc4480b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:15.075 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:15.005+0000 7fc459ffb640 1 -- 172.21.15.89:0/3106515515 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 9121+0+0 (secure 0 0 0) 0x7fc4541089b0 con 0x7fc45c106c20 2024-09-21T03:42:15.076 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:15.005+0000 7fc45bfff640 1 --2- 172.21.15.89:0/3106515515 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7fc4480b6510 0x7fc4480b8a20 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fc44c0058c0 tx=0x7fc44c00a2d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:15.076 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:15.008+0000 7fc459ffb640 1 -- 172.21.15.89:0/3106515515 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fc4540ceec0 con 0x7fc45c106c20 2024-09-21T03:42:15.211 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:42:14 smithi114 ceph-mon[34140]: pgmap v445: 129 pgs: 129 active+clean; 258 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 2.6 MiB/s wr, 605 op/s 2024-09-21T03:42:15.232 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:15.230+0000 7fc4624c9640 1 -- 172.21.15.89:0/3106515515 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc45c102400 con 0x7fc45c106c20 2024-09-21T03:42:15.233 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:15.231+0000 7fc459ffb640 1 -- 172.21.15.89:0/3106515515 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11912 (secure 0 0 0) 0x7fc45c102400 con 0x7fc45c106c20 2024-09-21T03:42:15.233 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-09-21T03:42:15.233 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":26,"btime":"2024-09-21T03:39:26:058611+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14721,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6835/2341538514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":2341538514},{"type":"v1","addr":"172.21.15.89:6835","nonce":2341538514}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14733,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6837/146708592","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6836","nonce":146708592},{"type":"v1","addr":"172.21.15.89:6837","nonce":146708592}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14781,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6839/2269965499","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6838","nonce":2269965499},{"type":"v1","addr":"172.21.15.89:6839","nonce":2269965499}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14793,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6841/650850383","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6840","nonce":650850383},{"type":"v1","addr":"172.21.15.89:6841","nonce":650850383}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24565,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6833/752336912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6832","nonce":752336912},{"type":"v1","addr":"172.21.15.114:6833","nonce":752336912}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24575,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6833/127607331","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6832","nonce":127607331},{"type":"v1","addr":"172.21.15.110:6833","nonce":127607331}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T03:37:13.570694+0000","modified":"2024-09-21T03:39:25.333954+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24638,"mds_1":24631,"mds_2":24614},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24593":{"gid":24593,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.110:6835/671820654","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6834","nonce":671820654},{"type":"v1","addr":"172.21.15.110:6835","nonce":671820654}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24604":{"gid":24604,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.114:6835/1049714548","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6834","nonce":1049714548},{"type":"v1","addr":"172.21.15.114:6835","nonce":1049714548}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24610":{"gid":24610,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.114:6837/1941213818","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6836","nonce":1941213818},{"type":"v1","addr":"172.21.15.114:6837","nonce":1941213818}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24614":{"gid":24614,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":15,"addr":"172.21.15.110:6837/4047124589","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6836","nonce":4047124589},{"type":"v1","addr":"172.21.15.110:6837","nonce":4047124589}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24631":{"gid":24631,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":15,"addr":"172.21.15.114:6839/3427369944","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6838","nonce":3427369944},{"type":"v1","addr":"172.21.15.114:6839","nonce":3427369944}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24638":{"gid":24638,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.110:6839/132366709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6838","nonce":132366709},{"type":"v1","addr":"172.21.15.110:6839","nonce":132366709}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24638,"qdb_cluster":[24638,24631,24614]},"id":1}]} 2024-09-21T03:42:15.234 INFO:teuthology.orchestra.run.smithi089.stderr:dumped fsmap epoch 26 2024-09-21T03:42:15.235 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:15.233+0000 7fc4624c9640 1 -- 172.21.15.89:0/3106515515 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7fc4480b6510 msgr2=0x7fc4480b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:15.236 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:15.233+0000 7fc4624c9640 1 --2- 172.21.15.89:0/3106515515 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7fc4480b6510 0x7fc4480b8a20 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fc44c0058c0 tx=0x7fc44c00a2d0 comp rx=0 tx=0).stop 2024-09-21T03:42:15.236 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:15.233+0000 7fc4624c9640 1 -- 172.21.15.89:0/3106515515 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc45c106c20 msgr2=0x7fc45c1a1d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:15.236 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:15.233+0000 7fc4624c9640 1 --2- 172.21.15.89:0/3106515515 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc45c106c20 0x7fc45c1a1d90 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7fc454005cc0 tx=0x7fc454032a10 comp rx=0 tx=0).stop 2024-09-21T03:42:15.236 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:15.233+0000 7fc4624c9640 1 -- 172.21.15.89:0/3106515515 shutdown_connections 2024-09-21T03:42:15.236 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:15.233+0000 7fc4624c9640 1 --2- 172.21.15.89:0/3106515515 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7fc4480b6510 0x7fc4480b8a20 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:15.236 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:15.233+0000 7fc4624c9640 1 --2- 172.21.15.89:0/3106515515 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc45c106c20 0x7fc45c1a1d90 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:15.236 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:15.233+0000 7fc4624c9640 1 --2- 172.21.15.89:0/3106515515 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc45c1062d0 0x7fc45c19caf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:15.236 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:15.233+0000 7fc4624c9640 1 --2- 172.21.15.89:0/3106515515 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc45c101ff0 0x7fc45c19c550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:15.236 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:15.233+0000 7fc4624c9640 1 -- 172.21.15.89:0/3106515515 >> 172.21.15.89:0/3106515515 conn(0x7fc45c079e30 msgr2=0x7fc45c107360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:42:15.236 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:15.233+0000 7fc4624c9640 1 -- 172.21.15.89:0/3106515515 shutdown_connections 2024-09-21T03:42:15.236 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:15.233+0000 7fc4624c9640 1 -- 172.21.15.89:0/3106515515 wait complete. 2024-09-21T03:42:15.250 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:42:14 smithi110 ceph-mon[34000]: pgmap v445: 129 pgs: 129 active+clean; 258 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 2.6 MiB/s wr, 605 op/s 2024-09-21T03:42:15.387 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:15.372+0000 7ff91e7ea640 1 -- 172.21.15.89:0/1128183219 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff918107690 msgr2=0x7ff918107aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:15.387 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:15.372+0000 7ff91e7ea640 1 --2- 172.21.15.89:0/1128183219 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff918107690 0x7ff918107aa0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7ff908009900 tx=0x7ff90802a410 comp rx=0 tx=0).stop 2024-09-21T03:42:15.387 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:15.372+0000 7ff91e7ea640 1 -- 172.21.15.89:0/1128183219 shutdown_connections 2024-09-21T03:42:15.387 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:15.372+0000 7ff91e7ea640 1 --2- 172.21.15.89:0/1128183219 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff918103950 0x7ff91810f500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:15.387 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:15.372+0000 7ff91e7ea640 1 --2- 172.21.15.89:0/1128183219 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff918103000 0x7ff918103410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:15.387 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:15.372+0000 7ff91e7ea640 1 --2- 172.21.15.89:0/1128183219 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff918107690 0x7ff918107aa0 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:15.387 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:15.372+0000 7ff91e7ea640 1 -- 172.21.15.89:0/1128183219 >> 172.21.15.89:0/1128183219 conn(0x7ff918079ea0 msgr2=0x7ff91807a2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:42:15.388 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:15.372+0000 7ff91e7ea640 1 -- 172.21.15.89:0/1128183219 shutdown_connections 2024-09-21T03:42:15.388 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:15.372+0000 7ff91e7ea640 1 -- 172.21.15.89:0/1128183219 wait complete. 2024-09-21T03:42:15.388 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:15.373+0000 7ff91e7ea640 1 Processor -- start 2024-09-21T03:42:15.388 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:15.373+0000 7ff91e7ea640 1 -- start start 2024-09-21T03:42:15.388 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:15.373+0000 7ff91e7ea640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff918103000 0x7ff91819b300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:15.388 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:15.373+0000 7ff91e7ea640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff918103950 0x7ff91819b840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:15.388 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:15.373+0000 7ff91e7ea640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff918107690 0x7ff91819bd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:15.388 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:15.373+0000 7ff91e7ea640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7ff9181128f0 con 0x7ff918103000 2024-09-21T03:42:15.388 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:15.373+0000 7ff91e7ea640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7ff918112770 con 0x7ff918103950 2024-09-21T03:42:15.388 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:15.373+0000 7ff91e7ea640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7ff918112a70 con 0x7ff918107690 2024-09-21T03:42:15.388 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:15.373+0000 7ff917fff640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff918103000 0x7ff91819b300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:15.388 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:15.373+0000 7ff917fff640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff918103000 0x7ff91819b300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:49842/0 (socket says 172.21.15.89:49842) 2024-09-21T03:42:15.388 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:15.373+0000 7ff917fff640 1 -- 172.21.15.89:0/1189117383 learned_addr learned my addr 172.21.15.89:0/1189117383 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-21T03:42:15.389 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:15.373+0000 7ff91cd87640 1 --2- 172.21.15.89:0/1189117383 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff918107690 0x7ff91819bd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:15.389 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:15.373+0000 7ff90ffff640 1 --2- 172.21.15.89:0/1189117383 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff918103950 0x7ff91819b840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:15.389 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:15.374+0000 7ff917fff640 1 -- 172.21.15.89:0/1189117383 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff918107690 msgr2=0x7ff91819bd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:15.389 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:15.374+0000 7ff917fff640 1 --2- 172.21.15.89:0/1189117383 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff918107690 0x7ff91819bd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:15.389 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:15.374+0000 7ff917fff640 1 -- 172.21.15.89:0/1189117383 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff918103950 msgr2=0x7ff91819b840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:15.389 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:15.374+0000 7ff917fff640 1 --2- 172.21.15.89:0/1189117383 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff918103950 0x7ff91819b840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:15.389 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:15.374+0000 7ff917fff640 1 -- 172.21.15.89:0/1189117383 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9181a65c0 con 0x7ff918103000 2024-09-21T03:42:15.389 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:15.374+0000 7ff90ffff640 1 --2- 172.21.15.89:0/1189117383 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff918103950 0x7ff91819b840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:42:15.389 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:15.374+0000 7ff917fff640 1 --2- 172.21.15.89:0/1189117383 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff918103000 0x7ff91819b300 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7ff908009a20 tx=0x7ff908004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:15.389 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:15.374+0000 7ff915ffb640 1 -- 172.21.15.89:0/1189117383 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff90803f030 con 0x7ff918103000 2024-09-21T03:42:15.389 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:15.374+0000 7ff915ffb640 1 -- 172.21.15.89:0/1189117383 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff9080035f0 con 0x7ff918103000 2024-09-21T03:42:15.389 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:15.374+0000 7ff91e7ea640 1 -- 172.21.15.89:0/1189117383 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff9181a6850 con 0x7ff918103000 2024-09-21T03:42:15.389 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:15.374+0000 7ff915ffb640 1 -- 172.21.15.89:0/1189117383 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff908035070 con 0x7ff918103000 2024-09-21T03:42:15.389 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:15.374+0000 7ff91e7ea640 1 -- 172.21.15.89:0/1189117383 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9181a6d30 con 0x7ff918103000 2024-09-21T03:42:15.390 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:15.375+0000 7ff91e7ea640 1 -- 172.21.15.89:0/1189117383 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff918103410 con 0x7ff918103000 2024-09-21T03:42:15.390 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:15.376+0000 7ff915ffb640 1 -- 172.21.15.89:0/1189117383 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 21) ==== 154512+0+0 (secure 0 0 0) 0x7ff908003aa0 con 0x7ff918103000 2024-09-21T03:42:15.390 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:15.378+0000 7ff915ffb640 1 --2- 172.21.15.89:0/1189117383 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7ff9040b6510 0x7ff9040b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:15.390 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:15.378+0000 7ff90ffff640 1 --2- 172.21.15.89:0/1189117383 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7ff9040b6510 0x7ff9040b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:15.390 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:15.378+0000 7ff915ffb640 1 -- 172.21.15.89:0/1189117383 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 9121+0+0 (secure 0 0 0) 0x7ff908108640 con 0x7ff918103000 2024-09-21T03:42:15.390 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:15.378+0000 7ff90ffff640 1 --2- 172.21.15.89:0/1189117383 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7ff9040b6510 0x7ff9040b8a20 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7ff9181054f0 tx=0x7ff900005ec0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:15.390 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:15.381+0000 7ff915ffb640 1 -- 172.21.15.89:0/1189117383 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7ff9080ceb50 con 0x7ff918103000 2024-09-21T03:42:15.604 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:15.601+0000 7ff91e7ea640 1 -- 172.21.15.89:0/1189117383 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff918102320 con 0x7ff918103000 2024-09-21T03:42:15.604 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:15.602+0000 7ff915ffb640 1 -- 172.21.15.89:0/1189117383 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11912 (secure 0 0 0) 0x7ff9080d2f30 con 0x7ff918103000 2024-09-21T03:42:15.605 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-09-21T03:42:15.605 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":26,"btime":"2024-09-21T03:39:26:058611+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14721,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6835/2341538514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":2341538514},{"type":"v1","addr":"172.21.15.89:6835","nonce":2341538514}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14733,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6837/146708592","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6836","nonce":146708592},{"type":"v1","addr":"172.21.15.89:6837","nonce":146708592}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14781,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6839/2269965499","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6838","nonce":2269965499},{"type":"v1","addr":"172.21.15.89:6839","nonce":2269965499}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14793,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6841/650850383","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6840","nonce":650850383},{"type":"v1","addr":"172.21.15.89:6841","nonce":650850383}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24565,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6833/752336912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6832","nonce":752336912},{"type":"v1","addr":"172.21.15.114:6833","nonce":752336912}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24575,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6833/127607331","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6832","nonce":127607331},{"type":"v1","addr":"172.21.15.110:6833","nonce":127607331}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T03:37:13.570694+0000","modified":"2024-09-21T03:39:25.333954+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24638,"mds_1":24631,"mds_2":24614},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24593":{"gid":24593,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.110:6835/671820654","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6834","nonce":671820654},{"type":"v1","addr":"172.21.15.110:6835","nonce":671820654}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24604":{"gid":24604,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.114:6835/1049714548","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6834","nonce":1049714548},{"type":"v1","addr":"172.21.15.114:6835","nonce":1049714548}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24610":{"gid":24610,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.114:6837/1941213818","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6836","nonce":1941213818},{"type":"v1","addr":"172.21.15.114:6837","nonce":1941213818}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24614":{"gid":24614,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":15,"addr":"172.21.15.110:6837/4047124589","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6836","nonce":4047124589},{"type":"v1","addr":"172.21.15.110:6837","nonce":4047124589}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24631":{"gid":24631,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":15,"addr":"172.21.15.114:6839/3427369944","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6838","nonce":3427369944},{"type":"v1","addr":"172.21.15.114:6839","nonce":3427369944}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24638":{"gid":24638,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.110:6839/132366709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6838","nonce":132366709},{"type":"v1","addr":"172.21.15.110:6839","nonce":132366709}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24638,"qdb_cluster":[24638,24631,24614]},"id":1}]} 2024-09-21T03:42:15.605 INFO:teuthology.orchestra.run.smithi089.stderr:dumped fsmap epoch 26 2024-09-21T03:42:15.607 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:15.604+0000 7ff91e7ea640 1 -- 172.21.15.89:0/1189117383 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7ff9040b6510 msgr2=0x7ff9040b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:15.607 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:15.604+0000 7ff91e7ea640 1 --2- 172.21.15.89:0/1189117383 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7ff9040b6510 0x7ff9040b8a20 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7ff9181054f0 tx=0x7ff900005ec0 comp rx=0 tx=0).stop 2024-09-21T03:42:15.608 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:15.604+0000 7ff91e7ea640 1 -- 172.21.15.89:0/1189117383 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff918103000 msgr2=0x7ff91819b300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:15.608 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:15.604+0000 7ff91e7ea640 1 --2- 172.21.15.89:0/1189117383 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff918103000 0x7ff91819b300 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7ff908009a20 tx=0x7ff908004270 comp rx=0 tx=0).stop 2024-09-21T03:42:15.608 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:15.604+0000 7ff91e7ea640 1 -- 172.21.15.89:0/1189117383 shutdown_connections 2024-09-21T03:42:15.608 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:15.604+0000 7ff91e7ea640 1 --2- 172.21.15.89:0/1189117383 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7ff9040b6510 0x7ff9040b8a20 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:15.608 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:15.604+0000 7ff91e7ea640 1 --2- 172.21.15.89:0/1189117383 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff918103950 0x7ff91819b840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:15.608 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:15.604+0000 7ff91e7ea640 1 --2- 172.21.15.89:0/1189117383 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff918107690 0x7ff91819bd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:15.608 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:15.604+0000 7ff91e7ea640 1 --2- 172.21.15.89:0/1189117383 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff918103000 0x7ff91819b300 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:15.608 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:15.604+0000 7ff91e7ea640 1 -- 172.21.15.89:0/1189117383 >> 172.21.15.89:0/1189117383 conn(0x7ff918079ea0 msgr2=0x7ff9181065a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:42:15.608 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:15.604+0000 7ff91e7ea640 1 -- 172.21.15.89:0/1189117383 shutdown_connections 2024-09-21T03:42:15.608 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:15.604+0000 7ff91e7ea640 1 -- 172.21.15.89:0/1189117383 wait complete. 2024-09-21T03:42:16.146 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24638 quiesce db --cancel --all' 2024-09-21T03:42:16.146 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 7c374098-77c9-11ef-baf5-efdc52797490 -- timeout 105 ceph tell mds.24638 quiesce db --cancel --all 2024-09-21T03:42:16.193 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:42:15 smithi089 ceph-mon[30928]: from='client.? 172.21.15.89:0/3106515515' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:42:16.193 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:42:15 smithi089 ceph-mon[30928]: from='client.? 172.21.15.89:0/1189117383' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:42:16.210 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:42:15 smithi114 ceph-mon[34140]: from='client.? 172.21.15.89:0/3106515515' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:42:16.211 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:42:15 smithi114 ceph-mon[34140]: from='client.? 172.21.15.89:0/1189117383' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:42:16.250 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:42:15 smithi110 ceph-mon[34000]: from='client.? 172.21.15.89:0/3106515515' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:42:16.250 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:42:15 smithi110 ceph-mon[34000]: from='client.? 172.21.15.89:0/1189117383' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:42:16.473 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/7c374098-77c9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:42:16.903 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24638 quiesce db / --timeout 90 --expiration 147.8 --await-for 45.0' 2024-09-21T03:42:16.904 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 7c374098-77c9-11ef-baf5-efdc52797490 -- timeout 105 ceph tell mds.24638 quiesce db / --timeout 90 --expiration 147.8 --await-for 45.0 2024-09-21T03:42:17.152 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:42:16 smithi089 ceph-mon[30928]: pgmap v446: 129 pgs: 129 active+clean; 258 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 1.3 MiB/s wr, 283 op/s 2024-09-21T03:42:17.152 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:42:16 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:42:17.153 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:42:16 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:42:17.153 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:42:16 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:42:17.153 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:42:16 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:42:17.211 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:42:16 smithi114 ceph-mon[34140]: pgmap v446: 129 pgs: 129 active+clean; 258 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 1.3 MiB/s wr, 283 op/s 2024-09-21T03:42:17.211 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:42:16 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:42:17.211 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:42:16 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:42:17.211 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:42:16 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:42:17.211 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:42:16 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:42:17.250 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:42:16 smithi110 ceph-mon[34000]: pgmap v446: 129 pgs: 129 active+clean; 258 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 1.3 MiB/s wr, 283 op/s 2024-09-21T03:42:17.250 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:42:16 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:42:17.250 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:42:16 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:42:17.250 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:42:16 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:42:17.250 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:42:16 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' 2024-09-21T03:42:17.364 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:17.357+0000 7f888f5d4640 1 -- 172.21.15.89:0/1249864212 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8888108d00 msgr2=0x7f888810f530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:17.364 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:17.357+0000 7f888f5d4640 1 --2- 172.21.15.89:0/1249864212 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8888108d00 0x7f888810f530 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f8880009ca0 tx=0x7f888002a8c0 comp rx=0 tx=0).stop 2024-09-21T03:42:17.364 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:17.357+0000 7f888f5d4640 1 -- 172.21.15.89:0/1249864212 shutdown_connections 2024-09-21T03:42:17.364 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:17.357+0000 7f888f5d4640 1 --2- 172.21.15.89:0/1249864212 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8888108d00 0x7f888810f530 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:17.364 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:17.357+0000 7f888f5d4640 1 --2- 172.21.15.89:0/1249864212 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f88881083b0 0x7f88881087c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:17.365 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:17.357+0000 7f888f5d4640 1 --2- 172.21.15.89:0/1249864212 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8888102ac0 0x7f8888102ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:17.365 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:17.357+0000 7f888f5d4640 1 -- 172.21.15.89:0/1249864212 >> 172.21.15.89:0/1249864212 conn(0x7f8888079720 msgr2=0x7f8888079b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:42:17.365 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:17.357+0000 7f888f5d4640 1 -- 172.21.15.89:0/1249864212 shutdown_connections 2024-09-21T03:42:17.365 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:17.357+0000 7f888f5d4640 1 -- 172.21.15.89:0/1249864212 wait complete. 2024-09-21T03:42:17.365 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:17.358+0000 7f888f5d4640 1 Processor -- start 2024-09-21T03:42:17.365 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:17.358+0000 7f888f5d4640 1 -- start start 2024-09-21T03:42:17.365 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:17.358+0000 7f888f5d4640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8888102ac0 0x7f888819b1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:17.365 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:17.358+0000 7f888f5d4640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f88881083b0 0x7f888819b710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:17.365 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:17.358+0000 7f888f5d4640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8888108d00 0x7f888819bc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:17.365 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:17.358+0000 7f888f5d4640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f88881128c0 con 0x7f8888108d00 2024-09-21T03:42:17.365 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:17.358+0000 7f888f5d4640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f8888112740 con 0x7f88881083b0 2024-09-21T03:42:17.366 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:17.358+0000 7f888f5d4640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f8888112a40 con 0x7f8888102ac0 2024-09-21T03:42:17.366 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:17.358+0000 7f888db71640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8888108d00 0x7f888819bc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:17.366 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:17.358+0000 7f888db71640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8888108d00 0x7f888819bc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:49860/0 (socket says 172.21.15.89:49860) 2024-09-21T03:42:17.366 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:17.358+0000 7f888db71640 1 -- 172.21.15.89:0/1686702654 learned_addr learned my addr 172.21.15.89:0/1686702654 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-21T03:42:17.366 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:17.358+0000 7f888d370640 1 --2- 172.21.15.89:0/1686702654 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8888102ac0 0x7f888819b1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:17.366 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:17.358+0000 7f888cb6f640 1 --2- 172.21.15.89:0/1686702654 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f88881083b0 0x7f888819b710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:17.366 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:17.358+0000 7f888db71640 1 -- 172.21.15.89:0/1686702654 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8888102ac0 msgr2=0x7f888819b1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:17.366 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:17.358+0000 7f888db71640 1 --2- 172.21.15.89:0/1686702654 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8888102ac0 0x7f888819b1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:17.366 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:17.358+0000 7f888db71640 1 -- 172.21.15.89:0/1686702654 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f88881083b0 msgr2=0x7f888819b710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:17.366 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:17.358+0000 7f888db71640 1 --2- 172.21.15.89:0/1686702654 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f88881083b0 0x7f888819b710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:17.366 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:17.358+0000 7f888db71640 1 -- 172.21.15.89:0/1686702654 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88881a6490 con 0x7f8888108d00 2024-09-21T03:42:17.366 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:17.358+0000 7f888cb6f640 1 --2- 172.21.15.89:0/1686702654 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f88881083b0 0x7f888819b710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:42:17.367 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:17.359+0000 7f888db71640 1 --2- 172.21.15.89:0/1686702654 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8888108d00 0x7f888819bc50 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f8880005cc0 tx=0x7f8880032a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:17.367 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:17.359+0000 7f887e7fc640 1 -- 172.21.15.89:0/1686702654 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8880032e00 con 0x7f8888108d00 2024-09-21T03:42:17.367 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:17.359+0000 7f887e7fc640 1 -- 172.21.15.89:0/1686702654 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f88800044d0 con 0x7f8888108d00 2024-09-21T03:42:17.367 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:17.359+0000 7f888f5d4640 1 -- 172.21.15.89:0/1686702654 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f88881a6720 con 0x7f8888108d00 2024-09-21T03:42:17.367 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:17.359+0000 7f887e7fc640 1 -- 172.21.15.89:0/1686702654 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8880030b70 con 0x7f8888108d00 2024-09-21T03:42:17.367 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:17.359+0000 7f888f5d4640 1 -- 172.21.15.89:0/1686702654 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88881a6cb0 con 0x7f8888108d00 2024-09-21T03:42:17.371 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:17.362+0000 7f8895b48740 1 Processor -- start 2024-09-21T03:42:17.371 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:17.362+0000 7f887e7fc640 1 -- 172.21.15.89:0/1686702654 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 21) ==== 154512+0+0 (secure 0 0 0) 0x7f8880004680 con 0x7f8888108d00 2024-09-21T03:42:17.371 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:17.362+0000 7f8895b48740 1 -- start start 2024-09-21T03:42:17.371 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:17.363+0000 7f8895b48740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x563f8b3b12d0 0x563f8b758310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:17.371 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:17.363+0000 7f8895b48740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563f8b388410 0x563f8b758850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:17.371 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:17.363+0000 7f8895b48740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563f8b3905f0 0x563f8b758d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:17.371 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:17.363+0000 7f8895b48740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x563f8b6cedd0 con 0x563f8b388410 2024-09-21T03:42:17.371 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:17.363+0000 7f8895b48740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x563f8b6be200 con 0x563f8b3905f0 2024-09-21T03:42:17.372 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:17.363+0000 7f8895b48740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x563f8b6c2010 con 0x563f8b3b12d0 2024-09-21T03:42:17.372 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:17.363+0000 7f888d370640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563f8b388410 0x563f8b758850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:17.372 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:17.363+0000 7f888d370640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563f8b388410 0x563f8b758850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:49862/0 (socket says 172.21.15.89:49862) 2024-09-21T03:42:17.372 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:17.363+0000 7f888d370640 1 -- 172.21.15.89:0/1808882923 learned_addr learned my addr 172.21.15.89:0/1808882923 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-21T03:42:17.372 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:17.363+0000 7f887e7fc640 1 --2- 172.21.15.89:0/1686702654 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f88740b6590 0x7f88740b8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:17.372 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:17.363+0000 7f888cb6f640 1 --2- 172.21.15.89:0/1808882923 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x563f8b3b12d0 0x563f8b758310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:17.372 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:17.363+0000 7f888d370640 1 -- 172.21.15.89:0/1808882923 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x563f8b3b12d0 msgr2=0x563f8b758310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:42:17.372 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:17.363+0000 7f888d370640 1 --2- 172.21.15.89:0/1808882923 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x563f8b3b12d0 0x563f8b758310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:17.372 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:17.363+0000 7f888d370640 1 -- 172.21.15.89:0/1808882923 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563f8b3905f0 msgr2=0x563f8b758d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:42:17.372 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:17.363+0000 7f888d370640 1 --2- 172.21.15.89:0/1808882923 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563f8b3905f0 0x563f8b758d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:17.372 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:17.363+0000 7f888d370640 1 -- 172.21.15.89:0/1808882923 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563f8b41c8b0 con 0x563f8b388410 2024-09-21T03:42:17.372 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:17.363+0000 7f888cb6f640 1 --2- 172.21.15.89:0/1808882923 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x563f8b3b12d0 0x563f8b758310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:42:17.372 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:17.364+0000 7f888db71640 1 --2- 172.21.15.89:0/1686702654 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f88740b6590 0x7f88740b8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:17.372 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:17.364+0000 7f888d370640 1 --2- 172.21.15.89:0/1808882923 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563f8b388410 0x563f8b758850 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f888400bc90 tx=0x7f88840107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5baca185568f65b9 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:17.372 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:17.364+0000 7f887e7fc640 1 -- 172.21.15.89:0/1686702654 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 9121+0+0 (secure 0 0 0) 0x7f8880108870 con 0x7f8888108d00 2024-09-21T03:42:17.373 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:17.364+0000 7f88737fe640 1 -- 172.21.15.89:0/1808882923 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8884005020 con 0x563f8b388410 2024-09-21T03:42:17.373 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:17.364+0000 7f888db71640 1 --2- 172.21.15.89:0/1686702654 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f88740b6590 0x7f88740b8aa0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f88800cae40 tx=0x7f88800cacd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:17.373 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:17.364+0000 7f88737fe640 1 -- 172.21.15.89:0/1808882923 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f88840047a0 con 0x563f8b388410 2024-09-21T03:42:17.373 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:17.364+0000 7f88737fe640 1 -- 172.21.15.89:0/1808882923 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88840116a0 con 0x563f8b388410 2024-09-21T03:42:17.373 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:17.365+0000 7f8895b48740 1 -- 172.21.15.89:0/1808882923 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563f8b388410 msgr2=0x563f8b758850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:17.373 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:17.365+0000 7f8895b48740 1 --2- 172.21.15.89:0/1808882923 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563f8b388410 0x563f8b758850 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f888400bc90 tx=0x7f88840107a0 comp rx=0 tx=0).stop 2024-09-21T03:42:17.373 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:17.365+0000 7f8895b48740 1 -- 172.21.15.89:0/1808882923 shutdown_connections 2024-09-21T03:42:17.373 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:17.365+0000 7f8895b48740 1 --2- 172.21.15.89:0/1808882923 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563f8b3905f0 0x563f8b758d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:17.373 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:17.365+0000 7f8895b48740 1 --2- 172.21.15.89:0/1808882923 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563f8b388410 0x563f8b758850 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:17.373 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:17.365+0000 7f8895b48740 1 --2- 172.21.15.89:0/1808882923 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x563f8b3b12d0 0x563f8b758310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:17.373 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:17.365+0000 7f8895b48740 1 -- 172.21.15.89:0/1808882923 >> 172.21.15.89:0/1808882923 conn(0x563f8b3a56c0 msgr2=0x563f8b756200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:42:17.373 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:17.365+0000 7f8895b48740 1 -- 172.21.15.89:0/1808882923 shutdown_connections 2024-09-21T03:42:17.373 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:17.365+0000 7f8895b48740 1 -- 172.21.15.89:0/1808882923 wait complete. 2024-09-21T03:42:17.374 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:17.366+0000 7f8895b48740 1 -- start start 2024-09-21T03:42:17.374 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:17.366+0000 7f8895b48740 1 Processor -- start 2024-09-21T03:42:17.374 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:17.366+0000 7f8895b48740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563f8b388410 0x563f8b768390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:17.374 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:17.366+0000 7f8895b48740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x563f8b3905f0 0x563f8b768950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:17.374 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:17.366+0000 7f8895b48740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563f8b3b12d0 0x563f8b774180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:17.374 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:17.366+0000 7f8895b48740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x563f8b51f810 con 0x563f8b388410 2024-09-21T03:42:17.374 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:17.366+0000 7f8895b48740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x563f8b659080 con 0x563f8b3b12d0 2024-09-21T03:42:17.374 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:17.366+0000 7f8895b48740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x563f8b76ae50 con 0x563f8b3905f0 2024-09-21T03:42:17.374 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:17.366+0000 7f8895b48740 10 client.0 _ll_register_callbacks cb 0x563f8b669960 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:42:17.374 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:17.367+0000 7f888cb6f640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563f8b388410 0x563f8b768390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:17.374 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:17.367+0000 7f888cb6f640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563f8b388410 0x563f8b768390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:49866/0 (socket says 172.21.15.89:49866) 2024-09-21T03:42:17.374 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:17.367+0000 7f888d370640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x563f8b3905f0 0x563f8b768950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:17.374 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:17.367+0000 7f888cb6f640 1 -- 172.21.15.89:0/2819380929 learned_addr learned my addr 172.21.15.89:0/2819380929 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-21T03:42:17.374 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:17.367+0000 7f888d370640 1 -- 172.21.15.89:0/2819380929 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563f8b3b12d0 msgr2=0x563f8b774180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:42:17.374 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:17.367+0000 7f888d370640 1 --2- 172.21.15.89:0/2819380929 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563f8b3b12d0 0x563f8b774180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:17.375 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:17.367+0000 7f888d370640 1 -- 172.21.15.89:0/2819380929 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563f8b388410 msgr2=0x563f8b768390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:42:17.375 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:17.367+0000 7f888d370640 1 --2- 172.21.15.89:0/2819380929 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563f8b388410 0x563f8b768390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:17.375 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:17.367+0000 7f888d370640 1 -- 172.21.15.89:0/2819380929 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563f8b663460 con 0x563f8b3905f0 2024-09-21T03:42:17.375 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:17.367+0000 7f888d370640 1 --2- 172.21.15.89:0/2819380929 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x563f8b3905f0 0x563f8b768950 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f8884000c00 tx=0x7f8884004b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1270aa30fba899bb server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:17.375 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:17.367+0000 7f8870ff9640 10 client.0 ms_handle_connect on v2:172.21.15.114:3300/0 2024-09-21T03:42:17.375 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:17.367+0000 7f8870ff9640 1 -- 172.21.15.89:0/2819380929 <== mon.1 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8884028020 con 0x563f8b3905f0 2024-09-21T03:42:17.375 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:17.367+0000 7f8895b48740 1 -- 172.21.15.89:0/2819380929 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f88840047a0 con 0x563f8b3905f0 2024-09-21T03:42:17.375 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:17.367+0000 7f8870ff9640 1 -- 172.21.15.89:0/2819380929 <== mon.1 v2:172.21.15.114:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 9121+0+0 (secure 0 0 0) 0x7f8884005210 con 0x563f8b3905f0 2024-09-21T03:42:17.375 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:17.368+0000 7f8870ff9640 1 -- 172.21.15.89:0/2819380929 <== mon.1 v2:172.21.15.114:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f8884005410 con 0x563f8b3905f0 2024-09-21T03:42:17.375 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:17.368+0000 7f8895b48740 10 client.25447 fetch_fsmap learned FSMap version 26 2024-09-21T03:42:17.375 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:17.368+0000 7f8895b48740 1 -- 172.21.15.89:0/2819380929 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x563f8b776ea0 con 0x563f8b3905f0 2024-09-21T03:42:17.375 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:17.368+0000 7f8870ff9640 1 -- 172.21.15.89:0/2819380929 <== mon.1 v2:172.21.15.114:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f88840047a0 con 0x563f8b3905f0 2024-09-21T03:42:17.375 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:17.368+0000 7f8870ff9640 1 -- 172.21.15.89:0/2819380929 <== mon.1 v2:172.21.15.114:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f8884003ba0 con 0x563f8b3905f0 2024-09-21T03:42:17.376 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:17.368+0000 7f8895b48740 10 client.25447 fetch_fsmap finished waiting for FSMap version 26 2024-09-21T03:42:17.376 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:17.368+0000 7f8895b48740 10 client.25447 resolve_mds: validated gid 24638 aka daemon mds.b 2024-09-21T03:42:17.376 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:17.368+0000 7f8895b48740 20 client.25447 populate_metadata read hostname 'smithi089' 2024-09-21T03:42:17.376 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:17.368+0000 7f8895b48740 1 --2- 172.21.15.89:0/2819380929 >> [v2:172.21.15.110:6838/132366709,v1:172.21.15.110:6839/132366709] conn(0x563f8b36fc50 0x563f8b77a260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:17.376 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:17.368+0000 7f8895b48740 4 client.25447 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:42:17.376 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:17.368+0000 7f8895b48740 1 -- 172.21.15.89:0/2819380929 --> [v2:172.21.15.110:6838/132366709,v1:172.21.15.110:6839/132366709] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563f8b6d0d20 con 0x563f8b36fc50 2024-09-21T03:42:17.376 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:17.368+0000 7f888cb6f640 1 --2- 172.21.15.89:0/2819380929 >> [v2:172.21.15.110:6838/132366709,v1:172.21.15.110:6839/132366709] conn(0x563f8b36fc50 0x563f8b77a260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:17.376 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:17.369+0000 7f8870ff9640 10 client.25447 ms_handle_connect on v2:172.21.15.110:6838/132366709 2024-09-21T03:42:17.376 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:17.369+0000 7f888cb6f640 1 --2- 172.21.15.89:0/2819380929 >> [v2:172.21.15.110:6838/132366709,v1:172.21.15.110:6839/132366709] conn(0x563f8b36fc50 0x563f8b77a260 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3c58d3ea661f027c server_cookie=91a0af3a521ff755 in_seq=0 out_seq=0 2024-09-21T03:42:17.377 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:17.370+0000 7f8870ff9640 1 -- 172.21.15.89:0/2819380929 <== mds.0 v2:172.21.15.110:6838/132366709 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x563f8b6d0d20 con 0x563f8b36fc50 2024-09-21T03:42:17.377 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:17.370+0000 7f8870ff9640 10 client.25447 handle_command_reply: tid=0 2024-09-21T03:42:17.377 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:17.370+0000 7f8895b48740 1 client.25447 shutdown 2024-09-21T03:42:17.377 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:17.370+0000 7f8895b48740 1 -- 172.21.15.89:0/2819380929 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x563f8b3905f0 msgr2=0x563f8b768950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:17.377 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:17.370+0000 7f8895b48740 1 --2- 172.21.15.89:0/2819380929 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x563f8b3905f0 0x563f8b768950 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f8884000c00 tx=0x7f8884004b80 comp rx=0 tx=0).stop 2024-09-21T03:42:17.377 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:17.370+0000 7f8895b48740 1 -- 172.21.15.89:0/2819380929 shutdown_connections 2024-09-21T03:42:17.377 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:17.370+0000 7f8895b48740 1 --2- 172.21.15.89:0/2819380929 >> [v2:172.21.15.110:6838/132366709,v1:172.21.15.110:6839/132366709] conn(0x563f8b36fc50 0x563f8b77a260 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:17.377 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:17.370+0000 7f8895b48740 1 --2- 172.21.15.89:0/2819380929 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563f8b3b12d0 0x563f8b774180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:17.378 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:17.370+0000 7f8895b48740 1 --2- 172.21.15.89:0/2819380929 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x563f8b3905f0 0x563f8b768950 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:17.378 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:17.370+0000 7f8870ff9640 0 client.25447 ms_handle_reset on v2:172.21.15.110:6838/132366709 2024-09-21T03:42:17.378 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:17.370+0000 7f8895b48740 1 --2- 172.21.15.89:0/2819380929 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563f8b388410 0x563f8b768390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:17.378 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:17.370+0000 7f8895b48740 1 -- 172.21.15.89:0/2819380929 >> 172.21.15.89:0/2819380929 conn(0x563f8b3a56c0 msgr2=0x563f8b759d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:42:17.378 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:17.370+0000 7f8895b48740 1 -- 172.21.15.89:0/2819380929 shutdown_connections 2024-09-21T03:42:17.378 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:17.370+0000 7f8895b48740 1 -- 172.21.15.89:0/2819380929 wait complete. 2024-09-21T03:42:17.378 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:17.370+0000 7f8895b48740 20 client.25447 trim_cache size 0 max 16384 2024-09-21T03:42:17.391 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:17.388+0000 7f8895b48740 1 Processor -- start 2024-09-21T03:42:17.391 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:17.389+0000 7f8895b48740 1 -- start start 2024-09-21T03:42:17.392 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:17.389+0000 7f8895b48740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563f8b388410 0x563f8b756360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:17.392 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:17.389+0000 7f8895b48740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x563f8b3905f0 0x563f8b774080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:17.392 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:17.389+0000 7f8895b48740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563f8b3b12d0 0x563f8b759a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:17.392 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:17.389+0000 7f8895b48740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f88881b10e0 con 0x563f8b388410 2024-09-21T03:42:17.392 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:17.389+0000 7f8895b48740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f88881ae9b0 con 0x563f8b3b12d0 2024-09-21T03:42:17.392 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:17.389+0000 7f8895b48740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f88881ac800 con 0x563f8b3905f0 2024-09-21T03:42:17.392 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:17.389+0000 7f888cb6f640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563f8b388410 0x563f8b756360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:17.392 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:17.389+0000 7f888cb6f640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563f8b388410 0x563f8b756360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:49870/0 (socket says 172.21.15.89:49870) 2024-09-21T03:42:17.392 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:17.389+0000 7f888cb6f640 1 -- 172.21.15.89:0/3584370518 learned_addr learned my addr 172.21.15.89:0/3584370518 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-21T03:42:17.393 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:17.389+0000 7f888d370640 1 --2- 172.21.15.89:0/3584370518 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x563f8b3905f0 0x563f8b774080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:17.393 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:17.389+0000 7f888cb6f640 1 -- 172.21.15.89:0/3584370518 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x563f8b3905f0 msgr2=0x563f8b774080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:42:17.393 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:17.389+0000 7f888cb6f640 1 --2- 172.21.15.89:0/3584370518 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x563f8b3905f0 0x563f8b774080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:17.393 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:17.389+0000 7f888cb6f640 1 -- 172.21.15.89:0/3584370518 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563f8b3b12d0 msgr2=0x563f8b759a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:42:17.393 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:17.389+0000 7f888cb6f640 1 --2- 172.21.15.89:0/3584370518 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563f8b3b12d0 0x563f8b759a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:17.393 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:17.389+0000 7f888cb6f640 1 -- 172.21.15.89:0/3584370518 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563f8b76a8d0 con 0x563f8b388410 2024-09-21T03:42:17.393 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:17.390+0000 7f888cb6f640 1 --2- 172.21.15.89:0/3584370518 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563f8b388410 0x563f8b756360 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x563f8b77a860 tx=0x7f8878012950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9097c4a275805e55 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:17.393 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:17.390+0000 7f8872ffd640 1 -- 172.21.15.89:0/3584370518 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8878004220 con 0x563f8b388410 2024-09-21T03:42:17.393 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:17.390+0000 7f8872ffd640 1 -- 172.21.15.89:0/3584370518 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f887801e030 con 0x563f8b388410 2024-09-21T03:42:17.393 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:17.390+0000 7f8872ffd640 1 -- 172.21.15.89:0/3584370518 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8878011810 con 0x563f8b388410 2024-09-21T03:42:17.394 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:17.391+0000 7f8895b48740 1 -- 172.21.15.89:0/3584370518 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563f8b388410 msgr2=0x563f8b756360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:17.394 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:17.391+0000 7f8895b48740 1 --2- 172.21.15.89:0/3584370518 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563f8b388410 0x563f8b756360 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x563f8b77a860 tx=0x7f8878012950 comp rx=0 tx=0).stop 2024-09-21T03:42:17.394 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:17.391+0000 7f8895b48740 1 -- 172.21.15.89:0/3584370518 shutdown_connections 2024-09-21T03:42:17.394 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:17.391+0000 7f8895b48740 1 --2- 172.21.15.89:0/3584370518 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563f8b3b12d0 0x563f8b759a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:17.394 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:17.391+0000 7f8895b48740 1 --2- 172.21.15.89:0/3584370518 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x563f8b3905f0 0x563f8b774080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:17.394 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:17.391+0000 7f8895b48740 1 --2- 172.21.15.89:0/3584370518 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563f8b388410 0x563f8b756360 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:17.394 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:17.391+0000 7f8895b48740 1 -- 172.21.15.89:0/3584370518 >> 172.21.15.89:0/3584370518 conn(0x563f8b3a56c0 msgr2=0x563f8b759340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:42:17.395 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:17.391+0000 7f8895b48740 1 -- 172.21.15.89:0/3584370518 shutdown_connections 2024-09-21T03:42:17.395 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:17.391+0000 7f8895b48740 1 -- 172.21.15.89:0/3584370518 wait complete. 2024-09-21T03:42:17.395 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:17.392+0000 7f8895b48740 1 -- start start 2024-09-21T03:42:17.395 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:17.392+0000 7f8895b48740 1 Processor -- start 2024-09-21T03:42:17.396 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:17.392+0000 7f8895b48740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563f8b388410 0x563f8b770af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:17.396 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:17.392+0000 7f8895b48740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563f8b3905f0 0x563f8b76d010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:17.397 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:17.392+0000 7f8895b48740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x563f8b3b12d0 0x563f8b76d720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:17.397 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:17.392+0000 7f8895b48740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f88881ace70 con 0x563f8b388410 2024-09-21T03:42:17.397 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:17.392+0000 7f8895b48740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f88881ae2e0 con 0x563f8b3905f0 2024-09-21T03:42:17.397 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:17.393+0000 7f8895b48740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f88881ad540 con 0x563f8b3b12d0 2024-09-21T03:42:17.397 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:17.393+0000 7f8895b48740 10 client.0 _ll_register_callbacks cb 0x7f888400b8e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:42:17.397 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:17.393+0000 7f888cb6f640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563f8b388410 0x563f8b770af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:17.397 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:17.393+0000 7f888cb6f640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563f8b388410 0x563f8b770af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:49878/0 (socket says 172.21.15.89:49878) 2024-09-21T03:42:17.397 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:17.393+0000 7f888cb6f640 1 -- 172.21.15.89:0/3447606805 learned_addr learned my addr 172.21.15.89:0/3447606805 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-21T03:42:17.397 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:17.393+0000 7f888d370640 1 --2- 172.21.15.89:0/3447606805 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563f8b3905f0 0x563f8b76d010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:17.397 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:17.393+0000 7f888cb6f640 1 -- 172.21.15.89:0/3447606805 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x563f8b3b12d0 msgr2=0x563f8b76d720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:42:17.397 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:17.393+0000 7f888cb6f640 1 --2- 172.21.15.89:0/3447606805 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x563f8b3b12d0 0x563f8b76d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:17.397 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:17.393+0000 7f888cb6f640 1 -- 172.21.15.89:0/3447606805 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563f8b3905f0 msgr2=0x563f8b76d010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:42:17.397 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:17.393+0000 7f888cb6f640 1 --2- 172.21.15.89:0/3447606805 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563f8b3905f0 0x563f8b76d010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:17.397 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:17.393+0000 7f888cb6f640 1 -- 172.21.15.89:0/3447606805 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563f8b75de50 con 0x563f8b388410 2024-09-21T03:42:17.398 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:17.393+0000 7f888d370640 1 --2- 172.21.15.89:0/3447606805 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563f8b3905f0 0x563f8b76d010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:42:17.398 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:17.393+0000 7f888cb6f640 1 --2- 172.21.15.89:0/3447606805 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563f8b388410 0x563f8b770af0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f8878004180 tx=0x7f88780041b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db25a59c9ca4a7aa server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:17.398 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:17.393+0000 7f8870ff9640 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2024-09-21T03:42:17.398 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:17.394+0000 7f8870ff9640 1 -- 172.21.15.89:0/3447606805 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8878032060 con 0x563f8b388410 2024-09-21T03:42:17.398 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:17.394+0000 7f8870ff9640 1 -- 172.21.15.89:0/3447606805 <== mon.0 v2:172.21.15.89:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 9121+0+0 (secure 0 0 0) 0x7f88780037f0 con 0x563f8b388410 2024-09-21T03:42:17.398 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:17.394+0000 7f8895b48740 1 -- 172.21.15.89:0/3447606805 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f887801e030 con 0x563f8b388410 2024-09-21T03:42:17.398 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:17.394+0000 7f8870ff9640 1 -- 172.21.15.89:0/3447606805 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f8878004410 con 0x563f8b388410 2024-09-21T03:42:17.398 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:17.394+0000 7f8895b48740 10 client.15897 fetch_fsmap learned FSMap version 26 2024-09-21T03:42:17.398 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:17.394+0000 7f8895b48740 1 -- 172.21.15.89:0/3447606805 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x563f8b759100 con 0x563f8b388410 2024-09-21T03:42:17.400 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:17.395+0000 7f8870ff9640 1 -- 172.21.15.89:0/3447606805 <== mon.0 v2:172.21.15.89:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f887801e030 con 0x563f8b388410 2024-09-21T03:42:17.400 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:17.395+0000 7f8870ff9640 1 -- 172.21.15.89:0/3447606805 <== mon.0 v2:172.21.15.89:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f887802c030 con 0x563f8b388410 2024-09-21T03:42:17.400 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:17.395+0000 7f8895b48740 10 client.15897 fetch_fsmap finished waiting for FSMap version 26 2024-09-21T03:42:17.400 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:17.395+0000 7f8895b48740 10 client.15897 resolve_mds: validated gid 24638 aka daemon mds.b 2024-09-21T03:42:17.400 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:17.395+0000 7f8895b48740 20 client.15897 populate_metadata read hostname 'smithi089' 2024-09-21T03:42:17.400 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:17.395+0000 7f8895b48740 1 --2- 172.21.15.89:0/3447606805 >> [v2:172.21.15.110:6838/132366709,v1:172.21.15.110:6839/132366709] conn(0x563f8b345000 0x563f8b7e7e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:17.400 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:17.395+0000 7f8895b48740 4 client.15897 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "quiesce db", "cancel": true, "all": true}] 2024-09-21T03:42:17.401 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:17.395+0000 7f8895b48740 1 -- 172.21.15.89:0/3447606805 --> [v2:172.21.15.110:6838/132366709,v1:172.21.15.110:6839/132366709] -- command(tid 0: {"prefix": "quiesce db", "cancel": true, "all": true}) -- 0x563f8b6cc7d0 con 0x563f8b345000 2024-09-21T03:42:17.401 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:17.395+0000 7f888d370640 1 --2- 172.21.15.89:0/3447606805 >> [v2:172.21.15.110:6838/132366709,v1:172.21.15.110:6839/132366709] conn(0x563f8b345000 0x563f8b7e7e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:17.401 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:17.396+0000 7f8870ff9640 10 client.15897 ms_handle_connect on v2:172.21.15.110:6838/132366709 2024-09-21T03:42:17.401 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:17.396+0000 7f888d370640 1 --2- 172.21.15.89:0/3447606805 >> [v2:172.21.15.110:6838/132366709,v1:172.21.15.110:6839/132366709] conn(0x563f8b345000 0x563f8b7e7e90 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c425bfad040248b2 server_cookie=da4daf57db3227da in_seq=0 out_seq=0 2024-09-21T03:42:17.401 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:17.396+0000 7f8870ff9640 1 -- 172.21.15.89:0/3447606805 <== mds.0 v2:172.21.15.110:6838/132366709 1 ==== command_reply(tid 0: 0 ) ==== 8+0+80 (crc 0 0 0) 0x563f8b6cc7d0 con 0x563f8b345000 2024-09-21T03:42:17.401 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:17.396+0000 7f8870ff9640 10 client.15897 handle_command_reply: tid=0 2024-09-21T03:42:17.401 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:17.396+0000 7f8895b48740 1 client.15897 shutdown 2024-09-21T03:42:17.401 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:17.396+0000 7f8895b48740 1 -- 172.21.15.89:0/3447606805 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563f8b388410 msgr2=0x563f8b770af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:17.401 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:17.396+0000 7f8895b48740 1 --2- 172.21.15.89:0/3447606805 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563f8b388410 0x563f8b770af0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f8878004180 tx=0x7f88780041b0 comp rx=0 tx=0).stop 2024-09-21T03:42:17.401 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:17.397+0000 7f8895b48740 1 -- 172.21.15.89:0/3447606805 shutdown_connections 2024-09-21T03:42:17.401 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:17.397+0000 7f8895b48740 1 --2- 172.21.15.89:0/3447606805 >> [v2:172.21.15.110:6838/132366709,v1:172.21.15.110:6839/132366709] conn(0x563f8b345000 0x563f8b7e7e90 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:17.401 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:17.397+0000 7f8895b48740 1 --2- 172.21.15.89:0/3447606805 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563f8b3905f0 0x563f8b76d010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:17.401 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:17.397+0000 7f8870ff9640 0 client.15897 ms_handle_reset on v2:172.21.15.110:6838/132366709 2024-09-21T03:42:17.401 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:17.397+0000 7f8895b48740 1 --2- 172.21.15.89:0/3447606805 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x563f8b3b12d0 0x563f8b76d720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:17.402 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:17.397+0000 7f8895b48740 1 --2- 172.21.15.89:0/3447606805 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563f8b388410 0x563f8b770af0 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:17.402 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:17.397+0000 7f8895b48740 1 -- 172.21.15.89:0/3447606805 >> 172.21.15.89:0/3447606805 conn(0x563f8b3a56c0 msgr2=0x563f8b756130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:42:17.402 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:17.397+0000 7f8895b48740 1 -- 172.21.15.89:0/3447606805 shutdown_connections 2024-09-21T03:42:17.402 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:17.397+0000 7f8895b48740 1 -- 172.21.15.89:0/3447606805 wait complete. 2024-09-21T03:42:17.402 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:17.397+0000 7f8895b48740 20 client.15897 trim_cache size 0 max 16384 2024-09-21T03:42:17.402 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-09-21T03:42:17.402 INFO:teuthology.orchestra.run.smithi089.stdout: "epoch": 26, 2024-09-21T03:42:17.402 INFO:teuthology.orchestra.run.smithi089.stdout: "leader": 24638, 2024-09-21T03:42:17.402 INFO:teuthology.orchestra.run.smithi089.stdout: "set_version": 12, 2024-09-21T03:42:17.402 INFO:teuthology.orchestra.run.smithi089.stdout: "sets": {} 2024-09-21T03:42:17.402 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-09-21T03:42:17.403 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:17.398+0000 7f888f5d4640 1 -- 172.21.15.89:0/1686702654 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f88740b6590 msgr2=0x7f88740b8aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:17.403 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:17.398+0000 7f888f5d4640 1 --2- 172.21.15.89:0/1686702654 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f88740b6590 0x7f88740b8aa0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f88800cae40 tx=0x7f88800cacd0 comp rx=0 tx=0).stop 2024-09-21T03:42:17.404 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:17.398+0000 7f888f5d4640 1 -- 172.21.15.89:0/1686702654 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8888108d00 msgr2=0x7f888819bc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:17.404 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:17.398+0000 7f888f5d4640 1 --2- 172.21.15.89:0/1686702654 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8888108d00 0x7f888819bc50 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f8880005cc0 tx=0x7f8880032a10 comp rx=0 tx=0).stop 2024-09-21T03:42:17.404 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:17.398+0000 7f888f5d4640 1 -- 172.21.15.89:0/1686702654 shutdown_connections 2024-09-21T03:42:17.404 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:17.398+0000 7f888f5d4640 1 --2- 172.21.15.89:0/1686702654 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f88740b6590 0x7f88740b8aa0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:17.404 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:17.398+0000 7f888f5d4640 1 --2- 172.21.15.89:0/1686702654 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f88881083b0 0x7f888819b710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:17.404 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:17.398+0000 7f888f5d4640 1 --2- 172.21.15.89:0/1686702654 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8888108d00 0x7f888819bc50 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:17.404 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:17.398+0000 7f888f5d4640 1 --2- 172.21.15.89:0/1686702654 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8888102ac0 0x7f888819b1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:17.404 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:17.398+0000 7f888f5d4640 1 -- 172.21.15.89:0/1686702654 >> 172.21.15.89:0/1686702654 conn(0x7f8888079720 msgr2=0x7f8888106460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:42:17.404 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:17.398+0000 7f888f5d4640 1 -- 172.21.15.89:0/1686702654 shutdown_connections 2024-09-21T03:42:17.404 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:17.398+0000 7f888f5d4640 1 -- 172.21.15.89:0/1686702654 wait complete. 2024-09-21T03:42:17.405 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/7c374098-77c9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:42:18.337 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:42:18.335+0000 7f6e18a86640 1 -- 172.21.15.89:0/576320333 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6e141025c0 msgr2=0x7f6e14102a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:18.338 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:42:18.335+0000 7f6e18a86640 1 --2- 172.21.15.89:0/576320333 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6e141025c0 0x7f6e14102a30 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f6e04009900 tx=0x7f6e0402a410 comp rx=0 tx=0).stop 2024-09-21T03:42:18.338 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:42:18.335+0000 7f6e18a86640 1 -- 172.21.15.89:0/576320333 shutdown_connections 2024-09-21T03:42:18.338 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:42:18.335+0000 7f6e18a86640 1 --2- 172.21.15.89:0/576320333 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6e14103fe0 0x7f6e1410a980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:18.338 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:42:18.335+0000 7f6e18a86640 1 --2- 172.21.15.89:0/576320333 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6e14103640 0x7f6e14103a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:18.338 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:42:18.335+0000 7f6e18a86640 1 --2- 172.21.15.89:0/576320333 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6e141025c0 0x7f6e14102a30 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:18.338 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:42:18.335+0000 7f6e18a86640 1 -- 172.21.15.89:0/576320333 >> 172.21.15.89:0/576320333 conn(0x7f6e140fdf00 msgr2=0x7f6e141003b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:42:18.338 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:42:18.335+0000 7f6e18a86640 1 -- 172.21.15.89:0/576320333 shutdown_connections 2024-09-21T03:42:18.338 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:42:18.335+0000 7f6e18a86640 1 -- 172.21.15.89:0/576320333 wait complete. 2024-09-21T03:42:18.340 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:42:18.336+0000 7f6e18a86640 1 Processor -- start 2024-09-21T03:42:18.341 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:42:18.336+0000 7f6e18a86640 1 -- start start 2024-09-21T03:42:18.341 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:42:18.336+0000 7f6e18a86640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6e141025c0 0x7f6e14198da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:18.341 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:42:18.336+0000 7f6e18a86640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6e14103640 0x7f6e14199410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:18.341 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:42:18.336+0000 7f6e18a86640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6e14103fe0 0x7f6e14199fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:18.341 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:42:18.336+0000 7f6e18a86640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f6e1410e0b0 con 0x7f6e14103fe0 2024-09-21T03:42:18.341 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:42:18.336+0000 7f6e18a86640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f6e1410df30 con 0x7f6e141025c0 2024-09-21T03:42:18.341 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:42:18.336+0000 7f6e18a86640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f6e1410e230 con 0x7f6e14103640 2024-09-21T03:42:18.341 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:42:18.336+0000 7f6e12d9d640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6e14103fe0 0x7f6e14199fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:18.341 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:42:18.336+0000 7f6e12d9d640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6e14103fe0 0x7f6e14199fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:49890/0 (socket says 172.21.15.89:49890) 2024-09-21T03:42:18.341 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:42:18.336+0000 7f6e12d9d640 1 -- 172.21.15.89:0/174256819 learned_addr learned my addr 172.21.15.89:0/174256819 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-21T03:42:18.341 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:42:18.336+0000 7f6e1259c640 1 --2- 172.21.15.89:0/174256819 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6e141025c0 0x7f6e14198da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:18.341 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:42:18.336+0000 7f6e09d9b640 1 --2- 172.21.15.89:0/174256819 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6e14103640 0x7f6e14199410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:18.341 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:42:18.336+0000 7f6e12d9d640 1 -- 172.21.15.89:0/174256819 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6e14103640 msgr2=0x7f6e14199410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:18.341 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:42:18.337+0000 7f6e12d9d640 1 --2- 172.21.15.89:0/174256819 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6e14103640 0x7f6e14199410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:18.342 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:42:18.337+0000 7f6e12d9d640 1 -- 172.21.15.89:0/174256819 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6e141025c0 msgr2=0x7f6e14198da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:18.342 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:42:18.337+0000 7f6e12d9d640 1 --2- 172.21.15.89:0/174256819 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6e141025c0 0x7f6e14198da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:18.342 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:42:18.337+0000 7f6e12d9d640 1 -- 172.21.15.89:0/174256819 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e1419a7e0 con 0x7f6e14103fe0 2024-09-21T03:42:18.342 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:42:18.337+0000 7f6e09d9b640 1 --2- 172.21.15.89:0/174256819 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6e14103640 0x7f6e14199410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:42:18.342 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:42:18.337+0000 7f6e12d9d640 1 --2- 172.21.15.89:0/174256819 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6e14103fe0 0x7f6e14199fc0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f6e0c00cad0 tx=0x7f6e0c00cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:18.342 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:42:18.337+0000 7f6e0bfff640 1 -- 172.21.15.89:0/174256819 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e0c00ae20 con 0x7f6e14103fe0 2024-09-21T03:42:18.342 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:42:18.337+0000 7f6e0bfff640 1 -- 172.21.15.89:0/174256819 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6e0c005020 con 0x7f6e14103fe0 2024-09-21T03:42:18.342 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:42:18.337+0000 7f6e18a86640 1 -- 172.21.15.89:0/174256819 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6e1419aad0 con 0x7f6e14103fe0 2024-09-21T03:42:18.342 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:42:18.337+0000 7f6e0bfff640 1 -- 172.21.15.89:0/174256819 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e0c00f6b0 con 0x7f6e14103fe0 2024-09-21T03:42:18.342 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:42:18.337+0000 7f6e18a86640 1 -- 172.21.15.89:0/174256819 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6e141a11c0 con 0x7f6e14103fe0 2024-09-21T03:42:18.349 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:42:18.339+0000 7f6e0bfff640 1 -- 172.21.15.89:0/174256819 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 21) ==== 154512+0+0 (secure 0 0 0) 0x7f6e0c00f910 con 0x7f6e14103fe0 2024-09-21T03:42:18.349 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:42:18.340+0000 7f6e1f9ed740 1 Processor -- start 2024-09-21T03:42:18.349 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:42:18.340+0000 7f6e1f9ed740 1 -- start start 2024-09-21T03:42:18.349 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:42:18.340+0000 7f6e1f9ed740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5651be9331b0 0x5651becdae50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:18.349 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:42:18.340+0000 7f6e1f9ed740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5651be90a2f0 0x5651becdb4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:18.349 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:42:18.340+0000 7f6e1f9ed740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5651be9124d0 0x5651becdc010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:18.349 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:42:18.340+0000 7f6e1f9ed740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x5651bec4d6f0 con 0x5651be90a2f0 2024-09-21T03:42:18.350 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:42:18.340+0000 7f6e1f9ed740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x5651bec3cb20 con 0x5651be9331b0 2024-09-21T03:42:18.350 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:42:18.340+0000 7f6e1f9ed740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x5651bec40930 con 0x5651be9124d0 2024-09-21T03:42:18.350 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:42:18.340+0000 7f6e1259c640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5651be90a2f0 0x5651becdb4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:18.350 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:42:18.340+0000 7f6e1259c640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5651be90a2f0 0x5651becdb4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:49906/0 (socket says 172.21.15.89:49906) 2024-09-21T03:42:18.350 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:42:18.340+0000 7f6e1259c640 1 -- 172.21.15.89:0/2049989704 learned_addr learned my addr 172.21.15.89:0/2049989704 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-21T03:42:18.350 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:42:18.340+0000 7f6e09d9b640 1 --2- 172.21.15.89:0/2049989704 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5651be9124d0 0x5651becdc010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:18.350 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:42:18.340+0000 7f6e09d9b640 1 --2- 172.21.15.89:0/2049989704 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5651be9331b0 0x5651becdae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:18.350 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:42:18.341+0000 7f6e0bfff640 1 --2- 172.21.15.89:0/174256819 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f6e000b6510 0x7f6e000b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:18.350 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:42:18.341+0000 7f6e1259c640 1 -- 172.21.15.89:0/2049989704 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5651be9124d0 msgr2=0x5651becdc010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:42:18.350 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:42:18.341+0000 7f6e1259c640 1 --2- 172.21.15.89:0/2049989704 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5651be9124d0 0x5651becdc010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:18.350 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:42:18.341+0000 7f6e1259c640 1 -- 172.21.15.89:0/2049989704 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5651be9331b0 msgr2=0x5651becdae50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:42:18.350 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:42:18.341+0000 7f6e1259c640 1 --2- 172.21.15.89:0/2049989704 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5651be9331b0 0x5651becdae50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:18.350 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:42:18.341+0000 7f6e1259c640 1 -- 172.21.15.89:0/2049989704 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5651bec02530 con 0x5651be90a2f0 2024-09-21T03:42:18.350 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:42:18.341+0000 7f6e12d9d640 1 --2- 172.21.15.89:0/174256819 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f6e000b6510 0x7f6e000b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:18.351 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:42:18.341+0000 7f6e1259c640 1 --2- 172.21.15.89:0/2049989704 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5651be90a2f0 0x5651becdb4c0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f6e0402a930 tx=0x7f6e04005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41b39f4e68724089 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:18.351 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:42:18.341+0000 7f6e0bfff640 1 -- 172.21.15.89:0/174256819 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 9121+0+0 (secure 0 0 0) 0x7f6e0c0e7a70 con 0x7f6e14103fe0 2024-09-21T03:42:18.351 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:42:18.341+0000 7f6e08d99640 1 -- 172.21.15.89:0/2049989704 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e04035030 con 0x5651be90a2f0 2024-09-21T03:42:18.351 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:42:18.341+0000 7f6e08d99640 1 -- 172.21.15.89:0/2049989704 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6e04040070 con 0x5651be90a2f0 2024-09-21T03:42:18.351 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:42:18.341+0000 7f6e08d99640 1 -- 172.21.15.89:0/2049989704 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e04003970 con 0x5651be90a2f0 2024-09-21T03:42:18.351 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:42:18.342+0000 7f6e1f9ed740 1 -- 172.21.15.89:0/2049989704 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5651be90a2f0 msgr2=0x5651becdb4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:18.351 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:42:18.342+0000 7f6e12d9d640 1 --2- 172.21.15.89:0/174256819 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f6e000b6510 0x7f6e000b8a20 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f6e0c0ad580 tx=0x7f6e0c0de320 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:18.351 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:42:18.342+0000 7f6e1f9ed740 1 --2- 172.21.15.89:0/2049989704 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5651be90a2f0 0x5651becdb4c0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f6e0402a930 tx=0x7f6e04005fd0 comp rx=0 tx=0).stop 2024-09-21T03:42:18.351 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:42:18.342+0000 7f6e1f9ed740 1 -- 172.21.15.89:0/2049989704 shutdown_connections 2024-09-21T03:42:18.351 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:42:18.342+0000 7f6e1f9ed740 1 --2- 172.21.15.89:0/2049989704 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5651be9124d0 0x5651becdc010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:18.351 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:42:18.342+0000 7f6e1f9ed740 1 --2- 172.21.15.89:0/2049989704 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5651be90a2f0 0x5651becdb4c0 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:18.351 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:42:18.342+0000 7f6e1f9ed740 1 --2- 172.21.15.89:0/2049989704 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5651be9331b0 0x5651becdae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:18.351 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:42:18.342+0000 7f6e1f9ed740 1 -- 172.21.15.89:0/2049989704 >> 172.21.15.89:0/2049989704 conn(0x5651be927300 msgr2=0x5651becd68e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:42:18.351 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:42:18.342+0000 7f6e1f9ed740 1 -- 172.21.15.89:0/2049989704 shutdown_connections 2024-09-21T03:42:18.351 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:42:18.342+0000 7f6e1f9ed740 1 -- 172.21.15.89:0/2049989704 wait complete. 2024-09-21T03:42:18.352 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:42:18.342+0000 7f6e1f9ed740 1 -- start start 2024-09-21T03:42:18.352 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:42:18.343+0000 7f6e1f9ed740 1 Processor -- start 2024-09-21T03:42:18.352 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:42:18.343+0000 7f6e1f9ed740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5651be90a2f0 0x5651bece45a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:18.352 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:42:18.343+0000 7f6e1f9ed740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5651be9124d0 0x5651bece4c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:18.352 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:42:18.343+0000 7f6e1f9ed740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5651be9331b0 0x5651bece57d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:18.352 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:42:18.343+0000 7f6e1f9ed740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x5651beb74ab0 con 0x5651be90a2f0 2024-09-21T03:42:18.352 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:42:18.343+0000 7f6e1f9ed740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x5651bebe2730 con 0x5651be9124d0 2024-09-21T03:42:18.352 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:42:18.343+0000 7f6e1f9ed740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x5651becdcad0 con 0x5651be9331b0 2024-09-21T03:42:18.352 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:42:18.343+0000 7f6e09d9b640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5651be90a2f0 0x5651bece45a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:18.352 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:42:18.343+0000 7f6e1f9ed740 10 client.0 _ll_register_callbacks cb 0x5651bec366b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:42:18.352 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:42:18.343+0000 7f6e09d9b640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5651be90a2f0 0x5651bece45a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:49916/0 (socket says 172.21.15.89:49916) 2024-09-21T03:42:18.352 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:42:18.343+0000 7f6e09d9b640 1 -- 172.21.15.89:0/2419928634 learned_addr learned my addr 172.21.15.89:0/2419928634 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-21T03:42:18.352 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:42:18.343+0000 7f6e1259c640 1 --2- 172.21.15.89:0/2419928634 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5651be9124d0 0x5651bece4c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:18.352 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:42:18.343+0000 7f6e09d9b640 1 --2- 172.21.15.89:0/2419928634 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5651be9331b0 0x5651bece57d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:18.353 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:42:18.343+0000 7f6e09d9b640 1 -- 172.21.15.89:0/2419928634 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5651be9331b0 msgr2=0x5651bece57d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:42:18.353 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:42:18.343+0000 7f6e09d9b640 1 --2- 172.21.15.89:0/2419928634 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5651be9331b0 0x5651bece57d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:18.353 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:42:18.343+0000 7f6e09d9b640 1 -- 172.21.15.89:0/2419928634 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5651be9124d0 msgr2=0x5651bece4c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:42:18.353 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:42:18.343+0000 7f6e09d9b640 1 --2- 172.21.15.89:0/2419928634 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5651be9124d0 0x5651bece4c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:18.353 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:42:18.343+0000 7f6e09d9b640 1 -- 172.21.15.89:0/2419928634 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5651be99e790 con 0x5651be90a2f0 2024-09-21T03:42:18.353 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:42:18.343+0000 7f6e1259c640 1 --2- 172.21.15.89:0/2419928634 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5651be9124d0 0x5651bece4c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:42:18.353 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:42:18.343+0000 7f6e09d9b640 1 --2- 172.21.15.89:0/2419928634 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5651be90a2f0 0x5651bece45a0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f6dfc0058c0 tx=0x7f6dfc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e070ed272a033aa1 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:18.353 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:42:18.343+0000 7f6df9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2024-09-21T03:42:18.353 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:42:18.343+0000 7f6df9ffb640 1 -- 172.21.15.89:0/2419928634 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dfc0132d0 con 0x5651be90a2f0 2024-09-21T03:42:18.353 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:42:18.344+0000 7f6df9ffb640 1 -- 172.21.15.89:0/2419928634 <== mon.0 v2:172.21.15.89:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 9121+0+0 (secure 0 0 0) 0x7f6dfc005020 con 0x5651be90a2f0 2024-09-21T03:42:18.353 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:42:18.344+0000 7f6e1f9ed740 1 -- 172.21.15.89:0/2419928634 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6e04040070 con 0x5651be90a2f0 2024-09-21T03:42:18.353 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:42:18.344+0000 7f6df9ffb640 1 -- 172.21.15.89:0/2419928634 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f6dfc028050 con 0x5651be90a2f0 2024-09-21T03:42:18.353 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:42:18.344+0000 7f6e1f9ed740 10 client.15909 fetch_fsmap learned FSMap version 26 2024-09-21T03:42:18.353 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:42:18.344+0000 7f6e1f9ed740 1 -- 172.21.15.89:0/2419928634 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5651bea17620 con 0x5651be90a2f0 2024-09-21T03:42:18.354 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:42:18.344+0000 7f6df9ffb640 1 -- 172.21.15.89:0/2419928634 <== mon.0 v2:172.21.15.89:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6e04040070 con 0x5651be90a2f0 2024-09-21T03:42:18.354 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:42:18.344+0000 7f6df9ffb640 1 -- 172.21.15.89:0/2419928634 <== mon.0 v2:172.21.15.89:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f6dfc01e030 con 0x5651be90a2f0 2024-09-21T03:42:18.354 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:42:18.344+0000 7f6e1f9ed740 10 client.15909 fetch_fsmap finished waiting for FSMap version 26 2024-09-21T03:42:18.354 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:42:18.344+0000 7f6e1f9ed740 10 client.15909 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T03:42:18.354 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:42:18.344+0000 7f6e1f9ed740 20 client.15909 populate_metadata read hostname 'smithi089' 2024-09-21T03:42:18.354 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:42:18.344+0000 7f6e1f9ed740 1 --2- 172.21.15.89:0/2419928634 >> [v2:172.21.15.110:6838/132366709,v1:172.21.15.110:6839/132366709] conn(0x5651be8f1d80 0x5651becf1e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:18.354 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:42:18.344+0000 7f6e1f9ed740 4 client.15909 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:42:18.354 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:42:18.344+0000 7f6e1f9ed740 1 -- 172.21.15.89:0/2419928634 --> [v2:172.21.15.110:6838/132366709,v1:172.21.15.110:6839/132366709] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5651bec4f640 con 0x5651be8f1d80 2024-09-21T03:42:18.354 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:42:18.344+0000 7f6e1259c640 1 --2- 172.21.15.89:0/2419928634 >> [v2:172.21.15.110:6838/132366709,v1:172.21.15.110:6839/132366709] conn(0x5651be8f1d80 0x5651becf1e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:18.354 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:42:18.345+0000 7f6df9ffb640 10 client.15909 ms_handle_connect on v2:172.21.15.110:6838/132366709 2024-09-21T03:42:18.354 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:42:18.345+0000 7f6e1259c640 1 --2- 172.21.15.89:0/2419928634 >> [v2:172.21.15.110:6838/132366709,v1:172.21.15.110:6839/132366709] conn(0x5651be8f1d80 0x5651becf1e00 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3420c9d158c143d4 server_cookie=9e1aa732574599d1 in_seq=0 out_seq=0 2024-09-21T03:42:18.354 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:42:18.346+0000 7f6df9ffb640 1 -- 172.21.15.89:0/2419928634 <== mds.0 v2:172.21.15.110:6838/132366709 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5651bec4f640 con 0x5651be8f1d80 2024-09-21T03:42:18.354 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:42:18.346+0000 7f6df9ffb640 10 client.15909 handle_command_reply: tid=0 2024-09-21T03:42:18.355 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:42:18.346+0000 7f6e1f9ed740 1 client.15909 shutdown 2024-09-21T03:42:18.355 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:42:18.346+0000 7f6e1f9ed740 1 -- 172.21.15.89:0/2419928634 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5651be90a2f0 msgr2=0x5651bece45a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:18.355 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:42:18.346+0000 7f6e1f9ed740 1 --2- 172.21.15.89:0/2419928634 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5651be90a2f0 0x5651bece45a0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f6dfc0058c0 tx=0x7f6dfc0046c0 comp rx=0 tx=0).stop 2024-09-21T03:42:18.355 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:42:18.346+0000 7f6e1f9ed740 1 -- 172.21.15.89:0/2419928634 shutdown_connections 2024-09-21T03:42:18.355 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:42:18.346+0000 7f6e1f9ed740 1 --2- 172.21.15.89:0/2419928634 >> [v2:172.21.15.110:6838/132366709,v1:172.21.15.110:6839/132366709] conn(0x5651be8f1d80 0x5651becf1e00 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:18.355 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:42:18.346+0000 7f6e1f9ed740 1 --2- 172.21.15.89:0/2419928634 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5651be9124d0 0x5651bece4c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:18.355 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:42:18.346+0000 7f6df9ffb640 0 client.15909 ms_handle_reset on v2:172.21.15.110:6838/132366709 2024-09-21T03:42:18.355 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:42:18.346+0000 7f6e1f9ed740 1 --2- 172.21.15.89:0/2419928634 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5651be9331b0 0x5651bece57d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:18.355 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:42:18.346+0000 7f6e1f9ed740 1 --2- 172.21.15.89:0/2419928634 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5651be90a2f0 0x5651bece45a0 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:18.355 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:42:18.346+0000 7f6e1f9ed740 1 -- 172.21.15.89:0/2419928634 >> 172.21.15.89:0/2419928634 conn(0x5651be927300 msgr2=0x5651becdbec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:42:18.355 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:42:18.346+0000 7f6e1f9ed740 1 -- 172.21.15.89:0/2419928634 shutdown_connections 2024-09-21T03:42:18.355 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:42:18.346+0000 7f6e1f9ed740 1 -- 172.21.15.89:0/2419928634 wait complete. 2024-09-21T03:42:18.355 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:42:18.346+0000 7f6e1f9ed740 20 client.15909 trim_cache size 0 max 16384 2024-09-21T03:42:18.364 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:42:18.360+0000 7f6e1f9ed740 1 Processor -- start 2024-09-21T03:42:18.364 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:42:18.360+0000 7f6e1f9ed740 1 -- start start 2024-09-21T03:42:18.364 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:42:18.361+0000 7f6e1f9ed740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5651be8f1d80 0x5651bece3200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:18.364 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:42:18.361+0000 7f6e1f9ed740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5651be90a2f0 0x5651bece37c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:18.364 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:42:18.361+0000 7f6e1f9ed740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5651be9124d0 0x5651becdadf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:18.364 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:42:18.361+0000 7f6e1f9ed740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f6e141add20 con 0x5651be90a2f0 2024-09-21T03:42:18.364 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:42:18.361+0000 7f6e1f9ed740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f6e141ab5f0 con 0x5651be8f1d80 2024-09-21T03:42:18.364 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:42:18.361+0000 7f6e1f9ed740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f6e141a9440 con 0x5651be9124d0 2024-09-21T03:42:18.364 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:42:18.361+0000 7f6e1259c640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5651be90a2f0 0x5651bece37c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:18.364 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:42:18.361+0000 7f6e1259c640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5651be90a2f0 0x5651bece37c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:49930/0 (socket says 172.21.15.89:49930) 2024-09-21T03:42:18.364 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:42:18.361+0000 7f6e1259c640 1 -- 172.21.15.89:0/4114021773 learned_addr learned my addr 172.21.15.89:0/4114021773 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-21T03:42:18.365 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:42:18.361+0000 7f6e09d9b640 1 --2- 172.21.15.89:0/4114021773 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5651be8f1d80 0x5651bece3200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:18.365 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:42:18.361+0000 7f6e1259c640 1 -- 172.21.15.89:0/4114021773 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5651be9124d0 msgr2=0x5651becdadf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:42:18.365 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:42:18.361+0000 7f6e1259c640 1 --2- 172.21.15.89:0/4114021773 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5651be9124d0 0x5651becdadf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:18.365 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:42:18.361+0000 7f6e1259c640 1 -- 172.21.15.89:0/4114021773 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5651be8f1d80 msgr2=0x5651bece3200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:42:18.365 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:42:18.361+0000 7f6e1259c640 1 --2- 172.21.15.89:0/4114021773 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5651be8f1d80 0x5651bece3200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:18.365 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:42:18.361+0000 7f6e1259c640 1 -- 172.21.15.89:0/4114021773 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5651becefae0 con 0x5651be90a2f0 2024-09-21T03:42:18.365 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:42:18.361+0000 7f6e09d9b640 1 --2- 172.21.15.89:0/4114021773 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5651be8f1d80 0x5651bece3200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:42:18.365 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:42:18.361+0000 7f6e1259c640 1 --2- 172.21.15.89:0/4114021773 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5651be90a2f0 0x5651bece37c0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f6e040057c0 tx=0x7f6e0403e040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71d3e439550b2220 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:18.365 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:42:18.361+0000 7f6dfbfff640 1 -- 172.21.15.89:0/4114021773 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e04035040 con 0x5651be90a2f0 2024-09-21T03:42:18.365 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:42:18.361+0000 7f6dfbfff640 1 -- 172.21.15.89:0/4114021773 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6e04040040 con 0x5651be90a2f0 2024-09-21T03:42:18.365 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:42:18.361+0000 7f6dfbfff640 1 -- 172.21.15.89:0/4114021773 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e0403cda0 con 0x5651be90a2f0 2024-09-21T03:42:18.369 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:42:18.362+0000 7f6e1f9ed740 1 -- 172.21.15.89:0/4114021773 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5651be90a2f0 msgr2=0x5651bece37c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:18.369 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:42:18.362+0000 7f6e1f9ed740 1 --2- 172.21.15.89:0/4114021773 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5651be90a2f0 0x5651bece37c0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f6e040057c0 tx=0x7f6e0403e040 comp rx=0 tx=0).stop 2024-09-21T03:42:18.369 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:42:18.362+0000 7f6e1f9ed740 1 -- 172.21.15.89:0/4114021773 shutdown_connections 2024-09-21T03:42:18.369 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:42:18.362+0000 7f6e1f9ed740 1 --2- 172.21.15.89:0/4114021773 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5651be9124d0 0x5651becdadf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:18.369 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:42:18.362+0000 7f6e1f9ed740 1 --2- 172.21.15.89:0/4114021773 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5651be90a2f0 0x5651bece37c0 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:18.369 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:42:18.362+0000 7f6e1f9ed740 1 --2- 172.21.15.89:0/4114021773 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5651be8f1d80 0x5651bece3200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:18.369 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:42:18.362+0000 7f6e1f9ed740 1 -- 172.21.15.89:0/4114021773 >> 172.21.15.89:0/4114021773 conn(0x5651be927300 msgr2=0x5651bece2af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:42:18.369 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:42:18.362+0000 7f6e1f9ed740 1 -- 172.21.15.89:0/4114021773 shutdown_connections 2024-09-21T03:42:18.369 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:42:18.362+0000 7f6e1f9ed740 1 -- 172.21.15.89:0/4114021773 wait complete. 2024-09-21T03:42:18.369 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:42:18.362+0000 7f6e1f9ed740 1 -- start start 2024-09-21T03:42:18.370 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:42:18.362+0000 7f6e1f9ed740 1 Processor -- start 2024-09-21T03:42:18.370 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:42:18.363+0000 7f6e1f9ed740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5651be8f1d80 0x5651bece9490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:18.370 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:42:18.363+0000 7f6e1f9ed740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5651be90a2f0 0x5651bece9b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:18.370 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:42:18.363+0000 7f6e1f9ed740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5651be9124d0 0x5651becea6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:18.370 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:42:18.363+0000 7f6e1f9ed740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f6e141a9ab0 con 0x5651be8f1d80 2024-09-21T03:42:18.370 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:42:18.363+0000 7f6e1f9ed740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f6e141aaf20 con 0x5651be9124d0 2024-09-21T03:42:18.370 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:42:18.363+0000 7f6e1f9ed740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f6e141aa180 con 0x5651be90a2f0 2024-09-21T03:42:18.370 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:42:18.363+0000 7f6e1f9ed740 10 client.0 _ll_register_callbacks cb 0x7f6dfc013a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:42:18.370 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:42:18.363+0000 7f6e09d9b640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5651be8f1d80 0x5651bece9490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:18.370 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:42:18.363+0000 7f6e09d9b640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5651be8f1d80 0x5651bece9490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:49932/0 (socket says 172.21.15.89:49932) 2024-09-21T03:42:18.370 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:42:18.363+0000 7f6e09d9b640 1 -- 172.21.15.89:0/4134665569 learned_addr learned my addr 172.21.15.89:0/4134665569 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-21T03:42:18.370 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:42:18.363+0000 7f6e1259c640 1 --2- 172.21.15.89:0/4134665569 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5651be90a2f0 0x5651bece9b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:18.370 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:42:18.363+0000 7f6e09d9b640 1 --2- 172.21.15.89:0/4134665569 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5651be9124d0 0x5651becea6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:18.370 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:42:18.363+0000 7f6e09d9b640 1 -- 172.21.15.89:0/4134665569 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5651be90a2f0 msgr2=0x5651bece9b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:42:18.371 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:42:18.363+0000 7f6e09d9b640 1 --2- 172.21.15.89:0/4134665569 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5651be90a2f0 0x5651bece9b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:18.371 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:42:18.363+0000 7f6e09d9b640 1 -- 172.21.15.89:0/4134665569 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5651be9124d0 msgr2=0x5651becea6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:42:18.371 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:42:18.363+0000 7f6e09d9b640 1 --2- 172.21.15.89:0/4134665569 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5651be9124d0 0x5651becea6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:18.371 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:42:18.363+0000 7f6e09d9b640 1 -- 172.21.15.89:0/4134665569 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5651beceae20 con 0x5651be8f1d80 2024-09-21T03:42:18.371 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:42:18.363+0000 7f6e09d9b640 1 --2- 172.21.15.89:0/4134665569 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5651be8f1d80 0x5651bece9490 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f6dfc009fd0 tx=0x7f6dfc001e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a060e88b73050ae0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:18.371 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:42:18.363+0000 7f6df9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2024-09-21T03:42:18.371 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:42:18.363+0000 7f6df9ffb640 1 -- 172.21.15.89:0/4134665569 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dfc025040 con 0x5651be8f1d80 2024-09-21T03:42:18.371 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:42:18.363+0000 7f6df9ffb640 1 -- 172.21.15.89:0/4134665569 <== mon.0 v2:172.21.15.89:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 9121+0+0 (secure 0 0 0) 0x7f6dfc004930 con 0x5651be8f1d80 2024-09-21T03:42:18.371 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:42:18.364+0000 7f6e1f9ed740 1 -- 172.21.15.89:0/4134665569 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6e04040040 con 0x5651be8f1d80 2024-09-21T03:42:18.371 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:42:18.364+0000 7f6df9ffb640 1 -- 172.21.15.89:0/4134665569 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f6dfc03c050 con 0x5651be8f1d80 2024-09-21T03:42:18.371 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:42:18.364+0000 7f6e1f9ed740 10 client.15915 fetch_fsmap learned FSMap version 26 2024-09-21T03:42:18.371 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:42:18.364+0000 7f6e1f9ed740 1 -- 172.21.15.89:0/4134665569 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5651becec210 con 0x5651be8f1d80 2024-09-21T03:42:18.372 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:42:18.364+0000 7f6df9ffb640 1 -- 172.21.15.89:0/4134665569 <== mon.0 v2:172.21.15.89:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6e04040040 con 0x5651be8f1d80 2024-09-21T03:42:18.372 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:42:18.364+0000 7f6df9ffb640 1 -- 172.21.15.89:0/4134665569 <== mon.0 v2:172.21.15.89:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f6dfc01e040 con 0x5651be8f1d80 2024-09-21T03:42:18.372 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:42:18.364+0000 7f6e1f9ed740 10 client.15915 fetch_fsmap finished waiting for FSMap version 26 2024-09-21T03:42:18.372 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:42:18.364+0000 7f6e1f9ed740 10 client.15915 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T03:42:18.372 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:42:18.364+0000 7f6e1f9ed740 20 client.15915 populate_metadata read hostname 'smithi089' 2024-09-21T03:42:18.372 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:42:18.364+0000 7f6e1f9ed740 1 --2- 172.21.15.89:0/4134665569 >> [v2:172.21.15.110:6838/132366709,v1:172.21.15.110:6839/132366709] conn(0x5651be9331b0 0x5651bed5ec70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:18.372 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:42:18.364+0000 7f6e1f9ed740 4 client.15915 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T03:42:18.372 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:42:18.364+0000 7f6e1f9ed740 1 -- 172.21.15.89:0/4134665569 --> [v2:172.21.15.110:6838/132366709,v1:172.21.15.110:6839/132366709] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5651bec4b0f0 con 0x5651be9331b0 2024-09-21T03:42:18.372 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:42:18.364+0000 7f6e1259c640 1 --2- 172.21.15.89:0/4134665569 >> [v2:172.21.15.110:6838/132366709,v1:172.21.15.110:6839/132366709] conn(0x5651be9331b0 0x5651bed5ec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:18.372 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:42:18.365+0000 7f6e1259c640 1 --2- 172.21.15.89:0/4134665569 >> [v2:172.21.15.110:6838/132366709,v1:172.21.15.110:6839/132366709] conn(0x5651be9331b0 0x5651bed5ec70 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=16db08aeb0c268af server_cookie=ac00e1c52bcbcf91 in_seq=0 out_seq=0 2024-09-21T03:42:18.372 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:42:18.365+0000 7f6df9ffb640 10 client.15915 ms_handle_connect on v2:172.21.15.110:6838/132366709 2024-09-21T03:42:18.379 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:42:18.376+0000 7f6df9ffb640 1 -- 172.21.15.89:0/4134665569 <== mds.0 v2:172.21.15.110:6838/132366709 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5651bec4b0f0 con 0x5651be9331b0 2024-09-21T03:42:18.380 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:42:18.377+0000 7f6df9ffb640 10 client.15915 handle_command_reply: tid=0 2024-09-21T03:42:18.380 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:42:18.377+0000 7f6e1f9ed740 1 client.15915 shutdown 2024-09-21T03:42:18.380 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:42:18.377+0000 7f6e1f9ed740 1 -- 172.21.15.89:0/4134665569 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5651be8f1d80 msgr2=0x5651bece9490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:18.380 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:42:18.377+0000 7f6e1f9ed740 1 --2- 172.21.15.89:0/4134665569 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5651be8f1d80 0x5651bece9490 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f6dfc009fd0 tx=0x7f6dfc001e50 comp rx=0 tx=0).stop 2024-09-21T03:42:18.380 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:42:18.377+0000 7f6e1f9ed740 1 -- 172.21.15.89:0/4134665569 shutdown_connections 2024-09-21T03:42:18.381 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:42:18.377+0000 7f6e1f9ed740 1 --2- 172.21.15.89:0/4134665569 >> [v2:172.21.15.110:6838/132366709,v1:172.21.15.110:6839/132366709] conn(0x5651be9331b0 0x5651bed5ec70 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:18.382 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:42:18.377+0000 7f6e1f9ed740 1 --2- 172.21.15.89:0/4134665569 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5651be9124d0 0x5651becea6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:18.382 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:42:18.377+0000 7f6e1f9ed740 1 --2- 172.21.15.89:0/4134665569 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5651be90a2f0 0x5651bece9b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:18.382 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:42:18.377+0000 7f6e1f9ed740 1 --2- 172.21.15.89:0/4134665569 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5651be8f1d80 0x5651bece9490 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:18.382 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:42:18.377+0000 7f6df9ffb640 0 client.15915 ms_handle_reset on v2:172.21.15.110:6838/132366709 2024-09-21T03:42:18.382 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:42:18.377+0000 7f6e1f9ed740 1 -- 172.21.15.89:0/4134665569 >> 172.21.15.89:0/4134665569 conn(0x5651be927300 msgr2=0x5651bece2980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:42:18.382 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:42:18.377+0000 7f6e1f9ed740 1 -- 172.21.15.89:0/4134665569 shutdown_connections 2024-09-21T03:42:18.382 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:42:18.377+0000 7f6e1f9ed740 1 -- 172.21.15.89:0/4134665569 wait complete. 2024-09-21T03:42:18.382 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:42:18.377+0000 7f6e1f9ed740 20 client.15915 trim_cache size 0 max 16384 2024-09-21T03:42:18.382 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stdout:{ 2024-09-21T03:42:18.382 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stdout: "message": "", 2024-09-21T03:42:18.383 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stdout: "return_code": 0 2024-09-21T03:42:18.383 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stdout:} 2024-09-21T03:42:18.383 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:42:18.377+0000 7f6e18a86640 1 -- 172.21.15.89:0/174256819 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f6e000b6510 msgr2=0x7f6e000b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:18.383 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:42:18.377+0000 7f6e18a86640 1 --2- 172.21.15.89:0/174256819 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f6e000b6510 0x7f6e000b8a20 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f6e0c0ad580 tx=0x7f6e0c0de320 comp rx=0 tx=0).stop 2024-09-21T03:42:18.384 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:42:18.377+0000 7f6e18a86640 1 -- 172.21.15.89:0/174256819 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6e14103fe0 msgr2=0x7f6e14199fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:18.384 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:42:18.377+0000 7f6e18a86640 1 --2- 172.21.15.89:0/174256819 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6e14103fe0 0x7f6e14199fc0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f6e0c00cad0 tx=0x7f6e0c00cf90 comp rx=0 tx=0).stop 2024-09-21T03:42:18.384 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:42:18.378+0000 7f6e18a86640 1 -- 172.21.15.89:0/174256819 shutdown_connections 2024-09-21T03:42:18.384 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:42:18.378+0000 7f6e18a86640 1 --2- 172.21.15.89:0/174256819 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f6e000b6510 0x7f6e000b8a20 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:18.384 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:42:18.378+0000 7f6e18a86640 1 --2- 172.21.15.89:0/174256819 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6e14103fe0 0x7f6e14199fc0 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:18.384 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:42:18.378+0000 7f6e18a86640 1 --2- 172.21.15.89:0/174256819 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6e14103640 0x7f6e14199410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:18.384 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:42:18.378+0000 7f6e18a86640 1 --2- 172.21.15.89:0/174256819 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6e141025c0 0x7f6e14198da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:18.384 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:42:18.378+0000 7f6e18a86640 1 -- 172.21.15.89:0/174256819 >> 172.21.15.89:0/174256819 conn(0x7f6e140fdf00 msgr2=0x7f6e14106960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:42:18.384 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:42:18.378+0000 7f6e18a86640 1 -- 172.21.15.89:0/174256819 shutdown_connections 2024-09-21T03:42:18.384 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-21T03:42:18.378+0000 7f6e18a86640 1 -- 172.21.15.89:0/174256819 wait complete. 2024-09-21T03:42:18.685 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:18.682+0000 7f193c519640 1 -- 172.21.15.89:0/1953385009 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f19341084d0 msgr2=0x7f19341088e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:18.685 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:18.682+0000 7f193c519640 1 --2- 172.21.15.89:0/1953385009 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f19341084d0 0x7f19341088e0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f1928009900 tx=0x7f192802a410 comp rx=0 tx=0).stop 2024-09-21T03:42:18.685 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:18.682+0000 7f193c519640 1 -- 172.21.15.89:0/1953385009 shutdown_connections 2024-09-21T03:42:18.685 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:18.682+0000 7f193c519640 1 --2- 172.21.15.89:0/1953385009 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1934108e20 0x7f193410f650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:18.685 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:18.682+0000 7f193c519640 1 --2- 172.21.15.89:0/1953385009 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f19341084d0 0x7f19341088e0 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:18.685 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:18.682+0000 7f193c519640 1 --2- 172.21.15.89:0/1953385009 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1934102a80 0x7f1934102e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:18.685 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:18.682+0000 7f193c519640 1 -- 172.21.15.89:0/1953385009 >> 172.21.15.89:0/1953385009 conn(0x7f1934079720 msgr2=0x7f1934079b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:42:18.685 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:18.682+0000 7f193c519640 1 -- 172.21.15.89:0/1953385009 shutdown_connections 2024-09-21T03:42:18.685 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:18.682+0000 7f193c519640 1 -- 172.21.15.89:0/1953385009 wait complete. 2024-09-21T03:42:18.687 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:18.683+0000 7f193c519640 1 Processor -- start 2024-09-21T03:42:18.687 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:18.683+0000 7f193c519640 1 -- start start 2024-09-21T03:42:18.687 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:18.683+0000 7f193c519640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1934102a80 0x7f193419a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:18.687 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:18.683+0000 7f193c519640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f19341084d0 0x7f193419aa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:18.687 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:18.683+0000 7f193c519640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1934108e20 0x7f193419fd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:18.687 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:18.683+0000 7f193c519640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f1934112990 con 0x7f1934102a80 2024-09-21T03:42:18.687 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:18.683+0000 7f193c519640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f1934112810 con 0x7f19341084d0 2024-09-21T03:42:18.687 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:18.683+0000 7f193c519640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f1934112b10 con 0x7f1934108e20 2024-09-21T03:42:18.687 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:18.683+0000 7f193a2b5640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1934102a80 0x7f193419a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:18.687 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:18.683+0000 7f193a2b5640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1934102a80 0x7f193419a4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:53128/0 (socket says 172.21.15.89:53128) 2024-09-21T03:42:18.687 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:18.683+0000 7f193a2b5640 1 -- 172.21.15.89:0/3880837163 learned_addr learned my addr 172.21.15.89:0/3880837163 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-21T03:42:18.687 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:18.683+0000 7f1939ab4640 1 --2- 172.21.15.89:0/3880837163 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f19341084d0 0x7f193419aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:18.688 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:18.683+0000 7f193aab6640 1 --2- 172.21.15.89:0/3880837163 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1934108e20 0x7f193419fd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:18.688 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:18.684+0000 7f193a2b5640 1 -- 172.21.15.89:0/3880837163 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1934108e20 msgr2=0x7f193419fd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:18.688 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:18.684+0000 7f193a2b5640 1 --2- 172.21.15.89:0/3880837163 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1934108e20 0x7f193419fd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:18.688 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:18.684+0000 7f193a2b5640 1 -- 172.21.15.89:0/3880837163 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f19341084d0 msgr2=0x7f193419aa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:18.688 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:18.684+0000 7f193a2b5640 1 --2- 172.21.15.89:0/3880837163 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f19341084d0 0x7f193419aa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:18.688 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:18.684+0000 7f193a2b5640 1 -- 172.21.15.89:0/3880837163 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19341a0440 con 0x7f1934102a80 2024-09-21T03:42:18.688 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:18.684+0000 7f193a2b5640 1 --2- 172.21.15.89:0/3880837163 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1934102a80 0x7f193419a4d0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f192400d770 tx=0x7f192400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:18.688 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:18.684+0000 7f19337fe640 1 -- 172.21.15.89:0/3880837163 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1924017070 con 0x7f1934102a80 2024-09-21T03:42:18.688 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:18.684+0000 7f193c519640 1 -- 172.21.15.89:0/3880837163 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f19341a0730 con 0x7f1934102a80 2024-09-21T03:42:18.688 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:18.684+0000 7f19337fe640 1 -- 172.21.15.89:0/3880837163 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f19240054b0 con 0x7f1934102a80 2024-09-21T03:42:18.688 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:18.684+0000 7f19337fe640 1 -- 172.21.15.89:0/3880837163 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1924004c60 con 0x7f1934102a80 2024-09-21T03:42:18.688 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:18.684+0000 7f193c519640 1 -- 172.21.15.89:0/3880837163 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19341a6e90 con 0x7f1934102a80 2024-09-21T03:42:18.689 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:18.687+0000 7f1942a8d740 1 Processor -- start 2024-09-21T03:42:18.691 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:18.687+0000 7f1942a8d740 1 -- start start 2024-09-21T03:42:18.691 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:18.687+0000 7f19337fe640 1 -- 172.21.15.89:0/3880837163 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 21) ==== 154512+0+0 (secure 0 0 0) 0x7f1924004e00 con 0x7f1934102a80 2024-09-21T03:42:18.691 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:18.687+0000 7f1942a8d740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c3213eb840 0x55c321792600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:18.691 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:18.687+0000 7f1942a8d740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c3213c2980 0x55c321792b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:18.691 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:18.687+0000 7f1942a8d740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c3213cab60 0x55c321793080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:18.691 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:18.687+0000 7f1942a8d740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55c321708b90 con 0x55c3213cab60 2024-09-21T03:42:18.691 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:18.687+0000 7f1942a8d740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55c3216f7fc0 con 0x55c3213c2980 2024-09-21T03:42:18.691 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:18.687+0000 7f1942a8d740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55c3216fbdd0 con 0x55c3213eb840 2024-09-21T03:42:18.691 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:18.688+0000 7f1939ab4640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c3213cab60 0x55c321793080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:18.691 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:18.688+0000 7f1939ab4640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c3213cab60 0x55c321793080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:53138/0 (socket says 172.21.15.89:53138) 2024-09-21T03:42:18.691 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:18.688+0000 7f1939ab4640 1 -- 172.21.15.89:0/3021285636 learned_addr learned my addr 172.21.15.89:0/3021285636 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-21T03:42:18.691 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:18.688+0000 7f193aab6640 1 --2- 172.21.15.89:0/3021285636 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c3213eb840 0x55c321792600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:18.692 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:18.688+0000 7f193a2b5640 1 --2- 172.21.15.89:0/3021285636 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c3213c2980 0x55c321792b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:18.692 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:18.688+0000 7f1939ab4640 1 -- 172.21.15.89:0/3021285636 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c3213eb840 msgr2=0x55c321792600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:42:18.692 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:18.688+0000 7f1939ab4640 1 --2- 172.21.15.89:0/3021285636 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c3213eb840 0x55c321792600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:18.692 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:18.688+0000 7f1939ab4640 1 -- 172.21.15.89:0/3021285636 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c3213c2980 msgr2=0x55c321792b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:42:18.692 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:18.688+0000 7f1939ab4640 1 --2- 172.21.15.89:0/3021285636 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c3213c2980 0x55c321792b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:18.692 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:18.688+0000 7f1939ab4640 1 -- 172.21.15.89:0/3021285636 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c32167f060 con 0x55c3213cab60 2024-09-21T03:42:18.692 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:18.688+0000 7f1939ab4640 1 --2- 172.21.15.89:0/3021285636 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c3213cab60 0x55c321793080 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f19280057c0 tx=0x7f1928031ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77de21125123ca0c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:18.692 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:18.688+0000 7f19337fe640 1 --2- 172.21.15.89:0/3880837163 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f191c0b6510 0x7f191c0b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:18.692 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:18.688+0000 7f1930ff9640 1 -- 172.21.15.89:0/3021285636 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1928040070 con 0x55c3213cab60 2024-09-21T03:42:18.692 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:18.688+0000 7f1930ff9640 1 -- 172.21.15.89:0/3021285636 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1928031e50 con 0x55c3213cab60 2024-09-21T03:42:18.692 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:18.688+0000 7f193aab6640 1 --2- 172.21.15.89:0/3880837163 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f191c0b6510 0x7f191c0b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:18.692 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:18.688+0000 7f19337fe640 1 -- 172.21.15.89:0/3880837163 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 9121+0+0 (secure 0 0 0) 0x7f19240e7040 con 0x7f1934102a80 2024-09-21T03:42:18.692 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:18.689+0000 7f1930ff9640 1 -- 172.21.15.89:0/3021285636 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1928035040 con 0x55c3213cab60 2024-09-21T03:42:18.695 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:18.689+0000 7f1942a8d740 1 -- 172.21.15.89:0/3021285636 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c3213cab60 msgr2=0x55c321793080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:18.695 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:18.689+0000 7f1942a8d740 1 --2- 172.21.15.89:0/3021285636 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c3213cab60 0x55c321793080 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f19280057c0 tx=0x7f1928031ba0 comp rx=0 tx=0).stop 2024-09-21T03:42:18.695 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:18.689+0000 7f193aab6640 1 --2- 172.21.15.89:0/3880837163 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f191c0b6510 0x7f191c0b8a20 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f192c005fd0 tx=0x7f192c009600 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:18.696 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:18.689+0000 7f1942a8d740 1 -- 172.21.15.89:0/3021285636 shutdown_connections 2024-09-21T03:42:18.696 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:18.689+0000 7f1942a8d740 1 --2- 172.21.15.89:0/3021285636 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c3213c2980 0x55c321792b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:18.696 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:18.689+0000 7f1942a8d740 1 --2- 172.21.15.89:0/3021285636 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c3213cab60 0x55c321793080 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:18.696 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:18.689+0000 7f1942a8d740 1 --2- 172.21.15.89:0/3021285636 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c3213eb840 0x55c321792600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:18.696 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:18.689+0000 7f1942a8d740 1 -- 172.21.15.89:0/3021285636 >> 172.21.15.89:0/3021285636 conn(0x55c3213df990 msgr2=0x55c3217904f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:42:18.696 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:18.690+0000 7f1942a8d740 1 -- 172.21.15.89:0/3021285636 shutdown_connections 2024-09-21T03:42:18.696 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:18.690+0000 7f1942a8d740 1 -- 172.21.15.89:0/3021285636 wait complete. 2024-09-21T03:42:18.696 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:18.690+0000 7f1942a8d740 1 -- start start 2024-09-21T03:42:18.696 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:18.690+0000 7f1942a8d740 1 Processor -- start 2024-09-21T03:42:18.696 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:18.690+0000 7f1942a8d740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c3213c2980 0x55c3217a24e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:18.696 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:18.690+0000 7f1942a8d740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c3213cab60 0x55c3217a2a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:18.696 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:18.690+0000 7f1942a8d740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c3213eb840 0x55c3217ae460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:18.696 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:18.690+0000 7f1942a8d740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55c3215597a0 con 0x55c3213eb840 2024-09-21T03:42:18.696 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:18.690+0000 7f1942a8d740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55c32169d0a0 con 0x55c3213c2980 2024-09-21T03:42:18.696 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:18.690+0000 7f1942a8d740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55c3217a5100 con 0x55c3213cab60 2024-09-21T03:42:18.697 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:18.690+0000 7f1942a8d740 10 client.0 _ll_register_callbacks cb 0x55c32167a3c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:42:18.697 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:18.690+0000 7f193aab6640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c3213eb840 0x55c3217ae460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:18.697 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:18.691+0000 7f193aab6640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c3213eb840 0x55c3217ae460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:53150/0 (socket says 172.21.15.89:53150) 2024-09-21T03:42:18.697 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:18.691+0000 7f193aab6640 1 -- 172.21.15.89:0/1835092156 learned_addr learned my addr 172.21.15.89:0/1835092156 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-21T03:42:18.697 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:18.691+0000 7f1939ab4640 1 --2- 172.21.15.89:0/1835092156 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c3213cab60 0x55c3217a2a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:18.697 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:18.691+0000 7f193a2b5640 1 --2- 172.21.15.89:0/1835092156 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c3213c2980 0x55c3217a24e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:18.697 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:18.691+0000 7f193aab6640 1 -- 172.21.15.89:0/1835092156 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c3213cab60 msgr2=0x55c3217a2a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:42:18.697 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:18.691+0000 7f193aab6640 1 --2- 172.21.15.89:0/1835092156 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c3213cab60 0x55c3217a2a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:18.697 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:18.691+0000 7f193aab6640 1 -- 172.21.15.89:0/1835092156 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c3213c2980 msgr2=0x55c3217a24e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:42:18.697 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:18.691+0000 7f193aab6640 1 --2- 172.21.15.89:0/1835092156 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c3213c2980 0x55c3217a24e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:18.697 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:18.691+0000 7f193aab6640 1 -- 172.21.15.89:0/1835092156 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c32168b880 con 0x55c3213eb840 2024-09-21T03:42:18.697 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:18.691+0000 7f193aab6640 1 --2- 172.21.15.89:0/1835092156 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c3213eb840 0x55c3217ae460 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f192c005c40 tx=0x7f192c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ffcc52c4ff6e9477 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:18.697 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:18.691+0000 7f1919ffb640 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2024-09-21T03:42:18.697 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:18.691+0000 7f1919ffb640 1 -- 172.21.15.89:0/1835092156 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f192c020620 con 0x55c3213eb840 2024-09-21T03:42:18.698 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:18.691+0000 7f1919ffb640 1 -- 172.21.15.89:0/1835092156 <== mon.0 v2:172.21.15.89:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 9121+0+0 (secure 0 0 0) 0x7f192c0207c0 con 0x55c3213eb840 2024-09-21T03:42:18.698 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:18.691+0000 7f1942a8d740 1 -- 172.21.15.89:0/1835092156 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1928031e50 con 0x55c3213eb840 2024-09-21T03:42:18.698 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:18.692+0000 7f1919ffb640 1 -- 172.21.15.89:0/1835092156 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f192c033050 con 0x55c3213eb840 2024-09-21T03:42:18.698 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:18.692+0000 7f1942a8d740 10 client.15927 fetch_fsmap learned FSMap version 26 2024-09-21T03:42:18.698 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:18.692+0000 7f1942a8d740 1 -- 172.21.15.89:0/1835092156 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55c321465b90 con 0x55c3213eb840 2024-09-21T03:42:18.698 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:18.692+0000 7f1919ffb640 1 -- 172.21.15.89:0/1835092156 <== mon.0 v2:172.21.15.89:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1928031e50 con 0x55c3213eb840 2024-09-21T03:42:18.698 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:18.692+0000 7f1919ffb640 1 -- 172.21.15.89:0/1835092156 <== mon.0 v2:172.21.15.89:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f192c00f040 con 0x55c3213eb840 2024-09-21T03:42:18.698 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:18.692+0000 7f1942a8d740 10 client.15927 fetch_fsmap finished waiting for FSMap version 26 2024-09-21T03:42:18.698 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:18.692+0000 7f1942a8d740 10 client.15927 resolve_mds: validated gid 24638 aka daemon mds.b 2024-09-21T03:42:18.698 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:18.692+0000 7f1942a8d740 20 client.15927 populate_metadata read hostname 'smithi089' 2024-09-21T03:42:18.698 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:18.692+0000 7f1942a8d740 1 --2- 172.21.15.89:0/1835092156 >> [v2:172.21.15.110:6838/132366709,v1:172.21.15.110:6839/132366709] conn(0x55c3213aa1c0 0x55c3217b44e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:18.698 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:18.692+0000 7f1942a8d740 4 client.15927 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:42:18.698 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:18.692+0000 7f1942a8d740 1 -- 172.21.15.89:0/1835092156 --> [v2:172.21.15.110:6838/132366709,v1:172.21.15.110:6839/132366709] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c32170aae0 con 0x55c3213aa1c0 2024-09-21T03:42:18.698 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:18.692+0000 7f193a2b5640 1 --2- 172.21.15.89:0/1835092156 >> [v2:172.21.15.110:6838/132366709,v1:172.21.15.110:6839/132366709] conn(0x55c3213aa1c0 0x55c3217b44e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:18.698 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:18.693+0000 7f193a2b5640 1 --2- 172.21.15.89:0/1835092156 >> [v2:172.21.15.110:6838/132366709,v1:172.21.15.110:6839/132366709] conn(0x55c3213aa1c0 0x55c3217b44e0 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e23b778fd362b82c server_cookie=2d18ea3f4912a4c4 in_seq=0 out_seq=0 2024-09-21T03:42:18.698 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:18.693+0000 7f1919ffb640 10 client.15927 ms_handle_connect on v2:172.21.15.110:6838/132366709 2024-09-21T03:42:18.700 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:18.693+0000 7f1919ffb640 1 -- 172.21.15.89:0/1835092156 <== mds.0 v2:172.21.15.110:6838/132366709 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55c32170aae0 con 0x55c3213aa1c0 2024-09-21T03:42:18.700 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:18.694+0000 7f1919ffb640 10 client.15927 handle_command_reply: tid=0 2024-09-21T03:42:18.700 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:18.694+0000 7f1942a8d740 1 client.15927 shutdown 2024-09-21T03:42:18.700 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:18.694+0000 7f1942a8d740 1 -- 172.21.15.89:0/1835092156 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c3213eb840 msgr2=0x55c3217ae460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:18.700 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:18.694+0000 7f1942a8d740 1 --2- 172.21.15.89:0/1835092156 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c3213eb840 0x55c3217ae460 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f192c005c40 tx=0x7f192c004a40 comp rx=0 tx=0).stop 2024-09-21T03:42:18.700 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:18.694+0000 7f1942a8d740 1 -- 172.21.15.89:0/1835092156 shutdown_connections 2024-09-21T03:42:18.700 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:18.694+0000 7f1942a8d740 1 --2- 172.21.15.89:0/1835092156 >> [v2:172.21.15.110:6838/132366709,v1:172.21.15.110:6839/132366709] conn(0x55c3213aa1c0 0x55c3217b44e0 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:18.700 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:18.694+0000 7f1942a8d740 1 --2- 172.21.15.89:0/1835092156 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c3213eb840 0x55c3217ae460 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:18.700 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:18.694+0000 7f1942a8d740 1 --2- 172.21.15.89:0/1835092156 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c3213cab60 0x55c3217a2a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:18.700 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:18.694+0000 7f1942a8d740 1 --2- 172.21.15.89:0/1835092156 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c3213c2980 0x55c3217a24e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:18.700 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:18.694+0000 7f1942a8d740 1 -- 172.21.15.89:0/1835092156 >> 172.21.15.89:0/1835092156 conn(0x55c3213df990 msgr2=0x55c32178ea90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:42:18.701 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:18.694+0000 7f1919ffb640 0 client.15927 ms_handle_reset on v2:172.21.15.110:6838/132366709 2024-09-21T03:42:18.701 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:18.694+0000 7f1942a8d740 1 -- 172.21.15.89:0/1835092156 shutdown_connections 2024-09-21T03:42:18.701 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:18.694+0000 7f1942a8d740 1 -- 172.21.15.89:0/1835092156 wait complete. 2024-09-21T03:42:18.701 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:18.694+0000 7f1942a8d740 20 client.15927 trim_cache size 0 max 16384 2024-09-21T03:42:18.713 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:18.711+0000 7f1942a8d740 1 Processor -- start 2024-09-21T03:42:18.713 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:18.711+0000 7f1942a8d740 1 -- start start 2024-09-21T03:42:18.713 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:18.711+0000 7f1942a8d740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c3213c2980 0x55c3217a3f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:18.713 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:18.711+0000 7f1942a8d740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c3213cab60 0x55c3217ae360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:18.714 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:18.711+0000 7f1942a8d740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c3213eb840 0x55c3217ae8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:18.714 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:18.711+0000 7f1942a8d740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f19341b1060 con 0x55c3213cab60 2024-09-21T03:42:18.714 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:18.711+0000 7f1942a8d740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f19341ae930 con 0x55c3213c2980 2024-09-21T03:42:18.714 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:18.711+0000 7f1942a8d740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f19341ac780 con 0x55c3213eb840 2024-09-21T03:42:18.714 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:18.711+0000 7f1939ab4640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c3213cab60 0x55c3217ae360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:18.714 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:18.711+0000 7f1939ab4640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c3213cab60 0x55c3217ae360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:53160/0 (socket says 172.21.15.89:53160) 2024-09-21T03:42:18.714 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:18.711+0000 7f1939ab4640 1 -- 172.21.15.89:0/983286263 learned_addr learned my addr 172.21.15.89:0/983286263 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-21T03:42:18.717 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:18.711+0000 7f193a2b5640 1 --2- 172.21.15.89:0/983286263 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c3213c2980 0x55c3217a3f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:18.717 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:18.711+0000 7f1939ab4640 1 -- 172.21.15.89:0/983286263 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c3213eb840 msgr2=0x55c3217ae8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:42:18.718 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:18.711+0000 7f1939ab4640 1 --2- 172.21.15.89:0/983286263 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c3213eb840 0x55c3217ae8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:18.718 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:18.711+0000 7f1939ab4640 1 -- 172.21.15.89:0/983286263 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c3213c2980 msgr2=0x55c3217a3f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:42:18.718 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:18.711+0000 7f1939ab4640 1 --2- 172.21.15.89:0/983286263 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c3213c2980 0x55c3217a3f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:18.718 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:18.711+0000 7f1939ab4640 1 -- 172.21.15.89:0/983286263 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c32178fbf0 con 0x55c3213cab60 2024-09-21T03:42:18.718 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:18.711+0000 7f193a2b5640 1 --2- 172.21.15.89:0/983286263 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c3213c2980 0x55c3217a3f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:42:18.718 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:18.712+0000 7f1939ab4640 1 --2- 172.21.15.89:0/983286263 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c3213cab60 0x55c3217ae360 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f1928009900 tx=0x7f192802ab10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da4e4e2f2ca858a6 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:18.718 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:18.712+0000 7f191bfff640 1 -- 172.21.15.89:0/983286263 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1928040040 con 0x55c3213cab60 2024-09-21T03:42:18.718 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:18.712+0000 7f191bfff640 1 -- 172.21.15.89:0/983286263 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f19280044d0 con 0x55c3213cab60 2024-09-21T03:42:18.718 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:18.712+0000 7f191bfff640 1 -- 172.21.15.89:0/983286263 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1928035040 con 0x55c3213cab60 2024-09-21T03:42:18.718 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:18.712+0000 7f1942a8d740 1 -- 172.21.15.89:0/983286263 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c3213cab60 msgr2=0x55c3217ae360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:18.718 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:18.712+0000 7f1942a8d740 1 --2- 172.21.15.89:0/983286263 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c3213cab60 0x55c3217ae360 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f1928009900 tx=0x7f192802ab10 comp rx=0 tx=0).stop 2024-09-21T03:42:18.718 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:18.712+0000 7f1942a8d740 1 -- 172.21.15.89:0/983286263 shutdown_connections 2024-09-21T03:42:18.718 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:18.712+0000 7f1942a8d740 1 --2- 172.21.15.89:0/983286263 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c3213eb840 0x55c3217ae8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:18.718 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:18.712+0000 7f1942a8d740 1 --2- 172.21.15.89:0/983286263 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c3213cab60 0x55c3217ae360 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:18.719 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:18.712+0000 7f1942a8d740 1 --2- 172.21.15.89:0/983286263 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c3213c2980 0x55c3217a3f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:18.719 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:18.712+0000 7f1942a8d740 1 -- 172.21.15.89:0/983286263 >> 172.21.15.89:0/983286263 conn(0x55c3213df990 msgr2=0x55c3217a3800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:42:18.719 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:18.713+0000 7f1942a8d740 1 -- 172.21.15.89:0/983286263 shutdown_connections 2024-09-21T03:42:18.719 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:18.713+0000 7f1942a8d740 1 -- 172.21.15.89:0/983286263 wait complete. 2024-09-21T03:42:18.719 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:18.713+0000 7f1942a8d740 1 -- start start 2024-09-21T03:42:18.719 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:18.713+0000 7f1942a8d740 1 Processor -- start 2024-09-21T03:42:18.719 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:18.713+0000 7f1942a8d740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c3213c2980 0x55c3217aa8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:18.719 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:18.713+0000 7f1942a8d740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c3213cab60 0x55c3217a6df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:18.719 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:18.713+0000 7f1942a8d740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c3213eb840 0x55c3217a7330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:18.719 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:18.713+0000 7f1942a8d740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f19341acdf0 con 0x55c3213cab60 2024-09-21T03:42:18.719 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:18.713+0000 7f1942a8d740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f19341ae260 con 0x55c3213c2980 2024-09-21T03:42:18.719 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:18.713+0000 7f1942a8d740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f19341ad4c0 con 0x55c3213eb840 2024-09-21T03:42:18.719 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:18.713+0000 7f1942a8d740 10 client.0 _ll_register_callbacks cb 0x7f192c009df0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:42:18.719 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:18.713+0000 7f1939ab4640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c3213cab60 0x55c3217a6df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:18.719 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:18.713+0000 7f1939ab4640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c3213cab60 0x55c3217a6df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:53176/0 (socket says 172.21.15.89:53176) 2024-09-21T03:42:18.720 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:18.713+0000 7f1939ab4640 1 -- 172.21.15.89:0/65354904 learned_addr learned my addr 172.21.15.89:0/65354904 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-21T03:42:18.720 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:18.713+0000 7f193a2b5640 1 --2- 172.21.15.89:0/65354904 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c3213c2980 0x55c3217aa8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:18.720 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:18.714+0000 7f193aab6640 1 --2- 172.21.15.89:0/65354904 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c3213eb840 0x55c3217a7330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:18.720 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:18.714+0000 7f1939ab4640 1 -- 172.21.15.89:0/65354904 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c3213eb840 msgr2=0x55c3217a7330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:42:18.720 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:18.714+0000 7f1939ab4640 1 --2- 172.21.15.89:0/65354904 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c3213eb840 0x55c3217a7330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:18.720 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:18.714+0000 7f1939ab4640 1 -- 172.21.15.89:0/65354904 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c3213c2980 msgr2=0x55c3217aa8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:42:18.720 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:18.714+0000 7f1939ab4640 1 --2- 172.21.15.89:0/65354904 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c3213c2980 0x55c3217aa8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:18.720 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:18.714+0000 7f1939ab4640 1 -- 172.21.15.89:0/65354904 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c3217a7870 con 0x55c3213cab60 2024-09-21T03:42:18.720 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:18.714+0000 7f193aab6640 1 --2- 172.21.15.89:0/65354904 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c3213eb840 0x55c3217a7330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:42:18.720 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:18.714+0000 7f1939ab4640 1 --2- 172.21.15.89:0/65354904 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c3213cab60 0x55c3217a6df0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f192802a930 tx=0x7f1928031630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c69b2e3fd1893910 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:18.720 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:18.714+0000 7f1919ffb640 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2024-09-21T03:42:18.720 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:18.714+0000 7f1919ffb640 1 -- 172.21.15.89:0/65354904 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1928054070 con 0x55c3213cab60 2024-09-21T03:42:18.720 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:18.714+0000 7f1919ffb640 1 -- 172.21.15.89:0/65354904 <== mon.0 v2:172.21.15.89:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 9121+0+0 (secure 0 0 0) 0x7f192803c850 con 0x55c3213cab60 2024-09-21T03:42:18.720 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:18.714+0000 7f1942a8d740 1 -- 172.21.15.89:0/65354904 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f19280044d0 con 0x55c3213cab60 2024-09-21T03:42:18.721 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:18.714+0000 7f1919ffb640 1 -- 172.21.15.89:0/65354904 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f1928035070 con 0x55c3213cab60 2024-09-21T03:42:18.721 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:18.714+0000 7f1942a8d740 10 client.15933 fetch_fsmap learned FSMap version 26 2024-09-21T03:42:18.721 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:18.714+0000 7f1942a8d740 1 -- 172.21.15.89:0/65354904 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55c32181fa70 con 0x55c3213cab60 2024-09-21T03:42:18.721 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:18.715+0000 7f1919ffb640 1 -- 172.21.15.89:0/65354904 <== mon.0 v2:172.21.15.89:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f19280044d0 con 0x55c3213cab60 2024-09-21T03:42:18.721 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:18.715+0000 7f1919ffb640 1 -- 172.21.15.89:0/65354904 <== mon.0 v2:172.21.15.89:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f1928059920 con 0x55c3213cab60 2024-09-21T03:42:18.721 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:18.715+0000 7f1942a8d740 10 client.15933 fetch_fsmap finished waiting for FSMap version 26 2024-09-21T03:42:18.721 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:18.715+0000 7f1942a8d740 10 client.15933 resolve_mds: validated gid 24638 aka daemon mds.b 2024-09-21T03:42:18.721 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:18.715+0000 7f1942a8d740 20 client.15933 populate_metadata read hostname 'smithi089' 2024-09-21T03:42:18.721 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:18.715+0000 7f1942a8d740 1 --2- 172.21.15.89:0/65354904 >> [v2:172.21.15.110:6838/132366709,v1:172.21.15.110:6839/132366709] conn(0x55c32137f570 0x55c321821d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:18.721 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:18.715+0000 7f1942a8d740 4 client.15933 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 147.8, "await_for": 45.0}] 2024-09-21T03:42:18.721 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:18.715+0000 7f1942a8d740 1 -- 172.21.15.89:0/65354904 --> [v2:172.21.15.110:6838/132366709,v1:172.21.15.110:6839/132366709] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 147.8, "await_for": 45.0}) -- 0x55c321706590 con 0x55c32137f570 2024-09-21T03:42:18.721 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:18.715+0000 7f193a2b5640 1 --2- 172.21.15.89:0/65354904 >> [v2:172.21.15.110:6838/132366709,v1:172.21.15.110:6839/132366709] conn(0x55c32137f570 0x55c321821d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:18.721 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:18.715+0000 7f1919ffb640 10 client.15933 ms_handle_connect on v2:172.21.15.110:6838/132366709 2024-09-21T03:42:18.721 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:18.715+0000 7f193a2b5640 1 --2- 172.21.15.89:0/65354904 >> [v2:172.21.15.110:6838/132366709,v1:172.21.15.110:6839/132366709] conn(0x55c32137f570 0x55c321821d10 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6d516e0644d8981c server_cookie=cd54f66e54ad2a8d in_seq=0 out_seq=0 2024-09-21T03:42:18.725 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:18.722+0000 7f1919ffb640 1 -- 172.21.15.89:0/65354904 <== mds.0 v2:172.21.15.110:6838/132366709 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55c321706590 con 0x55c32137f570 2024-09-21T03:42:18.725 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:18.722+0000 7f1919ffb640 10 client.15933 handle_command_reply: tid=0 2024-09-21T03:42:18.725 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:18.722+0000 7f1942a8d740 1 client.15933 shutdown 2024-09-21T03:42:18.725 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:18.722+0000 7f1942a8d740 1 -- 172.21.15.89:0/65354904 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c3213cab60 msgr2=0x55c3217a6df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:18.725 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:18.722+0000 7f1942a8d740 1 --2- 172.21.15.89:0/65354904 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c3213cab60 0x55c3217a6df0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f192802a930 tx=0x7f1928031630 comp rx=0 tx=0).stop 2024-09-21T03:42:18.725 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:18.722+0000 7f1942a8d740 1 -- 172.21.15.89:0/65354904 shutdown_connections 2024-09-21T03:42:18.725 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:18.722+0000 7f1942a8d740 1 --2- 172.21.15.89:0/65354904 >> [v2:172.21.15.110:6838/132366709,v1:172.21.15.110:6839/132366709] conn(0x55c32137f570 0x55c321821d10 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:18.725 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:18.722+0000 7f1942a8d740 1 --2- 172.21.15.89:0/65354904 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c3213eb840 0x55c3217a7330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:18.726 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:18.722+0000 7f1942a8d740 1 --2- 172.21.15.89:0/65354904 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c3213cab60 0x55c3217a6df0 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:18.726 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:18.722+0000 7f1942a8d740 1 --2- 172.21.15.89:0/65354904 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c3213c2980 0x55c3217aa8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:18.726 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:18.722+0000 7f1942a8d740 1 -- 172.21.15.89:0/65354904 >> 172.21.15.89:0/65354904 conn(0x55c3213df990 msgr2=0x55c321798080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:42:18.726 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:18.722+0000 7f1919ffb640 0 client.15933 ms_handle_reset on v2:172.21.15.110:6838/132366709 2024-09-21T03:42:18.726 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:18.722+0000 7f1942a8d740 1 -- 172.21.15.89:0/65354904 shutdown_connections 2024-09-21T03:42:18.726 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:18.722+0000 7f1942a8d740 1 -- 172.21.15.89:0/65354904 wait complete. 2024-09-21T03:42:18.726 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:18.723+0000 7f1942a8d740 20 client.15933 trim_cache size 0 max 16384 2024-09-21T03:42:18.726 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-09-21T03:42:18.726 INFO:teuthology.orchestra.run.smithi089.stdout: "epoch": 26, 2024-09-21T03:42:18.726 INFO:teuthology.orchestra.run.smithi089.stdout: "leader": 24638, 2024-09-21T03:42:18.726 INFO:teuthology.orchestra.run.smithi089.stdout: "set_version": 14, 2024-09-21T03:42:18.727 INFO:teuthology.orchestra.run.smithi089.stdout: "sets": { 2024-09-21T03:42:18.727 INFO:teuthology.orchestra.run.smithi089.stdout: "18e032f0": { 2024-09-21T03:42:18.727 INFO:teuthology.orchestra.run.smithi089.stdout: "version": 14, 2024-09-21T03:42:18.727 INFO:teuthology.orchestra.run.smithi089.stdout: "age_ref": 0.0, 2024-09-21T03:42:18.727 INFO:teuthology.orchestra.run.smithi089.stdout: "state": { 2024-09-21T03:42:18.727 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "QUIESCED", 2024-09-21T03:42:18.727 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 0.0 2024-09-21T03:42:18.727 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:18.727 INFO:teuthology.orchestra.run.smithi089.stdout: "timeout": 90.0, 2024-09-21T03:42:18.727 INFO:teuthology.orchestra.run.smithi089.stdout: "expiration": 147.8, 2024-09-21T03:42:18.727 INFO:teuthology.orchestra.run.smithi089.stdout: "members": { 2024-09-21T03:42:18.727 INFO:teuthology.orchestra.run.smithi089.stdout: "file:/": { 2024-09-21T03:42:18.727 INFO:teuthology.orchestra.run.smithi089.stdout: "excluded": false, 2024-09-21T03:42:18.727 INFO:teuthology.orchestra.run.smithi089.stdout: "state": { 2024-09-21T03:42:18.727 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "QUIESCED", 2024-09-21T03:42:18.727 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 0.0 2024-09-21T03:42:18.728 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-21T03:42:18.728 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-21T03:42:18.728 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-21T03:42:18.728 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-21T03:42:18.728 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-21T03:42:18.728 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-09-21T03:42:18.728 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:18.723+0000 7f193c519640 1 -- 172.21.15.89:0/3880837163 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f191c0b6510 msgr2=0x7f191c0b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:18.729 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:18.723+0000 7f193c519640 1 --2- 172.21.15.89:0/3880837163 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f191c0b6510 0x7f191c0b8a20 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f192c005fd0 tx=0x7f192c009600 comp rx=0 tx=0).stop 2024-09-21T03:42:18.729 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:18.723+0000 7f193c519640 1 -- 172.21.15.89:0/3880837163 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1934102a80 msgr2=0x7f193419a4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:18.729 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:18.723+0000 7f193c519640 1 --2- 172.21.15.89:0/3880837163 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1934102a80 0x7f193419a4d0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f192400d770 tx=0x7f192400dc30 comp rx=0 tx=0).stop 2024-09-21T03:42:18.729 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:18.723+0000 7f193c519640 1 -- 172.21.15.89:0/3880837163 shutdown_connections 2024-09-21T03:42:18.729 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:18.723+0000 7f193c519640 1 --2- 172.21.15.89:0/3880837163 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f191c0b6510 0x7f191c0b8a20 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:18.729 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:18.723+0000 7f193c519640 1 --2- 172.21.15.89:0/3880837163 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f19341084d0 0x7f193419aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:18.729 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:18.723+0000 7f193c519640 1 --2- 172.21.15.89:0/3880837163 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1934108e20 0x7f193419fd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:18.729 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:18.723+0000 7f193c519640 1 --2- 172.21.15.89:0/3880837163 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1934102a80 0x7f193419a4d0 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:18.729 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:18.723+0000 7f193c519640 1 -- 172.21.15.89:0/3880837163 >> 172.21.15.89:0/3880837163 conn(0x7f1934079720 msgr2=0x7f193406bec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:42:18.729 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:18.723+0000 7f193c519640 1 -- 172.21.15.89:0/3880837163 shutdown_connections 2024-09-21T03:42:18.729 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:18.723+0000 7f193c519640 1 -- 172.21.15.89:0/3880837163 wait complete. 2024-09-21T03:42:19.039 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:42:18 smithi089 ceph-mon[30928]: pgmap v447: 129 pgs: 129 active+clean; 182 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 733 KiB/s wr, 162 op/s 2024-09-21T03:42:19.211 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:42:18 smithi114 ceph-mon[34140]: pgmap v447: 129 pgs: 129 active+clean; 182 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 733 KiB/s wr, 162 op/s 2024-09-21T03:42:19.250 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:42:18 smithi110 ceph-mon[34000]: pgmap v447: 129 pgs: 129 active+clean; 182 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 733 KiB/s wr, 162 op/s 2024-09-21T03:42:19.569 INFO:tasks.quiescer:done joining Quiescers 2024-09-21T03:42:19.569 DEBUG:teuthology.run_tasks:Unwinding manager background_exec 2024-09-21T03:42:19.578 INFO:teuthology.task.background_exec:Stopping background command on ubuntu@smithi089.front.sepia.ceph.com 2024-09-21T03:42:19.587 DEBUG:teuthology.run_tasks:Unwinding manager check-counter 2024-09-21T03:42:20.367 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T03:42:20.489 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:20.484+0000 7f85e269b640 1 -- 172.21.15.89:0/1301271746 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f85dc1056c0 msgr2=0x7f85dc107ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:20.489 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:20.484+0000 7f85e269b640 1 --2- 172.21.15.89:0/1301271746 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f85dc1056c0 0x7f85dc107ba0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f85d0009900 tx=0x7f85d002a410 comp rx=0 tx=0).stop 2024-09-21T03:42:20.489 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:20.484+0000 7f85e269b640 1 -- 172.21.15.89:0/1301271746 shutdown_connections 2024-09-21T03:42:20.489 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:20.484+0000 7f85e269b640 1 --2- 172.21.15.89:0/1301271746 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f85dc1082e0 0x7f85dc10a7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:20.489 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:20.484+0000 7f85e269b640 1 --2- 172.21.15.89:0/1301271746 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f85dc1056c0 0x7f85dc107ba0 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:20.489 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:20.484+0000 7f85e269b640 1 --2- 172.21.15.89:0/1301271746 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f85dc102c40 0x7f85dc105080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:20.489 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:20.484+0000 7f85e269b640 1 -- 172.21.15.89:0/1301271746 >> 172.21.15.89:0/1301271746 conn(0x7f85dc0fd690 msgr2=0x7f85dc0ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:42:20.489 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:20.484+0000 7f85e269b640 1 -- 172.21.15.89:0/1301271746 shutdown_connections 2024-09-21T03:42:20.489 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:20.484+0000 7f85e269b640 1 -- 172.21.15.89:0/1301271746 wait complete. 2024-09-21T03:42:20.490 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:20.485+0000 7f85e269b640 1 Processor -- start 2024-09-21T03:42:20.490 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:20.485+0000 7f85e269b640 1 -- start start 2024-09-21T03:42:20.490 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:20.485+0000 7f85e269b640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f85dc102c40 0x7f85dc102090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:20.490 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:20.485+0000 7f85e269b640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f85dc1056c0 0x7f85dc19eca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:20.490 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:20.485+0000 7f85e269b640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f85dc1082e0 0x7f85dc1a11f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:20.490 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:20.485+0000 7f85e269b640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f85dc10e0a0 con 0x7f85dc1056c0 2024-09-21T03:42:20.490 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:20.485+0000 7f85e269b640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f85dc10df20 con 0x7f85dc102c40 2024-09-21T03:42:20.490 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:20.485+0000 7f85e269b640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f85dc10e220 con 0x7f85dc1082e0 2024-09-21T03:42:20.490 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:20.485+0000 7f85db7fe640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f85dc1056c0 0x7f85dc19eca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:20.490 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:20.485+0000 7f85db7fe640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f85dc1056c0 0x7f85dc19eca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:53198/0 (socket says 172.21.15.89:53198) 2024-09-21T03:42:20.490 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:20.485+0000 7f85db7fe640 1 -- 172.21.15.89:0/953536057 learned_addr learned my addr 172.21.15.89:0/953536057 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-21T03:42:20.490 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:20.486+0000 7f85e0c38640 1 --2- 172.21.15.89:0/953536057 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f85dc1082e0 0x7f85dc1a11f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:20.490 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:20.486+0000 7f85dbfff640 1 --2- 172.21.15.89:0/953536057 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f85dc102c40 0x7f85dc102090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:20.490 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:20.486+0000 7f85db7fe640 1 -- 172.21.15.89:0/953536057 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f85dc1082e0 msgr2=0x7f85dc1a11f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:20.490 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:20.486+0000 7f85db7fe640 1 --2- 172.21.15.89:0/953536057 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f85dc1082e0 0x7f85dc1a11f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:20.490 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:20.486+0000 7f85db7fe640 1 -- 172.21.15.89:0/953536057 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f85dc102c40 msgr2=0x7f85dc102090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:20.491 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:20.486+0000 7f85db7fe640 1 --2- 172.21.15.89:0/953536057 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f85dc102c40 0x7f85dc102090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:20.491 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:20.486+0000 7f85db7fe640 1 -- 172.21.15.89:0/953536057 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85dc1a19c0 con 0x7f85dc1056c0 2024-09-21T03:42:20.491 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:20.486+0000 7f85e0c38640 1 --2- 172.21.15.89:0/953536057 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f85dc1082e0 0x7f85dc1a11f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:42:20.491 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:20.486+0000 7f85db7fe640 1 --2- 172.21.15.89:0/953536057 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f85dc1056c0 0x7f85dc19eca0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f85d0009230 tx=0x7f85d00042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:20.491 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:20.486+0000 7f85d97fa640 1 -- 172.21.15.89:0/953536057 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85d003f030 con 0x7f85dc1056c0 2024-09-21T03:42:20.491 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:20.486+0000 7f85d97fa640 1 -- 172.21.15.89:0/953536057 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f85d0003c90 con 0x7f85dc1056c0 2024-09-21T03:42:20.491 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:20.486+0000 7f85e269b640 1 -- 172.21.15.89:0/953536057 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f85dc1a1c50 con 0x7f85dc1056c0 2024-09-21T03:42:20.491 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:20.486+0000 7f85d97fa640 1 -- 172.21.15.89:0/953536057 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85d0035070 con 0x7f85dc1056c0 2024-09-21T03:42:20.491 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:20.486+0000 7f85e269b640 1 -- 172.21.15.89:0/953536057 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85dc1a2140 con 0x7f85dc1056c0 2024-09-21T03:42:20.492 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:20.487+0000 7f85e269b640 1 -- 172.21.15.89:0/953536057 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f85dc0ff1b0 con 0x7f85dc1056c0 2024-09-21T03:42:20.492 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:20.489+0000 7f85d97fa640 1 -- 172.21.15.89:0/953536057 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 21) ==== 154512+0+0 (secure 0 0 0) 0x7f85d0042030 con 0x7f85dc1056c0 2024-09-21T03:42:20.492 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:20.490+0000 7f85d97fa640 1 --2- 172.21.15.89:0/953536057 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f85c40b6510 0x7f85c40b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:20.492 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:20.490+0000 7f85dbfff640 1 --2- 172.21.15.89:0/953536057 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f85c40b6510 0x7f85c40b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:20.492 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:20.490+0000 7f85d97fa640 1 -- 172.21.15.89:0/953536057 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 9121+0+0 (secure 0 0 0) 0x7f85d01080b0 con 0x7f85dc1056c0 2024-09-21T03:42:20.492 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:20.490+0000 7f85dbfff640 1 --2- 172.21.15.89:0/953536057 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f85c40b6510 0x7f85c40b8a20 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f85cc005d00 tx=0x7f85cc00b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:20.495 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:20.493+0000 7f85d97fa640 1 -- 172.21.15.89:0/953536057 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f85d00ce5c0 con 0x7f85dc1056c0 2024-09-21T03:42:20.674 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:20.672+0000 7f85e269b640 1 -- 172.21.15.89:0/953536057 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f85dc02e890 con 0x7f85dc1056c0 2024-09-21T03:42:20.676 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:20.673+0000 7f85d97fa640 1 -- 172.21.15.89:0/953536057 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v84) ==== 74+0+24810 (secure 0 0 0) 0x7f85d00d29a0 con 0x7f85dc1056c0 2024-09-21T03:42:20.676 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-09-21T03:42:20.676 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":84,"fsid":"7c374098-77c9-11ef-baf5-efdc52797490","created":"2024-09-21T03:28:56.681458+0000","modified":"2024-09-21T03:38:53.661606+0000","last_up_change":"2024-09-21T03:36:23.368379+0000","last_in_change":"2024-09-21T03:36:00.131331+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T03:32:37.834339+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-21T03:37:09.443341+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"74","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-21T03:37:11.467821+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"74","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-21T03:37:17.884066+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"27ca5ed8-17d1-4f8e-a7ae-084a914721a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6802","nonce":990539042},{"type":"v1","addr":"172.21.15.89:6803","nonce":990539042}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6804","nonce":990539042},{"type":"v1","addr":"172.21.15.89:6805","nonce":990539042}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6808","nonce":990539042},{"type":"v1","addr":"172.21.15.89:6809","nonce":990539042}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6806","nonce":990539042},{"type":"v1","addr":"172.21.15.89:6807","nonce":990539042}]},"public_addr":"172.21.15.89:6803/990539042","cluster_addr":"172.21.15.89:6805/990539042","heartbeat_back_addr":"172.21.15.89:6809/990539042","heartbeat_front_addr":"172.21.15.89:6807/990539042","state":["exists","up"]},{"osd":1,"uuid":"5176a910-5cf9-49d7-aedb-a20cf36ea075","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6800","nonce":3235852628},{"type":"v1","addr":"172.21.15.110:6801","nonce":3235852628}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6802","nonce":3235852628},{"type":"v1","addr":"172.21.15.110:6803","nonce":3235852628}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6806","nonce":3235852628},{"type":"v1","addr":"172.21.15.110:6807","nonce":3235852628}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6804","nonce":3235852628},{"type":"v1","addr":"172.21.15.110:6805","nonce":3235852628}]},"public_addr":"172.21.15.110:6801/3235852628","cluster_addr":"172.21.15.110:6803/3235852628","heartbeat_back_addr":"172.21.15.110:6807/3235852628","heartbeat_front_addr":"172.21.15.110:6805/3235852628","state":["exists","up"]},{"osd":2,"uuid":"d4977df1-b118-4d7e-b9a9-626a8accc812","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6800","nonce":169452122},{"type":"v1","addr":"172.21.15.114:6801","nonce":169452122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6802","nonce":169452122},{"type":"v1","addr":"172.21.15.114:6803","nonce":169452122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6806","nonce":169452122},{"type":"v1","addr":"172.21.15.114:6807","nonce":169452122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6804","nonce":169452122},{"type":"v1","addr":"172.21.15.114:6805","nonce":169452122}]},"public_addr":"172.21.15.114:6801/169452122","cluster_addr":"172.21.15.114:6803/169452122","heartbeat_back_addr":"172.21.15.114:6807/169452122","heartbeat_front_addr":"172.21.15.114:6805/169452122","state":["exists","up"]},{"osd":3,"uuid":"2c7ae1b4-0bf2-4954-81ed-55d03eab82c6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6810","nonce":3016980877},{"type":"v1","addr":"172.21.15.89:6811","nonce":3016980877}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6812","nonce":3016980877},{"type":"v1","addr":"172.21.15.89:6813","nonce":3016980877}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6816","nonce":3016980877},{"type":"v1","addr":"172.21.15.89:6817","nonce":3016980877}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6814","nonce":3016980877},{"type":"v1","addr":"172.21.15.89:6815","nonce":3016980877}]},"public_addr":"172.21.15.89:6811/3016980877","cluster_addr":"172.21.15.89:6813/3016980877","heartbeat_back_addr":"172.21.15.89:6817/3016980877","heartbeat_front_addr":"172.21.15.89:6815/3016980877","state":["exists","up"]},{"osd":4,"uuid":"8e65399d-6e55-4da4-98ba-f805f0c655a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":30,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6808","nonce":3217880476},{"type":"v1","addr":"172.21.15.110:6809","nonce":3217880476}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6810","nonce":3217880476},{"type":"v1","addr":"172.21.15.110:6811","nonce":3217880476}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6814","nonce":3217880476},{"type":"v1","addr":"172.21.15.110:6815","nonce":3217880476}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6812","nonce":3217880476},{"type":"v1","addr":"172.21.15.110:6813","nonce":3217880476}]},"public_addr":"172.21.15.110:6809/3217880476","cluster_addr":"172.21.15.110:6811/3217880476","heartbeat_back_addr":"172.21.15.110:6815/3217880476","heartbeat_front_addr":"172.21.15.110:6813/3217880476","state":["exists","up"]},{"osd":5,"uuid":"b9ec2889-46bd-40e5-8446-73ab28271c67","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6808","nonce":1144140707},{"type":"v1","addr":"172.21.15.114:6809","nonce":1144140707}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6810","nonce":1144140707},{"type":"v1","addr":"172.21.15.114:6811","nonce":1144140707}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6814","nonce":1144140707},{"type":"v1","addr":"172.21.15.114:6815","nonce":1144140707}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6812","nonce":1144140707},{"type":"v1","addr":"172.21.15.114:6813","nonce":1144140707}]},"public_addr":"172.21.15.114:6809/1144140707","cluster_addr":"172.21.15.114:6811/1144140707","heartbeat_back_addr":"172.21.15.114:6815/1144140707","heartbeat_front_addr":"172.21.15.114:6813/1144140707","state":["exists","up"]},{"osd":6,"uuid":"9b79989e-adb0-42b2-b2f6-a2dce56b66e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":41,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6818","nonce":1522586255},{"type":"v1","addr":"172.21.15.89:6819","nonce":1522586255}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6820","nonce":1522586255},{"type":"v1","addr":"172.21.15.89:6821","nonce":1522586255}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6824","nonce":1522586255},{"type":"v1","addr":"172.21.15.89:6825","nonce":1522586255}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6822","nonce":1522586255},{"type":"v1","addr":"172.21.15.89:6823","nonce":1522586255}]},"public_addr":"172.21.15.89:6819/1522586255","cluster_addr":"172.21.15.89:6821/1522586255","heartbeat_back_addr":"172.21.15.89:6825/1522586255","heartbeat_front_addr":"172.21.15.89:6823/1522586255","state":["exists","up"]},{"osd":7,"uuid":"b381821f-141c-47a1-be65-f57ca730391b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6816","nonce":1331818323},{"type":"v1","addr":"172.21.15.110:6817","nonce":1331818323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6818","nonce":1331818323},{"type":"v1","addr":"172.21.15.110:6819","nonce":1331818323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6822","nonce":1331818323},{"type":"v1","addr":"172.21.15.110:6823","nonce":1331818323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6820","nonce":1331818323},{"type":"v1","addr":"172.21.15.110:6821","nonce":1331818323}]},"public_addr":"172.21.15.110:6817/1331818323","cluster_addr":"172.21.15.110:6819/1331818323","heartbeat_back_addr":"172.21.15.110:6823/1331818323","heartbeat_front_addr":"172.21.15.110:6821/1331818323","state":["exists","up"]},{"osd":8,"uuid":"1bc2a5b4-15f3-456f-9d67-e51307a27bcc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6816","nonce":2138556699},{"type":"v1","addr":"172.21.15.114:6817","nonce":2138556699}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6818","nonce":2138556699},{"type":"v1","addr":"172.21.15.114:6819","nonce":2138556699}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6822","nonce":2138556699},{"type":"v1","addr":"172.21.15.114:6823","nonce":2138556699}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6820","nonce":2138556699},{"type":"v1","addr":"172.21.15.114:6821","nonce":2138556699}]},"public_addr":"172.21.15.114:6817/2138556699","cluster_addr":"172.21.15.114:6819/2138556699","heartbeat_back_addr":"172.21.15.114:6823/2138556699","heartbeat_front_addr":"172.21.15.114:6821/2138556699","state":["exists","up"]},{"osd":9,"uuid":"91f5fd29-d4a7-4800-ac7a-e83b13add89e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6826","nonce":1730253068},{"type":"v1","addr":"172.21.15.89:6827","nonce":1730253068}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6828","nonce":1730253068},{"type":"v1","addr":"172.21.15.89:6829","nonce":1730253068}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6832","nonce":1730253068},{"type":"v1","addr":"172.21.15.89:6833","nonce":1730253068}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6830","nonce":1730253068},{"type":"v1","addr":"172.21.15.89:6831","nonce":1730253068}]},"public_addr":"172.21.15.89:6827/1730253068","cluster_addr":"172.21.15.89:6829/1730253068","heartbeat_back_addr":"172.21.15.89:6833/1730253068","heartbeat_front_addr":"172.21.15.89:6831/1730253068","state":["exists","up"]},{"osd":10,"uuid":"01390499-2a4d-4188-b57a-fcb10c668e2c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6824","nonce":2904016117},{"type":"v1","addr":"172.21.15.110:6825","nonce":2904016117}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6826","nonce":2904016117},{"type":"v1","addr":"172.21.15.110:6827","nonce":2904016117}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6830","nonce":2904016117},{"type":"v1","addr":"172.21.15.110:6831","nonce":2904016117}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6828","nonce":2904016117},{"type":"v1","addr":"172.21.15.110:6829","nonce":2904016117}]},"public_addr":"172.21.15.110:6825/2904016117","cluster_addr":"172.21.15.110:6827/2904016117","heartbeat_back_addr":"172.21.15.110:6831/2904016117","heartbeat_front_addr":"172.21.15.110:6829/2904016117","state":["exists","up"]},{"osd":11,"uuid":"c3c0f38a-c3d3-415d-a9f8-875d5b756e35","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6824","nonce":3749854486},{"type":"v1","addr":"172.21.15.114:6825","nonce":3749854486}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6826","nonce":3749854486},{"type":"v1","addr":"172.21.15.114:6827","nonce":3749854486}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6830","nonce":3749854486},{"type":"v1","addr":"172.21.15.114:6831","nonce":3749854486}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6828","nonce":3749854486},{"type":"v1","addr":"172.21.15.114:6829","nonce":3749854486}]},"public_addr":"172.21.15.114:6825/3749854486","cluster_addr":"172.21.15.114:6827/3749854486","heartbeat_back_addr":"172.21.15.114:6831/3749854486","heartbeat_front_addr":"172.21.15.114:6829/3749854486","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:31:46.218908+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:32:10.338230+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:32:34.830596+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:33:00.816102+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:33:25.223838+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:33:50.188230+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:34:16.564255+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:34:41.498230+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:35:05.826322+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:35:31.234450+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:35:56.474373+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:36:21.741254+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.c","mappings":[{"from":2,"to":4}]},{"pgid":"2.e","mappings":[{"from":2,"to":1}]},{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.1b","mappings":[{"from":5,"to":9}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]},{"pgid":"4.6","mappings":[{"from":9,"to":11}]},{"pgid":"4.12","mappings":[{"from":9,"to":10}]},{"pgid":"4.13","mappings":[{"from":9,"to":8}]},{"pgid":"4.1b","mappings":[{"from":5,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.89:6801/655110910":"2024-09-22T03:29:51.519106+0000","172.21.15.89:0/1618544983":"2024-09-22T03:29:51.519106+0000","172.21.15.89:0/1265502758":"2024-09-22T03:29:51.519106+0000","172.21.15.89:0/4017684419":"2024-09-22T03:29:51.519106+0000","172.21.15.89:6801/3312712420":"2024-09-22T03:29:26.004686+0000","172.21.15.89:6800/3312712420":"2024-09-22T03:29:26.004686+0000","172.21.15.89:0/2056576483":"2024-09-22T03:29:26.004686+0000","172.21.15.89:6800/655110910":"2024-09-22T03:29:51.519106+0000","172.21.15.89:0/2382866302":"2024-09-22T03:29:26.004686+0000","172.21.15.89:0/1584351801":"2024-09-22T03:29:26.004686+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T03:42:20.678 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:20.676+0000 7f85e269b640 1 -- 172.21.15.89:0/953536057 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f85c40b6510 msgr2=0x7f85c40b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:20.678 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:20.676+0000 7f85e269b640 1 --2- 172.21.15.89:0/953536057 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f85c40b6510 0x7f85c40b8a20 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f85cc005d00 tx=0x7f85cc00b410 comp rx=0 tx=0).stop 2024-09-21T03:42:20.678 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:20.676+0000 7f85e269b640 1 -- 172.21.15.89:0/953536057 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f85dc1056c0 msgr2=0x7f85dc19eca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:20.678 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:20.676+0000 7f85e269b640 1 --2- 172.21.15.89:0/953536057 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f85dc1056c0 0x7f85dc19eca0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f85d0009230 tx=0x7f85d00042f0 comp rx=0 tx=0).stop 2024-09-21T03:42:20.678 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:20.676+0000 7f85e269b640 1 -- 172.21.15.89:0/953536057 shutdown_connections 2024-09-21T03:42:20.678 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:20.676+0000 7f85e269b640 1 --2- 172.21.15.89:0/953536057 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f85c40b6510 0x7f85c40b8a20 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:20.678 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:20.676+0000 7f85e269b640 1 --2- 172.21.15.89:0/953536057 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f85dc1082e0 0x7f85dc1a11f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:20.679 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:20.676+0000 7f85e269b640 1 --2- 172.21.15.89:0/953536057 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f85dc1056c0 0x7f85dc19eca0 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:20.679 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:20.676+0000 7f85e269b640 1 --2- 172.21.15.89:0/953536057 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f85dc102c40 0x7f85dc102090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:20.679 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:20.676+0000 7f85e269b640 1 -- 172.21.15.89:0/953536057 >> 172.21.15.89:0/953536057 conn(0x7f85dc0fd690 msgr2=0x7f85dc103bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:42:20.679 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:20.676+0000 7f85e269b640 1 -- 172.21.15.89:0/953536057 shutdown_connections 2024-09-21T03:42:20.679 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:20.676+0000 7f85e269b640 1 -- 172.21.15.89:0/953536057 wait complete. 2024-09-21T03:42:20.690 INFO:tasks.check_counter.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-21T03:32:37.834339+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-09-21T03:37:09.443341+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '74', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-09-21T03:37:11.467821+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '74', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-09-21T03:37:17.884066+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '80', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-09-21T03:42:20.690 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-21T03:42:20.804 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:20.802+0000 7f06d19a1640 1 -- 172.21.15.89:0/777383038 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f06cc1056c0 msgr2=0x7f06cc107ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:20.805 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:20.802+0000 7f06d19a1640 1 --2- 172.21.15.89:0/777383038 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f06cc1056c0 0x7f06cc107ba0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f06b4009900 tx=0x7f06b402a410 comp rx=0 tx=0).stop 2024-09-21T03:42:20.805 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:20.802+0000 7f06d19a1640 1 -- 172.21.15.89:0/777383038 shutdown_connections 2024-09-21T03:42:20.805 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:20.802+0000 7f06d19a1640 1 --2- 172.21.15.89:0/777383038 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f06cc1082e0 0x7f06cc10a7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:20.805 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:20.802+0000 7f06d19a1640 1 --2- 172.21.15.89:0/777383038 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f06cc1056c0 0x7f06cc107ba0 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:20.805 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:20.802+0000 7f06d19a1640 1 --2- 172.21.15.89:0/777383038 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f06cc102c60 0x7f06cc105080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:20.805 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:20.802+0000 7f06d19a1640 1 -- 172.21.15.89:0/777383038 >> 172.21.15.89:0/777383038 conn(0x7f06cc0fd690 msgr2=0x7f06cc0ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:42:20.805 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:20.802+0000 7f06d19a1640 1 -- 172.21.15.89:0/777383038 shutdown_connections 2024-09-21T03:42:20.805 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:20.802+0000 7f06d19a1640 1 -- 172.21.15.89:0/777383038 wait complete. 2024-09-21T03:42:20.805 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:20.802+0000 7f06d19a1640 1 Processor -- start 2024-09-21T03:42:20.807 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:20.803+0000 7f06d19a1640 1 -- start start 2024-09-21T03:42:20.807 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:20.803+0000 7f06d19a1640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f06cc102c60 0x7f06cc102100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:20.807 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:20.803+0000 7f06d19a1640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f06cc1056c0 0x7f06cc19ec00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:20.807 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:20.803+0000 7f06d19a1640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f06cc1082e0 0x7f06cc1a1150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:20.807 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:20.803+0000 7f06d19a1640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f06cc10e0b0 con 0x7f06cc102c60 2024-09-21T03:42:20.807 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:20.803+0000 7f06d19a1640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f06cc10df30 con 0x7f06cc1056c0 2024-09-21T03:42:20.807 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:20.803+0000 7f06d19a1640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f06cc10e230 con 0x7f06cc1082e0 2024-09-21T03:42:20.808 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:20.803+0000 7f06caffd640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f06cc102c60 0x7f06cc102100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:20.808 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:20.803+0000 7f06caffd640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f06cc102c60 0x7f06cc102100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:53226/0 (socket says 172.21.15.89:53226) 2024-09-21T03:42:20.808 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:20.803+0000 7f06caffd640 1 -- 172.21.15.89:0/2219579257 learned_addr learned my addr 172.21.15.89:0/2219579257 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-21T03:42:20.808 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:20.803+0000 7f06ca7fc640 1 --2- 172.21.15.89:0/2219579257 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f06cc1056c0 0x7f06cc19ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:20.808 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:20.803+0000 7f06cb7fe640 1 --2- 172.21.15.89:0/2219579257 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f06cc1082e0 0x7f06cc1a1150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:20.808 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:20.803+0000 7f06caffd640 1 -- 172.21.15.89:0/2219579257 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f06cc1082e0 msgr2=0x7f06cc1a1150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:20.808 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:20.803+0000 7f06caffd640 1 --2- 172.21.15.89:0/2219579257 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f06cc1082e0 0x7f06cc1a1150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:20.808 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:20.803+0000 7f06caffd640 1 -- 172.21.15.89:0/2219579257 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f06cc1056c0 msgr2=0x7f06cc19ec00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:20.808 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:20.803+0000 7f06caffd640 1 --2- 172.21.15.89:0/2219579257 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f06cc1056c0 0x7f06cc19ec00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:20.808 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:20.803+0000 7f06caffd640 1 -- 172.21.15.89:0/2219579257 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06cc1a1950 con 0x7f06cc102c60 2024-09-21T03:42:20.808 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:20.803+0000 7f06cb7fe640 1 --2- 172.21.15.89:0/2219579257 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f06cc1082e0 0x7f06cc1a1150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:42:20.808 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:20.803+0000 7f06caffd640 1 --2- 172.21.15.89:0/2219579257 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f06cc102c60 0x7f06cc102100 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f06bc00ea80 tx=0x7f06bc00ef40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:20.808 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:20.804+0000 7f06d099f640 1 -- 172.21.15.89:0/2219579257 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06bc00ce40 con 0x7f06cc102c60 2024-09-21T03:42:20.808 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:20.804+0000 7f06d099f640 1 -- 172.21.15.89:0/2219579257 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f06bc005020 con 0x7f06cc102c60 2024-09-21T03:42:20.809 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:20.804+0000 7f06d19a1640 1 -- 172.21.15.89:0/2219579257 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f06cc1a1c40 con 0x7f06cc102c60 2024-09-21T03:42:20.809 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:20.804+0000 7f06d099f640 1 -- 172.21.15.89:0/2219579257 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06bc0116d0 con 0x7f06cc102c60 2024-09-21T03:42:20.809 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:20.804+0000 7f06d19a1640 1 -- 172.21.15.89:0/2219579257 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06cc1a2140 con 0x7f06cc102c60 2024-09-21T03:42:20.809 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:20.805+0000 7f06d19a1640 1 -- 172.21.15.89:0/2219579257 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f06cc0ff1b0 con 0x7f06cc102c60 2024-09-21T03:42:20.809 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:20.806+0000 7f06d099f640 1 -- 172.21.15.89:0/2219579257 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 21) ==== 154512+0+0 (secure 0 0 0) 0x7f06bc004a40 con 0x7f06cc102c60 2024-09-21T03:42:20.809 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:20.807+0000 7f06d099f640 1 --2- 172.21.15.89:0/2219579257 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f06b80b6510 0x7f06b80b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:20.809 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:20.807+0000 7f06ca7fc640 1 --2- 172.21.15.89:0/2219579257 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f06b80b6510 0x7f06b80b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:20.810 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:20.807+0000 7f06d099f640 1 -- 172.21.15.89:0/2219579257 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 9121+0+0 (secure 0 0 0) 0x7f06bc0e7c50 con 0x7f06cc102c60 2024-09-21T03:42:20.810 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:20.808+0000 7f06ca7fc640 1 --2- 172.21.15.89:0/2219579257 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f06b80b6510 0x7f06b80b8a20 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f06b40057c0 tx=0x7f06b4000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:20.813 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:20.811+0000 7f06d099f640 1 -- 172.21.15.89:0/2219579257 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f06bc0ed020 con 0x7f06cc102c60 2024-09-21T03:42:20.996 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:20.994+0000 7f06d19a1640 1 -- 172.21.15.89:0/2219579257 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f06cc063ff0 con 0x7f06cc102c60 2024-09-21T03:42:20.996 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:20.994+0000 7f06d099f640 1 -- 172.21.15.89:0/2219579257 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v84) ==== 93+0+10 (secure 0 0 0) 0x7f06bc016030 con 0x7f06cc102c60 2024-09-21T03:42:20.997 INFO:teuthology.orchestra.run.smithi089.stdout:pg_num: 1 2024-09-21T03:42:20.999 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:20.996+0000 7f06d19a1640 1 -- 172.21.15.89:0/2219579257 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f06b80b6510 msgr2=0x7f06b80b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:20.999 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:20.996+0000 7f06d19a1640 1 --2- 172.21.15.89:0/2219579257 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f06b80b6510 0x7f06b80b8a20 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f06b40057c0 tx=0x7f06b4000bc0 comp rx=0 tx=0).stop 2024-09-21T03:42:20.999 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:20.996+0000 7f06d19a1640 1 -- 172.21.15.89:0/2219579257 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f06cc102c60 msgr2=0x7f06cc102100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:20.999 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:20.996+0000 7f06d19a1640 1 --2- 172.21.15.89:0/2219579257 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f06cc102c60 0x7f06cc102100 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f06bc00ea80 tx=0x7f06bc00ef40 comp rx=0 tx=0).stop 2024-09-21T03:42:20.999 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:20.996+0000 7f06d19a1640 1 -- 172.21.15.89:0/2219579257 shutdown_connections 2024-09-21T03:42:20.999 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:20.996+0000 7f06d19a1640 1 --2- 172.21.15.89:0/2219579257 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f06b80b6510 0x7f06b80b8a20 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:20.999 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:20.996+0000 7f06d19a1640 1 --2- 172.21.15.89:0/2219579257 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f06cc1056c0 0x7f06cc19ec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:20.999 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:20.996+0000 7f06d19a1640 1 --2- 172.21.15.89:0/2219579257 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f06cc1082e0 0x7f06cc1a1150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:20.999 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:20.996+0000 7f06d19a1640 1 --2- 172.21.15.89:0/2219579257 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f06cc102c60 0x7f06cc102100 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:20.999 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:20.996+0000 7f06d19a1640 1 -- 172.21.15.89:0/2219579257 >> 172.21.15.89:0/2219579257 conn(0x7f06cc0fd690 msgr2=0x7f06cc103b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:42:21.000 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:20.996+0000 7f06d19a1640 1 -- 172.21.15.89:0/2219579257 shutdown_connections 2024-09-21T03:42:21.000 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:20.996+0000 7f06d19a1640 1 -- 172.21.15.89:0/2219579257 wait complete. 2024-09-21T03:42:21.009 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:42:20 smithi089 ceph-mon[30928]: pgmap v448: 129 pgs: 129 active+clean; 1.1 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 770 KiB/s wr, 178 op/s 2024-09-21T03:42:21.009 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:42:20 smithi089 ceph-mon[30928]: from='client.? 172.21.15.89:0/953536057' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T03:42:21.010 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-09-21T03:42:21.125 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:21.120+0000 7fe884913640 1 -- 172.21.15.89:0/3593569783 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe880103620 msgr2=0x7fe880103a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:21.125 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:21.120+0000 7fe884913640 1 --2- 172.21.15.89:0/3593569783 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe880103620 0x7fe880103a30 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7fe870009900 tx=0x7fe87002a410 comp rx=0 tx=0).stop 2024-09-21T03:42:21.126 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:21.120+0000 7fe884913640 1 -- 172.21.15.89:0/3593569783 shutdown_connections 2024-09-21T03:42:21.126 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:21.120+0000 7fe884913640 1 --2- 172.21.15.89:0/3593569783 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe880103fc0 0x7fe88010a960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:21.126 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:21.120+0000 7fe884913640 1 --2- 172.21.15.89:0/3593569783 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe880103620 0x7fe880103a30 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:21.126 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:21.120+0000 7fe884913640 1 --2- 172.21.15.89:0/3593569783 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe8801025a0 0x7fe880102a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:21.126 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:21.120+0000 7fe884913640 1 -- 172.21.15.89:0/3593569783 >> 172.21.15.89:0/3593569783 conn(0x7fe8800fdee0 msgr2=0x7fe880100390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:42:21.126 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:21.120+0000 7fe884913640 1 -- 172.21.15.89:0/3593569783 shutdown_connections 2024-09-21T03:42:21.126 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:21.120+0000 7fe884913640 1 -- 172.21.15.89:0/3593569783 wait complete. 2024-09-21T03:42:21.126 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:21.121+0000 7fe884913640 1 Processor -- start 2024-09-21T03:42:21.126 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:21.121+0000 7fe884913640 1 -- start start 2024-09-21T03:42:21.126 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:21.121+0000 7fe884913640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe8801025a0 0x7fe880199e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:21.126 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:21.121+0000 7fe884913640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe880103620 0x7fe88019a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:21.127 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:21.121+0000 7fe884913640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe880103fc0 0x7fe88019b080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:21.127 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:21.121+0000 7fe884913640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fe88010f1e0 con 0x7fe880103fc0 2024-09-21T03:42:21.127 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:21.121+0000 7fe884913640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fe88010f060 con 0x7fe8801025a0 2024-09-21T03:42:21.127 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:21.121+0000 7fe884913640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fe88010f360 con 0x7fe880103620 2024-09-21T03:42:21.127 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:21.121+0000 7fe87ed9d640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe880103fc0 0x7fe88019b080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:21.127 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:21.121+0000 7fe87ed9d640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe880103fc0 0x7fe88019b080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:53248/0 (socket says 172.21.15.89:53248) 2024-09-21T03:42:21.127 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:21.121+0000 7fe87ed9d640 1 -- 172.21.15.89:0/2534929872 learned_addr learned my addr 172.21.15.89:0/2534929872 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-21T03:42:21.127 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:21.121+0000 7fe87e59c640 1 --2- 172.21.15.89:0/2534929872 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe8801025a0 0x7fe880199e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:21.127 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:21.121+0000 7fe87dd9b640 1 --2- 172.21.15.89:0/2534929872 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe880103620 0x7fe88019a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:21.127 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:21.121+0000 7fe87ed9d640 1 -- 172.21.15.89:0/2534929872 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe880103620 msgr2=0x7fe88019a4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:21.128 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:21.121+0000 7fe87ed9d640 1 --2- 172.21.15.89:0/2534929872 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe880103620 0x7fe88019a4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:21.128 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:21.122+0000 7fe87ed9d640 1 -- 172.21.15.89:0/2534929872 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe8801025a0 msgr2=0x7fe880199e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:21.128 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:21.122+0000 7fe87ed9d640 1 --2- 172.21.15.89:0/2534929872 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe8801025a0 0x7fe880199e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:21.128 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:21.122+0000 7fe87ed9d640 1 -- 172.21.15.89:0/2534929872 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe88019b8a0 con 0x7fe880103fc0 2024-09-21T03:42:21.128 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:21.122+0000 7fe87dd9b640 1 --2- 172.21.15.89:0/2534929872 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe880103620 0x7fe88019a4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:42:21.128 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:21.122+0000 7fe87e59c640 1 --2- 172.21.15.89:0/2534929872 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe8801025a0 0x7fe880199e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:42:21.128 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:21.122+0000 7fe87ed9d640 1 --2- 172.21.15.89:0/2534929872 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe880103fc0 0x7fe88019b080 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7fe87800eaa0 tx=0x7fe87800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:21.128 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:21.122+0000 7fe8777fe640 1 -- 172.21.15.89:0/2534929872 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe87800cdb0 con 0x7fe880103fc0 2024-09-21T03:42:21.128 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:21.122+0000 7fe8777fe640 1 -- 172.21.15.89:0/2534929872 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe878005020 con 0x7fe880103fc0 2024-09-21T03:42:21.128 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:21.122+0000 7fe884913640 1 -- 172.21.15.89:0/2534929872 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe88019bb90 con 0x7fe880103fc0 2024-09-21T03:42:21.128 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:21.122+0000 7fe8777fe640 1 -- 172.21.15.89:0/2534929872 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8780116d0 con 0x7fe880103fc0 2024-09-21T03:42:21.128 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:21.122+0000 7fe884913640 1 -- 172.21.15.89:0/2534929872 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8801a2280 con 0x7fe880103fc0 2024-09-21T03:42:21.128 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:21.123+0000 7fe884913640 1 -- 172.21.15.89:0/2534929872 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe880102a10 con 0x7fe880103fc0 2024-09-21T03:42:21.129 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:21.124+0000 7fe8777fe640 1 -- 172.21.15.89:0/2534929872 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 21) ==== 154512+0+0 (secure 0 0 0) 0x7fe8780051d0 con 0x7fe880103fc0 2024-09-21T03:42:21.129 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:21.125+0000 7fe8777fe640 1 --2- 172.21.15.89:0/2534929872 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7fe86c0b6510 0x7fe86c0b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:21.129 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:21.125+0000 7fe87e59c640 1 --2- 172.21.15.89:0/2534929872 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7fe86c0b6510 0x7fe86c0b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:21.129 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:21.125+0000 7fe8777fe640 1 -- 172.21.15.89:0/2534929872 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 9121+0+0 (secure 0 0 0) 0x7fe8780e7cc0 con 0x7fe880103fc0 2024-09-21T03:42:21.129 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:21.126+0000 7fe87e59c640 1 --2- 172.21.15.89:0/2534929872 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7fe86c0b6510 0x7fe86c0b8a20 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fe868009c20 tx=0x7fe868005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:21.130 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:21.128+0000 7fe8777fe640 1 -- 172.21.15.89:0/2534929872 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fe878016030 con 0x7fe880103fc0 2024-09-21T03:42:21.211 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:42:20 smithi114 ceph-mon[34140]: pgmap v448: 129 pgs: 129 active+clean; 1.1 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 770 KiB/s wr, 178 op/s 2024-09-21T03:42:21.211 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:42:20 smithi114 ceph-mon[34140]: from='client.? 172.21.15.89:0/953536057' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T03:42:21.250 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:42:20 smithi110 ceph-mon[34000]: pgmap v448: 129 pgs: 129 active+clean; 1.1 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 770 KiB/s wr, 178 op/s 2024-09-21T03:42:21.250 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:42:20 smithi110 ceph-mon[34000]: from='client.? 172.21.15.89:0/953536057' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T03:42:21.307 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:21.304+0000 7fe884913640 1 -- 172.21.15.89:0/2534929872 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fe88002e640 con 0x7fe880103fc0 2024-09-21T03:42:21.307 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:21.305+0000 7fe8777fe640 1 -- 172.21.15.89:0/2534929872 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v84) ==== 104+0+11 (secure 0 0 0) 0x7fe8780ae1d0 con 0x7fe880103fc0 2024-09-21T03:42:21.307 INFO:teuthology.orchestra.run.smithi089.stdout:pg_num: 32 2024-09-21T03:42:21.310 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:21.307+0000 7fe884913640 1 -- 172.21.15.89:0/2534929872 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7fe86c0b6510 msgr2=0x7fe86c0b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:21.310 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:21.307+0000 7fe884913640 1 --2- 172.21.15.89:0/2534929872 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7fe86c0b6510 0x7fe86c0b8a20 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fe868009c20 tx=0x7fe868005f90 comp rx=0 tx=0).stop 2024-09-21T03:42:21.310 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:21.307+0000 7fe884913640 1 -- 172.21.15.89:0/2534929872 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe880103fc0 msgr2=0x7fe88019b080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:21.310 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:21.307+0000 7fe884913640 1 --2- 172.21.15.89:0/2534929872 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe880103fc0 0x7fe88019b080 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7fe87800eaa0 tx=0x7fe87800ef60 comp rx=0 tx=0).stop 2024-09-21T03:42:21.310 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:21.307+0000 7fe884913640 1 -- 172.21.15.89:0/2534929872 shutdown_connections 2024-09-21T03:42:21.311 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:21.307+0000 7fe884913640 1 --2- 172.21.15.89:0/2534929872 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7fe86c0b6510 0x7fe86c0b8a20 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:21.311 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:21.307+0000 7fe884913640 1 --2- 172.21.15.89:0/2534929872 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe880103fc0 0x7fe88019b080 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:21.311 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:21.307+0000 7fe884913640 1 --2- 172.21.15.89:0/2534929872 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe880103620 0x7fe88019a4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:21.311 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:21.307+0000 7fe884913640 1 --2- 172.21.15.89:0/2534929872 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe8801025a0 0x7fe880199e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:21.311 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:21.307+0000 7fe884913640 1 -- 172.21.15.89:0/2534929872 >> 172.21.15.89:0/2534929872 conn(0x7fe8800fdee0 msgr2=0x7fe880106940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:42:21.311 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:21.308+0000 7fe884913640 1 -- 172.21.15.89:0/2534929872 shutdown_connections 2024-09-21T03:42:21.311 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:21.308+0000 7fe884913640 1 -- 172.21.15.89:0/2534929872 wait complete. 2024-09-21T03:42:21.321 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-09-21T03:42:21.436 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:21.431+0000 7f7457663640 1 -- 172.21.15.89:0/606784042 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7450103600 msgr2=0x7f7450103a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:21.436 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:21.431+0000 7f7457663640 1 --2- 172.21.15.89:0/606784042 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7450103600 0x7f7450103a10 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f7440009900 tx=0x7f744002a410 comp rx=0 tx=0).stop 2024-09-21T03:42:21.436 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:21.431+0000 7f7457663640 1 -- 172.21.15.89:0/606784042 shutdown_connections 2024-09-21T03:42:21.436 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:21.431+0000 7f7457663640 1 --2- 172.21.15.89:0/606784042 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7450103fa0 0x7f745010a940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:21.436 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:21.431+0000 7f7457663640 1 --2- 172.21.15.89:0/606784042 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7450103600 0x7f7450103a10 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:21.436 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:21.431+0000 7f7457663640 1 --2- 172.21.15.89:0/606784042 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7450102580 0x7f74501029f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:21.436 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:21.431+0000 7f7457663640 1 -- 172.21.15.89:0/606784042 >> 172.21.15.89:0/606784042 conn(0x7f74500fdee0 msgr2=0x7f7450100370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:42:21.437 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:21.431+0000 7f7457663640 1 -- 172.21.15.89:0/606784042 shutdown_connections 2024-09-21T03:42:21.437 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:21.431+0000 7f7457663640 1 -- 172.21.15.89:0/606784042 wait complete. 2024-09-21T03:42:21.437 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:21.432+0000 7f7457663640 1 Processor -- start 2024-09-21T03:42:21.437 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:21.432+0000 7f7457663640 1 -- start start 2024-09-21T03:42:21.437 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:21.432+0000 7f7457663640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7450102580 0x7f7450199e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:21.437 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:21.432+0000 7f7457663640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7450103600 0x7f745019a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:21.437 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:21.432+0000 7f7457663640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7450103fa0 0x7f745019b030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:21.437 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:21.432+0000 7f7457663640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f745010f1c0 con 0x7f7450102580 2024-09-21T03:42:21.437 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:21.432+0000 7f7457663640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f745010f040 con 0x7f7450103fa0 2024-09-21T03:42:21.437 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:21.432+0000 7f7457663640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f745010f340 con 0x7f7450103600 2024-09-21T03:42:21.437 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:21.432+0000 7f74553ff640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7450102580 0x7f7450199e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:21.437 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:21.432+0000 7f74553ff640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7450102580 0x7f7450199e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:53266/0 (socket says 172.21.15.89:53266) 2024-09-21T03:42:21.437 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:21.432+0000 7f74553ff640 1 -- 172.21.15.89:0/2103975775 learned_addr learned my addr 172.21.15.89:0/2103975775 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-21T03:42:21.437 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:21.433+0000 7f7454bfe640 1 --2- 172.21.15.89:0/2103975775 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7450103600 0x7f745019a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:21.437 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:21.433+0000 7f74553ff640 1 -- 172.21.15.89:0/2103975775 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7450103600 msgr2=0x7f745019a480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:21.438 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:21.433+0000 7f74553ff640 1 --2- 172.21.15.89:0/2103975775 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7450103600 0x7f745019a480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:21.438 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:21.433+0000 7f74553ff640 1 -- 172.21.15.89:0/2103975775 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7450103fa0 msgr2=0x7f745019b030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:21.438 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:21.433+0000 7f7455c00640 1 --2- 172.21.15.89:0/2103975775 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7450103fa0 0x7f745019b030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:21.438 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:21.433+0000 7f74553ff640 1 --2- 172.21.15.89:0/2103975775 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7450103fa0 0x7f745019b030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:21.438 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:21.433+0000 7f74553ff640 1 -- 172.21.15.89:0/2103975775 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f745019b850 con 0x7f7450102580 2024-09-21T03:42:21.438 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:21.433+0000 7f74553ff640 1 --2- 172.21.15.89:0/2103975775 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7450102580 0x7f7450199e10 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f744c00db80 tx=0x7f744c00b630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:21.438 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:21.433+0000 7f74467fc640 1 -- 172.21.15.89:0/2103975775 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f744c017070 con 0x7f7450102580 2024-09-21T03:42:21.438 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:21.433+0000 7f74467fc640 1 -- 172.21.15.89:0/2103975775 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f744c004020 con 0x7f7450102580 2024-09-21T03:42:21.438 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:21.433+0000 7f7457663640 1 -- 172.21.15.89:0/2103975775 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f745019bb40 con 0x7f7450102580 2024-09-21T03:42:21.438 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:21.433+0000 7f74467fc640 1 -- 172.21.15.89:0/2103975775 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f744c00be60 con 0x7f7450102580 2024-09-21T03:42:21.438 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:21.433+0000 7f7457663640 1 -- 172.21.15.89:0/2103975775 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f74501a2230 con 0x7f7450102580 2024-09-21T03:42:21.439 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:21.434+0000 7f7457663640 1 -- 172.21.15.89:0/2103975775 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7450109750 con 0x7f7450102580 2024-09-21T03:42:21.439 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:21.436+0000 7f74467fc640 1 -- 172.21.15.89:0/2103975775 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 21) ==== 154512+0+0 (secure 0 0 0) 0x7f744c003b70 con 0x7f7450102580 2024-09-21T03:42:21.439 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:21.436+0000 7f74467fc640 1 --2- 172.21.15.89:0/2103975775 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f743c0b6590 0x7f743c0b8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:21.439 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:21.437+0000 7f7454bfe640 1 --2- 172.21.15.89:0/2103975775 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f743c0b6590 0x7f743c0b8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:21.440 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:21.437+0000 7f74467fc640 1 -- 172.21.15.89:0/2103975775 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 9121+0+0 (secure 0 0 0) 0x7f744c0e7b90 con 0x7f7450102580 2024-09-21T03:42:21.440 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:21.437+0000 7f7454bfe640 1 --2- 172.21.15.89:0/2103975775 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f743c0b6590 0x7f743c0b8aa0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f7440000c00 tx=0x7f7440005720 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:21.442 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:21.439+0000 7f74467fc640 1 -- 172.21.15.89:0/2103975775 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f744c0ae120 con 0x7f7450102580 2024-09-21T03:42:21.621 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:21.619+0000 7f7457663640 1 -- 172.21.15.89:0/2103975775 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f7450063fc0 con 0x7f7450102580 2024-09-21T03:42:21.621 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:21.619+0000 7f74467fc640 1 -- 172.21.15.89:0/2103975775 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v84) ==== 100+0+11 (secure 0 0 0) 0x7f744c0b2500 con 0x7f7450102580 2024-09-21T03:42:21.622 INFO:teuthology.orchestra.run.smithi089.stdout:pg_num: 64 2024-09-21T03:42:21.623 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:21.621+0000 7f7457663640 1 -- 172.21.15.89:0/2103975775 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f743c0b6590 msgr2=0x7f743c0b8aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:21.624 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:21.621+0000 7f7457663640 1 --2- 172.21.15.89:0/2103975775 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f743c0b6590 0x7f743c0b8aa0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f7440000c00 tx=0x7f7440005720 comp rx=0 tx=0).stop 2024-09-21T03:42:21.624 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:21.621+0000 7f7457663640 1 -- 172.21.15.89:0/2103975775 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7450102580 msgr2=0x7f7450199e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:21.624 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:21.621+0000 7f7457663640 1 --2- 172.21.15.89:0/2103975775 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7450102580 0x7f7450199e10 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f744c00db80 tx=0x7f744c00b630 comp rx=0 tx=0).stop 2024-09-21T03:42:21.624 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:21.621+0000 7f7457663640 1 -- 172.21.15.89:0/2103975775 shutdown_connections 2024-09-21T03:42:21.624 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:21.621+0000 7f7457663640 1 --2- 172.21.15.89:0/2103975775 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f743c0b6590 0x7f743c0b8aa0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:21.624 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:21.621+0000 7f7457663640 1 --2- 172.21.15.89:0/2103975775 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7450103fa0 0x7f745019b030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:21.624 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:21.621+0000 7f7457663640 1 --2- 172.21.15.89:0/2103975775 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7450103600 0x7f745019a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:21.624 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:21.621+0000 7f7457663640 1 --2- 172.21.15.89:0/2103975775 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7450102580 0x7f7450199e10 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:21.624 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:21.621+0000 7f7457663640 1 -- 172.21.15.89:0/2103975775 >> 172.21.15.89:0/2103975775 conn(0x7f74500fdee0 msgr2=0x7f7450106920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:42:21.624 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:21.621+0000 7f7457663640 1 -- 172.21.15.89:0/2103975775 shutdown_connections 2024-09-21T03:42:21.624 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:21.621+0000 7f7457663640 1 -- 172.21.15.89:0/2103975775 wait complete. 2024-09-21T03:42:21.635 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-09-21T03:42:21.747 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:21.745+0000 7fe79ce4d640 1 -- 172.21.15.89:0/249485751 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe798103600 msgr2=0x7fe798103a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:21.748 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:21.745+0000 7fe79ce4d640 1 --2- 172.21.15.89:0/249485751 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe798103600 0x7fe798103a10 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7fe780009900 tx=0x7fe78002a410 comp rx=0 tx=0).stop 2024-09-21T03:42:21.748 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:21.745+0000 7fe79ce4d640 1 -- 172.21.15.89:0/249485751 shutdown_connections 2024-09-21T03:42:21.748 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:21.745+0000 7fe79ce4d640 1 --2- 172.21.15.89:0/249485751 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe798103fa0 0x7fe79810a940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:21.748 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:21.745+0000 7fe79ce4d640 1 --2- 172.21.15.89:0/249485751 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe798103600 0x7fe798103a10 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:21.748 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:21.745+0000 7fe79ce4d640 1 --2- 172.21.15.89:0/249485751 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe798102580 0x7fe7981029f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:21.748 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:21.745+0000 7fe79ce4d640 1 -- 172.21.15.89:0/249485751 >> 172.21.15.89:0/249485751 conn(0x7fe7980fdee0 msgr2=0x7fe798100370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:42:21.748 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:21.745+0000 7fe79ce4d640 1 -- 172.21.15.89:0/249485751 shutdown_connections 2024-09-21T03:42:21.748 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:21.745+0000 7fe79ce4d640 1 -- 172.21.15.89:0/249485751 wait complete. 2024-09-21T03:42:21.750 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:21.746+0000 7fe79ce4d640 1 Processor -- start 2024-09-21T03:42:21.750 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:21.746+0000 7fe79ce4d640 1 -- start start 2024-09-21T03:42:21.750 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:21.746+0000 7fe79ce4d640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe798102580 0x7fe798197c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:21.751 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:21.746+0000 7fe79ce4d640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe798103600 0x7fe798198270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:21.751 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:21.746+0000 7fe79ce4d640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe798103fa0 0x7fe798198e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:21.751 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:21.746+0000 7fe79ce4d640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fe79810cfb0 con 0x7fe798102580 2024-09-21T03:42:21.751 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:21.746+0000 7fe79ce4d640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fe79810ce30 con 0x7fe798103600 2024-09-21T03:42:21.751 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:21.746+0000 7fe79ce4d640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fe79810d130 con 0x7fe798103fa0 2024-09-21T03:42:21.751 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:21.746+0000 7fe79659c640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe798102580 0x7fe798197c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:21.751 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:21.746+0000 7fe79659c640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe798102580 0x7fe798197c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:53276/0 (socket says 172.21.15.89:53276) 2024-09-21T03:42:21.751 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:21.746+0000 7fe79659c640 1 -- 172.21.15.89:0/1527658912 learned_addr learned my addr 172.21.15.89:0/1527658912 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-21T03:42:21.751 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:21.746+0000 7fe796d9d640 1 --2- 172.21.15.89:0/1527658912 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe798103fa0 0x7fe798198e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:21.751 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:21.746+0000 7fe795d9b640 1 --2- 172.21.15.89:0/1527658912 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe798103600 0x7fe798198270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:21.751 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:21.746+0000 7fe79659c640 1 -- 172.21.15.89:0/1527658912 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe798103fa0 msgr2=0x7fe798198e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:21.751 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:21.746+0000 7fe79659c640 1 --2- 172.21.15.89:0/1527658912 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe798103fa0 0x7fe798198e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:21.752 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:21.746+0000 7fe79659c640 1 -- 172.21.15.89:0/1527658912 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe798103600 msgr2=0x7fe798198270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:21.752 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:21.746+0000 7fe79659c640 1 --2- 172.21.15.89:0/1527658912 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe798103600 0x7fe798198270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:21.752 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:21.746+0000 7fe79659c640 1 -- 172.21.15.89:0/1527658912 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe798199640 con 0x7fe798102580 2024-09-21T03:42:21.752 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:21.746+0000 7fe796d9d640 1 --2- 172.21.15.89:0/1527658912 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe798103fa0 0x7fe798198e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:42:21.752 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:21.746+0000 7fe795d9b640 1 --2- 172.21.15.89:0/1527658912 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe798103600 0x7fe798198270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:42:21.752 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:21.747+0000 7fe79659c640 1 --2- 172.21.15.89:0/1527658912 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe798102580 0x7fe798197c00 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7fe79000db80 tx=0x7fe79000b630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:21.752 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:21.747+0000 7fe78f7fe640 1 -- 172.21.15.89:0/1527658912 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe790017070 con 0x7fe798102580 2024-09-21T03:42:21.752 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:21.747+0000 7fe78f7fe640 1 -- 172.21.15.89:0/1527658912 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe790004020 con 0x7fe798102580 2024-09-21T03:42:21.752 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:21.747+0000 7fe78f7fe640 1 -- 172.21.15.89:0/1527658912 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe79000be60 con 0x7fe798102580 2024-09-21T03:42:21.752 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:21.747+0000 7fe79ce4d640 1 -- 172.21.15.89:0/1527658912 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe798199930 con 0x7fe798102580 2024-09-21T03:42:21.752 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:21.747+0000 7fe79ce4d640 1 -- 172.21.15.89:0/1527658912 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7981a0020 con 0x7fe798102580 2024-09-21T03:42:21.752 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:21.748+0000 7fe79ce4d640 1 -- 172.21.15.89:0/1527658912 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe798109750 con 0x7fe798102580 2024-09-21T03:42:21.753 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:21.749+0000 7fe78f7fe640 1 -- 172.21.15.89:0/1527658912 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 21) ==== 154512+0+0 (secure 0 0 0) 0x7fe790003b70 con 0x7fe798102580 2024-09-21T03:42:21.753 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:21.750+0000 7fe78f7fe640 1 --2- 172.21.15.89:0/1527658912 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7fe7840b87a0 0x7fe7840bacb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:21.753 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:21.750+0000 7fe78f7fe640 1 -- 172.21.15.89:0/1527658912 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 9121+0+0 (secure 0 0 0) 0x7fe7900e7b90 con 0x7fe798102580 2024-09-21T03:42:21.753 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:21.750+0000 7fe795d9b640 1 --2- 172.21.15.89:0/1527658912 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7fe7840b87a0 0x7fe7840bacb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:21.753 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:21.751+0000 7fe795d9b640 1 --2- 172.21.15.89:0/1527658912 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7fe7840b87a0 0x7fe7840bacb0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fe780005f70 tx=0x7fe780000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:21.755 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:21.753+0000 7fe78f7fe640 1 -- 172.21.15.89:0/1527658912 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fe7900ae120 con 0x7fe798102580 2024-09-21T03:42:21.935 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:21.932+0000 7fe79ce4d640 1 -- 172.21.15.89:0/1527658912 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7fe79802e890 con 0x7fe798102580 2024-09-21T03:42:21.935 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:21.933+0000 7fe78f7fe640 1 -- 172.21.15.89:0/1527658912 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v84) ==== 103+0+11 (secure 0 0 0) 0x7fe7900b2500 con 0x7fe798102580 2024-09-21T03:42:21.935 INFO:teuthology.orchestra.run.smithi089.stdout:pg_num: 32 2024-09-21T03:42:21.937 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:21.935+0000 7fe79ce4d640 1 -- 172.21.15.89:0/1527658912 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7fe7840b87a0 msgr2=0x7fe7840bacb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:21.938 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:21.935+0000 7fe79ce4d640 1 --2- 172.21.15.89:0/1527658912 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7fe7840b87a0 0x7fe7840bacb0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fe780005f70 tx=0x7fe780000bc0 comp rx=0 tx=0).stop 2024-09-21T03:42:21.938 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:21.935+0000 7fe79ce4d640 1 -- 172.21.15.89:0/1527658912 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe798102580 msgr2=0x7fe798197c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:21.938 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:21.935+0000 7fe79ce4d640 1 --2- 172.21.15.89:0/1527658912 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe798102580 0x7fe798197c00 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7fe79000db80 tx=0x7fe79000b630 comp rx=0 tx=0).stop 2024-09-21T03:42:21.938 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:21.935+0000 7fe79ce4d640 1 -- 172.21.15.89:0/1527658912 shutdown_connections 2024-09-21T03:42:21.938 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:21.935+0000 7fe79ce4d640 1 --2- 172.21.15.89:0/1527658912 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7fe7840b87a0 0x7fe7840bacb0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:21.938 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:21.935+0000 7fe79ce4d640 1 --2- 172.21.15.89:0/1527658912 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe798103600 0x7fe798198270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:21.938 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:21.935+0000 7fe79ce4d640 1 --2- 172.21.15.89:0/1527658912 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe798103fa0 0x7fe798198e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:21.938 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:21.935+0000 7fe79ce4d640 1 --2- 172.21.15.89:0/1527658912 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe798102580 0x7fe798197c00 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:21.938 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:21.935+0000 7fe79ce4d640 1 -- 172.21.15.89:0/1527658912 >> 172.21.15.89:0/1527658912 conn(0x7fe7980fdee0 msgr2=0x7fe798106920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:42:21.938 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:21.935+0000 7fe79ce4d640 1 -- 172.21.15.89:0/1527658912 shutdown_connections 2024-09-21T03:42:21.939 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:21.935+0000 7fe79ce4d640 1 -- 172.21.15.89:0/1527658912 wait complete. 2024-09-21T03:42:21.949 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json-pretty 2024-09-21T03:42:22.061 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:22.055+0000 7fad25951640 1 -- 172.21.15.89:0/41584723 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fad20103fa0 msgr2=0x7fad2010a940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:22.061 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:22.055+0000 7fad25951640 1 --2- 172.21.15.89:0/41584723 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fad20103fa0 0x7fad2010a940 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7fad18009ca0 tx=0x7fad1802a8c0 comp rx=0 tx=0).stop 2024-09-21T03:42:22.062 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:22.055+0000 7fad25951640 1 -- 172.21.15.89:0/41584723 shutdown_connections 2024-09-21T03:42:22.062 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:22.055+0000 7fad25951640 1 --2- 172.21.15.89:0/41584723 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fad20103fa0 0x7fad2010a940 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:22.062 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:22.055+0000 7fad25951640 1 --2- 172.21.15.89:0/41584723 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fad20103600 0x7fad20103a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:22.062 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:22.055+0000 7fad25951640 1 --2- 172.21.15.89:0/41584723 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fad20102580 0x7fad201029f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:22.062 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:22.055+0000 7fad25951640 1 -- 172.21.15.89:0/41584723 >> 172.21.15.89:0/41584723 conn(0x7fad200fdee0 msgr2=0x7fad20100370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:42:22.063 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:22.055+0000 7fad25951640 1 -- 172.21.15.89:0/41584723 shutdown_connections 2024-09-21T03:42:22.063 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:22.055+0000 7fad25951640 1 -- 172.21.15.89:0/41584723 wait complete. 2024-09-21T03:42:22.063 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:22.056+0000 7fad25951640 1 Processor -- start 2024-09-21T03:42:22.063 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:22.056+0000 7fad25951640 1 -- start start 2024-09-21T03:42:22.063 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:22.056+0000 7fad25951640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fad20102580 0x7fad20199e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:22.063 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:22.056+0000 7fad25951640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fad20103600 0x7fad2019a490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:22.063 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:22.056+0000 7fad25951640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fad20103fa0 0x7fad2019afd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:22.063 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:22.056+0000 7fad25951640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fad2010e010 con 0x7fad20102580 2024-09-21T03:42:22.064 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:22.056+0000 7fad25951640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fad2010de90 con 0x7fad20103600 2024-09-21T03:42:22.064 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:22.056+0000 7fad25951640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fad2010e190 con 0x7fad20103fa0 2024-09-21T03:42:22.064 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:22.056+0000 7fad1effd640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fad20102580 0x7fad20199e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:22.064 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:22.056+0000 7fad1effd640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fad20102580 0x7fad20199e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:53296/0 (socket says 172.21.15.89:53296) 2024-09-21T03:42:22.064 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:22.056+0000 7fad1effd640 1 -- 172.21.15.89:0/998049441 learned_addr learned my addr 172.21.15.89:0/998049441 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-21T03:42:22.064 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:22.057+0000 7fad1e7fc640 1 --2- 172.21.15.89:0/998049441 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fad20103600 0x7fad2019a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:22.064 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:22.057+0000 7fad1effd640 1 -- 172.21.15.89:0/998049441 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fad20103fa0 msgr2=0x7fad2019afd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:22.064 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:22.057+0000 7fad1f7fe640 1 --2- 172.21.15.89:0/998049441 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fad20103fa0 0x7fad2019afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:22.064 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:22.057+0000 7fad1effd640 1 --2- 172.21.15.89:0/998049441 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fad20103fa0 0x7fad2019afd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:22.065 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:22.057+0000 7fad1effd640 1 -- 172.21.15.89:0/998049441 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fad20103600 msgr2=0x7fad2019a490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:22.065 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:22.057+0000 7fad1effd640 1 --2- 172.21.15.89:0/998049441 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fad20103600 0x7fad2019a490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:22.065 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:22.057+0000 7fad1effd640 1 -- 172.21.15.89:0/998049441 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad2019b760 con 0x7fad20102580 2024-09-21T03:42:22.065 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:22.057+0000 7fad1effd640 1 --2- 172.21.15.89:0/998049441 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fad20102580 0x7fad20199e20 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7fad1000e9f0 tx=0x7fad1000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:22.065 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:22.057+0000 7fad2494f640 1 -- 172.21.15.89:0/998049441 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad1000ccc0 con 0x7fad20102580 2024-09-21T03:42:22.066 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:22.057+0000 7fad2494f640 1 -- 172.21.15.89:0/998049441 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fad100108c0 con 0x7fad20102580 2024-09-21T03:42:22.066 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:22.057+0000 7fad25951640 1 -- 172.21.15.89:0/998049441 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fad2019ba50 con 0x7fad20102580 2024-09-21T03:42:22.066 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:22.057+0000 7fad2494f640 1 -- 172.21.15.89:0/998049441 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad10018750 con 0x7fad20102580 2024-09-21T03:42:22.066 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:22.057+0000 7fad25951640 1 -- 172.21.15.89:0/998049441 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fad201a21d0 con 0x7fad20102580 2024-09-21T03:42:22.066 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:22.058+0000 7fad25951640 1 -- 172.21.15.89:0/998049441 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fad201029f0 con 0x7fad20102580 2024-09-21T03:42:22.067 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:22.060+0000 7fad2494f640 1 -- 172.21.15.89:0/998049441 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 21) ==== 154512+0+0 (secure 0 0 0) 0x7fad10010430 con 0x7fad20102580 2024-09-21T03:42:22.067 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:22.060+0000 7fad2494f640 1 --2- 172.21.15.89:0/998049441 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7fad080b6510 0x7fad080b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:22.067 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:22.060+0000 7fad1e7fc640 1 --2- 172.21.15.89:0/998049441 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7fad080b6510 0x7fad080b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:22.067 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:22.060+0000 7fad2494f640 1 -- 172.21.15.89:0/998049441 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 9121+0+0 (secure 0 0 0) 0x7fad100e7af0 con 0x7fad20102580 2024-09-21T03:42:22.068 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:22.061+0000 7fad1e7fc640 1 --2- 172.21.15.89:0/998049441 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7fad080b6510 0x7fad080b8a20 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fad1400abe0 tx=0x7fad1400a2d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:22.068 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:22.064+0000 7fad2494f640 1 -- 172.21.15.89:0/998049441 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fad100ae000 con 0x7fad20102580 2024-09-21T03:42:22.211 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:42:21 smithi114 ceph-mon[34140]: from='client.? 172.21.15.89:0/2219579257' entity='client.admin' cmd={"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} : dispatch 2024-09-21T03:42:22.211 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:42:21 smithi114 ceph-mon[34140]: from='client.? 172.21.15.89:0/2534929872' entity='client.admin' cmd={"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} : dispatch 2024-09-21T03:42:22.211 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:42:21 smithi114 ceph-mon[34140]: from='client.? 172.21.15.89:0/2103975775' entity='client.admin' cmd={"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} : dispatch 2024-09-21T03:42:22.250 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:42:21 smithi110 ceph-mon[34000]: from='client.? 172.21.15.89:0/2219579257' entity='client.admin' cmd={"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} : dispatch 2024-09-21T03:42:22.250 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:42:21 smithi110 ceph-mon[34000]: from='client.? 172.21.15.89:0/2534929872' entity='client.admin' cmd={"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} : dispatch 2024-09-21T03:42:22.250 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:42:21 smithi110 ceph-mon[34000]: from='client.? 172.21.15.89:0/2103975775' entity='client.admin' cmd={"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} : dispatch 2024-09-21T03:42:22.293 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:22.290+0000 7fad25951640 1 -- 172.21.15.89:0/998049441 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json-pretty"} v 0) -- 0x7fad201a2960 con 0x7fad20102580 2024-09-21T03:42:22.301 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:22.298+0000 7fad2494f640 1 -- 172.21.15.89:0/998049441 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json-pretty"}]=0 v21) ==== 81+0+833523 (secure 0 0 0) 0x7fad100b23e0 con 0x7fad20102580 2024-09-21T03:42:22.310 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-09-21T03:42:22.310 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-09-21T03:42:22.310 INFO:teuthology.orchestra.run.smithi089.stdout: "epoch": 21, 2024-09-21T03:42:22.310 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:22.310 INFO:teuthology.orchestra.run.smithi089.stdout: "active_gid": 14152, 2024-09-21T03:42:22.311 INFO:teuthology.orchestra.run.smithi089.stdout: "active_name": "x", 2024-09-21T03:42:22.311 INFO:teuthology.orchestra.run.smithi089.stdout: "active_addrs": { 2024-09-21T03:42:22.311 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-09-21T03:42:22.311 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-21T03:42:22.311 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-09-21T03:42:22.311 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6800", 2024-09-21T03:42:22.311 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 1591532358 2024-09-21T03:42:22.311 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.311 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-21T03:42:22.311 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-09-21T03:42:22.311 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6801", 2024-09-21T03:42:22.311 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 1591532358 2024-09-21T03:42:22.311 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-21T03:42:22.311 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-21T03:42:22.312 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.312 INFO:teuthology.orchestra.run.smithi089.stdout: "active_addr": "172.21.15.89:6801/1591532358", 2024-09-21T03:42:22.312 INFO:teuthology.orchestra.run.smithi089.stdout: "active_change": "2024-09-21T03:29:51.519235+0000", 2024-09-21T03:42:22.312 INFO:teuthology.orchestra.run.smithi089.stdout: "active_mgr_features": 4540719139924082687, 2024-09-21T03:42:22.312 INFO:teuthology.orchestra.run.smithi089.stdout: "available": true, 2024-09-21T03:42:22.312 INFO:teuthology.orchestra.run.smithi089.stdout: "standbys": [ 2024-09-21T03:42:22.312 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-21T03:42:22.312 INFO:teuthology.orchestra.run.smithi089.stdout: "gid": 24112, 2024-09-21T03:42:22.312 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "z", 2024-09-21T03:42:22.312 INFO:teuthology.orchestra.run.smithi089.stdout: "mgr_features": 4540719139924082687, 2024-09-21T03:42:22.312 INFO:teuthology.orchestra.run.smithi089.stdout: "available_modules": [ 2024-09-21T03:42:22.312 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-21T03:42:22.312 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "alerts", 2024-09-21T03:42:22.312 INFO:teuthology.orchestra.run.smithi089.stdout: "can_run": true, 2024-09-21T03:42:22.313 INFO:teuthology.orchestra.run.smithi089.stdout: "error_string": "", 2024-09-21T03:42:22.313 INFO:teuthology.orchestra.run.smithi089.stdout: "module_options": { 2024-09-21T03:42:22.313 INFO:teuthology.orchestra.run.smithi089.stdout: "interval": { 2024-09-21T03:42:22.313 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "interval", 2024-09-21T03:42:22.313 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "secs", 2024-09-21T03:42:22.313 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.313 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:22.313 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "60", 2024-09-21T03:42:22.313 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.313 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.313 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.313 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "How frequently to reexamine health status", 2024-09-21T03:42:22.313 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.313 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.313 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.314 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.314 INFO:teuthology.orchestra.run.smithi089.stdout: "log_level": { 2024-09-21T03:42:22.314 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_level", 2024-09-21T03:42:22.314 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:22.314 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.314 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:22.314 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-21T03:42:22.314 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.314 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.314 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-21T03:42:22.314 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2024-09-21T03:42:22.314 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2024-09-21T03:42:22.314 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2024-09-21T03:42:22.314 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2024-09-21T03:42:22.315 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2024-09-21T03:42:22.315 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2024-09-21T03:42:22.315 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-21T03:42:22.315 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:22.315 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.315 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.315 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.315 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.315 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster": { 2024-09-21T03:42:22.315 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster", 2024-09-21T03:42:22.315 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-21T03:42:22.315 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.315 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:22.315 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-21T03:42:22.316 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.316 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.316 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.316 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:22.316 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.316 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.316 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.316 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.316 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster_level": { 2024-09-21T03:42:22.316 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster_level", 2024-09-21T03:42:22.316 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:22.316 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.316 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:22.316 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "info", 2024-09-21T03:42:22.316 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.317 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.317 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-21T03:42:22.317 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2024-09-21T03:42:22.317 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2024-09-21T03:42:22.317 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2024-09-21T03:42:22.317 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2024-09-21T03:42:22.317 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2024-09-21T03:42:22.317 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2024-09-21T03:42:22.317 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-21T03:42:22.317 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:22.317 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.317 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.317 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.317 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.317 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_file": { 2024-09-21T03:42:22.317 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_file", 2024-09-21T03:42:22.318 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-21T03:42:22.318 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.318 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:22.318 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-21T03:42:22.318 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.318 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.318 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.318 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:22.318 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.318 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.318 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.318 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.318 INFO:teuthology.orchestra.run.smithi089.stdout: "smtp_destination": { 2024-09-21T03:42:22.318 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "smtp_destination", 2024-09-21T03:42:22.319 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:22.319 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.319 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:22.319 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-21T03:42:22.319 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.319 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.319 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.319 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Email address to send alerts to, use commas to separate multiple", 2024-09-21T03:42:22.319 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.319 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.319 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.319 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.319 INFO:teuthology.orchestra.run.smithi089.stdout: "smtp_from_name": { 2024-09-21T03:42:22.319 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "smtp_from_name", 2024-09-21T03:42:22.319 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:22.320 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.320 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:22.320 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "Ceph", 2024-09-21T03:42:22.320 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.320 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.320 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.320 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Email From: name", 2024-09-21T03:42:22.320 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.320 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.320 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.320 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.320 INFO:teuthology.orchestra.run.smithi089.stdout: "smtp_host": { 2024-09-21T03:42:22.320 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "smtp_host", 2024-09-21T03:42:22.320 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:22.320 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.321 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:22.321 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-21T03:42:22.321 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.321 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.321 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.321 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "SMTP server", 2024-09-21T03:42:22.321 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.321 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.321 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.321 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.321 INFO:teuthology.orchestra.run.smithi089.stdout: "smtp_password": { 2024-09-21T03:42:22.321 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "smtp_password", 2024-09-21T03:42:22.321 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:22.321 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.321 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:22.321 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-21T03:42:22.322 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.322 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.322 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.322 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Password to authenticate with", 2024-09-21T03:42:22.322 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.322 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.322 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.322 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.322 INFO:teuthology.orchestra.run.smithi089.stdout: "smtp_port": { 2024-09-21T03:42:22.322 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "smtp_port", 2024-09-21T03:42:22.322 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2024-09-21T03:42:22.322 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.322 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:22.322 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "465", 2024-09-21T03:42:22.322 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.323 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.323 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.323 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "SMTP port", 2024-09-21T03:42:22.323 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.323 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.323 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.323 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.323 INFO:teuthology.orchestra.run.smithi089.stdout: "smtp_sender": { 2024-09-21T03:42:22.323 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "smtp_sender", 2024-09-21T03:42:22.323 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:22.323 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.323 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:22.323 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-21T03:42:22.323 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.323 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.323 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.324 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "SMTP envelope sender", 2024-09-21T03:42:22.324 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.324 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.324 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.324 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.324 INFO:teuthology.orchestra.run.smithi089.stdout: "smtp_ssl": { 2024-09-21T03:42:22.324 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "smtp_ssl", 2024-09-21T03:42:22.324 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-21T03:42:22.324 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.324 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:22.324 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "True", 2024-09-21T03:42:22.324 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.324 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.324 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.325 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Use SSL to connect to SMTP server", 2024-09-21T03:42:22.325 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.325 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.325 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.325 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.325 INFO:teuthology.orchestra.run.smithi089.stdout: "smtp_user": { 2024-09-21T03:42:22.325 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "smtp_user", 2024-09-21T03:42:22.325 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:22.325 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.325 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:22.325 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-21T03:42:22.325 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.325 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.325 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.326 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "User to authenticate as", 2024-09-21T03:42:22.326 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.326 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.326 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.326 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.326 INFO:teuthology.orchestra.run.smithi089.stdout: "sqlite3_killpoint": { 2024-09-21T03:42:22.326 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:42:22.326 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2024-09-21T03:42:22.326 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "dev", 2024-09-21T03:42:22.326 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:22.326 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "0", 2024-09-21T03:42:22.326 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.326 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.326 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.327 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:22.327 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.327 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.327 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.327 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-21T03:42:22.327 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-21T03:42:22.327 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.327 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-21T03:42:22.327 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "balancer", 2024-09-21T03:42:22.327 INFO:teuthology.orchestra.run.smithi089.stdout: "can_run": true, 2024-09-21T03:42:22.327 INFO:teuthology.orchestra.run.smithi089.stdout: "error_string": "", 2024-09-21T03:42:22.327 INFO:teuthology.orchestra.run.smithi089.stdout: "module_options": { 2024-09-21T03:42:22.327 INFO:teuthology.orchestra.run.smithi089.stdout: "active": { 2024-09-21T03:42:22.327 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "active", 2024-09-21T03:42:22.328 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-21T03:42:22.328 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.328 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:22.328 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "True", 2024-09-21T03:42:22.328 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.328 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.328 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.328 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "automatically balance PGs across cluster", 2024-09-21T03:42:22.328 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.328 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.328 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.328 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.328 INFO:teuthology.orchestra.run.smithi089.stdout: "begin_time": { 2024-09-21T03:42:22.328 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "begin_time", 2024-09-21T03:42:22.328 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:22.329 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.329 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:22.329 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "0000", 2024-09-21T03:42:22.329 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.329 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.329 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.329 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "beginning time of day to automatically balance", 2024-09-21T03:42:22.329 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-21T03:42:22.329 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.329 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.329 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.329 INFO:teuthology.orchestra.run.smithi089.stdout: "begin_weekday": { 2024-09-21T03:42:22.329 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "begin_weekday", 2024-09-21T03:42:22.329 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "uint", 2024-09-21T03:42:22.329 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.330 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:22.330 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "0", 2024-09-21T03:42:22.330 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "0", 2024-09-21T03:42:22.330 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "6", 2024-09-21T03:42:22.330 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.330 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-09-21T03:42:22.330 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-21T03:42:22.330 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.330 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.330 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.330 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_compat_max_iterations": { 2024-09-21T03:42:22.330 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "crush_compat_max_iterations", 2024-09-21T03:42:22.330 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "uint", 2024-09-21T03:42:22.330 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.331 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:22.331 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "25", 2024-09-21T03:42:22.331 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "1", 2024-09-21T03:42:22.331 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "250", 2024-09-21T03:42:22.331 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.331 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-09-21T03:42:22.331 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.331 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.331 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.331 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.331 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_compat_metrics": { 2024-09-21T03:42:22.331 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "crush_compat_metrics", 2024-09-21T03:42:22.331 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:22.331 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.332 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:22.332 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "pgs,objects,bytes", 2024-09-21T03:42:22.332 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.332 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.332 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.332 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-09-21T03:42:22.332 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-09-21T03:42:22.332 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.332 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.332 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.332 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_compat_step": { 2024-09-21T03:42:22.332 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "crush_compat_step", 2024-09-21T03:42:22.332 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "float", 2024-09-21T03:42:22.332 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.332 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:22.333 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "0.5", 2024-09-21T03:42:22.333 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "0.001", 2024-09-21T03:42:22.333 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "0.999", 2024-09-21T03:42:22.333 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.333 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "aggressiveness of optimization", 2024-09-21T03:42:22.333 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-09-21T03:42:22.333 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.333 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.333 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.333 INFO:teuthology.orchestra.run.smithi089.stdout: "end_time": { 2024-09-21T03:42:22.333 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "end_time", 2024-09-21T03:42:22.333 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:22.333 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.333 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:22.333 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "2359", 2024-09-21T03:42:22.333 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.334 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.334 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.334 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "ending time of day to automatically balance", 2024-09-21T03:42:22.334 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-21T03:42:22.334 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.334 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.334 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.334 INFO:teuthology.orchestra.run.smithi089.stdout: "end_weekday": { 2024-09-21T03:42:22.334 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "end_weekday", 2024-09-21T03:42:22.334 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "uint", 2024-09-21T03:42:22.334 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.334 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:22.334 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "0", 2024-09-21T03:42:22.334 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "0", 2024-09-21T03:42:22.334 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "6", 2024-09-21T03:42:22.334 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.335 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-09-21T03:42:22.335 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-21T03:42:22.335 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.335 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.335 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.335 INFO:teuthology.orchestra.run.smithi089.stdout: "log_level": { 2024-09-21T03:42:22.335 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_level", 2024-09-21T03:42:22.335 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:22.335 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.335 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:22.335 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-21T03:42:22.335 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.335 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.335 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-21T03:42:22.336 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2024-09-21T03:42:22.336 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2024-09-21T03:42:22.336 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2024-09-21T03:42:22.336 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2024-09-21T03:42:22.336 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2024-09-21T03:42:22.336 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2024-09-21T03:42:22.336 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-21T03:42:22.336 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:22.336 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.336 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.336 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.336 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.336 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster": { 2024-09-21T03:42:22.337 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster", 2024-09-21T03:42:22.337 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-21T03:42:22.337 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.337 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:22.337 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-21T03:42:22.337 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.337 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.337 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.337 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:22.337 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.337 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.337 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.337 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.337 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster_level": { 2024-09-21T03:42:22.337 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster_level", 2024-09-21T03:42:22.338 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:22.338 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.338 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:22.338 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "info", 2024-09-21T03:42:22.338 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.338 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.338 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-21T03:42:22.338 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2024-09-21T03:42:22.338 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2024-09-21T03:42:22.338 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2024-09-21T03:42:22.338 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2024-09-21T03:42:22.338 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2024-09-21T03:42:22.338 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2024-09-21T03:42:22.338 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-21T03:42:22.339 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:22.339 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.339 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.339 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.339 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.339 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_file": { 2024-09-21T03:42:22.339 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_file", 2024-09-21T03:42:22.339 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-21T03:42:22.339 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.339 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:22.339 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-21T03:42:22.339 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.339 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.339 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.340 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:22.340 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.340 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.340 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.340 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.340 INFO:teuthology.orchestra.run.smithi089.stdout: "min_score": { 2024-09-21T03:42:22.340 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "min_score", 2024-09-21T03:42:22.340 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "float", 2024-09-21T03:42:22.340 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.340 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:22.340 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "0", 2024-09-21T03:42:22.340 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.340 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.340 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.341 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-09-21T03:42:22.341 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.341 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.341 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.341 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.341 INFO:teuthology.orchestra.run.smithi089.stdout: "mode": { 2024-09-21T03:42:22.341 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "mode", 2024-09-21T03:42:22.341 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:22.341 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.341 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:22.341 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "upmap", 2024-09-21T03:42:22.341 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.341 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.341 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-21T03:42:22.341 INFO:teuthology.orchestra.run.smithi089.stdout: "crush-compat", 2024-09-21T03:42:22.342 INFO:teuthology.orchestra.run.smithi089.stdout: "none", 2024-09-21T03:42:22.342 INFO:teuthology.orchestra.run.smithi089.stdout: "read", 2024-09-21T03:42:22.342 INFO:teuthology.orchestra.run.smithi089.stdout: "upmap", 2024-09-21T03:42:22.342 INFO:teuthology.orchestra.run.smithi089.stdout: "upmap-read" 2024-09-21T03:42:22.342 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-21T03:42:22.342 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Balancer mode", 2024-09-21T03:42:22.342 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.342 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.342 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.342 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.342 INFO:teuthology.orchestra.run.smithi089.stdout: "pool_ids": { 2024-09-21T03:42:22.342 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "pool_ids", 2024-09-21T03:42:22.342 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:22.342 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.343 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:22.343 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-21T03:42:22.343 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.343 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.343 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.343 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-09-21T03:42:22.343 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.343 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.343 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.343 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.343 INFO:teuthology.orchestra.run.smithi089.stdout: "sleep_interval": { 2024-09-21T03:42:22.343 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "sleep_interval", 2024-09-21T03:42:22.343 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "secs", 2024-09-21T03:42:22.343 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.343 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:22.343 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "60", 2024-09-21T03:42:22.344 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.344 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.344 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.344 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-09-21T03:42:22.344 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.344 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.344 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.344 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.344 INFO:teuthology.orchestra.run.smithi089.stdout: "sqlite3_killpoint": { 2024-09-21T03:42:22.344 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:42:22.344 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2024-09-21T03:42:22.344 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "dev", 2024-09-21T03:42:22.344 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:22.344 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "0", 2024-09-21T03:42:22.344 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.345 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.345 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.345 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:22.345 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.345 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.345 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.345 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.345 INFO:teuthology.orchestra.run.smithi089.stdout: "upmap_max_deviation": { 2024-09-21T03:42:22.345 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "upmap_max_deviation", 2024-09-21T03:42:22.345 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2024-09-21T03:42:22.345 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.345 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:22.345 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "5", 2024-09-21T03:42:22.345 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "1", 2024-09-21T03:42:22.346 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.346 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.346 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "deviation below which no optimization is attempted", 2024-09-21T03:42:22.346 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-09-21T03:42:22.346 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.346 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.346 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.346 INFO:teuthology.orchestra.run.smithi089.stdout: "upmap_max_optimizations": { 2024-09-21T03:42:22.346 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "upmap_max_optimizations", 2024-09-21T03:42:22.346 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "uint", 2024-09-21T03:42:22.346 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.346 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:22.346 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "10", 2024-09-21T03:42:22.346 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.347 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.347 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.347 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-09-21T03:42:22.347 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.347 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.347 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.347 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-21T03:42:22.347 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-21T03:42:22.347 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.347 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-21T03:42:22.347 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "cephadm", 2024-09-21T03:42:22.347 INFO:teuthology.orchestra.run.smithi089.stdout: "can_run": true, 2024-09-21T03:42:22.347 INFO:teuthology.orchestra.run.smithi089.stdout: "error_string": "", 2024-09-21T03:42:22.347 INFO:teuthology.orchestra.run.smithi089.stdout: "module_options": { 2024-09-21T03:42:22.347 INFO:teuthology.orchestra.run.smithi089.stdout: "agent_down_multiplier": { 2024-09-21T03:42:22.347 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "agent_down_multiplier", 2024-09-21T03:42:22.348 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "float", 2024-09-21T03:42:22.348 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.348 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:22.348 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "3.0", 2024-09-21T03:42:22.348 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.348 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.348 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.348 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-09-21T03:42:22.348 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.348 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.348 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.348 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.348 INFO:teuthology.orchestra.run.smithi089.stdout: "agent_refresh_rate": { 2024-09-21T03:42:22.349 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "agent_refresh_rate", 2024-09-21T03:42:22.349 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "secs", 2024-09-21T03:42:22.349 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.349 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:22.349 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "20", 2024-09-21T03:42:22.349 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.349 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.349 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.349 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-09-21T03:42:22.349 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.349 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.349 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.349 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.349 INFO:teuthology.orchestra.run.smithi089.stdout: "agent_starting_port": { 2024-09-21T03:42:22.349 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "agent_starting_port", 2024-09-21T03:42:22.349 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2024-09-21T03:42:22.350 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.350 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:22.350 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "4721", 2024-09-21T03:42:22.350 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.350 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.350 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.350 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-09-21T03:42:22.350 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.350 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.350 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.350 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.350 INFO:teuthology.orchestra.run.smithi089.stdout: "allow_ptrace": { 2024-09-21T03:42:22.350 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "allow_ptrace", 2024-09-21T03:42:22.350 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-21T03:42:22.351 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.351 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:22.351 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-21T03:42:22.351 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.351 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.351 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.351 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-09-21T03:42:22.351 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-09-21T03:42:22.351 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.351 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.351 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.351 INFO:teuthology.orchestra.run.smithi089.stdout: "autotune_interval": { 2024-09-21T03:42:22.351 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "autotune_interval", 2024-09-21T03:42:22.351 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "secs", 2024-09-21T03:42:22.351 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.352 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:22.352 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "600", 2024-09-21T03:42:22.352 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.352 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.352 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.352 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "how frequently to autotune daemon memory", 2024-09-21T03:42:22.352 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.352 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.352 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.352 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.352 INFO:teuthology.orchestra.run.smithi089.stdout: "autotune_memory_target_ratio": { 2024-09-21T03:42:22.352 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "autotune_memory_target_ratio", 2024-09-21T03:42:22.352 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "float", 2024-09-21T03:42:22.352 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.352 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:22.353 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "0.7", 2024-09-21T03:42:22.353 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.353 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.353 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.353 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-09-21T03:42:22.353 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.353 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.353 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.353 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.353 INFO:teuthology.orchestra.run.smithi089.stdout: "cephadm_log_destination": { 2024-09-21T03:42:22.353 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "cephadm_log_destination", 2024-09-21T03:42:22.353 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:22.353 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.353 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:22.353 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-21T03:42:22.353 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.354 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.354 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-21T03:42:22.354 INFO:teuthology.orchestra.run.smithi089.stdout: "file", 2024-09-21T03:42:22.354 INFO:teuthology.orchestra.run.smithi089.stdout: "file,syslog", 2024-09-21T03:42:22.354 INFO:teuthology.orchestra.run.smithi089.stdout: "syslog" 2024-09-21T03:42:22.354 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-21T03:42:22.354 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-09-21T03:42:22.354 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.354 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.354 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.354 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.354 INFO:teuthology.orchestra.run.smithi089.stdout: "cgroups_split": { 2024-09-21T03:42:22.354 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "cgroups_split", 2024-09-21T03:42:22.354 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-21T03:42:22.354 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.355 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:22.355 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "True", 2024-09-21T03:42:22.355 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.355 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.355 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.355 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-09-21T03:42:22.355 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.355 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.355 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.355 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.355 INFO:teuthology.orchestra.run.smithi089.stdout: "config_checks_enabled": { 2024-09-21T03:42:22.355 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "config_checks_enabled", 2024-09-21T03:42:22.355 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-21T03:42:22.355 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.355 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:22.355 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-21T03:42:22.356 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.356 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.356 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.356 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-09-21T03:42:22.356 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.356 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.356 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.356 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.356 INFO:teuthology.orchestra.run.smithi089.stdout: "config_dashboard": { 2024-09-21T03:42:22.356 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "config_dashboard", 2024-09-21T03:42:22.356 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-21T03:42:22.356 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.356 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:22.356 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "True", 2024-09-21T03:42:22.356 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.356 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.357 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.357 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-09-21T03:42:22.357 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.357 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.357 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.357 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.357 INFO:teuthology.orchestra.run.smithi089.stdout: "container_image_alertmanager": { 2024-09-21T03:42:22.357 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "container_image_alertmanager", 2024-09-21T03:42:22.357 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:22.357 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.357 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:22.357 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.27.0", 2024-09-21T03:42:22.357 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.357 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.357 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.358 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Prometheus container image", 2024-09-21T03:42:22.358 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.358 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.358 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.358 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.358 INFO:teuthology.orchestra.run.smithi089.stdout: "container_image_base": { 2024-09-21T03:42:22.358 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "container_image_base", 2024-09-21T03:42:22.358 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:22.358 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.358 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:22.358 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "quay.io/ceph/ceph", 2024-09-21T03:42:22.358 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.358 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.358 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.359 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Container image name, without the tag", 2024-09-21T03:42:22.359 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.359 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.359 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.359 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.359 INFO:teuthology.orchestra.run.smithi089.stdout: "container_image_elasticsearch": { 2024-09-21T03:42:22.359 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "container_image_elasticsearch", 2024-09-21T03:42:22.359 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:22.359 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.359 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:22.359 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-09-21T03:42:22.359 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.359 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.359 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.359 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "elasticsearch container image", 2024-09-21T03:42:22.360 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.360 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.360 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.360 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.360 INFO:teuthology.orchestra.run.smithi089.stdout: "container_image_grafana": { 2024-09-21T03:42:22.360 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "container_image_grafana", 2024-09-21T03:42:22.360 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:22.360 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.360 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:22.360 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "quay.io/ceph/grafana:10.4.8", 2024-09-21T03:42:22.360 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.360 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.360 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.360 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Prometheus container image", 2024-09-21T03:42:22.360 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.361 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.361 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.361 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.361 INFO:teuthology.orchestra.run.smithi089.stdout: "container_image_haproxy": { 2024-09-21T03:42:22.361 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "container_image_haproxy", 2024-09-21T03:42:22.361 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:22.361 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.361 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:22.361 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-09-21T03:42:22.361 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.361 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.361 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.361 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "HAproxy container image", 2024-09-21T03:42:22.361 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.361 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.361 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.362 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.362 INFO:teuthology.orchestra.run.smithi089.stdout: "container_image_jaeger_agent": { 2024-09-21T03:42:22.362 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "container_image_jaeger_agent", 2024-09-21T03:42:22.362 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:22.362 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.362 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:22.362 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-09-21T03:42:22.362 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.362 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.362 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.362 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Jaeger agent container image", 2024-09-21T03:42:22.362 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.362 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.362 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.362 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.363 INFO:teuthology.orchestra.run.smithi089.stdout: "container_image_jaeger_collector": { 2024-09-21T03:42:22.363 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "container_image_jaeger_collector", 2024-09-21T03:42:22.363 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:22.363 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.363 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:22.363 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-09-21T03:42:22.363 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.363 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.363 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.363 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Jaeger collector container image", 2024-09-21T03:42:22.363 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.363 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.363 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.363 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.363 INFO:teuthology.orchestra.run.smithi089.stdout: "container_image_jaeger_query": { 2024-09-21T03:42:22.364 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "container_image_jaeger_query", 2024-09-21T03:42:22.364 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:22.364 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.364 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:22.364 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-09-21T03:42:22.364 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.364 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.364 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.364 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Jaeger query container image", 2024-09-21T03:42:22.364 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.364 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.364 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.364 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.364 INFO:teuthology.orchestra.run.smithi089.stdout: "container_image_keepalived": { 2024-09-21T03:42:22.364 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "container_image_keepalived", 2024-09-21T03:42:22.365 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:22.365 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.365 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:22.365 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-09-21T03:42:22.365 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.365 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.365 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.365 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Keepalived container image", 2024-09-21T03:42:22.365 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.365 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.365 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.365 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.365 INFO:teuthology.orchestra.run.smithi089.stdout: "container_image_loki": { 2024-09-21T03:42:22.365 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "container_image_loki", 2024-09-21T03:42:22.366 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:22.366 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.366 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:22.366 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "docker.io/grafana/loki:3.0.0", 2024-09-21T03:42:22.366 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.366 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.366 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.366 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Loki container image", 2024-09-21T03:42:22.366 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.366 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.366 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.366 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.366 INFO:teuthology.orchestra.run.smithi089.stdout: "container_image_nginx": { 2024-09-21T03:42:22.366 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "container_image_nginx", 2024-09-21T03:42:22.367 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:22.367 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.367 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:22.367 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "quay.io/ceph/nginx:sclorg-nginx-126", 2024-09-21T03:42:22.367 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.367 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.367 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.367 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Nginx container image", 2024-09-21T03:42:22.367 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.367 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.367 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.367 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.367 INFO:teuthology.orchestra.run.smithi089.stdout: "container_image_node_exporter": { 2024-09-21T03:42:22.367 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "container_image_node_exporter", 2024-09-21T03:42:22.367 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:22.368 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.368 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:22.368 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.7.0", 2024-09-21T03:42:22.368 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.368 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.368 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.368 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Prometheus container image", 2024-09-21T03:42:22.368 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.368 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.368 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.368 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.368 INFO:teuthology.orchestra.run.smithi089.stdout: "container_image_nvmeof": { 2024-09-21T03:42:22.368 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "container_image_nvmeof", 2024-09-21T03:42:22.368 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:22.369 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.369 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:22.369 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.17", 2024-09-21T03:42:22.369 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.369 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.369 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.369 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Nvme-of container image", 2024-09-21T03:42:22.369 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.369 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.369 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.369 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.369 INFO:teuthology.orchestra.run.smithi089.stdout: "container_image_oauth2_proxy": { 2024-09-21T03:42:22.369 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "container_image_oauth2_proxy", 2024-09-21T03:42:22.369 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:22.369 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.369 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:22.370 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "quay.io/oauth2-proxy/oauth2-proxy:v7.6.0", 2024-09-21T03:42:22.370 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.370 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.370 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.370 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "oauth2-proxy container image", 2024-09-21T03:42:22.370 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.370 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.370 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.370 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.370 INFO:teuthology.orchestra.run.smithi089.stdout: "container_image_prometheus": { 2024-09-21T03:42:22.370 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "container_image_prometheus", 2024-09-21T03:42:22.370 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:22.370 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.370 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:22.370 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "quay.io/prometheus/prometheus:v2.51.0", 2024-09-21T03:42:22.370 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.371 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.371 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.371 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Prometheus container image", 2024-09-21T03:42:22.371 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.371 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.371 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.371 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.371 INFO:teuthology.orchestra.run.smithi089.stdout: "container_image_promtail": { 2024-09-21T03:42:22.371 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "container_image_promtail", 2024-09-21T03:42:22.371 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:22.371 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.371 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:22.371 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "docker.io/grafana/promtail:3.0.0", 2024-09-21T03:42:22.371 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.372 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.372 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.372 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Promtail container image", 2024-09-21T03:42:22.372 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.372 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.372 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.372 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.372 INFO:teuthology.orchestra.run.smithi089.stdout: "container_image_samba": { 2024-09-21T03:42:22.372 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "container_image_samba", 2024-09-21T03:42:22.372 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:22.372 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.372 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:22.372 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-09-21T03:42:22.372 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.372 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.372 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.373 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Samba/SMB container image", 2024-09-21T03:42:22.373 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.373 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.373 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.373 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.373 INFO:teuthology.orchestra.run.smithi089.stdout: "container_image_samba_metrics": { 2024-09-21T03:42:22.373 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "container_image_samba_metrics", 2024-09-21T03:42:22.373 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:22.373 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.373 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:22.373 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "quay.io/samba.org/samba-metrics:latest", 2024-09-21T03:42:22.373 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.373 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.373 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.373 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Samba/SMB metrics exporter container image", 2024-09-21T03:42:22.374 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.374 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.374 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.374 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.374 INFO:teuthology.orchestra.run.smithi089.stdout: "container_image_snmp_gateway": { 2024-09-21T03:42:22.374 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "container_image_snmp_gateway", 2024-09-21T03:42:22.374 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:22.374 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.374 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:22.374 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-09-21T03:42:22.374 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.374 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.374 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.374 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "SNMP Gateway container image", 2024-09-21T03:42:22.374 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.375 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.375 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.375 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.375 INFO:teuthology.orchestra.run.smithi089.stdout: "container_init": { 2024-09-21T03:42:22.375 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "container_init", 2024-09-21T03:42:22.375 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-21T03:42:22.375 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.375 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:22.375 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "True", 2024-09-21T03:42:22.375 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.375 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.375 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.375 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Run podman/docker with `--init`", 2024-09-21T03:42:22.375 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.375 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.376 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.376 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.376 INFO:teuthology.orchestra.run.smithi089.stdout: "daemon_cache_timeout": { 2024-09-21T03:42:22.376 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "daemon_cache_timeout", 2024-09-21T03:42:22.376 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "secs", 2024-09-21T03:42:22.376 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.376 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:22.376 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "600", 2024-09-21T03:42:22.376 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.376 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.376 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.376 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-09-21T03:42:22.376 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.376 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.376 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.376 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.377 INFO:teuthology.orchestra.run.smithi089.stdout: "default_cephadm_command_timeout": { 2024-09-21T03:42:22.377 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "default_cephadm_command_timeout", 2024-09-21T03:42:22.377 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2024-09-21T03:42:22.377 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.377 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:22.377 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "900", 2024-09-21T03:42:22.377 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.377 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.377 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.377 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-09-21T03:42:22.377 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.377 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.377 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.377 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.378 INFO:teuthology.orchestra.run.smithi089.stdout: "default_registry": { 2024-09-21T03:42:22.378 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "default_registry", 2024-09-21T03:42:22.378 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:22.378 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.378 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:22.378 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "docker.io", 2024-09-21T03:42:22.378 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.378 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.378 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.378 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-09-21T03:42:22.378 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.378 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.378 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.378 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.379 INFO:teuthology.orchestra.run.smithi089.stdout: "device_cache_timeout": { 2024-09-21T03:42:22.379 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "device_cache_timeout", 2024-09-21T03:42:22.379 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "secs", 2024-09-21T03:42:22.379 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.379 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:22.379 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "1800", 2024-09-21T03:42:22.379 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.379 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.379 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.379 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "seconds to cache device inventory", 2024-09-21T03:42:22.379 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.379 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.379 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.379 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.379 INFO:teuthology.orchestra.run.smithi089.stdout: "device_enhanced_scan": { 2024-09-21T03:42:22.379 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "device_enhanced_scan", 2024-09-21T03:42:22.380 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-21T03:42:22.380 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.380 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:22.380 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-21T03:42:22.380 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.380 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.380 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.380 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Use libstoragemgmt during device scans", 2024-09-21T03:42:22.380 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.380 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.380 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.380 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.380 INFO:teuthology.orchestra.run.smithi089.stdout: "facts_cache_timeout": { 2024-09-21T03:42:22.380 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "facts_cache_timeout", 2024-09-21T03:42:22.380 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "secs", 2024-09-21T03:42:22.380 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.381 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:22.381 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "60", 2024-09-21T03:42:22.381 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.381 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.381 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.381 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "seconds to cache host facts data", 2024-09-21T03:42:22.381 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.381 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.381 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.381 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.381 INFO:teuthology.orchestra.run.smithi089.stdout: "grafana_dashboards_path": { 2024-09-21T03:42:22.381 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "grafana_dashboards_path", 2024-09-21T03:42:22.381 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:22.381 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.381 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:22.381 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-09-21T03:42:22.382 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.382 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.382 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.382 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-09-21T03:42:22.382 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.382 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.382 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.382 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.382 INFO:teuthology.orchestra.run.smithi089.stdout: "host_check_interval": { 2024-09-21T03:42:22.382 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "host_check_interval", 2024-09-21T03:42:22.382 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "secs", 2024-09-21T03:42:22.382 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.382 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:22.383 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "600", 2024-09-21T03:42:22.383 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.383 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.383 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.383 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "how frequently to perform a host check", 2024-09-21T03:42:22.383 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.383 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.383 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.383 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.383 INFO:teuthology.orchestra.run.smithi089.stdout: "hw_monitoring": { 2024-09-21T03:42:22.383 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "hw_monitoring", 2024-09-21T03:42:22.383 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-21T03:42:22.383 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.383 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:22.384 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-21T03:42:22.384 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.384 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.384 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.384 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-09-21T03:42:22.384 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.384 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.384 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.384 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.384 INFO:teuthology.orchestra.run.smithi089.stdout: "inventory_list_all": { 2024-09-21T03:42:22.384 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "inventory_list_all", 2024-09-21T03:42:22.384 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-21T03:42:22.384 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.384 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:22.385 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-21T03:42:22.385 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.385 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.385 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.385 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-09-21T03:42:22.385 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.385 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.385 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.385 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.385 INFO:teuthology.orchestra.run.smithi089.stdout: "log_level": { 2024-09-21T03:42:22.385 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_level", 2024-09-21T03:42:22.385 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:22.385 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.385 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:22.385 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-21T03:42:22.386 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.386 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.386 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-21T03:42:22.386 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2024-09-21T03:42:22.386 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2024-09-21T03:42:22.386 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2024-09-21T03:42:22.386 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2024-09-21T03:42:22.386 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2024-09-21T03:42:22.386 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2024-09-21T03:42:22.386 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-21T03:42:22.386 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:22.386 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.386 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.386 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.386 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.387 INFO:teuthology.orchestra.run.smithi089.stdout: "log_refresh_metadata": { 2024-09-21T03:42:22.387 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_refresh_metadata", 2024-09-21T03:42:22.387 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-21T03:42:22.387 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.387 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:22.387 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-21T03:42:22.387 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.387 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.387 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.387 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-09-21T03:42:22.387 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.387 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.387 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.388 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.388 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster": { 2024-09-21T03:42:22.388 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster", 2024-09-21T03:42:22.388 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-21T03:42:22.388 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.388 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:22.388 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "True", 2024-09-21T03:42:22.388 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.388 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.388 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.388 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-09-21T03:42:22.388 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.388 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.388 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.388 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.389 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster_level": { 2024-09-21T03:42:22.389 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster_level", 2024-09-21T03:42:22.389 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:22.389 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.389 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:22.389 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "info", 2024-09-21T03:42:22.389 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.389 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.389 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-21T03:42:22.389 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2024-09-21T03:42:22.389 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2024-09-21T03:42:22.389 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2024-09-21T03:42:22.389 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2024-09-21T03:42:22.389 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2024-09-21T03:42:22.389 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2024-09-21T03:42:22.390 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-21T03:42:22.390 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:22.390 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.390 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.390 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.390 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.390 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_file": { 2024-09-21T03:42:22.390 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_file", 2024-09-21T03:42:22.390 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-21T03:42:22.390 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.390 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:22.390 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-21T03:42:22.390 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.390 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.390 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.391 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:22.391 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.391 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.391 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.391 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.391 INFO:teuthology.orchestra.run.smithi089.stdout: "manage_etc_ceph_ceph_conf": { 2024-09-21T03:42:22.391 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-09-21T03:42:22.391 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-21T03:42:22.391 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.391 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:22.391 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-21T03:42:22.391 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.391 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.391 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.391 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-09-21T03:42:22.392 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.392 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.392 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.392 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.392 INFO:teuthology.orchestra.run.smithi089.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-09-21T03:42:22.392 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-09-21T03:42:22.392 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:22.392 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.392 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:22.392 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "*", 2024-09-21T03:42:22.392 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.392 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.392 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.392 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-09-21T03:42:22.393 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.393 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.393 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.393 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.393 INFO:teuthology.orchestra.run.smithi089.stdout: "max_count_per_host": { 2024-09-21T03:42:22.393 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "max_count_per_host", 2024-09-21T03:42:22.393 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2024-09-21T03:42:22.393 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.393 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:22.393 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "10", 2024-09-21T03:42:22.393 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.393 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.393 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.393 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "max number of daemons per service per host", 2024-09-21T03:42:22.393 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.394 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.394 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.394 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.394 INFO:teuthology.orchestra.run.smithi089.stdout: "max_osd_draining_count": { 2024-09-21T03:42:22.394 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "max_osd_draining_count", 2024-09-21T03:42:22.394 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2024-09-21T03:42:22.394 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.394 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:22.394 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "10", 2024-09-21T03:42:22.394 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.394 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.394 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.394 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-09-21T03:42:22.394 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.395 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.395 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.395 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.395 INFO:teuthology.orchestra.run.smithi089.stdout: "migration_current": { 2024-09-21T03:42:22.395 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "migration_current", 2024-09-21T03:42:22.395 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2024-09-21T03:42:22.395 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.395 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:22.395 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-21T03:42:22.395 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.395 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.395 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.395 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "internal - do not modify", 2024-09-21T03:42:22.395 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.395 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.395 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.396 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.396 INFO:teuthology.orchestra.run.smithi089.stdout: "mode": { 2024-09-21T03:42:22.396 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "mode", 2024-09-21T03:42:22.396 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:22.396 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.396 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:22.396 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "root", 2024-09-21T03:42:22.396 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.396 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.396 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-21T03:42:22.396 INFO:teuthology.orchestra.run.smithi089.stdout: "cephadm-package", 2024-09-21T03:42:22.396 INFO:teuthology.orchestra.run.smithi089.stdout: "root" 2024-09-21T03:42:22.396 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-21T03:42:22.396 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "mode for remote execution of cephadm", 2024-09-21T03:42:22.396 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.396 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.396 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.397 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.397 INFO:teuthology.orchestra.run.smithi089.stdout: "oob_default_addr": { 2024-09-21T03:42:22.397 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "oob_default_addr", 2024-09-21T03:42:22.397 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:22.397 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.397 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:22.397 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "169.254.1.1", 2024-09-21T03:42:22.397 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.397 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.397 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.397 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Default address for RedFish API (oob management).", 2024-09-21T03:42:22.397 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.397 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.397 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.397 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.397 INFO:teuthology.orchestra.run.smithi089.stdout: "prometheus_alerts_path": { 2024-09-21T03:42:22.398 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "prometheus_alerts_path", 2024-09-21T03:42:22.398 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:22.398 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.398 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:22.398 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-09-21T03:42:22.398 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.398 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.398 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.398 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-09-21T03:42:22.398 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.398 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.398 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.398 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.398 INFO:teuthology.orchestra.run.smithi089.stdout: "registry_insecure": { 2024-09-21T03:42:22.398 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "registry_insecure", 2024-09-21T03:42:22.398 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-21T03:42:22.399 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.399 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:22.399 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-21T03:42:22.399 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.399 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.399 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.399 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-09-21T03:42:22.399 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.399 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.399 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.399 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.399 INFO:teuthology.orchestra.run.smithi089.stdout: "registry_password": { 2024-09-21T03:42:22.399 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "registry_password", 2024-09-21T03:42:22.399 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:22.400 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.400 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:22.400 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-21T03:42:22.400 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.400 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.400 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.400 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-09-21T03:42:22.400 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.400 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.400 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.400 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.400 INFO:teuthology.orchestra.run.smithi089.stdout: "registry_url": { 2024-09-21T03:42:22.400 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "registry_url", 2024-09-21T03:42:22.400 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:22.401 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.401 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:22.401 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-21T03:42:22.401 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.401 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.401 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.401 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-09-21T03:42:22.401 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.401 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.401 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.401 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.401 INFO:teuthology.orchestra.run.smithi089.stdout: "registry_username": { 2024-09-21T03:42:22.401 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "registry_username", 2024-09-21T03:42:22.401 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:22.401 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.401 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:22.401 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-21T03:42:22.402 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.402 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.402 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.402 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-09-21T03:42:22.402 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.402 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.402 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.402 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.402 INFO:teuthology.orchestra.run.smithi089.stdout: "secure_monitoring_stack": { 2024-09-21T03:42:22.402 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "secure_monitoring_stack", 2024-09-21T03:42:22.402 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-21T03:42:22.402 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.402 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:22.402 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-21T03:42:22.402 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.402 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.403 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.403 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-09-21T03:42:22.403 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.403 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.403 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.403 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.403 INFO:teuthology.orchestra.run.smithi089.stdout: "service_discovery_port": { 2024-09-21T03:42:22.403 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "service_discovery_port", 2024-09-21T03:42:22.403 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2024-09-21T03:42:22.403 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.403 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:22.403 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "8765", 2024-09-21T03:42:22.403 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.403 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.404 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.404 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "cephadm service discovery port", 2024-09-21T03:42:22.404 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.404 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.404 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.404 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.404 INFO:teuthology.orchestra.run.smithi089.stdout: "sqlite3_killpoint": { 2024-09-21T03:42:22.404 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:42:22.404 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2024-09-21T03:42:22.404 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "dev", 2024-09-21T03:42:22.404 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:22.404 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "0", 2024-09-21T03:42:22.404 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.404 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.405 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.405 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:22.405 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.405 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.405 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.405 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.405 INFO:teuthology.orchestra.run.smithi089.stdout: "ssh_config_file": { 2024-09-21T03:42:22.405 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "ssh_config_file", 2024-09-21T03:42:22.405 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:22.405 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.405 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:22.405 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-21T03:42:22.405 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.405 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.405 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.406 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-09-21T03:42:22.406 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.406 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.406 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.406 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.406 INFO:teuthology.orchestra.run.smithi089.stdout: "ssh_keepalive_count_max": { 2024-09-21T03:42:22.406 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "ssh_keepalive_count_max", 2024-09-21T03:42:22.406 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2024-09-21T03:42:22.406 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.406 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:22.406 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "3", 2024-09-21T03:42:22.406 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.406 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.406 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.407 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "How many times ssh connections can fail liveness checks before the host is marked offline", 2024-09-21T03:42:22.407 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.407 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.407 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.407 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.407 INFO:teuthology.orchestra.run.smithi089.stdout: "ssh_keepalive_interval": { 2024-09-21T03:42:22.407 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "ssh_keepalive_interval", 2024-09-21T03:42:22.407 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2024-09-21T03:42:22.407 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.407 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:22.407 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "7", 2024-09-21T03:42:22.407 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.407 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.407 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.408 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "How often ssh connections are checked for liveness", 2024-09-21T03:42:22.408 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.408 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.408 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.408 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.408 INFO:teuthology.orchestra.run.smithi089.stdout: "use_agent": { 2024-09-21T03:42:22.408 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "use_agent", 2024-09-21T03:42:22.408 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-21T03:42:22.408 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.408 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:22.408 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-21T03:42:22.408 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.408 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.408 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.408 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-09-21T03:42:22.409 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.409 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.409 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.409 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.409 INFO:teuthology.orchestra.run.smithi089.stdout: "use_repo_digest": { 2024-09-21T03:42:22.409 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "use_repo_digest", 2024-09-21T03:42:22.409 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-21T03:42:22.409 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.409 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:22.409 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "True", 2024-09-21T03:42:22.409 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.409 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.409 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.409 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-09-21T03:42:22.409 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.410 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.410 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.410 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.410 INFO:teuthology.orchestra.run.smithi089.stdout: "warn_on_failed_host_check": { 2024-09-21T03:42:22.410 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "warn_on_failed_host_check", 2024-09-21T03:42:22.410 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-21T03:42:22.410 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.410 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:22.410 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "True", 2024-09-21T03:42:22.410 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.410 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.410 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.410 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "raise a health warning if the host check fails", 2024-09-21T03:42:22.410 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.411 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.411 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.411 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.411 INFO:teuthology.orchestra.run.smithi089.stdout: "warn_on_stray_daemons": { 2024-09-21T03:42:22.411 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "warn_on_stray_daemons", 2024-09-21T03:42:22.411 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-21T03:42:22.411 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.411 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:22.411 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "True", 2024-09-21T03:42:22.411 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.411 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.411 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.411 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-09-21T03:42:22.411 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.411 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.412 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.412 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.412 INFO:teuthology.orchestra.run.smithi089.stdout: "warn_on_stray_hosts": { 2024-09-21T03:42:22.412 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "warn_on_stray_hosts", 2024-09-21T03:42:22.412 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-21T03:42:22.412 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.412 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:22.412 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "True", 2024-09-21T03:42:22.412 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.412 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.412 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.412 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-09-21T03:42:22.412 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.412 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.412 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.413 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-21T03:42:22.413 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-21T03:42:22.413 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.413 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-21T03:42:22.413 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "crash", 2024-09-21T03:42:22.413 INFO:teuthology.orchestra.run.smithi089.stdout: "can_run": true, 2024-09-21T03:42:22.413 INFO:teuthology.orchestra.run.smithi089.stdout: "error_string": "", 2024-09-21T03:42:22.413 INFO:teuthology.orchestra.run.smithi089.stdout: "module_options": { 2024-09-21T03:42:22.413 INFO:teuthology.orchestra.run.smithi089.stdout: "log_level": { 2024-09-21T03:42:22.413 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_level", 2024-09-21T03:42:22.413 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:22.413 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.413 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:22.413 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-21T03:42:22.413 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.414 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.414 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-21T03:42:22.414 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2024-09-21T03:42:22.414 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2024-09-21T03:42:22.414 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2024-09-21T03:42:22.414 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2024-09-21T03:42:22.414 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2024-09-21T03:42:22.414 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2024-09-21T03:42:22.414 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-21T03:42:22.414 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:22.414 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.414 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.414 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.414 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.414 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster": { 2024-09-21T03:42:22.415 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster", 2024-09-21T03:42:22.415 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-21T03:42:22.415 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.415 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:22.415 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-21T03:42:22.415 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.415 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.415 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.415 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:22.415 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.415 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.415 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.415 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.415 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster_level": { 2024-09-21T03:42:22.415 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster_level", 2024-09-21T03:42:22.416 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:22.416 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.416 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:22.416 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "info", 2024-09-21T03:42:22.416 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.416 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.416 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-21T03:42:22.416 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2024-09-21T03:42:22.416 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2024-09-21T03:42:22.416 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2024-09-21T03:42:22.416 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2024-09-21T03:42:22.416 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2024-09-21T03:42:22.416 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2024-09-21T03:42:22.416 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-21T03:42:22.416 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:22.416 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.417 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.417 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.417 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.417 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_file": { 2024-09-21T03:42:22.417 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_file", 2024-09-21T03:42:22.417 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-21T03:42:22.417 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.417 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:22.417 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-21T03:42:22.417 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.417 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.417 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.417 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:22.418 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.418 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.418 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.418 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.418 INFO:teuthology.orchestra.run.smithi089.stdout: "retain_interval": { 2024-09-21T03:42:22.418 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "retain_interval", 2024-09-21T03:42:22.418 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "secs", 2024-09-21T03:42:22.418 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.418 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:22.418 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "31536000", 2024-09-21T03:42:22.418 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.418 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.418 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.418 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "how long to retain crashes before pruning them", 2024-09-21T03:42:22.419 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.419 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.419 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.419 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.419 INFO:teuthology.orchestra.run.smithi089.stdout: "sqlite3_killpoint": { 2024-09-21T03:42:22.419 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:42:22.419 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2024-09-21T03:42:22.419 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "dev", 2024-09-21T03:42:22.419 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:22.419 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "0", 2024-09-21T03:42:22.419 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.419 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.419 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.419 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:22.419 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.419 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.420 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.420 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.420 INFO:teuthology.orchestra.run.smithi089.stdout: "warn_recent_interval": { 2024-09-21T03:42:22.420 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "warn_recent_interval", 2024-09-21T03:42:22.420 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "secs", 2024-09-21T03:42:22.420 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.420 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:22.420 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "1209600", 2024-09-21T03:42:22.420 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.420 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.420 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.420 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "time interval in which to warn about recent crashes", 2024-09-21T03:42:22.420 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.420 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.420 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.421 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-21T03:42:22.421 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-21T03:42:22.421 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.421 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-21T03:42:22.421 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "dashboard", 2024-09-21T03:42:22.421 INFO:teuthology.orchestra.run.smithi089.stdout: "can_run": true, 2024-09-21T03:42:22.421 INFO:teuthology.orchestra.run.smithi089.stdout: "error_string": "", 2024-09-21T03:42:22.421 INFO:teuthology.orchestra.run.smithi089.stdout: "module_options": { 2024-09-21T03:42:22.421 INFO:teuthology.orchestra.run.smithi089.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-09-21T03:42:22.421 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-09-21T03:42:22.421 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2024-09-21T03:42:22.421 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.421 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:22.421 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "10", 2024-09-21T03:42:22.421 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.421 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.422 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.422 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:22.422 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.422 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.422 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.422 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.422 INFO:teuthology.orchestra.run.smithi089.stdout: "ALERTMANAGER_API_HOST": { 2024-09-21T03:42:22.422 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "ALERTMANAGER_API_HOST", 2024-09-21T03:42:22.422 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:22.422 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.422 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:22.422 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-21T03:42:22.422 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.422 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.423 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.423 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:22.423 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.423 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.423 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.423 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.423 INFO:teuthology.orchestra.run.smithi089.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-09-21T03:42:22.423 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-09-21T03:42:22.423 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-21T03:42:22.423 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.423 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:22.423 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "True", 2024-09-21T03:42:22.423 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.423 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.424 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.424 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:22.424 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.424 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.424 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.424 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.424 INFO:teuthology.orchestra.run.smithi089.stdout: "AUDIT_API_ENABLED": { 2024-09-21T03:42:22.424 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "AUDIT_API_ENABLED", 2024-09-21T03:42:22.424 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-21T03:42:22.424 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.424 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:22.424 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-21T03:42:22.424 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.424 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.424 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.425 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:22.425 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.425 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.425 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.425 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.425 INFO:teuthology.orchestra.run.smithi089.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-09-21T03:42:22.425 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-09-21T03:42:22.425 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-21T03:42:22.425 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.425 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:22.425 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "True", 2024-09-21T03:42:22.425 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.425 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.425 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.425 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:22.425 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.425 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.426 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.426 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.426 INFO:teuthology.orchestra.run.smithi089.stdout: "ENABLE_BROWSABLE_API": { 2024-09-21T03:42:22.426 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "ENABLE_BROWSABLE_API", 2024-09-21T03:42:22.426 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-21T03:42:22.426 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.426 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:22.426 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "True", 2024-09-21T03:42:22.426 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.426 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.426 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.426 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:22.426 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.427 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.427 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.427 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.427 INFO:teuthology.orchestra.run.smithi089.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-09-21T03:42:22.427 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-09-21T03:42:22.427 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-21T03:42:22.427 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.427 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:22.427 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "True", 2024-09-21T03:42:22.427 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.427 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.427 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.427 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:22.427 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.427 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.427 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.428 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.428 INFO:teuthology.orchestra.run.smithi089.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-09-21T03:42:22.428 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-09-21T03:42:22.428 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-21T03:42:22.428 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.428 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:22.428 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "True", 2024-09-21T03:42:22.428 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.428 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.428 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.428 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:22.428 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.428 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.428 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.429 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.429 INFO:teuthology.orchestra.run.smithi089.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-09-21T03:42:22.429 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-09-21T03:42:22.429 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-21T03:42:22.429 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.429 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:22.429 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "True", 2024-09-21T03:42:22.429 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.429 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.429 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.429 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:22.429 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.429 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.429 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.429 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.430 INFO:teuthology.orchestra.run.smithi089.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-09-21T03:42:22.430 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-09-21T03:42:22.430 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-21T03:42:22.430 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.430 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:22.430 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "True", 2024-09-21T03:42:22.430 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.430 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.430 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.430 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:22.430 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.430 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.430 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.431 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.431 INFO:teuthology.orchestra.run.smithi089.stdout: "FEATURE_TOGGLE_NFS": { 2024-09-21T03:42:22.431 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-09-21T03:42:22.431 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-21T03:42:22.431 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.431 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:22.431 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "True", 2024-09-21T03:42:22.431 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.431 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.431 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.431 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:22.431 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.431 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.431 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.431 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.432 INFO:teuthology.orchestra.run.smithi089.stdout: "FEATURE_TOGGLE_RBD": { 2024-09-21T03:42:22.432 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-09-21T03:42:22.432 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-21T03:42:22.432 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.432 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:22.432 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "True", 2024-09-21T03:42:22.432 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.432 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.432 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.432 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:22.432 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.432 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.432 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.432 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.432 INFO:teuthology.orchestra.run.smithi089.stdout: "FEATURE_TOGGLE_RGW": { 2024-09-21T03:42:22.433 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-09-21T03:42:22.433 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-21T03:42:22.433 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.433 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:22.433 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "True", 2024-09-21T03:42:22.433 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.433 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.433 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.433 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:22.433 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.433 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.433 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.433 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.433 INFO:teuthology.orchestra.run.smithi089.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-09-21T03:42:22.433 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-09-21T03:42:22.434 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:22.434 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.434 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:22.434 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-21T03:42:22.434 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.434 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.434 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.434 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:22.434 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.434 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.434 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.434 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.434 INFO:teuthology.orchestra.run.smithi089.stdout: "GRAFANA_API_PASSWORD": { 2024-09-21T03:42:22.434 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "GRAFANA_API_PASSWORD", 2024-09-21T03:42:22.435 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:22.435 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.435 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:22.435 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "admin", 2024-09-21T03:42:22.435 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.435 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.435 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.435 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:22.435 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.435 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.435 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.435 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.435 INFO:teuthology.orchestra.run.smithi089.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-09-21T03:42:22.435 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-09-21T03:42:22.435 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-21T03:42:22.436 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.436 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:22.436 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "True", 2024-09-21T03:42:22.436 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.436 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.436 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.436 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:22.436 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.436 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.436 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.436 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.436 INFO:teuthology.orchestra.run.smithi089.stdout: "GRAFANA_API_URL": { 2024-09-21T03:42:22.436 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "GRAFANA_API_URL", 2024-09-21T03:42:22.436 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:22.436 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.437 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:22.437 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-21T03:42:22.437 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.437 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.437 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.437 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:22.437 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.437 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.437 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.437 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.437 INFO:teuthology.orchestra.run.smithi089.stdout: "GRAFANA_API_USERNAME": { 2024-09-21T03:42:22.437 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "GRAFANA_API_USERNAME", 2024-09-21T03:42:22.437 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:22.437 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.437 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:22.437 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "admin", 2024-09-21T03:42:22.438 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.438 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.438 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.438 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:22.438 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.438 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.438 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.438 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.438 INFO:teuthology.orchestra.run.smithi089.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-09-21T03:42:22.438 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-09-21T03:42:22.438 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:22.438 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.438 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:22.438 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-21T03:42:22.438 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.439 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.439 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.439 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:22.439 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.439 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.439 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.439 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.439 INFO:teuthology.orchestra.run.smithi089.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-09-21T03:42:22.439 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-09-21T03:42:22.439 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-21T03:42:22.439 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.439 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:22.439 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-21T03:42:22.439 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.439 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.440 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.440 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:22.440 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.440 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.440 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.440 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.440 INFO:teuthology.orchestra.run.smithi089.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-09-21T03:42:22.440 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-09-21T03:42:22.440 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-21T03:42:22.440 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.440 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:22.440 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "True", 2024-09-21T03:42:22.440 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.440 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.441 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.441 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:22.441 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.441 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.441 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.441 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.441 INFO:teuthology.orchestra.run.smithi089.stdout: "ISSUE_TRACKER_API_KEY": { 2024-09-21T03:42:22.441 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-09-21T03:42:22.441 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:22.441 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.441 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:22.441 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-21T03:42:22.441 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.441 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.441 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.442 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:22.442 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.442 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.442 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.442 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.442 INFO:teuthology.orchestra.run.smithi089.stdout: "MANAGED_BY_CLUSTERS": { 2024-09-21T03:42:22.442 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-09-21T03:42:22.442 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:22.442 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.442 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:22.442 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "[]", 2024-09-21T03:42:22.442 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.442 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.442 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.443 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:22.443 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.443 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.443 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.443 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.443 INFO:teuthology.orchestra.run.smithi089.stdout: "MULTICLUSTER_CONFIG": { 2024-09-21T03:42:22.443 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "MULTICLUSTER_CONFIG", 2024-09-21T03:42:22.443 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:22.443 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.443 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:22.443 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "{}", 2024-09-21T03:42:22.443 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.443 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.443 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.444 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:22.444 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.444 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.444 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.444 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.444 INFO:teuthology.orchestra.run.smithi089.stdout: "PROMETHEUS_API_HOST": { 2024-09-21T03:42:22.444 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "PROMETHEUS_API_HOST", 2024-09-21T03:42:22.444 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:22.444 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.444 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:22.444 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-21T03:42:22.444 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.444 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.444 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.444 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:22.445 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.445 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.445 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.445 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.445 INFO:teuthology.orchestra.run.smithi089.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-09-21T03:42:22.445 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-09-21T03:42:22.445 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-21T03:42:22.445 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.445 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:22.445 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "True", 2024-09-21T03:42:22.445 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.445 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.445 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.445 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:22.445 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.445 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.446 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.446 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.446 INFO:teuthology.orchestra.run.smithi089.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-09-21T03:42:22.446 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-09-21T03:42:22.446 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-21T03:42:22.446 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.446 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:22.446 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-21T03:42:22.446 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.446 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.446 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.446 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:22.446 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.446 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.446 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.447 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.447 INFO:teuthology.orchestra.run.smithi089.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-09-21T03:42:22.447 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-09-21T03:42:22.447 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-21T03:42:22.447 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.447 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:22.447 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-21T03:42:22.447 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.447 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.447 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.447 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:22.447 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.447 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.447 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.448 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.448 INFO:teuthology.orchestra.run.smithi089.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-09-21T03:42:22.448 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-09-21T03:42:22.448 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-21T03:42:22.448 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.448 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:22.448 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "True", 2024-09-21T03:42:22.448 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.448 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.448 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.448 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:22.448 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.448 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.448 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.448 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.449 INFO:teuthology.orchestra.run.smithi089.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-09-21T03:42:22.449 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-09-21T03:42:22.449 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-21T03:42:22.449 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.449 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:22.449 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "True", 2024-09-21T03:42:22.449 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.449 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.449 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.449 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:22.449 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.449 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.449 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.449 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.449 INFO:teuthology.orchestra.run.smithi089.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-09-21T03:42:22.450 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-09-21T03:42:22.450 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-21T03:42:22.450 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.450 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:22.450 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-21T03:42:22.450 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.450 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.450 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.450 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:22.450 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.450 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.450 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.450 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.450 INFO:teuthology.orchestra.run.smithi089.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-09-21T03:42:22.450 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-09-21T03:42:22.451 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-21T03:42:22.451 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.451 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:22.451 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-21T03:42:22.451 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.451 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.451 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.451 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:22.451 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.451 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.451 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.451 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.451 INFO:teuthology.orchestra.run.smithi089.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-09-21T03:42:22.451 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-09-21T03:42:22.452 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-21T03:42:22.452 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.452 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:22.452 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-21T03:42:22.452 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.452 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.452 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.452 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:22.452 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.452 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.452 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.452 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.452 INFO:teuthology.orchestra.run.smithi089.stdout: "PWD_POLICY_ENABLED": { 2024-09-21T03:42:22.452 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "PWD_POLICY_ENABLED", 2024-09-21T03:42:22.453 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-21T03:42:22.453 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.453 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:22.453 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "True", 2024-09-21T03:42:22.453 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.453 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.453 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.453 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:22.453 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.453 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.453 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.453 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.453 INFO:teuthology.orchestra.run.smithi089.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-09-21T03:42:22.453 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-09-21T03:42:22.453 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:22.454 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.454 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:22.454 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-09-21T03:42:22.454 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.454 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.454 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.454 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:22.454 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.454 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.454 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.454 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.454 INFO:teuthology.orchestra.run.smithi089.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-09-21T03:42:22.454 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-09-21T03:42:22.454 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2024-09-21T03:42:22.454 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.455 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:22.455 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "10", 2024-09-21T03:42:22.455 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.455 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.455 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.455 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:22.455 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.455 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.455 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.455 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.455 INFO:teuthology.orchestra.run.smithi089.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-09-21T03:42:22.455 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-09-21T03:42:22.455 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2024-09-21T03:42:22.456 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.456 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:22.456 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "8", 2024-09-21T03:42:22.456 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.456 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.456 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.456 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:22.456 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.456 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.456 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.456 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.456 INFO:teuthology.orchestra.run.smithi089.stdout: "REST_REQUESTS_TIMEOUT": { 2024-09-21T03:42:22.456 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-09-21T03:42:22.456 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2024-09-21T03:42:22.456 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.457 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:22.457 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "45", 2024-09-21T03:42:22.457 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.457 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.457 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.457 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:22.457 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.457 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.457 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.457 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.457 INFO:teuthology.orchestra.run.smithi089.stdout: "RGW_API_ACCESS_KEY": { 2024-09-21T03:42:22.457 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "RGW_API_ACCESS_KEY", 2024-09-21T03:42:22.457 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:22.457 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.457 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:22.458 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-21T03:42:22.458 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.458 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.458 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.458 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:22.458 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.458 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.458 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.458 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.458 INFO:teuthology.orchestra.run.smithi089.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-09-21T03:42:22.458 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-09-21T03:42:22.458 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:22.458 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.458 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:22.459 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "admin", 2024-09-21T03:42:22.459 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.459 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.459 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.459 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:22.459 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.459 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.459 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.459 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.459 INFO:teuthology.orchestra.run.smithi089.stdout: "RGW_API_SECRET_KEY": { 2024-09-21T03:42:22.459 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "RGW_API_SECRET_KEY", 2024-09-21T03:42:22.459 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:22.459 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.459 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:22.459 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-21T03:42:22.460 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.460 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.460 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.460 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:22.460 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.460 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.460 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.460 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.460 INFO:teuthology.orchestra.run.smithi089.stdout: "RGW_API_SSL_VERIFY": { 2024-09-21T03:42:22.460 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "RGW_API_SSL_VERIFY", 2024-09-21T03:42:22.460 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-21T03:42:22.460 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.460 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:22.460 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "True", 2024-09-21T03:42:22.460 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.461 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.461 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.461 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:22.461 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.461 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.461 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.461 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.461 INFO:teuthology.orchestra.run.smithi089.stdout: "UNSAFE_TLS_v1_2": { 2024-09-21T03:42:22.461 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "UNSAFE_TLS_v1_2", 2024-09-21T03:42:22.461 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-21T03:42:22.461 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.461 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:22.461 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-21T03:42:22.461 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.461 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.461 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.462 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:22.462 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.462 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.462 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.462 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.462 INFO:teuthology.orchestra.run.smithi089.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-09-21T03:42:22.462 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-09-21T03:42:22.462 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2024-09-21T03:42:22.462 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.462 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:22.462 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "0", 2024-09-21T03:42:22.462 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.462 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.462 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.462 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:22.463 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.463 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.463 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.463 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.463 INFO:teuthology.orchestra.run.smithi089.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-09-21T03:42:22.463 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-09-21T03:42:22.463 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2024-09-21T03:42:22.463 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.463 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:22.463 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "10", 2024-09-21T03:42:22.463 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.463 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.463 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.463 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:22.464 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.464 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.464 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.464 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.464 INFO:teuthology.orchestra.run.smithi089.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-09-21T03:42:22.464 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-09-21T03:42:22.464 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2024-09-21T03:42:22.464 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.464 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:22.464 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "5", 2024-09-21T03:42:22.464 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.464 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.464 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.464 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:22.465 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.465 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.465 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.465 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.465 INFO:teuthology.orchestra.run.smithi089.stdout: "cross_origin_url": { 2024-09-21T03:42:22.465 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "cross_origin_url", 2024-09-21T03:42:22.465 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:22.465 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.465 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:22.465 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-21T03:42:22.465 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.465 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.465 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.465 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:22.465 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.466 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.466 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.466 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.466 INFO:teuthology.orchestra.run.smithi089.stdout: "crt_file": { 2024-09-21T03:42:22.466 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "crt_file", 2024-09-21T03:42:22.466 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:22.466 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.466 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:22.466 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-21T03:42:22.466 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.466 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.466 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.466 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:22.466 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.467 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.467 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.467 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.467 INFO:teuthology.orchestra.run.smithi089.stdout: "debug": { 2024-09-21T03:42:22.467 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "debug", 2024-09-21T03:42:22.467 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-21T03:42:22.467 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.467 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:22.467 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-21T03:42:22.467 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.467 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.467 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.467 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Enable/disable debug options", 2024-09-21T03:42:22.467 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.467 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.468 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.468 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.468 INFO:teuthology.orchestra.run.smithi089.stdout: "jwt_token_ttl": { 2024-09-21T03:42:22.468 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "jwt_token_ttl", 2024-09-21T03:42:22.468 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2024-09-21T03:42:22.468 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.468 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:22.468 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "28800", 2024-09-21T03:42:22.468 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.468 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.468 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.468 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:22.469 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.469 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.469 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.469 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.469 INFO:teuthology.orchestra.run.smithi089.stdout: "key_file": { 2024-09-21T03:42:22.469 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "key_file", 2024-09-21T03:42:22.469 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:22.469 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.469 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:22.469 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-21T03:42:22.469 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.469 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.469 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.469 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:22.470 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.470 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.470 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.470 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.470 INFO:teuthology.orchestra.run.smithi089.stdout: "log_level": { 2024-09-21T03:42:22.470 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_level", 2024-09-21T03:42:22.470 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:22.470 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.470 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:22.470 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-21T03:42:22.470 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.470 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.470 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-21T03:42:22.470 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2024-09-21T03:42:22.470 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2024-09-21T03:42:22.471 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2024-09-21T03:42:22.471 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2024-09-21T03:42:22.471 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2024-09-21T03:42:22.471 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2024-09-21T03:42:22.471 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-21T03:42:22.471 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:22.471 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.471 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.471 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.471 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.471 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster": { 2024-09-21T03:42:22.471 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster", 2024-09-21T03:42:22.471 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-21T03:42:22.471 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.472 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:22.472 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-21T03:42:22.472 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.472 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.472 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.472 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:22.472 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.472 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.472 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.472 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.472 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster_level": { 2024-09-21T03:42:22.472 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster_level", 2024-09-21T03:42:22.472 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:22.472 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.472 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:22.472 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "info", 2024-09-21T03:42:22.473 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.473 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.473 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-21T03:42:22.473 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2024-09-21T03:42:22.473 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2024-09-21T03:42:22.473 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2024-09-21T03:42:22.473 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2024-09-21T03:42:22.473 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2024-09-21T03:42:22.473 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2024-09-21T03:42:22.473 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-21T03:42:22.473 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:22.473 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.473 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.473 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.474 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.474 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_file": { 2024-09-21T03:42:22.474 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_file", 2024-09-21T03:42:22.474 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-21T03:42:22.474 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.474 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:22.474 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-21T03:42:22.474 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.474 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.474 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.474 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:22.474 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.474 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.474 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.475 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.475 INFO:teuthology.orchestra.run.smithi089.stdout: "motd": { 2024-09-21T03:42:22.475 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "motd", 2024-09-21T03:42:22.475 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:22.475 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.475 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:22.475 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-21T03:42:22.475 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.475 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.475 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.475 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "The message of the day", 2024-09-21T03:42:22.475 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.475 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.475 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.475 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.476 INFO:teuthology.orchestra.run.smithi089.stdout: "redirect_resolve_ip_addr": { 2024-09-21T03:42:22.476 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "redirect_resolve_ip_addr", 2024-09-21T03:42:22.476 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-21T03:42:22.476 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.476 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:22.476 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-21T03:42:22.476 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.476 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.476 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.476 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:22.476 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.476 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.476 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.477 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.477 INFO:teuthology.orchestra.run.smithi089.stdout: "server_addr": { 2024-09-21T03:42:22.477 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "server_addr", 2024-09-21T03:42:22.477 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:22.477 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.477 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:22.477 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "::", 2024-09-21T03:42:22.477 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.477 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.477 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.477 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:22.477 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.477 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.477 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.477 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.477 INFO:teuthology.orchestra.run.smithi089.stdout: "server_port": { 2024-09-21T03:42:22.478 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "server_port", 2024-09-21T03:42:22.478 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2024-09-21T03:42:22.478 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.478 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:22.478 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "8080", 2024-09-21T03:42:22.478 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.478 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.478 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.478 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:22.478 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.478 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.478 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.478 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.478 INFO:teuthology.orchestra.run.smithi089.stdout: "sqlite3_killpoint": { 2024-09-21T03:42:22.478 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:42:22.479 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2024-09-21T03:42:22.479 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "dev", 2024-09-21T03:42:22.479 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:22.479 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "0", 2024-09-21T03:42:22.479 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.479 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.479 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.479 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:22.479 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.479 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.479 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.479 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.479 INFO:teuthology.orchestra.run.smithi089.stdout: "ssl": { 2024-09-21T03:42:22.479 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "ssl", 2024-09-21T03:42:22.479 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-21T03:42:22.479 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.479 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:22.480 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "True", 2024-09-21T03:42:22.480 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.480 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.480 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.480 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:22.480 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.480 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.480 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.480 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.480 INFO:teuthology.orchestra.run.smithi089.stdout: "ssl_server_port": { 2024-09-21T03:42:22.480 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "ssl_server_port", 2024-09-21T03:42:22.480 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2024-09-21T03:42:22.480 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.480 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:22.480 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "8443", 2024-09-21T03:42:22.480 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.481 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.481 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.481 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:22.481 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.481 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.481 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.481 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.481 INFO:teuthology.orchestra.run.smithi089.stdout: "sso_oauth2": { 2024-09-21T03:42:22.481 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "sso_oauth2", 2024-09-21T03:42:22.481 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-21T03:42:22.481 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.481 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:22.481 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-21T03:42:22.482 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.482 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.482 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.482 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:22.482 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.482 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.482 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.482 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.482 INFO:teuthology.orchestra.run.smithi089.stdout: "standby_behaviour": { 2024-09-21T03:42:22.482 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "standby_behaviour", 2024-09-21T03:42:22.482 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:22.482 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.482 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:22.482 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "redirect", 2024-09-21T03:42:22.482 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.483 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.483 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-21T03:42:22.483 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2024-09-21T03:42:22.483 INFO:teuthology.orchestra.run.smithi089.stdout: "redirect" 2024-09-21T03:42:22.483 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-21T03:42:22.483 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:22.483 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.483 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.483 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.483 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.483 INFO:teuthology.orchestra.run.smithi089.stdout: "standby_error_status_code": { 2024-09-21T03:42:22.483 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "standby_error_status_code", 2024-09-21T03:42:22.483 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2024-09-21T03:42:22.483 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.483 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:22.483 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "500", 2024-09-21T03:42:22.484 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "400", 2024-09-21T03:42:22.484 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "599", 2024-09-21T03:42:22.484 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.484 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:22.484 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.484 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.484 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.484 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.484 INFO:teuthology.orchestra.run.smithi089.stdout: "url_prefix": { 2024-09-21T03:42:22.484 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "url_prefix", 2024-09-21T03:42:22.484 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:22.484 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.484 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:22.484 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-21T03:42:22.484 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.484 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.485 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.485 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:22.485 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.485 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.485 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.485 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-21T03:42:22.485 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-21T03:42:22.485 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.485 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-21T03:42:22.485 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "devicehealth", 2024-09-21T03:42:22.485 INFO:teuthology.orchestra.run.smithi089.stdout: "can_run": true, 2024-09-21T03:42:22.485 INFO:teuthology.orchestra.run.smithi089.stdout: "error_string": "", 2024-09-21T03:42:22.485 INFO:teuthology.orchestra.run.smithi089.stdout: "module_options": { 2024-09-21T03:42:22.485 INFO:teuthology.orchestra.run.smithi089.stdout: "enable_monitoring": { 2024-09-21T03:42:22.485 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "enable_monitoring", 2024-09-21T03:42:22.485 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-21T03:42:22.486 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.486 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:22.486 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "True", 2024-09-21T03:42:22.486 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.486 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.486 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.486 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "monitor device health metrics", 2024-09-21T03:42:22.486 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.486 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.486 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.486 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.486 INFO:teuthology.orchestra.run.smithi089.stdout: "log_level": { 2024-09-21T03:42:22.486 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_level", 2024-09-21T03:42:22.486 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:22.487 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.487 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:22.487 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-21T03:42:22.487 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.487 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.487 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-21T03:42:22.487 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2024-09-21T03:42:22.487 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2024-09-21T03:42:22.487 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2024-09-21T03:42:22.487 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2024-09-21T03:42:22.487 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2024-09-21T03:42:22.487 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2024-09-21T03:42:22.487 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-21T03:42:22.487 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:22.487 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.488 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.488 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.489 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.489 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster": { 2024-09-21T03:42:22.489 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster", 2024-09-21T03:42:22.489 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-21T03:42:22.489 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.489 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:22.490 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-21T03:42:22.490 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.490 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.490 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.490 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:22.490 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.490 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.490 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.490 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.490 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster_level": { 2024-09-21T03:42:22.490 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster_level", 2024-09-21T03:42:22.490 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:22.490 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.490 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:22.490 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "info", 2024-09-21T03:42:22.491 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.491 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.491 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-21T03:42:22.491 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2024-09-21T03:42:22.491 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2024-09-21T03:42:22.491 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2024-09-21T03:42:22.491 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2024-09-21T03:42:22.491 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2024-09-21T03:42:22.491 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2024-09-21T03:42:22.491 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-21T03:42:22.491 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:22.491 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.491 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.492 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.492 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.492 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_file": { 2024-09-21T03:42:22.492 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_file", 2024-09-21T03:42:22.492 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-21T03:42:22.492 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.492 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:22.492 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-21T03:42:22.492 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.492 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.492 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.492 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:22.492 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.492 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.492 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.493 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.493 INFO:teuthology.orchestra.run.smithi089.stdout: "mark_out_threshold": { 2024-09-21T03:42:22.493 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "mark_out_threshold", 2024-09-21T03:42:22.493 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "secs", 2024-09-21T03:42:22.493 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.493 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:22.493 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "2419200", 2024-09-21T03:42:22.493 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.493 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.493 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.493 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-09-21T03:42:22.493 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.493 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.493 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.494 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.494 INFO:teuthology.orchestra.run.smithi089.stdout: "pool_name": { 2024-09-21T03:42:22.494 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "pool_name", 2024-09-21T03:42:22.494 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:22.494 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.494 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:22.494 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "device_health_metrics", 2024-09-21T03:42:22.494 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.494 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.494 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.494 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "name of pool in which to store device health metrics", 2024-09-21T03:42:22.494 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.494 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.494 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.494 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.495 INFO:teuthology.orchestra.run.smithi089.stdout: "retention_period": { 2024-09-21T03:42:22.495 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "retention_period", 2024-09-21T03:42:22.495 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "secs", 2024-09-21T03:42:22.495 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.495 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:22.495 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "15552000", 2024-09-21T03:42:22.495 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.495 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.495 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.495 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "how long to retain device health metrics", 2024-09-21T03:42:22.495 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.495 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.495 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.496 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.496 INFO:teuthology.orchestra.run.smithi089.stdout: "scrape_frequency": { 2024-09-21T03:42:22.496 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "scrape_frequency", 2024-09-21T03:42:22.496 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "secs", 2024-09-21T03:42:22.496 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.496 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:22.496 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "86400", 2024-09-21T03:42:22.496 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.496 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.496 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.496 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "how frequently to scrape device health metrics", 2024-09-21T03:42:22.496 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.496 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.496 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.497 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.497 INFO:teuthology.orchestra.run.smithi089.stdout: "self_heal": { 2024-09-21T03:42:22.497 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "self_heal", 2024-09-21T03:42:22.497 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-21T03:42:22.497 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.497 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:22.497 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "True", 2024-09-21T03:42:22.497 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.497 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.497 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.497 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-09-21T03:42:22.497 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.497 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.497 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.497 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.498 INFO:teuthology.orchestra.run.smithi089.stdout: "sleep_interval": { 2024-09-21T03:42:22.498 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "sleep_interval", 2024-09-21T03:42:22.498 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "secs", 2024-09-21T03:42:22.498 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.498 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:22.498 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "600", 2024-09-21T03:42:22.498 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.498 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.498 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.498 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "how frequently to wake up and check device health", 2024-09-21T03:42:22.498 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.498 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.498 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.498 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.498 INFO:teuthology.orchestra.run.smithi089.stdout: "sqlite3_killpoint": { 2024-09-21T03:42:22.499 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:42:22.499 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2024-09-21T03:42:22.499 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "dev", 2024-09-21T03:42:22.499 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:22.499 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "0", 2024-09-21T03:42:22.499 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.499 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.499 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.499 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:22.499 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.499 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.499 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.499 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.499 INFO:teuthology.orchestra.run.smithi089.stdout: "warn_threshold": { 2024-09-21T03:42:22.500 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "warn_threshold", 2024-09-21T03:42:22.500 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "secs", 2024-09-21T03:42:22.500 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.500 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:22.500 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "7257600", 2024-09-21T03:42:22.500 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.500 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.500 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.500 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-09-21T03:42:22.500 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.500 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.500 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.500 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-21T03:42:22.500 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-21T03:42:22.501 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.501 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-21T03:42:22.501 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "diskprediction_local", 2024-09-21T03:42:22.501 INFO:teuthology.orchestra.run.smithi089.stdout: "can_run": true, 2024-09-21T03:42:22.501 INFO:teuthology.orchestra.run.smithi089.stdout: "error_string": "", 2024-09-21T03:42:22.501 INFO:teuthology.orchestra.run.smithi089.stdout: "module_options": { 2024-09-21T03:42:22.501 INFO:teuthology.orchestra.run.smithi089.stdout: "log_level": { 2024-09-21T03:42:22.501 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_level", 2024-09-21T03:42:22.501 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:22.501 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.501 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:22.501 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-21T03:42:22.501 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.502 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.502 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-21T03:42:22.502 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2024-09-21T03:42:22.502 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2024-09-21T03:42:22.502 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2024-09-21T03:42:22.502 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2024-09-21T03:42:22.502 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2024-09-21T03:42:22.502 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2024-09-21T03:42:22.502 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-21T03:42:22.502 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:22.502 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.502 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.502 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.502 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.503 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster": { 2024-09-21T03:42:22.503 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster", 2024-09-21T03:42:22.503 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-21T03:42:22.503 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.503 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:22.503 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-21T03:42:22.503 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.503 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.503 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.503 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:22.503 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.503 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.503 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.503 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.504 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster_level": { 2024-09-21T03:42:22.504 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster_level", 2024-09-21T03:42:22.504 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:22.504 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.504 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:22.504 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "info", 2024-09-21T03:42:22.504 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.504 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.504 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-21T03:42:22.504 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2024-09-21T03:42:22.504 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2024-09-21T03:42:22.504 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2024-09-21T03:42:22.504 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2024-09-21T03:42:22.504 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2024-09-21T03:42:22.504 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2024-09-21T03:42:22.505 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-21T03:42:22.505 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:22.505 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.505 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.505 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.505 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.505 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_file": { 2024-09-21T03:42:22.505 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_file", 2024-09-21T03:42:22.505 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-21T03:42:22.505 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.505 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:22.505 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-21T03:42:22.505 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.505 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.506 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.506 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:22.506 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.506 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.506 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.506 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.506 INFO:teuthology.orchestra.run.smithi089.stdout: "predict_interval": { 2024-09-21T03:42:22.506 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "predict_interval", 2024-09-21T03:42:22.506 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:22.506 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.506 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:22.506 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "86400", 2024-09-21T03:42:22.506 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.506 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.507 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.507 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:22.507 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.507 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.507 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.507 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.507 INFO:teuthology.orchestra.run.smithi089.stdout: "predictor_model": { 2024-09-21T03:42:22.507 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "predictor_model", 2024-09-21T03:42:22.507 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:22.507 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.507 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:22.507 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "prophetstor", 2024-09-21T03:42:22.507 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.507 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.507 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.508 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:22.508 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.508 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.508 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.508 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.508 INFO:teuthology.orchestra.run.smithi089.stdout: "sleep_interval": { 2024-09-21T03:42:22.508 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "sleep_interval", 2024-09-21T03:42:22.508 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:22.508 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.508 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:22.508 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "600", 2024-09-21T03:42:22.508 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.508 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.508 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.509 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:22.509 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.509 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.509 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.509 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.509 INFO:teuthology.orchestra.run.smithi089.stdout: "sqlite3_killpoint": { 2024-09-21T03:42:22.509 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:42:22.509 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2024-09-21T03:42:22.509 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "dev", 2024-09-21T03:42:22.509 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:22.509 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "0", 2024-09-21T03:42:22.509 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.509 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.509 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.510 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:22.510 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.510 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.510 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.510 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-21T03:42:22.510 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-21T03:42:22.510 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.510 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-21T03:42:22.510 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "influx", 2024-09-21T03:42:22.510 INFO:teuthology.orchestra.run.smithi089.stdout: "can_run": false, 2024-09-21T03:42:22.510 INFO:teuthology.orchestra.run.smithi089.stdout: "error_string": "influxdb python module not found", 2024-09-21T03:42:22.510 INFO:teuthology.orchestra.run.smithi089.stdout: "module_options": { 2024-09-21T03:42:22.510 INFO:teuthology.orchestra.run.smithi089.stdout: "batch_size": { 2024-09-21T03:42:22.510 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "batch_size", 2024-09-21T03:42:22.510 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2024-09-21T03:42:22.511 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.511 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:22.511 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "5000", 2024-09-21T03:42:22.511 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.511 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.511 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.511 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-09-21T03:42:22.511 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.511 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.511 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.511 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.511 INFO:teuthology.orchestra.run.smithi089.stdout: "database": { 2024-09-21T03:42:22.511 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "database", 2024-09-21T03:42:22.511 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:22.511 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.511 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:22.512 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "ceph", 2024-09-21T03:42:22.512 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.512 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.512 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.512 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-09-21T03:42:22.512 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.512 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.512 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.512 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.512 INFO:teuthology.orchestra.run.smithi089.stdout: "hostname": { 2024-09-21T03:42:22.512 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "hostname", 2024-09-21T03:42:22.512 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:22.512 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.512 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:22.512 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-21T03:42:22.512 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.512 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.513 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.513 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "InfluxDB server hostname", 2024-09-21T03:42:22.513 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.513 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.513 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.513 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.513 INFO:teuthology.orchestra.run.smithi089.stdout: "interval": { 2024-09-21T03:42:22.513 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "interval", 2024-09-21T03:42:22.513 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "secs", 2024-09-21T03:42:22.513 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.513 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:22.513 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "30", 2024-09-21T03:42:22.513 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "5", 2024-09-21T03:42:22.513 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.513 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.513 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-09-21T03:42:22.514 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.514 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.514 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.514 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.514 INFO:teuthology.orchestra.run.smithi089.stdout: "log_level": { 2024-09-21T03:42:22.514 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_level", 2024-09-21T03:42:22.514 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:22.514 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.514 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:22.514 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-21T03:42:22.514 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.514 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.514 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-21T03:42:22.514 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2024-09-21T03:42:22.514 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2024-09-21T03:42:22.515 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2024-09-21T03:42:22.515 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2024-09-21T03:42:22.515 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2024-09-21T03:42:22.515 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2024-09-21T03:42:22.515 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-21T03:42:22.515 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:22.515 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.515 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.515 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.515 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.515 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster": { 2024-09-21T03:42:22.515 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster", 2024-09-21T03:42:22.515 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-21T03:42:22.515 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.515 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:22.515 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-21T03:42:22.516 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.516 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.516 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.516 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:22.516 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.516 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.516 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.516 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.516 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster_level": { 2024-09-21T03:42:22.516 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster_level", 2024-09-21T03:42:22.516 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:22.516 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.516 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:22.516 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "info", 2024-09-21T03:42:22.516 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.516 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.517 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-21T03:42:22.517 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2024-09-21T03:42:22.517 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2024-09-21T03:42:22.517 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2024-09-21T03:42:22.517 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2024-09-21T03:42:22.517 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2024-09-21T03:42:22.517 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2024-09-21T03:42:22.517 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-21T03:42:22.517 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:22.517 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.517 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.517 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.517 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.517 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_file": { 2024-09-21T03:42:22.517 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_file", 2024-09-21T03:42:22.517 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-21T03:42:22.517 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.518 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:22.518 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-21T03:42:22.518 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.518 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.518 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.518 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:22.518 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.518 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.518 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.518 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.518 INFO:teuthology.orchestra.run.smithi089.stdout: "password": { 2024-09-21T03:42:22.518 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "password", 2024-09-21T03:42:22.518 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:22.518 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.518 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:22.518 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-21T03:42:22.519 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.519 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.519 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.519 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "password of InfluxDB server user", 2024-09-21T03:42:22.519 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.519 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.519 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.519 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.519 INFO:teuthology.orchestra.run.smithi089.stdout: "port": { 2024-09-21T03:42:22.519 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "port", 2024-09-21T03:42:22.519 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2024-09-21T03:42:22.519 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.519 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:22.519 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "8086", 2024-09-21T03:42:22.519 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.519 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.519 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.520 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "InfluxDB server port", 2024-09-21T03:42:22.520 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.520 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.520 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.520 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.520 INFO:teuthology.orchestra.run.smithi089.stdout: "sqlite3_killpoint": { 2024-09-21T03:42:22.520 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:42:22.520 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2024-09-21T03:42:22.520 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "dev", 2024-09-21T03:42:22.520 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:22.520 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "0", 2024-09-21T03:42:22.520 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.520 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.520 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.520 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:22.521 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.521 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.521 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.521 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.521 INFO:teuthology.orchestra.run.smithi089.stdout: "ssl": { 2024-09-21T03:42:22.521 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "ssl", 2024-09-21T03:42:22.521 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:22.521 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.521 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:22.521 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "false", 2024-09-21T03:42:22.521 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.521 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.521 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.521 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-09-21T03:42:22.521 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.521 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.522 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.522 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.522 INFO:teuthology.orchestra.run.smithi089.stdout: "threads": { 2024-09-21T03:42:22.522 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "threads", 2024-09-21T03:42:22.522 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2024-09-21T03:42:22.522 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.522 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:22.522 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "5", 2024-09-21T03:42:22.522 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "1", 2024-09-21T03:42:22.522 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "32", 2024-09-21T03:42:22.522 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.522 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-09-21T03:42:22.522 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.522 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.522 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.523 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.523 INFO:teuthology.orchestra.run.smithi089.stdout: "username": { 2024-09-21T03:42:22.523 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "username", 2024-09-21T03:42:22.523 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:22.523 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.523 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:22.523 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-21T03:42:22.523 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.523 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.523 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.523 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "username of InfluxDB server user", 2024-09-21T03:42:22.523 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.523 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.523 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.524 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.524 INFO:teuthology.orchestra.run.smithi089.stdout: "verify_ssl": { 2024-09-21T03:42:22.524 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "verify_ssl", 2024-09-21T03:42:22.524 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:22.524 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.524 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:22.524 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "true", 2024-09-21T03:42:22.524 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.524 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.524 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.524 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-09-21T03:42:22.524 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.524 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.524 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.524 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-21T03:42:22.524 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-21T03:42:22.525 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.525 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-21T03:42:22.525 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "insights", 2024-09-21T03:42:22.525 INFO:teuthology.orchestra.run.smithi089.stdout: "can_run": true, 2024-09-21T03:42:22.525 INFO:teuthology.orchestra.run.smithi089.stdout: "error_string": "", 2024-09-21T03:42:22.525 INFO:teuthology.orchestra.run.smithi089.stdout: "module_options": { 2024-09-21T03:42:22.525 INFO:teuthology.orchestra.run.smithi089.stdout: "log_level": { 2024-09-21T03:42:22.525 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_level", 2024-09-21T03:42:22.525 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:22.525 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.525 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:22.525 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-21T03:42:22.525 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.525 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.525 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-21T03:42:22.526 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2024-09-21T03:42:22.526 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2024-09-21T03:42:22.526 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2024-09-21T03:42:22.526 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2024-09-21T03:42:22.526 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2024-09-21T03:42:22.526 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2024-09-21T03:42:22.526 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-21T03:42:22.526 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:22.526 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.526 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.526 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.526 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.526 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster": { 2024-09-21T03:42:22.527 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster", 2024-09-21T03:42:22.527 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-21T03:42:22.527 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.527 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:22.527 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-21T03:42:22.527 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.527 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.527 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.527 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:22.527 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.527 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.527 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.527 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.528 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster_level": { 2024-09-21T03:42:22.528 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster_level", 2024-09-21T03:42:22.528 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:22.528 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.528 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:22.528 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "info", 2024-09-21T03:42:22.528 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.528 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.528 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-21T03:42:22.528 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2024-09-21T03:42:22.528 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2024-09-21T03:42:22.528 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2024-09-21T03:42:22.528 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2024-09-21T03:42:22.528 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2024-09-21T03:42:22.529 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2024-09-21T03:42:22.529 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-21T03:42:22.529 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:22.529 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.529 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.529 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.529 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.529 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_file": { 2024-09-21T03:42:22.529 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_file", 2024-09-21T03:42:22.529 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-21T03:42:22.529 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.529 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:22.529 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-21T03:42:22.530 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.530 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.530 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.530 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:22.530 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.530 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.530 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.530 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.530 INFO:teuthology.orchestra.run.smithi089.stdout: "sqlite3_killpoint": { 2024-09-21T03:42:22.530 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:42:22.530 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2024-09-21T03:42:22.530 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "dev", 2024-09-21T03:42:22.530 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:22.531 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "0", 2024-09-21T03:42:22.531 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.531 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.531 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.531 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:22.531 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.531 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.531 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.531 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-21T03:42:22.531 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-21T03:42:22.531 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.531 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-21T03:42:22.532 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "iostat", 2024-09-21T03:42:22.532 INFO:teuthology.orchestra.run.smithi089.stdout: "can_run": true, 2024-09-21T03:42:22.532 INFO:teuthology.orchestra.run.smithi089.stdout: "error_string": "", 2024-09-21T03:42:22.532 INFO:teuthology.orchestra.run.smithi089.stdout: "module_options": { 2024-09-21T03:42:22.532 INFO:teuthology.orchestra.run.smithi089.stdout: "log_level": { 2024-09-21T03:42:22.532 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_level", 2024-09-21T03:42:22.532 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:22.532 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.532 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:22.532 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-21T03:42:22.532 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.532 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.532 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-21T03:42:22.532 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2024-09-21T03:42:22.533 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2024-09-21T03:42:22.533 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2024-09-21T03:42:22.533 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2024-09-21T03:42:22.533 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2024-09-21T03:42:22.533 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2024-09-21T03:42:22.533 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-21T03:42:22.533 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:22.533 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.533 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.533 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.533 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.533 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster": { 2024-09-21T03:42:22.533 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster", 2024-09-21T03:42:22.533 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-21T03:42:22.534 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.534 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:22.534 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-21T03:42:22.534 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.534 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.534 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.534 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:22.534 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.534 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.534 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.535 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.535 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster_level": { 2024-09-21T03:42:22.535 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster_level", 2024-09-21T03:42:22.535 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:22.535 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.535 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:22.535 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "info", 2024-09-21T03:42:22.535 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.535 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.535 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-21T03:42:22.535 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2024-09-21T03:42:22.535 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2024-09-21T03:42:22.536 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2024-09-21T03:42:22.536 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2024-09-21T03:42:22.536 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2024-09-21T03:42:22.536 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2024-09-21T03:42:22.536 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-21T03:42:22.536 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:22.536 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.536 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.536 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.536 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.536 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_file": { 2024-09-21T03:42:22.536 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_file", 2024-09-21T03:42:22.536 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-21T03:42:22.537 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.537 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:22.537 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-21T03:42:22.537 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.537 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.537 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.537 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:22.537 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.537 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.537 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.537 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.537 INFO:teuthology.orchestra.run.smithi089.stdout: "sqlite3_killpoint": { 2024-09-21T03:42:22.537 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:42:22.538 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2024-09-21T03:42:22.538 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "dev", 2024-09-21T03:42:22.538 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:22.538 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "0", 2024-09-21T03:42:22.538 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.538 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.538 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.538 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:22.538 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.538 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.538 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.538 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-21T03:42:22.538 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-21T03:42:22.539 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.539 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-21T03:42:22.539 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "k8sevents", 2024-09-21T03:42:22.539 INFO:teuthology.orchestra.run.smithi089.stdout: "can_run": true, 2024-09-21T03:42:22.539 INFO:teuthology.orchestra.run.smithi089.stdout: "error_string": "", 2024-09-21T03:42:22.539 INFO:teuthology.orchestra.run.smithi089.stdout: "module_options": { 2024-09-21T03:42:22.539 INFO:teuthology.orchestra.run.smithi089.stdout: "ceph_event_retention_days": { 2024-09-21T03:42:22.539 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "ceph_event_retention_days", 2024-09-21T03:42:22.539 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2024-09-21T03:42:22.539 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.539 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:22.540 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "7", 2024-09-21T03:42:22.540 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.540 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.540 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.540 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Days to hold ceph event information within local cache", 2024-09-21T03:42:22.540 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.540 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.540 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.540 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.540 INFO:teuthology.orchestra.run.smithi089.stdout: "config_check_secs": { 2024-09-21T03:42:22.540 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "config_check_secs", 2024-09-21T03:42:22.541 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2024-09-21T03:42:22.541 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.541 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:22.541 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "10", 2024-09-21T03:42:22.541 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "10", 2024-09-21T03:42:22.541 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.541 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.541 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-09-21T03:42:22.541 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.541 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.541 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.541 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.542 INFO:teuthology.orchestra.run.smithi089.stdout: "log_level": { 2024-09-21T03:42:22.542 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_level", 2024-09-21T03:42:22.542 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:22.542 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.542 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:22.542 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-21T03:42:22.542 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.542 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.542 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-21T03:42:22.542 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2024-09-21T03:42:22.542 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2024-09-21T03:42:22.543 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2024-09-21T03:42:22.543 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2024-09-21T03:42:22.543 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2024-09-21T03:42:22.543 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2024-09-21T03:42:22.543 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-21T03:42:22.543 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:22.543 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.543 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.543 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.543 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.544 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster": { 2024-09-21T03:42:22.544 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster", 2024-09-21T03:42:22.544 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-21T03:42:22.544 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.544 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:22.544 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-21T03:42:22.544 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.544 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.544 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.544 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:22.544 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.544 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.545 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.545 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.545 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster_level": { 2024-09-21T03:42:22.545 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster_level", 2024-09-21T03:42:22.545 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:22.545 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.545 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:22.545 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "info", 2024-09-21T03:42:22.545 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.545 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.545 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-21T03:42:22.546 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2024-09-21T03:42:22.546 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2024-09-21T03:42:22.546 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2024-09-21T03:42:22.546 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2024-09-21T03:42:22.546 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2024-09-21T03:42:22.546 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2024-09-21T03:42:22.546 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-21T03:42:22.546 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:22.546 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.546 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.546 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.546 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.547 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_file": { 2024-09-21T03:42:22.547 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_file", 2024-09-21T03:42:22.547 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-21T03:42:22.547 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.547 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:22.547 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-21T03:42:22.547 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.547 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.547 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.547 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:22.547 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.547 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.547 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.548 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.548 INFO:teuthology.orchestra.run.smithi089.stdout: "sqlite3_killpoint": { 2024-09-21T03:42:22.548 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:42:22.548 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2024-09-21T03:42:22.548 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "dev", 2024-09-21T03:42:22.548 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:22.548 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "0", 2024-09-21T03:42:22.548 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.548 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.548 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.548 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:22.548 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.548 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.548 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.549 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-21T03:42:22.549 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-21T03:42:22.549 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.549 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-21T03:42:22.549 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "localpool", 2024-09-21T03:42:22.549 INFO:teuthology.orchestra.run.smithi089.stdout: "can_run": true, 2024-09-21T03:42:22.549 INFO:teuthology.orchestra.run.smithi089.stdout: "error_string": "", 2024-09-21T03:42:22.549 INFO:teuthology.orchestra.run.smithi089.stdout: "module_options": { 2024-09-21T03:42:22.549 INFO:teuthology.orchestra.run.smithi089.stdout: "failure_domain": { 2024-09-21T03:42:22.549 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "failure_domain", 2024-09-21T03:42:22.549 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:22.549 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.549 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:22.550 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "host", 2024-09-21T03:42:22.550 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.550 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.550 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.550 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "failure domain for any created local pool", 2024-09-21T03:42:22.550 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-09-21T03:42:22.550 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.550 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.550 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.550 INFO:teuthology.orchestra.run.smithi089.stdout: "log_level": { 2024-09-21T03:42:22.550 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_level", 2024-09-21T03:42:22.550 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:22.551 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.551 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:22.551 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-21T03:42:22.551 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.551 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.551 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-21T03:42:22.551 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2024-09-21T03:42:22.551 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2024-09-21T03:42:22.551 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2024-09-21T03:42:22.551 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2024-09-21T03:42:22.551 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2024-09-21T03:42:22.551 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2024-09-21T03:42:22.552 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-21T03:42:22.552 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:22.552 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.552 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.552 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.552 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.552 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster": { 2024-09-21T03:42:22.552 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster", 2024-09-21T03:42:22.552 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-21T03:42:22.552 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.552 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:22.552 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-21T03:42:22.552 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.552 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.553 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.553 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:22.553 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.553 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.553 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.553 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.553 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster_level": { 2024-09-21T03:42:22.553 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster_level", 2024-09-21T03:42:22.553 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:22.553 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.553 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:22.553 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "info", 2024-09-21T03:42:22.554 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.554 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.554 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-21T03:42:22.554 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2024-09-21T03:42:22.554 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2024-09-21T03:42:22.554 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2024-09-21T03:42:22.554 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2024-09-21T03:42:22.554 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2024-09-21T03:42:22.554 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2024-09-21T03:42:22.554 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-21T03:42:22.554 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:22.554 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.554 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.555 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.555 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.555 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_file": { 2024-09-21T03:42:22.555 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_file", 2024-09-21T03:42:22.555 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-21T03:42:22.555 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.555 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:22.555 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-21T03:42:22.555 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.555 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.556 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.556 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:22.556 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.556 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.556 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.556 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.556 INFO:teuthology.orchestra.run.smithi089.stdout: "min_size": { 2024-09-21T03:42:22.556 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "min_size", 2024-09-21T03:42:22.556 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2024-09-21T03:42:22.556 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.556 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:22.557 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-21T03:42:22.557 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.557 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.557 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.557 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "default min_size for any created local pool", 2024-09-21T03:42:22.557 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-09-21T03:42:22.557 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.557 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.557 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.557 INFO:teuthology.orchestra.run.smithi089.stdout: "num_rep": { 2024-09-21T03:42:22.557 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "num_rep", 2024-09-21T03:42:22.557 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2024-09-21T03:42:22.558 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.558 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:22.558 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "3", 2024-09-21T03:42:22.558 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.558 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.558 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.558 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "default replica count for any created local pool", 2024-09-21T03:42:22.558 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.558 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.558 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.558 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.558 INFO:teuthology.orchestra.run.smithi089.stdout: "pg_num": { 2024-09-21T03:42:22.559 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "pg_num", 2024-09-21T03:42:22.559 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2024-09-21T03:42:22.559 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.559 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:22.559 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "128", 2024-09-21T03:42:22.559 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.559 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.559 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.559 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "default pg_num for any created local pool", 2024-09-21T03:42:22.559 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.559 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.559 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.559 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.560 INFO:teuthology.orchestra.run.smithi089.stdout: "prefix": { 2024-09-21T03:42:22.560 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "prefix", 2024-09-21T03:42:22.560 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:22.560 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.560 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:22.560 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-21T03:42:22.560 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.560 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.560 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.560 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "name prefix for any created local pool", 2024-09-21T03:42:22.560 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.560 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.560 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.561 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.561 INFO:teuthology.orchestra.run.smithi089.stdout: "sqlite3_killpoint": { 2024-09-21T03:42:22.561 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:42:22.561 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2024-09-21T03:42:22.561 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "dev", 2024-09-21T03:42:22.561 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:22.561 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "0", 2024-09-21T03:42:22.561 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.561 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.561 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.561 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:22.562 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.562 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.562 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.562 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.562 INFO:teuthology.orchestra.run.smithi089.stdout: "subtree": { 2024-09-21T03:42:22.562 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "subtree", 2024-09-21T03:42:22.562 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:22.562 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.562 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:22.562 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "rack", 2024-09-21T03:42:22.562 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.562 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.563 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.563 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "CRUSH level for which to create a local pool", 2024-09-21T03:42:22.563 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-09-21T03:42:22.563 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.563 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.563 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-21T03:42:22.563 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-21T03:42:22.563 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.563 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-21T03:42:22.563 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "mds_autoscaler", 2024-09-21T03:42:22.563 INFO:teuthology.orchestra.run.smithi089.stdout: "can_run": true, 2024-09-21T03:42:22.563 INFO:teuthology.orchestra.run.smithi089.stdout: "error_string": "", 2024-09-21T03:42:22.563 INFO:teuthology.orchestra.run.smithi089.stdout: "module_options": { 2024-09-21T03:42:22.564 INFO:teuthology.orchestra.run.smithi089.stdout: "log_level": { 2024-09-21T03:42:22.564 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_level", 2024-09-21T03:42:22.564 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:22.564 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.564 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:22.564 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-21T03:42:22.564 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.564 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.564 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-21T03:42:22.564 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2024-09-21T03:42:22.564 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2024-09-21T03:42:22.564 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2024-09-21T03:42:22.564 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2024-09-21T03:42:22.564 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2024-09-21T03:42:22.564 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2024-09-21T03:42:22.565 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-21T03:42:22.565 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:22.565 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.565 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.565 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.565 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.565 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster": { 2024-09-21T03:42:22.565 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster", 2024-09-21T03:42:22.565 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-21T03:42:22.565 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.566 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:22.566 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-21T03:42:22.566 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.566 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.566 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.566 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:22.566 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.566 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.566 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.566 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.566 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster_level": { 2024-09-21T03:42:22.566 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster_level", 2024-09-21T03:42:22.567 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:22.567 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.567 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:22.567 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "info", 2024-09-21T03:42:22.567 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.567 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.567 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-21T03:42:22.567 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2024-09-21T03:42:22.567 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2024-09-21T03:42:22.567 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2024-09-21T03:42:22.567 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2024-09-21T03:42:22.568 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2024-09-21T03:42:22.568 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2024-09-21T03:42:22.568 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-21T03:42:22.568 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:22.568 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.568 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.568 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.568 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.568 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_file": { 2024-09-21T03:42:22.568 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_file", 2024-09-21T03:42:22.568 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-21T03:42:22.568 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.569 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:22.569 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-21T03:42:22.569 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.569 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.569 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.569 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:22.569 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.569 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.569 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.569 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.569 INFO:teuthology.orchestra.run.smithi089.stdout: "sqlite3_killpoint": { 2024-09-21T03:42:22.569 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:42:22.569 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2024-09-21T03:42:22.570 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "dev", 2024-09-21T03:42:22.570 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:22.570 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "0", 2024-09-21T03:42:22.570 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.570 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.570 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.570 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:22.570 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.570 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.570 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.570 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-21T03:42:22.570 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-21T03:42:22.571 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.571 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-21T03:42:22.571 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "mirroring", 2024-09-21T03:42:22.571 INFO:teuthology.orchestra.run.smithi089.stdout: "can_run": true, 2024-09-21T03:42:22.571 INFO:teuthology.orchestra.run.smithi089.stdout: "error_string": "", 2024-09-21T03:42:22.571 INFO:teuthology.orchestra.run.smithi089.stdout: "module_options": { 2024-09-21T03:42:22.571 INFO:teuthology.orchestra.run.smithi089.stdout: "log_level": { 2024-09-21T03:42:22.571 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_level", 2024-09-21T03:42:22.571 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:22.571 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.571 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:22.571 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-21T03:42:22.572 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.572 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.572 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-21T03:42:22.572 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2024-09-21T03:42:22.572 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2024-09-21T03:42:22.572 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2024-09-21T03:42:22.572 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2024-09-21T03:42:22.572 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2024-09-21T03:42:22.572 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2024-09-21T03:42:22.572 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-21T03:42:22.572 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:22.572 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.573 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.573 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.573 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.573 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster": { 2024-09-21T03:42:22.573 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster", 2024-09-21T03:42:22.573 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-21T03:42:22.573 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.573 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:22.573 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-21T03:42:22.573 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.573 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.574 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.574 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:22.574 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.574 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.574 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.574 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.574 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster_level": { 2024-09-21T03:42:22.574 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster_level", 2024-09-21T03:42:22.574 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:22.574 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.574 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:22.574 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "info", 2024-09-21T03:42:22.575 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.575 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.575 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-21T03:42:22.575 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2024-09-21T03:42:22.575 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2024-09-21T03:42:22.575 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2024-09-21T03:42:22.575 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2024-09-21T03:42:22.575 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2024-09-21T03:42:22.575 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2024-09-21T03:42:22.575 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-21T03:42:22.575 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:22.575 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.575 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.576 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.576 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.576 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_file": { 2024-09-21T03:42:22.576 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_file", 2024-09-21T03:42:22.576 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-21T03:42:22.576 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.576 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:22.576 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-21T03:42:22.576 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.576 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.576 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.576 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:22.576 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.576 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.577 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.577 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.577 INFO:teuthology.orchestra.run.smithi089.stdout: "sqlite3_killpoint": { 2024-09-21T03:42:22.577 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:42:22.577 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2024-09-21T03:42:22.577 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "dev", 2024-09-21T03:42:22.577 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:22.577 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "0", 2024-09-21T03:42:22.577 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.577 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.577 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.577 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:22.577 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.578 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.578 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.578 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-21T03:42:22.578 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-21T03:42:22.578 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.578 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-21T03:42:22.578 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "nfs", 2024-09-21T03:42:22.578 INFO:teuthology.orchestra.run.smithi089.stdout: "can_run": true, 2024-09-21T03:42:22.578 INFO:teuthology.orchestra.run.smithi089.stdout: "error_string": "", 2024-09-21T03:42:22.578 INFO:teuthology.orchestra.run.smithi089.stdout: "module_options": { 2024-09-21T03:42:22.578 INFO:teuthology.orchestra.run.smithi089.stdout: "log_level": { 2024-09-21T03:42:22.578 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_level", 2024-09-21T03:42:22.578 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:22.579 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.579 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:22.579 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-21T03:42:22.579 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.579 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.579 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-21T03:42:22.579 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2024-09-21T03:42:22.579 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2024-09-21T03:42:22.579 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2024-09-21T03:42:22.579 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2024-09-21T03:42:22.579 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2024-09-21T03:42:22.579 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2024-09-21T03:42:22.580 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-21T03:42:22.580 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:22.580 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.580 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.580 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.580 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.580 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster": { 2024-09-21T03:42:22.580 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster", 2024-09-21T03:42:22.580 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-21T03:42:22.580 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.580 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:22.581 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-21T03:42:22.581 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.581 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.581 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.581 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:22.581 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.581 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.581 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.581 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.581 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster_level": { 2024-09-21T03:42:22.581 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster_level", 2024-09-21T03:42:22.581 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:22.581 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.582 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:22.582 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "info", 2024-09-21T03:42:22.582 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.582 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.582 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-21T03:42:22.582 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2024-09-21T03:42:22.582 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2024-09-21T03:42:22.582 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2024-09-21T03:42:22.582 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2024-09-21T03:42:22.582 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2024-09-21T03:42:22.582 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2024-09-21T03:42:22.582 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-21T03:42:22.582 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:22.583 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.583 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.583 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.583 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.583 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_file": { 2024-09-21T03:42:22.583 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_file", 2024-09-21T03:42:22.583 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-21T03:42:22.583 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.583 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:22.583 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-21T03:42:22.583 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.583 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.583 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.584 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:22.584 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.584 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.584 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.584 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.584 INFO:teuthology.orchestra.run.smithi089.stdout: "sqlite3_killpoint": { 2024-09-21T03:42:22.584 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:42:22.584 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2024-09-21T03:42:22.584 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "dev", 2024-09-21T03:42:22.584 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:22.584 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "0", 2024-09-21T03:42:22.584 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.584 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.585 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.585 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:22.585 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.585 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.585 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.585 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-21T03:42:22.585 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-21T03:42:22.585 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.585 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-21T03:42:22.585 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "orchestrator", 2024-09-21T03:42:22.585 INFO:teuthology.orchestra.run.smithi089.stdout: "can_run": true, 2024-09-21T03:42:22.585 INFO:teuthology.orchestra.run.smithi089.stdout: "error_string": "", 2024-09-21T03:42:22.586 INFO:teuthology.orchestra.run.smithi089.stdout: "module_options": { 2024-09-21T03:42:22.586 INFO:teuthology.orchestra.run.smithi089.stdout: "fail_fs": { 2024-09-21T03:42:22.586 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "fail_fs", 2024-09-21T03:42:22.586 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-21T03:42:22.586 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.586 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:22.586 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-21T03:42:22.586 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.586 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.586 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.586 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-09-21T03:42:22.586 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.586 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.587 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.587 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.587 INFO:teuthology.orchestra.run.smithi089.stdout: "log_level": { 2024-09-21T03:42:22.587 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_level", 2024-09-21T03:42:22.587 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:22.587 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.587 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:22.587 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-21T03:42:22.587 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.587 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.587 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-21T03:42:22.587 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2024-09-21T03:42:22.587 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2024-09-21T03:42:22.588 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2024-09-21T03:42:22.588 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2024-09-21T03:42:22.588 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2024-09-21T03:42:22.588 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2024-09-21T03:42:22.588 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-21T03:42:22.588 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:22.588 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.588 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.588 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.588 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.588 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster": { 2024-09-21T03:42:22.588 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster", 2024-09-21T03:42:22.588 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-21T03:42:22.588 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.589 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:22.589 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-21T03:42:22.589 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.589 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.589 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.589 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:22.589 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.589 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.589 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.589 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.589 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster_level": { 2024-09-21T03:42:22.589 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster_level", 2024-09-21T03:42:22.589 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:22.589 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.590 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:22.590 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "info", 2024-09-21T03:42:22.590 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.590 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.590 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-21T03:42:22.590 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2024-09-21T03:42:22.590 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2024-09-21T03:42:22.590 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2024-09-21T03:42:22.590 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2024-09-21T03:42:22.590 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2024-09-21T03:42:22.590 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2024-09-21T03:42:22.590 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-21T03:42:22.590 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:22.591 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.591 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.591 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.591 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.591 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_file": { 2024-09-21T03:42:22.591 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_file", 2024-09-21T03:42:22.591 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-21T03:42:22.591 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.591 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:22.591 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-21T03:42:22.591 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.591 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.592 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.592 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:22.592 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.592 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.592 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.592 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.592 INFO:teuthology.orchestra.run.smithi089.stdout: "orchestrator": { 2024-09-21T03:42:22.592 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "orchestrator", 2024-09-21T03:42:22.592 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:22.592 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.592 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:22.592 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-21T03:42:22.592 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.592 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.593 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-21T03:42:22.593 INFO:teuthology.orchestra.run.smithi089.stdout: "cephadm", 2024-09-21T03:42:22.593 INFO:teuthology.orchestra.run.smithi089.stdout: "rook", 2024-09-21T03:42:22.593 INFO:teuthology.orchestra.run.smithi089.stdout: "test_orchestrator" 2024-09-21T03:42:22.593 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-21T03:42:22.593 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Orchestrator backend", 2024-09-21T03:42:22.593 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.593 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.593 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.593 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.593 INFO:teuthology.orchestra.run.smithi089.stdout: "sqlite3_killpoint": { 2024-09-21T03:42:22.593 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:42:22.593 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2024-09-21T03:42:22.593 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "dev", 2024-09-21T03:42:22.594 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:22.594 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "0", 2024-09-21T03:42:22.594 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.594 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.594 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.594 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:22.594 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.594 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.594 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.594 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-21T03:42:22.594 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-21T03:42:22.594 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.594 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-21T03:42:22.595 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "osd_perf_query", 2024-09-21T03:42:22.595 INFO:teuthology.orchestra.run.smithi089.stdout: "can_run": true, 2024-09-21T03:42:22.595 INFO:teuthology.orchestra.run.smithi089.stdout: "error_string": "", 2024-09-21T03:42:22.595 INFO:teuthology.orchestra.run.smithi089.stdout: "module_options": { 2024-09-21T03:42:22.595 INFO:teuthology.orchestra.run.smithi089.stdout: "log_level": { 2024-09-21T03:42:22.595 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_level", 2024-09-21T03:42:22.595 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:22.595 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.595 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:22.595 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-21T03:42:22.595 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.595 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.595 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-21T03:42:22.596 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2024-09-21T03:42:22.596 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2024-09-21T03:42:22.596 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2024-09-21T03:42:22.596 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2024-09-21T03:42:22.596 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2024-09-21T03:42:22.596 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2024-09-21T03:42:22.596 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-21T03:42:22.596 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:22.596 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.596 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.596 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.596 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.597 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster": { 2024-09-21T03:42:22.597 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster", 2024-09-21T03:42:22.597 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-21T03:42:22.597 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.597 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:22.597 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-21T03:42:22.597 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.597 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.597 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.597 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:22.597 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.597 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.598 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.598 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.598 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster_level": { 2024-09-21T03:42:22.598 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster_level", 2024-09-21T03:42:22.598 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:22.598 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.598 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:22.598 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "info", 2024-09-21T03:42:22.598 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.598 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.598 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-21T03:42:22.598 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2024-09-21T03:42:22.599 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2024-09-21T03:42:22.599 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2024-09-21T03:42:22.599 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2024-09-21T03:42:22.599 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2024-09-21T03:42:22.599 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2024-09-21T03:42:22.599 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-21T03:42:22.599 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:22.599 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.599 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.599 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.599 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.599 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_file": { 2024-09-21T03:42:22.599 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_file", 2024-09-21T03:42:22.600 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-21T03:42:22.600 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.600 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:22.600 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-21T03:42:22.600 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.600 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.600 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.600 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:22.600 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.600 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.600 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.600 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.600 INFO:teuthology.orchestra.run.smithi089.stdout: "sqlite3_killpoint": { 2024-09-21T03:42:22.600 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:42:22.601 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2024-09-21T03:42:22.601 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "dev", 2024-09-21T03:42:22.601 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:22.601 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "0", 2024-09-21T03:42:22.601 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.601 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.601 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.601 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:22.601 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.601 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.601 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.601 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-21T03:42:22.601 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-21T03:42:22.602 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.602 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-21T03:42:22.602 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "osd_support", 2024-09-21T03:42:22.602 INFO:teuthology.orchestra.run.smithi089.stdout: "can_run": true, 2024-09-21T03:42:22.602 INFO:teuthology.orchestra.run.smithi089.stdout: "error_string": "", 2024-09-21T03:42:22.602 INFO:teuthology.orchestra.run.smithi089.stdout: "module_options": { 2024-09-21T03:42:22.602 INFO:teuthology.orchestra.run.smithi089.stdout: "log_level": { 2024-09-21T03:42:22.602 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_level", 2024-09-21T03:42:22.602 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:22.602 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.602 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:22.603 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-21T03:42:22.603 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.603 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.603 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-21T03:42:22.603 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2024-09-21T03:42:22.603 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2024-09-21T03:42:22.603 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2024-09-21T03:42:22.603 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2024-09-21T03:42:22.603 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2024-09-21T03:42:22.603 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2024-09-21T03:42:22.603 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-21T03:42:22.603 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:22.604 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.604 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.604 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.604 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.604 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster": { 2024-09-21T03:42:22.604 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster", 2024-09-21T03:42:22.604 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-21T03:42:22.604 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.604 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:22.604 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-21T03:42:22.604 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.605 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.605 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.605 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:22.605 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.605 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.605 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.605 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.605 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster_level": { 2024-09-21T03:42:22.605 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster_level", 2024-09-21T03:42:22.605 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:22.605 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.605 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:22.606 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "info", 2024-09-21T03:42:22.606 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.606 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.606 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-21T03:42:22.606 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2024-09-21T03:42:22.606 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2024-09-21T03:42:22.606 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2024-09-21T03:42:22.606 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2024-09-21T03:42:22.606 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2024-09-21T03:42:22.606 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2024-09-21T03:42:22.606 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-21T03:42:22.607 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:22.607 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.607 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.607 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.607 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.607 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_file": { 2024-09-21T03:42:22.607 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_file", 2024-09-21T03:42:22.607 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-21T03:42:22.607 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.607 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:22.607 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-21T03:42:22.608 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.608 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.608 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.608 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:22.608 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.608 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.608 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.608 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.608 INFO:teuthology.orchestra.run.smithi089.stdout: "sqlite3_killpoint": { 2024-09-21T03:42:22.608 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:42:22.609 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2024-09-21T03:42:22.609 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "dev", 2024-09-21T03:42:22.609 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:22.609 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "0", 2024-09-21T03:42:22.609 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.609 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.609 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.609 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:22.609 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.609 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.609 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.610 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-21T03:42:22.610 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-21T03:42:22.610 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.610 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-21T03:42:22.610 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "pg_autoscaler", 2024-09-21T03:42:22.610 INFO:teuthology.orchestra.run.smithi089.stdout: "can_run": true, 2024-09-21T03:42:22.610 INFO:teuthology.orchestra.run.smithi089.stdout: "error_string": "", 2024-09-21T03:42:22.610 INFO:teuthology.orchestra.run.smithi089.stdout: "module_options": { 2024-09-21T03:42:22.610 INFO:teuthology.orchestra.run.smithi089.stdout: "log_level": { 2024-09-21T03:42:22.610 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_level", 2024-09-21T03:42:22.610 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:22.610 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.611 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:22.611 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-21T03:42:22.611 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.611 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.611 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-21T03:42:22.611 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2024-09-21T03:42:22.611 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2024-09-21T03:42:22.611 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2024-09-21T03:42:22.611 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2024-09-21T03:42:22.611 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2024-09-21T03:42:22.611 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2024-09-21T03:42:22.611 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-21T03:42:22.611 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:22.612 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.612 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.612 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.612 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.612 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster": { 2024-09-21T03:42:22.612 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster", 2024-09-21T03:42:22.612 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-21T03:42:22.612 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.612 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:22.612 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-21T03:42:22.612 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.612 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.613 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.613 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:22.613 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.613 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.613 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.613 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.613 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster_level": { 2024-09-21T03:42:22.613 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster_level", 2024-09-21T03:42:22.613 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:22.613 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.614 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:22.614 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "info", 2024-09-21T03:42:22.614 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.614 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.614 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-21T03:42:22.614 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2024-09-21T03:42:22.614 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2024-09-21T03:42:22.614 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2024-09-21T03:42:22.614 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2024-09-21T03:42:22.614 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2024-09-21T03:42:22.614 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2024-09-21T03:42:22.615 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-21T03:42:22.615 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:22.615 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.615 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.615 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.615 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.615 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_file": { 2024-09-21T03:42:22.615 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_file", 2024-09-21T03:42:22.615 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-21T03:42:22.615 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.615 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:22.616 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-21T03:42:22.616 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.616 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.616 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.616 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:22.616 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.616 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.616 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.616 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.616 INFO:teuthology.orchestra.run.smithi089.stdout: "sleep_interval": { 2024-09-21T03:42:22.616 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "sleep_interval", 2024-09-21T03:42:22.616 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "secs", 2024-09-21T03:42:22.616 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.617 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:22.617 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "60", 2024-09-21T03:42:22.617 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.617 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.617 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.617 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:22.617 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.617 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.617 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.617 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.617 INFO:teuthology.orchestra.run.smithi089.stdout: "sqlite3_killpoint": { 2024-09-21T03:42:22.617 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:42:22.617 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2024-09-21T03:42:22.617 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "dev", 2024-09-21T03:42:22.618 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:22.618 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "0", 2024-09-21T03:42:22.618 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.618 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.618 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.618 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:22.618 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.618 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.618 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.618 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.618 INFO:teuthology.orchestra.run.smithi089.stdout: "threshold": { 2024-09-21T03:42:22.619 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "threshold", 2024-09-21T03:42:22.619 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "float", 2024-09-21T03:42:22.619 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.619 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:22.619 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "3.0", 2024-09-21T03:42:22.619 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "1.0", 2024-09-21T03:42:22.619 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.619 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.619 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "scaling threshold", 2024-09-21T03:42:22.619 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-09-21T03:42:22.619 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.619 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.620 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-21T03:42:22.620 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-21T03:42:22.620 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.620 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-21T03:42:22.620 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "progress", 2024-09-21T03:42:22.620 INFO:teuthology.orchestra.run.smithi089.stdout: "can_run": true, 2024-09-21T03:42:22.620 INFO:teuthology.orchestra.run.smithi089.stdout: "error_string": "", 2024-09-21T03:42:22.620 INFO:teuthology.orchestra.run.smithi089.stdout: "module_options": { 2024-09-21T03:42:22.620 INFO:teuthology.orchestra.run.smithi089.stdout: "allow_pg_recovery_event": { 2024-09-21T03:42:22.620 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "allow_pg_recovery_event", 2024-09-21T03:42:22.620 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-21T03:42:22.620 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.620 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:22.621 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-21T03:42:22.621 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.621 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.621 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.621 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "allow the module to show pg recovery progress", 2024-09-21T03:42:22.621 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.621 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.621 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.621 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.621 INFO:teuthology.orchestra.run.smithi089.stdout: "enabled": { 2024-09-21T03:42:22.621 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "enabled", 2024-09-21T03:42:22.621 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-21T03:42:22.621 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.622 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:22.622 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "True", 2024-09-21T03:42:22.622 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.622 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.622 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.622 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:22.622 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.622 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.622 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.622 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.622 INFO:teuthology.orchestra.run.smithi089.stdout: "log_level": { 2024-09-21T03:42:22.622 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_level", 2024-09-21T03:42:22.622 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:22.623 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.623 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:22.623 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-21T03:42:22.623 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.623 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.623 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-21T03:42:22.623 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2024-09-21T03:42:22.623 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2024-09-21T03:42:22.623 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2024-09-21T03:42:22.623 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2024-09-21T03:42:22.623 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2024-09-21T03:42:22.623 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2024-09-21T03:42:22.623 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-21T03:42:22.623 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:22.624 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.624 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.624 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.624 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.624 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster": { 2024-09-21T03:42:22.624 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster", 2024-09-21T03:42:22.624 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-21T03:42:22.624 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.624 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:22.624 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-21T03:42:22.624 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.624 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.624 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.624 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:22.624 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.625 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.625 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.625 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.625 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster_level": { 2024-09-21T03:42:22.625 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster_level", 2024-09-21T03:42:22.625 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:22.625 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.625 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:22.625 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "info", 2024-09-21T03:42:22.625 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.625 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.625 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-21T03:42:22.625 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2024-09-21T03:42:22.625 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2024-09-21T03:42:22.626 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2024-09-21T03:42:22.626 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2024-09-21T03:42:22.626 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2024-09-21T03:42:22.626 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2024-09-21T03:42:22.626 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-21T03:42:22.626 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:22.626 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.626 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.626 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.626 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.626 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_file": { 2024-09-21T03:42:22.626 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_file", 2024-09-21T03:42:22.627 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-21T03:42:22.627 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.627 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:22.627 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-21T03:42:22.627 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.627 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.627 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.627 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:22.627 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.627 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.627 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.627 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.627 INFO:teuthology.orchestra.run.smithi089.stdout: "max_completed_events": { 2024-09-21T03:42:22.628 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "max_completed_events", 2024-09-21T03:42:22.628 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2024-09-21T03:42:22.628 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.628 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:22.628 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "50", 2024-09-21T03:42:22.628 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.628 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.628 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.628 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "number of past completed events to remember", 2024-09-21T03:42:22.628 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.629 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.629 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.629 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.629 INFO:teuthology.orchestra.run.smithi089.stdout: "sleep_interval": { 2024-09-21T03:42:22.629 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "sleep_interval", 2024-09-21T03:42:22.629 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "secs", 2024-09-21T03:42:22.629 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.629 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:22.629 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "5", 2024-09-21T03:42:22.629 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.629 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.629 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.630 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "how long the module is going to sleep", 2024-09-21T03:42:22.630 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.630 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.630 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.630 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.630 INFO:teuthology.orchestra.run.smithi089.stdout: "sqlite3_killpoint": { 2024-09-21T03:42:22.630 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:42:22.630 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2024-09-21T03:42:22.630 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "dev", 2024-09-21T03:42:22.630 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:22.630 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "0", 2024-09-21T03:42:22.630 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.630 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.631 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.631 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:22.631 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.631 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.631 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.631 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-21T03:42:22.631 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-21T03:42:22.631 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.631 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-21T03:42:22.631 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "prometheus", 2024-09-21T03:42:22.631 INFO:teuthology.orchestra.run.smithi089.stdout: "can_run": true, 2024-09-21T03:42:22.631 INFO:teuthology.orchestra.run.smithi089.stdout: "error_string": "", 2024-09-21T03:42:22.631 INFO:teuthology.orchestra.run.smithi089.stdout: "module_options": { 2024-09-21T03:42:22.632 INFO:teuthology.orchestra.run.smithi089.stdout: "cache": { 2024-09-21T03:42:22.632 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "cache", 2024-09-21T03:42:22.632 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-21T03:42:22.632 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.632 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:22.632 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "True", 2024-09-21T03:42:22.632 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.632 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.632 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.632 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:22.632 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.632 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.632 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.632 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.633 INFO:teuthology.orchestra.run.smithi089.stdout: "exclude_perf_counters": { 2024-09-21T03:42:22.633 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "exclude_perf_counters", 2024-09-21T03:42:22.633 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-21T03:42:22.633 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.633 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:22.633 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "True", 2024-09-21T03:42:22.633 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.633 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.633 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.633 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-09-21T03:42:22.633 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-09-21T03:42:22.633 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.634 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.634 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.634 INFO:teuthology.orchestra.run.smithi089.stdout: "log_level": { 2024-09-21T03:42:22.634 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_level", 2024-09-21T03:42:22.634 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:22.634 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.634 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:22.634 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-21T03:42:22.634 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.634 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.634 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-21T03:42:22.634 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2024-09-21T03:42:22.634 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2024-09-21T03:42:22.635 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2024-09-21T03:42:22.635 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2024-09-21T03:42:22.635 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2024-09-21T03:42:22.635 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2024-09-21T03:42:22.635 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-21T03:42:22.635 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:22.635 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.635 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.635 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.635 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.635 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster": { 2024-09-21T03:42:22.635 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster", 2024-09-21T03:42:22.635 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-21T03:42:22.636 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.636 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:22.636 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-21T03:42:22.636 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.636 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.636 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.636 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:22.636 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.636 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.636 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.636 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.636 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster_level": { 2024-09-21T03:42:22.636 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster_level", 2024-09-21T03:42:22.636 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:22.637 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.637 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:22.637 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "info", 2024-09-21T03:42:22.637 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.637 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.637 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-21T03:42:22.637 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2024-09-21T03:42:22.637 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2024-09-21T03:42:22.637 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2024-09-21T03:42:22.637 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2024-09-21T03:42:22.637 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2024-09-21T03:42:22.637 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2024-09-21T03:42:22.638 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-21T03:42:22.638 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:22.638 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.638 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.638 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.638 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.638 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_file": { 2024-09-21T03:42:22.638 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_file", 2024-09-21T03:42:22.638 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-21T03:42:22.638 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.638 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:22.638 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-21T03:42:22.639 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.639 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.639 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.639 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:22.639 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.639 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.639 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.639 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.639 INFO:teuthology.orchestra.run.smithi089.stdout: "rbd_stats_pools": { 2024-09-21T03:42:22.639 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "rbd_stats_pools", 2024-09-21T03:42:22.639 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:22.639 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.639 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:22.640 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-21T03:42:22.640 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.640 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.640 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.640 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:22.640 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.640 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.640 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.640 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.640 INFO:teuthology.orchestra.run.smithi089.stdout: "rbd_stats_pools_refresh_interval": { 2024-09-21T03:42:22.640 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-09-21T03:42:22.640 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2024-09-21T03:42:22.640 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.640 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:22.641 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "300", 2024-09-21T03:42:22.641 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.641 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.641 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.641 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:22.641 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.641 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.641 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.641 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.641 INFO:teuthology.orchestra.run.smithi089.stdout: "scrape_interval": { 2024-09-21T03:42:22.641 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "scrape_interval", 2024-09-21T03:42:22.641 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "float", 2024-09-21T03:42:22.641 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.641 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:22.641 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "15.0", 2024-09-21T03:42:22.642 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.642 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.642 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.642 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:22.642 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.642 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.642 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.642 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.642 INFO:teuthology.orchestra.run.smithi089.stdout: "server_addr": { 2024-09-21T03:42:22.642 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "server_addr", 2024-09-21T03:42:22.642 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:22.642 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.642 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:22.643 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "::", 2024-09-21T03:42:22.643 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.643 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.643 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.643 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-09-21T03:42:22.643 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.643 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.643 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.643 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.643 INFO:teuthology.orchestra.run.smithi089.stdout: "server_port": { 2024-09-21T03:42:22.643 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "server_port", 2024-09-21T03:42:22.643 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2024-09-21T03:42:22.643 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.644 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:22.644 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "9283", 2024-09-21T03:42:22.644 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.644 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.644 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.644 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-09-21T03:42:22.644 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.644 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.644 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.644 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.644 INFO:teuthology.orchestra.run.smithi089.stdout: "sqlite3_killpoint": { 2024-09-21T03:42:22.644 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:42:22.645 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2024-09-21T03:42:22.645 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "dev", 2024-09-21T03:42:22.645 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:22.645 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "0", 2024-09-21T03:42:22.645 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.645 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.645 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.645 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:22.645 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.645 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.645 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.645 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.646 INFO:teuthology.orchestra.run.smithi089.stdout: "stale_cache_strategy": { 2024-09-21T03:42:22.646 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "stale_cache_strategy", 2024-09-21T03:42:22.646 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:22.646 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.646 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:22.646 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "log", 2024-09-21T03:42:22.646 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.646 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.646 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.646 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:22.646 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.647 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.647 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.647 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.647 INFO:teuthology.orchestra.run.smithi089.stdout: "standby_behaviour": { 2024-09-21T03:42:22.647 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "standby_behaviour", 2024-09-21T03:42:22.647 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:22.647 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.647 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:22.647 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "default", 2024-09-21T03:42:22.647 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.647 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.647 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-21T03:42:22.648 INFO:teuthology.orchestra.run.smithi089.stdout: "default", 2024-09-21T03:42:22.648 INFO:teuthology.orchestra.run.smithi089.stdout: "error" 2024-09-21T03:42:22.648 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-21T03:42:22.648 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:22.648 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.648 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.648 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.648 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.648 INFO:teuthology.orchestra.run.smithi089.stdout: "standby_error_status_code": { 2024-09-21T03:42:22.648 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "standby_error_status_code", 2024-09-21T03:42:22.648 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2024-09-21T03:42:22.648 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.649 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:22.649 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "500", 2024-09-21T03:42:22.649 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "400", 2024-09-21T03:42:22.649 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "599", 2024-09-21T03:42:22.649 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.649 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:22.649 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.649 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.649 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.649 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-21T03:42:22.649 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-21T03:42:22.649 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.650 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-21T03:42:22.650 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "rbd_support", 2024-09-21T03:42:22.650 INFO:teuthology.orchestra.run.smithi089.stdout: "can_run": true, 2024-09-21T03:42:22.650 INFO:teuthology.orchestra.run.smithi089.stdout: "error_string": "", 2024-09-21T03:42:22.650 INFO:teuthology.orchestra.run.smithi089.stdout: "module_options": { 2024-09-21T03:42:22.650 INFO:teuthology.orchestra.run.smithi089.stdout: "log_level": { 2024-09-21T03:42:22.650 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_level", 2024-09-21T03:42:22.650 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:22.650 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.650 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:22.650 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-21T03:42:22.651 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.651 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.651 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-21T03:42:22.651 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2024-09-21T03:42:22.651 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2024-09-21T03:42:22.651 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2024-09-21T03:42:22.651 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2024-09-21T03:42:22.651 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2024-09-21T03:42:22.651 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2024-09-21T03:42:22.651 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-21T03:42:22.651 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:22.651 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.651 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.651 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.652 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.652 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster": { 2024-09-21T03:42:22.652 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster", 2024-09-21T03:42:22.652 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-21T03:42:22.652 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.652 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:22.652 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-21T03:42:22.652 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.652 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.652 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.652 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:22.652 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.652 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.652 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.653 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.653 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster_level": { 2024-09-21T03:42:22.653 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster_level", 2024-09-21T03:42:22.653 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:22.653 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.653 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:22.653 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "info", 2024-09-21T03:42:22.653 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.653 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.653 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-21T03:42:22.653 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2024-09-21T03:42:22.653 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2024-09-21T03:42:22.653 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2024-09-21T03:42:22.653 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2024-09-21T03:42:22.654 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2024-09-21T03:42:22.654 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2024-09-21T03:42:22.654 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-21T03:42:22.654 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:22.654 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.654 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.654 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.654 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.654 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_file": { 2024-09-21T03:42:22.654 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_file", 2024-09-21T03:42:22.654 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-21T03:42:22.655 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.655 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:22.655 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-21T03:42:22.655 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.655 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.655 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.655 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:22.655 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.655 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.655 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.655 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.655 INFO:teuthology.orchestra.run.smithi089.stdout: "max_concurrent_snap_create": { 2024-09-21T03:42:22.656 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "max_concurrent_snap_create", 2024-09-21T03:42:22.656 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2024-09-21T03:42:22.656 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.656 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:22.656 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "10", 2024-09-21T03:42:22.656 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.656 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.656 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.656 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:22.656 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.656 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.656 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.656 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.657 INFO:teuthology.orchestra.run.smithi089.stdout: "mirror_snapshot_schedule": { 2024-09-21T03:42:22.657 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "mirror_snapshot_schedule", 2024-09-21T03:42:22.657 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:22.657 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.657 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:22.657 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-21T03:42:22.657 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.657 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.657 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.657 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:22.658 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.658 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.658 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.658 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.658 INFO:teuthology.orchestra.run.smithi089.stdout: "sqlite3_killpoint": { 2024-09-21T03:42:22.658 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:42:22.658 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2024-09-21T03:42:22.658 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "dev", 2024-09-21T03:42:22.658 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:22.658 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "0", 2024-09-21T03:42:22.658 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.659 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.659 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.659 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:22.659 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.659 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.659 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.659 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.659 INFO:teuthology.orchestra.run.smithi089.stdout: "trash_purge_schedule": { 2024-09-21T03:42:22.659 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "trash_purge_schedule", 2024-09-21T03:42:22.659 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:22.659 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.659 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:22.660 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-21T03:42:22.660 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.660 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.660 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.660 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:22.660 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.660 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.660 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.660 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-21T03:42:22.660 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-21T03:42:22.660 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.660 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-21T03:42:22.660 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "restful", 2024-09-21T03:42:22.661 INFO:teuthology.orchestra.run.smithi089.stdout: "can_run": true, 2024-09-21T03:42:22.661 INFO:teuthology.orchestra.run.smithi089.stdout: "error_string": "", 2024-09-21T03:42:22.661 INFO:teuthology.orchestra.run.smithi089.stdout: "module_options": { 2024-09-21T03:42:22.661 INFO:teuthology.orchestra.run.smithi089.stdout: "enable_auth": { 2024-09-21T03:42:22.661 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "enable_auth", 2024-09-21T03:42:22.661 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-21T03:42:22.661 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.661 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:22.661 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "True", 2024-09-21T03:42:22.661 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.661 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.661 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.662 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:22.662 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.662 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.662 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.662 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.662 INFO:teuthology.orchestra.run.smithi089.stdout: "key_file": { 2024-09-21T03:42:22.662 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "key_file", 2024-09-21T03:42:22.662 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:22.662 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.662 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:22.662 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-21T03:42:22.662 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.663 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.663 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.663 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:22.663 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.663 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.663 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.663 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.663 INFO:teuthology.orchestra.run.smithi089.stdout: "log_level": { 2024-09-21T03:42:22.663 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_level", 2024-09-21T03:42:22.663 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:22.663 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.663 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:22.663 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-21T03:42:22.664 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.664 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.664 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-21T03:42:22.664 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2024-09-21T03:42:22.664 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2024-09-21T03:42:22.664 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2024-09-21T03:42:22.664 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2024-09-21T03:42:22.664 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2024-09-21T03:42:22.664 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2024-09-21T03:42:22.664 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-21T03:42:22.664 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:22.665 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.665 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.665 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.665 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.665 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster": { 2024-09-21T03:42:22.665 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster", 2024-09-21T03:42:22.665 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-21T03:42:22.665 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.665 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:22.665 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-21T03:42:22.665 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.665 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.665 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.666 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:22.666 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.666 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.666 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.666 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.666 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster_level": { 2024-09-21T03:42:22.666 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster_level", 2024-09-21T03:42:22.666 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:22.666 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.666 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:22.667 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "info", 2024-09-21T03:42:22.667 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.667 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.667 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-21T03:42:22.667 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2024-09-21T03:42:22.667 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2024-09-21T03:42:22.667 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2024-09-21T03:42:22.667 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2024-09-21T03:42:22.667 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2024-09-21T03:42:22.667 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2024-09-21T03:42:22.667 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-21T03:42:22.667 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:22.667 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.668 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.668 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.668 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.668 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_file": { 2024-09-21T03:42:22.668 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_file", 2024-09-21T03:42:22.668 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-21T03:42:22.668 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.668 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:22.668 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-21T03:42:22.668 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.668 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.668 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.668 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:22.668 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.669 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.669 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.669 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.669 INFO:teuthology.orchestra.run.smithi089.stdout: "max_requests": { 2024-09-21T03:42:22.669 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "max_requests", 2024-09-21T03:42:22.669 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2024-09-21T03:42:22.669 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.669 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:22.669 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "500", 2024-09-21T03:42:22.669 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.669 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.669 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.669 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Maximum number of requests to keep in memory. When new request comes in, the oldest request will be removed if the number of requests exceeds the max request number.if un-finished request is removed, error message will be logged in the ceph-mgr log.", 2024-09-21T03:42:22.669 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.670 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.670 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.670 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.670 INFO:teuthology.orchestra.run.smithi089.stdout: "server_addr": { 2024-09-21T03:42:22.670 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "server_addr", 2024-09-21T03:42:22.670 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:22.670 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.670 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:22.670 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-21T03:42:22.670 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.670 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.670 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.670 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:22.671 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.671 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.671 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.671 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.671 INFO:teuthology.orchestra.run.smithi089.stdout: "server_port": { 2024-09-21T03:42:22.671 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "server_port", 2024-09-21T03:42:22.671 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:22.671 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.671 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:22.671 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-21T03:42:22.671 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.671 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.672 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.672 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:22.672 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.672 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.672 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.672 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.672 INFO:teuthology.orchestra.run.smithi089.stdout: "sqlite3_killpoint": { 2024-09-21T03:42:22.672 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:42:22.672 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2024-09-21T03:42:22.672 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "dev", 2024-09-21T03:42:22.672 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:22.672 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "0", 2024-09-21T03:42:22.672 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.673 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.673 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.673 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:22.673 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.673 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.673 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.673 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-21T03:42:22.673 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-21T03:42:22.673 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.673 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-21T03:42:22.673 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "rgw", 2024-09-21T03:42:22.673 INFO:teuthology.orchestra.run.smithi089.stdout: "can_run": true, 2024-09-21T03:42:22.673 INFO:teuthology.orchestra.run.smithi089.stdout: "error_string": "", 2024-09-21T03:42:22.674 INFO:teuthology.orchestra.run.smithi089.stdout: "module_options": { 2024-09-21T03:42:22.674 INFO:teuthology.orchestra.run.smithi089.stdout: "log_level": { 2024-09-21T03:42:22.674 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_level", 2024-09-21T03:42:22.674 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:22.674 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.674 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:22.674 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-21T03:42:22.674 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.674 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.674 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-21T03:42:22.674 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2024-09-21T03:42:22.674 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2024-09-21T03:42:22.675 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2024-09-21T03:42:22.675 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2024-09-21T03:42:22.675 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2024-09-21T03:42:22.675 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2024-09-21T03:42:22.675 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-21T03:42:22.675 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:22.675 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.675 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.675 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.675 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.675 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster": { 2024-09-21T03:42:22.676 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster", 2024-09-21T03:42:22.676 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-21T03:42:22.676 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.676 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:22.676 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-21T03:42:22.676 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.676 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.676 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.676 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:22.676 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.676 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.676 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.677 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.677 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster_level": { 2024-09-21T03:42:22.677 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster_level", 2024-09-21T03:42:22.677 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:22.677 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.677 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:22.677 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "info", 2024-09-21T03:42:22.677 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.677 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.677 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-21T03:42:22.677 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2024-09-21T03:42:22.677 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2024-09-21T03:42:22.677 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2024-09-21T03:42:22.677 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2024-09-21T03:42:22.678 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2024-09-21T03:42:22.678 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2024-09-21T03:42:22.678 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-21T03:42:22.678 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:22.678 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.678 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.678 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.678 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.678 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_file": { 2024-09-21T03:42:22.678 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_file", 2024-09-21T03:42:22.678 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-21T03:42:22.678 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.679 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:22.679 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-21T03:42:22.679 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.679 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.679 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.679 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:22.679 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.679 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.679 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.679 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.679 INFO:teuthology.orchestra.run.smithi089.stdout: "secondary_zone_period_retry_limit": { 2024-09-21T03:42:22.679 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "secondary_zone_period_retry_limit", 2024-09-21T03:42:22.680 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2024-09-21T03:42:22.680 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.680 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:22.680 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "5", 2024-09-21T03:42:22.680 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.680 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.680 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.680 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "RGW module period update retry limit for secondary site", 2024-09-21T03:42:22.680 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.680 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.680 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.680 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.681 INFO:teuthology.orchestra.run.smithi089.stdout: "sqlite3_killpoint": { 2024-09-21T03:42:22.681 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:42:22.681 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2024-09-21T03:42:22.681 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "dev", 2024-09-21T03:42:22.681 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:22.681 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "0", 2024-09-21T03:42:22.681 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.681 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.681 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.681 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:22.681 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.681 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.682 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.682 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-21T03:42:22.682 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-21T03:42:22.682 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.682 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-21T03:42:22.682 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "rook", 2024-09-21T03:42:22.682 INFO:teuthology.orchestra.run.smithi089.stdout: "can_run": true, 2024-09-21T03:42:22.682 INFO:teuthology.orchestra.run.smithi089.stdout: "error_string": "", 2024-09-21T03:42:22.682 INFO:teuthology.orchestra.run.smithi089.stdout: "module_options": { 2024-09-21T03:42:22.682 INFO:teuthology.orchestra.run.smithi089.stdout: "log_level": { 2024-09-21T03:42:22.682 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_level", 2024-09-21T03:42:22.682 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:22.682 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.683 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:22.683 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-21T03:42:22.683 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.683 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.683 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-21T03:42:22.683 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2024-09-21T03:42:22.683 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2024-09-21T03:42:22.683 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2024-09-21T03:42:22.683 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2024-09-21T03:42:22.683 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2024-09-21T03:42:22.683 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2024-09-21T03:42:22.683 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-21T03:42:22.683 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:22.684 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.684 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.684 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.684 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.684 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster": { 2024-09-21T03:42:22.684 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster", 2024-09-21T03:42:22.684 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-21T03:42:22.684 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.684 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:22.684 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-21T03:42:22.684 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.684 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.684 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.685 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:22.685 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.685 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.685 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.685 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.685 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster_level": { 2024-09-21T03:42:22.685 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster_level", 2024-09-21T03:42:22.685 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:22.685 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.685 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:22.685 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "info", 2024-09-21T03:42:22.685 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.686 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.686 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-21T03:42:22.686 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2024-09-21T03:42:22.686 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2024-09-21T03:42:22.686 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2024-09-21T03:42:22.686 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2024-09-21T03:42:22.686 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2024-09-21T03:42:22.686 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2024-09-21T03:42:22.686 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-21T03:42:22.686 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:22.686 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.686 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.686 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.686 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.687 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_file": { 2024-09-21T03:42:22.687 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_file", 2024-09-21T03:42:22.687 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-21T03:42:22.687 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.687 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:22.687 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-21T03:42:22.687 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.687 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.687 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.687 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:22.687 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.687 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.688 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.688 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.688 INFO:teuthology.orchestra.run.smithi089.stdout: "sqlite3_killpoint": { 2024-09-21T03:42:22.688 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:42:22.688 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2024-09-21T03:42:22.688 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "dev", 2024-09-21T03:42:22.688 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:22.688 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "0", 2024-09-21T03:42:22.688 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.688 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.689 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.689 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:22.689 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.689 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.689 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.689 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.689 INFO:teuthology.orchestra.run.smithi089.stdout: "storage_class": { 2024-09-21T03:42:22.689 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "storage_class", 2024-09-21T03:42:22.689 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:22.689 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.689 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:22.689 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "local", 2024-09-21T03:42:22.689 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.690 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.690 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.690 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-09-21T03:42:22.690 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.690 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.690 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.690 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-21T03:42:22.690 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-21T03:42:22.690 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.690 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-21T03:42:22.690 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "selftest", 2024-09-21T03:42:22.690 INFO:teuthology.orchestra.run.smithi089.stdout: "can_run": true, 2024-09-21T03:42:22.691 INFO:teuthology.orchestra.run.smithi089.stdout: "error_string": "", 2024-09-21T03:42:22.691 INFO:teuthology.orchestra.run.smithi089.stdout: "module_options": { 2024-09-21T03:42:22.691 INFO:teuthology.orchestra.run.smithi089.stdout: "log_level": { 2024-09-21T03:42:22.691 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_level", 2024-09-21T03:42:22.691 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:22.691 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.691 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:22.691 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-21T03:42:22.691 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.691 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.691 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-21T03:42:22.691 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2024-09-21T03:42:22.691 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2024-09-21T03:42:22.691 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2024-09-21T03:42:22.692 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2024-09-21T03:42:22.692 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2024-09-21T03:42:22.692 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2024-09-21T03:42:22.692 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-21T03:42:22.692 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:22.692 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.692 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.692 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.692 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.692 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster": { 2024-09-21T03:42:22.692 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster", 2024-09-21T03:42:22.692 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-21T03:42:22.692 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.692 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:22.693 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-21T03:42:22.693 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.693 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.693 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.693 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:22.693 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.693 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.693 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.693 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.693 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster_level": { 2024-09-21T03:42:22.693 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster_level", 2024-09-21T03:42:22.693 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:22.693 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.693 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:22.694 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "info", 2024-09-21T03:42:22.694 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.694 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.694 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-21T03:42:22.694 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2024-09-21T03:42:22.694 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2024-09-21T03:42:22.694 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2024-09-21T03:42:22.694 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2024-09-21T03:42:22.694 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2024-09-21T03:42:22.694 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2024-09-21T03:42:22.694 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-21T03:42:22.695 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:22.695 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.695 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.695 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.695 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.695 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_file": { 2024-09-21T03:42:22.695 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_file", 2024-09-21T03:42:22.695 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-21T03:42:22.695 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.695 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:22.695 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-21T03:42:22.695 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.696 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.696 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.696 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:22.696 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.696 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.696 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.696 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.696 INFO:teuthology.orchestra.run.smithi089.stdout: "roption1": { 2024-09-21T03:42:22.696 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "roption1", 2024-09-21T03:42:22.696 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:22.696 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.696 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:22.696 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-21T03:42:22.696 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.697 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.697 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.697 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:22.697 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.697 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.697 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.697 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.697 INFO:teuthology.orchestra.run.smithi089.stdout: "roption2": { 2024-09-21T03:42:22.697 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "roption2", 2024-09-21T03:42:22.697 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:22.697 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.697 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:22.697 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "xyz", 2024-09-21T03:42:22.698 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.698 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.698 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.698 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:22.698 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.698 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.698 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.698 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.698 INFO:teuthology.orchestra.run.smithi089.stdout: "rwoption1": { 2024-09-21T03:42:22.698 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "rwoption1", 2024-09-21T03:42:22.698 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:22.698 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.698 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:22.699 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-21T03:42:22.699 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.699 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.699 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.699 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:22.699 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.699 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.699 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.699 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.699 INFO:teuthology.orchestra.run.smithi089.stdout: "rwoption2": { 2024-09-21T03:42:22.699 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "rwoption2", 2024-09-21T03:42:22.699 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2024-09-21T03:42:22.699 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.699 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:22.700 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-21T03:42:22.700 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.700 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.700 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.700 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:22.700 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.700 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.700 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.700 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.700 INFO:teuthology.orchestra.run.smithi089.stdout: "rwoption3": { 2024-09-21T03:42:22.700 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "rwoption3", 2024-09-21T03:42:22.700 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "float", 2024-09-21T03:42:22.700 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.700 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:22.701 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-21T03:42:22.701 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.701 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.701 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.701 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:22.701 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.701 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.701 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.701 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.701 INFO:teuthology.orchestra.run.smithi089.stdout: "rwoption4": { 2024-09-21T03:42:22.701 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "rwoption4", 2024-09-21T03:42:22.701 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:22.701 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.701 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:22.702 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-21T03:42:22.702 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.702 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.702 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.702 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:22.702 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.702 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.702 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.702 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.702 INFO:teuthology.orchestra.run.smithi089.stdout: "rwoption5": { 2024-09-21T03:42:22.702 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "rwoption5", 2024-09-21T03:42:22.702 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-21T03:42:22.702 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.702 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:22.703 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-21T03:42:22.703 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.703 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.703 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.703 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:22.703 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.703 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.703 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.703 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.703 INFO:teuthology.orchestra.run.smithi089.stdout: "rwoption6": { 2024-09-21T03:42:22.703 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "rwoption6", 2024-09-21T03:42:22.703 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-21T03:42:22.704 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.704 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:22.704 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "True", 2024-09-21T03:42:22.704 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.704 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.704 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.704 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:22.704 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.704 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.704 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.704 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.704 INFO:teuthology.orchestra.run.smithi089.stdout: "rwoption7": { 2024-09-21T03:42:22.705 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "rwoption7", 2024-09-21T03:42:22.705 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2024-09-21T03:42:22.705 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.705 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:22.705 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-21T03:42:22.705 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "1", 2024-09-21T03:42:22.705 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "42", 2024-09-21T03:42:22.705 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.705 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:22.705 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.705 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.706 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.706 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.706 INFO:teuthology.orchestra.run.smithi089.stdout: "sqlite3_killpoint": { 2024-09-21T03:42:22.706 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:42:22.706 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2024-09-21T03:42:22.706 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "dev", 2024-09-21T03:42:22.706 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:22.706 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "0", 2024-09-21T03:42:22.706 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.706 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.706 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.707 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:22.707 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.707 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.707 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.707 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.707 INFO:teuthology.orchestra.run.smithi089.stdout: "testkey": { 2024-09-21T03:42:22.707 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "testkey", 2024-09-21T03:42:22.707 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:22.707 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.707 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:22.707 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-21T03:42:22.708 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.708 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.708 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.708 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:22.708 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.708 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.708 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.708 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.708 INFO:teuthology.orchestra.run.smithi089.stdout: "testlkey": { 2024-09-21T03:42:22.708 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "testlkey", 2024-09-21T03:42:22.708 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:22.709 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.709 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:22.709 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-21T03:42:22.709 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.709 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.709 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.709 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:22.709 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.709 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.709 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.709 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.709 INFO:teuthology.orchestra.run.smithi089.stdout: "testnewline": { 2024-09-21T03:42:22.710 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "testnewline", 2024-09-21T03:42:22.710 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:22.710 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.710 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:22.710 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-21T03:42:22.710 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.710 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.710 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.710 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:22.710 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.710 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.711 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.711 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-21T03:42:22.711 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-21T03:42:22.711 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.711 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-21T03:42:22.711 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "smb", 2024-09-21T03:42:22.711 INFO:teuthology.orchestra.run.smithi089.stdout: "can_run": true, 2024-09-21T03:42:22.711 INFO:teuthology.orchestra.run.smithi089.stdout: "error_string": "", 2024-09-21T03:42:22.711 INFO:teuthology.orchestra.run.smithi089.stdout: "module_options": { 2024-09-21T03:42:22.711 INFO:teuthology.orchestra.run.smithi089.stdout: "internal_store_backend": { 2024-09-21T03:42:22.711 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "internal_store_backend", 2024-09-21T03:42:22.711 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:22.712 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "dev", 2024-09-21T03:42:22.712 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:22.712 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-21T03:42:22.712 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.712 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.712 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.712 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "set internal store backend. for develoment and testing only", 2024-09-21T03:42:22.712 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.712 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.712 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.712 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.713 INFO:teuthology.orchestra.run.smithi089.stdout: "log_level": { 2024-09-21T03:42:22.713 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_level", 2024-09-21T03:42:22.713 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:22.713 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.713 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:22.713 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-21T03:42:22.713 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.713 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.713 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-21T03:42:22.713 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2024-09-21T03:42:22.713 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2024-09-21T03:42:22.714 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2024-09-21T03:42:22.714 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2024-09-21T03:42:22.714 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2024-09-21T03:42:22.714 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2024-09-21T03:42:22.714 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-21T03:42:22.714 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:22.714 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.714 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.714 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.714 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.714 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster": { 2024-09-21T03:42:22.715 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster", 2024-09-21T03:42:22.715 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-21T03:42:22.715 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.715 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:22.715 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-21T03:42:22.715 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.715 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.715 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.715 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:22.715 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.715 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.715 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.716 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.716 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster_level": { 2024-09-21T03:42:22.716 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster_level", 2024-09-21T03:42:22.716 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:22.716 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.716 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:22.716 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "info", 2024-09-21T03:42:22.716 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.716 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.716 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-21T03:42:22.716 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2024-09-21T03:42:22.716 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2024-09-21T03:42:22.717 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2024-09-21T03:42:22.717 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2024-09-21T03:42:22.717 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2024-09-21T03:42:22.717 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2024-09-21T03:42:22.717 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-21T03:42:22.717 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:22.717 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.717 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.717 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.717 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.717 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_file": { 2024-09-21T03:42:22.717 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_file", 2024-09-21T03:42:22.717 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-21T03:42:22.717 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.717 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:22.718 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-21T03:42:22.718 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.718 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.718 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.718 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:22.718 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.718 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.718 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.718 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.718 INFO:teuthology.orchestra.run.smithi089.stdout: "sqlite3_killpoint": { 2024-09-21T03:42:22.718 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:42:22.718 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2024-09-21T03:42:22.718 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "dev", 2024-09-21T03:42:22.719 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:22.719 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "0", 2024-09-21T03:42:22.719 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.719 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.719 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.719 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:22.719 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.719 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.719 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.719 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.719 INFO:teuthology.orchestra.run.smithi089.stdout: "update_orchestration": { 2024-09-21T03:42:22.719 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "update_orchestration", 2024-09-21T03:42:22.720 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-21T03:42:22.720 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.720 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:22.720 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "True", 2024-09-21T03:42:22.720 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.720 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.720 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.720 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "automatically update orchestration when smb resources are changed", 2024-09-21T03:42:22.720 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.720 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.720 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.720 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-21T03:42:22.720 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-21T03:42:22.720 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.721 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-21T03:42:22.721 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "snap_schedule", 2024-09-21T03:42:22.721 INFO:teuthology.orchestra.run.smithi089.stdout: "can_run": true, 2024-09-21T03:42:22.721 INFO:teuthology.orchestra.run.smithi089.stdout: "error_string": "", 2024-09-21T03:42:22.721 INFO:teuthology.orchestra.run.smithi089.stdout: "module_options": { 2024-09-21T03:42:22.721 INFO:teuthology.orchestra.run.smithi089.stdout: "allow_m_granularity": { 2024-09-21T03:42:22.721 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "allow_m_granularity", 2024-09-21T03:42:22.721 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-21T03:42:22.721 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.721 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:22.721 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-21T03:42:22.721 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.721 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.721 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.722 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "allow minute scheduled snapshots", 2024-09-21T03:42:22.722 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.722 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.722 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.722 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.722 INFO:teuthology.orchestra.run.smithi089.stdout: "dump_on_update": { 2024-09-21T03:42:22.722 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "dump_on_update", 2024-09-21T03:42:22.722 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-21T03:42:22.722 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.722 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:22.722 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-21T03:42:22.722 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.722 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.723 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.723 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "dump database to debug log on update", 2024-09-21T03:42:22.723 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.723 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.723 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.723 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.723 INFO:teuthology.orchestra.run.smithi089.stdout: "log_level": { 2024-09-21T03:42:22.723 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_level", 2024-09-21T03:42:22.723 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:22.723 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.723 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:22.724 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-21T03:42:22.724 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.724 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.724 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-21T03:42:22.724 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2024-09-21T03:42:22.724 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2024-09-21T03:42:22.724 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2024-09-21T03:42:22.724 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2024-09-21T03:42:22.724 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2024-09-21T03:42:22.724 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2024-09-21T03:42:22.724 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-21T03:42:22.724 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:22.724 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.724 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.725 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.725 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.725 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster": { 2024-09-21T03:42:22.725 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster", 2024-09-21T03:42:22.725 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-21T03:42:22.725 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.725 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:22.725 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-21T03:42:22.725 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.725 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.725 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.725 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:22.726 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.726 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.726 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.726 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.726 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster_level": { 2024-09-21T03:42:22.726 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster_level", 2024-09-21T03:42:22.726 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:22.726 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.726 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:22.726 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "info", 2024-09-21T03:42:22.726 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.727 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.727 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-21T03:42:22.727 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2024-09-21T03:42:22.727 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2024-09-21T03:42:22.727 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2024-09-21T03:42:22.727 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2024-09-21T03:42:22.727 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2024-09-21T03:42:22.727 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2024-09-21T03:42:22.727 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-21T03:42:22.727 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:22.727 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.727 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.727 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.727 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.728 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_file": { 2024-09-21T03:42:22.728 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_file", 2024-09-21T03:42:22.728 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-21T03:42:22.728 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.728 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:22.728 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-21T03:42:22.728 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.728 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.728 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.728 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:22.728 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.728 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.728 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.728 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.729 INFO:teuthology.orchestra.run.smithi089.stdout: "sqlite3_killpoint": { 2024-09-21T03:42:22.729 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:42:22.729 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2024-09-21T03:42:22.729 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "dev", 2024-09-21T03:42:22.729 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:22.729 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "0", 2024-09-21T03:42:22.729 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.729 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.729 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.729 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:22.729 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.729 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.729 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.729 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-21T03:42:22.730 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-21T03:42:22.730 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.730 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-21T03:42:22.730 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "stats", 2024-09-21T03:42:22.730 INFO:teuthology.orchestra.run.smithi089.stdout: "can_run": true, 2024-09-21T03:42:22.730 INFO:teuthology.orchestra.run.smithi089.stdout: "error_string": "", 2024-09-21T03:42:22.730 INFO:teuthology.orchestra.run.smithi089.stdout: "module_options": { 2024-09-21T03:42:22.730 INFO:teuthology.orchestra.run.smithi089.stdout: "log_level": { 2024-09-21T03:42:22.730 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_level", 2024-09-21T03:42:22.730 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:22.730 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.730 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:22.731 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-21T03:42:22.731 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.731 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.731 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-21T03:42:22.731 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2024-09-21T03:42:22.731 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2024-09-21T03:42:22.731 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2024-09-21T03:42:22.731 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2024-09-21T03:42:22.731 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2024-09-21T03:42:22.731 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2024-09-21T03:42:22.731 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-21T03:42:22.731 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:22.731 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.732 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.732 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.732 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.732 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster": { 2024-09-21T03:42:22.732 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster", 2024-09-21T03:42:22.732 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-21T03:42:22.732 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.732 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:22.732 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-21T03:42:22.732 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.732 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.732 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.732 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:22.733 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.733 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.733 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.733 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.733 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster_level": { 2024-09-21T03:42:22.733 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster_level", 2024-09-21T03:42:22.733 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:22.733 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.733 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:22.733 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "info", 2024-09-21T03:42:22.733 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.733 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.733 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-21T03:42:22.734 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2024-09-21T03:42:22.734 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2024-09-21T03:42:22.734 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2024-09-21T03:42:22.734 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2024-09-21T03:42:22.734 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2024-09-21T03:42:22.734 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2024-09-21T03:42:22.734 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-21T03:42:22.734 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:22.734 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.734 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.734 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.734 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.734 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_file": { 2024-09-21T03:42:22.735 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_file", 2024-09-21T03:42:22.735 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-21T03:42:22.735 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.735 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:22.735 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-21T03:42:22.735 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.735 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.735 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.735 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:22.735 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.735 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.736 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.736 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.736 INFO:teuthology.orchestra.run.smithi089.stdout: "sqlite3_killpoint": { 2024-09-21T03:42:22.736 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:42:22.736 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2024-09-21T03:42:22.736 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "dev", 2024-09-21T03:42:22.736 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:22.736 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "0", 2024-09-21T03:42:22.736 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.736 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.736 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.736 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:22.736 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.737 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.737 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.737 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-21T03:42:22.737 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-21T03:42:22.737 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.737 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-21T03:42:22.737 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "status", 2024-09-21T03:42:22.737 INFO:teuthology.orchestra.run.smithi089.stdout: "can_run": true, 2024-09-21T03:42:22.737 INFO:teuthology.orchestra.run.smithi089.stdout: "error_string": "", 2024-09-21T03:42:22.737 INFO:teuthology.orchestra.run.smithi089.stdout: "module_options": { 2024-09-21T03:42:22.737 INFO:teuthology.orchestra.run.smithi089.stdout: "log_level": { 2024-09-21T03:42:22.737 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_level", 2024-09-21T03:42:22.738 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:22.738 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.738 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:22.738 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-21T03:42:22.738 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.738 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.738 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-21T03:42:22.738 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2024-09-21T03:42:22.738 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2024-09-21T03:42:22.738 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2024-09-21T03:42:22.738 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2024-09-21T03:42:22.738 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2024-09-21T03:42:22.739 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2024-09-21T03:42:22.739 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-21T03:42:22.739 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:22.739 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.739 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.739 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.739 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.739 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster": { 2024-09-21T03:42:22.739 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster", 2024-09-21T03:42:22.739 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-21T03:42:22.739 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.740 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:22.740 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-21T03:42:22.740 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.740 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.740 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.740 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:22.740 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.740 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.740 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.740 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.740 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster_level": { 2024-09-21T03:42:22.740 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster_level", 2024-09-21T03:42:22.740 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:22.741 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.741 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:22.741 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "info", 2024-09-21T03:42:22.741 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.741 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.741 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-21T03:42:22.741 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2024-09-21T03:42:22.741 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2024-09-21T03:42:22.741 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2024-09-21T03:42:22.741 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2024-09-21T03:42:22.741 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2024-09-21T03:42:22.742 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2024-09-21T03:42:22.742 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-21T03:42:22.742 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:22.742 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.742 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.742 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.742 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.742 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_file": { 2024-09-21T03:42:22.742 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_file", 2024-09-21T03:42:22.742 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-21T03:42:22.742 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.743 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:22.743 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-21T03:42:22.743 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.743 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.743 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.743 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:22.743 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.743 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.743 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.743 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.743 INFO:teuthology.orchestra.run.smithi089.stdout: "sqlite3_killpoint": { 2024-09-21T03:42:22.744 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:42:22.744 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2024-09-21T03:42:22.744 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "dev", 2024-09-21T03:42:22.744 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:22.744 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "0", 2024-09-21T03:42:22.744 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.744 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.744 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.744 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:22.744 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.744 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.745 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.745 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-21T03:42:22.745 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-21T03:42:22.745 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.745 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-21T03:42:22.745 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "telegraf", 2024-09-21T03:42:22.745 INFO:teuthology.orchestra.run.smithi089.stdout: "can_run": true, 2024-09-21T03:42:22.745 INFO:teuthology.orchestra.run.smithi089.stdout: "error_string": "", 2024-09-21T03:42:22.745 INFO:teuthology.orchestra.run.smithi089.stdout: "module_options": { 2024-09-21T03:42:22.745 INFO:teuthology.orchestra.run.smithi089.stdout: "address": { 2024-09-21T03:42:22.746 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "address", 2024-09-21T03:42:22.746 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:22.746 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.746 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:22.746 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-09-21T03:42:22.746 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.746 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.746 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.746 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:22.746 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.747 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.747 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.747 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.747 INFO:teuthology.orchestra.run.smithi089.stdout: "interval": { 2024-09-21T03:42:22.747 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "interval", 2024-09-21T03:42:22.747 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "secs", 2024-09-21T03:42:22.747 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.747 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:22.747 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "15", 2024-09-21T03:42:22.747 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.747 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.747 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.747 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:22.748 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.748 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.748 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.748 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.748 INFO:teuthology.orchestra.run.smithi089.stdout: "log_level": { 2024-09-21T03:42:22.748 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_level", 2024-09-21T03:42:22.748 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:22.748 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.748 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:22.748 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-21T03:42:22.748 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.748 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.748 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-21T03:42:22.748 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2024-09-21T03:42:22.749 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2024-09-21T03:42:22.749 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2024-09-21T03:42:22.749 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2024-09-21T03:42:22.749 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2024-09-21T03:42:22.749 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2024-09-21T03:42:22.749 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-21T03:42:22.749 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:22.749 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.749 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.749 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.749 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.749 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster": { 2024-09-21T03:42:22.749 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster", 2024-09-21T03:42:22.749 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-21T03:42:22.749 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.750 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:22.750 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-21T03:42:22.750 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.750 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.750 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.750 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:22.750 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.750 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.750 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.750 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.750 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster_level": { 2024-09-21T03:42:22.750 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster_level", 2024-09-21T03:42:22.750 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:22.751 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.751 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:22.751 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "info", 2024-09-21T03:42:22.751 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.751 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.751 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-21T03:42:22.751 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2024-09-21T03:42:22.751 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2024-09-21T03:42:22.751 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2024-09-21T03:42:22.751 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2024-09-21T03:42:22.751 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2024-09-21T03:42:22.751 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2024-09-21T03:42:22.751 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-21T03:42:22.752 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:22.752 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.752 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.752 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.752 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.752 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_file": { 2024-09-21T03:42:22.752 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_file", 2024-09-21T03:42:22.752 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-21T03:42:22.752 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.752 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:22.752 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-21T03:42:22.752 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.752 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.753 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.753 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:22.753 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.753 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.753 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.753 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.753 INFO:teuthology.orchestra.run.smithi089.stdout: "sqlite3_killpoint": { 2024-09-21T03:42:22.753 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:42:22.753 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2024-09-21T03:42:22.753 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "dev", 2024-09-21T03:42:22.753 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:22.753 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "0", 2024-09-21T03:42:22.753 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.754 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.754 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.754 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:22.754 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.754 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.754 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.754 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-21T03:42:22.754 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-21T03:42:22.754 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.754 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-21T03:42:22.754 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "telemetry", 2024-09-21T03:42:22.754 INFO:teuthology.orchestra.run.smithi089.stdout: "can_run": true, 2024-09-21T03:42:22.755 INFO:teuthology.orchestra.run.smithi089.stdout: "error_string": "", 2024-09-21T03:42:22.755 INFO:teuthology.orchestra.run.smithi089.stdout: "module_options": { 2024-09-21T03:42:22.755 INFO:teuthology.orchestra.run.smithi089.stdout: "channel_basic": { 2024-09-21T03:42:22.755 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "channel_basic", 2024-09-21T03:42:22.755 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-21T03:42:22.755 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.755 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:22.755 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "True", 2024-09-21T03:42:22.755 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.755 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.755 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.755 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Share basic cluster information (size, version)", 2024-09-21T03:42:22.756 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.756 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.756 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.756 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.756 INFO:teuthology.orchestra.run.smithi089.stdout: "channel_crash": { 2024-09-21T03:42:22.756 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "channel_crash", 2024-09-21T03:42:22.756 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-21T03:42:22.756 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.756 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:22.756 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "True", 2024-09-21T03:42:22.756 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.757 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.757 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.757 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-09-21T03:42:22.757 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.757 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.757 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.757 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.757 INFO:teuthology.orchestra.run.smithi089.stdout: "channel_device": { 2024-09-21T03:42:22.757 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "channel_device", 2024-09-21T03:42:22.757 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-21T03:42:22.757 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.758 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:22.758 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "True", 2024-09-21T03:42:22.758 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.758 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.758 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.758 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-09-21T03:42:22.758 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.758 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.758 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.758 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.758 INFO:teuthology.orchestra.run.smithi089.stdout: "channel_ident": { 2024-09-21T03:42:22.758 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "channel_ident", 2024-09-21T03:42:22.758 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-21T03:42:22.759 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.759 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:22.759 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-21T03:42:22.759 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.759 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.759 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.759 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-09-21T03:42:22.759 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.759 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.759 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.759 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.760 INFO:teuthology.orchestra.run.smithi089.stdout: "channel_perf": { 2024-09-21T03:42:22.760 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "channel_perf", 2024-09-21T03:42:22.760 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-21T03:42:22.760 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.760 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:22.760 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-21T03:42:22.760 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.760 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.760 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.760 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Share various performance metrics of a cluster", 2024-09-21T03:42:22.760 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.760 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.760 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.761 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.761 INFO:teuthology.orchestra.run.smithi089.stdout: "contact": { 2024-09-21T03:42:22.761 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "contact", 2024-09-21T03:42:22.761 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:22.761 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.761 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:22.761 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-21T03:42:22.761 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.761 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.761 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.761 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:22.762 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.762 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.762 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.762 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.762 INFO:teuthology.orchestra.run.smithi089.stdout: "description": { 2024-09-21T03:42:22.762 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "description", 2024-09-21T03:42:22.762 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:22.762 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.762 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:22.762 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-21T03:42:22.762 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.762 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.762 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.763 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:22.763 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.763 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.763 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.763 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.763 INFO:teuthology.orchestra.run.smithi089.stdout: "device_url": { 2024-09-21T03:42:22.763 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "device_url", 2024-09-21T03:42:22.763 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:22.763 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.763 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:22.763 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-09-21T03:42:22.763 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.763 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.763 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.764 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:22.764 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.764 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.764 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.764 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.764 INFO:teuthology.orchestra.run.smithi089.stdout: "enabled": { 2024-09-21T03:42:22.764 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "enabled", 2024-09-21T03:42:22.764 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-21T03:42:22.764 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.764 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:22.764 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-21T03:42:22.764 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.764 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.764 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.765 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:22.765 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.765 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.765 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.765 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.765 INFO:teuthology.orchestra.run.smithi089.stdout: "interval": { 2024-09-21T03:42:22.765 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "interval", 2024-09-21T03:42:22.765 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2024-09-21T03:42:22.765 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.765 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:22.765 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "24", 2024-09-21T03:42:22.765 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "8", 2024-09-21T03:42:22.766 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.766 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.766 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:22.766 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.766 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.766 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.766 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.766 INFO:teuthology.orchestra.run.smithi089.stdout: "last_opt_revision": { 2024-09-21T03:42:22.766 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "last_opt_revision", 2024-09-21T03:42:22.766 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2024-09-21T03:42:22.766 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.766 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:22.766 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "1", 2024-09-21T03:42:22.767 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.767 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.767 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.767 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:22.767 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.767 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.767 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.767 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.767 INFO:teuthology.orchestra.run.smithi089.stdout: "leaderboard": { 2024-09-21T03:42:22.767 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "leaderboard", 2024-09-21T03:42:22.767 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-21T03:42:22.767 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.767 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:22.768 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-21T03:42:22.768 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.768 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.768 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.768 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:22.768 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.768 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.768 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.768 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.768 INFO:teuthology.orchestra.run.smithi089.stdout: "leaderboard_description": { 2024-09-21T03:42:22.768 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "leaderboard_description", 2024-09-21T03:42:22.768 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:22.768 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.769 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:22.769 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-21T03:42:22.769 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.769 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.769 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.769 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:22.769 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.769 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.769 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.769 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.769 INFO:teuthology.orchestra.run.smithi089.stdout: "log_level": { 2024-09-21T03:42:22.770 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_level", 2024-09-21T03:42:22.770 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:22.770 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.770 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:22.770 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-21T03:42:22.770 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.770 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.770 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-21T03:42:22.770 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2024-09-21T03:42:22.770 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2024-09-21T03:42:22.770 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2024-09-21T03:42:22.771 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2024-09-21T03:42:22.771 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2024-09-21T03:42:22.771 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2024-09-21T03:42:22.771 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-21T03:42:22.771 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:22.771 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.771 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.771 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.771 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.771 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster": { 2024-09-21T03:42:22.771 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster", 2024-09-21T03:42:22.771 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-21T03:42:22.772 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.772 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:22.772 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-21T03:42:22.772 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.772 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.772 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.772 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:22.772 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.772 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.772 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.772 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.772 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster_level": { 2024-09-21T03:42:22.773 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster_level", 2024-09-21T03:42:22.773 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:22.773 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.773 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:22.773 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "info", 2024-09-21T03:42:22.773 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.773 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.773 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-21T03:42:22.773 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2024-09-21T03:42:22.773 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2024-09-21T03:42:22.773 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2024-09-21T03:42:22.773 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2024-09-21T03:42:22.774 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2024-09-21T03:42:22.774 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2024-09-21T03:42:22.774 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-21T03:42:22.774 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:22.774 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.774 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.774 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.774 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.774 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_file": { 2024-09-21T03:42:22.774 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_file", 2024-09-21T03:42:22.774 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-21T03:42:22.775 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.775 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:22.775 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-21T03:42:22.775 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.775 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.775 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.775 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:22.775 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.775 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.775 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.775 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.776 INFO:teuthology.orchestra.run.smithi089.stdout: "organization": { 2024-09-21T03:42:22.776 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "organization", 2024-09-21T03:42:22.776 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:22.776 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.776 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:22.776 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-21T03:42:22.776 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.776 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.776 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.776 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:22.776 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.776 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.777 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.777 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.777 INFO:teuthology.orchestra.run.smithi089.stdout: "proxy": { 2024-09-21T03:42:22.777 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "proxy", 2024-09-21T03:42:22.777 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:22.777 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.777 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:22.777 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-21T03:42:22.777 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.777 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.777 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.777 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:22.777 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.777 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.778 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.778 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.778 INFO:teuthology.orchestra.run.smithi089.stdout: "sqlite3_killpoint": { 2024-09-21T03:42:22.778 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:42:22.778 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2024-09-21T03:42:22.778 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "dev", 2024-09-21T03:42:22.778 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:22.778 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "0", 2024-09-21T03:42:22.778 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.778 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.778 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.778 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:22.779 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.779 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.779 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.779 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.779 INFO:teuthology.orchestra.run.smithi089.stdout: "url": { 2024-09-21T03:42:22.779 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "url", 2024-09-21T03:42:22.779 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:22.779 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.779 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:22.779 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-09-21T03:42:22.779 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.779 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.780 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.780 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:22.780 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.780 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.780 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.780 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-21T03:42:22.780 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-21T03:42:22.780 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.780 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-21T03:42:22.780 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "test_orchestrator", 2024-09-21T03:42:22.780 INFO:teuthology.orchestra.run.smithi089.stdout: "can_run": true, 2024-09-21T03:42:22.780 INFO:teuthology.orchestra.run.smithi089.stdout: "error_string": "", 2024-09-21T03:42:22.780 INFO:teuthology.orchestra.run.smithi089.stdout: "module_options": { 2024-09-21T03:42:22.780 INFO:teuthology.orchestra.run.smithi089.stdout: "log_level": { 2024-09-21T03:42:22.781 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_level", 2024-09-21T03:42:22.781 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:22.781 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.781 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:22.781 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-21T03:42:22.781 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.781 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.781 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-21T03:42:22.781 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2024-09-21T03:42:22.781 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2024-09-21T03:42:22.781 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2024-09-21T03:42:22.781 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2024-09-21T03:42:22.781 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2024-09-21T03:42:22.781 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2024-09-21T03:42:22.782 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-21T03:42:22.782 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:22.782 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.782 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.782 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.782 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.782 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster": { 2024-09-21T03:42:22.782 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster", 2024-09-21T03:42:22.782 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-21T03:42:22.782 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.782 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:22.782 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-21T03:42:22.783 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.783 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.783 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.783 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:22.783 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.783 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.783 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.783 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.783 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster_level": { 2024-09-21T03:42:22.783 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster_level", 2024-09-21T03:42:22.783 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:22.783 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.783 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:22.784 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "info", 2024-09-21T03:42:22.784 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.784 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.784 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-21T03:42:22.784 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2024-09-21T03:42:22.784 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2024-09-21T03:42:22.784 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2024-09-21T03:42:22.784 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2024-09-21T03:42:22.784 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2024-09-21T03:42:22.784 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2024-09-21T03:42:22.784 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-21T03:42:22.784 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:22.784 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.785 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.785 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.785 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.785 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_file": { 2024-09-21T03:42:22.785 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_file", 2024-09-21T03:42:22.785 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-21T03:42:22.785 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.785 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:22.785 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-21T03:42:22.785 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.785 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.785 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.786 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:22.786 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.786 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.786 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.786 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.786 INFO:teuthology.orchestra.run.smithi089.stdout: "sqlite3_killpoint": { 2024-09-21T03:42:22.786 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:42:22.786 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2024-09-21T03:42:22.786 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "dev", 2024-09-21T03:42:22.786 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:22.786 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "0", 2024-09-21T03:42:22.786 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.787 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.787 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.787 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:22.787 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.787 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.787 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.787 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-21T03:42:22.787 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-21T03:42:22.787 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.787 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-21T03:42:22.787 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "volumes", 2024-09-21T03:42:22.787 INFO:teuthology.orchestra.run.smithi089.stdout: "can_run": true, 2024-09-21T03:42:22.787 INFO:teuthology.orchestra.run.smithi089.stdout: "error_string": "", 2024-09-21T03:42:22.788 INFO:teuthology.orchestra.run.smithi089.stdout: "module_options": { 2024-09-21T03:42:22.788 INFO:teuthology.orchestra.run.smithi089.stdout: "log_level": { 2024-09-21T03:42:22.788 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_level", 2024-09-21T03:42:22.788 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:22.788 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.788 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:22.788 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-21T03:42:22.788 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.788 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.788 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-21T03:42:22.788 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2024-09-21T03:42:22.788 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2024-09-21T03:42:22.788 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2024-09-21T03:42:22.788 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2024-09-21T03:42:22.789 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2024-09-21T03:42:22.789 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2024-09-21T03:42:22.789 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-21T03:42:22.789 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:22.789 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.789 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.789 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.789 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.789 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster": { 2024-09-21T03:42:22.789 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster", 2024-09-21T03:42:22.789 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-21T03:42:22.789 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.789 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:22.789 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-21T03:42:22.790 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.790 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.790 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.790 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:22.790 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.790 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.790 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.790 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.790 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster_level": { 2024-09-21T03:42:22.790 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster_level", 2024-09-21T03:42:22.790 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:22.790 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.791 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:22.791 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "info", 2024-09-21T03:42:22.791 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.791 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.791 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-21T03:42:22.791 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2024-09-21T03:42:22.791 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2024-09-21T03:42:22.791 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2024-09-21T03:42:22.791 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2024-09-21T03:42:22.791 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2024-09-21T03:42:22.791 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2024-09-21T03:42:22.791 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-21T03:42:22.791 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:22.791 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.792 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.792 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.792 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.792 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_file": { 2024-09-21T03:42:22.792 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_file", 2024-09-21T03:42:22.792 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-21T03:42:22.792 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.792 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:22.792 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-21T03:42:22.792 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.792 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.792 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.792 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:22.792 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.793 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.793 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.793 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.793 INFO:teuthology.orchestra.run.smithi089.stdout: "max_concurrent_clones": { 2024-09-21T03:42:22.793 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "max_concurrent_clones", 2024-09-21T03:42:22.793 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2024-09-21T03:42:22.793 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.793 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:22.793 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "4", 2024-09-21T03:42:22.793 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.793 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.793 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.793 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Number of asynchronous cloner threads", 2024-09-21T03:42:22.794 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.794 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.794 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.794 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.794 INFO:teuthology.orchestra.run.smithi089.stdout: "periodic_async_work": { 2024-09-21T03:42:22.794 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "periodic_async_work", 2024-09-21T03:42:22.794 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-21T03:42:22.794 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.794 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:22.794 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-21T03:42:22.794 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.794 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.794 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.794 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Periodically check for async work", 2024-09-21T03:42:22.795 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.795 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.795 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.795 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.795 INFO:teuthology.orchestra.run.smithi089.stdout: "snapshot_clone_delay": { 2024-09-21T03:42:22.795 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "snapshot_clone_delay", 2024-09-21T03:42:22.795 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2024-09-21T03:42:22.795 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.795 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:22.795 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "0", 2024-09-21T03:42:22.795 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.796 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.796 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.796 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-09-21T03:42:22.796 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.796 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.796 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.796 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.796 INFO:teuthology.orchestra.run.smithi089.stdout: "snapshot_clone_no_wait": { 2024-09-21T03:42:22.796 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "snapshot_clone_no_wait", 2024-09-21T03:42:22.796 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-21T03:42:22.796 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.796 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:22.796 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "True", 2024-09-21T03:42:22.797 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.797 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.797 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.797 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-09-21T03:42:22.797 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.797 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.797 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.797 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.797 INFO:teuthology.orchestra.run.smithi089.stdout: "sqlite3_killpoint": { 2024-09-21T03:42:22.797 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:42:22.797 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2024-09-21T03:42:22.797 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "dev", 2024-09-21T03:42:22.797 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:22.798 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "0", 2024-09-21T03:42:22.798 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.798 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.798 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.798 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:22.798 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.798 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.798 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.798 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-21T03:42:22.798 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-21T03:42:22.798 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.798 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-21T03:42:22.798 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "zabbix", 2024-09-21T03:42:22.799 INFO:teuthology.orchestra.run.smithi089.stdout: "can_run": true, 2024-09-21T03:42:22.799 INFO:teuthology.orchestra.run.smithi089.stdout: "error_string": "", 2024-09-21T03:42:22.799 INFO:teuthology.orchestra.run.smithi089.stdout: "module_options": { 2024-09-21T03:42:22.799 INFO:teuthology.orchestra.run.smithi089.stdout: "discovery_interval": { 2024-09-21T03:42:22.799 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "discovery_interval", 2024-09-21T03:42:22.799 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "uint", 2024-09-21T03:42:22.799 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.799 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:22.799 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "100", 2024-09-21T03:42:22.799 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.799 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.799 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.799 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:22.800 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.800 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.800 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.800 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.800 INFO:teuthology.orchestra.run.smithi089.stdout: "identifier": { 2024-09-21T03:42:22.800 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "identifier", 2024-09-21T03:42:22.800 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:22.800 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.800 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:22.800 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-21T03:42:22.800 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.800 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.800 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.801 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:22.801 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.801 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.801 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.801 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.801 INFO:teuthology.orchestra.run.smithi089.stdout: "interval": { 2024-09-21T03:42:22.801 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "interval", 2024-09-21T03:42:22.801 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "secs", 2024-09-21T03:42:22.801 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.801 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:22.801 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "60", 2024-09-21T03:42:22.801 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.802 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.802 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.802 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:22.802 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.802 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.802 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.802 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.802 INFO:teuthology.orchestra.run.smithi089.stdout: "log_level": { 2024-09-21T03:42:22.802 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_level", 2024-09-21T03:42:22.802 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:22.802 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.802 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:22.802 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-21T03:42:22.803 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.803 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.803 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-21T03:42:22.803 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2024-09-21T03:42:22.803 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2024-09-21T03:42:22.803 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2024-09-21T03:42:22.803 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2024-09-21T03:42:22.803 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2024-09-21T03:42:22.803 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2024-09-21T03:42:22.803 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-21T03:42:22.803 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:22.803 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.803 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.803 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.804 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.804 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster": { 2024-09-21T03:42:22.804 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster", 2024-09-21T03:42:22.804 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-21T03:42:22.804 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.804 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:22.804 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-21T03:42:22.804 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.804 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.804 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.804 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:22.804 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.804 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.804 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.805 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.805 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster_level": { 2024-09-21T03:42:22.805 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster_level", 2024-09-21T03:42:22.805 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:22.805 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.805 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:22.805 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "info", 2024-09-21T03:42:22.805 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.805 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.805 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-21T03:42:22.805 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2024-09-21T03:42:22.805 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2024-09-21T03:42:22.805 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2024-09-21T03:42:22.806 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2024-09-21T03:42:22.806 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2024-09-21T03:42:22.806 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2024-09-21T03:42:22.806 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-21T03:42:22.806 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:22.806 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.806 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.806 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.806 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.806 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_file": { 2024-09-21T03:42:22.806 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_file", 2024-09-21T03:42:22.806 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-21T03:42:22.806 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.806 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:22.807 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-21T03:42:22.807 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.807 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.807 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.807 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:22.807 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.807 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.807 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.807 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.807 INFO:teuthology.orchestra.run.smithi089.stdout: "sqlite3_killpoint": { 2024-09-21T03:42:22.807 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:42:22.807 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2024-09-21T03:42:22.807 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "dev", 2024-09-21T03:42:22.807 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:22.808 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "0", 2024-09-21T03:42:22.808 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.808 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.808 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.808 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:22.808 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.808 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.808 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.808 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.808 INFO:teuthology.orchestra.run.smithi089.stdout: "zabbix_host": { 2024-09-21T03:42:22.808 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "zabbix_host", 2024-09-21T03:42:22.808 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:22.808 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.809 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:22.809 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-21T03:42:22.809 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.809 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.809 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.809 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:22.809 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.809 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.809 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.809 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.809 INFO:teuthology.orchestra.run.smithi089.stdout: "zabbix_port": { 2024-09-21T03:42:22.809 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "zabbix_port", 2024-09-21T03:42:22.809 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2024-09-21T03:42:22.809 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.810 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:22.810 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "10051", 2024-09-21T03:42:22.810 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.810 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.810 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.810 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:22.810 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.810 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.810 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.810 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.810 INFO:teuthology.orchestra.run.smithi089.stdout: "zabbix_sender": { 2024-09-21T03:42:22.810 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "zabbix_sender", 2024-09-21T03:42:22.810 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:22.811 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.811 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:22.811 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-09-21T03:42:22.811 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.811 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.811 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.811 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:22.811 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.811 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.811 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.811 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-21T03:42:22.811 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-21T03:42:22.811 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-21T03:42:22.812 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-21T03:42:22.812 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.812 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-21T03:42:22.812 INFO:teuthology.orchestra.run.smithi089.stdout: "gid": 24134, 2024-09-21T03:42:22.812 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "y", 2024-09-21T03:42:22.812 INFO:teuthology.orchestra.run.smithi089.stdout: "mgr_features": 4540719139924082687, 2024-09-21T03:42:22.812 INFO:teuthology.orchestra.run.smithi089.stdout: "available_modules": [ 2024-09-21T03:42:22.812 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-21T03:42:22.812 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "alerts", 2024-09-21T03:42:22.812 INFO:teuthology.orchestra.run.smithi089.stdout: "can_run": true, 2024-09-21T03:42:22.812 INFO:teuthology.orchestra.run.smithi089.stdout: "error_string": "", 2024-09-21T03:42:22.812 INFO:teuthology.orchestra.run.smithi089.stdout: "module_options": { 2024-09-21T03:42:22.812 INFO:teuthology.orchestra.run.smithi089.stdout: "interval": { 2024-09-21T03:42:22.813 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "interval", 2024-09-21T03:42:22.813 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "secs", 2024-09-21T03:42:22.813 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.813 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:22.813 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "60", 2024-09-21T03:42:22.813 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.813 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.813 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.813 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "How frequently to reexamine health status", 2024-09-21T03:42:22.813 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.813 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.813 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.814 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.814 INFO:teuthology.orchestra.run.smithi089.stdout: "log_level": { 2024-09-21T03:42:22.814 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_level", 2024-09-21T03:42:22.814 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:22.814 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.814 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:22.814 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-21T03:42:22.814 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.814 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.814 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-21T03:42:22.814 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2024-09-21T03:42:22.814 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2024-09-21T03:42:22.815 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2024-09-21T03:42:22.815 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2024-09-21T03:42:22.815 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2024-09-21T03:42:22.815 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2024-09-21T03:42:22.815 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-21T03:42:22.815 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:22.815 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.815 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.815 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.815 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.815 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster": { 2024-09-21T03:42:22.815 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster", 2024-09-21T03:42:22.816 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-21T03:42:22.816 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.816 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:22.816 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-21T03:42:22.816 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.816 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.816 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.816 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:22.816 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.816 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.816 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.816 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.816 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster_level": { 2024-09-21T03:42:22.817 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster_level", 2024-09-21T03:42:22.817 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:22.817 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.817 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:22.817 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "info", 2024-09-21T03:42:22.817 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.817 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.817 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-21T03:42:22.817 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2024-09-21T03:42:22.817 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2024-09-21T03:42:22.817 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2024-09-21T03:42:22.817 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2024-09-21T03:42:22.818 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2024-09-21T03:42:22.818 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2024-09-21T03:42:22.818 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-21T03:42:22.818 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:22.818 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.818 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.818 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.818 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.818 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_file": { 2024-09-21T03:42:22.818 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_file", 2024-09-21T03:42:22.818 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-21T03:42:22.818 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.818 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:22.818 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-21T03:42:22.819 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.819 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.819 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.819 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:22.819 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.819 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.819 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.819 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.819 INFO:teuthology.orchestra.run.smithi089.stdout: "smtp_destination": { 2024-09-21T03:42:22.819 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "smtp_destination", 2024-09-21T03:42:22.819 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:22.819 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.819 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:22.820 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-21T03:42:22.820 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.820 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.820 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.820 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Email address to send alerts to, use commas to separate multiple", 2024-09-21T03:42:22.820 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.820 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.820 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.820 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.820 INFO:teuthology.orchestra.run.smithi089.stdout: "smtp_from_name": { 2024-09-21T03:42:22.820 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "smtp_from_name", 2024-09-21T03:42:22.820 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:22.820 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.820 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:22.821 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "Ceph", 2024-09-21T03:42:22.821 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.821 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.821 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.821 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Email From: name", 2024-09-21T03:42:22.821 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.821 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.821 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.821 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.821 INFO:teuthology.orchestra.run.smithi089.stdout: "smtp_host": { 2024-09-21T03:42:22.821 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "smtp_host", 2024-09-21T03:42:22.821 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:22.821 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.821 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:22.821 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-21T03:42:22.822 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.822 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.822 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.822 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "SMTP server", 2024-09-21T03:42:22.822 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.822 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.822 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.822 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.822 INFO:teuthology.orchestra.run.smithi089.stdout: "smtp_password": { 2024-09-21T03:42:22.822 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "smtp_password", 2024-09-21T03:42:22.822 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:22.822 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.822 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:22.823 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-21T03:42:22.823 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.823 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.823 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.823 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Password to authenticate with", 2024-09-21T03:42:22.823 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.823 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.823 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.823 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.823 INFO:teuthology.orchestra.run.smithi089.stdout: "smtp_port": { 2024-09-21T03:42:22.823 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "smtp_port", 2024-09-21T03:42:22.823 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2024-09-21T03:42:22.823 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.824 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:22.824 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "465", 2024-09-21T03:42:22.824 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.824 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.824 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.824 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "SMTP port", 2024-09-21T03:42:22.824 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.824 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.824 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.824 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.824 INFO:teuthology.orchestra.run.smithi089.stdout: "smtp_sender": { 2024-09-21T03:42:22.824 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "smtp_sender", 2024-09-21T03:42:22.824 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:22.824 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.825 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:22.825 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-21T03:42:22.825 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.825 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.825 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.825 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "SMTP envelope sender", 2024-09-21T03:42:22.825 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.825 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.825 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.825 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.825 INFO:teuthology.orchestra.run.smithi089.stdout: "smtp_ssl": { 2024-09-21T03:42:22.825 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "smtp_ssl", 2024-09-21T03:42:22.825 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-21T03:42:22.825 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.826 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:22.826 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "True", 2024-09-21T03:42:22.826 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.826 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.826 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.826 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Use SSL to connect to SMTP server", 2024-09-21T03:42:22.826 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.826 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.826 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.826 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.826 INFO:teuthology.orchestra.run.smithi089.stdout: "smtp_user": { 2024-09-21T03:42:22.827 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "smtp_user", 2024-09-21T03:42:22.827 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:22.827 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.827 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:22.827 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-21T03:42:22.827 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.827 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.827 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.827 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "User to authenticate as", 2024-09-21T03:42:22.827 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.827 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.827 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.827 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.827 INFO:teuthology.orchestra.run.smithi089.stdout: "sqlite3_killpoint": { 2024-09-21T03:42:22.828 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:42:22.828 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2024-09-21T03:42:22.828 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "dev", 2024-09-21T03:42:22.828 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:22.828 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "0", 2024-09-21T03:42:22.828 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.828 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.828 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.828 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:22.828 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.828 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.828 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.828 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-21T03:42:22.829 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-21T03:42:22.829 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.829 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-21T03:42:22.829 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "balancer", 2024-09-21T03:42:22.829 INFO:teuthology.orchestra.run.smithi089.stdout: "can_run": true, 2024-09-21T03:42:22.829 INFO:teuthology.orchestra.run.smithi089.stdout: "error_string": "", 2024-09-21T03:42:22.829 INFO:teuthology.orchestra.run.smithi089.stdout: "module_options": { 2024-09-21T03:42:22.829 INFO:teuthology.orchestra.run.smithi089.stdout: "active": { 2024-09-21T03:42:22.829 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "active", 2024-09-21T03:42:22.829 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-21T03:42:22.829 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.829 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:22.829 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "True", 2024-09-21T03:42:22.829 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.830 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.830 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.830 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "automatically balance PGs across cluster", 2024-09-21T03:42:22.830 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.830 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.830 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.830 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.830 INFO:teuthology.orchestra.run.smithi089.stdout: "begin_time": { 2024-09-21T03:42:22.830 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "begin_time", 2024-09-21T03:42:22.830 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:22.830 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.830 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:22.830 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "0000", 2024-09-21T03:42:22.831 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.831 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.831 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.831 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "beginning time of day to automatically balance", 2024-09-21T03:42:22.831 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-21T03:42:22.831 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.831 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.831 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.831 INFO:teuthology.orchestra.run.smithi089.stdout: "begin_weekday": { 2024-09-21T03:42:22.831 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "begin_weekday", 2024-09-21T03:42:22.831 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "uint", 2024-09-21T03:42:22.831 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.831 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:22.832 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "0", 2024-09-21T03:42:22.832 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "0", 2024-09-21T03:42:22.832 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "6", 2024-09-21T03:42:22.832 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.832 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-09-21T03:42:22.832 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-21T03:42:22.832 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.832 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.832 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.832 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_compat_max_iterations": { 2024-09-21T03:42:22.832 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "crush_compat_max_iterations", 2024-09-21T03:42:22.832 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "uint", 2024-09-21T03:42:22.833 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.833 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:22.833 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "25", 2024-09-21T03:42:22.833 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "1", 2024-09-21T03:42:22.833 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "250", 2024-09-21T03:42:22.833 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.833 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-09-21T03:42:22.833 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.833 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.833 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.833 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.833 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_compat_metrics": { 2024-09-21T03:42:22.833 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "crush_compat_metrics", 2024-09-21T03:42:22.834 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:22.834 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.834 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:22.834 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "pgs,objects,bytes", 2024-09-21T03:42:22.834 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.834 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.834 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.834 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-09-21T03:42:22.834 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-09-21T03:42:22.834 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.834 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.834 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.834 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_compat_step": { 2024-09-21T03:42:22.835 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "crush_compat_step", 2024-09-21T03:42:22.835 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "float", 2024-09-21T03:42:22.835 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.835 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:22.835 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "0.5", 2024-09-21T03:42:22.835 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "0.001", 2024-09-21T03:42:22.835 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "0.999", 2024-09-21T03:42:22.835 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.835 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "aggressiveness of optimization", 2024-09-21T03:42:22.835 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-09-21T03:42:22.835 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.835 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.835 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.836 INFO:teuthology.orchestra.run.smithi089.stdout: "end_time": { 2024-09-21T03:42:22.836 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "end_time", 2024-09-21T03:42:22.836 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:22.836 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.836 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:22.836 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "2359", 2024-09-21T03:42:22.836 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.836 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.836 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.836 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "ending time of day to automatically balance", 2024-09-21T03:42:22.836 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-21T03:42:22.836 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.836 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.836 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.837 INFO:teuthology.orchestra.run.smithi089.stdout: "end_weekday": { 2024-09-21T03:42:22.837 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "end_weekday", 2024-09-21T03:42:22.837 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "uint", 2024-09-21T03:42:22.837 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.837 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:22.837 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "0", 2024-09-21T03:42:22.837 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "0", 2024-09-21T03:42:22.837 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "6", 2024-09-21T03:42:22.837 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.837 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-09-21T03:42:22.837 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-21T03:42:22.837 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.837 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.837 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.838 INFO:teuthology.orchestra.run.smithi089.stdout: "log_level": { 2024-09-21T03:42:22.838 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_level", 2024-09-21T03:42:22.838 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:22.838 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.838 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:22.838 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-21T03:42:22.838 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.838 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.838 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-21T03:42:22.838 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2024-09-21T03:42:22.838 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2024-09-21T03:42:22.838 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2024-09-21T03:42:22.838 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2024-09-21T03:42:22.838 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2024-09-21T03:42:22.838 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2024-09-21T03:42:22.839 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-21T03:42:22.839 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:22.839 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.839 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.839 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.839 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.839 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster": { 2024-09-21T03:42:22.839 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster", 2024-09-21T03:42:22.839 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-21T03:42:22.839 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.839 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:22.839 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-21T03:42:22.839 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.840 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.840 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.840 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:22.840 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.840 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.840 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.840 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.840 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster_level": { 2024-09-21T03:42:22.840 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster_level", 2024-09-21T03:42:22.840 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:22.840 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.840 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:22.840 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "info", 2024-09-21T03:42:22.840 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.841 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.841 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-21T03:42:22.841 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2024-09-21T03:42:22.841 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2024-09-21T03:42:22.841 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2024-09-21T03:42:22.841 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2024-09-21T03:42:22.841 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2024-09-21T03:42:22.841 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2024-09-21T03:42:22.841 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-21T03:42:22.841 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:22.841 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.841 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.841 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.841 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.842 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_file": { 2024-09-21T03:42:22.842 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_file", 2024-09-21T03:42:22.842 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-21T03:42:22.842 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.842 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:22.842 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-21T03:42:22.842 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.842 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.842 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.842 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:22.842 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.842 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.842 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.843 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.843 INFO:teuthology.orchestra.run.smithi089.stdout: "min_score": { 2024-09-21T03:42:22.843 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "min_score", 2024-09-21T03:42:22.843 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "float", 2024-09-21T03:42:22.843 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.843 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:22.843 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "0", 2024-09-21T03:42:22.843 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.843 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.843 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.843 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-09-21T03:42:22.843 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.843 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.844 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.844 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.844 INFO:teuthology.orchestra.run.smithi089.stdout: "mode": { 2024-09-21T03:42:22.844 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "mode", 2024-09-21T03:42:22.844 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:22.844 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.844 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:22.844 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "upmap", 2024-09-21T03:42:22.844 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.844 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.844 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-21T03:42:22.844 INFO:teuthology.orchestra.run.smithi089.stdout: "crush-compat", 2024-09-21T03:42:22.844 INFO:teuthology.orchestra.run.smithi089.stdout: "none", 2024-09-21T03:42:22.845 INFO:teuthology.orchestra.run.smithi089.stdout: "read", 2024-09-21T03:42:22.845 INFO:teuthology.orchestra.run.smithi089.stdout: "upmap", 2024-09-21T03:42:22.845 INFO:teuthology.orchestra.run.smithi089.stdout: "upmap-read" 2024-09-21T03:42:22.845 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-21T03:42:22.845 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Balancer mode", 2024-09-21T03:42:22.845 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.845 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.845 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.845 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.845 INFO:teuthology.orchestra.run.smithi089.stdout: "pool_ids": { 2024-09-21T03:42:22.845 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "pool_ids", 2024-09-21T03:42:22.845 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:22.846 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.846 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:22.846 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-21T03:42:22.846 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.846 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.846 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.846 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-09-21T03:42:22.846 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.846 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.846 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.846 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.846 INFO:teuthology.orchestra.run.smithi089.stdout: "sleep_interval": { 2024-09-21T03:42:22.846 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "sleep_interval", 2024-09-21T03:42:22.847 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "secs", 2024-09-21T03:42:22.847 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.847 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:22.847 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "60", 2024-09-21T03:42:22.847 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.847 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.847 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.847 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-09-21T03:42:22.847 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.847 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.847 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.847 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.847 INFO:teuthology.orchestra.run.smithi089.stdout: "sqlite3_killpoint": { 2024-09-21T03:42:22.848 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:42:22.848 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2024-09-21T03:42:22.848 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "dev", 2024-09-21T03:42:22.848 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:22.848 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "0", 2024-09-21T03:42:22.848 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.848 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.848 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.848 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:22.848 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.848 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.848 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.848 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.848 INFO:teuthology.orchestra.run.smithi089.stdout: "upmap_max_deviation": { 2024-09-21T03:42:22.848 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "upmap_max_deviation", 2024-09-21T03:42:22.849 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2024-09-21T03:42:22.849 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.849 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:22.849 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "5", 2024-09-21T03:42:22.849 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "1", 2024-09-21T03:42:22.849 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.849 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.849 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "deviation below which no optimization is attempted", 2024-09-21T03:42:22.849 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-09-21T03:42:22.849 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.849 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.849 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.850 INFO:teuthology.orchestra.run.smithi089.stdout: "upmap_max_optimizations": { 2024-09-21T03:42:22.850 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "upmap_max_optimizations", 2024-09-21T03:42:22.850 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "uint", 2024-09-21T03:42:22.850 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.850 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:22.850 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "10", 2024-09-21T03:42:22.850 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.850 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.850 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.850 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-09-21T03:42:22.850 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.850 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.851 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.851 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-21T03:42:22.851 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-21T03:42:22.851 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.851 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-21T03:42:22.851 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "cephadm", 2024-09-21T03:42:22.851 INFO:teuthology.orchestra.run.smithi089.stdout: "can_run": true, 2024-09-21T03:42:22.851 INFO:teuthology.orchestra.run.smithi089.stdout: "error_string": "", 2024-09-21T03:42:22.851 INFO:teuthology.orchestra.run.smithi089.stdout: "module_options": { 2024-09-21T03:42:22.851 INFO:teuthology.orchestra.run.smithi089.stdout: "agent_down_multiplier": { 2024-09-21T03:42:22.851 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "agent_down_multiplier", 2024-09-21T03:42:22.851 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "float", 2024-09-21T03:42:22.851 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.851 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:22.852 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "3.0", 2024-09-21T03:42:22.852 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.852 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.852 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.852 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-09-21T03:42:22.852 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.852 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.852 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.852 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.852 INFO:teuthology.orchestra.run.smithi089.stdout: "agent_refresh_rate": { 2024-09-21T03:42:22.852 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "agent_refresh_rate", 2024-09-21T03:42:22.852 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "secs", 2024-09-21T03:42:22.852 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.853 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:22.853 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "20", 2024-09-21T03:42:22.853 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.853 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.853 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.853 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-09-21T03:42:22.853 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.853 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.853 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.853 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.853 INFO:teuthology.orchestra.run.smithi089.stdout: "agent_starting_port": { 2024-09-21T03:42:22.853 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "agent_starting_port", 2024-09-21T03:42:22.853 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2024-09-21T03:42:22.853 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.854 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:22.854 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "4721", 2024-09-21T03:42:22.854 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.854 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.854 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.854 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-09-21T03:42:22.854 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.854 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.854 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.854 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.854 INFO:teuthology.orchestra.run.smithi089.stdout: "allow_ptrace": { 2024-09-21T03:42:22.854 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "allow_ptrace", 2024-09-21T03:42:22.855 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-21T03:42:22.855 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.855 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:22.855 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-21T03:42:22.855 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.855 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.855 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.855 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-09-21T03:42:22.855 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-09-21T03:42:22.855 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.855 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.855 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.855 INFO:teuthology.orchestra.run.smithi089.stdout: "autotune_interval": { 2024-09-21T03:42:22.856 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "autotune_interval", 2024-09-21T03:42:22.856 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "secs", 2024-09-21T03:42:22.856 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.856 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:22.856 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "600", 2024-09-21T03:42:22.856 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.856 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.856 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.856 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "how frequently to autotune daemon memory", 2024-09-21T03:42:22.856 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.856 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.856 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.857 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.857 INFO:teuthology.orchestra.run.smithi089.stdout: "autotune_memory_target_ratio": { 2024-09-21T03:42:22.857 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "autotune_memory_target_ratio", 2024-09-21T03:42:22.857 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "float", 2024-09-21T03:42:22.857 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.857 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:22.857 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "0.7", 2024-09-21T03:42:22.857 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.857 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.857 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.857 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-09-21T03:42:22.857 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.857 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.858 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.858 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.858 INFO:teuthology.orchestra.run.smithi089.stdout: "cephadm_log_destination": { 2024-09-21T03:42:22.858 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "cephadm_log_destination", 2024-09-21T03:42:22.858 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:22.858 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.858 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:22.858 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-21T03:42:22.858 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.858 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.858 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-21T03:42:22.859 INFO:teuthology.orchestra.run.smithi089.stdout: "file", 2024-09-21T03:42:22.859 INFO:teuthology.orchestra.run.smithi089.stdout: "file,syslog", 2024-09-21T03:42:22.859 INFO:teuthology.orchestra.run.smithi089.stdout: "syslog" 2024-09-21T03:42:22.859 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-21T03:42:22.859 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-09-21T03:42:22.859 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.859 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.859 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.859 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.859 INFO:teuthology.orchestra.run.smithi089.stdout: "cgroups_split": { 2024-09-21T03:42:22.859 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "cgroups_split", 2024-09-21T03:42:22.859 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-21T03:42:22.859 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.860 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:22.860 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "True", 2024-09-21T03:42:22.860 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.860 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.860 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.860 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-09-21T03:42:22.860 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.860 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.860 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.860 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.860 INFO:teuthology.orchestra.run.smithi089.stdout: "config_checks_enabled": { 2024-09-21T03:42:22.860 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "config_checks_enabled", 2024-09-21T03:42:22.861 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-21T03:42:22.861 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.861 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:22.861 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-21T03:42:22.861 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.861 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.861 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.861 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-09-21T03:42:22.861 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.861 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.861 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.861 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.861 INFO:teuthology.orchestra.run.smithi089.stdout: "config_dashboard": { 2024-09-21T03:42:22.862 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "config_dashboard", 2024-09-21T03:42:22.862 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-21T03:42:22.862 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.862 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:22.862 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "True", 2024-09-21T03:42:22.862 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.862 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.862 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.862 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-09-21T03:42:22.862 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.862 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.863 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.863 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.863 INFO:teuthology.orchestra.run.smithi089.stdout: "container_image_alertmanager": { 2024-09-21T03:42:22.863 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "container_image_alertmanager", 2024-09-21T03:42:22.863 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:22.863 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.863 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:22.863 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.27.0", 2024-09-21T03:42:22.863 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.863 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.863 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.863 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Prometheus container image", 2024-09-21T03:42:22.864 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.864 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.864 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.864 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.864 INFO:teuthology.orchestra.run.smithi089.stdout: "container_image_base": { 2024-09-21T03:42:22.864 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "container_image_base", 2024-09-21T03:42:22.864 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:22.864 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.864 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:22.864 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "quay.io/ceph/ceph", 2024-09-21T03:42:22.864 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.865 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.865 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.865 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Container image name, without the tag", 2024-09-21T03:42:22.865 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.865 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.865 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.865 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.865 INFO:teuthology.orchestra.run.smithi089.stdout: "container_image_elasticsearch": { 2024-09-21T03:42:22.865 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "container_image_elasticsearch", 2024-09-21T03:42:22.865 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:22.865 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.865 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:22.866 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-09-21T03:42:22.866 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.866 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.866 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.866 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "elasticsearch container image", 2024-09-21T03:42:22.866 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.866 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.866 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.866 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.866 INFO:teuthology.orchestra.run.smithi089.stdout: "container_image_grafana": { 2024-09-21T03:42:22.866 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "container_image_grafana", 2024-09-21T03:42:22.867 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:22.867 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.867 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:22.867 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "quay.io/ceph/grafana:10.4.8", 2024-09-21T03:42:22.867 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.867 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.867 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.867 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Prometheus container image", 2024-09-21T03:42:22.867 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.867 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.867 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.867 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.868 INFO:teuthology.orchestra.run.smithi089.stdout: "container_image_haproxy": { 2024-09-21T03:42:22.868 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "container_image_haproxy", 2024-09-21T03:42:22.868 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:22.868 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.868 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:22.868 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-09-21T03:42:22.868 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.868 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.868 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.868 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "HAproxy container image", 2024-09-21T03:42:22.868 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.868 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.868 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.868 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.869 INFO:teuthology.orchestra.run.smithi089.stdout: "container_image_jaeger_agent": { 2024-09-21T03:42:22.869 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "container_image_jaeger_agent", 2024-09-21T03:42:22.869 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:22.869 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.869 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:22.869 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-09-21T03:42:22.869 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.869 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.869 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.869 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Jaeger agent container image", 2024-09-21T03:42:22.869 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.869 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.869 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.870 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.870 INFO:teuthology.orchestra.run.smithi089.stdout: "container_image_jaeger_collector": { 2024-09-21T03:42:22.870 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "container_image_jaeger_collector", 2024-09-21T03:42:22.870 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:22.870 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.870 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:22.870 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-09-21T03:42:22.870 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.870 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.870 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.871 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Jaeger collector container image", 2024-09-21T03:42:22.871 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.871 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.871 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.871 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.871 INFO:teuthology.orchestra.run.smithi089.stdout: "container_image_jaeger_query": { 2024-09-21T03:42:22.871 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "container_image_jaeger_query", 2024-09-21T03:42:22.871 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:22.871 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.871 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:22.871 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-09-21T03:42:22.871 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.871 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.872 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.872 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Jaeger query container image", 2024-09-21T03:42:22.872 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.872 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.872 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.872 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.872 INFO:teuthology.orchestra.run.smithi089.stdout: "container_image_keepalived": { 2024-09-21T03:42:22.872 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "container_image_keepalived", 2024-09-21T03:42:22.872 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:22.872 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.872 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:22.872 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-09-21T03:42:22.872 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.872 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.873 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.873 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Keepalived container image", 2024-09-21T03:42:22.873 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.873 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.873 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.873 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.873 INFO:teuthology.orchestra.run.smithi089.stdout: "container_image_loki": { 2024-09-21T03:42:22.873 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "container_image_loki", 2024-09-21T03:42:22.873 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:22.873 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.873 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:22.873 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "docker.io/grafana/loki:3.0.0", 2024-09-21T03:42:22.873 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.874 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.874 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.874 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Loki container image", 2024-09-21T03:42:22.874 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.874 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.874 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.874 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.874 INFO:teuthology.orchestra.run.smithi089.stdout: "container_image_nginx": { 2024-09-21T03:42:22.874 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "container_image_nginx", 2024-09-21T03:42:22.874 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:22.874 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.874 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:22.875 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "quay.io/ceph/nginx:sclorg-nginx-126", 2024-09-21T03:42:22.875 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.875 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.875 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.875 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Nginx container image", 2024-09-21T03:42:22.875 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.875 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.875 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.875 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.875 INFO:teuthology.orchestra.run.smithi089.stdout: "container_image_node_exporter": { 2024-09-21T03:42:22.875 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "container_image_node_exporter", 2024-09-21T03:42:22.875 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:22.875 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.876 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:22.876 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.7.0", 2024-09-21T03:42:22.876 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.876 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.876 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.876 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Prometheus container image", 2024-09-21T03:42:22.876 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.876 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.876 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.876 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.876 INFO:teuthology.orchestra.run.smithi089.stdout: "container_image_nvmeof": { 2024-09-21T03:42:22.876 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "container_image_nvmeof", 2024-09-21T03:42:22.877 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:22.877 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.877 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:22.877 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.17", 2024-09-21T03:42:22.877 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.877 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.877 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.877 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Nvme-of container image", 2024-09-21T03:42:22.877 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.877 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.877 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.877 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.877 INFO:teuthology.orchestra.run.smithi089.stdout: "container_image_oauth2_proxy": { 2024-09-21T03:42:22.877 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "container_image_oauth2_proxy", 2024-09-21T03:42:22.878 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:22.878 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.878 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:22.878 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "quay.io/oauth2-proxy/oauth2-proxy:v7.6.0", 2024-09-21T03:42:22.878 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.878 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.878 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.878 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "oauth2-proxy container image", 2024-09-21T03:42:22.878 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.878 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.878 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.878 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.879 INFO:teuthology.orchestra.run.smithi089.stdout: "container_image_prometheus": { 2024-09-21T03:42:22.879 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "container_image_prometheus", 2024-09-21T03:42:22.879 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:22.879 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.879 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:22.879 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "quay.io/prometheus/prometheus:v2.51.0", 2024-09-21T03:42:22.879 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.879 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.879 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.879 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Prometheus container image", 2024-09-21T03:42:22.879 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.879 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.879 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.880 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.880 INFO:teuthology.orchestra.run.smithi089.stdout: "container_image_promtail": { 2024-09-21T03:42:22.880 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "container_image_promtail", 2024-09-21T03:42:22.880 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:22.880 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.880 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:22.880 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "docker.io/grafana/promtail:3.0.0", 2024-09-21T03:42:22.880 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.880 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.880 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.880 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Promtail container image", 2024-09-21T03:42:22.880 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.881 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.881 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.881 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.881 INFO:teuthology.orchestra.run.smithi089.stdout: "container_image_samba": { 2024-09-21T03:42:22.881 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "container_image_samba", 2024-09-21T03:42:22.881 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:22.881 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.881 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:22.881 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-09-21T03:42:22.881 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.881 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.881 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.882 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Samba/SMB container image", 2024-09-21T03:42:22.882 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.882 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.882 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.882 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.882 INFO:teuthology.orchestra.run.smithi089.stdout: "container_image_samba_metrics": { 2024-09-21T03:42:22.882 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "container_image_samba_metrics", 2024-09-21T03:42:22.882 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:22.882 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.882 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:22.882 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "quay.io/samba.org/samba-metrics:latest", 2024-09-21T03:42:22.882 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.883 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.883 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.883 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Samba/SMB metrics exporter container image", 2024-09-21T03:42:22.883 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.883 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.883 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.883 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.883 INFO:teuthology.orchestra.run.smithi089.stdout: "container_image_snmp_gateway": { 2024-09-21T03:42:22.883 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "container_image_snmp_gateway", 2024-09-21T03:42:22.883 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:22.883 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.883 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:22.884 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-09-21T03:42:22.884 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.884 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.884 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.884 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "SNMP Gateway container image", 2024-09-21T03:42:22.884 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.884 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.884 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.884 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.884 INFO:teuthology.orchestra.run.smithi089.stdout: "container_init": { 2024-09-21T03:42:22.884 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "container_init", 2024-09-21T03:42:22.884 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-21T03:42:22.885 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.885 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:22.885 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "True", 2024-09-21T03:42:22.885 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.885 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.885 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.885 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Run podman/docker with `--init`", 2024-09-21T03:42:22.885 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.885 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.885 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.885 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.885 INFO:teuthology.orchestra.run.smithi089.stdout: "daemon_cache_timeout": { 2024-09-21T03:42:22.885 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "daemon_cache_timeout", 2024-09-21T03:42:22.886 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "secs", 2024-09-21T03:42:22.886 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.886 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:22.886 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "600", 2024-09-21T03:42:22.886 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.886 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.886 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.886 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-09-21T03:42:22.886 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.886 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.886 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.886 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.886 INFO:teuthology.orchestra.run.smithi089.stdout: "default_cephadm_command_timeout": { 2024-09-21T03:42:22.887 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "default_cephadm_command_timeout", 2024-09-21T03:42:22.887 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2024-09-21T03:42:22.887 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.887 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:22.887 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "900", 2024-09-21T03:42:22.887 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.887 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.887 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.887 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-09-21T03:42:22.887 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.887 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.887 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.887 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.888 INFO:teuthology.orchestra.run.smithi089.stdout: "default_registry": { 2024-09-21T03:42:22.888 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "default_registry", 2024-09-21T03:42:22.888 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:22.888 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.888 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:22.888 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "docker.io", 2024-09-21T03:42:22.888 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.888 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.888 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.888 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-09-21T03:42:22.888 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.888 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.888 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.888 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.889 INFO:teuthology.orchestra.run.smithi089.stdout: "device_cache_timeout": { 2024-09-21T03:42:22.889 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "device_cache_timeout", 2024-09-21T03:42:22.889 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "secs", 2024-09-21T03:42:22.889 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.889 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:22.889 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "1800", 2024-09-21T03:42:22.889 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.889 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.889 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.889 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "seconds to cache device inventory", 2024-09-21T03:42:22.889 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.889 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.889 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.890 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.890 INFO:teuthology.orchestra.run.smithi089.stdout: "device_enhanced_scan": { 2024-09-21T03:42:22.890 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "device_enhanced_scan", 2024-09-21T03:42:22.890 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-21T03:42:22.890 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.890 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:22.890 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-21T03:42:22.890 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.890 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.890 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.890 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Use libstoragemgmt during device scans", 2024-09-21T03:42:22.890 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.891 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.891 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.891 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.891 INFO:teuthology.orchestra.run.smithi089.stdout: "facts_cache_timeout": { 2024-09-21T03:42:22.891 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "facts_cache_timeout", 2024-09-21T03:42:22.891 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "secs", 2024-09-21T03:42:22.891 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.891 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:22.891 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "60", 2024-09-21T03:42:22.891 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.891 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.891 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.891 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "seconds to cache host facts data", 2024-09-21T03:42:22.892 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.892 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.892 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.892 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.892 INFO:teuthology.orchestra.run.smithi089.stdout: "grafana_dashboards_path": { 2024-09-21T03:42:22.892 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "grafana_dashboards_path", 2024-09-21T03:42:22.892 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:22.892 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.892 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:22.892 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-09-21T03:42:22.892 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.892 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.892 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.893 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-09-21T03:42:22.893 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.893 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.893 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.893 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.893 INFO:teuthology.orchestra.run.smithi089.stdout: "host_check_interval": { 2024-09-21T03:42:22.893 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "host_check_interval", 2024-09-21T03:42:22.893 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "secs", 2024-09-21T03:42:22.893 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.893 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:22.893 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "600", 2024-09-21T03:42:22.894 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.894 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.894 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.894 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "how frequently to perform a host check", 2024-09-21T03:42:22.894 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.894 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.894 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.894 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.894 INFO:teuthology.orchestra.run.smithi089.stdout: "hw_monitoring": { 2024-09-21T03:42:22.894 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "hw_monitoring", 2024-09-21T03:42:22.894 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-21T03:42:22.894 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.894 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:22.895 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-21T03:42:22.895 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.895 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.895 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.895 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-09-21T03:42:22.895 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.895 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.895 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.895 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.895 INFO:teuthology.orchestra.run.smithi089.stdout: "inventory_list_all": { 2024-09-21T03:42:22.896 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "inventory_list_all", 2024-09-21T03:42:22.896 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-21T03:42:22.896 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.896 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:22.896 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-21T03:42:22.896 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.896 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.896 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.896 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-09-21T03:42:22.896 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.896 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.896 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.897 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.897 INFO:teuthology.orchestra.run.smithi089.stdout: "log_level": { 2024-09-21T03:42:22.897 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_level", 2024-09-21T03:42:22.897 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:22.897 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.897 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:22.897 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-21T03:42:22.897 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.897 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.897 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-21T03:42:22.897 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2024-09-21T03:42:22.897 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2024-09-21T03:42:22.897 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2024-09-21T03:42:22.898 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2024-09-21T03:42:22.898 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2024-09-21T03:42:22.898 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2024-09-21T03:42:22.898 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-21T03:42:22.898 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:22.898 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.898 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.898 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.898 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.898 INFO:teuthology.orchestra.run.smithi089.stdout: "log_refresh_metadata": { 2024-09-21T03:42:22.898 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_refresh_metadata", 2024-09-21T03:42:22.898 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-21T03:42:22.898 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.899 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:22.899 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-21T03:42:22.899 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.899 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.899 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.899 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-09-21T03:42:22.899 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.899 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.899 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.899 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.899 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster": { 2024-09-21T03:42:22.899 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster", 2024-09-21T03:42:22.900 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-21T03:42:22.900 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.900 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:22.900 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "True", 2024-09-21T03:42:22.900 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.900 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.900 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.900 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-09-21T03:42:22.900 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.900 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.900 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.900 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.900 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster_level": { 2024-09-21T03:42:22.900 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster_level", 2024-09-21T03:42:22.901 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:22.901 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.901 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:22.901 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "info", 2024-09-21T03:42:22.901 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.901 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.901 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-21T03:42:22.901 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2024-09-21T03:42:22.901 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2024-09-21T03:42:22.901 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2024-09-21T03:42:22.901 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2024-09-21T03:42:22.901 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2024-09-21T03:42:22.901 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2024-09-21T03:42:22.901 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-21T03:42:22.902 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:22.902 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.902 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.902 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.902 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.902 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_file": { 2024-09-21T03:42:22.902 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_file", 2024-09-21T03:42:22.902 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-21T03:42:22.902 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.902 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:22.902 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-21T03:42:22.902 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.902 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.903 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.903 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:22.903 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.903 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.903 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.903 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.903 INFO:teuthology.orchestra.run.smithi089.stdout: "manage_etc_ceph_ceph_conf": { 2024-09-21T03:42:22.903 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-09-21T03:42:22.903 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-21T03:42:22.903 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.903 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:22.903 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-21T03:42:22.903 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.903 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.904 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.904 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-09-21T03:42:22.904 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.904 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.904 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.904 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.904 INFO:teuthology.orchestra.run.smithi089.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-09-21T03:42:22.904 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-09-21T03:42:22.904 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:22.904 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.904 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:22.904 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "*", 2024-09-21T03:42:22.904 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.904 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.905 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.905 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-09-21T03:42:22.905 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.905 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.905 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.905 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.905 INFO:teuthology.orchestra.run.smithi089.stdout: "max_count_per_host": { 2024-09-21T03:42:22.905 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "max_count_per_host", 2024-09-21T03:42:22.905 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2024-09-21T03:42:22.905 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.905 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:22.905 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "10", 2024-09-21T03:42:22.905 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.905 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.906 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.906 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "max number of daemons per service per host", 2024-09-21T03:42:22.906 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.906 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.906 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.906 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.906 INFO:teuthology.orchestra.run.smithi089.stdout: "max_osd_draining_count": { 2024-09-21T03:42:22.906 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "max_osd_draining_count", 2024-09-21T03:42:22.906 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2024-09-21T03:42:22.906 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.906 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:22.906 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "10", 2024-09-21T03:42:22.906 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.907 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.907 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.907 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-09-21T03:42:22.907 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.907 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.907 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.907 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.907 INFO:teuthology.orchestra.run.smithi089.stdout: "migration_current": { 2024-09-21T03:42:22.907 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "migration_current", 2024-09-21T03:42:22.907 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2024-09-21T03:42:22.907 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.907 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:22.907 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-21T03:42:22.908 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.908 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.908 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.908 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "internal - do not modify", 2024-09-21T03:42:22.908 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.908 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.908 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.908 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.908 INFO:teuthology.orchestra.run.smithi089.stdout: "mode": { 2024-09-21T03:42:22.908 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "mode", 2024-09-21T03:42:22.908 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:22.908 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.908 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:22.908 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "root", 2024-09-21T03:42:22.909 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.909 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.909 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-21T03:42:22.909 INFO:teuthology.orchestra.run.smithi089.stdout: "cephadm-package", 2024-09-21T03:42:22.909 INFO:teuthology.orchestra.run.smithi089.stdout: "root" 2024-09-21T03:42:22.909 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-21T03:42:22.909 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "mode for remote execution of cephadm", 2024-09-21T03:42:22.909 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.909 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.909 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.909 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.909 INFO:teuthology.orchestra.run.smithi089.stdout: "oob_default_addr": { 2024-09-21T03:42:22.909 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "oob_default_addr", 2024-09-21T03:42:22.909 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:22.910 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.910 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:22.910 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "169.254.1.1", 2024-09-21T03:42:22.910 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.910 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.910 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.910 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Default address for RedFish API (oob management).", 2024-09-21T03:42:22.910 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.910 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.910 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.910 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.910 INFO:teuthology.orchestra.run.smithi089.stdout: "prometheus_alerts_path": { 2024-09-21T03:42:22.910 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "prometheus_alerts_path", 2024-09-21T03:42:22.911 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:22.911 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.911 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:22.911 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-09-21T03:42:22.911 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.911 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.911 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.911 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-09-21T03:42:22.911 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.911 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.911 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.911 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.911 INFO:teuthology.orchestra.run.smithi089.stdout: "registry_insecure": { 2024-09-21T03:42:22.912 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "registry_insecure", 2024-09-21T03:42:22.912 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-21T03:42:22.912 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.912 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:22.912 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-21T03:42:22.912 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.912 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.912 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.912 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-09-21T03:42:22.912 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.912 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.912 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.912 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.913 INFO:teuthology.orchestra.run.smithi089.stdout: "registry_password": { 2024-09-21T03:42:22.913 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "registry_password", 2024-09-21T03:42:22.913 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:22.913 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.913 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:22.913 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-21T03:42:22.913 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.913 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.913 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.913 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-09-21T03:42:22.913 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.913 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.913 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.913 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.914 INFO:teuthology.orchestra.run.smithi089.stdout: "registry_url": { 2024-09-21T03:42:22.914 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "registry_url", 2024-09-21T03:42:22.914 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:22.914 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.914 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:22.914 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-21T03:42:22.914 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.914 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.914 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.914 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-09-21T03:42:22.914 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.914 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.914 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.915 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.915 INFO:teuthology.orchestra.run.smithi089.stdout: "registry_username": { 2024-09-21T03:42:22.915 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "registry_username", 2024-09-21T03:42:22.915 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:22.915 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.915 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:22.915 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-21T03:42:22.915 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.915 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.915 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.915 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-09-21T03:42:22.915 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.915 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.916 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.916 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.916 INFO:teuthology.orchestra.run.smithi089.stdout: "secure_monitoring_stack": { 2024-09-21T03:42:22.916 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "secure_monitoring_stack", 2024-09-21T03:42:22.916 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-21T03:42:22.916 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.916 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:22.916 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-21T03:42:22.916 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.916 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.916 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.916 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-09-21T03:42:22.916 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.917 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.917 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.917 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.917 INFO:teuthology.orchestra.run.smithi089.stdout: "service_discovery_port": { 2024-09-21T03:42:22.917 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "service_discovery_port", 2024-09-21T03:42:22.917 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2024-09-21T03:42:22.917 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.917 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:22.917 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "8765", 2024-09-21T03:42:22.917 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.917 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.917 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.917 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "cephadm service discovery port", 2024-09-21T03:42:22.918 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.918 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.918 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.918 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.918 INFO:teuthology.orchestra.run.smithi089.stdout: "sqlite3_killpoint": { 2024-09-21T03:42:22.918 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:42:22.918 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2024-09-21T03:42:22.918 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "dev", 2024-09-21T03:42:22.918 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:22.918 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "0", 2024-09-21T03:42:22.918 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.918 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.919 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.919 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:22.919 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.919 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.919 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.919 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.919 INFO:teuthology.orchestra.run.smithi089.stdout: "ssh_config_file": { 2024-09-21T03:42:22.919 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "ssh_config_file", 2024-09-21T03:42:22.919 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:22.919 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.919 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:22.919 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-21T03:42:22.919 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.920 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.920 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.920 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-09-21T03:42:22.920 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.920 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.920 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.920 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.920 INFO:teuthology.orchestra.run.smithi089.stdout: "ssh_keepalive_count_max": { 2024-09-21T03:42:22.920 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "ssh_keepalive_count_max", 2024-09-21T03:42:22.920 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2024-09-21T03:42:22.920 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.921 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:22.921 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "3", 2024-09-21T03:42:22.921 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.921 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.921 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.921 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "How many times ssh connections can fail liveness checks before the host is marked offline", 2024-09-21T03:42:22.921 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.921 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.921 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.921 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.921 INFO:teuthology.orchestra.run.smithi089.stdout: "ssh_keepalive_interval": { 2024-09-21T03:42:22.922 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "ssh_keepalive_interval", 2024-09-21T03:42:22.922 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2024-09-21T03:42:22.922 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.922 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:22.922 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "7", 2024-09-21T03:42:22.922 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.922 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.922 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.922 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "How often ssh connections are checked for liveness", 2024-09-21T03:42:22.922 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.922 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.922 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.922 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.923 INFO:teuthology.orchestra.run.smithi089.stdout: "use_agent": { 2024-09-21T03:42:22.923 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "use_agent", 2024-09-21T03:42:22.923 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-21T03:42:22.923 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.923 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:22.923 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-21T03:42:22.923 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.923 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.923 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.923 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-09-21T03:42:22.923 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.923 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.924 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.924 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.924 INFO:teuthology.orchestra.run.smithi089.stdout: "use_repo_digest": { 2024-09-21T03:42:22.924 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "use_repo_digest", 2024-09-21T03:42:22.924 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-21T03:42:22.924 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.924 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:22.924 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "True", 2024-09-21T03:42:22.924 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.924 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.924 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.924 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-09-21T03:42:22.925 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.925 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.925 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.925 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.925 INFO:teuthology.orchestra.run.smithi089.stdout: "warn_on_failed_host_check": { 2024-09-21T03:42:22.925 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "warn_on_failed_host_check", 2024-09-21T03:42:22.925 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-21T03:42:22.925 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.925 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:22.925 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "True", 2024-09-21T03:42:22.925 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.926 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.926 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.926 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "raise a health warning if the host check fails", 2024-09-21T03:42:22.926 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.926 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.926 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.926 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.926 INFO:teuthology.orchestra.run.smithi089.stdout: "warn_on_stray_daemons": { 2024-09-21T03:42:22.926 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "warn_on_stray_daemons", 2024-09-21T03:42:22.926 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-21T03:42:22.926 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.926 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:22.927 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "True", 2024-09-21T03:42:22.927 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.927 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.927 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.927 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-09-21T03:42:22.927 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.927 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.927 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.927 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.927 INFO:teuthology.orchestra.run.smithi089.stdout: "warn_on_stray_hosts": { 2024-09-21T03:42:22.927 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "warn_on_stray_hosts", 2024-09-21T03:42:22.928 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-21T03:42:22.928 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.928 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:22.928 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "True", 2024-09-21T03:42:22.928 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.928 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.928 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.928 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-09-21T03:42:22.928 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.928 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.928 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.929 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-21T03:42:22.929 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-21T03:42:22.929 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.929 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-21T03:42:22.929 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "crash", 2024-09-21T03:42:22.929 INFO:teuthology.orchestra.run.smithi089.stdout: "can_run": true, 2024-09-21T03:42:22.929 INFO:teuthology.orchestra.run.smithi089.stdout: "error_string": "", 2024-09-21T03:42:22.929 INFO:teuthology.orchestra.run.smithi089.stdout: "module_options": { 2024-09-21T03:42:22.929 INFO:teuthology.orchestra.run.smithi089.stdout: "log_level": { 2024-09-21T03:42:22.929 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_level", 2024-09-21T03:42:22.929 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:22.930 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.930 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:22.930 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-21T03:42:22.930 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.930 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.930 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-21T03:42:22.930 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2024-09-21T03:42:22.930 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2024-09-21T03:42:22.930 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2024-09-21T03:42:22.930 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2024-09-21T03:42:22.930 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2024-09-21T03:42:22.931 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2024-09-21T03:42:22.931 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-21T03:42:22.931 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:22.931 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.931 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.931 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.931 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.931 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster": { 2024-09-21T03:42:22.931 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster", 2024-09-21T03:42:22.931 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-21T03:42:22.931 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.931 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:22.931 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-21T03:42:22.932 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.932 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.932 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.932 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:22.932 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.932 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.932 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.932 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.932 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster_level": { 2024-09-21T03:42:22.932 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster_level", 2024-09-21T03:42:22.932 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:22.932 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.933 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:22.933 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "info", 2024-09-21T03:42:22.933 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.933 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.933 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-21T03:42:22.933 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2024-09-21T03:42:22.933 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2024-09-21T03:42:22.933 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2024-09-21T03:42:22.933 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2024-09-21T03:42:22.933 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2024-09-21T03:42:22.933 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2024-09-21T03:42:22.933 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-21T03:42:22.933 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:22.934 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.934 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.934 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.934 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.934 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_file": { 2024-09-21T03:42:22.934 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_file", 2024-09-21T03:42:22.934 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-21T03:42:22.934 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.934 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:22.934 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-21T03:42:22.934 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.934 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.935 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.935 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:22.935 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.935 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.935 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.935 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.935 INFO:teuthology.orchestra.run.smithi089.stdout: "retain_interval": { 2024-09-21T03:42:22.935 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "retain_interval", 2024-09-21T03:42:22.935 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "secs", 2024-09-21T03:42:22.935 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.935 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:22.935 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "31536000", 2024-09-21T03:42:22.936 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.936 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.936 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.936 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "how long to retain crashes before pruning them", 2024-09-21T03:42:22.936 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.936 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.936 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.936 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.936 INFO:teuthology.orchestra.run.smithi089.stdout: "sqlite3_killpoint": { 2024-09-21T03:42:22.936 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:42:22.936 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2024-09-21T03:42:22.936 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "dev", 2024-09-21T03:42:22.936 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:22.937 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "0", 2024-09-21T03:42:22.937 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.937 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.937 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.937 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:22.937 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.937 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.937 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.937 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.937 INFO:teuthology.orchestra.run.smithi089.stdout: "warn_recent_interval": { 2024-09-21T03:42:22.937 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "warn_recent_interval", 2024-09-21T03:42:22.937 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "secs", 2024-09-21T03:42:22.937 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.937 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:22.938 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "1209600", 2024-09-21T03:42:22.938 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.938 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.938 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.938 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "time interval in which to warn about recent crashes", 2024-09-21T03:42:22.938 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.938 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.938 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.938 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-21T03:42:22.938 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-21T03:42:22.938 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.938 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-21T03:42:22.938 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "dashboard", 2024-09-21T03:42:22.939 INFO:teuthology.orchestra.run.smithi089.stdout: "can_run": true, 2024-09-21T03:42:22.939 INFO:teuthology.orchestra.run.smithi089.stdout: "error_string": "", 2024-09-21T03:42:22.939 INFO:teuthology.orchestra.run.smithi089.stdout: "module_options": { 2024-09-21T03:42:22.939 INFO:teuthology.orchestra.run.smithi089.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-09-21T03:42:22.939 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-09-21T03:42:22.939 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2024-09-21T03:42:22.939 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.939 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:22.939 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "10", 2024-09-21T03:42:22.939 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.939 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.939 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.939 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:22.940 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.940 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.940 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.940 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.940 INFO:teuthology.orchestra.run.smithi089.stdout: "ALERTMANAGER_API_HOST": { 2024-09-21T03:42:22.940 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "ALERTMANAGER_API_HOST", 2024-09-21T03:42:22.940 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:22.940 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.940 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:22.940 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-21T03:42:22.940 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.940 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.940 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.940 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:22.940 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.941 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.941 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.941 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.941 INFO:teuthology.orchestra.run.smithi089.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-09-21T03:42:22.941 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-09-21T03:42:22.941 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-21T03:42:22.941 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.941 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:22.941 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "True", 2024-09-21T03:42:22.941 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.941 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.941 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.941 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:22.942 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.942 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.942 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.942 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.942 INFO:teuthology.orchestra.run.smithi089.stdout: "AUDIT_API_ENABLED": { 2024-09-21T03:42:22.942 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "AUDIT_API_ENABLED", 2024-09-21T03:42:22.942 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-21T03:42:22.942 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.942 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:22.942 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-21T03:42:22.942 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.943 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.943 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.943 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:22.943 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.943 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.943 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.943 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.943 INFO:teuthology.orchestra.run.smithi089.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-09-21T03:42:22.943 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-09-21T03:42:22.943 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-21T03:42:22.943 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.943 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:22.944 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "True", 2024-09-21T03:42:22.944 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.944 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.944 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.944 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:22.944 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.944 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.944 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.944 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.944 INFO:teuthology.orchestra.run.smithi089.stdout: "ENABLE_BROWSABLE_API": { 2024-09-21T03:42:22.944 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "ENABLE_BROWSABLE_API", 2024-09-21T03:42:22.944 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-21T03:42:22.945 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.945 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:22.945 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "True", 2024-09-21T03:42:22.945 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.945 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.945 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.945 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:22.945 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.945 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.945 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.945 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.945 INFO:teuthology.orchestra.run.smithi089.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-09-21T03:42:22.945 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-09-21T03:42:22.946 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-21T03:42:22.946 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.946 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:22.946 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "True", 2024-09-21T03:42:22.946 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.946 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.946 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.946 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:22.946 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.946 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.946 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.946 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.947 INFO:teuthology.orchestra.run.smithi089.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-09-21T03:42:22.947 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-09-21T03:42:22.947 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-21T03:42:22.947 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.947 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:22.947 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "True", 2024-09-21T03:42:22.947 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.947 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.947 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.947 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:22.947 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.947 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.948 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.948 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.948 INFO:teuthology.orchestra.run.smithi089.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-09-21T03:42:22.948 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-09-21T03:42:22.948 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-21T03:42:22.948 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.948 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:22.948 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "True", 2024-09-21T03:42:22.948 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.948 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.948 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.948 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:22.948 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.949 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.949 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.949 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.949 INFO:teuthology.orchestra.run.smithi089.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-09-21T03:42:22.949 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-09-21T03:42:22.949 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-21T03:42:22.949 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.949 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:22.949 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "True", 2024-09-21T03:42:22.949 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.949 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.949 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.949 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:22.950 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.950 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.950 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.950 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.950 INFO:teuthology.orchestra.run.smithi089.stdout: "FEATURE_TOGGLE_NFS": { 2024-09-21T03:42:22.950 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-09-21T03:42:22.950 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-21T03:42:22.950 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.950 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:22.950 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "True", 2024-09-21T03:42:22.950 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.951 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.951 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.951 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:22.951 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.951 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.951 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.951 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.951 INFO:teuthology.orchestra.run.smithi089.stdout: "FEATURE_TOGGLE_RBD": { 2024-09-21T03:42:22.951 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-09-21T03:42:22.951 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-21T03:42:22.951 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.951 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:22.951 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "True", 2024-09-21T03:42:22.952 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.952 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.952 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.952 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:22.952 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.952 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.952 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.952 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.952 INFO:teuthology.orchestra.run.smithi089.stdout: "FEATURE_TOGGLE_RGW": { 2024-09-21T03:42:22.952 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-09-21T03:42:22.952 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-21T03:42:22.952 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.953 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:22.953 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "True", 2024-09-21T03:42:22.953 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.953 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.953 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.953 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:22.953 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.953 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.953 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.953 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.953 INFO:teuthology.orchestra.run.smithi089.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-09-21T03:42:22.953 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-09-21T03:42:22.953 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:22.954 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.954 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:22.954 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-21T03:42:22.954 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.954 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.954 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.954 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:22.954 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.954 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.954 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.954 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.954 INFO:teuthology.orchestra.run.smithi089.stdout: "GRAFANA_API_PASSWORD": { 2024-09-21T03:42:22.955 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "GRAFANA_API_PASSWORD", 2024-09-21T03:42:22.955 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:22.955 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.955 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:22.955 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "admin", 2024-09-21T03:42:22.955 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.955 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.955 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.955 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:22.955 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.955 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.955 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.956 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.956 INFO:teuthology.orchestra.run.smithi089.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-09-21T03:42:22.956 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-09-21T03:42:22.956 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-21T03:42:22.956 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.956 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:22.956 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "True", 2024-09-21T03:42:22.956 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.956 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.956 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.956 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:22.957 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.957 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.957 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.957 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.957 INFO:teuthology.orchestra.run.smithi089.stdout: "GRAFANA_API_URL": { 2024-09-21T03:42:22.957 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "GRAFANA_API_URL", 2024-09-21T03:42:22.957 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:22.957 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.957 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:22.957 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-21T03:42:22.957 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.957 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.957 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.958 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:22.958 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.958 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.958 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.958 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.958 INFO:teuthology.orchestra.run.smithi089.stdout: "GRAFANA_API_USERNAME": { 2024-09-21T03:42:22.958 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "GRAFANA_API_USERNAME", 2024-09-21T03:42:22.958 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:22.958 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.958 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:22.958 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "admin", 2024-09-21T03:42:22.958 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.959 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.959 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.959 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:22.959 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.959 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.959 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.959 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.959 INFO:teuthology.orchestra.run.smithi089.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-09-21T03:42:22.959 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-09-21T03:42:22.959 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:22.959 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.960 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:22.960 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-21T03:42:22.960 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.960 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.960 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.960 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:22.960 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.960 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.960 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.960 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.960 INFO:teuthology.orchestra.run.smithi089.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-09-21T03:42:22.960 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-09-21T03:42:22.960 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-21T03:42:22.960 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.961 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:22.961 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-21T03:42:22.961 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.961 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.961 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.961 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:22.961 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.961 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.961 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.961 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.961 INFO:teuthology.orchestra.run.smithi089.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-09-21T03:42:22.961 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-09-21T03:42:22.961 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-21T03:42:22.962 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.962 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:22.962 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "True", 2024-09-21T03:42:22.962 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.962 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.962 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.962 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:22.962 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.962 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.962 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.962 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.962 INFO:teuthology.orchestra.run.smithi089.stdout: "ISSUE_TRACKER_API_KEY": { 2024-09-21T03:42:22.962 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-09-21T03:42:22.963 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:22.963 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.963 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:22.963 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-21T03:42:22.963 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.963 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.963 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.963 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:22.963 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.963 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.963 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.963 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.963 INFO:teuthology.orchestra.run.smithi089.stdout: "MANAGED_BY_CLUSTERS": { 2024-09-21T03:42:22.964 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-09-21T03:42:22.964 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:22.964 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.964 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:22.964 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "[]", 2024-09-21T03:42:22.964 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.964 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.964 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.964 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:22.964 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.964 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.964 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.964 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.965 INFO:teuthology.orchestra.run.smithi089.stdout: "MULTICLUSTER_CONFIG": { 2024-09-21T03:42:22.965 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "MULTICLUSTER_CONFIG", 2024-09-21T03:42:22.965 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:22.965 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.965 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:22.965 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "{}", 2024-09-21T03:42:22.965 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.965 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.965 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.965 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:22.965 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.965 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.966 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.966 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.966 INFO:teuthology.orchestra.run.smithi089.stdout: "PROMETHEUS_API_HOST": { 2024-09-21T03:42:22.966 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "PROMETHEUS_API_HOST", 2024-09-21T03:42:22.966 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:22.966 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.966 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:22.966 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-21T03:42:22.966 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.966 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.966 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.966 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:22.967 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.967 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.967 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.967 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.967 INFO:teuthology.orchestra.run.smithi089.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-09-21T03:42:22.967 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-09-21T03:42:22.967 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-21T03:42:22.967 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.967 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:22.967 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "True", 2024-09-21T03:42:22.968 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.968 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.968 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.968 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:22.968 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.968 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.968 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.968 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.968 INFO:teuthology.orchestra.run.smithi089.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-09-21T03:42:22.968 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-09-21T03:42:22.968 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-21T03:42:22.969 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.969 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:22.969 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-21T03:42:22.969 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.969 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.969 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.969 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:22.969 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.969 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.969 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.969 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.969 INFO:teuthology.orchestra.run.smithi089.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-09-21T03:42:22.969 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-09-21T03:42:22.970 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-21T03:42:22.970 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.970 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:22.970 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-21T03:42:22.970 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.970 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.970 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.970 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:22.970 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.970 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.970 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.970 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.970 INFO:teuthology.orchestra.run.smithi089.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-09-21T03:42:22.971 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-09-21T03:42:22.971 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-21T03:42:22.971 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.971 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:22.971 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "True", 2024-09-21T03:42:22.971 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.971 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.971 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.971 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:22.971 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.971 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.971 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.971 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.972 INFO:teuthology.orchestra.run.smithi089.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-09-21T03:42:22.972 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-09-21T03:42:22.972 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-21T03:42:22.972 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.972 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:22.972 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "True", 2024-09-21T03:42:22.972 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.972 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.972 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.972 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:22.972 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.972 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.972 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.973 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.973 INFO:teuthology.orchestra.run.smithi089.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-09-21T03:42:22.973 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-09-21T03:42:22.973 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-21T03:42:22.973 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.973 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:22.973 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-21T03:42:22.973 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.973 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.973 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.973 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:22.973 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.973 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.974 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.974 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.974 INFO:teuthology.orchestra.run.smithi089.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-09-21T03:42:22.974 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-09-21T03:42:22.974 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-21T03:42:22.974 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.974 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:22.974 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-21T03:42:22.974 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.974 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.974 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.975 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:22.975 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.975 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.975 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.975 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.975 INFO:teuthology.orchestra.run.smithi089.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-09-21T03:42:22.975 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-09-21T03:42:22.975 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-21T03:42:22.975 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.975 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:22.975 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-21T03:42:22.975 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.975 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.975 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.976 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:22.976 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.976 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.976 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.976 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.976 INFO:teuthology.orchestra.run.smithi089.stdout: "PWD_POLICY_ENABLED": { 2024-09-21T03:42:22.976 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "PWD_POLICY_ENABLED", 2024-09-21T03:42:22.976 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-21T03:42:22.976 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.976 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:22.976 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "True", 2024-09-21T03:42:22.977 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.977 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.977 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.977 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:22.977 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.977 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.977 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.977 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.977 INFO:teuthology.orchestra.run.smithi089.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-09-21T03:42:22.977 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-09-21T03:42:22.977 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:22.977 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.977 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:22.978 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-09-21T03:42:22.978 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.978 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.978 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.978 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:22.978 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.978 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.978 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.978 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.978 INFO:teuthology.orchestra.run.smithi089.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-09-21T03:42:22.978 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-09-21T03:42:22.978 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2024-09-21T03:42:22.978 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.979 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:22.979 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "10", 2024-09-21T03:42:22.979 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.979 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.979 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.979 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:22.979 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.979 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.979 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.979 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.980 INFO:teuthology.orchestra.run.smithi089.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-09-21T03:42:22.980 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-09-21T03:42:22.980 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2024-09-21T03:42:22.980 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.980 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:22.980 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "8", 2024-09-21T03:42:22.980 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.980 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.980 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.980 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:22.980 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.980 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.980 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.980 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.981 INFO:teuthology.orchestra.run.smithi089.stdout: "REST_REQUESTS_TIMEOUT": { 2024-09-21T03:42:22.981 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-09-21T03:42:22.981 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2024-09-21T03:42:22.981 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.981 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:22.981 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "45", 2024-09-21T03:42:22.981 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.981 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.981 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.981 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:22.981 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.981 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.982 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.982 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.982 INFO:teuthology.orchestra.run.smithi089.stdout: "RGW_API_ACCESS_KEY": { 2024-09-21T03:42:22.982 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "RGW_API_ACCESS_KEY", 2024-09-21T03:42:22.982 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:22.982 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.982 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:22.982 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-21T03:42:22.982 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.982 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.982 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.982 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:22.982 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.983 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.983 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.983 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.983 INFO:teuthology.orchestra.run.smithi089.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-09-21T03:42:22.983 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-09-21T03:42:22.983 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:22.983 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.983 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:22.983 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "admin", 2024-09-21T03:42:22.983 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.983 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.983 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.983 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:22.984 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.984 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.984 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.984 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.984 INFO:teuthology.orchestra.run.smithi089.stdout: "RGW_API_SECRET_KEY": { 2024-09-21T03:42:22.984 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "RGW_API_SECRET_KEY", 2024-09-21T03:42:22.984 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:22.984 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.984 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:22.984 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-21T03:42:22.984 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.984 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.984 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.985 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:22.985 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.985 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.985 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.985 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.985 INFO:teuthology.orchestra.run.smithi089.stdout: "RGW_API_SSL_VERIFY": { 2024-09-21T03:42:22.985 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "RGW_API_SSL_VERIFY", 2024-09-21T03:42:22.985 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-21T03:42:22.985 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.985 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:22.985 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "True", 2024-09-21T03:42:22.985 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.985 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.986 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.986 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:22.986 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.986 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.986 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.986 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.986 INFO:teuthology.orchestra.run.smithi089.stdout: "UNSAFE_TLS_v1_2": { 2024-09-21T03:42:22.986 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "UNSAFE_TLS_v1_2", 2024-09-21T03:42:22.986 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-21T03:42:22.986 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.986 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:22.986 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-21T03:42:22.986 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.987 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.987 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.987 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:22.987 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.987 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.987 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.987 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.987 INFO:teuthology.orchestra.run.smithi089.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-09-21T03:42:22.987 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-09-21T03:42:22.987 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2024-09-21T03:42:22.987 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.987 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:22.987 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "0", 2024-09-21T03:42:22.987 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.988 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.988 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.988 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:22.988 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.988 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.988 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.988 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.988 INFO:teuthology.orchestra.run.smithi089.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-09-21T03:42:22.988 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-09-21T03:42:22.988 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2024-09-21T03:42:22.988 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.988 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:22.988 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "10", 2024-09-21T03:42:22.988 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.989 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.989 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.989 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:22.989 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.989 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.989 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.989 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.989 INFO:teuthology.orchestra.run.smithi089.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-09-21T03:42:22.989 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-09-21T03:42:22.989 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2024-09-21T03:42:22.989 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.989 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:22.989 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "5", 2024-09-21T03:42:22.989 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.990 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.990 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.990 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:22.990 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.990 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.990 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.990 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.990 INFO:teuthology.orchestra.run.smithi089.stdout: "cross_origin_url": { 2024-09-21T03:42:22.990 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "cross_origin_url", 2024-09-21T03:42:22.990 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:22.990 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.990 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:22.990 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-21T03:42:22.990 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.991 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.991 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.991 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:22.991 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.991 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.991 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.991 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.991 INFO:teuthology.orchestra.run.smithi089.stdout: "crt_file": { 2024-09-21T03:42:22.991 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "crt_file", 2024-09-21T03:42:22.991 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:22.991 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.991 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:22.991 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-21T03:42:22.991 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.992 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.992 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.992 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:22.992 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.992 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.992 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.992 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.992 INFO:teuthology.orchestra.run.smithi089.stdout: "debug": { 2024-09-21T03:42:22.992 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "debug", 2024-09-21T03:42:22.992 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-21T03:42:22.992 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.992 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:22.993 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-21T03:42:22.993 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.993 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.993 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.993 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Enable/disable debug options", 2024-09-21T03:42:22.993 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.993 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.993 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.993 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.993 INFO:teuthology.orchestra.run.smithi089.stdout: "jwt_token_ttl": { 2024-09-21T03:42:22.993 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "jwt_token_ttl", 2024-09-21T03:42:22.993 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2024-09-21T03:42:22.993 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.993 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:22.994 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "28800", 2024-09-21T03:42:22.994 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.994 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.994 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.994 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:22.994 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.994 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.994 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.994 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.994 INFO:teuthology.orchestra.run.smithi089.stdout: "key_file": { 2024-09-21T03:42:22.994 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "key_file", 2024-09-21T03:42:22.994 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:22.995 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.995 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:22.995 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-21T03:42:22.995 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.995 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.995 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.995 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:22.995 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.995 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.995 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.995 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.995 INFO:teuthology.orchestra.run.smithi089.stdout: "log_level": { 2024-09-21T03:42:22.996 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_level", 2024-09-21T03:42:22.996 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:22.996 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.996 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:22.996 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-21T03:42:22.996 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.996 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.996 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-21T03:42:22.996 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2024-09-21T03:42:22.996 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2024-09-21T03:42:22.996 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2024-09-21T03:42:22.996 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2024-09-21T03:42:22.996 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2024-09-21T03:42:22.996 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2024-09-21T03:42:22.997 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-21T03:42:22.997 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:22.997 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.997 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.997 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.997 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.997 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster": { 2024-09-21T03:42:22.997 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster", 2024-09-21T03:42:22.997 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-21T03:42:22.997 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.997 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:22.997 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-21T03:42:22.997 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.997 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.998 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:22.998 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:22.998 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.998 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.998 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:22.998 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:22.998 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster_level": { 2024-09-21T03:42:22.998 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster_level", 2024-09-21T03:42:22.998 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:22.998 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:22.998 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:22.998 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "info", 2024-09-21T03:42:22.999 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:22.999 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:22.999 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-21T03:42:22.999 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2024-09-21T03:42:22.999 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2024-09-21T03:42:22.999 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2024-09-21T03:42:22.999 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2024-09-21T03:42:22.999 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2024-09-21T03:42:22.999 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2024-09-21T03:42:22.999 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-21T03:42:22.999 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:22.999 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:22.999 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:22.999 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.000 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.000 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_file": { 2024-09-21T03:42:23.000 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_file", 2024-09-21T03:42:23.000 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-21T03:42:23.000 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.000 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:23.000 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-21T03:42:23.000 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.000 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.000 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.000 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.000 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.000 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.000 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.001 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.001 INFO:teuthology.orchestra.run.smithi089.stdout: "motd": { 2024-09-21T03:42:23.001 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "motd", 2024-09-21T03:42:23.001 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:23.001 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.001 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:23.001 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-21T03:42:23.001 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.001 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.001 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.001 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "The message of the day", 2024-09-21T03:42:23.001 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.001 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.001 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.002 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.002 INFO:teuthology.orchestra.run.smithi089.stdout: "redirect_resolve_ip_addr": { 2024-09-21T03:42:23.002 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "redirect_resolve_ip_addr", 2024-09-21T03:42:23.002 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-21T03:42:23.002 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.002 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:23.002 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-21T03:42:23.002 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.002 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.002 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.002 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.002 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.002 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.002 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.003 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.003 INFO:teuthology.orchestra.run.smithi089.stdout: "server_addr": { 2024-09-21T03:42:23.003 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "server_addr", 2024-09-21T03:42:23.003 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:23.003 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.003 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:23.003 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "::", 2024-09-21T03:42:23.003 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.003 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.003 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.003 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.003 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.004 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.004 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.004 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.004 INFO:teuthology.orchestra.run.smithi089.stdout: "server_port": { 2024-09-21T03:42:23.004 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "server_port", 2024-09-21T03:42:23.004 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2024-09-21T03:42:23.004 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.004 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:23.004 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "8080", 2024-09-21T03:42:23.004 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.004 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.004 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.005 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.005 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.005 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.005 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.005 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.005 INFO:teuthology.orchestra.run.smithi089.stdout: "sqlite3_killpoint": { 2024-09-21T03:42:23.005 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:42:23.005 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2024-09-21T03:42:23.005 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "dev", 2024-09-21T03:42:23.005 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:23.005 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "0", 2024-09-21T03:42:23.006 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.006 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.006 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.006 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.006 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.006 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.006 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.006 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.006 INFO:teuthology.orchestra.run.smithi089.stdout: "ssl": { 2024-09-21T03:42:23.006 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "ssl", 2024-09-21T03:42:23.006 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-21T03:42:23.006 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.007 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:23.007 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "True", 2024-09-21T03:42:23.007 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.007 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.007 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.007 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.007 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.007 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.007 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.007 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.007 INFO:teuthology.orchestra.run.smithi089.stdout: "ssl_server_port": { 2024-09-21T03:42:23.007 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "ssl_server_port", 2024-09-21T03:42:23.008 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2024-09-21T03:42:23.008 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.008 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:23.008 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "8443", 2024-09-21T03:42:23.008 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.008 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.008 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.008 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.008 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.008 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.008 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.008 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.009 INFO:teuthology.orchestra.run.smithi089.stdout: "sso_oauth2": { 2024-09-21T03:42:23.009 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "sso_oauth2", 2024-09-21T03:42:23.009 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-21T03:42:23.009 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.009 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:23.009 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-21T03:42:23.009 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.009 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.009 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.009 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.010 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.010 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.010 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.010 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.010 INFO:teuthology.orchestra.run.smithi089.stdout: "standby_behaviour": { 2024-09-21T03:42:23.010 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "standby_behaviour", 2024-09-21T03:42:23.010 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:23.010 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.010 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:23.010 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "redirect", 2024-09-21T03:42:23.010 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.010 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.011 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-21T03:42:23.011 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2024-09-21T03:42:23.011 INFO:teuthology.orchestra.run.smithi089.stdout: "redirect" 2024-09-21T03:42:23.011 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-21T03:42:23.011 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.011 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.011 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.011 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.011 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.011 INFO:teuthology.orchestra.run.smithi089.stdout: "standby_error_status_code": { 2024-09-21T03:42:23.011 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "standby_error_status_code", 2024-09-21T03:42:23.011 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2024-09-21T03:42:23.011 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.012 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:23.012 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "500", 2024-09-21T03:42:23.012 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "400", 2024-09-21T03:42:23.012 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "599", 2024-09-21T03:42:23.012 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.012 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.012 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.012 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.012 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.012 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.012 INFO:teuthology.orchestra.run.smithi089.stdout: "url_prefix": { 2024-09-21T03:42:23.012 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "url_prefix", 2024-09-21T03:42:23.012 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:23.013 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.013 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:23.013 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-21T03:42:23.013 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.013 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.013 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.013 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.013 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.013 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.013 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.013 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-21T03:42:23.013 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-21T03:42:23.013 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.014 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-21T03:42:23.014 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "devicehealth", 2024-09-21T03:42:23.014 INFO:teuthology.orchestra.run.smithi089.stdout: "can_run": true, 2024-09-21T03:42:23.014 INFO:teuthology.orchestra.run.smithi089.stdout: "error_string": "", 2024-09-21T03:42:23.014 INFO:teuthology.orchestra.run.smithi089.stdout: "module_options": { 2024-09-21T03:42:23.014 INFO:teuthology.orchestra.run.smithi089.stdout: "enable_monitoring": { 2024-09-21T03:42:23.014 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "enable_monitoring", 2024-09-21T03:42:23.014 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-21T03:42:23.014 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.014 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:23.014 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "True", 2024-09-21T03:42:23.014 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.015 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.015 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.015 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "monitor device health metrics", 2024-09-21T03:42:23.015 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.015 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.015 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.015 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.015 INFO:teuthology.orchestra.run.smithi089.stdout: "log_level": { 2024-09-21T03:42:23.015 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_level", 2024-09-21T03:42:23.015 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:23.015 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.015 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:23.016 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-21T03:42:23.016 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.016 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.016 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-21T03:42:23.016 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2024-09-21T03:42:23.016 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2024-09-21T03:42:23.016 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2024-09-21T03:42:23.016 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2024-09-21T03:42:23.016 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2024-09-21T03:42:23.016 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2024-09-21T03:42:23.016 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-21T03:42:23.016 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.016 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.016 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.017 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.017 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.017 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster": { 2024-09-21T03:42:23.017 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster", 2024-09-21T03:42:23.017 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-21T03:42:23.017 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.017 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:23.017 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-21T03:42:23.017 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.017 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.017 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.017 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.017 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.017 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.018 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.018 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.018 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster_level": { 2024-09-21T03:42:23.018 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster_level", 2024-09-21T03:42:23.018 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:23.018 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.018 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:23.018 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "info", 2024-09-21T03:42:23.018 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.018 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.018 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-21T03:42:23.018 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2024-09-21T03:42:23.019 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2024-09-21T03:42:23.019 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2024-09-21T03:42:23.019 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2024-09-21T03:42:23.019 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2024-09-21T03:42:23.019 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2024-09-21T03:42:23.019 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-21T03:42:23.019 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.019 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.019 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.019 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.019 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.019 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_file": { 2024-09-21T03:42:23.020 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_file", 2024-09-21T03:42:23.020 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-21T03:42:23.020 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.020 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:23.020 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-21T03:42:23.020 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.020 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.020 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.020 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.020 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.020 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.020 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.020 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.021 INFO:teuthology.orchestra.run.smithi089.stdout: "mark_out_threshold": { 2024-09-21T03:42:23.021 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "mark_out_threshold", 2024-09-21T03:42:23.021 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "secs", 2024-09-21T03:42:23.021 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.021 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:23.021 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "2419200", 2024-09-21T03:42:23.021 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.021 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.021 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.021 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-09-21T03:42:23.021 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.021 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.021 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.022 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.022 INFO:teuthology.orchestra.run.smithi089.stdout: "pool_name": { 2024-09-21T03:42:23.022 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "pool_name", 2024-09-21T03:42:23.022 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:23.022 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.022 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:23.022 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "device_health_metrics", 2024-09-21T03:42:23.022 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.022 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.022 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.022 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "name of pool in which to store device health metrics", 2024-09-21T03:42:23.022 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.022 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.023 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.023 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.023 INFO:teuthology.orchestra.run.smithi089.stdout: "retention_period": { 2024-09-21T03:42:23.023 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "retention_period", 2024-09-21T03:42:23.023 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "secs", 2024-09-21T03:42:23.023 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.023 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:23.023 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "15552000", 2024-09-21T03:42:23.023 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.023 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.023 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.024 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "how long to retain device health metrics", 2024-09-21T03:42:23.024 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.024 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.024 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.024 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.024 INFO:teuthology.orchestra.run.smithi089.stdout: "scrape_frequency": { 2024-09-21T03:42:23.024 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "scrape_frequency", 2024-09-21T03:42:23.024 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "secs", 2024-09-21T03:42:23.024 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.024 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:23.024 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "86400", 2024-09-21T03:42:23.024 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.025 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.025 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.025 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "how frequently to scrape device health metrics", 2024-09-21T03:42:23.025 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.025 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.025 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.025 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.025 INFO:teuthology.orchestra.run.smithi089.stdout: "self_heal": { 2024-09-21T03:42:23.025 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "self_heal", 2024-09-21T03:42:23.025 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-21T03:42:23.025 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.025 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:23.026 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "True", 2024-09-21T03:42:23.026 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.026 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.026 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.026 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-09-21T03:42:23.026 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.026 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.026 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.026 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.026 INFO:teuthology.orchestra.run.smithi089.stdout: "sleep_interval": { 2024-09-21T03:42:23.027 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "sleep_interval", 2024-09-21T03:42:23.027 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "secs", 2024-09-21T03:42:23.027 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.027 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:23.027 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "600", 2024-09-21T03:42:23.027 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.027 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.027 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.027 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "how frequently to wake up and check device health", 2024-09-21T03:42:23.027 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.027 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.027 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.027 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.028 INFO:teuthology.orchestra.run.smithi089.stdout: "sqlite3_killpoint": { 2024-09-21T03:42:23.028 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:42:23.028 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2024-09-21T03:42:23.028 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "dev", 2024-09-21T03:42:23.028 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:23.028 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "0", 2024-09-21T03:42:23.028 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.028 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.028 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.028 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.028 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.028 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.029 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.029 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.029 INFO:teuthology.orchestra.run.smithi089.stdout: "warn_threshold": { 2024-09-21T03:42:23.029 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "warn_threshold", 2024-09-21T03:42:23.029 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "secs", 2024-09-21T03:42:23.029 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.029 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:23.029 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "7257600", 2024-09-21T03:42:23.029 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.029 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.029 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.029 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-09-21T03:42:23.029 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.030 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.030 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.030 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-21T03:42:23.030 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-21T03:42:23.030 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.030 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-21T03:42:23.030 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "diskprediction_local", 2024-09-21T03:42:23.030 INFO:teuthology.orchestra.run.smithi089.stdout: "can_run": true, 2024-09-21T03:42:23.030 INFO:teuthology.orchestra.run.smithi089.stdout: "error_string": "", 2024-09-21T03:42:23.030 INFO:teuthology.orchestra.run.smithi089.stdout: "module_options": { 2024-09-21T03:42:23.030 INFO:teuthology.orchestra.run.smithi089.stdout: "log_level": { 2024-09-21T03:42:23.031 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_level", 2024-09-21T03:42:23.031 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:23.031 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.031 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:23.031 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-21T03:42:23.031 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.031 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.031 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-21T03:42:23.031 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2024-09-21T03:42:23.031 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2024-09-21T03:42:23.031 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2024-09-21T03:42:23.031 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2024-09-21T03:42:23.031 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2024-09-21T03:42:23.032 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2024-09-21T03:42:23.032 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-21T03:42:23.032 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.032 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.032 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.032 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.032 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.032 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster": { 2024-09-21T03:42:23.032 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster", 2024-09-21T03:42:23.032 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-21T03:42:23.032 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.032 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:23.033 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-21T03:42:23.033 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.033 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.033 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.033 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.033 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.033 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.033 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.033 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.033 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster_level": { 2024-09-21T03:42:23.033 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster_level", 2024-09-21T03:42:23.033 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:23.034 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.034 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:23.034 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "info", 2024-09-21T03:42:23.034 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.034 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.034 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-21T03:42:23.034 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2024-09-21T03:42:23.034 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2024-09-21T03:42:23.034 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2024-09-21T03:42:23.034 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2024-09-21T03:42:23.034 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2024-09-21T03:42:23.034 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2024-09-21T03:42:23.035 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-21T03:42:23.035 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.035 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.035 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.035 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.035 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.035 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_file": { 2024-09-21T03:42:23.035 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_file", 2024-09-21T03:42:23.035 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-21T03:42:23.035 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.035 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:23.035 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-21T03:42:23.035 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.036 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.036 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.036 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.036 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.036 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.036 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.036 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.036 INFO:teuthology.orchestra.run.smithi089.stdout: "predict_interval": { 2024-09-21T03:42:23.036 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "predict_interval", 2024-09-21T03:42:23.036 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:23.036 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.036 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:23.037 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "86400", 2024-09-21T03:42:23.037 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.037 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.037 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.037 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.037 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.037 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.037 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.037 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.037 INFO:teuthology.orchestra.run.smithi089.stdout: "predictor_model": { 2024-09-21T03:42:23.037 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "predictor_model", 2024-09-21T03:42:23.037 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:23.038 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.038 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:23.038 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "prophetstor", 2024-09-21T03:42:23.038 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.038 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.038 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.038 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.038 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.038 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.038 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.038 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.038 INFO:teuthology.orchestra.run.smithi089.stdout: "sleep_interval": { 2024-09-21T03:42:23.039 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "sleep_interval", 2024-09-21T03:42:23.039 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:23.039 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.039 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:23.039 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "600", 2024-09-21T03:42:23.039 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.039 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.039 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.039 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.039 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.039 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.039 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.039 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.040 INFO:teuthology.orchestra.run.smithi089.stdout: "sqlite3_killpoint": { 2024-09-21T03:42:23.040 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:42:23.040 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2024-09-21T03:42:23.040 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "dev", 2024-09-21T03:42:23.040 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:23.040 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "0", 2024-09-21T03:42:23.040 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.040 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.040 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.040 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.040 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.040 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.041 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.041 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-21T03:42:23.041 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-21T03:42:23.041 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.041 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-21T03:42:23.041 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "influx", 2024-09-21T03:42:23.041 INFO:teuthology.orchestra.run.smithi089.stdout: "can_run": false, 2024-09-21T03:42:23.041 INFO:teuthology.orchestra.run.smithi089.stdout: "error_string": "influxdb python module not found", 2024-09-21T03:42:23.041 INFO:teuthology.orchestra.run.smithi089.stdout: "module_options": { 2024-09-21T03:42:23.041 INFO:teuthology.orchestra.run.smithi089.stdout: "batch_size": { 2024-09-21T03:42:23.041 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "batch_size", 2024-09-21T03:42:23.041 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2024-09-21T03:42:23.041 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.042 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:23.042 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "5000", 2024-09-21T03:42:23.042 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.042 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.042 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.042 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-09-21T03:42:23.042 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.042 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.042 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.042 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.042 INFO:teuthology.orchestra.run.smithi089.stdout: "database": { 2024-09-21T03:42:23.042 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "database", 2024-09-21T03:42:23.043 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:23.043 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.043 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:23.043 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "ceph", 2024-09-21T03:42:23.043 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.043 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.043 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.043 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-09-21T03:42:23.043 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.043 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.043 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.043 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.043 INFO:teuthology.orchestra.run.smithi089.stdout: "hostname": { 2024-09-21T03:42:23.044 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "hostname", 2024-09-21T03:42:23.044 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:23.044 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.044 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:23.044 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-21T03:42:23.044 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.044 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.044 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.044 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "InfluxDB server hostname", 2024-09-21T03:42:23.044 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.044 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.044 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.044 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.045 INFO:teuthology.orchestra.run.smithi089.stdout: "interval": { 2024-09-21T03:42:23.045 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "interval", 2024-09-21T03:42:23.045 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "secs", 2024-09-21T03:42:23.045 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.045 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:23.045 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "30", 2024-09-21T03:42:23.045 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "5", 2024-09-21T03:42:23.045 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.045 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.045 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-09-21T03:42:23.045 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.045 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.046 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.046 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.046 INFO:teuthology.orchestra.run.smithi089.stdout: "log_level": { 2024-09-21T03:42:23.046 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_level", 2024-09-21T03:42:23.046 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:23.046 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.046 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:23.046 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-21T03:42:23.046 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.046 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.046 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-21T03:42:23.046 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2024-09-21T03:42:23.047 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2024-09-21T03:42:23.047 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2024-09-21T03:42:23.047 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2024-09-21T03:42:23.047 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2024-09-21T03:42:23.047 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2024-09-21T03:42:23.047 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-21T03:42:23.047 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.047 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.047 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.047 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.048 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.048 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster": { 2024-09-21T03:42:23.048 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster", 2024-09-21T03:42:23.048 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-21T03:42:23.048 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.048 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:23.048 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-21T03:42:23.048 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.048 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.048 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.048 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.049 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.049 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.049 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.049 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.049 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster_level": { 2024-09-21T03:42:23.049 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster_level", 2024-09-21T03:42:23.049 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:23.049 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.049 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:23.049 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "info", 2024-09-21T03:42:23.049 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.049 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.049 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-21T03:42:23.050 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2024-09-21T03:42:23.050 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2024-09-21T03:42:23.050 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2024-09-21T03:42:23.050 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2024-09-21T03:42:23.050 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2024-09-21T03:42:23.050 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2024-09-21T03:42:23.050 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-21T03:42:23.050 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.050 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.050 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.050 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.050 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.050 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_file": { 2024-09-21T03:42:23.051 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_file", 2024-09-21T03:42:23.051 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-21T03:42:23.051 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.051 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:23.051 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-21T03:42:23.051 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.051 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.051 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.051 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.051 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.051 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.051 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.052 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.052 INFO:teuthology.orchestra.run.smithi089.stdout: "password": { 2024-09-21T03:42:23.052 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "password", 2024-09-21T03:42:23.052 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:23.052 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.052 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:23.052 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-21T03:42:23.052 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.052 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.052 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.052 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "password of InfluxDB server user", 2024-09-21T03:42:23.052 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.053 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.053 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.053 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.053 INFO:teuthology.orchestra.run.smithi089.stdout: "port": { 2024-09-21T03:42:23.053 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "port", 2024-09-21T03:42:23.053 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2024-09-21T03:42:23.053 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.053 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:23.053 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "8086", 2024-09-21T03:42:23.053 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.053 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.053 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.053 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "InfluxDB server port", 2024-09-21T03:42:23.054 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.054 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.054 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.054 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.054 INFO:teuthology.orchestra.run.smithi089.stdout: "sqlite3_killpoint": { 2024-09-21T03:42:23.054 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:42:23.054 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2024-09-21T03:42:23.054 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "dev", 2024-09-21T03:42:23.054 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:23.054 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "0", 2024-09-21T03:42:23.055 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.055 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.055 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.055 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.055 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.055 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.055 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.055 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.055 INFO:teuthology.orchestra.run.smithi089.stdout: "ssl": { 2024-09-21T03:42:23.055 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "ssl", 2024-09-21T03:42:23.055 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:23.056 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.056 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:23.056 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "false", 2024-09-21T03:42:23.056 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.056 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.056 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.056 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-09-21T03:42:23.056 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.056 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.056 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.056 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.057 INFO:teuthology.orchestra.run.smithi089.stdout: "threads": { 2024-09-21T03:42:23.057 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "threads", 2024-09-21T03:42:23.057 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2024-09-21T03:42:23.057 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.057 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:23.057 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "5", 2024-09-21T03:42:23.057 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "1", 2024-09-21T03:42:23.057 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "32", 2024-09-21T03:42:23.057 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.057 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-09-21T03:42:23.057 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.057 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.057 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.058 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.058 INFO:teuthology.orchestra.run.smithi089.stdout: "username": { 2024-09-21T03:42:23.058 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "username", 2024-09-21T03:42:23.058 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:23.058 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.058 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:23.058 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-21T03:42:23.058 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.058 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.058 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.058 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "username of InfluxDB server user", 2024-09-21T03:42:23.058 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.058 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.059 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.059 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.059 INFO:teuthology.orchestra.run.smithi089.stdout: "verify_ssl": { 2024-09-21T03:42:23.059 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "verify_ssl", 2024-09-21T03:42:23.059 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:23.059 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.059 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:23.059 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "true", 2024-09-21T03:42:23.059 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.059 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.059 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.059 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-09-21T03:42:23.059 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.060 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.060 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.060 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-21T03:42:23.060 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-21T03:42:23.060 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.060 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-21T03:42:23.060 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "insights", 2024-09-21T03:42:23.060 INFO:teuthology.orchestra.run.smithi089.stdout: "can_run": true, 2024-09-21T03:42:23.060 INFO:teuthology.orchestra.run.smithi089.stdout: "error_string": "", 2024-09-21T03:42:23.060 INFO:teuthology.orchestra.run.smithi089.stdout: "module_options": { 2024-09-21T03:42:23.060 INFO:teuthology.orchestra.run.smithi089.stdout: "log_level": { 2024-09-21T03:42:23.060 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_level", 2024-09-21T03:42:23.060 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:23.061 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.061 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:23.061 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-21T03:42:23.061 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.061 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.061 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-21T03:42:23.061 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2024-09-21T03:42:23.061 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2024-09-21T03:42:23.061 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2024-09-21T03:42:23.061 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2024-09-21T03:42:23.061 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2024-09-21T03:42:23.061 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2024-09-21T03:42:23.061 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-21T03:42:23.062 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.062 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.062 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.062 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.062 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.062 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster": { 2024-09-21T03:42:23.062 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster", 2024-09-21T03:42:23.062 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-21T03:42:23.062 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.062 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:23.062 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-21T03:42:23.062 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.062 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.063 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.063 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.063 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.063 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.063 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.063 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.063 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster_level": { 2024-09-21T03:42:23.063 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster_level", 2024-09-21T03:42:23.063 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:23.063 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.063 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:23.063 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "info", 2024-09-21T03:42:23.064 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.064 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.064 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-21T03:42:23.064 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2024-09-21T03:42:23.064 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2024-09-21T03:42:23.064 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2024-09-21T03:42:23.064 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2024-09-21T03:42:23.064 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2024-09-21T03:42:23.064 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2024-09-21T03:42:23.064 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-21T03:42:23.064 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.064 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.064 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.064 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.065 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.065 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_file": { 2024-09-21T03:42:23.065 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_file", 2024-09-21T03:42:23.065 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-21T03:42:23.065 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.065 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:23.065 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-21T03:42:23.065 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.065 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.065 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.065 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.066 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.066 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.066 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.066 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.066 INFO:teuthology.orchestra.run.smithi089.stdout: "sqlite3_killpoint": { 2024-09-21T03:42:23.066 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:42:23.066 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2024-09-21T03:42:23.066 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "dev", 2024-09-21T03:42:23.066 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:23.066 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "0", 2024-09-21T03:42:23.066 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.066 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.067 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.067 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.067 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.067 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.067 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.067 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-21T03:42:23.067 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-21T03:42:23.067 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.067 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-21T03:42:23.067 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "iostat", 2024-09-21T03:42:23.067 INFO:teuthology.orchestra.run.smithi089.stdout: "can_run": true, 2024-09-21T03:42:23.067 INFO:teuthology.orchestra.run.smithi089.stdout: "error_string": "", 2024-09-21T03:42:23.068 INFO:teuthology.orchestra.run.smithi089.stdout: "module_options": { 2024-09-21T03:42:23.068 INFO:teuthology.orchestra.run.smithi089.stdout: "log_level": { 2024-09-21T03:42:23.068 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_level", 2024-09-21T03:42:23.068 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:23.068 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.068 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:23.068 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-21T03:42:23.068 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.068 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.068 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-21T03:42:23.069 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2024-09-21T03:42:23.069 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2024-09-21T03:42:23.069 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2024-09-21T03:42:23.069 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2024-09-21T03:42:23.069 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2024-09-21T03:42:23.069 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2024-09-21T03:42:23.069 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-21T03:42:23.069 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.069 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.069 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.069 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.069 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.070 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster": { 2024-09-21T03:42:23.070 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster", 2024-09-21T03:42:23.070 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-21T03:42:23.070 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.070 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:23.070 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-21T03:42:23.070 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.070 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.070 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.070 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.070 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.070 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.071 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.071 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.071 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster_level": { 2024-09-21T03:42:23.071 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster_level", 2024-09-21T03:42:23.071 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:23.071 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.071 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:23.071 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "info", 2024-09-21T03:42:23.071 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.071 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.071 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-21T03:42:23.071 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2024-09-21T03:42:23.072 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2024-09-21T03:42:23.072 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2024-09-21T03:42:23.072 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2024-09-21T03:42:23.072 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2024-09-21T03:42:23.072 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2024-09-21T03:42:23.072 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-21T03:42:23.072 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.072 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.072 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.072 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.072 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.073 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_file": { 2024-09-21T03:42:23.073 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_file", 2024-09-21T03:42:23.073 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-21T03:42:23.073 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.073 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:23.073 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-21T03:42:23.073 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.073 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.073 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.073 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.073 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.073 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.074 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.074 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.074 INFO:teuthology.orchestra.run.smithi089.stdout: "sqlite3_killpoint": { 2024-09-21T03:42:23.074 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:42:23.074 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2024-09-21T03:42:23.074 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "dev", 2024-09-21T03:42:23.074 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:23.074 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "0", 2024-09-21T03:42:23.074 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.074 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.074 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.074 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.075 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.075 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.075 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.075 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-21T03:42:23.075 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-21T03:42:23.075 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.075 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-21T03:42:23.075 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "k8sevents", 2024-09-21T03:42:23.075 INFO:teuthology.orchestra.run.smithi089.stdout: "can_run": true, 2024-09-21T03:42:23.075 INFO:teuthology.orchestra.run.smithi089.stdout: "error_string": "", 2024-09-21T03:42:23.075 INFO:teuthology.orchestra.run.smithi089.stdout: "module_options": { 2024-09-21T03:42:23.075 INFO:teuthology.orchestra.run.smithi089.stdout: "ceph_event_retention_days": { 2024-09-21T03:42:23.075 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "ceph_event_retention_days", 2024-09-21T03:42:23.076 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2024-09-21T03:42:23.076 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.076 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:23.076 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "7", 2024-09-21T03:42:23.076 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.076 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.076 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.076 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Days to hold ceph event information within local cache", 2024-09-21T03:42:23.076 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.076 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.076 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.077 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.077 INFO:teuthology.orchestra.run.smithi089.stdout: "config_check_secs": { 2024-09-21T03:42:23.077 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "config_check_secs", 2024-09-21T03:42:23.077 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2024-09-21T03:42:23.077 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.077 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:23.077 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "10", 2024-09-21T03:42:23.077 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "10", 2024-09-21T03:42:23.077 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.077 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.077 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-09-21T03:42:23.077 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.077 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.078 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.078 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.078 INFO:teuthology.orchestra.run.smithi089.stdout: "log_level": { 2024-09-21T03:42:23.078 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_level", 2024-09-21T03:42:23.078 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:23.078 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.078 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:23.078 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-21T03:42:23.078 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.078 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.078 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-21T03:42:23.079 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2024-09-21T03:42:23.079 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2024-09-21T03:42:23.079 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2024-09-21T03:42:23.079 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2024-09-21T03:42:23.079 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2024-09-21T03:42:23.079 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2024-09-21T03:42:23.079 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-21T03:42:23.079 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.079 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.079 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.079 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.079 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.079 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster": { 2024-09-21T03:42:23.080 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster", 2024-09-21T03:42:23.080 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-21T03:42:23.080 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.080 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:23.080 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-21T03:42:23.080 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.080 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.080 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.080 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.080 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.080 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.080 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.080 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.081 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster_level": { 2024-09-21T03:42:23.081 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster_level", 2024-09-21T03:42:23.081 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:23.081 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.081 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:23.081 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "info", 2024-09-21T03:42:23.081 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.081 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.081 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-21T03:42:23.081 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2024-09-21T03:42:23.081 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2024-09-21T03:42:23.081 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2024-09-21T03:42:23.082 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2024-09-21T03:42:23.082 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2024-09-21T03:42:23.082 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2024-09-21T03:42:23.082 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-21T03:42:23.082 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.082 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.082 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.082 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.082 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.082 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_file": { 2024-09-21T03:42:23.082 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_file", 2024-09-21T03:42:23.083 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-21T03:42:23.083 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.083 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:23.083 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-21T03:42:23.083 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.083 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.083 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.083 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.083 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.083 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.083 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.083 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.084 INFO:teuthology.orchestra.run.smithi089.stdout: "sqlite3_killpoint": { 2024-09-21T03:42:23.084 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:42:23.084 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2024-09-21T03:42:23.084 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "dev", 2024-09-21T03:42:23.084 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:23.084 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "0", 2024-09-21T03:42:23.084 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.084 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.084 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.084 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.084 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.084 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.084 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.085 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-21T03:42:23.085 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-21T03:42:23.085 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.085 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-21T03:42:23.085 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "localpool", 2024-09-21T03:42:23.085 INFO:teuthology.orchestra.run.smithi089.stdout: "can_run": true, 2024-09-21T03:42:23.085 INFO:teuthology.orchestra.run.smithi089.stdout: "error_string": "", 2024-09-21T03:42:23.085 INFO:teuthology.orchestra.run.smithi089.stdout: "module_options": { 2024-09-21T03:42:23.085 INFO:teuthology.orchestra.run.smithi089.stdout: "failure_domain": { 2024-09-21T03:42:23.085 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "failure_domain", 2024-09-21T03:42:23.085 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:23.085 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.086 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:23.086 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "host", 2024-09-21T03:42:23.086 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.086 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.086 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.086 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "failure domain for any created local pool", 2024-09-21T03:42:23.086 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-09-21T03:42:23.086 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.086 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.086 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.086 INFO:teuthology.orchestra.run.smithi089.stdout: "log_level": { 2024-09-21T03:42:23.087 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_level", 2024-09-21T03:42:23.087 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:23.087 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.087 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:23.087 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-21T03:42:23.087 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.087 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.087 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-21T03:42:23.087 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2024-09-21T03:42:23.087 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2024-09-21T03:42:23.087 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2024-09-21T03:42:23.087 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2024-09-21T03:42:23.087 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2024-09-21T03:42:23.087 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2024-09-21T03:42:23.088 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-21T03:42:23.088 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.088 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.088 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.088 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.088 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.088 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster": { 2024-09-21T03:42:23.088 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster", 2024-09-21T03:42:23.088 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-21T03:42:23.088 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.088 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:23.088 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-21T03:42:23.088 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.088 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.089 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.089 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.089 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.089 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.089 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.089 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.089 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster_level": { 2024-09-21T03:42:23.089 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster_level", 2024-09-21T03:42:23.089 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:23.089 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.089 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:23.089 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "info", 2024-09-21T03:42:23.089 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.089 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.090 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-21T03:42:23.090 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2024-09-21T03:42:23.090 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2024-09-21T03:42:23.090 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2024-09-21T03:42:23.090 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2024-09-21T03:42:23.090 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2024-09-21T03:42:23.090 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2024-09-21T03:42:23.090 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-21T03:42:23.090 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.090 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.090 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.090 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.090 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.091 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_file": { 2024-09-21T03:42:23.091 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_file", 2024-09-21T03:42:23.091 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-21T03:42:23.091 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.091 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:23.091 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-21T03:42:23.091 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.091 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.091 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.091 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.091 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.091 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.091 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.091 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.092 INFO:teuthology.orchestra.run.smithi089.stdout: "min_size": { 2024-09-21T03:42:23.092 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "min_size", 2024-09-21T03:42:23.092 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2024-09-21T03:42:23.092 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.092 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:23.092 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-21T03:42:23.092 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.092 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.092 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.092 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "default min_size for any created local pool", 2024-09-21T03:42:23.092 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-09-21T03:42:23.092 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.092 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.093 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.093 INFO:teuthology.orchestra.run.smithi089.stdout: "num_rep": { 2024-09-21T03:42:23.093 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "num_rep", 2024-09-21T03:42:23.093 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2024-09-21T03:42:23.093 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.093 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:23.093 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "3", 2024-09-21T03:42:23.093 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.093 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.093 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.093 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "default replica count for any created local pool", 2024-09-21T03:42:23.093 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.093 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.093 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.094 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.094 INFO:teuthology.orchestra.run.smithi089.stdout: "pg_num": { 2024-09-21T03:42:23.094 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "pg_num", 2024-09-21T03:42:23.094 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2024-09-21T03:42:23.094 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.094 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:23.094 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "128", 2024-09-21T03:42:23.094 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.094 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.094 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.094 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "default pg_num for any created local pool", 2024-09-21T03:42:23.095 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.095 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.095 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.095 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.095 INFO:teuthology.orchestra.run.smithi089.stdout: "prefix": { 2024-09-21T03:42:23.095 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "prefix", 2024-09-21T03:42:23.095 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:23.095 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.095 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:23.095 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-21T03:42:23.095 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.095 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.096 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.096 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "name prefix for any created local pool", 2024-09-21T03:42:23.096 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.096 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.096 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.096 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.096 INFO:teuthology.orchestra.run.smithi089.stdout: "sqlite3_killpoint": { 2024-09-21T03:42:23.096 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:42:23.096 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2024-09-21T03:42:23.096 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "dev", 2024-09-21T03:42:23.096 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:23.096 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "0", 2024-09-21T03:42:23.096 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.097 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.097 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.097 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.097 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.097 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.097 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.097 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.097 INFO:teuthology.orchestra.run.smithi089.stdout: "subtree": { 2024-09-21T03:42:23.097 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "subtree", 2024-09-21T03:42:23.097 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:23.097 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.097 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:23.097 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "rack", 2024-09-21T03:42:23.098 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.098 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.098 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.098 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "CRUSH level for which to create a local pool", 2024-09-21T03:42:23.098 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-09-21T03:42:23.098 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.098 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.098 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-21T03:42:23.098 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-21T03:42:23.098 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.098 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-21T03:42:23.098 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "mds_autoscaler", 2024-09-21T03:42:23.098 INFO:teuthology.orchestra.run.smithi089.stdout: "can_run": true, 2024-09-21T03:42:23.099 INFO:teuthology.orchestra.run.smithi089.stdout: "error_string": "", 2024-09-21T03:42:23.099 INFO:teuthology.orchestra.run.smithi089.stdout: "module_options": { 2024-09-21T03:42:23.099 INFO:teuthology.orchestra.run.smithi089.stdout: "log_level": { 2024-09-21T03:42:23.099 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_level", 2024-09-21T03:42:23.099 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:23.099 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.099 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:23.099 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-21T03:42:23.099 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.099 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.099 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-21T03:42:23.099 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2024-09-21T03:42:23.099 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2024-09-21T03:42:23.099 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2024-09-21T03:42:23.100 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2024-09-21T03:42:23.100 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2024-09-21T03:42:23.100 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2024-09-21T03:42:23.100 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-21T03:42:23.100 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.100 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.100 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.100 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.100 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.100 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster": { 2024-09-21T03:42:23.100 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster", 2024-09-21T03:42:23.100 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-21T03:42:23.101 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.101 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:23.101 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-21T03:42:23.101 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.101 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.101 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.101 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.101 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.101 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.101 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.101 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.102 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster_level": { 2024-09-21T03:42:23.102 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster_level", 2024-09-21T03:42:23.102 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:23.102 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.102 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:23.102 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "info", 2024-09-21T03:42:23.102 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.102 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.102 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-21T03:42:23.102 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2024-09-21T03:42:23.102 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2024-09-21T03:42:23.102 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2024-09-21T03:42:23.103 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2024-09-21T03:42:23.103 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2024-09-21T03:42:23.103 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2024-09-21T03:42:23.103 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-21T03:42:23.103 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.103 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.103 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.103 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.103 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.103 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_file": { 2024-09-21T03:42:23.103 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_file", 2024-09-21T03:42:23.103 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-21T03:42:23.103 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.103 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:23.105 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-21T03:42:23.105 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.105 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.105 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.105 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.106 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.106 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.106 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.106 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.106 INFO:teuthology.orchestra.run.smithi089.stdout: "sqlite3_killpoint": { 2024-09-21T03:42:23.106 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:42:23.106 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2024-09-21T03:42:23.106 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "dev", 2024-09-21T03:42:23.106 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:23.106 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "0", 2024-09-21T03:42:23.106 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.106 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.107 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.107 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.107 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.107 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.107 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.107 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-21T03:42:23.107 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-21T03:42:23.107 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.107 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-21T03:42:23.107 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "mirroring", 2024-09-21T03:42:23.107 INFO:teuthology.orchestra.run.smithi089.stdout: "can_run": true, 2024-09-21T03:42:23.108 INFO:teuthology.orchestra.run.smithi089.stdout: "error_string": "", 2024-09-21T03:42:23.108 INFO:teuthology.orchestra.run.smithi089.stdout: "module_options": { 2024-09-21T03:42:23.108 INFO:teuthology.orchestra.run.smithi089.stdout: "log_level": { 2024-09-21T03:42:23.108 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_level", 2024-09-21T03:42:23.108 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:23.108 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.108 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:23.108 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-21T03:42:23.108 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.108 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.108 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-21T03:42:23.108 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2024-09-21T03:42:23.109 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2024-09-21T03:42:23.109 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2024-09-21T03:42:23.109 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2024-09-21T03:42:23.109 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2024-09-21T03:42:23.109 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2024-09-21T03:42:23.109 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-21T03:42:23.109 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.109 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.109 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.109 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.109 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.109 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster": { 2024-09-21T03:42:23.109 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster", 2024-09-21T03:42:23.110 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-21T03:42:23.110 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.110 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:23.110 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-21T03:42:23.110 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.110 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.110 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.110 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.110 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.110 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.110 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.110 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.110 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster_level": { 2024-09-21T03:42:23.111 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster_level", 2024-09-21T03:42:23.111 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:23.111 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.111 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:23.111 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "info", 2024-09-21T03:42:23.111 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.111 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.111 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-21T03:42:23.111 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2024-09-21T03:42:23.111 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2024-09-21T03:42:23.111 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2024-09-21T03:42:23.111 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2024-09-21T03:42:23.111 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2024-09-21T03:42:23.112 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2024-09-21T03:42:23.112 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-21T03:42:23.112 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.112 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.112 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.112 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.112 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.112 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_file": { 2024-09-21T03:42:23.112 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_file", 2024-09-21T03:42:23.112 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-21T03:42:23.112 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.112 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:23.112 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-21T03:42:23.112 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.113 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.113 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.113 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.113 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.113 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.113 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.113 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.113 INFO:teuthology.orchestra.run.smithi089.stdout: "sqlite3_killpoint": { 2024-09-21T03:42:23.113 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:42:23.113 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2024-09-21T03:42:23.113 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "dev", 2024-09-21T03:42:23.113 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:23.113 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "0", 2024-09-21T03:42:23.114 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.114 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.114 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.114 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.114 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.114 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.114 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.114 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-21T03:42:23.114 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-21T03:42:23.114 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.115 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-21T03:42:23.115 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "nfs", 2024-09-21T03:42:23.115 INFO:teuthology.orchestra.run.smithi089.stdout: "can_run": true, 2024-09-21T03:42:23.115 INFO:teuthology.orchestra.run.smithi089.stdout: "error_string": "", 2024-09-21T03:42:23.115 INFO:teuthology.orchestra.run.smithi089.stdout: "module_options": { 2024-09-21T03:42:23.115 INFO:teuthology.orchestra.run.smithi089.stdout: "log_level": { 2024-09-21T03:42:23.115 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_level", 2024-09-21T03:42:23.115 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:23.115 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.115 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:23.115 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-21T03:42:23.115 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.115 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.116 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-21T03:42:23.116 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2024-09-21T03:42:23.116 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2024-09-21T03:42:23.116 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2024-09-21T03:42:23.116 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2024-09-21T03:42:23.116 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2024-09-21T03:42:23.116 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2024-09-21T03:42:23.116 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-21T03:42:23.116 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.116 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.116 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.116 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.116 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.117 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster": { 2024-09-21T03:42:23.117 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster", 2024-09-21T03:42:23.117 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-21T03:42:23.117 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.117 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:23.117 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-21T03:42:23.117 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.117 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.117 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.117 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.117 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.117 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.118 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.118 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.118 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster_level": { 2024-09-21T03:42:23.118 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster_level", 2024-09-21T03:42:23.118 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:23.118 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.118 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:23.118 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "info", 2024-09-21T03:42:23.118 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.118 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.118 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-21T03:42:23.119 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2024-09-21T03:42:23.119 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2024-09-21T03:42:23.119 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2024-09-21T03:42:23.119 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2024-09-21T03:42:23.119 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2024-09-21T03:42:23.119 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2024-09-21T03:42:23.119 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-21T03:42:23.119 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.119 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.119 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.119 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.119 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.120 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_file": { 2024-09-21T03:42:23.120 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_file", 2024-09-21T03:42:23.120 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-21T03:42:23.120 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.120 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:23.120 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-21T03:42:23.120 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.120 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.120 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.120 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.120 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.120 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.120 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.120 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.121 INFO:teuthology.orchestra.run.smithi089.stdout: "sqlite3_killpoint": { 2024-09-21T03:42:23.121 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:42:23.121 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2024-09-21T03:42:23.121 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "dev", 2024-09-21T03:42:23.121 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:23.121 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "0", 2024-09-21T03:42:23.121 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.121 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.121 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.121 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.121 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.121 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.121 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.122 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-21T03:42:23.122 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-21T03:42:23.122 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.122 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-21T03:42:23.122 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "orchestrator", 2024-09-21T03:42:23.122 INFO:teuthology.orchestra.run.smithi089.stdout: "can_run": true, 2024-09-21T03:42:23.122 INFO:teuthology.orchestra.run.smithi089.stdout: "error_string": "", 2024-09-21T03:42:23.122 INFO:teuthology.orchestra.run.smithi089.stdout: "module_options": { 2024-09-21T03:42:23.122 INFO:teuthology.orchestra.run.smithi089.stdout: "fail_fs": { 2024-09-21T03:42:23.122 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "fail_fs", 2024-09-21T03:42:23.122 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-21T03:42:23.122 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.123 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:23.123 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-21T03:42:23.123 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.123 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.123 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.123 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-09-21T03:42:23.123 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.123 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.123 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.123 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.123 INFO:teuthology.orchestra.run.smithi089.stdout: "log_level": { 2024-09-21T03:42:23.123 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_level", 2024-09-21T03:42:23.123 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:23.124 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.124 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:23.124 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-21T03:42:23.124 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.124 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.124 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-21T03:42:23.124 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2024-09-21T03:42:23.124 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2024-09-21T03:42:23.124 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2024-09-21T03:42:23.124 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2024-09-21T03:42:23.124 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2024-09-21T03:42:23.124 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2024-09-21T03:42:23.125 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-21T03:42:23.125 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.125 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.125 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.125 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.125 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.125 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster": { 2024-09-21T03:42:23.125 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster", 2024-09-21T03:42:23.125 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-21T03:42:23.125 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.125 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:23.126 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-21T03:42:23.126 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.126 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.126 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.126 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.126 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.126 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.126 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.126 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.126 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster_level": { 2024-09-21T03:42:23.126 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster_level", 2024-09-21T03:42:23.127 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:23.127 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.127 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:23.127 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "info", 2024-09-21T03:42:23.127 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.127 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.127 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-21T03:42:23.127 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2024-09-21T03:42:23.127 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2024-09-21T03:42:23.127 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2024-09-21T03:42:23.127 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2024-09-21T03:42:23.128 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2024-09-21T03:42:23.128 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2024-09-21T03:42:23.128 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-21T03:42:23.128 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.128 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.128 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.128 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.128 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.128 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_file": { 2024-09-21T03:42:23.128 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_file", 2024-09-21T03:42:23.128 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-21T03:42:23.128 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.128 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:23.129 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-21T03:42:23.129 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.129 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.129 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.129 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.129 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.129 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.129 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.129 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.129 INFO:teuthology.orchestra.run.smithi089.stdout: "orchestrator": { 2024-09-21T03:42:23.129 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "orchestrator", 2024-09-21T03:42:23.129 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:23.129 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.130 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:23.130 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-21T03:42:23.130 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.130 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.130 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-21T03:42:23.130 INFO:teuthology.orchestra.run.smithi089.stdout: "cephadm", 2024-09-21T03:42:23.130 INFO:teuthology.orchestra.run.smithi089.stdout: "rook", 2024-09-21T03:42:23.130 INFO:teuthology.orchestra.run.smithi089.stdout: "test_orchestrator" 2024-09-21T03:42:23.130 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-21T03:42:23.130 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Orchestrator backend", 2024-09-21T03:42:23.130 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.131 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.131 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.131 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.131 INFO:teuthology.orchestra.run.smithi089.stdout: "sqlite3_killpoint": { 2024-09-21T03:42:23.131 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:42:23.131 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2024-09-21T03:42:23.131 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "dev", 2024-09-21T03:42:23.131 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:23.131 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "0", 2024-09-21T03:42:23.131 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.131 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.131 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.131 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.132 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.132 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.132 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.132 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-21T03:42:23.132 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-21T03:42:23.132 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.132 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-21T03:42:23.132 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "osd_perf_query", 2024-09-21T03:42:23.132 INFO:teuthology.orchestra.run.smithi089.stdout: "can_run": true, 2024-09-21T03:42:23.132 INFO:teuthology.orchestra.run.smithi089.stdout: "error_string": "", 2024-09-21T03:42:23.132 INFO:teuthology.orchestra.run.smithi089.stdout: "module_options": { 2024-09-21T03:42:23.132 INFO:teuthology.orchestra.run.smithi089.stdout: "log_level": { 2024-09-21T03:42:23.133 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_level", 2024-09-21T03:42:23.133 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:23.133 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.133 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:23.133 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-21T03:42:23.133 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.133 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.133 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-21T03:42:23.133 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2024-09-21T03:42:23.133 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2024-09-21T03:42:23.133 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2024-09-21T03:42:23.133 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2024-09-21T03:42:23.134 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2024-09-21T03:42:23.134 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2024-09-21T03:42:23.134 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-21T03:42:23.134 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.134 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.134 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.134 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.134 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.134 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster": { 2024-09-21T03:42:23.134 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster", 2024-09-21T03:42:23.134 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-21T03:42:23.134 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.135 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:23.135 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-21T03:42:23.135 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.135 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.135 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.135 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.135 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.135 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.135 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.135 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.135 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster_level": { 2024-09-21T03:42:23.135 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster_level", 2024-09-21T03:42:23.135 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:23.136 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.136 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:23.136 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "info", 2024-09-21T03:42:23.136 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.136 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.136 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-21T03:42:23.136 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2024-09-21T03:42:23.136 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2024-09-21T03:42:23.136 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2024-09-21T03:42:23.136 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2024-09-21T03:42:23.136 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2024-09-21T03:42:23.136 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2024-09-21T03:42:23.137 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-21T03:42:23.137 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.137 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.137 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.137 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.137 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.137 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_file": { 2024-09-21T03:42:23.137 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_file", 2024-09-21T03:42:23.137 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-21T03:42:23.137 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.137 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:23.138 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-21T03:42:23.138 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.138 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.138 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.138 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.138 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.138 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.138 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.138 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.138 INFO:teuthology.orchestra.run.smithi089.stdout: "sqlite3_killpoint": { 2024-09-21T03:42:23.138 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:42:23.138 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2024-09-21T03:42:23.138 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "dev", 2024-09-21T03:42:23.139 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:23.139 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "0", 2024-09-21T03:42:23.139 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.139 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.139 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.139 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.139 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.139 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.139 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.139 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-21T03:42:23.139 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-21T03:42:23.139 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.140 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-21T03:42:23.140 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "osd_support", 2024-09-21T03:42:23.140 INFO:teuthology.orchestra.run.smithi089.stdout: "can_run": true, 2024-09-21T03:42:23.140 INFO:teuthology.orchestra.run.smithi089.stdout: "error_string": "", 2024-09-21T03:42:23.140 INFO:teuthology.orchestra.run.smithi089.stdout: "module_options": { 2024-09-21T03:42:23.140 INFO:teuthology.orchestra.run.smithi089.stdout: "log_level": { 2024-09-21T03:42:23.140 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_level", 2024-09-21T03:42:23.140 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:23.140 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.140 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:23.140 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-21T03:42:23.140 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.140 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.141 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-21T03:42:23.141 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2024-09-21T03:42:23.141 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2024-09-21T03:42:23.141 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2024-09-21T03:42:23.141 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2024-09-21T03:42:23.141 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2024-09-21T03:42:23.141 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2024-09-21T03:42:23.141 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-21T03:42:23.141 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.141 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.141 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.142 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.142 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.142 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster": { 2024-09-21T03:42:23.142 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster", 2024-09-21T03:42:23.142 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-21T03:42:23.142 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.142 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:23.142 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-21T03:42:23.142 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.142 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.142 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.143 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.143 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.143 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.143 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.143 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.143 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster_level": { 2024-09-21T03:42:23.143 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster_level", 2024-09-21T03:42:23.143 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:23.143 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.143 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:23.143 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "info", 2024-09-21T03:42:23.143 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.143 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.144 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-21T03:42:23.144 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2024-09-21T03:42:23.144 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2024-09-21T03:42:23.144 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2024-09-21T03:42:23.144 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2024-09-21T03:42:23.144 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2024-09-21T03:42:23.144 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2024-09-21T03:42:23.144 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-21T03:42:23.144 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.144 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.144 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.144 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.145 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.145 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_file": { 2024-09-21T03:42:23.145 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_file", 2024-09-21T03:42:23.145 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-21T03:42:23.145 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.145 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:23.145 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-21T03:42:23.145 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.145 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.145 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.145 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.145 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.145 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.146 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.146 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.146 INFO:teuthology.orchestra.run.smithi089.stdout: "sqlite3_killpoint": { 2024-09-21T03:42:23.146 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:42:23.146 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2024-09-21T03:42:23.146 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "dev", 2024-09-21T03:42:23.146 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:23.146 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "0", 2024-09-21T03:42:23.146 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.146 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.146 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.146 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.146 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.147 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.147 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.147 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-21T03:42:23.147 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-21T03:42:23.147 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.147 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-21T03:42:23.147 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "pg_autoscaler", 2024-09-21T03:42:23.147 INFO:teuthology.orchestra.run.smithi089.stdout: "can_run": true, 2024-09-21T03:42:23.147 INFO:teuthology.orchestra.run.smithi089.stdout: "error_string": "", 2024-09-21T03:42:23.147 INFO:teuthology.orchestra.run.smithi089.stdout: "module_options": { 2024-09-21T03:42:23.147 INFO:teuthology.orchestra.run.smithi089.stdout: "log_level": { 2024-09-21T03:42:23.147 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_level", 2024-09-21T03:42:23.147 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:23.148 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.148 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:23.148 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-21T03:42:23.148 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.148 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.148 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-21T03:42:23.148 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2024-09-21T03:42:23.148 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2024-09-21T03:42:23.148 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2024-09-21T03:42:23.148 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2024-09-21T03:42:23.148 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2024-09-21T03:42:23.148 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2024-09-21T03:42:23.148 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-21T03:42:23.148 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.148 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.149 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.149 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.149 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.149 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster": { 2024-09-21T03:42:23.149 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster", 2024-09-21T03:42:23.149 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-21T03:42:23.149 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.149 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:23.149 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-21T03:42:23.149 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.149 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.149 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.149 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.149 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.150 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.150 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.150 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.150 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster_level": { 2024-09-21T03:42:23.150 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster_level", 2024-09-21T03:42:23.150 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:23.150 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.150 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:23.150 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "info", 2024-09-21T03:42:23.150 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.150 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.150 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-21T03:42:23.150 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2024-09-21T03:42:23.151 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2024-09-21T03:42:23.151 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2024-09-21T03:42:23.151 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2024-09-21T03:42:23.151 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2024-09-21T03:42:23.151 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2024-09-21T03:42:23.151 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-21T03:42:23.151 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.151 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.151 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.151 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.151 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.151 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_file": { 2024-09-21T03:42:23.151 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_file", 2024-09-21T03:42:23.151 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-21T03:42:23.152 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.152 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:23.152 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-21T03:42:23.152 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.152 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.152 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.152 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.152 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.152 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.152 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.152 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.152 INFO:teuthology.orchestra.run.smithi089.stdout: "sleep_interval": { 2024-09-21T03:42:23.152 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "sleep_interval", 2024-09-21T03:42:23.153 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "secs", 2024-09-21T03:42:23.153 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.153 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:23.153 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "60", 2024-09-21T03:42:23.153 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.153 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.153 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.153 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.153 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.153 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.153 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.153 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.153 INFO:teuthology.orchestra.run.smithi089.stdout: "sqlite3_killpoint": { 2024-09-21T03:42:23.153 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:42:23.154 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2024-09-21T03:42:23.154 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "dev", 2024-09-21T03:42:23.154 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:23.154 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "0", 2024-09-21T03:42:23.154 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.154 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.154 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.154 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.154 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.154 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.154 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.154 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.154 INFO:teuthology.orchestra.run.smithi089.stdout: "threshold": { 2024-09-21T03:42:23.154 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "threshold", 2024-09-21T03:42:23.155 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "float", 2024-09-21T03:42:23.155 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.155 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:23.155 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "3.0", 2024-09-21T03:42:23.155 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "1.0", 2024-09-21T03:42:23.155 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.155 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.155 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "scaling threshold", 2024-09-21T03:42:23.155 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-09-21T03:42:23.155 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.155 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.155 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-21T03:42:23.156 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-21T03:42:23.156 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.156 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-21T03:42:23.156 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "progress", 2024-09-21T03:42:23.156 INFO:teuthology.orchestra.run.smithi089.stdout: "can_run": true, 2024-09-21T03:42:23.156 INFO:teuthology.orchestra.run.smithi089.stdout: "error_string": "", 2024-09-21T03:42:23.156 INFO:teuthology.orchestra.run.smithi089.stdout: "module_options": { 2024-09-21T03:42:23.156 INFO:teuthology.orchestra.run.smithi089.stdout: "allow_pg_recovery_event": { 2024-09-21T03:42:23.156 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "allow_pg_recovery_event", 2024-09-21T03:42:23.156 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-21T03:42:23.156 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.156 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:23.156 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-21T03:42:23.157 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.157 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.157 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.157 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "allow the module to show pg recovery progress", 2024-09-21T03:42:23.157 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.157 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.157 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.157 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.157 INFO:teuthology.orchestra.run.smithi089.stdout: "enabled": { 2024-09-21T03:42:23.157 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "enabled", 2024-09-21T03:42:23.157 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-21T03:42:23.157 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.157 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:23.157 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "True", 2024-09-21T03:42:23.157 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.158 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.158 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.158 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.158 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.158 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.158 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.158 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.158 INFO:teuthology.orchestra.run.smithi089.stdout: "log_level": { 2024-09-21T03:42:23.158 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_level", 2024-09-21T03:42:23.158 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:23.158 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.158 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:23.159 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-21T03:42:23.159 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.159 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.159 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-21T03:42:23.159 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2024-09-21T03:42:23.159 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2024-09-21T03:42:23.159 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2024-09-21T03:42:23.159 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2024-09-21T03:42:23.159 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2024-09-21T03:42:23.159 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2024-09-21T03:42:23.159 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-21T03:42:23.159 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.160 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.160 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.160 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.160 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.160 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster": { 2024-09-21T03:42:23.160 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster", 2024-09-21T03:42:23.160 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-21T03:42:23.160 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.160 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:23.160 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-21T03:42:23.160 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.160 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.160 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.161 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.161 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.161 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.161 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.161 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.161 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster_level": { 2024-09-21T03:42:23.161 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster_level", 2024-09-21T03:42:23.161 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:23.161 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.161 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:23.161 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "info", 2024-09-21T03:42:23.161 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.161 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.162 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-21T03:42:23.162 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2024-09-21T03:42:23.162 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2024-09-21T03:42:23.162 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2024-09-21T03:42:23.162 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2024-09-21T03:42:23.162 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2024-09-21T03:42:23.162 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2024-09-21T03:42:23.162 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-21T03:42:23.162 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.162 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.162 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.162 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.162 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.163 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_file": { 2024-09-21T03:42:23.163 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_file", 2024-09-21T03:42:23.163 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-21T03:42:23.163 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.163 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:23.163 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-21T03:42:23.163 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.163 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.163 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.163 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.163 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.163 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.163 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.163 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.164 INFO:teuthology.orchestra.run.smithi089.stdout: "max_completed_events": { 2024-09-21T03:42:23.164 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "max_completed_events", 2024-09-21T03:42:23.164 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2024-09-21T03:42:23.164 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.164 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:23.164 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "50", 2024-09-21T03:42:23.164 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.164 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.164 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.164 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "number of past completed events to remember", 2024-09-21T03:42:23.165 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.165 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.165 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.165 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.165 INFO:teuthology.orchestra.run.smithi089.stdout: "sleep_interval": { 2024-09-21T03:42:23.165 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "sleep_interval", 2024-09-21T03:42:23.165 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "secs", 2024-09-21T03:42:23.165 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.165 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:23.165 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "5", 2024-09-21T03:42:23.166 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.166 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.166 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.166 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "how long the module is going to sleep", 2024-09-21T03:42:23.166 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.166 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.166 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.166 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.166 INFO:teuthology.orchestra.run.smithi089.stdout: "sqlite3_killpoint": { 2024-09-21T03:42:23.166 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:42:23.166 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2024-09-21T03:42:23.167 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "dev", 2024-09-21T03:42:23.167 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:23.167 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "0", 2024-09-21T03:42:23.167 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.167 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.167 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.167 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.167 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.167 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.167 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.167 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-21T03:42:23.167 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-21T03:42:23.167 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.168 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-21T03:42:23.168 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "prometheus", 2024-09-21T03:42:23.168 INFO:teuthology.orchestra.run.smithi089.stdout: "can_run": true, 2024-09-21T03:42:23.168 INFO:teuthology.orchestra.run.smithi089.stdout: "error_string": "", 2024-09-21T03:42:23.168 INFO:teuthology.orchestra.run.smithi089.stdout: "module_options": { 2024-09-21T03:42:23.168 INFO:teuthology.orchestra.run.smithi089.stdout: "cache": { 2024-09-21T03:42:23.168 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "cache", 2024-09-21T03:42:23.168 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-21T03:42:23.168 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.168 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:23.168 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "True", 2024-09-21T03:42:23.168 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.168 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.169 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.169 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.169 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.169 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.169 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.169 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.169 INFO:teuthology.orchestra.run.smithi089.stdout: "exclude_perf_counters": { 2024-09-21T03:42:23.169 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "exclude_perf_counters", 2024-09-21T03:42:23.169 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-21T03:42:23.169 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.169 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:23.169 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "True", 2024-09-21T03:42:23.170 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.170 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.170 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.170 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-09-21T03:42:23.170 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-09-21T03:42:23.170 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.170 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.170 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.170 INFO:teuthology.orchestra.run.smithi089.stdout: "log_level": { 2024-09-21T03:42:23.170 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_level", 2024-09-21T03:42:23.170 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:23.170 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.171 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:23.171 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-21T03:42:23.171 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.171 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.171 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-21T03:42:23.171 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2024-09-21T03:42:23.171 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2024-09-21T03:42:23.171 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2024-09-21T03:42:23.171 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2024-09-21T03:42:23.171 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2024-09-21T03:42:23.171 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2024-09-21T03:42:23.171 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-21T03:42:23.171 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.172 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.172 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.172 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.172 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.172 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster": { 2024-09-21T03:42:23.172 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster", 2024-09-21T03:42:23.172 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-21T03:42:23.172 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.172 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:23.172 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-21T03:42:23.172 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.172 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.172 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.173 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.173 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.173 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.173 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.173 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.173 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster_level": { 2024-09-21T03:42:23.173 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster_level", 2024-09-21T03:42:23.173 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:23.173 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.173 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:23.173 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "info", 2024-09-21T03:42:23.173 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.174 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.174 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-21T03:42:23.174 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2024-09-21T03:42:23.174 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2024-09-21T03:42:23.174 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2024-09-21T03:42:23.174 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2024-09-21T03:42:23.174 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2024-09-21T03:42:23.174 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2024-09-21T03:42:23.174 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-21T03:42:23.174 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.174 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.174 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.175 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.175 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.175 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_file": { 2024-09-21T03:42:23.175 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_file", 2024-09-21T03:42:23.175 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-21T03:42:23.175 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.175 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:23.175 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-21T03:42:23.175 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.175 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.175 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.176 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.176 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.176 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.176 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.176 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.176 INFO:teuthology.orchestra.run.smithi089.stdout: "rbd_stats_pools": { 2024-09-21T03:42:23.176 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "rbd_stats_pools", 2024-09-21T03:42:23.176 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:23.176 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.176 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:23.176 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-21T03:42:23.177 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.177 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.177 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.177 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.177 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.177 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.177 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.177 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.177 INFO:teuthology.orchestra.run.smithi089.stdout: "rbd_stats_pools_refresh_interval": { 2024-09-21T03:42:23.177 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-09-21T03:42:23.177 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2024-09-21T03:42:23.177 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.178 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:23.178 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "300", 2024-09-21T03:42:23.178 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.178 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.178 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.178 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.178 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.178 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.178 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.178 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.178 INFO:teuthology.orchestra.run.smithi089.stdout: "scrape_interval": { 2024-09-21T03:42:23.178 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "scrape_interval", 2024-09-21T03:42:23.179 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "float", 2024-09-21T03:42:23.179 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.179 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:23.179 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "15.0", 2024-09-21T03:42:23.179 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.179 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.179 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.179 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.179 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.179 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.179 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.179 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.179 INFO:teuthology.orchestra.run.smithi089.stdout: "server_addr": { 2024-09-21T03:42:23.180 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "server_addr", 2024-09-21T03:42:23.180 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:23.180 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.180 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:23.180 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "::", 2024-09-21T03:42:23.180 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.180 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.180 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.180 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-09-21T03:42:23.180 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.181 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.181 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.181 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.181 INFO:teuthology.orchestra.run.smithi089.stdout: "server_port": { 2024-09-21T03:42:23.181 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "server_port", 2024-09-21T03:42:23.181 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2024-09-21T03:42:23.181 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.181 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:23.181 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "9283", 2024-09-21T03:42:23.181 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.181 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.181 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.181 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-09-21T03:42:23.182 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.182 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.182 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.182 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.182 INFO:teuthology.orchestra.run.smithi089.stdout: "sqlite3_killpoint": { 2024-09-21T03:42:23.182 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:42:23.182 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2024-09-21T03:42:23.182 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "dev", 2024-09-21T03:42:23.182 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:23.182 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "0", 2024-09-21T03:42:23.182 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.183 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.183 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.183 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.183 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.183 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.183 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.183 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.183 INFO:teuthology.orchestra.run.smithi089.stdout: "stale_cache_strategy": { 2024-09-21T03:42:23.183 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "stale_cache_strategy", 2024-09-21T03:42:23.183 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:23.183 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.183 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:23.184 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "log", 2024-09-21T03:42:23.184 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.184 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.184 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.184 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.184 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.184 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.184 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.184 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.184 INFO:teuthology.orchestra.run.smithi089.stdout: "standby_behaviour": { 2024-09-21T03:42:23.184 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "standby_behaviour", 2024-09-21T03:42:23.184 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:23.184 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.185 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:23.185 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "default", 2024-09-21T03:42:23.185 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.185 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.185 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-21T03:42:23.185 INFO:teuthology.orchestra.run.smithi089.stdout: "default", 2024-09-21T03:42:23.185 INFO:teuthology.orchestra.run.smithi089.stdout: "error" 2024-09-21T03:42:23.185 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-21T03:42:23.185 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.185 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.185 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.185 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.186 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.186 INFO:teuthology.orchestra.run.smithi089.stdout: "standby_error_status_code": { 2024-09-21T03:42:23.186 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "standby_error_status_code", 2024-09-21T03:42:23.186 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2024-09-21T03:42:23.186 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.186 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:23.186 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "500", 2024-09-21T03:42:23.186 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "400", 2024-09-21T03:42:23.186 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "599", 2024-09-21T03:42:23.186 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.187 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.187 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.187 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.187 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.187 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-21T03:42:23.187 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-21T03:42:23.187 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.187 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-21T03:42:23.187 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "rbd_support", 2024-09-21T03:42:23.187 INFO:teuthology.orchestra.run.smithi089.stdout: "can_run": true, 2024-09-21T03:42:23.187 INFO:teuthology.orchestra.run.smithi089.stdout: "error_string": "", 2024-09-21T03:42:23.188 INFO:teuthology.orchestra.run.smithi089.stdout: "module_options": { 2024-09-21T03:42:23.188 INFO:teuthology.orchestra.run.smithi089.stdout: "log_level": { 2024-09-21T03:42:23.188 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_level", 2024-09-21T03:42:23.188 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:23.188 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.188 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:23.188 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-21T03:42:23.188 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.188 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.188 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-21T03:42:23.188 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2024-09-21T03:42:23.188 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2024-09-21T03:42:23.188 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2024-09-21T03:42:23.189 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2024-09-21T03:42:23.189 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2024-09-21T03:42:23.189 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2024-09-21T03:42:23.189 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-21T03:42:23.189 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.189 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.189 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.189 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.189 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.189 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster": { 2024-09-21T03:42:23.189 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster", 2024-09-21T03:42:23.190 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-21T03:42:23.190 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.190 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:23.190 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-21T03:42:23.190 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.190 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.190 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.190 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.190 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.190 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.190 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.190 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.190 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster_level": { 2024-09-21T03:42:23.191 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster_level", 2024-09-21T03:42:23.191 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:23.191 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.191 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:23.191 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "info", 2024-09-21T03:42:23.191 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.191 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.191 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-21T03:42:23.191 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2024-09-21T03:42:23.191 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2024-09-21T03:42:23.191 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2024-09-21T03:42:23.191 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2024-09-21T03:42:23.191 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2024-09-21T03:42:23.192 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2024-09-21T03:42:23.192 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-21T03:42:23.192 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.192 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.192 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.192 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.192 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.192 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_file": { 2024-09-21T03:42:23.192 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_file", 2024-09-21T03:42:23.192 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-21T03:42:23.192 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.192 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:23.192 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-21T03:42:23.193 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.193 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.193 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.193 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.193 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.193 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.193 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.193 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.193 INFO:teuthology.orchestra.run.smithi089.stdout: "max_concurrent_snap_create": { 2024-09-21T03:42:23.193 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "max_concurrent_snap_create", 2024-09-21T03:42:23.193 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2024-09-21T03:42:23.193 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.193 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:23.194 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "10", 2024-09-21T03:42:23.194 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.194 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.194 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.194 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.194 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.194 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.194 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.194 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.194 INFO:teuthology.orchestra.run.smithi089.stdout: "mirror_snapshot_schedule": { 2024-09-21T03:42:23.194 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "mirror_snapshot_schedule", 2024-09-21T03:42:23.194 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:23.194 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.195 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:23.195 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-21T03:42:23.195 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.195 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.195 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.195 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.195 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.195 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.195 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.195 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.195 INFO:teuthology.orchestra.run.smithi089.stdout: "sqlite3_killpoint": { 2024-09-21T03:42:23.196 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:42:23.196 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2024-09-21T03:42:23.196 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "dev", 2024-09-21T03:42:23.196 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:23.196 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "0", 2024-09-21T03:42:23.196 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.196 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.196 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.196 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.196 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.196 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.197 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.197 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.197 INFO:teuthology.orchestra.run.smithi089.stdout: "trash_purge_schedule": { 2024-09-21T03:42:23.197 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "trash_purge_schedule", 2024-09-21T03:42:23.197 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:23.197 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.197 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:23.197 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-21T03:42:23.197 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.197 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.197 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.197 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.197 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.198 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.198 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.198 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-21T03:42:23.198 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-21T03:42:23.198 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.198 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-21T03:42:23.198 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "restful", 2024-09-21T03:42:23.198 INFO:teuthology.orchestra.run.smithi089.stdout: "can_run": true, 2024-09-21T03:42:23.198 INFO:teuthology.orchestra.run.smithi089.stdout: "error_string": "", 2024-09-21T03:42:23.198 INFO:teuthology.orchestra.run.smithi089.stdout: "module_options": { 2024-09-21T03:42:23.198 INFO:teuthology.orchestra.run.smithi089.stdout: "enable_auth": { 2024-09-21T03:42:23.198 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "enable_auth", 2024-09-21T03:42:23.198 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-21T03:42:23.198 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.199 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:23.199 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "True", 2024-09-21T03:42:23.199 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.199 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.199 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.199 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.199 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.199 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.199 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.199 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.199 INFO:teuthology.orchestra.run.smithi089.stdout: "key_file": { 2024-09-21T03:42:23.199 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "key_file", 2024-09-21T03:42:23.199 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:23.199 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.200 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:23.200 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-21T03:42:23.200 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.200 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.200 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.200 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.200 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.200 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.200 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.200 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.200 INFO:teuthology.orchestra.run.smithi089.stdout: "log_level": { 2024-09-21T03:42:23.200 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_level", 2024-09-21T03:42:23.201 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:23.201 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.201 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:23.201 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-21T03:42:23.201 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.201 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.201 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-21T03:42:23.201 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2024-09-21T03:42:23.201 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2024-09-21T03:42:23.201 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2024-09-21T03:42:23.201 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2024-09-21T03:42:23.201 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2024-09-21T03:42:23.201 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2024-09-21T03:42:23.201 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-21T03:42:23.202 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.202 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.202 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.202 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.202 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.202 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster": { 2024-09-21T03:42:23.202 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster", 2024-09-21T03:42:23.202 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-21T03:42:23.202 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.202 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:23.202 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-21T03:42:23.202 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.203 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.203 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.203 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.203 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.203 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.203 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.203 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.203 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster_level": { 2024-09-21T03:42:23.203 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster_level", 2024-09-21T03:42:23.203 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:23.203 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.203 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:23.203 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "info", 2024-09-21T03:42:23.204 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.204 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.204 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-21T03:42:23.204 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2024-09-21T03:42:23.204 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2024-09-21T03:42:23.204 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2024-09-21T03:42:23.204 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2024-09-21T03:42:23.204 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2024-09-21T03:42:23.204 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2024-09-21T03:42:23.204 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-21T03:42:23.204 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.204 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.204 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.205 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.205 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.205 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_file": { 2024-09-21T03:42:23.205 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_file", 2024-09-21T03:42:23.205 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-21T03:42:23.205 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.205 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:23.205 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-21T03:42:23.205 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.205 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.205 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.206 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.206 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.206 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.206 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.206 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.206 INFO:teuthology.orchestra.run.smithi089.stdout: "max_requests": { 2024-09-21T03:42:23.206 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "max_requests", 2024-09-21T03:42:23.206 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2024-09-21T03:42:23.206 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.206 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:23.206 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "500", 2024-09-21T03:42:23.206 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.206 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.207 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.207 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Maximum number of requests to keep in memory. When new request comes in, the oldest request will be removed if the number of requests exceeds the max request number.if un-finished request is removed, error message will be logged in the ceph-mgr log.", 2024-09-21T03:42:23.207 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.207 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.207 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.207 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.207 INFO:teuthology.orchestra.run.smithi089.stdout: "server_addr": { 2024-09-21T03:42:23.207 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "server_addr", 2024-09-21T03:42:23.207 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:23.207 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.207 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:23.207 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-21T03:42:23.207 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.208 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.208 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.208 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.208 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.208 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.208 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.208 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.208 INFO:teuthology.orchestra.run.smithi089.stdout: "server_port": { 2024-09-21T03:42:23.208 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "server_port", 2024-09-21T03:42:23.208 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:23.208 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.208 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:23.208 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-21T03:42:23.209 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.209 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.209 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.209 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.209 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.209 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.209 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.209 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.209 INFO:teuthology.orchestra.run.smithi089.stdout: "sqlite3_killpoint": { 2024-09-21T03:42:23.209 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:42:23.209 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2024-09-21T03:42:23.209 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "dev", 2024-09-21T03:42:23.209 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:23.210 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "0", 2024-09-21T03:42:23.210 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.210 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.210 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.210 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.210 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.210 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.210 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.210 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-21T03:42:23.210 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-21T03:42:23.210 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.210 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-21T03:42:23.211 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "rgw", 2024-09-21T03:42:23.211 INFO:teuthology.orchestra.run.smithi089.stdout: "can_run": true, 2024-09-21T03:42:23.211 INFO:teuthology.orchestra.run.smithi089.stdout: "error_string": "", 2024-09-21T03:42:23.211 INFO:teuthology.orchestra.run.smithi089.stdout: "module_options": { 2024-09-21T03:42:23.211 INFO:teuthology.orchestra.run.smithi089.stdout: "log_level": { 2024-09-21T03:42:23.211 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_level", 2024-09-21T03:42:23.211 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:23.211 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.211 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:23.211 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-21T03:42:23.211 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.211 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.211 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-21T03:42:23.212 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2024-09-21T03:42:23.212 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2024-09-21T03:42:23.212 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2024-09-21T03:42:23.212 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2024-09-21T03:42:23.212 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2024-09-21T03:42:23.212 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2024-09-21T03:42:23.212 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-21T03:42:23.212 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.212 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.212 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.212 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.212 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.212 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster": { 2024-09-21T03:42:23.213 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster", 2024-09-21T03:42:23.213 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-21T03:42:23.213 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.213 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:23.213 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-21T03:42:23.213 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.213 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.213 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.213 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.213 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.213 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.214 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.214 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.214 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster_level": { 2024-09-21T03:42:23.214 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster_level", 2024-09-21T03:42:23.214 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:23.214 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.214 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:23.214 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "info", 2024-09-21T03:42:23.214 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.214 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.214 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-21T03:42:23.215 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2024-09-21T03:42:23.215 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2024-09-21T03:42:23.215 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2024-09-21T03:42:23.215 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2024-09-21T03:42:23.215 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2024-09-21T03:42:23.215 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2024-09-21T03:42:23.215 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-21T03:42:23.215 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.215 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.215 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.215 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.216 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.216 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_file": { 2024-09-21T03:42:23.216 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_file", 2024-09-21T03:42:23.216 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-21T03:42:23.216 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.216 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:23.216 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-21T03:42:23.216 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.216 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.216 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.216 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.216 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.217 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.217 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.217 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.217 INFO:teuthology.orchestra.run.smithi089.stdout: "secondary_zone_period_retry_limit": { 2024-09-21T03:42:23.217 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "secondary_zone_period_retry_limit", 2024-09-21T03:42:23.217 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2024-09-21T03:42:23.217 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.217 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:23.217 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "5", 2024-09-21T03:42:23.217 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.217 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.218 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.218 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "RGW module period update retry limit for secondary site", 2024-09-21T03:42:23.218 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.218 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.218 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.218 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.218 INFO:teuthology.orchestra.run.smithi089.stdout: "sqlite3_killpoint": { 2024-09-21T03:42:23.218 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:42:23.218 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2024-09-21T03:42:23.218 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "dev", 2024-09-21T03:42:23.218 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:23.218 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "0", 2024-09-21T03:42:23.219 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.219 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.219 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.219 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.219 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.219 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.219 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.219 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-21T03:42:23.219 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-21T03:42:23.219 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.219 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-21T03:42:23.219 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "rook", 2024-09-21T03:42:23.219 INFO:teuthology.orchestra.run.smithi089.stdout: "can_run": true, 2024-09-21T03:42:23.220 INFO:teuthology.orchestra.run.smithi089.stdout: "error_string": "", 2024-09-21T03:42:23.220 INFO:teuthology.orchestra.run.smithi089.stdout: "module_options": { 2024-09-21T03:42:23.220 INFO:teuthology.orchestra.run.smithi089.stdout: "log_level": { 2024-09-21T03:42:23.220 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_level", 2024-09-21T03:42:23.220 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:23.220 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.220 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:23.220 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-21T03:42:23.220 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.220 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.220 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-21T03:42:23.220 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2024-09-21T03:42:23.220 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2024-09-21T03:42:23.221 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2024-09-21T03:42:23.221 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2024-09-21T03:42:23.221 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2024-09-21T03:42:23.221 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2024-09-21T03:42:23.221 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-21T03:42:23.221 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.221 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.221 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.221 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.221 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.221 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster": { 2024-09-21T03:42:23.221 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster", 2024-09-21T03:42:23.222 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-21T03:42:23.222 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.222 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:23.222 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-21T03:42:23.222 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.222 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.222 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.222 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.222 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.222 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.222 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.222 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.223 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster_level": { 2024-09-21T03:42:23.223 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster_level", 2024-09-21T03:42:23.223 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:23.223 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.223 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:23.223 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "info", 2024-09-21T03:42:23.223 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.223 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.223 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-21T03:42:23.223 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2024-09-21T03:42:23.223 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2024-09-21T03:42:23.223 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2024-09-21T03:42:23.223 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2024-09-21T03:42:23.224 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2024-09-21T03:42:23.224 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2024-09-21T03:42:23.224 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-21T03:42:23.224 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.224 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.224 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.224 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.224 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.224 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_file": { 2024-09-21T03:42:23.224 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_file", 2024-09-21T03:42:23.224 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-21T03:42:23.224 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.225 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:23.225 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-21T03:42:23.225 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.225 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.225 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.225 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.225 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.225 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.225 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.225 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.225 INFO:teuthology.orchestra.run.smithi089.stdout: "sqlite3_killpoint": { 2024-09-21T03:42:23.225 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:42:23.226 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2024-09-21T03:42:23.226 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "dev", 2024-09-21T03:42:23.226 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:23.226 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "0", 2024-09-21T03:42:23.226 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.226 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.226 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.226 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.226 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.226 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.226 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.227 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.227 INFO:teuthology.orchestra.run.smithi089.stdout: "storage_class": { 2024-09-21T03:42:23.227 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "storage_class", 2024-09-21T03:42:23.227 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:23.227 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.227 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:23.227 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "local", 2024-09-21T03:42:23.227 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.227 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.227 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.227 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-09-21T03:42:23.227 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.228 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.228 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.228 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-21T03:42:23.228 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-21T03:42:23.228 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.228 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-21T03:42:23.228 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "selftest", 2024-09-21T03:42:23.228 INFO:teuthology.orchestra.run.smithi089.stdout: "can_run": true, 2024-09-21T03:42:23.228 INFO:teuthology.orchestra.run.smithi089.stdout: "error_string": "", 2024-09-21T03:42:23.228 INFO:teuthology.orchestra.run.smithi089.stdout: "module_options": { 2024-09-21T03:42:23.228 INFO:teuthology.orchestra.run.smithi089.stdout: "log_level": { 2024-09-21T03:42:23.228 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_level", 2024-09-21T03:42:23.229 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:23.229 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.229 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:23.229 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-21T03:42:23.229 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.229 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.229 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-21T03:42:23.229 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2024-09-21T03:42:23.229 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2024-09-21T03:42:23.229 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2024-09-21T03:42:23.229 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2024-09-21T03:42:23.229 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2024-09-21T03:42:23.229 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2024-09-21T03:42:23.230 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-21T03:42:23.230 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.230 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.230 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.230 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.230 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.230 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster": { 2024-09-21T03:42:23.230 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster", 2024-09-21T03:42:23.230 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-21T03:42:23.230 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.230 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:23.231 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-21T03:42:23.231 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.231 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.231 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.231 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.231 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.231 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.231 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.231 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.231 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster_level": { 2024-09-21T03:42:23.231 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster_level", 2024-09-21T03:42:23.231 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:23.231 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.232 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:23.232 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "info", 2024-09-21T03:42:23.232 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.232 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.232 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-21T03:42:23.232 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2024-09-21T03:42:23.232 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2024-09-21T03:42:23.232 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2024-09-21T03:42:23.232 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2024-09-21T03:42:23.232 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2024-09-21T03:42:23.232 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2024-09-21T03:42:23.233 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-21T03:42:23.233 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.233 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.233 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.233 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.233 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.233 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_file": { 2024-09-21T03:42:23.233 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_file", 2024-09-21T03:42:23.233 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-21T03:42:23.233 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.233 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:23.234 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-21T03:42:23.234 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.234 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.234 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.234 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.234 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.234 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.234 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.234 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.234 INFO:teuthology.orchestra.run.smithi089.stdout: "roption1": { 2024-09-21T03:42:23.234 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "roption1", 2024-09-21T03:42:23.234 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:23.235 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.235 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:23.235 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-21T03:42:23.235 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.235 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.235 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.235 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.235 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.235 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.235 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.235 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.235 INFO:teuthology.orchestra.run.smithi089.stdout: "roption2": { 2024-09-21T03:42:23.236 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "roption2", 2024-09-21T03:42:23.236 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:23.236 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.236 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:23.236 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "xyz", 2024-09-21T03:42:23.236 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.236 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.236 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.236 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.236 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.236 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.237 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.237 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.237 INFO:teuthology.orchestra.run.smithi089.stdout: "rwoption1": { 2024-09-21T03:42:23.237 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "rwoption1", 2024-09-21T03:42:23.237 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:23.237 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.237 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:23.237 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-21T03:42:23.237 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.237 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.237 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.237 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.238 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.238 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.238 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.238 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.238 INFO:teuthology.orchestra.run.smithi089.stdout: "rwoption2": { 2024-09-21T03:42:23.238 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "rwoption2", 2024-09-21T03:42:23.238 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2024-09-21T03:42:23.238 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.238 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:23.238 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-21T03:42:23.238 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.239 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.239 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.239 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.239 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.239 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.239 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.239 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.239 INFO:teuthology.orchestra.run.smithi089.stdout: "rwoption3": { 2024-09-21T03:42:23.239 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "rwoption3", 2024-09-21T03:42:23.239 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "float", 2024-09-21T03:42:23.239 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.239 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:23.240 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-21T03:42:23.240 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.240 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.240 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.240 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.240 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.240 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.240 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.240 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.240 INFO:teuthology.orchestra.run.smithi089.stdout: "rwoption4": { 2024-09-21T03:42:23.240 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "rwoption4", 2024-09-21T03:42:23.240 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:23.241 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.241 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:23.241 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-21T03:42:23.241 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.241 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.241 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.241 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.241 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.241 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.241 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.241 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.241 INFO:teuthology.orchestra.run.smithi089.stdout: "rwoption5": { 2024-09-21T03:42:23.241 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "rwoption5", 2024-09-21T03:42:23.242 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-21T03:42:23.242 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.242 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:23.242 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-21T03:42:23.242 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.242 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.242 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.242 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.242 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.242 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.242 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.242 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.243 INFO:teuthology.orchestra.run.smithi089.stdout: "rwoption6": { 2024-09-21T03:42:23.243 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "rwoption6", 2024-09-21T03:42:23.243 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-21T03:42:23.243 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.243 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:23.243 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "True", 2024-09-21T03:42:23.243 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.243 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.243 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.243 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.243 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.243 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.243 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.244 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.244 INFO:teuthology.orchestra.run.smithi089.stdout: "rwoption7": { 2024-09-21T03:42:23.244 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "rwoption7", 2024-09-21T03:42:23.244 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2024-09-21T03:42:23.244 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.244 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:23.244 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-21T03:42:23.244 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "1", 2024-09-21T03:42:23.244 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "42", 2024-09-21T03:42:23.244 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.244 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.244 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.244 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.245 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.245 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.245 INFO:teuthology.orchestra.run.smithi089.stdout: "sqlite3_killpoint": { 2024-09-21T03:42:23.245 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:42:23.245 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2024-09-21T03:42:23.245 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "dev", 2024-09-21T03:42:23.245 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:23.245 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "0", 2024-09-21T03:42:23.245 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.245 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.245 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.245 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.245 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.246 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.246 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.246 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.246 INFO:teuthology.orchestra.run.smithi089.stdout: "testkey": { 2024-09-21T03:42:23.246 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "testkey", 2024-09-21T03:42:23.246 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:23.246 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.246 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:23.246 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-21T03:42:23.246 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.246 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.246 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.246 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.247 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.247 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.247 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.247 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.247 INFO:teuthology.orchestra.run.smithi089.stdout: "testlkey": { 2024-09-21T03:42:23.247 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "testlkey", 2024-09-21T03:42:23.247 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:23.247 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.247 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:23.247 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-21T03:42:23.247 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.247 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.247 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.248 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.248 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.248 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.248 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.248 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.248 INFO:teuthology.orchestra.run.smithi089.stdout: "testnewline": { 2024-09-21T03:42:23.248 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "testnewline", 2024-09-21T03:42:23.248 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:23.248 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.248 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:23.248 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-21T03:42:23.248 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.249 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.249 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.249 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.249 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.249 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.249 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.249 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-21T03:42:23.249 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-21T03:42:23.249 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.249 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-21T03:42:23.249 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "smb", 2024-09-21T03:42:23.249 INFO:teuthology.orchestra.run.smithi089.stdout: "can_run": true, 2024-09-21T03:42:23.249 INFO:teuthology.orchestra.run.smithi089.stdout: "error_string": "", 2024-09-21T03:42:23.250 INFO:teuthology.orchestra.run.smithi089.stdout: "module_options": { 2024-09-21T03:42:23.250 INFO:teuthology.orchestra.run.smithi089.stdout: "internal_store_backend": { 2024-09-21T03:42:23.250 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "internal_store_backend", 2024-09-21T03:42:23.250 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:23.250 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "dev", 2024-09-21T03:42:23.250 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:23.250 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-21T03:42:23.250 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.250 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.250 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.250 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "set internal store backend. for develoment and testing only", 2024-09-21T03:42:23.250 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.250 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.251 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.251 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.251 INFO:teuthology.orchestra.run.smithi089.stdout: "log_level": { 2024-09-21T03:42:23.251 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_level", 2024-09-21T03:42:23.251 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:23.251 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.251 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:23.251 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-21T03:42:23.251 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.251 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.251 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-21T03:42:23.251 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2024-09-21T03:42:23.251 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2024-09-21T03:42:23.252 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2024-09-21T03:42:23.252 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2024-09-21T03:42:23.252 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2024-09-21T03:42:23.252 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2024-09-21T03:42:23.252 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-21T03:42:23.252 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.252 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.252 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.252 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.252 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.252 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster": { 2024-09-21T03:42:23.252 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster", 2024-09-21T03:42:23.253 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-21T03:42:23.253 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.253 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:23.253 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-21T03:42:23.253 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.253 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.253 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.253 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.253 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.253 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.253 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.253 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.253 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster_level": { 2024-09-21T03:42:23.254 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster_level", 2024-09-21T03:42:23.254 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:23.254 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.254 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:23.254 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "info", 2024-09-21T03:42:23.254 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.254 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.254 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-21T03:42:23.254 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2024-09-21T03:42:23.254 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2024-09-21T03:42:23.254 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2024-09-21T03:42:23.254 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2024-09-21T03:42:23.255 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2024-09-21T03:42:23.255 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2024-09-21T03:42:23.255 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-21T03:42:23.255 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.255 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.255 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.255 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.255 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.255 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_file": { 2024-09-21T03:42:23.255 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_file", 2024-09-21T03:42:23.255 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-21T03:42:23.255 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.255 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:23.256 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-21T03:42:23.256 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.256 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.256 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.256 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.256 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.256 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.256 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.256 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.256 INFO:teuthology.orchestra.run.smithi089.stdout: "sqlite3_killpoint": { 2024-09-21T03:42:23.256 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:42:23.256 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2024-09-21T03:42:23.256 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "dev", 2024-09-21T03:42:23.256 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:23.257 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "0", 2024-09-21T03:42:23.257 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.257 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.257 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.257 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.257 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.257 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.257 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.257 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.257 INFO:teuthology.orchestra.run.smithi089.stdout: "update_orchestration": { 2024-09-21T03:42:23.257 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "update_orchestration", 2024-09-21T03:42:23.257 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-21T03:42:23.257 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.257 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:23.258 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "True", 2024-09-21T03:42:23.258 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.258 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.258 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.258 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "automatically update orchestration when smb resources are changed", 2024-09-21T03:42:23.258 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.258 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.258 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.258 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-21T03:42:23.258 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-21T03:42:23.258 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.259 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-21T03:42:23.259 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "snap_schedule", 2024-09-21T03:42:23.259 INFO:teuthology.orchestra.run.smithi089.stdout: "can_run": true, 2024-09-21T03:42:23.259 INFO:teuthology.orchestra.run.smithi089.stdout: "error_string": "", 2024-09-21T03:42:23.259 INFO:teuthology.orchestra.run.smithi089.stdout: "module_options": { 2024-09-21T03:42:23.259 INFO:teuthology.orchestra.run.smithi089.stdout: "allow_m_granularity": { 2024-09-21T03:42:23.259 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "allow_m_granularity", 2024-09-21T03:42:23.259 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-21T03:42:23.259 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.259 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:23.259 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-21T03:42:23.259 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.259 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.260 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.260 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "allow minute scheduled snapshots", 2024-09-21T03:42:23.260 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.260 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.260 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.260 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.260 INFO:teuthology.orchestra.run.smithi089.stdout: "dump_on_update": { 2024-09-21T03:42:23.260 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "dump_on_update", 2024-09-21T03:42:23.260 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-21T03:42:23.260 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.260 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:23.260 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-21T03:42:23.260 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.261 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.261 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.261 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "dump database to debug log on update", 2024-09-21T03:42:23.261 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.261 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.261 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.261 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.261 INFO:teuthology.orchestra.run.smithi089.stdout: "log_level": { 2024-09-21T03:42:23.261 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_level", 2024-09-21T03:42:23.261 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:23.261 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.261 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:23.261 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-21T03:42:23.261 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.262 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.262 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-21T03:42:23.262 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2024-09-21T03:42:23.262 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2024-09-21T03:42:23.262 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2024-09-21T03:42:23.262 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2024-09-21T03:42:23.262 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2024-09-21T03:42:23.262 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2024-09-21T03:42:23.262 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-21T03:42:23.262 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.262 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.263 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.263 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.263 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.263 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster": { 2024-09-21T03:42:23.263 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster", 2024-09-21T03:42:23.263 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-21T03:42:23.263 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.263 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:23.263 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-21T03:42:23.263 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.263 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.263 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.264 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.264 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.264 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.264 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.264 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.264 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster_level": { 2024-09-21T03:42:23.264 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster_level", 2024-09-21T03:42:23.264 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:23.264 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.264 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:23.264 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "info", 2024-09-21T03:42:23.264 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.264 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.265 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-21T03:42:23.265 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2024-09-21T03:42:23.265 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2024-09-21T03:42:23.265 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2024-09-21T03:42:23.265 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2024-09-21T03:42:23.265 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2024-09-21T03:42:23.265 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2024-09-21T03:42:23.265 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-21T03:42:23.265 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.265 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.265 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.265 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.265 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.266 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_file": { 2024-09-21T03:42:23.266 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_file", 2024-09-21T03:42:23.266 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-21T03:42:23.266 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.266 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:23.266 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-21T03:42:23.266 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.266 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.266 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.266 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.266 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.266 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.266 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.267 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.267 INFO:teuthology.orchestra.run.smithi089.stdout: "sqlite3_killpoint": { 2024-09-21T03:42:23.267 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:42:23.267 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2024-09-21T03:42:23.267 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "dev", 2024-09-21T03:42:23.267 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:23.267 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "0", 2024-09-21T03:42:23.267 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.267 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.267 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.267 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.267 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.267 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.268 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.268 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-21T03:42:23.268 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-21T03:42:23.268 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.268 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-21T03:42:23.268 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "stats", 2024-09-21T03:42:23.268 INFO:teuthology.orchestra.run.smithi089.stdout: "can_run": true, 2024-09-21T03:42:23.268 INFO:teuthology.orchestra.run.smithi089.stdout: "error_string": "", 2024-09-21T03:42:23.268 INFO:teuthology.orchestra.run.smithi089.stdout: "module_options": { 2024-09-21T03:42:23.268 INFO:teuthology.orchestra.run.smithi089.stdout: "log_level": { 2024-09-21T03:42:23.268 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_level", 2024-09-21T03:42:23.268 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:23.268 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.268 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:23.269 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-21T03:42:23.269 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.269 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.269 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-21T03:42:23.269 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2024-09-21T03:42:23.269 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2024-09-21T03:42:23.269 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2024-09-21T03:42:23.269 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2024-09-21T03:42:23.269 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2024-09-21T03:42:23.269 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2024-09-21T03:42:23.269 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-21T03:42:23.269 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.269 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.269 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.270 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.270 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.270 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster": { 2024-09-21T03:42:23.270 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster", 2024-09-21T03:42:23.270 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-21T03:42:23.270 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.270 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:23.270 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-21T03:42:23.270 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.270 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.270 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.270 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.270 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.270 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.271 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.271 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.271 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster_level": { 2024-09-21T03:42:23.271 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster_level", 2024-09-21T03:42:23.271 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:23.271 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.271 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:23.271 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "info", 2024-09-21T03:42:23.271 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.271 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.271 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-21T03:42:23.272 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2024-09-21T03:42:23.272 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2024-09-21T03:42:23.272 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2024-09-21T03:42:23.272 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2024-09-21T03:42:23.272 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2024-09-21T03:42:23.272 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2024-09-21T03:42:23.272 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-21T03:42:23.272 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.272 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.272 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.272 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.273 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.273 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_file": { 2024-09-21T03:42:23.273 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_file", 2024-09-21T03:42:23.273 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-21T03:42:23.273 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.273 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:23.273 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-21T03:42:23.273 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.273 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.273 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.273 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.273 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.273 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.273 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.274 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.274 INFO:teuthology.orchestra.run.smithi089.stdout: "sqlite3_killpoint": { 2024-09-21T03:42:23.274 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:42:23.274 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2024-09-21T03:42:23.274 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "dev", 2024-09-21T03:42:23.274 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:23.274 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "0", 2024-09-21T03:42:23.274 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.274 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.274 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.274 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.274 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.274 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.275 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.275 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-21T03:42:23.275 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-21T03:42:23.275 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.275 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-21T03:42:23.275 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "status", 2024-09-21T03:42:23.275 INFO:teuthology.orchestra.run.smithi089.stdout: "can_run": true, 2024-09-21T03:42:23.275 INFO:teuthology.orchestra.run.smithi089.stdout: "error_string": "", 2024-09-21T03:42:23.275 INFO:teuthology.orchestra.run.smithi089.stdout: "module_options": { 2024-09-21T03:42:23.275 INFO:teuthology.orchestra.run.smithi089.stdout: "log_level": { 2024-09-21T03:42:23.275 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_level", 2024-09-21T03:42:23.275 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:23.276 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.276 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:23.276 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-21T03:42:23.276 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.276 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.276 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-21T03:42:23.276 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2024-09-21T03:42:23.276 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2024-09-21T03:42:23.276 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2024-09-21T03:42:23.276 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2024-09-21T03:42:23.276 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2024-09-21T03:42:23.277 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2024-09-21T03:42:23.277 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-21T03:42:23.277 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.277 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.277 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.277 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.277 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.277 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster": { 2024-09-21T03:42:23.277 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster", 2024-09-21T03:42:23.277 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-21T03:42:23.277 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.278 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:23.278 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-21T03:42:23.278 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.278 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.278 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.278 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.278 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.278 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.278 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.278 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.278 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster_level": { 2024-09-21T03:42:23.279 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster_level", 2024-09-21T03:42:23.279 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:23.279 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.279 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:23.279 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "info", 2024-09-21T03:42:23.279 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.279 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.279 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-21T03:42:23.279 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2024-09-21T03:42:23.279 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2024-09-21T03:42:23.279 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2024-09-21T03:42:23.279 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2024-09-21T03:42:23.280 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2024-09-21T03:42:23.280 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2024-09-21T03:42:23.280 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-21T03:42:23.280 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.280 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.280 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.280 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.280 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.280 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_file": { 2024-09-21T03:42:23.280 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_file", 2024-09-21T03:42:23.280 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-21T03:42:23.280 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.281 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:23.281 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-21T03:42:23.281 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.281 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.281 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.281 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.281 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.281 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.281 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.281 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.281 INFO:teuthology.orchestra.run.smithi089.stdout: "sqlite3_killpoint": { 2024-09-21T03:42:23.281 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:42:23.281 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2024-09-21T03:42:23.282 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "dev", 2024-09-21T03:42:23.282 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:23.282 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "0", 2024-09-21T03:42:23.282 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.282 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.282 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.282 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.282 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.282 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.282 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.282 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-21T03:42:23.282 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-21T03:42:23.283 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.283 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-21T03:42:23.283 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "telegraf", 2024-09-21T03:42:23.283 INFO:teuthology.orchestra.run.smithi089.stdout: "can_run": true, 2024-09-21T03:42:23.283 INFO:teuthology.orchestra.run.smithi089.stdout: "error_string": "", 2024-09-21T03:42:23.283 INFO:teuthology.orchestra.run.smithi089.stdout: "module_options": { 2024-09-21T03:42:23.283 INFO:teuthology.orchestra.run.smithi089.stdout: "address": { 2024-09-21T03:42:23.283 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "address", 2024-09-21T03:42:23.283 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:23.283 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.283 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:23.283 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-09-21T03:42:23.283 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.284 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.284 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.284 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.284 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.284 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.284 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.284 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.284 INFO:teuthology.orchestra.run.smithi089.stdout: "interval": { 2024-09-21T03:42:23.284 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "interval", 2024-09-21T03:42:23.284 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "secs", 2024-09-21T03:42:23.284 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.284 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:23.285 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "15", 2024-09-21T03:42:23.285 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.285 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.285 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.285 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.285 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.285 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.285 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.285 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.285 INFO:teuthology.orchestra.run.smithi089.stdout: "log_level": { 2024-09-21T03:42:23.285 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_level", 2024-09-21T03:42:23.285 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:23.286 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.286 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:23.286 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-21T03:42:23.286 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.286 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.286 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-21T03:42:23.286 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2024-09-21T03:42:23.286 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2024-09-21T03:42:23.286 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2024-09-21T03:42:23.286 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2024-09-21T03:42:23.286 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2024-09-21T03:42:23.286 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2024-09-21T03:42:23.287 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-21T03:42:23.287 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.287 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.287 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.287 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.287 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.287 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster": { 2024-09-21T03:42:23.287 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster", 2024-09-21T03:42:23.287 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-21T03:42:23.287 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.287 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:23.287 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-21T03:42:23.287 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.288 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.288 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.288 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.288 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.288 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.288 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.288 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.288 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster_level": { 2024-09-21T03:42:23.288 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster_level", 2024-09-21T03:42:23.288 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:23.288 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.288 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:23.288 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "info", 2024-09-21T03:42:23.289 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.289 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.289 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-21T03:42:23.289 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2024-09-21T03:42:23.289 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2024-09-21T03:42:23.289 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2024-09-21T03:42:23.289 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2024-09-21T03:42:23.289 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2024-09-21T03:42:23.289 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2024-09-21T03:42:23.289 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-21T03:42:23.289 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.290 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.290 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.290 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.290 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.290 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_file": { 2024-09-21T03:42:23.290 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_file", 2024-09-21T03:42:23.290 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-21T03:42:23.290 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.290 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:23.290 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-21T03:42:23.291 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.291 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.291 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.291 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.291 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.291 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.291 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.291 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.291 INFO:teuthology.orchestra.run.smithi089.stdout: "sqlite3_killpoint": { 2024-09-21T03:42:23.291 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:42:23.291 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2024-09-21T03:42:23.291 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "dev", 2024-09-21T03:42:23.291 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:23.292 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "0", 2024-09-21T03:42:23.292 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.292 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.292 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.292 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.292 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.292 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.292 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.292 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-21T03:42:23.292 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-21T03:42:23.292 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.293 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-21T03:42:23.293 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "telemetry", 2024-09-21T03:42:23.293 INFO:teuthology.orchestra.run.smithi089.stdout: "can_run": true, 2024-09-21T03:42:23.293 INFO:teuthology.orchestra.run.smithi089.stdout: "error_string": "", 2024-09-21T03:42:23.293 INFO:teuthology.orchestra.run.smithi089.stdout: "module_options": { 2024-09-21T03:42:23.293 INFO:teuthology.orchestra.run.smithi089.stdout: "channel_basic": { 2024-09-21T03:42:23.293 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "channel_basic", 2024-09-21T03:42:23.293 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-21T03:42:23.293 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.293 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:23.293 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "True", 2024-09-21T03:42:23.293 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.293 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.294 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.294 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Share basic cluster information (size, version)", 2024-09-21T03:42:23.294 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.294 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.294 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.294 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.294 INFO:teuthology.orchestra.run.smithi089.stdout: "channel_crash": { 2024-09-21T03:42:23.294 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "channel_crash", 2024-09-21T03:42:23.294 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-21T03:42:23.294 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.294 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:23.294 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "True", 2024-09-21T03:42:23.295 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.295 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.295 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.295 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-09-21T03:42:23.295 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.295 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.295 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.295 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.295 INFO:teuthology.orchestra.run.smithi089.stdout: "channel_device": { 2024-09-21T03:42:23.295 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "channel_device", 2024-09-21T03:42:23.295 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-21T03:42:23.296 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.296 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:23.296 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "True", 2024-09-21T03:42:23.296 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.296 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.296 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.296 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-09-21T03:42:23.296 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.296 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.296 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.296 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.296 INFO:teuthology.orchestra.run.smithi089.stdout: "channel_ident": { 2024-09-21T03:42:23.296 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "channel_ident", 2024-09-21T03:42:23.297 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-21T03:42:23.297 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.297 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:23.297 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-21T03:42:23.297 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.297 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.297 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.297 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-09-21T03:42:23.297 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.297 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.297 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.297 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.297 INFO:teuthology.orchestra.run.smithi089.stdout: "channel_perf": { 2024-09-21T03:42:23.298 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "channel_perf", 2024-09-21T03:42:23.298 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-21T03:42:23.298 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.298 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:23.298 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-21T03:42:23.298 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.298 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.298 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.298 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Share various performance metrics of a cluster", 2024-09-21T03:42:23.298 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.298 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.298 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.298 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.299 INFO:teuthology.orchestra.run.smithi089.stdout: "contact": { 2024-09-21T03:42:23.299 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "contact", 2024-09-21T03:42:23.299 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:23.299 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.299 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:23.299 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-21T03:42:23.299 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.299 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.299 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.299 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.299 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.299 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.300 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.300 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.300 INFO:teuthology.orchestra.run.smithi089.stdout: "description": { 2024-09-21T03:42:23.300 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "description", 2024-09-21T03:42:23.300 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:23.300 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.300 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:23.300 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-21T03:42:23.300 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.300 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.300 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.301 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.301 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.301 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.301 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.301 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.301 INFO:teuthology.orchestra.run.smithi089.stdout: "device_url": { 2024-09-21T03:42:23.301 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "device_url", 2024-09-21T03:42:23.301 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:23.301 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.301 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:23.301 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-09-21T03:42:23.301 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.302 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.302 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.302 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.302 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.302 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.302 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.302 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.302 INFO:teuthology.orchestra.run.smithi089.stdout: "enabled": { 2024-09-21T03:42:23.302 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "enabled", 2024-09-21T03:42:23.302 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-21T03:42:23.302 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.302 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:23.303 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-21T03:42:23.303 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.303 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.303 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.303 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.303 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.303 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.303 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.303 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.303 INFO:teuthology.orchestra.run.smithi089.stdout: "interval": { 2024-09-21T03:42:23.303 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "interval", 2024-09-21T03:42:23.303 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2024-09-21T03:42:23.303 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.304 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:23.304 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "24", 2024-09-21T03:42:23.304 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "8", 2024-09-21T03:42:23.304 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.304 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.304 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.304 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.304 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.304 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.304 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.304 INFO:teuthology.orchestra.run.smithi089.stdout: "last_opt_revision": { 2024-09-21T03:42:23.304 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "last_opt_revision", 2024-09-21T03:42:23.305 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2024-09-21T03:42:23.305 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.305 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:23.305 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "1", 2024-09-21T03:42:23.305 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.305 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.305 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.305 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.305 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.305 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.305 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.306 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.306 INFO:teuthology.orchestra.run.smithi089.stdout: "leaderboard": { 2024-09-21T03:42:23.306 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "leaderboard", 2024-09-21T03:42:23.306 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-21T03:42:23.306 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.306 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:23.306 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-21T03:42:23.306 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.306 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.306 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.306 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.307 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.307 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.307 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.307 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.307 INFO:teuthology.orchestra.run.smithi089.stdout: "leaderboard_description": { 2024-09-21T03:42:23.307 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "leaderboard_description", 2024-09-21T03:42:23.307 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:23.307 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.307 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:23.307 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-21T03:42:23.307 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.307 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.308 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.308 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.308 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.308 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.308 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.308 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.308 INFO:teuthology.orchestra.run.smithi089.stdout: "log_level": { 2024-09-21T03:42:23.308 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_level", 2024-09-21T03:42:23.308 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:23.308 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.308 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:23.308 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-21T03:42:23.309 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.309 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.309 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-21T03:42:23.309 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2024-09-21T03:42:23.309 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2024-09-21T03:42:23.309 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2024-09-21T03:42:23.309 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2024-09-21T03:42:23.309 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2024-09-21T03:42:23.309 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2024-09-21T03:42:23.309 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-21T03:42:23.309 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.310 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.310 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.310 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.310 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.310 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster": { 2024-09-21T03:42:23.310 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster", 2024-09-21T03:42:23.310 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-21T03:42:23.310 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.310 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:23.310 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-21T03:42:23.310 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.310 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.310 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.311 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.311 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.311 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.311 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.311 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.311 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster_level": { 2024-09-21T03:42:23.311 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster_level", 2024-09-21T03:42:23.311 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:23.311 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.311 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:23.311 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "info", 2024-09-21T03:42:23.311 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.312 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.312 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-21T03:42:23.312 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2024-09-21T03:42:23.312 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2024-09-21T03:42:23.312 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2024-09-21T03:42:23.312 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2024-09-21T03:42:23.312 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2024-09-21T03:42:23.312 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2024-09-21T03:42:23.312 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-21T03:42:23.312 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.312 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.312 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.313 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.313 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.313 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_file": { 2024-09-21T03:42:23.313 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_file", 2024-09-21T03:42:23.313 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-21T03:42:23.313 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.313 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:23.313 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-21T03:42:23.313 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.313 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.313 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.313 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.313 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.313 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.314 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.314 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.314 INFO:teuthology.orchestra.run.smithi089.stdout: "organization": { 2024-09-21T03:42:23.314 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "organization", 2024-09-21T03:42:23.314 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:23.314 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.314 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:23.314 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-21T03:42:23.314 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.314 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.314 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.314 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.314 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.315 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.315 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.315 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.315 INFO:teuthology.orchestra.run.smithi089.stdout: "proxy": { 2024-09-21T03:42:23.315 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "proxy", 2024-09-21T03:42:23.315 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:23.315 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.315 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:23.315 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-21T03:42:23.315 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.315 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.315 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.315 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.316 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.316 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.316 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.316 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.316 INFO:teuthology.orchestra.run.smithi089.stdout: "sqlite3_killpoint": { 2024-09-21T03:42:23.316 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:42:23.316 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2024-09-21T03:42:23.316 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "dev", 2024-09-21T03:42:23.316 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:23.316 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "0", 2024-09-21T03:42:23.316 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.316 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.316 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.316 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.317 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.317 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.317 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.317 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.317 INFO:teuthology.orchestra.run.smithi089.stdout: "url": { 2024-09-21T03:42:23.317 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "url", 2024-09-21T03:42:23.317 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:23.317 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.317 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:23.317 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-09-21T03:42:23.317 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.317 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.317 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.318 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.318 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.318 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.318 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.318 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-21T03:42:23.318 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-21T03:42:23.318 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.318 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-21T03:42:23.318 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "test_orchestrator", 2024-09-21T03:42:23.318 INFO:teuthology.orchestra.run.smithi089.stdout: "can_run": true, 2024-09-21T03:42:23.318 INFO:teuthology.orchestra.run.smithi089.stdout: "error_string": "", 2024-09-21T03:42:23.318 INFO:teuthology.orchestra.run.smithi089.stdout: "module_options": { 2024-09-21T03:42:23.318 INFO:teuthology.orchestra.run.smithi089.stdout: "log_level": { 2024-09-21T03:42:23.319 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_level", 2024-09-21T03:42:23.319 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:23.319 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.319 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:23.319 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-21T03:42:23.319 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.319 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.319 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-21T03:42:23.319 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2024-09-21T03:42:23.319 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2024-09-21T03:42:23.319 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2024-09-21T03:42:23.319 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2024-09-21T03:42:23.320 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2024-09-21T03:42:23.320 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2024-09-21T03:42:23.320 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-21T03:42:23.320 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.320 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.320 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.320 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.320 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.320 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster": { 2024-09-21T03:42:23.320 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster", 2024-09-21T03:42:23.320 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-21T03:42:23.320 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.320 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:23.320 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-21T03:42:23.321 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.321 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.321 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.321 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.321 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.321 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.321 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.321 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.321 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster_level": { 2024-09-21T03:42:23.321 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster_level", 2024-09-21T03:42:23.321 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:23.321 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.321 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:23.322 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "info", 2024-09-21T03:42:23.322 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.322 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.322 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-21T03:42:23.322 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2024-09-21T03:42:23.322 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2024-09-21T03:42:23.322 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2024-09-21T03:42:23.322 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2024-09-21T03:42:23.322 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2024-09-21T03:42:23.322 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2024-09-21T03:42:23.322 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-21T03:42:23.322 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.322 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.323 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.323 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.323 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.323 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_file": { 2024-09-21T03:42:23.323 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_file", 2024-09-21T03:42:23.323 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-21T03:42:23.323 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.323 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:23.323 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-21T03:42:23.323 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.323 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.323 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.323 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.323 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.324 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.324 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.324 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.324 INFO:teuthology.orchestra.run.smithi089.stdout: "sqlite3_killpoint": { 2024-09-21T03:42:23.324 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:42:23.324 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2024-09-21T03:42:23.324 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "dev", 2024-09-21T03:42:23.324 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:23.324 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "0", 2024-09-21T03:42:23.324 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.324 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.324 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.324 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.325 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.325 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.325 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.325 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-21T03:42:23.325 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-21T03:42:23.325 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.325 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-21T03:42:23.325 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "volumes", 2024-09-21T03:42:23.325 INFO:teuthology.orchestra.run.smithi089.stdout: "can_run": true, 2024-09-21T03:42:23.325 INFO:teuthology.orchestra.run.smithi089.stdout: "error_string": "", 2024-09-21T03:42:23.325 INFO:teuthology.orchestra.run.smithi089.stdout: "module_options": { 2024-09-21T03:42:23.325 INFO:teuthology.orchestra.run.smithi089.stdout: "log_level": { 2024-09-21T03:42:23.325 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_level", 2024-09-21T03:42:23.326 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:23.326 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.326 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:23.326 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-21T03:42:23.326 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.326 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.326 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-21T03:42:23.326 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2024-09-21T03:42:23.326 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2024-09-21T03:42:23.326 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2024-09-21T03:42:23.326 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2024-09-21T03:42:23.326 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2024-09-21T03:42:23.326 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2024-09-21T03:42:23.327 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-21T03:42:23.327 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.327 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.327 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.327 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.327 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.327 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster": { 2024-09-21T03:42:23.327 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster", 2024-09-21T03:42:23.327 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-21T03:42:23.327 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.327 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:23.327 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-21T03:42:23.327 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.328 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.328 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.328 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.328 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.328 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.328 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.328 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.328 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster_level": { 2024-09-21T03:42:23.328 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster_level", 2024-09-21T03:42:23.328 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:23.328 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.328 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:23.328 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "info", 2024-09-21T03:42:23.329 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.329 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.329 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-21T03:42:23.329 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2024-09-21T03:42:23.329 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2024-09-21T03:42:23.329 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2024-09-21T03:42:23.329 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2024-09-21T03:42:23.329 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2024-09-21T03:42:23.329 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2024-09-21T03:42:23.329 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-21T03:42:23.329 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.329 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.329 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.330 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.330 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.330 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_file": { 2024-09-21T03:42:23.330 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_file", 2024-09-21T03:42:23.330 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-21T03:42:23.330 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.330 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:23.330 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-21T03:42:23.330 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.330 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.330 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.331 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.331 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.331 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.331 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.331 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.331 INFO:teuthology.orchestra.run.smithi089.stdout: "max_concurrent_clones": { 2024-09-21T03:42:23.331 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "max_concurrent_clones", 2024-09-21T03:42:23.331 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2024-09-21T03:42:23.331 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.331 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:23.331 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "4", 2024-09-21T03:42:23.332 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.332 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.332 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.332 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Number of asynchronous cloner threads", 2024-09-21T03:42:23.332 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.332 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.332 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.332 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.332 INFO:teuthology.orchestra.run.smithi089.stdout: "periodic_async_work": { 2024-09-21T03:42:23.332 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "periodic_async_work", 2024-09-21T03:42:23.332 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-21T03:42:23.332 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.333 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:23.333 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-21T03:42:23.333 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.333 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.333 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.333 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Periodically check for async work", 2024-09-21T03:42:23.333 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.333 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.333 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.333 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.333 INFO:teuthology.orchestra.run.smithi089.stdout: "snapshot_clone_delay": { 2024-09-21T03:42:23.334 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "snapshot_clone_delay", 2024-09-21T03:42:23.334 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2024-09-21T03:42:23.334 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.334 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:23.334 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "0", 2024-09-21T03:42:23.334 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.334 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.334 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.334 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-09-21T03:42:23.334 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.334 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.334 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.335 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.335 INFO:teuthology.orchestra.run.smithi089.stdout: "snapshot_clone_no_wait": { 2024-09-21T03:42:23.335 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "snapshot_clone_no_wait", 2024-09-21T03:42:23.335 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-21T03:42:23.335 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.335 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:23.335 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "True", 2024-09-21T03:42:23.335 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.335 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.335 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.336 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-09-21T03:42:23.336 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.336 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.336 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.336 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.336 INFO:teuthology.orchestra.run.smithi089.stdout: "sqlite3_killpoint": { 2024-09-21T03:42:23.336 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:42:23.336 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2024-09-21T03:42:23.336 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "dev", 2024-09-21T03:42:23.336 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:23.336 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "0", 2024-09-21T03:42:23.336 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.337 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.337 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.337 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.337 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.337 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.337 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.337 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-21T03:42:23.337 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-21T03:42:23.337 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.337 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-21T03:42:23.337 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "zabbix", 2024-09-21T03:42:23.338 INFO:teuthology.orchestra.run.smithi089.stdout: "can_run": true, 2024-09-21T03:42:23.338 INFO:teuthology.orchestra.run.smithi089.stdout: "error_string": "", 2024-09-21T03:42:23.338 INFO:teuthology.orchestra.run.smithi089.stdout: "module_options": { 2024-09-21T03:42:23.338 INFO:teuthology.orchestra.run.smithi089.stdout: "discovery_interval": { 2024-09-21T03:42:23.338 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "discovery_interval", 2024-09-21T03:42:23.338 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "uint", 2024-09-21T03:42:23.338 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.338 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:23.338 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "100", 2024-09-21T03:42:23.338 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.338 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.339 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.339 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.339 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.339 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.339 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.339 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.339 INFO:teuthology.orchestra.run.smithi089.stdout: "identifier": { 2024-09-21T03:42:23.339 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "identifier", 2024-09-21T03:42:23.339 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:23.339 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.339 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:23.339 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-21T03:42:23.340 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.340 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.340 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.340 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.340 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.340 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.340 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.340 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.340 INFO:teuthology.orchestra.run.smithi089.stdout: "interval": { 2024-09-21T03:42:23.340 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "interval", 2024-09-21T03:42:23.340 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "secs", 2024-09-21T03:42:23.340 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.340 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:23.340 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "60", 2024-09-21T03:42:23.341 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.341 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.341 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.341 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.341 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.341 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.341 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.341 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.341 INFO:teuthology.orchestra.run.smithi089.stdout: "log_level": { 2024-09-21T03:42:23.341 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_level", 2024-09-21T03:42:23.341 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:23.341 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.341 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:23.342 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-21T03:42:23.342 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.342 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.342 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-21T03:42:23.342 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2024-09-21T03:42:23.342 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2024-09-21T03:42:23.342 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2024-09-21T03:42:23.342 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2024-09-21T03:42:23.342 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2024-09-21T03:42:23.342 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2024-09-21T03:42:23.342 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-21T03:42:23.342 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.343 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.343 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.343 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.343 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.343 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster": { 2024-09-21T03:42:23.343 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster", 2024-09-21T03:42:23.343 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-21T03:42:23.343 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.343 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:23.343 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-21T03:42:23.343 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.343 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.343 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.344 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.344 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.344 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.344 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.344 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.344 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster_level": { 2024-09-21T03:42:23.344 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster_level", 2024-09-21T03:42:23.344 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:23.344 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.344 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:23.344 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "info", 2024-09-21T03:42:23.344 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.344 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.345 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-21T03:42:23.345 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2024-09-21T03:42:23.345 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2024-09-21T03:42:23.345 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2024-09-21T03:42:23.345 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2024-09-21T03:42:23.345 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2024-09-21T03:42:23.345 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2024-09-21T03:42:23.345 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-21T03:42:23.345 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.345 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.345 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.345 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.346 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.346 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_file": { 2024-09-21T03:42:23.346 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_file", 2024-09-21T03:42:23.346 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-21T03:42:23.346 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.346 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:23.346 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-21T03:42:23.346 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.346 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.346 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.347 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.347 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.347 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.347 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.347 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.347 INFO:teuthology.orchestra.run.smithi089.stdout: "sqlite3_killpoint": { 2024-09-21T03:42:23.347 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:42:23.347 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2024-09-21T03:42:23.347 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "dev", 2024-09-21T03:42:23.347 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:23.347 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "0", 2024-09-21T03:42:23.347 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.348 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.348 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.348 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.348 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.348 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.348 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.348 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.348 INFO:teuthology.orchestra.run.smithi089.stdout: "zabbix_host": { 2024-09-21T03:42:23.348 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "zabbix_host", 2024-09-21T03:42:23.348 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:23.348 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.349 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:23.349 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-21T03:42:23.349 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.349 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.349 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.349 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.349 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.349 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.349 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.349 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.349 INFO:teuthology.orchestra.run.smithi089.stdout: "zabbix_port": { 2024-09-21T03:42:23.349 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "zabbix_port", 2024-09-21T03:42:23.349 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2024-09-21T03:42:23.350 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.350 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:23.350 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "10051", 2024-09-21T03:42:23.350 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.350 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.350 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.350 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.350 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.350 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.350 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.350 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.351 INFO:teuthology.orchestra.run.smithi089.stdout: "zabbix_sender": { 2024-09-21T03:42:23.351 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "zabbix_sender", 2024-09-21T03:42:23.351 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:23.351 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.351 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:23.351 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-09-21T03:42:23.351 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.351 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.351 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.351 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.351 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.351 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.352 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.352 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-21T03:42:23.352 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-21T03:42:23.352 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-21T03:42:23.352 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-21T03:42:23.352 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-21T03:42:23.352 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-21T03:42:23.352 INFO:teuthology.orchestra.run.smithi089.stdout: "modules": [ 2024-09-21T03:42:23.352 INFO:teuthology.orchestra.run.smithi089.stdout: "cephadm", 2024-09-21T03:42:23.352 INFO:teuthology.orchestra.run.smithi089.stdout: "dashboard", 2024-09-21T03:42:23.352 INFO:teuthology.orchestra.run.smithi089.stdout: "iostat", 2024-09-21T03:42:23.353 INFO:teuthology.orchestra.run.smithi089.stdout: "nfs", 2024-09-21T03:42:23.353 INFO:teuthology.orchestra.run.smithi089.stdout: "restful" 2024-09-21T03:42:23.353 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-21T03:42:23.353 INFO:teuthology.orchestra.run.smithi089.stdout: "available_modules": [ 2024-09-21T03:42:23.353 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-21T03:42:23.353 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "alerts", 2024-09-21T03:42:23.353 INFO:teuthology.orchestra.run.smithi089.stdout: "can_run": true, 2024-09-21T03:42:23.353 INFO:teuthology.orchestra.run.smithi089.stdout: "error_string": "", 2024-09-21T03:42:23.353 INFO:teuthology.orchestra.run.smithi089.stdout: "module_options": { 2024-09-21T03:42:23.353 INFO:teuthology.orchestra.run.smithi089.stdout: "interval": { 2024-09-21T03:42:23.353 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "interval", 2024-09-21T03:42:23.353 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "secs", 2024-09-21T03:42:23.354 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.354 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:23.354 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "60", 2024-09-21T03:42:23.354 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.354 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.354 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.354 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "How frequently to reexamine health status", 2024-09-21T03:42:23.354 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.354 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.354 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.354 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.355 INFO:teuthology.orchestra.run.smithi089.stdout: "log_level": { 2024-09-21T03:42:23.355 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_level", 2024-09-21T03:42:23.355 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:23.355 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.355 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:23.355 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-21T03:42:23.355 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.355 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.355 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-21T03:42:23.355 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2024-09-21T03:42:23.355 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2024-09-21T03:42:23.355 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2024-09-21T03:42:23.356 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2024-09-21T03:42:23.356 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2024-09-21T03:42:23.356 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2024-09-21T03:42:23.356 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-21T03:42:23.356 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.356 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.356 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.356 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.356 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.356 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster": { 2024-09-21T03:42:23.356 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster", 2024-09-21T03:42:23.356 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-21T03:42:23.356 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.357 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:23.357 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-21T03:42:23.357 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.357 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.357 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.357 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.357 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.357 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.357 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.357 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.357 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster_level": { 2024-09-21T03:42:23.357 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster_level", 2024-09-21T03:42:23.357 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:23.357 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.358 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:23.358 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "info", 2024-09-21T03:42:23.358 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.358 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.358 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-21T03:42:23.358 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2024-09-21T03:42:23.358 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2024-09-21T03:42:23.358 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2024-09-21T03:42:23.358 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2024-09-21T03:42:23.358 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2024-09-21T03:42:23.358 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2024-09-21T03:42:23.358 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-21T03:42:23.359 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.359 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.359 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.359 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.359 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.359 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_file": { 2024-09-21T03:42:23.359 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_file", 2024-09-21T03:42:23.359 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-21T03:42:23.359 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.359 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:23.359 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-21T03:42:23.359 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.359 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.360 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.360 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.360 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.360 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.360 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.360 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.360 INFO:teuthology.orchestra.run.smithi089.stdout: "smtp_destination": { 2024-09-21T03:42:23.360 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "smtp_destination", 2024-09-21T03:42:23.360 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:23.360 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.360 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:23.360 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-21T03:42:23.361 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.361 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.361 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.361 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Email address to send alerts to, use commas to separate multiple", 2024-09-21T03:42:23.361 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.361 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.361 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.361 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.361 INFO:teuthology.orchestra.run.smithi089.stdout: "smtp_from_name": { 2024-09-21T03:42:23.361 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "smtp_from_name", 2024-09-21T03:42:23.361 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:23.361 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.361 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:23.362 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "Ceph", 2024-09-21T03:42:23.362 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.362 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.362 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.362 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Email From: name", 2024-09-21T03:42:23.362 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.362 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.362 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.362 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.362 INFO:teuthology.orchestra.run.smithi089.stdout: "smtp_host": { 2024-09-21T03:42:23.362 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "smtp_host", 2024-09-21T03:42:23.362 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:23.362 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.363 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:23.363 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-21T03:42:23.363 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.363 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.363 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.363 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "SMTP server", 2024-09-21T03:42:23.363 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.363 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.363 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.363 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.363 INFO:teuthology.orchestra.run.smithi089.stdout: "smtp_password": { 2024-09-21T03:42:23.364 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "smtp_password", 2024-09-21T03:42:23.364 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:23.364 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.364 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:23.364 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-21T03:42:23.364 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.364 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.364 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.364 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Password to authenticate with", 2024-09-21T03:42:23.364 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.364 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.364 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.364 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.365 INFO:teuthology.orchestra.run.smithi089.stdout: "smtp_port": { 2024-09-21T03:42:23.365 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "smtp_port", 2024-09-21T03:42:23.365 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2024-09-21T03:42:23.365 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.365 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:23.365 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "465", 2024-09-21T03:42:23.365 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.365 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.365 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.365 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "SMTP port", 2024-09-21T03:42:23.365 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.365 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.365 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.366 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.366 INFO:teuthology.orchestra.run.smithi089.stdout: "smtp_sender": { 2024-09-21T03:42:23.366 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "smtp_sender", 2024-09-21T03:42:23.366 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:23.366 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.366 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:23.366 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-21T03:42:23.366 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.366 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.366 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.366 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "SMTP envelope sender", 2024-09-21T03:42:23.366 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.366 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.367 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.367 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.367 INFO:teuthology.orchestra.run.smithi089.stdout: "smtp_ssl": { 2024-09-21T03:42:23.367 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "smtp_ssl", 2024-09-21T03:42:23.367 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-21T03:42:23.367 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.367 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:23.367 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "True", 2024-09-21T03:42:23.367 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.367 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.367 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.367 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Use SSL to connect to SMTP server", 2024-09-21T03:42:23.367 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.368 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.368 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.368 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.368 INFO:teuthology.orchestra.run.smithi089.stdout: "smtp_user": { 2024-09-21T03:42:23.368 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "smtp_user", 2024-09-21T03:42:23.368 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:23.368 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.368 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:23.368 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-21T03:42:23.368 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.368 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.368 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.368 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "User to authenticate as", 2024-09-21T03:42:23.368 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.368 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.369 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.369 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.369 INFO:teuthology.orchestra.run.smithi089.stdout: "sqlite3_killpoint": { 2024-09-21T03:42:23.369 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:42:23.369 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2024-09-21T03:42:23.369 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "dev", 2024-09-21T03:42:23.369 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:23.369 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "0", 2024-09-21T03:42:23.369 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.369 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.369 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.369 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.369 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.369 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.370 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.370 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-21T03:42:23.370 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-21T03:42:23.370 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.370 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-21T03:42:23.370 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "balancer", 2024-09-21T03:42:23.370 INFO:teuthology.orchestra.run.smithi089.stdout: "can_run": true, 2024-09-21T03:42:23.370 INFO:teuthology.orchestra.run.smithi089.stdout: "error_string": "", 2024-09-21T03:42:23.370 INFO:teuthology.orchestra.run.smithi089.stdout: "module_options": { 2024-09-21T03:42:23.370 INFO:teuthology.orchestra.run.smithi089.stdout: "active": { 2024-09-21T03:42:23.370 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "active", 2024-09-21T03:42:23.370 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-21T03:42:23.370 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.370 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:23.371 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "True", 2024-09-21T03:42:23.371 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.371 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.371 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.371 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "automatically balance PGs across cluster", 2024-09-21T03:42:23.371 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.371 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.371 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.371 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.371 INFO:teuthology.orchestra.run.smithi089.stdout: "begin_time": { 2024-09-21T03:42:23.371 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "begin_time", 2024-09-21T03:42:23.371 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:23.371 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.372 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:23.372 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "0000", 2024-09-21T03:42:23.372 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.372 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.372 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.372 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "beginning time of day to automatically balance", 2024-09-21T03:42:23.372 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-21T03:42:23.372 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.372 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.372 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.372 INFO:teuthology.orchestra.run.smithi089.stdout: "begin_weekday": { 2024-09-21T03:42:23.372 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "begin_weekday", 2024-09-21T03:42:23.372 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "uint", 2024-09-21T03:42:23.373 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.373 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:23.373 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "0", 2024-09-21T03:42:23.373 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "0", 2024-09-21T03:42:23.373 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "6", 2024-09-21T03:42:23.373 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.373 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-09-21T03:42:23.373 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-21T03:42:23.373 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.373 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.373 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.373 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_compat_max_iterations": { 2024-09-21T03:42:23.373 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "crush_compat_max_iterations", 2024-09-21T03:42:23.373 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "uint", 2024-09-21T03:42:23.374 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.374 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:23.374 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "25", 2024-09-21T03:42:23.374 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "1", 2024-09-21T03:42:23.374 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "250", 2024-09-21T03:42:23.374 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.374 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-09-21T03:42:23.374 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.374 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.374 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.374 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.374 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_compat_metrics": { 2024-09-21T03:42:23.375 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "crush_compat_metrics", 2024-09-21T03:42:23.375 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:23.375 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.375 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:23.375 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "pgs,objects,bytes", 2024-09-21T03:42:23.375 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.375 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.375 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.375 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-09-21T03:42:23.375 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-09-21T03:42:23.375 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.375 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.376 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.376 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_compat_step": { 2024-09-21T03:42:23.376 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "crush_compat_step", 2024-09-21T03:42:23.376 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "float", 2024-09-21T03:42:23.376 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.376 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:23.376 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "0.5", 2024-09-21T03:42:23.376 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "0.001", 2024-09-21T03:42:23.376 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "0.999", 2024-09-21T03:42:23.376 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.376 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "aggressiveness of optimization", 2024-09-21T03:42:23.376 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-09-21T03:42:23.377 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.377 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.377 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.377 INFO:teuthology.orchestra.run.smithi089.stdout: "end_time": { 2024-09-21T03:42:23.377 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "end_time", 2024-09-21T03:42:23.377 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:23.377 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.377 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:23.377 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "2359", 2024-09-21T03:42:23.377 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.377 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.377 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.377 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "ending time of day to automatically balance", 2024-09-21T03:42:23.378 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-21T03:42:23.378 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.378 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.378 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.378 INFO:teuthology.orchestra.run.smithi089.stdout: "end_weekday": { 2024-09-21T03:42:23.378 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "end_weekday", 2024-09-21T03:42:23.378 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "uint", 2024-09-21T03:42:23.378 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.378 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:23.378 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "0", 2024-09-21T03:42:23.378 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "0", 2024-09-21T03:42:23.378 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "6", 2024-09-21T03:42:23.378 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.379 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-09-21T03:42:23.379 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-21T03:42:23.379 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.379 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.379 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.379 INFO:teuthology.orchestra.run.smithi089.stdout: "log_level": { 2024-09-21T03:42:23.379 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_level", 2024-09-21T03:42:23.379 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:23.379 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.379 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:23.379 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-21T03:42:23.379 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.380 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.380 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-21T03:42:23.380 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2024-09-21T03:42:23.380 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2024-09-21T03:42:23.380 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2024-09-21T03:42:23.380 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2024-09-21T03:42:23.380 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2024-09-21T03:42:23.380 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2024-09-21T03:42:23.380 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-21T03:42:23.380 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.380 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.380 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.380 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.381 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.381 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster": { 2024-09-21T03:42:23.381 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster", 2024-09-21T03:42:23.381 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-21T03:42:23.381 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.381 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:23.381 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-21T03:42:23.381 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.381 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.381 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.381 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.381 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.381 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.381 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.382 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.382 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster_level": { 2024-09-21T03:42:23.382 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster_level", 2024-09-21T03:42:23.382 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:23.382 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.382 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:23.382 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "info", 2024-09-21T03:42:23.382 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.382 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.382 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-21T03:42:23.382 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2024-09-21T03:42:23.382 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2024-09-21T03:42:23.382 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2024-09-21T03:42:23.383 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2024-09-21T03:42:23.383 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2024-09-21T03:42:23.383 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2024-09-21T03:42:23.383 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-21T03:42:23.383 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.383 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.383 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.383 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.383 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.383 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_file": { 2024-09-21T03:42:23.383 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_file", 2024-09-21T03:42:23.383 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-21T03:42:23.383 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.383 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:23.384 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-21T03:42:23.384 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.384 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.384 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.384 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.384 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.384 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.384 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.384 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.384 INFO:teuthology.orchestra.run.smithi089.stdout: "min_score": { 2024-09-21T03:42:23.384 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "min_score", 2024-09-21T03:42:23.384 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "float", 2024-09-21T03:42:23.384 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.385 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:23.385 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "0", 2024-09-21T03:42:23.385 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.385 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.385 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.385 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-09-21T03:42:23.385 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.385 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.385 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.385 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.385 INFO:teuthology.orchestra.run.smithi089.stdout: "mode": { 2024-09-21T03:42:23.385 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "mode", 2024-09-21T03:42:23.385 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:23.386 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.386 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:23.386 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "upmap", 2024-09-21T03:42:23.386 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.386 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.386 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-21T03:42:23.386 INFO:teuthology.orchestra.run.smithi089.stdout: "crush-compat", 2024-09-21T03:42:23.386 INFO:teuthology.orchestra.run.smithi089.stdout: "none", 2024-09-21T03:42:23.386 INFO:teuthology.orchestra.run.smithi089.stdout: "read", 2024-09-21T03:42:23.386 INFO:teuthology.orchestra.run.smithi089.stdout: "upmap", 2024-09-21T03:42:23.386 INFO:teuthology.orchestra.run.smithi089.stdout: "upmap-read" 2024-09-21T03:42:23.386 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-21T03:42:23.387 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Balancer mode", 2024-09-21T03:42:23.387 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.387 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.387 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.387 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.387 INFO:teuthology.orchestra.run.smithi089.stdout: "pool_ids": { 2024-09-21T03:42:23.387 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "pool_ids", 2024-09-21T03:42:23.387 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:23.387 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.387 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:23.387 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-21T03:42:23.387 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.387 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.388 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.388 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-09-21T03:42:23.388 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.388 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.388 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.388 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.388 INFO:teuthology.orchestra.run.smithi089.stdout: "sleep_interval": { 2024-09-21T03:42:23.388 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "sleep_interval", 2024-09-21T03:42:23.388 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "secs", 2024-09-21T03:42:23.388 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.388 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:23.388 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "60", 2024-09-21T03:42:23.388 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.389 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.389 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.389 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-09-21T03:42:23.389 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.389 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.389 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.389 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.389 INFO:teuthology.orchestra.run.smithi089.stdout: "sqlite3_killpoint": { 2024-09-21T03:42:23.389 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:42:23.389 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2024-09-21T03:42:23.389 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "dev", 2024-09-21T03:42:23.389 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:23.389 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "0", 2024-09-21T03:42:23.390 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.390 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.390 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.390 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.390 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.390 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.390 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.390 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.390 INFO:teuthology.orchestra.run.smithi089.stdout: "upmap_max_deviation": { 2024-09-21T03:42:23.390 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "upmap_max_deviation", 2024-09-21T03:42:23.390 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2024-09-21T03:42:23.391 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.391 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:23.391 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "5", 2024-09-21T03:42:23.391 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "1", 2024-09-21T03:42:23.391 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.391 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.391 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "deviation below which no optimization is attempted", 2024-09-21T03:42:23.391 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-09-21T03:42:23.391 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.391 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.391 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.391 INFO:teuthology.orchestra.run.smithi089.stdout: "upmap_max_optimizations": { 2024-09-21T03:42:23.391 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "upmap_max_optimizations", 2024-09-21T03:42:23.392 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "uint", 2024-09-21T03:42:23.392 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.392 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:23.392 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "10", 2024-09-21T03:42:23.392 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.392 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.392 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.392 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-09-21T03:42:23.392 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.392 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.392 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.392 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-21T03:42:23.393 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-21T03:42:23.393 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.393 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-21T03:42:23.393 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "cephadm", 2024-09-21T03:42:23.393 INFO:teuthology.orchestra.run.smithi089.stdout: "can_run": true, 2024-09-21T03:42:23.393 INFO:teuthology.orchestra.run.smithi089.stdout: "error_string": "", 2024-09-21T03:42:23.393 INFO:teuthology.orchestra.run.smithi089.stdout: "module_options": { 2024-09-21T03:42:23.393 INFO:teuthology.orchestra.run.smithi089.stdout: "agent_down_multiplier": { 2024-09-21T03:42:23.393 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "agent_down_multiplier", 2024-09-21T03:42:23.393 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "float", 2024-09-21T03:42:23.393 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.393 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:23.394 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "3.0", 2024-09-21T03:42:23.394 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.394 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.394 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.394 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-09-21T03:42:23.394 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.394 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.394 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.394 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.394 INFO:teuthology.orchestra.run.smithi089.stdout: "agent_refresh_rate": { 2024-09-21T03:42:23.394 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "agent_refresh_rate", 2024-09-21T03:42:23.395 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "secs", 2024-09-21T03:42:23.395 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.395 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:23.395 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "20", 2024-09-21T03:42:23.395 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.395 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.395 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.395 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-09-21T03:42:23.395 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.395 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.395 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.395 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.396 INFO:teuthology.orchestra.run.smithi089.stdout: "agent_starting_port": { 2024-09-21T03:42:23.396 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "agent_starting_port", 2024-09-21T03:42:23.396 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2024-09-21T03:42:23.396 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.396 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:23.396 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "4721", 2024-09-21T03:42:23.396 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.396 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.396 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.396 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-09-21T03:42:23.396 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.396 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.397 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.397 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.397 INFO:teuthology.orchestra.run.smithi089.stdout: "allow_ptrace": { 2024-09-21T03:42:23.397 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "allow_ptrace", 2024-09-21T03:42:23.397 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-21T03:42:23.397 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.397 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:23.397 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-21T03:42:23.397 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.397 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.397 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.397 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-09-21T03:42:23.397 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-09-21T03:42:23.398 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.398 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.398 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.398 INFO:teuthology.orchestra.run.smithi089.stdout: "autotune_interval": { 2024-09-21T03:42:23.398 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "autotune_interval", 2024-09-21T03:42:23.398 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "secs", 2024-09-21T03:42:23.398 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.398 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:23.398 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "600", 2024-09-21T03:42:23.398 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.399 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.399 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.399 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "how frequently to autotune daemon memory", 2024-09-21T03:42:23.399 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.399 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.399 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.399 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.399 INFO:teuthology.orchestra.run.smithi089.stdout: "autotune_memory_target_ratio": { 2024-09-21T03:42:23.399 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "autotune_memory_target_ratio", 2024-09-21T03:42:23.399 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "float", 2024-09-21T03:42:23.399 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.399 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:23.399 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "0.7", 2024-09-21T03:42:23.399 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.400 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.400 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.400 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-09-21T03:42:23.400 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.400 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.400 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.400 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.400 INFO:teuthology.orchestra.run.smithi089.stdout: "cephadm_log_destination": { 2024-09-21T03:42:23.400 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "cephadm_log_destination", 2024-09-21T03:42:23.400 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:23.400 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.400 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:23.400 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-21T03:42:23.401 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.401 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.401 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-21T03:42:23.401 INFO:teuthology.orchestra.run.smithi089.stdout: "file", 2024-09-21T03:42:23.401 INFO:teuthology.orchestra.run.smithi089.stdout: "file,syslog", 2024-09-21T03:42:23.401 INFO:teuthology.orchestra.run.smithi089.stdout: "syslog" 2024-09-21T03:42:23.401 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-21T03:42:23.401 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-09-21T03:42:23.401 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.401 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.401 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.401 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.401 INFO:teuthology.orchestra.run.smithi089.stdout: "cgroups_split": { 2024-09-21T03:42:23.402 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "cgroups_split", 2024-09-21T03:42:23.402 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-21T03:42:23.402 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.402 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:23.402 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "True", 2024-09-21T03:42:23.402 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.402 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.402 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.402 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-09-21T03:42:23.402 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.402 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.402 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.402 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.403 INFO:teuthology.orchestra.run.smithi089.stdout: "config_checks_enabled": { 2024-09-21T03:42:23.403 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "config_checks_enabled", 2024-09-21T03:42:23.403 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-21T03:42:23.403 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.403 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:23.403 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-21T03:42:23.403 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.403 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.403 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.403 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-09-21T03:42:23.403 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.403 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.403 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.403 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.404 INFO:teuthology.orchestra.run.smithi089.stdout: "config_dashboard": { 2024-09-21T03:42:23.404 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "config_dashboard", 2024-09-21T03:42:23.404 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-21T03:42:23.404 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.404 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:23.404 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "True", 2024-09-21T03:42:23.404 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.404 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.404 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.404 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-09-21T03:42:23.404 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.404 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.404 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.405 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.405 INFO:teuthology.orchestra.run.smithi089.stdout: "container_image_alertmanager": { 2024-09-21T03:42:23.405 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "container_image_alertmanager", 2024-09-21T03:42:23.405 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:23.405 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.405 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:23.405 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.27.0", 2024-09-21T03:42:23.405 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.405 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.405 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.405 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Prometheus container image", 2024-09-21T03:42:23.405 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.405 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.406 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.406 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.406 INFO:teuthology.orchestra.run.smithi089.stdout: "container_image_base": { 2024-09-21T03:42:23.406 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "container_image_base", 2024-09-21T03:42:23.406 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:23.406 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.406 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:23.406 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "quay.io/ceph/ceph", 2024-09-21T03:42:23.406 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.406 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.406 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.406 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Container image name, without the tag", 2024-09-21T03:42:23.406 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.407 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.407 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.407 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.407 INFO:teuthology.orchestra.run.smithi089.stdout: "container_image_elasticsearch": { 2024-09-21T03:42:23.407 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "container_image_elasticsearch", 2024-09-21T03:42:23.407 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:23.407 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.407 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:23.407 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-09-21T03:42:23.407 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.407 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.407 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.407 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "elasticsearch container image", 2024-09-21T03:42:23.407 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.408 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.408 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.408 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.408 INFO:teuthology.orchestra.run.smithi089.stdout: "container_image_grafana": { 2024-09-21T03:42:23.408 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "container_image_grafana", 2024-09-21T03:42:23.408 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:23.408 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.408 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:23.408 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "quay.io/ceph/grafana:10.4.8", 2024-09-21T03:42:23.408 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.408 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.408 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.408 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Prometheus container image", 2024-09-21T03:42:23.408 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.409 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.409 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.409 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.409 INFO:teuthology.orchestra.run.smithi089.stdout: "container_image_haproxy": { 2024-09-21T03:42:23.409 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "container_image_haproxy", 2024-09-21T03:42:23.409 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:23.409 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.409 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:23.409 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-09-21T03:42:23.409 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.409 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.409 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.409 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "HAproxy container image", 2024-09-21T03:42:23.409 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.410 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.410 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.410 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.410 INFO:teuthology.orchestra.run.smithi089.stdout: "container_image_jaeger_agent": { 2024-09-21T03:42:23.410 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "container_image_jaeger_agent", 2024-09-21T03:42:23.410 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:23.410 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.410 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:23.410 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-09-21T03:42:23.410 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.410 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.410 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.410 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Jaeger agent container image", 2024-09-21T03:42:23.411 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.411 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.411 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.411 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.411 INFO:teuthology.orchestra.run.smithi089.stdout: "container_image_jaeger_collector": { 2024-09-21T03:42:23.411 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "container_image_jaeger_collector", 2024-09-21T03:42:23.411 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:23.411 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.411 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:23.411 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-09-21T03:42:23.411 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.411 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.412 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.412 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Jaeger collector container image", 2024-09-21T03:42:23.412 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.412 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.412 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.412 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.412 INFO:teuthology.orchestra.run.smithi089.stdout: "container_image_jaeger_query": { 2024-09-21T03:42:23.412 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "container_image_jaeger_query", 2024-09-21T03:42:23.412 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:23.412 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.412 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:23.412 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-09-21T03:42:23.412 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.412 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.413 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.413 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Jaeger query container image", 2024-09-21T03:42:23.413 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.413 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.413 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.413 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.413 INFO:teuthology.orchestra.run.smithi089.stdout: "container_image_keepalived": { 2024-09-21T03:42:23.413 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "container_image_keepalived", 2024-09-21T03:42:23.413 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:23.413 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.413 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:23.413 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-09-21T03:42:23.414 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.414 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.414 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.414 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Keepalived container image", 2024-09-21T03:42:23.414 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.414 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.414 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.414 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.414 INFO:teuthology.orchestra.run.smithi089.stdout: "container_image_loki": { 2024-09-21T03:42:23.414 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "container_image_loki", 2024-09-21T03:42:23.414 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:23.414 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.415 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:23.415 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "docker.io/grafana/loki:3.0.0", 2024-09-21T03:42:23.415 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.415 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.415 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.415 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Loki container image", 2024-09-21T03:42:23.415 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.415 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.415 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.415 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.415 INFO:teuthology.orchestra.run.smithi089.stdout: "container_image_nginx": { 2024-09-21T03:42:23.415 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "container_image_nginx", 2024-09-21T03:42:23.416 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:23.416 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.416 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:23.416 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "quay.io/ceph/nginx:sclorg-nginx-126", 2024-09-21T03:42:23.416 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.416 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.416 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.416 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Nginx container image", 2024-09-21T03:42:23.416 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.416 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.416 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.416 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.416 INFO:teuthology.orchestra.run.smithi089.stdout: "container_image_node_exporter": { 2024-09-21T03:42:23.416 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "container_image_node_exporter", 2024-09-21T03:42:23.417 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:23.417 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.417 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:23.417 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.7.0", 2024-09-21T03:42:23.417 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.417 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.417 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.417 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Prometheus container image", 2024-09-21T03:42:23.417 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.417 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.417 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.417 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.417 INFO:teuthology.orchestra.run.smithi089.stdout: "container_image_nvmeof": { 2024-09-21T03:42:23.417 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "container_image_nvmeof", 2024-09-21T03:42:23.418 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:23.418 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.418 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:23.418 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.17", 2024-09-21T03:42:23.418 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.418 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.418 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.418 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Nvme-of container image", 2024-09-21T03:42:23.418 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.418 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.418 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.418 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.418 INFO:teuthology.orchestra.run.smithi089.stdout: "container_image_oauth2_proxy": { 2024-09-21T03:42:23.419 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "container_image_oauth2_proxy", 2024-09-21T03:42:23.419 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:23.419 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.419 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:23.419 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "quay.io/oauth2-proxy/oauth2-proxy:v7.6.0", 2024-09-21T03:42:23.419 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.419 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.419 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.419 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "oauth2-proxy container image", 2024-09-21T03:42:23.419 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.419 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.420 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.420 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.420 INFO:teuthology.orchestra.run.smithi089.stdout: "container_image_prometheus": { 2024-09-21T03:42:23.420 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "container_image_prometheus", 2024-09-21T03:42:23.420 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:23.420 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.420 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:23.420 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "quay.io/prometheus/prometheus:v2.51.0", 2024-09-21T03:42:23.420 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.420 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.420 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.420 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Prometheus container image", 2024-09-21T03:42:23.420 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.420 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.421 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.421 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.421 INFO:teuthology.orchestra.run.smithi089.stdout: "container_image_promtail": { 2024-09-21T03:42:23.421 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "container_image_promtail", 2024-09-21T03:42:23.421 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:23.421 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.421 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:23.421 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "docker.io/grafana/promtail:3.0.0", 2024-09-21T03:42:23.421 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.421 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.421 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.421 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Promtail container image", 2024-09-21T03:42:23.421 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.421 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.422 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.422 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.422 INFO:teuthology.orchestra.run.smithi089.stdout: "container_image_samba": { 2024-09-21T03:42:23.422 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "container_image_samba", 2024-09-21T03:42:23.422 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:23.422 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.422 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:23.422 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-09-21T03:42:23.422 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.422 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.422 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.422 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Samba/SMB container image", 2024-09-21T03:42:23.422 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.423 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.423 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.423 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.423 INFO:teuthology.orchestra.run.smithi089.stdout: "container_image_samba_metrics": { 2024-09-21T03:42:23.423 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "container_image_samba_metrics", 2024-09-21T03:42:23.423 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:23.423 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.423 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:23.423 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "quay.io/samba.org/samba-metrics:latest", 2024-09-21T03:42:23.423 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.423 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.423 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.423 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Samba/SMB metrics exporter container image", 2024-09-21T03:42:23.424 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.424 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.424 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.424 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.424 INFO:teuthology.orchestra.run.smithi089.stdout: "container_image_snmp_gateway": { 2024-09-21T03:42:23.424 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "container_image_snmp_gateway", 2024-09-21T03:42:23.424 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:23.424 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.424 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:23.424 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-09-21T03:42:23.424 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.424 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.424 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.425 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "SNMP Gateway container image", 2024-09-21T03:42:23.425 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.425 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.425 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.425 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.425 INFO:teuthology.orchestra.run.smithi089.stdout: "container_init": { 2024-09-21T03:42:23.425 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "container_init", 2024-09-21T03:42:23.425 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-21T03:42:23.425 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.425 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:23.425 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "True", 2024-09-21T03:42:23.425 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.425 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.426 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.426 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Run podman/docker with `--init`", 2024-09-21T03:42:23.426 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.426 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.426 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.426 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.426 INFO:teuthology.orchestra.run.smithi089.stdout: "daemon_cache_timeout": { 2024-09-21T03:42:23.426 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "daemon_cache_timeout", 2024-09-21T03:42:23.426 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "secs", 2024-09-21T03:42:23.426 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.426 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:23.426 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "600", 2024-09-21T03:42:23.426 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.427 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.427 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.427 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-09-21T03:42:23.427 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.427 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.427 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.427 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.427 INFO:teuthology.orchestra.run.smithi089.stdout: "default_cephadm_command_timeout": { 2024-09-21T03:42:23.427 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "default_cephadm_command_timeout", 2024-09-21T03:42:23.427 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2024-09-21T03:42:23.427 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.427 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:23.427 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "900", 2024-09-21T03:42:23.427 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.428 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.428 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.428 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-09-21T03:42:23.428 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.428 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.428 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.428 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.428 INFO:teuthology.orchestra.run.smithi089.stdout: "default_registry": { 2024-09-21T03:42:23.428 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "default_registry", 2024-09-21T03:42:23.428 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:23.428 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.428 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:23.428 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "docker.io", 2024-09-21T03:42:23.428 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.429 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.429 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.429 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-09-21T03:42:23.429 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.429 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.429 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.429 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.429 INFO:teuthology.orchestra.run.smithi089.stdout: "device_cache_timeout": { 2024-09-21T03:42:23.429 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "device_cache_timeout", 2024-09-21T03:42:23.429 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "secs", 2024-09-21T03:42:23.429 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.429 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:23.429 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "1800", 2024-09-21T03:42:23.429 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.430 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.430 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.430 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "seconds to cache device inventory", 2024-09-21T03:42:23.430 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.430 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.430 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.430 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.430 INFO:teuthology.orchestra.run.smithi089.stdout: "device_enhanced_scan": { 2024-09-21T03:42:23.430 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "device_enhanced_scan", 2024-09-21T03:42:23.430 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-21T03:42:23.430 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.430 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:23.430 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-21T03:42:23.431 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.431 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.431 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.431 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Use libstoragemgmt during device scans", 2024-09-21T03:42:23.431 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.431 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.431 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.431 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.431 INFO:teuthology.orchestra.run.smithi089.stdout: "facts_cache_timeout": { 2024-09-21T03:42:23.431 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "facts_cache_timeout", 2024-09-21T03:42:23.431 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "secs", 2024-09-21T03:42:23.431 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.431 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:23.431 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "60", 2024-09-21T03:42:23.432 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.432 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.432 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.432 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "seconds to cache host facts data", 2024-09-21T03:42:23.432 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.432 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.432 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.432 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.432 INFO:teuthology.orchestra.run.smithi089.stdout: "grafana_dashboards_path": { 2024-09-21T03:42:23.432 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "grafana_dashboards_path", 2024-09-21T03:42:23.432 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:23.432 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.432 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:23.432 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-09-21T03:42:23.433 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.433 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.433 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.433 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-09-21T03:42:23.433 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.433 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.433 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.433 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.433 INFO:teuthology.orchestra.run.smithi089.stdout: "host_check_interval": { 2024-09-21T03:42:23.433 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "host_check_interval", 2024-09-21T03:42:23.433 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "secs", 2024-09-21T03:42:23.433 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.433 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:23.433 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "600", 2024-09-21T03:42:23.434 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.434 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.434 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.434 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "how frequently to perform a host check", 2024-09-21T03:42:23.434 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.434 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.434 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.434 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.434 INFO:teuthology.orchestra.run.smithi089.stdout: "hw_monitoring": { 2024-09-21T03:42:23.434 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "hw_monitoring", 2024-09-21T03:42:23.434 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-21T03:42:23.435 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.435 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:23.435 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-21T03:42:23.435 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.435 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.435 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.435 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-09-21T03:42:23.435 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.435 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.435 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.435 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.435 INFO:teuthology.orchestra.run.smithi089.stdout: "inventory_list_all": { 2024-09-21T03:42:23.436 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "inventory_list_all", 2024-09-21T03:42:23.436 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-21T03:42:23.436 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.436 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:23.436 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-21T03:42:23.436 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.436 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.436 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.436 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-09-21T03:42:23.436 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.436 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.436 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.437 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.437 INFO:teuthology.orchestra.run.smithi089.stdout: "log_level": { 2024-09-21T03:42:23.437 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_level", 2024-09-21T03:42:23.437 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:23.437 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.437 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:23.437 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-21T03:42:23.437 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.437 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.437 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-21T03:42:23.437 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2024-09-21T03:42:23.437 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2024-09-21T03:42:23.437 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2024-09-21T03:42:23.438 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2024-09-21T03:42:23.438 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2024-09-21T03:42:23.438 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2024-09-21T03:42:23.438 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-21T03:42:23.438 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.438 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.438 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.438 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.438 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.438 INFO:teuthology.orchestra.run.smithi089.stdout: "log_refresh_metadata": { 2024-09-21T03:42:23.438 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_refresh_metadata", 2024-09-21T03:42:23.438 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-21T03:42:23.438 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.439 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:23.439 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-21T03:42:23.439 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.439 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.439 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.439 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-09-21T03:42:23.439 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.439 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.439 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.439 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.439 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster": { 2024-09-21T03:42:23.439 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster", 2024-09-21T03:42:23.439 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-21T03:42:23.440 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.440 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:23.440 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "True", 2024-09-21T03:42:23.440 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.440 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.440 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.440 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-09-21T03:42:23.440 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.440 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.440 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.440 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.440 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster_level": { 2024-09-21T03:42:23.441 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster_level", 2024-09-21T03:42:23.441 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:23.441 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.441 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:23.441 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "info", 2024-09-21T03:42:23.441 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.441 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.441 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-21T03:42:23.441 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2024-09-21T03:42:23.441 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2024-09-21T03:42:23.441 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2024-09-21T03:42:23.442 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2024-09-21T03:42:23.442 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2024-09-21T03:42:23.442 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2024-09-21T03:42:23.442 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-21T03:42:23.442 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.442 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.442 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.442 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.442 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.442 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_file": { 2024-09-21T03:42:23.442 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_file", 2024-09-21T03:42:23.443 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-21T03:42:23.443 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.443 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:23.443 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-21T03:42:23.443 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.443 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.443 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.443 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.443 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.443 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.443 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.443 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.443 INFO:teuthology.orchestra.run.smithi089.stdout: "manage_etc_ceph_ceph_conf": { 2024-09-21T03:42:23.443 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-09-21T03:42:23.444 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-21T03:42:23.444 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.444 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:23.444 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-21T03:42:23.444 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.444 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.444 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.444 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-09-21T03:42:23.444 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.444 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.444 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.445 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.445 INFO:teuthology.orchestra.run.smithi089.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-09-21T03:42:23.445 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-09-21T03:42:23.445 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:23.445 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.445 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:23.445 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "*", 2024-09-21T03:42:23.445 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.445 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.445 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.445 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-09-21T03:42:23.445 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.445 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.446 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.446 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.446 INFO:teuthology.orchestra.run.smithi089.stdout: "max_count_per_host": { 2024-09-21T03:42:23.446 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "max_count_per_host", 2024-09-21T03:42:23.446 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2024-09-21T03:42:23.446 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.446 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:23.446 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "10", 2024-09-21T03:42:23.446 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.446 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.446 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.446 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "max number of daemons per service per host", 2024-09-21T03:42:23.447 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.447 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.447 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.447 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.447 INFO:teuthology.orchestra.run.smithi089.stdout: "max_osd_draining_count": { 2024-09-21T03:42:23.447 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "max_osd_draining_count", 2024-09-21T03:42:23.447 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2024-09-21T03:42:23.447 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.447 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:23.447 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "10", 2024-09-21T03:42:23.447 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.447 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.447 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.447 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-09-21T03:42:23.448 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.448 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.448 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.448 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.448 INFO:teuthology.orchestra.run.smithi089.stdout: "migration_current": { 2024-09-21T03:42:23.448 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "migration_current", 2024-09-21T03:42:23.448 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2024-09-21T03:42:23.448 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.448 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:23.448 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-21T03:42:23.448 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.448 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.448 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.448 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "internal - do not modify", 2024-09-21T03:42:23.448 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.449 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.449 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.449 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.449 INFO:teuthology.orchestra.run.smithi089.stdout: "mode": { 2024-09-21T03:42:23.449 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "mode", 2024-09-21T03:42:23.449 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:23.449 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.449 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:23.449 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "root", 2024-09-21T03:42:23.449 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.449 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.449 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-21T03:42:23.449 INFO:teuthology.orchestra.run.smithi089.stdout: "cephadm-package", 2024-09-21T03:42:23.449 INFO:teuthology.orchestra.run.smithi089.stdout: "root" 2024-09-21T03:42:23.450 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-21T03:42:23.450 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "mode for remote execution of cephadm", 2024-09-21T03:42:23.450 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.450 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.450 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.450 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.450 INFO:teuthology.orchestra.run.smithi089.stdout: "oob_default_addr": { 2024-09-21T03:42:23.450 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "oob_default_addr", 2024-09-21T03:42:23.450 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:23.450 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.450 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:23.450 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "169.254.1.1", 2024-09-21T03:42:23.450 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.451 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.451 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.451 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Default address for RedFish API (oob management).", 2024-09-21T03:42:23.451 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.451 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.451 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.451 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.451 INFO:teuthology.orchestra.run.smithi089.stdout: "prometheus_alerts_path": { 2024-09-21T03:42:23.451 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "prometheus_alerts_path", 2024-09-21T03:42:23.451 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:23.451 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.451 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:23.451 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-09-21T03:42:23.451 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.452 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.452 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.452 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-09-21T03:42:23.452 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.452 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.452 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.452 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.452 INFO:teuthology.orchestra.run.smithi089.stdout: "registry_insecure": { 2024-09-21T03:42:23.452 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "registry_insecure", 2024-09-21T03:42:23.452 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-21T03:42:23.452 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.452 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:23.452 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-21T03:42:23.453 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.453 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.453 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.453 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-09-21T03:42:23.453 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.453 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.453 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.453 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.453 INFO:teuthology.orchestra.run.smithi089.stdout: "registry_password": { 2024-09-21T03:42:23.453 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "registry_password", 2024-09-21T03:42:23.453 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:23.453 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.453 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:23.453 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-21T03:42:23.454 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.454 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.454 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.454 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-09-21T03:42:23.454 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.454 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.454 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.454 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.454 INFO:teuthology.orchestra.run.smithi089.stdout: "registry_url": { 2024-09-21T03:42:23.454 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "registry_url", 2024-09-21T03:42:23.454 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:23.455 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.455 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:23.455 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-21T03:42:23.455 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.455 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.455 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.455 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-09-21T03:42:23.455 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.455 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.455 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.455 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.455 INFO:teuthology.orchestra.run.smithi089.stdout: "registry_username": { 2024-09-21T03:42:23.455 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "registry_username", 2024-09-21T03:42:23.456 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:23.456 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.456 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:23.456 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-21T03:42:23.456 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.456 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.456 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.456 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-09-21T03:42:23.456 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.456 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.456 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.456 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.456 INFO:teuthology.orchestra.run.smithi089.stdout: "secure_monitoring_stack": { 2024-09-21T03:42:23.456 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "secure_monitoring_stack", 2024-09-21T03:42:23.457 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-21T03:42:23.457 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.457 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:23.457 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-21T03:42:23.457 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.457 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.457 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.457 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-09-21T03:42:23.457 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.457 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.457 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.457 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.457 INFO:teuthology.orchestra.run.smithi089.stdout: "service_discovery_port": { 2024-09-21T03:42:23.458 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "service_discovery_port", 2024-09-21T03:42:23.458 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2024-09-21T03:42:23.458 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.458 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:23.458 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "8765", 2024-09-21T03:42:23.458 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.458 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.458 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.458 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "cephadm service discovery port", 2024-09-21T03:42:23.458 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.458 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.458 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.458 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.459 INFO:teuthology.orchestra.run.smithi089.stdout: "sqlite3_killpoint": { 2024-09-21T03:42:23.459 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:42:23.459 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2024-09-21T03:42:23.459 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "dev", 2024-09-21T03:42:23.459 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:23.459 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "0", 2024-09-21T03:42:23.459 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.459 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.459 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.459 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.459 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.459 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.459 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.459 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.459 INFO:teuthology.orchestra.run.smithi089.stdout: "ssh_config_file": { 2024-09-21T03:42:23.460 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "ssh_config_file", 2024-09-21T03:42:23.460 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:23.460 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.460 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:23.460 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-21T03:42:23.460 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.460 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.460 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.460 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-09-21T03:42:23.460 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.460 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.460 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.461 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.461 INFO:teuthology.orchestra.run.smithi089.stdout: "ssh_keepalive_count_max": { 2024-09-21T03:42:23.461 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "ssh_keepalive_count_max", 2024-09-21T03:42:23.461 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2024-09-21T03:42:23.461 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.461 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:23.461 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "3", 2024-09-21T03:42:23.461 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.461 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.461 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.461 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "How many times ssh connections can fail liveness checks before the host is marked offline", 2024-09-21T03:42:23.461 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.462 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.462 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.462 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.462 INFO:teuthology.orchestra.run.smithi089.stdout: "ssh_keepalive_interval": { 2024-09-21T03:42:23.462 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "ssh_keepalive_interval", 2024-09-21T03:42:23.462 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2024-09-21T03:42:23.462 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.462 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:23.462 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "7", 2024-09-21T03:42:23.462 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.462 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.462 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.462 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "How often ssh connections are checked for liveness", 2024-09-21T03:42:23.462 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.462 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.463 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.463 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.463 INFO:teuthology.orchestra.run.smithi089.stdout: "use_agent": { 2024-09-21T03:42:23.463 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "use_agent", 2024-09-21T03:42:23.463 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-21T03:42:23.463 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.463 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:23.463 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-21T03:42:23.463 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.463 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.463 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.463 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-09-21T03:42:23.463 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.463 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.464 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.464 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.464 INFO:teuthology.orchestra.run.smithi089.stdout: "use_repo_digest": { 2024-09-21T03:42:23.464 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "use_repo_digest", 2024-09-21T03:42:23.464 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-21T03:42:23.464 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.464 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:23.464 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "True", 2024-09-21T03:42:23.464 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.464 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.464 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.464 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-09-21T03:42:23.464 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.464 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.465 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.465 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.465 INFO:teuthology.orchestra.run.smithi089.stdout: "warn_on_failed_host_check": { 2024-09-21T03:42:23.465 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "warn_on_failed_host_check", 2024-09-21T03:42:23.465 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-21T03:42:23.465 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.465 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:23.465 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "True", 2024-09-21T03:42:23.465 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.465 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.465 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.465 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "raise a health warning if the host check fails", 2024-09-21T03:42:23.465 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.466 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.466 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.466 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.466 INFO:teuthology.orchestra.run.smithi089.stdout: "warn_on_stray_daemons": { 2024-09-21T03:42:23.466 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "warn_on_stray_daemons", 2024-09-21T03:42:23.466 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-21T03:42:23.466 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.466 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:23.466 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "True", 2024-09-21T03:42:23.466 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.466 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.466 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.467 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-09-21T03:42:23.467 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.467 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.467 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.467 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.467 INFO:teuthology.orchestra.run.smithi089.stdout: "warn_on_stray_hosts": { 2024-09-21T03:42:23.467 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "warn_on_stray_hosts", 2024-09-21T03:42:23.467 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-21T03:42:23.467 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.467 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:23.467 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "True", 2024-09-21T03:42:23.467 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.468 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.468 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.468 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-09-21T03:42:23.468 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.468 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.468 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.468 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-21T03:42:23.468 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-21T03:42:23.468 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.468 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-21T03:42:23.468 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "crash", 2024-09-21T03:42:23.468 INFO:teuthology.orchestra.run.smithi089.stdout: "can_run": true, 2024-09-21T03:42:23.469 INFO:teuthology.orchestra.run.smithi089.stdout: "error_string": "", 2024-09-21T03:42:23.469 INFO:teuthology.orchestra.run.smithi089.stdout: "module_options": { 2024-09-21T03:42:23.469 INFO:teuthology.orchestra.run.smithi089.stdout: "log_level": { 2024-09-21T03:42:23.469 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_level", 2024-09-21T03:42:23.469 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:23.469 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.469 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:23.469 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-21T03:42:23.469 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.469 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.469 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-21T03:42:23.469 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2024-09-21T03:42:23.469 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2024-09-21T03:42:23.469 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2024-09-21T03:42:23.470 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2024-09-21T03:42:23.470 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2024-09-21T03:42:23.470 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2024-09-21T03:42:23.470 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-21T03:42:23.470 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.470 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.470 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.470 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.470 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.470 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster": { 2024-09-21T03:42:23.470 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster", 2024-09-21T03:42:23.470 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-21T03:42:23.470 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.471 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:23.471 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-21T03:42:23.471 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.471 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.471 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.471 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.471 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.471 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.471 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.471 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.471 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster_level": { 2024-09-21T03:42:23.471 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster_level", 2024-09-21T03:42:23.471 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:23.471 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.472 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:23.472 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "info", 2024-09-21T03:42:23.472 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.472 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.472 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-21T03:42:23.472 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2024-09-21T03:42:23.472 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2024-09-21T03:42:23.472 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2024-09-21T03:42:23.472 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2024-09-21T03:42:23.472 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2024-09-21T03:42:23.472 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2024-09-21T03:42:23.472 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-21T03:42:23.472 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.473 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.473 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.473 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.473 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.473 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_file": { 2024-09-21T03:42:23.473 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_file", 2024-09-21T03:42:23.473 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-21T03:42:23.473 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.473 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:23.473 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-21T03:42:23.473 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.473 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.473 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.473 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.474 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.474 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.474 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.474 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.474 INFO:teuthology.orchestra.run.smithi089.stdout: "retain_interval": { 2024-09-21T03:42:23.474 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "retain_interval", 2024-09-21T03:42:23.474 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "secs", 2024-09-21T03:42:23.474 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.474 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:23.474 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "31536000", 2024-09-21T03:42:23.474 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.475 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.475 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.475 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "how long to retain crashes before pruning them", 2024-09-21T03:42:23.475 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.475 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.475 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.475 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.475 INFO:teuthology.orchestra.run.smithi089.stdout: "sqlite3_killpoint": { 2024-09-21T03:42:23.475 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:42:23.475 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2024-09-21T03:42:23.475 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "dev", 2024-09-21T03:42:23.476 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:23.476 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "0", 2024-09-21T03:42:23.476 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.476 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.476 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.476 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.476 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.476 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.476 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.476 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.476 INFO:teuthology.orchestra.run.smithi089.stdout: "warn_recent_interval": { 2024-09-21T03:42:23.476 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "warn_recent_interval", 2024-09-21T03:42:23.476 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "secs", 2024-09-21T03:42:23.477 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.477 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:23.477 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "1209600", 2024-09-21T03:42:23.477 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.477 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.477 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.477 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "time interval in which to warn about recent crashes", 2024-09-21T03:42:23.477 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.477 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.477 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.477 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-21T03:42:23.477 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-21T03:42:23.478 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.478 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-21T03:42:23.478 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "dashboard", 2024-09-21T03:42:23.478 INFO:teuthology.orchestra.run.smithi089.stdout: "can_run": true, 2024-09-21T03:42:23.478 INFO:teuthology.orchestra.run.smithi089.stdout: "error_string": "", 2024-09-21T03:42:23.478 INFO:teuthology.orchestra.run.smithi089.stdout: "module_options": { 2024-09-21T03:42:23.478 INFO:teuthology.orchestra.run.smithi089.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-09-21T03:42:23.478 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-09-21T03:42:23.478 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2024-09-21T03:42:23.478 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.478 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:23.478 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "10", 2024-09-21T03:42:23.479 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.479 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.479 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.479 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.479 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.479 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.479 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.479 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.479 INFO:teuthology.orchestra.run.smithi089.stdout: "ALERTMANAGER_API_HOST": { 2024-09-21T03:42:23.479 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "ALERTMANAGER_API_HOST", 2024-09-21T03:42:23.479 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:23.479 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.480 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:23.480 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-21T03:42:23.480 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.480 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.480 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.480 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.480 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.480 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.480 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.480 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.480 INFO:teuthology.orchestra.run.smithi089.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-09-21T03:42:23.480 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-09-21T03:42:23.480 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-21T03:42:23.481 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.481 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:23.481 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "True", 2024-09-21T03:42:23.481 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.481 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.481 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.481 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.481 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.481 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.481 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.481 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.481 INFO:teuthology.orchestra.run.smithi089.stdout: "AUDIT_API_ENABLED": { 2024-09-21T03:42:23.482 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "AUDIT_API_ENABLED", 2024-09-21T03:42:23.482 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-21T03:42:23.482 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.482 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:23.482 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-21T03:42:23.482 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.482 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.482 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.482 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.482 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.482 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.482 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.483 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.483 INFO:teuthology.orchestra.run.smithi089.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-09-21T03:42:23.483 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-09-21T03:42:23.483 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-21T03:42:23.483 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.483 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:23.483 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "True", 2024-09-21T03:42:23.483 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.483 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.483 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.483 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.483 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.483 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.484 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.484 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.484 INFO:teuthology.orchestra.run.smithi089.stdout: "ENABLE_BROWSABLE_API": { 2024-09-21T03:42:23.484 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "ENABLE_BROWSABLE_API", 2024-09-21T03:42:23.484 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-21T03:42:23.484 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.484 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:23.484 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "True", 2024-09-21T03:42:23.484 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.484 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.484 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.484 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.484 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.485 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.485 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.485 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.485 INFO:teuthology.orchestra.run.smithi089.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-09-21T03:42:23.485 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-09-21T03:42:23.485 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-21T03:42:23.485 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.485 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:23.485 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "True", 2024-09-21T03:42:23.485 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.485 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.485 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.486 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.486 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.486 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.486 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.486 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.486 INFO:teuthology.orchestra.run.smithi089.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-09-21T03:42:23.486 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-09-21T03:42:23.486 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-21T03:42:23.486 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.486 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:23.486 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "True", 2024-09-21T03:42:23.487 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.487 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.487 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.487 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.487 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.487 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.487 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.487 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.487 INFO:teuthology.orchestra.run.smithi089.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-09-21T03:42:23.487 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-09-21T03:42:23.487 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-21T03:42:23.487 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.488 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:23.488 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "True", 2024-09-21T03:42:23.488 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.488 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.488 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.488 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.488 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.488 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.488 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.488 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.488 INFO:teuthology.orchestra.run.smithi089.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-09-21T03:42:23.488 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-09-21T03:42:23.488 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-21T03:42:23.489 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.489 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:23.489 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "True", 2024-09-21T03:42:23.489 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.489 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.489 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.489 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.489 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.489 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.489 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.489 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.489 INFO:teuthology.orchestra.run.smithi089.stdout: "FEATURE_TOGGLE_NFS": { 2024-09-21T03:42:23.489 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-09-21T03:42:23.490 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-21T03:42:23.490 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.490 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:23.490 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "True", 2024-09-21T03:42:23.490 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.490 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.490 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.490 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.490 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.490 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.490 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.490 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.491 INFO:teuthology.orchestra.run.smithi089.stdout: "FEATURE_TOGGLE_RBD": { 2024-09-21T03:42:23.491 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-09-21T03:42:23.491 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-21T03:42:23.491 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.491 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:23.491 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "True", 2024-09-21T03:42:23.491 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.491 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.491 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.491 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.491 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.491 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.492 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.492 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.492 INFO:teuthology.orchestra.run.smithi089.stdout: "FEATURE_TOGGLE_RGW": { 2024-09-21T03:42:23.492 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-09-21T03:42:23.492 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-21T03:42:23.492 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.492 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:23.492 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "True", 2024-09-21T03:42:23.492 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.492 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.492 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.492 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.492 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.492 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.493 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.493 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.493 INFO:teuthology.orchestra.run.smithi089.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-09-21T03:42:23.493 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-09-21T03:42:23.493 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:23.493 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.493 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:23.493 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-21T03:42:23.493 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.493 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.493 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.493 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.493 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.493 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.494 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.494 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.494 INFO:teuthology.orchestra.run.smithi089.stdout: "GRAFANA_API_PASSWORD": { 2024-09-21T03:42:23.494 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "GRAFANA_API_PASSWORD", 2024-09-21T03:42:23.494 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:23.494 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.494 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:23.494 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "admin", 2024-09-21T03:42:23.494 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.494 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.494 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.494 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.495 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.495 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.495 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.495 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.495 INFO:teuthology.orchestra.run.smithi089.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-09-21T03:42:23.495 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-09-21T03:42:23.495 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-21T03:42:23.495 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.495 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:23.495 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "True", 2024-09-21T03:42:23.495 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.495 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.496 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.496 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.496 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.496 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.496 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.496 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.496 INFO:teuthology.orchestra.run.smithi089.stdout: "GRAFANA_API_URL": { 2024-09-21T03:42:23.496 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "GRAFANA_API_URL", 2024-09-21T03:42:23.496 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:23.496 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.496 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:23.496 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-21T03:42:23.496 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.497 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.497 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.497 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.497 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.497 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.497 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.497 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.497 INFO:teuthology.orchestra.run.smithi089.stdout: "GRAFANA_API_USERNAME": { 2024-09-21T03:42:23.497 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "GRAFANA_API_USERNAME", 2024-09-21T03:42:23.497 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:23.497 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.497 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:23.498 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "admin", 2024-09-21T03:42:23.498 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.498 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.498 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.498 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.498 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.498 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.498 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.498 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.498 INFO:teuthology.orchestra.run.smithi089.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-09-21T03:42:23.498 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-09-21T03:42:23.498 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:23.499 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.499 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:23.499 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-21T03:42:23.499 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.499 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.499 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.499 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.499 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.499 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.499 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.499 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.499 INFO:teuthology.orchestra.run.smithi089.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-09-21T03:42:23.499 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-09-21T03:42:23.500 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-21T03:42:23.500 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.500 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:23.500 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-21T03:42:23.500 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.500 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.500 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.500 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.500 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.500 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.500 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.500 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.501 INFO:teuthology.orchestra.run.smithi089.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-09-21T03:42:23.501 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-09-21T03:42:23.501 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-21T03:42:23.501 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.501 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:23.501 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "True", 2024-09-21T03:42:23.501 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.501 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.501 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.501 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.501 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.501 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.501 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.501 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.502 INFO:teuthology.orchestra.run.smithi089.stdout: "ISSUE_TRACKER_API_KEY": { 2024-09-21T03:42:23.502 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-09-21T03:42:23.502 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:23.502 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.502 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:23.502 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-21T03:42:23.502 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.502 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.502 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.502 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.502 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.502 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.502 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.503 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.503 INFO:teuthology.orchestra.run.smithi089.stdout: "MANAGED_BY_CLUSTERS": { 2024-09-21T03:42:23.503 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-09-21T03:42:23.503 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:23.503 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.503 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:23.503 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "[]", 2024-09-21T03:42:23.503 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.503 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.503 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.503 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.503 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.503 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.504 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.504 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.504 INFO:teuthology.orchestra.run.smithi089.stdout: "MULTICLUSTER_CONFIG": { 2024-09-21T03:42:23.504 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "MULTICLUSTER_CONFIG", 2024-09-21T03:42:23.504 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:23.504 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.504 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:23.504 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "{}", 2024-09-21T03:42:23.504 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.504 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.504 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.504 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.504 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.505 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.505 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.505 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.505 INFO:teuthology.orchestra.run.smithi089.stdout: "PROMETHEUS_API_HOST": { 2024-09-21T03:42:23.505 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "PROMETHEUS_API_HOST", 2024-09-21T03:42:23.505 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:23.505 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.505 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:23.505 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-21T03:42:23.505 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.505 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.505 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.506 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.506 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.506 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.506 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.506 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.506 INFO:teuthology.orchestra.run.smithi089.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-09-21T03:42:23.506 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-09-21T03:42:23.506 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-21T03:42:23.506 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.506 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:23.506 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "True", 2024-09-21T03:42:23.506 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.507 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.507 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.507 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.507 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.507 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.507 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.507 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.507 INFO:teuthology.orchestra.run.smithi089.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-09-21T03:42:23.507 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-09-21T03:42:23.507 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-21T03:42:23.507 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.507 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:23.508 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-21T03:42:23.508 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.508 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.508 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.508 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.508 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.508 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.508 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.508 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.508 INFO:teuthology.orchestra.run.smithi089.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-09-21T03:42:23.508 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-09-21T03:42:23.509 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-21T03:42:23.509 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.509 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:23.509 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-21T03:42:23.509 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.509 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.509 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.509 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.509 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.509 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.509 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.509 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.509 INFO:teuthology.orchestra.run.smithi089.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-09-21T03:42:23.510 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-09-21T03:42:23.510 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-21T03:42:23.510 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.510 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:23.510 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "True", 2024-09-21T03:42:23.510 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.510 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.510 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.510 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.510 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.510 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.510 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.510 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.511 INFO:teuthology.orchestra.run.smithi089.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-09-21T03:42:23.511 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-09-21T03:42:23.511 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-21T03:42:23.511 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.511 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:23.511 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "True", 2024-09-21T03:42:23.511 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.511 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.511 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.511 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.511 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.511 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.512 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.512 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.512 INFO:teuthology.orchestra.run.smithi089.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-09-21T03:42:23.512 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-09-21T03:42:23.512 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-21T03:42:23.512 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.512 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:23.512 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-21T03:42:23.512 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.512 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.512 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.512 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.512 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.513 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.513 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.513 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.513 INFO:teuthology.orchestra.run.smithi089.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-09-21T03:42:23.513 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-09-21T03:42:23.513 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-21T03:42:23.513 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.513 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:23.513 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-21T03:42:23.513 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.513 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.513 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.513 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.513 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.514 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.514 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.514 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.514 INFO:teuthology.orchestra.run.smithi089.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-09-21T03:42:23.514 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-09-21T03:42:23.514 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-21T03:42:23.514 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.514 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:23.514 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-21T03:42:23.514 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.514 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.514 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.515 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.515 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.515 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.515 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.515 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.515 INFO:teuthology.orchestra.run.smithi089.stdout: "PWD_POLICY_ENABLED": { 2024-09-21T03:42:23.515 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "PWD_POLICY_ENABLED", 2024-09-21T03:42:23.515 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-21T03:42:23.515 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.515 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:23.515 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "True", 2024-09-21T03:42:23.515 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.516 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.516 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.516 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.516 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.516 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.516 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.516 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.516 INFO:teuthology.orchestra.run.smithi089.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-09-21T03:42:23.516 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-09-21T03:42:23.516 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:23.516 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.516 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:23.516 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-09-21T03:42:23.516 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.516 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.517 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.517 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.517 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.517 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.517 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.517 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.517 INFO:teuthology.orchestra.run.smithi089.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-09-21T03:42:23.517 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-09-21T03:42:23.517 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2024-09-21T03:42:23.517 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.517 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:23.517 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "10", 2024-09-21T03:42:23.517 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.518 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.518 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.518 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.518 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.518 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.518 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.518 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.518 INFO:teuthology.orchestra.run.smithi089.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-09-21T03:42:23.518 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-09-21T03:42:23.518 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2024-09-21T03:42:23.518 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.518 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:23.518 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "8", 2024-09-21T03:42:23.518 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.519 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.519 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.519 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.519 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.519 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.519 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.519 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.519 INFO:teuthology.orchestra.run.smithi089.stdout: "REST_REQUESTS_TIMEOUT": { 2024-09-21T03:42:23.519 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-09-21T03:42:23.519 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2024-09-21T03:42:23.519 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.519 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:23.519 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "45", 2024-09-21T03:42:23.520 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.520 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.520 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.520 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.520 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.520 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.520 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.520 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.520 INFO:teuthology.orchestra.run.smithi089.stdout: "RGW_API_ACCESS_KEY": { 2024-09-21T03:42:23.520 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "RGW_API_ACCESS_KEY", 2024-09-21T03:42:23.520 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:23.520 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.520 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:23.520 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-21T03:42:23.521 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.521 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.521 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.521 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.521 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.521 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.521 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.521 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.521 INFO:teuthology.orchestra.run.smithi089.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-09-21T03:42:23.521 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-09-21T03:42:23.521 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:23.521 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.521 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:23.521 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "admin", 2024-09-21T03:42:23.522 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.522 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.522 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.522 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.522 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.522 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.522 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.522 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.522 INFO:teuthology.orchestra.run.smithi089.stdout: "RGW_API_SECRET_KEY": { 2024-09-21T03:42:23.522 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "RGW_API_SECRET_KEY", 2024-09-21T03:42:23.522 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:23.522 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.522 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:23.523 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-21T03:42:23.523 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.523 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.523 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.523 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.523 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.523 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.523 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.523 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.523 INFO:teuthology.orchestra.run.smithi089.stdout: "RGW_API_SSL_VERIFY": { 2024-09-21T03:42:23.523 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "RGW_API_SSL_VERIFY", 2024-09-21T03:42:23.523 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-21T03:42:23.524 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.524 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:23.524 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "True", 2024-09-21T03:42:23.524 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.524 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.524 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.524 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.524 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.524 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.524 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.524 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.524 INFO:teuthology.orchestra.run.smithi089.stdout: "UNSAFE_TLS_v1_2": { 2024-09-21T03:42:23.524 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "UNSAFE_TLS_v1_2", 2024-09-21T03:42:23.525 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-21T03:42:23.525 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.525 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:23.525 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-21T03:42:23.525 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.525 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.525 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.525 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.525 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.525 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.525 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.525 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.525 INFO:teuthology.orchestra.run.smithi089.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-09-21T03:42:23.526 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-09-21T03:42:23.526 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2024-09-21T03:42:23.526 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.526 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:23.526 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "0", 2024-09-21T03:42:23.526 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.526 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.526 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.526 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.526 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.526 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.526 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.526 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.526 INFO:teuthology.orchestra.run.smithi089.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-09-21T03:42:23.527 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-09-21T03:42:23.527 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2024-09-21T03:42:23.527 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.527 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:23.527 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "10", 2024-09-21T03:42:23.527 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.527 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.527 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.527 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.527 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.527 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.527 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.527 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.528 INFO:teuthology.orchestra.run.smithi089.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-09-21T03:42:23.528 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-09-21T03:42:23.528 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2024-09-21T03:42:23.528 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.528 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:23.528 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "5", 2024-09-21T03:42:23.528 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.528 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.528 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.528 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.528 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.528 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.528 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.529 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.529 INFO:teuthology.orchestra.run.smithi089.stdout: "cross_origin_url": { 2024-09-21T03:42:23.529 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "cross_origin_url", 2024-09-21T03:42:23.529 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:23.529 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.529 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:23.529 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-21T03:42:23.529 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.529 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.529 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.529 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.529 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.529 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.530 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.530 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.530 INFO:teuthology.orchestra.run.smithi089.stdout: "crt_file": { 2024-09-21T03:42:23.530 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "crt_file", 2024-09-21T03:42:23.530 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:23.530 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.530 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:23.530 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-21T03:42:23.530 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.530 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.530 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.530 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.531 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.531 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.531 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.531 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.531 INFO:teuthology.orchestra.run.smithi089.stdout: "debug": { 2024-09-21T03:42:23.531 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "debug", 2024-09-21T03:42:23.531 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-21T03:42:23.531 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.531 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:23.531 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-21T03:42:23.531 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.531 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.532 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.532 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Enable/disable debug options", 2024-09-21T03:42:23.532 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.532 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.532 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.532 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.532 INFO:teuthology.orchestra.run.smithi089.stdout: "jwt_token_ttl": { 2024-09-21T03:42:23.532 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "jwt_token_ttl", 2024-09-21T03:42:23.532 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2024-09-21T03:42:23.532 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.532 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:23.532 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "28800", 2024-09-21T03:42:23.533 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.533 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.533 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.533 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.533 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.533 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.533 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.533 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.533 INFO:teuthology.orchestra.run.smithi089.stdout: "key_file": { 2024-09-21T03:42:23.533 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "key_file", 2024-09-21T03:42:23.533 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:23.533 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.533 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:23.534 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-21T03:42:23.534 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.534 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.534 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.534 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.534 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.534 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.534 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.534 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.534 INFO:teuthology.orchestra.run.smithi089.stdout: "log_level": { 2024-09-21T03:42:23.534 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_level", 2024-09-21T03:42:23.534 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:23.534 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.535 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:23.535 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-21T03:42:23.535 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.535 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.535 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-21T03:42:23.535 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2024-09-21T03:42:23.535 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2024-09-21T03:42:23.535 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2024-09-21T03:42:23.535 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2024-09-21T03:42:23.535 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2024-09-21T03:42:23.535 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2024-09-21T03:42:23.535 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-21T03:42:23.536 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.536 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.536 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.536 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.536 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.536 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster": { 2024-09-21T03:42:23.536 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster", 2024-09-21T03:42:23.536 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-21T03:42:23.536 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.536 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:23.536 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-21T03:42:23.536 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.537 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.537 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.537 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.537 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.537 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.537 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.537 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.537 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster_level": { 2024-09-21T03:42:23.537 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster_level", 2024-09-21T03:42:23.537 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:23.537 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.537 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:23.537 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "info", 2024-09-21T03:42:23.537 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.538 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.538 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-21T03:42:23.538 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2024-09-21T03:42:23.538 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2024-09-21T03:42:23.538 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2024-09-21T03:42:23.538 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2024-09-21T03:42:23.538 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2024-09-21T03:42:23.538 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2024-09-21T03:42:23.538 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-21T03:42:23.538 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.539 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.539 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.539 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.539 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.539 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_file": { 2024-09-21T03:42:23.539 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_file", 2024-09-21T03:42:23.539 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-21T03:42:23.539 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.539 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:23.539 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-21T03:42:23.539 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.539 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.540 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.540 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.540 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.540 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.540 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.540 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.540 INFO:teuthology.orchestra.run.smithi089.stdout: "motd": { 2024-09-21T03:42:23.540 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "motd", 2024-09-21T03:42:23.540 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:23.540 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.540 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:23.540 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-21T03:42:23.540 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.541 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.541 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.541 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "The message of the day", 2024-09-21T03:42:23.541 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.541 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.541 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.541 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.541 INFO:teuthology.orchestra.run.smithi089.stdout: "redirect_resolve_ip_addr": { 2024-09-21T03:42:23.541 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "redirect_resolve_ip_addr", 2024-09-21T03:42:23.541 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-21T03:42:23.541 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.541 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:23.542 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-21T03:42:23.542 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.542 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.542 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.542 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.542 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.542 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.542 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.542 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.542 INFO:teuthology.orchestra.run.smithi089.stdout: "server_addr": { 2024-09-21T03:42:23.542 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "server_addr", 2024-09-21T03:42:23.543 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:23.543 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.543 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:23.543 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "::", 2024-09-21T03:42:23.543 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.543 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.543 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.543 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.543 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.543 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.543 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.543 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.544 INFO:teuthology.orchestra.run.smithi089.stdout: "server_port": { 2024-09-21T03:42:23.544 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "server_port", 2024-09-21T03:42:23.544 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2024-09-21T03:42:23.544 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.544 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:23.544 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "8080", 2024-09-21T03:42:23.544 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.544 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.544 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.544 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.544 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.544 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.544 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.545 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.545 INFO:teuthology.orchestra.run.smithi089.stdout: "sqlite3_killpoint": { 2024-09-21T03:42:23.545 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:42:23.545 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2024-09-21T03:42:23.545 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "dev", 2024-09-21T03:42:23.545 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:23.545 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "0", 2024-09-21T03:42:23.545 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.545 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.545 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.545 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.545 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.545 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.546 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.546 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.546 INFO:teuthology.orchestra.run.smithi089.stdout: "ssl": { 2024-09-21T03:42:23.546 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "ssl", 2024-09-21T03:42:23.546 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-21T03:42:23.546 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.546 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:23.546 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "True", 2024-09-21T03:42:23.546 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.546 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.546 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.546 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.547 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.547 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.547 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.547 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.547 INFO:teuthology.orchestra.run.smithi089.stdout: "ssl_server_port": { 2024-09-21T03:42:23.547 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "ssl_server_port", 2024-09-21T03:42:23.547 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2024-09-21T03:42:23.547 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.547 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:23.547 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "8443", 2024-09-21T03:42:23.547 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.547 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.547 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.548 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.548 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.548 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.548 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.548 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.548 INFO:teuthology.orchestra.run.smithi089.stdout: "sso_oauth2": { 2024-09-21T03:42:23.548 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "sso_oauth2", 2024-09-21T03:42:23.548 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-21T03:42:23.548 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.548 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:23.548 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-21T03:42:23.548 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.549 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.549 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.549 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.549 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.549 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.549 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.549 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.549 INFO:teuthology.orchestra.run.smithi089.stdout: "standby_behaviour": { 2024-09-21T03:42:23.549 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "standby_behaviour", 2024-09-21T03:42:23.549 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:23.549 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.549 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:23.550 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "redirect", 2024-09-21T03:42:23.550 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.550 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.550 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-21T03:42:23.550 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2024-09-21T03:42:23.550 INFO:teuthology.orchestra.run.smithi089.stdout: "redirect" 2024-09-21T03:42:23.550 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-21T03:42:23.550 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.550 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.550 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.550 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.551 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.551 INFO:teuthology.orchestra.run.smithi089.stdout: "standby_error_status_code": { 2024-09-21T03:42:23.551 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "standby_error_status_code", 2024-09-21T03:42:23.551 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2024-09-21T03:42:23.551 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.551 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:23.551 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "500", 2024-09-21T03:42:23.551 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "400", 2024-09-21T03:42:23.551 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "599", 2024-09-21T03:42:23.551 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.551 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.551 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.551 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.552 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.552 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.552 INFO:teuthology.orchestra.run.smithi089.stdout: "url_prefix": { 2024-09-21T03:42:23.552 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "url_prefix", 2024-09-21T03:42:23.552 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:23.552 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.552 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:23.552 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-21T03:42:23.552 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.552 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.552 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.552 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.552 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.552 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.552 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.553 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-21T03:42:23.553 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-21T03:42:23.553 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.553 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-21T03:42:23.553 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "devicehealth", 2024-09-21T03:42:23.553 INFO:teuthology.orchestra.run.smithi089.stdout: "can_run": true, 2024-09-21T03:42:23.553 INFO:teuthology.orchestra.run.smithi089.stdout: "error_string": "", 2024-09-21T03:42:23.553 INFO:teuthology.orchestra.run.smithi089.stdout: "module_options": { 2024-09-21T03:42:23.553 INFO:teuthology.orchestra.run.smithi089.stdout: "enable_monitoring": { 2024-09-21T03:42:23.553 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "enable_monitoring", 2024-09-21T03:42:23.553 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-21T03:42:23.553 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.553 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:23.554 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "True", 2024-09-21T03:42:23.554 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.554 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.554 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.554 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "monitor device health metrics", 2024-09-21T03:42:23.554 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.554 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.554 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.554 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.554 INFO:teuthology.orchestra.run.smithi089.stdout: "log_level": { 2024-09-21T03:42:23.554 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_level", 2024-09-21T03:42:23.554 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:23.554 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.555 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:23.555 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-21T03:42:23.555 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.555 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.555 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-21T03:42:23.555 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2024-09-21T03:42:23.555 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2024-09-21T03:42:23.555 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2024-09-21T03:42:23.555 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2024-09-21T03:42:23.555 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2024-09-21T03:42:23.555 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2024-09-21T03:42:23.555 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-21T03:42:23.556 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.556 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.556 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.556 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.556 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.556 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster": { 2024-09-21T03:42:23.556 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster", 2024-09-21T03:42:23.556 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-21T03:42:23.556 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.556 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:23.556 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-21T03:42:23.556 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.556 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.556 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.557 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.557 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.557 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.557 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.557 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.557 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster_level": { 2024-09-21T03:42:23.557 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster_level", 2024-09-21T03:42:23.557 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:23.557 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.557 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:23.557 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "info", 2024-09-21T03:42:23.557 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.558 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.558 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-21T03:42:23.558 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2024-09-21T03:42:23.558 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2024-09-21T03:42:23.558 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2024-09-21T03:42:23.558 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2024-09-21T03:42:23.558 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2024-09-21T03:42:23.558 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2024-09-21T03:42:23.558 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-21T03:42:23.558 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.558 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.559 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.559 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.559 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.559 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_file": { 2024-09-21T03:42:23.559 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_file", 2024-09-21T03:42:23.559 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-21T03:42:23.559 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.559 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:23.559 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-21T03:42:23.559 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.559 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.559 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.560 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.560 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.560 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.560 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.560 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.560 INFO:teuthology.orchestra.run.smithi089.stdout: "mark_out_threshold": { 2024-09-21T03:42:23.560 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "mark_out_threshold", 2024-09-21T03:42:23.560 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "secs", 2024-09-21T03:42:23.560 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.560 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:23.560 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "2419200", 2024-09-21T03:42:23.560 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.561 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.561 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.561 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-09-21T03:42:23.561 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.561 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.561 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.561 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.561 INFO:teuthology.orchestra.run.smithi089.stdout: "pool_name": { 2024-09-21T03:42:23.561 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "pool_name", 2024-09-21T03:42:23.561 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:23.561 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.562 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:23.562 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "device_health_metrics", 2024-09-21T03:42:23.562 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.562 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.562 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.562 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "name of pool in which to store device health metrics", 2024-09-21T03:42:23.562 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.562 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.562 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.562 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.562 INFO:teuthology.orchestra.run.smithi089.stdout: "retention_period": { 2024-09-21T03:42:23.562 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "retention_period", 2024-09-21T03:42:23.563 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "secs", 2024-09-21T03:42:23.563 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.563 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:23.563 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "15552000", 2024-09-21T03:42:23.563 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.563 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.563 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.563 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "how long to retain device health metrics", 2024-09-21T03:42:23.563 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.563 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.563 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.564 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.564 INFO:teuthology.orchestra.run.smithi089.stdout: "scrape_frequency": { 2024-09-21T03:42:23.564 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "scrape_frequency", 2024-09-21T03:42:23.564 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "secs", 2024-09-21T03:42:23.564 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.564 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:23.564 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "86400", 2024-09-21T03:42:23.564 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.564 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.564 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.564 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "how frequently to scrape device health metrics", 2024-09-21T03:42:23.565 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.565 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.565 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.565 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.565 INFO:teuthology.orchestra.run.smithi089.stdout: "self_heal": { 2024-09-21T03:42:23.565 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "self_heal", 2024-09-21T03:42:23.565 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-21T03:42:23.565 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.565 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:23.565 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "True", 2024-09-21T03:42:23.565 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.565 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.566 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.566 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-09-21T03:42:23.566 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.566 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.566 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.566 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.566 INFO:teuthology.orchestra.run.smithi089.stdout: "sleep_interval": { 2024-09-21T03:42:23.566 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "sleep_interval", 2024-09-21T03:42:23.566 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "secs", 2024-09-21T03:42:23.566 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.567 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:23.567 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "600", 2024-09-21T03:42:23.567 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.567 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.567 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.567 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "how frequently to wake up and check device health", 2024-09-21T03:42:23.567 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.567 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.567 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.567 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.567 INFO:teuthology.orchestra.run.smithi089.stdout: "sqlite3_killpoint": { 2024-09-21T03:42:23.567 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:42:23.568 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2024-09-21T03:42:23.568 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "dev", 2024-09-21T03:42:23.568 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:23.568 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "0", 2024-09-21T03:42:23.568 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.568 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.568 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.568 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.568 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.568 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.568 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.568 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.569 INFO:teuthology.orchestra.run.smithi089.stdout: "warn_threshold": { 2024-09-21T03:42:23.569 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "warn_threshold", 2024-09-21T03:42:23.569 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "secs", 2024-09-21T03:42:23.569 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.569 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:23.569 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "7257600", 2024-09-21T03:42:23.569 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.569 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.569 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.569 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-09-21T03:42:23.569 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.569 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.570 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.570 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-21T03:42:23.570 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-21T03:42:23.570 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.570 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-21T03:42:23.570 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "diskprediction_local", 2024-09-21T03:42:23.570 INFO:teuthology.orchestra.run.smithi089.stdout: "can_run": true, 2024-09-21T03:42:23.570 INFO:teuthology.orchestra.run.smithi089.stdout: "error_string": "", 2024-09-21T03:42:23.570 INFO:teuthology.orchestra.run.smithi089.stdout: "module_options": { 2024-09-21T03:42:23.570 INFO:teuthology.orchestra.run.smithi089.stdout: "log_level": { 2024-09-21T03:42:23.570 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_level", 2024-09-21T03:42:23.570 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:23.570 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.571 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:23.571 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-21T03:42:23.571 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.571 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.571 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-21T03:42:23.571 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2024-09-21T03:42:23.571 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2024-09-21T03:42:23.571 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2024-09-21T03:42:23.571 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2024-09-21T03:42:23.571 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2024-09-21T03:42:23.571 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2024-09-21T03:42:23.571 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-21T03:42:23.571 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.572 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.572 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.572 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.572 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.572 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster": { 2024-09-21T03:42:23.572 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster", 2024-09-21T03:42:23.572 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-21T03:42:23.572 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.572 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:23.572 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-21T03:42:23.572 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.572 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.572 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.573 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.573 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.573 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.573 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.573 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.573 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster_level": { 2024-09-21T03:42:23.573 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster_level", 2024-09-21T03:42:23.573 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:23.573 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.573 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:23.573 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "info", 2024-09-21T03:42:23.573 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.573 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.573 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-21T03:42:23.574 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2024-09-21T03:42:23.574 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2024-09-21T03:42:23.574 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2024-09-21T03:42:23.574 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2024-09-21T03:42:23.574 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2024-09-21T03:42:23.574 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2024-09-21T03:42:23.574 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-21T03:42:23.574 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.574 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.574 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.574 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.575 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.575 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_file": { 2024-09-21T03:42:23.575 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_file", 2024-09-21T03:42:23.575 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-21T03:42:23.575 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.575 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:23.575 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-21T03:42:23.575 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.575 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.575 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.575 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.576 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.576 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.576 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.576 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.576 INFO:teuthology.orchestra.run.smithi089.stdout: "predict_interval": { 2024-09-21T03:42:23.576 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "predict_interval", 2024-09-21T03:42:23.576 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:23.576 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.576 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:23.576 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "86400", 2024-09-21T03:42:23.576 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.577 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.577 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.577 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.577 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.577 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.577 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.577 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.577 INFO:teuthology.orchestra.run.smithi089.stdout: "predictor_model": { 2024-09-21T03:42:23.577 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "predictor_model", 2024-09-21T03:42:23.577 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:23.577 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.577 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:23.578 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "prophetstor", 2024-09-21T03:42:23.578 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.578 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.578 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.578 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.578 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.578 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.578 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.578 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.578 INFO:teuthology.orchestra.run.smithi089.stdout: "sleep_interval": { 2024-09-21T03:42:23.578 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "sleep_interval", 2024-09-21T03:42:23.578 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:23.579 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.579 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:23.579 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "600", 2024-09-21T03:42:23.579 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.579 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.579 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.579 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.579 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.579 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.579 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.579 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.579 INFO:teuthology.orchestra.run.smithi089.stdout: "sqlite3_killpoint": { 2024-09-21T03:42:23.580 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:42:23.580 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2024-09-21T03:42:23.580 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "dev", 2024-09-21T03:42:23.580 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:23.580 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "0", 2024-09-21T03:42:23.580 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.580 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.580 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.580 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.580 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.580 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.580 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.581 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-21T03:42:23.581 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-21T03:42:23.581 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.581 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-21T03:42:23.581 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "influx", 2024-09-21T03:42:23.581 INFO:teuthology.orchestra.run.smithi089.stdout: "can_run": false, 2024-09-21T03:42:23.581 INFO:teuthology.orchestra.run.smithi089.stdout: "error_string": "influxdb python module not found", 2024-09-21T03:42:23.581 INFO:teuthology.orchestra.run.smithi089.stdout: "module_options": { 2024-09-21T03:42:23.581 INFO:teuthology.orchestra.run.smithi089.stdout: "batch_size": { 2024-09-21T03:42:23.581 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "batch_size", 2024-09-21T03:42:23.581 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2024-09-21T03:42:23.581 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.582 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:23.582 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "5000", 2024-09-21T03:42:23.582 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.582 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.582 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.582 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-09-21T03:42:23.582 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.582 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.582 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.582 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.582 INFO:teuthology.orchestra.run.smithi089.stdout: "database": { 2024-09-21T03:42:23.582 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "database", 2024-09-21T03:42:23.583 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:23.583 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.583 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:23.583 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "ceph", 2024-09-21T03:42:23.583 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.583 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.583 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.583 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-09-21T03:42:23.583 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.583 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.583 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.584 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.584 INFO:teuthology.orchestra.run.smithi089.stdout: "hostname": { 2024-09-21T03:42:23.584 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "hostname", 2024-09-21T03:42:23.584 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:23.584 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.584 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:23.584 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-21T03:42:23.584 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.584 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.584 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.584 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "InfluxDB server hostname", 2024-09-21T03:42:23.584 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.584 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.585 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.585 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.585 INFO:teuthology.orchestra.run.smithi089.stdout: "interval": { 2024-09-21T03:42:23.585 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "interval", 2024-09-21T03:42:23.585 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "secs", 2024-09-21T03:42:23.585 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.585 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:23.585 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "30", 2024-09-21T03:42:23.585 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "5", 2024-09-21T03:42:23.585 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.585 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.585 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-09-21T03:42:23.586 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.586 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.586 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.586 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.586 INFO:teuthology.orchestra.run.smithi089.stdout: "log_level": { 2024-09-21T03:42:23.586 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_level", 2024-09-21T03:42:23.586 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:23.586 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.586 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:23.586 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-21T03:42:23.586 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.586 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.587 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-21T03:42:23.587 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2024-09-21T03:42:23.587 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2024-09-21T03:42:23.587 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2024-09-21T03:42:23.587 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2024-09-21T03:42:23.587 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2024-09-21T03:42:23.587 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2024-09-21T03:42:23.587 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-21T03:42:23.587 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.587 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.587 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.588 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.588 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.588 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster": { 2024-09-21T03:42:23.588 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster", 2024-09-21T03:42:23.588 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-21T03:42:23.588 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.588 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:23.588 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-21T03:42:23.588 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.588 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.588 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.588 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.588 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.589 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.589 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.589 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.589 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster_level": { 2024-09-21T03:42:23.589 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster_level", 2024-09-21T03:42:23.589 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:23.589 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.589 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:23.589 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "info", 2024-09-21T03:42:23.589 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.589 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.589 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-21T03:42:23.590 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2024-09-21T03:42:23.590 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2024-09-21T03:42:23.590 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2024-09-21T03:42:23.590 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2024-09-21T03:42:23.590 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2024-09-21T03:42:23.590 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2024-09-21T03:42:23.590 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-21T03:42:23.590 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.590 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.590 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.590 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.591 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.591 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_file": { 2024-09-21T03:42:23.591 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_file", 2024-09-21T03:42:23.591 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-21T03:42:23.591 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.591 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:23.591 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-21T03:42:23.591 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.591 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.591 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.591 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.591 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.592 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.592 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.592 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.592 INFO:teuthology.orchestra.run.smithi089.stdout: "password": { 2024-09-21T03:42:23.592 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "password", 2024-09-21T03:42:23.592 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:23.592 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.592 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:23.592 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-21T03:42:23.592 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.592 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.592 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.593 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "password of InfluxDB server user", 2024-09-21T03:42:23.593 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.593 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.593 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.593 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.593 INFO:teuthology.orchestra.run.smithi089.stdout: "port": { 2024-09-21T03:42:23.593 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "port", 2024-09-21T03:42:23.593 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2024-09-21T03:42:23.593 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.593 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:23.593 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "8086", 2024-09-21T03:42:23.593 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.593 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.594 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.594 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "InfluxDB server port", 2024-09-21T03:42:23.594 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.594 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.594 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.594 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.594 INFO:teuthology.orchestra.run.smithi089.stdout: "sqlite3_killpoint": { 2024-09-21T03:42:23.594 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:42:23.594 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2024-09-21T03:42:23.594 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "dev", 2024-09-21T03:42:23.594 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:23.594 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "0", 2024-09-21T03:42:23.594 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.595 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.595 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.595 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.595 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.595 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.595 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.595 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.595 INFO:teuthology.orchestra.run.smithi089.stdout: "ssl": { 2024-09-21T03:42:23.595 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "ssl", 2024-09-21T03:42:23.595 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:23.595 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.595 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:23.596 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "false", 2024-09-21T03:42:23.596 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.596 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.596 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.596 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-09-21T03:42:23.596 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.596 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.596 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.596 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.596 INFO:teuthology.orchestra.run.smithi089.stdout: "threads": { 2024-09-21T03:42:23.596 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "threads", 2024-09-21T03:42:23.596 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2024-09-21T03:42:23.597 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.597 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:23.597 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "5", 2024-09-21T03:42:23.597 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "1", 2024-09-21T03:42:23.597 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "32", 2024-09-21T03:42:23.597 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.597 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-09-21T03:42:23.597 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.597 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.597 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.597 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.597 INFO:teuthology.orchestra.run.smithi089.stdout: "username": { 2024-09-21T03:42:23.598 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "username", 2024-09-21T03:42:23.598 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:23.598 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.598 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:23.598 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-21T03:42:23.598 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.598 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.598 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.598 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "username of InfluxDB server user", 2024-09-21T03:42:23.598 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.598 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.599 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.599 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.599 INFO:teuthology.orchestra.run.smithi089.stdout: "verify_ssl": { 2024-09-21T03:42:23.599 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "verify_ssl", 2024-09-21T03:42:23.599 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:23.599 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.599 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:23.599 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "true", 2024-09-21T03:42:23.599 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.599 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.599 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.599 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-09-21T03:42:23.600 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.600 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.600 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.600 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-21T03:42:23.600 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-21T03:42:23.600 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.600 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-21T03:42:23.600 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "insights", 2024-09-21T03:42:23.600 INFO:teuthology.orchestra.run.smithi089.stdout: "can_run": true, 2024-09-21T03:42:23.600 INFO:teuthology.orchestra.run.smithi089.stdout: "error_string": "", 2024-09-21T03:42:23.600 INFO:teuthology.orchestra.run.smithi089.stdout: "module_options": { 2024-09-21T03:42:23.600 INFO:teuthology.orchestra.run.smithi089.stdout: "log_level": { 2024-09-21T03:42:23.601 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_level", 2024-09-21T03:42:23.601 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:23.601 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.601 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:23.601 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-21T03:42:23.601 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.601 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.601 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-21T03:42:23.601 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2024-09-21T03:42:23.601 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2024-09-21T03:42:23.601 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2024-09-21T03:42:23.601 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2024-09-21T03:42:23.602 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2024-09-21T03:42:23.602 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2024-09-21T03:42:23.602 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-21T03:42:23.602 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.602 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.602 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.602 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.602 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.602 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster": { 2024-09-21T03:42:23.602 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster", 2024-09-21T03:42:23.602 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-21T03:42:23.602 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.603 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:23.603 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-21T03:42:23.603 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.603 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.603 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.603 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.603 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.603 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.603 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.603 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.603 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster_level": { 2024-09-21T03:42:23.603 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster_level", 2024-09-21T03:42:23.604 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:23.604 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.604 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:23.604 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "info", 2024-09-21T03:42:23.604 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.604 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.604 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-21T03:42:23.604 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2024-09-21T03:42:23.604 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2024-09-21T03:42:23.604 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2024-09-21T03:42:23.604 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2024-09-21T03:42:23.604 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2024-09-21T03:42:23.605 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2024-09-21T03:42:23.605 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-21T03:42:23.605 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.605 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.605 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.605 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.605 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.605 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_file": { 2024-09-21T03:42:23.605 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_file", 2024-09-21T03:42:23.605 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-21T03:42:23.605 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.605 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:23.605 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-21T03:42:23.606 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.606 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.606 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.606 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.606 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.606 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.606 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.606 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.606 INFO:teuthology.orchestra.run.smithi089.stdout: "sqlite3_killpoint": { 2024-09-21T03:42:23.606 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:42:23.606 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2024-09-21T03:42:23.606 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "dev", 2024-09-21T03:42:23.606 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:23.607 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "0", 2024-09-21T03:42:23.607 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.607 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.607 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.607 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.607 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.607 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.607 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.607 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-21T03:42:23.607 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-21T03:42:23.607 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.607 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-21T03:42:23.608 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "iostat", 2024-09-21T03:42:23.608 INFO:teuthology.orchestra.run.smithi089.stdout: "can_run": true, 2024-09-21T03:42:23.608 INFO:teuthology.orchestra.run.smithi089.stdout: "error_string": "", 2024-09-21T03:42:23.608 INFO:teuthology.orchestra.run.smithi089.stdout: "module_options": { 2024-09-21T03:42:23.608 INFO:teuthology.orchestra.run.smithi089.stdout: "log_level": { 2024-09-21T03:42:23.608 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_level", 2024-09-21T03:42:23.608 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:23.608 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.608 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:23.608 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-21T03:42:23.608 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.608 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.608 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-21T03:42:23.609 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2024-09-21T03:42:23.609 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2024-09-21T03:42:23.609 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2024-09-21T03:42:23.609 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2024-09-21T03:42:23.609 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2024-09-21T03:42:23.609 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2024-09-21T03:42:23.609 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-21T03:42:23.609 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.609 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.609 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.609 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.609 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.609 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster": { 2024-09-21T03:42:23.609 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster", 2024-09-21T03:42:23.610 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-21T03:42:23.610 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.610 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:23.610 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-21T03:42:23.610 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.610 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.610 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.610 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.610 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.610 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.610 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.610 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.610 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster_level": { 2024-09-21T03:42:23.611 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster_level", 2024-09-21T03:42:23.611 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:23.611 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.611 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:23.611 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "info", 2024-09-21T03:42:23.611 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.611 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.611 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-21T03:42:23.611 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2024-09-21T03:42:23.611 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2024-09-21T03:42:23.611 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2024-09-21T03:42:23.611 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2024-09-21T03:42:23.611 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2024-09-21T03:42:23.611 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2024-09-21T03:42:23.612 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-21T03:42:23.612 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.612 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.612 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.612 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.612 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.612 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_file": { 2024-09-21T03:42:23.612 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_file", 2024-09-21T03:42:23.612 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-21T03:42:23.612 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.612 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:23.612 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-21T03:42:23.612 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.612 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.613 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.613 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.613 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.613 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.613 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.613 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.613 INFO:teuthology.orchestra.run.smithi089.stdout: "sqlite3_killpoint": { 2024-09-21T03:42:23.613 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:42:23.613 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2024-09-21T03:42:23.613 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "dev", 2024-09-21T03:42:23.613 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:23.613 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "0", 2024-09-21T03:42:23.613 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.614 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.614 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.614 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.614 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.614 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.614 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.614 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-21T03:42:23.614 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-21T03:42:23.614 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.614 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-21T03:42:23.615 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "k8sevents", 2024-09-21T03:42:23.615 INFO:teuthology.orchestra.run.smithi089.stdout: "can_run": true, 2024-09-21T03:42:23.615 INFO:teuthology.orchestra.run.smithi089.stdout: "error_string": "", 2024-09-21T03:42:23.615 INFO:teuthology.orchestra.run.smithi089.stdout: "module_options": { 2024-09-21T03:42:23.615 INFO:teuthology.orchestra.run.smithi089.stdout: "ceph_event_retention_days": { 2024-09-21T03:42:23.615 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "ceph_event_retention_days", 2024-09-21T03:42:23.615 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2024-09-21T03:42:23.615 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.615 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:23.615 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "7", 2024-09-21T03:42:23.615 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.615 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.615 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.616 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Days to hold ceph event information within local cache", 2024-09-21T03:42:23.616 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.616 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.616 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.616 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.616 INFO:teuthology.orchestra.run.smithi089.stdout: "config_check_secs": { 2024-09-21T03:42:23.616 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "config_check_secs", 2024-09-21T03:42:23.616 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2024-09-21T03:42:23.616 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.616 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:23.616 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "10", 2024-09-21T03:42:23.616 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "10", 2024-09-21T03:42:23.616 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.617 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.617 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-09-21T03:42:23.617 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.617 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.617 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.617 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.617 INFO:teuthology.orchestra.run.smithi089.stdout: "log_level": { 2024-09-21T03:42:23.617 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_level", 2024-09-21T03:42:23.617 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:23.617 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.617 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:23.617 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-21T03:42:23.618 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.618 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.618 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-21T03:42:23.618 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2024-09-21T03:42:23.618 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2024-09-21T03:42:23.618 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2024-09-21T03:42:23.618 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2024-09-21T03:42:23.618 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2024-09-21T03:42:23.618 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2024-09-21T03:42:23.618 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-21T03:42:23.618 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.618 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.618 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.619 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.619 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.619 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster": { 2024-09-21T03:42:23.619 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster", 2024-09-21T03:42:23.619 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-21T03:42:23.619 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.619 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:23.619 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-21T03:42:23.619 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.619 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.619 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.619 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.619 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.619 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.620 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.620 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.620 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster_level": { 2024-09-21T03:42:23.620 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster_level", 2024-09-21T03:42:23.620 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:23.620 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.620 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:23.620 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "info", 2024-09-21T03:42:23.620 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.620 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.620 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-21T03:42:23.620 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2024-09-21T03:42:23.620 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2024-09-21T03:42:23.620 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2024-09-21T03:42:23.620 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2024-09-21T03:42:23.621 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2024-09-21T03:42:23.621 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2024-09-21T03:42:23.621 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-21T03:42:23.621 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.621 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.621 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.621 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.621 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.621 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_file": { 2024-09-21T03:42:23.621 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_file", 2024-09-21T03:42:23.621 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-21T03:42:23.621 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.621 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:23.622 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-21T03:42:23.622 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.622 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.622 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.622 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.622 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.622 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.622 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.622 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.622 INFO:teuthology.orchestra.run.smithi089.stdout: "sqlite3_killpoint": { 2024-09-21T03:42:23.622 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:42:23.622 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2024-09-21T03:42:23.622 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "dev", 2024-09-21T03:42:23.623 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:23.623 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "0", 2024-09-21T03:42:23.623 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.623 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.623 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.623 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.623 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.623 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.623 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.623 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-21T03:42:23.623 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-21T03:42:23.623 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.623 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-21T03:42:23.624 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "localpool", 2024-09-21T03:42:23.624 INFO:teuthology.orchestra.run.smithi089.stdout: "can_run": true, 2024-09-21T03:42:23.624 INFO:teuthology.orchestra.run.smithi089.stdout: "error_string": "", 2024-09-21T03:42:23.624 INFO:teuthology.orchestra.run.smithi089.stdout: "module_options": { 2024-09-21T03:42:23.624 INFO:teuthology.orchestra.run.smithi089.stdout: "failure_domain": { 2024-09-21T03:42:23.624 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "failure_domain", 2024-09-21T03:42:23.624 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:23.624 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.624 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:23.624 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "host", 2024-09-21T03:42:23.624 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.624 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.625 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.625 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "failure domain for any created local pool", 2024-09-21T03:42:23.625 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-09-21T03:42:23.625 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.625 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.625 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.625 INFO:teuthology.orchestra.run.smithi089.stdout: "log_level": { 2024-09-21T03:42:23.625 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_level", 2024-09-21T03:42:23.625 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:23.625 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.625 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:23.626 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-21T03:42:23.626 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.626 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.626 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-21T03:42:23.626 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2024-09-21T03:42:23.626 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2024-09-21T03:42:23.626 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2024-09-21T03:42:23.626 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2024-09-21T03:42:23.626 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2024-09-21T03:42:23.626 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2024-09-21T03:42:23.626 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-21T03:42:23.627 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.627 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.627 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.627 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.627 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.627 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster": { 2024-09-21T03:42:23.627 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster", 2024-09-21T03:42:23.627 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-21T03:42:23.627 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.627 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:23.627 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-21T03:42:23.627 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.627 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.627 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.628 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.628 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.628 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.628 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.628 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.628 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster_level": { 2024-09-21T03:42:23.628 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster_level", 2024-09-21T03:42:23.628 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:23.628 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.628 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:23.628 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "info", 2024-09-21T03:42:23.628 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.628 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.628 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-21T03:42:23.629 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2024-09-21T03:42:23.629 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2024-09-21T03:42:23.629 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2024-09-21T03:42:23.629 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2024-09-21T03:42:23.629 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2024-09-21T03:42:23.629 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2024-09-21T03:42:23.629 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-21T03:42:23.629 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.629 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.629 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.629 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.629 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.629 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_file": { 2024-09-21T03:42:23.630 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_file", 2024-09-21T03:42:23.630 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-21T03:42:23.630 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.630 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:23.630 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-21T03:42:23.630 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.630 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.630 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.630 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.630 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.630 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.630 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.631 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.631 INFO:teuthology.orchestra.run.smithi089.stdout: "min_size": { 2024-09-21T03:42:23.631 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "min_size", 2024-09-21T03:42:23.631 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2024-09-21T03:42:23.631 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.631 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:23.631 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-21T03:42:23.631 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.631 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.631 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.631 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "default min_size for any created local pool", 2024-09-21T03:42:23.631 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-09-21T03:42:23.632 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.632 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.632 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.632 INFO:teuthology.orchestra.run.smithi089.stdout: "num_rep": { 2024-09-21T03:42:23.632 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "num_rep", 2024-09-21T03:42:23.632 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2024-09-21T03:42:23.632 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.632 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:23.632 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "3", 2024-09-21T03:42:23.632 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.632 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.633 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.633 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "default replica count for any created local pool", 2024-09-21T03:42:23.633 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.633 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.633 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.633 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.633 INFO:teuthology.orchestra.run.smithi089.stdout: "pg_num": { 2024-09-21T03:42:23.633 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "pg_num", 2024-09-21T03:42:23.633 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2024-09-21T03:42:23.633 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.633 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:23.633 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "128", 2024-09-21T03:42:23.634 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.634 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.634 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.634 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "default pg_num for any created local pool", 2024-09-21T03:42:23.634 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.634 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.634 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.634 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.634 INFO:teuthology.orchestra.run.smithi089.stdout: "prefix": { 2024-09-21T03:42:23.634 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "prefix", 2024-09-21T03:42:23.634 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:23.634 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.635 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:23.635 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-21T03:42:23.635 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.635 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.635 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.635 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "name prefix for any created local pool", 2024-09-21T03:42:23.635 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.635 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.635 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.635 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.635 INFO:teuthology.orchestra.run.smithi089.stdout: "sqlite3_killpoint": { 2024-09-21T03:42:23.635 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:42:23.636 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2024-09-21T03:42:23.636 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "dev", 2024-09-21T03:42:23.636 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:23.636 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "0", 2024-09-21T03:42:23.636 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.636 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.636 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.636 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.636 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.636 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.636 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.636 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.636 INFO:teuthology.orchestra.run.smithi089.stdout: "subtree": { 2024-09-21T03:42:23.637 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "subtree", 2024-09-21T03:42:23.637 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:23.637 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.637 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:23.637 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "rack", 2024-09-21T03:42:23.637 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.637 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.637 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.637 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "CRUSH level for which to create a local pool", 2024-09-21T03:42:23.637 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-09-21T03:42:23.637 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.637 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.637 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-21T03:42:23.638 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-21T03:42:23.638 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.638 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-21T03:42:23.638 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "mds_autoscaler", 2024-09-21T03:42:23.638 INFO:teuthology.orchestra.run.smithi089.stdout: "can_run": true, 2024-09-21T03:42:23.638 INFO:teuthology.orchestra.run.smithi089.stdout: "error_string": "", 2024-09-21T03:42:23.638 INFO:teuthology.orchestra.run.smithi089.stdout: "module_options": { 2024-09-21T03:42:23.638 INFO:teuthology.orchestra.run.smithi089.stdout: "log_level": { 2024-09-21T03:42:23.638 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_level", 2024-09-21T03:42:23.638 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:23.638 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.639 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:23.639 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-21T03:42:23.639 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.639 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.639 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-21T03:42:23.639 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2024-09-21T03:42:23.639 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2024-09-21T03:42:23.639 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2024-09-21T03:42:23.639 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2024-09-21T03:42:23.639 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2024-09-21T03:42:23.639 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2024-09-21T03:42:23.639 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-21T03:42:23.639 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.640 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.640 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.640 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.640 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.640 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster": { 2024-09-21T03:42:23.640 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster", 2024-09-21T03:42:23.640 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-21T03:42:23.640 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.640 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:23.640 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-21T03:42:23.640 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.640 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.641 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.641 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.641 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.641 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.641 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.641 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.641 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster_level": { 2024-09-21T03:42:23.641 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster_level", 2024-09-21T03:42:23.641 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:23.641 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.641 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:23.641 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "info", 2024-09-21T03:42:23.641 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.642 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.642 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-21T03:42:23.642 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2024-09-21T03:42:23.642 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2024-09-21T03:42:23.642 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2024-09-21T03:42:23.642 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2024-09-21T03:42:23.642 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2024-09-21T03:42:23.642 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2024-09-21T03:42:23.642 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-21T03:42:23.642 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.643 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.643 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.643 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.643 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.643 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_file": { 2024-09-21T03:42:23.643 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_file", 2024-09-21T03:42:23.643 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-21T03:42:23.643 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.643 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:23.643 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-21T03:42:23.643 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.643 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.644 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.644 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.644 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.644 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.644 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.644 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.644 INFO:teuthology.orchestra.run.smithi089.stdout: "sqlite3_killpoint": { 2024-09-21T03:42:23.644 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:42:23.644 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2024-09-21T03:42:23.644 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "dev", 2024-09-21T03:42:23.644 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:23.644 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "0", 2024-09-21T03:42:23.644 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.644 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.645 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.645 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.645 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.645 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.645 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.645 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-21T03:42:23.645 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-21T03:42:23.645 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.645 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-21T03:42:23.645 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "mirroring", 2024-09-21T03:42:23.645 INFO:teuthology.orchestra.run.smithi089.stdout: "can_run": true, 2024-09-21T03:42:23.645 INFO:teuthology.orchestra.run.smithi089.stdout: "error_string": "", 2024-09-21T03:42:23.646 INFO:teuthology.orchestra.run.smithi089.stdout: "module_options": { 2024-09-21T03:42:23.646 INFO:teuthology.orchestra.run.smithi089.stdout: "log_level": { 2024-09-21T03:42:23.646 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_level", 2024-09-21T03:42:23.646 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:23.646 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.646 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:23.646 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-21T03:42:23.646 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.646 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.646 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-21T03:42:23.646 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2024-09-21T03:42:23.646 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2024-09-21T03:42:23.647 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2024-09-21T03:42:23.647 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2024-09-21T03:42:23.647 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2024-09-21T03:42:23.647 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2024-09-21T03:42:23.647 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-21T03:42:23.647 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.647 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.647 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.647 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.647 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.647 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster": { 2024-09-21T03:42:23.648 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster", 2024-09-21T03:42:23.648 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-21T03:42:23.648 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.648 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:23.648 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-21T03:42:23.648 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.648 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.648 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.648 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.648 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.648 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.648 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.648 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.649 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster_level": { 2024-09-21T03:42:23.649 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster_level", 2024-09-21T03:42:23.649 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:23.649 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.649 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:23.649 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "info", 2024-09-21T03:42:23.649 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.649 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.649 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-21T03:42:23.649 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2024-09-21T03:42:23.649 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2024-09-21T03:42:23.649 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2024-09-21T03:42:23.650 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2024-09-21T03:42:23.650 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2024-09-21T03:42:23.650 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2024-09-21T03:42:23.650 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-21T03:42:23.650 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.650 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.650 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.650 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.650 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.650 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_file": { 2024-09-21T03:42:23.650 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_file", 2024-09-21T03:42:23.650 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-21T03:42:23.651 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.651 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:23.651 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-21T03:42:23.651 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.651 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.651 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.651 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.651 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.651 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.651 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.651 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.651 INFO:teuthology.orchestra.run.smithi089.stdout: "sqlite3_killpoint": { 2024-09-21T03:42:23.652 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:42:23.652 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2024-09-21T03:42:23.652 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "dev", 2024-09-21T03:42:23.652 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:23.652 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "0", 2024-09-21T03:42:23.652 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.652 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.652 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.652 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.652 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.652 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.652 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.652 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-21T03:42:23.653 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-21T03:42:23.653 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.653 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-21T03:42:23.653 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "nfs", 2024-09-21T03:42:23.653 INFO:teuthology.orchestra.run.smithi089.stdout: "can_run": true, 2024-09-21T03:42:23.653 INFO:teuthology.orchestra.run.smithi089.stdout: "error_string": "", 2024-09-21T03:42:23.653 INFO:teuthology.orchestra.run.smithi089.stdout: "module_options": { 2024-09-21T03:42:23.653 INFO:teuthology.orchestra.run.smithi089.stdout: "log_level": { 2024-09-21T03:42:23.653 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_level", 2024-09-21T03:42:23.653 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:23.653 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.653 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:23.653 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-21T03:42:23.654 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.654 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.654 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-21T03:42:23.654 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2024-09-21T03:42:23.654 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2024-09-21T03:42:23.654 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2024-09-21T03:42:23.654 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2024-09-21T03:42:23.654 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2024-09-21T03:42:23.654 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2024-09-21T03:42:23.654 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-21T03:42:23.654 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.654 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.655 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.655 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.655 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.655 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster": { 2024-09-21T03:42:23.655 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster", 2024-09-21T03:42:23.655 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-21T03:42:23.655 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.655 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:23.655 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-21T03:42:23.655 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.655 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.655 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.656 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.656 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.656 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.656 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.656 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.656 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster_level": { 2024-09-21T03:42:23.656 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster_level", 2024-09-21T03:42:23.656 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:23.656 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.656 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:23.656 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "info", 2024-09-21T03:42:23.657 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.657 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.657 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-21T03:42:23.657 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2024-09-21T03:42:23.657 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2024-09-21T03:42:23.657 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2024-09-21T03:42:23.657 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2024-09-21T03:42:23.657 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2024-09-21T03:42:23.657 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2024-09-21T03:42:23.657 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-21T03:42:23.657 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.657 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.657 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.658 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.658 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.658 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_file": { 2024-09-21T03:42:23.658 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_file", 2024-09-21T03:42:23.658 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-21T03:42:23.658 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.658 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:23.658 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-21T03:42:23.658 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.658 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.658 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.658 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.658 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.659 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.659 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.659 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.659 INFO:teuthology.orchestra.run.smithi089.stdout: "sqlite3_killpoint": { 2024-09-21T03:42:23.659 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:42:23.659 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2024-09-21T03:42:23.659 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "dev", 2024-09-21T03:42:23.659 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:23.659 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "0", 2024-09-21T03:42:23.659 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.659 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.659 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.659 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.660 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.660 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.660 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.660 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-21T03:42:23.660 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-21T03:42:23.660 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.660 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-21T03:42:23.660 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "orchestrator", 2024-09-21T03:42:23.660 INFO:teuthology.orchestra.run.smithi089.stdout: "can_run": true, 2024-09-21T03:42:23.660 INFO:teuthology.orchestra.run.smithi089.stdout: "error_string": "", 2024-09-21T03:42:23.660 INFO:teuthology.orchestra.run.smithi089.stdout: "module_options": { 2024-09-21T03:42:23.660 INFO:teuthology.orchestra.run.smithi089.stdout: "fail_fs": { 2024-09-21T03:42:23.660 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "fail_fs", 2024-09-21T03:42:23.661 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-21T03:42:23.661 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.661 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:23.661 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-21T03:42:23.661 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.661 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.661 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.661 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-09-21T03:42:23.661 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.661 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.661 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.661 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.661 INFO:teuthology.orchestra.run.smithi089.stdout: "log_level": { 2024-09-21T03:42:23.662 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_level", 2024-09-21T03:42:23.662 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:23.662 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.662 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:23.662 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-21T03:42:23.662 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.662 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.662 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-21T03:42:23.662 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2024-09-21T03:42:23.662 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2024-09-21T03:42:23.662 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2024-09-21T03:42:23.662 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2024-09-21T03:42:23.662 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2024-09-21T03:42:23.663 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2024-09-21T03:42:23.663 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-21T03:42:23.663 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.663 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.663 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.663 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.663 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.663 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster": { 2024-09-21T03:42:23.663 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster", 2024-09-21T03:42:23.663 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-21T03:42:23.663 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.663 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:23.664 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-21T03:42:23.664 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.664 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.664 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.664 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.664 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.664 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.664 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.664 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.664 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster_level": { 2024-09-21T03:42:23.664 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster_level", 2024-09-21T03:42:23.665 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:23.665 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.665 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:23.665 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "info", 2024-09-21T03:42:23.665 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.665 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.665 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-21T03:42:23.665 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2024-09-21T03:42:23.665 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2024-09-21T03:42:23.665 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2024-09-21T03:42:23.665 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2024-09-21T03:42:23.665 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2024-09-21T03:42:23.665 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2024-09-21T03:42:23.666 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-21T03:42:23.666 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.666 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.666 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.666 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.666 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.666 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_file": { 2024-09-21T03:42:23.666 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_file", 2024-09-21T03:42:23.666 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-21T03:42:23.666 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.666 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:23.666 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-21T03:42:23.667 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.667 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.667 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.667 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.667 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.667 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.667 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.667 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.667 INFO:teuthology.orchestra.run.smithi089.stdout: "orchestrator": { 2024-09-21T03:42:23.667 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "orchestrator", 2024-09-21T03:42:23.667 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:23.667 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.667 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:23.667 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-21T03:42:23.668 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.668 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.668 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-21T03:42:23.668 INFO:teuthology.orchestra.run.smithi089.stdout: "cephadm", 2024-09-21T03:42:23.668 INFO:teuthology.orchestra.run.smithi089.stdout: "rook", 2024-09-21T03:42:23.668 INFO:teuthology.orchestra.run.smithi089.stdout: "test_orchestrator" 2024-09-21T03:42:23.668 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-21T03:42:23.668 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Orchestrator backend", 2024-09-21T03:42:23.668 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.668 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.668 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.668 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.669 INFO:teuthology.orchestra.run.smithi089.stdout: "sqlite3_killpoint": { 2024-09-21T03:42:23.669 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:42:23.669 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2024-09-21T03:42:23.669 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "dev", 2024-09-21T03:42:23.669 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:23.669 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "0", 2024-09-21T03:42:23.669 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.669 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.669 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.669 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.669 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.669 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.669 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.670 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-21T03:42:23.670 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-21T03:42:23.670 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.670 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-21T03:42:23.670 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "osd_perf_query", 2024-09-21T03:42:23.670 INFO:teuthology.orchestra.run.smithi089.stdout: "can_run": true, 2024-09-21T03:42:23.670 INFO:teuthology.orchestra.run.smithi089.stdout: "error_string": "", 2024-09-21T03:42:23.670 INFO:teuthology.orchestra.run.smithi089.stdout: "module_options": { 2024-09-21T03:42:23.670 INFO:teuthology.orchestra.run.smithi089.stdout: "log_level": { 2024-09-21T03:42:23.670 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_level", 2024-09-21T03:42:23.670 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:23.670 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.670 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:23.671 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-21T03:42:23.671 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.671 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.671 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-21T03:42:23.671 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2024-09-21T03:42:23.671 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2024-09-21T03:42:23.671 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2024-09-21T03:42:23.671 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2024-09-21T03:42:23.671 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2024-09-21T03:42:23.671 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2024-09-21T03:42:23.671 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-21T03:42:23.671 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.672 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.672 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.672 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.672 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.672 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster": { 2024-09-21T03:42:23.672 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster", 2024-09-21T03:42:23.672 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-21T03:42:23.672 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.672 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:23.672 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-21T03:42:23.672 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.672 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.672 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.673 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.673 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.673 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.673 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.673 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.673 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster_level": { 2024-09-21T03:42:23.673 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster_level", 2024-09-21T03:42:23.673 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:23.673 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.673 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:23.673 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "info", 2024-09-21T03:42:23.673 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.673 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.674 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-21T03:42:23.674 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2024-09-21T03:42:23.674 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2024-09-21T03:42:23.674 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2024-09-21T03:42:23.674 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2024-09-21T03:42:23.674 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2024-09-21T03:42:23.674 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2024-09-21T03:42:23.674 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-21T03:42:23.674 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.674 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.674 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.674 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.674 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.675 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_file": { 2024-09-21T03:42:23.675 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_file", 2024-09-21T03:42:23.675 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-21T03:42:23.675 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.675 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:23.675 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-21T03:42:23.675 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.675 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.675 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.675 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.675 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.676 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.676 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.676 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.676 INFO:teuthology.orchestra.run.smithi089.stdout: "sqlite3_killpoint": { 2024-09-21T03:42:23.676 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:42:23.676 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2024-09-21T03:42:23.676 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "dev", 2024-09-21T03:42:23.676 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:23.676 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "0", 2024-09-21T03:42:23.676 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.676 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.676 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.676 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.677 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.677 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.677 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.677 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-21T03:42:23.677 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-21T03:42:23.677 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.677 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-21T03:42:23.677 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "osd_support", 2024-09-21T03:42:23.677 INFO:teuthology.orchestra.run.smithi089.stdout: "can_run": true, 2024-09-21T03:42:23.677 INFO:teuthology.orchestra.run.smithi089.stdout: "error_string": "", 2024-09-21T03:42:23.677 INFO:teuthology.orchestra.run.smithi089.stdout: "module_options": { 2024-09-21T03:42:23.677 INFO:teuthology.orchestra.run.smithi089.stdout: "log_level": { 2024-09-21T03:42:23.677 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_level", 2024-09-21T03:42:23.678 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:23.678 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.678 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:23.678 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-21T03:42:23.678 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.678 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.678 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-21T03:42:23.678 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2024-09-21T03:42:23.678 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2024-09-21T03:42:23.678 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2024-09-21T03:42:23.679 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2024-09-21T03:42:23.679 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2024-09-21T03:42:23.679 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2024-09-21T03:42:23.679 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-21T03:42:23.679 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.679 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.679 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.679 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.679 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.679 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster": { 2024-09-21T03:42:23.679 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster", 2024-09-21T03:42:23.679 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-21T03:42:23.679 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.680 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:23.680 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-21T03:42:23.680 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.680 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.680 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.680 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.680 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.680 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.680 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.680 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.680 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster_level": { 2024-09-21T03:42:23.680 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster_level", 2024-09-21T03:42:23.680 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:23.681 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.681 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:23.681 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "info", 2024-09-21T03:42:23.681 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.681 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.681 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-21T03:42:23.681 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2024-09-21T03:42:23.681 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2024-09-21T03:42:23.681 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2024-09-21T03:42:23.681 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2024-09-21T03:42:23.681 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2024-09-21T03:42:23.681 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2024-09-21T03:42:23.681 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-21T03:42:23.681 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.682 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.682 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.682 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.682 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.682 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_file": { 2024-09-21T03:42:23.682 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_file", 2024-09-21T03:42:23.682 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-21T03:42:23.682 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.682 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:23.682 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-21T03:42:23.682 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.682 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.682 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.683 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.683 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.683 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.683 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.683 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.683 INFO:teuthology.orchestra.run.smithi089.stdout: "sqlite3_killpoint": { 2024-09-21T03:42:23.683 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:42:23.683 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2024-09-21T03:42:23.683 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "dev", 2024-09-21T03:42:23.683 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:23.683 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "0", 2024-09-21T03:42:23.683 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.684 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.684 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.684 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.684 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.684 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.684 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.684 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-21T03:42:23.684 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-21T03:42:23.684 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.684 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-21T03:42:23.684 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "pg_autoscaler", 2024-09-21T03:42:23.684 INFO:teuthology.orchestra.run.smithi089.stdout: "can_run": true, 2024-09-21T03:42:23.684 INFO:teuthology.orchestra.run.smithi089.stdout: "error_string": "", 2024-09-21T03:42:23.685 INFO:teuthology.orchestra.run.smithi089.stdout: "module_options": { 2024-09-21T03:42:23.685 INFO:teuthology.orchestra.run.smithi089.stdout: "log_level": { 2024-09-21T03:42:23.685 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_level", 2024-09-21T03:42:23.685 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:23.685 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.685 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:23.685 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-21T03:42:23.685 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.685 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.685 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-21T03:42:23.685 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2024-09-21T03:42:23.685 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2024-09-21T03:42:23.686 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2024-09-21T03:42:23.686 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2024-09-21T03:42:23.686 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2024-09-21T03:42:23.686 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2024-09-21T03:42:23.686 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-21T03:42:23.686 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.686 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.686 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.686 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.686 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.686 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster": { 2024-09-21T03:42:23.686 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster", 2024-09-21T03:42:23.687 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-21T03:42:23.687 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.687 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:23.687 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-21T03:42:23.687 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.687 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.687 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.687 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.687 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.687 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.687 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.687 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.687 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster_level": { 2024-09-21T03:42:23.687 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster_level", 2024-09-21T03:42:23.688 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:23.688 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.688 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:23.688 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "info", 2024-09-21T03:42:23.688 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.688 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.688 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-21T03:42:23.688 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2024-09-21T03:42:23.688 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2024-09-21T03:42:23.688 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2024-09-21T03:42:23.688 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2024-09-21T03:42:23.688 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2024-09-21T03:42:23.688 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2024-09-21T03:42:23.688 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-21T03:42:23.689 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.689 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.689 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.689 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.689 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.689 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_file": { 2024-09-21T03:42:23.689 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_file", 2024-09-21T03:42:23.689 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-21T03:42:23.689 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.689 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:23.689 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-21T03:42:23.689 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.689 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.690 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.690 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.690 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.690 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.690 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.690 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.690 INFO:teuthology.orchestra.run.smithi089.stdout: "sleep_interval": { 2024-09-21T03:42:23.690 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "sleep_interval", 2024-09-21T03:42:23.690 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "secs", 2024-09-21T03:42:23.690 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.690 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:23.690 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "60", 2024-09-21T03:42:23.690 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.690 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.691 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.691 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.691 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.691 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.691 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.691 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.691 INFO:teuthology.orchestra.run.smithi089.stdout: "sqlite3_killpoint": { 2024-09-21T03:42:23.691 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:42:23.691 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2024-09-21T03:42:23.691 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "dev", 2024-09-21T03:42:23.691 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:23.691 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "0", 2024-09-21T03:42:23.691 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.692 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.692 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.692 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.692 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.692 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.692 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.692 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.692 INFO:teuthology.orchestra.run.smithi089.stdout: "threshold": { 2024-09-21T03:42:23.692 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "threshold", 2024-09-21T03:42:23.692 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "float", 2024-09-21T03:42:23.692 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.692 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:23.692 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "3.0", 2024-09-21T03:42:23.692 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "1.0", 2024-09-21T03:42:23.693 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.693 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.693 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "scaling threshold", 2024-09-21T03:42:23.693 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-09-21T03:42:23.693 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.693 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.693 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-21T03:42:23.693 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-21T03:42:23.693 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.693 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-21T03:42:23.693 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "progress", 2024-09-21T03:42:23.693 INFO:teuthology.orchestra.run.smithi089.stdout: "can_run": true, 2024-09-21T03:42:23.693 INFO:teuthology.orchestra.run.smithi089.stdout: "error_string": "", 2024-09-21T03:42:23.693 INFO:teuthology.orchestra.run.smithi089.stdout: "module_options": { 2024-09-21T03:42:23.694 INFO:teuthology.orchestra.run.smithi089.stdout: "allow_pg_recovery_event": { 2024-09-21T03:42:23.694 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "allow_pg_recovery_event", 2024-09-21T03:42:23.694 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-21T03:42:23.694 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.694 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:23.694 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-21T03:42:23.694 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.694 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.694 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.694 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "allow the module to show pg recovery progress", 2024-09-21T03:42:23.694 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.694 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.695 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.695 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.695 INFO:teuthology.orchestra.run.smithi089.stdout: "enabled": { 2024-09-21T03:42:23.695 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "enabled", 2024-09-21T03:42:23.695 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-21T03:42:23.695 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.695 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:23.695 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "True", 2024-09-21T03:42:23.695 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.695 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.695 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.695 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.696 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.696 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.696 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.696 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.696 INFO:teuthology.orchestra.run.smithi089.stdout: "log_level": { 2024-09-21T03:42:23.696 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_level", 2024-09-21T03:42:23.696 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:23.696 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.696 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:23.696 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-21T03:42:23.696 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.696 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.696 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-21T03:42:23.697 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2024-09-21T03:42:23.697 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2024-09-21T03:42:23.697 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2024-09-21T03:42:23.697 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2024-09-21T03:42:23.697 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2024-09-21T03:42:23.697 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2024-09-21T03:42:23.697 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-21T03:42:23.697 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.697 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.697 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.697 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.697 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.697 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster": { 2024-09-21T03:42:23.697 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster", 2024-09-21T03:42:23.698 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-21T03:42:23.698 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.698 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:23.698 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-21T03:42:23.698 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.698 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.698 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.698 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.698 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.698 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.698 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.698 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.698 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster_level": { 2024-09-21T03:42:23.698 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster_level", 2024-09-21T03:42:23.699 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:23.699 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.699 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:23.699 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "info", 2024-09-21T03:42:23.699 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.699 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.699 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-21T03:42:23.699 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2024-09-21T03:42:23.699 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2024-09-21T03:42:23.699 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2024-09-21T03:42:23.699 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2024-09-21T03:42:23.699 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2024-09-21T03:42:23.699 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2024-09-21T03:42:23.699 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-21T03:42:23.700 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.700 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.700 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.700 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.700 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.700 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_file": { 2024-09-21T03:42:23.700 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_file", 2024-09-21T03:42:23.700 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-21T03:42:23.700 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.700 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:23.700 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-21T03:42:23.700 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.700 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.701 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.701 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.701 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.701 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.701 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.701 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.701 INFO:teuthology.orchestra.run.smithi089.stdout: "max_completed_events": { 2024-09-21T03:42:23.701 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "max_completed_events", 2024-09-21T03:42:23.701 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2024-09-21T03:42:23.701 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.701 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:23.701 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "50", 2024-09-21T03:42:23.701 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.702 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.702 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.702 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "number of past completed events to remember", 2024-09-21T03:42:23.702 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.702 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.702 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.702 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.702 INFO:teuthology.orchestra.run.smithi089.stdout: "sleep_interval": { 2024-09-21T03:42:23.702 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "sleep_interval", 2024-09-21T03:42:23.702 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "secs", 2024-09-21T03:42:23.702 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.702 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:23.702 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "5", 2024-09-21T03:42:23.703 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.703 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.703 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.703 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "how long the module is going to sleep", 2024-09-21T03:42:23.703 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.703 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.703 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.703 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.703 INFO:teuthology.orchestra.run.smithi089.stdout: "sqlite3_killpoint": { 2024-09-21T03:42:23.703 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:42:23.703 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2024-09-21T03:42:23.704 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "dev", 2024-09-21T03:42:23.704 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:23.704 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "0", 2024-09-21T03:42:23.704 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.704 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.704 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.704 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.704 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.704 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.704 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.704 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-21T03:42:23.704 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-21T03:42:23.704 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.705 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-21T03:42:23.705 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "prometheus", 2024-09-21T03:42:23.705 INFO:teuthology.orchestra.run.smithi089.stdout: "can_run": true, 2024-09-21T03:42:23.705 INFO:teuthology.orchestra.run.smithi089.stdout: "error_string": "", 2024-09-21T03:42:23.705 INFO:teuthology.orchestra.run.smithi089.stdout: "module_options": { 2024-09-21T03:42:23.705 INFO:teuthology.orchestra.run.smithi089.stdout: "cache": { 2024-09-21T03:42:23.705 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "cache", 2024-09-21T03:42:23.705 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-21T03:42:23.705 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.705 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:23.705 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "True", 2024-09-21T03:42:23.705 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.705 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.706 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.706 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.706 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.706 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.706 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.706 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.706 INFO:teuthology.orchestra.run.smithi089.stdout: "exclude_perf_counters": { 2024-09-21T03:42:23.706 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "exclude_perf_counters", 2024-09-21T03:42:23.706 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-21T03:42:23.706 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.706 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:23.706 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "True", 2024-09-21T03:42:23.707 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.707 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.707 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.707 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-09-21T03:42:23.707 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-09-21T03:42:23.707 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.707 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.707 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.707 INFO:teuthology.orchestra.run.smithi089.stdout: "log_level": { 2024-09-21T03:42:23.707 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_level", 2024-09-21T03:42:23.707 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:23.707 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.708 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:23.708 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-21T03:42:23.708 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.708 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.708 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-21T03:42:23.708 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2024-09-21T03:42:23.708 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2024-09-21T03:42:23.708 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2024-09-21T03:42:23.708 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2024-09-21T03:42:23.708 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2024-09-21T03:42:23.708 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2024-09-21T03:42:23.708 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-21T03:42:23.708 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.709 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.709 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.709 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.709 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.709 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster": { 2024-09-21T03:42:23.709 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster", 2024-09-21T03:42:23.709 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-21T03:42:23.709 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.709 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:23.709 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-21T03:42:23.709 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.709 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.710 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.710 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.710 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.710 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.710 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.710 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.710 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster_level": { 2024-09-21T03:42:23.710 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster_level", 2024-09-21T03:42:23.710 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:23.710 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.710 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:23.710 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "info", 2024-09-21T03:42:23.710 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.711 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.711 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-21T03:42:23.711 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2024-09-21T03:42:23.711 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2024-09-21T03:42:23.711 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2024-09-21T03:42:23.711 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2024-09-21T03:42:23.711 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2024-09-21T03:42:23.711 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2024-09-21T03:42:23.711 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-21T03:42:23.711 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.711 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.711 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.712 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.712 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.712 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_file": { 2024-09-21T03:42:23.712 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_file", 2024-09-21T03:42:23.712 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-21T03:42:23.712 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.712 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:23.712 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-21T03:42:23.712 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.712 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.712 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.712 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.712 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.713 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.713 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.713 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.713 INFO:teuthology.orchestra.run.smithi089.stdout: "rbd_stats_pools": { 2024-09-21T03:42:23.713 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "rbd_stats_pools", 2024-09-21T03:42:23.713 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:23.713 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.713 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:23.713 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-21T03:42:23.713 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.713 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.713 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.713 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.713 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.714 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.714 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.714 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.714 INFO:teuthology.orchestra.run.smithi089.stdout: "rbd_stats_pools_refresh_interval": { 2024-09-21T03:42:23.714 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-09-21T03:42:23.714 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2024-09-21T03:42:23.714 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.714 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:23.714 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "300", 2024-09-21T03:42:23.714 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.714 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.714 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.715 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.715 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.715 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.715 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.715 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.715 INFO:teuthology.orchestra.run.smithi089.stdout: "scrape_interval": { 2024-09-21T03:42:23.715 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "scrape_interval", 2024-09-21T03:42:23.715 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "float", 2024-09-21T03:42:23.715 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.715 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:23.715 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "15.0", 2024-09-21T03:42:23.715 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.716 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.716 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.716 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.716 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.716 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.716 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.716 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.716 INFO:teuthology.orchestra.run.smithi089.stdout: "server_addr": { 2024-09-21T03:42:23.716 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "server_addr", 2024-09-21T03:42:23.716 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:23.716 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.716 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:23.716 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "::", 2024-09-21T03:42:23.716 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.717 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.717 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.717 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-09-21T03:42:23.717 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.717 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.717 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.717 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.717 INFO:teuthology.orchestra.run.smithi089.stdout: "server_port": { 2024-09-21T03:42:23.717 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "server_port", 2024-09-21T03:42:23.717 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2024-09-21T03:42:23.717 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.717 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:23.717 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "9283", 2024-09-21T03:42:23.717 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.718 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.718 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.718 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-09-21T03:42:23.718 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.718 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.718 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.718 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.718 INFO:teuthology.orchestra.run.smithi089.stdout: "sqlite3_killpoint": { 2024-09-21T03:42:23.718 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:42:23.718 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2024-09-21T03:42:23.718 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "dev", 2024-09-21T03:42:23.718 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:23.718 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "0", 2024-09-21T03:42:23.719 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.719 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.719 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.719 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.719 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.719 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.719 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.719 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.719 INFO:teuthology.orchestra.run.smithi089.stdout: "stale_cache_strategy": { 2024-09-21T03:42:23.719 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "stale_cache_strategy", 2024-09-21T03:42:23.719 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:23.719 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.719 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:23.719 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "log", 2024-09-21T03:42:23.720 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.720 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.720 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.720 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.720 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.720 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.720 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.720 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.720 INFO:teuthology.orchestra.run.smithi089.stdout: "standby_behaviour": { 2024-09-21T03:42:23.720 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "standby_behaviour", 2024-09-21T03:42:23.720 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:23.720 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.720 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:23.721 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "default", 2024-09-21T03:42:23.721 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.721 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.721 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-21T03:42:23.721 INFO:teuthology.orchestra.run.smithi089.stdout: "default", 2024-09-21T03:42:23.721 INFO:teuthology.orchestra.run.smithi089.stdout: "error" 2024-09-21T03:42:23.721 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-21T03:42:23.721 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.721 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.721 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.721 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.721 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.721 INFO:teuthology.orchestra.run.smithi089.stdout: "standby_error_status_code": { 2024-09-21T03:42:23.721 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "standby_error_status_code", 2024-09-21T03:42:23.722 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2024-09-21T03:42:23.722 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.722 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:23.722 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "500", 2024-09-21T03:42:23.722 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "400", 2024-09-21T03:42:23.722 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "599", 2024-09-21T03:42:23.722 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.722 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.722 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.722 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.722 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.722 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-21T03:42:23.722 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-21T03:42:23.723 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.723 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-21T03:42:23.723 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "rbd_support", 2024-09-21T03:42:23.723 INFO:teuthology.orchestra.run.smithi089.stdout: "can_run": true, 2024-09-21T03:42:23.723 INFO:teuthology.orchestra.run.smithi089.stdout: "error_string": "", 2024-09-21T03:42:23.723 INFO:teuthology.orchestra.run.smithi089.stdout: "module_options": { 2024-09-21T03:42:23.723 INFO:teuthology.orchestra.run.smithi089.stdout: "log_level": { 2024-09-21T03:42:23.723 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_level", 2024-09-21T03:42:23.723 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:23.723 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.723 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:23.723 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-21T03:42:23.724 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.724 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.724 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-21T03:42:23.724 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2024-09-21T03:42:23.724 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2024-09-21T03:42:23.724 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2024-09-21T03:42:23.724 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2024-09-21T03:42:23.724 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2024-09-21T03:42:23.724 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2024-09-21T03:42:23.724 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-21T03:42:23.724 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.724 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.725 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.725 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.725 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.725 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster": { 2024-09-21T03:42:23.725 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster", 2024-09-21T03:42:23.725 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-21T03:42:23.725 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.725 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:23.725 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-21T03:42:23.725 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.725 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.725 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.726 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.726 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.726 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.726 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.726 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.726 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster_level": { 2024-09-21T03:42:23.726 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster_level", 2024-09-21T03:42:23.726 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:23.726 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.726 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:23.726 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "info", 2024-09-21T03:42:23.727 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.727 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.727 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-21T03:42:23.727 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2024-09-21T03:42:23.727 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2024-09-21T03:42:23.727 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2024-09-21T03:42:23.727 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2024-09-21T03:42:23.727 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2024-09-21T03:42:23.727 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2024-09-21T03:42:23.727 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-21T03:42:23.728 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.728 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.728 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.728 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.728 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.728 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_file": { 2024-09-21T03:42:23.728 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_file", 2024-09-21T03:42:23.729 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-21T03:42:23.729 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.729 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:23.729 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-21T03:42:23.729 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.729 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.729 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.729 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.729 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.729 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.729 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.730 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.730 INFO:teuthology.orchestra.run.smithi089.stdout: "max_concurrent_snap_create": { 2024-09-21T03:42:23.730 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "max_concurrent_snap_create", 2024-09-21T03:42:23.730 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2024-09-21T03:42:23.730 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.730 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:23.730 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "10", 2024-09-21T03:42:23.730 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.730 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.730 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.730 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.730 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.731 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.731 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.731 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.731 INFO:teuthology.orchestra.run.smithi089.stdout: "mirror_snapshot_schedule": { 2024-09-21T03:42:23.731 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "mirror_snapshot_schedule", 2024-09-21T03:42:23.731 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:23.731 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.731 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:23.731 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-21T03:42:23.731 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.731 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.731 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.731 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.732 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.732 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.732 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.732 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.732 INFO:teuthology.orchestra.run.smithi089.stdout: "sqlite3_killpoint": { 2024-09-21T03:42:23.732 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:42:23.732 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2024-09-21T03:42:23.732 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "dev", 2024-09-21T03:42:23.732 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:23.732 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "0", 2024-09-21T03:42:23.732 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.732 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.733 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.733 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.733 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.733 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.733 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.733 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.733 INFO:teuthology.orchestra.run.smithi089.stdout: "trash_purge_schedule": { 2024-09-21T03:42:23.733 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "trash_purge_schedule", 2024-09-21T03:42:23.733 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:23.733 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.733 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:23.733 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-21T03:42:23.734 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.734 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.734 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.734 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.734 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.734 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.734 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.734 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-21T03:42:23.734 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-21T03:42:23.734 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.734 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-21T03:42:23.734 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "restful", 2024-09-21T03:42:23.735 INFO:teuthology.orchestra.run.smithi089.stdout: "can_run": true, 2024-09-21T03:42:23.735 INFO:teuthology.orchestra.run.smithi089.stdout: "error_string": "", 2024-09-21T03:42:23.735 INFO:teuthology.orchestra.run.smithi089.stdout: "module_options": { 2024-09-21T03:42:23.735 INFO:teuthology.orchestra.run.smithi089.stdout: "enable_auth": { 2024-09-21T03:42:23.735 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "enable_auth", 2024-09-21T03:42:23.735 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-21T03:42:23.735 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.735 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:23.735 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "True", 2024-09-21T03:42:23.735 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.735 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.735 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.735 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.736 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.736 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.736 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.736 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.736 INFO:teuthology.orchestra.run.smithi089.stdout: "key_file": { 2024-09-21T03:42:23.736 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "key_file", 2024-09-21T03:42:23.736 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:23.736 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.736 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:23.736 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-21T03:42:23.736 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.737 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.737 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.737 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.737 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.737 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.737 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.737 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.737 INFO:teuthology.orchestra.run.smithi089.stdout: "log_level": { 2024-09-21T03:42:23.737 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_level", 2024-09-21T03:42:23.737 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:23.737 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.737 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:23.738 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-21T03:42:23.738 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.738 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.738 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-21T03:42:23.738 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2024-09-21T03:42:23.738 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2024-09-21T03:42:23.738 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2024-09-21T03:42:23.738 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2024-09-21T03:42:23.738 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2024-09-21T03:42:23.738 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2024-09-21T03:42:23.738 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-21T03:42:23.738 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.739 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.739 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.739 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.739 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.739 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster": { 2024-09-21T03:42:23.739 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster", 2024-09-21T03:42:23.739 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-21T03:42:23.739 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.739 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:23.739 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-21T03:42:23.739 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.739 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.740 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.740 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.740 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.740 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.740 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.740 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.740 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster_level": { 2024-09-21T03:42:23.740 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster_level", 2024-09-21T03:42:23.740 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:23.740 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.740 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:23.740 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "info", 2024-09-21T03:42:23.740 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.741 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.741 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-21T03:42:23.741 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2024-09-21T03:42:23.741 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2024-09-21T03:42:23.741 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2024-09-21T03:42:23.741 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2024-09-21T03:42:23.741 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2024-09-21T03:42:23.741 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2024-09-21T03:42:23.741 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-21T03:42:23.741 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.741 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.741 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.742 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.742 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.742 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_file": { 2024-09-21T03:42:23.742 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_file", 2024-09-21T03:42:23.742 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-21T03:42:23.742 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.742 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:23.742 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-21T03:42:23.742 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.742 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.742 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.742 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.743 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.743 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.743 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.743 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.743 INFO:teuthology.orchestra.run.smithi089.stdout: "max_requests": { 2024-09-21T03:42:23.743 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "max_requests", 2024-09-21T03:42:23.743 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2024-09-21T03:42:23.743 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.743 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:23.743 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "500", 2024-09-21T03:42:23.743 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.743 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.743 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.744 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Maximum number of requests to keep in memory. When new request comes in, the oldest request will be removed if the number of requests exceeds the max request number.if un-finished request is removed, error message will be logged in the ceph-mgr log.", 2024-09-21T03:42:23.744 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.744 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.744 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.744 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.744 INFO:teuthology.orchestra.run.smithi089.stdout: "server_addr": { 2024-09-21T03:42:23.744 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "server_addr", 2024-09-21T03:42:23.744 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:23.744 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.744 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:23.744 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-21T03:42:23.744 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.744 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.744 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.745 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.745 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.745 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.745 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.745 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.745 INFO:teuthology.orchestra.run.smithi089.stdout: "server_port": { 2024-09-21T03:42:23.745 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "server_port", 2024-09-21T03:42:23.745 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:23.745 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.745 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:23.745 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-21T03:42:23.745 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.746 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.746 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.746 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.746 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.746 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.746 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.746 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.746 INFO:teuthology.orchestra.run.smithi089.stdout: "sqlite3_killpoint": { 2024-09-21T03:42:23.746 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:42:23.746 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2024-09-21T03:42:23.746 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "dev", 2024-09-21T03:42:23.747 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:23.747 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "0", 2024-09-21T03:42:23.747 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.747 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.747 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.747 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.747 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.747 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.747 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.747 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-21T03:42:23.747 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-21T03:42:23.747 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.747 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-21T03:42:23.748 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "rgw", 2024-09-21T03:42:23.748 INFO:teuthology.orchestra.run.smithi089.stdout: "can_run": true, 2024-09-21T03:42:23.748 INFO:teuthology.orchestra.run.smithi089.stdout: "error_string": "", 2024-09-21T03:42:23.748 INFO:teuthology.orchestra.run.smithi089.stdout: "module_options": { 2024-09-21T03:42:23.748 INFO:teuthology.orchestra.run.smithi089.stdout: "log_level": { 2024-09-21T03:42:23.748 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_level", 2024-09-21T03:42:23.748 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:23.748 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.748 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:23.748 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-21T03:42:23.748 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.748 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.749 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-21T03:42:23.749 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2024-09-21T03:42:23.749 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2024-09-21T03:42:23.749 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2024-09-21T03:42:23.749 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2024-09-21T03:42:23.749 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2024-09-21T03:42:23.749 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2024-09-21T03:42:23.749 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-21T03:42:23.749 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.749 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.749 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.749 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.749 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.749 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster": { 2024-09-21T03:42:23.750 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster", 2024-09-21T03:42:23.750 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-21T03:42:23.750 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.750 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:23.750 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-21T03:42:23.750 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.750 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.750 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.750 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.750 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.750 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.750 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.751 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.751 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster_level": { 2024-09-21T03:42:23.751 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster_level", 2024-09-21T03:42:23.751 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:23.751 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.751 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:23.751 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "info", 2024-09-21T03:42:23.751 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.751 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.751 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-21T03:42:23.751 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2024-09-21T03:42:23.751 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2024-09-21T03:42:23.751 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2024-09-21T03:42:23.752 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2024-09-21T03:42:23.752 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2024-09-21T03:42:23.752 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2024-09-21T03:42:23.752 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-21T03:42:23.752 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.752 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.752 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.752 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.752 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.752 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_file": { 2024-09-21T03:42:23.752 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_file", 2024-09-21T03:42:23.752 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-21T03:42:23.753 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.753 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:23.753 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-21T03:42:23.753 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.753 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.753 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.753 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.753 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.753 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.753 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.753 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.753 INFO:teuthology.orchestra.run.smithi089.stdout: "secondary_zone_period_retry_limit": { 2024-09-21T03:42:23.754 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "secondary_zone_period_retry_limit", 2024-09-21T03:42:23.754 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2024-09-21T03:42:23.754 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.754 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:23.754 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "5", 2024-09-21T03:42:23.754 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.754 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.754 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.754 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "RGW module period update retry limit for secondary site", 2024-09-21T03:42:23.754 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.754 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.754 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.754 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.755 INFO:teuthology.orchestra.run.smithi089.stdout: "sqlite3_killpoint": { 2024-09-21T03:42:23.755 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:42:23.755 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2024-09-21T03:42:23.755 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "dev", 2024-09-21T03:42:23.755 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:23.755 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "0", 2024-09-21T03:42:23.755 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.755 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.755 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.755 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.755 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.755 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.755 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.755 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-21T03:42:23.756 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-21T03:42:23.756 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.756 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-21T03:42:23.756 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "rook", 2024-09-21T03:42:23.756 INFO:teuthology.orchestra.run.smithi089.stdout: "can_run": true, 2024-09-21T03:42:23.756 INFO:teuthology.orchestra.run.smithi089.stdout: "error_string": "", 2024-09-21T03:42:23.756 INFO:teuthology.orchestra.run.smithi089.stdout: "module_options": { 2024-09-21T03:42:23.756 INFO:teuthology.orchestra.run.smithi089.stdout: "log_level": { 2024-09-21T03:42:23.756 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_level", 2024-09-21T03:42:23.756 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:23.757 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.757 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:23.757 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-21T03:42:23.757 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.757 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.757 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-21T03:42:23.757 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2024-09-21T03:42:23.757 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2024-09-21T03:42:23.757 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2024-09-21T03:42:23.757 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2024-09-21T03:42:23.757 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2024-09-21T03:42:23.757 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2024-09-21T03:42:23.757 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-21T03:42:23.757 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.757 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.758 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.758 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.758 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.758 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster": { 2024-09-21T03:42:23.758 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster", 2024-09-21T03:42:23.758 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-21T03:42:23.758 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.758 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:23.758 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-21T03:42:23.758 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.758 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.758 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.758 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.759 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.759 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.759 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.759 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.759 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster_level": { 2024-09-21T03:42:23.759 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster_level", 2024-09-21T03:42:23.759 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:23.759 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.759 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:23.759 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "info", 2024-09-21T03:42:23.759 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.759 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.759 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-21T03:42:23.760 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2024-09-21T03:42:23.760 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2024-09-21T03:42:23.760 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2024-09-21T03:42:23.760 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2024-09-21T03:42:23.760 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2024-09-21T03:42:23.760 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2024-09-21T03:42:23.760 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-21T03:42:23.760 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.760 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.760 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.760 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.760 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.760 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_file": { 2024-09-21T03:42:23.761 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_file", 2024-09-21T03:42:23.761 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-21T03:42:23.761 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.761 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:23.761 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-21T03:42:23.761 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.761 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.761 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.761 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.761 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.761 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.761 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.761 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.761 INFO:teuthology.orchestra.run.smithi089.stdout: "sqlite3_killpoint": { 2024-09-21T03:42:23.762 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:42:23.762 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2024-09-21T03:42:23.762 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "dev", 2024-09-21T03:42:23.762 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:23.762 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "0", 2024-09-21T03:42:23.762 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.762 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.762 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.762 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.762 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.762 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.762 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.762 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.763 INFO:teuthology.orchestra.run.smithi089.stdout: "storage_class": { 2024-09-21T03:42:23.763 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "storage_class", 2024-09-21T03:42:23.763 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:23.763 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.763 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:23.763 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "local", 2024-09-21T03:42:23.763 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.763 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.763 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.763 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-09-21T03:42:23.763 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.763 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.763 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.763 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-21T03:42:23.764 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-21T03:42:23.764 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.764 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-21T03:42:23.764 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "selftest", 2024-09-21T03:42:23.764 INFO:teuthology.orchestra.run.smithi089.stdout: "can_run": true, 2024-09-21T03:42:23.764 INFO:teuthology.orchestra.run.smithi089.stdout: "error_string": "", 2024-09-21T03:42:23.764 INFO:teuthology.orchestra.run.smithi089.stdout: "module_options": { 2024-09-21T03:42:23.764 INFO:teuthology.orchestra.run.smithi089.stdout: "log_level": { 2024-09-21T03:42:23.764 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_level", 2024-09-21T03:42:23.764 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:23.764 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.764 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:23.764 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-21T03:42:23.765 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.765 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.765 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-21T03:42:23.765 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2024-09-21T03:42:23.765 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2024-09-21T03:42:23.765 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2024-09-21T03:42:23.765 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2024-09-21T03:42:23.765 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2024-09-21T03:42:23.765 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2024-09-21T03:42:23.765 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-21T03:42:23.765 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.765 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.765 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.766 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.766 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.766 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster": { 2024-09-21T03:42:23.766 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster", 2024-09-21T03:42:23.766 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-21T03:42:23.766 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.766 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:23.766 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-21T03:42:23.766 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.766 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.766 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.766 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.766 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.766 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.767 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.767 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.767 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster_level": { 2024-09-21T03:42:23.767 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster_level", 2024-09-21T03:42:23.767 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:23.767 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.767 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:23.767 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "info", 2024-09-21T03:42:23.767 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.767 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.767 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-21T03:42:23.767 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2024-09-21T03:42:23.767 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2024-09-21T03:42:23.768 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2024-09-21T03:42:23.768 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2024-09-21T03:42:23.768 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2024-09-21T03:42:23.768 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2024-09-21T03:42:23.768 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-21T03:42:23.768 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.768 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.768 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.768 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.768 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.768 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_file": { 2024-09-21T03:42:23.768 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_file", 2024-09-21T03:42:23.768 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-21T03:42:23.768 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.769 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:23.769 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-21T03:42:23.769 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.769 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.769 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.769 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.769 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.769 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.769 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.769 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.769 INFO:teuthology.orchestra.run.smithi089.stdout: "roption1": { 2024-09-21T03:42:23.769 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "roption1", 2024-09-21T03:42:23.770 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:23.770 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.770 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:23.770 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-21T03:42:23.770 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.770 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.770 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.770 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.770 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.770 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.770 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.770 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.771 INFO:teuthology.orchestra.run.smithi089.stdout: "roption2": { 2024-09-21T03:42:23.771 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "roption2", 2024-09-21T03:42:23.771 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:23.771 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.771 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:23.771 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "xyz", 2024-09-21T03:42:23.771 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.771 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.771 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.771 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.771 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.772 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.772 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.772 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.772 INFO:teuthology.orchestra.run.smithi089.stdout: "rwoption1": { 2024-09-21T03:42:23.772 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "rwoption1", 2024-09-21T03:42:23.772 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:23.772 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.772 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:23.772 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-21T03:42:23.772 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.772 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.772 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.773 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.773 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.773 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.773 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.773 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.773 INFO:teuthology.orchestra.run.smithi089.stdout: "rwoption2": { 2024-09-21T03:42:23.773 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "rwoption2", 2024-09-21T03:42:23.773 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2024-09-21T03:42:23.773 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.773 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:23.773 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-21T03:42:23.773 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.774 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.774 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.774 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.774 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.774 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.774 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.774 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.774 INFO:teuthology.orchestra.run.smithi089.stdout: "rwoption3": { 2024-09-21T03:42:23.774 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "rwoption3", 2024-09-21T03:42:23.774 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "float", 2024-09-21T03:42:23.774 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.775 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:23.775 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-21T03:42:23.775 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.775 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.775 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.775 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.775 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.775 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.775 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.775 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.775 INFO:teuthology.orchestra.run.smithi089.stdout: "rwoption4": { 2024-09-21T03:42:23.776 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "rwoption4", 2024-09-21T03:42:23.776 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:23.776 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.776 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:23.776 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-21T03:42:23.776 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.776 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.776 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.776 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.776 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.776 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.776 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.776 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.777 INFO:teuthology.orchestra.run.smithi089.stdout: "rwoption5": { 2024-09-21T03:42:23.777 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "rwoption5", 2024-09-21T03:42:23.777 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-21T03:42:23.777 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.777 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:23.777 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-21T03:42:23.777 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.777 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.777 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.777 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.777 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.778 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.778 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.778 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.778 INFO:teuthology.orchestra.run.smithi089.stdout: "rwoption6": { 2024-09-21T03:42:23.778 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "rwoption6", 2024-09-21T03:42:23.778 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-21T03:42:23.778 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.778 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:23.778 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "True", 2024-09-21T03:42:23.778 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.778 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.778 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.779 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.779 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.779 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.779 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.779 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.779 INFO:teuthology.orchestra.run.smithi089.stdout: "rwoption7": { 2024-09-21T03:42:23.779 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "rwoption7", 2024-09-21T03:42:23.779 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2024-09-21T03:42:23.779 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.779 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:23.779 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-21T03:42:23.779 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "1", 2024-09-21T03:42:23.779 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "42", 2024-09-21T03:42:23.780 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.780 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.780 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.780 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.780 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.780 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.780 INFO:teuthology.orchestra.run.smithi089.stdout: "sqlite3_killpoint": { 2024-09-21T03:42:23.780 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:42:23.780 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2024-09-21T03:42:23.780 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "dev", 2024-09-21T03:42:23.780 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:23.780 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "0", 2024-09-21T03:42:23.781 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.781 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.781 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.781 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.781 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.781 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.781 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.781 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.781 INFO:teuthology.orchestra.run.smithi089.stdout: "testkey": { 2024-09-21T03:42:23.781 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "testkey", 2024-09-21T03:42:23.781 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:23.781 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.781 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:23.782 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-21T03:42:23.782 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.782 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.782 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.782 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.782 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.782 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.782 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.782 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.782 INFO:teuthology.orchestra.run.smithi089.stdout: "testlkey": { 2024-09-21T03:42:23.782 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "testlkey", 2024-09-21T03:42:23.783 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:23.783 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.783 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:23.783 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-21T03:42:23.783 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.783 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.783 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.783 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.783 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.783 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.783 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.783 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.783 INFO:teuthology.orchestra.run.smithi089.stdout: "testnewline": { 2024-09-21T03:42:23.784 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "testnewline", 2024-09-21T03:42:23.784 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:23.784 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.784 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:23.784 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-21T03:42:23.784 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.784 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.784 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.784 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.784 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.784 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.784 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.785 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-21T03:42:23.785 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-21T03:42:23.785 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.785 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-21T03:42:23.785 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "smb", 2024-09-21T03:42:23.785 INFO:teuthology.orchestra.run.smithi089.stdout: "can_run": true, 2024-09-21T03:42:23.785 INFO:teuthology.orchestra.run.smithi089.stdout: "error_string": "", 2024-09-21T03:42:23.785 INFO:teuthology.orchestra.run.smithi089.stdout: "module_options": { 2024-09-21T03:42:23.785 INFO:teuthology.orchestra.run.smithi089.stdout: "internal_store_backend": { 2024-09-21T03:42:23.785 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "internal_store_backend", 2024-09-21T03:42:23.785 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:23.785 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "dev", 2024-09-21T03:42:23.786 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:23.786 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-21T03:42:23.786 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.786 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.786 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.786 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "set internal store backend. for develoment and testing only", 2024-09-21T03:42:23.786 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.786 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.786 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.786 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.786 INFO:teuthology.orchestra.run.smithi089.stdout: "log_level": { 2024-09-21T03:42:23.786 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_level", 2024-09-21T03:42:23.787 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:23.787 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.787 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:23.787 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-21T03:42:23.787 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.787 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.787 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-21T03:42:23.787 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2024-09-21T03:42:23.787 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2024-09-21T03:42:23.787 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2024-09-21T03:42:23.787 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2024-09-21T03:42:23.787 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2024-09-21T03:42:23.787 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2024-09-21T03:42:23.787 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-21T03:42:23.788 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.788 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.788 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.788 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.788 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.788 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster": { 2024-09-21T03:42:23.788 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster", 2024-09-21T03:42:23.788 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-21T03:42:23.788 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.788 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:23.788 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-21T03:42:23.788 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.789 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.789 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.789 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.789 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.789 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.789 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.789 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.789 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster_level": { 2024-09-21T03:42:23.789 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster_level", 2024-09-21T03:42:23.789 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:23.789 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.789 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:23.789 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "info", 2024-09-21T03:42:23.790 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.790 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.790 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-21T03:42:23.790 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2024-09-21T03:42:23.790 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2024-09-21T03:42:23.790 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2024-09-21T03:42:23.790 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2024-09-21T03:42:23.790 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2024-09-21T03:42:23.790 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2024-09-21T03:42:23.790 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-21T03:42:23.790 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.790 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.791 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.791 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.791 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.791 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_file": { 2024-09-21T03:42:23.791 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_file", 2024-09-21T03:42:23.791 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-21T03:42:23.791 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.791 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:23.791 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-21T03:42:23.791 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.791 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.792 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.792 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.792 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.792 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.792 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.792 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.792 INFO:teuthology.orchestra.run.smithi089.stdout: "sqlite3_killpoint": { 2024-09-21T03:42:23.792 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:42:23.792 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2024-09-21T03:42:23.792 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "dev", 2024-09-21T03:42:23.792 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:23.793 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "0", 2024-09-21T03:42:23.793 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.793 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.793 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.793 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.793 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.793 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.793 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.793 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.793 INFO:teuthology.orchestra.run.smithi089.stdout: "update_orchestration": { 2024-09-21T03:42:23.793 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "update_orchestration", 2024-09-21T03:42:23.794 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-21T03:42:23.794 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.794 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:23.794 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "True", 2024-09-21T03:42:23.794 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.794 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.794 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.794 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "automatically update orchestration when smb resources are changed", 2024-09-21T03:42:23.794 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.794 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.794 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.794 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-21T03:42:23.794 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-21T03:42:23.795 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.795 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-21T03:42:23.795 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "snap_schedule", 2024-09-21T03:42:23.795 INFO:teuthology.orchestra.run.smithi089.stdout: "can_run": true, 2024-09-21T03:42:23.795 INFO:teuthology.orchestra.run.smithi089.stdout: "error_string": "", 2024-09-21T03:42:23.795 INFO:teuthology.orchestra.run.smithi089.stdout: "module_options": { 2024-09-21T03:42:23.795 INFO:teuthology.orchestra.run.smithi089.stdout: "allow_m_granularity": { 2024-09-21T03:42:23.795 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "allow_m_granularity", 2024-09-21T03:42:23.795 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-21T03:42:23.795 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.795 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:23.795 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-21T03:42:23.796 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.796 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.796 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.796 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "allow minute scheduled snapshots", 2024-09-21T03:42:23.796 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.796 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.796 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.796 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.796 INFO:teuthology.orchestra.run.smithi089.stdout: "dump_on_update": { 2024-09-21T03:42:23.796 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "dump_on_update", 2024-09-21T03:42:23.796 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-21T03:42:23.796 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.797 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:23.797 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-21T03:42:23.797 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.797 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.797 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.797 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "dump database to debug log on update", 2024-09-21T03:42:23.797 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.797 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.797 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.797 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.797 INFO:teuthology.orchestra.run.smithi089.stdout: "log_level": { 2024-09-21T03:42:23.797 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_level", 2024-09-21T03:42:23.797 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:23.798 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.798 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:23.798 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-21T03:42:23.798 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.798 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.798 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-21T03:42:23.798 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2024-09-21T03:42:23.798 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2024-09-21T03:42:23.798 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2024-09-21T03:42:23.798 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2024-09-21T03:42:23.798 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2024-09-21T03:42:23.798 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2024-09-21T03:42:23.799 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-21T03:42:23.799 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.799 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.799 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.799 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.799 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.799 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster": { 2024-09-21T03:42:23.799 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster", 2024-09-21T03:42:23.799 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-21T03:42:23.799 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.799 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:23.799 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-21T03:42:23.799 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.800 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.800 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.800 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.800 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.800 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.800 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.800 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.800 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster_level": { 2024-09-21T03:42:23.800 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster_level", 2024-09-21T03:42:23.800 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:23.800 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.800 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:23.801 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "info", 2024-09-21T03:42:23.801 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.801 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.801 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-21T03:42:23.801 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2024-09-21T03:42:23.801 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2024-09-21T03:42:23.801 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2024-09-21T03:42:23.801 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2024-09-21T03:42:23.801 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2024-09-21T03:42:23.801 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2024-09-21T03:42:23.801 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-21T03:42:23.801 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.801 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.802 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.802 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.802 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.802 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_file": { 2024-09-21T03:42:23.802 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_file", 2024-09-21T03:42:23.802 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-21T03:42:23.802 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.802 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:23.802 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-21T03:42:23.802 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.802 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.802 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.803 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.803 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.803 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.803 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.803 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.803 INFO:teuthology.orchestra.run.smithi089.stdout: "sqlite3_killpoint": { 2024-09-21T03:42:23.803 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:42:23.803 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2024-09-21T03:42:23.803 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "dev", 2024-09-21T03:42:23.803 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:23.803 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "0", 2024-09-21T03:42:23.803 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.804 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.804 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.804 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.804 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.804 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.804 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.804 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-21T03:42:23.804 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-21T03:42:23.804 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.804 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-21T03:42:23.804 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "stats", 2024-09-21T03:42:23.804 INFO:teuthology.orchestra.run.smithi089.stdout: "can_run": true, 2024-09-21T03:42:23.804 INFO:teuthology.orchestra.run.smithi089.stdout: "error_string": "", 2024-09-21T03:42:23.805 INFO:teuthology.orchestra.run.smithi089.stdout: "module_options": { 2024-09-21T03:42:23.805 INFO:teuthology.orchestra.run.smithi089.stdout: "log_level": { 2024-09-21T03:42:23.805 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_level", 2024-09-21T03:42:23.805 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:23.805 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.805 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:23.805 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-21T03:42:23.805 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.805 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.805 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-21T03:42:23.805 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2024-09-21T03:42:23.805 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2024-09-21T03:42:23.806 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2024-09-21T03:42:23.806 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2024-09-21T03:42:23.806 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2024-09-21T03:42:23.806 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2024-09-21T03:42:23.806 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-21T03:42:23.806 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.806 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.806 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.806 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.806 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.806 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster": { 2024-09-21T03:42:23.806 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster", 2024-09-21T03:42:23.807 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-21T03:42:23.807 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.807 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:23.807 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-21T03:42:23.807 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.807 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.807 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.807 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.807 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.807 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.807 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.807 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.808 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster_level": { 2024-09-21T03:42:23.808 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster_level", 2024-09-21T03:42:23.808 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:23.808 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.808 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:23.808 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "info", 2024-09-21T03:42:23.808 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.808 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.808 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-21T03:42:23.808 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2024-09-21T03:42:23.808 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2024-09-21T03:42:23.808 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2024-09-21T03:42:23.808 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2024-09-21T03:42:23.808 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2024-09-21T03:42:23.809 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2024-09-21T03:42:23.809 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-21T03:42:23.809 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.809 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.809 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.809 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.809 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.809 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_file": { 2024-09-21T03:42:23.809 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_file", 2024-09-21T03:42:23.809 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-21T03:42:23.809 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.809 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:23.810 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-21T03:42:23.810 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.810 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.810 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.810 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.810 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.810 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.810 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.810 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.810 INFO:teuthology.orchestra.run.smithi089.stdout: "sqlite3_killpoint": { 2024-09-21T03:42:23.810 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:42:23.810 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2024-09-21T03:42:23.810 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "dev", 2024-09-21T03:42:23.811 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:23.811 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "0", 2024-09-21T03:42:23.811 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.811 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.811 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.811 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.811 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.811 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.811 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.811 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-21T03:42:23.811 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-21T03:42:23.811 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.811 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-21T03:42:23.811 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "status", 2024-09-21T03:42:23.812 INFO:teuthology.orchestra.run.smithi089.stdout: "can_run": true, 2024-09-21T03:42:23.812 INFO:teuthology.orchestra.run.smithi089.stdout: "error_string": "", 2024-09-21T03:42:23.812 INFO:teuthology.orchestra.run.smithi089.stdout: "module_options": { 2024-09-21T03:42:23.812 INFO:teuthology.orchestra.run.smithi089.stdout: "log_level": { 2024-09-21T03:42:23.812 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_level", 2024-09-21T03:42:23.812 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:23.812 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.812 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:23.812 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-21T03:42:23.812 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.812 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.812 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-21T03:42:23.812 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2024-09-21T03:42:23.813 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2024-09-21T03:42:23.813 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2024-09-21T03:42:23.813 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2024-09-21T03:42:23.813 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2024-09-21T03:42:23.813 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2024-09-21T03:42:23.813 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-21T03:42:23.813 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.813 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.813 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.813 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.813 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.813 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster": { 2024-09-21T03:42:23.813 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster", 2024-09-21T03:42:23.814 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-21T03:42:23.814 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.814 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:23.814 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-21T03:42:23.814 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.814 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.814 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.814 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.814 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.814 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.814 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.814 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.815 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster_level": { 2024-09-21T03:42:23.815 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster_level", 2024-09-21T03:42:23.815 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:23.815 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.815 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:23.815 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "info", 2024-09-21T03:42:23.815 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.815 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.815 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-21T03:42:23.815 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2024-09-21T03:42:23.815 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2024-09-21T03:42:23.815 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2024-09-21T03:42:23.815 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2024-09-21T03:42:23.816 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2024-09-21T03:42:23.816 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2024-09-21T03:42:23.816 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-21T03:42:23.816 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.816 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.816 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.816 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.816 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.816 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_file": { 2024-09-21T03:42:23.816 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_file", 2024-09-21T03:42:23.816 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-21T03:42:23.816 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.816 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:23.817 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-21T03:42:23.817 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.817 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.817 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.817 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.817 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.817 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.817 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.817 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.817 INFO:teuthology.orchestra.run.smithi089.stdout: "sqlite3_killpoint": { 2024-09-21T03:42:23.817 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:42:23.817 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2024-09-21T03:42:23.817 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "dev", 2024-09-21T03:42:23.818 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:23.818 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "0", 2024-09-21T03:42:23.818 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.818 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.818 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.818 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.818 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.818 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.818 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.818 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-21T03:42:23.818 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-21T03:42:23.819 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.819 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-21T03:42:23.819 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "telegraf", 2024-09-21T03:42:23.819 INFO:teuthology.orchestra.run.smithi089.stdout: "can_run": true, 2024-09-21T03:42:23.819 INFO:teuthology.orchestra.run.smithi089.stdout: "error_string": "", 2024-09-21T03:42:23.819 INFO:teuthology.orchestra.run.smithi089.stdout: "module_options": { 2024-09-21T03:42:23.819 INFO:teuthology.orchestra.run.smithi089.stdout: "address": { 2024-09-21T03:42:23.819 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "address", 2024-09-21T03:42:23.819 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:23.819 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.819 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:23.819 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-09-21T03:42:23.820 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.820 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.820 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.820 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.820 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.820 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.820 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.820 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.820 INFO:teuthology.orchestra.run.smithi089.stdout: "interval": { 2024-09-21T03:42:23.820 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "interval", 2024-09-21T03:42:23.820 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "secs", 2024-09-21T03:42:23.820 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.820 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:23.820 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "15", 2024-09-21T03:42:23.821 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.821 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.821 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.821 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.821 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.821 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.821 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.821 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.821 INFO:teuthology.orchestra.run.smithi089.stdout: "log_level": { 2024-09-21T03:42:23.821 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_level", 2024-09-21T03:42:23.821 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:23.821 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.821 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:23.821 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-21T03:42:23.822 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.822 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.822 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-21T03:42:23.822 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2024-09-21T03:42:23.822 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2024-09-21T03:42:23.822 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2024-09-21T03:42:23.822 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2024-09-21T03:42:23.822 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2024-09-21T03:42:23.822 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2024-09-21T03:42:23.822 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-21T03:42:23.822 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.822 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.823 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.823 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.823 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.823 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster": { 2024-09-21T03:42:23.823 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster", 2024-09-21T03:42:23.823 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-21T03:42:23.823 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.823 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:23.823 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-21T03:42:23.823 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.823 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.823 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.824 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.824 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.824 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.824 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.824 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.824 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster_level": { 2024-09-21T03:42:23.824 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster_level", 2024-09-21T03:42:23.824 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:23.824 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.824 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:23.824 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "info", 2024-09-21T03:42:23.824 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.825 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.825 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-21T03:42:23.825 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2024-09-21T03:42:23.825 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2024-09-21T03:42:23.825 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2024-09-21T03:42:23.825 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2024-09-21T03:42:23.825 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2024-09-21T03:42:23.825 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2024-09-21T03:42:23.825 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-21T03:42:23.825 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.825 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.826 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.826 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.826 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.826 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_file": { 2024-09-21T03:42:23.826 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_file", 2024-09-21T03:42:23.826 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-21T03:42:23.826 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.826 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:23.826 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-21T03:42:23.826 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.826 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.826 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.827 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.827 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.827 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.827 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.827 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.827 INFO:teuthology.orchestra.run.smithi089.stdout: "sqlite3_killpoint": { 2024-09-21T03:42:23.827 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:42:23.827 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2024-09-21T03:42:23.827 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "dev", 2024-09-21T03:42:23.827 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:23.827 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "0", 2024-09-21T03:42:23.827 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.828 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.828 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.828 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.828 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.828 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.828 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.828 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-21T03:42:23.828 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-21T03:42:23.828 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.828 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-21T03:42:23.828 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "telemetry", 2024-09-21T03:42:23.828 INFO:teuthology.orchestra.run.smithi089.stdout: "can_run": true, 2024-09-21T03:42:23.828 INFO:teuthology.orchestra.run.smithi089.stdout: "error_string": "", 2024-09-21T03:42:23.829 INFO:teuthology.orchestra.run.smithi089.stdout: "module_options": { 2024-09-21T03:42:23.829 INFO:teuthology.orchestra.run.smithi089.stdout: "channel_basic": { 2024-09-21T03:42:23.829 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "channel_basic", 2024-09-21T03:42:23.829 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-21T03:42:23.829 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.829 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:23.829 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "True", 2024-09-21T03:42:23.829 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.829 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.829 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.829 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Share basic cluster information (size, version)", 2024-09-21T03:42:23.829 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.830 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.830 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.830 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.830 INFO:teuthology.orchestra.run.smithi089.stdout: "channel_crash": { 2024-09-21T03:42:23.830 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "channel_crash", 2024-09-21T03:42:23.830 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-21T03:42:23.830 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.830 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:23.830 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "True", 2024-09-21T03:42:23.830 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.830 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.830 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.831 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-09-21T03:42:23.831 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.831 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.831 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.831 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.831 INFO:teuthology.orchestra.run.smithi089.stdout: "channel_device": { 2024-09-21T03:42:23.831 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "channel_device", 2024-09-21T03:42:23.831 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-21T03:42:23.831 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.831 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:23.831 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "True", 2024-09-21T03:42:23.831 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.832 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.832 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.832 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-09-21T03:42:23.832 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.832 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.832 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.832 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.832 INFO:teuthology.orchestra.run.smithi089.stdout: "channel_ident": { 2024-09-21T03:42:23.832 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "channel_ident", 2024-09-21T03:42:23.832 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-21T03:42:23.832 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.832 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:23.832 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-21T03:42:23.833 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.833 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.833 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.833 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-09-21T03:42:23.833 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.833 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.833 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.833 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.833 INFO:teuthology.orchestra.run.smithi089.stdout: "channel_perf": { 2024-09-21T03:42:23.833 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "channel_perf", 2024-09-21T03:42:23.833 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-21T03:42:23.833 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.834 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:23.834 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-21T03:42:23.834 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.834 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.834 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.834 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Share various performance metrics of a cluster", 2024-09-21T03:42:23.834 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.834 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.834 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.834 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.834 INFO:teuthology.orchestra.run.smithi089.stdout: "contact": { 2024-09-21T03:42:23.834 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "contact", 2024-09-21T03:42:23.835 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:23.835 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.835 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:23.835 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-21T03:42:23.835 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.835 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.835 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.835 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.835 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.835 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.835 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.835 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.836 INFO:teuthology.orchestra.run.smithi089.stdout: "description": { 2024-09-21T03:42:23.836 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "description", 2024-09-21T03:42:23.836 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:23.836 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.836 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:23.836 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-21T03:42:23.836 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.836 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.836 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.836 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.836 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.836 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.836 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.836 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.837 INFO:teuthology.orchestra.run.smithi089.stdout: "device_url": { 2024-09-21T03:42:23.837 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "device_url", 2024-09-21T03:42:23.837 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:23.837 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.837 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:23.837 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-09-21T03:42:23.837 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.837 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.837 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.837 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.837 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.837 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.838 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.838 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.838 INFO:teuthology.orchestra.run.smithi089.stdout: "enabled": { 2024-09-21T03:42:23.838 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "enabled", 2024-09-21T03:42:23.838 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-21T03:42:23.838 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.838 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:23.838 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-21T03:42:23.838 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.838 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.838 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.839 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.839 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.839 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.839 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.839 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.839 INFO:teuthology.orchestra.run.smithi089.stdout: "interval": { 2024-09-21T03:42:23.839 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "interval", 2024-09-21T03:42:23.839 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2024-09-21T03:42:23.839 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.839 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:23.839 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "24", 2024-09-21T03:42:23.839 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "8", 2024-09-21T03:42:23.839 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.840 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.840 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.840 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.840 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.840 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.840 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.840 INFO:teuthology.orchestra.run.smithi089.stdout: "last_opt_revision": { 2024-09-21T03:42:23.840 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "last_opt_revision", 2024-09-21T03:42:23.840 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2024-09-21T03:42:23.840 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.840 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:23.840 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "1", 2024-09-21T03:42:23.841 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.841 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.841 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.841 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.841 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.841 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.841 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.841 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.841 INFO:teuthology.orchestra.run.smithi089.stdout: "leaderboard": { 2024-09-21T03:42:23.841 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "leaderboard", 2024-09-21T03:42:23.841 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-21T03:42:23.841 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.841 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:23.841 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-21T03:42:23.842 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.842 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.842 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.842 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.842 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.842 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.842 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.842 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.842 INFO:teuthology.orchestra.run.smithi089.stdout: "leaderboard_description": { 2024-09-21T03:42:23.842 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "leaderboard_description", 2024-09-21T03:42:23.842 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:23.842 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.843 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:23.843 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-21T03:42:23.843 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.843 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.843 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.843 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.843 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.843 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.843 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.843 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.843 INFO:teuthology.orchestra.run.smithi089.stdout: "log_level": { 2024-09-21T03:42:23.844 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_level", 2024-09-21T03:42:23.844 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:23.844 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.844 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:23.844 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-21T03:42:23.844 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.844 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.844 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-21T03:42:23.844 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2024-09-21T03:42:23.844 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2024-09-21T03:42:23.844 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2024-09-21T03:42:23.844 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2024-09-21T03:42:23.844 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2024-09-21T03:42:23.845 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2024-09-21T03:42:23.845 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-21T03:42:23.845 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.845 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.845 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.845 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.845 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.845 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster": { 2024-09-21T03:42:23.845 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster", 2024-09-21T03:42:23.845 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-21T03:42:23.845 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.845 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:23.845 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-21T03:42:23.846 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.846 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.846 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.846 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.846 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.846 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.846 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.846 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.846 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster_level": { 2024-09-21T03:42:23.846 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster_level", 2024-09-21T03:42:23.846 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:23.847 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.847 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:23.847 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "info", 2024-09-21T03:42:23.847 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.847 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.847 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-21T03:42:23.847 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2024-09-21T03:42:23.847 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2024-09-21T03:42:23.847 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2024-09-21T03:42:23.847 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2024-09-21T03:42:23.847 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2024-09-21T03:42:23.848 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2024-09-21T03:42:23.848 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-21T03:42:23.848 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.848 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.848 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.848 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.848 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.848 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_file": { 2024-09-21T03:42:23.848 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_file", 2024-09-21T03:42:23.848 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-21T03:42:23.848 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.848 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:23.849 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-21T03:42:23.849 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.849 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.849 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.849 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.849 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.849 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.849 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.849 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.849 INFO:teuthology.orchestra.run.smithi089.stdout: "organization": { 2024-09-21T03:42:23.849 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "organization", 2024-09-21T03:42:23.849 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:23.850 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.850 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:23.850 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-21T03:42:23.850 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.850 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.850 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.850 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.850 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.850 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.850 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.850 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.850 INFO:teuthology.orchestra.run.smithi089.stdout: "proxy": { 2024-09-21T03:42:23.851 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "proxy", 2024-09-21T03:42:23.851 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:23.851 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.851 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:23.851 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-21T03:42:23.851 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.851 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.851 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.851 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.851 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.851 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.851 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.852 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.852 INFO:teuthology.orchestra.run.smithi089.stdout: "sqlite3_killpoint": { 2024-09-21T03:42:23.852 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:42:23.852 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2024-09-21T03:42:23.852 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "dev", 2024-09-21T03:42:23.852 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:23.852 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "0", 2024-09-21T03:42:23.852 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.852 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.852 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.852 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.852 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.852 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.853 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.853 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.853 INFO:teuthology.orchestra.run.smithi089.stdout: "url": { 2024-09-21T03:42:23.853 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "url", 2024-09-21T03:42:23.853 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:23.853 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.853 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:23.853 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-09-21T03:42:23.853 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.853 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.853 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.853 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.854 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.854 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.854 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.854 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-21T03:42:23.854 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-21T03:42:23.854 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.854 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-21T03:42:23.854 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "test_orchestrator", 2024-09-21T03:42:23.854 INFO:teuthology.orchestra.run.smithi089.stdout: "can_run": true, 2024-09-21T03:42:23.854 INFO:teuthology.orchestra.run.smithi089.stdout: "error_string": "", 2024-09-21T03:42:23.854 INFO:teuthology.orchestra.run.smithi089.stdout: "module_options": { 2024-09-21T03:42:23.854 INFO:teuthology.orchestra.run.smithi089.stdout: "log_level": { 2024-09-21T03:42:23.855 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_level", 2024-09-21T03:42:23.855 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:23.855 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.855 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:23.855 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-21T03:42:23.855 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.855 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.855 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-21T03:42:23.855 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2024-09-21T03:42:23.855 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2024-09-21T03:42:23.855 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2024-09-21T03:42:23.855 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2024-09-21T03:42:23.856 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2024-09-21T03:42:23.856 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2024-09-21T03:42:23.856 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-21T03:42:23.856 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.856 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.856 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.856 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.856 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.856 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster": { 2024-09-21T03:42:23.856 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster", 2024-09-21T03:42:23.856 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-21T03:42:23.856 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.857 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:23.857 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-21T03:42:23.857 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.857 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.857 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.857 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.857 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.857 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.857 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.857 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.857 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster_level": { 2024-09-21T03:42:23.857 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster_level", 2024-09-21T03:42:23.857 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:23.857 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.858 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:23.858 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "info", 2024-09-21T03:42:23.858 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.858 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.858 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-21T03:42:23.858 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2024-09-21T03:42:23.858 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2024-09-21T03:42:23.858 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2024-09-21T03:42:23.858 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2024-09-21T03:42:23.858 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2024-09-21T03:42:23.858 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2024-09-21T03:42:23.858 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-21T03:42:23.859 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.859 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.859 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.859 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.859 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.859 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_file": { 2024-09-21T03:42:23.859 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_file", 2024-09-21T03:42:23.859 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-21T03:42:23.859 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.859 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:23.859 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-21T03:42:23.859 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.860 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.860 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.860 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.860 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.860 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.860 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.860 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.860 INFO:teuthology.orchestra.run.smithi089.stdout: "sqlite3_killpoint": { 2024-09-21T03:42:23.860 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:42:23.860 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2024-09-21T03:42:23.860 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "dev", 2024-09-21T03:42:23.860 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:23.861 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "0", 2024-09-21T03:42:23.861 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.861 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.861 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.861 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.861 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.861 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.861 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.861 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-21T03:42:23.861 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-21T03:42:23.861 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.861 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-21T03:42:23.862 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "volumes", 2024-09-21T03:42:23.862 INFO:teuthology.orchestra.run.smithi089.stdout: "can_run": true, 2024-09-21T03:42:23.862 INFO:teuthology.orchestra.run.smithi089.stdout: "error_string": "", 2024-09-21T03:42:23.862 INFO:teuthology.orchestra.run.smithi089.stdout: "module_options": { 2024-09-21T03:42:23.862 INFO:teuthology.orchestra.run.smithi089.stdout: "log_level": { 2024-09-21T03:42:23.862 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_level", 2024-09-21T03:42:23.862 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:23.862 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.862 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:23.862 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-21T03:42:23.862 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.862 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.862 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-21T03:42:23.863 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2024-09-21T03:42:23.863 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2024-09-21T03:42:23.863 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2024-09-21T03:42:23.863 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2024-09-21T03:42:23.863 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2024-09-21T03:42:23.863 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2024-09-21T03:42:23.863 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-21T03:42:23.863 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.863 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.863 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.863 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.863 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.863 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster": { 2024-09-21T03:42:23.864 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster", 2024-09-21T03:42:23.864 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-21T03:42:23.864 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.864 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:23.864 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-21T03:42:23.864 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.864 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.864 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.864 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.864 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.864 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.864 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.865 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.865 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster_level": { 2024-09-21T03:42:23.865 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster_level", 2024-09-21T03:42:23.865 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:23.865 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.865 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:23.865 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "info", 2024-09-21T03:42:23.865 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.865 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.865 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-21T03:42:23.865 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2024-09-21T03:42:23.865 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2024-09-21T03:42:23.865 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2024-09-21T03:42:23.866 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2024-09-21T03:42:23.866 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2024-09-21T03:42:23.866 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2024-09-21T03:42:23.866 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-21T03:42:23.866 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.866 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.866 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.866 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.866 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.866 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_file": { 2024-09-21T03:42:23.866 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_file", 2024-09-21T03:42:23.866 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-21T03:42:23.866 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.867 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:23.867 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-21T03:42:23.867 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.867 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.867 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.867 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.867 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.867 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.867 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.867 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.867 INFO:teuthology.orchestra.run.smithi089.stdout: "max_concurrent_clones": { 2024-09-21T03:42:23.867 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "max_concurrent_clones", 2024-09-21T03:42:23.868 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2024-09-21T03:42:23.868 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.868 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:23.868 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "4", 2024-09-21T03:42:23.868 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.868 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.868 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.868 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Number of asynchronous cloner threads", 2024-09-21T03:42:23.868 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.868 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.868 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.868 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.869 INFO:teuthology.orchestra.run.smithi089.stdout: "periodic_async_work": { 2024-09-21T03:42:23.869 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "periodic_async_work", 2024-09-21T03:42:23.869 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-21T03:42:23.869 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.869 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:23.869 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-21T03:42:23.869 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.869 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.869 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.869 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Periodically check for async work", 2024-09-21T03:42:23.869 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.869 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.869 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.869 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.870 INFO:teuthology.orchestra.run.smithi089.stdout: "snapshot_clone_delay": { 2024-09-21T03:42:23.870 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "snapshot_clone_delay", 2024-09-21T03:42:23.870 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2024-09-21T03:42:23.870 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.870 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:23.870 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "0", 2024-09-21T03:42:23.870 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.870 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.870 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.870 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-09-21T03:42:23.870 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.870 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.870 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.871 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.871 INFO:teuthology.orchestra.run.smithi089.stdout: "snapshot_clone_no_wait": { 2024-09-21T03:42:23.871 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "snapshot_clone_no_wait", 2024-09-21T03:42:23.871 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-21T03:42:23.871 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.871 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:23.871 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "True", 2024-09-21T03:42:23.871 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.871 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.871 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.871 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-09-21T03:42:23.871 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.871 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.872 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.872 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.872 INFO:teuthology.orchestra.run.smithi089.stdout: "sqlite3_killpoint": { 2024-09-21T03:42:23.872 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:42:23.872 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2024-09-21T03:42:23.872 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "dev", 2024-09-21T03:42:23.872 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:23.872 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "0", 2024-09-21T03:42:23.872 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.872 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.872 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.872 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.872 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.872 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.873 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.873 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-21T03:42:23.873 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-21T03:42:23.873 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.873 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-21T03:42:23.873 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "zabbix", 2024-09-21T03:42:23.873 INFO:teuthology.orchestra.run.smithi089.stdout: "can_run": true, 2024-09-21T03:42:23.873 INFO:teuthology.orchestra.run.smithi089.stdout: "error_string": "", 2024-09-21T03:42:23.873 INFO:teuthology.orchestra.run.smithi089.stdout: "module_options": { 2024-09-21T03:42:23.873 INFO:teuthology.orchestra.run.smithi089.stdout: "discovery_interval": { 2024-09-21T03:42:23.873 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "discovery_interval", 2024-09-21T03:42:23.873 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "uint", 2024-09-21T03:42:23.873 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.873 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:23.874 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "100", 2024-09-21T03:42:23.874 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.874 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.874 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.874 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.874 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.874 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.874 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.874 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.874 INFO:teuthology.orchestra.run.smithi089.stdout: "identifier": { 2024-09-21T03:42:23.874 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "identifier", 2024-09-21T03:42:23.874 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:23.874 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.875 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:23.875 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-21T03:42:23.875 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.875 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.875 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.875 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.875 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.875 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.875 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.875 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.875 INFO:teuthology.orchestra.run.smithi089.stdout: "interval": { 2024-09-21T03:42:23.875 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "interval", 2024-09-21T03:42:23.875 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "secs", 2024-09-21T03:42:23.876 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.876 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:23.876 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "60", 2024-09-21T03:42:23.876 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.876 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.876 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.876 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.876 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.876 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.876 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.876 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.876 INFO:teuthology.orchestra.run.smithi089.stdout: "log_level": { 2024-09-21T03:42:23.876 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_level", 2024-09-21T03:42:23.877 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:23.877 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.877 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:23.877 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-21T03:42:23.877 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.877 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.877 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-21T03:42:23.877 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2024-09-21T03:42:23.877 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2024-09-21T03:42:23.877 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2024-09-21T03:42:23.877 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2024-09-21T03:42:23.877 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2024-09-21T03:42:23.878 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2024-09-21T03:42:23.878 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-21T03:42:23.878 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.878 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.878 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.878 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.878 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.878 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster": { 2024-09-21T03:42:23.878 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster", 2024-09-21T03:42:23.878 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-21T03:42:23.878 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.878 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:23.879 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-21T03:42:23.879 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.879 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.879 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.879 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.879 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.879 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.879 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.879 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.879 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster_level": { 2024-09-21T03:42:23.879 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster_level", 2024-09-21T03:42:23.879 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:23.879 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.879 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:23.880 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "info", 2024-09-21T03:42:23.880 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.880 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.880 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-21T03:42:23.880 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2024-09-21T03:42:23.880 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2024-09-21T03:42:23.880 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2024-09-21T03:42:23.880 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2024-09-21T03:42:23.880 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2024-09-21T03:42:23.880 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2024-09-21T03:42:23.880 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-21T03:42:23.880 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.880 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.881 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.881 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.881 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.881 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_file": { 2024-09-21T03:42:23.881 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_file", 2024-09-21T03:42:23.881 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-21T03:42:23.881 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.881 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:23.881 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-21T03:42:23.881 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.881 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.881 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.882 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.882 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.882 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.882 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.882 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.882 INFO:teuthology.orchestra.run.smithi089.stdout: "sqlite3_killpoint": { 2024-09-21T03:42:23.882 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:42:23.882 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2024-09-21T03:42:23.882 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "dev", 2024-09-21T03:42:23.882 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-21T03:42:23.882 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "0", 2024-09-21T03:42:23.882 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.882 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.882 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.883 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.883 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.883 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.883 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.883 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.883 INFO:teuthology.orchestra.run.smithi089.stdout: "zabbix_host": { 2024-09-21T03:42:23.883 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "zabbix_host", 2024-09-21T03:42:23.883 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:23.883 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.883 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:23.883 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-21T03:42:23.883 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.883 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.884 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.884 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.884 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.884 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.884 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.884 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.884 INFO:teuthology.orchestra.run.smithi089.stdout: "zabbix_port": { 2024-09-21T03:42:23.884 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "zabbix_port", 2024-09-21T03:42:23.884 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2024-09-21T03:42:23.884 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.884 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:23.884 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "10051", 2024-09-21T03:42:23.884 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.885 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.885 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.885 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.885 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.885 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.885 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.885 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.885 INFO:teuthology.orchestra.run.smithi089.stdout: "zabbix_sender": { 2024-09-21T03:42:23.885 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "zabbix_sender", 2024-09-21T03:42:23.885 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-21T03:42:23.885 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-21T03:42:23.885 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-21T03:42:23.886 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-09-21T03:42:23.886 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-21T03:42:23.886 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-21T03:42:23.886 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-21T03:42:23.886 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-21T03:42:23.886 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-21T03:42:23.886 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-21T03:42:23.886 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-21T03:42:23.886 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-21T03:42:23.886 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-21T03:42:23.886 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-21T03:42:23.886 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-21T03:42:23.887 INFO:teuthology.orchestra.run.smithi089.stdout: "services": { 2024-09-21T03:42:23.887 INFO:teuthology.orchestra.run.smithi089.stdout: "dashboard": "https://172.21.15.89:8443/" 2024-09-21T03:42:23.887 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.887 INFO:teuthology.orchestra.run.smithi089.stdout: "always_on_modules": { 2024-09-21T03:42:23.887 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus": [ 2024-09-21T03:42:23.887 INFO:teuthology.orchestra.run.smithi089.stdout: "balancer", 2024-09-21T03:42:23.887 INFO:teuthology.orchestra.run.smithi089.stdout: "crash", 2024-09-21T03:42:23.887 INFO:teuthology.orchestra.run.smithi089.stdout: "devicehealth", 2024-09-21T03:42:23.887 INFO:teuthology.orchestra.run.smithi089.stdout: "orchestrator", 2024-09-21T03:42:23.887 INFO:teuthology.orchestra.run.smithi089.stdout: "pg_autoscaler", 2024-09-21T03:42:23.887 INFO:teuthology.orchestra.run.smithi089.stdout: "progress", 2024-09-21T03:42:23.887 INFO:teuthology.orchestra.run.smithi089.stdout: "rbd_support", 2024-09-21T03:42:23.887 INFO:teuthology.orchestra.run.smithi089.stdout: "status", 2024-09-21T03:42:23.887 INFO:teuthology.orchestra.run.smithi089.stdout: "telemetry", 2024-09-21T03:42:23.888 INFO:teuthology.orchestra.run.smithi089.stdout: "volumes" 2024-09-21T03:42:23.888 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-21T03:42:23.888 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific": [ 2024-09-21T03:42:23.888 INFO:teuthology.orchestra.run.smithi089.stdout: "balancer", 2024-09-21T03:42:23.888 INFO:teuthology.orchestra.run.smithi089.stdout: "crash", 2024-09-21T03:42:23.888 INFO:teuthology.orchestra.run.smithi089.stdout: "devicehealth", 2024-09-21T03:42:23.888 INFO:teuthology.orchestra.run.smithi089.stdout: "orchestrator", 2024-09-21T03:42:23.888 INFO:teuthology.orchestra.run.smithi089.stdout: "pg_autoscaler", 2024-09-21T03:42:23.888 INFO:teuthology.orchestra.run.smithi089.stdout: "progress", 2024-09-21T03:42:23.888 INFO:teuthology.orchestra.run.smithi089.stdout: "rbd_support", 2024-09-21T03:42:23.888 INFO:teuthology.orchestra.run.smithi089.stdout: "status", 2024-09-21T03:42:23.888 INFO:teuthology.orchestra.run.smithi089.stdout: "telemetry", 2024-09-21T03:42:23.888 INFO:teuthology.orchestra.run.smithi089.stdout: "volumes" 2024-09-21T03:42:23.888 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-21T03:42:23.889 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy": [ 2024-09-21T03:42:23.889 INFO:teuthology.orchestra.run.smithi089.stdout: "balancer", 2024-09-21T03:42:23.889 INFO:teuthology.orchestra.run.smithi089.stdout: "crash", 2024-09-21T03:42:23.889 INFO:teuthology.orchestra.run.smithi089.stdout: "devicehealth", 2024-09-21T03:42:23.889 INFO:teuthology.orchestra.run.smithi089.stdout: "orchestrator", 2024-09-21T03:42:23.889 INFO:teuthology.orchestra.run.smithi089.stdout: "pg_autoscaler", 2024-09-21T03:42:23.889 INFO:teuthology.orchestra.run.smithi089.stdout: "progress", 2024-09-21T03:42:23.889 INFO:teuthology.orchestra.run.smithi089.stdout: "rbd_support", 2024-09-21T03:42:23.889 INFO:teuthology.orchestra.run.smithi089.stdout: "status", 2024-09-21T03:42:23.889 INFO:teuthology.orchestra.run.smithi089.stdout: "telemetry", 2024-09-21T03:42:23.889 INFO:teuthology.orchestra.run.smithi089.stdout: "volumes" 2024-09-21T03:42:23.889 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-21T03:42:23.890 INFO:teuthology.orchestra.run.smithi089.stdout: "reef": [ 2024-09-21T03:42:23.890 INFO:teuthology.orchestra.run.smithi089.stdout: "balancer", 2024-09-21T03:42:23.890 INFO:teuthology.orchestra.run.smithi089.stdout: "crash", 2024-09-21T03:42:23.890 INFO:teuthology.orchestra.run.smithi089.stdout: "devicehealth", 2024-09-21T03:42:23.890 INFO:teuthology.orchestra.run.smithi089.stdout: "orchestrator", 2024-09-21T03:42:23.890 INFO:teuthology.orchestra.run.smithi089.stdout: "pg_autoscaler", 2024-09-21T03:42:23.890 INFO:teuthology.orchestra.run.smithi089.stdout: "progress", 2024-09-21T03:42:23.890 INFO:teuthology.orchestra.run.smithi089.stdout: "rbd_support", 2024-09-21T03:42:23.890 INFO:teuthology.orchestra.run.smithi089.stdout: "status", 2024-09-21T03:42:23.890 INFO:teuthology.orchestra.run.smithi089.stdout: "telemetry", 2024-09-21T03:42:23.890 INFO:teuthology.orchestra.run.smithi089.stdout: "volumes" 2024-09-21T03:42:23.891 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-21T03:42:23.891 INFO:teuthology.orchestra.run.smithi089.stdout: "squid": [ 2024-09-21T03:42:23.891 INFO:teuthology.orchestra.run.smithi089.stdout: "balancer", 2024-09-21T03:42:23.891 INFO:teuthology.orchestra.run.smithi089.stdout: "crash", 2024-09-21T03:42:23.891 INFO:teuthology.orchestra.run.smithi089.stdout: "devicehealth", 2024-09-21T03:42:23.891 INFO:teuthology.orchestra.run.smithi089.stdout: "orchestrator", 2024-09-21T03:42:23.891 INFO:teuthology.orchestra.run.smithi089.stdout: "pg_autoscaler", 2024-09-21T03:42:23.891 INFO:teuthology.orchestra.run.smithi089.stdout: "progress", 2024-09-21T03:42:23.891 INFO:teuthology.orchestra.run.smithi089.stdout: "rbd_support", 2024-09-21T03:42:23.891 INFO:teuthology.orchestra.run.smithi089.stdout: "status", 2024-09-21T03:42:23.891 INFO:teuthology.orchestra.run.smithi089.stdout: "telemetry", 2024-09-21T03:42:23.892 INFO:teuthology.orchestra.run.smithi089.stdout: "volumes" 2024-09-21T03:42:23.892 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-21T03:42:23.892 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.892 INFO:teuthology.orchestra.run.smithi089.stdout: "last_failure_osd_epoch": 3, 2024-09-21T03:42:23.892 INFO:teuthology.orchestra.run.smithi089.stdout: "active_clients": [ 2024-09-21T03:42:23.892 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-21T03:42:23.892 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "devicehealth", 2024-09-21T03:42:23.892 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-09-21T03:42:23.892 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-21T03:42:23.892 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-09-21T03:42:23.892 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:0", 2024-09-21T03:42:23.893 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 3874566624 2024-09-21T03:42:23.893 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-21T03:42:23.893 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-21T03:42:23.893 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.893 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-21T03:42:23.893 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "libcephsqlite", 2024-09-21T03:42:23.893 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-09-21T03:42:23.893 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-21T03:42:23.893 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-09-21T03:42:23.893 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:0", 2024-09-21T03:42:23.893 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 1821892963 2024-09-21T03:42:23.893 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-21T03:42:23.893 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-21T03:42:23.894 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.894 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-21T03:42:23.894 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "rbd_support", 2024-09-21T03:42:23.894 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-09-21T03:42:23.894 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-21T03:42:23.894 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-09-21T03:42:23.894 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:0", 2024-09-21T03:42:23.894 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 2892236499 2024-09-21T03:42:23.894 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-21T03:42:23.894 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-21T03:42:23.894 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:23.894 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-21T03:42:23.895 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "volumes", 2024-09-21T03:42:23.895 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-09-21T03:42:23.895 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-21T03:42:23.895 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-09-21T03:42:23.895 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:0", 2024-09-21T03:42:23.895 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 1955004802 2024-09-21T03:42:23.895 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-21T03:42:23.895 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-21T03:42:23.895 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-21T03:42:23.895 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-21T03:42:23.895 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-09-21T03:42:23.896 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:22.305+0000 7fad25951640 1 -- 172.21.15.89:0/998049441 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7fad080b6510 msgr2=0x7fad080b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:23.896 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:22.305+0000 7fad25951640 1 --2- 172.21.15.89:0/998049441 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7fad080b6510 0x7fad080b8a20 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fad1400abe0 tx=0x7fad1400a2d0 comp rx=0 tx=0).stop 2024-09-21T03:42:23.896 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:22.305+0000 7fad25951640 1 -- 172.21.15.89:0/998049441 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fad20102580 msgr2=0x7fad20199e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:23.896 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:22.305+0000 7fad25951640 1 --2- 172.21.15.89:0/998049441 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fad20102580 0x7fad20199e20 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7fad1000e9f0 tx=0x7fad1000edf0 comp rx=0 tx=0).stop 2024-09-21T03:42:23.896 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:22.305+0000 7fad25951640 1 -- 172.21.15.89:0/998049441 shutdown_connections 2024-09-21T03:42:23.897 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:22.305+0000 7fad25951640 1 --2- 172.21.15.89:0/998049441 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7fad080b6510 0x7fad080b8a20 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:23.897 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:22.305+0000 7fad25951640 1 --2- 172.21.15.89:0/998049441 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fad20103600 0x7fad2019a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:23.897 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:22.305+0000 7fad25951640 1 --2- 172.21.15.89:0/998049441 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fad20103fa0 0x7fad2019afd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:23.897 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:22.305+0000 7fad25951640 1 --2- 172.21.15.89:0/998049441 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fad20102580 0x7fad20199e20 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:23.897 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:22.305+0000 7fad25951640 1 -- 172.21.15.89:0/998049441 >> 172.21.15.89:0/998049441 conn(0x7fad200fdee0 msgr2=0x7fad200ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:42:23.897 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:22.305+0000 7fad25951640 1 -- 172.21.15.89:0/998049441 shutdown_connections 2024-09-21T03:42:23.897 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:22.305+0000 7fad25951640 1 -- 172.21.15.89:0/998049441 wait complete. 2024-09-21T03:42:23.897 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:42:21 smithi089 ceph-mon[30928]: from='client.? 172.21.15.89:0/2219579257' entity='client.admin' cmd={"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} : dispatch 2024-09-21T03:42:23.897 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:42:21 smithi089 ceph-mon[30928]: from='client.? 172.21.15.89:0/2534929872' entity='client.admin' cmd={"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} : dispatch 2024-09-21T03:42:23.897 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:42:21 smithi089 ceph-mon[30928]: from='client.? 172.21.15.89:0/2103975775' entity='client.admin' cmd={"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} : dispatch 2024-09-21T03:42:23.947 DEBUG:tasks.check_counter:Getting stats from a 2024-09-21T03:42:23.947 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 7c374098-77c9-11ef-baf5-efdc52797490 -- ceph daemon mds.a perf dump 2024-09-21T03:42:23.948 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:42:22 smithi089 ceph-mon[30928]: pgmap v449: 129 pgs: 129 active+clean; 1.1 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 745 KiB/s wr, 168 op/s 2024-09-21T03:42:23.948 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:42:22 smithi089 ceph-mon[30928]: from='client.? 172.21.15.89:0/1527658912' entity='client.admin' cmd={"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} : dispatch 2024-09-21T03:42:23.948 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:42:22 smithi089 ceph-mon[30928]: from='client.? 172.21.15.89:0/998049441' entity='client.admin' cmd={"prefix": "mgr dump", "format": "json-pretty"} : dispatch 2024-09-21T03:42:23.949 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:42:22 smithi110 ceph-mon[34000]: pgmap v449: 129 pgs: 129 active+clean; 1.1 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 745 KiB/s wr, 168 op/s 2024-09-21T03:42:23.949 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:42:22 smithi110 ceph-mon[34000]: from='client.? 172.21.15.89:0/1527658912' entity='client.admin' cmd={"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} : dispatch 2024-09-21T03:42:23.949 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:42:22 smithi110 ceph-mon[34000]: from='client.? 172.21.15.89:0/998049441' entity='client.admin' cmd={"prefix": "mgr dump", "format": "json-pretty"} : dispatch 2024-09-21T03:42:23.949 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:42:22 smithi114 ceph-mon[34140]: pgmap v449: 129 pgs: 129 active+clean; 1.1 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 745 KiB/s wr, 168 op/s 2024-09-21T03:42:23.949 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:42:22 smithi114 ceph-mon[34140]: from='client.? 172.21.15.89:0/1527658912' entity='client.admin' cmd={"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} : dispatch 2024-09-21T03:42:23.949 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:42:22 smithi114 ceph-mon[34140]: from='client.? 172.21.15.89:0/998049441' entity='client.admin' cmd={"prefix": "mgr dump", "format": "json-pretty"} : dispatch 2024-09-21T03:42:24.207 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/7c374098-77c9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:42:24.743 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-09-21T03:42:24.743 INFO:teuthology.orchestra.run.smithi089.stdout: "AsyncMessenger::Worker-0": { 2024-09-21T03:42:24.743 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_recv_messages": 94, 2024-09-21T03:42:24.743 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_send_messages": 89, 2024-09-21T03:42:24.743 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_recv_bytes": 800880, 2024-09-21T03:42:24.743 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_send_bytes": 53952, 2024-09-21T03:42:24.743 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_created_connections": 4, 2024-09-21T03:42:24.743 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_active_connections": 0, 2024-09-21T03:42:24.744 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_running_total_time": 0.020315536, 2024-09-21T03:42:24.744 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_running_send_time": 0.007468203, 2024-09-21T03:42:24.744 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_running_recv_time": 0.018989418, 2024-09-21T03:42:24.744 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-21T03:42:24.744 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T03:42:24.744 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 88, 2024-09-21T03:42:24.744 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.004518319, 2024-09-21T03:42:24.744 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000051344 2024-09-21T03:42:24.744 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:24.744 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_handle_ack_lat": { 2024-09-21T03:42:24.744 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2024-09-21T03:42:24.744 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2024-09-21T03:42:24.744 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2024-09-21T03:42:24.745 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:24.745 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_recv_encrypted_bytes": 800880, 2024-09-21T03:42:24.745 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_send_encrypted_bytes": 53952 2024-09-21T03:42:24.745 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:24.745 INFO:teuthology.orchestra.run.smithi089.stdout: "AsyncMessenger::Worker-1": { 2024-09-21T03:42:24.745 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_recv_messages": 5, 2024-09-21T03:42:24.745 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_send_messages": 70, 2024-09-21T03:42:24.745 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_recv_bytes": 3120, 2024-09-21T03:42:24.745 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_send_bytes": 92608, 2024-09-21T03:42:24.745 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_created_connections": 3, 2024-09-21T03:42:24.745 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_active_connections": 2, 2024-09-21T03:42:24.745 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_running_total_time": 0.009624029, 2024-09-21T03:42:24.745 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_running_send_time": 0.005435270, 2024-09-21T03:42:24.745 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_running_recv_time": 0.003592658, 2024-09-21T03:42:24.746 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-21T03:42:24.746 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T03:42:24.746 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 69, 2024-09-21T03:42:24.746 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.007339686, 2024-09-21T03:42:24.746 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000106372 2024-09-21T03:42:24.746 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:24.746 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_handle_ack_lat": { 2024-09-21T03:42:24.746 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2024-09-21T03:42:24.746 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2024-09-21T03:42:24.746 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2024-09-21T03:42:24.746 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:24.746 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_recv_encrypted_bytes": 3120, 2024-09-21T03:42:24.746 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_send_encrypted_bytes": 92608 2024-09-21T03:42:24.747 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:24.747 INFO:teuthology.orchestra.run.smithi089.stdout: "AsyncMessenger::Worker-2": { 2024-09-21T03:42:24.747 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_recv_messages": 0, 2024-09-21T03:42:24.747 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_send_messages": 2, 2024-09-21T03:42:24.747 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_recv_bytes": 0, 2024-09-21T03:42:24.747 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_send_bytes": 0, 2024-09-21T03:42:24.747 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_created_connections": 2, 2024-09-21T03:42:24.747 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_active_connections": 1, 2024-09-21T03:42:24.747 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_running_total_time": 0.000278664, 2024-09-21T03:42:24.747 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_running_send_time": 0.000000000, 2024-09-21T03:42:24.747 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_running_recv_time": 0.000224414, 2024-09-21T03:42:24.747 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-21T03:42:24.747 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T03:42:24.747 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2024-09-21T03:42:24.747 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2024-09-21T03:42:24.748 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2024-09-21T03:42:24.748 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:24.748 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_handle_ack_lat": { 2024-09-21T03:42:24.748 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2024-09-21T03:42:24.748 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2024-09-21T03:42:24.748 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2024-09-21T03:42:24.748 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:24.748 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-21T03:42:24.748 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_send_encrypted_bytes": 0 2024-09-21T03:42:24.748 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:24.748 INFO:teuthology.orchestra.run.smithi089.stdout: "cct": { 2024-09-21T03:42:24.748 INFO:teuthology.orchestra.run.smithi089.stdout: "total_workers": 0, 2024-09-21T03:42:24.748 INFO:teuthology.orchestra.run.smithi089.stdout: "unhealthy_workers": 0 2024-09-21T03:42:24.748 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:24.749 INFO:teuthology.orchestra.run.smithi089.stdout: "mempool": { 2024-09-21T03:42:24.749 INFO:teuthology.orchestra.run.smithi089.stdout: "bloom_filter_bytes": 0, 2024-09-21T03:42:24.749 INFO:teuthology.orchestra.run.smithi089.stdout: "bloom_filter_items": 0, 2024-09-21T03:42:24.749 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_alloc_bytes": 0, 2024-09-21T03:42:24.749 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_alloc_items": 0, 2024-09-21T03:42:24.749 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_cache_data_bytes": 0, 2024-09-21T03:42:24.749 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_cache_data_items": 0, 2024-09-21T03:42:24.749 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-21T03:42:24.749 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_cache_onode_items": 0, 2024-09-21T03:42:24.749 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-21T03:42:24.749 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_cache_meta_items": 0, 2024-09-21T03:42:24.749 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_cache_other_bytes": 0, 2024-09-21T03:42:24.749 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_cache_other_items": 0, 2024-09-21T03:42:24.749 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-21T03:42:24.750 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_cache_buffer_items": 0, 2024-09-21T03:42:24.750 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_extent_bytes": 0, 2024-09-21T03:42:24.750 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_extent_items": 0, 2024-09-21T03:42:24.750 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_blob_bytes": 0, 2024-09-21T03:42:24.750 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_blob_items": 0, 2024-09-21T03:42:24.750 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-21T03:42:24.750 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_shared_blob_items": 0, 2024-09-21T03:42:24.750 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-21T03:42:24.750 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_inline_bl_items": 0, 2024-09-21T03:42:24.750 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_fsck_bytes": 0, 2024-09-21T03:42:24.750 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_fsck_items": 0, 2024-09-21T03:42:24.750 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_txc_bytes": 0, 2024-09-21T03:42:24.750 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_txc_items": 0, 2024-09-21T03:42:24.750 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-21T03:42:24.750 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_writing_deferred_items": 0, 2024-09-21T03:42:24.751 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_writing_bytes": 0, 2024-09-21T03:42:24.751 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_writing_items": 0, 2024-09-21T03:42:24.751 INFO:teuthology.orchestra.run.smithi089.stdout: "bluefs_bytes": 0, 2024-09-21T03:42:24.751 INFO:teuthology.orchestra.run.smithi089.stdout: "bluefs_items": 0, 2024-09-21T03:42:24.751 INFO:teuthology.orchestra.run.smithi089.stdout: "bluefs_file_reader_bytes": 0, 2024-09-21T03:42:24.751 INFO:teuthology.orchestra.run.smithi089.stdout: "bluefs_file_reader_items": 0, 2024-09-21T03:42:24.751 INFO:teuthology.orchestra.run.smithi089.stdout: "bluefs_file_writer_bytes": 0, 2024-09-21T03:42:24.751 INFO:teuthology.orchestra.run.smithi089.stdout: "bluefs_file_writer_items": 0, 2024-09-21T03:42:24.751 INFO:teuthology.orchestra.run.smithi089.stdout: "buffer_anon_bytes": 148841, 2024-09-21T03:42:24.751 INFO:teuthology.orchestra.run.smithi089.stdout: "buffer_anon_items": 28, 2024-09-21T03:42:24.751 INFO:teuthology.orchestra.run.smithi089.stdout: "buffer_meta_bytes": 0, 2024-09-21T03:42:24.751 INFO:teuthology.orchestra.run.smithi089.stdout: "buffer_meta_items": 0, 2024-09-21T03:42:24.751 INFO:teuthology.orchestra.run.smithi089.stdout: "osd_bytes": 0, 2024-09-21T03:42:24.751 INFO:teuthology.orchestra.run.smithi089.stdout: "osd_items": 0, 2024-09-21T03:42:24.751 INFO:teuthology.orchestra.run.smithi089.stdout: "osd_mapbl_bytes": 0, 2024-09-21T03:42:24.752 INFO:teuthology.orchestra.run.smithi089.stdout: "osd_mapbl_items": 0, 2024-09-21T03:42:24.752 INFO:teuthology.orchestra.run.smithi089.stdout: "osd_pglog_bytes": 0, 2024-09-21T03:42:24.752 INFO:teuthology.orchestra.run.smithi089.stdout: "osd_pglog_items": 0, 2024-09-21T03:42:24.752 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap_bytes": 0, 2024-09-21T03:42:24.752 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap_items": 0, 2024-09-21T03:42:24.752 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap_mapping_bytes": 0, 2024-09-21T03:42:24.752 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap_mapping_items": 0, 2024-09-21T03:42:24.752 INFO:teuthology.orchestra.run.smithi089.stdout: "pgmap_bytes": 0, 2024-09-21T03:42:24.752 INFO:teuthology.orchestra.run.smithi089.stdout: "pgmap_items": 0, 2024-09-21T03:42:24.752 INFO:teuthology.orchestra.run.smithi089.stdout: "mds_co_bytes": 968, 2024-09-21T03:42:24.752 INFO:teuthology.orchestra.run.smithi089.stdout: "mds_co_items": 2, 2024-09-21T03:42:24.752 INFO:teuthology.orchestra.run.smithi089.stdout: "unittest_1_bytes": 0, 2024-09-21T03:42:24.752 INFO:teuthology.orchestra.run.smithi089.stdout: "unittest_1_items": 0, 2024-09-21T03:42:24.752 INFO:teuthology.orchestra.run.smithi089.stdout: "unittest_2_bytes": 0, 2024-09-21T03:42:24.752 INFO:teuthology.orchestra.run.smithi089.stdout: "unittest_2_items": 0 2024-09-21T03:42:24.753 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:24.753 INFO:teuthology.orchestra.run.smithi089.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-21T03:42:24.753 INFO:teuthology.orchestra.run.smithi089.stdout: "val": 0, 2024-09-21T03:42:24.753 INFO:teuthology.orchestra.run.smithi089.stdout: "max": 104857600, 2024-09-21T03:42:24.753 INFO:teuthology.orchestra.run.smithi089.stdout: "get_started": 0, 2024-09-21T03:42:24.753 INFO:teuthology.orchestra.run.smithi089.stdout: "get": 95, 2024-09-21T03:42:24.753 INFO:teuthology.orchestra.run.smithi089.stdout: "get_sum": 787654, 2024-09-21T03:42:24.753 INFO:teuthology.orchestra.run.smithi089.stdout: "get_or_fail_fail": 0, 2024-09-21T03:42:24.753 INFO:teuthology.orchestra.run.smithi089.stdout: "get_or_fail_success": 95, 2024-09-21T03:42:24.753 INFO:teuthology.orchestra.run.smithi089.stdout: "take": 0, 2024-09-21T03:42:24.753 INFO:teuthology.orchestra.run.smithi089.stdout: "take_sum": 0, 2024-09-21T03:42:24.753 INFO:teuthology.orchestra.run.smithi089.stdout: "put": 95, 2024-09-21T03:42:24.753 INFO:teuthology.orchestra.run.smithi089.stdout: "put_sum": 787654, 2024-09-21T03:42:24.753 INFO:teuthology.orchestra.run.smithi089.stdout: "wait": { 2024-09-21T03:42:24.753 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2024-09-21T03:42:24.753 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2024-09-21T03:42:24.754 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2024-09-21T03:42:24.754 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-21T03:42:24.754 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-21T03:42:24.754 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-09-21T03:42:25.211 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:42:24 smithi114 ceph-mon[34140]: pgmap v450: 129 pgs: 129 active+clean; 1.1 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 745 KiB/s wr, 171 op/s 2024-09-21T03:42:25.250 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:42:24 smithi110 ceph-mon[34000]: pgmap v450: 129 pgs: 129 active+clean; 1.1 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 745 KiB/s wr, 171 op/s 2024-09-21T03:42:25.311 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:42:24 smithi089 ceph-mon[30928]: pgmap v450: 129 pgs: 129 active+clean; 1.1 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 745 KiB/s wr, 171 op/s 2024-09-21T03:42:25.516 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.a 2024-09-21T03:42:25.516 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.a 2024-09-21T03:42:25.516 DEBUG:tasks.check_counter:Getting stats from d 2024-09-21T03:42:25.516 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 7c374098-77c9-11ef-baf5-efdc52797490 -- ceph daemon mds.d perf dump 2024-09-21T03:42:25.763 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/7c374098-77c9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:42:26.277 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-09-21T03:42:26.278 INFO:teuthology.orchestra.run.smithi089.stdout: "AsyncMessenger::Worker-0": { 2024-09-21T03:42:26.278 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_recv_messages": 4, 2024-09-21T03:42:26.278 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_send_messages": 4, 2024-09-21T03:42:26.278 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_recv_bytes": 2960, 2024-09-21T03:42:26.278 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_send_bytes": 448, 2024-09-21T03:42:26.278 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_created_connections": 4, 2024-09-21T03:42:26.278 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_active_connections": 0, 2024-09-21T03:42:26.278 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_running_total_time": 0.002950487, 2024-09-21T03:42:26.278 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_running_send_time": 0.000045418, 2024-09-21T03:42:26.278 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_running_recv_time": 0.002092214, 2024-09-21T03:42:26.278 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-21T03:42:26.278 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T03:42:26.279 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 3, 2024-09-21T03:42:26.279 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.003872562, 2024-09-21T03:42:26.279 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.001290854 2024-09-21T03:42:26.279 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:26.279 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_handle_ack_lat": { 2024-09-21T03:42:26.279 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2024-09-21T03:42:26.279 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2024-09-21T03:42:26.279 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2024-09-21T03:42:26.279 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:26.279 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_recv_encrypted_bytes": 2960, 2024-09-21T03:42:26.279 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_send_encrypted_bytes": 448 2024-09-21T03:42:26.279 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:26.279 INFO:teuthology.orchestra.run.smithi089.stdout: "AsyncMessenger::Worker-1": { 2024-09-21T03:42:26.279 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_recv_messages": 1, 2024-09-21T03:42:26.280 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_send_messages": 72, 2024-09-21T03:42:26.280 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_recv_bytes": 160, 2024-09-21T03:42:26.280 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_send_bytes": 92864, 2024-09-21T03:42:26.280 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_created_connections": 3, 2024-09-21T03:42:26.280 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_active_connections": 2, 2024-09-21T03:42:26.280 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_running_total_time": 0.007250814, 2024-09-21T03:42:26.280 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_running_send_time": 0.005115295, 2024-09-21T03:42:26.280 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_running_recv_time": 0.001388581, 2024-09-21T03:42:26.280 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-21T03:42:26.280 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T03:42:26.280 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 70, 2024-09-21T03:42:26.280 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.003436225, 2024-09-21T03:42:26.280 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000049088 2024-09-21T03:42:26.280 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:26.280 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_handle_ack_lat": { 2024-09-21T03:42:26.281 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2024-09-21T03:42:26.281 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2024-09-21T03:42:26.281 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2024-09-21T03:42:26.281 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:26.281 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_recv_encrypted_bytes": 160, 2024-09-21T03:42:26.281 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_send_encrypted_bytes": 92864 2024-09-21T03:42:26.281 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:26.281 INFO:teuthology.orchestra.run.smithi089.stdout: "AsyncMessenger::Worker-2": { 2024-09-21T03:42:26.281 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_recv_messages": 101, 2024-09-21T03:42:26.281 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_send_messages": 96, 2024-09-21T03:42:26.281 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_recv_bytes": 802848, 2024-09-21T03:42:26.281 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_send_bytes": 57952, 2024-09-21T03:42:26.281 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_created_connections": 2, 2024-09-21T03:42:26.281 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_active_connections": 1, 2024-09-21T03:42:26.282 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_running_total_time": 0.021183984, 2024-09-21T03:42:26.282 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_running_send_time": 0.008538437, 2024-09-21T03:42:26.282 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_running_recv_time": 0.019265938, 2024-09-21T03:42:26.282 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-21T03:42:26.282 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T03:42:26.282 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 95, 2024-09-21T03:42:26.282 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.005128053, 2024-09-21T03:42:26.282 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000053979 2024-09-21T03:42:26.282 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:26.282 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_handle_ack_lat": { 2024-09-21T03:42:26.282 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2024-09-21T03:42:26.282 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2024-09-21T03:42:26.282 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2024-09-21T03:42:26.282 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:26.283 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_recv_encrypted_bytes": 802848, 2024-09-21T03:42:26.283 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_send_encrypted_bytes": 57952 2024-09-21T03:42:26.283 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:26.283 INFO:teuthology.orchestra.run.smithi089.stdout: "cct": { 2024-09-21T03:42:26.283 INFO:teuthology.orchestra.run.smithi089.stdout: "total_workers": 0, 2024-09-21T03:42:26.283 INFO:teuthology.orchestra.run.smithi089.stdout: "unhealthy_workers": 0 2024-09-21T03:42:26.283 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:26.283 INFO:teuthology.orchestra.run.smithi089.stdout: "mempool": { 2024-09-21T03:42:26.283 INFO:teuthology.orchestra.run.smithi089.stdout: "bloom_filter_bytes": 0, 2024-09-21T03:42:26.283 INFO:teuthology.orchestra.run.smithi089.stdout: "bloom_filter_items": 0, 2024-09-21T03:42:26.283 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_alloc_bytes": 0, 2024-09-21T03:42:26.283 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_alloc_items": 0, 2024-09-21T03:42:26.283 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_cache_data_bytes": 0, 2024-09-21T03:42:26.283 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_cache_data_items": 0, 2024-09-21T03:42:26.283 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-21T03:42:26.283 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_cache_onode_items": 0, 2024-09-21T03:42:26.284 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-21T03:42:26.284 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_cache_meta_items": 0, 2024-09-21T03:42:26.284 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_cache_other_bytes": 0, 2024-09-21T03:42:26.284 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_cache_other_items": 0, 2024-09-21T03:42:26.284 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-21T03:42:26.284 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_cache_buffer_items": 0, 2024-09-21T03:42:26.284 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_extent_bytes": 0, 2024-09-21T03:42:26.284 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_extent_items": 0, 2024-09-21T03:42:26.284 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_blob_bytes": 0, 2024-09-21T03:42:26.284 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_blob_items": 0, 2024-09-21T03:42:26.284 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-21T03:42:26.284 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_shared_blob_items": 0, 2024-09-21T03:42:26.284 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-21T03:42:26.284 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_inline_bl_items": 0, 2024-09-21T03:42:26.284 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_fsck_bytes": 0, 2024-09-21T03:42:26.284 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_fsck_items": 0, 2024-09-21T03:42:26.285 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_txc_bytes": 0, 2024-09-21T03:42:26.285 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_txc_items": 0, 2024-09-21T03:42:26.285 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-21T03:42:26.285 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_writing_deferred_items": 0, 2024-09-21T03:42:26.285 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_writing_bytes": 0, 2024-09-21T03:42:26.285 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_writing_items": 0, 2024-09-21T03:42:26.285 INFO:teuthology.orchestra.run.smithi089.stdout: "bluefs_bytes": 0, 2024-09-21T03:42:26.285 INFO:teuthology.orchestra.run.smithi089.stdout: "bluefs_items": 0, 2024-09-21T03:42:26.285 INFO:teuthology.orchestra.run.smithi089.stdout: "bluefs_file_reader_bytes": 0, 2024-09-21T03:42:26.285 INFO:teuthology.orchestra.run.smithi089.stdout: "bluefs_file_reader_items": 0, 2024-09-21T03:42:26.285 INFO:teuthology.orchestra.run.smithi089.stdout: "bluefs_file_writer_bytes": 0, 2024-09-21T03:42:26.285 INFO:teuthology.orchestra.run.smithi089.stdout: "bluefs_file_writer_items": 0, 2024-09-21T03:42:26.285 INFO:teuthology.orchestra.run.smithi089.stdout: "buffer_anon_bytes": 148823, 2024-09-21T03:42:26.285 INFO:teuthology.orchestra.run.smithi089.stdout: "buffer_anon_items": 27, 2024-09-21T03:42:26.285 INFO:teuthology.orchestra.run.smithi089.stdout: "buffer_meta_bytes": 0, 2024-09-21T03:42:26.285 INFO:teuthology.orchestra.run.smithi089.stdout: "buffer_meta_items": 0, 2024-09-21T03:42:26.286 INFO:teuthology.orchestra.run.smithi089.stdout: "osd_bytes": 0, 2024-09-21T03:42:26.286 INFO:teuthology.orchestra.run.smithi089.stdout: "osd_items": 0, 2024-09-21T03:42:26.286 INFO:teuthology.orchestra.run.smithi089.stdout: "osd_mapbl_bytes": 0, 2024-09-21T03:42:26.286 INFO:teuthology.orchestra.run.smithi089.stdout: "osd_mapbl_items": 0, 2024-09-21T03:42:26.286 INFO:teuthology.orchestra.run.smithi089.stdout: "osd_pglog_bytes": 0, 2024-09-21T03:42:26.286 INFO:teuthology.orchestra.run.smithi089.stdout: "osd_pglog_items": 0, 2024-09-21T03:42:26.286 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap_bytes": 0, 2024-09-21T03:42:26.286 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap_items": 0, 2024-09-21T03:42:26.286 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap_mapping_bytes": 0, 2024-09-21T03:42:26.286 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap_mapping_items": 0, 2024-09-21T03:42:26.286 INFO:teuthology.orchestra.run.smithi089.stdout: "pgmap_bytes": 0, 2024-09-21T03:42:26.286 INFO:teuthology.orchestra.run.smithi089.stdout: "pgmap_items": 0, 2024-09-21T03:42:26.286 INFO:teuthology.orchestra.run.smithi089.stdout: "mds_co_bytes": 968, 2024-09-21T03:42:26.286 INFO:teuthology.orchestra.run.smithi089.stdout: "mds_co_items": 2, 2024-09-21T03:42:26.287 INFO:teuthology.orchestra.run.smithi089.stdout: "unittest_1_bytes": 0, 2024-09-21T03:42:26.287 INFO:teuthology.orchestra.run.smithi089.stdout: "unittest_1_items": 0, 2024-09-21T03:42:26.287 INFO:teuthology.orchestra.run.smithi089.stdout: "unittest_2_bytes": 0, 2024-09-21T03:42:26.287 INFO:teuthology.orchestra.run.smithi089.stdout: "unittest_2_items": 0 2024-09-21T03:42:26.287 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:26.287 INFO:teuthology.orchestra.run.smithi089.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-21T03:42:26.287 INFO:teuthology.orchestra.run.smithi089.stdout: "val": 0, 2024-09-21T03:42:26.287 INFO:teuthology.orchestra.run.smithi089.stdout: "max": 104857600, 2024-09-21T03:42:26.287 INFO:teuthology.orchestra.run.smithi089.stdout: "get_started": 0, 2024-09-21T03:42:26.287 INFO:teuthology.orchestra.run.smithi089.stdout: "get": 102, 2024-09-21T03:42:26.287 INFO:teuthology.orchestra.run.smithi089.stdout: "get_sum": 788628, 2024-09-21T03:42:26.287 INFO:teuthology.orchestra.run.smithi089.stdout: "get_or_fail_fail": 0, 2024-09-21T03:42:26.287 INFO:teuthology.orchestra.run.smithi089.stdout: "get_or_fail_success": 102, 2024-09-21T03:42:26.287 INFO:teuthology.orchestra.run.smithi089.stdout: "take": 0, 2024-09-21T03:42:26.287 INFO:teuthology.orchestra.run.smithi089.stdout: "take_sum": 0, 2024-09-21T03:42:26.287 INFO:teuthology.orchestra.run.smithi089.stdout: "put": 102, 2024-09-21T03:42:26.287 INFO:teuthology.orchestra.run.smithi089.stdout: "put_sum": 788628, 2024-09-21T03:42:26.288 INFO:teuthology.orchestra.run.smithi089.stdout: "wait": { 2024-09-21T03:42:26.288 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2024-09-21T03:42:26.288 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2024-09-21T03:42:26.288 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2024-09-21T03:42:26.288 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-21T03:42:26.288 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-21T03:42:26.288 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-09-21T03:42:27.021 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.d 2024-09-21T03:42:27.021 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.d 2024-09-21T03:42:27.021 DEBUG:tasks.check_counter:Getting stats from g 2024-09-21T03:42:27.022 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 7c374098-77c9-11ef-baf5-efdc52797490 -- ceph daemon mds.g perf dump 2024-09-21T03:42:27.211 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:42:26 smithi114 ceph-mon[34140]: pgmap v451: 129 pgs: 129 active+clean; 1.1 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 557 KiB/s wr, 128 op/s 2024-09-21T03:42:27.250 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:42:26 smithi110 ceph-mon[34000]: pgmap v451: 129 pgs: 129 active+clean; 1.1 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 557 KiB/s wr, 128 op/s 2024-09-21T03:42:27.271 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/7c374098-77c9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:42:27.311 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:42:26 smithi089 ceph-mon[30928]: pgmap v451: 129 pgs: 129 active+clean; 1.1 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 557 KiB/s wr, 128 op/s 2024-09-21T03:42:27.814 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-09-21T03:42:27.815 INFO:teuthology.orchestra.run.smithi089.stdout: "AsyncMessenger::Worker-0": { 2024-09-21T03:42:27.815 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_recv_messages": 101, 2024-09-21T03:42:27.815 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_send_messages": 94, 2024-09-21T03:42:27.815 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_recv_bytes": 804720, 2024-09-21T03:42:27.815 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_send_bytes": 55840, 2024-09-21T03:42:27.815 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_created_connections": 4, 2024-09-21T03:42:27.815 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_active_connections": 0, 2024-09-21T03:42:27.815 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_running_total_time": 0.023730149, 2024-09-21T03:42:27.815 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_running_send_time": 0.008141252, 2024-09-21T03:42:27.815 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_running_recv_time": 0.021772633, 2024-09-21T03:42:27.816 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-21T03:42:27.816 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T03:42:27.816 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 94, 2024-09-21T03:42:27.816 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.008822774, 2024-09-21T03:42:27.816 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000093859 2024-09-21T03:42:27.816 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:27.816 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_handle_ack_lat": { 2024-09-21T03:42:27.816 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2024-09-21T03:42:27.816 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2024-09-21T03:42:27.816 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2024-09-21T03:42:27.816 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:27.816 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_recv_encrypted_bytes": 804720, 2024-09-21T03:42:27.816 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_send_encrypted_bytes": 55840 2024-09-21T03:42:27.816 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:27.816 INFO:teuthology.orchestra.run.smithi089.stdout: "AsyncMessenger::Worker-1": { 2024-09-21T03:42:27.817 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_recv_messages": 1, 2024-09-21T03:42:27.817 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_send_messages": 69, 2024-09-21T03:42:27.817 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_recv_bytes": 160, 2024-09-21T03:42:27.817 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_send_bytes": 92336, 2024-09-21T03:42:27.817 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_created_connections": 3, 2024-09-21T03:42:27.817 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_active_connections": 2, 2024-09-21T03:42:27.817 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_running_total_time": 0.007773475, 2024-09-21T03:42:27.817 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_running_send_time": 0.005690963, 2024-09-21T03:42:27.817 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_running_recv_time": 0.001383208, 2024-09-21T03:42:27.817 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-21T03:42:27.817 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T03:42:27.817 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 67, 2024-09-21T03:42:27.817 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.003162238, 2024-09-21T03:42:27.817 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000047197 2024-09-21T03:42:27.818 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:27.818 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_handle_ack_lat": { 2024-09-21T03:42:27.818 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2024-09-21T03:42:27.818 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2024-09-21T03:42:27.818 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2024-09-21T03:42:27.818 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:27.818 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_recv_encrypted_bytes": 160, 2024-09-21T03:42:27.818 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_send_encrypted_bytes": 92336 2024-09-21T03:42:27.818 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:27.818 INFO:teuthology.orchestra.run.smithi089.stdout: "AsyncMessenger::Worker-2": { 2024-09-21T03:42:27.818 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_recv_messages": 0, 2024-09-21T03:42:27.818 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_send_messages": 2, 2024-09-21T03:42:27.818 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_recv_bytes": 0, 2024-09-21T03:42:27.819 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_send_bytes": 0, 2024-09-21T03:42:27.819 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_created_connections": 2, 2024-09-21T03:42:27.819 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_active_connections": 1, 2024-09-21T03:42:27.819 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_running_total_time": 0.000352624, 2024-09-21T03:42:27.819 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_running_send_time": 0.000000000, 2024-09-21T03:42:27.819 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_running_recv_time": 0.000299513, 2024-09-21T03:42:27.819 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-21T03:42:27.819 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T03:42:27.819 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2024-09-21T03:42:27.819 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2024-09-21T03:42:27.819 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2024-09-21T03:42:27.819 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:27.819 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_handle_ack_lat": { 2024-09-21T03:42:27.820 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2024-09-21T03:42:27.820 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2024-09-21T03:42:27.820 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2024-09-21T03:42:27.820 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:27.820 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-21T03:42:27.820 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_send_encrypted_bytes": 0 2024-09-21T03:42:27.820 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:27.820 INFO:teuthology.orchestra.run.smithi089.stdout: "cct": { 2024-09-21T03:42:27.820 INFO:teuthology.orchestra.run.smithi089.stdout: "total_workers": 0, 2024-09-21T03:42:27.820 INFO:teuthology.orchestra.run.smithi089.stdout: "unhealthy_workers": 0 2024-09-21T03:42:27.820 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:27.820 INFO:teuthology.orchestra.run.smithi089.stdout: "mempool": { 2024-09-21T03:42:27.820 INFO:teuthology.orchestra.run.smithi089.stdout: "bloom_filter_bytes": 0, 2024-09-21T03:42:27.820 INFO:teuthology.orchestra.run.smithi089.stdout: "bloom_filter_items": 0, 2024-09-21T03:42:27.821 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_alloc_bytes": 0, 2024-09-21T03:42:27.821 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_alloc_items": 0, 2024-09-21T03:42:27.821 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_cache_data_bytes": 0, 2024-09-21T03:42:27.821 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_cache_data_items": 0, 2024-09-21T03:42:27.821 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-21T03:42:27.821 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_cache_onode_items": 0, 2024-09-21T03:42:27.821 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-21T03:42:27.821 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_cache_meta_items": 0, 2024-09-21T03:42:27.821 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_cache_other_bytes": 0, 2024-09-21T03:42:27.821 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_cache_other_items": 0, 2024-09-21T03:42:27.821 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-21T03:42:27.821 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_cache_buffer_items": 0, 2024-09-21T03:42:27.821 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_extent_bytes": 0, 2024-09-21T03:42:27.821 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_extent_items": 0, 2024-09-21T03:42:27.821 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_blob_bytes": 0, 2024-09-21T03:42:27.822 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_blob_items": 0, 2024-09-21T03:42:27.822 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-21T03:42:27.822 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_shared_blob_items": 0, 2024-09-21T03:42:27.822 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-21T03:42:27.822 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_inline_bl_items": 0, 2024-09-21T03:42:27.822 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_fsck_bytes": 0, 2024-09-21T03:42:27.822 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_fsck_items": 0, 2024-09-21T03:42:27.822 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_txc_bytes": 0, 2024-09-21T03:42:27.822 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_txc_items": 0, 2024-09-21T03:42:27.822 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-21T03:42:27.822 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_writing_deferred_items": 0, 2024-09-21T03:42:27.822 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_writing_bytes": 0, 2024-09-21T03:42:27.822 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_writing_items": 0, 2024-09-21T03:42:27.823 INFO:teuthology.orchestra.run.smithi089.stdout: "bluefs_bytes": 0, 2024-09-21T03:42:27.823 INFO:teuthology.orchestra.run.smithi089.stdout: "bluefs_items": 0, 2024-09-21T03:42:27.823 INFO:teuthology.orchestra.run.smithi089.stdout: "bluefs_file_reader_bytes": 0, 2024-09-21T03:42:27.823 INFO:teuthology.orchestra.run.smithi089.stdout: "bluefs_file_reader_items": 0, 2024-09-21T03:42:27.823 INFO:teuthology.orchestra.run.smithi089.stdout: "bluefs_file_writer_bytes": 0, 2024-09-21T03:42:27.823 INFO:teuthology.orchestra.run.smithi089.stdout: "bluefs_file_writer_items": 0, 2024-09-21T03:42:27.823 INFO:teuthology.orchestra.run.smithi089.stdout: "buffer_anon_bytes": 148863, 2024-09-21T03:42:27.823 INFO:teuthology.orchestra.run.smithi089.stdout: "buffer_anon_items": 28, 2024-09-21T03:42:27.823 INFO:teuthology.orchestra.run.smithi089.stdout: "buffer_meta_bytes": 0, 2024-09-21T03:42:27.823 INFO:teuthology.orchestra.run.smithi089.stdout: "buffer_meta_items": 0, 2024-09-21T03:42:27.823 INFO:teuthology.orchestra.run.smithi089.stdout: "osd_bytes": 0, 2024-09-21T03:42:27.823 INFO:teuthology.orchestra.run.smithi089.stdout: "osd_items": 0, 2024-09-21T03:42:27.823 INFO:teuthology.orchestra.run.smithi089.stdout: "osd_mapbl_bytes": 0, 2024-09-21T03:42:27.823 INFO:teuthology.orchestra.run.smithi089.stdout: "osd_mapbl_items": 0, 2024-09-21T03:42:27.824 INFO:teuthology.orchestra.run.smithi089.stdout: "osd_pglog_bytes": 0, 2024-09-21T03:42:27.824 INFO:teuthology.orchestra.run.smithi089.stdout: "osd_pglog_items": 0, 2024-09-21T03:42:27.824 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap_bytes": 0, 2024-09-21T03:42:27.824 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap_items": 0, 2024-09-21T03:42:27.824 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap_mapping_bytes": 0, 2024-09-21T03:42:27.824 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap_mapping_items": 0, 2024-09-21T03:42:27.824 INFO:teuthology.orchestra.run.smithi089.stdout: "pgmap_bytes": 0, 2024-09-21T03:42:27.824 INFO:teuthology.orchestra.run.smithi089.stdout: "pgmap_items": 0, 2024-09-21T03:42:27.824 INFO:teuthology.orchestra.run.smithi089.stdout: "mds_co_bytes": 968, 2024-09-21T03:42:27.824 INFO:teuthology.orchestra.run.smithi089.stdout: "mds_co_items": 2, 2024-09-21T03:42:27.824 INFO:teuthology.orchestra.run.smithi089.stdout: "unittest_1_bytes": 0, 2024-09-21T03:42:27.824 INFO:teuthology.orchestra.run.smithi089.stdout: "unittest_1_items": 0, 2024-09-21T03:42:27.824 INFO:teuthology.orchestra.run.smithi089.stdout: "unittest_2_bytes": 0, 2024-09-21T03:42:27.824 INFO:teuthology.orchestra.run.smithi089.stdout: "unittest_2_items": 0 2024-09-21T03:42:27.825 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:27.825 INFO:teuthology.orchestra.run.smithi089.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-21T03:42:27.825 INFO:teuthology.orchestra.run.smithi089.stdout: "val": 0, 2024-09-21T03:42:27.825 INFO:teuthology.orchestra.run.smithi089.stdout: "max": 104857600, 2024-09-21T03:42:27.825 INFO:teuthology.orchestra.run.smithi089.stdout: "get_started": 0, 2024-09-21T03:42:27.825 INFO:teuthology.orchestra.run.smithi089.stdout: "get": 98, 2024-09-21T03:42:27.825 INFO:teuthology.orchestra.run.smithi089.stdout: "get_sum": 788108, 2024-09-21T03:42:27.825 INFO:teuthology.orchestra.run.smithi089.stdout: "get_or_fail_fail": 0, 2024-09-21T03:42:27.825 INFO:teuthology.orchestra.run.smithi089.stdout: "get_or_fail_success": 98, 2024-09-21T03:42:27.825 INFO:teuthology.orchestra.run.smithi089.stdout: "take": 0, 2024-09-21T03:42:27.825 INFO:teuthology.orchestra.run.smithi089.stdout: "take_sum": 0, 2024-09-21T03:42:27.825 INFO:teuthology.orchestra.run.smithi089.stdout: "put": 98, 2024-09-21T03:42:27.825 INFO:teuthology.orchestra.run.smithi089.stdout: "put_sum": 788108, 2024-09-21T03:42:27.825 INFO:teuthology.orchestra.run.smithi089.stdout: "wait": { 2024-09-21T03:42:27.826 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2024-09-21T03:42:27.826 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2024-09-21T03:42:27.826 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2024-09-21T03:42:27.826 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-21T03:42:27.826 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-21T03:42:27.826 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-09-21T03:42:28.605 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.g 2024-09-21T03:42:28.605 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.g 2024-09-21T03:42:28.605 DEBUG:tasks.check_counter:Getting stats from j 2024-09-21T03:42:28.606 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 7c374098-77c9-11ef-baf5-efdc52797490 -- ceph daemon mds.j perf dump 2024-09-21T03:42:28.851 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/7c374098-77c9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:42:29.211 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:42:28 smithi114 ceph-mon[34140]: pgmap v452: 129 pgs: 129 active+clean; 1.1 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 217 KiB/s wr, 66 op/s 2024-09-21T03:42:29.241 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:42:28 smithi089 ceph-mon[30928]: pgmap v452: 129 pgs: 129 active+clean; 1.1 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 217 KiB/s wr, 66 op/s 2024-09-21T03:42:29.250 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:42:28 smithi110 ceph-mon[34000]: pgmap v452: 129 pgs: 129 active+clean; 1.1 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 217 KiB/s wr, 66 op/s 2024-09-21T03:42:29.359 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-09-21T03:42:29.359 INFO:teuthology.orchestra.run.smithi089.stdout: "AsyncMessenger::Worker-0": { 2024-09-21T03:42:29.359 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_recv_messages": 0, 2024-09-21T03:42:29.360 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_send_messages": 2, 2024-09-21T03:42:29.360 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_recv_bytes": 0, 2024-09-21T03:42:29.360 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_send_bytes": 0, 2024-09-21T03:42:29.360 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_created_connections": 4, 2024-09-21T03:42:29.360 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_active_connections": 0, 2024-09-21T03:42:29.360 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_running_total_time": 0.001050306, 2024-09-21T03:42:29.360 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_running_send_time": 0.000000000, 2024-09-21T03:42:29.360 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_running_recv_time": 0.000169951, 2024-09-21T03:42:29.360 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-21T03:42:29.360 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T03:42:29.360 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2024-09-21T03:42:29.360 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2024-09-21T03:42:29.360 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2024-09-21T03:42:29.360 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:29.361 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_handle_ack_lat": { 2024-09-21T03:42:29.361 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2024-09-21T03:42:29.361 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2024-09-21T03:42:29.361 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2024-09-21T03:42:29.361 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:29.361 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-21T03:42:29.361 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_send_encrypted_bytes": 0 2024-09-21T03:42:29.361 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:29.361 INFO:teuthology.orchestra.run.smithi089.stdout: "AsyncMessenger::Worker-1": { 2024-09-21T03:42:29.361 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_recv_messages": 102, 2024-09-21T03:42:29.361 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_send_messages": 97, 2024-09-21T03:42:29.361 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_recv_bytes": 803120, 2024-09-21T03:42:29.361 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_send_bytes": 58592, 2024-09-21T03:42:29.361 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_created_connections": 2, 2024-09-21T03:42:29.361 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_active_connections": 1, 2024-09-21T03:42:29.362 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_running_total_time": 0.022666535, 2024-09-21T03:42:29.362 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_running_send_time": 0.008266549, 2024-09-21T03:42:29.362 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_running_recv_time": 0.022584056, 2024-09-21T03:42:29.362 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-21T03:42:29.362 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T03:42:29.362 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 96, 2024-09-21T03:42:29.362 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.005673221, 2024-09-21T03:42:29.362 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000059096 2024-09-21T03:42:29.362 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:29.362 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_handle_ack_lat": { 2024-09-21T03:42:29.362 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2024-09-21T03:42:29.362 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2024-09-21T03:42:29.362 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2024-09-21T03:42:29.362 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:29.363 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_recv_encrypted_bytes": 803120, 2024-09-21T03:42:29.363 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_send_encrypted_bytes": 58592 2024-09-21T03:42:29.363 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:29.363 INFO:teuthology.orchestra.run.smithi089.stdout: "AsyncMessenger::Worker-2": { 2024-09-21T03:42:29.363 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_recv_messages": 5, 2024-09-21T03:42:29.363 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_send_messages": 75, 2024-09-21T03:42:29.363 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_recv_bytes": 3120, 2024-09-21T03:42:29.363 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_send_bytes": 93488, 2024-09-21T03:42:29.363 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_created_connections": 3, 2024-09-21T03:42:29.363 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_active_connections": 2, 2024-09-21T03:42:29.363 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_running_total_time": 0.008797977, 2024-09-21T03:42:29.363 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_running_send_time": 0.005017388, 2024-09-21T03:42:29.363 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_running_recv_time": 0.003178194, 2024-09-21T03:42:29.363 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-21T03:42:29.364 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T03:42:29.364 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 74, 2024-09-21T03:42:29.364 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.008518405, 2024-09-21T03:42:29.364 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000115113 2024-09-21T03:42:29.364 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:29.364 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_handle_ack_lat": { 2024-09-21T03:42:29.364 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2024-09-21T03:42:29.364 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2024-09-21T03:42:29.364 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2024-09-21T03:42:29.364 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:29.364 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_recv_encrypted_bytes": 3120, 2024-09-21T03:42:29.364 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_send_encrypted_bytes": 93488 2024-09-21T03:42:29.364 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:29.364 INFO:teuthology.orchestra.run.smithi089.stdout: "cct": { 2024-09-21T03:42:29.364 INFO:teuthology.orchestra.run.smithi089.stdout: "total_workers": 0, 2024-09-21T03:42:29.365 INFO:teuthology.orchestra.run.smithi089.stdout: "unhealthy_workers": 0 2024-09-21T03:42:29.365 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:29.365 INFO:teuthology.orchestra.run.smithi089.stdout: "mempool": { 2024-09-21T03:42:29.365 INFO:teuthology.orchestra.run.smithi089.stdout: "bloom_filter_bytes": 0, 2024-09-21T03:42:29.365 INFO:teuthology.orchestra.run.smithi089.stdout: "bloom_filter_items": 0, 2024-09-21T03:42:29.365 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_alloc_bytes": 0, 2024-09-21T03:42:29.365 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_alloc_items": 0, 2024-09-21T03:42:29.365 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_cache_data_bytes": 0, 2024-09-21T03:42:29.365 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_cache_data_items": 0, 2024-09-21T03:42:29.365 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-21T03:42:29.365 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_cache_onode_items": 0, 2024-09-21T03:42:29.365 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-21T03:42:29.365 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_cache_meta_items": 0, 2024-09-21T03:42:29.365 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_cache_other_bytes": 0, 2024-09-21T03:42:29.365 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_cache_other_items": 0, 2024-09-21T03:42:29.366 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-21T03:42:29.366 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_cache_buffer_items": 0, 2024-09-21T03:42:29.366 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_extent_bytes": 0, 2024-09-21T03:42:29.366 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_extent_items": 0, 2024-09-21T03:42:29.366 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_blob_bytes": 0, 2024-09-21T03:42:29.366 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_blob_items": 0, 2024-09-21T03:42:29.366 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-21T03:42:29.366 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_shared_blob_items": 0, 2024-09-21T03:42:29.366 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-21T03:42:29.366 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_inline_bl_items": 0, 2024-09-21T03:42:29.366 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_fsck_bytes": 0, 2024-09-21T03:42:29.366 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_fsck_items": 0, 2024-09-21T03:42:29.366 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_txc_bytes": 0, 2024-09-21T03:42:29.366 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_txc_items": 0, 2024-09-21T03:42:29.366 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-21T03:42:29.367 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_writing_deferred_items": 0, 2024-09-21T03:42:29.367 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_writing_bytes": 0, 2024-09-21T03:42:29.367 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_writing_items": 0, 2024-09-21T03:42:29.367 INFO:teuthology.orchestra.run.smithi089.stdout: "bluefs_bytes": 0, 2024-09-21T03:42:29.367 INFO:teuthology.orchestra.run.smithi089.stdout: "bluefs_items": 0, 2024-09-21T03:42:29.367 INFO:teuthology.orchestra.run.smithi089.stdout: "bluefs_file_reader_bytes": 0, 2024-09-21T03:42:29.367 INFO:teuthology.orchestra.run.smithi089.stdout: "bluefs_file_reader_items": 0, 2024-09-21T03:42:29.367 INFO:teuthology.orchestra.run.smithi089.stdout: "bluefs_file_writer_bytes": 0, 2024-09-21T03:42:29.367 INFO:teuthology.orchestra.run.smithi089.stdout: "bluefs_file_writer_items": 0, 2024-09-21T03:42:29.367 INFO:teuthology.orchestra.run.smithi089.stdout: "buffer_anon_bytes": 148863, 2024-09-21T03:42:29.367 INFO:teuthology.orchestra.run.smithi089.stdout: "buffer_anon_items": 28, 2024-09-21T03:42:29.367 INFO:teuthology.orchestra.run.smithi089.stdout: "buffer_meta_bytes": 0, 2024-09-21T03:42:29.367 INFO:teuthology.orchestra.run.smithi089.stdout: "buffer_meta_items": 0, 2024-09-21T03:42:29.367 INFO:teuthology.orchestra.run.smithi089.stdout: "osd_bytes": 0, 2024-09-21T03:42:29.368 INFO:teuthology.orchestra.run.smithi089.stdout: "osd_items": 0, 2024-09-21T03:42:29.368 INFO:teuthology.orchestra.run.smithi089.stdout: "osd_mapbl_bytes": 0, 2024-09-21T03:42:29.368 INFO:teuthology.orchestra.run.smithi089.stdout: "osd_mapbl_items": 0, 2024-09-21T03:42:29.368 INFO:teuthology.orchestra.run.smithi089.stdout: "osd_pglog_bytes": 0, 2024-09-21T03:42:29.368 INFO:teuthology.orchestra.run.smithi089.stdout: "osd_pglog_items": 0, 2024-09-21T03:42:29.368 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap_bytes": 0, 2024-09-21T03:42:29.368 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap_items": 0, 2024-09-21T03:42:29.368 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap_mapping_bytes": 0, 2024-09-21T03:42:29.368 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap_mapping_items": 0, 2024-09-21T03:42:29.368 INFO:teuthology.orchestra.run.smithi089.stdout: "pgmap_bytes": 0, 2024-09-21T03:42:29.368 INFO:teuthology.orchestra.run.smithi089.stdout: "pgmap_items": 0, 2024-09-21T03:42:29.368 INFO:teuthology.orchestra.run.smithi089.stdout: "mds_co_bytes": 968, 2024-09-21T03:42:29.368 INFO:teuthology.orchestra.run.smithi089.stdout: "mds_co_items": 2, 2024-09-21T03:42:29.368 INFO:teuthology.orchestra.run.smithi089.stdout: "unittest_1_bytes": 0, 2024-09-21T03:42:29.368 INFO:teuthology.orchestra.run.smithi089.stdout: "unittest_1_items": 0, 2024-09-21T03:42:29.369 INFO:teuthology.orchestra.run.smithi089.stdout: "unittest_2_bytes": 0, 2024-09-21T03:42:29.369 INFO:teuthology.orchestra.run.smithi089.stdout: "unittest_2_items": 0 2024-09-21T03:42:29.369 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T03:42:29.369 INFO:teuthology.orchestra.run.smithi089.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-21T03:42:29.369 INFO:teuthology.orchestra.run.smithi089.stdout: "val": 0, 2024-09-21T03:42:29.369 INFO:teuthology.orchestra.run.smithi089.stdout: "max": 104857600, 2024-09-21T03:42:29.369 INFO:teuthology.orchestra.run.smithi089.stdout: "get_started": 0, 2024-09-21T03:42:29.369 INFO:teuthology.orchestra.run.smithi089.stdout: "get": 103, 2024-09-21T03:42:29.369 INFO:teuthology.orchestra.run.smithi089.stdout: "get_sum": 788758, 2024-09-21T03:42:29.369 INFO:teuthology.orchestra.run.smithi089.stdout: "get_or_fail_fail": 0, 2024-09-21T03:42:29.369 INFO:teuthology.orchestra.run.smithi089.stdout: "get_or_fail_success": 103, 2024-09-21T03:42:29.369 INFO:teuthology.orchestra.run.smithi089.stdout: "take": 0, 2024-09-21T03:42:29.369 INFO:teuthology.orchestra.run.smithi089.stdout: "take_sum": 0, 2024-09-21T03:42:29.369 INFO:teuthology.orchestra.run.smithi089.stdout: "put": 103, 2024-09-21T03:42:29.370 INFO:teuthology.orchestra.run.smithi089.stdout: "put_sum": 788758, 2024-09-21T03:42:29.370 INFO:teuthology.orchestra.run.smithi089.stdout: "wait": { 2024-09-21T03:42:29.370 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2024-09-21T03:42:29.370 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2024-09-21T03:42:29.370 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2024-09-21T03:42:29.370 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-21T03:42:29.370 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-21T03:42:29.370 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-09-21T03:42:30.092 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.j 2024-09-21T03:42:30.092 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.j 2024-09-21T03:42:30.092 DEBUG:tasks.check_counter:Getting stats from b 2024-09-21T03:42:30.093 DEBUG:teuthology.orchestra.run.smithi110:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 7c374098-77c9-11ef-baf5-efdc52797490 -- ceph daemon mds.b perf dump 2024-09-21T03:42:30.350 INFO:teuthology.orchestra.run.smithi110.stderr:Inferring config /var/lib/ceph/7c374098-77c9-11ef-baf5-efdc52797490/mon.b/config 2024-09-21T03:42:30.961 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-21T03:42:30.961 INFO:teuthology.orchestra.run.smithi110.stdout: "AsyncMessenger::Worker-0": { 2024-09-21T03:42:30.962 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_recv_messages": 983, 2024-09-21T03:42:30.962 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_send_messages": 682, 2024-09-21T03:42:30.962 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_recv_bytes": 332649, 2024-09-21T03:42:30.962 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_send_bytes": 1164629, 2024-09-21T03:42:30.962 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_created_connections": 36, 2024-09-21T03:42:30.962 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_active_connections": 6, 2024-09-21T03:42:30.962 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_running_total_time": 0.138098326, 2024-09-21T03:42:30.962 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_running_send_time": 0.043950516, 2024-09-21T03:42:30.962 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_running_recv_time": 0.051596826, 2024-09-21T03:42:30.962 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_running_fast_dispatch_time": 0.036523994, 2024-09-21T03:42:30.962 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T03:42:30.962 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 678, 2024-09-21T03:42:30.962 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.324984043, 2024-09-21T03:42:30.962 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000479327 2024-09-21T03:42:30.962 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-21T03:42:30.963 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_handle_ack_lat": { 2024-09-21T03:42:30.963 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 799, 2024-09-21T03:42:30.963 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000189309, 2024-09-21T03:42:30.963 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000236 2024-09-21T03:42:30.963 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-21T03:42:30.963 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_recv_encrypted_bytes": 103232, 2024-09-21T03:42:30.963 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_send_encrypted_bytes": 34512 2024-09-21T03:42:30.963 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-21T03:42:30.963 INFO:teuthology.orchestra.run.smithi110.stdout: "AsyncMessenger::Worker-1": { 2024-09-21T03:42:30.963 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_recv_messages": 629, 2024-09-21T03:42:30.963 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_send_messages": 561, 2024-09-21T03:42:30.963 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_recv_bytes": 1003660, 2024-09-21T03:42:30.963 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_send_bytes": 461355, 2024-09-21T03:42:30.963 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_created_connections": 27, 2024-09-21T03:42:30.964 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_active_connections": 5, 2024-09-21T03:42:30.964 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_running_total_time": 0.078849137, 2024-09-21T03:42:30.964 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_running_send_time": 0.031057154, 2024-09-21T03:42:30.964 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_running_recv_time": 0.051507912, 2024-09-21T03:42:30.964 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_running_fast_dispatch_time": 0.008139547, 2024-09-21T03:42:30.964 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T03:42:30.964 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 563, 2024-09-21T03:42:30.964 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.126178285, 2024-09-21T03:42:30.964 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000224117 2024-09-21T03:42:30.964 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-21T03:42:30.964 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_handle_ack_lat": { 2024-09-21T03:42:30.964 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 367, 2024-09-21T03:42:30.964 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000077834, 2024-09-21T03:42:30.964 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000212 2024-09-21T03:42:30.964 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-21T03:42:30.965 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_recv_encrypted_bytes": 852048, 2024-09-21T03:42:30.965 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_send_encrypted_bytes": 63264 2024-09-21T03:42:30.965 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-21T03:42:30.965 INFO:teuthology.orchestra.run.smithi110.stdout: "AsyncMessenger::Worker-2": { 2024-09-21T03:42:30.965 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_recv_messages": 520, 2024-09-21T03:42:30.965 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_send_messages": 572, 2024-09-21T03:42:30.965 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_recv_bytes": 180111, 2024-09-21T03:42:30.965 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_send_bytes": 471128, 2024-09-21T03:42:30.965 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_created_connections": 15, 2024-09-21T03:42:30.965 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_active_connections": 6, 2024-09-21T03:42:30.965 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_running_total_time": 0.061748337, 2024-09-21T03:42:30.965 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_running_send_time": 0.027854423, 2024-09-21T03:42:30.965 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_running_recv_time": 0.029760565, 2024-09-21T03:42:30.965 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_running_fast_dispatch_time": 0.007243963, 2024-09-21T03:42:30.965 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T03:42:30.966 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 570, 2024-09-21T03:42:30.966 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.179366969, 2024-09-21T03:42:30.966 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000314678 2024-09-21T03:42:30.966 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-21T03:42:30.966 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_handle_ack_lat": { 2024-09-21T03:42:30.966 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 471, 2024-09-21T03:42:30.966 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000127723, 2024-09-21T03:42:30.966 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000271 2024-09-21T03:42:30.966 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-21T03:42:30.966 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_recv_encrypted_bytes": 160, 2024-09-21T03:42:30.966 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_send_encrypted_bytes": 206816 2024-09-21T03:42:30.966 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-21T03:42:30.966 INFO:teuthology.orchestra.run.smithi110.stdout: "cct": { 2024-09-21T03:42:30.966 INFO:teuthology.orchestra.run.smithi110.stdout: "total_workers": 1, 2024-09-21T03:42:30.967 INFO:teuthology.orchestra.run.smithi110.stdout: "unhealthy_workers": 0 2024-09-21T03:42:30.967 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-21T03:42:30.967 INFO:teuthology.orchestra.run.smithi110.stdout: "finisher-MDSRank": { 2024-09-21T03:42:30.967 INFO:teuthology.orchestra.run.smithi110.stdout: "queue_len": 0, 2024-09-21T03:42:30.967 INFO:teuthology.orchestra.run.smithi110.stdout: "complete_latency": { 2024-09-21T03:42:30.967 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 471, 2024-09-21T03:42:30.967 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.117888763, 2024-09-21T03:42:30.967 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000250294 2024-09-21T03:42:30.967 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-21T03:42:30.967 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-21T03:42:30.967 INFO:teuthology.orchestra.run.smithi110.stdout: "finisher-PurgeQueue": { 2024-09-21T03:42:30.967 INFO:teuthology.orchestra.run.smithi110.stdout: "queue_len": 0, 2024-09-21T03:42:30.967 INFO:teuthology.orchestra.run.smithi110.stdout: "complete_latency": { 2024-09-21T03:42:30.967 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 89, 2024-09-21T03:42:30.967 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.003981004, 2024-09-21T03:42:30.968 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000044730 2024-09-21T03:42:30.968 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-21T03:42:30.968 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-21T03:42:30.968 INFO:teuthology.orchestra.run.smithi110.stdout: "mds": { 2024-09-21T03:42:30.968 INFO:teuthology.orchestra.run.smithi110.stdout: "request": 75, 2024-09-21T03:42:30.968 INFO:teuthology.orchestra.run.smithi110.stdout: "reply": 75, 2024-09-21T03:42:30.968 INFO:teuthology.orchestra.run.smithi110.stdout: "reply_latency": { 2024-09-21T03:42:30.968 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 75, 2024-09-21T03:42:30.968 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.081631924, 2024-09-21T03:42:30.968 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.001088425 2024-09-21T03:42:30.968 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-21T03:42:30.968 INFO:teuthology.orchestra.run.smithi110.stdout: "slow_reply": 0, 2024-09-21T03:42:30.968 INFO:teuthology.orchestra.run.smithi110.stdout: "forward": 0, 2024-09-21T03:42:30.968 INFO:teuthology.orchestra.run.smithi110.stdout: "dir_fetch_complete": 0, 2024-09-21T03:42:30.968 INFO:teuthology.orchestra.run.smithi110.stdout: "dir_fetch_keys": 0, 2024-09-21T03:42:30.969 INFO:teuthology.orchestra.run.smithi110.stdout: "dir_commit": 81, 2024-09-21T03:42:30.969 INFO:teuthology.orchestra.run.smithi110.stdout: "dir_split": 0, 2024-09-21T03:42:30.969 INFO:teuthology.orchestra.run.smithi110.stdout: "dir_merge": 0, 2024-09-21T03:42:30.969 INFO:teuthology.orchestra.run.smithi110.stdout: "inodes": 24, 2024-09-21T03:42:30.969 INFO:teuthology.orchestra.run.smithi110.stdout: "inodes_top": 4, 2024-09-21T03:42:30.969 INFO:teuthology.orchestra.run.smithi110.stdout: "inodes_bottom": 20, 2024-09-21T03:42:30.969 INFO:teuthology.orchestra.run.smithi110.stdout: "inodes_pin_tail": 0, 2024-09-21T03:42:30.969 INFO:teuthology.orchestra.run.smithi110.stdout: "inodes_pinned": 21, 2024-09-21T03:42:30.969 INFO:teuthology.orchestra.run.smithi110.stdout: "inodes_expired": 0, 2024-09-21T03:42:30.969 INFO:teuthology.orchestra.run.smithi110.stdout: "inodes_with_caps": 2, 2024-09-21T03:42:30.969 INFO:teuthology.orchestra.run.smithi110.stdout: "caps": 2, 2024-09-21T03:42:30.969 INFO:teuthology.orchestra.run.smithi110.stdout: "subtrees": 2, 2024-09-21T03:42:30.969 INFO:teuthology.orchestra.run.smithi110.stdout: "traverse": 121, 2024-09-21T03:42:30.969 INFO:teuthology.orchestra.run.smithi110.stdout: "traverse_hit": 108, 2024-09-21T03:42:30.970 INFO:teuthology.orchestra.run.smithi110.stdout: "traverse_forward": 0, 2024-09-21T03:42:30.970 INFO:teuthology.orchestra.run.smithi110.stdout: "traverse_discover": 0, 2024-09-21T03:42:30.970 INFO:teuthology.orchestra.run.smithi110.stdout: "traverse_dir_fetch": 0, 2024-09-21T03:42:30.970 INFO:teuthology.orchestra.run.smithi110.stdout: "traverse_remote_ino": 0, 2024-09-21T03:42:30.970 INFO:teuthology.orchestra.run.smithi110.stdout: "traverse_lock": 0, 2024-09-21T03:42:30.970 INFO:teuthology.orchestra.run.smithi110.stdout: "load_cent": 0, 2024-09-21T03:42:30.970 INFO:teuthology.orchestra.run.smithi110.stdout: "q": 0, 2024-09-21T03:42:30.970 INFO:teuthology.orchestra.run.smithi110.stdout: "exported": 0, 2024-09-21T03:42:30.970 INFO:teuthology.orchestra.run.smithi110.stdout: "exported_inodes": 0, 2024-09-21T03:42:30.970 INFO:teuthology.orchestra.run.smithi110.stdout: "imported": 0, 2024-09-21T03:42:30.970 INFO:teuthology.orchestra.run.smithi110.stdout: "imported_inodes": 0, 2024-09-21T03:42:30.970 INFO:teuthology.orchestra.run.smithi110.stdout: "openino_dir_fetch": 0, 2024-09-21T03:42:30.970 INFO:teuthology.orchestra.run.smithi110.stdout: "openino_backtrace_fetch": 0, 2024-09-21T03:42:30.970 INFO:teuthology.orchestra.run.smithi110.stdout: "openino_peer_discover": 0, 2024-09-21T03:42:30.971 INFO:teuthology.orchestra.run.smithi110.stdout: "root_rfiles": 4, 2024-09-21T03:42:30.971 INFO:teuthology.orchestra.run.smithi110.stdout: "root_rbytes": 234, 2024-09-21T03:42:30.971 INFO:teuthology.orchestra.run.smithi110.stdout: "root_rsnaps": 0, 2024-09-21T03:42:30.971 INFO:teuthology.orchestra.run.smithi110.stdout: "scrub_backtrace_fetch": 0, 2024-09-21T03:42:30.971 INFO:teuthology.orchestra.run.smithi110.stdout: "scrub_set_tag": 0, 2024-09-21T03:42:30.971 INFO:teuthology.orchestra.run.smithi110.stdout: "scrub_backtrace_repaired": 0, 2024-09-21T03:42:30.971 INFO:teuthology.orchestra.run.smithi110.stdout: "scrub_inotable_repaired": 0, 2024-09-21T03:42:30.971 INFO:teuthology.orchestra.run.smithi110.stdout: "scrub_dir_inodes": 0, 2024-09-21T03:42:30.971 INFO:teuthology.orchestra.run.smithi110.stdout: "scrub_dir_base_inodes": 0, 2024-09-21T03:42:30.971 INFO:teuthology.orchestra.run.smithi110.stdout: "scrub_dirfrag_rstats": 0, 2024-09-21T03:42:30.971 INFO:teuthology.orchestra.run.smithi110.stdout: "scrub_file_inodes": 0, 2024-09-21T03:42:30.971 INFO:teuthology.orchestra.run.smithi110.stdout: "handle_inode_file_caps": 0, 2024-09-21T03:42:30.971 INFO:teuthology.orchestra.run.smithi110.stdout: "ceph_cap_op_revoke": 36, 2024-09-21T03:42:30.971 INFO:teuthology.orchestra.run.smithi110.stdout: "ceph_cap_op_grant": 35, 2024-09-21T03:42:30.971 INFO:teuthology.orchestra.run.smithi110.stdout: "ceph_cap_op_trunc": 0, 2024-09-21T03:42:30.972 INFO:teuthology.orchestra.run.smithi110.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-21T03:42:30.972 INFO:teuthology.orchestra.run.smithi110.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-21T03:42:30.972 INFO:teuthology.orchestra.run.smithi110.stdout: "handle_client_caps": 106, 2024-09-21T03:42:30.972 INFO:teuthology.orchestra.run.smithi110.stdout: "handle_client_caps_dirty": 69, 2024-09-21T03:42:30.972 INFO:teuthology.orchestra.run.smithi110.stdout: "handle_client_cap_release": 2, 2024-09-21T03:42:30.972 INFO:teuthology.orchestra.run.smithi110.stdout: "process_request_cap_release": 34 2024-09-21T03:42:30.972 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-21T03:42:30.972 INFO:teuthology.orchestra.run.smithi110.stdout: "mds_cache": { 2024-09-21T03:42:30.972 INFO:teuthology.orchestra.run.smithi110.stdout: "dir_update": 0, 2024-09-21T03:42:30.972 INFO:teuthology.orchestra.run.smithi110.stdout: "dir_update_receipt": 0, 2024-09-21T03:42:30.972 INFO:teuthology.orchestra.run.smithi110.stdout: "dir_try_discover": 0, 2024-09-21T03:42:30.972 INFO:teuthology.orchestra.run.smithi110.stdout: "dir_send_discover": 0, 2024-09-21T03:42:30.972 INFO:teuthology.orchestra.run.smithi110.stdout: "dir_handle_discover": 6, 2024-09-21T03:42:30.972 INFO:teuthology.orchestra.run.smithi110.stdout: "num_strays": 0, 2024-09-21T03:42:30.972 INFO:teuthology.orchestra.run.smithi110.stdout: "num_strays_delayed": 0, 2024-09-21T03:42:30.973 INFO:teuthology.orchestra.run.smithi110.stdout: "num_strays_enqueuing": 0, 2024-09-21T03:42:30.973 INFO:teuthology.orchestra.run.smithi110.stdout: "strays_created": 6, 2024-09-21T03:42:30.973 INFO:teuthology.orchestra.run.smithi110.stdout: "strays_enqueued": 6, 2024-09-21T03:42:30.973 INFO:teuthology.orchestra.run.smithi110.stdout: "strays_reintegrated": 0, 2024-09-21T03:42:30.973 INFO:teuthology.orchestra.run.smithi110.stdout: "strays_migrated": 0, 2024-09-21T03:42:30.973 INFO:teuthology.orchestra.run.smithi110.stdout: "num_recovering_processing": 0, 2024-09-21T03:42:30.973 INFO:teuthology.orchestra.run.smithi110.stdout: "num_recovering_enqueued": 0, 2024-09-21T03:42:30.973 INFO:teuthology.orchestra.run.smithi110.stdout: "num_recovering_prioritized": 0, 2024-09-21T03:42:30.973 INFO:teuthology.orchestra.run.smithi110.stdout: "recovery_started": 0, 2024-09-21T03:42:30.973 INFO:teuthology.orchestra.run.smithi110.stdout: "recovery_completed": 0, 2024-09-21T03:42:30.973 INFO:teuthology.orchestra.run.smithi110.stdout: "ireq_quiesce_path": 3, 2024-09-21T03:42:30.973 INFO:teuthology.orchestra.run.smithi110.stdout: "ireq_quiesce_inode": 45, 2024-09-21T03:42:30.973 INFO:teuthology.orchestra.run.smithi110.stdout: "ireq_enqueue_scrub": 0, 2024-09-21T03:42:30.973 INFO:teuthology.orchestra.run.smithi110.stdout: "ireq_exportdir": 0, 2024-09-21T03:42:30.974 INFO:teuthology.orchestra.run.smithi110.stdout: "ireq_flush": 0, 2024-09-21T03:42:30.974 INFO:teuthology.orchestra.run.smithi110.stdout: "ireq_fragmentdir": 0, 2024-09-21T03:42:30.974 INFO:teuthology.orchestra.run.smithi110.stdout: "ireq_fragstats": 0, 2024-09-21T03:42:30.974 INFO:teuthology.orchestra.run.smithi110.stdout: "ireq_inodestats": 0, 2024-09-21T03:42:30.974 INFO:teuthology.orchestra.run.smithi110.stdout: "uninline_started": 0, 2024-09-21T03:42:30.974 INFO:teuthology.orchestra.run.smithi110.stdout: "uninline_succeeded": 0, 2024-09-21T03:42:30.974 INFO:teuthology.orchestra.run.smithi110.stdout: "uninline_write_failed": 0 2024-09-21T03:42:30.974 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-21T03:42:30.974 INFO:teuthology.orchestra.run.smithi110.stdout: "mds_log": { 2024-09-21T03:42:30.974 INFO:teuthology.orchestra.run.smithi110.stdout: "evlrg": 0, 2024-09-21T03:42:30.974 INFO:teuthology.orchestra.run.smithi110.stdout: "evadd": 161, 2024-09-21T03:42:30.974 INFO:teuthology.orchestra.run.smithi110.stdout: "evex": 156, 2024-09-21T03:42:30.974 INFO:teuthology.orchestra.run.smithi110.stdout: "evtrm": 156, 2024-09-21T03:42:30.974 INFO:teuthology.orchestra.run.smithi110.stdout: "ev": 5, 2024-09-21T03:42:30.974 INFO:teuthology.orchestra.run.smithi110.stdout: "evexg": 0, 2024-09-21T03:42:30.975 INFO:teuthology.orchestra.run.smithi110.stdout: "evexd": 16, 2024-09-21T03:42:30.975 INFO:teuthology.orchestra.run.smithi110.stdout: "segadd": 15, 2024-09-21T03:42:30.975 INFO:teuthology.orchestra.run.smithi110.stdout: "segex": 14, 2024-09-21T03:42:30.975 INFO:teuthology.orchestra.run.smithi110.stdout: "segtrm": 14, 2024-09-21T03:42:30.975 INFO:teuthology.orchestra.run.smithi110.stdout: "seg": 1, 2024-09-21T03:42:30.975 INFO:teuthology.orchestra.run.smithi110.stdout: "segmjr": 1, 2024-09-21T03:42:30.975 INFO:teuthology.orchestra.run.smithi110.stdout: "segexg": 0, 2024-09-21T03:42:30.975 INFO:teuthology.orchestra.run.smithi110.stdout: "segexd": 1, 2024-09-21T03:42:30.975 INFO:teuthology.orchestra.run.smithi110.stdout: "expos": 4775340, 2024-09-21T03:42:30.975 INFO:teuthology.orchestra.run.smithi110.stdout: "wrpos": 4786905, 2024-09-21T03:42:30.975 INFO:teuthology.orchestra.run.smithi110.stdout: "rdpos": 0, 2024-09-21T03:42:30.975 INFO:teuthology.orchestra.run.smithi110.stdout: "jlat": { 2024-09-21T03:42:30.975 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 129, 2024-09-21T03:42:30.975 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.605650070, 2024-09-21T03:42:30.975 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.004694961 2024-09-21T03:42:30.976 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-21T03:42:30.976 INFO:teuthology.orchestra.run.smithi110.stdout: "replayed": 0 2024-09-21T03:42:30.976 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-21T03:42:30.976 INFO:teuthology.orchestra.run.smithi110.stdout: "mds_mem": { 2024-09-21T03:42:30.976 INFO:teuthology.orchestra.run.smithi110.stdout: "ino": 23, 2024-09-21T03:42:30.976 INFO:teuthology.orchestra.run.smithi110.stdout: "ino+": 29, 2024-09-21T03:42:30.976 INFO:teuthology.orchestra.run.smithi110.stdout: "ino-": 6, 2024-09-21T03:42:30.976 INFO:teuthology.orchestra.run.smithi110.stdout: "dir": 18, 2024-09-21T03:42:30.976 INFO:teuthology.orchestra.run.smithi110.stdout: "dir+": 20, 2024-09-21T03:42:30.976 INFO:teuthology.orchestra.run.smithi110.stdout: "dir-": 2, 2024-09-21T03:42:30.976 INFO:teuthology.orchestra.run.smithi110.stdout: "dn": 24, 2024-09-21T03:42:30.976 INFO:teuthology.orchestra.run.smithi110.stdout: "dn+": 34, 2024-09-21T03:42:30.976 INFO:teuthology.orchestra.run.smithi110.stdout: "dn-": 10, 2024-09-21T03:42:30.976 INFO:teuthology.orchestra.run.smithi110.stdout: "cap": 2, 2024-09-21T03:42:30.976 INFO:teuthology.orchestra.run.smithi110.stdout: "cap+": 21, 2024-09-21T03:42:30.977 INFO:teuthology.orchestra.run.smithi110.stdout: "cap-": 19, 2024-09-21T03:42:30.977 INFO:teuthology.orchestra.run.smithi110.stdout: "rss": 123112, 2024-09-21T03:42:30.977 INFO:teuthology.orchestra.run.smithi110.stdout: "heap": 337461248 2024-09-21T03:42:30.977 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-21T03:42:30.977 INFO:teuthology.orchestra.run.smithi110.stdout: "mds_server": { 2024-09-21T03:42:30.977 INFO:teuthology.orchestra.run.smithi110.stdout: "dispatch_client_request": 100, 2024-09-21T03:42:30.977 INFO:teuthology.orchestra.run.smithi110.stdout: "dispatch_server_request": 0, 2024-09-21T03:42:30.977 INFO:teuthology.orchestra.run.smithi110.stdout: "handle_client_request": 75, 2024-09-21T03:42:30.977 INFO:teuthology.orchestra.run.smithi110.stdout: "handle_client_session": 19, 2024-09-21T03:42:30.977 INFO:teuthology.orchestra.run.smithi110.stdout: "handle_peer_request": 0, 2024-09-21T03:42:30.977 INFO:teuthology.orchestra.run.smithi110.stdout: "req_create_latency": { 2024-09-21T03:42:30.977 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 8, 2024-09-21T03:42:30.977 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.008016853, 2024-09-21T03:42:30.977 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.001002106 2024-09-21T03:42:30.978 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-21T03:42:30.978 INFO:teuthology.orchestra.run.smithi110.stdout: "req_getattr_latency": { 2024-09-21T03:42:30.978 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 30, 2024-09-21T03:42:30.978 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.008877951, 2024-09-21T03:42:30.978 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000295931 2024-09-21T03:42:30.978 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-21T03:42:30.978 INFO:teuthology.orchestra.run.smithi110.stdout: "req_getfilelock_latency": { 2024-09-21T03:42:30.978 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-09-21T03:42:30.978 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-09-21T03:42:30.978 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-09-21T03:42:30.978 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-21T03:42:30.978 INFO:teuthology.orchestra.run.smithi110.stdout: "req_link_latency": { 2024-09-21T03:42:30.978 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-09-21T03:42:30.978 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-09-21T03:42:30.978 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-09-21T03:42:30.979 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-21T03:42:30.979 INFO:teuthology.orchestra.run.smithi110.stdout: "req_lookup_latency": { 2024-09-21T03:42:30.979 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 6, 2024-09-21T03:42:30.979 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.004348836, 2024-09-21T03:42:30.979 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000724806 2024-09-21T03:42:30.979 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-21T03:42:30.979 INFO:teuthology.orchestra.run.smithi110.stdout: "req_lookuphash_latency": { 2024-09-21T03:42:30.979 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-09-21T03:42:30.979 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-09-21T03:42:30.979 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-09-21T03:42:30.979 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-21T03:42:30.979 INFO:teuthology.orchestra.run.smithi110.stdout: "req_lookupino_latency": { 2024-09-21T03:42:30.979 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-09-21T03:42:30.979 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-09-21T03:42:30.980 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-09-21T03:42:30.980 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-21T03:42:30.980 INFO:teuthology.orchestra.run.smithi110.stdout: "req_lookupname_latency": { 2024-09-21T03:42:30.980 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-09-21T03:42:30.980 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-09-21T03:42:30.980 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-09-21T03:42:30.980 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-21T03:42:30.980 INFO:teuthology.orchestra.run.smithi110.stdout: "req_lookupparent_latency": { 2024-09-21T03:42:30.980 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-09-21T03:42:30.980 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-09-21T03:42:30.980 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-09-21T03:42:30.980 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-21T03:42:30.980 INFO:teuthology.orchestra.run.smithi110.stdout: "req_lookupsnap_latency": { 2024-09-21T03:42:30.980 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-09-21T03:42:30.980 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-09-21T03:42:30.981 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-09-21T03:42:30.981 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-21T03:42:30.981 INFO:teuthology.orchestra.run.smithi110.stdout: "req_lssnap_latency": { 2024-09-21T03:42:30.981 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-09-21T03:42:30.981 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-09-21T03:42:30.981 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-09-21T03:42:30.981 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-21T03:42:30.982 INFO:teuthology.orchestra.run.smithi110.stdout: "req_mkdir_latency": { 2024-09-21T03:42:30.982 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 8, 2024-09-21T03:42:30.982 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.015146030, 2024-09-21T03:42:30.982 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.001893253 2024-09-21T03:42:30.982 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-21T03:42:30.982 INFO:teuthology.orchestra.run.smithi110.stdout: "req_mknod_latency": { 2024-09-21T03:42:30.982 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-09-21T03:42:30.982 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-09-21T03:42:30.982 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-09-21T03:42:30.982 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-21T03:42:30.982 INFO:teuthology.orchestra.run.smithi110.stdout: "req_mksnap_latency": { 2024-09-21T03:42:30.982 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-09-21T03:42:30.982 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-09-21T03:42:30.983 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-09-21T03:42:30.983 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-21T03:42:30.983 INFO:teuthology.orchestra.run.smithi110.stdout: "req_open_latency": { 2024-09-21T03:42:30.983 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-09-21T03:42:30.983 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-09-21T03:42:30.983 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-09-21T03:42:30.983 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-21T03:42:30.983 INFO:teuthology.orchestra.run.smithi110.stdout: "req_readdir_latency": { 2024-09-21T03:42:30.983 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 2, 2024-09-21T03:42:30.983 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.003177500, 2024-09-21T03:42:30.983 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.001588750 2024-09-21T03:42:30.983 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-21T03:42:30.983 INFO:teuthology.orchestra.run.smithi110.stdout: "req_rename_latency": { 2024-09-21T03:42:30.983 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 2, 2024-09-21T03:42:30.984 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.002196601, 2024-09-21T03:42:30.984 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.001098300 2024-09-21T03:42:30.984 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-21T03:42:30.984 INFO:teuthology.orchestra.run.smithi110.stdout: "req_renamesnap_latency": { 2024-09-21T03:42:30.984 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-09-21T03:42:30.984 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-09-21T03:42:30.984 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-09-21T03:42:30.984 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-21T03:42:30.984 INFO:teuthology.orchestra.run.smithi110.stdout: "req_snapdiff_latency": { 2024-09-21T03:42:30.984 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-09-21T03:42:30.984 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-09-21T03:42:30.984 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-09-21T03:42:30.984 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-21T03:42:30.984 INFO:teuthology.orchestra.run.smithi110.stdout: "req_rmdir_latency": { 2024-09-21T03:42:30.984 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 2, 2024-09-21T03:42:30.985 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.007960971, 2024-09-21T03:42:30.985 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.003980485 2024-09-21T03:42:30.985 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-21T03:42:30.985 INFO:teuthology.orchestra.run.smithi110.stdout: "req_rmsnap_latency": { 2024-09-21T03:42:30.985 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-09-21T03:42:30.985 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-09-21T03:42:30.985 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-09-21T03:42:30.985 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-21T03:42:30.985 INFO:teuthology.orchestra.run.smithi110.stdout: "req_rmxattr_latency": { 2024-09-21T03:42:30.985 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-09-21T03:42:30.985 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-09-21T03:42:30.985 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-09-21T03:42:30.985 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-21T03:42:30.985 INFO:teuthology.orchestra.run.smithi110.stdout: "req_setattr_latency": { 2024-09-21T03:42:30.985 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 2, 2024-09-21T03:42:30.986 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.001809850, 2024-09-21T03:42:30.986 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000904925 2024-09-21T03:42:30.986 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-21T03:42:30.986 INFO:teuthology.orchestra.run.smithi110.stdout: "req_setdirlayout_latency": { 2024-09-21T03:42:30.986 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-09-21T03:42:30.986 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-09-21T03:42:30.986 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-09-21T03:42:30.986 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-21T03:42:30.986 INFO:teuthology.orchestra.run.smithi110.stdout: "req_setfilelock_latency": { 2024-09-21T03:42:30.986 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-09-21T03:42:30.986 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-09-21T03:42:30.986 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-09-21T03:42:30.986 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-21T03:42:30.986 INFO:teuthology.orchestra.run.smithi110.stdout: "req_setlayout_latency": { 2024-09-21T03:42:30.987 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-09-21T03:42:30.987 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-09-21T03:42:30.987 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-09-21T03:42:30.987 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-21T03:42:30.987 INFO:teuthology.orchestra.run.smithi110.stdout: "req_setxattr_latency": { 2024-09-21T03:42:30.987 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 11, 2024-09-21T03:42:30.987 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.025755562, 2024-09-21T03:42:30.987 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.002341414 2024-09-21T03:42:30.987 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-21T03:42:30.987 INFO:teuthology.orchestra.run.smithi110.stdout: "req_symlink_latency": { 2024-09-21T03:42:30.987 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-09-21T03:42:30.987 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-09-21T03:42:30.987 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-09-21T03:42:30.988 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-21T03:42:30.988 INFO:teuthology.orchestra.run.smithi110.stdout: "req_unlink_latency": { 2024-09-21T03:42:30.988 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 4, 2024-09-21T03:42:30.988 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.004341770, 2024-09-21T03:42:30.988 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.001085442 2024-09-21T03:42:30.988 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-21T03:42:30.988 INFO:teuthology.orchestra.run.smithi110.stdout: "cap_revoke_eviction": 0, 2024-09-21T03:42:30.988 INFO:teuthology.orchestra.run.smithi110.stdout: "cap_acquisition_throttle": 0, 2024-09-21T03:42:30.988 INFO:teuthology.orchestra.run.smithi110.stdout: "req_getvxattr_latency": { 2024-09-21T03:42:30.988 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-09-21T03:42:30.988 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-09-21T03:42:30.988 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-09-21T03:42:30.988 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-21T03:42:30.988 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-21T03:42:30.988 INFO:teuthology.orchestra.run.smithi110.stdout: "mds_sessions": { 2024-09-21T03:42:30.989 INFO:teuthology.orchestra.run.smithi110.stdout: "session_count": 1, 2024-09-21T03:42:30.989 INFO:teuthology.orchestra.run.smithi110.stdout: "session_add": 4, 2024-09-21T03:42:30.989 INFO:teuthology.orchestra.run.smithi110.stdout: "session_remove": 3, 2024-09-21T03:42:30.989 INFO:teuthology.orchestra.run.smithi110.stdout: "sessions_open": 1, 2024-09-21T03:42:30.989 INFO:teuthology.orchestra.run.smithi110.stdout: "sessions_stale": 0, 2024-09-21T03:42:30.989 INFO:teuthology.orchestra.run.smithi110.stdout: "total_load": 24, 2024-09-21T03:42:30.989 INFO:teuthology.orchestra.run.smithi110.stdout: "average_load": 24, 2024-09-21T03:42:30.989 INFO:teuthology.orchestra.run.smithi110.stdout: "avg_session_uptime": 185, 2024-09-21T03:42:30.989 INFO:teuthology.orchestra.run.smithi110.stdout: "mdthresh_evicted": 0 2024-09-21T03:42:30.989 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-21T03:42:30.989 INFO:teuthology.orchestra.run.smithi110.stdout: "mempool": { 2024-09-21T03:42:30.989 INFO:teuthology.orchestra.run.smithi110.stdout: "bloom_filter_bytes": 0, 2024-09-21T03:42:30.989 INFO:teuthology.orchestra.run.smithi110.stdout: "bloom_filter_items": 0, 2024-09-21T03:42:30.989 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_alloc_bytes": 0, 2024-09-21T03:42:30.990 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_alloc_items": 0, 2024-09-21T03:42:30.990 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_cache_data_bytes": 0, 2024-09-21T03:42:30.990 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_cache_data_items": 0, 2024-09-21T03:42:30.990 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-21T03:42:30.990 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_cache_onode_items": 0, 2024-09-21T03:42:30.990 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-21T03:42:30.990 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_cache_meta_items": 0, 2024-09-21T03:42:30.990 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_cache_other_bytes": 0, 2024-09-21T03:42:30.990 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_cache_other_items": 0, 2024-09-21T03:42:30.990 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-21T03:42:30.990 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_cache_buffer_items": 0, 2024-09-21T03:42:30.990 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_extent_bytes": 0, 2024-09-21T03:42:30.990 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_extent_items": 0, 2024-09-21T03:42:30.990 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_blob_bytes": 0, 2024-09-21T03:42:30.991 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_blob_items": 0, 2024-09-21T03:42:30.991 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-21T03:42:30.991 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_shared_blob_items": 0, 2024-09-21T03:42:30.991 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-21T03:42:30.991 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_inline_bl_items": 0, 2024-09-21T03:42:30.991 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_fsck_bytes": 0, 2024-09-21T03:42:30.991 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_fsck_items": 0, 2024-09-21T03:42:30.991 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_txc_bytes": 0, 2024-09-21T03:42:30.991 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_txc_items": 0, 2024-09-21T03:42:30.991 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-21T03:42:30.991 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_writing_deferred_items": 0, 2024-09-21T03:42:30.991 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_writing_bytes": 0, 2024-09-21T03:42:30.991 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_writing_items": 0, 2024-09-21T03:42:30.991 INFO:teuthology.orchestra.run.smithi110.stdout: "bluefs_bytes": 0, 2024-09-21T03:42:30.991 INFO:teuthology.orchestra.run.smithi110.stdout: "bluefs_items": 0, 2024-09-21T03:42:30.991 INFO:teuthology.orchestra.run.smithi110.stdout: "bluefs_file_reader_bytes": 0, 2024-09-21T03:42:30.992 INFO:teuthology.orchestra.run.smithi110.stdout: "bluefs_file_reader_items": 0, 2024-09-21T03:42:30.992 INFO:teuthology.orchestra.run.smithi110.stdout: "bluefs_file_writer_bytes": 0, 2024-09-21T03:42:30.992 INFO:teuthology.orchestra.run.smithi110.stdout: "bluefs_file_writer_items": 0, 2024-09-21T03:42:30.992 INFO:teuthology.orchestra.run.smithi110.stdout: "buffer_anon_bytes": 294247, 2024-09-21T03:42:30.992 INFO:teuthology.orchestra.run.smithi110.stdout: "buffer_anon_items": 97, 2024-09-21T03:42:30.992 INFO:teuthology.orchestra.run.smithi110.stdout: "buffer_meta_bytes": 0, 2024-09-21T03:42:30.992 INFO:teuthology.orchestra.run.smithi110.stdout: "buffer_meta_items": 0, 2024-09-21T03:42:30.992 INFO:teuthology.orchestra.run.smithi110.stdout: "osd_bytes": 0, 2024-09-21T03:42:30.992 INFO:teuthology.orchestra.run.smithi110.stdout: "osd_items": 0, 2024-09-21T03:42:30.992 INFO:teuthology.orchestra.run.smithi110.stdout: "osd_mapbl_bytes": 0, 2024-09-21T03:42:30.992 INFO:teuthology.orchestra.run.smithi110.stdout: "osd_mapbl_items": 0, 2024-09-21T03:42:30.992 INFO:teuthology.orchestra.run.smithi110.stdout: "osd_pglog_bytes": 0, 2024-09-21T03:42:30.992 INFO:teuthology.orchestra.run.smithi110.stdout: "osd_pglog_items": 0, 2024-09-21T03:42:30.992 INFO:teuthology.orchestra.run.smithi110.stdout: "osdmap_bytes": 8760, 2024-09-21T03:42:30.992 INFO:teuthology.orchestra.run.smithi110.stdout: "osdmap_items": 164, 2024-09-21T03:42:30.992 INFO:teuthology.orchestra.run.smithi110.stdout: "osdmap_mapping_bytes": 0, 2024-09-21T03:42:30.993 INFO:teuthology.orchestra.run.smithi110.stdout: "osdmap_mapping_items": 0, 2024-09-21T03:42:30.993 INFO:teuthology.orchestra.run.smithi110.stdout: "pgmap_bytes": 0, 2024-09-21T03:42:30.993 INFO:teuthology.orchestra.run.smithi110.stdout: "pgmap_items": 0, 2024-09-21T03:42:30.993 INFO:teuthology.orchestra.run.smithi110.stdout: "mds_co_bytes": 95274, 2024-09-21T03:42:30.993 INFO:teuthology.orchestra.run.smithi110.stdout: "mds_co_items": 668, 2024-09-21T03:42:30.993 INFO:teuthology.orchestra.run.smithi110.stdout: "unittest_1_bytes": 0, 2024-09-21T03:42:30.993 INFO:teuthology.orchestra.run.smithi110.stdout: "unittest_1_items": 0, 2024-09-21T03:42:30.993 INFO:teuthology.orchestra.run.smithi110.stdout: "unittest_2_bytes": 0, 2024-09-21T03:42:30.993 INFO:teuthology.orchestra.run.smithi110.stdout: "unittest_2_items": 0 2024-09-21T03:42:30.993 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-21T03:42:30.993 INFO:teuthology.orchestra.run.smithi110.stdout: "objecter": { 2024-09-21T03:42:30.993 INFO:teuthology.orchestra.run.smithi110.stdout: "op_active": 0, 2024-09-21T03:42:30.993 INFO:teuthology.orchestra.run.smithi110.stdout: "op_laggy": 0, 2024-09-21T03:42:30.993 INFO:teuthology.orchestra.run.smithi110.stdout: "op_send": 923, 2024-09-21T03:42:30.993 INFO:teuthology.orchestra.run.smithi110.stdout: "op_send_bytes": 687610, 2024-09-21T03:42:30.993 INFO:teuthology.orchestra.run.smithi110.stdout: "op_resend": 37, 2024-09-21T03:42:30.993 INFO:teuthology.orchestra.run.smithi110.stdout: "op_reply": 923, 2024-09-21T03:42:30.994 INFO:teuthology.orchestra.run.smithi110.stdout: "op_latency": { 2024-09-21T03:42:30.994 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 923, 2024-09-21T03:42:30.994 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 22.343607634, 2024-09-21T03:42:30.994 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.024207592 2024-09-21T03:42:30.994 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-21T03:42:30.994 INFO:teuthology.orchestra.run.smithi110.stdout: "op_inflight": 0, 2024-09-21T03:42:30.994 INFO:teuthology.orchestra.run.smithi110.stdout: "oplen_avg": { 2024-09-21T03:42:30.994 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 923, 2024-09-21T03:42:30.994 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 1152 2024-09-21T03:42:30.994 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-21T03:42:30.994 INFO:teuthology.orchestra.run.smithi110.stdout: "op": 923, 2024-09-21T03:42:30.994 INFO:teuthology.orchestra.run.smithi110.stdout: "op_r": 2, 2024-09-21T03:42:30.994 INFO:teuthology.orchestra.run.smithi110.stdout: "op_w": 921, 2024-09-21T03:42:30.994 INFO:teuthology.orchestra.run.smithi110.stdout: "op_rmw": 0, 2024-09-21T03:42:30.994 INFO:teuthology.orchestra.run.smithi110.stdout: "op_pg": 0, 2024-09-21T03:42:30.995 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_stat": 61, 2024-09-21T03:42:30.995 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_create": 38, 2024-09-21T03:42:30.995 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_read": 2, 2024-09-21T03:42:30.995 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_write": 131, 2024-09-21T03:42:30.995 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_writefull": 41, 2024-09-21T03:42:30.995 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_writesame": 0, 2024-09-21T03:42:30.995 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_append": 0, 2024-09-21T03:42:30.995 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_zero": 0, 2024-09-21T03:42:30.995 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_truncate": 0, 2024-09-21T03:42:30.995 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_delete": 585, 2024-09-21T03:42:30.995 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_mapext": 0, 2024-09-21T03:42:30.995 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_sparse_read": 0, 2024-09-21T03:42:30.995 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_clonerange": 0, 2024-09-21T03:42:30.995 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_getxattr": 0, 2024-09-21T03:42:30.996 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_setxattr": 68, 2024-09-21T03:42:30.996 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_cmpxattr": 0, 2024-09-21T03:42:30.996 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_rmxattr": 0, 2024-09-21T03:42:30.996 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_resetxattrs": 0, 2024-09-21T03:42:30.996 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_call": 0, 2024-09-21T03:42:30.996 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_watch": 0, 2024-09-21T03:42:30.996 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_notify": 0, 2024-09-21T03:42:30.996 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_src_cmpxattr": 0, 2024-09-21T03:42:30.996 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_pgls": 0, 2024-09-21T03:42:30.996 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_pgls_filter": 0, 2024-09-21T03:42:30.996 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_other": 0, 2024-09-21T03:42:30.996 INFO:teuthology.orchestra.run.smithi110.stdout: "linger_active": 0, 2024-09-21T03:42:30.996 INFO:teuthology.orchestra.run.smithi110.stdout: "linger_send": 0, 2024-09-21T03:42:30.996 INFO:teuthology.orchestra.run.smithi110.stdout: "linger_resend": 0, 2024-09-21T03:42:30.996 INFO:teuthology.orchestra.run.smithi110.stdout: "linger_ping": 0, 2024-09-21T03:42:30.996 INFO:teuthology.orchestra.run.smithi110.stdout: "poolop_active": 0, 2024-09-21T03:42:30.997 INFO:teuthology.orchestra.run.smithi110.stdout: "poolop_send": 0, 2024-09-21T03:42:30.997 INFO:teuthology.orchestra.run.smithi110.stdout: "poolop_resend": 0, 2024-09-21T03:42:30.997 INFO:teuthology.orchestra.run.smithi110.stdout: "poolstat_active": 0, 2024-09-21T03:42:30.997 INFO:teuthology.orchestra.run.smithi110.stdout: "poolstat_send": 0, 2024-09-21T03:42:30.997 INFO:teuthology.orchestra.run.smithi110.stdout: "poolstat_resend": 0, 2024-09-21T03:42:30.997 INFO:teuthology.orchestra.run.smithi110.stdout: "statfs_active": 0, 2024-09-21T03:42:30.997 INFO:teuthology.orchestra.run.smithi110.stdout: "statfs_send": 0, 2024-09-21T03:42:30.997 INFO:teuthology.orchestra.run.smithi110.stdout: "statfs_resend": 0, 2024-09-21T03:42:30.997 INFO:teuthology.orchestra.run.smithi110.stdout: "command_active": 0, 2024-09-21T03:42:30.997 INFO:teuthology.orchestra.run.smithi110.stdout: "command_send": 0, 2024-09-21T03:42:30.997 INFO:teuthology.orchestra.run.smithi110.stdout: "command_resend": 0, 2024-09-21T03:42:30.997 INFO:teuthology.orchestra.run.smithi110.stdout: "map_epoch": 84, 2024-09-21T03:42:30.997 INFO:teuthology.orchestra.run.smithi110.stdout: "map_full": 0, 2024-09-21T03:42:30.997 INFO:teuthology.orchestra.run.smithi110.stdout: "map_inc": 10, 2024-09-21T03:42:30.997 INFO:teuthology.orchestra.run.smithi110.stdout: "osd_sessions": 12, 2024-09-21T03:42:30.997 INFO:teuthology.orchestra.run.smithi110.stdout: "osd_session_open": 12, 2024-09-21T03:42:30.998 INFO:teuthology.orchestra.run.smithi110.stdout: "osd_session_close": 0, 2024-09-21T03:42:30.998 INFO:teuthology.orchestra.run.smithi110.stdout: "osd_laggy": 0, 2024-09-21T03:42:30.998 INFO:teuthology.orchestra.run.smithi110.stdout: "omap_wr": 209, 2024-09-21T03:42:30.998 INFO:teuthology.orchestra.run.smithi110.stdout: "omap_rd": 0, 2024-09-21T03:42:30.998 INFO:teuthology.orchestra.run.smithi110.stdout: "omap_del": 17 2024-09-21T03:42:30.998 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-21T03:42:30.998 INFO:teuthology.orchestra.run.smithi110.stdout: "oft": { 2024-09-21T03:42:30.998 INFO:teuthology.orchestra.run.smithi110.stdout: "omap_total_objs": 1, 2024-09-21T03:42:30.998 INFO:teuthology.orchestra.run.smithi110.stdout: "omap_total_kv_pairs": 0, 2024-09-21T03:42:30.998 INFO:teuthology.orchestra.run.smithi110.stdout: "omap_total_updates": 0, 2024-09-21T03:42:30.998 INFO:teuthology.orchestra.run.smithi110.stdout: "omap_total_removes": 0 2024-09-21T03:42:30.998 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-21T03:42:30.998 INFO:teuthology.orchestra.run.smithi110.stdout: "purge_queue": { 2024-09-21T03:42:30.998 INFO:teuthology.orchestra.run.smithi110.stdout: "pq_executing_ops": 0, 2024-09-21T03:42:30.998 INFO:teuthology.orchestra.run.smithi110.stdout: "pq_executing_ops_high_water": 70, 2024-09-21T03:42:30.998 INFO:teuthology.orchestra.run.smithi110.stdout: "pq_executing": 0, 2024-09-21T03:42:30.999 INFO:teuthology.orchestra.run.smithi110.stdout: "pq_executing_high_water": 4, 2024-09-21T03:42:30.999 INFO:teuthology.orchestra.run.smithi110.stdout: "pq_executed_ops": 73, 2024-09-21T03:42:30.999 INFO:teuthology.orchestra.run.smithi110.stdout: "pq_executed": 6, 2024-09-21T03:42:30.999 INFO:teuthology.orchestra.run.smithi110.stdout: "pq_item_in_journal": 0 2024-09-21T03:42:30.999 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-21T03:42:30.999 INFO:teuthology.orchestra.run.smithi110.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-21T03:42:30.999 INFO:teuthology.orchestra.run.smithi110.stdout: "val": 0, 2024-09-21T03:42:30.999 INFO:teuthology.orchestra.run.smithi110.stdout: "max": 104857600, 2024-09-21T03:42:30.999 INFO:teuthology.orchestra.run.smithi110.stdout: "get_started": 0, 2024-09-21T03:42:30.999 INFO:teuthology.orchestra.run.smithi110.stdout: "get": 2128, 2024-09-21T03:42:30.999 INFO:teuthology.orchestra.run.smithi110.stdout: "get_sum": 1303513, 2024-09-21T03:42:30.999 INFO:teuthology.orchestra.run.smithi110.stdout: "get_or_fail_fail": 0, 2024-09-21T03:42:30.999 INFO:teuthology.orchestra.run.smithi110.stdout: "get_or_fail_success": 2128, 2024-09-21T03:42:30.999 INFO:teuthology.orchestra.run.smithi110.stdout: "take": 0, 2024-09-21T03:42:30.999 INFO:teuthology.orchestra.run.smithi110.stdout: "take_sum": 0, 2024-09-21T03:42:31.000 INFO:teuthology.orchestra.run.smithi110.stdout: "put": 2128, 2024-09-21T03:42:31.000 INFO:teuthology.orchestra.run.smithi110.stdout: "put_sum": 1303513, 2024-09-21T03:42:31.000 INFO:teuthology.orchestra.run.smithi110.stdout: "wait": { 2024-09-21T03:42:31.000 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-09-21T03:42:31.000 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-09-21T03:42:31.000 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-09-21T03:42:31.000 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-21T03:42:31.000 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-21T03:42:31.000 INFO:teuthology.orchestra.run.smithi110.stdout: "throttle-objecter_bytes": { 2024-09-21T03:42:31.000 INFO:teuthology.orchestra.run.smithi110.stdout: "val": 0, 2024-09-21T03:42:31.000 INFO:teuthology.orchestra.run.smithi110.stdout: "max": 104857600, 2024-09-21T03:42:31.000 INFO:teuthology.orchestra.run.smithi110.stdout: "get_started": 0, 2024-09-21T03:42:31.000 INFO:teuthology.orchestra.run.smithi110.stdout: "get": 0, 2024-09-21T03:42:31.000 INFO:teuthology.orchestra.run.smithi110.stdout: "get_sum": 0, 2024-09-21T03:42:31.000 INFO:teuthology.orchestra.run.smithi110.stdout: "get_or_fail_fail": 0, 2024-09-21T03:42:31.000 INFO:teuthology.orchestra.run.smithi110.stdout: "get_or_fail_success": 0, 2024-09-21T03:42:31.001 INFO:teuthology.orchestra.run.smithi110.stdout: "take": 923, 2024-09-21T03:42:31.001 INFO:teuthology.orchestra.run.smithi110.stdout: "take_sum": 688248, 2024-09-21T03:42:31.001 INFO:teuthology.orchestra.run.smithi110.stdout: "put": 923, 2024-09-21T03:42:31.001 INFO:teuthology.orchestra.run.smithi110.stdout: "put_sum": 688248, 2024-09-21T03:42:31.001 INFO:teuthology.orchestra.run.smithi110.stdout: "wait": { 2024-09-21T03:42:31.001 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-09-21T03:42:31.001 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-09-21T03:42:31.001 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-09-21T03:42:31.001 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-21T03:42:31.001 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-21T03:42:31.001 INFO:teuthology.orchestra.run.smithi110.stdout: "throttle-objecter_ops": { 2024-09-21T03:42:31.001 INFO:teuthology.orchestra.run.smithi110.stdout: "val": 0, 2024-09-21T03:42:31.001 INFO:teuthology.orchestra.run.smithi110.stdout: "max": 1024, 2024-09-21T03:42:31.001 INFO:teuthology.orchestra.run.smithi110.stdout: "get_started": 0, 2024-09-21T03:42:31.001 INFO:teuthology.orchestra.run.smithi110.stdout: "get": 0, 2024-09-21T03:42:31.002 INFO:teuthology.orchestra.run.smithi110.stdout: "get_sum": 0, 2024-09-21T03:42:31.002 INFO:teuthology.orchestra.run.smithi110.stdout: "get_or_fail_fail": 0, 2024-09-21T03:42:31.002 INFO:teuthology.orchestra.run.smithi110.stdout: "get_or_fail_success": 0, 2024-09-21T03:42:31.002 INFO:teuthology.orchestra.run.smithi110.stdout: "take": 923, 2024-09-21T03:42:31.002 INFO:teuthology.orchestra.run.smithi110.stdout: "take_sum": 923, 2024-09-21T03:42:31.002 INFO:teuthology.orchestra.run.smithi110.stdout: "put": 923, 2024-09-21T03:42:31.002 INFO:teuthology.orchestra.run.smithi110.stdout: "put_sum": 923, 2024-09-21T03:42:31.002 INFO:teuthology.orchestra.run.smithi110.stdout: "wait": { 2024-09-21T03:42:31.002 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-09-21T03:42:31.002 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-09-21T03:42:31.002 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-09-21T03:42:31.002 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-21T03:42:31.002 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-21T03:42:31.002 INFO:teuthology.orchestra.run.smithi110.stdout: "throttle-write_buf_throttle": { 2024-09-21T03:42:31.002 INFO:teuthology.orchestra.run.smithi110.stdout: "val": 0, 2024-09-21T03:42:31.003 INFO:teuthology.orchestra.run.smithi110.stdout: "max": 3758096384, 2024-09-21T03:42:31.003 INFO:teuthology.orchestra.run.smithi110.stdout: "get_started": 0, 2024-09-21T03:42:31.003 INFO:teuthology.orchestra.run.smithi110.stdout: "get": 6, 2024-09-21T03:42:31.003 INFO:teuthology.orchestra.run.smithi110.stdout: "get_sum": 638, 2024-09-21T03:42:31.003 INFO:teuthology.orchestra.run.smithi110.stdout: "get_or_fail_fail": 0, 2024-09-21T03:42:31.003 INFO:teuthology.orchestra.run.smithi110.stdout: "get_or_fail_success": 6, 2024-09-21T03:42:31.003 INFO:teuthology.orchestra.run.smithi110.stdout: "take": 0, 2024-09-21T03:42:31.003 INFO:teuthology.orchestra.run.smithi110.stdout: "take_sum": 0, 2024-09-21T03:42:31.003 INFO:teuthology.orchestra.run.smithi110.stdout: "put": 2, 2024-09-21T03:42:31.003 INFO:teuthology.orchestra.run.smithi110.stdout: "put_sum": 638, 2024-09-21T03:42:31.003 INFO:teuthology.orchestra.run.smithi110.stdout: "wait": { 2024-09-21T03:42:31.003 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-09-21T03:42:31.003 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-09-21T03:42:31.003 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-09-21T03:42:31.003 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-21T03:42:31.004 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-21T03:42:31.004 INFO:teuthology.orchestra.run.smithi110.stdout: "throttle-write_buf_throttle-0x55f22d8c5450": { 2024-09-21T03:42:31.004 INFO:teuthology.orchestra.run.smithi110.stdout: "val": 0, 2024-09-21T03:42:31.004 INFO:teuthology.orchestra.run.smithi110.stdout: "max": 3758096384, 2024-09-21T03:42:31.004 INFO:teuthology.orchestra.run.smithi110.stdout: "get_started": 0, 2024-09-21T03:42:31.004 INFO:teuthology.orchestra.run.smithi110.stdout: "get": 161, 2024-09-21T03:42:31.004 INFO:teuthology.orchestra.run.smithi110.stdout: "get_sum": 592601, 2024-09-21T03:42:31.004 INFO:teuthology.orchestra.run.smithi110.stdout: "get_or_fail_fail": 0, 2024-09-21T03:42:31.004 INFO:teuthology.orchestra.run.smithi110.stdout: "get_or_fail_success": 161, 2024-09-21T03:42:31.004 INFO:teuthology.orchestra.run.smithi110.stdout: "take": 0, 2024-09-21T03:42:31.004 INFO:teuthology.orchestra.run.smithi110.stdout: "take_sum": 0, 2024-09-21T03:42:31.004 INFO:teuthology.orchestra.run.smithi110.stdout: "put": 129, 2024-09-21T03:42:31.004 INFO:teuthology.orchestra.run.smithi110.stdout: "put_sum": 592601, 2024-09-21T03:42:31.004 INFO:teuthology.orchestra.run.smithi110.stdout: "wait": { 2024-09-21T03:42:31.004 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-09-21T03:42:31.005 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-09-21T03:42:31.005 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-09-21T03:42:31.005 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-21T03:42:31.005 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-21T03:42:31.005 INFO:teuthology.orchestra.run.smithi110.stdout: "trackedop": { 2024-09-21T03:42:31.005 INFO:teuthology.orchestra.run.smithi110.stdout: "slow_ops_count": 48 2024-09-21T03:42:31.005 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-21T03:42:31.005 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-21T03:42:31.005 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:42:30 smithi110 ceph-mon[34000]: pgmap v453: 129 pgs: 129 active+clean; 1.1 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 47 KiB/s wr, 26 op/s 2024-09-21T03:42:31.061 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:42:30 smithi089 ceph-mon[30928]: pgmap v453: 129 pgs: 129 active+clean; 1.1 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 47 KiB/s wr, 26 op/s 2024-09-21T03:42:31.211 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:42:30 smithi114 ceph-mon[34140]: pgmap v453: 129 pgs: 129 active+clean; 1.1 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 47 KiB/s wr, 26 op/s 2024-09-21T03:42:31.710 INFO:tasks.check_counter:Daemon mds.b mds.exported=0 2024-09-21T03:42:31.710 INFO:tasks.check_counter:Daemon mds.b mds.imported=0 2024-09-21T03:42:31.710 DEBUG:tasks.check_counter:Getting stats from e 2024-09-21T03:42:31.710 DEBUG:teuthology.orchestra.run.smithi110:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 7c374098-77c9-11ef-baf5-efdc52797490 -- ceph daemon mds.e perf dump 2024-09-21T03:42:31.956 INFO:teuthology.orchestra.run.smithi110.stderr:Inferring config /var/lib/ceph/7c374098-77c9-11ef-baf5-efdc52797490/mon.b/config 2024-09-21T03:42:32.637 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-21T03:42:32.637 INFO:teuthology.orchestra.run.smithi110.stdout: "AsyncMessenger::Worker-0": { 2024-09-21T03:42:32.637 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_recv_messages": 290, 2024-09-21T03:42:32.638 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_send_messages": 292, 2024-09-21T03:42:32.638 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_recv_bytes": 78168, 2024-09-21T03:42:32.638 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_send_bytes": 92284, 2024-09-21T03:42:32.638 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_created_connections": 6, 2024-09-21T03:42:32.638 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_active_connections": 2, 2024-09-21T03:42:32.638 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_running_total_time": 0.034533321, 2024-09-21T03:42:32.638 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_running_send_time": 0.016619927, 2024-09-21T03:42:32.638 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_running_recv_time": 0.007180395, 2024-09-21T03:42:32.638 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_running_fast_dispatch_time": 0.007306791, 2024-09-21T03:42:32.638 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T03:42:32.638 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 290, 2024-09-21T03:42:32.638 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.013425939, 2024-09-21T03:42:32.638 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000046296 2024-09-21T03:42:32.638 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-21T03:42:32.639 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_handle_ack_lat": { 2024-09-21T03:42:32.639 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-09-21T03:42:32.639 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-09-21T03:42:32.639 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-09-21T03:42:32.639 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-21T03:42:32.639 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-21T03:42:32.639 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_send_encrypted_bytes": 0 2024-09-21T03:42:32.639 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-21T03:42:32.639 INFO:teuthology.orchestra.run.smithi110.stdout: "AsyncMessenger::Worker-1": { 2024-09-21T03:42:32.639 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_recv_messages": 307, 2024-09-21T03:42:32.639 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_send_messages": 298, 2024-09-21T03:42:32.639 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_recv_bytes": 877386, 2024-09-21T03:42:32.639 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_send_bytes": 121001, 2024-09-21T03:42:32.639 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_created_connections": 4, 2024-09-21T03:42:32.640 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_active_connections": 3, 2024-09-21T03:42:32.640 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_running_total_time": 0.046869924, 2024-09-21T03:42:32.640 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_running_send_time": 0.017693619, 2024-09-21T03:42:32.640 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_running_recv_time": 0.029536189, 2024-09-21T03:42:32.640 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_running_fast_dispatch_time": 0.007137050, 2024-09-21T03:42:32.640 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T03:42:32.640 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 298, 2024-09-21T03:42:32.640 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.018505908, 2024-09-21T03:42:32.640 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000062100 2024-09-21T03:42:32.640 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-21T03:42:32.640 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_handle_ack_lat": { 2024-09-21T03:42:32.640 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-09-21T03:42:32.641 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-09-21T03:42:32.641 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-09-21T03:42:32.641 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-21T03:42:32.641 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_recv_encrypted_bytes": 830624, 2024-09-21T03:42:32.641 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_send_encrypted_bytes": 60576 2024-09-21T03:42:32.641 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-21T03:42:32.641 INFO:teuthology.orchestra.run.smithi110.stdout: "AsyncMessenger::Worker-2": { 2024-09-21T03:42:32.641 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_recv_messages": 284, 2024-09-21T03:42:32.641 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_send_messages": 356, 2024-09-21T03:42:32.641 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_recv_bytes": 95248, 2024-09-21T03:42:32.641 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_send_bytes": 286602, 2024-09-21T03:42:32.641 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_created_connections": 4, 2024-09-21T03:42:32.641 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_active_connections": 3, 2024-09-21T03:42:32.641 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_running_total_time": 0.050389495, 2024-09-21T03:42:32.641 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_running_send_time": 0.022944226, 2024-09-21T03:42:32.641 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_running_recv_time": 0.011112014, 2024-09-21T03:42:32.642 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_running_fast_dispatch_time": 0.012168728, 2024-09-21T03:42:32.642 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T03:42:32.642 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 354, 2024-09-21T03:42:32.642 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.024168107, 2024-09-21T03:42:32.642 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000068271 2024-09-21T03:42:32.642 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-21T03:42:32.642 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_handle_ack_lat": { 2024-09-21T03:42:32.642 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-09-21T03:42:32.642 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-09-21T03:42:32.642 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-09-21T03:42:32.642 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-21T03:42:32.642 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_recv_encrypted_bytes": 160, 2024-09-21T03:42:32.642 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_send_encrypted_bytes": 196608 2024-09-21T03:42:32.642 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-21T03:42:32.642 INFO:teuthology.orchestra.run.smithi110.stdout: "cct": { 2024-09-21T03:42:32.643 INFO:teuthology.orchestra.run.smithi110.stdout: "total_workers": 1, 2024-09-21T03:42:32.643 INFO:teuthology.orchestra.run.smithi110.stdout: "unhealthy_workers": 0 2024-09-21T03:42:32.643 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-21T03:42:32.643 INFO:teuthology.orchestra.run.smithi110.stdout: "finisher-MDSRank": { 2024-09-21T03:42:32.643 INFO:teuthology.orchestra.run.smithi110.stdout: "queue_len": 0, 2024-09-21T03:42:32.643 INFO:teuthology.orchestra.run.smithi110.stdout: "complete_latency": { 2024-09-21T03:42:32.643 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 1318, 2024-09-21T03:42:32.643 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.042885101, 2024-09-21T03:42:32.643 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000032538 2024-09-21T03:42:32.643 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-21T03:42:32.643 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-21T03:42:32.643 INFO:teuthology.orchestra.run.smithi110.stdout: "finisher-PurgeQueue": { 2024-09-21T03:42:32.643 INFO:teuthology.orchestra.run.smithi110.stdout: "queue_len": 0, 2024-09-21T03:42:32.643 INFO:teuthology.orchestra.run.smithi110.stdout: "complete_latency": { 2024-09-21T03:42:32.643 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-09-21T03:42:32.644 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-09-21T03:42:32.644 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-09-21T03:42:32.644 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-21T03:42:32.644 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-21T03:42:32.644 INFO:teuthology.orchestra.run.smithi110.stdout: "mds": { 2024-09-21T03:42:32.644 INFO:teuthology.orchestra.run.smithi110.stdout: "request": 0, 2024-09-21T03:42:32.644 INFO:teuthology.orchestra.run.smithi110.stdout: "reply": 0, 2024-09-21T03:42:32.644 INFO:teuthology.orchestra.run.smithi110.stdout: "reply_latency": { 2024-09-21T03:42:32.644 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-09-21T03:42:32.644 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-09-21T03:42:32.644 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-09-21T03:42:32.644 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-21T03:42:32.644 INFO:teuthology.orchestra.run.smithi110.stdout: "slow_reply": 0, 2024-09-21T03:42:32.644 INFO:teuthology.orchestra.run.smithi110.stdout: "forward": 0, 2024-09-21T03:42:32.644 INFO:teuthology.orchestra.run.smithi110.stdout: "dir_fetch_complete": 0, 2024-09-21T03:42:32.645 INFO:teuthology.orchestra.run.smithi110.stdout: "dir_fetch_keys": 0, 2024-09-21T03:42:32.645 INFO:teuthology.orchestra.run.smithi110.stdout: "dir_commit": 0, 2024-09-21T03:42:32.645 INFO:teuthology.orchestra.run.smithi110.stdout: "dir_split": 0, 2024-09-21T03:42:32.645 INFO:teuthology.orchestra.run.smithi110.stdout: "dir_merge": 0, 2024-09-21T03:42:32.645 INFO:teuthology.orchestra.run.smithi110.stdout: "inodes": 0, 2024-09-21T03:42:32.645 INFO:teuthology.orchestra.run.smithi110.stdout: "inodes_top": 0, 2024-09-21T03:42:32.645 INFO:teuthology.orchestra.run.smithi110.stdout: "inodes_bottom": 0, 2024-09-21T03:42:32.645 INFO:teuthology.orchestra.run.smithi110.stdout: "inodes_pin_tail": 0, 2024-09-21T03:42:32.645 INFO:teuthology.orchestra.run.smithi110.stdout: "inodes_pinned": 0, 2024-09-21T03:42:32.645 INFO:teuthology.orchestra.run.smithi110.stdout: "inodes_expired": 0, 2024-09-21T03:42:32.645 INFO:teuthology.orchestra.run.smithi110.stdout: "inodes_with_caps": 0, 2024-09-21T03:42:32.645 INFO:teuthology.orchestra.run.smithi110.stdout: "caps": 0, 2024-09-21T03:42:32.645 INFO:teuthology.orchestra.run.smithi110.stdout: "subtrees": 1, 2024-09-21T03:42:32.646 INFO:teuthology.orchestra.run.smithi110.stdout: "traverse": 0, 2024-09-21T03:42:32.646 INFO:teuthology.orchestra.run.smithi110.stdout: "traverse_hit": 0, 2024-09-21T03:42:32.646 INFO:teuthology.orchestra.run.smithi110.stdout: "traverse_forward": 0, 2024-09-21T03:42:32.646 INFO:teuthology.orchestra.run.smithi110.stdout: "traverse_discover": 0, 2024-09-21T03:42:32.646 INFO:teuthology.orchestra.run.smithi110.stdout: "traverse_dir_fetch": 0, 2024-09-21T03:42:32.646 INFO:teuthology.orchestra.run.smithi110.stdout: "traverse_remote_ino": 0, 2024-09-21T03:42:32.646 INFO:teuthology.orchestra.run.smithi110.stdout: "traverse_lock": 0, 2024-09-21T03:42:32.646 INFO:teuthology.orchestra.run.smithi110.stdout: "load_cent": 0, 2024-09-21T03:42:32.646 INFO:teuthology.orchestra.run.smithi110.stdout: "q": 0, 2024-09-21T03:42:32.646 INFO:teuthology.orchestra.run.smithi110.stdout: "exported": 0, 2024-09-21T03:42:32.646 INFO:teuthology.orchestra.run.smithi110.stdout: "exported_inodes": 0, 2024-09-21T03:42:32.646 INFO:teuthology.orchestra.run.smithi110.stdout: "imported": 0, 2024-09-21T03:42:32.646 INFO:teuthology.orchestra.run.smithi110.stdout: "imported_inodes": 0, 2024-09-21T03:42:32.647 INFO:teuthology.orchestra.run.smithi110.stdout: "openino_dir_fetch": 0, 2024-09-21T03:42:32.647 INFO:teuthology.orchestra.run.smithi110.stdout: "openino_backtrace_fetch": 0, 2024-09-21T03:42:32.647 INFO:teuthology.orchestra.run.smithi110.stdout: "openino_peer_discover": 0, 2024-09-21T03:42:32.647 INFO:teuthology.orchestra.run.smithi110.stdout: "root_rfiles": 0, 2024-09-21T03:42:32.647 INFO:teuthology.orchestra.run.smithi110.stdout: "root_rbytes": 0, 2024-09-21T03:42:32.647 INFO:teuthology.orchestra.run.smithi110.stdout: "root_rsnaps": 0, 2024-09-21T03:42:32.647 INFO:teuthology.orchestra.run.smithi110.stdout: "scrub_backtrace_fetch": 0, 2024-09-21T03:42:32.647 INFO:teuthology.orchestra.run.smithi110.stdout: "scrub_set_tag": 0, 2024-09-21T03:42:32.647 INFO:teuthology.orchestra.run.smithi110.stdout: "scrub_backtrace_repaired": 0, 2024-09-21T03:42:32.647 INFO:teuthology.orchestra.run.smithi110.stdout: "scrub_inotable_repaired": 0, 2024-09-21T03:42:32.647 INFO:teuthology.orchestra.run.smithi110.stdout: "scrub_dir_inodes": 0, 2024-09-21T03:42:32.647 INFO:teuthology.orchestra.run.smithi110.stdout: "scrub_dir_base_inodes": 0, 2024-09-21T03:42:32.647 INFO:teuthology.orchestra.run.smithi110.stdout: "scrub_dirfrag_rstats": 0, 2024-09-21T03:42:32.647 INFO:teuthology.orchestra.run.smithi110.stdout: "scrub_file_inodes": 0, 2024-09-21T03:42:32.647 INFO:teuthology.orchestra.run.smithi110.stdout: "handle_inode_file_caps": 0, 2024-09-21T03:42:32.648 INFO:teuthology.orchestra.run.smithi110.stdout: "ceph_cap_op_revoke": 0, 2024-09-21T03:42:32.648 INFO:teuthology.orchestra.run.smithi110.stdout: "ceph_cap_op_grant": 0, 2024-09-21T03:42:32.648 INFO:teuthology.orchestra.run.smithi110.stdout: "ceph_cap_op_trunc": 0, 2024-09-21T03:42:32.648 INFO:teuthology.orchestra.run.smithi110.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-21T03:42:32.648 INFO:teuthology.orchestra.run.smithi110.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-21T03:42:32.648 INFO:teuthology.orchestra.run.smithi110.stdout: "handle_client_caps": 0, 2024-09-21T03:42:32.648 INFO:teuthology.orchestra.run.smithi110.stdout: "handle_client_caps_dirty": 0, 2024-09-21T03:42:32.648 INFO:teuthology.orchestra.run.smithi110.stdout: "handle_client_cap_release": 0, 2024-09-21T03:42:32.648 INFO:teuthology.orchestra.run.smithi110.stdout: "process_request_cap_release": 0 2024-09-21T03:42:32.648 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-21T03:42:32.648 INFO:teuthology.orchestra.run.smithi110.stdout: "mds_cache": { 2024-09-21T03:42:32.648 INFO:teuthology.orchestra.run.smithi110.stdout: "dir_update": 0, 2024-09-21T03:42:32.648 INFO:teuthology.orchestra.run.smithi110.stdout: "dir_update_receipt": 0, 2024-09-21T03:42:32.648 INFO:teuthology.orchestra.run.smithi110.stdout: "dir_try_discover": 0, 2024-09-21T03:42:32.648 INFO:teuthology.orchestra.run.smithi110.stdout: "dir_send_discover": 0, 2024-09-21T03:42:32.649 INFO:teuthology.orchestra.run.smithi110.stdout: "dir_handle_discover": 0, 2024-09-21T03:42:32.649 INFO:teuthology.orchestra.run.smithi110.stdout: "num_strays": 0, 2024-09-21T03:42:32.649 INFO:teuthology.orchestra.run.smithi110.stdout: "num_strays_delayed": 0, 2024-09-21T03:42:32.649 INFO:teuthology.orchestra.run.smithi110.stdout: "num_strays_enqueuing": 0, 2024-09-21T03:42:32.649 INFO:teuthology.orchestra.run.smithi110.stdout: "strays_created": 0, 2024-09-21T03:42:32.649 INFO:teuthology.orchestra.run.smithi110.stdout: "strays_enqueued": 0, 2024-09-21T03:42:32.649 INFO:teuthology.orchestra.run.smithi110.stdout: "strays_reintegrated": 0, 2024-09-21T03:42:32.649 INFO:teuthology.orchestra.run.smithi110.stdout: "strays_migrated": 0, 2024-09-21T03:42:32.649 INFO:teuthology.orchestra.run.smithi110.stdout: "num_recovering_processing": 0, 2024-09-21T03:42:32.649 INFO:teuthology.orchestra.run.smithi110.stdout: "num_recovering_enqueued": 0, 2024-09-21T03:42:32.649 INFO:teuthology.orchestra.run.smithi110.stdout: "num_recovering_prioritized": 0, 2024-09-21T03:42:32.649 INFO:teuthology.orchestra.run.smithi110.stdout: "recovery_started": 0, 2024-09-21T03:42:32.649 INFO:teuthology.orchestra.run.smithi110.stdout: "recovery_completed": 0, 2024-09-21T03:42:32.649 INFO:teuthology.orchestra.run.smithi110.stdout: "ireq_quiesce_path": 0, 2024-09-21T03:42:32.649 INFO:teuthology.orchestra.run.smithi110.stdout: "ireq_quiesce_inode": 0, 2024-09-21T03:42:32.649 INFO:teuthology.orchestra.run.smithi110.stdout: "ireq_enqueue_scrub": 0, 2024-09-21T03:42:32.650 INFO:teuthology.orchestra.run.smithi110.stdout: "ireq_exportdir": 0, 2024-09-21T03:42:32.650 INFO:teuthology.orchestra.run.smithi110.stdout: "ireq_flush": 0, 2024-09-21T03:42:32.650 INFO:teuthology.orchestra.run.smithi110.stdout: "ireq_fragmentdir": 0, 2024-09-21T03:42:32.650 INFO:teuthology.orchestra.run.smithi110.stdout: "ireq_fragstats": 0, 2024-09-21T03:42:32.650 INFO:teuthology.orchestra.run.smithi110.stdout: "ireq_inodestats": 0, 2024-09-21T03:42:32.650 INFO:teuthology.orchestra.run.smithi110.stdout: "uninline_started": 0, 2024-09-21T03:42:32.650 INFO:teuthology.orchestra.run.smithi110.stdout: "uninline_succeeded": 0, 2024-09-21T03:42:32.650 INFO:teuthology.orchestra.run.smithi110.stdout: "uninline_write_failed": 0 2024-09-21T03:42:32.650 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-21T03:42:32.650 INFO:teuthology.orchestra.run.smithi110.stdout: "mds_log": { 2024-09-21T03:42:32.650 INFO:teuthology.orchestra.run.smithi110.stdout: "evlrg": 0, 2024-09-21T03:42:32.650 INFO:teuthology.orchestra.run.smithi110.stdout: "evadd": 0, 2024-09-21T03:42:32.650 INFO:teuthology.orchestra.run.smithi110.stdout: "evex": 0, 2024-09-21T03:42:32.650 INFO:teuthology.orchestra.run.smithi110.stdout: "evtrm": 0, 2024-09-21T03:42:32.650 INFO:teuthology.orchestra.run.smithi110.stdout: "ev": 5, 2024-09-21T03:42:32.650 INFO:teuthology.orchestra.run.smithi110.stdout: "evexg": 0, 2024-09-21T03:42:32.651 INFO:teuthology.orchestra.run.smithi110.stdout: "evexd": 0, 2024-09-21T03:42:32.651 INFO:teuthology.orchestra.run.smithi110.stdout: "segadd": 0, 2024-09-21T03:42:32.651 INFO:teuthology.orchestra.run.smithi110.stdout: "segex": 0, 2024-09-21T03:42:32.651 INFO:teuthology.orchestra.run.smithi110.stdout: "segtrm": 0, 2024-09-21T03:42:32.651 INFO:teuthology.orchestra.run.smithi110.stdout: "seg": 2, 2024-09-21T03:42:32.651 INFO:teuthology.orchestra.run.smithi110.stdout: "segmjr": 2, 2024-09-21T03:42:32.651 INFO:teuthology.orchestra.run.smithi110.stdout: "segexg": 0, 2024-09-21T03:42:32.651 INFO:teuthology.orchestra.run.smithi110.stdout: "segexd": 0, 2024-09-21T03:42:32.651 INFO:teuthology.orchestra.run.smithi110.stdout: "expos": 4194304, 2024-09-21T03:42:32.651 INFO:teuthology.orchestra.run.smithi110.stdout: "wrpos": 4195917, 2024-09-21T03:42:32.651 INFO:teuthology.orchestra.run.smithi110.stdout: "rdpos": 4195776, 2024-09-21T03:42:32.651 INFO:teuthology.orchestra.run.smithi110.stdout: "jlat": { 2024-09-21T03:42:32.651 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-09-21T03:42:32.651 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-09-21T03:42:32.651 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-09-21T03:42:32.651 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-21T03:42:32.652 INFO:teuthology.orchestra.run.smithi110.stdout: "replayed": 5 2024-09-21T03:42:32.652 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-21T03:42:32.652 INFO:teuthology.orchestra.run.smithi110.stdout: "mds_mem": { 2024-09-21T03:42:32.652 INFO:teuthology.orchestra.run.smithi110.stdout: "ino": 3, 2024-09-21T03:42:32.652 INFO:teuthology.orchestra.run.smithi110.stdout: "ino+": 3, 2024-09-21T03:42:32.652 INFO:teuthology.orchestra.run.smithi110.stdout: "ino-": 0, 2024-09-21T03:42:32.652 INFO:teuthology.orchestra.run.smithi110.stdout: "dir": 1, 2024-09-21T03:42:32.652 INFO:teuthology.orchestra.run.smithi110.stdout: "dir+": 1, 2024-09-21T03:42:32.652 INFO:teuthology.orchestra.run.smithi110.stdout: "dir-": 0, 2024-09-21T03:42:32.652 INFO:teuthology.orchestra.run.smithi110.stdout: "dn": 0, 2024-09-21T03:42:32.652 INFO:teuthology.orchestra.run.smithi110.stdout: "dn+": 0, 2024-09-21T03:42:32.652 INFO:teuthology.orchestra.run.smithi110.stdout: "dn-": 0, 2024-09-21T03:42:32.652 INFO:teuthology.orchestra.run.smithi110.stdout: "cap": 0, 2024-09-21T03:42:32.652 INFO:teuthology.orchestra.run.smithi110.stdout: "cap+": 0, 2024-09-21T03:42:32.652 INFO:teuthology.orchestra.run.smithi110.stdout: "cap-": 0, 2024-09-21T03:42:32.652 INFO:teuthology.orchestra.run.smithi110.stdout: "rss": 118716, 2024-09-21T03:42:32.653 INFO:teuthology.orchestra.run.smithi110.stdout: "heap": 315371520 2024-09-21T03:42:32.653 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-21T03:42:32.653 INFO:teuthology.orchestra.run.smithi110.stdout: "mds_server": { 2024-09-21T03:42:32.653 INFO:teuthology.orchestra.run.smithi110.stdout: "dispatch_client_request": 0, 2024-09-21T03:42:32.653 INFO:teuthology.orchestra.run.smithi110.stdout: "dispatch_server_request": 0, 2024-09-21T03:42:32.653 INFO:teuthology.orchestra.run.smithi110.stdout: "handle_client_request": 0, 2024-09-21T03:42:32.653 INFO:teuthology.orchestra.run.smithi110.stdout: "handle_client_session": 0, 2024-09-21T03:42:32.653 INFO:teuthology.orchestra.run.smithi110.stdout: "handle_peer_request": 0, 2024-09-21T03:42:32.653 INFO:teuthology.orchestra.run.smithi110.stdout: "req_create_latency": { 2024-09-21T03:42:32.653 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-09-21T03:42:32.653 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-09-21T03:42:32.653 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-09-21T03:42:32.653 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-21T03:42:32.653 INFO:teuthology.orchestra.run.smithi110.stdout: "req_getattr_latency": { 2024-09-21T03:42:32.653 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-09-21T03:42:32.653 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-09-21T03:42:32.654 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-09-21T03:42:32.654 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-21T03:42:32.654 INFO:teuthology.orchestra.run.smithi110.stdout: "req_getfilelock_latency": { 2024-09-21T03:42:32.654 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-09-21T03:42:32.654 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-09-21T03:42:32.654 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-09-21T03:42:32.654 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-21T03:42:32.654 INFO:teuthology.orchestra.run.smithi110.stdout: "req_link_latency": { 2024-09-21T03:42:32.654 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-09-21T03:42:32.654 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-09-21T03:42:32.654 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-09-21T03:42:32.654 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-21T03:42:32.654 INFO:teuthology.orchestra.run.smithi110.stdout: "req_lookup_latency": { 2024-09-21T03:42:32.655 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-09-21T03:42:32.655 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-09-21T03:42:32.655 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-09-21T03:42:32.655 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-21T03:42:32.655 INFO:teuthology.orchestra.run.smithi110.stdout: "req_lookuphash_latency": { 2024-09-21T03:42:32.655 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-09-21T03:42:32.655 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-09-21T03:42:32.655 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-09-21T03:42:32.655 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-21T03:42:32.655 INFO:teuthology.orchestra.run.smithi110.stdout: "req_lookupino_latency": { 2024-09-21T03:42:32.655 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-09-21T03:42:32.655 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-09-21T03:42:32.655 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-09-21T03:42:32.655 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-21T03:42:32.655 INFO:teuthology.orchestra.run.smithi110.stdout: "req_lookupname_latency": { 2024-09-21T03:42:32.655 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-09-21T03:42:32.656 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-09-21T03:42:32.656 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-09-21T03:42:32.656 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-21T03:42:32.656 INFO:teuthology.orchestra.run.smithi110.stdout: "req_lookupparent_latency": { 2024-09-21T03:42:32.656 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-09-21T03:42:32.656 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-09-21T03:42:32.656 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-09-21T03:42:32.656 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-21T03:42:32.656 INFO:teuthology.orchestra.run.smithi110.stdout: "req_lookupsnap_latency": { 2024-09-21T03:42:32.656 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-09-21T03:42:32.656 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-09-21T03:42:32.656 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-09-21T03:42:32.656 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-21T03:42:32.656 INFO:teuthology.orchestra.run.smithi110.stdout: "req_lssnap_latency": { 2024-09-21T03:42:32.656 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-09-21T03:42:32.657 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-09-21T03:42:32.657 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-09-21T03:42:32.657 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-21T03:42:32.657 INFO:teuthology.orchestra.run.smithi110.stdout: "req_mkdir_latency": { 2024-09-21T03:42:32.657 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-09-21T03:42:32.657 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-09-21T03:42:32.657 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-09-21T03:42:32.657 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-21T03:42:32.657 INFO:teuthology.orchestra.run.smithi110.stdout: "req_mknod_latency": { 2024-09-21T03:42:32.657 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-09-21T03:42:32.657 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-09-21T03:42:32.657 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-09-21T03:42:32.657 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-21T03:42:32.657 INFO:teuthology.orchestra.run.smithi110.stdout: "req_mksnap_latency": { 2024-09-21T03:42:32.657 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-09-21T03:42:32.657 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-09-21T03:42:32.658 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-09-21T03:42:32.658 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-21T03:42:32.658 INFO:teuthology.orchestra.run.smithi110.stdout: "req_open_latency": { 2024-09-21T03:42:32.658 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-09-21T03:42:32.658 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-09-21T03:42:32.658 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-09-21T03:42:32.658 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-21T03:42:32.658 INFO:teuthology.orchestra.run.smithi110.stdout: "req_readdir_latency": { 2024-09-21T03:42:32.658 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-09-21T03:42:32.658 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-09-21T03:42:32.658 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-09-21T03:42:32.658 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-21T03:42:32.658 INFO:teuthology.orchestra.run.smithi110.stdout: "req_rename_latency": { 2024-09-21T03:42:32.658 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-09-21T03:42:32.658 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-09-21T03:42:32.659 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-09-21T03:42:32.659 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-21T03:42:32.659 INFO:teuthology.orchestra.run.smithi110.stdout: "req_renamesnap_latency": { 2024-09-21T03:42:32.659 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-09-21T03:42:32.659 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-09-21T03:42:32.659 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-09-21T03:42:32.659 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-21T03:42:32.659 INFO:teuthology.orchestra.run.smithi110.stdout: "req_snapdiff_latency": { 2024-09-21T03:42:32.659 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-09-21T03:42:32.659 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-09-21T03:42:32.659 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-09-21T03:42:32.659 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-21T03:42:32.659 INFO:teuthology.orchestra.run.smithi110.stdout: "req_rmdir_latency": { 2024-09-21T03:42:32.659 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-09-21T03:42:32.659 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-09-21T03:42:32.660 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-09-21T03:42:32.660 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-21T03:42:32.660 INFO:teuthology.orchestra.run.smithi110.stdout: "req_rmsnap_latency": { 2024-09-21T03:42:32.660 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-09-21T03:42:32.660 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-09-21T03:42:32.660 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-09-21T03:42:32.660 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-21T03:42:32.660 INFO:teuthology.orchestra.run.smithi110.stdout: "req_rmxattr_latency": { 2024-09-21T03:42:32.660 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-09-21T03:42:32.660 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-09-21T03:42:32.660 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-09-21T03:42:32.660 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-21T03:42:32.660 INFO:teuthology.orchestra.run.smithi110.stdout: "req_setattr_latency": { 2024-09-21T03:42:32.660 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-09-21T03:42:32.660 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-09-21T03:42:32.660 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-09-21T03:42:32.661 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-21T03:42:32.661 INFO:teuthology.orchestra.run.smithi110.stdout: "req_setdirlayout_latency": { 2024-09-21T03:42:32.661 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-09-21T03:42:32.661 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-09-21T03:42:32.661 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-09-21T03:42:32.661 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-21T03:42:32.661 INFO:teuthology.orchestra.run.smithi110.stdout: "req_setfilelock_latency": { 2024-09-21T03:42:32.661 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-09-21T03:42:32.661 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-09-21T03:42:32.661 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-09-21T03:42:32.661 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-21T03:42:32.661 INFO:teuthology.orchestra.run.smithi110.stdout: "req_setlayout_latency": { 2024-09-21T03:42:32.661 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-09-21T03:42:32.661 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-09-21T03:42:32.662 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-09-21T03:42:32.662 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-21T03:42:32.662 INFO:teuthology.orchestra.run.smithi110.stdout: "req_setxattr_latency": { 2024-09-21T03:42:32.662 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-09-21T03:42:32.662 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-09-21T03:42:32.662 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-09-21T03:42:32.662 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-21T03:42:32.662 INFO:teuthology.orchestra.run.smithi110.stdout: "req_symlink_latency": { 2024-09-21T03:42:32.662 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-09-21T03:42:32.662 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-09-21T03:42:32.662 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-09-21T03:42:32.662 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-21T03:42:32.662 INFO:teuthology.orchestra.run.smithi110.stdout: "req_unlink_latency": { 2024-09-21T03:42:32.662 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-09-21T03:42:32.662 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-09-21T03:42:32.663 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-09-21T03:42:32.663 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-21T03:42:32.663 INFO:teuthology.orchestra.run.smithi110.stdout: "cap_revoke_eviction": 0, 2024-09-21T03:42:32.663 INFO:teuthology.orchestra.run.smithi110.stdout: "cap_acquisition_throttle": 0, 2024-09-21T03:42:32.663 INFO:teuthology.orchestra.run.smithi110.stdout: "req_getvxattr_latency": { 2024-09-21T03:42:32.663 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-09-21T03:42:32.663 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-09-21T03:42:32.663 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-09-21T03:42:32.663 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-21T03:42:32.663 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-21T03:42:32.663 INFO:teuthology.orchestra.run.smithi110.stdout: "mds_sessions": { 2024-09-21T03:42:32.663 INFO:teuthology.orchestra.run.smithi110.stdout: "session_count": 1, 2024-09-21T03:42:32.663 INFO:teuthology.orchestra.run.smithi110.stdout: "session_add": 2, 2024-09-21T03:42:32.663 INFO:teuthology.orchestra.run.smithi110.stdout: "session_remove": 1, 2024-09-21T03:42:32.663 INFO:teuthology.orchestra.run.smithi110.stdout: "sessions_open": 2, 2024-09-21T03:42:32.664 INFO:teuthology.orchestra.run.smithi110.stdout: "sessions_stale": 0, 2024-09-21T03:42:32.664 INFO:teuthology.orchestra.run.smithi110.stdout: "total_load": 0, 2024-09-21T03:42:32.664 INFO:teuthology.orchestra.run.smithi110.stdout: "average_load": 0, 2024-09-21T03:42:32.664 INFO:teuthology.orchestra.run.smithi110.stdout: "avg_session_uptime": 2363, 2024-09-21T03:42:32.664 INFO:teuthology.orchestra.run.smithi110.stdout: "mdthresh_evicted": 0 2024-09-21T03:42:32.664 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-21T03:42:32.664 INFO:teuthology.orchestra.run.smithi110.stdout: "mempool": { 2024-09-21T03:42:32.664 INFO:teuthology.orchestra.run.smithi110.stdout: "bloom_filter_bytes": 0, 2024-09-21T03:42:32.664 INFO:teuthology.orchestra.run.smithi110.stdout: "bloom_filter_items": 0, 2024-09-21T03:42:32.664 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_alloc_bytes": 0, 2024-09-21T03:42:32.664 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_alloc_items": 0, 2024-09-21T03:42:32.664 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_cache_data_bytes": 0, 2024-09-21T03:42:32.664 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_cache_data_items": 0, 2024-09-21T03:42:32.664 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-21T03:42:32.665 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_cache_onode_items": 0, 2024-09-21T03:42:32.665 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-21T03:42:32.665 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_cache_meta_items": 0, 2024-09-21T03:42:32.665 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_cache_other_bytes": 0, 2024-09-21T03:42:32.665 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_cache_other_items": 0, 2024-09-21T03:42:32.665 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-21T03:42:32.665 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_cache_buffer_items": 0, 2024-09-21T03:42:32.665 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_extent_bytes": 0, 2024-09-21T03:42:32.665 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_extent_items": 0, 2024-09-21T03:42:32.665 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_blob_bytes": 0, 2024-09-21T03:42:32.665 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_blob_items": 0, 2024-09-21T03:42:32.665 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-21T03:42:32.665 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_shared_blob_items": 0, 2024-09-21T03:42:32.665 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-21T03:42:32.666 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_inline_bl_items": 0, 2024-09-21T03:42:32.666 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_fsck_bytes": 0, 2024-09-21T03:42:32.666 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_fsck_items": 0, 2024-09-21T03:42:32.666 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_txc_bytes": 0, 2024-09-21T03:42:32.666 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_txc_items": 0, 2024-09-21T03:42:32.666 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-21T03:42:32.666 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_writing_deferred_items": 0, 2024-09-21T03:42:32.666 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_writing_bytes": 0, 2024-09-21T03:42:32.666 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_writing_items": 0, 2024-09-21T03:42:32.666 INFO:teuthology.orchestra.run.smithi110.stdout: "bluefs_bytes": 0, 2024-09-21T03:42:32.666 INFO:teuthology.orchestra.run.smithi110.stdout: "bluefs_items": 0, 2024-09-21T03:42:32.666 INFO:teuthology.orchestra.run.smithi110.stdout: "bluefs_file_reader_bytes": 0, 2024-09-21T03:42:32.666 INFO:teuthology.orchestra.run.smithi110.stdout: "bluefs_file_reader_items": 0, 2024-09-21T03:42:32.666 INFO:teuthology.orchestra.run.smithi110.stdout: "bluefs_file_writer_bytes": 0, 2024-09-21T03:42:32.666 INFO:teuthology.orchestra.run.smithi110.stdout: "bluefs_file_writer_items": 0, 2024-09-21T03:42:32.667 INFO:teuthology.orchestra.run.smithi110.stdout: "buffer_anon_bytes": 189279, 2024-09-21T03:42:32.667 INFO:teuthology.orchestra.run.smithi110.stdout: "buffer_anon_items": 49, 2024-09-21T03:42:32.667 INFO:teuthology.orchestra.run.smithi110.stdout: "buffer_meta_bytes": 0, 2024-09-21T03:42:32.667 INFO:teuthology.orchestra.run.smithi110.stdout: "buffer_meta_items": 0, 2024-09-21T03:42:32.667 INFO:teuthology.orchestra.run.smithi110.stdout: "osd_bytes": 0, 2024-09-21T03:42:32.667 INFO:teuthology.orchestra.run.smithi110.stdout: "osd_items": 0, 2024-09-21T03:42:32.667 INFO:teuthology.orchestra.run.smithi110.stdout: "osd_mapbl_bytes": 0, 2024-09-21T03:42:32.667 INFO:teuthology.orchestra.run.smithi110.stdout: "osd_mapbl_items": 0, 2024-09-21T03:42:32.667 INFO:teuthology.orchestra.run.smithi110.stdout: "osd_pglog_bytes": 0, 2024-09-21T03:42:32.667 INFO:teuthology.orchestra.run.smithi110.stdout: "osd_pglog_items": 0, 2024-09-21T03:42:32.667 INFO:teuthology.orchestra.run.smithi110.stdout: "osdmap_bytes": 8760, 2024-09-21T03:42:32.667 INFO:teuthology.orchestra.run.smithi110.stdout: "osdmap_items": 164, 2024-09-21T03:42:32.667 INFO:teuthology.orchestra.run.smithi110.stdout: "osdmap_mapping_bytes": 0, 2024-09-21T03:42:32.667 INFO:teuthology.orchestra.run.smithi110.stdout: "osdmap_mapping_items": 0, 2024-09-21T03:42:32.668 INFO:teuthology.orchestra.run.smithi110.stdout: "pgmap_bytes": 0, 2024-09-21T03:42:32.668 INFO:teuthology.orchestra.run.smithi110.stdout: "pgmap_items": 0, 2024-09-21T03:42:32.668 INFO:teuthology.orchestra.run.smithi110.stdout: "mds_co_bytes": 8672, 2024-09-21T03:42:32.668 INFO:teuthology.orchestra.run.smithi110.stdout: "mds_co_items": 13, 2024-09-21T03:42:32.668 INFO:teuthology.orchestra.run.smithi110.stdout: "unittest_1_bytes": 0, 2024-09-21T03:42:32.668 INFO:teuthology.orchestra.run.smithi110.stdout: "unittest_1_items": 0, 2024-09-21T03:42:32.668 INFO:teuthology.orchestra.run.smithi110.stdout: "unittest_2_bytes": 0, 2024-09-21T03:42:32.668 INFO:teuthology.orchestra.run.smithi110.stdout: "unittest_2_items": 0 2024-09-21T03:42:32.668 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-21T03:42:32.668 INFO:teuthology.orchestra.run.smithi110.stdout: "objecter": { 2024-09-21T03:42:32.668 INFO:teuthology.orchestra.run.smithi110.stdout: "op_active": 0, 2024-09-21T03:42:32.668 INFO:teuthology.orchestra.run.smithi110.stdout: "op_laggy": 0, 2024-09-21T03:42:32.668 INFO:teuthology.orchestra.run.smithi110.stdout: "op_send": 763, 2024-09-21T03:42:32.668 INFO:teuthology.orchestra.run.smithi110.stdout: "op_send_bytes": 21, 2024-09-21T03:42:32.668 INFO:teuthology.orchestra.run.smithi110.stdout: "op_resend": 3, 2024-09-21T03:42:32.669 INFO:teuthology.orchestra.run.smithi110.stdout: "op_reply": 763, 2024-09-21T03:42:32.669 INFO:teuthology.orchestra.run.smithi110.stdout: "op_latency": { 2024-09-21T03:42:32.669 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 763, 2024-09-21T03:42:32.669 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 1.239978172, 2024-09-21T03:42:32.669 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.001625135 2024-09-21T03:42:32.669 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-21T03:42:32.669 INFO:teuthology.orchestra.run.smithi110.stdout: "op_inflight": 0, 2024-09-21T03:42:32.669 INFO:teuthology.orchestra.run.smithi110.stdout: "oplen_avg": { 2024-09-21T03:42:32.669 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 763, 2024-09-21T03:42:32.669 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 764 2024-09-21T03:42:32.669 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-21T03:42:32.669 INFO:teuthology.orchestra.run.smithi110.stdout: "op": 763, 2024-09-21T03:42:32.669 INFO:teuthology.orchestra.run.smithi110.stdout: "op_r": 763, 2024-09-21T03:42:32.669 INFO:teuthology.orchestra.run.smithi110.stdout: "op_w": 0, 2024-09-21T03:42:32.670 INFO:teuthology.orchestra.run.smithi110.stdout: "op_rmw": 0, 2024-09-21T03:42:32.670 INFO:teuthology.orchestra.run.smithi110.stdout: "op_pg": 0, 2024-09-21T03:42:32.670 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_stat": 471, 2024-09-21T03:42:32.670 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_create": 0, 2024-09-21T03:42:32.670 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_read": 290, 2024-09-21T03:42:32.670 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_write": 0, 2024-09-21T03:42:32.670 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_writefull": 0, 2024-09-21T03:42:32.670 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_writesame": 0, 2024-09-21T03:42:32.670 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_append": 0, 2024-09-21T03:42:32.670 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_zero": 0, 2024-09-21T03:42:32.670 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_truncate": 0, 2024-09-21T03:42:32.670 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_delete": 0, 2024-09-21T03:42:32.670 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_mapext": 0, 2024-09-21T03:42:32.670 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_sparse_read": 0, 2024-09-21T03:42:32.671 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_clonerange": 0, 2024-09-21T03:42:32.671 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_getxattr": 1, 2024-09-21T03:42:32.671 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_setxattr": 0, 2024-09-21T03:42:32.671 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_cmpxattr": 0, 2024-09-21T03:42:32.671 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_rmxattr": 0, 2024-09-21T03:42:32.671 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_resetxattrs": 0, 2024-09-21T03:42:32.671 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_call": 0, 2024-09-21T03:42:32.671 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_watch": 0, 2024-09-21T03:42:32.671 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_notify": 0, 2024-09-21T03:42:32.671 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_src_cmpxattr": 0, 2024-09-21T03:42:32.671 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_pgls": 0, 2024-09-21T03:42:32.671 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_pgls_filter": 0, 2024-09-21T03:42:32.671 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_other": 0, 2024-09-21T03:42:32.671 INFO:teuthology.orchestra.run.smithi110.stdout: "linger_active": 0, 2024-09-21T03:42:32.671 INFO:teuthology.orchestra.run.smithi110.stdout: "linger_send": 0, 2024-09-21T03:42:32.672 INFO:teuthology.orchestra.run.smithi110.stdout: "linger_resend": 0, 2024-09-21T03:42:32.672 INFO:teuthology.orchestra.run.smithi110.stdout: "linger_ping": 0, 2024-09-21T03:42:32.672 INFO:teuthology.orchestra.run.smithi110.stdout: "poolop_active": 0, 2024-09-21T03:42:32.672 INFO:teuthology.orchestra.run.smithi110.stdout: "poolop_send": 0, 2024-09-21T03:42:32.672 INFO:teuthology.orchestra.run.smithi110.stdout: "poolop_resend": 0, 2024-09-21T03:42:32.672 INFO:teuthology.orchestra.run.smithi110.stdout: "poolstat_active": 0, 2024-09-21T03:42:32.672 INFO:teuthology.orchestra.run.smithi110.stdout: "poolstat_send": 0, 2024-09-21T03:42:32.672 INFO:teuthology.orchestra.run.smithi110.stdout: "poolstat_resend": 0, 2024-09-21T03:42:32.672 INFO:teuthology.orchestra.run.smithi110.stdout: "statfs_active": 0, 2024-09-21T03:42:32.672 INFO:teuthology.orchestra.run.smithi110.stdout: "statfs_send": 0, 2024-09-21T03:42:32.672 INFO:teuthology.orchestra.run.smithi110.stdout: "statfs_resend": 0, 2024-09-21T03:42:32.672 INFO:teuthology.orchestra.run.smithi110.stdout: "command_active": 0, 2024-09-21T03:42:32.672 INFO:teuthology.orchestra.run.smithi110.stdout: "command_send": 0, 2024-09-21T03:42:32.672 INFO:teuthology.orchestra.run.smithi110.stdout: "command_resend": 0, 2024-09-21T03:42:32.673 INFO:teuthology.orchestra.run.smithi110.stdout: "map_epoch": 84, 2024-09-21T03:42:32.673 INFO:teuthology.orchestra.run.smithi110.stdout: "map_full": 0, 2024-09-21T03:42:32.673 INFO:teuthology.orchestra.run.smithi110.stdout: "map_inc": 4, 2024-09-21T03:42:32.673 INFO:teuthology.orchestra.run.smithi110.stdout: "osd_sessions": 5, 2024-09-21T03:42:32.673 INFO:teuthology.orchestra.run.smithi110.stdout: "osd_session_open": 5, 2024-09-21T03:42:32.673 INFO:teuthology.orchestra.run.smithi110.stdout: "osd_session_close": 0, 2024-09-21T03:42:32.673 INFO:teuthology.orchestra.run.smithi110.stdout: "osd_laggy": 0, 2024-09-21T03:42:32.673 INFO:teuthology.orchestra.run.smithi110.stdout: "omap_wr": 0, 2024-09-21T03:42:32.673 INFO:teuthology.orchestra.run.smithi110.stdout: "omap_rd": 2, 2024-09-21T03:42:32.673 INFO:teuthology.orchestra.run.smithi110.stdout: "omap_del": 0 2024-09-21T03:42:32.673 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-21T03:42:32.673 INFO:teuthology.orchestra.run.smithi110.stdout: "oft": { 2024-09-21T03:42:32.673 INFO:teuthology.orchestra.run.smithi110.stdout: "omap_total_objs": 0, 2024-09-21T03:42:32.673 INFO:teuthology.orchestra.run.smithi110.stdout: "omap_total_kv_pairs": 0, 2024-09-21T03:42:32.673 INFO:teuthology.orchestra.run.smithi110.stdout: "omap_total_updates": 0, 2024-09-21T03:42:32.673 INFO:teuthology.orchestra.run.smithi110.stdout: "omap_total_removes": 0 2024-09-21T03:42:32.674 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-21T03:42:32.674 INFO:teuthology.orchestra.run.smithi110.stdout: "purge_queue": { 2024-09-21T03:42:32.674 INFO:teuthology.orchestra.run.smithi110.stdout: "pq_executing_ops": 0, 2024-09-21T03:42:32.674 INFO:teuthology.orchestra.run.smithi110.stdout: "pq_executing_ops_high_water": 0, 2024-09-21T03:42:32.674 INFO:teuthology.orchestra.run.smithi110.stdout: "pq_executing": 0, 2024-09-21T03:42:32.674 INFO:teuthology.orchestra.run.smithi110.stdout: "pq_executing_high_water": 0, 2024-09-21T03:42:32.674 INFO:teuthology.orchestra.run.smithi110.stdout: "pq_executed_ops": 0, 2024-09-21T03:42:32.674 INFO:teuthology.orchestra.run.smithi110.stdout: "pq_executed": 0, 2024-09-21T03:42:32.674 INFO:teuthology.orchestra.run.smithi110.stdout: "pq_item_in_journal": 0 2024-09-21T03:42:32.674 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-21T03:42:32.674 INFO:teuthology.orchestra.run.smithi110.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-21T03:42:32.674 INFO:teuthology.orchestra.run.smithi110.stdout: "val": 0, 2024-09-21T03:42:32.674 INFO:teuthology.orchestra.run.smithi110.stdout: "max": 104857600, 2024-09-21T03:42:32.675 INFO:teuthology.orchestra.run.smithi110.stdout: "get_started": 0, 2024-09-21T03:42:32.675 INFO:teuthology.orchestra.run.smithi110.stdout: "get": 877, 2024-09-21T03:42:32.675 INFO:teuthology.orchestra.run.smithi110.stdout: "get_sum": 963158, 2024-09-21T03:42:32.675 INFO:teuthology.orchestra.run.smithi110.stdout: "get_or_fail_fail": 0, 2024-09-21T03:42:32.675 INFO:teuthology.orchestra.run.smithi110.stdout: "get_or_fail_success": 877, 2024-09-21T03:42:32.675 INFO:teuthology.orchestra.run.smithi110.stdout: "take": 0, 2024-09-21T03:42:32.675 INFO:teuthology.orchestra.run.smithi110.stdout: "take_sum": 0, 2024-09-21T03:42:32.675 INFO:teuthology.orchestra.run.smithi110.stdout: "put": 877, 2024-09-21T03:42:32.675 INFO:teuthology.orchestra.run.smithi110.stdout: "put_sum": 963158, 2024-09-21T03:42:32.675 INFO:teuthology.orchestra.run.smithi110.stdout: "wait": { 2024-09-21T03:42:32.675 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-09-21T03:42:32.675 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-09-21T03:42:32.675 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-09-21T03:42:32.675 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-21T03:42:32.675 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-21T03:42:32.676 INFO:teuthology.orchestra.run.smithi110.stdout: "throttle-objecter_bytes": { 2024-09-21T03:42:32.676 INFO:teuthology.orchestra.run.smithi110.stdout: "val": 0, 2024-09-21T03:42:32.676 INFO:teuthology.orchestra.run.smithi110.stdout: "max": 104857600, 2024-09-21T03:42:32.676 INFO:teuthology.orchestra.run.smithi110.stdout: "get_started": 0, 2024-09-21T03:42:32.676 INFO:teuthology.orchestra.run.smithi110.stdout: "get": 0, 2024-09-21T03:42:32.676 INFO:teuthology.orchestra.run.smithi110.stdout: "get_sum": 0, 2024-09-21T03:42:32.676 INFO:teuthology.orchestra.run.smithi110.stdout: "get_or_fail_fail": 0, 2024-09-21T03:42:32.676 INFO:teuthology.orchestra.run.smithi110.stdout: "get_or_fail_success": 0, 2024-09-21T03:42:32.676 INFO:teuthology.orchestra.run.smithi110.stdout: "take": 763, 2024-09-21T03:42:32.676 INFO:teuthology.orchestra.run.smithi110.stdout: "take_sum": 1618, 2024-09-21T03:42:32.676 INFO:teuthology.orchestra.run.smithi110.stdout: "put": 763, 2024-09-21T03:42:32.676 INFO:teuthology.orchestra.run.smithi110.stdout: "put_sum": 1618, 2024-09-21T03:42:32.676 INFO:teuthology.orchestra.run.smithi110.stdout: "wait": { 2024-09-21T03:42:32.676 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-09-21T03:42:32.676 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-09-21T03:42:32.677 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-09-21T03:42:32.677 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-21T03:42:32.677 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-21T03:42:32.677 INFO:teuthology.orchestra.run.smithi110.stdout: "throttle-objecter_ops": { 2024-09-21T03:42:32.677 INFO:teuthology.orchestra.run.smithi110.stdout: "val": 0, 2024-09-21T03:42:32.677 INFO:teuthology.orchestra.run.smithi110.stdout: "max": 1024, 2024-09-21T03:42:32.677 INFO:teuthology.orchestra.run.smithi110.stdout: "get_started": 0, 2024-09-21T03:42:32.677 INFO:teuthology.orchestra.run.smithi110.stdout: "get": 0, 2024-09-21T03:42:32.677 INFO:teuthology.orchestra.run.smithi110.stdout: "get_sum": 0, 2024-09-21T03:42:32.677 INFO:teuthology.orchestra.run.smithi110.stdout: "get_or_fail_fail": 0, 2024-09-21T03:42:32.677 INFO:teuthology.orchestra.run.smithi110.stdout: "get_or_fail_success": 0, 2024-09-21T03:42:32.677 INFO:teuthology.orchestra.run.smithi110.stdout: "take": 763, 2024-09-21T03:42:32.677 INFO:teuthology.orchestra.run.smithi110.stdout: "take_sum": 763, 2024-09-21T03:42:32.677 INFO:teuthology.orchestra.run.smithi110.stdout: "put": 763, 2024-09-21T03:42:32.677 INFO:teuthology.orchestra.run.smithi110.stdout: "put_sum": 763, 2024-09-21T03:42:32.678 INFO:teuthology.orchestra.run.smithi110.stdout: "wait": { 2024-09-21T03:42:32.678 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-09-21T03:42:32.678 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-09-21T03:42:32.678 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-09-21T03:42:32.678 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-21T03:42:32.678 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-21T03:42:32.678 INFO:teuthology.orchestra.run.smithi110.stdout: "throttle-write_buf_throttle": { 2024-09-21T03:42:32.678 INFO:teuthology.orchestra.run.smithi110.stdout: "val": 0, 2024-09-21T03:42:32.678 INFO:teuthology.orchestra.run.smithi110.stdout: "max": 3758096384, 2024-09-21T03:42:32.678 INFO:teuthology.orchestra.run.smithi110.stdout: "get_started": 0, 2024-09-21T03:42:32.678 INFO:teuthology.orchestra.run.smithi110.stdout: "get": 0, 2024-09-21T03:42:32.678 INFO:teuthology.orchestra.run.smithi110.stdout: "get_sum": 0, 2024-09-21T03:42:32.678 INFO:teuthology.orchestra.run.smithi110.stdout: "get_or_fail_fail": 0, 2024-09-21T03:42:32.679 INFO:teuthology.orchestra.run.smithi110.stdout: "get_or_fail_success": 0, 2024-09-21T03:42:32.679 INFO:teuthology.orchestra.run.smithi110.stdout: "take": 0, 2024-09-21T03:42:32.679 INFO:teuthology.orchestra.run.smithi110.stdout: "take_sum": 0, 2024-09-21T03:42:32.679 INFO:teuthology.orchestra.run.smithi110.stdout: "put": 0, 2024-09-21T03:42:32.679 INFO:teuthology.orchestra.run.smithi110.stdout: "put_sum": 0, 2024-09-21T03:42:32.679 INFO:teuthology.orchestra.run.smithi110.stdout: "wait": { 2024-09-21T03:42:32.679 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-09-21T03:42:32.679 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-09-21T03:42:32.679 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-09-21T03:42:32.679 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-21T03:42:32.679 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-21T03:42:32.679 INFO:teuthology.orchestra.run.smithi110.stdout: "throttle-write_buf_throttle-0x55de27567450": { 2024-09-21T03:42:32.679 INFO:teuthology.orchestra.run.smithi110.stdout: "val": 0, 2024-09-21T03:42:32.679 INFO:teuthology.orchestra.run.smithi110.stdout: "max": 3758096384, 2024-09-21T03:42:32.680 INFO:teuthology.orchestra.run.smithi110.stdout: "get_started": 0, 2024-09-21T03:42:32.680 INFO:teuthology.orchestra.run.smithi110.stdout: "get": 0, 2024-09-21T03:42:32.680 INFO:teuthology.orchestra.run.smithi110.stdout: "get_sum": 0, 2024-09-21T03:42:32.680 INFO:teuthology.orchestra.run.smithi110.stdout: "get_or_fail_fail": 0, 2024-09-21T03:42:32.680 INFO:teuthology.orchestra.run.smithi110.stdout: "get_or_fail_success": 0, 2024-09-21T03:42:32.680 INFO:teuthology.orchestra.run.smithi110.stdout: "take": 0, 2024-09-21T03:42:32.680 INFO:teuthology.orchestra.run.smithi110.stdout: "take_sum": 0, 2024-09-21T03:42:32.680 INFO:teuthology.orchestra.run.smithi110.stdout: "put": 0, 2024-09-21T03:42:32.680 INFO:teuthology.orchestra.run.smithi110.stdout: "put_sum": 0, 2024-09-21T03:42:32.680 INFO:teuthology.orchestra.run.smithi110.stdout: "wait": { 2024-09-21T03:42:32.680 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-09-21T03:42:32.680 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-09-21T03:42:32.680 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-09-21T03:42:32.680 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-21T03:42:32.680 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-21T03:42:32.681 INFO:teuthology.orchestra.run.smithi110.stdout: "trackedop": { 2024-09-21T03:42:32.681 INFO:teuthology.orchestra.run.smithi110.stdout: "slow_ops_count": 0 2024-09-21T03:42:32.681 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-21T03:42:32.681 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-21T03:42:33.211 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:42:32 smithi114 ceph-mon[34140]: pgmap v454: 129 pgs: 129 active+clean; 1.1 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 682 B/s wr, 7 op/s 2024-09-21T03:42:33.250 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:42:32 smithi110 ceph-mon[34000]: pgmap v454: 129 pgs: 129 active+clean; 1.1 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 682 B/s wr, 7 op/s 2024-09-21T03:42:33.311 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:42:32 smithi089 ceph-mon[30928]: pgmap v454: 129 pgs: 129 active+clean; 1.1 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 682 B/s wr, 7 op/s 2024-09-21T03:42:33.472 INFO:tasks.check_counter:Daemon mds.e mds.exported=0 2024-09-21T03:42:33.473 INFO:tasks.check_counter:Daemon mds.e mds.imported=0 2024-09-21T03:42:33.473 DEBUG:tasks.check_counter:Getting stats from h 2024-09-21T03:42:33.473 DEBUG:teuthology.orchestra.run.smithi110:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 7c374098-77c9-11ef-baf5-efdc52797490 -- ceph daemon mds.h perf dump 2024-09-21T03:42:33.717 INFO:teuthology.orchestra.run.smithi110.stderr:Inferring config /var/lib/ceph/7c374098-77c9-11ef-baf5-efdc52797490/mon.b/config 2024-09-21T03:42:34.235 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-21T03:42:34.235 INFO:teuthology.orchestra.run.smithi110.stdout: "AsyncMessenger::Worker-0": { 2024-09-21T03:42:34.235 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_recv_messages": 165, 2024-09-21T03:42:34.235 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_send_messages": 156, 2024-09-21T03:42:34.235 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_recv_bytes": 842266, 2024-09-21T03:42:34.235 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_send_bytes": 121622, 2024-09-21T03:42:34.235 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_created_connections": 12, 2024-09-21T03:42:34.235 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_active_connections": 3, 2024-09-21T03:42:34.235 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_running_total_time": 0.030713714, 2024-09-21T03:42:34.235 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_running_send_time": 0.010555432, 2024-09-21T03:42:34.235 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_running_recv_time": 0.029065037, 2024-09-21T03:42:34.236 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_running_fast_dispatch_time": 0.000285286, 2024-09-21T03:42:34.236 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T03:42:34.236 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 155, 2024-09-21T03:42:34.236 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.015926521, 2024-09-21T03:42:34.236 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000102751 2024-09-21T03:42:34.236 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-21T03:42:34.236 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_handle_ack_lat": { 2024-09-21T03:42:34.236 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 12, 2024-09-21T03:42:34.236 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000002700, 2024-09-21T03:42:34.236 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000225 2024-09-21T03:42:34.236 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-21T03:42:34.236 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_recv_encrypted_bytes": 837136, 2024-09-21T03:42:34.236 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_send_encrypted_bytes": 63648 2024-09-21T03:42:34.236 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-21T03:42:34.236 INFO:teuthology.orchestra.run.smithi110.stdout: "AsyncMessenger::Worker-1": { 2024-09-21T03:42:34.237 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_recv_messages": 32, 2024-09-21T03:42:34.237 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_send_messages": 101, 2024-09-21T03:42:34.237 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_recv_bytes": 10870, 2024-09-21T03:42:34.237 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_send_bytes": 214473, 2024-09-21T03:42:34.237 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_created_connections": 7, 2024-09-21T03:42:34.237 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_active_connections": 4, 2024-09-21T03:42:34.237 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_running_total_time": 0.012073549, 2024-09-21T03:42:34.237 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_running_send_time": 0.005483803, 2024-09-21T03:42:34.237 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_running_recv_time": 0.005477907, 2024-09-21T03:42:34.237 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_running_fast_dispatch_time": 0.000461754, 2024-09-21T03:42:34.237 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T03:42:34.237 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 100, 2024-09-21T03:42:34.237 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.015687796, 2024-09-21T03:42:34.237 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000156877 2024-09-21T03:42:34.237 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-21T03:42:34.237 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_handle_ack_lat": { 2024-09-21T03:42:34.238 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 21, 2024-09-21T03:42:34.238 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000003974, 2024-09-21T03:42:34.238 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000189 2024-09-21T03:42:34.238 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-21T03:42:34.238 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_recv_encrypted_bytes": 6496, 2024-09-21T03:42:34.238 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_send_encrypted_bytes": 201424 2024-09-21T03:42:34.238 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-21T03:42:34.238 INFO:teuthology.orchestra.run.smithi110.stdout: "AsyncMessenger::Worker-2": { 2024-09-21T03:42:34.238 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_recv_messages": 167, 2024-09-21T03:42:34.238 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_send_messages": 220, 2024-09-21T03:42:34.238 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_recv_bytes": 44830, 2024-09-21T03:42:34.238 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_send_bytes": 113714, 2024-09-21T03:42:34.238 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_created_connections": 8, 2024-09-21T03:42:34.239 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_active_connections": 4, 2024-09-21T03:42:34.239 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_running_total_time": 0.029037184, 2024-09-21T03:42:34.239 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_running_send_time": 0.014388047, 2024-09-21T03:42:34.239 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_running_recv_time": 0.012354686, 2024-09-21T03:42:34.239 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_running_fast_dispatch_time": 0.001696572, 2024-09-21T03:42:34.239 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T03:42:34.239 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 218, 2024-09-21T03:42:34.239 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.019889424, 2024-09-21T03:42:34.239 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000091235 2024-09-21T03:42:34.239 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-21T03:42:34.239 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_handle_ack_lat": { 2024-09-21T03:42:34.239 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 350, 2024-09-21T03:42:34.239 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000097334, 2024-09-21T03:42:34.239 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000278 2024-09-21T03:42:34.239 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-21T03:42:34.239 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-21T03:42:34.240 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_send_encrypted_bytes": 0 2024-09-21T03:42:34.240 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-21T03:42:34.240 INFO:teuthology.orchestra.run.smithi110.stdout: "cct": { 2024-09-21T03:42:34.240 INFO:teuthology.orchestra.run.smithi110.stdout: "total_workers": 1, 2024-09-21T03:42:34.240 INFO:teuthology.orchestra.run.smithi110.stdout: "unhealthy_workers": 0 2024-09-21T03:42:34.240 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-21T03:42:34.240 INFO:teuthology.orchestra.run.smithi110.stdout: "finisher-MDSRank": { 2024-09-21T03:42:34.240 INFO:teuthology.orchestra.run.smithi110.stdout: "queue_len": 0, 2024-09-21T03:42:34.240 INFO:teuthology.orchestra.run.smithi110.stdout: "complete_latency": { 2024-09-21T03:42:34.240 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 47, 2024-09-21T03:42:34.240 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.001381698, 2024-09-21T03:42:34.240 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000029397 2024-09-21T03:42:34.240 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-21T03:42:34.240 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-21T03:42:34.240 INFO:teuthology.orchestra.run.smithi110.stdout: "finisher-PurgeQueue": { 2024-09-21T03:42:34.240 INFO:teuthology.orchestra.run.smithi110.stdout: "queue_len": 0, 2024-09-21T03:42:34.241 INFO:teuthology.orchestra.run.smithi110.stdout: "complete_latency": { 2024-09-21T03:42:34.241 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 2, 2024-09-21T03:42:34.241 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000021027, 2024-09-21T03:42:34.241 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000010513 2024-09-21T03:42:34.241 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-21T03:42:34.241 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-21T03:42:34.241 INFO:teuthology.orchestra.run.smithi110.stdout: "mds": { 2024-09-21T03:42:34.241 INFO:teuthology.orchestra.run.smithi110.stdout: "request": 4, 2024-09-21T03:42:34.241 INFO:teuthology.orchestra.run.smithi110.stdout: "reply": 3, 2024-09-21T03:42:34.241 INFO:teuthology.orchestra.run.smithi110.stdout: "reply_latency": { 2024-09-21T03:42:34.241 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 3, 2024-09-21T03:42:34.241 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.001379784, 2024-09-21T03:42:34.241 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000459928 2024-09-21T03:42:34.241 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-21T03:42:34.241 INFO:teuthology.orchestra.run.smithi110.stdout: "slow_reply": 0, 2024-09-21T03:42:34.241 INFO:teuthology.orchestra.run.smithi110.stdout: "forward": 1, 2024-09-21T03:42:34.241 INFO:teuthology.orchestra.run.smithi110.stdout: "dir_fetch_complete": 0, 2024-09-21T03:42:34.242 INFO:teuthology.orchestra.run.smithi110.stdout: "dir_fetch_keys": 0, 2024-09-21T03:42:34.242 INFO:teuthology.orchestra.run.smithi110.stdout: "dir_commit": 11, 2024-09-21T03:42:34.242 INFO:teuthology.orchestra.run.smithi110.stdout: "dir_split": 0, 2024-09-21T03:42:34.242 INFO:teuthology.orchestra.run.smithi110.stdout: "dir_merge": 0, 2024-09-21T03:42:34.242 INFO:teuthology.orchestra.run.smithi110.stdout: "inodes": 12, 2024-09-21T03:42:34.242 INFO:teuthology.orchestra.run.smithi110.stdout: "inodes_top": 0, 2024-09-21T03:42:34.242 INFO:teuthology.orchestra.run.smithi110.stdout: "inodes_bottom": 12, 2024-09-21T03:42:34.242 INFO:teuthology.orchestra.run.smithi110.stdout: "inodes_pin_tail": 0, 2024-09-21T03:42:34.242 INFO:teuthology.orchestra.run.smithi110.stdout: "inodes_pinned": 12, 2024-09-21T03:42:34.242 INFO:teuthology.orchestra.run.smithi110.stdout: "inodes_expired": 0, 2024-09-21T03:42:34.242 INFO:teuthology.orchestra.run.smithi110.stdout: "inodes_with_caps": 0, 2024-09-21T03:42:34.242 INFO:teuthology.orchestra.run.smithi110.stdout: "caps": 0, 2024-09-21T03:42:34.242 INFO:teuthology.orchestra.run.smithi110.stdout: "subtrees": 2, 2024-09-21T03:42:34.242 INFO:teuthology.orchestra.run.smithi110.stdout: "traverse": 16, 2024-09-21T03:42:34.242 INFO:teuthology.orchestra.run.smithi110.stdout: "traverse_hit": 14, 2024-09-21T03:42:34.242 INFO:teuthology.orchestra.run.smithi110.stdout: "traverse_forward": 0, 2024-09-21T03:42:34.243 INFO:teuthology.orchestra.run.smithi110.stdout: "traverse_discover": 2, 2024-09-21T03:42:34.243 INFO:teuthology.orchestra.run.smithi110.stdout: "traverse_dir_fetch": 0, 2024-09-21T03:42:34.243 INFO:teuthology.orchestra.run.smithi110.stdout: "traverse_remote_ino": 0, 2024-09-21T03:42:34.243 INFO:teuthology.orchestra.run.smithi110.stdout: "traverse_lock": 0, 2024-09-21T03:42:34.243 INFO:teuthology.orchestra.run.smithi110.stdout: "load_cent": 0, 2024-09-21T03:42:34.243 INFO:teuthology.orchestra.run.smithi110.stdout: "q": 0, 2024-09-21T03:42:34.243 INFO:teuthology.orchestra.run.smithi110.stdout: "exported": 0, 2024-09-21T03:42:34.243 INFO:teuthology.orchestra.run.smithi110.stdout: "exported_inodes": 0, 2024-09-21T03:42:34.243 INFO:teuthology.orchestra.run.smithi110.stdout: "imported": 0, 2024-09-21T03:42:34.243 INFO:teuthology.orchestra.run.smithi110.stdout: "imported_inodes": 0, 2024-09-21T03:42:34.243 INFO:teuthology.orchestra.run.smithi110.stdout: "openino_dir_fetch": 0, 2024-09-21T03:42:34.243 INFO:teuthology.orchestra.run.smithi110.stdout: "openino_backtrace_fetch": 0, 2024-09-21T03:42:34.243 INFO:teuthology.orchestra.run.smithi110.stdout: "openino_peer_discover": 0, 2024-09-21T03:42:34.243 INFO:teuthology.orchestra.run.smithi110.stdout: "root_rfiles": 8, 2024-09-21T03:42:34.243 INFO:teuthology.orchestra.run.smithi110.stdout: "root_rbytes": 202106210, 2024-09-21T03:42:34.243 INFO:teuthology.orchestra.run.smithi110.stdout: "root_rsnaps": 0, 2024-09-21T03:42:34.243 INFO:teuthology.orchestra.run.smithi110.stdout: "scrub_backtrace_fetch": 0, 2024-09-21T03:42:34.244 INFO:teuthology.orchestra.run.smithi110.stdout: "scrub_set_tag": 0, 2024-09-21T03:42:34.244 INFO:teuthology.orchestra.run.smithi110.stdout: "scrub_backtrace_repaired": 0, 2024-09-21T03:42:34.244 INFO:teuthology.orchestra.run.smithi110.stdout: "scrub_inotable_repaired": 0, 2024-09-21T03:42:34.244 INFO:teuthology.orchestra.run.smithi110.stdout: "scrub_dir_inodes": 0, 2024-09-21T03:42:34.244 INFO:teuthology.orchestra.run.smithi110.stdout: "scrub_dir_base_inodes": 0, 2024-09-21T03:42:34.244 INFO:teuthology.orchestra.run.smithi110.stdout: "scrub_dirfrag_rstats": 0, 2024-09-21T03:42:34.244 INFO:teuthology.orchestra.run.smithi110.stdout: "scrub_file_inodes": 0, 2024-09-21T03:42:34.244 INFO:teuthology.orchestra.run.smithi110.stdout: "handle_inode_file_caps": 0, 2024-09-21T03:42:34.244 INFO:teuthology.orchestra.run.smithi110.stdout: "ceph_cap_op_revoke": 3, 2024-09-21T03:42:34.244 INFO:teuthology.orchestra.run.smithi110.stdout: "ceph_cap_op_grant": 0, 2024-09-21T03:42:34.244 INFO:teuthology.orchestra.run.smithi110.stdout: "ceph_cap_op_trunc": 0, 2024-09-21T03:42:34.244 INFO:teuthology.orchestra.run.smithi110.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-21T03:42:34.244 INFO:teuthology.orchestra.run.smithi110.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-21T03:42:34.244 INFO:teuthology.orchestra.run.smithi110.stdout: "handle_client_caps": 3, 2024-09-21T03:42:34.244 INFO:teuthology.orchestra.run.smithi110.stdout: "handle_client_caps_dirty": 0, 2024-09-21T03:42:34.244 INFO:teuthology.orchestra.run.smithi110.stdout: "handle_client_cap_release": 0, 2024-09-21T03:42:34.244 INFO:teuthology.orchestra.run.smithi110.stdout: "process_request_cap_release": 0 2024-09-21T03:42:34.245 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-21T03:42:34.245 INFO:teuthology.orchestra.run.smithi110.stdout: "mds_cache": { 2024-09-21T03:42:34.245 INFO:teuthology.orchestra.run.smithi110.stdout: "dir_update": 0, 2024-09-21T03:42:34.245 INFO:teuthology.orchestra.run.smithi110.stdout: "dir_update_receipt": 0, 2024-09-21T03:42:34.245 INFO:teuthology.orchestra.run.smithi110.stdout: "dir_try_discover": 0, 2024-09-21T03:42:34.245 INFO:teuthology.orchestra.run.smithi110.stdout: "dir_send_discover": 3, 2024-09-21T03:42:34.245 INFO:teuthology.orchestra.run.smithi110.stdout: "dir_handle_discover": 0, 2024-09-21T03:42:34.245 INFO:teuthology.orchestra.run.smithi110.stdout: "num_strays": 0, 2024-09-21T03:42:34.245 INFO:teuthology.orchestra.run.smithi110.stdout: "num_strays_delayed": 0, 2024-09-21T03:42:34.245 INFO:teuthology.orchestra.run.smithi110.stdout: "num_strays_enqueuing": 0, 2024-09-21T03:42:34.245 INFO:teuthology.orchestra.run.smithi110.stdout: "strays_created": 0, 2024-09-21T03:42:34.245 INFO:teuthology.orchestra.run.smithi110.stdout: "strays_enqueued": 0, 2024-09-21T03:42:34.245 INFO:teuthology.orchestra.run.smithi110.stdout: "strays_reintegrated": 0, 2024-09-21T03:42:34.245 INFO:teuthology.orchestra.run.smithi110.stdout: "strays_migrated": 0, 2024-09-21T03:42:34.245 INFO:teuthology.orchestra.run.smithi110.stdout: "num_recovering_processing": 0, 2024-09-21T03:42:34.245 INFO:teuthology.orchestra.run.smithi110.stdout: "num_recovering_enqueued": 0, 2024-09-21T03:42:34.245 INFO:teuthology.orchestra.run.smithi110.stdout: "num_recovering_prioritized": 0, 2024-09-21T03:42:34.246 INFO:teuthology.orchestra.run.smithi110.stdout: "recovery_started": 0, 2024-09-21T03:42:34.246 INFO:teuthology.orchestra.run.smithi110.stdout: "recovery_completed": 0, 2024-09-21T03:42:34.246 INFO:teuthology.orchestra.run.smithi110.stdout: "ireq_quiesce_path": 3, 2024-09-21T03:42:34.246 INFO:teuthology.orchestra.run.smithi110.stdout: "ireq_quiesce_inode": 9, 2024-09-21T03:42:34.246 INFO:teuthology.orchestra.run.smithi110.stdout: "ireq_enqueue_scrub": 0, 2024-09-21T03:42:34.246 INFO:teuthology.orchestra.run.smithi110.stdout: "ireq_exportdir": 0, 2024-09-21T03:42:34.246 INFO:teuthology.orchestra.run.smithi110.stdout: "ireq_flush": 0, 2024-09-21T03:42:34.246 INFO:teuthology.orchestra.run.smithi110.stdout: "ireq_fragmentdir": 0, 2024-09-21T03:42:34.246 INFO:teuthology.orchestra.run.smithi110.stdout: "ireq_fragstats": 0, 2024-09-21T03:42:34.246 INFO:teuthology.orchestra.run.smithi110.stdout: "ireq_inodestats": 0, 2024-09-21T03:42:34.246 INFO:teuthology.orchestra.run.smithi110.stdout: "uninline_started": 0, 2024-09-21T03:42:34.246 INFO:teuthology.orchestra.run.smithi110.stdout: "uninline_succeeded": 0, 2024-09-21T03:42:34.246 INFO:teuthology.orchestra.run.smithi110.stdout: "uninline_write_failed": 0 2024-09-21T03:42:34.246 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-21T03:42:34.246 INFO:teuthology.orchestra.run.smithi110.stdout: "mds_log": { 2024-09-21T03:42:34.246 INFO:teuthology.orchestra.run.smithi110.stdout: "evlrg": 0, 2024-09-21T03:42:34.247 INFO:teuthology.orchestra.run.smithi110.stdout: "evadd": 5, 2024-09-21T03:42:34.247 INFO:teuthology.orchestra.run.smithi110.stdout: "evex": 0, 2024-09-21T03:42:34.247 INFO:teuthology.orchestra.run.smithi110.stdout: "evtrm": 0, 2024-09-21T03:42:34.247 INFO:teuthology.orchestra.run.smithi110.stdout: "ev": 5, 2024-09-21T03:42:34.247 INFO:teuthology.orchestra.run.smithi110.stdout: "evexg": 0, 2024-09-21T03:42:34.247 INFO:teuthology.orchestra.run.smithi110.stdout: "evexd": 0, 2024-09-21T03:42:34.247 INFO:teuthology.orchestra.run.smithi110.stdout: "segadd": 2, 2024-09-21T03:42:34.247 INFO:teuthology.orchestra.run.smithi110.stdout: "segex": 0, 2024-09-21T03:42:34.247 INFO:teuthology.orchestra.run.smithi110.stdout: "segtrm": 0, 2024-09-21T03:42:34.247 INFO:teuthology.orchestra.run.smithi110.stdout: "seg": 2, 2024-09-21T03:42:34.247 INFO:teuthology.orchestra.run.smithi110.stdout: "segmjr": 2, 2024-09-21T03:42:34.247 INFO:teuthology.orchestra.run.smithi110.stdout: "segexg": 0, 2024-09-21T03:42:34.247 INFO:teuthology.orchestra.run.smithi110.stdout: "segexd": 0, 2024-09-21T03:42:34.247 INFO:teuthology.orchestra.run.smithi110.stdout: "expos": 4194304, 2024-09-21T03:42:34.247 INFO:teuthology.orchestra.run.smithi110.stdout: "wrpos": 4195917, 2024-09-21T03:42:34.247 INFO:teuthology.orchestra.run.smithi110.stdout: "rdpos": 0, 2024-09-21T03:42:34.247 INFO:teuthology.orchestra.run.smithi110.stdout: "jlat": { 2024-09-21T03:42:34.247 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 4, 2024-09-21T03:42:34.248 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.007382627, 2024-09-21T03:42:34.248 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.001845656 2024-09-21T03:42:34.248 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-21T03:42:34.248 INFO:teuthology.orchestra.run.smithi110.stdout: "replayed": 0 2024-09-21T03:42:34.248 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-21T03:42:34.248 INFO:teuthology.orchestra.run.smithi110.stdout: "mds_mem": { 2024-09-21T03:42:34.248 INFO:teuthology.orchestra.run.smithi110.stdout: "ino": 15, 2024-09-21T03:42:34.248 INFO:teuthology.orchestra.run.smithi110.stdout: "ino+": 15, 2024-09-21T03:42:34.248 INFO:teuthology.orchestra.run.smithi110.stdout: "ino-": 0, 2024-09-21T03:42:34.248 INFO:teuthology.orchestra.run.smithi110.stdout: "dir": 13, 2024-09-21T03:42:34.248 INFO:teuthology.orchestra.run.smithi110.stdout: "dir+": 14, 2024-09-21T03:42:34.248 INFO:teuthology.orchestra.run.smithi110.stdout: "dir-": 1, 2024-09-21T03:42:34.248 INFO:teuthology.orchestra.run.smithi110.stdout: "dn": 12, 2024-09-21T03:42:34.248 INFO:teuthology.orchestra.run.smithi110.stdout: "dn+": 12, 2024-09-21T03:42:34.248 INFO:teuthology.orchestra.run.smithi110.stdout: "dn-": 0, 2024-09-21T03:42:34.249 INFO:teuthology.orchestra.run.smithi110.stdout: "cap": 0, 2024-09-21T03:42:34.249 INFO:teuthology.orchestra.run.smithi110.stdout: "cap+": 3, 2024-09-21T03:42:34.249 INFO:teuthology.orchestra.run.smithi110.stdout: "cap-": 3, 2024-09-21T03:42:34.249 INFO:teuthology.orchestra.run.smithi110.stdout: "rss": 117716, 2024-09-21T03:42:34.249 INFO:teuthology.orchestra.run.smithi110.stdout: "heap": 325857280 2024-09-21T03:42:34.249 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-21T03:42:34.249 INFO:teuthology.orchestra.run.smithi110.stdout: "mds_server": { 2024-09-21T03:42:34.249 INFO:teuthology.orchestra.run.smithi110.stdout: "dispatch_client_request": 6, 2024-09-21T03:42:34.249 INFO:teuthology.orchestra.run.smithi110.stdout: "dispatch_server_request": 0, 2024-09-21T03:42:34.249 INFO:teuthology.orchestra.run.smithi110.stdout: "handle_client_request": 4, 2024-09-21T03:42:34.249 INFO:teuthology.orchestra.run.smithi110.stdout: "handle_client_session": 8, 2024-09-21T03:42:34.249 INFO:teuthology.orchestra.run.smithi110.stdout: "handle_peer_request": 0, 2024-09-21T03:42:34.249 INFO:teuthology.orchestra.run.smithi110.stdout: "req_create_latency": { 2024-09-21T03:42:34.249 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-09-21T03:42:34.249 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-09-21T03:42:34.249 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-09-21T03:42:34.249 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-21T03:42:34.249 INFO:teuthology.orchestra.run.smithi110.stdout: "req_getattr_latency": { 2024-09-21T03:42:34.250 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 1, 2024-09-21T03:42:34.250 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000225065, 2024-09-21T03:42:34.250 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000225065 2024-09-21T03:42:34.250 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-21T03:42:34.250 INFO:teuthology.orchestra.run.smithi110.stdout: "req_getfilelock_latency": { 2024-09-21T03:42:34.250 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-09-21T03:42:34.250 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-09-21T03:42:34.250 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-09-21T03:42:34.250 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-21T03:42:34.250 INFO:teuthology.orchestra.run.smithi110.stdout: "req_link_latency": { 2024-09-21T03:42:34.250 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-09-21T03:42:34.250 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-09-21T03:42:34.250 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-09-21T03:42:34.250 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-21T03:42:34.250 INFO:teuthology.orchestra.run.smithi110.stdout: "req_lookup_latency": { 2024-09-21T03:42:34.250 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 2, 2024-09-21T03:42:34.251 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.001154719, 2024-09-21T03:42:34.251 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000577359 2024-09-21T03:42:34.251 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-21T03:42:34.251 INFO:teuthology.orchestra.run.smithi110.stdout: "req_lookuphash_latency": { 2024-09-21T03:42:34.251 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-09-21T03:42:34.251 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-09-21T03:42:34.251 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-09-21T03:42:34.251 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-21T03:42:34.251 INFO:teuthology.orchestra.run.smithi110.stdout: "req_lookupino_latency": { 2024-09-21T03:42:34.251 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-09-21T03:42:34.251 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-09-21T03:42:34.251 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-09-21T03:42:34.251 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-21T03:42:34.251 INFO:teuthology.orchestra.run.smithi110.stdout: "req_lookupname_latency": { 2024-09-21T03:42:34.251 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-09-21T03:42:34.251 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-09-21T03:42:34.252 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-09-21T03:42:34.252 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-21T03:42:34.252 INFO:teuthology.orchestra.run.smithi110.stdout: "req_lookupparent_latency": { 2024-09-21T03:42:34.252 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-09-21T03:42:34.252 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-09-21T03:42:34.252 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-09-21T03:42:34.252 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-21T03:42:34.252 INFO:teuthology.orchestra.run.smithi110.stdout: "req_lookupsnap_latency": { 2024-09-21T03:42:34.252 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-09-21T03:42:34.252 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-09-21T03:42:34.252 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-09-21T03:42:34.252 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-21T03:42:34.252 INFO:teuthology.orchestra.run.smithi110.stdout: "req_lssnap_latency": { 2024-09-21T03:42:34.252 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-09-21T03:42:34.252 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-09-21T03:42:34.253 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-09-21T03:42:34.253 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-21T03:42:34.253 INFO:teuthology.orchestra.run.smithi110.stdout: "req_mkdir_latency": { 2024-09-21T03:42:34.253 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-09-21T03:42:34.253 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-09-21T03:42:34.253 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-09-21T03:42:34.253 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-21T03:42:34.253 INFO:teuthology.orchestra.run.smithi110.stdout: "req_mknod_latency": { 2024-09-21T03:42:34.253 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-09-21T03:42:34.253 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-09-21T03:42:34.253 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-09-21T03:42:34.253 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-21T03:42:34.254 INFO:teuthology.orchestra.run.smithi110.stdout: "req_mksnap_latency": { 2024-09-21T03:42:34.254 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-09-21T03:42:34.254 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-09-21T03:42:34.254 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-09-21T03:42:34.254 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-21T03:42:34.254 INFO:teuthology.orchestra.run.smithi110.stdout: "req_open_latency": { 2024-09-21T03:42:34.254 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-09-21T03:42:34.254 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-09-21T03:42:34.254 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-09-21T03:42:34.254 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-21T03:42:34.254 INFO:teuthology.orchestra.run.smithi110.stdout: "req_readdir_latency": { 2024-09-21T03:42:34.254 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-09-21T03:42:34.254 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-09-21T03:42:34.254 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-09-21T03:42:34.254 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-21T03:42:34.255 INFO:teuthology.orchestra.run.smithi110.stdout: "req_rename_latency": { 2024-09-21T03:42:34.255 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-09-21T03:42:34.255 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-09-21T03:42:34.255 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-09-21T03:42:34.255 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-21T03:42:34.255 INFO:teuthology.orchestra.run.smithi110.stdout: "req_renamesnap_latency": { 2024-09-21T03:42:34.255 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-09-21T03:42:34.255 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-09-21T03:42:34.255 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-09-21T03:42:34.255 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-21T03:42:34.255 INFO:teuthology.orchestra.run.smithi110.stdout: "req_snapdiff_latency": { 2024-09-21T03:42:34.255 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-09-21T03:42:34.255 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-09-21T03:42:34.255 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-09-21T03:42:34.255 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-21T03:42:34.256 INFO:teuthology.orchestra.run.smithi110.stdout: "req_rmdir_latency": { 2024-09-21T03:42:34.256 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-09-21T03:42:34.256 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-09-21T03:42:34.256 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-09-21T03:42:34.256 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-21T03:42:34.256 INFO:teuthology.orchestra.run.smithi110.stdout: "req_rmsnap_latency": { 2024-09-21T03:42:34.256 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-09-21T03:42:34.256 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-09-21T03:42:34.256 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-09-21T03:42:34.256 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-21T03:42:34.256 INFO:teuthology.orchestra.run.smithi110.stdout: "req_rmxattr_latency": { 2024-09-21T03:42:34.256 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-09-21T03:42:34.256 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-09-21T03:42:34.256 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-09-21T03:42:34.257 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-21T03:42:34.257 INFO:teuthology.orchestra.run.smithi110.stdout: "req_setattr_latency": { 2024-09-21T03:42:34.257 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-09-21T03:42:34.257 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-09-21T03:42:34.257 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-09-21T03:42:34.257 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-21T03:42:34.257 INFO:teuthology.orchestra.run.smithi110.stdout: "req_setdirlayout_latency": { 2024-09-21T03:42:34.257 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-09-21T03:42:34.257 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-09-21T03:42:34.257 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-09-21T03:42:34.257 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-21T03:42:34.257 INFO:teuthology.orchestra.run.smithi110.stdout: "req_setfilelock_latency": { 2024-09-21T03:42:34.257 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-09-21T03:42:34.257 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-09-21T03:42:34.257 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-09-21T03:42:34.258 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-21T03:42:34.258 INFO:teuthology.orchestra.run.smithi110.stdout: "req_setlayout_latency": { 2024-09-21T03:42:34.258 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-09-21T03:42:34.258 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-09-21T03:42:34.258 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-09-21T03:42:34.258 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-21T03:42:34.258 INFO:teuthology.orchestra.run.smithi110.stdout: "req_setxattr_latency": { 2024-09-21T03:42:34.258 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-09-21T03:42:34.258 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-09-21T03:42:34.258 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-09-21T03:42:34.258 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-21T03:42:34.258 INFO:teuthology.orchestra.run.smithi110.stdout: "req_symlink_latency": { 2024-09-21T03:42:34.258 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-09-21T03:42:34.259 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-09-21T03:42:34.259 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-09-21T03:42:34.259 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-21T03:42:34.259 INFO:teuthology.orchestra.run.smithi110.stdout: "req_unlink_latency": { 2024-09-21T03:42:34.259 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-09-21T03:42:34.259 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-09-21T03:42:34.259 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-09-21T03:42:34.259 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-21T03:42:34.259 INFO:teuthology.orchestra.run.smithi110.stdout: "cap_revoke_eviction": 0, 2024-09-21T03:42:34.259 INFO:teuthology.orchestra.run.smithi110.stdout: "cap_acquisition_throttle": 0, 2024-09-21T03:42:34.259 INFO:teuthology.orchestra.run.smithi110.stdout: "req_getvxattr_latency": { 2024-09-21T03:42:34.259 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-09-21T03:42:34.259 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-09-21T03:42:34.259 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-09-21T03:42:34.259 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-21T03:42:34.260 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-21T03:42:34.260 INFO:teuthology.orchestra.run.smithi110.stdout: "mds_sessions": { 2024-09-21T03:42:34.260 INFO:teuthology.orchestra.run.smithi110.stdout: "session_count": 1, 2024-09-21T03:42:34.260 INFO:teuthology.orchestra.run.smithi110.stdout: "session_add": 2, 2024-09-21T03:42:34.260 INFO:teuthology.orchestra.run.smithi110.stdout: "session_remove": 1, 2024-09-21T03:42:34.260 INFO:teuthology.orchestra.run.smithi110.stdout: "sessions_open": 1, 2024-09-21T03:42:34.260 INFO:teuthology.orchestra.run.smithi110.stdout: "sessions_stale": 0, 2024-09-21T03:42:34.260 INFO:teuthology.orchestra.run.smithi110.stdout: "total_load": 2, 2024-09-21T03:42:34.260 INFO:teuthology.orchestra.run.smithi110.stdout: "average_load": 1, 2024-09-21T03:42:34.260 INFO:teuthology.orchestra.run.smithi110.stdout: "avg_session_uptime": 188, 2024-09-21T03:42:34.260 INFO:teuthology.orchestra.run.smithi110.stdout: "mdthresh_evicted": 0 2024-09-21T03:42:34.260 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-21T03:42:34.260 INFO:teuthology.orchestra.run.smithi110.stdout: "mempool": { 2024-09-21T03:42:34.260 INFO:teuthology.orchestra.run.smithi110.stdout: "bloom_filter_bytes": 0, 2024-09-21T03:42:34.260 INFO:teuthology.orchestra.run.smithi110.stdout: "bloom_filter_items": 0, 2024-09-21T03:42:34.261 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_alloc_bytes": 0, 2024-09-21T03:42:34.261 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_alloc_items": 0, 2024-09-21T03:42:34.261 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_cache_data_bytes": 0, 2024-09-21T03:42:34.261 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_cache_data_items": 0, 2024-09-21T03:42:34.261 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-21T03:42:34.261 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_cache_onode_items": 0, 2024-09-21T03:42:34.261 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-21T03:42:34.261 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_cache_meta_items": 0, 2024-09-21T03:42:34.261 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_cache_other_bytes": 0, 2024-09-21T03:42:34.261 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_cache_other_items": 0, 2024-09-21T03:42:34.261 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-21T03:42:34.261 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_cache_buffer_items": 0, 2024-09-21T03:42:34.261 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_extent_bytes": 0, 2024-09-21T03:42:34.261 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_extent_items": 0, 2024-09-21T03:42:34.261 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_blob_bytes": 0, 2024-09-21T03:42:34.262 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_blob_items": 0, 2024-09-21T03:42:34.262 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-21T03:42:34.262 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_shared_blob_items": 0, 2024-09-21T03:42:34.262 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-21T03:42:34.262 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_inline_bl_items": 0, 2024-09-21T03:42:34.262 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_fsck_bytes": 0, 2024-09-21T03:42:34.262 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_fsck_items": 0, 2024-09-21T03:42:34.262 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_txc_bytes": 0, 2024-09-21T03:42:34.262 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_txc_items": 0, 2024-09-21T03:42:34.262 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-21T03:42:34.262 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_writing_deferred_items": 0, 2024-09-21T03:42:34.262 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_writing_bytes": 0, 2024-09-21T03:42:34.262 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_writing_items": 0, 2024-09-21T03:42:34.262 INFO:teuthology.orchestra.run.smithi110.stdout: "bluefs_bytes": 0, 2024-09-21T03:42:34.263 INFO:teuthology.orchestra.run.smithi110.stdout: "bluefs_items": 0, 2024-09-21T03:42:34.263 INFO:teuthology.orchestra.run.smithi110.stdout: "bluefs_file_reader_bytes": 0, 2024-09-21T03:42:34.263 INFO:teuthology.orchestra.run.smithi110.stdout: "bluefs_file_reader_items": 0, 2024-09-21T03:42:34.263 INFO:teuthology.orchestra.run.smithi110.stdout: "bluefs_file_writer_bytes": 0, 2024-09-21T03:42:34.263 INFO:teuthology.orchestra.run.smithi110.stdout: "bluefs_file_writer_items": 0, 2024-09-21T03:42:34.263 INFO:teuthology.orchestra.run.smithi110.stdout: "buffer_anon_bytes": 238647, 2024-09-21T03:42:34.263 INFO:teuthology.orchestra.run.smithi110.stdout: "buffer_anon_items": 77, 2024-09-21T03:42:34.263 INFO:teuthology.orchestra.run.smithi110.stdout: "buffer_meta_bytes": 88, 2024-09-21T03:42:34.263 INFO:teuthology.orchestra.run.smithi110.stdout: "buffer_meta_items": 1, 2024-09-21T03:42:34.263 INFO:teuthology.orchestra.run.smithi110.stdout: "osd_bytes": 0, 2024-09-21T03:42:34.263 INFO:teuthology.orchestra.run.smithi110.stdout: "osd_items": 0, 2024-09-21T03:42:34.263 INFO:teuthology.orchestra.run.smithi110.stdout: "osd_mapbl_bytes": 0, 2024-09-21T03:42:34.263 INFO:teuthology.orchestra.run.smithi110.stdout: "osd_mapbl_items": 0, 2024-09-21T03:42:34.264 INFO:teuthology.orchestra.run.smithi110.stdout: "osd_pglog_bytes": 0, 2024-09-21T03:42:34.264 INFO:teuthology.orchestra.run.smithi110.stdout: "osd_pglog_items": 0, 2024-09-21T03:42:34.264 INFO:teuthology.orchestra.run.smithi110.stdout: "osdmap_bytes": 8760, 2024-09-21T03:42:34.264 INFO:teuthology.orchestra.run.smithi110.stdout: "osdmap_items": 164, 2024-09-21T03:42:34.264 INFO:teuthology.orchestra.run.smithi110.stdout: "osdmap_mapping_bytes": 0, 2024-09-21T03:42:34.264 INFO:teuthology.orchestra.run.smithi110.stdout: "osdmap_mapping_items": 0, 2024-09-21T03:42:34.264 INFO:teuthology.orchestra.run.smithi110.stdout: "pgmap_bytes": 0, 2024-09-21T03:42:34.264 INFO:teuthology.orchestra.run.smithi110.stdout: "pgmap_items": 0, 2024-09-21T03:42:34.264 INFO:teuthology.orchestra.run.smithi110.stdout: "mds_co_bytes": 59688, 2024-09-21T03:42:34.264 INFO:teuthology.orchestra.run.smithi110.stdout: "mds_co_items": 234, 2024-09-21T03:42:34.264 INFO:teuthology.orchestra.run.smithi110.stdout: "unittest_1_bytes": 0, 2024-09-21T03:42:34.264 INFO:teuthology.orchestra.run.smithi110.stdout: "unittest_1_items": 0, 2024-09-21T03:42:34.264 INFO:teuthology.orchestra.run.smithi110.stdout: "unittest_2_bytes": 0, 2024-09-21T03:42:34.264 INFO:teuthology.orchestra.run.smithi110.stdout: "unittest_2_items": 0 2024-09-21T03:42:34.264 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-21T03:42:34.264 INFO:teuthology.orchestra.run.smithi110.stdout: "objecter": { 2024-09-21T03:42:34.265 INFO:teuthology.orchestra.run.smithi110.stdout: "op_active": 0, 2024-09-21T03:42:34.265 INFO:teuthology.orchestra.run.smithi110.stdout: "op_laggy": 0, 2024-09-21T03:42:34.265 INFO:teuthology.orchestra.run.smithi110.stdout: "op_send": 42, 2024-09-21T03:42:34.265 INFO:teuthology.orchestra.run.smithi110.stdout: "op_send_bytes": 12140, 2024-09-21T03:42:34.265 INFO:teuthology.orchestra.run.smithi110.stdout: "op_resend": 33, 2024-09-21T03:42:34.265 INFO:teuthology.orchestra.run.smithi110.stdout: "op_reply": 42, 2024-09-21T03:42:34.265 INFO:teuthology.orchestra.run.smithi110.stdout: "op_latency": { 2024-09-21T03:42:34.265 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 42, 2024-09-21T03:42:34.265 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.203996433, 2024-09-21T03:42:34.265 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.004857057 2024-09-21T03:42:34.265 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-21T03:42:34.265 INFO:teuthology.orchestra.run.smithi110.stdout: "op_inflight": 0, 2024-09-21T03:42:34.265 INFO:teuthology.orchestra.run.smithi110.stdout: "oplen_avg": { 2024-09-21T03:42:34.265 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 42, 2024-09-21T03:42:34.265 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 66 2024-09-21T03:42:34.265 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-21T03:42:34.265 INFO:teuthology.orchestra.run.smithi110.stdout: "op": 42, 2024-09-21T03:42:34.266 INFO:teuthology.orchestra.run.smithi110.stdout: "op_r": 0, 2024-09-21T03:42:34.266 INFO:teuthology.orchestra.run.smithi110.stdout: "op_w": 42, 2024-09-21T03:42:34.266 INFO:teuthology.orchestra.run.smithi110.stdout: "op_rmw": 0, 2024-09-21T03:42:34.266 INFO:teuthology.orchestra.run.smithi110.stdout: "op_pg": 0, 2024-09-21T03:42:34.266 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_stat": 0, 2024-09-21T03:42:34.266 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_create": 10, 2024-09-21T03:42:34.266 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_read": 0, 2024-09-21T03:42:34.266 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_write": 4, 2024-09-21T03:42:34.266 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_writefull": 7, 2024-09-21T03:42:34.266 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_writesame": 0, 2024-09-21T03:42:34.266 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_append": 0, 2024-09-21T03:42:34.266 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_zero": 0, 2024-09-21T03:42:34.266 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_truncate": 0, 2024-09-21T03:42:34.267 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_delete": 6, 2024-09-21T03:42:34.267 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_mapext": 0, 2024-09-21T03:42:34.267 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_sparse_read": 0, 2024-09-21T03:42:34.267 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_clonerange": 0, 2024-09-21T03:42:34.267 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_getxattr": 0, 2024-09-21T03:42:34.267 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_setxattr": 20, 2024-09-21T03:42:34.267 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_cmpxattr": 0, 2024-09-21T03:42:34.267 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_rmxattr": 0, 2024-09-21T03:42:34.267 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_resetxattrs": 0, 2024-09-21T03:42:34.267 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_call": 0, 2024-09-21T03:42:34.267 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_watch": 0, 2024-09-21T03:42:34.267 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_notify": 0, 2024-09-21T03:42:34.267 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_src_cmpxattr": 0, 2024-09-21T03:42:34.267 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_pgls": 0, 2024-09-21T03:42:34.267 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_pgls_filter": 0, 2024-09-21T03:42:34.267 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_other": 0, 2024-09-21T03:42:34.267 INFO:teuthology.orchestra.run.smithi110.stdout: "linger_active": 0, 2024-09-21T03:42:34.268 INFO:teuthology.orchestra.run.smithi110.stdout: "linger_send": 0, 2024-09-21T03:42:34.268 INFO:teuthology.orchestra.run.smithi110.stdout: "linger_resend": 0, 2024-09-21T03:42:34.268 INFO:teuthology.orchestra.run.smithi110.stdout: "linger_ping": 0, 2024-09-21T03:42:34.268 INFO:teuthology.orchestra.run.smithi110.stdout: "poolop_active": 0, 2024-09-21T03:42:34.268 INFO:teuthology.orchestra.run.smithi110.stdout: "poolop_send": 0, 2024-09-21T03:42:34.268 INFO:teuthology.orchestra.run.smithi110.stdout: "poolop_resend": 0, 2024-09-21T03:42:34.268 INFO:teuthology.orchestra.run.smithi110.stdout: "poolstat_active": 0, 2024-09-21T03:42:34.268 INFO:teuthology.orchestra.run.smithi110.stdout: "poolstat_send": 0, 2024-09-21T03:42:34.268 INFO:teuthology.orchestra.run.smithi110.stdout: "poolstat_resend": 0, 2024-09-21T03:42:34.268 INFO:teuthology.orchestra.run.smithi110.stdout: "statfs_active": 0, 2024-09-21T03:42:34.268 INFO:teuthology.orchestra.run.smithi110.stdout: "statfs_send": 0, 2024-09-21T03:42:34.268 INFO:teuthology.orchestra.run.smithi110.stdout: "statfs_resend": 0, 2024-09-21T03:42:34.268 INFO:teuthology.orchestra.run.smithi110.stdout: "command_active": 0, 2024-09-21T03:42:34.268 INFO:teuthology.orchestra.run.smithi110.stdout: "command_send": 0, 2024-09-21T03:42:34.268 INFO:teuthology.orchestra.run.smithi110.stdout: "command_resend": 0, 2024-09-21T03:42:34.268 INFO:teuthology.orchestra.run.smithi110.stdout: "map_epoch": 84, 2024-09-21T03:42:34.269 INFO:teuthology.orchestra.run.smithi110.stdout: "map_full": 0, 2024-09-21T03:42:34.269 INFO:teuthology.orchestra.run.smithi110.stdout: "map_inc": 4, 2024-09-21T03:42:34.269 INFO:teuthology.orchestra.run.smithi110.stdout: "osd_sessions": 8, 2024-09-21T03:42:34.269 INFO:teuthology.orchestra.run.smithi110.stdout: "osd_session_open": 8, 2024-09-21T03:42:34.269 INFO:teuthology.orchestra.run.smithi110.stdout: "osd_session_close": 0, 2024-09-21T03:42:34.269 INFO:teuthology.orchestra.run.smithi110.stdout: "osd_laggy": 0, 2024-09-21T03:42:34.269 INFO:teuthology.orchestra.run.smithi110.stdout: "omap_wr": 17, 2024-09-21T03:42:34.269 INFO:teuthology.orchestra.run.smithi110.stdout: "omap_rd": 0, 2024-09-21T03:42:34.269 INFO:teuthology.orchestra.run.smithi110.stdout: "omap_del": 2 2024-09-21T03:42:34.269 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-21T03:42:34.269 INFO:teuthology.orchestra.run.smithi110.stdout: "oft": { 2024-09-21T03:42:34.269 INFO:teuthology.orchestra.run.smithi110.stdout: "omap_total_objs": 1, 2024-09-21T03:42:34.269 INFO:teuthology.orchestra.run.smithi110.stdout: "omap_total_kv_pairs": 0, 2024-09-21T03:42:34.269 INFO:teuthology.orchestra.run.smithi110.stdout: "omap_total_updates": 0, 2024-09-21T03:42:34.269 INFO:teuthology.orchestra.run.smithi110.stdout: "omap_total_removes": 0 2024-09-21T03:42:34.269 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-21T03:42:34.269 INFO:teuthology.orchestra.run.smithi110.stdout: "purge_queue": { 2024-09-21T03:42:34.270 INFO:teuthology.orchestra.run.smithi110.stdout: "pq_executing_ops": 0, 2024-09-21T03:42:34.270 INFO:teuthology.orchestra.run.smithi110.stdout: "pq_executing_ops_high_water": 0, 2024-09-21T03:42:34.270 INFO:teuthology.orchestra.run.smithi110.stdout: "pq_executing": 0, 2024-09-21T03:42:34.270 INFO:teuthology.orchestra.run.smithi110.stdout: "pq_executing_high_water": 0, 2024-09-21T03:42:34.270 INFO:teuthology.orchestra.run.smithi110.stdout: "pq_executed_ops": 0, 2024-09-21T03:42:34.270 INFO:teuthology.orchestra.run.smithi110.stdout: "pq_executed": 0, 2024-09-21T03:42:34.270 INFO:teuthology.orchestra.run.smithi110.stdout: "pq_item_in_journal": 0 2024-09-21T03:42:34.270 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-21T03:42:34.270 INFO:teuthology.orchestra.run.smithi110.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-21T03:42:34.270 INFO:teuthology.orchestra.run.smithi110.stdout: "val": 0, 2024-09-21T03:42:34.270 INFO:teuthology.orchestra.run.smithi110.stdout: "max": 104857600, 2024-09-21T03:42:34.270 INFO:teuthology.orchestra.run.smithi110.stdout: "get_started": 0, 2024-09-21T03:42:34.270 INFO:teuthology.orchestra.run.smithi110.stdout: "get": 360, 2024-09-21T03:42:34.270 INFO:teuthology.orchestra.run.smithi110.stdout: "get_sum": 854973, 2024-09-21T03:42:34.270 INFO:teuthology.orchestra.run.smithi110.stdout: "get_or_fail_fail": 0, 2024-09-21T03:42:34.270 INFO:teuthology.orchestra.run.smithi110.stdout: "get_or_fail_success": 360, 2024-09-21T03:42:34.271 INFO:teuthology.orchestra.run.smithi110.stdout: "take": 0, 2024-09-21T03:42:34.271 INFO:teuthology.orchestra.run.smithi110.stdout: "take_sum": 0, 2024-09-21T03:42:34.271 INFO:teuthology.orchestra.run.smithi110.stdout: "put": 360, 2024-09-21T03:42:34.271 INFO:teuthology.orchestra.run.smithi110.stdout: "put_sum": 854973, 2024-09-21T03:42:34.271 INFO:teuthology.orchestra.run.smithi110.stdout: "wait": { 2024-09-21T03:42:34.271 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-09-21T03:42:34.271 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-09-21T03:42:34.271 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-09-21T03:42:34.271 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-21T03:42:34.271 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-21T03:42:34.271 INFO:teuthology.orchestra.run.smithi110.stdout: "throttle-objecter_bytes": { 2024-09-21T03:42:34.271 INFO:teuthology.orchestra.run.smithi110.stdout: "val": 0, 2024-09-21T03:42:34.271 INFO:teuthology.orchestra.run.smithi110.stdout: "max": 104857600, 2024-09-21T03:42:34.271 INFO:teuthology.orchestra.run.smithi110.stdout: "get_started": 0, 2024-09-21T03:42:34.271 INFO:teuthology.orchestra.run.smithi110.stdout: "get": 0, 2024-09-21T03:42:34.271 INFO:teuthology.orchestra.run.smithi110.stdout: "get_sum": 0, 2024-09-21T03:42:34.271 INFO:teuthology.orchestra.run.smithi110.stdout: "get_or_fail_fail": 0, 2024-09-21T03:42:34.272 INFO:teuthology.orchestra.run.smithi110.stdout: "get_or_fail_success": 0, 2024-09-21T03:42:34.272 INFO:teuthology.orchestra.run.smithi110.stdout: "take": 42, 2024-09-21T03:42:34.272 INFO:teuthology.orchestra.run.smithi110.stdout: "take_sum": 12140, 2024-09-21T03:42:34.272 INFO:teuthology.orchestra.run.smithi110.stdout: "put": 42, 2024-09-21T03:42:34.272 INFO:teuthology.orchestra.run.smithi110.stdout: "put_sum": 12140, 2024-09-21T03:42:34.272 INFO:teuthology.orchestra.run.smithi110.stdout: "wait": { 2024-09-21T03:42:34.272 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-09-21T03:42:34.272 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-09-21T03:42:34.272 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-09-21T03:42:34.272 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-21T03:42:34.272 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-21T03:42:34.272 INFO:teuthology.orchestra.run.smithi110.stdout: "throttle-objecter_ops": { 2024-09-21T03:42:34.272 INFO:teuthology.orchestra.run.smithi110.stdout: "val": 0, 2024-09-21T03:42:34.272 INFO:teuthology.orchestra.run.smithi110.stdout: "max": 1024, 2024-09-21T03:42:34.272 INFO:teuthology.orchestra.run.smithi110.stdout: "get_started": 0, 2024-09-21T03:42:34.272 INFO:teuthology.orchestra.run.smithi110.stdout: "get": 0, 2024-09-21T03:42:34.272 INFO:teuthology.orchestra.run.smithi110.stdout: "get_sum": 0, 2024-09-21T03:42:34.273 INFO:teuthology.orchestra.run.smithi110.stdout: "get_or_fail_fail": 0, 2024-09-21T03:42:34.273 INFO:teuthology.orchestra.run.smithi110.stdout: "get_or_fail_success": 0, 2024-09-21T03:42:34.273 INFO:teuthology.orchestra.run.smithi110.stdout: "take": 42, 2024-09-21T03:42:34.273 INFO:teuthology.orchestra.run.smithi110.stdout: "take_sum": 42, 2024-09-21T03:42:34.273 INFO:teuthology.orchestra.run.smithi110.stdout: "put": 42, 2024-09-21T03:42:34.273 INFO:teuthology.orchestra.run.smithi110.stdout: "put_sum": 42, 2024-09-21T03:42:34.273 INFO:teuthology.orchestra.run.smithi110.stdout: "wait": { 2024-09-21T03:42:34.273 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-09-21T03:42:34.273 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-09-21T03:42:34.273 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-09-21T03:42:34.273 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-21T03:42:34.273 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-21T03:42:34.273 INFO:teuthology.orchestra.run.smithi110.stdout: "throttle-write_buf_throttle": { 2024-09-21T03:42:34.273 INFO:teuthology.orchestra.run.smithi110.stdout: "val": 0, 2024-09-21T03:42:34.273 INFO:teuthology.orchestra.run.smithi110.stdout: "max": 3758096384, 2024-09-21T03:42:34.273 INFO:teuthology.orchestra.run.smithi110.stdout: "get_started": 0, 2024-09-21T03:42:34.273 INFO:teuthology.orchestra.run.smithi110.stdout: "get": 0, 2024-09-21T03:42:34.274 INFO:teuthology.orchestra.run.smithi110.stdout: "get_sum": 0, 2024-09-21T03:42:34.274 INFO:teuthology.orchestra.run.smithi110.stdout: "get_or_fail_fail": 0, 2024-09-21T03:42:34.274 INFO:teuthology.orchestra.run.smithi110.stdout: "get_or_fail_success": 0, 2024-09-21T03:42:34.274 INFO:teuthology.orchestra.run.smithi110.stdout: "take": 0, 2024-09-21T03:42:34.274 INFO:teuthology.orchestra.run.smithi110.stdout: "take_sum": 0, 2024-09-21T03:42:34.274 INFO:teuthology.orchestra.run.smithi110.stdout: "put": 0, 2024-09-21T03:42:34.274 INFO:teuthology.orchestra.run.smithi110.stdout: "put_sum": 0, 2024-09-21T03:42:34.274 INFO:teuthology.orchestra.run.smithi110.stdout: "wait": { 2024-09-21T03:42:34.274 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-09-21T03:42:34.274 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-09-21T03:42:34.274 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-09-21T03:42:34.274 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-21T03:42:34.274 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-21T03:42:34.274 INFO:teuthology.orchestra.run.smithi110.stdout: "throttle-write_buf_throttle-0x562d6b61d6c0": { 2024-09-21T03:42:34.274 INFO:teuthology.orchestra.run.smithi110.stdout: "val": 0, 2024-09-21T03:42:34.275 INFO:teuthology.orchestra.run.smithi110.stdout: "max": 3758096384, 2024-09-21T03:42:34.275 INFO:teuthology.orchestra.run.smithi110.stdout: "get_started": 0, 2024-09-21T03:42:34.275 INFO:teuthology.orchestra.run.smithi110.stdout: "get": 5, 2024-09-21T03:42:34.275 INFO:teuthology.orchestra.run.smithi110.stdout: "get_sum": 1613, 2024-09-21T03:42:34.275 INFO:teuthology.orchestra.run.smithi110.stdout: "get_or_fail_fail": 0, 2024-09-21T03:42:34.275 INFO:teuthology.orchestra.run.smithi110.stdout: "get_or_fail_success": 5, 2024-09-21T03:42:34.275 INFO:teuthology.orchestra.run.smithi110.stdout: "take": 0, 2024-09-21T03:42:34.275 INFO:teuthology.orchestra.run.smithi110.stdout: "take_sum": 0, 2024-09-21T03:42:34.275 INFO:teuthology.orchestra.run.smithi110.stdout: "put": 4, 2024-09-21T03:42:34.275 INFO:teuthology.orchestra.run.smithi110.stdout: "put_sum": 1613, 2024-09-21T03:42:34.275 INFO:teuthology.orchestra.run.smithi110.stdout: "wait": { 2024-09-21T03:42:34.275 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-09-21T03:42:34.275 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-09-21T03:42:34.275 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-09-21T03:42:34.275 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-21T03:42:34.275 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-21T03:42:34.275 INFO:teuthology.orchestra.run.smithi110.stdout: "trackedop": { 2024-09-21T03:42:34.276 INFO:teuthology.orchestra.run.smithi110.stdout: "slow_ops_count": 12 2024-09-21T03:42:34.276 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-21T03:42:34.276 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-21T03:42:35.022 INFO:tasks.check_counter:Daemon mds.h mds.exported=0 2024-09-21T03:42:35.022 INFO:tasks.check_counter:Daemon mds.h mds.imported=0 2024-09-21T03:42:35.022 DEBUG:tasks.check_counter:Getting stats from k 2024-09-21T03:42:35.023 DEBUG:teuthology.orchestra.run.smithi110:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 7c374098-77c9-11ef-baf5-efdc52797490 -- ceph daemon mds.k perf dump 2024-09-21T03:42:35.211 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:42:34 smithi114 ceph-mon[34140]: pgmap v455: 129 pgs: 129 active+clean; 1.1 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 682 B/s wr, 8 op/s 2024-09-21T03:42:35.250 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:42:34 smithi110 ceph-mon[34000]: pgmap v455: 129 pgs: 129 active+clean; 1.1 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 682 B/s wr, 8 op/s 2024-09-21T03:42:35.267 INFO:teuthology.orchestra.run.smithi110.stderr:Inferring config /var/lib/ceph/7c374098-77c9-11ef-baf5-efdc52797490/mon.b/config 2024-09-21T03:42:35.311 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:42:34 smithi089 ceph-mon[30928]: pgmap v455: 129 pgs: 129 active+clean; 1.1 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 682 B/s wr, 8 op/s 2024-09-21T03:42:35.893 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-21T03:42:35.894 INFO:teuthology.orchestra.run.smithi110.stdout: "AsyncMessenger::Worker-0": { 2024-09-21T03:42:35.894 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_recv_messages": 106, 2024-09-21T03:42:35.894 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_send_messages": 101, 2024-09-21T03:42:35.894 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_recv_bytes": 804832, 2024-09-21T03:42:35.894 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_send_bytes": 60848, 2024-09-21T03:42:35.894 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_created_connections": 4, 2024-09-21T03:42:35.894 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_active_connections": 0, 2024-09-21T03:42:35.894 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_running_total_time": 0.024364823, 2024-09-21T03:42:35.894 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_running_send_time": 0.008974768, 2024-09-21T03:42:35.894 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_running_recv_time": 0.023552004, 2024-09-21T03:42:35.895 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-21T03:42:35.895 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T03:42:35.895 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 100, 2024-09-21T03:42:35.895 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.006741301, 2024-09-21T03:42:35.895 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000067413 2024-09-21T03:42:35.895 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-21T03:42:35.895 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_handle_ack_lat": { 2024-09-21T03:42:35.895 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-09-21T03:42:35.895 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-09-21T03:42:35.895 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-09-21T03:42:35.895 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-21T03:42:35.895 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_recv_encrypted_bytes": 804832, 2024-09-21T03:42:35.895 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_send_encrypted_bytes": 60848 2024-09-21T03:42:35.895 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-21T03:42:35.896 INFO:teuthology.orchestra.run.smithi110.stdout: "AsyncMessenger::Worker-1": { 2024-09-21T03:42:35.896 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_recv_messages": 1, 2024-09-21T03:42:35.896 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_send_messages": 76, 2024-09-21T03:42:35.896 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_recv_bytes": 160, 2024-09-21T03:42:35.896 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_send_bytes": 93568, 2024-09-21T03:42:35.896 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_created_connections": 3, 2024-09-21T03:42:35.896 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_active_connections": 2, 2024-09-21T03:42:35.896 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_running_total_time": 0.007098218, 2024-09-21T03:42:35.896 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_running_send_time": 0.005081477, 2024-09-21T03:42:35.896 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_running_recv_time": 0.001186597, 2024-09-21T03:42:35.896 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-21T03:42:35.896 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T03:42:35.896 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 74, 2024-09-21T03:42:35.896 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.003843945, 2024-09-21T03:42:35.897 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000051945 2024-09-21T03:42:35.897 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-21T03:42:35.897 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_handle_ack_lat": { 2024-09-21T03:42:35.897 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-09-21T03:42:35.897 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-09-21T03:42:35.897 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-09-21T03:42:35.897 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-21T03:42:35.897 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_recv_encrypted_bytes": 160, 2024-09-21T03:42:35.897 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_send_encrypted_bytes": 93568 2024-09-21T03:42:35.897 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-21T03:42:35.897 INFO:teuthology.orchestra.run.smithi110.stdout: "AsyncMessenger::Worker-2": { 2024-09-21T03:42:35.897 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_recv_messages": 4, 2024-09-21T03:42:35.897 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_send_messages": 4, 2024-09-21T03:42:35.897 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_recv_bytes": 2960, 2024-09-21T03:42:35.897 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_send_bytes": 448, 2024-09-21T03:42:35.898 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_created_connections": 2, 2024-09-21T03:42:35.898 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_active_connections": 1, 2024-09-21T03:42:35.898 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_running_total_time": 0.002174816, 2024-09-21T03:42:35.898 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_running_send_time": 0.000047637, 2024-09-21T03:42:35.898 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_running_recv_time": 0.002121475, 2024-09-21T03:42:35.898 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-21T03:42:35.898 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T03:42:35.898 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 3, 2024-09-21T03:42:35.898 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.003741525, 2024-09-21T03:42:35.898 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.001247175 2024-09-21T03:42:35.898 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-21T03:42:35.898 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_handle_ack_lat": { 2024-09-21T03:42:35.898 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-09-21T03:42:35.898 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-09-21T03:42:35.899 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-09-21T03:42:35.899 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-21T03:42:35.899 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_recv_encrypted_bytes": 2960, 2024-09-21T03:42:35.899 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_send_encrypted_bytes": 448 2024-09-21T03:42:35.899 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-21T03:42:35.899 INFO:teuthology.orchestra.run.smithi110.stdout: "cct": { 2024-09-21T03:42:35.899 INFO:teuthology.orchestra.run.smithi110.stdout: "total_workers": 0, 2024-09-21T03:42:35.899 INFO:teuthology.orchestra.run.smithi110.stdout: "unhealthy_workers": 0 2024-09-21T03:42:35.899 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-21T03:42:35.899 INFO:teuthology.orchestra.run.smithi110.stdout: "mempool": { 2024-09-21T03:42:35.899 INFO:teuthology.orchestra.run.smithi110.stdout: "bloom_filter_bytes": 0, 2024-09-21T03:42:35.899 INFO:teuthology.orchestra.run.smithi110.stdout: "bloom_filter_items": 0, 2024-09-21T03:42:35.899 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_alloc_bytes": 0, 2024-09-21T03:42:35.899 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_alloc_items": 0, 2024-09-21T03:42:35.900 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_cache_data_bytes": 0, 2024-09-21T03:42:35.900 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_cache_data_items": 0, 2024-09-21T03:42:35.900 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-21T03:42:35.900 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_cache_onode_items": 0, 2024-09-21T03:42:35.900 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-21T03:42:35.900 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_cache_meta_items": 0, 2024-09-21T03:42:35.900 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_cache_other_bytes": 0, 2024-09-21T03:42:35.900 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_cache_other_items": 0, 2024-09-21T03:42:35.900 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-21T03:42:35.900 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_cache_buffer_items": 0, 2024-09-21T03:42:35.900 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_extent_bytes": 0, 2024-09-21T03:42:35.900 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_extent_items": 0, 2024-09-21T03:42:35.900 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_blob_bytes": 0, 2024-09-21T03:42:35.900 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_blob_items": 0, 2024-09-21T03:42:35.900 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-21T03:42:35.901 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_shared_blob_items": 0, 2024-09-21T03:42:35.901 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-21T03:42:35.901 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_inline_bl_items": 0, 2024-09-21T03:42:35.901 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_fsck_bytes": 0, 2024-09-21T03:42:35.901 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_fsck_items": 0, 2024-09-21T03:42:35.901 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_txc_bytes": 0, 2024-09-21T03:42:35.901 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_txc_items": 0, 2024-09-21T03:42:35.901 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-21T03:42:35.901 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_writing_deferred_items": 0, 2024-09-21T03:42:35.901 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_writing_bytes": 0, 2024-09-21T03:42:35.901 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_writing_items": 0, 2024-09-21T03:42:35.901 INFO:teuthology.orchestra.run.smithi110.stdout: "bluefs_bytes": 0, 2024-09-21T03:42:35.901 INFO:teuthology.orchestra.run.smithi110.stdout: "bluefs_items": 0, 2024-09-21T03:42:35.901 INFO:teuthology.orchestra.run.smithi110.stdout: "bluefs_file_reader_bytes": 0, 2024-09-21T03:42:35.902 INFO:teuthology.orchestra.run.smithi110.stdout: "bluefs_file_reader_items": 0, 2024-09-21T03:42:35.902 INFO:teuthology.orchestra.run.smithi110.stdout: "bluefs_file_writer_bytes": 0, 2024-09-21T03:42:35.902 INFO:teuthology.orchestra.run.smithi110.stdout: "bluefs_file_writer_items": 0, 2024-09-21T03:42:35.902 INFO:teuthology.orchestra.run.smithi110.stdout: "buffer_anon_bytes": 148863, 2024-09-21T03:42:35.902 INFO:teuthology.orchestra.run.smithi110.stdout: "buffer_anon_items": 28, 2024-09-21T03:42:35.902 INFO:teuthology.orchestra.run.smithi110.stdout: "buffer_meta_bytes": 0, 2024-09-21T03:42:35.902 INFO:teuthology.orchestra.run.smithi110.stdout: "buffer_meta_items": 0, 2024-09-21T03:42:35.902 INFO:teuthology.orchestra.run.smithi110.stdout: "osd_bytes": 0, 2024-09-21T03:42:35.902 INFO:teuthology.orchestra.run.smithi110.stdout: "osd_items": 0, 2024-09-21T03:42:35.902 INFO:teuthology.orchestra.run.smithi110.stdout: "osd_mapbl_bytes": 0, 2024-09-21T03:42:35.902 INFO:teuthology.orchestra.run.smithi110.stdout: "osd_mapbl_items": 0, 2024-09-21T03:42:35.902 INFO:teuthology.orchestra.run.smithi110.stdout: "osd_pglog_bytes": 0, 2024-09-21T03:42:35.902 INFO:teuthology.orchestra.run.smithi110.stdout: "osd_pglog_items": 0, 2024-09-21T03:42:35.902 INFO:teuthology.orchestra.run.smithi110.stdout: "osdmap_bytes": 0, 2024-09-21T03:42:35.902 INFO:teuthology.orchestra.run.smithi110.stdout: "osdmap_items": 0, 2024-09-21T03:42:35.903 INFO:teuthology.orchestra.run.smithi110.stdout: "osdmap_mapping_bytes": 0, 2024-09-21T03:42:35.903 INFO:teuthology.orchestra.run.smithi110.stdout: "osdmap_mapping_items": 0, 2024-09-21T03:42:35.903 INFO:teuthology.orchestra.run.smithi110.stdout: "pgmap_bytes": 0, 2024-09-21T03:42:35.903 INFO:teuthology.orchestra.run.smithi110.stdout: "pgmap_items": 0, 2024-09-21T03:42:35.903 INFO:teuthology.orchestra.run.smithi110.stdout: "mds_co_bytes": 968, 2024-09-21T03:42:35.903 INFO:teuthology.orchestra.run.smithi110.stdout: "mds_co_items": 2, 2024-09-21T03:42:35.903 INFO:teuthology.orchestra.run.smithi110.stdout: "unittest_1_bytes": 0, 2024-09-21T03:42:35.903 INFO:teuthology.orchestra.run.smithi110.stdout: "unittest_1_items": 0, 2024-09-21T03:42:35.903 INFO:teuthology.orchestra.run.smithi110.stdout: "unittest_2_bytes": 0, 2024-09-21T03:42:35.903 INFO:teuthology.orchestra.run.smithi110.stdout: "unittest_2_items": 0 2024-09-21T03:42:35.903 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-21T03:42:35.903 INFO:teuthology.orchestra.run.smithi110.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-21T03:42:35.903 INFO:teuthology.orchestra.run.smithi110.stdout: "val": 0, 2024-09-21T03:42:35.903 INFO:teuthology.orchestra.run.smithi110.stdout: "max": 104857600, 2024-09-21T03:42:35.903 INFO:teuthology.orchestra.run.smithi110.stdout: "get_started": 0, 2024-09-21T03:42:35.903 INFO:teuthology.orchestra.run.smithi110.stdout: "get": 107, 2024-09-21T03:42:35.903 INFO:teuthology.orchestra.run.smithi110.stdout: "get_sum": 789915, 2024-09-21T03:42:35.904 INFO:teuthology.orchestra.run.smithi110.stdout: "get_or_fail_fail": 0, 2024-09-21T03:42:35.904 INFO:teuthology.orchestra.run.smithi110.stdout: "get_or_fail_success": 107, 2024-09-21T03:42:35.904 INFO:teuthology.orchestra.run.smithi110.stdout: "take": 0, 2024-09-21T03:42:35.904 INFO:teuthology.orchestra.run.smithi110.stdout: "take_sum": 0, 2024-09-21T03:42:35.904 INFO:teuthology.orchestra.run.smithi110.stdout: "put": 107, 2024-09-21T03:42:35.904 INFO:teuthology.orchestra.run.smithi110.stdout: "put_sum": 789915, 2024-09-21T03:42:35.904 INFO:teuthology.orchestra.run.smithi110.stdout: "wait": { 2024-09-21T03:42:35.904 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-09-21T03:42:35.904 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-09-21T03:42:35.904 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-09-21T03:42:35.904 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-21T03:42:35.904 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-21T03:42:35.904 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-21T03:42:36.583 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.k 2024-09-21T03:42:36.583 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.k 2024-09-21T03:42:36.583 DEBUG:tasks.check_counter:Getting stats from c 2024-09-21T03:42:36.583 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 7c374098-77c9-11ef-baf5-efdc52797490 -- ceph daemon mds.c perf dump 2024-09-21T03:42:36.843 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/7c374098-77c9-11ef-baf5-efdc52797490/mon.c/config 2024-09-21T03:42:37.211 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:42:36 smithi114 ceph-mon[34140]: pgmap v456: 129 pgs: 129 active+clean; 1.1 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 170 B/s wr, 7 op/s 2024-09-21T03:42:37.250 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:42:36 smithi110 ceph-mon[34000]: pgmap v456: 129 pgs: 129 active+clean; 1.1 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 170 B/s wr, 7 op/s 2024-09-21T03:42:37.311 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:42:36 smithi089 ceph-mon[30928]: pgmap v456: 129 pgs: 129 active+clean; 1.1 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 170 B/s wr, 7 op/s 2024-09-21T03:42:37.387 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-21T03:42:37.387 INFO:teuthology.orchestra.run.smithi114.stdout: "AsyncMessenger::Worker-0": { 2024-09-21T03:42:37.387 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_recv_messages": 775, 2024-09-21T03:42:37.387 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_send_messages": 768, 2024-09-21T03:42:37.387 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_recv_bytes": 1614119, 2024-09-21T03:42:37.387 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_send_bytes": 270168, 2024-09-21T03:42:37.388 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_created_connections": 5, 2024-09-21T03:42:37.388 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_active_connections": 1, 2024-09-21T03:42:37.388 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_running_total_time": 0.100277904, 2024-09-21T03:42:37.388 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_running_send_time": 0.041271411, 2024-09-21T03:42:37.388 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_running_recv_time": 0.040640721, 2024-09-21T03:42:37.388 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_running_fast_dispatch_time": 0.020621094, 2024-09-21T03:42:37.388 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T03:42:37.388 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 767, 2024-09-21T03:42:37.388 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.031309216, 2024-09-21T03:42:37.388 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000040820 2024-09-21T03:42:37.388 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T03:42:37.388 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_handle_ack_lat": { 2024-09-21T03:42:37.388 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-09-21T03:42:37.388 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-09-21T03:42:37.388 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-09-21T03:42:37.388 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T03:42:37.389 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_recv_encrypted_bytes": 827552, 2024-09-21T03:42:37.389 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_send_encrypted_bytes": 59648 2024-09-21T03:42:37.389 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T03:42:37.389 INFO:teuthology.orchestra.run.smithi114.stdout: "AsyncMessenger::Worker-1": { 2024-09-21T03:42:37.389 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_recv_messages": 9, 2024-09-21T03:42:37.389 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_send_messages": 79, 2024-09-21T03:42:37.389 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_recv_bytes": 4776, 2024-09-21T03:42:37.389 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_send_bytes": 200042, 2024-09-21T03:42:37.389 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_created_connections": 5, 2024-09-21T03:42:37.389 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_active_connections": 4, 2024-09-21T03:42:37.389 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_running_total_time": 0.008751299, 2024-09-21T03:42:37.389 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_running_send_time": 0.004174406, 2024-09-21T03:42:37.389 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_running_recv_time": 0.003867684, 2024-09-21T03:42:37.389 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_running_fast_dispatch_time": 0.000096211, 2024-09-21T03:42:37.390 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T03:42:37.390 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 78, 2024-09-21T03:42:37.390 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.011184128, 2024-09-21T03:42:37.390 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000143386 2024-09-21T03:42:37.390 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T03:42:37.390 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_handle_ack_lat": { 2024-09-21T03:42:37.390 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-09-21T03:42:37.390 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-09-21T03:42:37.390 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-09-21T03:42:37.390 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T03:42:37.390 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_recv_encrypted_bytes": 3120, 2024-09-21T03:42:37.390 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_send_encrypted_bytes": 198704 2024-09-21T03:42:37.390 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T03:42:37.391 INFO:teuthology.orchestra.run.smithi114.stdout: "AsyncMessenger::Worker-2": { 2024-09-21T03:42:37.391 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_recv_messages": 291, 2024-09-21T03:42:37.391 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_send_messages": 293, 2024-09-21T03:42:37.391 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_recv_bytes": 71692, 2024-09-21T03:42:37.391 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_send_bytes": 92545, 2024-09-21T03:42:37.391 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_created_connections": 4, 2024-09-21T03:42:37.391 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_active_connections": 3, 2024-09-21T03:42:37.391 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_running_total_time": 0.035632076, 2024-09-21T03:42:37.391 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_running_send_time": 0.015907217, 2024-09-21T03:42:37.391 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_running_recv_time": 0.007142875, 2024-09-21T03:42:37.391 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_running_fast_dispatch_time": 0.009936220, 2024-09-21T03:42:37.391 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T03:42:37.391 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 291, 2024-09-21T03:42:37.391 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.012658595, 2024-09-21T03:42:37.392 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000043500 2024-09-21T03:42:37.392 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T03:42:37.392 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_handle_ack_lat": { 2024-09-21T03:42:37.392 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-09-21T03:42:37.392 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-09-21T03:42:37.392 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-09-21T03:42:37.392 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T03:42:37.392 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-21T03:42:37.392 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_send_encrypted_bytes": 0 2024-09-21T03:42:37.392 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T03:42:37.392 INFO:teuthology.orchestra.run.smithi114.stdout: "cct": { 2024-09-21T03:42:37.392 INFO:teuthology.orchestra.run.smithi114.stdout: "total_workers": 1, 2024-09-21T03:42:37.392 INFO:teuthology.orchestra.run.smithi114.stdout: "unhealthy_workers": 0 2024-09-21T03:42:37.392 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T03:42:37.393 INFO:teuthology.orchestra.run.smithi114.stdout: "finisher-MDSRank": { 2024-09-21T03:42:37.393 INFO:teuthology.orchestra.run.smithi114.stdout: "queue_len": 0, 2024-09-21T03:42:37.393 INFO:teuthology.orchestra.run.smithi114.stdout: "complete_latency": { 2024-09-21T03:42:37.393 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 1589, 2024-09-21T03:42:37.393 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.046802099, 2024-09-21T03:42:37.393 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000029453 2024-09-21T03:42:37.393 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T03:42:37.393 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T03:42:37.393 INFO:teuthology.orchestra.run.smithi114.stdout: "finisher-PurgeQueue": { 2024-09-21T03:42:37.393 INFO:teuthology.orchestra.run.smithi114.stdout: "queue_len": 0, 2024-09-21T03:42:37.393 INFO:teuthology.orchestra.run.smithi114.stdout: "complete_latency": { 2024-09-21T03:42:37.393 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-09-21T03:42:37.393 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-09-21T03:42:37.393 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-09-21T03:42:37.393 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T03:42:37.394 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T03:42:37.394 INFO:teuthology.orchestra.run.smithi114.stdout: "mds": { 2024-09-21T03:42:37.394 INFO:teuthology.orchestra.run.smithi114.stdout: "request": 0, 2024-09-21T03:42:37.394 INFO:teuthology.orchestra.run.smithi114.stdout: "reply": 0, 2024-09-21T03:42:37.394 INFO:teuthology.orchestra.run.smithi114.stdout: "reply_latency": { 2024-09-21T03:42:37.394 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-09-21T03:42:37.394 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-09-21T03:42:37.394 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-09-21T03:42:37.394 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T03:42:37.394 INFO:teuthology.orchestra.run.smithi114.stdout: "slow_reply": 0, 2024-09-21T03:42:37.394 INFO:teuthology.orchestra.run.smithi114.stdout: "forward": 0, 2024-09-21T03:42:37.394 INFO:teuthology.orchestra.run.smithi114.stdout: "dir_fetch_complete": 0, 2024-09-21T03:42:37.394 INFO:teuthology.orchestra.run.smithi114.stdout: "dir_fetch_keys": 0, 2024-09-21T03:42:37.394 INFO:teuthology.orchestra.run.smithi114.stdout: "dir_commit": 0, 2024-09-21T03:42:37.395 INFO:teuthology.orchestra.run.smithi114.stdout: "dir_split": 0, 2024-09-21T03:42:37.395 INFO:teuthology.orchestra.run.smithi114.stdout: "dir_merge": 0, 2024-09-21T03:42:37.395 INFO:teuthology.orchestra.run.smithi114.stdout: "inodes": 20, 2024-09-21T03:42:37.395 INFO:teuthology.orchestra.run.smithi114.stdout: "inodes_top": 9, 2024-09-21T03:42:37.395 INFO:teuthology.orchestra.run.smithi114.stdout: "inodes_bottom": 11, 2024-09-21T03:42:37.395 INFO:teuthology.orchestra.run.smithi114.stdout: "inodes_pin_tail": 0, 2024-09-21T03:42:37.395 INFO:teuthology.orchestra.run.smithi114.stdout: "inodes_pinned": 7, 2024-09-21T03:42:37.395 INFO:teuthology.orchestra.run.smithi114.stdout: "inodes_expired": 0, 2024-09-21T03:42:37.395 INFO:teuthology.orchestra.run.smithi114.stdout: "inodes_with_caps": 0, 2024-09-21T03:42:37.395 INFO:teuthology.orchestra.run.smithi114.stdout: "caps": 0, 2024-09-21T03:42:37.395 INFO:teuthology.orchestra.run.smithi114.stdout: "subtrees": 2, 2024-09-21T03:42:37.395 INFO:teuthology.orchestra.run.smithi114.stdout: "traverse": 0, 2024-09-21T03:42:37.395 INFO:teuthology.orchestra.run.smithi114.stdout: "traverse_hit": 0, 2024-09-21T03:42:37.395 INFO:teuthology.orchestra.run.smithi114.stdout: "traverse_forward": 0, 2024-09-21T03:42:37.396 INFO:teuthology.orchestra.run.smithi114.stdout: "traverse_discover": 0, 2024-09-21T03:42:37.396 INFO:teuthology.orchestra.run.smithi114.stdout: "traverse_dir_fetch": 0, 2024-09-21T03:42:37.396 INFO:teuthology.orchestra.run.smithi114.stdout: "traverse_remote_ino": 0, 2024-09-21T03:42:37.396 INFO:teuthology.orchestra.run.smithi114.stdout: "traverse_lock": 0, 2024-09-21T03:42:37.396 INFO:teuthology.orchestra.run.smithi114.stdout: "load_cent": 0, 2024-09-21T03:42:37.396 INFO:teuthology.orchestra.run.smithi114.stdout: "q": 0, 2024-09-21T03:42:37.396 INFO:teuthology.orchestra.run.smithi114.stdout: "exported": 0, 2024-09-21T03:42:37.396 INFO:teuthology.orchestra.run.smithi114.stdout: "exported_inodes": 0, 2024-09-21T03:42:37.396 INFO:teuthology.orchestra.run.smithi114.stdout: "imported": 0, 2024-09-21T03:42:37.396 INFO:teuthology.orchestra.run.smithi114.stdout: "imported_inodes": 0, 2024-09-21T03:42:37.396 INFO:teuthology.orchestra.run.smithi114.stdout: "openino_dir_fetch": 0, 2024-09-21T03:42:37.396 INFO:teuthology.orchestra.run.smithi114.stdout: "openino_backtrace_fetch": 0, 2024-09-21T03:42:37.396 INFO:teuthology.orchestra.run.smithi114.stdout: "openino_peer_discover": 0, 2024-09-21T03:42:37.396 INFO:teuthology.orchestra.run.smithi114.stdout: "root_rfiles": 4, 2024-09-21T03:42:37.396 INFO:teuthology.orchestra.run.smithi114.stdout: "root_rbytes": 234, 2024-09-21T03:42:37.397 INFO:teuthology.orchestra.run.smithi114.stdout: "root_rsnaps": 0, 2024-09-21T03:42:37.397 INFO:teuthology.orchestra.run.smithi114.stdout: "scrub_backtrace_fetch": 0, 2024-09-21T03:42:37.397 INFO:teuthology.orchestra.run.smithi114.stdout: "scrub_set_tag": 0, 2024-09-21T03:42:37.397 INFO:teuthology.orchestra.run.smithi114.stdout: "scrub_backtrace_repaired": 0, 2024-09-21T03:42:37.397 INFO:teuthology.orchestra.run.smithi114.stdout: "scrub_inotable_repaired": 0, 2024-09-21T03:42:37.397 INFO:teuthology.orchestra.run.smithi114.stdout: "scrub_dir_inodes": 0, 2024-09-21T03:42:37.397 INFO:teuthology.orchestra.run.smithi114.stdout: "scrub_dir_base_inodes": 0, 2024-09-21T03:42:37.397 INFO:teuthology.orchestra.run.smithi114.stdout: "scrub_dirfrag_rstats": 0, 2024-09-21T03:42:37.397 INFO:teuthology.orchestra.run.smithi114.stdout: "scrub_file_inodes": 0, 2024-09-21T03:42:37.397 INFO:teuthology.orchestra.run.smithi114.stdout: "handle_inode_file_caps": 0, 2024-09-21T03:42:37.397 INFO:teuthology.orchestra.run.smithi114.stdout: "ceph_cap_op_revoke": 0, 2024-09-21T03:42:37.397 INFO:teuthology.orchestra.run.smithi114.stdout: "ceph_cap_op_grant": 0, 2024-09-21T03:42:37.397 INFO:teuthology.orchestra.run.smithi114.stdout: "ceph_cap_op_trunc": 0, 2024-09-21T03:42:37.397 INFO:teuthology.orchestra.run.smithi114.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-21T03:42:37.397 INFO:teuthology.orchestra.run.smithi114.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-21T03:42:37.397 INFO:teuthology.orchestra.run.smithi114.stdout: "handle_client_caps": 0, 2024-09-21T03:42:37.397 INFO:teuthology.orchestra.run.smithi114.stdout: "handle_client_caps_dirty": 0, 2024-09-21T03:42:37.398 INFO:teuthology.orchestra.run.smithi114.stdout: "handle_client_cap_release": 0, 2024-09-21T03:42:37.398 INFO:teuthology.orchestra.run.smithi114.stdout: "process_request_cap_release": 0 2024-09-21T03:42:37.398 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T03:42:37.398 INFO:teuthology.orchestra.run.smithi114.stdout: "mds_cache": { 2024-09-21T03:42:37.398 INFO:teuthology.orchestra.run.smithi114.stdout: "dir_update": 0, 2024-09-21T03:42:37.398 INFO:teuthology.orchestra.run.smithi114.stdout: "dir_update_receipt": 0, 2024-09-21T03:42:37.398 INFO:teuthology.orchestra.run.smithi114.stdout: "dir_try_discover": 0, 2024-09-21T03:42:37.398 INFO:teuthology.orchestra.run.smithi114.stdout: "dir_send_discover": 0, 2024-09-21T03:42:37.398 INFO:teuthology.orchestra.run.smithi114.stdout: "dir_handle_discover": 0, 2024-09-21T03:42:37.398 INFO:teuthology.orchestra.run.smithi114.stdout: "num_strays": 0, 2024-09-21T03:42:37.398 INFO:teuthology.orchestra.run.smithi114.stdout: "num_strays_delayed": 0, 2024-09-21T03:42:37.398 INFO:teuthology.orchestra.run.smithi114.stdout: "num_strays_enqueuing": 0, 2024-09-21T03:42:37.398 INFO:teuthology.orchestra.run.smithi114.stdout: "strays_created": 0, 2024-09-21T03:42:37.398 INFO:teuthology.orchestra.run.smithi114.stdout: "strays_enqueued": 0, 2024-09-21T03:42:37.398 INFO:teuthology.orchestra.run.smithi114.stdout: "strays_reintegrated": 0, 2024-09-21T03:42:37.399 INFO:teuthology.orchestra.run.smithi114.stdout: "strays_migrated": 0, 2024-09-21T03:42:37.399 INFO:teuthology.orchestra.run.smithi114.stdout: "num_recovering_processing": 0, 2024-09-21T03:42:37.399 INFO:teuthology.orchestra.run.smithi114.stdout: "num_recovering_enqueued": 0, 2024-09-21T03:42:37.399 INFO:teuthology.orchestra.run.smithi114.stdout: "num_recovering_prioritized": 0, 2024-09-21T03:42:37.399 INFO:teuthology.orchestra.run.smithi114.stdout: "recovery_started": 0, 2024-09-21T03:42:37.399 INFO:teuthology.orchestra.run.smithi114.stdout: "recovery_completed": 0, 2024-09-21T03:42:37.399 INFO:teuthology.orchestra.run.smithi114.stdout: "ireq_quiesce_path": 0, 2024-09-21T03:42:37.399 INFO:teuthology.orchestra.run.smithi114.stdout: "ireq_quiesce_inode": 0, 2024-09-21T03:42:37.399 INFO:teuthology.orchestra.run.smithi114.stdout: "ireq_enqueue_scrub": 0, 2024-09-21T03:42:37.399 INFO:teuthology.orchestra.run.smithi114.stdout: "ireq_exportdir": 0, 2024-09-21T03:42:37.399 INFO:teuthology.orchestra.run.smithi114.stdout: "ireq_flush": 0, 2024-09-21T03:42:37.399 INFO:teuthology.orchestra.run.smithi114.stdout: "ireq_fragmentdir": 0, 2024-09-21T03:42:37.399 INFO:teuthology.orchestra.run.smithi114.stdout: "ireq_fragstats": 0, 2024-09-21T03:42:37.399 INFO:teuthology.orchestra.run.smithi114.stdout: "ireq_inodestats": 0, 2024-09-21T03:42:37.399 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_started": 0, 2024-09-21T03:42:37.399 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_succeeded": 0, 2024-09-21T03:42:37.399 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_write_failed": 0 2024-09-21T03:42:37.400 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T03:42:37.400 INFO:teuthology.orchestra.run.smithi114.stdout: "mds_log": { 2024-09-21T03:42:37.400 INFO:teuthology.orchestra.run.smithi114.stdout: "evlrg": 0, 2024-09-21T03:42:37.400 INFO:teuthology.orchestra.run.smithi114.stdout: "evadd": 0, 2024-09-21T03:42:37.400 INFO:teuthology.orchestra.run.smithi114.stdout: "evex": 0, 2024-09-21T03:42:37.400 INFO:teuthology.orchestra.run.smithi114.stdout: "evtrm": 0, 2024-09-21T03:42:37.400 INFO:teuthology.orchestra.run.smithi114.stdout: "ev": 161, 2024-09-21T03:42:37.400 INFO:teuthology.orchestra.run.smithi114.stdout: "evexg": 0, 2024-09-21T03:42:37.400 INFO:teuthology.orchestra.run.smithi114.stdout: "evexd": 0, 2024-09-21T03:42:37.400 INFO:teuthology.orchestra.run.smithi114.stdout: "segadd": 0, 2024-09-21T03:42:37.400 INFO:teuthology.orchestra.run.smithi114.stdout: "segex": 0, 2024-09-21T03:42:37.400 INFO:teuthology.orchestra.run.smithi114.stdout: "segtrm": 0, 2024-09-21T03:42:37.400 INFO:teuthology.orchestra.run.smithi114.stdout: "seg": 2, 2024-09-21T03:42:37.400 INFO:teuthology.orchestra.run.smithi114.stdout: "segmjr": 15, 2024-09-21T03:42:37.400 INFO:teuthology.orchestra.run.smithi114.stdout: "segexg": 0, 2024-09-21T03:42:37.400 INFO:teuthology.orchestra.run.smithi114.stdout: "segexd": 0, 2024-09-21T03:42:37.401 INFO:teuthology.orchestra.run.smithi114.stdout: "expos": 4775340, 2024-09-21T03:42:37.401 INFO:teuthology.orchestra.run.smithi114.stdout: "wrpos": 4786905, 2024-09-21T03:42:37.401 INFO:teuthology.orchestra.run.smithi114.stdout: "rdpos": 4786306, 2024-09-21T03:42:37.401 INFO:teuthology.orchestra.run.smithi114.stdout: "jlat": { 2024-09-21T03:42:37.401 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-09-21T03:42:37.401 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-09-21T03:42:37.401 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-09-21T03:42:37.401 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T03:42:37.401 INFO:teuthology.orchestra.run.smithi114.stdout: "replayed": 161 2024-09-21T03:42:37.401 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T03:42:37.401 INFO:teuthology.orchestra.run.smithi114.stdout: "mds_mem": { 2024-09-21T03:42:37.401 INFO:teuthology.orchestra.run.smithi114.stdout: "ino": 14, 2024-09-21T03:42:37.401 INFO:teuthology.orchestra.run.smithi114.stdout: "ino+": 20, 2024-09-21T03:42:37.401 INFO:teuthology.orchestra.run.smithi114.stdout: "ino-": 6, 2024-09-21T03:42:37.401 INFO:teuthology.orchestra.run.smithi114.stdout: "dir": 9, 2024-09-21T03:42:37.401 INFO:teuthology.orchestra.run.smithi114.stdout: "dir+": 11, 2024-09-21T03:42:37.401 INFO:teuthology.orchestra.run.smithi114.stdout: "dir-": 2, 2024-09-21T03:42:37.402 INFO:teuthology.orchestra.run.smithi114.stdout: "dn": 20, 2024-09-21T03:42:37.402 INFO:teuthology.orchestra.run.smithi114.stdout: "dn+": 25, 2024-09-21T03:42:37.402 INFO:teuthology.orchestra.run.smithi114.stdout: "dn-": 5, 2024-09-21T03:42:37.402 INFO:teuthology.orchestra.run.smithi114.stdout: "cap": 0, 2024-09-21T03:42:37.402 INFO:teuthology.orchestra.run.smithi114.stdout: "cap+": 0, 2024-09-21T03:42:37.402 INFO:teuthology.orchestra.run.smithi114.stdout: "cap-": 0, 2024-09-21T03:42:37.402 INFO:teuthology.orchestra.run.smithi114.stdout: "rss": 116752, 2024-09-21T03:42:37.402 INFO:teuthology.orchestra.run.smithi114.stdout: "heap": 316420096 2024-09-21T03:42:37.402 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T03:42:37.402 INFO:teuthology.orchestra.run.smithi114.stdout: "mds_server": { 2024-09-21T03:42:37.402 INFO:teuthology.orchestra.run.smithi114.stdout: "dispatch_client_request": 0, 2024-09-21T03:42:37.402 INFO:teuthology.orchestra.run.smithi114.stdout: "dispatch_server_request": 0, 2024-09-21T03:42:37.402 INFO:teuthology.orchestra.run.smithi114.stdout: "handle_client_request": 0, 2024-09-21T03:42:37.402 INFO:teuthology.orchestra.run.smithi114.stdout: "handle_client_session": 0, 2024-09-21T03:42:37.402 INFO:teuthology.orchestra.run.smithi114.stdout: "handle_peer_request": 0, 2024-09-21T03:42:37.402 INFO:teuthology.orchestra.run.smithi114.stdout: "req_create_latency": { 2024-09-21T03:42:37.403 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-09-21T03:42:37.403 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-09-21T03:42:37.403 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-09-21T03:42:37.403 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T03:42:37.403 INFO:teuthology.orchestra.run.smithi114.stdout: "req_getattr_latency": { 2024-09-21T03:42:37.403 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-09-21T03:42:37.403 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-09-21T03:42:37.403 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-09-21T03:42:37.403 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T03:42:37.403 INFO:teuthology.orchestra.run.smithi114.stdout: "req_getfilelock_latency": { 2024-09-21T03:42:37.403 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-09-21T03:42:37.403 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-09-21T03:42:37.403 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-09-21T03:42:37.403 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T03:42:37.404 INFO:teuthology.orchestra.run.smithi114.stdout: "req_link_latency": { 2024-09-21T03:42:37.404 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-09-21T03:42:37.404 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-09-21T03:42:37.404 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-09-21T03:42:37.404 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T03:42:37.404 INFO:teuthology.orchestra.run.smithi114.stdout: "req_lookup_latency": { 2024-09-21T03:42:37.404 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-09-21T03:42:37.404 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-09-21T03:42:37.404 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-09-21T03:42:37.404 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T03:42:37.404 INFO:teuthology.orchestra.run.smithi114.stdout: "req_lookuphash_latency": { 2024-09-21T03:42:37.404 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-09-21T03:42:37.404 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-09-21T03:42:37.404 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-09-21T03:42:37.404 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T03:42:37.404 INFO:teuthology.orchestra.run.smithi114.stdout: "req_lookupino_latency": { 2024-09-21T03:42:37.405 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-09-21T03:42:37.405 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-09-21T03:42:37.405 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-09-21T03:42:37.405 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T03:42:37.405 INFO:teuthology.orchestra.run.smithi114.stdout: "req_lookupname_latency": { 2024-09-21T03:42:37.405 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-09-21T03:42:37.405 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-09-21T03:42:37.405 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-09-21T03:42:37.405 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T03:42:37.405 INFO:teuthology.orchestra.run.smithi114.stdout: "req_lookupparent_latency": { 2024-09-21T03:42:37.405 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-09-21T03:42:37.405 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-09-21T03:42:37.405 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-09-21T03:42:37.405 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T03:42:37.406 INFO:teuthology.orchestra.run.smithi114.stdout: "req_lookupsnap_latency": { 2024-09-21T03:42:37.406 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-09-21T03:42:37.406 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-09-21T03:42:37.406 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-09-21T03:42:37.406 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T03:42:37.406 INFO:teuthology.orchestra.run.smithi114.stdout: "req_lssnap_latency": { 2024-09-21T03:42:37.406 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-09-21T03:42:37.406 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-09-21T03:42:37.407 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-09-21T03:42:37.407 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T03:42:37.407 INFO:teuthology.orchestra.run.smithi114.stdout: "req_mkdir_latency": { 2024-09-21T03:42:37.407 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-09-21T03:42:37.407 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-09-21T03:42:37.407 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-09-21T03:42:37.407 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T03:42:37.407 INFO:teuthology.orchestra.run.smithi114.stdout: "req_mknod_latency": { 2024-09-21T03:42:37.407 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-09-21T03:42:37.407 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-09-21T03:42:37.407 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-09-21T03:42:37.407 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T03:42:37.408 INFO:teuthology.orchestra.run.smithi114.stdout: "req_mksnap_latency": { 2024-09-21T03:42:37.408 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-09-21T03:42:37.408 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-09-21T03:42:37.408 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-09-21T03:42:37.408 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T03:42:37.408 INFO:teuthology.orchestra.run.smithi114.stdout: "req_open_latency": { 2024-09-21T03:42:37.408 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-09-21T03:42:37.408 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-09-21T03:42:37.408 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-09-21T03:42:37.408 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T03:42:37.408 INFO:teuthology.orchestra.run.smithi114.stdout: "req_readdir_latency": { 2024-09-21T03:42:37.408 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-09-21T03:42:37.408 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-09-21T03:42:37.408 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-09-21T03:42:37.409 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T03:42:37.409 INFO:teuthology.orchestra.run.smithi114.stdout: "req_rename_latency": { 2024-09-21T03:42:37.409 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-09-21T03:42:37.409 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-09-21T03:42:37.409 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-09-21T03:42:37.409 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T03:42:37.409 INFO:teuthology.orchestra.run.smithi114.stdout: "req_renamesnap_latency": { 2024-09-21T03:42:37.409 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-09-21T03:42:37.409 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-09-21T03:42:37.409 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-09-21T03:42:37.409 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T03:42:37.409 INFO:teuthology.orchestra.run.smithi114.stdout: "req_snapdiff_latency": { 2024-09-21T03:42:37.409 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-09-21T03:42:37.409 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-09-21T03:42:37.409 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-09-21T03:42:37.409 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T03:42:37.410 INFO:teuthology.orchestra.run.smithi114.stdout: "req_rmdir_latency": { 2024-09-21T03:42:37.410 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-09-21T03:42:37.410 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-09-21T03:42:37.410 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-09-21T03:42:37.410 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T03:42:37.410 INFO:teuthology.orchestra.run.smithi114.stdout: "req_rmsnap_latency": { 2024-09-21T03:42:37.410 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-09-21T03:42:37.410 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-09-21T03:42:37.410 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-09-21T03:42:37.410 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T03:42:37.410 INFO:teuthology.orchestra.run.smithi114.stdout: "req_rmxattr_latency": { 2024-09-21T03:42:37.410 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-09-21T03:42:37.410 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-09-21T03:42:37.410 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-09-21T03:42:37.410 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T03:42:37.411 INFO:teuthology.orchestra.run.smithi114.stdout: "req_setattr_latency": { 2024-09-21T03:42:37.411 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-09-21T03:42:37.411 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-09-21T03:42:37.411 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-09-21T03:42:37.411 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T03:42:37.411 INFO:teuthology.orchestra.run.smithi114.stdout: "req_setdirlayout_latency": { 2024-09-21T03:42:37.411 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-09-21T03:42:37.411 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-09-21T03:42:37.411 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-09-21T03:42:37.411 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T03:42:37.411 INFO:teuthology.orchestra.run.smithi114.stdout: "req_setfilelock_latency": { 2024-09-21T03:42:37.411 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-09-21T03:42:37.411 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-09-21T03:42:37.411 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-09-21T03:42:37.412 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T03:42:37.412 INFO:teuthology.orchestra.run.smithi114.stdout: "req_setlayout_latency": { 2024-09-21T03:42:37.412 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-09-21T03:42:37.412 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-09-21T03:42:37.412 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-09-21T03:42:37.412 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T03:42:37.412 INFO:teuthology.orchestra.run.smithi114.stdout: "req_setxattr_latency": { 2024-09-21T03:42:37.412 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-09-21T03:42:37.412 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-09-21T03:42:37.412 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-09-21T03:42:37.412 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T03:42:37.412 INFO:teuthology.orchestra.run.smithi114.stdout: "req_symlink_latency": { 2024-09-21T03:42:37.412 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-09-21T03:42:37.412 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-09-21T03:42:37.412 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-09-21T03:42:37.412 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T03:42:37.413 INFO:teuthology.orchestra.run.smithi114.stdout: "req_unlink_latency": { 2024-09-21T03:42:37.413 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-09-21T03:42:37.413 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-09-21T03:42:37.413 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-09-21T03:42:37.413 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T03:42:37.413 INFO:teuthology.orchestra.run.smithi114.stdout: "cap_revoke_eviction": 0, 2024-09-21T03:42:37.413 INFO:teuthology.orchestra.run.smithi114.stdout: "cap_acquisition_throttle": 0, 2024-09-21T03:42:37.413 INFO:teuthology.orchestra.run.smithi114.stdout: "req_getvxattr_latency": { 2024-09-21T03:42:37.413 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-09-21T03:42:37.413 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-09-21T03:42:37.413 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-09-21T03:42:37.413 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T03:42:37.413 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T03:42:37.413 INFO:teuthology.orchestra.run.smithi114.stdout: "mds_sessions": { 2024-09-21T03:42:37.413 INFO:teuthology.orchestra.run.smithi114.stdout: "session_count": 1, 2024-09-21T03:42:37.413 INFO:teuthology.orchestra.run.smithi114.stdout: "session_add": 4, 2024-09-21T03:42:37.414 INFO:teuthology.orchestra.run.smithi114.stdout: "session_remove": 3, 2024-09-21T03:42:37.414 INFO:teuthology.orchestra.run.smithi114.stdout: "sessions_open": 2, 2024-09-21T03:42:37.414 INFO:teuthology.orchestra.run.smithi114.stdout: "sessions_stale": 0, 2024-09-21T03:42:37.414 INFO:teuthology.orchestra.run.smithi114.stdout: "total_load": 0, 2024-09-21T03:42:37.414 INFO:teuthology.orchestra.run.smithi114.stdout: "average_load": 0, 2024-09-21T03:42:37.414 INFO:teuthology.orchestra.run.smithi114.stdout: "avg_session_uptime": 2312, 2024-09-21T03:42:37.414 INFO:teuthology.orchestra.run.smithi114.stdout: "mdthresh_evicted": 0 2024-09-21T03:42:37.414 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T03:42:37.414 INFO:teuthology.orchestra.run.smithi114.stdout: "mempool": { 2024-09-21T03:42:37.414 INFO:teuthology.orchestra.run.smithi114.stdout: "bloom_filter_bytes": 0, 2024-09-21T03:42:37.414 INFO:teuthology.orchestra.run.smithi114.stdout: "bloom_filter_items": 0, 2024-09-21T03:42:37.414 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_alloc_bytes": 0, 2024-09-21T03:42:37.414 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_alloc_items": 0, 2024-09-21T03:42:37.414 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_cache_data_bytes": 0, 2024-09-21T03:42:37.414 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_cache_data_items": 0, 2024-09-21T03:42:37.415 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-21T03:42:37.415 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_cache_onode_items": 0, 2024-09-21T03:42:37.415 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-21T03:42:37.415 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_cache_meta_items": 0, 2024-09-21T03:42:37.415 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_cache_other_bytes": 0, 2024-09-21T03:42:37.415 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_cache_other_items": 0, 2024-09-21T03:42:37.415 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-21T03:42:37.415 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_cache_buffer_items": 0, 2024-09-21T03:42:37.415 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_extent_bytes": 0, 2024-09-21T03:42:37.415 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_extent_items": 0, 2024-09-21T03:42:37.415 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_blob_bytes": 0, 2024-09-21T03:42:37.415 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_blob_items": 0, 2024-09-21T03:42:37.415 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-21T03:42:37.415 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_shared_blob_items": 0, 2024-09-21T03:42:37.415 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-21T03:42:37.415 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_inline_bl_items": 0, 2024-09-21T03:42:37.416 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_fsck_bytes": 0, 2024-09-21T03:42:37.416 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_fsck_items": 0, 2024-09-21T03:42:37.416 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_txc_bytes": 0, 2024-09-21T03:42:37.416 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_txc_items": 0, 2024-09-21T03:42:37.416 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-21T03:42:37.416 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_writing_deferred_items": 0, 2024-09-21T03:42:37.416 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_writing_bytes": 0, 2024-09-21T03:42:37.416 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_writing_items": 0, 2024-09-21T03:42:37.416 INFO:teuthology.orchestra.run.smithi114.stdout: "bluefs_bytes": 0, 2024-09-21T03:42:37.416 INFO:teuthology.orchestra.run.smithi114.stdout: "bluefs_items": 0, 2024-09-21T03:42:37.416 INFO:teuthology.orchestra.run.smithi114.stdout: "bluefs_file_reader_bytes": 0, 2024-09-21T03:42:37.416 INFO:teuthology.orchestra.run.smithi114.stdout: "bluefs_file_reader_items": 0, 2024-09-21T03:42:37.416 INFO:teuthology.orchestra.run.smithi114.stdout: "bluefs_file_writer_bytes": 0, 2024-09-21T03:42:37.416 INFO:teuthology.orchestra.run.smithi114.stdout: "bluefs_file_writer_items": 0, 2024-09-21T03:42:37.416 INFO:teuthology.orchestra.run.smithi114.stdout: "buffer_anon_bytes": 189279, 2024-09-21T03:42:37.416 INFO:teuthology.orchestra.run.smithi114.stdout: "buffer_anon_items": 49, 2024-09-21T03:42:37.416 INFO:teuthology.orchestra.run.smithi114.stdout: "buffer_meta_bytes": 0, 2024-09-21T03:42:37.417 INFO:teuthology.orchestra.run.smithi114.stdout: "buffer_meta_items": 0, 2024-09-21T03:42:37.417 INFO:teuthology.orchestra.run.smithi114.stdout: "osd_bytes": 0, 2024-09-21T03:42:37.417 INFO:teuthology.orchestra.run.smithi114.stdout: "osd_items": 0, 2024-09-21T03:42:37.417 INFO:teuthology.orchestra.run.smithi114.stdout: "osd_mapbl_bytes": 0, 2024-09-21T03:42:37.417 INFO:teuthology.orchestra.run.smithi114.stdout: "osd_mapbl_items": 0, 2024-09-21T03:42:37.417 INFO:teuthology.orchestra.run.smithi114.stdout: "osd_pglog_bytes": 0, 2024-09-21T03:42:37.417 INFO:teuthology.orchestra.run.smithi114.stdout: "osd_pglog_items": 0, 2024-09-21T03:42:37.417 INFO:teuthology.orchestra.run.smithi114.stdout: "osdmap_bytes": 8760, 2024-09-21T03:42:37.417 INFO:teuthology.orchestra.run.smithi114.stdout: "osdmap_items": 164, 2024-09-21T03:42:37.417 INFO:teuthology.orchestra.run.smithi114.stdout: "osdmap_mapping_bytes": 0, 2024-09-21T03:42:37.417 INFO:teuthology.orchestra.run.smithi114.stdout: "osdmap_mapping_items": 0, 2024-09-21T03:42:37.417 INFO:teuthology.orchestra.run.smithi114.stdout: "pgmap_bytes": 0, 2024-09-21T03:42:37.417 INFO:teuthology.orchestra.run.smithi114.stdout: "pgmap_items": 0, 2024-09-21T03:42:37.417 INFO:teuthology.orchestra.run.smithi114.stdout: "mds_co_bytes": 57650, 2024-09-21T03:42:37.417 INFO:teuthology.orchestra.run.smithi114.stdout: "mds_co_items": 333, 2024-09-21T03:42:37.418 INFO:teuthology.orchestra.run.smithi114.stdout: "unittest_1_bytes": 0, 2024-09-21T03:42:37.418 INFO:teuthology.orchestra.run.smithi114.stdout: "unittest_1_items": 0, 2024-09-21T03:42:37.418 INFO:teuthology.orchestra.run.smithi114.stdout: "unittest_2_bytes": 0, 2024-09-21T03:42:37.418 INFO:teuthology.orchestra.run.smithi114.stdout: "unittest_2_items": 0 2024-09-21T03:42:37.418 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T03:42:37.418 INFO:teuthology.orchestra.run.smithi114.stdout: "objecter": { 2024-09-21T03:42:37.418 INFO:teuthology.orchestra.run.smithi114.stdout: "op_active": 0, 2024-09-21T03:42:37.418 INFO:teuthology.orchestra.run.smithi114.stdout: "op_laggy": 0, 2024-09-21T03:42:37.418 INFO:teuthology.orchestra.run.smithi114.stdout: "op_send": 957, 2024-09-21T03:42:37.418 INFO:teuthology.orchestra.run.smithi114.stdout: "op_send_bytes": 26, 2024-09-21T03:42:37.418 INFO:teuthology.orchestra.run.smithi114.stdout: "op_resend": 4, 2024-09-21T03:42:37.418 INFO:teuthology.orchestra.run.smithi114.stdout: "op_reply": 957, 2024-09-21T03:42:37.419 INFO:teuthology.orchestra.run.smithi114.stdout: "op_latency": { 2024-09-21T03:42:37.419 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 957, 2024-09-21T03:42:37.419 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.817996046, 2024-09-21T03:42:37.419 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000854750 2024-09-21T03:42:37.419 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T03:42:37.419 INFO:teuthology.orchestra.run.smithi114.stdout: "op_inflight": 0, 2024-09-21T03:42:37.419 INFO:teuthology.orchestra.run.smithi114.stdout: "oplen_avg": { 2024-09-21T03:42:37.419 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 957, 2024-09-21T03:42:37.419 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 958 2024-09-21T03:42:37.419 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T03:42:37.419 INFO:teuthology.orchestra.run.smithi114.stdout: "op": 957, 2024-09-21T03:42:37.419 INFO:teuthology.orchestra.run.smithi114.stdout: "op_r": 957, 2024-09-21T03:42:37.419 INFO:teuthology.orchestra.run.smithi114.stdout: "op_w": 0, 2024-09-21T03:42:37.419 INFO:teuthology.orchestra.run.smithi114.stdout: "op_rmw": 0, 2024-09-21T03:42:37.419 INFO:teuthology.orchestra.run.smithi114.stdout: "op_pg": 0, 2024-09-21T03:42:37.420 INFO:teuthology.orchestra.run.smithi114.stdout: "osdop_stat": 576, 2024-09-21T03:42:37.420 INFO:teuthology.orchestra.run.smithi114.stdout: "osdop_create": 0, 2024-09-21T03:42:37.420 INFO:teuthology.orchestra.run.smithi114.stdout: "osdop_read": 378, 2024-09-21T03:42:37.420 INFO:teuthology.orchestra.run.smithi114.stdout: "osdop_write": 0, 2024-09-21T03:42:37.420 INFO:teuthology.orchestra.run.smithi114.stdout: "osdop_writefull": 0, 2024-09-21T03:42:37.420 INFO:teuthology.orchestra.run.smithi114.stdout: "osdop_writesame": 0, 2024-09-21T03:42:37.420 INFO:teuthology.orchestra.run.smithi114.stdout: "osdop_append": 0, 2024-09-21T03:42:37.420 INFO:teuthology.orchestra.run.smithi114.stdout: "osdop_zero": 0, 2024-09-21T03:42:37.420 INFO:teuthology.orchestra.run.smithi114.stdout: "osdop_truncate": 0, 2024-09-21T03:42:37.420 INFO:teuthology.orchestra.run.smithi114.stdout: "osdop_delete": 0, 2024-09-21T03:42:37.420 INFO:teuthology.orchestra.run.smithi114.stdout: "osdop_mapext": 0, 2024-09-21T03:42:37.420 INFO:teuthology.orchestra.run.smithi114.stdout: "osdop_sparse_read": 0, 2024-09-21T03:42:37.420 INFO:teuthology.orchestra.run.smithi114.stdout: "osdop_clonerange": 0, 2024-09-21T03:42:37.420 INFO:teuthology.orchestra.run.smithi114.stdout: "osdop_getxattr": 2, 2024-09-21T03:42:37.420 INFO:teuthology.orchestra.run.smithi114.stdout: "osdop_setxattr": 0, 2024-09-21T03:42:37.420 INFO:teuthology.orchestra.run.smithi114.stdout: "osdop_cmpxattr": 0, 2024-09-21T03:42:37.421 INFO:teuthology.orchestra.run.smithi114.stdout: "osdop_rmxattr": 0, 2024-09-21T03:42:37.421 INFO:teuthology.orchestra.run.smithi114.stdout: "osdop_resetxattrs": 0, 2024-09-21T03:42:37.421 INFO:teuthology.orchestra.run.smithi114.stdout: "osdop_call": 0, 2024-09-21T03:42:37.421 INFO:teuthology.orchestra.run.smithi114.stdout: "osdop_watch": 0, 2024-09-21T03:42:37.421 INFO:teuthology.orchestra.run.smithi114.stdout: "osdop_notify": 0, 2024-09-21T03:42:37.421 INFO:teuthology.orchestra.run.smithi114.stdout: "osdop_src_cmpxattr": 0, 2024-09-21T03:42:37.421 INFO:teuthology.orchestra.run.smithi114.stdout: "osdop_pgls": 0, 2024-09-21T03:42:37.421 INFO:teuthology.orchestra.run.smithi114.stdout: "osdop_pgls_filter": 0, 2024-09-21T03:42:37.421 INFO:teuthology.orchestra.run.smithi114.stdout: "osdop_other": 0, 2024-09-21T03:42:37.421 INFO:teuthology.orchestra.run.smithi114.stdout: "linger_active": 0, 2024-09-21T03:42:37.421 INFO:teuthology.orchestra.run.smithi114.stdout: "linger_send": 0, 2024-09-21T03:42:37.421 INFO:teuthology.orchestra.run.smithi114.stdout: "linger_resend": 0, 2024-09-21T03:42:37.421 INFO:teuthology.orchestra.run.smithi114.stdout: "linger_ping": 0, 2024-09-21T03:42:37.421 INFO:teuthology.orchestra.run.smithi114.stdout: "poolop_active": 0, 2024-09-21T03:42:37.421 INFO:teuthology.orchestra.run.smithi114.stdout: "poolop_send": 0, 2024-09-21T03:42:37.422 INFO:teuthology.orchestra.run.smithi114.stdout: "poolop_resend": 0, 2024-09-21T03:42:37.422 INFO:teuthology.orchestra.run.smithi114.stdout: "poolstat_active": 0, 2024-09-21T03:42:37.422 INFO:teuthology.orchestra.run.smithi114.stdout: "poolstat_send": 0, 2024-09-21T03:42:37.422 INFO:teuthology.orchestra.run.smithi114.stdout: "poolstat_resend": 0, 2024-09-21T03:42:37.422 INFO:teuthology.orchestra.run.smithi114.stdout: "statfs_active": 0, 2024-09-21T03:42:37.422 INFO:teuthology.orchestra.run.smithi114.stdout: "statfs_send": 0, 2024-09-21T03:42:37.422 INFO:teuthology.orchestra.run.smithi114.stdout: "statfs_resend": 0, 2024-09-21T03:42:37.422 INFO:teuthology.orchestra.run.smithi114.stdout: "command_active": 0, 2024-09-21T03:42:37.422 INFO:teuthology.orchestra.run.smithi114.stdout: "command_send": 0, 2024-09-21T03:42:37.422 INFO:teuthology.orchestra.run.smithi114.stdout: "command_resend": 0, 2024-09-21T03:42:37.422 INFO:teuthology.orchestra.run.smithi114.stdout: "map_epoch": 84, 2024-09-21T03:42:37.422 INFO:teuthology.orchestra.run.smithi114.stdout: "map_full": 0, 2024-09-21T03:42:37.422 INFO:teuthology.orchestra.run.smithi114.stdout: "map_inc": 4, 2024-09-21T03:42:37.423 INFO:teuthology.orchestra.run.smithi114.stdout: "osd_sessions": 5, 2024-09-21T03:42:37.423 INFO:teuthology.orchestra.run.smithi114.stdout: "osd_session_open": 5, 2024-09-21T03:42:37.423 INFO:teuthology.orchestra.run.smithi114.stdout: "osd_session_close": 0, 2024-09-21T03:42:37.423 INFO:teuthology.orchestra.run.smithi114.stdout: "osd_laggy": 0, 2024-09-21T03:42:37.423 INFO:teuthology.orchestra.run.smithi114.stdout: "omap_wr": 0, 2024-09-21T03:42:37.423 INFO:teuthology.orchestra.run.smithi114.stdout: "omap_rd": 2, 2024-09-21T03:42:37.423 INFO:teuthology.orchestra.run.smithi114.stdout: "omap_del": 0 2024-09-21T03:42:37.423 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T03:42:37.423 INFO:teuthology.orchestra.run.smithi114.stdout: "oft": { 2024-09-21T03:42:37.423 INFO:teuthology.orchestra.run.smithi114.stdout: "omap_total_objs": 0, 2024-09-21T03:42:37.423 INFO:teuthology.orchestra.run.smithi114.stdout: "omap_total_kv_pairs": 0, 2024-09-21T03:42:37.423 INFO:teuthology.orchestra.run.smithi114.stdout: "omap_total_updates": 0, 2024-09-21T03:42:37.423 INFO:teuthology.orchestra.run.smithi114.stdout: "omap_total_removes": 0 2024-09-21T03:42:37.423 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T03:42:37.424 INFO:teuthology.orchestra.run.smithi114.stdout: "purge_queue": { 2024-09-21T03:42:37.424 INFO:teuthology.orchestra.run.smithi114.stdout: "pq_executing_ops": 0, 2024-09-21T03:42:37.424 INFO:teuthology.orchestra.run.smithi114.stdout: "pq_executing_ops_high_water": 0, 2024-09-21T03:42:37.424 INFO:teuthology.orchestra.run.smithi114.stdout: "pq_executing": 0, 2024-09-21T03:42:37.424 INFO:teuthology.orchestra.run.smithi114.stdout: "pq_executing_high_water": 0, 2024-09-21T03:42:37.424 INFO:teuthology.orchestra.run.smithi114.stdout: "pq_executed_ops": 0, 2024-09-21T03:42:37.424 INFO:teuthology.orchestra.run.smithi114.stdout: "pq_executed": 0, 2024-09-21T03:42:37.424 INFO:teuthology.orchestra.run.smithi114.stdout: "pq_item_in_journal": 0 2024-09-21T03:42:37.424 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T03:42:37.424 INFO:teuthology.orchestra.run.smithi114.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-21T03:42:37.424 INFO:teuthology.orchestra.run.smithi114.stdout: "val": 0, 2024-09-21T03:42:37.424 INFO:teuthology.orchestra.run.smithi114.stdout: "max": 104857600, 2024-09-21T03:42:37.424 INFO:teuthology.orchestra.run.smithi114.stdout: "get_started": 0, 2024-09-21T03:42:37.424 INFO:teuthology.orchestra.run.smithi114.stdout: "get": 1071, 2024-09-21T03:42:37.425 INFO:teuthology.orchestra.run.smithi114.stdout: "get_sum": 1585489, 2024-09-21T03:42:37.425 INFO:teuthology.orchestra.run.smithi114.stdout: "get_or_fail_fail": 0, 2024-09-21T03:42:37.425 INFO:teuthology.orchestra.run.smithi114.stdout: "get_or_fail_success": 1071, 2024-09-21T03:42:37.425 INFO:teuthology.orchestra.run.smithi114.stdout: "take": 0, 2024-09-21T03:42:37.425 INFO:teuthology.orchestra.run.smithi114.stdout: "take_sum": 0, 2024-09-21T03:42:37.425 INFO:teuthology.orchestra.run.smithi114.stdout: "put": 1071, 2024-09-21T03:42:37.425 INFO:teuthology.orchestra.run.smithi114.stdout: "put_sum": 1585489, 2024-09-21T03:42:37.425 INFO:teuthology.orchestra.run.smithi114.stdout: "wait": { 2024-09-21T03:42:37.425 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-09-21T03:42:37.425 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-09-21T03:42:37.425 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-09-21T03:42:37.426 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T03:42:37.426 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T03:42:37.426 INFO:teuthology.orchestra.run.smithi114.stdout: "throttle-objecter_bytes": { 2024-09-21T03:42:37.426 INFO:teuthology.orchestra.run.smithi114.stdout: "val": 0, 2024-09-21T03:42:37.426 INFO:teuthology.orchestra.run.smithi114.stdout: "max": 104857600, 2024-09-21T03:42:37.426 INFO:teuthology.orchestra.run.smithi114.stdout: "get_started": 0, 2024-09-21T03:42:37.426 INFO:teuthology.orchestra.run.smithi114.stdout: "get": 0, 2024-09-21T03:42:37.426 INFO:teuthology.orchestra.run.smithi114.stdout: "get_sum": 0, 2024-09-21T03:42:37.426 INFO:teuthology.orchestra.run.smithi114.stdout: "get_or_fail_fail": 0, 2024-09-21T03:42:37.426 INFO:teuthology.orchestra.run.smithi114.stdout: "get_or_fail_success": 0, 2024-09-21T03:42:37.426 INFO:teuthology.orchestra.run.smithi114.stdout: "take": 957, 2024-09-21T03:42:37.426 INFO:teuthology.orchestra.run.smithi114.stdout: "take_sum": 592611, 2024-09-21T03:42:37.426 INFO:teuthology.orchestra.run.smithi114.stdout: "put": 957, 2024-09-21T03:42:37.427 INFO:teuthology.orchestra.run.smithi114.stdout: "put_sum": 592611, 2024-09-21T03:42:37.427 INFO:teuthology.orchestra.run.smithi114.stdout: "wait": { 2024-09-21T03:42:37.427 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-09-21T03:42:37.427 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-09-21T03:42:37.427 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-09-21T03:42:37.427 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T03:42:37.427 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T03:42:37.427 INFO:teuthology.orchestra.run.smithi114.stdout: "throttle-objecter_ops": { 2024-09-21T03:42:37.427 INFO:teuthology.orchestra.run.smithi114.stdout: "val": 0, 2024-09-21T03:42:37.427 INFO:teuthology.orchestra.run.smithi114.stdout: "max": 1024, 2024-09-21T03:42:37.427 INFO:teuthology.orchestra.run.smithi114.stdout: "get_started": 0, 2024-09-21T03:42:37.427 INFO:teuthology.orchestra.run.smithi114.stdout: "get": 0, 2024-09-21T03:42:37.427 INFO:teuthology.orchestra.run.smithi114.stdout: "get_sum": 0, 2024-09-21T03:42:37.428 INFO:teuthology.orchestra.run.smithi114.stdout: "get_or_fail_fail": 0, 2024-09-21T03:42:37.428 INFO:teuthology.orchestra.run.smithi114.stdout: "get_or_fail_success": 0, 2024-09-21T03:42:37.428 INFO:teuthology.orchestra.run.smithi114.stdout: "take": 957, 2024-09-21T03:42:37.428 INFO:teuthology.orchestra.run.smithi114.stdout: "take_sum": 957, 2024-09-21T03:42:37.428 INFO:teuthology.orchestra.run.smithi114.stdout: "put": 957, 2024-09-21T03:42:37.428 INFO:teuthology.orchestra.run.smithi114.stdout: "put_sum": 957, 2024-09-21T03:42:37.428 INFO:teuthology.orchestra.run.smithi114.stdout: "wait": { 2024-09-21T03:42:37.428 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-09-21T03:42:37.428 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-09-21T03:42:37.428 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-09-21T03:42:37.428 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T03:42:37.428 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T03:42:37.428 INFO:teuthology.orchestra.run.smithi114.stdout: "throttle-write_buf_throttle": { 2024-09-21T03:42:37.428 INFO:teuthology.orchestra.run.smithi114.stdout: "val": 0, 2024-09-21T03:42:37.429 INFO:teuthology.orchestra.run.smithi114.stdout: "max": 3758096384, 2024-09-21T03:42:37.429 INFO:teuthology.orchestra.run.smithi114.stdout: "get_started": 0, 2024-09-21T03:42:37.429 INFO:teuthology.orchestra.run.smithi114.stdout: "get": 0, 2024-09-21T03:42:37.429 INFO:teuthology.orchestra.run.smithi114.stdout: "get_sum": 0, 2024-09-21T03:42:37.429 INFO:teuthology.orchestra.run.smithi114.stdout: "get_or_fail_fail": 0, 2024-09-21T03:42:37.429 INFO:teuthology.orchestra.run.smithi114.stdout: "get_or_fail_success": 0, 2024-09-21T03:42:37.429 INFO:teuthology.orchestra.run.smithi114.stdout: "take": 0, 2024-09-21T03:42:37.429 INFO:teuthology.orchestra.run.smithi114.stdout: "take_sum": 0, 2024-09-21T03:42:37.429 INFO:teuthology.orchestra.run.smithi114.stdout: "put": 0, 2024-09-21T03:42:37.429 INFO:teuthology.orchestra.run.smithi114.stdout: "put_sum": 0, 2024-09-21T03:42:37.429 INFO:teuthology.orchestra.run.smithi114.stdout: "wait": { 2024-09-21T03:42:37.429 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-09-21T03:42:37.429 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-09-21T03:42:37.429 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-09-21T03:42:37.430 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T03:42:37.430 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T03:42:37.430 INFO:teuthology.orchestra.run.smithi114.stdout: "throttle-write_buf_throttle-0x564cac839520": { 2024-09-21T03:42:37.430 INFO:teuthology.orchestra.run.smithi114.stdout: "val": 0, 2024-09-21T03:42:37.430 INFO:teuthology.orchestra.run.smithi114.stdout: "max": 3758096384, 2024-09-21T03:42:37.430 INFO:teuthology.orchestra.run.smithi114.stdout: "get_started": 0, 2024-09-21T03:42:37.430 INFO:teuthology.orchestra.run.smithi114.stdout: "get": 0, 2024-09-21T03:42:37.430 INFO:teuthology.orchestra.run.smithi114.stdout: "get_sum": 0, 2024-09-21T03:42:37.430 INFO:teuthology.orchestra.run.smithi114.stdout: "get_or_fail_fail": 0, 2024-09-21T03:42:37.430 INFO:teuthology.orchestra.run.smithi114.stdout: "get_or_fail_success": 0, 2024-09-21T03:42:37.430 INFO:teuthology.orchestra.run.smithi114.stdout: "take": 0, 2024-09-21T03:42:37.430 INFO:teuthology.orchestra.run.smithi114.stdout: "take_sum": 0, 2024-09-21T03:42:37.431 INFO:teuthology.orchestra.run.smithi114.stdout: "put": 0, 2024-09-21T03:42:37.431 INFO:teuthology.orchestra.run.smithi114.stdout: "put_sum": 0, 2024-09-21T03:42:37.431 INFO:teuthology.orchestra.run.smithi114.stdout: "wait": { 2024-09-21T03:42:37.431 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-09-21T03:42:37.431 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-09-21T03:42:37.431 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-09-21T03:42:37.431 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T03:42:37.431 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T03:42:37.431 INFO:teuthology.orchestra.run.smithi114.stdout: "trackedop": { 2024-09-21T03:42:37.431 INFO:teuthology.orchestra.run.smithi114.stdout: "slow_ops_count": 0 2024-09-21T03:42:37.431 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T03:42:37.431 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-21T03:42:38.268 INFO:tasks.check_counter:Daemon mds.c mds.exported=0 2024-09-21T03:42:38.269 INFO:tasks.check_counter:Daemon mds.c mds.imported=0 2024-09-21T03:42:38.269 DEBUG:tasks.check_counter:Getting stats from f 2024-09-21T03:42:38.269 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 7c374098-77c9-11ef-baf5-efdc52797490 -- ceph daemon mds.f perf dump 2024-09-21T03:42:38.514 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/7c374098-77c9-11ef-baf5-efdc52797490/mon.c/config 2024-09-21T03:42:39.024 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-21T03:42:39.024 INFO:teuthology.orchestra.run.smithi114.stdout: "AsyncMessenger::Worker-0": { 2024-09-21T03:42:39.025 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_recv_messages": 404, 2024-09-21T03:42:39.025 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_send_messages": 397, 2024-09-21T03:42:39.025 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_recv_bytes": 925376, 2024-09-21T03:42:39.025 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_send_bytes": 152988, 2024-09-21T03:42:39.025 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_created_connections": 5, 2024-09-21T03:42:39.025 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_active_connections": 1, 2024-09-21T03:42:39.025 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_running_total_time": 0.069387248, 2024-09-21T03:42:39.025 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_running_send_time": 0.026644986, 2024-09-21T03:42:39.025 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_running_recv_time": 0.034460119, 2024-09-21T03:42:39.025 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_running_fast_dispatch_time": 0.012106566, 2024-09-21T03:42:39.025 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T03:42:39.025 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 396, 2024-09-21T03:42:39.025 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.023330610, 2024-09-21T03:42:39.025 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000058915 2024-09-21T03:42:39.025 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T03:42:39.026 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_handle_ack_lat": { 2024-09-21T03:42:39.026 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-09-21T03:42:39.026 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-09-21T03:42:39.026 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-09-21T03:42:39.026 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T03:42:39.026 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_recv_encrypted_bytes": 827936, 2024-09-21T03:42:39.026 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_send_encrypted_bytes": 60768 2024-09-21T03:42:39.026 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T03:42:39.026 INFO:teuthology.orchestra.run.smithi114.stdout: "AsyncMessenger::Worker-1": { 2024-09-21T03:42:39.026 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_recv_messages": 3, 2024-09-21T03:42:39.026 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_send_messages": 76, 2024-09-21T03:42:39.026 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_recv_bytes": 716, 2024-09-21T03:42:39.026 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_send_bytes": 199146, 2024-09-21T03:42:39.026 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_created_connections": 4, 2024-09-21T03:42:39.026 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_active_connections": 3, 2024-09-21T03:42:39.027 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_running_total_time": 0.006733301, 2024-09-21T03:42:39.027 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_running_send_time": 0.004287913, 2024-09-21T03:42:39.027 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_running_recv_time": 0.001568548, 2024-09-21T03:42:39.027 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_running_fast_dispatch_time": 0.000070562, 2024-09-21T03:42:39.027 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T03:42:39.027 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 74, 2024-09-21T03:42:39.027 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.005581177, 2024-09-21T03:42:39.027 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000075421 2024-09-21T03:42:39.027 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T03:42:39.027 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_handle_ack_lat": { 2024-09-21T03:42:39.027 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-09-21T03:42:39.027 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-09-21T03:42:39.027 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-09-21T03:42:39.027 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T03:42:39.028 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_recv_encrypted_bytes": 160, 2024-09-21T03:42:39.028 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_send_encrypted_bytes": 198448 2024-09-21T03:42:39.028 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T03:42:39.028 INFO:teuthology.orchestra.run.smithi114.stdout: "AsyncMessenger::Worker-2": { 2024-09-21T03:42:39.028 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_recv_messages": 500, 2024-09-21T03:42:39.028 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_send_messages": 500, 2024-09-21T03:42:39.028 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_recv_bytes": 132406, 2024-09-21T03:42:39.028 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_send_bytes": 158183, 2024-09-21T03:42:39.028 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_created_connections": 4, 2024-09-21T03:42:39.028 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_active_connections": 3, 2024-09-21T03:42:39.028 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_running_total_time": 0.049646959, 2024-09-21T03:42:39.028 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_running_send_time": 0.021151265, 2024-09-21T03:42:39.028 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_running_recv_time": 0.012630738, 2024-09-21T03:42:39.029 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_running_fast_dispatch_time": 0.013289087, 2024-09-21T03:42:39.029 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T03:42:39.029 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 499, 2024-09-21T03:42:39.029 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.029415049, 2024-09-21T03:42:39.029 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000058947 2024-09-21T03:42:39.029 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T03:42:39.029 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_handle_ack_lat": { 2024-09-21T03:42:39.029 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-09-21T03:42:39.029 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-09-21T03:42:39.029 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-09-21T03:42:39.029 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T03:42:39.029 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_recv_encrypted_bytes": 2960, 2024-09-21T03:42:39.029 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_send_encrypted_bytes": 448 2024-09-21T03:42:39.029 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T03:42:39.030 INFO:teuthology.orchestra.run.smithi114.stdout: "cct": { 2024-09-21T03:42:39.030 INFO:teuthology.orchestra.run.smithi114.stdout: "total_workers": 1, 2024-09-21T03:42:39.030 INFO:teuthology.orchestra.run.smithi114.stdout: "unhealthy_workers": 0 2024-09-21T03:42:39.030 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T03:42:39.030 INFO:teuthology.orchestra.run.smithi114.stdout: "finisher-MDSRank": { 2024-09-21T03:42:39.030 INFO:teuthology.orchestra.run.smithi114.stdout: "queue_len": 0, 2024-09-21T03:42:39.030 INFO:teuthology.orchestra.run.smithi114.stdout: "complete_latency": { 2024-09-21T03:42:39.030 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 1364, 2024-09-21T03:42:39.030 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.042405753, 2024-09-21T03:42:39.030 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000031089 2024-09-21T03:42:39.030 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T03:42:39.030 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T03:42:39.030 INFO:teuthology.orchestra.run.smithi114.stdout: "finisher-PurgeQueue": { 2024-09-21T03:42:39.030 INFO:teuthology.orchestra.run.smithi114.stdout: "queue_len": 0, 2024-09-21T03:42:39.030 INFO:teuthology.orchestra.run.smithi114.stdout: "complete_latency": { 2024-09-21T03:42:39.031 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-09-21T03:42:39.031 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-09-21T03:42:39.031 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-09-21T03:42:39.031 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T03:42:39.031 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T03:42:39.031 INFO:teuthology.orchestra.run.smithi114.stdout: "mds": { 2024-09-21T03:42:39.031 INFO:teuthology.orchestra.run.smithi114.stdout: "request": 0, 2024-09-21T03:42:39.031 INFO:teuthology.orchestra.run.smithi114.stdout: "reply": 0, 2024-09-21T03:42:39.031 INFO:teuthology.orchestra.run.smithi114.stdout: "reply_latency": { 2024-09-21T03:42:39.031 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-09-21T03:42:39.031 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-09-21T03:42:39.031 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-09-21T03:42:39.031 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T03:42:39.031 INFO:teuthology.orchestra.run.smithi114.stdout: "slow_reply": 0, 2024-09-21T03:42:39.031 INFO:teuthology.orchestra.run.smithi114.stdout: "forward": 0, 2024-09-21T03:42:39.032 INFO:teuthology.orchestra.run.smithi114.stdout: "dir_fetch_complete": 0, 2024-09-21T03:42:39.032 INFO:teuthology.orchestra.run.smithi114.stdout: "dir_fetch_keys": 0, 2024-09-21T03:42:39.032 INFO:teuthology.orchestra.run.smithi114.stdout: "dir_commit": 0, 2024-09-21T03:42:39.032 INFO:teuthology.orchestra.run.smithi114.stdout: "dir_split": 0, 2024-09-21T03:42:39.032 INFO:teuthology.orchestra.run.smithi114.stdout: "dir_merge": 0, 2024-09-21T03:42:39.032 INFO:teuthology.orchestra.run.smithi114.stdout: "inodes": 0, 2024-09-21T03:42:39.032 INFO:teuthology.orchestra.run.smithi114.stdout: "inodes_top": 0, 2024-09-21T03:42:39.032 INFO:teuthology.orchestra.run.smithi114.stdout: "inodes_bottom": 0, 2024-09-21T03:42:39.032 INFO:teuthology.orchestra.run.smithi114.stdout: "inodes_pin_tail": 0, 2024-09-21T03:42:39.032 INFO:teuthology.orchestra.run.smithi114.stdout: "inodes_pinned": 0, 2024-09-21T03:42:39.032 INFO:teuthology.orchestra.run.smithi114.stdout: "inodes_expired": 0, 2024-09-21T03:42:39.032 INFO:teuthology.orchestra.run.smithi114.stdout: "inodes_with_caps": 0, 2024-09-21T03:42:39.032 INFO:teuthology.orchestra.run.smithi114.stdout: "caps": 0, 2024-09-21T03:42:39.032 INFO:teuthology.orchestra.run.smithi114.stdout: "subtrees": 1, 2024-09-21T03:42:39.032 INFO:teuthology.orchestra.run.smithi114.stdout: "traverse": 0, 2024-09-21T03:42:39.032 INFO:teuthology.orchestra.run.smithi114.stdout: "traverse_hit": 0, 2024-09-21T03:42:39.033 INFO:teuthology.orchestra.run.smithi114.stdout: "traverse_forward": 0, 2024-09-21T03:42:39.033 INFO:teuthology.orchestra.run.smithi114.stdout: "traverse_discover": 0, 2024-09-21T03:42:39.033 INFO:teuthology.orchestra.run.smithi114.stdout: "traverse_dir_fetch": 0, 2024-09-21T03:42:39.033 INFO:teuthology.orchestra.run.smithi114.stdout: "traverse_remote_ino": 0, 2024-09-21T03:42:39.033 INFO:teuthology.orchestra.run.smithi114.stdout: "traverse_lock": 0, 2024-09-21T03:42:39.033 INFO:teuthology.orchestra.run.smithi114.stdout: "load_cent": 0, 2024-09-21T03:42:39.033 INFO:teuthology.orchestra.run.smithi114.stdout: "q": 0, 2024-09-21T03:42:39.033 INFO:teuthology.orchestra.run.smithi114.stdout: "exported": 0, 2024-09-21T03:42:39.033 INFO:teuthology.orchestra.run.smithi114.stdout: "exported_inodes": 0, 2024-09-21T03:42:39.033 INFO:teuthology.orchestra.run.smithi114.stdout: "imported": 0, 2024-09-21T03:42:39.033 INFO:teuthology.orchestra.run.smithi114.stdout: "imported_inodes": 0, 2024-09-21T03:42:39.033 INFO:teuthology.orchestra.run.smithi114.stdout: "openino_dir_fetch": 0, 2024-09-21T03:42:39.034 INFO:teuthology.orchestra.run.smithi114.stdout: "openino_backtrace_fetch": 0, 2024-09-21T03:42:39.034 INFO:teuthology.orchestra.run.smithi114.stdout: "openino_peer_discover": 0, 2024-09-21T03:42:39.034 INFO:teuthology.orchestra.run.smithi114.stdout: "root_rfiles": 0, 2024-09-21T03:42:39.034 INFO:teuthology.orchestra.run.smithi114.stdout: "root_rbytes": 0, 2024-09-21T03:42:39.034 INFO:teuthology.orchestra.run.smithi114.stdout: "root_rsnaps": 0, 2024-09-21T03:42:39.034 INFO:teuthology.orchestra.run.smithi114.stdout: "scrub_backtrace_fetch": 0, 2024-09-21T03:42:39.034 INFO:teuthology.orchestra.run.smithi114.stdout: "scrub_set_tag": 0, 2024-09-21T03:42:39.034 INFO:teuthology.orchestra.run.smithi114.stdout: "scrub_backtrace_repaired": 0, 2024-09-21T03:42:39.034 INFO:teuthology.orchestra.run.smithi114.stdout: "scrub_inotable_repaired": 0, 2024-09-21T03:42:39.034 INFO:teuthology.orchestra.run.smithi114.stdout: "scrub_dir_inodes": 0, 2024-09-21T03:42:39.034 INFO:teuthology.orchestra.run.smithi114.stdout: "scrub_dir_base_inodes": 0, 2024-09-21T03:42:39.034 INFO:teuthology.orchestra.run.smithi114.stdout: "scrub_dirfrag_rstats": 0, 2024-09-21T03:42:39.034 INFO:teuthology.orchestra.run.smithi114.stdout: "scrub_file_inodes": 0, 2024-09-21T03:42:39.034 INFO:teuthology.orchestra.run.smithi114.stdout: "handle_inode_file_caps": 0, 2024-09-21T03:42:39.034 INFO:teuthology.orchestra.run.smithi114.stdout: "ceph_cap_op_revoke": 0, 2024-09-21T03:42:39.035 INFO:teuthology.orchestra.run.smithi114.stdout: "ceph_cap_op_grant": 0, 2024-09-21T03:42:39.035 INFO:teuthology.orchestra.run.smithi114.stdout: "ceph_cap_op_trunc": 0, 2024-09-21T03:42:39.035 INFO:teuthology.orchestra.run.smithi114.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-21T03:42:39.035 INFO:teuthology.orchestra.run.smithi114.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-21T03:42:39.035 INFO:teuthology.orchestra.run.smithi114.stdout: "handle_client_caps": 0, 2024-09-21T03:42:39.035 INFO:teuthology.orchestra.run.smithi114.stdout: "handle_client_caps_dirty": 0, 2024-09-21T03:42:39.035 INFO:teuthology.orchestra.run.smithi114.stdout: "handle_client_cap_release": 0, 2024-09-21T03:42:39.035 INFO:teuthology.orchestra.run.smithi114.stdout: "process_request_cap_release": 0 2024-09-21T03:42:39.035 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T03:42:39.035 INFO:teuthology.orchestra.run.smithi114.stdout: "mds_cache": { 2024-09-21T03:42:39.035 INFO:teuthology.orchestra.run.smithi114.stdout: "dir_update": 0, 2024-09-21T03:42:39.035 INFO:teuthology.orchestra.run.smithi114.stdout: "dir_update_receipt": 0, 2024-09-21T03:42:39.035 INFO:teuthology.orchestra.run.smithi114.stdout: "dir_try_discover": 0, 2024-09-21T03:42:39.035 INFO:teuthology.orchestra.run.smithi114.stdout: "dir_send_discover": 0, 2024-09-21T03:42:39.036 INFO:teuthology.orchestra.run.smithi114.stdout: "dir_handle_discover": 0, 2024-09-21T03:42:39.036 INFO:teuthology.orchestra.run.smithi114.stdout: "num_strays": 0, 2024-09-21T03:42:39.036 INFO:teuthology.orchestra.run.smithi114.stdout: "num_strays_delayed": 0, 2024-09-21T03:42:39.036 INFO:teuthology.orchestra.run.smithi114.stdout: "num_strays_enqueuing": 0, 2024-09-21T03:42:39.036 INFO:teuthology.orchestra.run.smithi114.stdout: "strays_created": 0, 2024-09-21T03:42:39.036 INFO:teuthology.orchestra.run.smithi114.stdout: "strays_enqueued": 0, 2024-09-21T03:42:39.036 INFO:teuthology.orchestra.run.smithi114.stdout: "strays_reintegrated": 0, 2024-09-21T03:42:39.036 INFO:teuthology.orchestra.run.smithi114.stdout: "strays_migrated": 0, 2024-09-21T03:42:39.036 INFO:teuthology.orchestra.run.smithi114.stdout: "num_recovering_processing": 0, 2024-09-21T03:42:39.036 INFO:teuthology.orchestra.run.smithi114.stdout: "num_recovering_enqueued": 0, 2024-09-21T03:42:39.036 INFO:teuthology.orchestra.run.smithi114.stdout: "num_recovering_prioritized": 0, 2024-09-21T03:42:39.036 INFO:teuthology.orchestra.run.smithi114.stdout: "recovery_started": 0, 2024-09-21T03:42:39.036 INFO:teuthology.orchestra.run.smithi114.stdout: "recovery_completed": 0, 2024-09-21T03:42:39.036 INFO:teuthology.orchestra.run.smithi114.stdout: "ireq_quiesce_path": 0, 2024-09-21T03:42:39.036 INFO:teuthology.orchestra.run.smithi114.stdout: "ireq_quiesce_inode": 0, 2024-09-21T03:42:39.037 INFO:teuthology.orchestra.run.smithi114.stdout: "ireq_enqueue_scrub": 0, 2024-09-21T03:42:39.037 INFO:teuthology.orchestra.run.smithi114.stdout: "ireq_exportdir": 0, 2024-09-21T03:42:39.037 INFO:teuthology.orchestra.run.smithi114.stdout: "ireq_flush": 0, 2024-09-21T03:42:39.037 INFO:teuthology.orchestra.run.smithi114.stdout: "ireq_fragmentdir": 0, 2024-09-21T03:42:39.037 INFO:teuthology.orchestra.run.smithi114.stdout: "ireq_fragstats": 0, 2024-09-21T03:42:39.037 INFO:teuthology.orchestra.run.smithi114.stdout: "ireq_inodestats": 0, 2024-09-21T03:42:39.037 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_started": 0, 2024-09-21T03:42:39.037 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_succeeded": 0, 2024-09-21T03:42:39.037 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_write_failed": 0 2024-09-21T03:42:39.037 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T03:42:39.037 INFO:teuthology.orchestra.run.smithi114.stdout: "mds_log": { 2024-09-21T03:42:39.037 INFO:teuthology.orchestra.run.smithi114.stdout: "evlrg": 0, 2024-09-21T03:42:39.037 INFO:teuthology.orchestra.run.smithi114.stdout: "evadd": 0, 2024-09-21T03:42:39.037 INFO:teuthology.orchestra.run.smithi114.stdout: "evex": 0, 2024-09-21T03:42:39.037 INFO:teuthology.orchestra.run.smithi114.stdout: "evtrm": 0, 2024-09-21T03:42:39.037 INFO:teuthology.orchestra.run.smithi114.stdout: "ev": 7, 2024-09-21T03:42:39.038 INFO:teuthology.orchestra.run.smithi114.stdout: "evexg": 0, 2024-09-21T03:42:39.038 INFO:teuthology.orchestra.run.smithi114.stdout: "evexd": 0, 2024-09-21T03:42:39.038 INFO:teuthology.orchestra.run.smithi114.stdout: "segadd": 0, 2024-09-21T03:42:39.038 INFO:teuthology.orchestra.run.smithi114.stdout: "segex": 0, 2024-09-21T03:42:39.038 INFO:teuthology.orchestra.run.smithi114.stdout: "segtrm": 0, 2024-09-21T03:42:39.038 INFO:teuthology.orchestra.run.smithi114.stdout: "seg": 2, 2024-09-21T03:42:39.038 INFO:teuthology.orchestra.run.smithi114.stdout: "segmjr": 2, 2024-09-21T03:42:39.038 INFO:teuthology.orchestra.run.smithi114.stdout: "segexg": 0, 2024-09-21T03:42:39.038 INFO:teuthology.orchestra.run.smithi114.stdout: "segexd": 0, 2024-09-21T03:42:39.038 INFO:teuthology.orchestra.run.smithi114.stdout: "expos": 4194304, 2024-09-21T03:42:39.038 INFO:teuthology.orchestra.run.smithi114.stdout: "wrpos": 4196449, 2024-09-21T03:42:39.038 INFO:teuthology.orchestra.run.smithi114.stdout: "rdpos": 4196308, 2024-09-21T03:42:39.038 INFO:teuthology.orchestra.run.smithi114.stdout: "jlat": { 2024-09-21T03:42:39.038 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-09-21T03:42:39.038 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-09-21T03:42:39.039 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-09-21T03:42:39.039 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T03:42:39.039 INFO:teuthology.orchestra.run.smithi114.stdout: "replayed": 7 2024-09-21T03:42:39.039 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T03:42:39.039 INFO:teuthology.orchestra.run.smithi114.stdout: "mds_mem": { 2024-09-21T03:42:39.039 INFO:teuthology.orchestra.run.smithi114.stdout: "ino": 3, 2024-09-21T03:42:39.039 INFO:teuthology.orchestra.run.smithi114.stdout: "ino+": 3, 2024-09-21T03:42:39.039 INFO:teuthology.orchestra.run.smithi114.stdout: "ino-": 0, 2024-09-21T03:42:39.039 INFO:teuthology.orchestra.run.smithi114.stdout: "dir": 1, 2024-09-21T03:42:39.039 INFO:teuthology.orchestra.run.smithi114.stdout: "dir+": 1, 2024-09-21T03:42:39.039 INFO:teuthology.orchestra.run.smithi114.stdout: "dir-": 0, 2024-09-21T03:42:39.039 INFO:teuthology.orchestra.run.smithi114.stdout: "dn": 0, 2024-09-21T03:42:39.039 INFO:teuthology.orchestra.run.smithi114.stdout: "dn+": 0, 2024-09-21T03:42:39.039 INFO:teuthology.orchestra.run.smithi114.stdout: "dn-": 0, 2024-09-21T03:42:39.040 INFO:teuthology.orchestra.run.smithi114.stdout: "cap": 0, 2024-09-21T03:42:39.040 INFO:teuthology.orchestra.run.smithi114.stdout: "cap+": 0, 2024-09-21T03:42:39.040 INFO:teuthology.orchestra.run.smithi114.stdout: "cap-": 0, 2024-09-21T03:42:39.040 INFO:teuthology.orchestra.run.smithi114.stdout: "rss": 116776, 2024-09-21T03:42:39.040 INFO:teuthology.orchestra.run.smithi114.stdout: "heap": 315367424 2024-09-21T03:42:39.040 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T03:42:39.040 INFO:teuthology.orchestra.run.smithi114.stdout: "mds_server": { 2024-09-21T03:42:39.040 INFO:teuthology.orchestra.run.smithi114.stdout: "dispatch_client_request": 0, 2024-09-21T03:42:39.040 INFO:teuthology.orchestra.run.smithi114.stdout: "dispatch_server_request": 0, 2024-09-21T03:42:39.040 INFO:teuthology.orchestra.run.smithi114.stdout: "handle_client_request": 0, 2024-09-21T03:42:39.040 INFO:teuthology.orchestra.run.smithi114.stdout: "handle_client_session": 0, 2024-09-21T03:42:39.040 INFO:teuthology.orchestra.run.smithi114.stdout: "handle_peer_request": 0, 2024-09-21T03:42:39.041 INFO:teuthology.orchestra.run.smithi114.stdout: "req_create_latency": { 2024-09-21T03:42:39.041 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-09-21T03:42:39.041 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-09-21T03:42:39.041 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-09-21T03:42:39.041 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T03:42:39.041 INFO:teuthology.orchestra.run.smithi114.stdout: "req_getattr_latency": { 2024-09-21T03:42:39.041 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-09-21T03:42:39.041 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-09-21T03:42:39.041 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-09-21T03:42:39.041 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T03:42:39.041 INFO:teuthology.orchestra.run.smithi114.stdout: "req_getfilelock_latency": { 2024-09-21T03:42:39.041 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-09-21T03:42:39.041 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-09-21T03:42:39.042 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-09-21T03:42:39.042 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T03:42:39.042 INFO:teuthology.orchestra.run.smithi114.stdout: "req_link_latency": { 2024-09-21T03:42:39.042 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-09-21T03:42:39.042 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-09-21T03:42:39.042 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-09-21T03:42:39.042 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T03:42:39.042 INFO:teuthology.orchestra.run.smithi114.stdout: "req_lookup_latency": { 2024-09-21T03:42:39.042 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-09-21T03:42:39.042 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-09-21T03:42:39.042 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-09-21T03:42:39.042 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T03:42:39.042 INFO:teuthology.orchestra.run.smithi114.stdout: "req_lookuphash_latency": { 2024-09-21T03:42:39.043 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-09-21T03:42:39.043 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-09-21T03:42:39.043 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-09-21T03:42:39.043 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T03:42:39.043 INFO:teuthology.orchestra.run.smithi114.stdout: "req_lookupino_latency": { 2024-09-21T03:42:39.043 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-09-21T03:42:39.043 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-09-21T03:42:39.043 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-09-21T03:42:39.043 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T03:42:39.043 INFO:teuthology.orchestra.run.smithi114.stdout: "req_lookupname_latency": { 2024-09-21T03:42:39.043 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-09-21T03:42:39.043 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-09-21T03:42:39.043 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-09-21T03:42:39.044 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T03:42:39.044 INFO:teuthology.orchestra.run.smithi114.stdout: "req_lookupparent_latency": { 2024-09-21T03:42:39.044 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-09-21T03:42:39.044 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-09-21T03:42:39.044 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-09-21T03:42:39.044 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T03:42:39.044 INFO:teuthology.orchestra.run.smithi114.stdout: "req_lookupsnap_latency": { 2024-09-21T03:42:39.044 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-09-21T03:42:39.044 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-09-21T03:42:39.044 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-09-21T03:42:39.044 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T03:42:39.044 INFO:teuthology.orchestra.run.smithi114.stdout: "req_lssnap_latency": { 2024-09-21T03:42:39.044 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-09-21T03:42:39.044 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-09-21T03:42:39.045 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-09-21T03:42:39.045 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T03:42:39.045 INFO:teuthology.orchestra.run.smithi114.stdout: "req_mkdir_latency": { 2024-09-21T03:42:39.045 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-09-21T03:42:39.045 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-09-21T03:42:39.045 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-09-21T03:42:39.045 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T03:42:39.046 INFO:teuthology.orchestra.run.smithi114.stdout: "req_mknod_latency": { 2024-09-21T03:42:39.046 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-09-21T03:42:39.046 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-09-21T03:42:39.046 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-09-21T03:42:39.046 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T03:42:39.046 INFO:teuthology.orchestra.run.smithi114.stdout: "req_mksnap_latency": { 2024-09-21T03:42:39.046 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-09-21T03:42:39.046 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-09-21T03:42:39.046 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-09-21T03:42:39.046 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T03:42:39.046 INFO:teuthology.orchestra.run.smithi114.stdout: "req_open_latency": { 2024-09-21T03:42:39.046 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-09-21T03:42:39.046 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-09-21T03:42:39.046 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-09-21T03:42:39.047 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T03:42:39.047 INFO:teuthology.orchestra.run.smithi114.stdout: "req_readdir_latency": { 2024-09-21T03:42:39.047 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-09-21T03:42:39.047 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-09-21T03:42:39.047 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-09-21T03:42:39.047 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T03:42:39.047 INFO:teuthology.orchestra.run.smithi114.stdout: "req_rename_latency": { 2024-09-21T03:42:39.047 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-09-21T03:42:39.047 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-09-21T03:42:39.047 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-09-21T03:42:39.047 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T03:42:39.047 INFO:teuthology.orchestra.run.smithi114.stdout: "req_renamesnap_latency": { 2024-09-21T03:42:39.047 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-09-21T03:42:39.047 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-09-21T03:42:39.047 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-09-21T03:42:39.048 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T03:42:39.048 INFO:teuthology.orchestra.run.smithi114.stdout: "req_snapdiff_latency": { 2024-09-21T03:42:39.048 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-09-21T03:42:39.048 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-09-21T03:42:39.048 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-09-21T03:42:39.048 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T03:42:39.048 INFO:teuthology.orchestra.run.smithi114.stdout: "req_rmdir_latency": { 2024-09-21T03:42:39.048 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-09-21T03:42:39.048 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-09-21T03:42:39.048 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-09-21T03:42:39.048 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T03:42:39.048 INFO:teuthology.orchestra.run.smithi114.stdout: "req_rmsnap_latency": { 2024-09-21T03:42:39.048 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-09-21T03:42:39.048 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-09-21T03:42:39.048 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-09-21T03:42:39.049 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T03:42:39.049 INFO:teuthology.orchestra.run.smithi114.stdout: "req_rmxattr_latency": { 2024-09-21T03:42:39.049 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-09-21T03:42:39.049 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-09-21T03:42:39.049 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-09-21T03:42:39.049 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T03:42:39.049 INFO:teuthology.orchestra.run.smithi114.stdout: "req_setattr_latency": { 2024-09-21T03:42:39.049 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-09-21T03:42:39.049 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-09-21T03:42:39.049 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-09-21T03:42:39.049 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T03:42:39.049 INFO:teuthology.orchestra.run.smithi114.stdout: "req_setdirlayout_latency": { 2024-09-21T03:42:39.049 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-09-21T03:42:39.049 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-09-21T03:42:39.049 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-09-21T03:42:39.049 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T03:42:39.050 INFO:teuthology.orchestra.run.smithi114.stdout: "req_setfilelock_latency": { 2024-09-21T03:42:39.050 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-09-21T03:42:39.050 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-09-21T03:42:39.050 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-09-21T03:42:39.050 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T03:42:39.050 INFO:teuthology.orchestra.run.smithi114.stdout: "req_setlayout_latency": { 2024-09-21T03:42:39.050 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-09-21T03:42:39.050 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-09-21T03:42:39.050 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-09-21T03:42:39.050 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T03:42:39.050 INFO:teuthology.orchestra.run.smithi114.stdout: "req_setxattr_latency": { 2024-09-21T03:42:39.050 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-09-21T03:42:39.050 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-09-21T03:42:39.051 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-09-21T03:42:39.051 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T03:42:39.051 INFO:teuthology.orchestra.run.smithi114.stdout: "req_symlink_latency": { 2024-09-21T03:42:39.051 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-09-21T03:42:39.051 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-09-21T03:42:39.051 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-09-21T03:42:39.051 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T03:42:39.051 INFO:teuthology.orchestra.run.smithi114.stdout: "req_unlink_latency": { 2024-09-21T03:42:39.051 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-09-21T03:42:39.051 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-09-21T03:42:39.051 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-09-21T03:42:39.051 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T03:42:39.051 INFO:teuthology.orchestra.run.smithi114.stdout: "cap_revoke_eviction": 0, 2024-09-21T03:42:39.052 INFO:teuthology.orchestra.run.smithi114.stdout: "cap_acquisition_throttle": 0, 2024-09-21T03:42:39.052 INFO:teuthology.orchestra.run.smithi114.stdout: "req_getvxattr_latency": { 2024-09-21T03:42:39.052 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-09-21T03:42:39.052 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-09-21T03:42:39.052 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-09-21T03:42:39.052 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T03:42:39.052 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T03:42:39.052 INFO:teuthology.orchestra.run.smithi114.stdout: "mds_sessions": { 2024-09-21T03:42:39.052 INFO:teuthology.orchestra.run.smithi114.stdout: "session_count": 1, 2024-09-21T03:42:39.052 INFO:teuthology.orchestra.run.smithi114.stdout: "session_add": 3, 2024-09-21T03:42:39.052 INFO:teuthology.orchestra.run.smithi114.stdout: "session_remove": 2, 2024-09-21T03:42:39.052 INFO:teuthology.orchestra.run.smithi114.stdout: "sessions_open": 2, 2024-09-21T03:42:39.052 INFO:teuthology.orchestra.run.smithi114.stdout: "sessions_stale": 0, 2024-09-21T03:42:39.052 INFO:teuthology.orchestra.run.smithi114.stdout: "total_load": 0, 2024-09-21T03:42:39.052 INFO:teuthology.orchestra.run.smithi114.stdout: "average_load": 0, 2024-09-21T03:42:39.052 INFO:teuthology.orchestra.run.smithi114.stdout: "avg_session_uptime": 2315, 2024-09-21T03:42:39.053 INFO:teuthology.orchestra.run.smithi114.stdout: "mdthresh_evicted": 0 2024-09-21T03:42:39.053 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T03:42:39.053 INFO:teuthology.orchestra.run.smithi114.stdout: "mempool": { 2024-09-21T03:42:39.053 INFO:teuthology.orchestra.run.smithi114.stdout: "bloom_filter_bytes": 0, 2024-09-21T03:42:39.053 INFO:teuthology.orchestra.run.smithi114.stdout: "bloom_filter_items": 0, 2024-09-21T03:42:39.053 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_alloc_bytes": 0, 2024-09-21T03:42:39.053 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_alloc_items": 0, 2024-09-21T03:42:39.053 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_cache_data_bytes": 0, 2024-09-21T03:42:39.053 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_cache_data_items": 0, 2024-09-21T03:42:39.053 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-21T03:42:39.053 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_cache_onode_items": 0, 2024-09-21T03:42:39.053 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-21T03:42:39.053 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_cache_meta_items": 0, 2024-09-21T03:42:39.053 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_cache_other_bytes": 0, 2024-09-21T03:42:39.053 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_cache_other_items": 0, 2024-09-21T03:42:39.053 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-21T03:42:39.054 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_cache_buffer_items": 0, 2024-09-21T03:42:39.054 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_extent_bytes": 0, 2024-09-21T03:42:39.054 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_extent_items": 0, 2024-09-21T03:42:39.054 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_blob_bytes": 0, 2024-09-21T03:42:39.054 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_blob_items": 0, 2024-09-21T03:42:39.054 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-21T03:42:39.054 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_shared_blob_items": 0, 2024-09-21T03:42:39.054 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-21T03:42:39.054 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_inline_bl_items": 0, 2024-09-21T03:42:39.054 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_fsck_bytes": 0, 2024-09-21T03:42:39.054 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_fsck_items": 0, 2024-09-21T03:42:39.054 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_txc_bytes": 0, 2024-09-21T03:42:39.054 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_txc_items": 0, 2024-09-21T03:42:39.054 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-21T03:42:39.054 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_writing_deferred_items": 0, 2024-09-21T03:42:39.055 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_writing_bytes": 0, 2024-09-21T03:42:39.055 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_writing_items": 0, 2024-09-21T03:42:39.055 INFO:teuthology.orchestra.run.smithi114.stdout: "bluefs_bytes": 0, 2024-09-21T03:42:39.055 INFO:teuthology.orchestra.run.smithi114.stdout: "bluefs_items": 0, 2024-09-21T03:42:39.055 INFO:teuthology.orchestra.run.smithi114.stdout: "bluefs_file_reader_bytes": 0, 2024-09-21T03:42:39.055 INFO:teuthology.orchestra.run.smithi114.stdout: "bluefs_file_reader_items": 0, 2024-09-21T03:42:39.055 INFO:teuthology.orchestra.run.smithi114.stdout: "bluefs_file_writer_bytes": 0, 2024-09-21T03:42:39.055 INFO:teuthology.orchestra.run.smithi114.stdout: "bluefs_file_writer_items": 0, 2024-09-21T03:42:39.055 INFO:teuthology.orchestra.run.smithi114.stdout: "buffer_anon_bytes": 181216, 2024-09-21T03:42:39.055 INFO:teuthology.orchestra.run.smithi114.stdout: "buffer_anon_items": 46, 2024-09-21T03:42:39.055 INFO:teuthology.orchestra.run.smithi114.stdout: "buffer_meta_bytes": 0, 2024-09-21T03:42:39.055 INFO:teuthology.orchestra.run.smithi114.stdout: "buffer_meta_items": 0, 2024-09-21T03:42:39.055 INFO:teuthology.orchestra.run.smithi114.stdout: "osd_bytes": 0, 2024-09-21T03:42:39.055 INFO:teuthology.orchestra.run.smithi114.stdout: "osd_items": 0, 2024-09-21T03:42:39.055 INFO:teuthology.orchestra.run.smithi114.stdout: "osd_mapbl_bytes": 0, 2024-09-21T03:42:39.055 INFO:teuthology.orchestra.run.smithi114.stdout: "osd_mapbl_items": 0, 2024-09-21T03:42:39.056 INFO:teuthology.orchestra.run.smithi114.stdout: "osd_pglog_bytes": 0, 2024-09-21T03:42:39.056 INFO:teuthology.orchestra.run.smithi114.stdout: "osd_pglog_items": 0, 2024-09-21T03:42:39.056 INFO:teuthology.orchestra.run.smithi114.stdout: "osdmap_bytes": 8760, 2024-09-21T03:42:39.056 INFO:teuthology.orchestra.run.smithi114.stdout: "osdmap_items": 164, 2024-09-21T03:42:39.056 INFO:teuthology.orchestra.run.smithi114.stdout: "osdmap_mapping_bytes": 0, 2024-09-21T03:42:39.056 INFO:teuthology.orchestra.run.smithi114.stdout: "osdmap_mapping_items": 0, 2024-09-21T03:42:39.056 INFO:teuthology.orchestra.run.smithi114.stdout: "pgmap_bytes": 0, 2024-09-21T03:42:39.056 INFO:teuthology.orchestra.run.smithi114.stdout: "pgmap_items": 0, 2024-09-21T03:42:39.056 INFO:teuthology.orchestra.run.smithi114.stdout: "mds_co_bytes": 8672, 2024-09-21T03:42:39.056 INFO:teuthology.orchestra.run.smithi114.stdout: "mds_co_items": 13, 2024-09-21T03:42:39.056 INFO:teuthology.orchestra.run.smithi114.stdout: "unittest_1_bytes": 0, 2024-09-21T03:42:39.056 INFO:teuthology.orchestra.run.smithi114.stdout: "unittest_1_items": 0, 2024-09-21T03:42:39.056 INFO:teuthology.orchestra.run.smithi114.stdout: "unittest_2_bytes": 0, 2024-09-21T03:42:39.056 INFO:teuthology.orchestra.run.smithi114.stdout: "unittest_2_items": 0 2024-09-21T03:42:39.056 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T03:42:39.056 INFO:teuthology.orchestra.run.smithi114.stdout: "objecter": { 2024-09-21T03:42:39.057 INFO:teuthology.orchestra.run.smithi114.stdout: "op_active": 0, 2024-09-21T03:42:39.057 INFO:teuthology.orchestra.run.smithi114.stdout: "op_laggy": 0, 2024-09-21T03:42:39.057 INFO:teuthology.orchestra.run.smithi114.stdout: "op_send": 788, 2024-09-21T03:42:39.057 INFO:teuthology.orchestra.run.smithi114.stdout: "op_send_bytes": 21, 2024-09-21T03:42:39.057 INFO:teuthology.orchestra.run.smithi114.stdout: "op_resend": 3, 2024-09-21T03:42:39.057 INFO:teuthology.orchestra.run.smithi114.stdout: "op_reply": 788, 2024-09-21T03:42:39.057 INFO:teuthology.orchestra.run.smithi114.stdout: "op_latency": { 2024-09-21T03:42:39.057 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 788, 2024-09-21T03:42:39.057 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.793996182, 2024-09-21T03:42:39.057 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.001007609 2024-09-21T03:42:39.057 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T03:42:39.057 INFO:teuthology.orchestra.run.smithi114.stdout: "op_inflight": 0, 2024-09-21T03:42:39.057 INFO:teuthology.orchestra.run.smithi114.stdout: "oplen_avg": { 2024-09-21T03:42:39.057 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 788, 2024-09-21T03:42:39.057 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 789 2024-09-21T03:42:39.057 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T03:42:39.057 INFO:teuthology.orchestra.run.smithi114.stdout: "op": 788, 2024-09-21T03:42:39.058 INFO:teuthology.orchestra.run.smithi114.stdout: "op_r": 788, 2024-09-21T03:42:39.058 INFO:teuthology.orchestra.run.smithi114.stdout: "op_w": 0, 2024-09-21T03:42:39.058 INFO:teuthology.orchestra.run.smithi114.stdout: "op_rmw": 0, 2024-09-21T03:42:39.058 INFO:teuthology.orchestra.run.smithi114.stdout: "op_pg": 0, 2024-09-21T03:42:39.058 INFO:teuthology.orchestra.run.smithi114.stdout: "osdop_stat": 487, 2024-09-21T03:42:39.058 INFO:teuthology.orchestra.run.smithi114.stdout: "osdop_create": 0, 2024-09-21T03:42:39.058 INFO:teuthology.orchestra.run.smithi114.stdout: "osdop_read": 299, 2024-09-21T03:42:39.058 INFO:teuthology.orchestra.run.smithi114.stdout: "osdop_write": 0, 2024-09-21T03:42:39.058 INFO:teuthology.orchestra.run.smithi114.stdout: "osdop_writefull": 0, 2024-09-21T03:42:39.058 INFO:teuthology.orchestra.run.smithi114.stdout: "osdop_writesame": 0, 2024-09-21T03:42:39.058 INFO:teuthology.orchestra.run.smithi114.stdout: "osdop_append": 0, 2024-09-21T03:42:39.058 INFO:teuthology.orchestra.run.smithi114.stdout: "osdop_zero": 0, 2024-09-21T03:42:39.058 INFO:teuthology.orchestra.run.smithi114.stdout: "osdop_truncate": 0, 2024-09-21T03:42:39.058 INFO:teuthology.orchestra.run.smithi114.stdout: "osdop_delete": 0, 2024-09-21T03:42:39.059 INFO:teuthology.orchestra.run.smithi114.stdout: "osdop_mapext": 0, 2024-09-21T03:42:39.059 INFO:teuthology.orchestra.run.smithi114.stdout: "osdop_sparse_read": 0, 2024-09-21T03:42:39.059 INFO:teuthology.orchestra.run.smithi114.stdout: "osdop_clonerange": 0, 2024-09-21T03:42:39.059 INFO:teuthology.orchestra.run.smithi114.stdout: "osdop_getxattr": 1, 2024-09-21T03:42:39.059 INFO:teuthology.orchestra.run.smithi114.stdout: "osdop_setxattr": 0, 2024-09-21T03:42:39.059 INFO:teuthology.orchestra.run.smithi114.stdout: "osdop_cmpxattr": 0, 2024-09-21T03:42:39.059 INFO:teuthology.orchestra.run.smithi114.stdout: "osdop_rmxattr": 0, 2024-09-21T03:42:39.059 INFO:teuthology.orchestra.run.smithi114.stdout: "osdop_resetxattrs": 0, 2024-09-21T03:42:39.059 INFO:teuthology.orchestra.run.smithi114.stdout: "osdop_call": 0, 2024-09-21T03:42:39.059 INFO:teuthology.orchestra.run.smithi114.stdout: "osdop_watch": 0, 2024-09-21T03:42:39.059 INFO:teuthology.orchestra.run.smithi114.stdout: "osdop_notify": 0, 2024-09-21T03:42:39.059 INFO:teuthology.orchestra.run.smithi114.stdout: "osdop_src_cmpxattr": 0, 2024-09-21T03:42:39.059 INFO:teuthology.orchestra.run.smithi114.stdout: "osdop_pgls": 0, 2024-09-21T03:42:39.059 INFO:teuthology.orchestra.run.smithi114.stdout: "osdop_pgls_filter": 0, 2024-09-21T03:42:39.060 INFO:teuthology.orchestra.run.smithi114.stdout: "osdop_other": 0, 2024-09-21T03:42:39.060 INFO:teuthology.orchestra.run.smithi114.stdout: "linger_active": 0, 2024-09-21T03:42:39.060 INFO:teuthology.orchestra.run.smithi114.stdout: "linger_send": 0, 2024-09-21T03:42:39.060 INFO:teuthology.orchestra.run.smithi114.stdout: "linger_resend": 0, 2024-09-21T03:42:39.060 INFO:teuthology.orchestra.run.smithi114.stdout: "linger_ping": 0, 2024-09-21T03:42:39.060 INFO:teuthology.orchestra.run.smithi114.stdout: "poolop_active": 0, 2024-09-21T03:42:39.060 INFO:teuthology.orchestra.run.smithi114.stdout: "poolop_send": 0, 2024-09-21T03:42:39.060 INFO:teuthology.orchestra.run.smithi114.stdout: "poolop_resend": 0, 2024-09-21T03:42:39.060 INFO:teuthology.orchestra.run.smithi114.stdout: "poolstat_active": 0, 2024-09-21T03:42:39.060 INFO:teuthology.orchestra.run.smithi114.stdout: "poolstat_send": 0, 2024-09-21T03:42:39.060 INFO:teuthology.orchestra.run.smithi114.stdout: "poolstat_resend": 0, 2024-09-21T03:42:39.060 INFO:teuthology.orchestra.run.smithi114.stdout: "statfs_active": 0, 2024-09-21T03:42:39.060 INFO:teuthology.orchestra.run.smithi114.stdout: "statfs_send": 0, 2024-09-21T03:42:39.060 INFO:teuthology.orchestra.run.smithi114.stdout: "statfs_resend": 0, 2024-09-21T03:42:39.060 INFO:teuthology.orchestra.run.smithi114.stdout: "command_active": 0, 2024-09-21T03:42:39.061 INFO:teuthology.orchestra.run.smithi114.stdout: "command_send": 0, 2024-09-21T03:42:39.061 INFO:teuthology.orchestra.run.smithi114.stdout: "command_resend": 0, 2024-09-21T03:42:39.061 INFO:teuthology.orchestra.run.smithi114.stdout: "map_epoch": 84, 2024-09-21T03:42:39.061 INFO:teuthology.orchestra.run.smithi114.stdout: "map_full": 0, 2024-09-21T03:42:39.061 INFO:teuthology.orchestra.run.smithi114.stdout: "map_inc": 4, 2024-09-21T03:42:39.061 INFO:teuthology.orchestra.run.smithi114.stdout: "osd_sessions": 4, 2024-09-21T03:42:39.061 INFO:teuthology.orchestra.run.smithi114.stdout: "osd_session_open": 4, 2024-09-21T03:42:39.061 INFO:teuthology.orchestra.run.smithi114.stdout: "osd_session_close": 0, 2024-09-21T03:42:39.061 INFO:teuthology.orchestra.run.smithi114.stdout: "osd_laggy": 0, 2024-09-21T03:42:39.061 INFO:teuthology.orchestra.run.smithi114.stdout: "omap_wr": 0, 2024-09-21T03:42:39.061 INFO:teuthology.orchestra.run.smithi114.stdout: "omap_rd": 2, 2024-09-21T03:42:39.061 INFO:teuthology.orchestra.run.smithi114.stdout: "omap_del": 0 2024-09-21T03:42:39.061 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T03:42:39.061 INFO:teuthology.orchestra.run.smithi114.stdout: "oft": { 2024-09-21T03:42:39.061 INFO:teuthology.orchestra.run.smithi114.stdout: "omap_total_objs": 0, 2024-09-21T03:42:39.061 INFO:teuthology.orchestra.run.smithi114.stdout: "omap_total_kv_pairs": 0, 2024-09-21T03:42:39.062 INFO:teuthology.orchestra.run.smithi114.stdout: "omap_total_updates": 0, 2024-09-21T03:42:39.062 INFO:teuthology.orchestra.run.smithi114.stdout: "omap_total_removes": 0 2024-09-21T03:42:39.062 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T03:42:39.062 INFO:teuthology.orchestra.run.smithi114.stdout: "purge_queue": { 2024-09-21T03:42:39.062 INFO:teuthology.orchestra.run.smithi114.stdout: "pq_executing_ops": 0, 2024-09-21T03:42:39.062 INFO:teuthology.orchestra.run.smithi114.stdout: "pq_executing_ops_high_water": 0, 2024-09-21T03:42:39.062 INFO:teuthology.orchestra.run.smithi114.stdout: "pq_executing": 0, 2024-09-21T03:42:39.062 INFO:teuthology.orchestra.run.smithi114.stdout: "pq_executing_high_water": 0, 2024-09-21T03:42:39.062 INFO:teuthology.orchestra.run.smithi114.stdout: "pq_executed_ops": 0, 2024-09-21T03:42:39.062 INFO:teuthology.orchestra.run.smithi114.stdout: "pq_executed": 0, 2024-09-21T03:42:39.062 INFO:teuthology.orchestra.run.smithi114.stdout: "pq_item_in_journal": 0 2024-09-21T03:42:39.062 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T03:42:39.062 INFO:teuthology.orchestra.run.smithi114.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-21T03:42:39.062 INFO:teuthology.orchestra.run.smithi114.stdout: "val": 0, 2024-09-21T03:42:39.062 INFO:teuthology.orchestra.run.smithi114.stdout: "max": 104857600, 2024-09-21T03:42:39.063 INFO:teuthology.orchestra.run.smithi114.stdout: "get_started": 0, 2024-09-21T03:42:39.063 INFO:teuthology.orchestra.run.smithi114.stdout: "get": 903, 2024-09-21T03:42:39.063 INFO:teuthology.orchestra.run.smithi114.stdout: "get_sum": 968462, 2024-09-21T03:42:39.063 INFO:teuthology.orchestra.run.smithi114.stdout: "get_or_fail_fail": 0, 2024-09-21T03:42:39.063 INFO:teuthology.orchestra.run.smithi114.stdout: "get_or_fail_success": 903, 2024-09-21T03:42:39.063 INFO:teuthology.orchestra.run.smithi114.stdout: "take": 0, 2024-09-21T03:42:39.063 INFO:teuthology.orchestra.run.smithi114.stdout: "take_sum": 0, 2024-09-21T03:42:39.063 INFO:teuthology.orchestra.run.smithi114.stdout: "put": 903, 2024-09-21T03:42:39.063 INFO:teuthology.orchestra.run.smithi114.stdout: "put_sum": 968462, 2024-09-21T03:42:39.063 INFO:teuthology.orchestra.run.smithi114.stdout: "wait": { 2024-09-21T03:42:39.063 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-09-21T03:42:39.063 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-09-21T03:42:39.063 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-09-21T03:42:39.063 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T03:42:39.064 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T03:42:39.064 INFO:teuthology.orchestra.run.smithi114.stdout: "throttle-objecter_bytes": { 2024-09-21T03:42:39.064 INFO:teuthology.orchestra.run.smithi114.stdout: "val": 0, 2024-09-21T03:42:39.064 INFO:teuthology.orchestra.run.smithi114.stdout: "max": 104857600, 2024-09-21T03:42:39.064 INFO:teuthology.orchestra.run.smithi114.stdout: "get_started": 0, 2024-09-21T03:42:39.064 INFO:teuthology.orchestra.run.smithi114.stdout: "get": 0, 2024-09-21T03:42:39.064 INFO:teuthology.orchestra.run.smithi114.stdout: "get_sum": 0, 2024-09-21T03:42:39.064 INFO:teuthology.orchestra.run.smithi114.stdout: "get_or_fail_fail": 0, 2024-09-21T03:42:39.064 INFO:teuthology.orchestra.run.smithi114.stdout: "get_or_fail_success": 0, 2024-09-21T03:42:39.064 INFO:teuthology.orchestra.run.smithi114.stdout: "take": 788, 2024-09-21T03:42:39.064 INFO:teuthology.orchestra.run.smithi114.stdout: "take_sum": 2150, 2024-09-21T03:42:39.064 INFO:teuthology.orchestra.run.smithi114.stdout: "put": 788, 2024-09-21T03:42:39.064 INFO:teuthology.orchestra.run.smithi114.stdout: "put_sum": 2150, 2024-09-21T03:42:39.064 INFO:teuthology.orchestra.run.smithi114.stdout: "wait": { 2024-09-21T03:42:39.064 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-09-21T03:42:39.065 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-09-21T03:42:39.065 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-09-21T03:42:39.065 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T03:42:39.065 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T03:42:39.065 INFO:teuthology.orchestra.run.smithi114.stdout: "throttle-objecter_ops": { 2024-09-21T03:42:39.065 INFO:teuthology.orchestra.run.smithi114.stdout: "val": 0, 2024-09-21T03:42:39.065 INFO:teuthology.orchestra.run.smithi114.stdout: "max": 1024, 2024-09-21T03:42:39.065 INFO:teuthology.orchestra.run.smithi114.stdout: "get_started": 0, 2024-09-21T03:42:39.065 INFO:teuthology.orchestra.run.smithi114.stdout: "get": 0, 2024-09-21T03:42:39.065 INFO:teuthology.orchestra.run.smithi114.stdout: "get_sum": 0, 2024-09-21T03:42:39.065 INFO:teuthology.orchestra.run.smithi114.stdout: "get_or_fail_fail": 0, 2024-09-21T03:42:39.065 INFO:teuthology.orchestra.run.smithi114.stdout: "get_or_fail_success": 0, 2024-09-21T03:42:39.065 INFO:teuthology.orchestra.run.smithi114.stdout: "take": 788, 2024-09-21T03:42:39.065 INFO:teuthology.orchestra.run.smithi114.stdout: "take_sum": 788, 2024-09-21T03:42:39.065 INFO:teuthology.orchestra.run.smithi114.stdout: "put": 788, 2024-09-21T03:42:39.066 INFO:teuthology.orchestra.run.smithi114.stdout: "put_sum": 788, 2024-09-21T03:42:39.066 INFO:teuthology.orchestra.run.smithi114.stdout: "wait": { 2024-09-21T03:42:39.066 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-09-21T03:42:39.066 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-09-21T03:42:39.066 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-09-21T03:42:39.066 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T03:42:39.066 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T03:42:39.066 INFO:teuthology.orchestra.run.smithi114.stdout: "throttle-write_buf_throttle": { 2024-09-21T03:42:39.066 INFO:teuthology.orchestra.run.smithi114.stdout: "val": 0, 2024-09-21T03:42:39.066 INFO:teuthology.orchestra.run.smithi114.stdout: "max": 3758096384, 2024-09-21T03:42:39.066 INFO:teuthology.orchestra.run.smithi114.stdout: "get_started": 0, 2024-09-21T03:42:39.066 INFO:teuthology.orchestra.run.smithi114.stdout: "get": 0, 2024-09-21T03:42:39.066 INFO:teuthology.orchestra.run.smithi114.stdout: "get_sum": 0, 2024-09-21T03:42:39.066 INFO:teuthology.orchestra.run.smithi114.stdout: "get_or_fail_fail": 0, 2024-09-21T03:42:39.066 INFO:teuthology.orchestra.run.smithi114.stdout: "get_or_fail_success": 0, 2024-09-21T03:42:39.067 INFO:teuthology.orchestra.run.smithi114.stdout: "take": 0, 2024-09-21T03:42:39.067 INFO:teuthology.orchestra.run.smithi114.stdout: "take_sum": 0, 2024-09-21T03:42:39.067 INFO:teuthology.orchestra.run.smithi114.stdout: "put": 0, 2024-09-21T03:42:39.067 INFO:teuthology.orchestra.run.smithi114.stdout: "put_sum": 0, 2024-09-21T03:42:39.067 INFO:teuthology.orchestra.run.smithi114.stdout: "wait": { 2024-09-21T03:42:39.067 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-09-21T03:42:39.067 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-09-21T03:42:39.067 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-09-21T03:42:39.067 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T03:42:39.067 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T03:42:39.067 INFO:teuthology.orchestra.run.smithi114.stdout: "throttle-write_buf_throttle-0x564d5c1c5520": { 2024-09-21T03:42:39.067 INFO:teuthology.orchestra.run.smithi114.stdout: "val": 0, 2024-09-21T03:42:39.067 INFO:teuthology.orchestra.run.smithi114.stdout: "max": 3758096384, 2024-09-21T03:42:39.067 INFO:teuthology.orchestra.run.smithi114.stdout: "get_started": 0, 2024-09-21T03:42:39.067 INFO:teuthology.orchestra.run.smithi114.stdout: "get": 0, 2024-09-21T03:42:39.068 INFO:teuthology.orchestra.run.smithi114.stdout: "get_sum": 0, 2024-09-21T03:42:39.068 INFO:teuthology.orchestra.run.smithi114.stdout: "get_or_fail_fail": 0, 2024-09-21T03:42:39.068 INFO:teuthology.orchestra.run.smithi114.stdout: "get_or_fail_success": 0, 2024-09-21T03:42:39.068 INFO:teuthology.orchestra.run.smithi114.stdout: "take": 0, 2024-09-21T03:42:39.068 INFO:teuthology.orchestra.run.smithi114.stdout: "take_sum": 0, 2024-09-21T03:42:39.068 INFO:teuthology.orchestra.run.smithi114.stdout: "put": 0, 2024-09-21T03:42:39.068 INFO:teuthology.orchestra.run.smithi114.stdout: "put_sum": 0, 2024-09-21T03:42:39.068 INFO:teuthology.orchestra.run.smithi114.stdout: "wait": { 2024-09-21T03:42:39.068 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-09-21T03:42:39.068 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-09-21T03:42:39.068 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-09-21T03:42:39.068 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T03:42:39.068 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T03:42:39.068 INFO:teuthology.orchestra.run.smithi114.stdout: "trackedop": { 2024-09-21T03:42:39.068 INFO:teuthology.orchestra.run.smithi114.stdout: "slow_ops_count": 0 2024-09-21T03:42:39.069 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T03:42:39.069 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-21T03:42:39.210 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:42:38 smithi114 ceph-mon[34140]: pgmap v457: 129 pgs: 129 active+clean; 1.1 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-21T03:42:39.250 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:42:38 smithi110 ceph-mon[34000]: pgmap v457: 129 pgs: 129 active+clean; 1.1 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-21T03:42:39.311 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:42:38 smithi089 ceph-mon[30928]: pgmap v457: 129 pgs: 129 active+clean; 1.1 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-21T03:42:39.751 INFO:tasks.check_counter:Daemon mds.f mds.exported=0 2024-09-21T03:42:39.751 INFO:tasks.check_counter:Daemon mds.f mds.imported=0 2024-09-21T03:42:39.751 DEBUG:tasks.check_counter:Getting stats from i 2024-09-21T03:42:39.751 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 7c374098-77c9-11ef-baf5-efdc52797490 -- ceph daemon mds.i perf dump 2024-09-21T03:42:39.998 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/7c374098-77c9-11ef-baf5-efdc52797490/mon.c/config 2024-09-21T03:42:40.491 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-21T03:42:40.491 INFO:teuthology.orchestra.run.smithi114.stdout: "AsyncMessenger::Worker-0": { 2024-09-21T03:42:40.491 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_recv_messages": 384, 2024-09-21T03:42:40.491 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_send_messages": 388, 2024-09-21T03:42:40.491 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_recv_bytes": 905284, 2024-09-21T03:42:40.492 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_send_bytes": 202891, 2024-09-21T03:42:40.492 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_created_connections": 9, 2024-09-21T03:42:40.492 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_active_connections": 3, 2024-09-21T03:42:40.492 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_running_total_time": 0.056033290, 2024-09-21T03:42:40.492 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_running_send_time": 0.023364453, 2024-09-21T03:42:40.492 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_running_recv_time": 0.040820055, 2024-09-21T03:42:40.492 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_running_fast_dispatch_time": 0.002173864, 2024-09-21T03:42:40.492 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T03:42:40.492 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 387, 2024-09-21T03:42:40.492 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.027470771, 2024-09-21T03:42:40.492 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000070983 2024-09-21T03:42:40.492 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T03:42:40.492 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_handle_ack_lat": { 2024-09-21T03:42:40.492 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 442, 2024-09-21T03:42:40.493 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000106670, 2024-09-21T03:42:40.493 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000241 2024-09-21T03:42:40.493 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T03:42:40.493 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_recv_encrypted_bytes": 841664, 2024-09-21T03:42:40.493 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_send_encrypted_bytes": 64928 2024-09-21T03:42:40.493 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T03:42:40.493 INFO:teuthology.orchestra.run.smithi114.stdout: "AsyncMessenger::Worker-1": { 2024-09-21T03:42:40.493 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_recv_messages": 34, 2024-09-21T03:42:40.493 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_send_messages": 104, 2024-09-21T03:42:40.493 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_recv_bytes": 11184, 2024-09-21T03:42:40.493 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_send_bytes": 211141, 2024-09-21T03:42:40.493 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_created_connections": 8, 2024-09-21T03:42:40.493 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_active_connections": 5, 2024-09-21T03:42:40.493 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_running_total_time": 0.013326586, 2024-09-21T03:42:40.494 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_running_send_time": 0.006043823, 2024-09-21T03:42:40.494 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_running_recv_time": 0.006375495, 2024-09-21T03:42:40.494 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_running_fast_dispatch_time": 0.000375348, 2024-09-21T03:42:40.494 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T03:42:40.494 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 103, 2024-09-21T03:42:40.494 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.020836310, 2024-09-21T03:42:40.494 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000202294 2024-09-21T03:42:40.494 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T03:42:40.494 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_handle_ack_lat": { 2024-09-21T03:42:40.494 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 37, 2024-09-21T03:42:40.494 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000005489, 2024-09-21T03:42:40.494 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000148 2024-09-21T03:42:40.494 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T03:42:40.495 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_recv_encrypted_bytes": 4704, 2024-09-21T03:42:40.495 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_send_encrypted_bytes": 201808 2024-09-21T03:42:40.495 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T03:42:40.495 INFO:teuthology.orchestra.run.smithi114.stdout: "AsyncMessenger::Worker-2": { 2024-09-21T03:42:40.495 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_recv_messages": 24, 2024-09-21T03:42:40.495 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_send_messages": 26, 2024-09-21T03:42:40.495 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_recv_bytes": 5909, 2024-09-21T03:42:40.495 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_send_bytes": 63774, 2024-09-21T03:42:40.495 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_created_connections": 11, 2024-09-21T03:42:40.495 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_active_connections": 4, 2024-09-21T03:42:40.495 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_running_total_time": 0.007023593, 2024-09-21T03:42:40.495 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_running_send_time": 0.001014011, 2024-09-21T03:42:40.495 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_running_recv_time": 0.004917072, 2024-09-21T03:42:40.496 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_running_fast_dispatch_time": 0.000879591, 2024-09-21T03:42:40.496 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T03:42:40.496 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 24, 2024-09-21T03:42:40.496 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.025767064, 2024-09-21T03:42:40.496 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.001073627 2024-09-21T03:42:40.496 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T03:42:40.496 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_handle_ack_lat": { 2024-09-21T03:42:40.496 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 11, 2024-09-21T03:42:40.496 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000001605, 2024-09-21T03:42:40.496 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000145 2024-09-21T03:42:40.496 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T03:42:40.496 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_recv_encrypted_bytes": 928, 2024-09-21T03:42:40.496 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_send_encrypted_bytes": 736 2024-09-21T03:42:40.497 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T03:42:40.497 INFO:teuthology.orchestra.run.smithi114.stdout: "cct": { 2024-09-21T03:42:40.497 INFO:teuthology.orchestra.run.smithi114.stdout: "total_workers": 1, 2024-09-21T03:42:40.497 INFO:teuthology.orchestra.run.smithi114.stdout: "unhealthy_workers": 0 2024-09-21T03:42:40.497 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T03:42:40.497 INFO:teuthology.orchestra.run.smithi114.stdout: "finisher-MDSRank": { 2024-09-21T03:42:40.497 INFO:teuthology.orchestra.run.smithi114.stdout: "queue_len": 0, 2024-09-21T03:42:40.497 INFO:teuthology.orchestra.run.smithi114.stdout: "complete_latency": { 2024-09-21T03:42:40.497 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 54, 2024-09-21T03:42:40.497 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.001870470, 2024-09-21T03:42:40.497 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000034638 2024-09-21T03:42:40.497 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T03:42:40.497 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T03:42:40.497 INFO:teuthology.orchestra.run.smithi114.stdout: "finisher-PurgeQueue": { 2024-09-21T03:42:40.497 INFO:teuthology.orchestra.run.smithi114.stdout: "queue_len": 0, 2024-09-21T03:42:40.497 INFO:teuthology.orchestra.run.smithi114.stdout: "complete_latency": { 2024-09-21T03:42:40.498 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 2, 2024-09-21T03:42:40.498 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000037273, 2024-09-21T03:42:40.498 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000018636 2024-09-21T03:42:40.498 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T03:42:40.498 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T03:42:40.498 INFO:teuthology.orchestra.run.smithi114.stdout: "mds": { 2024-09-21T03:42:40.498 INFO:teuthology.orchestra.run.smithi114.stdout: "request": 7, 2024-09-21T03:42:40.498 INFO:teuthology.orchestra.run.smithi114.stdout: "reply": 5, 2024-09-21T03:42:40.498 INFO:teuthology.orchestra.run.smithi114.stdout: "reply_latency": { 2024-09-21T03:42:40.498 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 5, 2024-09-21T03:42:40.498 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.001859263, 2024-09-21T03:42:40.498 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000371852 2024-09-21T03:42:40.498 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T03:42:40.498 INFO:teuthology.orchestra.run.smithi114.stdout: "slow_reply": 0, 2024-09-21T03:42:40.499 INFO:teuthology.orchestra.run.smithi114.stdout: "forward": 2, 2024-09-21T03:42:40.499 INFO:teuthology.orchestra.run.smithi114.stdout: "dir_fetch_complete": 0, 2024-09-21T03:42:40.499 INFO:teuthology.orchestra.run.smithi114.stdout: "dir_fetch_keys": 0, 2024-09-21T03:42:40.499 INFO:teuthology.orchestra.run.smithi114.stdout: "dir_commit": 11, 2024-09-21T03:42:40.499 INFO:teuthology.orchestra.run.smithi114.stdout: "dir_split": 0, 2024-09-21T03:42:40.499 INFO:teuthology.orchestra.run.smithi114.stdout: "dir_merge": 0, 2024-09-21T03:42:40.499 INFO:teuthology.orchestra.run.smithi114.stdout: "inodes": 14, 2024-09-21T03:42:40.499 INFO:teuthology.orchestra.run.smithi114.stdout: "inodes_top": 0, 2024-09-21T03:42:40.499 INFO:teuthology.orchestra.run.smithi114.stdout: "inodes_bottom": 14, 2024-09-21T03:42:40.499 INFO:teuthology.orchestra.run.smithi114.stdout: "inodes_pin_tail": 0, 2024-09-21T03:42:40.499 INFO:teuthology.orchestra.run.smithi114.stdout: "inodes_pinned": 14, 2024-09-21T03:42:40.499 INFO:teuthology.orchestra.run.smithi114.stdout: "inodes_expired": 0, 2024-09-21T03:42:40.499 INFO:teuthology.orchestra.run.smithi114.stdout: "inodes_with_caps": 4, 2024-09-21T03:42:40.499 INFO:teuthology.orchestra.run.smithi114.stdout: "caps": 4, 2024-09-21T03:42:40.499 INFO:teuthology.orchestra.run.smithi114.stdout: "subtrees": 2, 2024-09-21T03:42:40.499 INFO:teuthology.orchestra.run.smithi114.stdout: "traverse": 18, 2024-09-21T03:42:40.500 INFO:teuthology.orchestra.run.smithi114.stdout: "traverse_hit": 14, 2024-09-21T03:42:40.500 INFO:teuthology.orchestra.run.smithi114.stdout: "traverse_forward": 0, 2024-09-21T03:42:40.500 INFO:teuthology.orchestra.run.smithi114.stdout: "traverse_discover": 2, 2024-09-21T03:42:40.500 INFO:teuthology.orchestra.run.smithi114.stdout: "traverse_dir_fetch": 0, 2024-09-21T03:42:40.500 INFO:teuthology.orchestra.run.smithi114.stdout: "traverse_remote_ino": 0, 2024-09-21T03:42:40.500 INFO:teuthology.orchestra.run.smithi114.stdout: "traverse_lock": 0, 2024-09-21T03:42:40.500 INFO:teuthology.orchestra.run.smithi114.stdout: "load_cent": 0, 2024-09-21T03:42:40.500 INFO:teuthology.orchestra.run.smithi114.stdout: "q": 0, 2024-09-21T03:42:40.500 INFO:teuthology.orchestra.run.smithi114.stdout: "exported": 0, 2024-09-21T03:42:40.500 INFO:teuthology.orchestra.run.smithi114.stdout: "exported_inodes": 0, 2024-09-21T03:42:40.500 INFO:teuthology.orchestra.run.smithi114.stdout: "imported": 0, 2024-09-21T03:42:40.500 INFO:teuthology.orchestra.run.smithi114.stdout: "imported_inodes": 0, 2024-09-21T03:42:40.500 INFO:teuthology.orchestra.run.smithi114.stdout: "openino_dir_fetch": 0, 2024-09-21T03:42:40.500 INFO:teuthology.orchestra.run.smithi114.stdout: "openino_backtrace_fetch": 0, 2024-09-21T03:42:40.500 INFO:teuthology.orchestra.run.smithi114.stdout: "openino_peer_discover": 0, 2024-09-21T03:42:40.500 INFO:teuthology.orchestra.run.smithi114.stdout: "root_rfiles": 8, 2024-09-21T03:42:40.501 INFO:teuthology.orchestra.run.smithi114.stdout: "root_rbytes": 202106210, 2024-09-21T03:42:40.501 INFO:teuthology.orchestra.run.smithi114.stdout: "root_rsnaps": 0, 2024-09-21T03:42:40.501 INFO:teuthology.orchestra.run.smithi114.stdout: "scrub_backtrace_fetch": 0, 2024-09-21T03:42:40.501 INFO:teuthology.orchestra.run.smithi114.stdout: "scrub_set_tag": 0, 2024-09-21T03:42:40.501 INFO:teuthology.orchestra.run.smithi114.stdout: "scrub_backtrace_repaired": 0, 2024-09-21T03:42:40.501 INFO:teuthology.orchestra.run.smithi114.stdout: "scrub_inotable_repaired": 0, 2024-09-21T03:42:40.501 INFO:teuthology.orchestra.run.smithi114.stdout: "scrub_dir_inodes": 0, 2024-09-21T03:42:40.501 INFO:teuthology.orchestra.run.smithi114.stdout: "scrub_dir_base_inodes": 0, 2024-09-21T03:42:40.501 INFO:teuthology.orchestra.run.smithi114.stdout: "scrub_dirfrag_rstats": 0, 2024-09-21T03:42:40.501 INFO:teuthology.orchestra.run.smithi114.stdout: "scrub_file_inodes": 0, 2024-09-21T03:42:40.501 INFO:teuthology.orchestra.run.smithi114.stdout: "handle_inode_file_caps": 0, 2024-09-21T03:42:40.501 INFO:teuthology.orchestra.run.smithi114.stdout: "ceph_cap_op_revoke": 4, 2024-09-21T03:42:40.501 INFO:teuthology.orchestra.run.smithi114.stdout: "ceph_cap_op_grant": 0, 2024-09-21T03:42:40.501 INFO:teuthology.orchestra.run.smithi114.stdout: "ceph_cap_op_trunc": 0, 2024-09-21T03:42:40.501 INFO:teuthology.orchestra.run.smithi114.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-21T03:42:40.502 INFO:teuthology.orchestra.run.smithi114.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-21T03:42:40.502 INFO:teuthology.orchestra.run.smithi114.stdout: "handle_client_caps": 4, 2024-09-21T03:42:40.502 INFO:teuthology.orchestra.run.smithi114.stdout: "handle_client_caps_dirty": 0, 2024-09-21T03:42:40.502 INFO:teuthology.orchestra.run.smithi114.stdout: "handle_client_cap_release": 0, 2024-09-21T03:42:40.502 INFO:teuthology.orchestra.run.smithi114.stdout: "process_request_cap_release": 2 2024-09-21T03:42:40.502 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T03:42:40.502 INFO:teuthology.orchestra.run.smithi114.stdout: "mds_cache": { 2024-09-21T03:42:40.502 INFO:teuthology.orchestra.run.smithi114.stdout: "dir_update": 0, 2024-09-21T03:42:40.502 INFO:teuthology.orchestra.run.smithi114.stdout: "dir_update_receipt": 0, 2024-09-21T03:42:40.502 INFO:teuthology.orchestra.run.smithi114.stdout: "dir_try_discover": 0, 2024-09-21T03:42:40.502 INFO:teuthology.orchestra.run.smithi114.stdout: "dir_send_discover": 3, 2024-09-21T03:42:40.502 INFO:teuthology.orchestra.run.smithi114.stdout: "dir_handle_discover": 0, 2024-09-21T03:42:40.502 INFO:teuthology.orchestra.run.smithi114.stdout: "num_strays": 0, 2024-09-21T03:42:40.502 INFO:teuthology.orchestra.run.smithi114.stdout: "num_strays_delayed": 0, 2024-09-21T03:42:40.502 INFO:teuthology.orchestra.run.smithi114.stdout: "num_strays_enqueuing": 0, 2024-09-21T03:42:40.503 INFO:teuthology.orchestra.run.smithi114.stdout: "strays_created": 0, 2024-09-21T03:42:40.503 INFO:teuthology.orchestra.run.smithi114.stdout: "strays_enqueued": 0, 2024-09-21T03:42:40.503 INFO:teuthology.orchestra.run.smithi114.stdout: "strays_reintegrated": 0, 2024-09-21T03:42:40.503 INFO:teuthology.orchestra.run.smithi114.stdout: "strays_migrated": 0, 2024-09-21T03:42:40.503 INFO:teuthology.orchestra.run.smithi114.stdout: "num_recovering_processing": 0, 2024-09-21T03:42:40.503 INFO:teuthology.orchestra.run.smithi114.stdout: "num_recovering_enqueued": 0, 2024-09-21T03:42:40.503 INFO:teuthology.orchestra.run.smithi114.stdout: "num_recovering_prioritized": 0, 2024-09-21T03:42:40.503 INFO:teuthology.orchestra.run.smithi114.stdout: "recovery_started": 0, 2024-09-21T03:42:40.503 INFO:teuthology.orchestra.run.smithi114.stdout: "recovery_completed": 0, 2024-09-21T03:42:40.503 INFO:teuthology.orchestra.run.smithi114.stdout: "ireq_quiesce_path": 3, 2024-09-21T03:42:40.503 INFO:teuthology.orchestra.run.smithi114.stdout: "ireq_quiesce_inode": 15, 2024-09-21T03:42:40.503 INFO:teuthology.orchestra.run.smithi114.stdout: "ireq_enqueue_scrub": 0, 2024-09-21T03:42:40.503 INFO:teuthology.orchestra.run.smithi114.stdout: "ireq_exportdir": 0, 2024-09-21T03:42:40.503 INFO:teuthology.orchestra.run.smithi114.stdout: "ireq_flush": 0, 2024-09-21T03:42:40.503 INFO:teuthology.orchestra.run.smithi114.stdout: "ireq_fragmentdir": 0, 2024-09-21T03:42:40.503 INFO:teuthology.orchestra.run.smithi114.stdout: "ireq_fragstats": 0, 2024-09-21T03:42:40.503 INFO:teuthology.orchestra.run.smithi114.stdout: "ireq_inodestats": 0, 2024-09-21T03:42:40.504 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_started": 0, 2024-09-21T03:42:40.504 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_succeeded": 0, 2024-09-21T03:42:40.504 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_write_failed": 0 2024-09-21T03:42:40.504 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T03:42:40.504 INFO:teuthology.orchestra.run.smithi114.stdout: "mds_log": { 2024-09-21T03:42:40.504 INFO:teuthology.orchestra.run.smithi114.stdout: "evlrg": 0, 2024-09-21T03:42:40.504 INFO:teuthology.orchestra.run.smithi114.stdout: "evadd": 7, 2024-09-21T03:42:40.504 INFO:teuthology.orchestra.run.smithi114.stdout: "evex": 0, 2024-09-21T03:42:40.504 INFO:teuthology.orchestra.run.smithi114.stdout: "evtrm": 0, 2024-09-21T03:42:40.504 INFO:teuthology.orchestra.run.smithi114.stdout: "ev": 7, 2024-09-21T03:42:40.504 INFO:teuthology.orchestra.run.smithi114.stdout: "evexg": 0, 2024-09-21T03:42:40.504 INFO:teuthology.orchestra.run.smithi114.stdout: "evexd": 0, 2024-09-21T03:42:40.504 INFO:teuthology.orchestra.run.smithi114.stdout: "segadd": 2, 2024-09-21T03:42:40.504 INFO:teuthology.orchestra.run.smithi114.stdout: "segex": 0, 2024-09-21T03:42:40.504 INFO:teuthology.orchestra.run.smithi114.stdout: "segtrm": 0, 2024-09-21T03:42:40.504 INFO:teuthology.orchestra.run.smithi114.stdout: "seg": 2, 2024-09-21T03:42:40.504 INFO:teuthology.orchestra.run.smithi114.stdout: "segmjr": 2, 2024-09-21T03:42:40.505 INFO:teuthology.orchestra.run.smithi114.stdout: "segexg": 0, 2024-09-21T03:42:40.505 INFO:teuthology.orchestra.run.smithi114.stdout: "segexd": 0, 2024-09-21T03:42:40.505 INFO:teuthology.orchestra.run.smithi114.stdout: "expos": 4194304, 2024-09-21T03:42:40.505 INFO:teuthology.orchestra.run.smithi114.stdout: "wrpos": 4196449, 2024-09-21T03:42:40.505 INFO:teuthology.orchestra.run.smithi114.stdout: "rdpos": 0, 2024-09-21T03:42:40.505 INFO:teuthology.orchestra.run.smithi114.stdout: "jlat": { 2024-09-21T03:42:40.505 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 6, 2024-09-21T03:42:40.505 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.011340031, 2024-09-21T03:42:40.505 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.001890005 2024-09-21T03:42:40.505 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T03:42:40.505 INFO:teuthology.orchestra.run.smithi114.stdout: "replayed": 0 2024-09-21T03:42:40.505 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T03:42:40.505 INFO:teuthology.orchestra.run.smithi114.stdout: "mds_mem": { 2024-09-21T03:42:40.505 INFO:teuthology.orchestra.run.smithi114.stdout: "ino": 17, 2024-09-21T03:42:40.506 INFO:teuthology.orchestra.run.smithi114.stdout: "ino+": 17, 2024-09-21T03:42:40.506 INFO:teuthology.orchestra.run.smithi114.stdout: "ino-": 0, 2024-09-21T03:42:40.506 INFO:teuthology.orchestra.run.smithi114.stdout: "dir": 15, 2024-09-21T03:42:40.506 INFO:teuthology.orchestra.run.smithi114.stdout: "dir+": 16, 2024-09-21T03:42:40.506 INFO:teuthology.orchestra.run.smithi114.stdout: "dir-": 1, 2024-09-21T03:42:40.506 INFO:teuthology.orchestra.run.smithi114.stdout: "dn": 14, 2024-09-21T03:42:40.506 INFO:teuthology.orchestra.run.smithi114.stdout: "dn+": 14, 2024-09-21T03:42:40.506 INFO:teuthology.orchestra.run.smithi114.stdout: "dn-": 0, 2024-09-21T03:42:40.506 INFO:teuthology.orchestra.run.smithi114.stdout: "cap": 4, 2024-09-21T03:42:40.506 INFO:teuthology.orchestra.run.smithi114.stdout: "cap+": 5, 2024-09-21T03:42:40.506 INFO:teuthology.orchestra.run.smithi114.stdout: "cap-": 1, 2024-09-21T03:42:40.506 INFO:teuthology.orchestra.run.smithi114.stdout: "rss": 120764, 2024-09-21T03:42:40.506 INFO:teuthology.orchestra.run.smithi114.stdout: "heap": 324808704 2024-09-21T03:42:40.506 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T03:42:40.506 INFO:teuthology.orchestra.run.smithi114.stdout: "mds_server": { 2024-09-21T03:42:40.506 INFO:teuthology.orchestra.run.smithi114.stdout: "dispatch_client_request": 9, 2024-09-21T03:42:40.507 INFO:teuthology.orchestra.run.smithi114.stdout: "dispatch_server_request": 0, 2024-09-21T03:42:40.507 INFO:teuthology.orchestra.run.smithi114.stdout: "handle_client_request": 7, 2024-09-21T03:42:40.507 INFO:teuthology.orchestra.run.smithi114.stdout: "handle_client_session": 12, 2024-09-21T03:42:40.507 INFO:teuthology.orchestra.run.smithi114.stdout: "handle_peer_request": 0, 2024-09-21T03:42:40.507 INFO:teuthology.orchestra.run.smithi114.stdout: "req_create_latency": { 2024-09-21T03:42:40.507 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-09-21T03:42:40.507 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-09-21T03:42:40.507 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-09-21T03:42:40.507 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T03:42:40.507 INFO:teuthology.orchestra.run.smithi114.stdout: "req_getattr_latency": { 2024-09-21T03:42:40.507 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 5, 2024-09-21T03:42:40.507 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.001859263, 2024-09-21T03:42:40.507 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000371852 2024-09-21T03:42:40.507 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T03:42:40.508 INFO:teuthology.orchestra.run.smithi114.stdout: "req_getfilelock_latency": { 2024-09-21T03:42:40.508 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-09-21T03:42:40.508 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-09-21T03:42:40.508 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-09-21T03:42:40.508 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T03:42:40.508 INFO:teuthology.orchestra.run.smithi114.stdout: "req_link_latency": { 2024-09-21T03:42:40.508 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-09-21T03:42:40.508 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-09-21T03:42:40.508 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-09-21T03:42:40.508 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T03:42:40.508 INFO:teuthology.orchestra.run.smithi114.stdout: "req_lookup_latency": { 2024-09-21T03:42:40.508 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-09-21T03:42:40.508 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-09-21T03:42:40.508 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-09-21T03:42:40.508 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T03:42:40.508 INFO:teuthology.orchestra.run.smithi114.stdout: "req_lookuphash_latency": { 2024-09-21T03:42:40.508 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-09-21T03:42:40.509 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-09-21T03:42:40.509 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-09-21T03:42:40.509 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T03:42:40.509 INFO:teuthology.orchestra.run.smithi114.stdout: "req_lookupino_latency": { 2024-09-21T03:42:40.509 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-09-21T03:42:40.509 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-09-21T03:42:40.509 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-09-21T03:42:40.509 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T03:42:40.509 INFO:teuthology.orchestra.run.smithi114.stdout: "req_lookupname_latency": { 2024-09-21T03:42:40.509 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-09-21T03:42:40.509 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-09-21T03:42:40.509 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-09-21T03:42:40.509 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T03:42:40.509 INFO:teuthology.orchestra.run.smithi114.stdout: "req_lookupparent_latency": { 2024-09-21T03:42:40.509 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-09-21T03:42:40.510 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-09-21T03:42:40.510 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-09-21T03:42:40.510 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T03:42:40.510 INFO:teuthology.orchestra.run.smithi114.stdout: "req_lookupsnap_latency": { 2024-09-21T03:42:40.510 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-09-21T03:42:40.510 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-09-21T03:42:40.510 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-09-21T03:42:40.510 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T03:42:40.510 INFO:teuthology.orchestra.run.smithi114.stdout: "req_lssnap_latency": { 2024-09-21T03:42:40.510 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-09-21T03:42:40.510 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-09-21T03:42:40.511 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-09-21T03:42:40.511 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T03:42:40.511 INFO:teuthology.orchestra.run.smithi114.stdout: "req_mkdir_latency": { 2024-09-21T03:42:40.511 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-09-21T03:42:40.511 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-09-21T03:42:40.511 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-09-21T03:42:40.511 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T03:42:40.511 INFO:teuthology.orchestra.run.smithi114.stdout: "req_mknod_latency": { 2024-09-21T03:42:40.511 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-09-21T03:42:40.511 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-09-21T03:42:40.511 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-09-21T03:42:40.511 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T03:42:40.512 INFO:teuthology.orchestra.run.smithi114.stdout: "req_mksnap_latency": { 2024-09-21T03:42:40.512 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-09-21T03:42:40.512 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-09-21T03:42:40.512 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-09-21T03:42:40.512 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T03:42:40.512 INFO:teuthology.orchestra.run.smithi114.stdout: "req_open_latency": { 2024-09-21T03:42:40.512 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-09-21T03:42:40.512 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-09-21T03:42:40.512 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-09-21T03:42:40.512 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T03:42:40.512 INFO:teuthology.orchestra.run.smithi114.stdout: "req_readdir_latency": { 2024-09-21T03:42:40.512 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-09-21T03:42:40.512 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-09-21T03:42:40.512 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-09-21T03:42:40.512 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T03:42:40.512 INFO:teuthology.orchestra.run.smithi114.stdout: "req_rename_latency": { 2024-09-21T03:42:40.513 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-09-21T03:42:40.513 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-09-21T03:42:40.513 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-09-21T03:42:40.513 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T03:42:40.513 INFO:teuthology.orchestra.run.smithi114.stdout: "req_renamesnap_latency": { 2024-09-21T03:42:40.513 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-09-21T03:42:40.513 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-09-21T03:42:40.513 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-09-21T03:42:40.513 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T03:42:40.513 INFO:teuthology.orchestra.run.smithi114.stdout: "req_snapdiff_latency": { 2024-09-21T03:42:40.513 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-09-21T03:42:40.513 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-09-21T03:42:40.513 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-09-21T03:42:40.513 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T03:42:40.513 INFO:teuthology.orchestra.run.smithi114.stdout: "req_rmdir_latency": { 2024-09-21T03:42:40.513 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-09-21T03:42:40.514 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-09-21T03:42:40.514 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-09-21T03:42:40.514 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T03:42:40.514 INFO:teuthology.orchestra.run.smithi114.stdout: "req_rmsnap_latency": { 2024-09-21T03:42:40.514 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-09-21T03:42:40.514 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-09-21T03:42:40.514 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-09-21T03:42:40.514 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T03:42:40.514 INFO:teuthology.orchestra.run.smithi114.stdout: "req_rmxattr_latency": { 2024-09-21T03:42:40.514 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-09-21T03:42:40.514 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-09-21T03:42:40.514 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-09-21T03:42:40.514 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T03:42:40.514 INFO:teuthology.orchestra.run.smithi114.stdout: "req_setattr_latency": { 2024-09-21T03:42:40.514 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-09-21T03:42:40.514 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-09-21T03:42:40.515 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-09-21T03:42:40.515 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T03:42:40.515 INFO:teuthology.orchestra.run.smithi114.stdout: "req_setdirlayout_latency": { 2024-09-21T03:42:40.515 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-09-21T03:42:40.515 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-09-21T03:42:40.515 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-09-21T03:42:40.515 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T03:42:40.515 INFO:teuthology.orchestra.run.smithi114.stdout: "req_setfilelock_latency": { 2024-09-21T03:42:40.515 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-09-21T03:42:40.515 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-09-21T03:42:40.515 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-09-21T03:42:40.515 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T03:42:40.515 INFO:teuthology.orchestra.run.smithi114.stdout: "req_setlayout_latency": { 2024-09-21T03:42:40.515 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-09-21T03:42:40.515 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-09-21T03:42:40.516 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-09-21T03:42:40.516 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T03:42:40.516 INFO:teuthology.orchestra.run.smithi114.stdout: "req_setxattr_latency": { 2024-09-21T03:42:40.516 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-09-21T03:42:40.516 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-09-21T03:42:40.516 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-09-21T03:42:40.516 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T03:42:40.516 INFO:teuthology.orchestra.run.smithi114.stdout: "req_symlink_latency": { 2024-09-21T03:42:40.516 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-09-21T03:42:40.516 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-09-21T03:42:40.516 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-09-21T03:42:40.516 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T03:42:40.516 INFO:teuthology.orchestra.run.smithi114.stdout: "req_unlink_latency": { 2024-09-21T03:42:40.516 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-09-21T03:42:40.516 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-09-21T03:42:40.516 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-09-21T03:42:40.516 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T03:42:40.517 INFO:teuthology.orchestra.run.smithi114.stdout: "cap_revoke_eviction": 0, 2024-09-21T03:42:40.517 INFO:teuthology.orchestra.run.smithi114.stdout: "cap_acquisition_throttle": 0, 2024-09-21T03:42:40.517 INFO:teuthology.orchestra.run.smithi114.stdout: "req_getvxattr_latency": { 2024-09-21T03:42:40.517 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-09-21T03:42:40.517 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-09-21T03:42:40.517 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-09-21T03:42:40.517 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T03:42:40.517 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T03:42:40.517 INFO:teuthology.orchestra.run.smithi114.stdout: "mds_sessions": { 2024-09-21T03:42:40.517 INFO:teuthology.orchestra.run.smithi114.stdout: "session_count": 1, 2024-09-21T03:42:40.517 INFO:teuthology.orchestra.run.smithi114.stdout: "session_add": 3, 2024-09-21T03:42:40.517 INFO:teuthology.orchestra.run.smithi114.stdout: "session_remove": 2, 2024-09-21T03:42:40.517 INFO:teuthology.orchestra.run.smithi114.stdout: "sessions_open": 1, 2024-09-21T03:42:40.517 INFO:teuthology.orchestra.run.smithi114.stdout: "sessions_stale": 0, 2024-09-21T03:42:40.517 INFO:teuthology.orchestra.run.smithi114.stdout: "total_load": 4, 2024-09-21T03:42:40.517 INFO:teuthology.orchestra.run.smithi114.stdout: "average_load": 4, 2024-09-21T03:42:40.517 INFO:teuthology.orchestra.run.smithi114.stdout: "avg_session_uptime": 196, 2024-09-21T03:42:40.518 INFO:teuthology.orchestra.run.smithi114.stdout: "mdthresh_evicted": 0 2024-09-21T03:42:40.518 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T03:42:40.518 INFO:teuthology.orchestra.run.smithi114.stdout: "mempool": { 2024-09-21T03:42:40.518 INFO:teuthology.orchestra.run.smithi114.stdout: "bloom_filter_bytes": 0, 2024-09-21T03:42:40.518 INFO:teuthology.orchestra.run.smithi114.stdout: "bloom_filter_items": 0, 2024-09-21T03:42:40.518 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_alloc_bytes": 0, 2024-09-21T03:42:40.518 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_alloc_items": 0, 2024-09-21T03:42:40.518 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_cache_data_bytes": 0, 2024-09-21T03:42:40.518 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_cache_data_items": 0, 2024-09-21T03:42:40.518 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-21T03:42:40.518 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_cache_onode_items": 0, 2024-09-21T03:42:40.518 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-21T03:42:40.518 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_cache_meta_items": 0, 2024-09-21T03:42:40.518 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_cache_other_bytes": 0, 2024-09-21T03:42:40.518 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_cache_other_items": 0, 2024-09-21T03:42:40.519 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-21T03:42:40.519 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_cache_buffer_items": 0, 2024-09-21T03:42:40.519 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_extent_bytes": 0, 2024-09-21T03:42:40.519 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_extent_items": 0, 2024-09-21T03:42:40.519 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_blob_bytes": 0, 2024-09-21T03:42:40.519 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_blob_items": 0, 2024-09-21T03:42:40.519 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-21T03:42:40.519 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_shared_blob_items": 0, 2024-09-21T03:42:40.519 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-21T03:42:40.519 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_inline_bl_items": 0, 2024-09-21T03:42:40.519 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_fsck_bytes": 0, 2024-09-21T03:42:40.519 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_fsck_items": 0, 2024-09-21T03:42:40.519 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_txc_bytes": 0, 2024-09-21T03:42:40.519 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_txc_items": 0, 2024-09-21T03:42:40.519 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-21T03:42:40.519 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_writing_deferred_items": 0, 2024-09-21T03:42:40.519 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_writing_bytes": 0, 2024-09-21T03:42:40.520 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_writing_items": 0, 2024-09-21T03:42:40.520 INFO:teuthology.orchestra.run.smithi114.stdout: "bluefs_bytes": 0, 2024-09-21T03:42:40.520 INFO:teuthology.orchestra.run.smithi114.stdout: "bluefs_items": 0, 2024-09-21T03:42:40.520 INFO:teuthology.orchestra.run.smithi114.stdout: "bluefs_file_reader_bytes": 0, 2024-09-21T03:42:40.520 INFO:teuthology.orchestra.run.smithi114.stdout: "bluefs_file_reader_items": 0, 2024-09-21T03:42:40.520 INFO:teuthology.orchestra.run.smithi114.stdout: "bluefs_file_writer_bytes": 0, 2024-09-21T03:42:40.520 INFO:teuthology.orchestra.run.smithi114.stdout: "bluefs_file_writer_items": 0, 2024-09-21T03:42:40.520 INFO:teuthology.orchestra.run.smithi114.stdout: "buffer_anon_bytes": 246515, 2024-09-21T03:42:40.520 INFO:teuthology.orchestra.run.smithi114.stdout: "buffer_anon_items": 78, 2024-09-21T03:42:40.520 INFO:teuthology.orchestra.run.smithi114.stdout: "buffer_meta_bytes": 0, 2024-09-21T03:42:40.520 INFO:teuthology.orchestra.run.smithi114.stdout: "buffer_meta_items": 0, 2024-09-21T03:42:40.520 INFO:teuthology.orchestra.run.smithi114.stdout: "osd_bytes": 0, 2024-09-21T03:42:40.520 INFO:teuthology.orchestra.run.smithi114.stdout: "osd_items": 0, 2024-09-21T03:42:40.520 INFO:teuthology.orchestra.run.smithi114.stdout: "osd_mapbl_bytes": 0, 2024-09-21T03:42:40.520 INFO:teuthology.orchestra.run.smithi114.stdout: "osd_mapbl_items": 0, 2024-09-21T03:42:40.520 INFO:teuthology.orchestra.run.smithi114.stdout: "osd_pglog_bytes": 0, 2024-09-21T03:42:40.520 INFO:teuthology.orchestra.run.smithi114.stdout: "osd_pglog_items": 0, 2024-09-21T03:42:40.521 INFO:teuthology.orchestra.run.smithi114.stdout: "osdmap_bytes": 8760, 2024-09-21T03:42:40.521 INFO:teuthology.orchestra.run.smithi114.stdout: "osdmap_items": 164, 2024-09-21T03:42:40.521 INFO:teuthology.orchestra.run.smithi114.stdout: "osdmap_mapping_bytes": 0, 2024-09-21T03:42:40.521 INFO:teuthology.orchestra.run.smithi114.stdout: "osdmap_mapping_items": 0, 2024-09-21T03:42:40.521 INFO:teuthology.orchestra.run.smithi114.stdout: "pgmap_bytes": 0, 2024-09-21T03:42:40.521 INFO:teuthology.orchestra.run.smithi114.stdout: "pgmap_items": 0, 2024-09-21T03:42:40.521 INFO:teuthology.orchestra.run.smithi114.stdout: "mds_co_bytes": 69609, 2024-09-21T03:42:40.521 INFO:teuthology.orchestra.run.smithi114.stdout: "mds_co_items": 331, 2024-09-21T03:42:40.521 INFO:teuthology.orchestra.run.smithi114.stdout: "unittest_1_bytes": 0, 2024-09-21T03:42:40.521 INFO:teuthology.orchestra.run.smithi114.stdout: "unittest_1_items": 0, 2024-09-21T03:42:40.521 INFO:teuthology.orchestra.run.smithi114.stdout: "unittest_2_bytes": 0, 2024-09-21T03:42:40.521 INFO:teuthology.orchestra.run.smithi114.stdout: "unittest_2_items": 0 2024-09-21T03:42:40.521 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T03:42:40.521 INFO:teuthology.orchestra.run.smithi114.stdout: "objecter": { 2024-09-21T03:42:40.521 INFO:teuthology.orchestra.run.smithi114.stdout: "op_active": 0, 2024-09-21T03:42:40.521 INFO:teuthology.orchestra.run.smithi114.stdout: "op_laggy": 0, 2024-09-21T03:42:40.522 INFO:teuthology.orchestra.run.smithi114.stdout: "op_send": 43, 2024-09-21T03:42:40.522 INFO:teuthology.orchestra.run.smithi114.stdout: "op_send_bytes": 12728, 2024-09-21T03:42:40.522 INFO:teuthology.orchestra.run.smithi114.stdout: "op_resend": 33, 2024-09-21T03:42:40.522 INFO:teuthology.orchestra.run.smithi114.stdout: "op_reply": 43, 2024-09-21T03:42:40.522 INFO:teuthology.orchestra.run.smithi114.stdout: "op_latency": { 2024-09-21T03:42:40.522 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 43, 2024-09-21T03:42:40.522 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.253998764, 2024-09-21T03:42:40.522 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.005906948 2024-09-21T03:42:40.522 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T03:42:40.522 INFO:teuthology.orchestra.run.smithi114.stdout: "op_inflight": 0, 2024-09-21T03:42:40.522 INFO:teuthology.orchestra.run.smithi114.stdout: "oplen_avg": { 2024-09-21T03:42:40.522 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 43, 2024-09-21T03:42:40.522 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 66 2024-09-21T03:42:40.522 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T03:42:40.522 INFO:teuthology.orchestra.run.smithi114.stdout: "op": 43, 2024-09-21T03:42:40.522 INFO:teuthology.orchestra.run.smithi114.stdout: "op_r": 0, 2024-09-21T03:42:40.523 INFO:teuthology.orchestra.run.smithi114.stdout: "op_w": 43, 2024-09-21T03:42:40.523 INFO:teuthology.orchestra.run.smithi114.stdout: "op_rmw": 0, 2024-09-21T03:42:40.523 INFO:teuthology.orchestra.run.smithi114.stdout: "op_pg": 0, 2024-09-21T03:42:40.523 INFO:teuthology.orchestra.run.smithi114.stdout: "osdop_stat": 0, 2024-09-21T03:42:40.523 INFO:teuthology.orchestra.run.smithi114.stdout: "osdop_create": 10, 2024-09-21T03:42:40.523 INFO:teuthology.orchestra.run.smithi114.stdout: "osdop_read": 0, 2024-09-21T03:42:40.523 INFO:teuthology.orchestra.run.smithi114.stdout: "osdop_write": 6, 2024-09-21T03:42:40.523 INFO:teuthology.orchestra.run.smithi114.stdout: "osdop_writefull": 7, 2024-09-21T03:42:40.523 INFO:teuthology.orchestra.run.smithi114.stdout: "osdop_writesame": 0, 2024-09-21T03:42:40.523 INFO:teuthology.orchestra.run.smithi114.stdout: "osdop_append": 0, 2024-09-21T03:42:40.523 INFO:teuthology.orchestra.run.smithi114.stdout: "osdop_zero": 0, 2024-09-21T03:42:40.523 INFO:teuthology.orchestra.run.smithi114.stdout: "osdop_truncate": 0, 2024-09-21T03:42:40.523 INFO:teuthology.orchestra.run.smithi114.stdout: "osdop_delete": 6, 2024-09-21T03:42:40.523 INFO:teuthology.orchestra.run.smithi114.stdout: "osdop_mapext": 0, 2024-09-21T03:42:40.523 INFO:teuthology.orchestra.run.smithi114.stdout: "osdop_sparse_read": 0, 2024-09-21T03:42:40.524 INFO:teuthology.orchestra.run.smithi114.stdout: "osdop_clonerange": 0, 2024-09-21T03:42:40.524 INFO:teuthology.orchestra.run.smithi114.stdout: "osdop_getxattr": 0, 2024-09-21T03:42:40.524 INFO:teuthology.orchestra.run.smithi114.stdout: "osdop_setxattr": 20, 2024-09-21T03:42:40.524 INFO:teuthology.orchestra.run.smithi114.stdout: "osdop_cmpxattr": 0, 2024-09-21T03:42:40.524 INFO:teuthology.orchestra.run.smithi114.stdout: "osdop_rmxattr": 0, 2024-09-21T03:42:40.524 INFO:teuthology.orchestra.run.smithi114.stdout: "osdop_resetxattrs": 0, 2024-09-21T03:42:40.524 INFO:teuthology.orchestra.run.smithi114.stdout: "osdop_call": 0, 2024-09-21T03:42:40.524 INFO:teuthology.orchestra.run.smithi114.stdout: "osdop_watch": 0, 2024-09-21T03:42:40.524 INFO:teuthology.orchestra.run.smithi114.stdout: "osdop_notify": 0, 2024-09-21T03:42:40.524 INFO:teuthology.orchestra.run.smithi114.stdout: "osdop_src_cmpxattr": 0, 2024-09-21T03:42:40.524 INFO:teuthology.orchestra.run.smithi114.stdout: "osdop_pgls": 0, 2024-09-21T03:42:40.524 INFO:teuthology.orchestra.run.smithi114.stdout: "osdop_pgls_filter": 0, 2024-09-21T03:42:40.524 INFO:teuthology.orchestra.run.smithi114.stdout: "osdop_other": 0, 2024-09-21T03:42:40.524 INFO:teuthology.orchestra.run.smithi114.stdout: "linger_active": 0, 2024-09-21T03:42:40.524 INFO:teuthology.orchestra.run.smithi114.stdout: "linger_send": 0, 2024-09-21T03:42:40.525 INFO:teuthology.orchestra.run.smithi114.stdout: "linger_resend": 0, 2024-09-21T03:42:40.525 INFO:teuthology.orchestra.run.smithi114.stdout: "linger_ping": 0, 2024-09-21T03:42:40.525 INFO:teuthology.orchestra.run.smithi114.stdout: "poolop_active": 0, 2024-09-21T03:42:40.525 INFO:teuthology.orchestra.run.smithi114.stdout: "poolop_send": 0, 2024-09-21T03:42:40.525 INFO:teuthology.orchestra.run.smithi114.stdout: "poolop_resend": 0, 2024-09-21T03:42:40.525 INFO:teuthology.orchestra.run.smithi114.stdout: "poolstat_active": 0, 2024-09-21T03:42:40.525 INFO:teuthology.orchestra.run.smithi114.stdout: "poolstat_send": 0, 2024-09-21T03:42:40.525 INFO:teuthology.orchestra.run.smithi114.stdout: "poolstat_resend": 0, 2024-09-21T03:42:40.525 INFO:teuthology.orchestra.run.smithi114.stdout: "statfs_active": 0, 2024-09-21T03:42:40.525 INFO:teuthology.orchestra.run.smithi114.stdout: "statfs_send": 0, 2024-09-21T03:42:40.525 INFO:teuthology.orchestra.run.smithi114.stdout: "statfs_resend": 0, 2024-09-21T03:42:40.525 INFO:teuthology.orchestra.run.smithi114.stdout: "command_active": 0, 2024-09-21T03:42:40.525 INFO:teuthology.orchestra.run.smithi114.stdout: "command_send": 0, 2024-09-21T03:42:40.525 INFO:teuthology.orchestra.run.smithi114.stdout: "command_resend": 0, 2024-09-21T03:42:40.526 INFO:teuthology.orchestra.run.smithi114.stdout: "map_epoch": 84, 2024-09-21T03:42:40.526 INFO:teuthology.orchestra.run.smithi114.stdout: "map_full": 0, 2024-09-21T03:42:40.526 INFO:teuthology.orchestra.run.smithi114.stdout: "map_inc": 4, 2024-09-21T03:42:40.526 INFO:teuthology.orchestra.run.smithi114.stdout: "osd_sessions": 9, 2024-09-21T03:42:40.526 INFO:teuthology.orchestra.run.smithi114.stdout: "osd_session_open": 9, 2024-09-21T03:42:40.526 INFO:teuthology.orchestra.run.smithi114.stdout: "osd_session_close": 0, 2024-09-21T03:42:40.526 INFO:teuthology.orchestra.run.smithi114.stdout: "osd_laggy": 0, 2024-09-21T03:42:40.526 INFO:teuthology.orchestra.run.smithi114.stdout: "omap_wr": 16, 2024-09-21T03:42:40.526 INFO:teuthology.orchestra.run.smithi114.stdout: "omap_rd": 0, 2024-09-21T03:42:40.526 INFO:teuthology.orchestra.run.smithi114.stdout: "omap_del": 1 2024-09-21T03:42:40.526 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T03:42:40.526 INFO:teuthology.orchestra.run.smithi114.stdout: "oft": { 2024-09-21T03:42:40.526 INFO:teuthology.orchestra.run.smithi114.stdout: "omap_total_objs": 1, 2024-09-21T03:42:40.526 INFO:teuthology.orchestra.run.smithi114.stdout: "omap_total_kv_pairs": 0, 2024-09-21T03:42:40.526 INFO:teuthology.orchestra.run.smithi114.stdout: "omap_total_updates": 0, 2024-09-21T03:42:40.526 INFO:teuthology.orchestra.run.smithi114.stdout: "omap_total_removes": 0 2024-09-21T03:42:40.527 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T03:42:40.527 INFO:teuthology.orchestra.run.smithi114.stdout: "purge_queue": { 2024-09-21T03:42:40.527 INFO:teuthology.orchestra.run.smithi114.stdout: "pq_executing_ops": 0, 2024-09-21T03:42:40.527 INFO:teuthology.orchestra.run.smithi114.stdout: "pq_executing_ops_high_water": 0, 2024-09-21T03:42:40.527 INFO:teuthology.orchestra.run.smithi114.stdout: "pq_executing": 0, 2024-09-21T03:42:40.527 INFO:teuthology.orchestra.run.smithi114.stdout: "pq_executing_high_water": 0, 2024-09-21T03:42:40.527 INFO:teuthology.orchestra.run.smithi114.stdout: "pq_executed_ops": 0, 2024-09-21T03:42:40.527 INFO:teuthology.orchestra.run.smithi114.stdout: "pq_executed": 0, 2024-09-21T03:42:40.527 INFO:teuthology.orchestra.run.smithi114.stdout: "pq_item_in_journal": 0 2024-09-21T03:42:40.527 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T03:42:40.527 INFO:teuthology.orchestra.run.smithi114.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-21T03:42:40.527 INFO:teuthology.orchestra.run.smithi114.stdout: "val": 0, 2024-09-21T03:42:40.527 INFO:teuthology.orchestra.run.smithi114.stdout: "max": 104857600, 2024-09-21T03:42:40.527 INFO:teuthology.orchestra.run.smithi114.stdout: "get_started": 0, 2024-09-21T03:42:40.527 INFO:teuthology.orchestra.run.smithi114.stdout: "get": 438, 2024-09-21T03:42:40.527 INFO:teuthology.orchestra.run.smithi114.stdout: "get_sum": 872244, 2024-09-21T03:42:40.528 INFO:teuthology.orchestra.run.smithi114.stdout: "get_or_fail_fail": 0, 2024-09-21T03:42:40.528 INFO:teuthology.orchestra.run.smithi114.stdout: "get_or_fail_success": 438, 2024-09-21T03:42:40.528 INFO:teuthology.orchestra.run.smithi114.stdout: "take": 0, 2024-09-21T03:42:40.528 INFO:teuthology.orchestra.run.smithi114.stdout: "take_sum": 0, 2024-09-21T03:42:40.528 INFO:teuthology.orchestra.run.smithi114.stdout: "put": 438, 2024-09-21T03:42:40.528 INFO:teuthology.orchestra.run.smithi114.stdout: "put_sum": 872244, 2024-09-21T03:42:40.528 INFO:teuthology.orchestra.run.smithi114.stdout: "wait": { 2024-09-21T03:42:40.528 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-09-21T03:42:40.528 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-09-21T03:42:40.528 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-09-21T03:42:40.528 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T03:42:40.528 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T03:42:40.528 INFO:teuthology.orchestra.run.smithi114.stdout: "throttle-objecter_bytes": { 2024-09-21T03:42:40.528 INFO:teuthology.orchestra.run.smithi114.stdout: "val": 0, 2024-09-21T03:42:40.528 INFO:teuthology.orchestra.run.smithi114.stdout: "max": 104857600, 2024-09-21T03:42:40.528 INFO:teuthology.orchestra.run.smithi114.stdout: "get_started": 0, 2024-09-21T03:42:40.528 INFO:teuthology.orchestra.run.smithi114.stdout: "get": 0, 2024-09-21T03:42:40.529 INFO:teuthology.orchestra.run.smithi114.stdout: "get_sum": 0, 2024-09-21T03:42:40.529 INFO:teuthology.orchestra.run.smithi114.stdout: "get_or_fail_fail": 0, 2024-09-21T03:42:40.529 INFO:teuthology.orchestra.run.smithi114.stdout: "get_or_fail_success": 0, 2024-09-21T03:42:40.529 INFO:teuthology.orchestra.run.smithi114.stdout: "take": 43, 2024-09-21T03:42:40.529 INFO:teuthology.orchestra.run.smithi114.stdout: "take_sum": 12728, 2024-09-21T03:42:40.529 INFO:teuthology.orchestra.run.smithi114.stdout: "put": 43, 2024-09-21T03:42:40.529 INFO:teuthology.orchestra.run.smithi114.stdout: "put_sum": 12728, 2024-09-21T03:42:40.529 INFO:teuthology.orchestra.run.smithi114.stdout: "wait": { 2024-09-21T03:42:40.529 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-09-21T03:42:40.529 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-09-21T03:42:40.529 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-09-21T03:42:40.529 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T03:42:40.529 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T03:42:40.529 INFO:teuthology.orchestra.run.smithi114.stdout: "throttle-objecter_ops": { 2024-09-21T03:42:40.529 INFO:teuthology.orchestra.run.smithi114.stdout: "val": 0, 2024-09-21T03:42:40.529 INFO:teuthology.orchestra.run.smithi114.stdout: "max": 1024, 2024-09-21T03:42:40.529 INFO:teuthology.orchestra.run.smithi114.stdout: "get_started": 0, 2024-09-21T03:42:40.530 INFO:teuthology.orchestra.run.smithi114.stdout: "get": 0, 2024-09-21T03:42:40.530 INFO:teuthology.orchestra.run.smithi114.stdout: "get_sum": 0, 2024-09-21T03:42:40.530 INFO:teuthology.orchestra.run.smithi114.stdout: "get_or_fail_fail": 0, 2024-09-21T03:42:40.530 INFO:teuthology.orchestra.run.smithi114.stdout: "get_or_fail_success": 0, 2024-09-21T03:42:40.530 INFO:teuthology.orchestra.run.smithi114.stdout: "take": 43, 2024-09-21T03:42:40.530 INFO:teuthology.orchestra.run.smithi114.stdout: "take_sum": 43, 2024-09-21T03:42:40.530 INFO:teuthology.orchestra.run.smithi114.stdout: "put": 43, 2024-09-21T03:42:40.530 INFO:teuthology.orchestra.run.smithi114.stdout: "put_sum": 43, 2024-09-21T03:42:40.530 INFO:teuthology.orchestra.run.smithi114.stdout: "wait": { 2024-09-21T03:42:40.530 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-09-21T03:42:40.530 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-09-21T03:42:40.530 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-09-21T03:42:40.530 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T03:42:40.530 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T03:42:40.530 INFO:teuthology.orchestra.run.smithi114.stdout: "throttle-write_buf_throttle": { 2024-09-21T03:42:40.531 INFO:teuthology.orchestra.run.smithi114.stdout: "val": 0, 2024-09-21T03:42:40.531 INFO:teuthology.orchestra.run.smithi114.stdout: "max": 3758096384, 2024-09-21T03:42:40.531 INFO:teuthology.orchestra.run.smithi114.stdout: "get_started": 0, 2024-09-21T03:42:40.531 INFO:teuthology.orchestra.run.smithi114.stdout: "get": 0, 2024-09-21T03:42:40.531 INFO:teuthology.orchestra.run.smithi114.stdout: "get_sum": 0, 2024-09-21T03:42:40.531 INFO:teuthology.orchestra.run.smithi114.stdout: "get_or_fail_fail": 0, 2024-09-21T03:42:40.531 INFO:teuthology.orchestra.run.smithi114.stdout: "get_or_fail_success": 0, 2024-09-21T03:42:40.531 INFO:teuthology.orchestra.run.smithi114.stdout: "take": 0, 2024-09-21T03:42:40.531 INFO:teuthology.orchestra.run.smithi114.stdout: "take_sum": 0, 2024-09-21T03:42:40.531 INFO:teuthology.orchestra.run.smithi114.stdout: "put": 0, 2024-09-21T03:42:40.531 INFO:teuthology.orchestra.run.smithi114.stdout: "put_sum": 0, 2024-09-21T03:42:40.531 INFO:teuthology.orchestra.run.smithi114.stdout: "wait": { 2024-09-21T03:42:40.531 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-09-21T03:42:40.531 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-09-21T03:42:40.531 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-09-21T03:42:40.531 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T03:42:40.532 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T03:42:40.532 INFO:teuthology.orchestra.run.smithi114.stdout: "throttle-write_buf_throttle-0x5625831b16c0": { 2024-09-21T03:42:40.532 INFO:teuthology.orchestra.run.smithi114.stdout: "val": 0, 2024-09-21T03:42:40.532 INFO:teuthology.orchestra.run.smithi114.stdout: "max": 3758096384, 2024-09-21T03:42:40.532 INFO:teuthology.orchestra.run.smithi114.stdout: "get_started": 0, 2024-09-21T03:42:40.532 INFO:teuthology.orchestra.run.smithi114.stdout: "get": 7, 2024-09-21T03:42:40.532 INFO:teuthology.orchestra.run.smithi114.stdout: "get_sum": 2145, 2024-09-21T03:42:40.532 INFO:teuthology.orchestra.run.smithi114.stdout: "get_or_fail_fail": 0, 2024-09-21T03:42:40.532 INFO:teuthology.orchestra.run.smithi114.stdout: "get_or_fail_success": 7, 2024-09-21T03:42:40.532 INFO:teuthology.orchestra.run.smithi114.stdout: "take": 0, 2024-09-21T03:42:40.532 INFO:teuthology.orchestra.run.smithi114.stdout: "take_sum": 0, 2024-09-21T03:42:40.532 INFO:teuthology.orchestra.run.smithi114.stdout: "put": 6, 2024-09-21T03:42:40.532 INFO:teuthology.orchestra.run.smithi114.stdout: "put_sum": 2145, 2024-09-21T03:42:40.532 INFO:teuthology.orchestra.run.smithi114.stdout: "wait": { 2024-09-21T03:42:40.532 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-09-21T03:42:40.532 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-09-21T03:42:40.532 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-09-21T03:42:40.533 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T03:42:40.533 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T03:42:40.533 INFO:teuthology.orchestra.run.smithi114.stdout: "trackedop": { 2024-09-21T03:42:40.533 INFO:teuthology.orchestra.run.smithi114.stdout: "slow_ops_count": 18 2024-09-21T03:42:40.533 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T03:42:40.533 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-21T03:42:41.211 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:42:40 smithi114 ceph-mon[34140]: pgmap v458: 129 pgs: 129 active+clean; 1.1 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-21T03:42:41.250 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:42:40 smithi110 ceph-mon[34000]: pgmap v458: 129 pgs: 129 active+clean; 1.1 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-21T03:42:41.286 INFO:tasks.check_counter:Daemon mds.i mds.exported=0 2024-09-21T03:42:41.286 INFO:tasks.check_counter:Daemon mds.i mds.imported=0 2024-09-21T03:42:41.287 DEBUG:tasks.check_counter:Getting stats from l 2024-09-21T03:42:41.287 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 7c374098-77c9-11ef-baf5-efdc52797490 -- ceph daemon mds.l perf dump 2024-09-21T03:42:41.311 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:42:40 smithi089 ceph-mon[30928]: pgmap v458: 129 pgs: 129 active+clean; 1.1 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-21T03:42:41.531 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/7c374098-77c9-11ef-baf5-efdc52797490/mon.c/config 2024-09-21T03:42:42.051 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-21T03:42:42.051 INFO:teuthology.orchestra.run.smithi114.stdout: "AsyncMessenger::Worker-0": { 2024-09-21T03:42:42.051 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_recv_messages": 4, 2024-09-21T03:42:42.051 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_send_messages": 4, 2024-09-21T03:42:42.051 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_recv_bytes": 2960, 2024-09-21T03:42:42.051 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_send_bytes": 448, 2024-09-21T03:42:42.052 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_created_connections": 4, 2024-09-21T03:42:42.052 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_active_connections": 0, 2024-09-21T03:42:42.052 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_running_total_time": 0.002949037, 2024-09-21T03:42:42.052 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_running_send_time": 0.000036128, 2024-09-21T03:42:42.052 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_running_recv_time": 0.002170535, 2024-09-21T03:42:42.052 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-21T03:42:42.052 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T03:42:42.052 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 3, 2024-09-21T03:42:42.052 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.003862999, 2024-09-21T03:42:42.052 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.001287666 2024-09-21T03:42:42.052 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T03:42:42.052 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_handle_ack_lat": { 2024-09-21T03:42:42.052 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-09-21T03:42:42.052 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-09-21T03:42:42.052 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-09-21T03:42:42.052 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T03:42:42.053 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_recv_encrypted_bytes": 2960, 2024-09-21T03:42:42.053 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_send_encrypted_bytes": 448 2024-09-21T03:42:42.053 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T03:42:42.053 INFO:teuthology.orchestra.run.smithi114.stdout: "AsyncMessenger::Worker-1": { 2024-09-21T03:42:42.053 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_recv_messages": 1, 2024-09-21T03:42:42.053 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_send_messages": 77, 2024-09-21T03:42:42.053 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_recv_bytes": 160, 2024-09-21T03:42:42.053 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_send_bytes": 93744, 2024-09-21T03:42:42.053 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_created_connections": 3, 2024-09-21T03:42:42.053 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_active_connections": 2, 2024-09-21T03:42:42.053 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_running_total_time": 0.007019623, 2024-09-21T03:42:42.053 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_running_send_time": 0.004971901, 2024-09-21T03:42:42.053 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_running_recv_time": 0.001102919, 2024-09-21T03:42:42.053 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-21T03:42:42.053 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T03:42:42.053 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 75, 2024-09-21T03:42:42.054 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.006195473, 2024-09-21T03:42:42.054 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000082606 2024-09-21T03:42:42.054 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T03:42:42.054 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_handle_ack_lat": { 2024-09-21T03:42:42.054 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-09-21T03:42:42.054 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-09-21T03:42:42.054 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-09-21T03:42:42.054 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T03:42:42.054 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_recv_encrypted_bytes": 160, 2024-09-21T03:42:42.054 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_send_encrypted_bytes": 93744 2024-09-21T03:42:42.054 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T03:42:42.054 INFO:teuthology.orchestra.run.smithi114.stdout: "AsyncMessenger::Worker-2": { 2024-09-21T03:42:42.054 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_recv_messages": 108, 2024-09-21T03:42:42.054 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_send_messages": 103, 2024-09-21T03:42:42.055 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_recv_bytes": 805376, 2024-09-21T03:42:42.055 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_send_bytes": 62128, 2024-09-21T03:42:42.055 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_created_connections": 2, 2024-09-21T03:42:42.055 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_active_connections": 1, 2024-09-21T03:42:42.055 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_running_total_time": 0.024436046, 2024-09-21T03:42:42.055 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_running_send_time": 0.009036796, 2024-09-21T03:42:42.055 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_running_recv_time": 0.024044657, 2024-09-21T03:42:42.055 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-21T03:42:42.055 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T03:42:42.055 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 102, 2024-09-21T03:42:42.055 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.006171573, 2024-09-21T03:42:42.055 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000060505 2024-09-21T03:42:42.055 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T03:42:42.055 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_handle_ack_lat": { 2024-09-21T03:42:42.055 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-09-21T03:42:42.055 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-09-21T03:42:42.056 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-09-21T03:42:42.056 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T03:42:42.056 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_recv_encrypted_bytes": 805376, 2024-09-21T03:42:42.056 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_send_encrypted_bytes": 62128 2024-09-21T03:42:42.056 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T03:42:42.056 INFO:teuthology.orchestra.run.smithi114.stdout: "cct": { 2024-09-21T03:42:42.056 INFO:teuthology.orchestra.run.smithi114.stdout: "total_workers": 0, 2024-09-21T03:42:42.056 INFO:teuthology.orchestra.run.smithi114.stdout: "unhealthy_workers": 0 2024-09-21T03:42:42.056 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T03:42:42.056 INFO:teuthology.orchestra.run.smithi114.stdout: "mempool": { 2024-09-21T03:42:42.056 INFO:teuthology.orchestra.run.smithi114.stdout: "bloom_filter_bytes": 0, 2024-09-21T03:42:42.056 INFO:teuthology.orchestra.run.smithi114.stdout: "bloom_filter_items": 0, 2024-09-21T03:42:42.056 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_alloc_bytes": 0, 2024-09-21T03:42:42.056 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_alloc_items": 0, 2024-09-21T03:42:42.057 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_cache_data_bytes": 0, 2024-09-21T03:42:42.057 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_cache_data_items": 0, 2024-09-21T03:42:42.057 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-21T03:42:42.057 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_cache_onode_items": 0, 2024-09-21T03:42:42.057 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-21T03:42:42.057 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_cache_meta_items": 0, 2024-09-21T03:42:42.057 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_cache_other_bytes": 0, 2024-09-21T03:42:42.057 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_cache_other_items": 0, 2024-09-21T03:42:42.057 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-21T03:42:42.057 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_cache_buffer_items": 0, 2024-09-21T03:42:42.057 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_extent_bytes": 0, 2024-09-21T03:42:42.057 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_extent_items": 0, 2024-09-21T03:42:42.057 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_blob_bytes": 0, 2024-09-21T03:42:42.057 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_blob_items": 0, 2024-09-21T03:42:42.058 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-21T03:42:42.058 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_shared_blob_items": 0, 2024-09-21T03:42:42.058 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-21T03:42:42.058 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_inline_bl_items": 0, 2024-09-21T03:42:42.058 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_fsck_bytes": 0, 2024-09-21T03:42:42.058 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_fsck_items": 0, 2024-09-21T03:42:42.058 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_txc_bytes": 0, 2024-09-21T03:42:42.058 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_txc_items": 0, 2024-09-21T03:42:42.058 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-21T03:42:42.058 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_writing_deferred_items": 0, 2024-09-21T03:42:42.058 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_writing_bytes": 0, 2024-09-21T03:42:42.058 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_writing_items": 0, 2024-09-21T03:42:42.058 INFO:teuthology.orchestra.run.smithi114.stdout: "bluefs_bytes": 0, 2024-09-21T03:42:42.058 INFO:teuthology.orchestra.run.smithi114.stdout: "bluefs_items": 0, 2024-09-21T03:42:42.058 INFO:teuthology.orchestra.run.smithi114.stdout: "bluefs_file_reader_bytes": 0, 2024-09-21T03:42:42.059 INFO:teuthology.orchestra.run.smithi114.stdout: "bluefs_file_reader_items": 0, 2024-09-21T03:42:42.059 INFO:teuthology.orchestra.run.smithi114.stdout: "bluefs_file_writer_bytes": 0, 2024-09-21T03:42:42.059 INFO:teuthology.orchestra.run.smithi114.stdout: "bluefs_file_writer_items": 0, 2024-09-21T03:42:42.059 INFO:teuthology.orchestra.run.smithi114.stdout: "buffer_anon_bytes": 148880, 2024-09-21T03:42:42.059 INFO:teuthology.orchestra.run.smithi114.stdout: "buffer_anon_items": 29, 2024-09-21T03:42:42.059 INFO:teuthology.orchestra.run.smithi114.stdout: "buffer_meta_bytes": 0, 2024-09-21T03:42:42.059 INFO:teuthology.orchestra.run.smithi114.stdout: "buffer_meta_items": 0, 2024-09-21T03:42:42.059 INFO:teuthology.orchestra.run.smithi114.stdout: "osd_bytes": 0, 2024-09-21T03:42:42.059 INFO:teuthology.orchestra.run.smithi114.stdout: "osd_items": 0, 2024-09-21T03:42:42.059 INFO:teuthology.orchestra.run.smithi114.stdout: "osd_mapbl_bytes": 0, 2024-09-21T03:42:42.059 INFO:teuthology.orchestra.run.smithi114.stdout: "osd_mapbl_items": 0, 2024-09-21T03:42:42.059 INFO:teuthology.orchestra.run.smithi114.stdout: "osd_pglog_bytes": 0, 2024-09-21T03:42:42.059 INFO:teuthology.orchestra.run.smithi114.stdout: "osd_pglog_items": 0, 2024-09-21T03:42:42.059 INFO:teuthology.orchestra.run.smithi114.stdout: "osdmap_bytes": 0, 2024-09-21T03:42:42.059 INFO:teuthology.orchestra.run.smithi114.stdout: "osdmap_items": 0, 2024-09-21T03:42:42.059 INFO:teuthology.orchestra.run.smithi114.stdout: "osdmap_mapping_bytes": 0, 2024-09-21T03:42:42.060 INFO:teuthology.orchestra.run.smithi114.stdout: "osdmap_mapping_items": 0, 2024-09-21T03:42:42.060 INFO:teuthology.orchestra.run.smithi114.stdout: "pgmap_bytes": 0, 2024-09-21T03:42:42.060 INFO:teuthology.orchestra.run.smithi114.stdout: "pgmap_items": 0, 2024-09-21T03:42:42.060 INFO:teuthology.orchestra.run.smithi114.stdout: "mds_co_bytes": 968, 2024-09-21T03:42:42.060 INFO:teuthology.orchestra.run.smithi114.stdout: "mds_co_items": 2, 2024-09-21T03:42:42.060 INFO:teuthology.orchestra.run.smithi114.stdout: "unittest_1_bytes": 0, 2024-09-21T03:42:42.060 INFO:teuthology.orchestra.run.smithi114.stdout: "unittest_1_items": 0, 2024-09-21T03:42:42.060 INFO:teuthology.orchestra.run.smithi114.stdout: "unittest_2_bytes": 0, 2024-09-21T03:42:42.060 INFO:teuthology.orchestra.run.smithi114.stdout: "unittest_2_items": 0 2024-09-21T03:42:42.060 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T03:42:42.060 INFO:teuthology.orchestra.run.smithi114.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-21T03:42:42.060 INFO:teuthology.orchestra.run.smithi114.stdout: "val": 0, 2024-09-21T03:42:42.060 INFO:teuthology.orchestra.run.smithi114.stdout: "max": 104857600, 2024-09-21T03:42:42.060 INFO:teuthology.orchestra.run.smithi114.stdout: "get_started": 0, 2024-09-21T03:42:42.060 INFO:teuthology.orchestra.run.smithi114.stdout: "get": 109, 2024-09-21T03:42:42.060 INFO:teuthology.orchestra.run.smithi114.stdout: "get_sum": 790175, 2024-09-21T03:42:42.061 INFO:teuthology.orchestra.run.smithi114.stdout: "get_or_fail_fail": 0, 2024-09-21T03:42:42.061 INFO:teuthology.orchestra.run.smithi114.stdout: "get_or_fail_success": 109, 2024-09-21T03:42:42.061 INFO:teuthology.orchestra.run.smithi114.stdout: "take": 0, 2024-09-21T03:42:42.061 INFO:teuthology.orchestra.run.smithi114.stdout: "take_sum": 0, 2024-09-21T03:42:42.061 INFO:teuthology.orchestra.run.smithi114.stdout: "put": 109, 2024-09-21T03:42:42.061 INFO:teuthology.orchestra.run.smithi114.stdout: "put_sum": 790175, 2024-09-21T03:42:42.061 INFO:teuthology.orchestra.run.smithi114.stdout: "wait": { 2024-09-21T03:42:42.061 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-09-21T03:42:42.061 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-09-21T03:42:42.061 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-09-21T03:42:42.061 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T03:42:42.061 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T03:42:42.061 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-21T03:42:42.830 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.l 2024-09-21T03:42:42.831 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.l 2024-09-21T03:42:42.831 DEBUG:teuthology.run_tasks:Unwinding manager ceph-fuse 2024-09-21T03:42:42.844 INFO:tasks.ceph_fuse:Unmounting ceph-fuse clients... 2024-09-21T03:42:42.844 DEBUG:teuthology.orchestra.run.smithi089:> set -ex 2024-09-21T03:42:42.844 DEBUG:teuthology.orchestra.run.smithi089:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T03:42:42.865 DEBUG:teuthology.orchestra.run.smithi089:> set -ex 2024-09-21T03:42:42.865 DEBUG:teuthology.orchestra.run.smithi089:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T03:42:42.924 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 7c374098-77c9-11ef-baf5-efdc52797490 -- ceph osd blocklist ls 2024-09-21T03:42:43.211 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:42:42 smithi114 ceph-mon[34140]: pgmap v459: 129 pgs: 129 active+clean; 1.1 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-21T03:42:43.225 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/7c374098-77c9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:42:43.250 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:42:42 smithi110 ceph-mon[34000]: pgmap v459: 129 pgs: 129 active+clean; 1.1 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-21T03:42:43.311 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:42:42 smithi089 ceph-mon[30928]: pgmap v459: 129 pgs: 129 active+clean; 1.1 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-21T03:42:43.710 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:43.702+0000 7f7be53f2640 1 -- 172.21.15.89:0/4211991126 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7be0108cf0 msgr2=0x7f7be010f520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:43.710 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:43.702+0000 7f7be53f2640 1 --2- 172.21.15.89:0/4211991126 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7be0108cf0 0x7f7be010f520 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f7bd8009ca0 tx=0x7f7bd802a8c0 comp rx=0 tx=0).stop 2024-09-21T03:42:43.710 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:43.702+0000 7f7be53f2640 1 -- 172.21.15.89:0/4211991126 shutdown_connections 2024-09-21T03:42:43.710 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:43.702+0000 7f7be53f2640 1 --2- 172.21.15.89:0/4211991126 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7be01083a0 0x7f7be01087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:43.710 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:43.702+0000 7f7be53f2640 1 --2- 172.21.15.89:0/4211991126 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7be0108cf0 0x7f7be010f520 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:43.710 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:43.702+0000 7f7be53f2640 1 --2- 172.21.15.89:0/4211991126 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7be0102a60 0x7f7be0102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:43.710 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:43.702+0000 7f7be53f2640 1 -- 172.21.15.89:0/4211991126 >> 172.21.15.89:0/4211991126 conn(0x7f7be0079730 msgr2=0x7f7be0079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:42:43.710 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:43.702+0000 7f7be53f2640 1 -- 172.21.15.89:0/4211991126 shutdown_connections 2024-09-21T03:42:43.710 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:43.702+0000 7f7be53f2640 1 -- 172.21.15.89:0/4211991126 wait complete. 2024-09-21T03:42:43.710 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:43.703+0000 7f7be53f2640 1 Processor -- start 2024-09-21T03:42:43.710 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:43.703+0000 7f7be53f2640 1 -- start start 2024-09-21T03:42:43.711 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:43.703+0000 7f7be53f2640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7be0102a60 0x7f7be019a350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:43.711 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:43.703+0000 7f7be53f2640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7be01083a0 0x7f7be019a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:43.711 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:43.703+0000 7f7be53f2640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7be0108cf0 0x7f7be019fb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:43.711 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:43.703+0000 7f7be53f2640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f7be0113960 con 0x7f7be0102a60 2024-09-21T03:42:43.711 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:43.703+0000 7f7be53f2640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f7be01137e0 con 0x7f7be01083a0 2024-09-21T03:42:43.711 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:43.703+0000 7f7be53f2640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f7be0113ae0 con 0x7f7be0108cf0 2024-09-21T03:42:43.711 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:43.703+0000 7f7bdeffd640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7be0102a60 0x7f7be019a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:43.711 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:43.703+0000 7f7bdeffd640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7be0102a60 0x7f7be019a350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:39798/0 (socket says 172.21.15.89:39798) 2024-09-21T03:42:43.711 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:43.703+0000 7f7bdeffd640 1 -- 172.21.15.89:0/3911490089 learned_addr learned my addr 172.21.15.89:0/3911490089 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-21T03:42:43.711 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:43.703+0000 7f7bd67fc640 1 --2- 172.21.15.89:0/3911490089 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7be01083a0 0x7f7be019a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:43.711 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:43.703+0000 7f7bdf7fe640 1 --2- 172.21.15.89:0/3911490089 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7be0108cf0 0x7f7be019fb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:43.711 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:43.704+0000 7f7bdeffd640 1 -- 172.21.15.89:0/3911490089 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7be0108cf0 msgr2=0x7f7be019fb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:43.711 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:43.704+0000 7f7bdeffd640 1 --2- 172.21.15.89:0/3911490089 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7be0108cf0 0x7f7be019fb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:43.711 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:43.704+0000 7f7bdeffd640 1 -- 172.21.15.89:0/3911490089 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7be01083a0 msgr2=0x7f7be019a8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:43.712 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:43.704+0000 7f7bdeffd640 1 --2- 172.21.15.89:0/3911490089 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7be01083a0 0x7f7be019a8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:43.712 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:43.704+0000 7f7bdeffd640 1 -- 172.21.15.89:0/3911490089 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7be01a0200 con 0x7f7be0102a60 2024-09-21T03:42:43.712 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:43.704+0000 7f7bd67fc640 1 --2- 172.21.15.89:0/3911490089 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7be01083a0 0x7f7be019a8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:42:43.712 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:43.704+0000 7f7bdeffd640 1 --2- 172.21.15.89:0/3911490089 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7be0102a60 0x7f7be019a350 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f7bd000d950 tx=0x7f7bd000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:43.712 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:43.704+0000 7f7bdcff9640 1 -- 172.21.15.89:0/3911490089 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bd0017070 con 0x7f7be0102a60 2024-09-21T03:42:43.712 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:43.704+0000 7f7be53f2640 1 -- 172.21.15.89:0/3911490089 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7be01a04f0 con 0x7f7be0102a60 2024-09-21T03:42:43.712 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:43.704+0000 7f7bdcff9640 1 -- 172.21.15.89:0/3911490089 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7bd0009c30 con 0x7f7be0102a60 2024-09-21T03:42:43.712 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:43.704+0000 7f7bdcff9640 1 -- 172.21.15.89:0/3911490089 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bd000f620 con 0x7f7be0102a60 2024-09-21T03:42:43.712 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:43.704+0000 7f7be53f2640 1 -- 172.21.15.89:0/3911490089 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7be01a6d10 con 0x7f7be0102a60 2024-09-21T03:42:43.712 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:43.705+0000 7f7be53f2640 1 -- 172.21.15.89:0/3911490089 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7be0102e70 con 0x7f7be0102a60 2024-09-21T03:42:43.712 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:43.707+0000 7f7bdcff9640 1 -- 172.21.15.89:0/3911490089 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 21) ==== 154512+0+0 (secure 0 0 0) 0x7f7bd0010460 con 0x7f7be0102a60 2024-09-21T03:42:43.712 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:43.708+0000 7f7bdcff9640 1 --2- 172.21.15.89:0/3911490089 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f7bc00b65c0 0x7f7bc00b8ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:43.712 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:43.708+0000 7f7bd67fc640 1 --2- 172.21.15.89:0/3911490089 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f7bc00b65c0 0x7f7bc00b8ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:43.712 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:43.708+0000 7f7bdcff9640 1 -- 172.21.15.89:0/3911490089 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 9121+0+0 (secure 0 0 0) 0x7f7bd00e72a0 con 0x7f7be0102a60 2024-09-21T03:42:43.713 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:43.708+0000 7f7bd67fc640 1 --2- 172.21.15.89:0/3911490089 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f7bc00b65c0 0x7f7bc00b8ad0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f7bc8005e10 tx=0x7f7bc8005da0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:43.714 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:43.712+0000 7f7bdcff9640 1 -- 172.21.15.89:0/3911490089 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f7bd00ad7b0 con 0x7f7be0102a60 2024-09-21T03:42:43.929 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:43.927+0000 7f7be53f2640 1 -- 172.21.15.89:0/3911490089 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f7be019b480 con 0x7f7be0102a60 2024-09-21T03:42:43.929 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:43.928+0000 7f7bdcff9640 1 -- 172.21.15.89:0/3911490089 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v84) ==== 81+0+590 (secure 0 0 0) 0x7f7bd00b1b90 con 0x7f7be0102a60 2024-09-21T03:42:43.930 INFO:teuthology.orchestra.run.smithi089.stdout:172.21.15.89:6801/655110910 2024-09-22T03:29:51.519106+0000 2024-09-21T03:42:43.930 INFO:teuthology.orchestra.run.smithi089.stdout:172.21.15.89:0/1618544983 2024-09-22T03:29:51.519106+0000 2024-09-21T03:42:43.930 INFO:teuthology.orchestra.run.smithi089.stdout:172.21.15.89:0/1265502758 2024-09-22T03:29:51.519106+0000 2024-09-21T03:42:43.930 INFO:teuthology.orchestra.run.smithi089.stdout:172.21.15.89:0/4017684419 2024-09-22T03:29:51.519106+0000 2024-09-21T03:42:43.930 INFO:teuthology.orchestra.run.smithi089.stdout:172.21.15.89:6801/3312712420 2024-09-22T03:29:26.004686+0000 2024-09-21T03:42:43.930 INFO:teuthology.orchestra.run.smithi089.stdout:172.21.15.89:6800/3312712420 2024-09-22T03:29:26.004686+0000 2024-09-21T03:42:43.930 INFO:teuthology.orchestra.run.smithi089.stdout:172.21.15.89:0/2056576483 2024-09-22T03:29:26.004686+0000 2024-09-21T03:42:43.930 INFO:teuthology.orchestra.run.smithi089.stdout:172.21.15.89:6800/655110910 2024-09-22T03:29:51.519106+0000 2024-09-21T03:42:43.930 INFO:teuthology.orchestra.run.smithi089.stdout:172.21.15.89:0/2382866302 2024-09-22T03:29:26.004686+0000 2024-09-21T03:42:43.930 INFO:teuthology.orchestra.run.smithi089.stdout:172.21.15.89:0/1584351801 2024-09-22T03:29:26.004686+0000 2024-09-21T03:42:43.930 INFO:teuthology.orchestra.run.smithi089.stderr:listed 10 entries 2024-09-21T03:42:43.932 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:43.931+0000 7f7be53f2640 1 -- 172.21.15.89:0/3911490089 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f7bc00b65c0 msgr2=0x7f7bc00b8ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:43.932 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:43.931+0000 7f7be53f2640 1 --2- 172.21.15.89:0/3911490089 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f7bc00b65c0 0x7f7bc00b8ad0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f7bc8005e10 tx=0x7f7bc8005da0 comp rx=0 tx=0).stop 2024-09-21T03:42:43.932 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:43.931+0000 7f7be53f2640 1 -- 172.21.15.89:0/3911490089 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7be0102a60 msgr2=0x7f7be019a350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:43.933 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:43.931+0000 7f7be53f2640 1 --2- 172.21.15.89:0/3911490089 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7be0102a60 0x7f7be019a350 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f7bd000d950 tx=0x7f7bd000de10 comp rx=0 tx=0).stop 2024-09-21T03:42:43.933 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:43.931+0000 7f7be53f2640 1 -- 172.21.15.89:0/3911490089 shutdown_connections 2024-09-21T03:42:43.933 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:43.931+0000 7f7be53f2640 1 --2- 172.21.15.89:0/3911490089 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f7bc00b65c0 0x7f7bc00b8ad0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:43.933 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:43.931+0000 7f7be53f2640 1 --2- 172.21.15.89:0/3911490089 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7be01083a0 0x7f7be019a8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:43.933 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:43.931+0000 7f7be53f2640 1 --2- 172.21.15.89:0/3911490089 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7be0108cf0 0x7f7be019fb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:43.933 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:43.931+0000 7f7be53f2640 1 --2- 172.21.15.89:0/3911490089 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7be0102a60 0x7f7be019a350 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:43.933 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:43.931+0000 7f7be53f2640 1 -- 172.21.15.89:0/3911490089 >> 172.21.15.89:0/3911490089 conn(0x7f7be0079730 msgr2=0x7f7be0104410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:42:43.933 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:43.931+0000 7f7be53f2640 1 -- 172.21.15.89:0/3911490089 shutdown_connections 2024-09-21T03:42:43.933 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:43.931+0000 7f7be53f2640 1 -- 172.21.15.89:0/3911490089 wait complete. 2024-09-21T03:42:44.250 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:42:43 smithi110 ceph-mon[34000]: from='client.? 172.21.15.89:0/3911490089' entity='client.admin' cmd={"prefix": "osd blocklist ls"} : dispatch 2024-09-21T03:42:44.311 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:42:43 smithi089 ceph-mon[30928]: from='client.? 172.21.15.89:0/3911490089' entity='client.admin' cmd={"prefix": "osd blocklist ls"} : dispatch 2024-09-21T03:42:44.461 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:42:43 smithi114 ceph-mon[34140]: from='client.? 172.21.15.89:0/3911490089' entity='client.admin' cmd={"prefix": "osd blocklist ls"} : dispatch 2024-09-21T03:42:44.736 DEBUG:teuthology.orchestra.run.smithi089:> set -ex 2024-09-21T03:42:44.736 DEBUG:teuthology.orchestra.run.smithi089:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T03:42:44.754 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 7c374098-77c9-11ef-baf5-efdc52797490 -- ceph osd blocklist ls 2024-09-21T03:42:45.060 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/7c374098-77c9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:42:45.217 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:42:44 smithi089 ceph-mon[30928]: pgmap v460: 129 pgs: 129 active+clean; 1.1 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-21T03:42:45.250 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:42:44 smithi110 ceph-mon[34000]: pgmap v460: 129 pgs: 129 active+clean; 1.1 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-21T03:42:45.461 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:42:44 smithi114 ceph-mon[34140]: pgmap v460: 129 pgs: 129 active+clean; 1.1 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-21T03:42:45.579 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:45.560+0000 7fb9d20e1640 1 -- 172.21.15.89:0/2485730875 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb9cc109df0 msgr2=0x7fb9cc110620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:45.580 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:45.560+0000 7fb9d20e1640 1 --2- 172.21.15.89:0/2485730875 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb9cc109df0 0x7fb9cc110620 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7fb9c4009ca0 tx=0x7fb9c402a8c0 comp rx=0 tx=0).stop 2024-09-21T03:42:45.580 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:45.560+0000 7fb9d20e1640 1 -- 172.21.15.89:0/2485730875 shutdown_connections 2024-09-21T03:42:45.580 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:45.560+0000 7fb9d20e1640 1 --2- 172.21.15.89:0/2485730875 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb9cc109df0 0x7fb9cc110620 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:45.580 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:45.560+0000 7fb9d20e1640 1 --2- 172.21.15.89:0/2485730875 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb9cc1094a0 0x7fb9cc1098b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:45.580 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:45.560+0000 7fb9d20e1640 1 --2- 172.21.15.89:0/2485730875 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb9cc103b60 0x7fb9cc103f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:45.580 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:45.560+0000 7fb9d20e1640 1 -- 172.21.15.89:0/2485730875 >> 172.21.15.89:0/2485730875 conn(0x7fb9cc07b850 msgr2=0x7fb9cc07bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:42:45.580 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:45.560+0000 7fb9d20e1640 1 -- 172.21.15.89:0/2485730875 shutdown_connections 2024-09-21T03:42:45.580 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:45.560+0000 7fb9d20e1640 1 -- 172.21.15.89:0/2485730875 wait complete. 2024-09-21T03:42:45.580 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:45.561+0000 7fb9d20e1640 1 Processor -- start 2024-09-21T03:42:45.580 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:45.561+0000 7fb9d20e1640 1 -- start start 2024-09-21T03:42:45.580 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:45.561+0000 7fb9d20e1640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb9cc103b60 0x7fb9cc19a3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:45.580 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:45.561+0000 7fb9d20e1640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb9cc1094a0 0x7fb9cc19a940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:45.580 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:45.561+0000 7fb9d20e1640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb9cc109df0 0x7fb9cc19fbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:45.581 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:45.561+0000 7fb9d20e1640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fb9cc1139b0 con 0x7fb9cc103b60 2024-09-21T03:42:45.581 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:45.561+0000 7fb9d20e1640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fb9cc113830 con 0x7fb9cc1094a0 2024-09-21T03:42:45.581 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:45.561+0000 7fb9d20e1640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fb9cc113b30 con 0x7fb9cc109df0 2024-09-21T03:42:45.581 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:45.561+0000 7fb9cb7fe640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb9cc103b60 0x7fb9cc19a3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:45.581 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:45.561+0000 7fb9cb7fe640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb9cc103b60 0x7fb9cc19a3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:39826/0 (socket says 172.21.15.89:39826) 2024-09-21T03:42:45.581 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:45.561+0000 7fb9cb7fe640 1 -- 172.21.15.89:0/345497769 learned_addr learned my addr 172.21.15.89:0/345497769 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-21T03:42:45.581 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:45.561+0000 7fb9cbfff640 1 --2- 172.21.15.89:0/345497769 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb9cc109df0 0x7fb9cc19fbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:45.581 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:45.561+0000 7fb9caffd640 1 --2- 172.21.15.89:0/345497769 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb9cc1094a0 0x7fb9cc19a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:45.581 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:45.561+0000 7fb9cb7fe640 1 -- 172.21.15.89:0/345497769 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb9cc109df0 msgr2=0x7fb9cc19fbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:45.581 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:45.561+0000 7fb9cb7fe640 1 --2- 172.21.15.89:0/345497769 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb9cc109df0 0x7fb9cc19fbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:45.581 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:45.561+0000 7fb9cb7fe640 1 -- 172.21.15.89:0/345497769 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb9cc1094a0 msgr2=0x7fb9cc19a940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:45.581 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:45.561+0000 7fb9cb7fe640 1 --2- 172.21.15.89:0/345497769 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb9cc1094a0 0x7fb9cc19a940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:45.581 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:45.561+0000 7fb9cb7fe640 1 -- 172.21.15.89:0/345497769 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9cc1a0310 con 0x7fb9cc103b60 2024-09-21T03:42:45.581 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:45.561+0000 7fb9cbfff640 1 --2- 172.21.15.89:0/345497769 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb9cc109df0 0x7fb9cc19fbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:42:45.582 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:45.562+0000 7fb9cb7fe640 1 --2- 172.21.15.89:0/345497769 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb9cc103b60 0x7fb9cc19a3a0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7fb9bc00c890 tx=0x7fb9bc00cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:45.582 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:45.562+0000 7fb9c8ff9640 1 -- 172.21.15.89:0/345497769 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9bc004220 con 0x7fb9cc103b60 2024-09-21T03:42:45.582 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:45.562+0000 7fb9c8ff9640 1 -- 172.21.15.89:0/345497769 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb9bc007e40 con 0x7fb9cc103b60 2024-09-21T03:42:45.582 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:45.562+0000 7fb9c8ff9640 1 -- 172.21.15.89:0/345497769 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9bc004a00 con 0x7fb9cc103b60 2024-09-21T03:42:45.582 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:45.562+0000 7fb9d20e1640 1 -- 172.21.15.89:0/345497769 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb9cc1a0600 con 0x7fb9cc103b60 2024-09-21T03:42:45.582 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:45.562+0000 7fb9d20e1640 1 -- 172.21.15.89:0/345497769 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9cc1a6d60 con 0x7fb9cc103b60 2024-09-21T03:42:45.582 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:45.563+0000 7fb9d20e1640 1 -- 172.21.15.89:0/345497769 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb9cc103f70 con 0x7fb9cc103b60 2024-09-21T03:42:45.582 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:45.564+0000 7fb9c8ff9640 1 -- 172.21.15.89:0/345497769 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 21) ==== 154512+0+0 (secure 0 0 0) 0x7fb9bc0079b0 con 0x7fb9cc103b60 2024-09-21T03:42:45.582 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:45.565+0000 7fb9c8ff9640 1 --2- 172.21.15.89:0/345497769 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7fb9b40b6510 0x7fb9b40b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:45.582 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:45.565+0000 7fb9caffd640 1 --2- 172.21.15.89:0/345497769 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7fb9b40b6510 0x7fb9b40b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:45.582 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:45.565+0000 7fb9c8ff9640 1 -- 172.21.15.89:0/345497769 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 9121+0+0 (secure 0 0 0) 0x7fb9bc0e7850 con 0x7fb9cc103b60 2024-09-21T03:42:45.582 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:45.566+0000 7fb9caffd640 1 --2- 172.21.15.89:0/345497769 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7fb9b40b6510 0x7fb9b40b8a20 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fb9c0005fd0 tx=0x7fb9c0005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:45.582 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:45.568+0000 7fb9c8ff9640 1 -- 172.21.15.89:0/345497769 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fb9bc014030 con 0x7fb9cc103b60 2024-09-21T03:42:45.761 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:45.758+0000 7fb9d20e1640 1 -- 172.21.15.89:0/345497769 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fb9cc19b4d0 con 0x7fb9cc103b60 2024-09-21T03:42:45.761 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:45.759+0000 7fb9c8ff9640 1 -- 172.21.15.89:0/345497769 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v84) ==== 81+0+590 (secure 0 0 0) 0x7fb9bc0add60 con 0x7fb9cc103b60 2024-09-21T03:42:45.761 INFO:teuthology.orchestra.run.smithi089.stdout:172.21.15.89:6801/655110910 2024-09-22T03:29:51.519106+0000 2024-09-21T03:42:45.761 INFO:teuthology.orchestra.run.smithi089.stdout:172.21.15.89:0/1618544983 2024-09-22T03:29:51.519106+0000 2024-09-21T03:42:45.761 INFO:teuthology.orchestra.run.smithi089.stdout:172.21.15.89:0/1265502758 2024-09-22T03:29:51.519106+0000 2024-09-21T03:42:45.761 INFO:teuthology.orchestra.run.smithi089.stdout:172.21.15.89:0/4017684419 2024-09-22T03:29:51.519106+0000 2024-09-21T03:42:45.761 INFO:teuthology.orchestra.run.smithi089.stdout:172.21.15.89:6801/3312712420 2024-09-22T03:29:26.004686+0000 2024-09-21T03:42:45.761 INFO:teuthology.orchestra.run.smithi089.stdout:172.21.15.89:6800/3312712420 2024-09-22T03:29:26.004686+0000 2024-09-21T03:42:45.762 INFO:teuthology.orchestra.run.smithi089.stdout:172.21.15.89:0/2056576483 2024-09-22T03:29:26.004686+0000 2024-09-21T03:42:45.762 INFO:teuthology.orchestra.run.smithi089.stdout:172.21.15.89:6800/655110910 2024-09-22T03:29:51.519106+0000 2024-09-21T03:42:45.762 INFO:teuthology.orchestra.run.smithi089.stdout:172.21.15.89:0/2382866302 2024-09-22T03:29:26.004686+0000 2024-09-21T03:42:45.762 INFO:teuthology.orchestra.run.smithi089.stdout:172.21.15.89:0/1584351801 2024-09-22T03:29:26.004686+0000 2024-09-21T03:42:45.762 INFO:teuthology.orchestra.run.smithi089.stderr:listed 10 entries 2024-09-21T03:42:45.763 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:45.761+0000 7fb9d20e1640 1 -- 172.21.15.89:0/345497769 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7fb9b40b6510 msgr2=0x7fb9b40b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:45.763 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:45.761+0000 7fb9d20e1640 1 --2- 172.21.15.89:0/345497769 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7fb9b40b6510 0x7fb9b40b8a20 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fb9c0005fd0 tx=0x7fb9c0005850 comp rx=0 tx=0).stop 2024-09-21T03:42:45.763 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:45.761+0000 7fb9d20e1640 1 -- 172.21.15.89:0/345497769 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb9cc103b60 msgr2=0x7fb9cc19a3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:45.763 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:45.761+0000 7fb9d20e1640 1 --2- 172.21.15.89:0/345497769 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb9cc103b60 0x7fb9cc19a3a0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7fb9bc00c890 tx=0x7fb9bc00cd50 comp rx=0 tx=0).stop 2024-09-21T03:42:45.763 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:45.761+0000 7fb9d20e1640 1 -- 172.21.15.89:0/345497769 shutdown_connections 2024-09-21T03:42:45.763 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:45.761+0000 7fb9d20e1640 1 --2- 172.21.15.89:0/345497769 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7fb9b40b6510 0x7fb9b40b8a20 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:45.763 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:45.761+0000 7fb9d20e1640 1 --2- 172.21.15.89:0/345497769 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb9cc1094a0 0x7fb9cc19a940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:45.764 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:45.761+0000 7fb9d20e1640 1 --2- 172.21.15.89:0/345497769 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb9cc109df0 0x7fb9cc19fbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:45.764 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:45.761+0000 7fb9d20e1640 1 --2- 172.21.15.89:0/345497769 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb9cc103b60 0x7fb9cc19a3a0 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:45.764 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:45.761+0000 7fb9d20e1640 1 -- 172.21.15.89:0/345497769 >> 172.21.15.89:0/345497769 conn(0x7fb9cc07b850 msgr2=0x7fb9cc107530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:42:45.764 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:45.761+0000 7fb9d20e1640 1 -- 172.21.15.89:0/345497769 shutdown_connections 2024-09-21T03:42:45.764 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:45.761+0000 7fb9d20e1640 1 -- 172.21.15.89:0/345497769 wait complete. 2024-09-21T03:42:46.061 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:42:45 smithi089 ceph-mon[30928]: from='client.? 172.21.15.89:0/345497769' entity='client.admin' cmd={"prefix": "osd blocklist ls"} : dispatch 2024-09-21T03:42:46.250 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:42:45 smithi110 ceph-mon[34000]: from='client.? 172.21.15.89:0/345497769' entity='client.admin' cmd={"prefix": "osd blocklist ls"} : dispatch 2024-09-21T03:42:46.437 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi089.front.sepia.ceph.com... 2024-09-21T03:42:46.438 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:42:46.438 DEBUG:teuthology.orchestra.run.smithi089:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-09-21T03:42:46.461 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:42:45 smithi114 ceph-mon[34140]: from='client.? 172.21.15.89:0/345497769' entity='client.admin' cmd={"prefix": "osd blocklist ls"} : dispatch 2024-09-21T03:42:46.464 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T03:42:46.465 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi089.stderr:ceph-fuse[84548]: fuse finished with error 0 and tester_r 0 2024-09-21T03:42:47.250 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:42:46 smithi110 ceph-mon[34000]: pgmap v461: 129 pgs: 129 active+clean; 1.1 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-09-21T03:42:47.311 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:42:46 smithi089 ceph-mon[30928]: pgmap v461: 129 pgs: 129 active+clean; 1.1 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-09-21T03:42:47.461 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:42:46 smithi114 ceph-mon[34140]: pgmap v461: 129 pgs: 129 active+clean; 1.1 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-09-21T03:42:49.250 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:42:48 smithi110 ceph-mon[34000]: pgmap v462: 129 pgs: 129 active+clean; 1.1 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 5 op/s 2024-09-21T03:42:49.311 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:42:48 smithi089 ceph-mon[30928]: pgmap v462: 129 pgs: 129 active+clean; 1.1 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 5 op/s 2024-09-21T03:42:49.461 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:42:48 smithi114 ceph-mon[34140]: pgmap v462: 129 pgs: 129 active+clean; 1.1 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 5 op/s 2024-09-21T03:42:51.250 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:42:50 smithi110 ceph-mon[34000]: pgmap v463: 129 pgs: 129 active+clean; 1.1 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 597 B/s wr, 8 op/s 2024-09-21T03:42:51.311 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:42:50 smithi089 ceph-mon[30928]: pgmap v463: 129 pgs: 129 active+clean; 1.1 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 597 B/s wr, 8 op/s 2024-09-21T03:42:51.461 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:42:50 smithi114 ceph-mon[34140]: pgmap v463: 129 pgs: 129 active+clean; 1.1 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 597 B/s wr, 8 op/s 2024-09-21T03:42:52.566 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi089.front.sepia.ceph.com 2024-09-21T03:42:52.566 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:42:52.567 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-21T03:42:52.588 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:42:52.588 DEBUG:teuthology.orchestra.run.smithi089:> ip netns list 2024-09-21T03:42:52.646 INFO:teuthology.orchestra.run.smithi089.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-21T03:42:52.647 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:42:52.647 DEBUG:teuthology.orchestra.run.smithi089:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-21T03:42:52.720 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:42:52.720 DEBUG:teuthology.orchestra.run.smithi089:> sudo ip link delete ceph-brx 2024-09-21T03:42:52.816 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-09-21T03:42:52.827 INFO:tasks.cephadm:Teardown begin 2024-09-21T03:42:52.827 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-21T03:42:52.827 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 7c374098-77c9-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T03:42:53.081 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/7c374098-77c9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:42:53.250 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:42:52 smithi110 ceph-mon[34000]: pgmap v464: 129 pgs: 129 active+clean; 1.1 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 597 B/s wr, 6 op/s 2024-09-21T03:42:53.311 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:42:52 smithi089 ceph-mon[30928]: pgmap v464: 129 pgs: 129 active+clean; 1.1 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 597 B/s wr, 6 op/s 2024-09-21T03:42:53.461 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:42:52 smithi114 ceph-mon[34140]: pgmap v464: 129 pgs: 129 active+clean; 1.1 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 597 B/s wr, 6 op/s 2024-09-21T03:42:53.584 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:53.579+0000 7fa2e12b9640 1 -- 172.21.15.89:0/945813463 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa2dc105b70 msgr2=0x7fa2dc10f600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:53.584 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:53.579+0000 7fa2e12b9640 1 --2- 172.21.15.89:0/945813463 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa2dc105b70 0x7fa2dc10f600 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7fa2d4009ca0 tx=0x7fa2d402a8c0 comp rx=0 tx=0).stop 2024-09-21T03:42:53.584 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:53.579+0000 7fa2e12b9640 1 -- 172.21.15.89:0/945813463 shutdown_connections 2024-09-21T03:42:53.584 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:53.579+0000 7fa2e12b9640 1 --2- 172.21.15.89:0/945813463 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa2dc105220 0x7fa2dc105630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:53.584 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:53.579+0000 7fa2e12b9640 1 --2- 172.21.15.89:0/945813463 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa2dc105b70 0x7fa2dc10f600 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:53.584 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:53.579+0000 7fa2e12b9640 1 --2- 172.21.15.89:0/945813463 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa2dc103210 0x7fa2dc103620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:53.584 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:53.579+0000 7fa2e12b9640 1 -- 172.21.15.89:0/945813463 >> 172.21.15.89:0/945813463 conn(0x7fa2dc079ea0 msgr2=0x7fa2dc07a2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:42:53.584 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:53.579+0000 7fa2e12b9640 1 -- 172.21.15.89:0/945813463 shutdown_connections 2024-09-21T03:42:53.585 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:53.579+0000 7fa2e12b9640 1 -- 172.21.15.89:0/945813463 wait complete. 2024-09-21T03:42:53.585 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:53.579+0000 7fa2e12b9640 1 Processor -- start 2024-09-21T03:42:53.585 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:53.580+0000 7fa2e12b9640 1 -- start start 2024-09-21T03:42:53.585 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:53.580+0000 7fa2e12b9640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa2dc103210 0x7fa2dc0feed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:53.585 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:53.580+0000 7fa2e12b9640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa2dc105220 0x7fa2dc0ff410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:53.585 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:53.580+0000 7fa2e12b9640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa2dc105b70 0x7fa2dc0ff950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:53.585 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:53.580+0000 7fa2e12b9640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fa2dc112930 con 0x7fa2dc105220 2024-09-21T03:42:53.585 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:53.580+0000 7fa2e12b9640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fa2dc1127b0 con 0x7fa2dc103210 2024-09-21T03:42:53.585 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:53.580+0000 7fa2e12b9640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fa2dc112ab0 con 0x7fa2dc105b70 2024-09-21T03:42:53.585 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:53.580+0000 7fa2da7fc640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa2dc105220 0x7fa2dc0ff410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:53.585 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:53.580+0000 7fa2da7fc640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa2dc105220 0x7fa2dc0ff410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:59324/0 (socket says 172.21.15.89:59324) 2024-09-21T03:42:53.585 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:53.580+0000 7fa2da7fc640 1 -- 172.21.15.89:0/1864103054 learned_addr learned my addr 172.21.15.89:0/1864103054 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-21T03:42:53.585 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:53.580+0000 7fa2daffd640 1 --2- 172.21.15.89:0/1864103054 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa2dc103210 0x7fa2dc0feed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:53.585 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:53.580+0000 7fa2db7fe640 1 --2- 172.21.15.89:0/1864103054 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa2dc105b70 0x7fa2dc0ff950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:53.586 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:53.580+0000 7fa2da7fc640 1 -- 172.21.15.89:0/1864103054 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa2dc105b70 msgr2=0x7fa2dc0ff950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:53.586 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:53.580+0000 7fa2da7fc640 1 --2- 172.21.15.89:0/1864103054 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa2dc105b70 0x7fa2dc0ff950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:53.586 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:53.580+0000 7fa2da7fc640 1 -- 172.21.15.89:0/1864103054 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa2dc103210 msgr2=0x7fa2dc0feed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:53.586 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:53.580+0000 7fa2da7fc640 1 --2- 172.21.15.89:0/1864103054 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa2dc103210 0x7fa2dc0feed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:53.586 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:53.580+0000 7fa2da7fc640 1 -- 172.21.15.89:0/1864103054 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2dc1a8710 con 0x7fa2dc105220 2024-09-21T03:42:53.586 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:53.580+0000 7fa2db7fe640 1 --2- 172.21.15.89:0/1864103054 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa2dc105b70 0x7fa2dc0ff950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:42:53.586 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:53.581+0000 7fa2daffd640 1 --2- 172.21.15.89:0/1864103054 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa2dc103210 0x7fa2dc0feed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:42:53.586 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:53.581+0000 7fa2da7fc640 1 --2- 172.21.15.89:0/1864103054 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa2dc105220 0x7fa2dc0ff410 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7fa2d000d750 tx=0x7fa2d000dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:53.586 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:53.581+0000 7fa2c3fff640 1 -- 172.21.15.89:0/1864103054 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2d0017070 con 0x7fa2dc105220 2024-09-21T03:42:53.586 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:53.581+0000 7fa2e12b9640 1 -- 172.21.15.89:0/1864103054 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa2dc1a8a00 con 0x7fa2dc105220 2024-09-21T03:42:53.586 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:53.581+0000 7fa2c3fff640 1 -- 172.21.15.89:0/1864103054 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa2d00054b0 con 0x7fa2dc105220 2024-09-21T03:42:53.586 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:53.581+0000 7fa2c3fff640 1 -- 172.21.15.89:0/1864103054 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2d0004bc0 con 0x7fa2dc105220 2024-09-21T03:42:53.586 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:53.581+0000 7fa2e12b9640 1 -- 172.21.15.89:0/1864103054 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2dc1a8ef0 con 0x7fa2dc105220 2024-09-21T03:42:53.587 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:53.582+0000 7fa2e12b9640 1 -- 172.21.15.89:0/1864103054 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa2dc103620 con 0x7fa2dc105220 2024-09-21T03:42:53.587 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:53.583+0000 7fa2c3fff640 1 -- 172.21.15.89:0/1864103054 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 21) ==== 154512+0+0 (secure 0 0 0) 0x7fa2d0004d60 con 0x7fa2dc105220 2024-09-21T03:42:53.587 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:53.584+0000 7fa2c3fff640 1 --2- 172.21.15.89:0/1864103054 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7fa2c40b6510 0x7fa2c40b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:53.587 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:53.584+0000 7fa2daffd640 1 --2- 172.21.15.89:0/1864103054 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7fa2c40b6510 0x7fa2c40b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:53.588 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:53.584+0000 7fa2c3fff640 1 -- 172.21.15.89:0/1864103054 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 9121+0+0 (secure 0 0 0) 0x7fa2d00e70b0 con 0x7fa2dc105220 2024-09-21T03:42:53.588 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:53.585+0000 7fa2daffd640 1 --2- 172.21.15.89:0/1864103054 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7fa2c40b6510 0x7fa2c40b8a20 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fa2dc100920 tx=0x7fa2cc00b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:53.591 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:53.589+0000 7fa2c3fff640 1 -- 172.21.15.89:0/1864103054 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fa2d00ad5c0 con 0x7fa2dc105220 2024-09-21T03:42:53.804 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:53.802+0000 7fa2e12b9640 1 -- 172.21.15.89:0/1864103054 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa2dc1a9190 con 0x7fa2dc105220 2024-09-21T03:42:53.805 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:53.803+0000 7fa2c3fff640 1 -- 172.21.15.89:0/1864103054 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11912 (secure 0 0 0) 0x7fa2d00b19a0 con 0x7fa2dc105220 2024-09-21T03:42:53.805 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-09-21T03:42:53.806 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":26,"btime":"2024-09-21T03:39:26:058611+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14721,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6835/2341538514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":2341538514},{"type":"v1","addr":"172.21.15.89:6835","nonce":2341538514}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14733,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6837/146708592","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6836","nonce":146708592},{"type":"v1","addr":"172.21.15.89:6837","nonce":146708592}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14781,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6839/2269965499","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6838","nonce":2269965499},{"type":"v1","addr":"172.21.15.89:6839","nonce":2269965499}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14793,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6841/650850383","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6840","nonce":650850383},{"type":"v1","addr":"172.21.15.89:6841","nonce":650850383}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24565,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6833/752336912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6832","nonce":752336912},{"type":"v1","addr":"172.21.15.114:6833","nonce":752336912}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24575,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6833/127607331","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6832","nonce":127607331},{"type":"v1","addr":"172.21.15.110:6833","nonce":127607331}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T03:37:13.570694+0000","modified":"2024-09-21T03:39:25.333954+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24638,"mds_1":24631,"mds_2":24614},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24593":{"gid":24593,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.110:6835/671820654","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6834","nonce":671820654},{"type":"v1","addr":"172.21.15.110:6835","nonce":671820654}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24604":{"gid":24604,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.114:6835/1049714548","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6834","nonce":1049714548},{"type":"v1","addr":"172.21.15.114:6835","nonce":1049714548}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24610":{"gid":24610,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.114:6837/1941213818","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6836","nonce":1941213818},{"type":"v1","addr":"172.21.15.114:6837","nonce":1941213818}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24614":{"gid":24614,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":15,"addr":"172.21.15.110:6837/4047124589","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6836","nonce":4047124589},{"type":"v1","addr":"172.21.15.110:6837","nonce":4047124589}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24631":{"gid":24631,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":15,"addr":"172.21.15.114:6839/3427369944","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6838","nonce":3427369944},{"type":"v1","addr":"172.21.15.114:6839","nonce":3427369944}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24638":{"gid":24638,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.110:6839/132366709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6838","nonce":132366709},{"type":"v1","addr":"172.21.15.110:6839","nonce":132366709}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24638,"qdb_cluster":[24638,24631,24614]},"id":1}]} 2024-09-21T03:42:53.806 INFO:teuthology.orchestra.run.smithi089.stderr:dumped fsmap epoch 26 2024-09-21T03:42:53.808 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:53.805+0000 7fa2e12b9640 1 -- 172.21.15.89:0/1864103054 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7fa2c40b6510 msgr2=0x7fa2c40b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:53.808 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:53.805+0000 7fa2e12b9640 1 --2- 172.21.15.89:0/1864103054 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7fa2c40b6510 0x7fa2c40b8a20 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fa2dc100920 tx=0x7fa2cc00b390 comp rx=0 tx=0).stop 2024-09-21T03:42:53.808 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:53.805+0000 7fa2e12b9640 1 -- 172.21.15.89:0/1864103054 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa2dc105220 msgr2=0x7fa2dc0ff410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:53.808 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:53.805+0000 7fa2e12b9640 1 --2- 172.21.15.89:0/1864103054 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa2dc105220 0x7fa2dc0ff410 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7fa2d000d750 tx=0x7fa2d000dc10 comp rx=0 tx=0).stop 2024-09-21T03:42:53.808 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:53.805+0000 7fa2e12b9640 1 -- 172.21.15.89:0/1864103054 shutdown_connections 2024-09-21T03:42:53.808 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:53.805+0000 7fa2e12b9640 1 --2- 172.21.15.89:0/1864103054 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7fa2c40b6510 0x7fa2c40b8a20 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:53.808 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:53.805+0000 7fa2e12b9640 1 --2- 172.21.15.89:0/1864103054 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa2dc105b70 0x7fa2dc0ff950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:53.808 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:53.805+0000 7fa2e12b9640 1 --2- 172.21.15.89:0/1864103054 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa2dc105220 0x7fa2dc0ff410 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:53.808 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:53.805+0000 7fa2e12b9640 1 --2- 172.21.15.89:0/1864103054 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa2dc103210 0x7fa2dc0feed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:53.808 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:53.805+0000 7fa2e12b9640 1 -- 172.21.15.89:0/1864103054 >> 172.21.15.89:0/1864103054 conn(0x7fa2dc079ea0 msgr2=0x7fa2dc106550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:42:53.808 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:53.805+0000 7fa2e12b9640 1 -- 172.21.15.89:0/1864103054 shutdown_connections 2024-09-21T03:42:53.808 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:53.805+0000 7fa2e12b9640 1 -- 172.21.15.89:0/1864103054 wait complete. 2024-09-21T03:42:54.061 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:42:53 smithi089 ceph-mon[30928]: from='client.? 172.21.15.89:0/1864103054' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:42:54.250 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:42:53 smithi110 ceph-mon[34000]: from='client.? 172.21.15.89:0/1864103054' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:42:54.461 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:42:53 smithi114 ceph-mon[34140]: from='client.? 172.21.15.89:0/1864103054' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:42:54.601 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 7c374098-77c9-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T03:42:54.850 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/7c374098-77c9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:42:55.250 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:42:54 smithi110 ceph-mon[34000]: pgmap v465: 129 pgs: 129 active+clean; 1.1 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 597 B/s wr, 6 op/s 2024-09-21T03:42:55.285 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:42:54 smithi089 ceph-mon[30928]: pgmap v465: 129 pgs: 129 active+clean; 1.1 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 597 B/s wr, 6 op/s 2024-09-21T03:42:55.385 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:55.370+0000 7f509cadd640 1 -- 172.21.15.89:0/1415052320 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f50981094a0 msgr2=0x7f50981098b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:55.385 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:55.370+0000 7f509cadd640 1 --2- 172.21.15.89:0/1415052320 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f50981094a0 0x7f50981098b0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f5080009900 tx=0x7f508002a410 comp rx=0 tx=0).stop 2024-09-21T03:42:55.385 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:55.371+0000 7f509cadd640 1 -- 172.21.15.89:0/1415052320 shutdown_connections 2024-09-21T03:42:55.385 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:55.371+0000 7f509cadd640 1 --2- 172.21.15.89:0/1415052320 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5098109df0 0x7f5098110620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:55.385 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:55.371+0000 7f509cadd640 1 --2- 172.21.15.89:0/1415052320 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f50981094a0 0x7f50981098b0 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:55.385 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:55.371+0000 7f509cadd640 1 --2- 172.21.15.89:0/1415052320 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5098103b60 0x7f5098103f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:55.385 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:55.371+0000 7f509cadd640 1 -- 172.21.15.89:0/1415052320 >> 172.21.15.89:0/1415052320 conn(0x7f509807b850 msgr2=0x7f509807bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:42:55.385 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:55.371+0000 7f509cadd640 1 -- 172.21.15.89:0/1415052320 shutdown_connections 2024-09-21T03:42:55.385 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:55.371+0000 7f509cadd640 1 -- 172.21.15.89:0/1415052320 wait complete. 2024-09-21T03:42:55.386 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:55.371+0000 7f509cadd640 1 Processor -- start 2024-09-21T03:42:55.386 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:55.371+0000 7f509cadd640 1 -- start start 2024-09-21T03:42:55.386 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:55.372+0000 7f509cadd640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5098103b60 0x7f5098075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:55.386 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:55.372+0000 7f509cadd640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f50981094a0 0x7f5098075c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:55.386 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:55.372+0000 7f509cadd640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5098109df0 0x7f5098076190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:55.386 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:55.372+0000 7f509cadd640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f5098113960 con 0x7f5098103b60 2024-09-21T03:42:55.386 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:55.372+0000 7f509cadd640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f50981137e0 con 0x7f50981094a0 2024-09-21T03:42:55.386 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:55.372+0000 7f509cadd640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f5098113ae0 con 0x7f5098109df0 2024-09-21T03:42:55.386 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:55.372+0000 7f509659c640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5098103b60 0x7f5098075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:55.386 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:55.372+0000 7f509659c640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5098103b60 0x7f5098075710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:59338/0 (socket says 172.21.15.89:59338) 2024-09-21T03:42:55.386 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:55.372+0000 7f509659c640 1 -- 172.21.15.89:0/855563190 learned_addr learned my addr 172.21.15.89:0/855563190 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-21T03:42:55.386 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:55.372+0000 7f5096d9d640 1 --2- 172.21.15.89:0/855563190 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5098109df0 0x7f5098076190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:55.386 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:55.372+0000 7f508ffff640 1 --2- 172.21.15.89:0/855563190 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f50981094a0 0x7f5098075c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:55.387 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:55.372+0000 7f509659c640 1 -- 172.21.15.89:0/855563190 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5098109df0 msgr2=0x7f5098076190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:55.387 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:55.372+0000 7f509659c640 1 --2- 172.21.15.89:0/855563190 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5098109df0 0x7f5098076190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:55.387 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:55.372+0000 7f509659c640 1 -- 172.21.15.89:0/855563190 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f50981094a0 msgr2=0x7f5098075c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:55.387 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:55.372+0000 7f509659c640 1 --2- 172.21.15.89:0/855563190 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f50981094a0 0x7f5098075c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:55.387 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:55.372+0000 7f509659c640 1 -- 172.21.15.89:0/855563190 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50981a86f0 con 0x7f5098103b60 2024-09-21T03:42:55.387 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:55.372+0000 7f509659c640 1 --2- 172.21.15.89:0/855563190 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5098103b60 0x7f5098075710 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f508800d950 tx=0x7f508800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:55.387 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:55.373+0000 7f508f7fe640 1 -- 172.21.15.89:0/855563190 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5088017070 con 0x7f5098103b60 2024-09-21T03:42:55.387 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:55.373+0000 7f508f7fe640 1 -- 172.21.15.89:0/855563190 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5088009c30 con 0x7f5098103b60 2024-09-21T03:42:55.387 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:55.373+0000 7f509cadd640 1 -- 172.21.15.89:0/855563190 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f50981a89e0 con 0x7f5098103b60 2024-09-21T03:42:55.387 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:55.373+0000 7f508f7fe640 1 -- 172.21.15.89:0/855563190 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f508800f620 con 0x7f5098103b60 2024-09-21T03:42:55.387 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:55.373+0000 7f509cadd640 1 -- 172.21.15.89:0/855563190 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50981a8ef0 con 0x7f5098103b60 2024-09-21T03:42:55.387 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:55.374+0000 7f509cadd640 1 -- 172.21.15.89:0/855563190 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5098103f70 con 0x7f5098103b60 2024-09-21T03:42:55.387 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:55.375+0000 7f508f7fe640 1 -- 172.21.15.89:0/855563190 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 21) ==== 154512+0+0 (secure 0 0 0) 0x7f5088010460 con 0x7f5098103b60 2024-09-21T03:42:55.387 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:55.376+0000 7f508f7fe640 1 --2- 172.21.15.89:0/855563190 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f50840b6510 0x7f50840b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:55.388 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:55.376+0000 7f508ffff640 1 --2- 172.21.15.89:0/855563190 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f50840b6510 0x7f50840b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:55.388 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:55.376+0000 7f508f7fe640 1 -- 172.21.15.89:0/855563190 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 9121+0+0 (secure 0 0 0) 0x7f50880e7350 con 0x7f5098103b60 2024-09-21T03:42:55.388 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:55.377+0000 7f508ffff640 1 --2- 172.21.15.89:0/855563190 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f50840b6510 0x7f50840b8a20 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f50800057c0 tx=0x7f5080000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:55.388 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:55.379+0000 7f508f7fe640 1 -- 172.21.15.89:0/855563190 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f50880ad860 con 0x7f5098103b60 2024-09-21T03:42:55.461 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:42:54 smithi114 ceph-mon[34140]: pgmap v465: 129 pgs: 129 active+clean; 1.1 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 597 B/s wr, 6 op/s 2024-09-21T03:42:55.594 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:55.592+0000 7f509cadd640 1 -- 172.21.15.89:0/855563190 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f509802e640 con 0x7f5098103b60 2024-09-21T03:42:55.595 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:55.593+0000 7f508f7fe640 1 -- 172.21.15.89:0/855563190 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11912 (secure 0 0 0) 0x7f5088009de0 con 0x7f5098103b60 2024-09-21T03:42:55.595 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-09-21T03:42:55.595 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":26,"btime":"2024-09-21T03:39:26:058611+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14721,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6835/2341538514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":2341538514},{"type":"v1","addr":"172.21.15.89:6835","nonce":2341538514}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14733,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6837/146708592","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6836","nonce":146708592},{"type":"v1","addr":"172.21.15.89:6837","nonce":146708592}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14781,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6839/2269965499","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6838","nonce":2269965499},{"type":"v1","addr":"172.21.15.89:6839","nonce":2269965499}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14793,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6841/650850383","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6840","nonce":650850383},{"type":"v1","addr":"172.21.15.89:6841","nonce":650850383}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24565,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6833/752336912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6832","nonce":752336912},{"type":"v1","addr":"172.21.15.114:6833","nonce":752336912}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24575,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6833/127607331","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6832","nonce":127607331},{"type":"v1","addr":"172.21.15.110:6833","nonce":127607331}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T03:37:13.570694+0000","modified":"2024-09-21T03:39:25.333954+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24638,"mds_1":24631,"mds_2":24614},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24593":{"gid":24593,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.110:6835/671820654","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6834","nonce":671820654},{"type":"v1","addr":"172.21.15.110:6835","nonce":671820654}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24604":{"gid":24604,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.114:6835/1049714548","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6834","nonce":1049714548},{"type":"v1","addr":"172.21.15.114:6835","nonce":1049714548}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24610":{"gid":24610,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.114:6837/1941213818","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6836","nonce":1941213818},{"type":"v1","addr":"172.21.15.114:6837","nonce":1941213818}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24614":{"gid":24614,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":15,"addr":"172.21.15.110:6837/4047124589","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6836","nonce":4047124589},{"type":"v1","addr":"172.21.15.110:6837","nonce":4047124589}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24631":{"gid":24631,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":15,"addr":"172.21.15.114:6839/3427369944","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6838","nonce":3427369944},{"type":"v1","addr":"172.21.15.114:6839","nonce":3427369944}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24638":{"gid":24638,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.110:6839/132366709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6838","nonce":132366709},{"type":"v1","addr":"172.21.15.110:6839","nonce":132366709}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24638,"qdb_cluster":[24638,24631,24614]},"id":1}]} 2024-09-21T03:42:55.596 INFO:teuthology.orchestra.run.smithi089.stderr:dumped fsmap epoch 26 2024-09-21T03:42:55.597 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:55.595+0000 7f509cadd640 1 -- 172.21.15.89:0/855563190 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f50840b6510 msgr2=0x7f50840b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:55.598 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:55.595+0000 7f509cadd640 1 --2- 172.21.15.89:0/855563190 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f50840b6510 0x7f50840b8a20 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f50800057c0 tx=0x7f5080000bc0 comp rx=0 tx=0).stop 2024-09-21T03:42:55.598 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:55.595+0000 7f509cadd640 1 -- 172.21.15.89:0/855563190 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5098103b60 msgr2=0x7f5098075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:55.598 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:55.595+0000 7f509cadd640 1 --2- 172.21.15.89:0/855563190 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5098103b60 0x7f5098075710 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f508800d950 tx=0x7f508800de10 comp rx=0 tx=0).stop 2024-09-21T03:42:55.598 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:55.595+0000 7f509cadd640 1 -- 172.21.15.89:0/855563190 shutdown_connections 2024-09-21T03:42:55.598 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:55.595+0000 7f509cadd640 1 --2- 172.21.15.89:0/855563190 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f50840b6510 0x7f50840b8a20 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:55.598 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:55.595+0000 7f509cadd640 1 --2- 172.21.15.89:0/855563190 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f50981094a0 0x7f5098075c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:55.598 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:55.595+0000 7f509cadd640 1 --2- 172.21.15.89:0/855563190 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5098109df0 0x7f5098076190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:55.598 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:55.595+0000 7f509cadd640 1 --2- 172.21.15.89:0/855563190 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5098103b60 0x7f5098075710 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:55.598 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:55.595+0000 7f509cadd640 1 -- 172.21.15.89:0/855563190 >> 172.21.15.89:0/855563190 conn(0x7f509807b850 msgr2=0x7f50981055c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:42:55.598 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:55.595+0000 7f509cadd640 1 -- 172.21.15.89:0/855563190 shutdown_connections 2024-09-21T03:42:55.598 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:55.595+0000 7f509cadd640 1 -- 172.21.15.89:0/855563190 wait complete. 2024-09-21T03:42:56.250 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:42:55 smithi110 ceph-mon[34000]: from='client.? 172.21.15.89:0/855563190' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:42:56.311 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:42:55 smithi089 ceph-mon[30928]: from='client.? 172.21.15.89:0/855563190' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:42:56.412 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 7c374098-77c9-11ef-baf5-efdc52797490 -- ceph osd dump --format=json 2024-09-21T03:42:56.461 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:42:55 smithi114 ceph-mon[34140]: from='client.? 172.21.15.89:0/855563190' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:42:56.660 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/7c374098-77c9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:42:57.061 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:42:57 smithi089 ceph-mon[30928]: pgmap v466: 129 pgs: 129 active+clean; 1.1 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 597 B/s wr, 7 op/s 2024-09-21T03:42:57.180 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:57.167+0000 7f5ac02ab640 1 -- 172.21.15.89:0/3220549682 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5ab81049c0 msgr2=0x7f5ab8104dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:57.180 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:57.167+0000 7f5ac02ab640 1 --2- 172.21.15.89:0/3220549682 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5ab81049c0 0x7f5ab8104dd0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f5ab4009ca0 tx=0x7f5ab402a8c0 comp rx=0 tx=0).stop 2024-09-21T03:42:57.180 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:57.167+0000 7f5ac02ab640 1 -- 172.21.15.89:0/3220549682 shutdown_connections 2024-09-21T03:42:57.180 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:57.167+0000 7f5ac02ab640 1 --2- 172.21.15.89:0/3220549682 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5ab8105310 0x7f5ab810d860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:57.180 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:57.167+0000 7f5ac02ab640 1 --2- 172.21.15.89:0/3220549682 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5ab810de20 0x7f5ab8110310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:57.180 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:57.167+0000 7f5ac02ab640 1 --2- 172.21.15.89:0/3220549682 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5ab81049c0 0x7f5ab8104dd0 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:57.181 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:57.167+0000 7f5ac02ab640 1 -- 172.21.15.89:0/3220549682 >> 172.21.15.89:0/3220549682 conn(0x7f5ab8079710 msgr2=0x7f5ab8079b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:42:57.181 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:57.167+0000 7f5ac02ab640 1 -- 172.21.15.89:0/3220549682 shutdown_connections 2024-09-21T03:42:57.181 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:57.167+0000 7f5ac02ab640 1 -- 172.21.15.89:0/3220549682 wait complete. 2024-09-21T03:42:57.181 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:57.168+0000 7f5ac02ab640 1 Processor -- start 2024-09-21T03:42:57.181 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:57.168+0000 7f5ac02ab640 1 -- start start 2024-09-21T03:42:57.181 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:57.168+0000 7f5ac02ab640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5ab81049c0 0x7f5ab819a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:57.181 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:57.168+0000 7f5ac02ab640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5ab8105310 0x7f5ab819a990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:57.181 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:57.168+0000 7f5ac02ab640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5ab810de20 0x7f5ab819fc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:57.181 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:57.168+0000 7f5ac02ab640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f5ab81139a0 con 0x7f5ab810de20 2024-09-21T03:42:57.181 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:57.168+0000 7f5ac02ab640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f5ab8113820 con 0x7f5ab81049c0 2024-09-21T03:42:57.181 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:57.168+0000 7f5ac02ab640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f5ab8113b20 con 0x7f5ab8105310 2024-09-21T03:42:57.181 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:57.168+0000 7f5abe848640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5ab810de20 0x7f5ab819fc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:57.181 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:57.168+0000 7f5abe848640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5ab810de20 0x7f5ab819fc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:59346/0 (socket says 172.21.15.89:59346) 2024-09-21T03:42:57.181 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:57.168+0000 7f5abe848640 1 -- 172.21.15.89:0/1672120906 learned_addr learned my addr 172.21.15.89:0/1672120906 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-21T03:42:57.181 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:57.169+0000 7f5abe047640 1 --2- 172.21.15.89:0/1672120906 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5ab81049c0 0x7f5ab819a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:57.182 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:57.169+0000 7f5abd846640 1 --2- 172.21.15.89:0/1672120906 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5ab8105310 0x7f5ab819a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:57.182 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:57.169+0000 7f5abe848640 1 -- 172.21.15.89:0/1672120906 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5ab8105310 msgr2=0x7f5ab819a990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:57.182 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:57.169+0000 7f5abe848640 1 --2- 172.21.15.89:0/1672120906 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5ab8105310 0x7f5ab819a990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:57.182 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:57.169+0000 7f5abe848640 1 -- 172.21.15.89:0/1672120906 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5ab81049c0 msgr2=0x7f5ab819a3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:57.182 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:57.169+0000 7f5abe848640 1 --2- 172.21.15.89:0/1672120906 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5ab81049c0 0x7f5ab819a3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:57.182 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:57.169+0000 7f5abe848640 1 -- 172.21.15.89:0/1672120906 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ab81a02a0 con 0x7f5ab810de20 2024-09-21T03:42:57.182 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:57.169+0000 7f5abe848640 1 --2- 172.21.15.89:0/1672120906 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5ab810de20 0x7f5ab819fc30 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f5ab000c970 tx=0x7f5ab000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:57.182 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:57.169+0000 7f5aaf7fe640 1 -- 172.21.15.89:0/1672120906 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ab0007bb0 con 0x7f5ab810de20 2024-09-21T03:42:57.182 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:57.169+0000 7f5aaf7fe640 1 -- 172.21.15.89:0/1672120906 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5ab0007d50 con 0x7f5ab810de20 2024-09-21T03:42:57.182 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:57.169+0000 7f5ac02ab640 1 -- 172.21.15.89:0/1672120906 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ab81a0590 con 0x7f5ab810de20 2024-09-21T03:42:57.182 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:57.169+0000 7f5aaf7fe640 1 -- 172.21.15.89:0/1672120906 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ab0018660 con 0x7f5ab810de20 2024-09-21T03:42:57.182 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:57.169+0000 7f5ac02ab640 1 -- 172.21.15.89:0/1672120906 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ab81a6db0 con 0x7f5ab810de20 2024-09-21T03:42:57.183 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:57.170+0000 7f5ac02ab640 1 -- 172.21.15.89:0/1672120906 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5ab806c280 con 0x7f5ab810de20 2024-09-21T03:42:57.183 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:57.172+0000 7f5aaf7fe640 1 -- 172.21.15.89:0/1672120906 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 21) ==== 154512+0+0 (secure 0 0 0) 0x7f5ab000f970 con 0x7f5ab810de20 2024-09-21T03:42:57.183 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:57.172+0000 7f5aaf7fe640 1 --2- 172.21.15.89:0/1672120906 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f5aa00b6540 0x7f5aa00b8a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:57.183 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:57.172+0000 7f5abe047640 1 --2- 172.21.15.89:0/1672120906 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f5aa00b6540 0x7f5aa00b8a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:57.183 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:57.172+0000 7f5aaf7fe640 1 -- 172.21.15.89:0/1672120906 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 9121+0+0 (secure 0 0 0) 0x7f5ab00e78f0 con 0x7f5ab810de20 2024-09-21T03:42:57.183 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:57.173+0000 7f5abe047640 1 --2- 172.21.15.89:0/1672120906 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f5aa00b6540 0x7f5aa00b8a50 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f5ab4009a60 tx=0x7f5ab4009890 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:57.183 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:57.175+0000 7f5aaf7fe640 1 -- 172.21.15.89:0/1672120906 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f5ab0056710 con 0x7f5ab810de20 2024-09-21T03:42:57.356 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:57.354+0000 7f5ac02ab640 1 -- 172.21.15.89:0/1672120906 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f5ab8104dd0 con 0x7f5ab810de20 2024-09-21T03:42:57.358 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:57.355+0000 7f5aaf7fe640 1 -- 172.21.15.89:0/1672120906 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v84) ==== 74+0+24810 (secure 0 0 0) 0x7f5ab0014030 con 0x7f5ab810de20 2024-09-21T03:42:57.358 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-09-21T03:42:57.358 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":84,"fsid":"7c374098-77c9-11ef-baf5-efdc52797490","created":"2024-09-21T03:28:56.681458+0000","modified":"2024-09-21T03:38:53.661606+0000","last_up_change":"2024-09-21T03:36:23.368379+0000","last_in_change":"2024-09-21T03:36:00.131331+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T03:32:37.834339+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-21T03:37:09.443341+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"74","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-21T03:37:11.467821+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"74","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-21T03:37:17.884066+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"27ca5ed8-17d1-4f8e-a7ae-084a914721a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6802","nonce":990539042},{"type":"v1","addr":"172.21.15.89:6803","nonce":990539042}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6804","nonce":990539042},{"type":"v1","addr":"172.21.15.89:6805","nonce":990539042}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6808","nonce":990539042},{"type":"v1","addr":"172.21.15.89:6809","nonce":990539042}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6806","nonce":990539042},{"type":"v1","addr":"172.21.15.89:6807","nonce":990539042}]},"public_addr":"172.21.15.89:6803/990539042","cluster_addr":"172.21.15.89:6805/990539042","heartbeat_back_addr":"172.21.15.89:6809/990539042","heartbeat_front_addr":"172.21.15.89:6807/990539042","state":["exists","up"]},{"osd":1,"uuid":"5176a910-5cf9-49d7-aedb-a20cf36ea075","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6800","nonce":3235852628},{"type":"v1","addr":"172.21.15.110:6801","nonce":3235852628}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6802","nonce":3235852628},{"type":"v1","addr":"172.21.15.110:6803","nonce":3235852628}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6806","nonce":3235852628},{"type":"v1","addr":"172.21.15.110:6807","nonce":3235852628}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6804","nonce":3235852628},{"type":"v1","addr":"172.21.15.110:6805","nonce":3235852628}]},"public_addr":"172.21.15.110:6801/3235852628","cluster_addr":"172.21.15.110:6803/3235852628","heartbeat_back_addr":"172.21.15.110:6807/3235852628","heartbeat_front_addr":"172.21.15.110:6805/3235852628","state":["exists","up"]},{"osd":2,"uuid":"d4977df1-b118-4d7e-b9a9-626a8accc812","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6800","nonce":169452122},{"type":"v1","addr":"172.21.15.114:6801","nonce":169452122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6802","nonce":169452122},{"type":"v1","addr":"172.21.15.114:6803","nonce":169452122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6806","nonce":169452122},{"type":"v1","addr":"172.21.15.114:6807","nonce":169452122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6804","nonce":169452122},{"type":"v1","addr":"172.21.15.114:6805","nonce":169452122}]},"public_addr":"172.21.15.114:6801/169452122","cluster_addr":"172.21.15.114:6803/169452122","heartbeat_back_addr":"172.21.15.114:6807/169452122","heartbeat_front_addr":"172.21.15.114:6805/169452122","state":["exists","up"]},{"osd":3,"uuid":"2c7ae1b4-0bf2-4954-81ed-55d03eab82c6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6810","nonce":3016980877},{"type":"v1","addr":"172.21.15.89:6811","nonce":3016980877}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6812","nonce":3016980877},{"type":"v1","addr":"172.21.15.89:6813","nonce":3016980877}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6816","nonce":3016980877},{"type":"v1","addr":"172.21.15.89:6817","nonce":3016980877}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6814","nonce":3016980877},{"type":"v1","addr":"172.21.15.89:6815","nonce":3016980877}]},"public_addr":"172.21.15.89:6811/3016980877","cluster_addr":"172.21.15.89:6813/3016980877","heartbeat_back_addr":"172.21.15.89:6817/3016980877","heartbeat_front_addr":"172.21.15.89:6815/3016980877","state":["exists","up"]},{"osd":4,"uuid":"8e65399d-6e55-4da4-98ba-f805f0c655a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":30,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6808","nonce":3217880476},{"type":"v1","addr":"172.21.15.110:6809","nonce":3217880476}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6810","nonce":3217880476},{"type":"v1","addr":"172.21.15.110:6811","nonce":3217880476}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6814","nonce":3217880476},{"type":"v1","addr":"172.21.15.110:6815","nonce":3217880476}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6812","nonce":3217880476},{"type":"v1","addr":"172.21.15.110:6813","nonce":3217880476}]},"public_addr":"172.21.15.110:6809/3217880476","cluster_addr":"172.21.15.110:6811/3217880476","heartbeat_back_addr":"172.21.15.110:6815/3217880476","heartbeat_front_addr":"172.21.15.110:6813/3217880476","state":["exists","up"]},{"osd":5,"uuid":"b9ec2889-46bd-40e5-8446-73ab28271c67","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6808","nonce":1144140707},{"type":"v1","addr":"172.21.15.114:6809","nonce":1144140707}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6810","nonce":1144140707},{"type":"v1","addr":"172.21.15.114:6811","nonce":1144140707}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6814","nonce":1144140707},{"type":"v1","addr":"172.21.15.114:6815","nonce":1144140707}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6812","nonce":1144140707},{"type":"v1","addr":"172.21.15.114:6813","nonce":1144140707}]},"public_addr":"172.21.15.114:6809/1144140707","cluster_addr":"172.21.15.114:6811/1144140707","heartbeat_back_addr":"172.21.15.114:6815/1144140707","heartbeat_front_addr":"172.21.15.114:6813/1144140707","state":["exists","up"]},{"osd":6,"uuid":"9b79989e-adb0-42b2-b2f6-a2dce56b66e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":41,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6818","nonce":1522586255},{"type":"v1","addr":"172.21.15.89:6819","nonce":1522586255}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6820","nonce":1522586255},{"type":"v1","addr":"172.21.15.89:6821","nonce":1522586255}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6824","nonce":1522586255},{"type":"v1","addr":"172.21.15.89:6825","nonce":1522586255}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6822","nonce":1522586255},{"type":"v1","addr":"172.21.15.89:6823","nonce":1522586255}]},"public_addr":"172.21.15.89:6819/1522586255","cluster_addr":"172.21.15.89:6821/1522586255","heartbeat_back_addr":"172.21.15.89:6825/1522586255","heartbeat_front_addr":"172.21.15.89:6823/1522586255","state":["exists","up"]},{"osd":7,"uuid":"b381821f-141c-47a1-be65-f57ca730391b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6816","nonce":1331818323},{"type":"v1","addr":"172.21.15.110:6817","nonce":1331818323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6818","nonce":1331818323},{"type":"v1","addr":"172.21.15.110:6819","nonce":1331818323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6822","nonce":1331818323},{"type":"v1","addr":"172.21.15.110:6823","nonce":1331818323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6820","nonce":1331818323},{"type":"v1","addr":"172.21.15.110:6821","nonce":1331818323}]},"public_addr":"172.21.15.110:6817/1331818323","cluster_addr":"172.21.15.110:6819/1331818323","heartbeat_back_addr":"172.21.15.110:6823/1331818323","heartbeat_front_addr":"172.21.15.110:6821/1331818323","state":["exists","up"]},{"osd":8,"uuid":"1bc2a5b4-15f3-456f-9d67-e51307a27bcc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6816","nonce":2138556699},{"type":"v1","addr":"172.21.15.114:6817","nonce":2138556699}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6818","nonce":2138556699},{"type":"v1","addr":"172.21.15.114:6819","nonce":2138556699}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6822","nonce":2138556699},{"type":"v1","addr":"172.21.15.114:6823","nonce":2138556699}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6820","nonce":2138556699},{"type":"v1","addr":"172.21.15.114:6821","nonce":2138556699}]},"public_addr":"172.21.15.114:6817/2138556699","cluster_addr":"172.21.15.114:6819/2138556699","heartbeat_back_addr":"172.21.15.114:6823/2138556699","heartbeat_front_addr":"172.21.15.114:6821/2138556699","state":["exists","up"]},{"osd":9,"uuid":"91f5fd29-d4a7-4800-ac7a-e83b13add89e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6826","nonce":1730253068},{"type":"v1","addr":"172.21.15.89:6827","nonce":1730253068}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6828","nonce":1730253068},{"type":"v1","addr":"172.21.15.89:6829","nonce":1730253068}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6832","nonce":1730253068},{"type":"v1","addr":"172.21.15.89:6833","nonce":1730253068}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6830","nonce":1730253068},{"type":"v1","addr":"172.21.15.89:6831","nonce":1730253068}]},"public_addr":"172.21.15.89:6827/1730253068","cluster_addr":"172.21.15.89:6829/1730253068","heartbeat_back_addr":"172.21.15.89:6833/1730253068","heartbeat_front_addr":"172.21.15.89:6831/1730253068","state":["exists","up"]},{"osd":10,"uuid":"01390499-2a4d-4188-b57a-fcb10c668e2c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6824","nonce":2904016117},{"type":"v1","addr":"172.21.15.110:6825","nonce":2904016117}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6826","nonce":2904016117},{"type":"v1","addr":"172.21.15.110:6827","nonce":2904016117}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6830","nonce":2904016117},{"type":"v1","addr":"172.21.15.110:6831","nonce":2904016117}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6828","nonce":2904016117},{"type":"v1","addr":"172.21.15.110:6829","nonce":2904016117}]},"public_addr":"172.21.15.110:6825/2904016117","cluster_addr":"172.21.15.110:6827/2904016117","heartbeat_back_addr":"172.21.15.110:6831/2904016117","heartbeat_front_addr":"172.21.15.110:6829/2904016117","state":["exists","up"]},{"osd":11,"uuid":"c3c0f38a-c3d3-415d-a9f8-875d5b756e35","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6824","nonce":3749854486},{"type":"v1","addr":"172.21.15.114:6825","nonce":3749854486}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6826","nonce":3749854486},{"type":"v1","addr":"172.21.15.114:6827","nonce":3749854486}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6830","nonce":3749854486},{"type":"v1","addr":"172.21.15.114:6831","nonce":3749854486}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6828","nonce":3749854486},{"type":"v1","addr":"172.21.15.114:6829","nonce":3749854486}]},"public_addr":"172.21.15.114:6825/3749854486","cluster_addr":"172.21.15.114:6827/3749854486","heartbeat_back_addr":"172.21.15.114:6831/3749854486","heartbeat_front_addr":"172.21.15.114:6829/3749854486","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:31:46.218908+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:32:10.338230+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:32:34.830596+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:33:00.816102+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:33:25.223838+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:33:50.188230+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:34:16.564255+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:34:41.498230+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:35:05.826322+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:35:31.234450+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:35:56.474373+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:36:21.741254+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.c","mappings":[{"from":2,"to":4}]},{"pgid":"2.e","mappings":[{"from":2,"to":1}]},{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.1b","mappings":[{"from":5,"to":9}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]},{"pgid":"4.6","mappings":[{"from":9,"to":11}]},{"pgid":"4.12","mappings":[{"from":9,"to":10}]},{"pgid":"4.13","mappings":[{"from":9,"to":8}]},{"pgid":"4.1b","mappings":[{"from":5,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.89:6801/655110910":"2024-09-22T03:29:51.519106+0000","172.21.15.89:0/1618544983":"2024-09-22T03:29:51.519106+0000","172.21.15.89:0/1265502758":"2024-09-22T03:29:51.519106+0000","172.21.15.89:0/4017684419":"2024-09-22T03:29:51.519106+0000","172.21.15.89:6801/3312712420":"2024-09-22T03:29:26.004686+0000","172.21.15.89:6800/3312712420":"2024-09-22T03:29:26.004686+0000","172.21.15.89:0/2056576483":"2024-09-22T03:29:26.004686+0000","172.21.15.89:6800/655110910":"2024-09-22T03:29:51.519106+0000","172.21.15.89:0/2382866302":"2024-09-22T03:29:26.004686+0000","172.21.15.89:0/1584351801":"2024-09-22T03:29:26.004686+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T03:42:57.360 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:57.358+0000 7f5ac02ab640 1 -- 172.21.15.89:0/1672120906 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f5aa00b6540 msgr2=0x7f5aa00b8a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:57.360 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:57.358+0000 7f5ac02ab640 1 --2- 172.21.15.89:0/1672120906 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f5aa00b6540 0x7f5aa00b8a50 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f5ab4009a60 tx=0x7f5ab4009890 comp rx=0 tx=0).stop 2024-09-21T03:42:57.361 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:57.358+0000 7f5ac02ab640 1 -- 172.21.15.89:0/1672120906 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5ab810de20 msgr2=0x7f5ab819fc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:57.361 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:57.358+0000 7f5ac02ab640 1 --2- 172.21.15.89:0/1672120906 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5ab810de20 0x7f5ab819fc30 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f5ab000c970 tx=0x7f5ab000ce30 comp rx=0 tx=0).stop 2024-09-21T03:42:57.361 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:57.358+0000 7f5ac02ab640 1 -- 172.21.15.89:0/1672120906 shutdown_connections 2024-09-21T03:42:57.361 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:57.358+0000 7f5ac02ab640 1 --2- 172.21.15.89:0/1672120906 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f5aa00b6540 0x7f5aa00b8a50 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:57.361 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:57.358+0000 7f5ac02ab640 1 --2- 172.21.15.89:0/1672120906 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5ab810de20 0x7f5ab819fc30 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:57.361 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:57.358+0000 7f5ac02ab640 1 --2- 172.21.15.89:0/1672120906 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5ab8105310 0x7f5ab819a990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:57.361 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:57.358+0000 7f5ac02ab640 1 --2- 172.21.15.89:0/1672120906 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5ab81049c0 0x7f5ab819a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:57.361 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:57.358+0000 7f5ac02ab640 1 -- 172.21.15.89:0/1672120906 >> 172.21.15.89:0/1672120906 conn(0x7f5ab8079710 msgr2=0x7f5ab8107940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:42:57.361 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:57.358+0000 7f5ac02ab640 1 -- 172.21.15.89:0/1672120906 shutdown_connections 2024-09-21T03:42:57.361 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:57.358+0000 7f5ac02ab640 1 -- 172.21.15.89:0/1672120906 wait complete. 2024-09-21T03:42:57.461 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:42:57 smithi114 ceph-mon[34140]: pgmap v466: 129 pgs: 129 active+clean; 1.1 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 597 B/s wr, 7 op/s 2024-09-21T03:42:57.500 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:42:57 smithi110 ceph-mon[34000]: pgmap v466: 129 pgs: 129 active+clean; 1.1 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 597 B/s wr, 7 op/s 2024-09-21T03:42:58.150 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 7c374098-77c9-11ef-baf5-efdc52797490 -- ceph fs fail cephfs 2024-09-21T03:42:58.311 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:42:58 smithi089 ceph-mon[30928]: from='client.? 172.21.15.89:0/1672120906' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T03:42:58.399 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/7c374098-77c9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:42:58.461 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:42:58 smithi114 ceph-mon[34140]: from='client.? 172.21.15.89:0/1672120906' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T03:42:58.500 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:42:58 smithi110 ceph-mon[34000]: from='client.? 172.21.15.89:0/1672120906' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T03:42:58.908 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:58.903+0000 7fd0d301c640 1 -- 172.21.15.89:0/3320271542 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd0cc109df0 msgr2=0x7fd0cc110620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:58.908 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:58.903+0000 7fd0d301c640 1 --2- 172.21.15.89:0/3320271542 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd0cc109df0 0x7fd0cc110620 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7fd0c4009ca0 tx=0x7fd0c402a8e0 comp rx=0 tx=0).stop 2024-09-21T03:42:58.908 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:58.903+0000 7fd0d301c640 1 -- 172.21.15.89:0/3320271542 shutdown_connections 2024-09-21T03:42:58.908 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:58.903+0000 7fd0d301c640 1 --2- 172.21.15.89:0/3320271542 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd0cc109df0 0x7fd0cc110620 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:58.908 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:58.903+0000 7fd0d301c640 1 --2- 172.21.15.89:0/3320271542 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd0cc1094a0 0x7fd0cc1098b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:58.908 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:58.903+0000 7fd0d301c640 1 --2- 172.21.15.89:0/3320271542 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd0cc103b60 0x7fd0cc103f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:58.908 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:58.903+0000 7fd0d301c640 1 -- 172.21.15.89:0/3320271542 >> 172.21.15.89:0/3320271542 conn(0x7fd0cc07b850 msgr2=0x7fd0cc07bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:42:58.909 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:58.903+0000 7fd0d301c640 1 -- 172.21.15.89:0/3320271542 shutdown_connections 2024-09-21T03:42:58.909 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:58.903+0000 7fd0d301c640 1 -- 172.21.15.89:0/3320271542 wait complete. 2024-09-21T03:42:58.909 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:58.904+0000 7fd0d301c640 1 Processor -- start 2024-09-21T03:42:58.909 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:58.904+0000 7fd0d301c640 1 -- start start 2024-09-21T03:42:58.909 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:58.904+0000 7fd0d301c640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd0cc103b60 0x7fd0cc19a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:58.909 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:58.904+0000 7fd0d301c640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd0cc1094a0 0x7fd0cc19a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:58.909 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:58.904+0000 7fd0d301c640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd0cc109df0 0x7fd0cc19fc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:58.909 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:58.904+0000 7fd0d301c640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fd0cc1139a0 con 0x7fd0cc109df0 2024-09-21T03:42:58.909 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:58.904+0000 7fd0d301c640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fd0cc113820 con 0x7fd0cc1094a0 2024-09-21T03:42:58.909 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:58.904+0000 7fd0d301c640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fd0cc113b20 con 0x7fd0cc103b60 2024-09-21T03:42:58.909 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:58.904+0000 7fd0d15b9640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd0cc109df0 0x7fd0cc19fc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:58.909 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:58.904+0000 7fd0d15b9640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd0cc109df0 0x7fd0cc19fc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:39080/0 (socket says 172.21.15.89:39080) 2024-09-21T03:42:58.909 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:58.904+0000 7fd0d15b9640 1 -- 172.21.15.89:0/479033211 learned_addr learned my addr 172.21.15.89:0/479033211 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-21T03:42:58.909 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:58.904+0000 7fd0d0db8640 1 --2- 172.21.15.89:0/479033211 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd0cc103b60 0x7fd0cc19a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:58.910 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:58.904+0000 7fd0cbfff640 1 --2- 172.21.15.89:0/479033211 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd0cc1094a0 0x7fd0cc19a9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:58.910 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:58.904+0000 7fd0d15b9640 1 -- 172.21.15.89:0/479033211 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd0cc103b60 msgr2=0x7fd0cc19a430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:58.910 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:58.904+0000 7fd0d15b9640 1 --2- 172.21.15.89:0/479033211 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd0cc103b60 0x7fd0cc19a430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:58.910 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:58.904+0000 7fd0d15b9640 1 -- 172.21.15.89:0/479033211 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd0cc1094a0 msgr2=0x7fd0cc19a9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:58.910 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:58.904+0000 7fd0d15b9640 1 --2- 172.21.15.89:0/479033211 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd0cc1094a0 0x7fd0cc19a9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:58.910 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:58.904+0000 7fd0d15b9640 1 -- 172.21.15.89:0/479033211 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0cc1a02e0 con 0x7fd0cc109df0 2024-09-21T03:42:58.910 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:58.904+0000 7fd0cbfff640 1 --2- 172.21.15.89:0/479033211 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd0cc1094a0 0x7fd0cc19a9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:42:58.910 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:58.905+0000 7fd0d15b9640 1 --2- 172.21.15.89:0/479033211 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd0cc109df0 0x7fd0cc19fc70 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7fd0c4006010 tx=0x7fd0c4003c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:58.910 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:58.905+0000 7fd0c9ffb640 1 -- 172.21.15.89:0/479033211 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0c4004240 con 0x7fd0cc109df0 2024-09-21T03:42:58.910 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:58.905+0000 7fd0c9ffb640 1 -- 172.21.15.89:0/479033211 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd0c40305e0 con 0x7fd0cc109df0 2024-09-21T03:42:58.910 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:58.905+0000 7fd0d301c640 1 -- 172.21.15.89:0/479033211 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd0cc1a0570 con 0x7fd0cc109df0 2024-09-21T03:42:58.910 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:58.905+0000 7fd0c9ffb640 1 -- 172.21.15.89:0/479033211 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0c403b660 con 0x7fd0cc109df0 2024-09-21T03:42:58.910 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:58.905+0000 7fd0d301c640 1 -- 172.21.15.89:0/479033211 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0cc1a0ab0 con 0x7fd0cc109df0 2024-09-21T03:42:58.911 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:58.906+0000 7fd0d301c640 1 -- 172.21.15.89:0/479033211 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd0cc103f70 con 0x7fd0cc109df0 2024-09-21T03:42:58.911 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:58.908+0000 7fd0c9ffb640 1 -- 172.21.15.89:0/479033211 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 21) ==== 154512+0+0 (secure 0 0 0) 0x7fd0c4030790 con 0x7fd0cc109df0 2024-09-21T03:42:58.911 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:58.909+0000 7fd0c9ffb640 1 --2- 172.21.15.89:0/479033211 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7fd0b40b6540 0x7fd0b40b8a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:58.911 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:58.909+0000 7fd0c9ffb640 1 -- 172.21.15.89:0/479033211 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 9121+0+0 (secure 0 0 0) 0x7fd0c4108cb0 con 0x7fd0cc109df0 2024-09-21T03:42:58.912 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:58.909+0000 7fd0d0db8640 1 --2- 172.21.15.89:0/479033211 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7fd0b40b6540 0x7fd0b40b8a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:58.912 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:58.910+0000 7fd0d0db8640 1 --2- 172.21.15.89:0/479033211 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7fd0b40b6540 0x7fd0b40b8a50 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fd0bc005e10 tx=0x7fd0bc005da0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:58.915 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:58.912+0000 7fd0c9ffb640 1 -- 172.21.15.89:0/479033211 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fd0c40cf1c0 con 0x7fd0cc109df0 2024-09-21T03:42:59.131 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:59.128+0000 7fd0d301c640 1 -- 172.21.15.89:0/479033211 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7fd0cc19b560 con 0x7fd0cc109df0 2024-09-21T03:42:59.131 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:42:59 smithi089 ceph-mon[30928]: pgmap v467: 129 pgs: 129 active+clean; 1.1 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 597 B/s wr, 6 op/s 2024-09-21T03:42:59.133 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:59.131+0000 7fd0c9ffb640 1 -- 172.21.15.89:0/479033211 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v27) ==== 161+0+0 (secure 0 0 0) 0x7fd0cc19b560 con 0x7fd0cc109df0 2024-09-21T03:42:59.133 INFO:teuthology.orchestra.run.smithi089.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-21T03:42:59.136 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:59.134+0000 7fd0d301c640 1 -- 172.21.15.89:0/479033211 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7fd0b40b6540 msgr2=0x7fd0b40b8a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:59.137 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:59.134+0000 7fd0d301c640 1 --2- 172.21.15.89:0/479033211 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7fd0b40b6540 0x7fd0b40b8a50 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fd0bc005e10 tx=0x7fd0bc005da0 comp rx=0 tx=0).stop 2024-09-21T03:42:59.137 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:59.134+0000 7fd0d301c640 1 -- 172.21.15.89:0/479033211 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd0cc109df0 msgr2=0x7fd0cc19fc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:59.137 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:59.134+0000 7fd0d301c640 1 --2- 172.21.15.89:0/479033211 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd0cc109df0 0x7fd0cc19fc70 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7fd0c4006010 tx=0x7fd0c4003c90 comp rx=0 tx=0).stop 2024-09-21T03:42:59.137 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:59.134+0000 7fd0d301c640 1 -- 172.21.15.89:0/479033211 shutdown_connections 2024-09-21T03:42:59.137 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:59.134+0000 7fd0d301c640 1 --2- 172.21.15.89:0/479033211 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7fd0b40b6540 0x7fd0b40b8a50 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:59.137 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:59.134+0000 7fd0d301c640 1 --2- 172.21.15.89:0/479033211 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd0cc1094a0 0x7fd0cc19a9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:59.137 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:59.134+0000 7fd0d301c640 1 --2- 172.21.15.89:0/479033211 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd0cc109df0 0x7fd0cc19fc70 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:59.137 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:59.134+0000 7fd0d301c640 1 --2- 172.21.15.89:0/479033211 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd0cc103b60 0x7fd0cc19a430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:59.137 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:59.134+0000 7fd0d301c640 1 -- 172.21.15.89:0/479033211 >> 172.21.15.89:0/479033211 conn(0x7fd0cc07b850 msgr2=0x7fd0cc1075e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:42:59.137 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:59.134+0000 7fd0d301c640 1 -- 172.21.15.89:0/479033211 shutdown_connections 2024-09-21T03:42:59.137 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:42:59.134+0000 7fd0d301c640 1 -- 172.21.15.89:0/479033211 wait complete. 2024-09-21T03:42:59.461 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:42:59 smithi114 ceph-mon[34140]: pgmap v467: 129 pgs: 129 active+clean; 1.1 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 597 B/s wr, 6 op/s 2024-09-21T03:42:59.461 INFO:journalctl@ceph.mds.c.smithi114.stdout:Sep 21 03:42:59 smithi114 ceph-7c374098-77c9-11ef-baf5-efdc52797490-mds-c[62970]: ignoring --setuser ceph since I am not root 2024-09-21T03:42:59.461 INFO:journalctl@ceph.mds.c.smithi114.stdout:Sep 21 03:42:59 smithi114 ceph-7c374098-77c9-11ef-baf5-efdc52797490-mds-c[62970]: ignoring --setgroup ceph since I am not root 2024-09-21T03:42:59.461 INFO:journalctl@ceph.mds.c.smithi114.stdout:Sep 21 03:42:59 smithi114 ceph-7c374098-77c9-11ef-baf5-efdc52797490-mds-c[62970]: starting mds.c at 2024-09-21T03:42:59.462 INFO:journalctl@ceph.mds.i.smithi114.stdout:Sep 21 03:42:59 smithi114 ceph-7c374098-77c9-11ef-baf5-efdc52797490-mds-i[63238]: ignoring --setuser ceph since I am not root 2024-09-21T03:42:59.462 INFO:journalctl@ceph.mds.i.smithi114.stdout:Sep 21 03:42:59 smithi114 ceph-7c374098-77c9-11ef-baf5-efdc52797490-mds-i[63238]: ignoring --setgroup ceph since I am not root 2024-09-21T03:42:59.462 INFO:journalctl@ceph.mds.i.smithi114.stdout:Sep 21 03:42:59 smithi114 ceph-mds[63242]: -- 172.21.15.114:0/3926337024 <== mon.1 v2:172.21.15.114:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5579953ef860 con 0x557999597180 2024-09-21T03:42:59.462 INFO:journalctl@ceph.mds.i.smithi114.stdout:Sep 21 03:42:59 smithi114 ceph-7c374098-77c9-11ef-baf5-efdc52797490-mds-i[63238]: starting mds.i at 2024-09-21T03:42:59.462 INFO:journalctl@ceph.mds.f.smithi114.stdout:Sep 21 03:42:59 smithi114 ceph-7c374098-77c9-11ef-baf5-efdc52797490-mds-f[62699]: ignoring --setuser ceph since I am not root 2024-09-21T03:42:59.462 INFO:journalctl@ceph.mds.f.smithi114.stdout:Sep 21 03:42:59 smithi114 ceph-7c374098-77c9-11ef-baf5-efdc52797490-mds-f[62699]: ignoring --setgroup ceph since I am not root 2024-09-21T03:42:59.462 INFO:journalctl@ceph.mds.f.smithi114.stdout:Sep 21 03:42:59 smithi114 ceph-mds[62703]: -- 172.21.15.114:0/2935989014 <== mon.1 v2:172.21.15.114:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5573010a9c20 con 0x557305251a80 2024-09-21T03:42:59.462 INFO:journalctl@ceph.mds.f.smithi114.stdout:Sep 21 03:42:59 smithi114 ceph-7c374098-77c9-11ef-baf5-efdc52797490-mds-f[62699]: starting mds.f at 2024-09-21T03:42:59.500 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:42:59 smithi110 ceph-mon[34000]: pgmap v467: 129 pgs: 129 active+clean; 1.1 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 597 B/s wr, 6 op/s 2024-09-21T03:42:59.500 INFO:journalctl@ceph.mds.e.smithi110.stdout:Sep 21 03:42:59 smithi110 ceph-7c374098-77c9-11ef-baf5-efdc52797490-mds-e[61549]: ignoring --setuser ceph since I am not root 2024-09-21T03:42:59.500 INFO:journalctl@ceph.mds.e.smithi110.stdout:Sep 21 03:42:59 smithi110 ceph-7c374098-77c9-11ef-baf5-efdc52797490-mds-e[61549]: ignoring --setgroup ceph since I am not root 2024-09-21T03:42:59.500 INFO:journalctl@ceph.mds.e.smithi110.stdout:Sep 21 03:42:59 smithi110 ceph-mds[61553]: -- 172.21.15.110:0/1897480087 <== mon.0 v2:172.21.15.89:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55f72d111860 con 0x55f7312b9180 2024-09-21T03:42:59.500 INFO:journalctl@ceph.mds.e.smithi110.stdout:Sep 21 03:42:59 smithi110 ceph-7c374098-77c9-11ef-baf5-efdc52797490-mds-e[61549]: starting mds.e at 2024-09-21T03:42:59.501 INFO:journalctl@ceph.mds.b.smithi110.stdout:Sep 21 03:42:59 smithi110 ceph-7c374098-77c9-11ef-baf5-efdc52797490-mds-b[62090]: ignoring --setuser ceph since I am not root 2024-09-21T03:42:59.501 INFO:journalctl@ceph.mds.b.smithi110.stdout:Sep 21 03:42:59 smithi110 ceph-7c374098-77c9-11ef-baf5-efdc52797490-mds-b[62090]: ignoring --setgroup ceph since I am not root 2024-09-21T03:42:59.501 INFO:journalctl@ceph.mds.b.smithi110.stdout:Sep 21 03:42:59 smithi110 ceph-mds[62094]: -- 172.21.15.110:0/972072370 <== mon.2 v2:172.21.15.110:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55d98ed6b860 con 0x55d992f13600 2024-09-21T03:42:59.501 INFO:journalctl@ceph.mds.b.smithi110.stdout:Sep 21 03:42:59 smithi110 ceph-7c374098-77c9-11ef-baf5-efdc52797490-mds-b[62090]: starting mds.b at 2024-09-21T03:42:59.501 INFO:journalctl@ceph.mds.h.smithi110.stdout:Sep 21 03:42:59 smithi110 ceph-7c374098-77c9-11ef-baf5-efdc52797490-mds-h[61819]: ignoring --setuser ceph since I am not root 2024-09-21T03:42:59.501 INFO:journalctl@ceph.mds.h.smithi110.stdout:Sep 21 03:42:59 smithi110 ceph-7c374098-77c9-11ef-baf5-efdc52797490-mds-h[61819]: ignoring --setgroup ceph since I am not root 2024-09-21T03:42:59.501 INFO:journalctl@ceph.mds.h.smithi110.stdout:Sep 21 03:42:59 smithi110 ceph-mds[61823]: -- 172.21.15.110:0/2491193521 <== mon.0 v2:172.21.15.89:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5600b1751860 con 0x5600b58f9a80 2024-09-21T03:42:59.501 INFO:journalctl@ceph.mds.h.smithi110.stdout:Sep 21 03:42:59 smithi110 ceph-7c374098-77c9-11ef-baf5-efdc52797490-mds-h[61819]: starting mds.h at 2024-09-21T03:42:59.504 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:42:59 smithi089 ceph-7c374098-77c9-11ef-baf5-efdc52797490-mon-a[30924]: 2024-09-21T03:42:59.129+0000 7fa868f4c640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-21T03:42:59.928 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 7c374098-77c9-11ef-baf5-efdc52797490 -- ceph fs rm cephfs --yes-i-really-mean-it 2024-09-21T03:43:00.178 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/7c374098-77c9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:43:00.300 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:43:00 smithi089 ceph-mon[30928]: from='client.? 172.21.15.89:0/479033211' entity='client.admin' cmd={"prefix": "fs fail", "fs_name": "cephfs"} : dispatch 2024-09-21T03:43:00.300 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:43:00 smithi089 ceph-mon[30928]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-09-21T03:43:00.300 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:43:00 smithi089 ceph-mon[30928]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-21T03:43:00.300 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:43:00 smithi089 ceph-mon[30928]: from='client.? 172.21.15.89:0/479033211' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-09-21T03:43:00.300 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:43:00 smithi089 ceph-mon[30928]: osdmap e85: 12 total, 12 up, 12 in 2024-09-21T03:43:00.300 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:43:00 smithi089 ceph-mon[30928]: fsmap cephfs:0/3 6 up:standby, 3 failed 2024-09-21T03:43:00.461 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:43:00 smithi114 ceph-mon[34140]: from='client.? 172.21.15.89:0/479033211' entity='client.admin' cmd={"prefix": "fs fail", "fs_name": "cephfs"} : dispatch 2024-09-21T03:43:00.461 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:43:00 smithi114 ceph-mon[34140]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-09-21T03:43:00.461 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:43:00 smithi114 ceph-mon[34140]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-21T03:43:00.461 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:43:00 smithi114 ceph-mon[34140]: from='client.? 172.21.15.89:0/479033211' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-09-21T03:43:00.461 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:43:00 smithi114 ceph-mon[34140]: osdmap e85: 12 total, 12 up, 12 in 2024-09-21T03:43:00.461 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:43:00 smithi114 ceph-mon[34140]: fsmap cephfs:0/3 6 up:standby, 3 failed 2024-09-21T03:43:00.500 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:43:00 smithi110 ceph-mon[34000]: from='client.? 172.21.15.89:0/479033211' entity='client.admin' cmd={"prefix": "fs fail", "fs_name": "cephfs"} : dispatch 2024-09-21T03:43:00.500 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:43:00 smithi110 ceph-mon[34000]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-09-21T03:43:00.500 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:43:00 smithi110 ceph-mon[34000]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-21T03:43:00.500 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:43:00 smithi110 ceph-mon[34000]: from='client.? 172.21.15.89:0/479033211' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-09-21T03:43:00.500 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:43:00 smithi110 ceph-mon[34000]: osdmap e85: 12 total, 12 up, 12 in 2024-09-21T03:43:00.501 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:43:00 smithi110 ceph-mon[34000]: fsmap cephfs:0/3 6 up:standby, 3 failed 2024-09-21T03:43:00.679 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:43:00.674+0000 7f09cf457640 1 -- 172.21.15.89:0/2210954815 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f09c8109df0 msgr2=0x7f09c8110620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:00.679 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:43:00.674+0000 7f09cf457640 1 --2- 172.21.15.89:0/2210954815 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f09c8109df0 0x7f09c8110620 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f09c0009ca0 tx=0x7f09c002a8c0 comp rx=0 tx=0).stop 2024-09-21T03:43:00.679 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:43:00.674+0000 7f09cf457640 1 -- 172.21.15.89:0/2210954815 shutdown_connections 2024-09-21T03:43:00.679 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:43:00.674+0000 7f09cf457640 1 --2- 172.21.15.89:0/2210954815 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f09c81094a0 0x7f09c81098b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:00.679 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:43:00.674+0000 7f09cf457640 1 --2- 172.21.15.89:0/2210954815 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f09c8109df0 0x7f09c8110620 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:00.679 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:43:00.674+0000 7f09cf457640 1 --2- 172.21.15.89:0/2210954815 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f09c8103b60 0x7f09c8103f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:00.679 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:43:00.674+0000 7f09cf457640 1 -- 172.21.15.89:0/2210954815 >> 172.21.15.89:0/2210954815 conn(0x7f09c807b850 msgr2=0x7f09c807bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:43:00.680 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:43:00.675+0000 7f09cf457640 1 -- 172.21.15.89:0/2210954815 shutdown_connections 2024-09-21T03:43:00.680 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:43:00.675+0000 7f09cf457640 1 -- 172.21.15.89:0/2210954815 wait complete. 2024-09-21T03:43:00.680 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:43:00.675+0000 7f09cf457640 1 Processor -- start 2024-09-21T03:43:00.680 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:43:00.675+0000 7f09cf457640 1 -- start start 2024-09-21T03:43:00.680 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:43:00.676+0000 7f09cf457640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f09c8103b60 0x7f09c819a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:00.680 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:43:00.676+0000 7f09cf457640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f09c81094a0 0x7f09c819aa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:00.680 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:43:00.676+0000 7f09cf457640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f09c8109df0 0x7f09c819fd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:00.680 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:43:00.676+0000 7f09cf457640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f09c81139b0 con 0x7f09c81094a0 2024-09-21T03:43:00.680 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:43:00.676+0000 7f09cf457640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f09c8113830 con 0x7f09c8109df0 2024-09-21T03:43:00.680 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:43:00.676+0000 7f09cf457640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f09c8113b30 con 0x7f09c8103b60 2024-09-21T03:43:00.680 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:43:00.676+0000 7f09cc9f2640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f09c81094a0 0x7f09c819aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:00.680 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:43:00.676+0000 7f09cc9f2640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f09c81094a0 0x7f09c819aa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:39110/0 (socket says 172.21.15.89:39110) 2024-09-21T03:43:00.680 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:43:00.676+0000 7f09cc9f2640 1 -- 172.21.15.89:0/1649827022 learned_addr learned my addr 172.21.15.89:0/1649827022 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-21T03:43:00.680 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:43:00.676+0000 7f09cd9f4640 1 --2- 172.21.15.89:0/1649827022 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f09c8109df0 0x7f09c819fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:00.681 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:43:00.676+0000 7f09cd1f3640 1 --2- 172.21.15.89:0/1649827022 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f09c8103b60 0x7f09c819a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:00.681 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:43:00.676+0000 7f09cc9f2640 1 -- 172.21.15.89:0/1649827022 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f09c8103b60 msgr2=0x7f09c819a4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:00.681 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:43:00.676+0000 7f09cc9f2640 1 --2- 172.21.15.89:0/1649827022 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f09c8103b60 0x7f09c819a4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:00.681 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:43:00.676+0000 7f09cc9f2640 1 -- 172.21.15.89:0/1649827022 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f09c8109df0 msgr2=0x7f09c819fd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:00.681 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:43:00.676+0000 7f09cc9f2640 1 --2- 172.21.15.89:0/1649827022 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f09c8109df0 0x7f09c819fd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:00.681 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:43:00.676+0000 7f09cc9f2640 1 -- 172.21.15.89:0/1649827022 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09c81a0430 con 0x7f09c81094a0 2024-09-21T03:43:00.681 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:43:00.676+0000 7f09cd9f4640 1 --2- 172.21.15.89:0/1649827022 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f09c8109df0 0x7f09c819fd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:43:00.681 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:43:00.676+0000 7f09cc9f2640 1 --2- 172.21.15.89:0/1649827022 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f09c81094a0 0x7f09c819aa60 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f09bc00d950 tx=0x7f09bc00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:43:00.681 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:43:00.676+0000 7f09ba7fc640 1 -- 172.21.15.89:0/1649827022 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09bc017070 con 0x7f09c81094a0 2024-09-21T03:43:00.681 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:43:00.676+0000 7f09ba7fc640 1 -- 172.21.15.89:0/1649827022 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f09bc009c30 con 0x7f09c81094a0 2024-09-21T03:43:00.681 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:43:00.676+0000 7f09cf457640 1 -- 172.21.15.89:0/1649827022 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f09c81a0720 con 0x7f09c81094a0 2024-09-21T03:43:00.681 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:43:00.676+0000 7f09ba7fc640 1 -- 172.21.15.89:0/1649827022 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09bc0047a0 con 0x7f09c81094a0 2024-09-21T03:43:00.681 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:43:00.676+0000 7f09cf457640 1 -- 172.21.15.89:0/1649827022 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09c81a6e80 con 0x7f09c81094a0 2024-09-21T03:43:00.682 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:43:00.677+0000 7f09cf457640 1 -- 172.21.15.89:0/1649827022 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f09c8103f70 con 0x7f09c81094a0 2024-09-21T03:43:00.682 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:43:00.679+0000 7f09ba7fc640 1 -- 172.21.15.89:0/1649827022 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 21) ==== 154512+0+0 (secure 0 0 0) 0x7f09bc004020 con 0x7f09c81094a0 2024-09-21T03:43:00.682 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:43:00.679+0000 7f09ba7fc640 1 --2- 172.21.15.89:0/1649827022 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f09b40b6510 0x7f09b40b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:00.683 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:43:00.680+0000 7f09cd1f3640 1 --2- 172.21.15.89:0/1649827022 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f09b40b6510 0x7f09b40b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:00.683 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:43:00.680+0000 7f09ba7fc640 1 -- 172.21.15.89:0/1649827022 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 9379+0+0 (secure 0 0 0) 0x7f09bc0e78c0 con 0x7f09c81094a0 2024-09-21T03:43:00.683 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:43:00.680+0000 7f09cd1f3640 1 --2- 172.21.15.89:0/1649827022 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f09b40b6510 0x7f09b40b8a20 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f09c40058c0 tx=0x7f09c4005c60 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:43:00.685 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:43:00.683+0000 7f09ba7fc640 1 -- 172.21.15.89:0/1649827022 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f09bc020020 con 0x7f09c81094a0 2024-09-21T03:43:00.898 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:43:00.896+0000 7f09cf457640 1 -- 172.21.15.89:0/1649827022 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f09c802e640 con 0x7f09c81094a0 2024-09-21T03:43:01.140 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:43:01.138+0000 7f09ba7fc640 1 -- 172.21.15.89:0/1649827022 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v29) ==== 104+0+0 (secure 0 0 0) 0x7f09bc0adcd0 con 0x7f09c81094a0 2024-09-21T03:43:01.143 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:43:01.140+0000 7f09cf457640 1 -- 172.21.15.89:0/1649827022 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f09b40b6510 msgr2=0x7f09b40b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:01.143 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:43:01.140+0000 7f09cf457640 1 --2- 172.21.15.89:0/1649827022 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f09b40b6510 0x7f09b40b8a20 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f09c40058c0 tx=0x7f09c4005c60 comp rx=0 tx=0).stop 2024-09-21T03:43:01.143 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:43:01.140+0000 7f09cf457640 1 -- 172.21.15.89:0/1649827022 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f09c81094a0 msgr2=0x7f09c819aa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:01.143 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:43:01.140+0000 7f09cf457640 1 --2- 172.21.15.89:0/1649827022 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f09c81094a0 0x7f09c819aa60 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f09bc00d950 tx=0x7f09bc00de10 comp rx=0 tx=0).stop 2024-09-21T03:43:01.143 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:43:01.140+0000 7f09cf457640 1 -- 172.21.15.89:0/1649827022 shutdown_connections 2024-09-21T03:43:01.143 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:43:01.140+0000 7f09cf457640 1 --2- 172.21.15.89:0/1649827022 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f09b40b6510 0x7f09b40b8a20 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:01.143 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:43:01.140+0000 7f09cf457640 1 --2- 172.21.15.89:0/1649827022 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f09c8109df0 0x7f09c819fd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:01.143 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:43:01.140+0000 7f09cf457640 1 --2- 172.21.15.89:0/1649827022 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f09c81094a0 0x7f09c819aa60 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:01.143 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:43:01.140+0000 7f09cf457640 1 --2- 172.21.15.89:0/1649827022 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f09c8103b60 0x7f09c819a4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:01.143 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:43:01.140+0000 7f09cf457640 1 -- 172.21.15.89:0/1649827022 >> 172.21.15.89:0/1649827022 conn(0x7f09c807b850 msgr2=0x7f09c8105610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:43:01.144 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:43:01.140+0000 7f09cf457640 1 -- 172.21.15.89:0/1649827022 shutdown_connections 2024-09-21T03:43:01.144 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:43:01.140+0000 7f09cf457640 1 -- 172.21.15.89:0/1649827022 wait complete. 2024-09-21T03:43:01.155 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:43:01 smithi089 ceph-mon[30928]: pgmap v469: 129 pgs: 129 active+clean; 1.1 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-21T03:43:01.155 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:43:01 smithi089 ceph-mon[30928]: mds.? [v2:172.21.15.114:6834/2357777249,v1:172.21.15.114:6837/2357777249] up:boot 2024-09-21T03:43:01.155 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:43:01 smithi089 ceph-mon[30928]: mds.? [v2:172.21.15.114:6835/3036289443,v1:172.21.15.114:6838/3036289443] up:boot 2024-09-21T03:43:01.155 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:43:01 smithi089 ceph-mon[30928]: mds.? [v2:172.21.15.114:6836/705437003,v1:172.21.15.114:6839/705437003] up:boot 2024-09-21T03:43:01.155 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:43:01 smithi089 ceph-mon[30928]: mds.? [v2:172.21.15.110:6834/1235091367,v1:172.21.15.110:6837/1235091367] up:boot 2024-09-21T03:43:01.155 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:43:01 smithi089 ceph-mon[30928]: mds.? [v2:172.21.15.110:6835/3857196354,v1:172.21.15.110:6838/3857196354] up:boot 2024-09-21T03:43:01.155 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:43:01 smithi089 ceph-mon[30928]: mds.? [v2:172.21.15.110:6836/2640285439,v1:172.21.15.110:6839/2640285439] up:boot 2024-09-21T03:43:01.155 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:43:01 smithi089 ceph-mon[30928]: fsmap cephfs:0/3 12 up:standby, 3 failed 2024-09-21T03:43:01.155 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:43:01 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "i"} : dispatch 2024-09-21T03:43:01.155 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:43:01 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "c"} : dispatch 2024-09-21T03:43:01.155 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:43:01 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "e"} : dispatch 2024-09-21T03:43:01.155 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:43:01 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "f"} : dispatch 2024-09-21T03:43:01.155 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:43:01 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "h"} : dispatch 2024-09-21T03:43:01.156 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:43:01 smithi089 ceph-mon[30928]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "b"} : dispatch 2024-09-21T03:43:01.156 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:43:01 smithi089 ceph-mon[30928]: from='client.? 172.21.15.89:0/1649827022' entity='client.admin' cmd={"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} : dispatch 2024-09-21T03:43:01.461 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:43:01 smithi114 ceph-mon[34140]: pgmap v469: 129 pgs: 129 active+clean; 1.1 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-21T03:43:01.461 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:43:01 smithi114 ceph-mon[34140]: mds.? [v2:172.21.15.114:6834/2357777249,v1:172.21.15.114:6837/2357777249] up:boot 2024-09-21T03:43:01.461 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:43:01 smithi114 ceph-mon[34140]: mds.? [v2:172.21.15.114:6835/3036289443,v1:172.21.15.114:6838/3036289443] up:boot 2024-09-21T03:43:01.461 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:43:01 smithi114 ceph-mon[34140]: mds.? [v2:172.21.15.114:6836/705437003,v1:172.21.15.114:6839/705437003] up:boot 2024-09-21T03:43:01.461 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:43:01 smithi114 ceph-mon[34140]: mds.? [v2:172.21.15.110:6834/1235091367,v1:172.21.15.110:6837/1235091367] up:boot 2024-09-21T03:43:01.461 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:43:01 smithi114 ceph-mon[34140]: mds.? [v2:172.21.15.110:6835/3857196354,v1:172.21.15.110:6838/3857196354] up:boot 2024-09-21T03:43:01.461 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:43:01 smithi114 ceph-mon[34140]: mds.? [v2:172.21.15.110:6836/2640285439,v1:172.21.15.110:6839/2640285439] up:boot 2024-09-21T03:43:01.461 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:43:01 smithi114 ceph-mon[34140]: fsmap cephfs:0/3 12 up:standby, 3 failed 2024-09-21T03:43:01.461 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:43:01 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "i"} : dispatch 2024-09-21T03:43:01.462 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:43:01 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "c"} : dispatch 2024-09-21T03:43:01.462 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:43:01 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "e"} : dispatch 2024-09-21T03:43:01.462 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:43:01 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "f"} : dispatch 2024-09-21T03:43:01.462 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:43:01 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "h"} : dispatch 2024-09-21T03:43:01.462 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:43:01 smithi114 ceph-mon[34140]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "b"} : dispatch 2024-09-21T03:43:01.462 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:43:01 smithi114 ceph-mon[34140]: from='client.? 172.21.15.89:0/1649827022' entity='client.admin' cmd={"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} : dispatch 2024-09-21T03:43:01.500 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:43:01 smithi110 ceph-mon[34000]: pgmap v469: 129 pgs: 129 active+clean; 1.1 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-21T03:43:01.500 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:43:01 smithi110 ceph-mon[34000]: mds.? [v2:172.21.15.114:6834/2357777249,v1:172.21.15.114:6837/2357777249] up:boot 2024-09-21T03:43:01.500 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:43:01 smithi110 ceph-mon[34000]: mds.? [v2:172.21.15.114:6835/3036289443,v1:172.21.15.114:6838/3036289443] up:boot 2024-09-21T03:43:01.500 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:43:01 smithi110 ceph-mon[34000]: mds.? [v2:172.21.15.114:6836/705437003,v1:172.21.15.114:6839/705437003] up:boot 2024-09-21T03:43:01.501 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:43:01 smithi110 ceph-mon[34000]: mds.? [v2:172.21.15.110:6834/1235091367,v1:172.21.15.110:6837/1235091367] up:boot 2024-09-21T03:43:01.501 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:43:01 smithi110 ceph-mon[34000]: mds.? [v2:172.21.15.110:6835/3857196354,v1:172.21.15.110:6838/3857196354] up:boot 2024-09-21T03:43:01.501 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:43:01 smithi110 ceph-mon[34000]: mds.? [v2:172.21.15.110:6836/2640285439,v1:172.21.15.110:6839/2640285439] up:boot 2024-09-21T03:43:01.501 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:43:01 smithi110 ceph-mon[34000]: fsmap cephfs:0/3 12 up:standby, 3 failed 2024-09-21T03:43:01.501 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:43:01 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "i"} : dispatch 2024-09-21T03:43:01.501 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:43:01 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "c"} : dispatch 2024-09-21T03:43:01.501 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:43:01 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "e"} : dispatch 2024-09-21T03:43:01.501 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:43:01 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "f"} : dispatch 2024-09-21T03:43:01.501 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:43:01 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "h"} : dispatch 2024-09-21T03:43:01.501 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:43:01 smithi110 ceph-mon[34000]: from='mgr.14152 172.21.15.89:0/1861734621' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "b"} : dispatch 2024-09-21T03:43:01.501 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:43:01 smithi110 ceph-mon[34000]: from='client.? 172.21.15.89:0/1649827022' entity='client.admin' cmd={"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} : dispatch 2024-09-21T03:43:01.898 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 7c374098-77c9-11ef-baf5-efdc52797490 -- ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-21T03:43:02.153 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:43:02 smithi089 ceph-mon[30928]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-09-21T03:43:02.153 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:43:02 smithi089 ceph-mon[30928]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-09-21T03:43:02.153 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:43:02 smithi089 ceph-mon[30928]: Cluster is now healthy 2024-09-21T03:43:02.153 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:43:02 smithi089 ceph-mon[30928]: from='client.? 172.21.15.89:0/1649827022' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-09-21T03:43:02.153 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:43:02 smithi089 ceph-mon[30928]: fsmap 12 up:standby 2024-09-21T03:43:02.153 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/7c374098-77c9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:43:02.461 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:43:02 smithi114 ceph-mon[34140]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-09-21T03:43:02.461 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:43:02 smithi114 ceph-mon[34140]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-09-21T03:43:02.461 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:43:02 smithi114 ceph-mon[34140]: Cluster is now healthy 2024-09-21T03:43:02.461 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:43:02 smithi114 ceph-mon[34140]: from='client.? 172.21.15.89:0/1649827022' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-09-21T03:43:02.461 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:43:02 smithi114 ceph-mon[34140]: fsmap 12 up:standby 2024-09-21T03:43:02.500 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:43:02 smithi110 ceph-mon[34000]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-09-21T03:43:02.500 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:43:02 smithi110 ceph-mon[34000]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-09-21T03:43:02.500 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:43:02 smithi110 ceph-mon[34000]: Cluster is now healthy 2024-09-21T03:43:02.500 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:43:02 smithi110 ceph-mon[34000]: from='client.? 172.21.15.89:0/1649827022' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-09-21T03:43:02.500 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:43:02 smithi110 ceph-mon[34000]: fsmap 12 up:standby 2024-09-21T03:43:02.648 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:43:02.640+0000 7f3b289cc640 1 -- 172.21.15.89:0/426848897 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3b24103b80 msgr2=0x7f3b24103f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:02.648 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:43:02.640+0000 7f3b289cc640 1 --2- 172.21.15.89:0/426848897 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3b24103b80 0x7f3b24103f90 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f3b1c009900 tx=0x7f3b1c02a410 comp rx=0 tx=0).stop 2024-09-21T03:43:02.648 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:43:02.640+0000 7f3b289cc640 1 -- 172.21.15.89:0/426848897 shutdown_connections 2024-09-21T03:43:02.648 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:43:02.640+0000 7f3b289cc640 1 --2- 172.21.15.89:0/426848897 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3b241094c0 0x7f3b241098d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:02.648 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:43:02.640+0000 7f3b289cc640 1 --2- 172.21.15.89:0/426848897 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3b24109e10 0x7f3b24110640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:02.649 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:43:02.640+0000 7f3b289cc640 1 --2- 172.21.15.89:0/426848897 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3b24103b80 0x7f3b24103f90 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:02.649 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:43:02.640+0000 7f3b289cc640 1 -- 172.21.15.89:0/426848897 >> 172.21.15.89:0/426848897 conn(0x7f3b240ff610 msgr2=0x7f3b24101a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:43:02.649 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:43:02.640+0000 7f3b289cc640 1 -- 172.21.15.89:0/426848897 shutdown_connections 2024-09-21T03:43:02.649 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:43:02.640+0000 7f3b289cc640 1 -- 172.21.15.89:0/426848897 wait complete. 2024-09-21T03:43:02.649 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:43:02.640+0000 7f3b289cc640 1 Processor -- start 2024-09-21T03:43:02.649 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:43:02.641+0000 7f3b289cc640 1 -- start start 2024-09-21T03:43:02.649 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:43:02.641+0000 7f3b289cc640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3b24103b80 0x7f3b2419a410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:02.649 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:43:02.641+0000 7f3b289cc640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3b241094c0 0x7f3b2419a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:02.649 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:43:02.641+0000 7f3b289cc640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3b24109e10 0x7f3b2419fc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:02.649 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:43:02.641+0000 7f3b289cc640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f3b241139e0 con 0x7f3b24103b80 2024-09-21T03:43:02.649 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:43:02.641+0000 7f3b289cc640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f3b24113860 con 0x7f3b241094c0 2024-09-21T03:43:02.649 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:43:02.641+0000 7f3b289cc640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f3b24113b60 con 0x7f3b24109e10 2024-09-21T03:43:02.649 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:43:02.641+0000 7f3b2259c640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3b24103b80 0x7f3b2419a410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:02.649 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:43:02.641+0000 7f3b2259c640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3b24103b80 0x7f3b2419a410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:39128/0 (socket says 172.21.15.89:39128) 2024-09-21T03:43:02.650 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:43:02.641+0000 7f3b2259c640 1 -- 172.21.15.89:0/1696998128 learned_addr learned my addr 172.21.15.89:0/1696998128 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-21T03:43:02.650 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:43:02.641+0000 7f3b2259c640 1 -- 172.21.15.89:0/1696998128 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3b24109e10 msgr2=0x7f3b2419fc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:02.650 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:43:02.641+0000 7f3b2259c640 1 --2- 172.21.15.89:0/1696998128 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3b24109e10 0x7f3b2419fc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:02.650 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:43:02.641+0000 7f3b2259c640 1 -- 172.21.15.89:0/1696998128 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3b241094c0 msgr2=0x7f3b2419a9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:02.650 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:43:02.641+0000 7f3b2259c640 1 --2- 172.21.15.89:0/1696998128 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3b241094c0 0x7f3b2419a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:02.650 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:43:02.641+0000 7f3b2259c640 1 -- 172.21.15.89:0/1696998128 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b241a0380 con 0x7f3b24103b80 2024-09-21T03:43:02.650 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:43:02.641+0000 7f3b2259c640 1 --2- 172.21.15.89:0/1696998128 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3b24103b80 0x7f3b2419a410 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f3b1c009a20 tx=0x7f3b1c004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:43:02.650 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:43:02.642+0000 7f3b137fe640 1 -- 172.21.15.89:0/1696998128 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b1c03f030 con 0x7f3b24103b80 2024-09-21T03:43:02.650 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:43:02.642+0000 7f3b289cc640 1 -- 172.21.15.89:0/1696998128 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3b241a0610 con 0x7f3b24103b80 2024-09-21T03:43:02.650 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:43:02.642+0000 7f3b137fe640 1 -- 172.21.15.89:0/1696998128 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3b1c0035f0 con 0x7f3b24103b80 2024-09-21T03:43:02.650 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:43:02.642+0000 7f3b137fe640 1 -- 172.21.15.89:0/1696998128 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b1c035070 con 0x7f3b24103b80 2024-09-21T03:43:02.650 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:43:02.642+0000 7f3b289cc640 1 -- 172.21.15.89:0/1696998128 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3b241a6dd0 con 0x7f3b24103b80 2024-09-21T03:43:02.651 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:43:02.643+0000 7f3b289cc640 1 -- 172.21.15.89:0/1696998128 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3b24103f90 con 0x7f3b24103b80 2024-09-21T03:43:02.651 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:43:02.644+0000 7f3b137fe640 1 -- 172.21.15.89:0/1696998128 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 21) ==== 154512+0+0 (secure 0 0 0) 0x7f3b1c003aa0 con 0x7f3b24103b80 2024-09-21T03:43:02.651 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:43:02.645+0000 7f3b137fe640 1 --2- 172.21.15.89:0/1696998128 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f3b0c0b6510 0x7f3b0c0b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:02.651 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:43:02.645+0000 7f3b21d9b640 1 --2- 172.21.15.89:0/1696998128 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f3b0c0b6510 0x7f3b0c0b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:02.651 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:43:02.645+0000 7f3b137fe640 1 -- 172.21.15.89:0/1696998128 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 9379+0+0 (secure 0 0 0) 0x7f3b1c1087e0 con 0x7f3b24103b80 2024-09-21T03:43:02.651 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:43:02.646+0000 7f3b21d9b640 1 --2- 172.21.15.89:0/1696998128 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f3b0c0b6510 0x7f3b0c0b8a20 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f3b24108ec0 tx=0x7f3b18005f70 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:43:02.651 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:43:02.648+0000 7f3b137fe640 1 -- 172.21.15.89:0/1696998128 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f3b1c0cebf0 con 0x7f3b24103b80 2024-09-21T03:43:02.828 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:43:02.825+0000 7f3b289cc640 1 -- 172.21.15.89:0/1696998128 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f3b2419b540 con 0x7f3b24103b80 2024-09-21T03:43:03.154 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:43:03.151+0000 7f3b137fe640 1 -- 172.21.15.89:0/1696998128 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v86) ==== 181+0+0 (secure 0 0 0) 0x7f3b1c0d2fd0 con 0x7f3b24103b80 2024-09-21T03:43:03.154 INFO:teuthology.orchestra.run.smithi089.stderr:pool 'cephfs_metadata' removed 2024-09-21T03:43:03.158 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:43:03.155+0000 7f3b289cc640 1 -- 172.21.15.89:0/1696998128 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f3b0c0b6510 msgr2=0x7f3b0c0b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:03.159 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:43:03.155+0000 7f3b289cc640 1 --2- 172.21.15.89:0/1696998128 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f3b0c0b6510 0x7f3b0c0b8a20 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f3b24108ec0 tx=0x7f3b18005f70 comp rx=0 tx=0).stop 2024-09-21T03:43:03.159 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:43:03.155+0000 7f3b289cc640 1 -- 172.21.15.89:0/1696998128 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3b24103b80 msgr2=0x7f3b2419a410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:03.159 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:43:03.155+0000 7f3b289cc640 1 --2- 172.21.15.89:0/1696998128 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3b24103b80 0x7f3b2419a410 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f3b1c009a20 tx=0x7f3b1c004270 comp rx=0 tx=0).stop 2024-09-21T03:43:03.159 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:43:03.156+0000 7f3b289cc640 1 -- 172.21.15.89:0/1696998128 shutdown_connections 2024-09-21T03:43:03.159 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:43:03.156+0000 7f3b289cc640 1 --2- 172.21.15.89:0/1696998128 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f3b0c0b6510 0x7f3b0c0b8a20 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:03.159 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:43:03.156+0000 7f3b289cc640 1 --2- 172.21.15.89:0/1696998128 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3b241094c0 0x7f3b2419a9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:03.159 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:43:03.156+0000 7f3b289cc640 1 --2- 172.21.15.89:0/1696998128 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3b24109e10 0x7f3b2419fc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:03.159 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:43:03.156+0000 7f3b289cc640 1 --2- 172.21.15.89:0/1696998128 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3b24103b80 0x7f3b2419a410 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:03.159 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:43:03.156+0000 7f3b289cc640 1 -- 172.21.15.89:0/1696998128 >> 172.21.15.89:0/1696998128 conn(0x7f3b240ff610 msgr2=0x7f3b24101a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:43:03.159 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:43:03.156+0000 7f3b289cc640 1 -- 172.21.15.89:0/1696998128 shutdown_connections 2024-09-21T03:43:03.159 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:43:03.157+0000 7f3b289cc640 1 -- 172.21.15.89:0/1696998128 wait complete. 2024-09-21T03:43:03.461 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:43:03 smithi114 ceph-mon[34140]: pgmap v470: 129 pgs: 129 active+clean; 1.1 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-21T03:43:03.461 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:43:03 smithi114 ceph-mon[34140]: from='client.? 172.21.15.89:0/1696998128' entity='client.admin' cmd={"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} : dispatch 2024-09-21T03:43:03.500 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:43:03 smithi110 ceph-mon[34000]: pgmap v470: 129 pgs: 129 active+clean; 1.1 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-21T03:43:03.500 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:43:03 smithi110 ceph-mon[34000]: from='client.? 172.21.15.89:0/1696998128' entity='client.admin' cmd={"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} : dispatch 2024-09-21T03:43:03.519 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:43:03 smithi089 ceph-mon[30928]: pgmap v470: 129 pgs: 129 active+clean; 1.1 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-21T03:43:03.519 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:43:03 smithi089 ceph-mon[30928]: from='client.? 172.21.15.89:0/1696998128' entity='client.admin' cmd={"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} : dispatch 2024-09-21T03:43:03.955 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 7c374098-77c9-11ef-baf5-efdc52797490 -- ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-21T03:43:04.209 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/7c374098-77c9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:43:04.311 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:43:04 smithi089 ceph-mon[30928]: from='client.? 172.21.15.89:0/1696998128' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-09-21T03:43:04.311 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:43:04 smithi089 ceph-mon[30928]: osdmap e86: 12 total, 12 up, 12 in 2024-09-21T03:43:04.461 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:43:04 smithi114 ceph-mon[34140]: from='client.? 172.21.15.89:0/1696998128' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-09-21T03:43:04.461 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:43:04 smithi114 ceph-mon[34140]: osdmap e86: 12 total, 12 up, 12 in 2024-09-21T03:43:04.500 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:43:04 smithi110 ceph-mon[34000]: from='client.? 172.21.15.89:0/1696998128' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-09-21T03:43:04.500 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:43:04 smithi110 ceph-mon[34000]: osdmap e86: 12 total, 12 up, 12 in 2024-09-21T03:43:04.735 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:43:04.724+0000 7f104ab29640 1 -- 172.21.15.89:0/4017008935 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1044109df0 msgr2=0x7f1044110620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:04.735 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:43:04.724+0000 7f104ab29640 1 --2- 172.21.15.89:0/4017008935 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1044109df0 0x7f1044110620 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f103c009ca0 tx=0x7f103c02a8c0 comp rx=0 tx=0).stop 2024-09-21T03:43:04.735 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:43:04.724+0000 7f104ab29640 1 -- 172.21.15.89:0/4017008935 shutdown_connections 2024-09-21T03:43:04.735 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:43:04.724+0000 7f104ab29640 1 --2- 172.21.15.89:0/4017008935 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f10441094a0 0x7f10441098b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:04.735 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:43:04.724+0000 7f104ab29640 1 --2- 172.21.15.89:0/4017008935 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1044109df0 0x7f1044110620 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:04.735 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:43:04.724+0000 7f104ab29640 1 --2- 172.21.15.89:0/4017008935 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1044103b60 0x7f1044103f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:04.735 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:43:04.724+0000 7f104ab29640 1 -- 172.21.15.89:0/4017008935 >> 172.21.15.89:0/4017008935 conn(0x7f104407b850 msgr2=0x7f104407bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:43:04.736 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:43:04.725+0000 7f104ab29640 1 -- 172.21.15.89:0/4017008935 shutdown_connections 2024-09-21T03:43:04.736 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:43:04.725+0000 7f104ab29640 1 -- 172.21.15.89:0/4017008935 wait complete. 2024-09-21T03:43:04.736 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:43:04.725+0000 7f104ab29640 1 Processor -- start 2024-09-21T03:43:04.736 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:43:04.725+0000 7f104ab29640 1 -- start start 2024-09-21T03:43:04.736 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:43:04.725+0000 7f104ab29640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1044103b60 0x7f104419a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:04.736 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:43:04.725+0000 7f104ab29640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f10441094a0 0x7f104419aa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:04.736 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:43:04.725+0000 7f104ab29640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1044109df0 0x7f104419fd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:04.736 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:43:04.725+0000 7f104ab29640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f10441139b0 con 0x7f1044103b60 2024-09-21T03:43:04.736 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:43:04.725+0000 7f104ab29640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f1044113830 con 0x7f1044109df0 2024-09-21T03:43:04.736 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:43:04.725+0000 7f104ab29640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f1044113b30 con 0x7f10441094a0 2024-09-21T03:43:04.736 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:43:04.726+0000 7f10488c5640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1044103b60 0x7f104419a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:04.736 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:43:04.726+0000 7f10488c5640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1044103b60 0x7f104419a4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:39144/0 (socket says 172.21.15.89:39144) 2024-09-21T03:43:04.737 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:43:04.726+0000 7f10488c5640 1 -- 172.21.15.89:0/2795519122 learned_addr learned my addr 172.21.15.89:0/2795519122 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-21T03:43:04.737 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:43:04.726+0000 7f1043fff640 1 --2- 172.21.15.89:0/2795519122 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f10441094a0 0x7f104419aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:04.737 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:43:04.726+0000 7f10490c6640 1 --2- 172.21.15.89:0/2795519122 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1044109df0 0x7f104419fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:04.737 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:43:04.726+0000 7f10488c5640 1 -- 172.21.15.89:0/2795519122 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f10441094a0 msgr2=0x7f104419aa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:04.737 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:43:04.726+0000 7f10488c5640 1 --2- 172.21.15.89:0/2795519122 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f10441094a0 0x7f104419aa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:04.737 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:43:04.726+0000 7f10488c5640 1 -- 172.21.15.89:0/2795519122 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1044109df0 msgr2=0x7f104419fd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:04.737 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:43:04.726+0000 7f10488c5640 1 --2- 172.21.15.89:0/2795519122 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1044109df0 0x7f104419fd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:04.737 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:43:04.726+0000 7f10488c5640 1 -- 172.21.15.89:0/2795519122 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10441a0430 con 0x7f1044103b60 2024-09-21T03:43:04.737 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:43:04.726+0000 7f10488c5640 1 --2- 172.21.15.89:0/2795519122 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1044103b60 0x7f104419a4c0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f103800c970 tx=0x7f103800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:43:04.737 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:43:04.726+0000 7f1041ffb640 1 -- 172.21.15.89:0/2795519122 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1038007bb0 con 0x7f1044103b60 2024-09-21T03:43:04.737 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:43:04.726+0000 7f1041ffb640 1 -- 172.21.15.89:0/2795519122 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1038007d50 con 0x7f1044103b60 2024-09-21T03:43:04.737 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:43:04.726+0000 7f104ab29640 1 -- 172.21.15.89:0/2795519122 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f10441a0720 con 0x7f1044103b60 2024-09-21T03:43:04.737 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:43:04.726+0000 7f1041ffb640 1 -- 172.21.15.89:0/2795519122 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1038004e40 con 0x7f1044103b60 2024-09-21T03:43:04.738 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:43:04.727+0000 7f104ab29640 1 -- 172.21.15.89:0/2795519122 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10441a6e80 con 0x7f1044103b60 2024-09-21T03:43:04.738 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:43:04.727+0000 7f104ab29640 1 -- 172.21.15.89:0/2795519122 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1044103f70 con 0x7f1044103b60 2024-09-21T03:43:04.738 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:43:04.730+0000 7f1041ffb640 1 -- 172.21.15.89:0/2795519122 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 21) ==== 154512+0+0 (secure 0 0 0) 0x7f1038005000 con 0x7f1044103b60 2024-09-21T03:43:04.738 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:43:04.730+0000 7f1041ffb640 1 --2- 172.21.15.89:0/2795519122 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f10340b6510 0x7f10340b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:04.738 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:43:04.731+0000 7f1043fff640 1 --2- 172.21.15.89:0/2795519122 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f10340b6510 0x7f10340b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:04.738 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:43:04.731+0000 7f1041ffb640 1 -- 172.21.15.89:0/2795519122 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8825+0+0 (secure 0 0 0) 0x7f10380e76f0 con 0x7f1044103b60 2024-09-21T03:43:04.738 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:43:04.731+0000 7f1043fff640 1 --2- 172.21.15.89:0/2795519122 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f10340b6510 0x7f10340b8a20 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f1030005fd0 tx=0x7f1030005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:43:04.738 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:43:04.734+0000 7f1041ffb640 1 -- 172.21.15.89:0/2795519122 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f1038014030 con 0x7f1044103b60 2024-09-21T03:43:04.918 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:43:04.916+0000 7f104ab29640 1 -- 172.21.15.89:0/2795519122 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f104419b5f0 con 0x7f1044103b60 2024-09-21T03:43:05.173 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:43:05.170+0000 7f1041ffb640 1 -- 172.21.15.89:0/2795519122 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v87) ==== 169+0+0 (secure 0 0 0) 0x7f10380add30 con 0x7f1044103b60 2024-09-21T03:43:05.173 INFO:teuthology.orchestra.run.smithi089.stderr:pool 'cephfs_data' removed 2024-09-21T03:43:05.173 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:43:05 smithi089 ceph-mon[30928]: pgmap v472: 97 pgs: 97 active+clean; 577 KiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:43:05.173 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:43:05 smithi089 ceph-mon[30928]: from='client.? 172.21.15.89:0/2795519122' entity='client.admin' cmd={"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} : dispatch 2024-09-21T03:43:05.177 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:43:05.174+0000 7f104ab29640 1 -- 172.21.15.89:0/2795519122 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f10340b6510 msgr2=0x7f10340b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:05.177 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:43:05.174+0000 7f104ab29640 1 --2- 172.21.15.89:0/2795519122 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f10340b6510 0x7f10340b8a20 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f1030005fd0 tx=0x7f1030005850 comp rx=0 tx=0).stop 2024-09-21T03:43:05.177 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:43:05.174+0000 7f104ab29640 1 -- 172.21.15.89:0/2795519122 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1044103b60 msgr2=0x7f104419a4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:05.177 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:43:05.174+0000 7f104ab29640 1 --2- 172.21.15.89:0/2795519122 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1044103b60 0x7f104419a4c0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f103800c970 tx=0x7f103800ce30 comp rx=0 tx=0).stop 2024-09-21T03:43:05.177 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:43:05.174+0000 7f104ab29640 1 -- 172.21.15.89:0/2795519122 shutdown_connections 2024-09-21T03:43:05.177 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:43:05.174+0000 7f104ab29640 1 --2- 172.21.15.89:0/2795519122 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f10340b6510 0x7f10340b8a20 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:05.177 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:43:05.174+0000 7f104ab29640 1 --2- 172.21.15.89:0/2795519122 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1044109df0 0x7f104419fd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:05.178 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:43:05.174+0000 7f104ab29640 1 --2- 172.21.15.89:0/2795519122 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f10441094a0 0x7f104419aa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:05.178 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:43:05.174+0000 7f104ab29640 1 --2- 172.21.15.89:0/2795519122 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1044103b60 0x7f104419a4c0 secure :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f103800c970 tx=0x7f103800ce30 comp rx=0 tx=0).stop 2024-09-21T03:43:05.178 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:43:05.174+0000 7f104ab29640 1 -- 172.21.15.89:0/2795519122 >> 172.21.15.89:0/2795519122 conn(0x7f104407b850 msgr2=0x7f1044105610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:43:05.178 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:43:05.174+0000 7f104ab29640 1 -- 172.21.15.89:0/2795519122 shutdown_connections 2024-09-21T03:43:05.178 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:43:05.174+0000 7f104ab29640 1 -- 172.21.15.89:0/2795519122 wait complete. 2024-09-21T03:43:05.461 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:43:05 smithi114 ceph-mon[34140]: pgmap v472: 97 pgs: 97 active+clean; 577 KiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:43:05.461 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:43:05 smithi114 ceph-mon[34140]: from='client.? 172.21.15.89:0/2795519122' entity='client.admin' cmd={"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} : dispatch 2024-09-21T03:43:05.500 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:43:05 smithi110 ceph-mon[34000]: pgmap v472: 97 pgs: 97 active+clean; 577 KiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:43:05.500 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:43:05 smithi110 ceph-mon[34000]: from='client.? 172.21.15.89:0/2795519122' entity='client.admin' cmd={"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} : dispatch 2024-09-21T03:43:05.945 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 7c374098-77c9-11ef-baf5-efdc52797490 -- ceph osd pool rm cephfs_data_ec cephfs_data_ec --yes-i-really-really-mean-it 2024-09-21T03:43:06.195 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/7c374098-77c9-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:43:06.457 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:43:06 smithi089 ceph-mon[30928]: from='client.? 172.21.15.89:0/2795519122' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-09-21T03:43:06.457 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:43:06 smithi089 ceph-mon[30928]: osdmap e87: 12 total, 12 up, 12 in 2024-09-21T03:43:06.460 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:43:06 smithi114 ceph-mon[34140]: from='client.? 172.21.15.89:0/2795519122' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-09-21T03:43:06.460 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:43:06 smithi114 ceph-mon[34140]: osdmap e87: 12 total, 12 up, 12 in 2024-09-21T03:43:06.500 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:43:06 smithi110 ceph-mon[34000]: from='client.? 172.21.15.89:0/2795519122' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-09-21T03:43:06.500 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:43:06 smithi110 ceph-mon[34000]: osdmap e87: 12 total, 12 up, 12 in 2024-09-21T03:43:06.705 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:43:06.703+0000 7f0af1605640 1 -- 172.21.15.89:0/4050818224 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0aec1083d0 msgr2=0x7f0aec1087e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:06.705 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:43:06.703+0000 7f0af1605640 1 --2- 172.21.15.89:0/4050818224 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0aec1083d0 0x7f0aec1087e0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f0ae0009900 tx=0x7f0ae002a410 comp rx=0 tx=0).stop 2024-09-21T03:43:06.706 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:43:06.703+0000 7f0af1605640 1 -- 172.21.15.89:0/4050818224 shutdown_connections 2024-09-21T03:43:06.706 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:43:06.703+0000 7f0af1605640 1 --2- 172.21.15.89:0/4050818224 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0aec108d20 0x7f0aec10f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:06.706 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:43:06.703+0000 7f0af1605640 1 --2- 172.21.15.89:0/4050818224 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0aec1083d0 0x7f0aec1087e0 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:06.706 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:43:06.703+0000 7f0af1605640 1 --2- 172.21.15.89:0/4050818224 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0aec102ae0 0x7f0aec102ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:06.706 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:43:06.703+0000 7f0af1605640 1 -- 172.21.15.89:0/4050818224 >> 172.21.15.89:0/4050818224 conn(0x7f0aec079790 msgr2=0x7f0aec079ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:43:06.706 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:43:06.703+0000 7f0af1605640 1 -- 172.21.15.89:0/4050818224 shutdown_connections 2024-09-21T03:43:06.706 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:43:06.703+0000 7f0af1605640 1 -- 172.21.15.89:0/4050818224 wait complete. 2024-09-21T03:43:06.708 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:43:06.703+0000 7f0af1605640 1 Processor -- start 2024-09-21T03:43:06.708 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:43:06.704+0000 7f0af1605640 1 -- start start 2024-09-21T03:43:06.708 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:43:06.704+0000 7f0af1605640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0aec102ae0 0x7f0aec19a3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:06.708 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:43:06.704+0000 7f0af1605640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0aec1083d0 0x7f0aec19a960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:06.708 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:43:06.704+0000 7f0af1605640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0aec108d20 0x7f0aec19fc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:06.708 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:43:06.704+0000 7f0af1605640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f0aec1139b0 con 0x7f0aec102ae0 2024-09-21T03:43:06.708 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:43:06.704+0000 7f0af1605640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f0aec113830 con 0x7f0aec1083d0 2024-09-21T03:43:06.708 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:43:06.704+0000 7f0af1605640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f0aec113b30 con 0x7f0aec108d20 2024-09-21T03:43:06.708 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:43:06.704+0000 7f0aeaffd640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0aec102ae0 0x7f0aec19a3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:06.708 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:43:06.704+0000 7f0aeaffd640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0aec102ae0 0x7f0aec19a3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:39170/0 (socket says 172.21.15.89:39170) 2024-09-21T03:43:06.709 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:43:06.704+0000 7f0aeaffd640 1 -- 172.21.15.89:0/2645202535 learned_addr learned my addr 172.21.15.89:0/2645202535 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-21T03:43:06.709 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:43:06.704+0000 7f0aeb7fe640 1 --2- 172.21.15.89:0/2645202535 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0aec108d20 0x7f0aec19fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:06.709 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:43:06.704+0000 7f0aea7fc640 1 --2- 172.21.15.89:0/2645202535 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0aec1083d0 0x7f0aec19a960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:06.709 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:43:06.705+0000 7f0aeaffd640 1 -- 172.21.15.89:0/2645202535 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0aec108d20 msgr2=0x7f0aec19fc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:06.709 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:43:06.705+0000 7f0aeaffd640 1 --2- 172.21.15.89:0/2645202535 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0aec108d20 0x7f0aec19fc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:06.709 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:43:06.705+0000 7f0aeaffd640 1 -- 172.21.15.89:0/2645202535 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0aec1083d0 msgr2=0x7f0aec19a960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:06.709 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:43:06.705+0000 7f0aeaffd640 1 --2- 172.21.15.89:0/2645202535 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0aec1083d0 0x7f0aec19a960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:06.709 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:43:06.705+0000 7f0aeaffd640 1 -- 172.21.15.89:0/2645202535 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0aec1a0330 con 0x7f0aec102ae0 2024-09-21T03:43:06.709 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:43:06.705+0000 7f0aeaffd640 1 --2- 172.21.15.89:0/2645202535 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0aec102ae0 0x7f0aec19a3c0 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f0adc00db80 tx=0x7f0adc00b630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:43:06.709 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:43:06.705+0000 7f0ad3fff640 1 -- 172.21.15.89:0/2645202535 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0adc017070 con 0x7f0aec102ae0 2024-09-21T03:43:06.709 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:43:06.705+0000 7f0af1605640 1 -- 172.21.15.89:0/2645202535 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0aec1a0620 con 0x7f0aec102ae0 2024-09-21T03:43:06.709 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:43:06.705+0000 7f0ad3fff640 1 -- 172.21.15.89:0/2645202535 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0adc004020 con 0x7f0aec102ae0 2024-09-21T03:43:06.710 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:43:06.705+0000 7f0ad3fff640 1 -- 172.21.15.89:0/2645202535 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0adc00be60 con 0x7f0aec102ae0 2024-09-21T03:43:06.710 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:43:06.705+0000 7f0af1605640 1 -- 172.21.15.89:0/2645202535 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0aec1a6d80 con 0x7f0aec102ae0 2024-09-21T03:43:06.710 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:43:06.706+0000 7f0af1605640 1 -- 172.21.15.89:0/2645202535 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0aec103d10 con 0x7f0aec102ae0 2024-09-21T03:43:06.710 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:43:06.708+0000 7f0ad3fff640 1 -- 172.21.15.89:0/2645202535 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 21) ==== 154512+0+0 (secure 0 0 0) 0x7f0adc003b70 con 0x7f0aec102ae0 2024-09-21T03:43:06.711 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:43:06.709+0000 7f0ad3fff640 1 --2- 172.21.15.89:0/2645202535 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f0ad40b6590 0x7f0ad40b8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:06.711 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:43:06.709+0000 7f0aea7fc640 1 --2- 172.21.15.89:0/2645202535 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f0ad40b6590 0x7f0ad40b8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:06.712 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:43:06.709+0000 7f0ad3fff640 1 -- 172.21.15.89:0/2645202535 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8324+0+0 (secure 0 0 0) 0x7f0adc0e77a0 con 0x7f0aec102ae0 2024-09-21T03:43:06.712 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:43:06.710+0000 7f0aea7fc640 1 --2- 172.21.15.89:0/2645202535 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f0ad40b6590 0x7f0ad40b8aa0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f0ae00057c0 tx=0x7f0ae0000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:43:06.715 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:43:06.713+0000 7f0ad3fff640 1 -- 172.21.15.89:0/2645202535 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f0adc0ae050 con 0x7f0aec102ae0 2024-09-21T03:43:06.894 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:43:06.892+0000 7f0af1605640 1 -- 172.21.15.89:0/2645202535 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true} v 0) -- 0x7f0aec102ef0 con 0x7f0aec102ae0 2024-09-21T03:43:07.185 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:43:07.183+0000 7f0ad3fff640 1 -- 172.21.15.89:0/2645202535 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data_ec' removed v88) ==== 178+0+0 (secure 0 0 0) 0x7f0adc0b2430 con 0x7f0aec102ae0 2024-09-21T03:43:07.185 INFO:teuthology.orchestra.run.smithi089.stderr:pool 'cephfs_data_ec' removed 2024-09-21T03:43:07.188 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:43:07.185+0000 7f0af1605640 1 -- 172.21.15.89:0/2645202535 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f0ad40b6590 msgr2=0x7f0ad40b8aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:07.188 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:43:07.185+0000 7f0af1605640 1 --2- 172.21.15.89:0/2645202535 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f0ad40b6590 0x7f0ad40b8aa0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f0ae00057c0 tx=0x7f0ae0000bc0 comp rx=0 tx=0).stop 2024-09-21T03:43:07.188 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:43:07.185+0000 7f0af1605640 1 -- 172.21.15.89:0/2645202535 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0aec102ae0 msgr2=0x7f0aec19a3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:07.188 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:43:07.185+0000 7f0af1605640 1 --2- 172.21.15.89:0/2645202535 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0aec102ae0 0x7f0aec19a3c0 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f0adc00db80 tx=0x7f0adc00b630 comp rx=0 tx=0).stop 2024-09-21T03:43:07.188 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:43:07.185+0000 7f0af1605640 1 -- 172.21.15.89:0/2645202535 shutdown_connections 2024-09-21T03:43:07.188 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:43:07.185+0000 7f0af1605640 1 --2- 172.21.15.89:0/2645202535 >> [v2:172.21.15.89:6800/1591532358,v1:172.21.15.89:6801/1591532358] conn(0x7f0ad40b6590 0x7f0ad40b8aa0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:07.188 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:43:07.185+0000 7f0af1605640 1 --2- 172.21.15.89:0/2645202535 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0aec1083d0 0x7f0aec19a960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:07.188 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:43:07.185+0000 7f0af1605640 1 --2- 172.21.15.89:0/2645202535 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0aec108d20 0x7f0aec19fc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:07.188 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:43:07.185+0000 7f0af1605640 1 --2- 172.21.15.89:0/2645202535 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0aec102ae0 0x7f0aec19a3c0 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:07.188 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:43:07.185+0000 7f0af1605640 1 -- 172.21.15.89:0/2645202535 >> 172.21.15.89:0/2645202535 conn(0x7f0aec079790 msgr2=0x7f0aec106520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:43:07.189 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:43:07.185+0000 7f0af1605640 1 -- 172.21.15.89:0/2645202535 shutdown_connections 2024-09-21T03:43:07.189 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T03:43:07.186+0000 7f0af1605640 1 -- 172.21.15.89:0/2645202535 wait complete. 2024-09-21T03:43:07.461 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:43:07 smithi114 ceph-mon[34140]: pgmap v474: 33 pgs: 33 active+clean; 577 KiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:43:07.461 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:43:07 smithi114 ceph-mon[34140]: from='client.? 172.21.15.89:0/2645202535' entity='client.admin' cmd={"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true} : dispatch 2024-09-21T03:43:07.500 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:43:07 smithi110 ceph-mon[34000]: pgmap v474: 33 pgs: 33 active+clean; 577 KiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:43:07.500 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:43:07 smithi110 ceph-mon[34000]: from='client.? 172.21.15.89:0/2645202535' entity='client.admin' cmd={"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true} : dispatch 2024-09-21T03:43:07.561 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:43:07 smithi089 ceph-mon[30928]: pgmap v474: 33 pgs: 33 active+clean; 577 KiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:43:07.561 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:43:07 smithi089 ceph-mon[30928]: from='client.? 172.21.15.89:0/2645202535' entity='client.admin' cmd={"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true} : dispatch 2024-09-21T03:43:08.180 DEBUG:teuthology.orchestra.run.smithi089:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-21T03:43:08.206 DEBUG:teuthology.orchestra.run.smithi110:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-21T03:43:08.233 DEBUG:teuthology.orchestra.run.smithi114:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-21T03:43:08.261 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-09-21T03:43:08.261 DEBUG:teuthology.orchestra.run.smithi089:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-09-21T03:43:08.277 DEBUG:teuthology.orchestra.run.smithi110:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-09-21T03:43:08.292 DEBUG:teuthology.orchestra.run.smithi114:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-09-21T03:43:08.316 INFO:tasks.cephadm:Stopping all daemons... 2024-09-21T03:43:08.316 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2024-09-21T03:43:08.317 DEBUG:teuthology.orchestra.run.smithi089:> sudo systemctl stop ceph-7c374098-77c9-11ef-baf5-efdc52797490@mon.a 2024-09-21T03:43:08.460 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:43:08 smithi114 ceph-mon[34140]: from='client.? 172.21.15.89:0/2645202535' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]': finished 2024-09-21T03:43:08.461 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:43:08 smithi114 ceph-mon[34140]: osdmap e88: 12 total, 12 up, 12 in 2024-09-21T03:43:08.500 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:43:08 smithi110 ceph-mon[34000]: from='client.? 172.21.15.89:0/2645202535' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]': finished 2024-09-21T03:43:08.500 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:43:08 smithi110 ceph-mon[34000]: osdmap e88: 12 total, 12 up, 12 in 2024-09-21T03:43:08.561 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:43:08 smithi089 ceph-mon[30928]: from='client.? 172.21.15.89:0/2645202535' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]': finished 2024-09-21T03:43:08.561 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:43:08 smithi089 ceph-mon[30928]: osdmap e88: 12 total, 12 up, 12 in 2024-09-21T03:43:08.562 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:43:08 smithi089 systemd[1]: Stopping Ceph mon.a for 7c374098-77c9-11ef-baf5-efdc52797490... 2024-09-21T03:43:09.050 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:43:08 smithi089 ceph-7c374098-77c9-11ef-baf5-efdc52797490-mon-a[30924]: 2024-09-21T03:43:08.621+0000 7fa86e757640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T03:43:09.050 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:43:08 smithi089 ceph-7c374098-77c9-11ef-baf5-efdc52797490-mon-a[30924]: 2024-09-21T03:43:08.621+0000 7fa86e757640 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2024-09-21T03:43:09.051 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:43:08 smithi089 podman[91373]: 2024-09-21 03:43:08.653110172 +0000 UTC m=+0.146808536 container died 768910a1020e482627b41daee6c7c4475f062377a34e6dbfd751444fd47ba7a1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410, name=ceph-7c374098-77c9-11ef-baf5-efdc52797490-mon-a, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-09-21T03:43:09.311 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:43:09 smithi089 podman[91373]: 2024-09-21 03:43:09.240695375 +0000 UTC m=+0.734393773 container cleanup 768910a1020e482627b41daee6c7c4475f062377a34e6dbfd751444fd47ba7a1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410, name=ceph-7c374098-77c9-11ef-baf5-efdc52797490-mon-a, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-09-21T03:43:09.311 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 21 03:43:09 smithi089 bash[91373]: ceph-7c374098-77c9-11ef-baf5-efdc52797490-mon-a 2024-09-21T03:43:09.666 DEBUG:teuthology.orchestra.run.smithi089:> sudo pkill -f 'journalctl -f -n 0 -u ceph-7c374098-77c9-11ef-baf5-efdc52797490@mon.a.service' 2024-09-21T03:43:09.715 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T03:43:09.715 INFO:tasks.cephadm.mon.a:Stopped mon.a 2024-09-21T03:43:09.715 INFO:tasks.cephadm.mon.c:Stopping mon.b... 2024-09-21T03:43:09.715 DEBUG:teuthology.orchestra.run.smithi110:> sudo systemctl stop ceph-7c374098-77c9-11ef-baf5-efdc52797490@mon.b 2024-09-21T03:43:10.000 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:43:09 smithi110 systemd[1]: Stopping Ceph mon.b for 7c374098-77c9-11ef-baf5-efdc52797490... 2024-09-21T03:43:10.500 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:43:10 smithi110 ceph-7c374098-77c9-11ef-baf5-efdc52797490-mon-b[33996]: 2024-09-21T03:43:10.031+0000 7f4206491640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T03:43:10.500 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:43:10 smithi110 ceph-7c374098-77c9-11ef-baf5-efdc52797490-mon-b[33996]: 2024-09-21T03:43:10.031+0000 7f4206491640 -1 mon.b@2(peon) e3 *** Got Signal Terminated *** 2024-09-21T03:43:10.500 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:43:10 smithi110 podman[64473]: 2024-09-21 03:43:10.235930046 +0000 UTC m=+0.330184044 container died 7e32224c6f9aa4dc9fd73c8be4c596132725591097fee7942c7cfd7aeafd46a7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-7c374098-77c9-11ef-baf5-efdc52797490-mon-b, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, ceph=True, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731) 2024-09-21T03:43:10.893 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:43:10 smithi110 podman[64473]: 2024-09-21 03:43:10.640299695 +0000 UTC m=+0.734553683 container cleanup 7e32224c6f9aa4dc9fd73c8be4c596132725591097fee7942c7cfd7aeafd46a7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-7c374098-77c9-11ef-baf5-efdc52797490-mon-b, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2) 2024-09-21T03:43:10.893 INFO:journalctl@ceph.mon.b.smithi110.stdout:Sep 21 03:43:10 smithi110 bash[64473]: ceph-7c374098-77c9-11ef-baf5-efdc52797490-mon-b 2024-09-21T03:43:11.065 DEBUG:teuthology.orchestra.run.smithi110:> sudo pkill -f 'journalctl -f -n 0 -u ceph-7c374098-77c9-11ef-baf5-efdc52797490@mon.b.service' 2024-09-21T03:43:11.108 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T03:43:11.108 INFO:tasks.cephadm.mon.c:Stopped mon.b 2024-09-21T03:43:11.109 INFO:tasks.cephadm.mon.c:Stopping mon.c... 2024-09-21T03:43:11.109 DEBUG:teuthology.orchestra.run.smithi114:> sudo systemctl stop ceph-7c374098-77c9-11ef-baf5-efdc52797490@mon.c 2024-09-21T03:43:11.423 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:43:11 smithi114 systemd[1]: Stopping Ceph mon.c for 7c374098-77c9-11ef-baf5-efdc52797490... 2024-09-21T03:43:11.711 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:43:11 smithi114 ceph-7c374098-77c9-11ef-baf5-efdc52797490-mon-c[34136]: 2024-09-21T03:43:11.420+0000 7fe8a74df640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T03:43:11.711 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:43:11 smithi114 ceph-7c374098-77c9-11ef-baf5-efdc52797490-mon-c[34136]: 2024-09-21T03:43:11.420+0000 7fe8a74df640 -1 mon.c@1(peon) e3 *** Got Signal Terminated *** 2024-09-21T03:43:11.711 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:43:11 smithi114 podman[65694]: 2024-09-21 03:43:11.574809209 +0000 UTC m=+0.271157188 container died 1205e7833f22c32e2ce7f554d863b6bc7df66e2260cb028062b7b1edd85e0abb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-7c374098-77c9-11ef-baf5-efdc52797490-mon-c, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.build-date=20240731, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-21T03:43:12.211 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:43:11 smithi114 podman[65694]: 2024-09-21 03:43:11.962822835 +0000 UTC m=+0.659170805 container cleanup 1205e7833f22c32e2ce7f554d863b6bc7df66e2260cb028062b7b1edd85e0abb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-7c374098-77c9-11ef-baf5-efdc52797490-mon-c, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-21T03:43:12.211 INFO:journalctl@ceph.mon.c.smithi114.stdout:Sep 21 03:43:11 smithi114 bash[65694]: ceph-7c374098-77c9-11ef-baf5-efdc52797490-mon-c 2024-09-21T03:43:12.402 DEBUG:teuthology.orchestra.run.smithi114:> sudo pkill -f 'journalctl -f -n 0 -u ceph-7c374098-77c9-11ef-baf5-efdc52797490@mon.c.service' 2024-09-21T03:43:12.444 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T03:43:12.445 INFO:tasks.cephadm.mon.c:Stopped mon.c 2024-09-21T03:43:12.445 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2024-09-21T03:43:12.445 DEBUG:teuthology.orchestra.run.smithi089:> sudo systemctl stop ceph-7c374098-77c9-11ef-baf5-efdc52797490@mgr.x 2024-09-21T03:43:12.747 INFO:journalctl@ceph.mgr.x.smithi089.stdout:Sep 21 03:43:12 smithi089 systemd[1]: Stopping Ceph mgr.x for 7c374098-77c9-11ef-baf5-efdc52797490... 2024-09-21T03:43:13.057 INFO:journalctl@ceph.mgr.x.smithi089.stdout:Sep 21 03:43:12 smithi089 ceph-7c374098-77c9-11ef-baf5-efdc52797490-mgr-x[31154]: 2024-09-21T03:43:12.744+0000 7f553e9d3640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mgr -n mgr.x -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T03:43:13.057 INFO:journalctl@ceph.mgr.x.smithi089.stdout:Sep 21 03:43:12 smithi089 podman[91494]: 2024-09-21 03:43:12.776553719 +0000 UTC m=+0.142286801 container died 5ac528282a8ca47b9972574f752ae5a3dd4a05fe49fe281c9cece957a5e6a224 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410, name=ceph-7c374098-77c9-11ef-baf5-efdc52797490-mgr-x, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0) 2024-09-21T03:43:13.311 INFO:journalctl@ceph.mgr.x.smithi089.stdout:Sep 21 03:43:13 smithi089 podman[91494]: 2024-09-21 03:43:13.264553899 +0000 UTC m=+0.630286968 container cleanup 5ac528282a8ca47b9972574f752ae5a3dd4a05fe49fe281c9cece957a5e6a224 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410, name=ceph-7c374098-77c9-11ef-baf5-efdc52797490-mgr-x, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2) 2024-09-21T03:43:13.311 INFO:journalctl@ceph.mgr.x.smithi089.stdout:Sep 21 03:43:13 smithi089 bash[91494]: ceph-7c374098-77c9-11ef-baf5-efdc52797490-mgr-x 2024-09-21T03:43:13.650 DEBUG:teuthology.orchestra.run.smithi089:> sudo pkill -f 'journalctl -f -n 0 -u ceph-7c374098-77c9-11ef-baf5-efdc52797490@mgr.x.service' 2024-09-21T03:43:13.685 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T03:43:13.685 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2024-09-21T03:43:13.686 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2024-09-21T03:43:13.686 DEBUG:teuthology.orchestra.run.smithi110:> sudo systemctl stop ceph-7c374098-77c9-11ef-baf5-efdc52797490@mgr.y 2024-09-21T03:43:14.000 INFO:journalctl@ceph.mgr.y.smithi110.stdout:Sep 21 03:43:13 smithi110 systemd[1]: Stopping Ceph mgr.y for 7c374098-77c9-11ef-baf5-efdc52797490... 2024-09-21T03:43:14.281 INFO:journalctl@ceph.mgr.y.smithi110.stdout:Sep 21 03:43:14 smithi110 podman[64597]: 2024-09-21 03:43:13.999847254 +0000 UTC m=+0.121972431 container died a8ac88b4b1dda616f9894d41e6bf8828602dff61d040e5b9fe38f3b9e38dbfb6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-7c374098-77c9-11ef-baf5-efdc52797490-mgr-y, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-09-21T03:43:14.665 INFO:journalctl@ceph.mgr.y.smithi110.stdout:Sep 21 03:43:14 smithi110 podman[64597]: 2024-09-21 03:43:14.362779536 +0000 UTC m=+0.484904715 container cleanup a8ac88b4b1dda616f9894d41e6bf8828602dff61d040e5b9fe38f3b9e38dbfb6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-7c374098-77c9-11ef-baf5-efdc52797490-mgr-y, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, ceph=True, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-21T03:43:14.665 INFO:journalctl@ceph.mgr.y.smithi110.stdout:Sep 21 03:43:14 smithi110 bash[64597]: ceph-7c374098-77c9-11ef-baf5-efdc52797490-mgr-y 2024-09-21T03:43:14.803 DEBUG:teuthology.orchestra.run.smithi110:> sudo pkill -f 'journalctl -f -n 0 -u ceph-7c374098-77c9-11ef-baf5-efdc52797490@mgr.y.service' 2024-09-21T03:43:14.838 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T03:43:14.838 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2024-09-21T03:43:14.838 INFO:tasks.cephadm.mgr.z:Stopping mgr.z... 2024-09-21T03:43:14.839 DEBUG:teuthology.orchestra.run.smithi114:> sudo systemctl stop ceph-7c374098-77c9-11ef-baf5-efdc52797490@mgr.z 2024-09-21T03:43:15.173 INFO:journalctl@ceph.mgr.z.smithi114.stdout:Sep 21 03:43:14 smithi114 systemd[1]: Stopping Ceph mgr.z for 7c374098-77c9-11ef-baf5-efdc52797490... 2024-09-21T03:43:15.461 INFO:journalctl@ceph.mgr.z.smithi114.stdout:Sep 21 03:43:15 smithi114 podman[65820]: 2024-09-21 03:43:15.171992566 +0000 UTC m=+0.138442733 container died 78bce9af8cd41306077d6b35dde07ede99d5cec0b61ff16efe9837b909d0391b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-7c374098-77c9-11ef-baf5-efdc52797490-mgr-z, io.buildah.version=1.37.2, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-21T03:43:15.847 INFO:journalctl@ceph.mgr.z.smithi114.stdout:Sep 21 03:43:15 smithi114 podman[65820]: 2024-09-21 03:43:15.560362116 +0000 UTC m=+0.526812276 container cleanup 78bce9af8cd41306077d6b35dde07ede99d5cec0b61ff16efe9837b909d0391b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-7c374098-77c9-11ef-baf5-efdc52797490-mgr-z, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, ceph=True, org.label-schema.license=GPLv2) 2024-09-21T03:43:15.848 INFO:journalctl@ceph.mgr.z.smithi114.stdout:Sep 21 03:43:15 smithi114 bash[65820]: ceph-7c374098-77c9-11ef-baf5-efdc52797490-mgr-z 2024-09-21T03:43:15.992 DEBUG:teuthology.orchestra.run.smithi114:> sudo pkill -f 'journalctl -f -n 0 -u ceph-7c374098-77c9-11ef-baf5-efdc52797490@mgr.z.service' 2024-09-21T03:43:16.026 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T03:43:16.026 INFO:tasks.cephadm.mgr.z:Stopped mgr.z 2024-09-21T03:43:16.026 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-09-21T03:43:16.026 DEBUG:teuthology.orchestra.run.smithi089:> sudo systemctl stop ceph-7c374098-77c9-11ef-baf5-efdc52797490@osd.0 2024-09-21T03:43:16.311 INFO:journalctl@ceph.osd.0.smithi089.stdout:Sep 21 03:43:16 smithi089 systemd[1]: Stopping Ceph osd.0 for 7c374098-77c9-11ef-baf5-efdc52797490... 2024-09-21T03:43:16.811 INFO:journalctl@ceph.osd.0.smithi089.stdout:Sep 21 03:43:16 smithi089 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-0[46605]: 2024-09-21T03:43:16.338+0000 7f543862a640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T03:43:16.811 INFO:journalctl@ceph.osd.0.smithi089.stdout:Sep 21 03:43:16 smithi089 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-0[46605]: 2024-09-21T03:43:16.338+0000 7f543862a640 -1 osd.0 88 *** Got signal Terminated *** 2024-09-21T03:43:16.811 INFO:journalctl@ceph.osd.0.smithi089.stdout:Sep 21 03:43:16 smithi089 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-0[46605]: 2024-09-21T03:43:16.338+0000 7f543862a640 -1 osd.0 88 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-21T03:43:22.106 INFO:journalctl@ceph.osd.0.smithi089.stdout:Sep 21 03:43:21 smithi089 podman[91616]: 2024-09-21 03:43:21.82242964 +0000 UTC m=+5.605058119 container died 0b2ede32cfc70d8c717fc2bb713490c13e42ef71bbda53d40edc36eeba6a8f61 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, ceph=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-09-21T03:43:22.561 INFO:journalctl@ceph.osd.0.smithi089.stdout:Sep 21 03:43:22 smithi089 podman[91616]: 2024-09-21 03:43:22.279863325 +0000 UTC m=+6.062491799 container cleanup 0b2ede32cfc70d8c717fc2bb713490c13e42ef71bbda53d40edc36eeba6a8f61 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS) 2024-09-21T03:43:22.561 INFO:journalctl@ceph.osd.0.smithi089.stdout:Sep 21 03:43:22 smithi089 bash[91616]: ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-0 2024-09-21T03:43:22.843 INFO:journalctl@ceph.osd.0.smithi089.stdout:Sep 21 03:43:22 smithi089 podman[91633]: 2024-09-21 03:43:22.56490498 +0000 UTC m=+0.743750814 container remove 0b2ede32cfc70d8c717fc2bb713490c13e42ef71bbda53d40edc36eeba6a8f61 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-0, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221) 2024-09-21T03:43:23.153 INFO:journalctl@ceph.osd.0.smithi089.stdout:Sep 21 03:43:22 smithi089 podman[91721]: 2024-09-21 03:43:22.797099616 +0000 UTC m=+0.015531921 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T03:43:23.153 INFO:journalctl@ceph.osd.0.smithi089.stdout:Sep 21 03:43:22 smithi089 podman[91721]: 2024-09-21 03:43:22.924681817 +0000 UTC m=+0.143114181 container create bac18953d983ad6e76a876af28e88c8f151d5dc29f60105770a6363a1703eeee (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-0-deactivate, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=) 2024-09-21T03:43:23.154 INFO:journalctl@ceph.osd.0.smithi089.stdout:Sep 21 03:43:23 smithi089 podman[91721]: 2024-09-21 03:43:23.152400177 +0000 UTC m=+0.370832480 container init bac18953d983ad6e76a876af28e88c8f151d5dc29f60105770a6363a1703eeee (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-0-deactivate, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221) 2024-09-21T03:43:23.562 INFO:journalctl@ceph.osd.0.smithi089.stdout:Sep 21 03:43:23 smithi089 podman[91721]: 2024-09-21 03:43:23.157291629 +0000 UTC m=+0.375723931 container start bac18953d983ad6e76a876af28e88c8f151d5dc29f60105770a6363a1703eeee (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-0-deactivate, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-21T03:43:23.562 INFO:journalctl@ceph.osd.0.smithi089.stdout:Sep 21 03:43:23 smithi089 podman[91721]: 2024-09-21 03:43:23.26108992 +0000 UTC m=+0.479522238 container attach bac18953d983ad6e76a876af28e88c8f151d5dc29f60105770a6363a1703eeee (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-0-deactivate, ceph=True, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-21T03:43:23.562 INFO:journalctl@ceph.osd.0.smithi089.stdout:Sep 21 03:43:23 smithi089 podman[91721]: 2024-09-21 03:43:23.340687139 +0000 UTC m=+0.559119444 container died bac18953d983ad6e76a876af28e88c8f151d5dc29f60105770a6363a1703eeee (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-0-deactivate, org.label-schema.schema-version=1.0, ceph=True, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-21T03:43:24.011 DEBUG:teuthology.orchestra.run.smithi089:> sudo pkill -f 'journalctl -f -n 0 -u ceph-7c374098-77c9-11ef-baf5-efdc52797490@osd.0.service' 2024-09-21T03:43:24.052 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T03:43:24.052 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-09-21T03:43:24.052 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-09-21T03:43:24.053 DEBUG:teuthology.orchestra.run.smithi110:> sudo systemctl stop ceph-7c374098-77c9-11ef-baf5-efdc52797490@osd.1 2024-09-21T03:43:24.369 INFO:journalctl@ceph.osd.1.smithi110.stdout:Sep 21 03:43:24 smithi110 systemd[1]: Stopping Ceph osd.1 for 7c374098-77c9-11ef-baf5-efdc52797490... 2024-09-21T03:43:24.750 INFO:journalctl@ceph.osd.1.smithi110.stdout:Sep 21 03:43:24 smithi110 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-1[37931]: 2024-09-21T03:43:24.366+0000 7f1f6a62e640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T03:43:24.750 INFO:journalctl@ceph.osd.1.smithi110.stdout:Sep 21 03:43:24 smithi110 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-1[37931]: 2024-09-21T03:43:24.366+0000 7f1f6a62e640 -1 osd.1 88 *** Got signal Terminated *** 2024-09-21T03:43:24.750 INFO:journalctl@ceph.osd.1.smithi110.stdout:Sep 21 03:43:24 smithi110 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-1[37931]: 2024-09-21T03:43:24.366+0000 7f1f6a62e640 -1 osd.1 88 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-21T03:43:30.162 INFO:journalctl@ceph.osd.1.smithi110.stdout:Sep 21 03:43:29 smithi110 podman[64721]: 2024-09-21 03:43:29.86579934 +0000 UTC m=+5.621061995 container died 63ad62231ff261c2e1b1e2d4311818adda841afa0ee010f14a11b53b71c4e990 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-1, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-21T03:43:30.500 INFO:journalctl@ceph.osd.1.smithi110.stdout:Sep 21 03:43:30 smithi110 podman[64721]: 2024-09-21 03:43:30.301392035 +0000 UTC m=+6.056654682 container cleanup 63ad62231ff261c2e1b1e2d4311818adda841afa0ee010f14a11b53b71c4e990 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-1, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , ceph=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, io.buildah.version=1.37.2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD) 2024-09-21T03:43:30.500 INFO:journalctl@ceph.osd.1.smithi110.stdout:Sep 21 03:43:30 smithi110 bash[64721]: ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-1 2024-09-21T03:43:30.958 INFO:journalctl@ceph.osd.1.smithi110.stdout:Sep 21 03:43:30 smithi110 podman[64737]: 2024-09-21 03:43:30.519691512 +0000 UTC m=+0.653811813 container remove 63ad62231ff261c2e1b1e2d4311818adda841afa0ee010f14a11b53b71c4e990 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-1, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=) 2024-09-21T03:43:31.233 INFO:journalctl@ceph.osd.1.smithi110.stdout:Sep 21 03:43:30 smithi110 podman[64824]: 2024-09-21 03:43:30.860258855 +0000 UTC m=+0.015553910 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T03:43:31.233 INFO:journalctl@ceph.osd.1.smithi110.stdout:Sep 21 03:43:30 smithi110 podman[64824]: 2024-09-21 03:43:30.970710822 +0000 UTC m=+0.126005880 container create 44149e29d02350ddd6ad24a5bd5aac72ca7ca7030c8293fe6cd22222646725ee (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-1-deactivate, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, ceph=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-21T03:43:31.500 INFO:journalctl@ceph.osd.1.smithi110.stdout:Sep 21 03:43:31 smithi110 podman[64824]: 2024-09-21 03:43:31.231714612 +0000 UTC m=+0.387009678 container init 44149e29d02350ddd6ad24a5bd5aac72ca7ca7030c8293fe6cd22222646725ee (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-1-deactivate, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True, GIT_CLEAN=True, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-21T03:43:31.500 INFO:journalctl@ceph.osd.1.smithi110.stdout:Sep 21 03:43:31 smithi110 podman[64824]: 2024-09-21 03:43:31.23522763 +0000 UTC m=+0.390522694 container start 44149e29d02350ddd6ad24a5bd5aac72ca7ca7030c8293fe6cd22222646725ee (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-1-deactivate, ceph=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731) 2024-09-21T03:43:31.500 INFO:journalctl@ceph.osd.1.smithi110.stdout:Sep 21 03:43:31 smithi110 podman[64824]: 2024-09-21 03:43:31.323472101 +0000 UTC m=+0.478767165 container attach 44149e29d02350ddd6ad24a5bd5aac72ca7ca7030c8293fe6cd22222646725ee (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-1-deactivate, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-09-21T03:43:31.501 INFO:journalctl@ceph.osd.1.smithi110.stdout:Sep 21 03:43:31 smithi110 podman[64824]: 2024-09-21 03:43:31.424773028 +0000 UTC m=+0.580068084 container died 44149e29d02350ddd6ad24a5bd5aac72ca7ca7030c8293fe6cd22222646725ee (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-1-deactivate, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_BRANCH=HEAD) 2024-09-21T03:43:32.041 DEBUG:teuthology.orchestra.run.smithi110:> sudo pkill -f 'journalctl -f -n 0 -u ceph-7c374098-77c9-11ef-baf5-efdc52797490@osd.1.service' 2024-09-21T03:43:32.081 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T03:43:32.081 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-09-21T03:43:32.081 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-09-21T03:43:32.081 DEBUG:teuthology.orchestra.run.smithi114:> sudo systemctl stop ceph-7c374098-77c9-11ef-baf5-efdc52797490@osd.2 2024-09-21T03:43:32.399 INFO:journalctl@ceph.osd.2.smithi114.stdout:Sep 21 03:43:32 smithi114 systemd[1]: Stopping Ceph osd.2 for 7c374098-77c9-11ef-baf5-efdc52797490... 2024-09-21T03:43:32.711 INFO:journalctl@ceph.osd.2.smithi114.stdout:Sep 21 03:43:32 smithi114 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-2[38568]: 2024-09-21T03:43:32.396+0000 7fd185158640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T03:43:32.711 INFO:journalctl@ceph.osd.2.smithi114.stdout:Sep 21 03:43:32 smithi114 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-2[38568]: 2024-09-21T03:43:32.396+0000 7fd185158640 -1 osd.2 88 *** Got signal Terminated *** 2024-09-21T03:43:32.711 INFO:journalctl@ceph.osd.2.smithi114.stdout:Sep 21 03:43:32 smithi114 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-2[38568]: 2024-09-21T03:43:32.396+0000 7fd185158640 -1 osd.2 88 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-21T03:43:38.211 INFO:journalctl@ceph.osd.2.smithi114.stdout:Sep 21 03:43:37 smithi114 podman[65948]: 2024-09-21 03:43:37.910530444 +0000 UTC m=+5.635102225 container died 3edce8a43c5b38bb44aff1a4af0dabda0244bf6b41ec90c3c9588b5766b13f55 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-2, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221) 2024-09-21T03:43:38.568 INFO:journalctl@ceph.osd.2.smithi114.stdout:Sep 21 03:43:38 smithi114 podman[65948]: 2024-09-21 03:43:38.348318196 +0000 UTC m=+6.072889982 container cleanup 3edce8a43c5b38bb44aff1a4af0dabda0244bf6b41ec90c3c9588b5766b13f55 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-2, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD) 2024-09-21T03:43:38.568 INFO:journalctl@ceph.osd.2.smithi114.stdout:Sep 21 03:43:38 smithi114 bash[65948]: ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-2 2024-09-21T03:43:38.961 INFO:journalctl@ceph.osd.2.smithi114.stdout:Sep 21 03:43:38 smithi114 podman[65963]: 2024-09-21 03:43:38.566532866 +0000 UTC m=+0.652903738 container remove 3edce8a43c5b38bb44aff1a4af0dabda0244bf6b41ec90c3c9588b5766b13f55 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, ceph=True) 2024-09-21T03:43:39.262 INFO:journalctl@ceph.osd.2.smithi114.stdout:Sep 21 03:43:39 smithi114 podman[66060]: 2024-09-21 03:43:39.000702306 +0000 UTC m=+0.103663700 container create a3b17b3907701836474221b2096b0f04b6fb20b07ae6b747bccbdb17559a1d70 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-2-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.license=GPLv2) 2024-09-21T03:43:39.263 INFO:journalctl@ceph.osd.2.smithi114.stdout:Sep 21 03:43:39 smithi114 podman[66060]: 2024-09-21 03:43:38.914558803 +0000 UTC m=+0.017520183 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T03:43:39.711 INFO:journalctl@ceph.osd.2.smithi114.stdout:Sep 21 03:43:39 smithi114 podman[66060]: 2024-09-21 03:43:39.261599341 +0000 UTC m=+0.364560722 container init a3b17b3907701836474221b2096b0f04b6fb20b07ae6b747bccbdb17559a1d70 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-2-deactivate, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.vendor=CentOS) 2024-09-21T03:43:39.711 INFO:journalctl@ceph.osd.2.smithi114.stdout:Sep 21 03:43:39 smithi114 podman[66060]: 2024-09-21 03:43:39.265259452 +0000 UTC m=+0.368220841 container start a3b17b3907701836474221b2096b0f04b6fb20b07ae6b747bccbdb17559a1d70 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-2-deactivate, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0) 2024-09-21T03:43:39.712 INFO:journalctl@ceph.osd.2.smithi114.stdout:Sep 21 03:43:39 smithi114 podman[66060]: 2024-09-21 03:43:39.361812561 +0000 UTC m=+0.464773951 container attach a3b17b3907701836474221b2096b0f04b6fb20b07ae6b747bccbdb17559a1d70 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-2-deactivate, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_CLEAN=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731) 2024-09-21T03:43:39.712 INFO:journalctl@ceph.osd.2.smithi114.stdout:Sep 21 03:43:39 smithi114 podman[66060]: 2024-09-21 03:43:39.4604516 +0000 UTC m=+0.563412981 container died a3b17b3907701836474221b2096b0f04b6fb20b07ae6b747bccbdb17559a1d70 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-2-deactivate, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-21T03:43:40.086 DEBUG:teuthology.orchestra.run.smithi114:> sudo pkill -f 'journalctl -f -n 0 -u ceph-7c374098-77c9-11ef-baf5-efdc52797490@osd.2.service' 2024-09-21T03:43:40.125 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T03:43:40.126 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-09-21T03:43:40.126 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-09-21T03:43:40.126 DEBUG:teuthology.orchestra.run.smithi089:> sudo systemctl stop ceph-7c374098-77c9-11ef-baf5-efdc52797490@osd.3 2024-09-21T03:43:40.438 INFO:journalctl@ceph.osd.3.smithi089.stdout:Sep 21 03:43:40 smithi089 systemd[1]: Stopping Ceph osd.3 for 7c374098-77c9-11ef-baf5-efdc52797490... 2024-09-21T03:43:40.811 INFO:journalctl@ceph.osd.3.smithi089.stdout:Sep 21 03:43:40 smithi089 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-3[52765]: 2024-09-21T03:43:40.435+0000 7fef24f37640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T03:43:40.811 INFO:journalctl@ceph.osd.3.smithi089.stdout:Sep 21 03:43:40 smithi089 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-3[52765]: 2024-09-21T03:43:40.435+0000 7fef24f37640 -1 osd.3 88 *** Got signal Terminated *** 2024-09-21T03:43:40.811 INFO:journalctl@ceph.osd.3.smithi089.stdout:Sep 21 03:43:40 smithi089 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-3[52765]: 2024-09-21T03:43:40.435+0000 7fef24f37640 -1 osd.3 88 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-21T03:43:44.061 INFO:journalctl@ceph.osd.3.smithi089.stdout:Sep 21 03:43:43 smithi089 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-3[52765]: 2024-09-21T03:43:43.605+0000 7fef21522640 -1 osd.3 88 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2024-09-21T03:43:16.998981+0000 front 2024-09-21T03:43:16.999010+0000 (oldest deadline 2024-09-21T03:43:42.899244+0000) 2024-09-21T03:43:44.750 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 21 03:43:44 smithi110 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-10[57089]: 2024-09-21T03:43:44.374+0000 7f9d479dc640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2024-09-21T03:43:21.264189+0000 front 2024-09-21T03:43:21.264118+0000 (oldest deadline 2024-09-21T03:43:43.564295+0000) 2024-09-21T03:43:45.061 INFO:journalctl@ceph.osd.3.smithi089.stdout:Sep 21 03:43:44 smithi089 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-3[52765]: 2024-09-21T03:43:44.634+0000 7fef21522640 -1 osd.3 88 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2024-09-21T03:43:16.998981+0000 front 2024-09-21T03:43:16.999010+0000 (oldest deadline 2024-09-21T03:43:42.899244+0000) 2024-09-21T03:43:45.750 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 21 03:43:45 smithi110 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-10[57089]: 2024-09-21T03:43:45.412+0000 7f9d479dc640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2024-09-21T03:43:21.264189+0000 front 2024-09-21T03:43:21.264118+0000 (oldest deadline 2024-09-21T03:43:43.564295+0000) 2024-09-21T03:43:45.811 INFO:journalctl@ceph.osd.9.smithi089.stdout:Sep 21 03:43:45 smithi089 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-9[65404]: 2024-09-21T03:43:45.330+0000 7f6c4e7c5640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2024-09-21T03:43:18.845419+0000 front 2024-09-21T03:43:18.845455+0000 (oldest deadline 2024-09-21T03:43:44.745556+0000) 2024-09-21T03:43:46.272 INFO:journalctl@ceph.osd.3.smithi089.stdout:Sep 21 03:43:45 smithi089 podman[91837]: 2024-09-21 03:43:45.970852306 +0000 UTC m=+5.645508053 container died ebca35608a2eb806f5b9d6ae596f444deeb0784d60f85158fc6e21e3c4f448b3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-3, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.license=GPLv2) 2024-09-21T03:43:46.561 INFO:journalctl@ceph.osd.3.smithi089.stdout:Sep 21 03:43:46 smithi089 podman[91837]: 2024-09-21 03:43:46.461608527 +0000 UTC m=+6.136264274 container cleanup ebca35608a2eb806f5b9d6ae596f444deeb0784d60f85158fc6e21e3c4f448b3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-3, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, ceph=True, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-21T03:43:46.561 INFO:journalctl@ceph.osd.3.smithi089.stdout:Sep 21 03:43:46 smithi089 bash[91837]: ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-3 2024-09-21T03:43:46.561 INFO:journalctl@ceph.osd.9.smithi089.stdout:Sep 21 03:43:46 smithi089 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-9[65404]: 2024-09-21T03:43:46.350+0000 7f6c4e7c5640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2024-09-21T03:43:18.845419+0000 front 2024-09-21T03:43:18.845455+0000 (oldest deadline 2024-09-21T03:43:44.745556+0000) 2024-09-21T03:43:46.750 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 21 03:43:46 smithi110 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-10[57089]: 2024-09-21T03:43:46.377+0000 7f9d479dc640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2024-09-21T03:43:21.264189+0000 front 2024-09-21T03:43:21.264118+0000 (oldest deadline 2024-09-21T03:43:43.564295+0000) 2024-09-21T03:43:47.041 INFO:journalctl@ceph.osd.3.smithi089.stdout:Sep 21 03:43:46 smithi089 podman[91853]: 2024-09-21 03:43:46.730145727 +0000 UTC m=+0.753043412 container remove ebca35608a2eb806f5b9d6ae596f444deeb0784d60f85158fc6e21e3c4f448b3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-3, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=) 2024-09-21T03:43:47.311 INFO:journalctl@ceph.osd.3.smithi089.stdout:Sep 21 03:43:47 smithi089 podman[91940]: 2024-09-21 03:43:46.942238312 +0000 UTC m=+0.015795521 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T03:43:47.311 INFO:journalctl@ceph.osd.3.smithi089.stdout:Sep 21 03:43:47 smithi089 podman[91940]: 2024-09-21 03:43:47.039478177 +0000 UTC m=+0.113035414 container create 5ead5f320fdfdc9343315342686602fe4e1e2542d1a856090b422a8af9cc4dc5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-3-deactivate, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=) 2024-09-21T03:43:47.311 INFO:journalctl@ceph.osd.3.smithi089.stdout:Sep 21 03:43:47 smithi089 podman[91940]: 2024-09-21 03:43:47.27565331 +0000 UTC m=+0.349210501 container init 5ead5f320fdfdc9343315342686602fe4e1e2542d1a856090b422a8af9cc4dc5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-3-deactivate, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, ceph=True) 2024-09-21T03:43:47.312 INFO:journalctl@ceph.osd.3.smithi089.stdout:Sep 21 03:43:47 smithi089 podman[91940]: 2024-09-21 03:43:47.279049414 +0000 UTC m=+0.352606607 container start 5ead5f320fdfdc9343315342686602fe4e1e2542d1a856090b422a8af9cc4dc5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-3-deactivate, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731) 2024-09-21T03:43:47.750 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 21 03:43:47 smithi110 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-10[57089]: 2024-09-21T03:43:47.329+0000 7f9d479dc640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2024-09-21T03:43:21.264189+0000 front 2024-09-21T03:43:21.264118+0000 (oldest deadline 2024-09-21T03:43:43.564295+0000) 2024-09-21T03:43:47.811 INFO:journalctl@ceph.osd.9.smithi089.stdout:Sep 21 03:43:47 smithi089 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-9[65404]: 2024-09-21T03:43:47.385+0000 7f6c4e7c5640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2024-09-21T03:43:18.845419+0000 front 2024-09-21T03:43:18.845455+0000 (oldest deadline 2024-09-21T03:43:44.745556+0000) 2024-09-21T03:43:47.811 INFO:journalctl@ceph.osd.3.smithi089.stdout:Sep 21 03:43:47 smithi089 podman[91940]: 2024-09-21 03:43:47.367768416 +0000 UTC m=+0.441325617 container attach 5ead5f320fdfdc9343315342686602fe4e1e2542d1a856090b422a8af9cc4dc5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-3-deactivate, GIT_CLEAN=True, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-21T03:43:47.811 INFO:journalctl@ceph.osd.3.smithi089.stdout:Sep 21 03:43:47 smithi089 podman[91940]: 2024-09-21 03:43:47.471647714 +0000 UTC m=+0.545204914 container died 5ead5f320fdfdc9343315342686602fe4e1e2542d1a856090b422a8af9cc4dc5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-3-deactivate, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_CLEAN=True, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=) 2024-09-21T03:43:48.151 DEBUG:teuthology.orchestra.run.smithi089:> sudo pkill -f 'journalctl -f -n 0 -u ceph-7c374098-77c9-11ef-baf5-efdc52797490@osd.3.service' 2024-09-21T03:43:48.190 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T03:43:48.190 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-09-21T03:43:48.190 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-09-21T03:43:48.190 DEBUG:teuthology.orchestra.run.smithi110:> sudo systemctl stop ceph-7c374098-77c9-11ef-baf5-efdc52797490@osd.4 2024-09-21T03:43:48.461 INFO:journalctl@ceph.osd.11.smithi114.stdout:Sep 21 03:43:48 smithi114 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-11[58313]: 2024-09-21T03:43:48.157+0000 7f7657ce9640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2024-09-21T03:43:19.721069+0000 front 2024-09-21T03:43:19.721119+0000 (oldest deadline 2024-09-21T03:43:47.321877+0000) 2024-09-21T03:43:48.500 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 21 03:43:48 smithi110 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-10[57089]: 2024-09-21T03:43:48.353+0000 7f9d479dc640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2024-09-21T03:43:21.264189+0000 front 2024-09-21T03:43:21.264118+0000 (oldest deadline 2024-09-21T03:43:43.564295+0000) 2024-09-21T03:43:48.500 INFO:journalctl@ceph.osd.4.smithi110.stdout:Sep 21 03:43:48 smithi110 systemd[1]: Stopping Ceph osd.4 for 7c374098-77c9-11ef-baf5-efdc52797490... 2024-09-21T03:43:48.500 INFO:journalctl@ceph.osd.4.smithi110.stdout:Sep 21 03:43:48 smithi110 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-4[44035]: 2024-09-21T03:43:48.463+0000 7fe43d325640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T03:43:48.500 INFO:journalctl@ceph.osd.4.smithi110.stdout:Sep 21 03:43:48 smithi110 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-4[44035]: 2024-09-21T03:43:48.463+0000 7fe43d325640 -1 osd.4 88 *** Got signal Terminated *** 2024-09-21T03:43:48.500 INFO:journalctl@ceph.osd.4.smithi110.stdout:Sep 21 03:43:48 smithi110 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-4[44035]: 2024-09-21T03:43:48.463+0000 7fe43d325640 -1 osd.4 88 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-21T03:43:48.561 INFO:journalctl@ceph.osd.9.smithi089.stdout:Sep 21 03:43:48 smithi089 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-9[65404]: 2024-09-21T03:43:48.342+0000 7f6c4e7c5640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2024-09-21T03:43:18.845419+0000 front 2024-09-21T03:43:18.845455+0000 (oldest deadline 2024-09-21T03:43:44.745556+0000) 2024-09-21T03:43:49.461 INFO:journalctl@ceph.osd.11.smithi114.stdout:Sep 21 03:43:49 smithi114 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-11[58313]: 2024-09-21T03:43:49.114+0000 7f7657ce9640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2024-09-21T03:43:19.721069+0000 front 2024-09-21T03:43:19.721119+0000 (oldest deadline 2024-09-21T03:43:47.321877+0000) 2024-09-21T03:43:49.750 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 21 03:43:49 smithi110 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-10[57089]: 2024-09-21T03:43:49.317+0000 7f9d479dc640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2024-09-21T03:43:21.264189+0000 front 2024-09-21T03:43:21.264118+0000 (oldest deadline 2024-09-21T03:43:43.564295+0000) 2024-09-21T03:43:49.811 INFO:journalctl@ceph.osd.9.smithi089.stdout:Sep 21 03:43:49 smithi089 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-9[65404]: 2024-09-21T03:43:49.362+0000 7f6c4e7c5640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2024-09-21T03:43:18.845419+0000 front 2024-09-21T03:43:18.845455+0000 (oldest deadline 2024-09-21T03:43:44.745556+0000) 2024-09-21T03:43:50.461 INFO:journalctl@ceph.osd.11.smithi114.stdout:Sep 21 03:43:50 smithi114 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-11[58313]: 2024-09-21T03:43:50.076+0000 7f7657ce9640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2024-09-21T03:43:19.721069+0000 front 2024-09-21T03:43:19.721119+0000 (oldest deadline 2024-09-21T03:43:47.321877+0000) 2024-09-21T03:43:50.750 INFO:journalctl@ceph.osd.4.smithi110.stdout:Sep 21 03:43:50 smithi110 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-4[44035]: 2024-09-21T03:43:50.525+0000 7fe439910640 -1 osd.4 88 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-21T03:43:28.221879+0000 front 2024-09-21T03:43:28.221932+0000 (oldest deadline 2024-09-21T03:43:50.521968+0000) 2024-09-21T03:43:50.750 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 21 03:43:50 smithi110 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-10[57089]: 2024-09-21T03:43:50.317+0000 7f9d479dc640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2024-09-21T03:43:21.264189+0000 front 2024-09-21T03:43:21.264118+0000 (oldest deadline 2024-09-21T03:43:43.564295+0000) 2024-09-21T03:43:50.750 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 21 03:43:50 smithi110 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-10[57089]: 2024-09-21T03:43:50.317+0000 7f9d479dc640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-21T03:43:25.265351+0000 front 2024-09-21T03:43:25.265386+0000 (oldest deadline 2024-09-21T03:43:49.965504+0000) 2024-09-21T03:43:50.811 INFO:journalctl@ceph.osd.9.smithi089.stdout:Sep 21 03:43:50 smithi089 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-9[65404]: 2024-09-21T03:43:50.409+0000 7f6c4e7c5640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2024-09-21T03:43:18.845419+0000 front 2024-09-21T03:43:18.845455+0000 (oldest deadline 2024-09-21T03:43:44.745556+0000) 2024-09-21T03:43:50.811 INFO:journalctl@ceph.osd.6.smithi089.stdout:Sep 21 03:43:50 smithi089 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-6[59191]: 2024-09-21T03:43:50.556+0000 7f845162a640 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-21T03:43:28.381812+0000 front 2024-09-21T03:43:28.381946+0000 (oldest deadline 2024-09-21T03:43:49.982463+0000) 2024-09-21T03:43:51.461 INFO:journalctl@ceph.osd.11.smithi114.stdout:Sep 21 03:43:51 smithi114 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-11[58313]: 2024-09-21T03:43:51.103+0000 7f7657ce9640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2024-09-21T03:43:19.721069+0000 front 2024-09-21T03:43:19.721119+0000 (oldest deadline 2024-09-21T03:43:47.321877+0000) 2024-09-21T03:43:51.548 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 21 03:43:51 smithi110 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-10[57089]: 2024-09-21T03:43:51.273+0000 7f9d479dc640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2024-09-21T03:43:21.264189+0000 front 2024-09-21T03:43:21.264118+0000 (oldest deadline 2024-09-21T03:43:43.564295+0000) 2024-09-21T03:43:51.548 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 21 03:43:51 smithi110 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-10[57089]: 2024-09-21T03:43:51.273+0000 7f9d479dc640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-21T03:43:25.265351+0000 front 2024-09-21T03:43:25.265386+0000 (oldest deadline 2024-09-21T03:43:49.965504+0000) 2024-09-21T03:43:51.811 INFO:journalctl@ceph.osd.6.smithi089.stdout:Sep 21 03:43:51 smithi089 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-6[59191]: 2024-09-21T03:43:51.525+0000 7f845162a640 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-21T03:43:28.381812+0000 front 2024-09-21T03:43:28.381946+0000 (oldest deadline 2024-09-21T03:43:49.982463+0000) 2024-09-21T03:43:51.811 INFO:journalctl@ceph.osd.9.smithi089.stdout:Sep 21 03:43:51 smithi089 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-9[65404]: 2024-09-21T03:43:51.383+0000 7f6c4e7c5640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2024-09-21T03:43:18.845419+0000 front 2024-09-21T03:43:18.845455+0000 (oldest deadline 2024-09-21T03:43:44.745556+0000) 2024-09-21T03:43:51.811 INFO:journalctl@ceph.osd.9.smithi089.stdout:Sep 21 03:43:51 smithi089 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-9[65404]: 2024-09-21T03:43:51.383+0000 7f6c4e7c5640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-21T03:43:24.746023+0000 front 2024-09-21T03:43:24.745970+0000 (oldest deadline 2024-09-21T03:43:50.646080+0000) 2024-09-21T03:43:51.961 INFO:journalctl@ceph.osd.5.smithi114.stdout:Sep 21 03:43:51 smithi114 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-5[45121]: 2024-09-21T03:43:51.665+0000 7fa94e1da640 -1 osd.5 88 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-21T03:43:28.513050+0000 front 2024-09-21T03:43:28.512888+0000 (oldest deadline 2024-09-21T03:43:50.812807+0000) 2024-09-21T03:43:52.000 INFO:journalctl@ceph.osd.4.smithi110.stdout:Sep 21 03:43:51 smithi110 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-4[44035]: 2024-09-21T03:43:51.545+0000 7fe439910640 -1 osd.4 88 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-21T03:43:28.221879+0000 front 2024-09-21T03:43:28.221932+0000 (oldest deadline 2024-09-21T03:43:50.521968+0000) 2024-09-21T03:43:52.461 INFO:journalctl@ceph.osd.8.smithi114.stdout:Sep 21 03:43:52 smithi114 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-8[51702]: 2024-09-21T03:43:52.289+0000 7f3e66f25640 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-21T03:43:23.926943+0000 front 2024-09-21T03:43:23.926870+0000 (oldest deadline 2024-09-21T03:43:52.127434+0000) 2024-09-21T03:43:52.461 INFO:journalctl@ceph.osd.11.smithi114.stdout:Sep 21 03:43:52 smithi114 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-11[58313]: 2024-09-21T03:43:52.142+0000 7f7657ce9640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2024-09-21T03:43:19.721069+0000 front 2024-09-21T03:43:19.721119+0000 (oldest deadline 2024-09-21T03:43:47.321877+0000) 2024-09-21T03:43:52.461 INFO:journalctl@ceph.osd.11.smithi114.stdout:Sep 21 03:43:52 smithi114 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-11[58313]: 2024-09-21T03:43:52.142+0000 7f7657ce9640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-21T03:43:27.322375+0000 front 2024-09-21T03:43:27.322426+0000 (oldest deadline 2024-09-21T03:43:51.422473+0000) 2024-09-21T03:43:52.516 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 21 03:43:52 smithi110 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-10[57089]: 2024-09-21T03:43:52.254+0000 7f9d479dc640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2024-09-21T03:43:21.264189+0000 front 2024-09-21T03:43:21.264118+0000 (oldest deadline 2024-09-21T03:43:43.564295+0000) 2024-09-21T03:43:52.516 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 21 03:43:52 smithi110 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-10[57089]: 2024-09-21T03:43:52.254+0000 7f9d479dc640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-21T03:43:25.265351+0000 front 2024-09-21T03:43:25.265386+0000 (oldest deadline 2024-09-21T03:43:49.965504+0000) 2024-09-21T03:43:52.712 INFO:journalctl@ceph.osd.6.smithi089.stdout:Sep 21 03:43:52 smithi089 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-6[59191]: 2024-09-21T03:43:52.483+0000 7f845162a640 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-21T03:43:28.381812+0000 front 2024-09-21T03:43:28.381946+0000 (oldest deadline 2024-09-21T03:43:49.982463+0000) 2024-09-21T03:43:52.713 INFO:journalctl@ceph.osd.9.smithi089.stdout:Sep 21 03:43:52 smithi089 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-9[65404]: 2024-09-21T03:43:52.391+0000 7f6c4e7c5640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2024-09-21T03:43:18.845419+0000 front 2024-09-21T03:43:18.845455+0000 (oldest deadline 2024-09-21T03:43:44.745556+0000) 2024-09-21T03:43:52.713 INFO:journalctl@ceph.osd.9.smithi089.stdout:Sep 21 03:43:52 smithi089 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-9[65404]: 2024-09-21T03:43:52.391+0000 7f6c4e7c5640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-21T03:43:24.746023+0000 front 2024-09-21T03:43:24.745970+0000 (oldest deadline 2024-09-21T03:43:50.646080+0000) 2024-09-21T03:43:52.961 INFO:journalctl@ceph.osd.5.smithi114.stdout:Sep 21 03:43:52 smithi114 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-5[45121]: 2024-09-21T03:43:52.685+0000 7fa94e1da640 -1 osd.5 88 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-21T03:43:28.513050+0000 front 2024-09-21T03:43:28.512888+0000 (oldest deadline 2024-09-21T03:43:50.812807+0000) 2024-09-21T03:43:53.000 INFO:journalctl@ceph.osd.4.smithi110.stdout:Sep 21 03:43:52 smithi110 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-4[44035]: 2024-09-21T03:43:52.515+0000 7fe439910640 -1 osd.4 88 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-21T03:43:28.221879+0000 front 2024-09-21T03:43:28.221932+0000 (oldest deadline 2024-09-21T03:43:50.521968+0000) 2024-09-21T03:43:53.461 INFO:journalctl@ceph.osd.8.smithi114.stdout:Sep 21 03:43:53 smithi114 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-8[51702]: 2024-09-21T03:43:53.337+0000 7f3e66f25640 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-21T03:43:23.926943+0000 front 2024-09-21T03:43:23.926870+0000 (oldest deadline 2024-09-21T03:43:52.127434+0000) 2024-09-21T03:43:53.462 INFO:journalctl@ceph.osd.11.smithi114.stdout:Sep 21 03:43:53 smithi114 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-11[58313]: 2024-09-21T03:43:53.160+0000 7f7657ce9640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2024-09-21T03:43:19.721069+0000 front 2024-09-21T03:43:19.721119+0000 (oldest deadline 2024-09-21T03:43:47.321877+0000) 2024-09-21T03:43:53.462 INFO:journalctl@ceph.osd.11.smithi114.stdout:Sep 21 03:43:53 smithi114 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-11[58313]: 2024-09-21T03:43:53.160+0000 7f7657ce9640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-21T03:43:27.322375+0000 front 2024-09-21T03:43:27.322426+0000 (oldest deadline 2024-09-21T03:43:51.422473+0000) 2024-09-21T03:43:53.546 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 21 03:43:53 smithi110 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-10[57089]: 2024-09-21T03:43:53.267+0000 7f9d479dc640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2024-09-21T03:43:21.264189+0000 front 2024-09-21T03:43:21.264118+0000 (oldest deadline 2024-09-21T03:43:43.564295+0000) 2024-09-21T03:43:53.547 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 21 03:43:53 smithi110 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-10[57089]: 2024-09-21T03:43:53.267+0000 7f9d479dc640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-21T03:43:25.265351+0000 front 2024-09-21T03:43:25.265386+0000 (oldest deadline 2024-09-21T03:43:49.965504+0000) 2024-09-21T03:43:53.811 INFO:journalctl@ceph.osd.6.smithi089.stdout:Sep 21 03:43:53 smithi089 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-6[59191]: 2024-09-21T03:43:53.507+0000 7f845162a640 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-21T03:43:28.381812+0000 front 2024-09-21T03:43:28.381946+0000 (oldest deadline 2024-09-21T03:43:49.982463+0000) 2024-09-21T03:43:53.811 INFO:journalctl@ceph.osd.9.smithi089.stdout:Sep 21 03:43:53 smithi089 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-9[65404]: 2024-09-21T03:43:53.383+0000 7f6c4e7c5640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2024-09-21T03:43:18.845419+0000 front 2024-09-21T03:43:18.845455+0000 (oldest deadline 2024-09-21T03:43:44.745556+0000) 2024-09-21T03:43:53.811 INFO:journalctl@ceph.osd.9.smithi089.stdout:Sep 21 03:43:53 smithi089 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-9[65404]: 2024-09-21T03:43:53.383+0000 7f6c4e7c5640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-21T03:43:24.746023+0000 front 2024-09-21T03:43:24.745970+0000 (oldest deadline 2024-09-21T03:43:50.646080+0000) 2024-09-21T03:43:53.961 INFO:journalctl@ceph.osd.5.smithi114.stdout:Sep 21 03:43:53 smithi114 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-5[45121]: 2024-09-21T03:43:53.665+0000 7fa94e1da640 -1 osd.5 88 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-21T03:43:28.513050+0000 front 2024-09-21T03:43:28.512888+0000 (oldest deadline 2024-09-21T03:43:50.812807+0000) 2024-09-21T03:43:54.250 INFO:journalctl@ceph.osd.4.smithi110.stdout:Sep 21 03:43:53 smithi110 podman[64946]: 2024-09-21 03:43:53.94134082 +0000 UTC m=+5.553028622 container died a2f1b2e2b128edab409194dee8b76a01f8f7320d0339d2e4543bb029818b4e93 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-4, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-21T03:43:54.461 INFO:journalctl@ceph.osd.8.smithi114.stdout:Sep 21 03:43:54 smithi114 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-8[51702]: 2024-09-21T03:43:54.307+0000 7f3e66f25640 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-21T03:43:23.926943+0000 front 2024-09-21T03:43:23.926870+0000 (oldest deadline 2024-09-21T03:43:52.127434+0000) 2024-09-21T03:43:54.461 INFO:journalctl@ceph.osd.11.smithi114.stdout:Sep 21 03:43:54 smithi114 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-11[58313]: 2024-09-21T03:43:54.186+0000 7f7657ce9640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2024-09-21T03:43:19.721069+0000 front 2024-09-21T03:43:19.721119+0000 (oldest deadline 2024-09-21T03:43:47.321877+0000) 2024-09-21T03:43:54.462 INFO:journalctl@ceph.osd.11.smithi114.stdout:Sep 21 03:43:54 smithi114 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-11[58313]: 2024-09-21T03:43:54.186+0000 7f7657ce9640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-21T03:43:27.322375+0000 front 2024-09-21T03:43:27.322426+0000 (oldest deadline 2024-09-21T03:43:51.422473+0000) 2024-09-21T03:43:54.584 INFO:journalctl@ceph.osd.7.smithi110.stdout:Sep 21 03:43:54 smithi110 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-7[50481]: 2024-09-21T03:43:54.448+0000 7fb9f2345640 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-21T03:43:29.081492+0000 front 2024-09-21T03:43:29.081521+0000 (oldest deadline 2024-09-21T03:43:53.781711+0000) 2024-09-21T03:43:54.584 INFO:journalctl@ceph.osd.4.smithi110.stdout:Sep 21 03:43:54 smithi110 podman[64946]: 2024-09-21 03:43:54.33922795 +0000 UTC m=+5.950915742 container cleanup a2f1b2e2b128edab409194dee8b76a01f8f7320d0339d2e4543bb029818b4e93 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-4, GIT_CLEAN=True, CEPH_POINT_RELEASE=, ceph=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0) 2024-09-21T03:43:54.584 INFO:journalctl@ceph.osd.4.smithi110.stdout:Sep 21 03:43:54 smithi110 bash[64946]: ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-4 2024-09-21T03:43:54.584 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 21 03:43:54 smithi110 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-10[57089]: 2024-09-21T03:43:54.277+0000 7f9d479dc640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2024-09-21T03:43:21.264189+0000 front 2024-09-21T03:43:21.264118+0000 (oldest deadline 2024-09-21T03:43:43.564295+0000) 2024-09-21T03:43:54.584 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 21 03:43:54 smithi110 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-10[57089]: 2024-09-21T03:43:54.277+0000 7f9d479dc640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-21T03:43:25.265351+0000 front 2024-09-21T03:43:25.265386+0000 (oldest deadline 2024-09-21T03:43:49.965504+0000) 2024-09-21T03:43:54.811 INFO:journalctl@ceph.osd.6.smithi089.stdout:Sep 21 03:43:54 smithi089 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-6[59191]: 2024-09-21T03:43:54.511+0000 7f845162a640 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-21T03:43:28.381812+0000 front 2024-09-21T03:43:28.381946+0000 (oldest deadline 2024-09-21T03:43:49.982463+0000) 2024-09-21T03:43:54.811 INFO:journalctl@ceph.osd.9.smithi089.stdout:Sep 21 03:43:54 smithi089 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-9[65404]: 2024-09-21T03:43:54.350+0000 7f6c4e7c5640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2024-09-21T03:43:18.845419+0000 front 2024-09-21T03:43:18.845455+0000 (oldest deadline 2024-09-21T03:43:44.745556+0000) 2024-09-21T03:43:54.812 INFO:journalctl@ceph.osd.9.smithi089.stdout:Sep 21 03:43:54 smithi089 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-9[65404]: 2024-09-21T03:43:54.350+0000 7f6c4e7c5640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-21T03:43:24.746023+0000 front 2024-09-21T03:43:24.745970+0000 (oldest deadline 2024-09-21T03:43:50.646080+0000) 2024-09-21T03:43:54.943 INFO:journalctl@ceph.osd.4.smithi110.stdout:Sep 21 03:43:54 smithi110 podman[64965]: 2024-09-21 03:43:54.582532441 +0000 UTC m=+0.637558875 container remove a2f1b2e2b128edab409194dee8b76a01f8f7320d0339d2e4543bb029818b4e93 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-4, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.build-date=20240731, ceph=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-09-21T03:43:54.961 INFO:journalctl@ceph.osd.5.smithi114.stdout:Sep 21 03:43:54 smithi114 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-5[45121]: 2024-09-21T03:43:54.648+0000 7fa94e1da640 -1 osd.5 88 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-21T03:43:28.513050+0000 front 2024-09-21T03:43:28.512888+0000 (oldest deadline 2024-09-21T03:43:50.812807+0000) 2024-09-21T03:43:55.204 INFO:journalctl@ceph.osd.4.smithi110.stdout:Sep 21 03:43:54 smithi110 podman[65051]: 2024-09-21 03:43:54.844574506 +0000 UTC m=+0.015428990 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T03:43:55.204 INFO:journalctl@ceph.osd.4.smithi110.stdout:Sep 21 03:43:54 smithi110 podman[65051]: 2024-09-21 03:43:54.966889958 +0000 UTC m=+0.137744429 container create 223bb1ee23f2b5f38a2bc9ea3ab75b8c575af11e302d3645d77b1603c9575369 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-4-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0) 2024-09-21T03:43:55.204 INFO:journalctl@ceph.osd.4.smithi110.stdout:Sep 21 03:43:55 smithi110 podman[65051]: 2024-09-21 03:43:55.202694065 +0000 UTC m=+0.373548541 container init 223bb1ee23f2b5f38a2bc9ea3ab75b8c575af11e302d3645d77b1603c9575369 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-4-deactivate, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, ceph=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221) 2024-09-21T03:43:55.461 INFO:journalctl@ceph.osd.8.smithi114.stdout:Sep 21 03:43:55 smithi114 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-8[51702]: 2024-09-21T03:43:55.333+0000 7f3e66f25640 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-21T03:43:23.926943+0000 front 2024-09-21T03:43:23.926870+0000 (oldest deadline 2024-09-21T03:43:52.127434+0000) 2024-09-21T03:43:55.461 INFO:journalctl@ceph.osd.11.smithi114.stdout:Sep 21 03:43:55 smithi114 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-11[58313]: 2024-09-21T03:43:55.204+0000 7f7657ce9640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2024-09-21T03:43:19.721069+0000 front 2024-09-21T03:43:19.721119+0000 (oldest deadline 2024-09-21T03:43:47.321877+0000) 2024-09-21T03:43:55.461 INFO:journalctl@ceph.osd.11.smithi114.stdout:Sep 21 03:43:55 smithi114 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-11[58313]: 2024-09-21T03:43:55.204+0000 7f7657ce9640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-21T03:43:27.322375+0000 front 2024-09-21T03:43:27.322426+0000 (oldest deadline 2024-09-21T03:43:51.422473+0000) 2024-09-21T03:43:55.474 INFO:journalctl@ceph.osd.4.smithi110.stdout:Sep 21 03:43:55 smithi110 podman[65051]: 2024-09-21 03:43:55.206152281 +0000 UTC m=+0.377006762 container start 223bb1ee23f2b5f38a2bc9ea3ab75b8c575af11e302d3645d77b1603c9575369 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-4-deactivate, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_CLEAN=True) 2024-09-21T03:43:55.474 INFO:journalctl@ceph.osd.4.smithi110.stdout:Sep 21 03:43:55 smithi110 podman[65051]: 2024-09-21 03:43:55.286191104 +0000 UTC m=+0.457045580 container attach 223bb1ee23f2b5f38a2bc9ea3ab75b8c575af11e302d3645d77b1603c9575369 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-4-deactivate, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-21T03:43:55.474 INFO:journalctl@ceph.osd.4.smithi110.stdout:Sep 21 03:43:55 smithi110 podman[65051]: 2024-09-21 03:43:55.402168464 +0000 UTC m=+0.573022954 container died 223bb1ee23f2b5f38a2bc9ea3ab75b8c575af11e302d3645d77b1603c9575369 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-4-deactivate, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, ceph=True, GIT_CLEAN=True, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux ) 2024-09-21T03:43:55.474 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 21 03:43:55 smithi110 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-10[57089]: 2024-09-21T03:43:55.272+0000 7f9d479dc640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2024-09-21T03:43:21.264189+0000 front 2024-09-21T03:43:21.264118+0000 (oldest deadline 2024-09-21T03:43:43.564295+0000) 2024-09-21T03:43:55.474 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 21 03:43:55 smithi110 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-10[57089]: 2024-09-21T03:43:55.272+0000 7f9d479dc640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-21T03:43:25.265351+0000 front 2024-09-21T03:43:25.265386+0000 (oldest deadline 2024-09-21T03:43:49.965504+0000) 2024-09-21T03:43:55.750 INFO:journalctl@ceph.osd.7.smithi110.stdout:Sep 21 03:43:55 smithi110 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-7[50481]: 2024-09-21T03:43:55.471+0000 7fb9f2345640 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-21T03:43:29.081492+0000 front 2024-09-21T03:43:29.081521+0000 (oldest deadline 2024-09-21T03:43:53.781711+0000) 2024-09-21T03:43:55.811 INFO:journalctl@ceph.osd.6.smithi089.stdout:Sep 21 03:43:55 smithi089 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-6[59191]: 2024-09-21T03:43:55.545+0000 7f845162a640 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-21T03:43:28.381812+0000 front 2024-09-21T03:43:28.381946+0000 (oldest deadline 2024-09-21T03:43:49.982463+0000) 2024-09-21T03:43:55.811 INFO:journalctl@ceph.osd.9.smithi089.stdout:Sep 21 03:43:55 smithi089 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-9[65404]: 2024-09-21T03:43:55.353+0000 7f6c4e7c5640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2024-09-21T03:43:18.845419+0000 front 2024-09-21T03:43:18.845455+0000 (oldest deadline 2024-09-21T03:43:44.745556+0000) 2024-09-21T03:43:55.811 INFO:journalctl@ceph.osd.9.smithi089.stdout:Sep 21 03:43:55 smithi089 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-9[65404]: 2024-09-21T03:43:55.353+0000 7f6c4e7c5640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-21T03:43:24.746023+0000 front 2024-09-21T03:43:24.745970+0000 (oldest deadline 2024-09-21T03:43:50.646080+0000) 2024-09-21T03:43:55.961 INFO:journalctl@ceph.osd.5.smithi114.stdout:Sep 21 03:43:55 smithi114 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-5[45121]: 2024-09-21T03:43:55.630+0000 7fa94e1da640 -1 osd.5 88 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-21T03:43:28.513050+0000 front 2024-09-21T03:43:28.512888+0000 (oldest deadline 2024-09-21T03:43:50.812807+0000) 2024-09-21T03:43:56.005 DEBUG:teuthology.orchestra.run.smithi110:> sudo pkill -f 'journalctl -f -n 0 -u ceph-7c374098-77c9-11ef-baf5-efdc52797490@osd.4.service' 2024-09-21T03:43:56.043 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T03:43:56.043 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-09-21T03:43:56.043 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-09-21T03:43:56.043 DEBUG:teuthology.orchestra.run.smithi114:> sudo systemctl stop ceph-7c374098-77c9-11ef-baf5-efdc52797490@osd.5 2024-09-21T03:43:56.333 INFO:journalctl@ceph.osd.5.smithi114.stdout:Sep 21 03:43:56 smithi114 systemd[1]: Stopping Ceph osd.5 for 7c374098-77c9-11ef-baf5-efdc52797490... 2024-09-21T03:43:56.333 INFO:journalctl@ceph.osd.11.smithi114.stdout:Sep 21 03:43:56 smithi114 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-11[58313]: 2024-09-21T03:43:56.186+0000 7f7657ce9640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2024-09-21T03:43:19.721069+0000 front 2024-09-21T03:43:19.721119+0000 (oldest deadline 2024-09-21T03:43:47.321877+0000) 2024-09-21T03:43:56.333 INFO:journalctl@ceph.osd.11.smithi114.stdout:Sep 21 03:43:56 smithi114 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-11[58313]: 2024-09-21T03:43:56.186+0000 7f7657ce9640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-21T03:43:27.322375+0000 front 2024-09-21T03:43:27.322426+0000 (oldest deadline 2024-09-21T03:43:51.422473+0000) 2024-09-21T03:43:56.647 INFO:journalctl@ceph.osd.8.smithi114.stdout:Sep 21 03:43:56 smithi114 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-8[51702]: 2024-09-21T03:43:56.330+0000 7f3e66f25640 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-21T03:43:23.926943+0000 front 2024-09-21T03:43:23.926870+0000 (oldest deadline 2024-09-21T03:43:52.127434+0000) 2024-09-21T03:43:56.647 INFO:journalctl@ceph.osd.5.smithi114.stdout:Sep 21 03:43:56 smithi114 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-5[45121]: 2024-09-21T03:43:56.359+0000 7fa955bf7640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T03:43:56.647 INFO:journalctl@ceph.osd.5.smithi114.stdout:Sep 21 03:43:56 smithi114 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-5[45121]: 2024-09-21T03:43:56.359+0000 7fa955bf7640 -1 osd.5 88 *** Got signal Terminated *** 2024-09-21T03:43:56.648 INFO:journalctl@ceph.osd.5.smithi114.stdout:Sep 21 03:43:56 smithi114 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-5[45121]: 2024-09-21T03:43:56.359+0000 7fa955bf7640 -1 osd.5 88 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-21T03:43:56.648 INFO:journalctl@ceph.osd.5.smithi114.stdout:Sep 21 03:43:56 smithi114 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-5[45121]: 2024-09-21T03:43:56.645+0000 7fa94e1da640 -1 osd.5 88 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-21T03:43:28.513050+0000 front 2024-09-21T03:43:28.512888+0000 (oldest deadline 2024-09-21T03:43:50.812807+0000) 2024-09-21T03:43:56.750 INFO:journalctl@ceph.osd.7.smithi110.stdout:Sep 21 03:43:56 smithi110 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-7[50481]: 2024-09-21T03:43:56.424+0000 7fb9f2345640 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-21T03:43:29.081492+0000 front 2024-09-21T03:43:29.081521+0000 (oldest deadline 2024-09-21T03:43:53.781711+0000) 2024-09-21T03:43:56.750 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 21 03:43:56 smithi110 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-10[57089]: 2024-09-21T03:43:56.284+0000 7f9d479dc640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2024-09-21T03:43:21.264189+0000 front 2024-09-21T03:43:21.264118+0000 (oldest deadline 2024-09-21T03:43:43.564295+0000) 2024-09-21T03:43:56.750 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 21 03:43:56 smithi110 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-10[57089]: 2024-09-21T03:43:56.284+0000 7f9d479dc640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-21T03:43:25.265351+0000 front 2024-09-21T03:43:25.265386+0000 (oldest deadline 2024-09-21T03:43:49.965504+0000) 2024-09-21T03:43:56.811 INFO:journalctl@ceph.osd.6.smithi089.stdout:Sep 21 03:43:56 smithi089 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-6[59191]: 2024-09-21T03:43:56.520+0000 7f845162a640 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-21T03:43:28.381812+0000 front 2024-09-21T03:43:28.381946+0000 (oldest deadline 2024-09-21T03:43:49.982463+0000) 2024-09-21T03:43:56.811 INFO:journalctl@ceph.osd.9.smithi089.stdout:Sep 21 03:43:56 smithi089 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-9[65404]: 2024-09-21T03:43:56.374+0000 7f6c4e7c5640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2024-09-21T03:43:18.845419+0000 front 2024-09-21T03:43:18.845455+0000 (oldest deadline 2024-09-21T03:43:44.745556+0000) 2024-09-21T03:43:56.811 INFO:journalctl@ceph.osd.9.smithi089.stdout:Sep 21 03:43:56 smithi089 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-9[65404]: 2024-09-21T03:43:56.374+0000 7f6c4e7c5640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-21T03:43:24.746023+0000 front 2024-09-21T03:43:24.745970+0000 (oldest deadline 2024-09-21T03:43:50.646080+0000) 2024-09-21T03:43:57.500 INFO:journalctl@ceph.osd.7.smithi110.stdout:Sep 21 03:43:57 smithi110 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-7[50481]: 2024-09-21T03:43:57.413+0000 7fb9f2345640 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-21T03:43:29.081492+0000 front 2024-09-21T03:43:29.081521+0000 (oldest deadline 2024-09-21T03:43:53.781711+0000) 2024-09-21T03:43:57.500 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 21 03:43:57 smithi110 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-10[57089]: 2024-09-21T03:43:57.236+0000 7f9d479dc640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2024-09-21T03:43:21.264189+0000 front 2024-09-21T03:43:21.264118+0000 (oldest deadline 2024-09-21T03:43:43.564295+0000) 2024-09-21T03:43:57.500 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 21 03:43:57 smithi110 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-10[57089]: 2024-09-21T03:43:57.236+0000 7f9d479dc640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-21T03:43:25.265351+0000 front 2024-09-21T03:43:25.265386+0000 (oldest deadline 2024-09-21T03:43:49.965504+0000) 2024-09-21T03:43:57.674 INFO:journalctl@ceph.osd.8.smithi114.stdout:Sep 21 03:43:57 smithi114 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-8[51702]: 2024-09-21T03:43:57.353+0000 7f3e66f25640 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-21T03:43:23.926943+0000 front 2024-09-21T03:43:23.926870+0000 (oldest deadline 2024-09-21T03:43:52.127434+0000) 2024-09-21T03:43:57.675 INFO:journalctl@ceph.osd.11.smithi114.stdout:Sep 21 03:43:57 smithi114 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-11[58313]: 2024-09-21T03:43:57.223+0000 7f7657ce9640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2024-09-21T03:43:19.721069+0000 front 2024-09-21T03:43:19.721119+0000 (oldest deadline 2024-09-21T03:43:47.321877+0000) 2024-09-21T03:43:57.675 INFO:journalctl@ceph.osd.11.smithi114.stdout:Sep 21 03:43:57 smithi114 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-11[58313]: 2024-09-21T03:43:57.223+0000 7f7657ce9640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-21T03:43:27.322375+0000 front 2024-09-21T03:43:27.322426+0000 (oldest deadline 2024-09-21T03:43:51.422473+0000) 2024-09-21T03:43:57.811 INFO:journalctl@ceph.osd.6.smithi089.stdout:Sep 21 03:43:57 smithi089 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-6[59191]: 2024-09-21T03:43:57.537+0000 7f845162a640 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-21T03:43:28.381812+0000 front 2024-09-21T03:43:28.381946+0000 (oldest deadline 2024-09-21T03:43:49.982463+0000) 2024-09-21T03:43:57.811 INFO:journalctl@ceph.osd.9.smithi089.stdout:Sep 21 03:43:57 smithi089 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-9[65404]: 2024-09-21T03:43:57.417+0000 7f6c4e7c5640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2024-09-21T03:43:18.845419+0000 front 2024-09-21T03:43:18.845455+0000 (oldest deadline 2024-09-21T03:43:44.745556+0000) 2024-09-21T03:43:57.811 INFO:journalctl@ceph.osd.9.smithi089.stdout:Sep 21 03:43:57 smithi089 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-9[65404]: 2024-09-21T03:43:57.417+0000 7f6c4e7c5640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-21T03:43:24.746023+0000 front 2024-09-21T03:43:24.745970+0000 (oldest deadline 2024-09-21T03:43:50.646080+0000) 2024-09-21T03:43:57.961 INFO:journalctl@ceph.osd.5.smithi114.stdout:Sep 21 03:43:57 smithi114 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-5[45121]: 2024-09-21T03:43:57.672+0000 7fa94e1da640 -1 osd.5 88 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-21T03:43:28.513050+0000 front 2024-09-21T03:43:28.512888+0000 (oldest deadline 2024-09-21T03:43:50.812807+0000) 2024-09-21T03:43:58.500 INFO:journalctl@ceph.osd.7.smithi110.stdout:Sep 21 03:43:58 smithi110 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-7[50481]: 2024-09-21T03:43:58.387+0000 7fb9f2345640 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-21T03:43:29.081492+0000 front 2024-09-21T03:43:29.081521+0000 (oldest deadline 2024-09-21T03:43:53.781711+0000) 2024-09-21T03:43:58.500 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 21 03:43:58 smithi110 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-10[57089]: 2024-09-21T03:43:58.227+0000 7f9d479dc640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2024-09-21T03:43:21.264189+0000 front 2024-09-21T03:43:21.264118+0000 (oldest deadline 2024-09-21T03:43:43.564295+0000) 2024-09-21T03:43:58.500 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 21 03:43:58 smithi110 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-10[57089]: 2024-09-21T03:43:58.227+0000 7f9d479dc640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-21T03:43:25.265351+0000 front 2024-09-21T03:43:25.265386+0000 (oldest deadline 2024-09-21T03:43:49.965504+0000) 2024-09-21T03:43:58.676 INFO:journalctl@ceph.osd.8.smithi114.stdout:Sep 21 03:43:58 smithi114 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-8[51702]: 2024-09-21T03:43:58.331+0000 7f3e66f25640 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-21T03:43:23.926943+0000 front 2024-09-21T03:43:23.926870+0000 (oldest deadline 2024-09-21T03:43:52.127434+0000) 2024-09-21T03:43:58.676 INFO:journalctl@ceph.osd.8.smithi114.stdout:Sep 21 03:43:58 smithi114 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-8[51702]: 2024-09-21T03:43:58.331+0000 7f3e66f25640 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.114:6804 osd.2 since back 2024-09-21T03:43:32.128066+0000 front 2024-09-21T03:43:32.127963+0000 (oldest deadline 2024-09-21T03:43:58.028025+0000) 2024-09-21T03:43:58.676 INFO:journalctl@ceph.osd.11.smithi114.stdout:Sep 21 03:43:58 smithi114 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-11[58313]: 2024-09-21T03:43:58.216+0000 7f7657ce9640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2024-09-21T03:43:19.721069+0000 front 2024-09-21T03:43:19.721119+0000 (oldest deadline 2024-09-21T03:43:47.321877+0000) 2024-09-21T03:43:58.676 INFO:journalctl@ceph.osd.11.smithi114.stdout:Sep 21 03:43:58 smithi114 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-11[58313]: 2024-09-21T03:43:58.216+0000 7f7657ce9640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-21T03:43:27.322375+0000 front 2024-09-21T03:43:27.322426+0000 (oldest deadline 2024-09-21T03:43:51.422473+0000) 2024-09-21T03:43:58.811 INFO:journalctl@ceph.osd.6.smithi089.stdout:Sep 21 03:43:58 smithi089 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-6[59191]: 2024-09-21T03:43:58.578+0000 7f845162a640 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-21T03:43:28.381812+0000 front 2024-09-21T03:43:28.381946+0000 (oldest deadline 2024-09-21T03:43:49.982463+0000) 2024-09-21T03:43:58.811 INFO:journalctl@ceph.osd.9.smithi089.stdout:Sep 21 03:43:58 smithi089 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-9[65404]: 2024-09-21T03:43:58.375+0000 7f6c4e7c5640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2024-09-21T03:43:18.845419+0000 front 2024-09-21T03:43:18.845455+0000 (oldest deadline 2024-09-21T03:43:44.745556+0000) 2024-09-21T03:43:58.811 INFO:journalctl@ceph.osd.9.smithi089.stdout:Sep 21 03:43:58 smithi089 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-9[65404]: 2024-09-21T03:43:58.375+0000 7f6c4e7c5640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-21T03:43:24.746023+0000 front 2024-09-21T03:43:24.745970+0000 (oldest deadline 2024-09-21T03:43:50.646080+0000) 2024-09-21T03:43:58.811 INFO:journalctl@ceph.osd.9.smithi089.stdout:Sep 21 03:43:58 smithi089 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-9[65404]: 2024-09-21T03:43:58.375+0000 7f6c4e7c5640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.114:6804 osd.2 since back 2024-09-21T03:43:34.147460+0000 front 2024-09-21T03:43:34.147364+0000 (oldest deadline 2024-09-21T03:43:58.247427+0000) 2024-09-21T03:43:58.961 INFO:journalctl@ceph.osd.5.smithi114.stdout:Sep 21 03:43:58 smithi114 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-5[45121]: 2024-09-21T03:43:58.674+0000 7fa94e1da640 -1 osd.5 88 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-21T03:43:28.513050+0000 front 2024-09-21T03:43:28.512888+0000 (oldest deadline 2024-09-21T03:43:50.812807+0000) 2024-09-21T03:43:58.961 INFO:journalctl@ceph.osd.5.smithi114.stdout:Sep 21 03:43:58 smithi114 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-5[45121]: 2024-09-21T03:43:58.674+0000 7fa94e1da640 -1 osd.5 88 heartbeat_check: no reply from 172.21.15.114:6804 osd.2 since back 2024-09-21T03:43:33.713881+0000 front 2024-09-21T03:43:33.713841+0000 (oldest deadline 2024-09-21T03:43:58.413935+0000) 2024-09-21T03:43:59.500 INFO:journalctl@ceph.osd.7.smithi110.stdout:Sep 21 03:43:59 smithi110 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-7[50481]: 2024-09-21T03:43:59.417+0000 7fb9f2345640 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-21T03:43:29.081492+0000 front 2024-09-21T03:43:29.081521+0000 (oldest deadline 2024-09-21T03:43:53.781711+0000) 2024-09-21T03:43:59.500 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 21 03:43:59 smithi110 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-10[57089]: 2024-09-21T03:43:59.201+0000 7f9d479dc640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2024-09-21T03:43:21.264189+0000 front 2024-09-21T03:43:21.264118+0000 (oldest deadline 2024-09-21T03:43:43.564295+0000) 2024-09-21T03:43:59.500 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 21 03:43:59 smithi110 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-10[57089]: 2024-09-21T03:43:59.201+0000 7f9d479dc640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-21T03:43:25.265351+0000 front 2024-09-21T03:43:25.265386+0000 (oldest deadline 2024-09-21T03:43:49.965504+0000) 2024-09-21T03:43:59.645 INFO:journalctl@ceph.osd.8.smithi114.stdout:Sep 21 03:43:59 smithi114 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-8[51702]: 2024-09-21T03:43:59.348+0000 7f3e66f25640 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-21T03:43:23.926943+0000 front 2024-09-21T03:43:23.926870+0000 (oldest deadline 2024-09-21T03:43:52.127434+0000) 2024-09-21T03:43:59.645 INFO:journalctl@ceph.osd.8.smithi114.stdout:Sep 21 03:43:59 smithi114 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-8[51702]: 2024-09-21T03:43:59.348+0000 7f3e66f25640 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.114:6804 osd.2 since back 2024-09-21T03:43:32.128066+0000 front 2024-09-21T03:43:32.127963+0000 (oldest deadline 2024-09-21T03:43:58.028025+0000) 2024-09-21T03:43:59.645 INFO:journalctl@ceph.osd.11.smithi114.stdout:Sep 21 03:43:59 smithi114 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-11[58313]: 2024-09-21T03:43:59.262+0000 7f7657ce9640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2024-09-21T03:43:19.721069+0000 front 2024-09-21T03:43:19.721119+0000 (oldest deadline 2024-09-21T03:43:47.321877+0000) 2024-09-21T03:43:59.645 INFO:journalctl@ceph.osd.11.smithi114.stdout:Sep 21 03:43:59 smithi114 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-11[58313]: 2024-09-21T03:43:59.262+0000 7f7657ce9640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-21T03:43:27.322375+0000 front 2024-09-21T03:43:27.322426+0000 (oldest deadline 2024-09-21T03:43:51.422473+0000) 2024-09-21T03:43:59.645 INFO:journalctl@ceph.osd.11.smithi114.stdout:Sep 21 03:43:59 smithi114 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-11[58313]: 2024-09-21T03:43:59.262+0000 7f7657ce9640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.114:6804 osd.2 since back 2024-09-21T03:43:37.224230+0000 front 2024-09-21T03:43:37.224148+0000 (oldest deadline 2024-09-21T03:43:58.924202+0000) 2024-09-21T03:43:59.811 INFO:journalctl@ceph.osd.6.smithi089.stdout:Sep 21 03:43:59 smithi089 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-6[59191]: 2024-09-21T03:43:59.533+0000 7f845162a640 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-21T03:43:28.381812+0000 front 2024-09-21T03:43:28.381946+0000 (oldest deadline 2024-09-21T03:43:49.982463+0000) 2024-09-21T03:43:59.811 INFO:journalctl@ceph.osd.9.smithi089.stdout:Sep 21 03:43:59 smithi089 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-9[65404]: 2024-09-21T03:43:59.409+0000 7f6c4e7c5640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2024-09-21T03:43:18.845419+0000 front 2024-09-21T03:43:18.845455+0000 (oldest deadline 2024-09-21T03:43:44.745556+0000) 2024-09-21T03:43:59.811 INFO:journalctl@ceph.osd.9.smithi089.stdout:Sep 21 03:43:59 smithi089 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-9[65404]: 2024-09-21T03:43:59.409+0000 7f6c4e7c5640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-21T03:43:24.746023+0000 front 2024-09-21T03:43:24.745970+0000 (oldest deadline 2024-09-21T03:43:50.646080+0000) 2024-09-21T03:43:59.811 INFO:journalctl@ceph.osd.9.smithi089.stdout:Sep 21 03:43:59 smithi089 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-9[65404]: 2024-09-21T03:43:59.409+0000 7f6c4e7c5640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.114:6804 osd.2 since back 2024-09-21T03:43:34.147460+0000 front 2024-09-21T03:43:34.147364+0000 (oldest deadline 2024-09-21T03:43:58.247427+0000) 2024-09-21T03:43:59.961 INFO:journalctl@ceph.osd.5.smithi114.stdout:Sep 21 03:43:59 smithi114 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-5[45121]: 2024-09-21T03:43:59.643+0000 7fa94e1da640 -1 osd.5 88 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-21T03:43:28.513050+0000 front 2024-09-21T03:43:28.512888+0000 (oldest deadline 2024-09-21T03:43:50.812807+0000) 2024-09-21T03:43:59.961 INFO:journalctl@ceph.osd.5.smithi114.stdout:Sep 21 03:43:59 smithi114 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-5[45121]: 2024-09-21T03:43:59.643+0000 7fa94e1da640 -1 osd.5 88 heartbeat_check: no reply from 172.21.15.114:6804 osd.2 since back 2024-09-21T03:43:33.713881+0000 front 2024-09-21T03:43:33.713841+0000 (oldest deadline 2024-09-21T03:43:58.413935+0000) 2024-09-21T03:44:00.500 INFO:journalctl@ceph.osd.7.smithi110.stdout:Sep 21 03:44:00 smithi110 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-7[50481]: 2024-09-21T03:44:00.458+0000 7fb9f2345640 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-21T03:43:29.081492+0000 front 2024-09-21T03:43:29.081521+0000 (oldest deadline 2024-09-21T03:43:53.781711+0000) 2024-09-21T03:44:00.500 INFO:journalctl@ceph.osd.7.smithi110.stdout:Sep 21 03:44:00 smithi110 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-7[50481]: 2024-09-21T03:44:00.458+0000 7fb9f2345640 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.114:6804 osd.2 since back 2024-09-21T03:43:37.283011+0000 front 2024-09-21T03:43:37.282954+0000 (oldest deadline 2024-09-21T03:43:59.582792+0000) 2024-09-21T03:44:00.500 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 21 03:44:00 smithi110 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-10[57089]: 2024-09-21T03:44:00.210+0000 7f9d479dc640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2024-09-21T03:43:21.264189+0000 front 2024-09-21T03:43:21.264118+0000 (oldest deadline 2024-09-21T03:43:43.564295+0000) 2024-09-21T03:44:00.500 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 21 03:44:00 smithi110 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-10[57089]: 2024-09-21T03:44:00.210+0000 7f9d479dc640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-21T03:43:25.265351+0000 front 2024-09-21T03:43:25.265386+0000 (oldest deadline 2024-09-21T03:43:49.965504+0000) 2024-09-21T03:44:00.501 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 21 03:44:00 smithi110 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-10[57089]: 2024-09-21T03:44:00.210+0000 7f9d479dc640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.114:6804 osd.2 since back 2024-09-21T03:43:33.966726+0000 front 2024-09-21T03:43:33.966884+0000 (oldest deadline 2024-09-21T03:43:59.866510+0000) 2024-09-21T03:44:00.602 INFO:journalctl@ceph.osd.8.smithi114.stdout:Sep 21 03:44:00 smithi114 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-8[51702]: 2024-09-21T03:44:00.356+0000 7f3e66f25640 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-21T03:43:23.926943+0000 front 2024-09-21T03:43:23.926870+0000 (oldest deadline 2024-09-21T03:43:52.127434+0000) 2024-09-21T03:44:00.602 INFO:journalctl@ceph.osd.8.smithi114.stdout:Sep 21 03:44:00 smithi114 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-8[51702]: 2024-09-21T03:44:00.356+0000 7f3e66f25640 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.114:6804 osd.2 since back 2024-09-21T03:43:32.128066+0000 front 2024-09-21T03:43:32.127963+0000 (oldest deadline 2024-09-21T03:43:58.028025+0000) 2024-09-21T03:44:00.602 INFO:journalctl@ceph.osd.11.smithi114.stdout:Sep 21 03:44:00 smithi114 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-11[58313]: 2024-09-21T03:44:00.284+0000 7f7657ce9640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2024-09-21T03:43:19.721069+0000 front 2024-09-21T03:43:19.721119+0000 (oldest deadline 2024-09-21T03:43:47.321877+0000) 2024-09-21T03:44:00.602 INFO:journalctl@ceph.osd.11.smithi114.stdout:Sep 21 03:44:00 smithi114 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-11[58313]: 2024-09-21T03:44:00.284+0000 7f7657ce9640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-21T03:43:27.322375+0000 front 2024-09-21T03:43:27.322426+0000 (oldest deadline 2024-09-21T03:43:51.422473+0000) 2024-09-21T03:44:00.603 INFO:journalctl@ceph.osd.11.smithi114.stdout:Sep 21 03:44:00 smithi114 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-11[58313]: 2024-09-21T03:44:00.284+0000 7f7657ce9640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.114:6804 osd.2 since back 2024-09-21T03:43:37.224230+0000 front 2024-09-21T03:43:37.224148+0000 (oldest deadline 2024-09-21T03:43:58.924202+0000) 2024-09-21T03:44:00.811 INFO:journalctl@ceph.osd.6.smithi089.stdout:Sep 21 03:44:00 smithi089 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-6[59191]: 2024-09-21T03:44:00.537+0000 7f845162a640 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-21T03:43:28.381812+0000 front 2024-09-21T03:43:28.381946+0000 (oldest deadline 2024-09-21T03:43:49.982463+0000) 2024-09-21T03:44:00.811 INFO:journalctl@ceph.osd.6.smithi089.stdout:Sep 21 03:44:00 smithi089 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-6[59191]: 2024-09-21T03:44:00.537+0000 7f845162a640 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.114:6804 osd.2 since back 2024-09-21T03:43:33.984420+0000 front 2024-09-21T03:43:33.984313+0000 (oldest deadline 2024-09-21T03:44:00.384680+0000) 2024-09-21T03:44:00.811 INFO:journalctl@ceph.osd.9.smithi089.stdout:Sep 21 03:44:00 smithi089 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-9[65404]: 2024-09-21T03:44:00.439+0000 7f6c4e7c5640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2024-09-21T03:43:18.845419+0000 front 2024-09-21T03:43:18.845455+0000 (oldest deadline 2024-09-21T03:43:44.745556+0000) 2024-09-21T03:44:00.811 INFO:journalctl@ceph.osd.9.smithi089.stdout:Sep 21 03:44:00 smithi089 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-9[65404]: 2024-09-21T03:44:00.439+0000 7f6c4e7c5640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-21T03:43:24.746023+0000 front 2024-09-21T03:43:24.745970+0000 (oldest deadline 2024-09-21T03:43:50.646080+0000) 2024-09-21T03:44:00.811 INFO:journalctl@ceph.osd.9.smithi089.stdout:Sep 21 03:44:00 smithi089 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-9[65404]: 2024-09-21T03:44:00.439+0000 7f6c4e7c5640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.114:6804 osd.2 since back 2024-09-21T03:43:34.147460+0000 front 2024-09-21T03:43:34.147364+0000 (oldest deadline 2024-09-21T03:43:58.247427+0000) 2024-09-21T03:44:00.961 INFO:journalctl@ceph.osd.5.smithi114.stdout:Sep 21 03:44:00 smithi114 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-5[45121]: 2024-09-21T03:44:00.600+0000 7fa94e1da640 -1 osd.5 88 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-21T03:43:28.513050+0000 front 2024-09-21T03:43:28.512888+0000 (oldest deadline 2024-09-21T03:43:50.812807+0000) 2024-09-21T03:44:00.961 INFO:journalctl@ceph.osd.5.smithi114.stdout:Sep 21 03:44:00 smithi114 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-5[45121]: 2024-09-21T03:44:00.600+0000 7fa94e1da640 -1 osd.5 88 heartbeat_check: no reply from 172.21.15.114:6804 osd.2 since back 2024-09-21T03:43:33.713881+0000 front 2024-09-21T03:43:33.713841+0000 (oldest deadline 2024-09-21T03:43:58.413935+0000) 2024-09-21T03:44:01.447 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 21 03:44:01 smithi110 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-10[57089]: 2024-09-21T03:44:01.187+0000 7f9d479dc640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2024-09-21T03:43:21.264189+0000 front 2024-09-21T03:43:21.264118+0000 (oldest deadline 2024-09-21T03:43:43.564295+0000) 2024-09-21T03:44:01.447 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 21 03:44:01 smithi110 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-10[57089]: 2024-09-21T03:44:01.187+0000 7f9d479dc640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-21T03:43:25.265351+0000 front 2024-09-21T03:43:25.265386+0000 (oldest deadline 2024-09-21T03:43:49.965504+0000) 2024-09-21T03:44:01.447 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 21 03:44:01 smithi110 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-10[57089]: 2024-09-21T03:44:01.187+0000 7f9d479dc640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.114:6804 osd.2 since back 2024-09-21T03:43:33.966726+0000 front 2024-09-21T03:43:33.966884+0000 (oldest deadline 2024-09-21T03:43:59.866510+0000) 2024-09-21T03:44:01.711 INFO:journalctl@ceph.osd.8.smithi114.stdout:Sep 21 03:44:01 smithi114 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-8[51702]: 2024-09-21T03:44:01.311+0000 7f3e66f25640 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-21T03:43:23.926943+0000 front 2024-09-21T03:43:23.926870+0000 (oldest deadline 2024-09-21T03:43:52.127434+0000) 2024-09-21T03:44:01.711 INFO:journalctl@ceph.osd.8.smithi114.stdout:Sep 21 03:44:01 smithi114 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-8[51702]: 2024-09-21T03:44:01.311+0000 7f3e66f25640 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.114:6804 osd.2 since back 2024-09-21T03:43:32.128066+0000 front 2024-09-21T03:43:32.127963+0000 (oldest deadline 2024-09-21T03:43:58.028025+0000) 2024-09-21T03:44:01.711 INFO:journalctl@ceph.osd.11.smithi114.stdout:Sep 21 03:44:01 smithi114 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-11[58313]: 2024-09-21T03:44:01.333+0000 7f7657ce9640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2024-09-21T03:43:19.721069+0000 front 2024-09-21T03:43:19.721119+0000 (oldest deadline 2024-09-21T03:43:47.321877+0000) 2024-09-21T03:44:01.711 INFO:journalctl@ceph.osd.11.smithi114.stdout:Sep 21 03:44:01 smithi114 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-11[58313]: 2024-09-21T03:44:01.333+0000 7f7657ce9640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-21T03:43:27.322375+0000 front 2024-09-21T03:43:27.322426+0000 (oldest deadline 2024-09-21T03:43:51.422473+0000) 2024-09-21T03:44:01.711 INFO:journalctl@ceph.osd.11.smithi114.stdout:Sep 21 03:44:01 smithi114 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-11[58313]: 2024-09-21T03:44:01.333+0000 7f7657ce9640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.114:6804 osd.2 since back 2024-09-21T03:43:37.224230+0000 front 2024-09-21T03:43:37.224148+0000 (oldest deadline 2024-09-21T03:43:58.924202+0000) 2024-09-21T03:44:01.750 INFO:journalctl@ceph.osd.7.smithi110.stdout:Sep 21 03:44:01 smithi110 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-7[50481]: 2024-09-21T03:44:01.445+0000 7fb9f2345640 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-21T03:43:29.081492+0000 front 2024-09-21T03:43:29.081521+0000 (oldest deadline 2024-09-21T03:43:53.781711+0000) 2024-09-21T03:44:01.750 INFO:journalctl@ceph.osd.7.smithi110.stdout:Sep 21 03:44:01 smithi110 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-7[50481]: 2024-09-21T03:44:01.445+0000 7fb9f2345640 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.114:6804 osd.2 since back 2024-09-21T03:43:37.283011+0000 front 2024-09-21T03:43:37.282954+0000 (oldest deadline 2024-09-21T03:43:59.582792+0000) 2024-09-21T03:44:01.811 INFO:journalctl@ceph.osd.6.smithi089.stdout:Sep 21 03:44:01 smithi089 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-6[59191]: 2024-09-21T03:44:01.532+0000 7f845162a640 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-21T03:43:28.381812+0000 front 2024-09-21T03:43:28.381946+0000 (oldest deadline 2024-09-21T03:43:49.982463+0000) 2024-09-21T03:44:01.811 INFO:journalctl@ceph.osd.6.smithi089.stdout:Sep 21 03:44:01 smithi089 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-6[59191]: 2024-09-21T03:44:01.532+0000 7f845162a640 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.114:6804 osd.2 since back 2024-09-21T03:43:33.984420+0000 front 2024-09-21T03:43:33.984313+0000 (oldest deadline 2024-09-21T03:44:00.384680+0000) 2024-09-21T03:44:01.811 INFO:journalctl@ceph.osd.9.smithi089.stdout:Sep 21 03:44:01 smithi089 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-9[65404]: 2024-09-21T03:44:01.441+0000 7f6c4e7c5640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2024-09-21T03:43:18.845419+0000 front 2024-09-21T03:43:18.845455+0000 (oldest deadline 2024-09-21T03:43:44.745556+0000) 2024-09-21T03:44:01.811 INFO:journalctl@ceph.osd.9.smithi089.stdout:Sep 21 03:44:01 smithi089 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-9[65404]: 2024-09-21T03:44:01.442+0000 7f6c4e7c5640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-21T03:43:24.746023+0000 front 2024-09-21T03:43:24.745970+0000 (oldest deadline 2024-09-21T03:43:50.646080+0000) 2024-09-21T03:44:01.812 INFO:journalctl@ceph.osd.9.smithi089.stdout:Sep 21 03:44:01 smithi089 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-9[65404]: 2024-09-21T03:44:01.442+0000 7f6c4e7c5640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.114:6804 osd.2 since back 2024-09-21T03:43:34.147460+0000 front 2024-09-21T03:43:34.147364+0000 (oldest deadline 2024-09-21T03:43:58.247427+0000) 2024-09-21T03:44:02.120 INFO:journalctl@ceph.osd.5.smithi114.stdout:Sep 21 03:44:01 smithi114 podman[66163]: 2024-09-21 03:44:01.86088499 +0000 UTC m=+5.620826467 container died 45bdbb1a62504681f2153a5defbfbd3b6505fcfbb0c3a3bfd4418ce98cebedc9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-5, GIT_CLEAN=True, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True) 2024-09-21T03:44:02.426 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 21 03:44:02 smithi110 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-10[57089]: 2024-09-21T03:44:02.173+0000 7f9d479dc640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2024-09-21T03:43:21.264189+0000 front 2024-09-21T03:43:21.264118+0000 (oldest deadline 2024-09-21T03:43:43.564295+0000) 2024-09-21T03:44:02.426 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 21 03:44:02 smithi110 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-10[57089]: 2024-09-21T03:44:02.173+0000 7f9d479dc640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-21T03:43:25.265351+0000 front 2024-09-21T03:43:25.265386+0000 (oldest deadline 2024-09-21T03:43:49.965504+0000) 2024-09-21T03:44:02.427 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 21 03:44:02 smithi110 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-10[57089]: 2024-09-21T03:44:02.173+0000 7f9d479dc640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.114:6804 osd.2 since back 2024-09-21T03:43:33.966726+0000 front 2024-09-21T03:43:33.966884+0000 (oldest deadline 2024-09-21T03:43:59.866510+0000) 2024-09-21T03:44:02.461 INFO:journalctl@ceph.osd.5.smithi114.stdout:Sep 21 03:44:02 smithi114 podman[66163]: 2024-09-21 03:44:02.218311466 +0000 UTC m=+5.978252936 container cleanup 45bdbb1a62504681f2153a5defbfbd3b6505fcfbb0c3a3bfd4418ce98cebedc9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-5, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD) 2024-09-21T03:44:02.461 INFO:journalctl@ceph.osd.5.smithi114.stdout:Sep 21 03:44:02 smithi114 bash[66163]: ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-5 2024-09-21T03:44:02.461 INFO:journalctl@ceph.osd.8.smithi114.stdout:Sep 21 03:44:02 smithi114 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-8[51702]: 2024-09-21T03:44:02.285+0000 7f3e66f25640 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-21T03:43:23.926943+0000 front 2024-09-21T03:43:23.926870+0000 (oldest deadline 2024-09-21T03:43:52.127434+0000) 2024-09-21T03:44:02.461 INFO:journalctl@ceph.osd.8.smithi114.stdout:Sep 21 03:44:02 smithi114 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-8[51702]: 2024-09-21T03:44:02.285+0000 7f3e66f25640 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.114:6804 osd.2 since back 2024-09-21T03:43:32.128066+0000 front 2024-09-21T03:43:32.127963+0000 (oldest deadline 2024-09-21T03:43:58.028025+0000) 2024-09-21T03:44:02.461 INFO:journalctl@ceph.osd.11.smithi114.stdout:Sep 21 03:44:02 smithi114 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-11[58313]: 2024-09-21T03:44:02.296+0000 7f7657ce9640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2024-09-21T03:43:19.721069+0000 front 2024-09-21T03:43:19.721119+0000 (oldest deadline 2024-09-21T03:43:47.321877+0000) 2024-09-21T03:44:02.461 INFO:journalctl@ceph.osd.11.smithi114.stdout:Sep 21 03:44:02 smithi114 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-11[58313]: 2024-09-21T03:44:02.296+0000 7f7657ce9640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-21T03:43:27.322375+0000 front 2024-09-21T03:43:27.322426+0000 (oldest deadline 2024-09-21T03:43:51.422473+0000) 2024-09-21T03:44:02.462 INFO:journalctl@ceph.osd.11.smithi114.stdout:Sep 21 03:44:02 smithi114 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-11[58313]: 2024-09-21T03:44:02.296+0000 7f7657ce9640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.114:6804 osd.2 since back 2024-09-21T03:43:37.224230+0000 front 2024-09-21T03:43:37.224148+0000 (oldest deadline 2024-09-21T03:43:58.924202+0000) 2024-09-21T03:44:02.750 INFO:journalctl@ceph.osd.7.smithi110.stdout:Sep 21 03:44:02 smithi110 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-7[50481]: 2024-09-21T03:44:02.424+0000 7fb9f2345640 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-21T03:43:29.081492+0000 front 2024-09-21T03:43:29.081521+0000 (oldest deadline 2024-09-21T03:43:53.781711+0000) 2024-09-21T03:44:02.750 INFO:journalctl@ceph.osd.7.smithi110.stdout:Sep 21 03:44:02 smithi110 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-7[50481]: 2024-09-21T03:44:02.424+0000 7fb9f2345640 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.114:6804 osd.2 since back 2024-09-21T03:43:37.283011+0000 front 2024-09-21T03:43:37.282954+0000 (oldest deadline 2024-09-21T03:43:59.582792+0000) 2024-09-21T03:44:02.811 INFO:journalctl@ceph.osd.6.smithi089.stdout:Sep 21 03:44:02 smithi089 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-6[59191]: 2024-09-21T03:44:02.506+0000 7f845162a640 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-21T03:43:28.381812+0000 front 2024-09-21T03:43:28.381946+0000 (oldest deadline 2024-09-21T03:43:49.982463+0000) 2024-09-21T03:44:02.811 INFO:journalctl@ceph.osd.6.smithi089.stdout:Sep 21 03:44:02 smithi089 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-6[59191]: 2024-09-21T03:44:02.506+0000 7f845162a640 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.114:6804 osd.2 since back 2024-09-21T03:43:33.984420+0000 front 2024-09-21T03:43:33.984313+0000 (oldest deadline 2024-09-21T03:44:00.384680+0000) 2024-09-21T03:44:02.811 INFO:journalctl@ceph.osd.9.smithi089.stdout:Sep 21 03:44:02 smithi089 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-9[65404]: 2024-09-21T03:44:02.393+0000 7f6c4e7c5640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2024-09-21T03:43:18.845419+0000 front 2024-09-21T03:43:18.845455+0000 (oldest deadline 2024-09-21T03:43:44.745556+0000) 2024-09-21T03:44:02.812 INFO:journalctl@ceph.osd.9.smithi089.stdout:Sep 21 03:44:02 smithi089 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-9[65404]: 2024-09-21T03:44:02.393+0000 7f6c4e7c5640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-21T03:43:24.746023+0000 front 2024-09-21T03:43:24.745970+0000 (oldest deadline 2024-09-21T03:43:50.646080+0000) 2024-09-21T03:44:02.812 INFO:journalctl@ceph.osd.9.smithi089.stdout:Sep 21 03:44:02 smithi089 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-9[65404]: 2024-09-21T03:44:02.393+0000 7f6c4e7c5640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.114:6804 osd.2 since back 2024-09-21T03:43:34.147460+0000 front 2024-09-21T03:43:34.147364+0000 (oldest deadline 2024-09-21T03:43:58.247427+0000) 2024-09-21T03:44:02.833 INFO:journalctl@ceph.osd.5.smithi114.stdout:Sep 21 03:44:02 smithi114 podman[66178]: 2024-09-21 03:44:02.479098371 +0000 UTC m=+0.621288600 container remove 45bdbb1a62504681f2153a5defbfbd3b6505fcfbb0c3a3bfd4418ce98cebedc9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-5, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-09-21T03:44:03.086 INFO:journalctl@ceph.osd.5.smithi114.stdout:Sep 21 03:44:02 smithi114 podman[66266]: 2024-09-21 03:44:02.734932958 +0000 UTC m=+0.015443930 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T03:44:03.086 INFO:journalctl@ceph.osd.5.smithi114.stdout:Sep 21 03:44:02 smithi114 podman[66266]: 2024-09-21 03:44:02.845597578 +0000 UTC m=+0.126108539 container create 9079a5476e59950ee1d31d6f0e2adeba1bb18b6af26e36b5cf043b11b4f2b171 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-5-deactivate, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, ceph=True, GIT_CLEAN=True) 2024-09-21T03:44:03.086 INFO:journalctl@ceph.osd.5.smithi114.stdout:Sep 21 03:44:03 smithi114 podman[66266]: 2024-09-21 03:44:03.081558389 +0000 UTC m=+0.362069347 container init 9079a5476e59950ee1d31d6f0e2adeba1bb18b6af26e36b5cf043b11b4f2b171 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-5-deactivate, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.build-date=20240731) 2024-09-21T03:44:03.087 INFO:journalctl@ceph.osd.5.smithi114.stdout:Sep 21 03:44:03 smithi114 podman[66266]: 2024-09-21 03:44:03.084785032 +0000 UTC m=+0.365295991 container start 9079a5476e59950ee1d31d6f0e2adeba1bb18b6af26e36b5cf043b11b4f2b171 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-5-deactivate, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-21T03:44:03.385 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 21 03:44:03 smithi110 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-10[57089]: 2024-09-21T03:44:03.124+0000 7f9d479dc640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2024-09-21T03:43:21.264189+0000 front 2024-09-21T03:43:21.264118+0000 (oldest deadline 2024-09-21T03:43:43.564295+0000) 2024-09-21T03:44:03.385 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 21 03:44:03 smithi110 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-10[57089]: 2024-09-21T03:44:03.124+0000 7f9d479dc640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-21T03:43:25.265351+0000 front 2024-09-21T03:43:25.265386+0000 (oldest deadline 2024-09-21T03:43:49.965504+0000) 2024-09-21T03:44:03.385 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 21 03:44:03 smithi110 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-10[57089]: 2024-09-21T03:44:03.124+0000 7f9d479dc640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.114:6804 osd.2 since back 2024-09-21T03:43:33.966726+0000 front 2024-09-21T03:43:33.966884+0000 (oldest deadline 2024-09-21T03:43:59.866510+0000) 2024-09-21T03:44:03.461 INFO:journalctl@ceph.osd.8.smithi114.stdout:Sep 21 03:44:03 smithi114 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-8[51702]: 2024-09-21T03:44:03.314+0000 7f3e66f25640 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-21T03:43:23.926943+0000 front 2024-09-21T03:43:23.926870+0000 (oldest deadline 2024-09-21T03:43:52.127434+0000) 2024-09-21T03:44:03.461 INFO:journalctl@ceph.osd.8.smithi114.stdout:Sep 21 03:44:03 smithi114 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-8[51702]: 2024-09-21T03:44:03.314+0000 7f3e66f25640 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.114:6804 osd.2 since back 2024-09-21T03:43:32.128066+0000 front 2024-09-21T03:43:32.127963+0000 (oldest deadline 2024-09-21T03:43:58.028025+0000) 2024-09-21T03:44:03.461 INFO:journalctl@ceph.osd.11.smithi114.stdout:Sep 21 03:44:03 smithi114 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-11[58313]: 2024-09-21T03:44:03.252+0000 7f7657ce9640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2024-09-21T03:43:19.721069+0000 front 2024-09-21T03:43:19.721119+0000 (oldest deadline 2024-09-21T03:43:47.321877+0000) 2024-09-21T03:44:03.461 INFO:journalctl@ceph.osd.11.smithi114.stdout:Sep 21 03:44:03 smithi114 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-11[58313]: 2024-09-21T03:44:03.252+0000 7f7657ce9640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-21T03:43:27.322375+0000 front 2024-09-21T03:43:27.322426+0000 (oldest deadline 2024-09-21T03:43:51.422473+0000) 2024-09-21T03:44:03.462 INFO:journalctl@ceph.osd.11.smithi114.stdout:Sep 21 03:44:03 smithi114 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-11[58313]: 2024-09-21T03:44:03.252+0000 7f7657ce9640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.114:6804 osd.2 since back 2024-09-21T03:43:37.224230+0000 front 2024-09-21T03:43:37.224148+0000 (oldest deadline 2024-09-21T03:43:58.924202+0000) 2024-09-21T03:44:03.462 INFO:journalctl@ceph.osd.5.smithi114.stdout:Sep 21 03:44:03 smithi114 podman[66266]: 2024-09-21 03:44:03.181583821 +0000 UTC m=+0.462094788 container attach 9079a5476e59950ee1d31d6f0e2adeba1bb18b6af26e36b5cf043b11b4f2b171 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-5-deactivate, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731) 2024-09-21T03:44:03.462 INFO:journalctl@ceph.osd.5.smithi114.stdout:Sep 21 03:44:03 smithi114 podman[66266]: 2024-09-21 03:44:03.267814281 +0000 UTC m=+0.548325255 container died 9079a5476e59950ee1d31d6f0e2adeba1bb18b6af26e36b5cf043b11b4f2b171 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-5-deactivate, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-21T03:44:03.750 INFO:journalctl@ceph.osd.7.smithi110.stdout:Sep 21 03:44:03 smithi110 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-7[50481]: 2024-09-21T03:44:03.383+0000 7fb9f2345640 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-21T03:43:29.081492+0000 front 2024-09-21T03:43:29.081521+0000 (oldest deadline 2024-09-21T03:43:53.781711+0000) 2024-09-21T03:44:03.750 INFO:journalctl@ceph.osd.7.smithi110.stdout:Sep 21 03:44:03 smithi110 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-7[50481]: 2024-09-21T03:44:03.383+0000 7fb9f2345640 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.114:6804 osd.2 since back 2024-09-21T03:43:37.283011+0000 front 2024-09-21T03:43:37.282954+0000 (oldest deadline 2024-09-21T03:43:59.582792+0000) 2024-09-21T03:44:03.811 INFO:journalctl@ceph.osd.6.smithi089.stdout:Sep 21 03:44:03 smithi089 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-6[59191]: 2024-09-21T03:44:03.491+0000 7f845162a640 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-21T03:43:28.381812+0000 front 2024-09-21T03:43:28.381946+0000 (oldest deadline 2024-09-21T03:43:49.982463+0000) 2024-09-21T03:44:03.811 INFO:journalctl@ceph.osd.6.smithi089.stdout:Sep 21 03:44:03 smithi089 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-6[59191]: 2024-09-21T03:44:03.491+0000 7f845162a640 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.114:6804 osd.2 since back 2024-09-21T03:43:33.984420+0000 front 2024-09-21T03:43:33.984313+0000 (oldest deadline 2024-09-21T03:44:00.384680+0000) 2024-09-21T03:44:03.811 INFO:journalctl@ceph.osd.9.smithi089.stdout:Sep 21 03:44:03 smithi089 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-9[65404]: 2024-09-21T03:44:03.414+0000 7f6c4e7c5640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2024-09-21T03:43:18.845419+0000 front 2024-09-21T03:43:18.845455+0000 (oldest deadline 2024-09-21T03:43:44.745556+0000) 2024-09-21T03:44:03.812 INFO:journalctl@ceph.osd.9.smithi089.stdout:Sep 21 03:44:03 smithi089 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-9[65404]: 2024-09-21T03:44:03.414+0000 7f6c4e7c5640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-21T03:43:24.746023+0000 front 2024-09-21T03:43:24.745970+0000 (oldest deadline 2024-09-21T03:43:50.646080+0000) 2024-09-21T03:44:03.812 INFO:journalctl@ceph.osd.9.smithi089.stdout:Sep 21 03:44:03 smithi089 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-9[65404]: 2024-09-21T03:44:03.414+0000 7f6c4e7c5640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.114:6804 osd.2 since back 2024-09-21T03:43:34.147460+0000 front 2024-09-21T03:43:34.147364+0000 (oldest deadline 2024-09-21T03:43:58.247427+0000) 2024-09-21T03:44:03.958 DEBUG:teuthology.orchestra.run.smithi114:> sudo pkill -f 'journalctl -f -n 0 -u ceph-7c374098-77c9-11ef-baf5-efdc52797490@osd.5.service' 2024-09-21T03:44:04.001 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T03:44:04.001 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-09-21T03:44:04.001 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2024-09-21T03:44:04.002 DEBUG:teuthology.orchestra.run.smithi089:> sudo systemctl stop ceph-7c374098-77c9-11ef-baf5-efdc52797490@osd.6 2024-09-21T03:44:04.302 INFO:journalctl@ceph.osd.6.smithi089.stdout:Sep 21 03:44:04 smithi089 systemd[1]: Stopping Ceph osd.6 for 7c374098-77c9-11ef-baf5-efdc52797490... 2024-09-21T03:44:04.385 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 21 03:44:04 smithi110 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-10[57089]: 2024-09-21T03:44:04.126+0000 7f9d479dc640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2024-09-21T03:43:21.264189+0000 front 2024-09-21T03:43:21.264118+0000 (oldest deadline 2024-09-21T03:43:43.564295+0000) 2024-09-21T03:44:04.385 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 21 03:44:04 smithi110 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-10[57089]: 2024-09-21T03:44:04.126+0000 7f9d479dc640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-21T03:43:25.265351+0000 front 2024-09-21T03:43:25.265386+0000 (oldest deadline 2024-09-21T03:43:49.965504+0000) 2024-09-21T03:44:04.385 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 21 03:44:04 smithi110 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-10[57089]: 2024-09-21T03:44:04.126+0000 7f9d479dc640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.114:6804 osd.2 since back 2024-09-21T03:43:33.966726+0000 front 2024-09-21T03:43:33.966884+0000 (oldest deadline 2024-09-21T03:43:59.866510+0000) 2024-09-21T03:44:04.561 INFO:journalctl@ceph.osd.9.smithi089.stdout:Sep 21 03:44:04 smithi089 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-9[65404]: 2024-09-21T03:44:04.438+0000 7f6c4e7c5640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2024-09-21T03:43:18.845419+0000 front 2024-09-21T03:43:18.845455+0000 (oldest deadline 2024-09-21T03:43:44.745556+0000) 2024-09-21T03:44:04.561 INFO:journalctl@ceph.osd.9.smithi089.stdout:Sep 21 03:44:04 smithi089 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-9[65404]: 2024-09-21T03:44:04.438+0000 7f6c4e7c5640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-21T03:43:24.746023+0000 front 2024-09-21T03:43:24.745970+0000 (oldest deadline 2024-09-21T03:43:50.646080+0000) 2024-09-21T03:44:04.561 INFO:journalctl@ceph.osd.9.smithi089.stdout:Sep 21 03:44:04 smithi089 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-9[65404]: 2024-09-21T03:44:04.438+0000 7f6c4e7c5640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.114:6804 osd.2 since back 2024-09-21T03:43:34.147460+0000 front 2024-09-21T03:43:34.147364+0000 (oldest deadline 2024-09-21T03:43:58.247427+0000) 2024-09-21T03:44:04.561 INFO:journalctl@ceph.osd.6.smithi089.stdout:Sep 21 03:44:04 smithi089 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-6[59191]: 2024-09-21T03:44:04.300+0000 7f8459047640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T03:44:04.561 INFO:journalctl@ceph.osd.6.smithi089.stdout:Sep 21 03:44:04 smithi089 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-6[59191]: 2024-09-21T03:44:04.300+0000 7f8459047640 -1 osd.6 88 *** Got signal Terminated *** 2024-09-21T03:44:04.561 INFO:journalctl@ceph.osd.6.smithi089.stdout:Sep 21 03:44:04 smithi089 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-6[59191]: 2024-09-21T03:44:04.300+0000 7f8459047640 -1 osd.6 88 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-21T03:44:04.562 INFO:journalctl@ceph.osd.6.smithi089.stdout:Sep 21 03:44:04 smithi089 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-6[59191]: 2024-09-21T03:44:04.474+0000 7f845162a640 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-21T03:43:28.381812+0000 front 2024-09-21T03:43:28.381946+0000 (oldest deadline 2024-09-21T03:43:49.982463+0000) 2024-09-21T03:44:04.562 INFO:journalctl@ceph.osd.6.smithi089.stdout:Sep 21 03:44:04 smithi089 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-6[59191]: 2024-09-21T03:44:04.474+0000 7f845162a640 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.114:6804 osd.2 since back 2024-09-21T03:43:33.984420+0000 front 2024-09-21T03:43:33.984313+0000 (oldest deadline 2024-09-21T03:44:00.384680+0000) 2024-09-21T03:44:04.711 INFO:journalctl@ceph.osd.8.smithi114.stdout:Sep 21 03:44:04 smithi114 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-8[51702]: 2024-09-21T03:44:04.359+0000 7f3e66f25640 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-21T03:43:23.926943+0000 front 2024-09-21T03:43:23.926870+0000 (oldest deadline 2024-09-21T03:43:52.127434+0000) 2024-09-21T03:44:04.711 INFO:journalctl@ceph.osd.8.smithi114.stdout:Sep 21 03:44:04 smithi114 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-8[51702]: 2024-09-21T03:44:04.359+0000 7f3e66f25640 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.114:6804 osd.2 since back 2024-09-21T03:43:32.128066+0000 front 2024-09-21T03:43:32.127963+0000 (oldest deadline 2024-09-21T03:43:58.028025+0000) 2024-09-21T03:44:04.711 INFO:journalctl@ceph.osd.11.smithi114.stdout:Sep 21 03:44:04 smithi114 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-11[58313]: 2024-09-21T03:44:04.278+0000 7f7657ce9640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2024-09-21T03:43:19.721069+0000 front 2024-09-21T03:43:19.721119+0000 (oldest deadline 2024-09-21T03:43:47.321877+0000) 2024-09-21T03:44:04.711 INFO:journalctl@ceph.osd.11.smithi114.stdout:Sep 21 03:44:04 smithi114 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-11[58313]: 2024-09-21T03:44:04.278+0000 7f7657ce9640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-21T03:43:27.322375+0000 front 2024-09-21T03:43:27.322426+0000 (oldest deadline 2024-09-21T03:43:51.422473+0000) 2024-09-21T03:44:04.711 INFO:journalctl@ceph.osd.11.smithi114.stdout:Sep 21 03:44:04 smithi114 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-11[58313]: 2024-09-21T03:44:04.279+0000 7f7657ce9640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.114:6804 osd.2 since back 2024-09-21T03:43:37.224230+0000 front 2024-09-21T03:43:37.224148+0000 (oldest deadline 2024-09-21T03:43:58.924202+0000) 2024-09-21T03:44:04.750 INFO:journalctl@ceph.osd.7.smithi110.stdout:Sep 21 03:44:04 smithi110 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-7[50481]: 2024-09-21T03:44:04.384+0000 7fb9f2345640 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-21T03:43:29.081492+0000 front 2024-09-21T03:43:29.081521+0000 (oldest deadline 2024-09-21T03:43:53.781711+0000) 2024-09-21T03:44:04.750 INFO:journalctl@ceph.osd.7.smithi110.stdout:Sep 21 03:44:04 smithi110 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-7[50481]: 2024-09-21T03:44:04.384+0000 7fb9f2345640 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.114:6804 osd.2 since back 2024-09-21T03:43:37.283011+0000 front 2024-09-21T03:43:37.282954+0000 (oldest deadline 2024-09-21T03:43:59.582792+0000) 2024-09-21T03:44:05.421 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 21 03:44:05 smithi110 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-10[57089]: 2024-09-21T03:44:05.147+0000 7f9d479dc640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2024-09-21T03:43:21.264189+0000 front 2024-09-21T03:43:21.264118+0000 (oldest deadline 2024-09-21T03:43:43.564295+0000) 2024-09-21T03:44:05.421 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 21 03:44:05 smithi110 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-10[57089]: 2024-09-21T03:44:05.147+0000 7f9d479dc640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-21T03:43:25.265351+0000 front 2024-09-21T03:43:25.265386+0000 (oldest deadline 2024-09-21T03:43:49.965504+0000) 2024-09-21T03:44:05.421 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 21 03:44:05 smithi110 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-10[57089]: 2024-09-21T03:44:05.147+0000 7f9d479dc640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.114:6804 osd.2 since back 2024-09-21T03:43:33.966726+0000 front 2024-09-21T03:43:33.966884+0000 (oldest deadline 2024-09-21T03:43:59.866510+0000) 2024-09-21T03:44:05.711 INFO:journalctl@ceph.osd.8.smithi114.stdout:Sep 21 03:44:05 smithi114 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-8[51702]: 2024-09-21T03:44:05.356+0000 7f3e66f25640 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-21T03:43:23.926943+0000 front 2024-09-21T03:43:23.926870+0000 (oldest deadline 2024-09-21T03:43:52.127434+0000) 2024-09-21T03:44:05.711 INFO:journalctl@ceph.osd.8.smithi114.stdout:Sep 21 03:44:05 smithi114 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-8[51702]: 2024-09-21T03:44:05.356+0000 7f3e66f25640 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.114:6804 osd.2 since back 2024-09-21T03:43:32.128066+0000 front 2024-09-21T03:43:32.127963+0000 (oldest deadline 2024-09-21T03:43:58.028025+0000) 2024-09-21T03:44:05.711 INFO:journalctl@ceph.osd.11.smithi114.stdout:Sep 21 03:44:05 smithi114 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-11[58313]: 2024-09-21T03:44:05.285+0000 7f7657ce9640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2024-09-21T03:43:19.721069+0000 front 2024-09-21T03:43:19.721119+0000 (oldest deadline 2024-09-21T03:43:47.321877+0000) 2024-09-21T03:44:05.711 INFO:journalctl@ceph.osd.11.smithi114.stdout:Sep 21 03:44:05 smithi114 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-11[58313]: 2024-09-21T03:44:05.285+0000 7f7657ce9640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-21T03:43:27.322375+0000 front 2024-09-21T03:43:27.322426+0000 (oldest deadline 2024-09-21T03:43:51.422473+0000) 2024-09-21T03:44:05.711 INFO:journalctl@ceph.osd.11.smithi114.stdout:Sep 21 03:44:05 smithi114 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-11[58313]: 2024-09-21T03:44:05.285+0000 7f7657ce9640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.114:6804 osd.2 since back 2024-09-21T03:43:37.224230+0000 front 2024-09-21T03:43:37.224148+0000 (oldest deadline 2024-09-21T03:43:58.924202+0000) 2024-09-21T03:44:05.750 INFO:journalctl@ceph.osd.7.smithi110.stdout:Sep 21 03:44:05 smithi110 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-7[50481]: 2024-09-21T03:44:05.419+0000 7fb9f2345640 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-21T03:43:29.081492+0000 front 2024-09-21T03:43:29.081521+0000 (oldest deadline 2024-09-21T03:43:53.781711+0000) 2024-09-21T03:44:05.750 INFO:journalctl@ceph.osd.7.smithi110.stdout:Sep 21 03:44:05 smithi110 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-7[50481]: 2024-09-21T03:44:05.419+0000 7fb9f2345640 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.114:6804 osd.2 since back 2024-09-21T03:43:37.283011+0000 front 2024-09-21T03:43:37.282954+0000 (oldest deadline 2024-09-21T03:43:59.582792+0000) 2024-09-21T03:44:05.811 INFO:journalctl@ceph.osd.6.smithi089.stdout:Sep 21 03:44:05 smithi089 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-6[59191]: 2024-09-21T03:44:05.467+0000 7f845162a640 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-21T03:43:28.381812+0000 front 2024-09-21T03:43:28.381946+0000 (oldest deadline 2024-09-21T03:43:49.982463+0000) 2024-09-21T03:44:05.811 INFO:journalctl@ceph.osd.6.smithi089.stdout:Sep 21 03:44:05 smithi089 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-6[59191]: 2024-09-21T03:44:05.467+0000 7f845162a640 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.114:6804 osd.2 since back 2024-09-21T03:43:33.984420+0000 front 2024-09-21T03:43:33.984313+0000 (oldest deadline 2024-09-21T03:44:00.384680+0000) 2024-09-21T03:44:05.811 INFO:journalctl@ceph.osd.9.smithi089.stdout:Sep 21 03:44:05 smithi089 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-9[65404]: 2024-09-21T03:44:05.479+0000 7f6c4e7c5640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2024-09-21T03:43:18.845419+0000 front 2024-09-21T03:43:18.845455+0000 (oldest deadline 2024-09-21T03:43:44.745556+0000) 2024-09-21T03:44:05.811 INFO:journalctl@ceph.osd.9.smithi089.stdout:Sep 21 03:44:05 smithi089 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-9[65404]: 2024-09-21T03:44:05.479+0000 7f6c4e7c5640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-21T03:43:24.746023+0000 front 2024-09-21T03:43:24.745970+0000 (oldest deadline 2024-09-21T03:43:50.646080+0000) 2024-09-21T03:44:05.812 INFO:journalctl@ceph.osd.9.smithi089.stdout:Sep 21 03:44:05 smithi089 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-9[65404]: 2024-09-21T03:44:05.479+0000 7f6c4e7c5640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.114:6804 osd.2 since back 2024-09-21T03:43:34.147460+0000 front 2024-09-21T03:43:34.147364+0000 (oldest deadline 2024-09-21T03:43:58.247427+0000) 2024-09-21T03:44:06.433 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 21 03:44:06 smithi110 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-10[57089]: 2024-09-21T03:44:06.142+0000 7f9d479dc640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2024-09-21T03:43:21.264189+0000 front 2024-09-21T03:43:21.264118+0000 (oldest deadline 2024-09-21T03:43:43.564295+0000) 2024-09-21T03:44:06.434 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 21 03:44:06 smithi110 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-10[57089]: 2024-09-21T03:44:06.142+0000 7f9d479dc640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-21T03:43:25.265351+0000 front 2024-09-21T03:43:25.265386+0000 (oldest deadline 2024-09-21T03:43:49.965504+0000) 2024-09-21T03:44:06.434 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 21 03:44:06 smithi110 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-10[57089]: 2024-09-21T03:44:06.142+0000 7f9d479dc640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.114:6804 osd.2 since back 2024-09-21T03:43:33.966726+0000 front 2024-09-21T03:43:33.966884+0000 (oldest deadline 2024-09-21T03:43:59.866510+0000) 2024-09-21T03:44:06.711 INFO:journalctl@ceph.osd.8.smithi114.stdout:Sep 21 03:44:06 smithi114 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-8[51702]: 2024-09-21T03:44:06.393+0000 7f3e66f25640 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-21T03:43:23.926943+0000 front 2024-09-21T03:43:23.926870+0000 (oldest deadline 2024-09-21T03:43:52.127434+0000) 2024-09-21T03:44:06.711 INFO:journalctl@ceph.osd.8.smithi114.stdout:Sep 21 03:44:06 smithi114 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-8[51702]: 2024-09-21T03:44:06.393+0000 7f3e66f25640 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.114:6804 osd.2 since back 2024-09-21T03:43:32.128066+0000 front 2024-09-21T03:43:32.127963+0000 (oldest deadline 2024-09-21T03:43:58.028025+0000) 2024-09-21T03:44:06.711 INFO:journalctl@ceph.osd.11.smithi114.stdout:Sep 21 03:44:06 smithi114 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-11[58313]: 2024-09-21T03:44:06.248+0000 7f7657ce9640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2024-09-21T03:43:19.721069+0000 front 2024-09-21T03:43:19.721119+0000 (oldest deadline 2024-09-21T03:43:47.321877+0000) 2024-09-21T03:44:06.711 INFO:journalctl@ceph.osd.11.smithi114.stdout:Sep 21 03:44:06 smithi114 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-11[58313]: 2024-09-21T03:44:06.248+0000 7f7657ce9640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-21T03:43:27.322375+0000 front 2024-09-21T03:43:27.322426+0000 (oldest deadline 2024-09-21T03:43:51.422473+0000) 2024-09-21T03:44:06.711 INFO:journalctl@ceph.osd.11.smithi114.stdout:Sep 21 03:44:06 smithi114 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-11[58313]: 2024-09-21T03:44:06.248+0000 7f7657ce9640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.114:6804 osd.2 since back 2024-09-21T03:43:37.224230+0000 front 2024-09-21T03:43:37.224148+0000 (oldest deadline 2024-09-21T03:43:58.924202+0000) 2024-09-21T03:44:06.750 INFO:journalctl@ceph.osd.7.smithi110.stdout:Sep 21 03:44:06 smithi110 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-7[50481]: 2024-09-21T03:44:06.432+0000 7fb9f2345640 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-21T03:43:29.081492+0000 front 2024-09-21T03:43:29.081521+0000 (oldest deadline 2024-09-21T03:43:53.781711+0000) 2024-09-21T03:44:06.750 INFO:journalctl@ceph.osd.7.smithi110.stdout:Sep 21 03:44:06 smithi110 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-7[50481]: 2024-09-21T03:44:06.432+0000 7fb9f2345640 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.114:6804 osd.2 since back 2024-09-21T03:43:37.283011+0000 front 2024-09-21T03:43:37.282954+0000 (oldest deadline 2024-09-21T03:43:59.582792+0000) 2024-09-21T03:44:06.811 INFO:journalctl@ceph.osd.6.smithi089.stdout:Sep 21 03:44:06 smithi089 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-6[59191]: 2024-09-21T03:44:06.451+0000 7f845162a640 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-21T03:43:28.381812+0000 front 2024-09-21T03:43:28.381946+0000 (oldest deadline 2024-09-21T03:43:49.982463+0000) 2024-09-21T03:44:06.811 INFO:journalctl@ceph.osd.6.smithi089.stdout:Sep 21 03:44:06 smithi089 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-6[59191]: 2024-09-21T03:44:06.451+0000 7f845162a640 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.114:6804 osd.2 since back 2024-09-21T03:43:33.984420+0000 front 2024-09-21T03:43:33.984313+0000 (oldest deadline 2024-09-21T03:44:00.384680+0000) 2024-09-21T03:44:06.811 INFO:journalctl@ceph.osd.9.smithi089.stdout:Sep 21 03:44:06 smithi089 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-9[65404]: 2024-09-21T03:44:06.471+0000 7f6c4e7c5640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2024-09-21T03:43:18.845419+0000 front 2024-09-21T03:43:18.845455+0000 (oldest deadline 2024-09-21T03:43:44.745556+0000) 2024-09-21T03:44:06.811 INFO:journalctl@ceph.osd.9.smithi089.stdout:Sep 21 03:44:06 smithi089 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-9[65404]: 2024-09-21T03:44:06.471+0000 7f6c4e7c5640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-21T03:43:24.746023+0000 front 2024-09-21T03:43:24.745970+0000 (oldest deadline 2024-09-21T03:43:50.646080+0000) 2024-09-21T03:44:06.811 INFO:journalctl@ceph.osd.9.smithi089.stdout:Sep 21 03:44:06 smithi089 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-9[65404]: 2024-09-21T03:44:06.471+0000 7f6c4e7c5640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.114:6804 osd.2 since back 2024-09-21T03:43:34.147460+0000 front 2024-09-21T03:43:34.147364+0000 (oldest deadline 2024-09-21T03:43:58.247427+0000) 2024-09-21T03:44:07.500 INFO:journalctl@ceph.osd.7.smithi110.stdout:Sep 21 03:44:07 smithi110 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-7[50481]: 2024-09-21T03:44:07.431+0000 7fb9f2345640 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-21T03:43:29.081492+0000 front 2024-09-21T03:43:29.081521+0000 (oldest deadline 2024-09-21T03:43:53.781711+0000) 2024-09-21T03:44:07.500 INFO:journalctl@ceph.osd.7.smithi110.stdout:Sep 21 03:44:07 smithi110 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-7[50481]: 2024-09-21T03:44:07.431+0000 7fb9f2345640 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.114:6804 osd.2 since back 2024-09-21T03:43:37.283011+0000 front 2024-09-21T03:43:37.282954+0000 (oldest deadline 2024-09-21T03:43:59.582792+0000) 2024-09-21T03:44:07.501 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 21 03:44:07 smithi110 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-10[57089]: 2024-09-21T03:44:07.190+0000 7f9d479dc640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2024-09-21T03:43:21.264189+0000 front 2024-09-21T03:43:21.264118+0000 (oldest deadline 2024-09-21T03:43:43.564295+0000) 2024-09-21T03:44:07.501 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 21 03:44:07 smithi110 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-10[57089]: 2024-09-21T03:44:07.190+0000 7f9d479dc640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-21T03:43:25.265351+0000 front 2024-09-21T03:43:25.265386+0000 (oldest deadline 2024-09-21T03:43:49.965504+0000) 2024-09-21T03:44:07.501 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 21 03:44:07 smithi110 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-10[57089]: 2024-09-21T03:44:07.190+0000 7f9d479dc640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.114:6804 osd.2 since back 2024-09-21T03:43:33.966726+0000 front 2024-09-21T03:43:33.966884+0000 (oldest deadline 2024-09-21T03:43:59.866510+0000) 2024-09-21T03:44:07.501 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 21 03:44:07 smithi110 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-10[57089]: 2024-09-21T03:44:07.190+0000 7f9d479dc640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.89:6814 osd.3 since back 2024-09-21T03:43:43.367532+0000 front 2024-09-21T03:43:43.367626+0000 (oldest deadline 2024-09-21T03:44:06.867392+0000) 2024-09-21T03:44:07.711 INFO:journalctl@ceph.osd.8.smithi114.stdout:Sep 21 03:44:07 smithi114 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-8[51702]: 2024-09-21T03:44:07.377+0000 7f3e66f25640 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-21T03:43:23.926943+0000 front 2024-09-21T03:43:23.926870+0000 (oldest deadline 2024-09-21T03:43:52.127434+0000) 2024-09-21T03:44:07.711 INFO:journalctl@ceph.osd.8.smithi114.stdout:Sep 21 03:44:07 smithi114 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-8[51702]: 2024-09-21T03:44:07.377+0000 7f3e66f25640 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.114:6804 osd.2 since back 2024-09-21T03:43:32.128066+0000 front 2024-09-21T03:43:32.127963+0000 (oldest deadline 2024-09-21T03:43:58.028025+0000) 2024-09-21T03:44:07.711 INFO:journalctl@ceph.osd.11.smithi114.stdout:Sep 21 03:44:07 smithi114 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-11[58313]: 2024-09-21T03:44:07.241+0000 7f7657ce9640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2024-09-21T03:43:19.721069+0000 front 2024-09-21T03:43:19.721119+0000 (oldest deadline 2024-09-21T03:43:47.321877+0000) 2024-09-21T03:44:07.711 INFO:journalctl@ceph.osd.11.smithi114.stdout:Sep 21 03:44:07 smithi114 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-11[58313]: 2024-09-21T03:44:07.241+0000 7f7657ce9640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-21T03:43:27.322375+0000 front 2024-09-21T03:43:27.322426+0000 (oldest deadline 2024-09-21T03:43:51.422473+0000) 2024-09-21T03:44:07.711 INFO:journalctl@ceph.osd.11.smithi114.stdout:Sep 21 03:44:07 smithi114 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-11[58313]: 2024-09-21T03:44:07.241+0000 7f7657ce9640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.114:6804 osd.2 since back 2024-09-21T03:43:37.224230+0000 front 2024-09-21T03:43:37.224148+0000 (oldest deadline 2024-09-21T03:43:58.924202+0000) 2024-09-21T03:44:07.811 INFO:journalctl@ceph.osd.6.smithi089.stdout:Sep 21 03:44:07 smithi089 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-6[59191]: 2024-09-21T03:44:07.422+0000 7f845162a640 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-21T03:43:28.381812+0000 front 2024-09-21T03:43:28.381946+0000 (oldest deadline 2024-09-21T03:43:49.982463+0000) 2024-09-21T03:44:07.811 INFO:journalctl@ceph.osd.6.smithi089.stdout:Sep 21 03:44:07 smithi089 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-6[59191]: 2024-09-21T03:44:07.422+0000 7f845162a640 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.114:6804 osd.2 since back 2024-09-21T03:43:33.984420+0000 front 2024-09-21T03:43:33.984313+0000 (oldest deadline 2024-09-21T03:44:00.384680+0000) 2024-09-21T03:44:07.811 INFO:journalctl@ceph.osd.6.smithi089.stdout:Sep 21 03:44:07 smithi089 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-6[59191]: 2024-09-21T03:44:07.422+0000 7f845162a640 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.89:6814 osd.3 since back 2024-09-21T03:43:44.485473+0000 front 2024-09-21T03:43:44.485403+0000 (oldest deadline 2024-09-21T03:44:07.385548+0000) 2024-09-21T03:44:07.811 INFO:journalctl@ceph.osd.9.smithi089.stdout:Sep 21 03:44:07 smithi089 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-9[65404]: 2024-09-21T03:44:07.502+0000 7f6c4e7c5640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2024-09-21T03:43:18.845419+0000 front 2024-09-21T03:43:18.845455+0000 (oldest deadline 2024-09-21T03:43:44.745556+0000) 2024-09-21T03:44:07.811 INFO:journalctl@ceph.osd.9.smithi089.stdout:Sep 21 03:44:07 smithi089 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-9[65404]: 2024-09-21T03:44:07.502+0000 7f6c4e7c5640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-21T03:43:24.746023+0000 front 2024-09-21T03:43:24.745970+0000 (oldest deadline 2024-09-21T03:43:50.646080+0000) 2024-09-21T03:44:07.811 INFO:journalctl@ceph.osd.9.smithi089.stdout:Sep 21 03:44:07 smithi089 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-9[65404]: 2024-09-21T03:44:07.502+0000 7f6c4e7c5640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.114:6804 osd.2 since back 2024-09-21T03:43:34.147460+0000 front 2024-09-21T03:43:34.147364+0000 (oldest deadline 2024-09-21T03:43:58.247427+0000) 2024-09-21T03:44:08.500 INFO:journalctl@ceph.osd.7.smithi110.stdout:Sep 21 03:44:08 smithi110 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-7[50481]: 2024-09-21T03:44:08.393+0000 7fb9f2345640 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-21T03:43:29.081492+0000 front 2024-09-21T03:43:29.081521+0000 (oldest deadline 2024-09-21T03:43:53.781711+0000) 2024-09-21T03:44:08.500 INFO:journalctl@ceph.osd.7.smithi110.stdout:Sep 21 03:44:08 smithi110 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-7[50481]: 2024-09-21T03:44:08.393+0000 7fb9f2345640 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.114:6804 osd.2 since back 2024-09-21T03:43:37.283011+0000 front 2024-09-21T03:43:37.282954+0000 (oldest deadline 2024-09-21T03:43:59.582792+0000) 2024-09-21T03:44:08.500 INFO:journalctl@ceph.osd.7.smithi110.stdout:Sep 21 03:44:08 smithi110 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-7[50481]: 2024-09-21T03:44:08.393+0000 7fb9f2345640 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.89:6814 osd.3 since back 2024-09-21T03:43:39.583357+0000 front 2024-09-21T03:43:39.583459+0000 (oldest deadline 2024-09-21T03:44:08.383601+0000) 2024-09-21T03:44:08.500 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 21 03:44:08 smithi110 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-10[57089]: 2024-09-21T03:44:08.199+0000 7f9d479dc640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2024-09-21T03:43:21.264189+0000 front 2024-09-21T03:43:21.264118+0000 (oldest deadline 2024-09-21T03:43:43.564295+0000) 2024-09-21T03:44:08.500 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 21 03:44:08 smithi110 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-10[57089]: 2024-09-21T03:44:08.199+0000 7f9d479dc640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-21T03:43:25.265351+0000 front 2024-09-21T03:43:25.265386+0000 (oldest deadline 2024-09-21T03:43:49.965504+0000) 2024-09-21T03:44:08.500 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 21 03:44:08 smithi110 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-10[57089]: 2024-09-21T03:44:08.199+0000 7f9d479dc640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.114:6804 osd.2 since back 2024-09-21T03:43:33.966726+0000 front 2024-09-21T03:43:33.966884+0000 (oldest deadline 2024-09-21T03:43:59.866510+0000) 2024-09-21T03:44:08.500 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 21 03:44:08 smithi110 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-10[57089]: 2024-09-21T03:44:08.199+0000 7f9d479dc640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.89:6814 osd.3 since back 2024-09-21T03:43:43.367532+0000 front 2024-09-21T03:43:43.367626+0000 (oldest deadline 2024-09-21T03:44:06.867392+0000) 2024-09-21T03:44:08.612 INFO:journalctl@ceph.osd.8.smithi114.stdout:Sep 21 03:44:08 smithi114 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-8[51702]: 2024-09-21T03:44:08.417+0000 7f3e66f25640 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-21T03:43:23.926943+0000 front 2024-09-21T03:43:23.926870+0000 (oldest deadline 2024-09-21T03:43:52.127434+0000) 2024-09-21T03:44:08.612 INFO:journalctl@ceph.osd.8.smithi114.stdout:Sep 21 03:44:08 smithi114 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-8[51702]: 2024-09-21T03:44:08.417+0000 7f3e66f25640 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.114:6804 osd.2 since back 2024-09-21T03:43:32.128066+0000 front 2024-09-21T03:43:32.127963+0000 (oldest deadline 2024-09-21T03:43:58.028025+0000) 2024-09-21T03:44:08.612 INFO:journalctl@ceph.osd.11.smithi114.stdout:Sep 21 03:44:08 smithi114 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-11[58313]: 2024-09-21T03:44:08.210+0000 7f7657ce9640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2024-09-21T03:43:19.721069+0000 front 2024-09-21T03:43:19.721119+0000 (oldest deadline 2024-09-21T03:43:47.321877+0000) 2024-09-21T03:44:08.612 INFO:journalctl@ceph.osd.11.smithi114.stdout:Sep 21 03:44:08 smithi114 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-11[58313]: 2024-09-21T03:44:08.210+0000 7f7657ce9640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-21T03:43:27.322375+0000 front 2024-09-21T03:43:27.322426+0000 (oldest deadline 2024-09-21T03:43:51.422473+0000) 2024-09-21T03:44:08.613 INFO:journalctl@ceph.osd.11.smithi114.stdout:Sep 21 03:44:08 smithi114 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-11[58313]: 2024-09-21T03:44:08.210+0000 7f7657ce9640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.114:6804 osd.2 since back 2024-09-21T03:43:37.224230+0000 front 2024-09-21T03:43:37.224148+0000 (oldest deadline 2024-09-21T03:43:58.924202+0000) 2024-09-21T03:44:08.811 INFO:journalctl@ceph.osd.6.smithi089.stdout:Sep 21 03:44:08 smithi089 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-6[59191]: 2024-09-21T03:44:08.447+0000 7f845162a640 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-21T03:43:28.381812+0000 front 2024-09-21T03:43:28.381946+0000 (oldest deadline 2024-09-21T03:43:49.982463+0000) 2024-09-21T03:44:08.811 INFO:journalctl@ceph.osd.6.smithi089.stdout:Sep 21 03:44:08 smithi089 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-6[59191]: 2024-09-21T03:44:08.447+0000 7f845162a640 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.114:6804 osd.2 since back 2024-09-21T03:43:33.984420+0000 front 2024-09-21T03:43:33.984313+0000 (oldest deadline 2024-09-21T03:44:00.384680+0000) 2024-09-21T03:44:08.811 INFO:journalctl@ceph.osd.6.smithi089.stdout:Sep 21 03:44:08 smithi089 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-6[59191]: 2024-09-21T03:44:08.447+0000 7f845162a640 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.89:6814 osd.3 since back 2024-09-21T03:43:44.485473+0000 front 2024-09-21T03:43:44.485403+0000 (oldest deadline 2024-09-21T03:44:07.385548+0000) 2024-09-21T03:44:08.811 INFO:journalctl@ceph.osd.9.smithi089.stdout:Sep 21 03:44:08 smithi089 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-9[65404]: 2024-09-21T03:44:08.538+0000 7f6c4e7c5640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2024-09-21T03:43:18.845419+0000 front 2024-09-21T03:43:18.845455+0000 (oldest deadline 2024-09-21T03:43:44.745556+0000) 2024-09-21T03:44:08.812 INFO:journalctl@ceph.osd.9.smithi089.stdout:Sep 21 03:44:08 smithi089 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-9[65404]: 2024-09-21T03:44:08.538+0000 7f6c4e7c5640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-21T03:43:24.746023+0000 front 2024-09-21T03:43:24.745970+0000 (oldest deadline 2024-09-21T03:43:50.646080+0000) 2024-09-21T03:44:08.812 INFO:journalctl@ceph.osd.9.smithi089.stdout:Sep 21 03:44:08 smithi089 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-9[65404]: 2024-09-21T03:44:08.538+0000 7f6c4e7c5640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.114:6804 osd.2 since back 2024-09-21T03:43:34.147460+0000 front 2024-09-21T03:43:34.147364+0000 (oldest deadline 2024-09-21T03:43:58.247427+0000) 2024-09-21T03:44:09.500 INFO:journalctl@ceph.osd.7.smithi110.stdout:Sep 21 03:44:09 smithi110 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-7[50481]: 2024-09-21T03:44:09.351+0000 7fb9f2345640 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-21T03:43:29.081492+0000 front 2024-09-21T03:43:29.081521+0000 (oldest deadline 2024-09-21T03:43:53.781711+0000) 2024-09-21T03:44:09.500 INFO:journalctl@ceph.osd.7.smithi110.stdout:Sep 21 03:44:09 smithi110 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-7[50481]: 2024-09-21T03:44:09.351+0000 7fb9f2345640 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.114:6804 osd.2 since back 2024-09-21T03:43:37.283011+0000 front 2024-09-21T03:43:37.282954+0000 (oldest deadline 2024-09-21T03:43:59.582792+0000) 2024-09-21T03:44:09.500 INFO:journalctl@ceph.osd.7.smithi110.stdout:Sep 21 03:44:09 smithi110 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-7[50481]: 2024-09-21T03:44:09.351+0000 7fb9f2345640 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.89:6814 osd.3 since back 2024-09-21T03:43:39.583357+0000 front 2024-09-21T03:43:39.583459+0000 (oldest deadline 2024-09-21T03:44:08.383601+0000) 2024-09-21T03:44:09.500 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 21 03:44:09 smithi110 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-10[57089]: 2024-09-21T03:44:09.217+0000 7f9d479dc640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2024-09-21T03:43:21.264189+0000 front 2024-09-21T03:43:21.264118+0000 (oldest deadline 2024-09-21T03:43:43.564295+0000) 2024-09-21T03:44:09.501 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 21 03:44:09 smithi110 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-10[57089]: 2024-09-21T03:44:09.217+0000 7f9d479dc640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-21T03:43:25.265351+0000 front 2024-09-21T03:43:25.265386+0000 (oldest deadline 2024-09-21T03:43:49.965504+0000) 2024-09-21T03:44:09.501 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 21 03:44:09 smithi110 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-10[57089]: 2024-09-21T03:44:09.217+0000 7f9d479dc640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.114:6804 osd.2 since back 2024-09-21T03:43:33.966726+0000 front 2024-09-21T03:43:33.966884+0000 (oldest deadline 2024-09-21T03:43:59.866510+0000) 2024-09-21T03:44:09.501 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 21 03:44:09 smithi110 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-10[57089]: 2024-09-21T03:44:09.217+0000 7f9d479dc640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.89:6814 osd.3 since back 2024-09-21T03:43:43.367532+0000 front 2024-09-21T03:43:43.367626+0000 (oldest deadline 2024-09-21T03:44:06.867392+0000) 2024-09-21T03:44:09.635 INFO:journalctl@ceph.osd.8.smithi114.stdout:Sep 21 03:44:09 smithi114 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-8[51702]: 2024-09-21T03:44:09.439+0000 7f3e66f25640 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-21T03:43:23.926943+0000 front 2024-09-21T03:43:23.926870+0000 (oldest deadline 2024-09-21T03:43:52.127434+0000) 2024-09-21T03:44:09.635 INFO:journalctl@ceph.osd.8.smithi114.stdout:Sep 21 03:44:09 smithi114 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-8[51702]: 2024-09-21T03:44:09.439+0000 7f3e66f25640 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.114:6804 osd.2 since back 2024-09-21T03:43:32.128066+0000 front 2024-09-21T03:43:32.127963+0000 (oldest deadline 2024-09-21T03:43:58.028025+0000) 2024-09-21T03:44:09.635 INFO:journalctl@ceph.osd.11.smithi114.stdout:Sep 21 03:44:09 smithi114 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-11[58313]: 2024-09-21T03:44:09.245+0000 7f7657ce9640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2024-09-21T03:43:19.721069+0000 front 2024-09-21T03:43:19.721119+0000 (oldest deadline 2024-09-21T03:43:47.321877+0000) 2024-09-21T03:44:09.635 INFO:journalctl@ceph.osd.11.smithi114.stdout:Sep 21 03:44:09 smithi114 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-11[58313]: 2024-09-21T03:44:09.245+0000 7f7657ce9640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-21T03:43:27.322375+0000 front 2024-09-21T03:43:27.322426+0000 (oldest deadline 2024-09-21T03:43:51.422473+0000) 2024-09-21T03:44:09.635 INFO:journalctl@ceph.osd.11.smithi114.stdout:Sep 21 03:44:09 smithi114 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-11[58313]: 2024-09-21T03:44:09.245+0000 7f7657ce9640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.114:6804 osd.2 since back 2024-09-21T03:43:37.224230+0000 front 2024-09-21T03:43:37.224148+0000 (oldest deadline 2024-09-21T03:43:58.924202+0000) 2024-09-21T03:44:09.811 INFO:journalctl@ceph.osd.9.smithi089.stdout:Sep 21 03:44:09 smithi089 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-9[65404]: 2024-09-21T03:44:09.530+0000 7f6c4e7c5640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2024-09-21T03:43:18.845419+0000 front 2024-09-21T03:43:18.845455+0000 (oldest deadline 2024-09-21T03:43:44.745556+0000) 2024-09-21T03:44:09.811 INFO:journalctl@ceph.osd.9.smithi089.stdout:Sep 21 03:44:09 smithi089 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-9[65404]: 2024-09-21T03:44:09.530+0000 7f6c4e7c5640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-21T03:43:24.746023+0000 front 2024-09-21T03:43:24.745970+0000 (oldest deadline 2024-09-21T03:43:50.646080+0000) 2024-09-21T03:44:09.811 INFO:journalctl@ceph.osd.9.smithi089.stdout:Sep 21 03:44:09 smithi089 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-9[65404]: 2024-09-21T03:44:09.530+0000 7f6c4e7c5640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.114:6804 osd.2 since back 2024-09-21T03:43:34.147460+0000 front 2024-09-21T03:43:34.147364+0000 (oldest deadline 2024-09-21T03:43:58.247427+0000) 2024-09-21T03:44:10.110 INFO:journalctl@ceph.osd.6.smithi089.stdout:Sep 21 03:44:09 smithi089 podman[92055]: 2024-09-21 03:44:09.817507981 +0000 UTC m=+5.623626169 container died 3a375799f7366c0586e3874975f6cdffb4b98e2995b2786d06244e6d80c17e7f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-6, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, ceph=True, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-21T03:44:10.461 INFO:journalctl@ceph.osd.8.smithi114.stdout:Sep 21 03:44:10 smithi114 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-8[51702]: 2024-09-21T03:44:10.391+0000 7f3e66f25640 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-21T03:43:23.926943+0000 front 2024-09-21T03:43:23.926870+0000 (oldest deadline 2024-09-21T03:43:52.127434+0000) 2024-09-21T03:44:10.461 INFO:journalctl@ceph.osd.8.smithi114.stdout:Sep 21 03:44:10 smithi114 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-8[51702]: 2024-09-21T03:44:10.391+0000 7f3e66f25640 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.114:6804 osd.2 since back 2024-09-21T03:43:32.128066+0000 front 2024-09-21T03:43:32.127963+0000 (oldest deadline 2024-09-21T03:43:58.028025+0000) 2024-09-21T03:44:10.461 INFO:journalctl@ceph.osd.8.smithi114.stdout:Sep 21 03:44:10 smithi114 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-8[51702]: 2024-09-21T03:44:10.391+0000 7f3e66f25640 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.89:6814 osd.3 since back 2024-09-21T03:43:45.029373+0000 front 2024-09-21T03:43:45.029438+0000 (oldest deadline 2024-09-21T03:44:10.329391+0000) 2024-09-21T03:44:10.461 INFO:journalctl@ceph.osd.11.smithi114.stdout:Sep 21 03:44:10 smithi114 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-11[58313]: 2024-09-21T03:44:10.208+0000 7f7657ce9640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2024-09-21T03:43:19.721069+0000 front 2024-09-21T03:43:19.721119+0000 (oldest deadline 2024-09-21T03:43:47.321877+0000) 2024-09-21T03:44:10.461 INFO:journalctl@ceph.osd.11.smithi114.stdout:Sep 21 03:44:10 smithi114 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-11[58313]: 2024-09-21T03:44:10.208+0000 7f7657ce9640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-21T03:43:27.322375+0000 front 2024-09-21T03:43:27.322426+0000 (oldest deadline 2024-09-21T03:43:51.422473+0000) 2024-09-21T03:44:10.461 INFO:journalctl@ceph.osd.11.smithi114.stdout:Sep 21 03:44:10 smithi114 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-11[58313]: 2024-09-21T03:44:10.208+0000 7f7657ce9640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.114:6804 osd.2 since back 2024-09-21T03:43:37.224230+0000 front 2024-09-21T03:43:37.224148+0000 (oldest deadline 2024-09-21T03:43:58.924202+0000) 2024-09-21T03:44:10.500 INFO:journalctl@ceph.osd.7.smithi110.stdout:Sep 21 03:44:10 smithi110 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-7[50481]: 2024-09-21T03:44:10.328+0000 7fb9f2345640 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-21T03:43:29.081492+0000 front 2024-09-21T03:43:29.081521+0000 (oldest deadline 2024-09-21T03:43:53.781711+0000) 2024-09-21T03:44:10.500 INFO:journalctl@ceph.osd.7.smithi110.stdout:Sep 21 03:44:10 smithi110 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-7[50481]: 2024-09-21T03:44:10.328+0000 7fb9f2345640 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.114:6804 osd.2 since back 2024-09-21T03:43:37.283011+0000 front 2024-09-21T03:43:37.282954+0000 (oldest deadline 2024-09-21T03:43:59.582792+0000) 2024-09-21T03:44:10.500 INFO:journalctl@ceph.osd.7.smithi110.stdout:Sep 21 03:44:10 smithi110 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-7[50481]: 2024-09-21T03:44:10.328+0000 7fb9f2345640 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.89:6814 osd.3 since back 2024-09-21T03:43:39.583357+0000 front 2024-09-21T03:43:39.583459+0000 (oldest deadline 2024-09-21T03:44:08.383601+0000) 2024-09-21T03:44:10.500 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 21 03:44:10 smithi110 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-10[57089]: 2024-09-21T03:44:10.199+0000 7f9d479dc640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2024-09-21T03:43:21.264189+0000 front 2024-09-21T03:43:21.264118+0000 (oldest deadline 2024-09-21T03:43:43.564295+0000) 2024-09-21T03:44:10.500 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 21 03:44:10 smithi110 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-10[57089]: 2024-09-21T03:44:10.199+0000 7f9d479dc640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-21T03:43:25.265351+0000 front 2024-09-21T03:43:25.265386+0000 (oldest deadline 2024-09-21T03:43:49.965504+0000) 2024-09-21T03:44:10.500 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 21 03:44:10 smithi110 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-10[57089]: 2024-09-21T03:44:10.199+0000 7f9d479dc640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.114:6804 osd.2 since back 2024-09-21T03:43:33.966726+0000 front 2024-09-21T03:43:33.966884+0000 (oldest deadline 2024-09-21T03:43:59.866510+0000) 2024-09-21T03:44:10.500 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 21 03:44:10 smithi110 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-10[57089]: 2024-09-21T03:44:10.199+0000 7f9d479dc640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.89:6814 osd.3 since back 2024-09-21T03:43:43.367532+0000 front 2024-09-21T03:43:43.367626+0000 (oldest deadline 2024-09-21T03:44:06.867392+0000) 2024-09-21T03:44:10.520 INFO:journalctl@ceph.osd.6.smithi089.stdout:Sep 21 03:44:10 smithi089 podman[92055]: 2024-09-21 03:44:10.334486299 +0000 UTC m=+6.140604532 container cleanup 3a375799f7366c0586e3874975f6cdffb4b98e2995b2786d06244e6d80c17e7f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-6, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221) 2024-09-21T03:44:10.521 INFO:journalctl@ceph.osd.6.smithi089.stdout:Sep 21 03:44:10 smithi089 bash[92055]: ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-6 2024-09-21T03:44:10.811 INFO:journalctl@ceph.osd.6.smithi089.stdout:Sep 21 03:44:10 smithi089 podman[92071]: 2024-09-21 03:44:10.51931392 +0000 UTC m=+0.694582507 container remove 3a375799f7366c0586e3874975f6cdffb4b98e2995b2786d06244e6d80c17e7f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-6, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.build-date=20240731, ceph=True) 2024-09-21T03:44:10.811 INFO:journalctl@ceph.osd.9.smithi089.stdout:Sep 21 03:44:10 smithi089 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-9[65404]: 2024-09-21T03:44:10.566+0000 7f6c4e7c5640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2024-09-21T03:43:18.845419+0000 front 2024-09-21T03:43:18.845455+0000 (oldest deadline 2024-09-21T03:43:44.745556+0000) 2024-09-21T03:44:10.811 INFO:journalctl@ceph.osd.9.smithi089.stdout:Sep 21 03:44:10 smithi089 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-9[65404]: 2024-09-21T03:44:10.566+0000 7f6c4e7c5640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-21T03:43:24.746023+0000 front 2024-09-21T03:43:24.745970+0000 (oldest deadline 2024-09-21T03:43:50.646080+0000) 2024-09-21T03:44:10.811 INFO:journalctl@ceph.osd.9.smithi089.stdout:Sep 21 03:44:10 smithi089 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-9[65404]: 2024-09-21T03:44:10.566+0000 7f6c4e7c5640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.114:6804 osd.2 since back 2024-09-21T03:43:34.147460+0000 front 2024-09-21T03:43:34.147364+0000 (oldest deadline 2024-09-21T03:43:58.247427+0000) 2024-09-21T03:44:10.812 INFO:journalctl@ceph.osd.9.smithi089.stdout:Sep 21 03:44:10 smithi089 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-9[65404]: 2024-09-21T03:44:10.566+0000 7f6c4e7c5640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.89:6814 osd.3 since back 2024-09-21T03:43:39.848901+0000 front 2024-09-21T03:43:39.848935+0000 (oldest deadline 2024-09-21T03:44:09.849669+0000) 2024-09-21T03:44:11.216 INFO:journalctl@ceph.osd.6.smithi089.stdout:Sep 21 03:44:10 smithi089 podman[92160]: 2024-09-21 03:44:10.829381903 +0000 UTC m=+0.015820048 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T03:44:11.216 INFO:journalctl@ceph.osd.6.smithi089.stdout:Sep 21 03:44:10 smithi089 podman[92160]: 2024-09-21 03:44:10.937083642 +0000 UTC m=+0.123521832 container create 8e0dd20bf93dafb08130ffeeca900bb37955b91af6e43bfd52d37229ca4dadb0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-6-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240731, GIT_CLEAN=True, io.buildah.version=1.37.2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS) 2024-09-21T03:44:11.461 INFO:journalctl@ceph.osd.8.smithi114.stdout:Sep 21 03:44:11 smithi114 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-8[51702]: 2024-09-21T03:44:11.394+0000 7f3e66f25640 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-21T03:43:23.926943+0000 front 2024-09-21T03:43:23.926870+0000 (oldest deadline 2024-09-21T03:43:52.127434+0000) 2024-09-21T03:44:11.461 INFO:journalctl@ceph.osd.8.smithi114.stdout:Sep 21 03:44:11 smithi114 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-8[51702]: 2024-09-21T03:44:11.394+0000 7f3e66f25640 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.114:6804 osd.2 since back 2024-09-21T03:43:32.128066+0000 front 2024-09-21T03:43:32.127963+0000 (oldest deadline 2024-09-21T03:43:58.028025+0000) 2024-09-21T03:44:11.461 INFO:journalctl@ceph.osd.8.smithi114.stdout:Sep 21 03:44:11 smithi114 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-8[51702]: 2024-09-21T03:44:11.394+0000 7f3e66f25640 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.89:6814 osd.3 since back 2024-09-21T03:43:45.029373+0000 front 2024-09-21T03:43:45.029438+0000 (oldest deadline 2024-09-21T03:44:10.329391+0000) 2024-09-21T03:44:11.461 INFO:journalctl@ceph.osd.11.smithi114.stdout:Sep 21 03:44:11 smithi114 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-11[58313]: 2024-09-21T03:44:11.193+0000 7f7657ce9640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2024-09-21T03:43:19.721069+0000 front 2024-09-21T03:43:19.721119+0000 (oldest deadline 2024-09-21T03:43:47.321877+0000) 2024-09-21T03:44:11.461 INFO:journalctl@ceph.osd.11.smithi114.stdout:Sep 21 03:44:11 smithi114 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-11[58313]: 2024-09-21T03:44:11.193+0000 7f7657ce9640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-21T03:43:27.322375+0000 front 2024-09-21T03:43:27.322426+0000 (oldest deadline 2024-09-21T03:43:51.422473+0000) 2024-09-21T03:44:11.461 INFO:journalctl@ceph.osd.11.smithi114.stdout:Sep 21 03:44:11 smithi114 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-11[58313]: 2024-09-21T03:44:11.193+0000 7f7657ce9640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.114:6804 osd.2 since back 2024-09-21T03:43:37.224230+0000 front 2024-09-21T03:43:37.224148+0000 (oldest deadline 2024-09-21T03:43:58.924202+0000) 2024-09-21T03:44:11.500 INFO:journalctl@ceph.osd.7.smithi110.stdout:Sep 21 03:44:11 smithi110 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-7[50481]: 2024-09-21T03:44:11.357+0000 7fb9f2345640 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-21T03:43:29.081492+0000 front 2024-09-21T03:43:29.081521+0000 (oldest deadline 2024-09-21T03:43:53.781711+0000) 2024-09-21T03:44:11.500 INFO:journalctl@ceph.osd.7.smithi110.stdout:Sep 21 03:44:11 smithi110 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-7[50481]: 2024-09-21T03:44:11.357+0000 7fb9f2345640 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.114:6804 osd.2 since back 2024-09-21T03:43:37.283011+0000 front 2024-09-21T03:43:37.282954+0000 (oldest deadline 2024-09-21T03:43:59.582792+0000) 2024-09-21T03:44:11.500 INFO:journalctl@ceph.osd.7.smithi110.stdout:Sep 21 03:44:11 smithi110 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-7[50481]: 2024-09-21T03:44:11.357+0000 7fb9f2345640 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.89:6814 osd.3 since back 2024-09-21T03:43:39.583357+0000 front 2024-09-21T03:43:39.583459+0000 (oldest deadline 2024-09-21T03:44:08.383601+0000) 2024-09-21T03:44:11.501 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 21 03:44:11 smithi110 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-10[57089]: 2024-09-21T03:44:11.189+0000 7f9d479dc640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2024-09-21T03:43:21.264189+0000 front 2024-09-21T03:43:21.264118+0000 (oldest deadline 2024-09-21T03:43:43.564295+0000) 2024-09-21T03:44:11.501 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 21 03:44:11 smithi110 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-10[57089]: 2024-09-21T03:44:11.189+0000 7f9d479dc640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-21T03:43:25.265351+0000 front 2024-09-21T03:43:25.265386+0000 (oldest deadline 2024-09-21T03:43:49.965504+0000) 2024-09-21T03:44:11.501 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 21 03:44:11 smithi110 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-10[57089]: 2024-09-21T03:44:11.189+0000 7f9d479dc640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.114:6804 osd.2 since back 2024-09-21T03:43:33.966726+0000 front 2024-09-21T03:43:33.966884+0000 (oldest deadline 2024-09-21T03:43:59.866510+0000) 2024-09-21T03:44:11.501 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 21 03:44:11 smithi110 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-10[57089]: 2024-09-21T03:44:11.189+0000 7f9d479dc640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.89:6814 osd.3 since back 2024-09-21T03:43:43.367532+0000 front 2024-09-21T03:43:43.367626+0000 (oldest deadline 2024-09-21T03:44:06.867392+0000) 2024-09-21T03:44:11.561 INFO:journalctl@ceph.osd.6.smithi089.stdout:Sep 21 03:44:11 smithi089 podman[92160]: 2024-09-21 03:44:11.214989261 +0000 UTC m=+0.401427414 container init 8e0dd20bf93dafb08130ffeeca900bb37955b91af6e43bfd52d37229ca4dadb0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-6-deactivate, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS) 2024-09-21T03:44:11.562 INFO:journalctl@ceph.osd.6.smithi089.stdout:Sep 21 03:44:11 smithi089 podman[92160]: 2024-09-21 03:44:11.218626353 +0000 UTC m=+0.405064507 container start 8e0dd20bf93dafb08130ffeeca900bb37955b91af6e43bfd52d37229ca4dadb0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-6-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , ceph=True) 2024-09-21T03:44:11.562 INFO:journalctl@ceph.osd.6.smithi089.stdout:Sep 21 03:44:11 smithi089 podman[92160]: 2024-09-21 03:44:11.323507706 +0000 UTC m=+0.509945868 container attach 8e0dd20bf93dafb08130ffeeca900bb37955b91af6e43bfd52d37229ca4dadb0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-6-deactivate, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux ) 2024-09-21T03:44:11.562 INFO:journalctl@ceph.osd.6.smithi089.stdout:Sep 21 03:44:11 smithi089 podman[92160]: 2024-09-21 03:44:11.406802047 +0000 UTC m=+0.593240193 container died 8e0dd20bf93dafb08130ffeeca900bb37955b91af6e43bfd52d37229ca4dadb0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-6-deactivate, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2) 2024-09-21T03:44:12.061 INFO:journalctl@ceph.osd.9.smithi089.stdout:Sep 21 03:44:11 smithi089 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-9[65404]: 2024-09-21T03:44:11.600+0000 7f6c4e7c5640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2024-09-21T03:43:18.845419+0000 front 2024-09-21T03:43:18.845455+0000 (oldest deadline 2024-09-21T03:43:44.745556+0000) 2024-09-21T03:44:12.061 INFO:journalctl@ceph.osd.9.smithi089.stdout:Sep 21 03:44:11 smithi089 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-9[65404]: 2024-09-21T03:44:11.600+0000 7f6c4e7c5640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-21T03:43:24.746023+0000 front 2024-09-21T03:43:24.745970+0000 (oldest deadline 2024-09-21T03:43:50.646080+0000) 2024-09-21T03:44:12.061 INFO:journalctl@ceph.osd.9.smithi089.stdout:Sep 21 03:44:11 smithi089 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-9[65404]: 2024-09-21T03:44:11.600+0000 7f6c4e7c5640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.114:6804 osd.2 since back 2024-09-21T03:43:34.147460+0000 front 2024-09-21T03:43:34.147364+0000 (oldest deadline 2024-09-21T03:43:58.247427+0000) 2024-09-21T03:44:12.061 INFO:journalctl@ceph.osd.9.smithi089.stdout:Sep 21 03:44:11 smithi089 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-9[65404]: 2024-09-21T03:44:11.600+0000 7f6c4e7c5640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.89:6814 osd.3 since back 2024-09-21T03:43:39.848901+0000 front 2024-09-21T03:43:39.848935+0000 (oldest deadline 2024-09-21T03:44:09.849669+0000) 2024-09-21T03:44:12.092 DEBUG:teuthology.orchestra.run.smithi089:> sudo pkill -f 'journalctl -f -n 0 -u ceph-7c374098-77c9-11ef-baf5-efdc52797490@osd.6.service' 2024-09-21T03:44:12.136 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T03:44:12.137 INFO:tasks.cephadm.osd.6:Stopped osd.6 2024-09-21T03:44:12.137 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2024-09-21T03:44:12.137 DEBUG:teuthology.orchestra.run.smithi110:> sudo systemctl stop ceph-7c374098-77c9-11ef-baf5-efdc52797490@osd.7 2024-09-21T03:44:12.438 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 21 03:44:12 smithi110 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-10[57089]: 2024-09-21T03:44:12.180+0000 7f9d479dc640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2024-09-21T03:43:21.264189+0000 front 2024-09-21T03:43:21.264118+0000 (oldest deadline 2024-09-21T03:43:43.564295+0000) 2024-09-21T03:44:12.438 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 21 03:44:12 smithi110 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-10[57089]: 2024-09-21T03:44:12.180+0000 7f9d479dc640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-21T03:43:25.265351+0000 front 2024-09-21T03:43:25.265386+0000 (oldest deadline 2024-09-21T03:43:49.965504+0000) 2024-09-21T03:44:12.438 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 21 03:44:12 smithi110 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-10[57089]: 2024-09-21T03:44:12.180+0000 7f9d479dc640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.114:6804 osd.2 since back 2024-09-21T03:43:33.966726+0000 front 2024-09-21T03:43:33.966884+0000 (oldest deadline 2024-09-21T03:43:59.866510+0000) 2024-09-21T03:44:12.438 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 21 03:44:12 smithi110 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-10[57089]: 2024-09-21T03:44:12.180+0000 7f9d479dc640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.89:6814 osd.3 since back 2024-09-21T03:43:43.367532+0000 front 2024-09-21T03:43:43.367626+0000 (oldest deadline 2024-09-21T03:44:06.867392+0000) 2024-09-21T03:44:12.438 INFO:journalctl@ceph.osd.7.smithi110.stdout:Sep 21 03:44:12 smithi110 systemd[1]: Stopping Ceph osd.7 for 7c374098-77c9-11ef-baf5-efdc52797490... 2024-09-21T03:44:12.438 INFO:journalctl@ceph.osd.7.smithi110.stdout:Sep 21 03:44:12 smithi110 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-7[50481]: 2024-09-21T03:44:12.337+0000 7fb9f2345640 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-21T03:43:29.081492+0000 front 2024-09-21T03:43:29.081521+0000 (oldest deadline 2024-09-21T03:43:53.781711+0000) 2024-09-21T03:44:12.438 INFO:journalctl@ceph.osd.7.smithi110.stdout:Sep 21 03:44:12 smithi110 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-7[50481]: 2024-09-21T03:44:12.337+0000 7fb9f2345640 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.114:6804 osd.2 since back 2024-09-21T03:43:37.283011+0000 front 2024-09-21T03:43:37.282954+0000 (oldest deadline 2024-09-21T03:43:59.582792+0000) 2024-09-21T03:44:12.439 INFO:journalctl@ceph.osd.7.smithi110.stdout:Sep 21 03:44:12 smithi110 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-7[50481]: 2024-09-21T03:44:12.337+0000 7fb9f2345640 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.89:6814 osd.3 since back 2024-09-21T03:43:39.583357+0000 front 2024-09-21T03:43:39.583459+0000 (oldest deadline 2024-09-21T03:44:08.383601+0000) 2024-09-21T03:44:12.439 INFO:journalctl@ceph.osd.7.smithi110.stdout:Sep 21 03:44:12 smithi110 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-7[50481]: 2024-09-21T03:44:12.435+0000 7fb9f9d62640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T03:44:12.439 INFO:journalctl@ceph.osd.7.smithi110.stdout:Sep 21 03:44:12 smithi110 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-7[50481]: 2024-09-21T03:44:12.435+0000 7fb9f9d62640 -1 osd.7 88 *** Got signal Terminated *** 2024-09-21T03:44:12.439 INFO:journalctl@ceph.osd.7.smithi110.stdout:Sep 21 03:44:12 smithi110 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-7[50481]: 2024-09-21T03:44:12.435+0000 7fb9f9d62640 -1 osd.7 88 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-21T03:44:12.461 INFO:journalctl@ceph.osd.8.smithi114.stdout:Sep 21 03:44:12 smithi114 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-8[51702]: 2024-09-21T03:44:12.412+0000 7f3e66f25640 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-21T03:43:23.926943+0000 front 2024-09-21T03:43:23.926870+0000 (oldest deadline 2024-09-21T03:43:52.127434+0000) 2024-09-21T03:44:12.461 INFO:journalctl@ceph.osd.8.smithi114.stdout:Sep 21 03:44:12 smithi114 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-8[51702]: 2024-09-21T03:44:12.412+0000 7f3e66f25640 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.114:6804 osd.2 since back 2024-09-21T03:43:32.128066+0000 front 2024-09-21T03:43:32.127963+0000 (oldest deadline 2024-09-21T03:43:58.028025+0000) 2024-09-21T03:44:12.461 INFO:journalctl@ceph.osd.8.smithi114.stdout:Sep 21 03:44:12 smithi114 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-8[51702]: 2024-09-21T03:44:12.412+0000 7f3e66f25640 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.89:6814 osd.3 since back 2024-09-21T03:43:45.029373+0000 front 2024-09-21T03:43:45.029438+0000 (oldest deadline 2024-09-21T03:44:10.329391+0000) 2024-09-21T03:44:12.461 INFO:journalctl@ceph.osd.11.smithi114.stdout:Sep 21 03:44:12 smithi114 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-11[58313]: 2024-09-21T03:44:12.167+0000 7f7657ce9640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2024-09-21T03:43:19.721069+0000 front 2024-09-21T03:43:19.721119+0000 (oldest deadline 2024-09-21T03:43:47.321877+0000) 2024-09-21T03:44:12.461 INFO:journalctl@ceph.osd.11.smithi114.stdout:Sep 21 03:44:12 smithi114 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-11[58313]: 2024-09-21T03:44:12.167+0000 7f7657ce9640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-21T03:43:27.322375+0000 front 2024-09-21T03:43:27.322426+0000 (oldest deadline 2024-09-21T03:43:51.422473+0000) 2024-09-21T03:44:12.461 INFO:journalctl@ceph.osd.11.smithi114.stdout:Sep 21 03:44:12 smithi114 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-11[58313]: 2024-09-21T03:44:12.167+0000 7f7657ce9640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.114:6804 osd.2 since back 2024-09-21T03:43:37.224230+0000 front 2024-09-21T03:43:37.224148+0000 (oldest deadline 2024-09-21T03:43:58.924202+0000) 2024-09-21T03:44:12.811 INFO:journalctl@ceph.osd.9.smithi089.stdout:Sep 21 03:44:12 smithi089 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-9[65404]: 2024-09-21T03:44:12.577+0000 7f6c4e7c5640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2024-09-21T03:43:18.845419+0000 front 2024-09-21T03:43:18.845455+0000 (oldest deadline 2024-09-21T03:43:44.745556+0000) 2024-09-21T03:44:12.812 INFO:journalctl@ceph.osd.9.smithi089.stdout:Sep 21 03:44:12 smithi089 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-9[65404]: 2024-09-21T03:44:12.577+0000 7f6c4e7c5640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-21T03:43:24.746023+0000 front 2024-09-21T03:43:24.745970+0000 (oldest deadline 2024-09-21T03:43:50.646080+0000) 2024-09-21T03:44:12.812 INFO:journalctl@ceph.osd.9.smithi089.stdout:Sep 21 03:44:12 smithi089 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-9[65404]: 2024-09-21T03:44:12.577+0000 7f6c4e7c5640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.114:6804 osd.2 since back 2024-09-21T03:43:34.147460+0000 front 2024-09-21T03:43:34.147364+0000 (oldest deadline 2024-09-21T03:43:58.247427+0000) 2024-09-21T03:44:12.812 INFO:journalctl@ceph.osd.9.smithi089.stdout:Sep 21 03:44:12 smithi089 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-9[65404]: 2024-09-21T03:44:12.577+0000 7f6c4e7c5640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.89:6814 osd.3 since back 2024-09-21T03:43:39.848901+0000 front 2024-09-21T03:43:39.848935+0000 (oldest deadline 2024-09-21T03:44:09.849669+0000) 2024-09-21T03:44:13.461 INFO:journalctl@ceph.osd.8.smithi114.stdout:Sep 21 03:44:13 smithi114 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-8[51702]: 2024-09-21T03:44:13.420+0000 7f3e66f25640 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-21T03:43:23.926943+0000 front 2024-09-21T03:43:23.926870+0000 (oldest deadline 2024-09-21T03:43:52.127434+0000) 2024-09-21T03:44:13.461 INFO:journalctl@ceph.osd.8.smithi114.stdout:Sep 21 03:44:13 smithi114 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-8[51702]: 2024-09-21T03:44:13.420+0000 7f3e66f25640 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.114:6804 osd.2 since back 2024-09-21T03:43:32.128066+0000 front 2024-09-21T03:43:32.127963+0000 (oldest deadline 2024-09-21T03:43:58.028025+0000) 2024-09-21T03:44:13.461 INFO:journalctl@ceph.osd.8.smithi114.stdout:Sep 21 03:44:13 smithi114 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-8[51702]: 2024-09-21T03:44:13.420+0000 7f3e66f25640 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.89:6814 osd.3 since back 2024-09-21T03:43:45.029373+0000 front 2024-09-21T03:43:45.029438+0000 (oldest deadline 2024-09-21T03:44:10.329391+0000) 2024-09-21T03:44:13.461 INFO:journalctl@ceph.osd.11.smithi114.stdout:Sep 21 03:44:13 smithi114 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-11[58313]: 2024-09-21T03:44:13.177+0000 7f7657ce9640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2024-09-21T03:43:19.721069+0000 front 2024-09-21T03:43:19.721119+0000 (oldest deadline 2024-09-21T03:43:47.321877+0000) 2024-09-21T03:44:13.461 INFO:journalctl@ceph.osd.11.smithi114.stdout:Sep 21 03:44:13 smithi114 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-11[58313]: 2024-09-21T03:44:13.177+0000 7f7657ce9640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-21T03:43:27.322375+0000 front 2024-09-21T03:43:27.322426+0000 (oldest deadline 2024-09-21T03:43:51.422473+0000) 2024-09-21T03:44:13.461 INFO:journalctl@ceph.osd.11.smithi114.stdout:Sep 21 03:44:13 smithi114 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-11[58313]: 2024-09-21T03:44:13.177+0000 7f7657ce9640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.114:6804 osd.2 since back 2024-09-21T03:43:37.224230+0000 front 2024-09-21T03:43:37.224148+0000 (oldest deadline 2024-09-21T03:43:58.924202+0000) 2024-09-21T03:44:13.500 INFO:journalctl@ceph.osd.7.smithi110.stdout:Sep 21 03:44:13 smithi110 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-7[50481]: 2024-09-21T03:44:13.306+0000 7fb9f2345640 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-21T03:43:29.081492+0000 front 2024-09-21T03:43:29.081521+0000 (oldest deadline 2024-09-21T03:43:53.781711+0000) 2024-09-21T03:44:13.501 INFO:journalctl@ceph.osd.7.smithi110.stdout:Sep 21 03:44:13 smithi110 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-7[50481]: 2024-09-21T03:44:13.306+0000 7fb9f2345640 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.114:6804 osd.2 since back 2024-09-21T03:43:37.283011+0000 front 2024-09-21T03:43:37.282954+0000 (oldest deadline 2024-09-21T03:43:59.582792+0000) 2024-09-21T03:44:13.501 INFO:journalctl@ceph.osd.7.smithi110.stdout:Sep 21 03:44:13 smithi110 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-7[50481]: 2024-09-21T03:44:13.306+0000 7fb9f2345640 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.89:6814 osd.3 since back 2024-09-21T03:43:39.583357+0000 front 2024-09-21T03:43:39.583459+0000 (oldest deadline 2024-09-21T03:44:08.383601+0000) 2024-09-21T03:44:13.501 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 21 03:44:13 smithi110 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-10[57089]: 2024-09-21T03:44:13.171+0000 7f9d479dc640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2024-09-21T03:43:21.264189+0000 front 2024-09-21T03:43:21.264118+0000 (oldest deadline 2024-09-21T03:43:43.564295+0000) 2024-09-21T03:44:13.501 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 21 03:44:13 smithi110 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-10[57089]: 2024-09-21T03:44:13.171+0000 7f9d479dc640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-21T03:43:25.265351+0000 front 2024-09-21T03:43:25.265386+0000 (oldest deadline 2024-09-21T03:43:49.965504+0000) 2024-09-21T03:44:13.501 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 21 03:44:13 smithi110 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-10[57089]: 2024-09-21T03:44:13.171+0000 7f9d479dc640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.114:6804 osd.2 since back 2024-09-21T03:43:33.966726+0000 front 2024-09-21T03:43:33.966884+0000 (oldest deadline 2024-09-21T03:43:59.866510+0000) 2024-09-21T03:44:13.502 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 21 03:44:13 smithi110 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-10[57089]: 2024-09-21T03:44:13.171+0000 7f9d479dc640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.89:6814 osd.3 since back 2024-09-21T03:43:43.367532+0000 front 2024-09-21T03:43:43.367626+0000 (oldest deadline 2024-09-21T03:44:06.867392+0000) 2024-09-21T03:44:14.061 INFO:journalctl@ceph.osd.9.smithi089.stdout:Sep 21 03:44:13 smithi089 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-9[65404]: 2024-09-21T03:44:13.564+0000 7f6c4e7c5640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2024-09-21T03:43:18.845419+0000 front 2024-09-21T03:43:18.845455+0000 (oldest deadline 2024-09-21T03:43:44.745556+0000) 2024-09-21T03:44:14.061 INFO:journalctl@ceph.osd.9.smithi089.stdout:Sep 21 03:44:13 smithi089 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-9[65404]: 2024-09-21T03:44:13.564+0000 7f6c4e7c5640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-21T03:43:24.746023+0000 front 2024-09-21T03:43:24.745970+0000 (oldest deadline 2024-09-21T03:43:50.646080+0000) 2024-09-21T03:44:14.061 INFO:journalctl@ceph.osd.9.smithi089.stdout:Sep 21 03:44:13 smithi089 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-9[65404]: 2024-09-21T03:44:13.564+0000 7f6c4e7c5640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.114:6804 osd.2 since back 2024-09-21T03:43:34.147460+0000 front 2024-09-21T03:43:34.147364+0000 (oldest deadline 2024-09-21T03:43:58.247427+0000) 2024-09-21T03:44:14.062 INFO:journalctl@ceph.osd.9.smithi089.stdout:Sep 21 03:44:13 smithi089 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-9[65404]: 2024-09-21T03:44:13.564+0000 7f6c4e7c5640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.89:6814 osd.3 since back 2024-09-21T03:43:39.848901+0000 front 2024-09-21T03:43:39.848935+0000 (oldest deadline 2024-09-21T03:44:09.849669+0000) 2024-09-21T03:44:14.461 INFO:journalctl@ceph.osd.8.smithi114.stdout:Sep 21 03:44:14 smithi114 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-8[51702]: 2024-09-21T03:44:14.378+0000 7f3e66f25640 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-21T03:43:23.926943+0000 front 2024-09-21T03:43:23.926870+0000 (oldest deadline 2024-09-21T03:43:52.127434+0000) 2024-09-21T03:44:14.461 INFO:journalctl@ceph.osd.8.smithi114.stdout:Sep 21 03:44:14 smithi114 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-8[51702]: 2024-09-21T03:44:14.378+0000 7f3e66f25640 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.114:6804 osd.2 since back 2024-09-21T03:43:32.128066+0000 front 2024-09-21T03:43:32.127963+0000 (oldest deadline 2024-09-21T03:43:58.028025+0000) 2024-09-21T03:44:14.461 INFO:journalctl@ceph.osd.8.smithi114.stdout:Sep 21 03:44:14 smithi114 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-8[51702]: 2024-09-21T03:44:14.378+0000 7f3e66f25640 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.89:6814 osd.3 since back 2024-09-21T03:43:45.029373+0000 front 2024-09-21T03:43:45.029438+0000 (oldest deadline 2024-09-21T03:44:10.329391+0000) 2024-09-21T03:44:14.461 INFO:journalctl@ceph.osd.11.smithi114.stdout:Sep 21 03:44:14 smithi114 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-11[58313]: 2024-09-21T03:44:14.129+0000 7f7657ce9640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2024-09-21T03:43:19.721069+0000 front 2024-09-21T03:43:19.721119+0000 (oldest deadline 2024-09-21T03:43:47.321877+0000) 2024-09-21T03:44:14.461 INFO:journalctl@ceph.osd.11.smithi114.stdout:Sep 21 03:44:14 smithi114 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-11[58313]: 2024-09-21T03:44:14.129+0000 7f7657ce9640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-21T03:43:27.322375+0000 front 2024-09-21T03:43:27.322426+0000 (oldest deadline 2024-09-21T03:43:51.422473+0000) 2024-09-21T03:44:14.461 INFO:journalctl@ceph.osd.11.smithi114.stdout:Sep 21 03:44:14 smithi114 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-11[58313]: 2024-09-21T03:44:14.129+0000 7f7657ce9640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.114:6804 osd.2 since back 2024-09-21T03:43:37.224230+0000 front 2024-09-21T03:43:37.224148+0000 (oldest deadline 2024-09-21T03:43:58.924202+0000) 2024-09-21T03:44:14.500 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 21 03:44:14 smithi110 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-10[57089]: 2024-09-21T03:44:14.188+0000 7f9d479dc640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2024-09-21T03:43:21.264189+0000 front 2024-09-21T03:43:21.264118+0000 (oldest deadline 2024-09-21T03:43:43.564295+0000) 2024-09-21T03:44:14.500 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 21 03:44:14 smithi110 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-10[57089]: 2024-09-21T03:44:14.188+0000 7f9d479dc640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-21T03:43:25.265351+0000 front 2024-09-21T03:43:25.265386+0000 (oldest deadline 2024-09-21T03:43:49.965504+0000) 2024-09-21T03:44:14.500 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 21 03:44:14 smithi110 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-10[57089]: 2024-09-21T03:44:14.188+0000 7f9d479dc640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.114:6804 osd.2 since back 2024-09-21T03:43:33.966726+0000 front 2024-09-21T03:43:33.966884+0000 (oldest deadline 2024-09-21T03:43:59.866510+0000) 2024-09-21T03:44:14.500 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 21 03:44:14 smithi110 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-10[57089]: 2024-09-21T03:44:14.188+0000 7f9d479dc640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.89:6814 osd.3 since back 2024-09-21T03:43:43.367532+0000 front 2024-09-21T03:43:43.367626+0000 (oldest deadline 2024-09-21T03:44:06.867392+0000) 2024-09-21T03:44:14.500 INFO:journalctl@ceph.osd.7.smithi110.stdout:Sep 21 03:44:14 smithi110 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-7[50481]: 2024-09-21T03:44:14.356+0000 7fb9f2345640 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-21T03:43:29.081492+0000 front 2024-09-21T03:43:29.081521+0000 (oldest deadline 2024-09-21T03:43:53.781711+0000) 2024-09-21T03:44:14.501 INFO:journalctl@ceph.osd.7.smithi110.stdout:Sep 21 03:44:14 smithi110 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-7[50481]: 2024-09-21T03:44:14.356+0000 7fb9f2345640 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.114:6804 osd.2 since back 2024-09-21T03:43:37.283011+0000 front 2024-09-21T03:43:37.282954+0000 (oldest deadline 2024-09-21T03:43:59.582792+0000) 2024-09-21T03:44:14.501 INFO:journalctl@ceph.osd.7.smithi110.stdout:Sep 21 03:44:14 smithi110 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-7[50481]: 2024-09-21T03:44:14.356+0000 7fb9f2345640 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.89:6814 osd.3 since back 2024-09-21T03:43:39.583357+0000 front 2024-09-21T03:43:39.583459+0000 (oldest deadline 2024-09-21T03:44:08.383601+0000) 2024-09-21T03:44:14.501 INFO:journalctl@ceph.osd.7.smithi110.stdout:Sep 21 03:44:14 smithi110 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-7[50481]: 2024-09-21T03:44:14.356+0000 7fb9f2345640 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.110:6812 osd.4 since back 2024-09-21T03:43:48.383949+0000 front 2024-09-21T03:43:48.384002+0000 (oldest deadline 2024-09-21T03:44:14.284020+0000) 2024-09-21T03:44:14.811 INFO:journalctl@ceph.osd.9.smithi089.stdout:Sep 21 03:44:14 smithi089 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-9[65404]: 2024-09-21T03:44:14.551+0000 7f6c4e7c5640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2024-09-21T03:43:18.845419+0000 front 2024-09-21T03:43:18.845455+0000 (oldest deadline 2024-09-21T03:43:44.745556+0000) 2024-09-21T03:44:14.811 INFO:journalctl@ceph.osd.9.smithi089.stdout:Sep 21 03:44:14 smithi089 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-9[65404]: 2024-09-21T03:44:14.551+0000 7f6c4e7c5640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-21T03:43:24.746023+0000 front 2024-09-21T03:43:24.745970+0000 (oldest deadline 2024-09-21T03:43:50.646080+0000) 2024-09-21T03:44:14.811 INFO:journalctl@ceph.osd.9.smithi089.stdout:Sep 21 03:44:14 smithi089 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-9[65404]: 2024-09-21T03:44:14.551+0000 7f6c4e7c5640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.114:6804 osd.2 since back 2024-09-21T03:43:34.147460+0000 front 2024-09-21T03:43:34.147364+0000 (oldest deadline 2024-09-21T03:43:58.247427+0000) 2024-09-21T03:44:14.811 INFO:journalctl@ceph.osd.9.smithi089.stdout:Sep 21 03:44:14 smithi089 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-9[65404]: 2024-09-21T03:44:14.551+0000 7f6c4e7c5640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.89:6814 osd.3 since back 2024-09-21T03:43:39.848901+0000 front 2024-09-21T03:43:39.848935+0000 (oldest deadline 2024-09-21T03:44:09.849669+0000) 2024-09-21T03:44:14.811 INFO:journalctl@ceph.osd.9.smithi089.stdout:Sep 21 03:44:14 smithi089 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-9[65404]: 2024-09-21T03:44:14.551+0000 7f6c4e7c5640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.110:6812 osd.4 since back 2024-09-21T03:43:49.850335+0000 front 2024-09-21T03:43:49.850307+0000 (oldest deadline 2024-09-21T03:44:13.950181+0000) 2024-09-21T03:44:15.461 INFO:journalctl@ceph.osd.8.smithi114.stdout:Sep 21 03:44:15 smithi114 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-8[51702]: 2024-09-21T03:44:15.339+0000 7f3e66f25640 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-21T03:43:23.926943+0000 front 2024-09-21T03:43:23.926870+0000 (oldest deadline 2024-09-21T03:43:52.127434+0000) 2024-09-21T03:44:15.461 INFO:journalctl@ceph.osd.8.smithi114.stdout:Sep 21 03:44:15 smithi114 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-8[51702]: 2024-09-21T03:44:15.339+0000 7f3e66f25640 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.114:6804 osd.2 since back 2024-09-21T03:43:32.128066+0000 front 2024-09-21T03:43:32.127963+0000 (oldest deadline 2024-09-21T03:43:58.028025+0000) 2024-09-21T03:44:15.461 INFO:journalctl@ceph.osd.8.smithi114.stdout:Sep 21 03:44:15 smithi114 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-8[51702]: 2024-09-21T03:44:15.339+0000 7f3e66f25640 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.89:6814 osd.3 since back 2024-09-21T03:43:45.029373+0000 front 2024-09-21T03:43:45.029438+0000 (oldest deadline 2024-09-21T03:44:10.329391+0000) 2024-09-21T03:44:15.461 INFO:journalctl@ceph.osd.11.smithi114.stdout:Sep 21 03:44:15 smithi114 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-11[58313]: 2024-09-21T03:44:15.097+0000 7f7657ce9640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2024-09-21T03:43:19.721069+0000 front 2024-09-21T03:43:19.721119+0000 (oldest deadline 2024-09-21T03:43:47.321877+0000) 2024-09-21T03:44:15.461 INFO:journalctl@ceph.osd.11.smithi114.stdout:Sep 21 03:44:15 smithi114 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-11[58313]: 2024-09-21T03:44:15.097+0000 7f7657ce9640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-21T03:43:27.322375+0000 front 2024-09-21T03:43:27.322426+0000 (oldest deadline 2024-09-21T03:43:51.422473+0000) 2024-09-21T03:44:15.461 INFO:journalctl@ceph.osd.11.smithi114.stdout:Sep 21 03:44:15 smithi114 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-11[58313]: 2024-09-21T03:44:15.097+0000 7f7657ce9640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.114:6804 osd.2 since back 2024-09-21T03:43:37.224230+0000 front 2024-09-21T03:43:37.224148+0000 (oldest deadline 2024-09-21T03:43:58.924202+0000) 2024-09-21T03:44:15.500 INFO:journalctl@ceph.osd.7.smithi110.stdout:Sep 21 03:44:15 smithi110 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-7[50481]: 2024-09-21T03:44:15.371+0000 7fb9f2345640 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-21T03:43:29.081492+0000 front 2024-09-21T03:43:29.081521+0000 (oldest deadline 2024-09-21T03:43:53.781711+0000) 2024-09-21T03:44:15.500 INFO:journalctl@ceph.osd.7.smithi110.stdout:Sep 21 03:44:15 smithi110 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-7[50481]: 2024-09-21T03:44:15.371+0000 7fb9f2345640 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.114:6804 osd.2 since back 2024-09-21T03:43:37.283011+0000 front 2024-09-21T03:43:37.282954+0000 (oldest deadline 2024-09-21T03:43:59.582792+0000) 2024-09-21T03:44:15.500 INFO:journalctl@ceph.osd.7.smithi110.stdout:Sep 21 03:44:15 smithi110 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-7[50481]: 2024-09-21T03:44:15.371+0000 7fb9f2345640 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.89:6814 osd.3 since back 2024-09-21T03:43:39.583357+0000 front 2024-09-21T03:43:39.583459+0000 (oldest deadline 2024-09-21T03:44:08.383601+0000) 2024-09-21T03:44:15.500 INFO:journalctl@ceph.osd.7.smithi110.stdout:Sep 21 03:44:15 smithi110 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-7[50481]: 2024-09-21T03:44:15.371+0000 7fb9f2345640 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.110:6812 osd.4 since back 2024-09-21T03:43:48.383949+0000 front 2024-09-21T03:43:48.384002+0000 (oldest deadline 2024-09-21T03:44:14.284020+0000) 2024-09-21T03:44:15.500 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 21 03:44:15 smithi110 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-10[57089]: 2024-09-21T03:44:15.146+0000 7f9d479dc640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2024-09-21T03:43:21.264189+0000 front 2024-09-21T03:43:21.264118+0000 (oldest deadline 2024-09-21T03:43:43.564295+0000) 2024-09-21T03:44:15.500 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 21 03:44:15 smithi110 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-10[57089]: 2024-09-21T03:44:15.146+0000 7f9d479dc640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-21T03:43:25.265351+0000 front 2024-09-21T03:43:25.265386+0000 (oldest deadline 2024-09-21T03:43:49.965504+0000) 2024-09-21T03:44:15.500 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 21 03:44:15 smithi110 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-10[57089]: 2024-09-21T03:44:15.146+0000 7f9d479dc640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.114:6804 osd.2 since back 2024-09-21T03:43:33.966726+0000 front 2024-09-21T03:43:33.966884+0000 (oldest deadline 2024-09-21T03:43:59.866510+0000) 2024-09-21T03:44:15.500 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 21 03:44:15 smithi110 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-10[57089]: 2024-09-21T03:44:15.146+0000 7f9d479dc640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.89:6814 osd.3 since back 2024-09-21T03:43:43.367532+0000 front 2024-09-21T03:43:43.367626+0000 (oldest deadline 2024-09-21T03:44:06.867392+0000) 2024-09-21T03:44:15.501 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 21 03:44:15 smithi110 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-10[57089]: 2024-09-21T03:44:15.146+0000 7f9d479dc640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.110:6812 osd.4 since back 2024-09-21T03:43:48.568210+0000 front 2024-09-21T03:43:48.568248+0000 (oldest deadline 2024-09-21T03:44:14.468315+0000) 2024-09-21T03:44:15.811 INFO:journalctl@ceph.osd.9.smithi089.stdout:Sep 21 03:44:15 smithi089 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-9[65404]: 2024-09-21T03:44:15.524+0000 7f6c4e7c5640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2024-09-21T03:43:18.845419+0000 front 2024-09-21T03:43:18.845455+0000 (oldest deadline 2024-09-21T03:43:44.745556+0000) 2024-09-21T03:44:15.811 INFO:journalctl@ceph.osd.9.smithi089.stdout:Sep 21 03:44:15 smithi089 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-9[65404]: 2024-09-21T03:44:15.524+0000 7f6c4e7c5640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-21T03:43:24.746023+0000 front 2024-09-21T03:43:24.745970+0000 (oldest deadline 2024-09-21T03:43:50.646080+0000) 2024-09-21T03:44:15.811 INFO:journalctl@ceph.osd.9.smithi089.stdout:Sep 21 03:44:15 smithi089 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-9[65404]: 2024-09-21T03:44:15.524+0000 7f6c4e7c5640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.114:6804 osd.2 since back 2024-09-21T03:43:34.147460+0000 front 2024-09-21T03:43:34.147364+0000 (oldest deadline 2024-09-21T03:43:58.247427+0000) 2024-09-21T03:44:15.811 INFO:journalctl@ceph.osd.9.smithi089.stdout:Sep 21 03:44:15 smithi089 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-9[65404]: 2024-09-21T03:44:15.524+0000 7f6c4e7c5640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.89:6814 osd.3 since back 2024-09-21T03:43:39.848901+0000 front 2024-09-21T03:43:39.848935+0000 (oldest deadline 2024-09-21T03:44:09.849669+0000) 2024-09-21T03:44:15.812 INFO:journalctl@ceph.osd.9.smithi089.stdout:Sep 21 03:44:15 smithi089 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-9[65404]: 2024-09-21T03:44:15.524+0000 7f6c4e7c5640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.110:6812 osd.4 since back 2024-09-21T03:43:49.850335+0000 front 2024-09-21T03:43:49.850307+0000 (oldest deadline 2024-09-21T03:44:13.950181+0000) 2024-09-21T03:44:16.461 INFO:journalctl@ceph.osd.8.smithi114.stdout:Sep 21 03:44:16 smithi114 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-8[51702]: 2024-09-21T03:44:16.306+0000 7f3e66f25640 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-21T03:43:23.926943+0000 front 2024-09-21T03:43:23.926870+0000 (oldest deadline 2024-09-21T03:43:52.127434+0000) 2024-09-21T03:44:16.461 INFO:journalctl@ceph.osd.8.smithi114.stdout:Sep 21 03:44:16 smithi114 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-8[51702]: 2024-09-21T03:44:16.306+0000 7f3e66f25640 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.114:6804 osd.2 since back 2024-09-21T03:43:32.128066+0000 front 2024-09-21T03:43:32.127963+0000 (oldest deadline 2024-09-21T03:43:58.028025+0000) 2024-09-21T03:44:16.461 INFO:journalctl@ceph.osd.8.smithi114.stdout:Sep 21 03:44:16 smithi114 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-8[51702]: 2024-09-21T03:44:16.306+0000 7f3e66f25640 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.89:6814 osd.3 since back 2024-09-21T03:43:45.029373+0000 front 2024-09-21T03:43:45.029438+0000 (oldest deadline 2024-09-21T03:44:10.329391+0000) 2024-09-21T03:44:16.461 INFO:journalctl@ceph.osd.8.smithi114.stdout:Sep 21 03:44:16 smithi114 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-8[51702]: 2024-09-21T03:44:16.306+0000 7f3e66f25640 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.110:6812 osd.4 since back 2024-09-21T03:43:50.330091+0000 front 2024-09-21T03:43:50.330000+0000 (oldest deadline 2024-09-21T03:44:16.229845+0000) 2024-09-21T03:44:16.462 INFO:journalctl@ceph.osd.11.smithi114.stdout:Sep 21 03:44:16 smithi114 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-11[58313]: 2024-09-21T03:44:16.115+0000 7f7657ce9640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2024-09-21T03:43:19.721069+0000 front 2024-09-21T03:43:19.721119+0000 (oldest deadline 2024-09-21T03:43:47.321877+0000) 2024-09-21T03:44:16.462 INFO:journalctl@ceph.osd.11.smithi114.stdout:Sep 21 03:44:16 smithi114 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-11[58313]: 2024-09-21T03:44:16.115+0000 7f7657ce9640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-21T03:43:27.322375+0000 front 2024-09-21T03:43:27.322426+0000 (oldest deadline 2024-09-21T03:43:51.422473+0000) 2024-09-21T03:44:16.462 INFO:journalctl@ceph.osd.11.smithi114.stdout:Sep 21 03:44:16 smithi114 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-11[58313]: 2024-09-21T03:44:16.115+0000 7f7657ce9640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.114:6804 osd.2 since back 2024-09-21T03:43:37.224230+0000 front 2024-09-21T03:43:37.224148+0000 (oldest deadline 2024-09-21T03:43:58.924202+0000) 2024-09-21T03:44:16.462 INFO:journalctl@ceph.osd.11.smithi114.stdout:Sep 21 03:44:16 smithi114 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-11[58313]: 2024-09-21T03:44:16.115+0000 7f7657ce9640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.110:6812 osd.4 since back 2024-09-21T03:43:50.626287+0000 front 2024-09-21T03:43:50.626333+0000 (oldest deadline 2024-09-21T03:44:15.326365+0000) 2024-09-21T03:44:16.500 INFO:journalctl@ceph.osd.7.smithi110.stdout:Sep 21 03:44:16 smithi110 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-7[50481]: 2024-09-21T03:44:16.390+0000 7fb9f2345640 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-21T03:43:29.081492+0000 front 2024-09-21T03:43:29.081521+0000 (oldest deadline 2024-09-21T03:43:53.781711+0000) 2024-09-21T03:44:16.500 INFO:journalctl@ceph.osd.7.smithi110.stdout:Sep 21 03:44:16 smithi110 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-7[50481]: 2024-09-21T03:44:16.390+0000 7fb9f2345640 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.114:6804 osd.2 since back 2024-09-21T03:43:37.283011+0000 front 2024-09-21T03:43:37.282954+0000 (oldest deadline 2024-09-21T03:43:59.582792+0000) 2024-09-21T03:44:16.500 INFO:journalctl@ceph.osd.7.smithi110.stdout:Sep 21 03:44:16 smithi110 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-7[50481]: 2024-09-21T03:44:16.390+0000 7fb9f2345640 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.89:6814 osd.3 since back 2024-09-21T03:43:39.583357+0000 front 2024-09-21T03:43:39.583459+0000 (oldest deadline 2024-09-21T03:44:08.383601+0000) 2024-09-21T03:44:16.500 INFO:journalctl@ceph.osd.7.smithi110.stdout:Sep 21 03:44:16 smithi110 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-7[50481]: 2024-09-21T03:44:16.390+0000 7fb9f2345640 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.110:6812 osd.4 since back 2024-09-21T03:43:48.383949+0000 front 2024-09-21T03:43:48.384002+0000 (oldest deadline 2024-09-21T03:44:14.284020+0000) 2024-09-21T03:44:16.500 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 21 03:44:16 smithi110 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-10[57089]: 2024-09-21T03:44:16.188+0000 7f9d479dc640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2024-09-21T03:43:21.264189+0000 front 2024-09-21T03:43:21.264118+0000 (oldest deadline 2024-09-21T03:43:43.564295+0000) 2024-09-21T03:44:16.500 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 21 03:44:16 smithi110 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-10[57089]: 2024-09-21T03:44:16.188+0000 7f9d479dc640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-21T03:43:25.265351+0000 front 2024-09-21T03:43:25.265386+0000 (oldest deadline 2024-09-21T03:43:49.965504+0000) 2024-09-21T03:44:16.500 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 21 03:44:16 smithi110 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-10[57089]: 2024-09-21T03:44:16.188+0000 7f9d479dc640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.114:6804 osd.2 since back 2024-09-21T03:43:33.966726+0000 front 2024-09-21T03:43:33.966884+0000 (oldest deadline 2024-09-21T03:43:59.866510+0000) 2024-09-21T03:44:16.500 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 21 03:44:16 smithi110 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-10[57089]: 2024-09-21T03:44:16.188+0000 7f9d479dc640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.89:6814 osd.3 since back 2024-09-21T03:43:43.367532+0000 front 2024-09-21T03:43:43.367626+0000 (oldest deadline 2024-09-21T03:44:06.867392+0000) 2024-09-21T03:44:16.501 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 21 03:44:16 smithi110 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-10[57089]: 2024-09-21T03:44:16.188+0000 7f9d479dc640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.110:6812 osd.4 since back 2024-09-21T03:43:48.568210+0000 front 2024-09-21T03:43:48.568248+0000 (oldest deadline 2024-09-21T03:44:14.468315+0000) 2024-09-21T03:44:16.811 INFO:journalctl@ceph.osd.9.smithi089.stdout:Sep 21 03:44:16 smithi089 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-9[65404]: 2024-09-21T03:44:16.483+0000 7f6c4e7c5640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2024-09-21T03:43:18.845419+0000 front 2024-09-21T03:43:18.845455+0000 (oldest deadline 2024-09-21T03:43:44.745556+0000) 2024-09-21T03:44:16.811 INFO:journalctl@ceph.osd.9.smithi089.stdout:Sep 21 03:44:16 smithi089 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-9[65404]: 2024-09-21T03:44:16.483+0000 7f6c4e7c5640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-21T03:43:24.746023+0000 front 2024-09-21T03:43:24.745970+0000 (oldest deadline 2024-09-21T03:43:50.646080+0000) 2024-09-21T03:44:16.811 INFO:journalctl@ceph.osd.9.smithi089.stdout:Sep 21 03:44:16 smithi089 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-9[65404]: 2024-09-21T03:44:16.483+0000 7f6c4e7c5640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.114:6804 osd.2 since back 2024-09-21T03:43:34.147460+0000 front 2024-09-21T03:43:34.147364+0000 (oldest deadline 2024-09-21T03:43:58.247427+0000) 2024-09-21T03:44:16.812 INFO:journalctl@ceph.osd.9.smithi089.stdout:Sep 21 03:44:16 smithi089 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-9[65404]: 2024-09-21T03:44:16.483+0000 7f6c4e7c5640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.89:6814 osd.3 since back 2024-09-21T03:43:39.848901+0000 front 2024-09-21T03:43:39.848935+0000 (oldest deadline 2024-09-21T03:44:09.849669+0000) 2024-09-21T03:44:16.812 INFO:journalctl@ceph.osd.9.smithi089.stdout:Sep 21 03:44:16 smithi089 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-9[65404]: 2024-09-21T03:44:16.483+0000 7f6c4e7c5640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.110:6812 osd.4 since back 2024-09-21T03:43:49.850335+0000 front 2024-09-21T03:43:49.850307+0000 (oldest deadline 2024-09-21T03:44:13.950181+0000) 2024-09-21T03:44:17.461 INFO:journalctl@ceph.osd.8.smithi114.stdout:Sep 21 03:44:17 smithi114 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-8[51702]: 2024-09-21T03:44:17.268+0000 7f3e66f25640 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-21T03:43:23.926943+0000 front 2024-09-21T03:43:23.926870+0000 (oldest deadline 2024-09-21T03:43:52.127434+0000) 2024-09-21T03:44:17.461 INFO:journalctl@ceph.osd.8.smithi114.stdout:Sep 21 03:44:17 smithi114 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-8[51702]: 2024-09-21T03:44:17.268+0000 7f3e66f25640 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.114:6804 osd.2 since back 2024-09-21T03:43:32.128066+0000 front 2024-09-21T03:43:32.127963+0000 (oldest deadline 2024-09-21T03:43:58.028025+0000) 2024-09-21T03:44:17.461 INFO:journalctl@ceph.osd.8.smithi114.stdout:Sep 21 03:44:17 smithi114 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-8[51702]: 2024-09-21T03:44:17.268+0000 7f3e66f25640 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.89:6814 osd.3 since back 2024-09-21T03:43:45.029373+0000 front 2024-09-21T03:43:45.029438+0000 (oldest deadline 2024-09-21T03:44:10.329391+0000) 2024-09-21T03:44:17.461 INFO:journalctl@ceph.osd.8.smithi114.stdout:Sep 21 03:44:17 smithi114 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-8[51702]: 2024-09-21T03:44:17.268+0000 7f3e66f25640 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.110:6812 osd.4 since back 2024-09-21T03:43:50.330091+0000 front 2024-09-21T03:43:50.330000+0000 (oldest deadline 2024-09-21T03:44:16.229845+0000) 2024-09-21T03:44:17.462 INFO:journalctl@ceph.osd.11.smithi114.stdout:Sep 21 03:44:17 smithi114 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-11[58313]: 2024-09-21T03:44:17.134+0000 7f7657ce9640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2024-09-21T03:43:19.721069+0000 front 2024-09-21T03:43:19.721119+0000 (oldest deadline 2024-09-21T03:43:47.321877+0000) 2024-09-21T03:44:17.462 INFO:journalctl@ceph.osd.11.smithi114.stdout:Sep 21 03:44:17 smithi114 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-11[58313]: 2024-09-21T03:44:17.134+0000 7f7657ce9640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-21T03:43:27.322375+0000 front 2024-09-21T03:43:27.322426+0000 (oldest deadline 2024-09-21T03:43:51.422473+0000) 2024-09-21T03:44:17.462 INFO:journalctl@ceph.osd.11.smithi114.stdout:Sep 21 03:44:17 smithi114 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-11[58313]: 2024-09-21T03:44:17.134+0000 7f7657ce9640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.114:6804 osd.2 since back 2024-09-21T03:43:37.224230+0000 front 2024-09-21T03:43:37.224148+0000 (oldest deadline 2024-09-21T03:43:58.924202+0000) 2024-09-21T03:44:17.462 INFO:journalctl@ceph.osd.11.smithi114.stdout:Sep 21 03:44:17 smithi114 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-11[58313]: 2024-09-21T03:44:17.134+0000 7f7657ce9640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.110:6812 osd.4 since back 2024-09-21T03:43:50.626287+0000 front 2024-09-21T03:43:50.626333+0000 (oldest deadline 2024-09-21T03:44:15.326365+0000) 2024-09-21T03:44:17.500 INFO:journalctl@ceph.osd.7.smithi110.stdout:Sep 21 03:44:17 smithi110 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-7[50481]: 2024-09-21T03:44:17.353+0000 7fb9f2345640 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-21T03:43:29.081492+0000 front 2024-09-21T03:43:29.081521+0000 (oldest deadline 2024-09-21T03:43:53.781711+0000) 2024-09-21T03:44:17.500 INFO:journalctl@ceph.osd.7.smithi110.stdout:Sep 21 03:44:17 smithi110 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-7[50481]: 2024-09-21T03:44:17.353+0000 7fb9f2345640 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.114:6804 osd.2 since back 2024-09-21T03:43:37.283011+0000 front 2024-09-21T03:43:37.282954+0000 (oldest deadline 2024-09-21T03:43:59.582792+0000) 2024-09-21T03:44:17.501 INFO:journalctl@ceph.osd.7.smithi110.stdout:Sep 21 03:44:17 smithi110 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-7[50481]: 2024-09-21T03:44:17.353+0000 7fb9f2345640 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.89:6814 osd.3 since back 2024-09-21T03:43:39.583357+0000 front 2024-09-21T03:43:39.583459+0000 (oldest deadline 2024-09-21T03:44:08.383601+0000) 2024-09-21T03:44:17.501 INFO:journalctl@ceph.osd.7.smithi110.stdout:Sep 21 03:44:17 smithi110 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-7[50481]: 2024-09-21T03:44:17.353+0000 7fb9f2345640 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.110:6812 osd.4 since back 2024-09-21T03:43:48.383949+0000 front 2024-09-21T03:43:48.384002+0000 (oldest deadline 2024-09-21T03:44:14.284020+0000) 2024-09-21T03:44:17.501 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 21 03:44:17 smithi110 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-10[57089]: 2024-09-21T03:44:17.234+0000 7f9d479dc640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2024-09-21T03:43:21.264189+0000 front 2024-09-21T03:43:21.264118+0000 (oldest deadline 2024-09-21T03:43:43.564295+0000) 2024-09-21T03:44:17.501 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 21 03:44:17 smithi110 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-10[57089]: 2024-09-21T03:44:17.234+0000 7f9d479dc640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-21T03:43:25.265351+0000 front 2024-09-21T03:43:25.265386+0000 (oldest deadline 2024-09-21T03:43:49.965504+0000) 2024-09-21T03:44:17.501 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 21 03:44:17 smithi110 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-10[57089]: 2024-09-21T03:44:17.235+0000 7f9d479dc640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.114:6804 osd.2 since back 2024-09-21T03:43:33.966726+0000 front 2024-09-21T03:43:33.966884+0000 (oldest deadline 2024-09-21T03:43:59.866510+0000) 2024-09-21T03:44:17.501 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 21 03:44:17 smithi110 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-10[57089]: 2024-09-21T03:44:17.235+0000 7f9d479dc640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.89:6814 osd.3 since back 2024-09-21T03:43:43.367532+0000 front 2024-09-21T03:43:43.367626+0000 (oldest deadline 2024-09-21T03:44:06.867392+0000) 2024-09-21T03:44:17.501 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 21 03:44:17 smithi110 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-10[57089]: 2024-09-21T03:44:17.235+0000 7f9d479dc640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.110:6812 osd.4 since back 2024-09-21T03:43:48.568210+0000 front 2024-09-21T03:43:48.568248+0000 (oldest deadline 2024-09-21T03:44:14.468315+0000) 2024-09-21T03:44:17.811 INFO:journalctl@ceph.osd.9.smithi089.stdout:Sep 21 03:44:17 smithi089 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-9[65404]: 2024-09-21T03:44:17.464+0000 7f6c4e7c5640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2024-09-21T03:43:18.845419+0000 front 2024-09-21T03:43:18.845455+0000 (oldest deadline 2024-09-21T03:43:44.745556+0000) 2024-09-21T03:44:17.811 INFO:journalctl@ceph.osd.9.smithi089.stdout:Sep 21 03:44:17 smithi089 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-9[65404]: 2024-09-21T03:44:17.464+0000 7f6c4e7c5640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-21T03:43:24.746023+0000 front 2024-09-21T03:43:24.745970+0000 (oldest deadline 2024-09-21T03:43:50.646080+0000) 2024-09-21T03:44:17.811 INFO:journalctl@ceph.osd.9.smithi089.stdout:Sep 21 03:44:17 smithi089 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-9[65404]: 2024-09-21T03:44:17.464+0000 7f6c4e7c5640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.114:6804 osd.2 since back 2024-09-21T03:43:34.147460+0000 front 2024-09-21T03:43:34.147364+0000 (oldest deadline 2024-09-21T03:43:58.247427+0000) 2024-09-21T03:44:17.812 INFO:journalctl@ceph.osd.9.smithi089.stdout:Sep 21 03:44:17 smithi089 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-9[65404]: 2024-09-21T03:44:17.464+0000 7f6c4e7c5640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.89:6814 osd.3 since back 2024-09-21T03:43:39.848901+0000 front 2024-09-21T03:43:39.848935+0000 (oldest deadline 2024-09-21T03:44:09.849669+0000) 2024-09-21T03:44:17.812 INFO:journalctl@ceph.osd.9.smithi089.stdout:Sep 21 03:44:17 smithi089 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-9[65404]: 2024-09-21T03:44:17.464+0000 7f6c4e7c5640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.110:6812 osd.4 since back 2024-09-21T03:43:49.850335+0000 front 2024-09-21T03:43:49.850307+0000 (oldest deadline 2024-09-21T03:44:13.950181+0000) 2024-09-21T03:44:18.213 INFO:journalctl@ceph.osd.7.smithi110.stdout:Sep 21 03:44:17 smithi110 podman[65153]: 2024-09-21 03:44:17.899733876 +0000 UTC m=+5.575673918 container died 5223b04ac522dba2028a5f3c52a66f80b6188d4a8e1dcb8fc01f23a19cc8bace (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-7, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221) 2024-09-21T03:44:18.461 INFO:journalctl@ceph.osd.8.smithi114.stdout:Sep 21 03:44:18 smithi114 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-8[51702]: 2024-09-21T03:44:18.259+0000 7f3e66f25640 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-21T03:43:23.926943+0000 front 2024-09-21T03:43:23.926870+0000 (oldest deadline 2024-09-21T03:43:52.127434+0000) 2024-09-21T03:44:18.461 INFO:journalctl@ceph.osd.8.smithi114.stdout:Sep 21 03:44:18 smithi114 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-8[51702]: 2024-09-21T03:44:18.259+0000 7f3e66f25640 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.114:6804 osd.2 since back 2024-09-21T03:43:32.128066+0000 front 2024-09-21T03:43:32.127963+0000 (oldest deadline 2024-09-21T03:43:58.028025+0000) 2024-09-21T03:44:18.461 INFO:journalctl@ceph.osd.8.smithi114.stdout:Sep 21 03:44:18 smithi114 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-8[51702]: 2024-09-21T03:44:18.259+0000 7f3e66f25640 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.89:6814 osd.3 since back 2024-09-21T03:43:45.029373+0000 front 2024-09-21T03:43:45.029438+0000 (oldest deadline 2024-09-21T03:44:10.329391+0000) 2024-09-21T03:44:18.461 INFO:journalctl@ceph.osd.8.smithi114.stdout:Sep 21 03:44:18 smithi114 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-8[51702]: 2024-09-21T03:44:18.259+0000 7f3e66f25640 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.110:6812 osd.4 since back 2024-09-21T03:43:50.330091+0000 front 2024-09-21T03:43:50.330000+0000 (oldest deadline 2024-09-21T03:44:16.229845+0000) 2024-09-21T03:44:18.462 INFO:journalctl@ceph.osd.11.smithi114.stdout:Sep 21 03:44:18 smithi114 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-11[58313]: 2024-09-21T03:44:18.175+0000 7f7657ce9640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2024-09-21T03:43:19.721069+0000 front 2024-09-21T03:43:19.721119+0000 (oldest deadline 2024-09-21T03:43:47.321877+0000) 2024-09-21T03:44:18.462 INFO:journalctl@ceph.osd.11.smithi114.stdout:Sep 21 03:44:18 smithi114 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-11[58313]: 2024-09-21T03:44:18.175+0000 7f7657ce9640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-21T03:43:27.322375+0000 front 2024-09-21T03:43:27.322426+0000 (oldest deadline 2024-09-21T03:43:51.422473+0000) 2024-09-21T03:44:18.462 INFO:journalctl@ceph.osd.11.smithi114.stdout:Sep 21 03:44:18 smithi114 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-11[58313]: 2024-09-21T03:44:18.175+0000 7f7657ce9640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.114:6804 osd.2 since back 2024-09-21T03:43:37.224230+0000 front 2024-09-21T03:43:37.224148+0000 (oldest deadline 2024-09-21T03:43:58.924202+0000) 2024-09-21T03:44:18.462 INFO:journalctl@ceph.osd.11.smithi114.stdout:Sep 21 03:44:18 smithi114 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-11[58313]: 2024-09-21T03:44:18.175+0000 7f7657ce9640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.110:6812 osd.4 since back 2024-09-21T03:43:50.626287+0000 front 2024-09-21T03:43:50.626333+0000 (oldest deadline 2024-09-21T03:44:15.326365+0000) 2024-09-21T03:44:18.500 INFO:journalctl@ceph.osd.7.smithi110.stdout:Sep 21 03:44:18 smithi110 podman[65153]: 2024-09-21 03:44:18.301607717 +0000 UTC m=+5.977547770 container cleanup 5223b04ac522dba2028a5f3c52a66f80b6188d4a8e1dcb8fc01f23a19cc8bace (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-7, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2) 2024-09-21T03:44:18.501 INFO:journalctl@ceph.osd.7.smithi110.stdout:Sep 21 03:44:18 smithi110 bash[65153]: ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-7 2024-09-21T03:44:18.501 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 21 03:44:18 smithi110 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-10[57089]: 2024-09-21T03:44:18.220+0000 7f9d479dc640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2024-09-21T03:43:21.264189+0000 front 2024-09-21T03:43:21.264118+0000 (oldest deadline 2024-09-21T03:43:43.564295+0000) 2024-09-21T03:44:18.501 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 21 03:44:18 smithi110 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-10[57089]: 2024-09-21T03:44:18.220+0000 7f9d479dc640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-21T03:43:25.265351+0000 front 2024-09-21T03:43:25.265386+0000 (oldest deadline 2024-09-21T03:43:49.965504+0000) 2024-09-21T03:44:18.501 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 21 03:44:18 smithi110 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-10[57089]: 2024-09-21T03:44:18.220+0000 7f9d479dc640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.114:6804 osd.2 since back 2024-09-21T03:43:33.966726+0000 front 2024-09-21T03:43:33.966884+0000 (oldest deadline 2024-09-21T03:43:59.866510+0000) 2024-09-21T03:44:18.501 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 21 03:44:18 smithi110 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-10[57089]: 2024-09-21T03:44:18.220+0000 7f9d479dc640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.89:6814 osd.3 since back 2024-09-21T03:43:43.367532+0000 front 2024-09-21T03:43:43.367626+0000 (oldest deadline 2024-09-21T03:44:06.867392+0000) 2024-09-21T03:44:18.501 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 21 03:44:18 smithi110 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-10[57089]: 2024-09-21T03:44:18.220+0000 7f9d479dc640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.110:6812 osd.4 since back 2024-09-21T03:43:48.568210+0000 front 2024-09-21T03:43:48.568248+0000 (oldest deadline 2024-09-21T03:44:14.468315+0000) 2024-09-21T03:44:18.811 INFO:journalctl@ceph.osd.9.smithi089.stdout:Sep 21 03:44:18 smithi089 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-9[65404]: 2024-09-21T03:44:18.511+0000 7f6c4e7c5640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2024-09-21T03:43:18.845419+0000 front 2024-09-21T03:43:18.845455+0000 (oldest deadline 2024-09-21T03:43:44.745556+0000) 2024-09-21T03:44:18.812 INFO:journalctl@ceph.osd.9.smithi089.stdout:Sep 21 03:44:18 smithi089 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-9[65404]: 2024-09-21T03:44:18.511+0000 7f6c4e7c5640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-21T03:43:24.746023+0000 front 2024-09-21T03:43:24.745970+0000 (oldest deadline 2024-09-21T03:43:50.646080+0000) 2024-09-21T03:44:18.812 INFO:journalctl@ceph.osd.9.smithi089.stdout:Sep 21 03:44:18 smithi089 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-9[65404]: 2024-09-21T03:44:18.511+0000 7f6c4e7c5640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.114:6804 osd.2 since back 2024-09-21T03:43:34.147460+0000 front 2024-09-21T03:43:34.147364+0000 (oldest deadline 2024-09-21T03:43:58.247427+0000) 2024-09-21T03:44:18.812 INFO:journalctl@ceph.osd.9.smithi089.stdout:Sep 21 03:44:18 smithi089 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-9[65404]: 2024-09-21T03:44:18.511+0000 7f6c4e7c5640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.89:6814 osd.3 since back 2024-09-21T03:43:39.848901+0000 front 2024-09-21T03:43:39.848935+0000 (oldest deadline 2024-09-21T03:44:09.849669+0000) 2024-09-21T03:44:18.812 INFO:journalctl@ceph.osd.9.smithi089.stdout:Sep 21 03:44:18 smithi089 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-9[65404]: 2024-09-21T03:44:18.511+0000 7f6c4e7c5640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.110:6812 osd.4 since back 2024-09-21T03:43:49.850335+0000 front 2024-09-21T03:43:49.850307+0000 (oldest deadline 2024-09-21T03:44:13.950181+0000) 2024-09-21T03:44:18.864 INFO:journalctl@ceph.osd.7.smithi110.stdout:Sep 21 03:44:18 smithi110 podman[65180]: 2024-09-21 03:44:18.528156931 +0000 UTC m=+0.625664714 container remove 5223b04ac522dba2028a5f3c52a66f80b6188d4a8e1dcb8fc01f23a19cc8bace (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-7, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.license=GPLv2) 2024-09-21T03:44:19.132 INFO:journalctl@ceph.osd.7.smithi110.stdout:Sep 21 03:44:18 smithi110 podman[65270]: 2024-09-21 03:44:18.764868369 +0000 UTC m=+0.015761537 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T03:44:19.132 INFO:journalctl@ceph.osd.7.smithi110.stdout:Sep 21 03:44:18 smithi110 podman[65270]: 2024-09-21 03:44:18.870455023 +0000 UTC m=+0.121348226 container create 2cd32a0b9ac3d6c14ef74ceb79fe614f62bdf2f32fcc3e7f2163f2615abae607 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-7-deactivate, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-21T03:44:19.461 INFO:journalctl@ceph.osd.8.smithi114.stdout:Sep 21 03:44:19 smithi114 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-8[51702]: 2024-09-21T03:44:19.262+0000 7f3e66f25640 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-21T03:43:23.926943+0000 front 2024-09-21T03:43:23.926870+0000 (oldest deadline 2024-09-21T03:43:52.127434+0000) 2024-09-21T03:44:19.461 INFO:journalctl@ceph.osd.8.smithi114.stdout:Sep 21 03:44:19 smithi114 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-8[51702]: 2024-09-21T03:44:19.262+0000 7f3e66f25640 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.114:6804 osd.2 since back 2024-09-21T03:43:32.128066+0000 front 2024-09-21T03:43:32.127963+0000 (oldest deadline 2024-09-21T03:43:58.028025+0000) 2024-09-21T03:44:19.461 INFO:journalctl@ceph.osd.8.smithi114.stdout:Sep 21 03:44:19 smithi114 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-8[51702]: 2024-09-21T03:44:19.262+0000 7f3e66f25640 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.89:6814 osd.3 since back 2024-09-21T03:43:45.029373+0000 front 2024-09-21T03:43:45.029438+0000 (oldest deadline 2024-09-21T03:44:10.329391+0000) 2024-09-21T03:44:19.461 INFO:journalctl@ceph.osd.8.smithi114.stdout:Sep 21 03:44:19 smithi114 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-8[51702]: 2024-09-21T03:44:19.262+0000 7f3e66f25640 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.110:6812 osd.4 since back 2024-09-21T03:43:50.330091+0000 front 2024-09-21T03:43:50.330000+0000 (oldest deadline 2024-09-21T03:44:16.229845+0000) 2024-09-21T03:44:19.462 INFO:journalctl@ceph.osd.11.smithi114.stdout:Sep 21 03:44:19 smithi114 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-11[58313]: 2024-09-21T03:44:19.185+0000 7f7657ce9640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2024-09-21T03:43:19.721069+0000 front 2024-09-21T03:43:19.721119+0000 (oldest deadline 2024-09-21T03:43:47.321877+0000) 2024-09-21T03:44:19.462 INFO:journalctl@ceph.osd.11.smithi114.stdout:Sep 21 03:44:19 smithi114 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-11[58313]: 2024-09-21T03:44:19.185+0000 7f7657ce9640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-21T03:43:27.322375+0000 front 2024-09-21T03:43:27.322426+0000 (oldest deadline 2024-09-21T03:43:51.422473+0000) 2024-09-21T03:44:19.462 INFO:journalctl@ceph.osd.11.smithi114.stdout:Sep 21 03:44:19 smithi114 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-11[58313]: 2024-09-21T03:44:19.185+0000 7f7657ce9640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.114:6804 osd.2 since back 2024-09-21T03:43:37.224230+0000 front 2024-09-21T03:43:37.224148+0000 (oldest deadline 2024-09-21T03:43:58.924202+0000) 2024-09-21T03:44:19.462 INFO:journalctl@ceph.osd.11.smithi114.stdout:Sep 21 03:44:19 smithi114 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-11[58313]: 2024-09-21T03:44:19.185+0000 7f7657ce9640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.110:6812 osd.4 since back 2024-09-21T03:43:50.626287+0000 front 2024-09-21T03:43:50.626333+0000 (oldest deadline 2024-09-21T03:44:15.326365+0000) 2024-09-21T03:44:19.500 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 21 03:44:19 smithi110 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-10[57089]: 2024-09-21T03:44:19.267+0000 7f9d479dc640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2024-09-21T03:43:21.264189+0000 front 2024-09-21T03:43:21.264118+0000 (oldest deadline 2024-09-21T03:43:43.564295+0000) 2024-09-21T03:44:19.500 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 21 03:44:19 smithi110 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-10[57089]: 2024-09-21T03:44:19.267+0000 7f9d479dc640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-21T03:43:25.265351+0000 front 2024-09-21T03:43:25.265386+0000 (oldest deadline 2024-09-21T03:43:49.965504+0000) 2024-09-21T03:44:19.500 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 21 03:44:19 smithi110 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-10[57089]: 2024-09-21T03:44:19.267+0000 7f9d479dc640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.114:6804 osd.2 since back 2024-09-21T03:43:33.966726+0000 front 2024-09-21T03:43:33.966884+0000 (oldest deadline 2024-09-21T03:43:59.866510+0000) 2024-09-21T03:44:19.500 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 21 03:44:19 smithi110 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-10[57089]: 2024-09-21T03:44:19.267+0000 7f9d479dc640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.89:6814 osd.3 since back 2024-09-21T03:43:43.367532+0000 front 2024-09-21T03:43:43.367626+0000 (oldest deadline 2024-09-21T03:44:06.867392+0000) 2024-09-21T03:44:19.500 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 21 03:44:19 smithi110 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-10[57089]: 2024-09-21T03:44:19.267+0000 7f9d479dc640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.110:6812 osd.4 since back 2024-09-21T03:43:48.568210+0000 front 2024-09-21T03:43:48.568248+0000 (oldest deadline 2024-09-21T03:44:14.468315+0000) 2024-09-21T03:44:19.501 INFO:journalctl@ceph.osd.7.smithi110.stdout:Sep 21 03:44:19 smithi110 podman[65270]: 2024-09-21 03:44:19.131366255 +0000 UTC m=+0.382259412 container init 2cd32a0b9ac3d6c14ef74ceb79fe614f62bdf2f32fcc3e7f2163f2615abae607 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-7-deactivate, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS) 2024-09-21T03:44:19.501 INFO:journalctl@ceph.osd.7.smithi110.stdout:Sep 21 03:44:19 smithi110 podman[65270]: 2024-09-21 03:44:19.134827662 +0000 UTC m=+0.385720820 container start 2cd32a0b9ac3d6c14ef74ceb79fe614f62bdf2f32fcc3e7f2163f2615abae607 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-7-deactivate, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_CLEAN=True, io.buildah.version=1.37.2, ceph=True, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS) 2024-09-21T03:44:19.501 INFO:journalctl@ceph.osd.7.smithi110.stdout:Sep 21 03:44:19 smithi110 podman[65270]: 2024-09-21 03:44:19.223114547 +0000 UTC m=+0.474007705 container attach 2cd32a0b9ac3d6c14ef74ceb79fe614f62bdf2f32fcc3e7f2163f2615abae607 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-7-deactivate, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-09-21T03:44:19.501 INFO:journalctl@ceph.osd.7.smithi110.stdout:Sep 21 03:44:19 smithi110 podman[65270]: 2024-09-21 03:44:19.326101412 +0000 UTC m=+0.576994578 container died 2cd32a0b9ac3d6c14ef74ceb79fe614f62bdf2f32fcc3e7f2163f2615abae607 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-7-deactivate, ceph=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731) 2024-09-21T03:44:19.811 INFO:journalctl@ceph.osd.9.smithi089.stdout:Sep 21 03:44:19 smithi089 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-9[65404]: 2024-09-21T03:44:19.488+0000 7f6c4e7c5640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2024-09-21T03:43:18.845419+0000 front 2024-09-21T03:43:18.845455+0000 (oldest deadline 2024-09-21T03:43:44.745556+0000) 2024-09-21T03:44:19.811 INFO:journalctl@ceph.osd.9.smithi089.stdout:Sep 21 03:44:19 smithi089 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-9[65404]: 2024-09-21T03:44:19.488+0000 7f6c4e7c5640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-21T03:43:24.746023+0000 front 2024-09-21T03:43:24.745970+0000 (oldest deadline 2024-09-21T03:43:50.646080+0000) 2024-09-21T03:44:19.811 INFO:journalctl@ceph.osd.9.smithi089.stdout:Sep 21 03:44:19 smithi089 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-9[65404]: 2024-09-21T03:44:19.488+0000 7f6c4e7c5640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.114:6804 osd.2 since back 2024-09-21T03:43:34.147460+0000 front 2024-09-21T03:43:34.147364+0000 (oldest deadline 2024-09-21T03:43:58.247427+0000) 2024-09-21T03:44:19.811 INFO:journalctl@ceph.osd.9.smithi089.stdout:Sep 21 03:44:19 smithi089 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-9[65404]: 2024-09-21T03:44:19.488+0000 7f6c4e7c5640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.89:6814 osd.3 since back 2024-09-21T03:43:39.848901+0000 front 2024-09-21T03:43:39.848935+0000 (oldest deadline 2024-09-21T03:44:09.849669+0000) 2024-09-21T03:44:19.812 INFO:journalctl@ceph.osd.9.smithi089.stdout:Sep 21 03:44:19 smithi089 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-9[65404]: 2024-09-21T03:44:19.488+0000 7f6c4e7c5640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.110:6812 osd.4 since back 2024-09-21T03:43:49.850335+0000 front 2024-09-21T03:43:49.850307+0000 (oldest deadline 2024-09-21T03:44:13.950181+0000) 2024-09-21T03:44:19.942 DEBUG:teuthology.orchestra.run.smithi110:> sudo pkill -f 'journalctl -f -n 0 -u ceph-7c374098-77c9-11ef-baf5-efdc52797490@osd.7.service' 2024-09-21T03:44:19.983 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T03:44:19.983 INFO:tasks.cephadm.osd.7:Stopped osd.7 2024-09-21T03:44:19.983 INFO:tasks.cephadm.osd.8:Stopping osd.8... 2024-09-21T03:44:19.984 DEBUG:teuthology.orchestra.run.smithi114:> sudo systemctl stop ceph-7c374098-77c9-11ef-baf5-efdc52797490@osd.8 2024-09-21T03:44:20.307 INFO:journalctl@ceph.osd.11.smithi114.stdout:Sep 21 03:44:20 smithi114 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-11[58313]: 2024-09-21T03:44:20.217+0000 7f7657ce9640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2024-09-21T03:43:19.721069+0000 front 2024-09-21T03:43:19.721119+0000 (oldest deadline 2024-09-21T03:43:47.321877+0000) 2024-09-21T03:44:20.307 INFO:journalctl@ceph.osd.11.smithi114.stdout:Sep 21 03:44:20 smithi114 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-11[58313]: 2024-09-21T03:44:20.217+0000 7f7657ce9640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-21T03:43:27.322375+0000 front 2024-09-21T03:43:27.322426+0000 (oldest deadline 2024-09-21T03:43:51.422473+0000) 2024-09-21T03:44:20.307 INFO:journalctl@ceph.osd.11.smithi114.stdout:Sep 21 03:44:20 smithi114 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-11[58313]: 2024-09-21T03:44:20.217+0000 7f7657ce9640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.114:6804 osd.2 since back 2024-09-21T03:43:37.224230+0000 front 2024-09-21T03:43:37.224148+0000 (oldest deadline 2024-09-21T03:43:58.924202+0000) 2024-09-21T03:44:20.307 INFO:journalctl@ceph.osd.11.smithi114.stdout:Sep 21 03:44:20 smithi114 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-11[58313]: 2024-09-21T03:44:20.217+0000 7f7657ce9640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.110:6812 osd.4 since back 2024-09-21T03:43:50.626287+0000 front 2024-09-21T03:43:50.626333+0000 (oldest deadline 2024-09-21T03:44:15.326365+0000) 2024-09-21T03:44:20.307 INFO:journalctl@ceph.osd.8.smithi114.stdout:Sep 21 03:44:20 smithi114 systemd[1]: Stopping Ceph osd.8 for 7c374098-77c9-11ef-baf5-efdc52797490... 2024-09-21T03:44:20.307 INFO:journalctl@ceph.osd.8.smithi114.stdout:Sep 21 03:44:20 smithi114 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-8[51702]: 2024-09-21T03:44:20.246+0000 7f3e66f25640 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-21T03:43:23.926943+0000 front 2024-09-21T03:43:23.926870+0000 (oldest deadline 2024-09-21T03:43:52.127434+0000) 2024-09-21T03:44:20.307 INFO:journalctl@ceph.osd.8.smithi114.stdout:Sep 21 03:44:20 smithi114 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-8[51702]: 2024-09-21T03:44:20.246+0000 7f3e66f25640 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.114:6804 osd.2 since back 2024-09-21T03:43:32.128066+0000 front 2024-09-21T03:43:32.127963+0000 (oldest deadline 2024-09-21T03:43:58.028025+0000) 2024-09-21T03:44:20.307 INFO:journalctl@ceph.osd.8.smithi114.stdout:Sep 21 03:44:20 smithi114 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-8[51702]: 2024-09-21T03:44:20.246+0000 7f3e66f25640 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.89:6814 osd.3 since back 2024-09-21T03:43:45.029373+0000 front 2024-09-21T03:43:45.029438+0000 (oldest deadline 2024-09-21T03:44:10.329391+0000) 2024-09-21T03:44:20.308 INFO:journalctl@ceph.osd.8.smithi114.stdout:Sep 21 03:44:20 smithi114 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-8[51702]: 2024-09-21T03:44:20.246+0000 7f3e66f25640 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.110:6812 osd.4 since back 2024-09-21T03:43:50.330091+0000 front 2024-09-21T03:43:50.330000+0000 (oldest deadline 2024-09-21T03:44:16.229845+0000) 2024-09-21T03:44:20.308 INFO:journalctl@ceph.osd.8.smithi114.stdout:Sep 21 03:44:20 smithi114 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-8[51702]: 2024-09-21T03:44:20.304+0000 7f3e6a93a640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.8 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T03:44:20.308 INFO:journalctl@ceph.osd.8.smithi114.stdout:Sep 21 03:44:20 smithi114 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-8[51702]: 2024-09-21T03:44:20.304+0000 7f3e6a93a640 -1 osd.8 88 *** Got signal Terminated *** 2024-09-21T03:44:20.308 INFO:journalctl@ceph.osd.8.smithi114.stdout:Sep 21 03:44:20 smithi114 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-8[51702]: 2024-09-21T03:44:20.304+0000 7f3e6a93a640 -1 osd.8 88 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-21T03:44:20.500 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 21 03:44:20 smithi110 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-10[57089]: 2024-09-21T03:44:20.223+0000 7f9d479dc640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2024-09-21T03:43:21.264189+0000 front 2024-09-21T03:43:21.264118+0000 (oldest deadline 2024-09-21T03:43:43.564295+0000) 2024-09-21T03:44:20.500 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 21 03:44:20 smithi110 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-10[57089]: 2024-09-21T03:44:20.223+0000 7f9d479dc640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-21T03:43:25.265351+0000 front 2024-09-21T03:43:25.265386+0000 (oldest deadline 2024-09-21T03:43:49.965504+0000) 2024-09-21T03:44:20.500 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 21 03:44:20 smithi110 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-10[57089]: 2024-09-21T03:44:20.223+0000 7f9d479dc640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.114:6804 osd.2 since back 2024-09-21T03:43:33.966726+0000 front 2024-09-21T03:43:33.966884+0000 (oldest deadline 2024-09-21T03:43:59.866510+0000) 2024-09-21T03:44:20.500 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 21 03:44:20 smithi110 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-10[57089]: 2024-09-21T03:44:20.223+0000 7f9d479dc640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.89:6814 osd.3 since back 2024-09-21T03:43:43.367532+0000 front 2024-09-21T03:43:43.367626+0000 (oldest deadline 2024-09-21T03:44:06.867392+0000) 2024-09-21T03:44:20.500 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 21 03:44:20 smithi110 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-10[57089]: 2024-09-21T03:44:20.223+0000 7f9d479dc640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.110:6812 osd.4 since back 2024-09-21T03:43:48.568210+0000 front 2024-09-21T03:43:48.568248+0000 (oldest deadline 2024-09-21T03:44:14.468315+0000) 2024-09-21T03:44:20.811 INFO:journalctl@ceph.osd.9.smithi089.stdout:Sep 21 03:44:20 smithi089 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-9[65404]: 2024-09-21T03:44:20.501+0000 7f6c4e7c5640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2024-09-21T03:43:18.845419+0000 front 2024-09-21T03:43:18.845455+0000 (oldest deadline 2024-09-21T03:43:44.745556+0000) 2024-09-21T03:44:20.811 INFO:journalctl@ceph.osd.9.smithi089.stdout:Sep 21 03:44:20 smithi089 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-9[65404]: 2024-09-21T03:44:20.501+0000 7f6c4e7c5640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-21T03:43:24.746023+0000 front 2024-09-21T03:43:24.745970+0000 (oldest deadline 2024-09-21T03:43:50.646080+0000) 2024-09-21T03:44:20.811 INFO:journalctl@ceph.osd.9.smithi089.stdout:Sep 21 03:44:20 smithi089 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-9[65404]: 2024-09-21T03:44:20.501+0000 7f6c4e7c5640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.114:6804 osd.2 since back 2024-09-21T03:43:34.147460+0000 front 2024-09-21T03:43:34.147364+0000 (oldest deadline 2024-09-21T03:43:58.247427+0000) 2024-09-21T03:44:20.812 INFO:journalctl@ceph.osd.9.smithi089.stdout:Sep 21 03:44:20 smithi089 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-9[65404]: 2024-09-21T03:44:20.501+0000 7f6c4e7c5640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.89:6814 osd.3 since back 2024-09-21T03:43:39.848901+0000 front 2024-09-21T03:43:39.848935+0000 (oldest deadline 2024-09-21T03:44:09.849669+0000) 2024-09-21T03:44:20.812 INFO:journalctl@ceph.osd.9.smithi089.stdout:Sep 21 03:44:20 smithi089 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-9[65404]: 2024-09-21T03:44:20.501+0000 7f6c4e7c5640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.110:6812 osd.4 since back 2024-09-21T03:43:49.850335+0000 front 2024-09-21T03:43:49.850307+0000 (oldest deadline 2024-09-21T03:44:13.950181+0000) 2024-09-21T03:44:21.500 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 21 03:44:21 smithi110 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-10[57089]: 2024-09-21T03:44:21.197+0000 7f9d479dc640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2024-09-21T03:43:21.264189+0000 front 2024-09-21T03:43:21.264118+0000 (oldest deadline 2024-09-21T03:43:43.564295+0000) 2024-09-21T03:44:21.598 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 21 03:44:21 smithi110 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-10[57089]: 2024-09-21T03:44:21.197+0000 7f9d479dc640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-21T03:43:25.265351+0000 front 2024-09-21T03:43:25.265386+0000 (oldest deadline 2024-09-21T03:43:49.965504+0000) 2024-09-21T03:44:21.598 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 21 03:44:21 smithi110 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-10[57089]: 2024-09-21T03:44:21.197+0000 7f9d479dc640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.114:6804 osd.2 since back 2024-09-21T03:43:33.966726+0000 front 2024-09-21T03:43:33.966884+0000 (oldest deadline 2024-09-21T03:43:59.866510+0000) 2024-09-21T03:44:21.598 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 21 03:44:21 smithi110 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-10[57089]: 2024-09-21T03:44:21.197+0000 7f9d479dc640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.89:6814 osd.3 since back 2024-09-21T03:43:43.367532+0000 front 2024-09-21T03:43:43.367626+0000 (oldest deadline 2024-09-21T03:44:06.867392+0000) 2024-09-21T03:44:21.598 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 21 03:44:21 smithi110 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-10[57089]: 2024-09-21T03:44:21.197+0000 7f9d479dc640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.110:6812 osd.4 since back 2024-09-21T03:43:48.568210+0000 front 2024-09-21T03:43:48.568248+0000 (oldest deadline 2024-09-21T03:44:14.468315+0000) 2024-09-21T03:44:21.712 INFO:journalctl@ceph.osd.8.smithi114.stdout:Sep 21 03:44:21 smithi114 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-8[51702]: 2024-09-21T03:44:21.256+0000 7f3e66f25640 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-21T03:43:23.926943+0000 front 2024-09-21T03:43:23.926870+0000 (oldest deadline 2024-09-21T03:43:52.127434+0000) 2024-09-21T03:44:21.712 INFO:journalctl@ceph.osd.8.smithi114.stdout:Sep 21 03:44:21 smithi114 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-8[51702]: 2024-09-21T03:44:21.256+0000 7f3e66f25640 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.114:6804 osd.2 since back 2024-09-21T03:43:32.128066+0000 front 2024-09-21T03:43:32.127963+0000 (oldest deadline 2024-09-21T03:43:58.028025+0000) 2024-09-21T03:44:21.712 INFO:journalctl@ceph.osd.8.smithi114.stdout:Sep 21 03:44:21 smithi114 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-8[51702]: 2024-09-21T03:44:21.256+0000 7f3e66f25640 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.89:6814 osd.3 since back 2024-09-21T03:43:45.029373+0000 front 2024-09-21T03:43:45.029438+0000 (oldest deadline 2024-09-21T03:44:10.329391+0000) 2024-09-21T03:44:21.712 INFO:journalctl@ceph.osd.8.smithi114.stdout:Sep 21 03:44:21 smithi114 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-8[51702]: 2024-09-21T03:44:21.256+0000 7f3e66f25640 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.110:6812 osd.4 since back 2024-09-21T03:43:50.330091+0000 front 2024-09-21T03:43:50.330000+0000 (oldest deadline 2024-09-21T03:44:16.229845+0000) 2024-09-21T03:44:21.712 INFO:journalctl@ceph.osd.11.smithi114.stdout:Sep 21 03:44:21 smithi114 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-11[58313]: 2024-09-21T03:44:21.258+0000 7f7657ce9640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2024-09-21T03:43:19.721069+0000 front 2024-09-21T03:43:19.721119+0000 (oldest deadline 2024-09-21T03:43:47.321877+0000) 2024-09-21T03:44:21.712 INFO:journalctl@ceph.osd.11.smithi114.stdout:Sep 21 03:44:21 smithi114 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-11[58313]: 2024-09-21T03:44:21.258+0000 7f7657ce9640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-21T03:43:27.322375+0000 front 2024-09-21T03:43:27.322426+0000 (oldest deadline 2024-09-21T03:43:51.422473+0000) 2024-09-21T03:44:21.712 INFO:journalctl@ceph.osd.11.smithi114.stdout:Sep 21 03:44:21 smithi114 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-11[58313]: 2024-09-21T03:44:21.258+0000 7f7657ce9640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.114:6804 osd.2 since back 2024-09-21T03:43:37.224230+0000 front 2024-09-21T03:43:37.224148+0000 (oldest deadline 2024-09-21T03:43:58.924202+0000) 2024-09-21T03:44:21.712 INFO:journalctl@ceph.osd.11.smithi114.stdout:Sep 21 03:44:21 smithi114 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-11[58313]: 2024-09-21T03:44:21.258+0000 7f7657ce9640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.110:6812 osd.4 since back 2024-09-21T03:43:50.626287+0000 front 2024-09-21T03:43:50.626333+0000 (oldest deadline 2024-09-21T03:44:15.326365+0000) 2024-09-21T03:44:21.811 INFO:journalctl@ceph.osd.9.smithi089.stdout:Sep 21 03:44:21 smithi089 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-9[65404]: 2024-09-21T03:44:21.474+0000 7f6c4e7c5640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2024-09-21T03:43:18.845419+0000 front 2024-09-21T03:43:18.845455+0000 (oldest deadline 2024-09-21T03:43:44.745556+0000) 2024-09-21T03:44:21.811 INFO:journalctl@ceph.osd.9.smithi089.stdout:Sep 21 03:44:21 smithi089 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-9[65404]: 2024-09-21T03:44:21.474+0000 7f6c4e7c5640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-21T03:43:24.746023+0000 front 2024-09-21T03:43:24.745970+0000 (oldest deadline 2024-09-21T03:43:50.646080+0000) 2024-09-21T03:44:21.811 INFO:journalctl@ceph.osd.9.smithi089.stdout:Sep 21 03:44:21 smithi089 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-9[65404]: 2024-09-21T03:44:21.474+0000 7f6c4e7c5640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.114:6804 osd.2 since back 2024-09-21T03:43:34.147460+0000 front 2024-09-21T03:43:34.147364+0000 (oldest deadline 2024-09-21T03:43:58.247427+0000) 2024-09-21T03:44:21.811 INFO:journalctl@ceph.osd.9.smithi089.stdout:Sep 21 03:44:21 smithi089 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-9[65404]: 2024-09-21T03:44:21.474+0000 7f6c4e7c5640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.89:6814 osd.3 since back 2024-09-21T03:43:39.848901+0000 front 2024-09-21T03:43:39.848935+0000 (oldest deadline 2024-09-21T03:44:09.849669+0000) 2024-09-21T03:44:21.811 INFO:journalctl@ceph.osd.9.smithi089.stdout:Sep 21 03:44:21 smithi089 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-9[65404]: 2024-09-21T03:44:21.474+0000 7f6c4e7c5640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.110:6812 osd.4 since back 2024-09-21T03:43:49.850335+0000 front 2024-09-21T03:43:49.850307+0000 (oldest deadline 2024-09-21T03:44:13.950181+0000) 2024-09-21T03:44:22.500 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 21 03:44:22 smithi110 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-10[57089]: 2024-09-21T03:44:22.238+0000 7f9d479dc640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2024-09-21T03:43:21.264189+0000 front 2024-09-21T03:43:21.264118+0000 (oldest deadline 2024-09-21T03:43:43.564295+0000) 2024-09-21T03:44:22.500 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 21 03:44:22 smithi110 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-10[57089]: 2024-09-21T03:44:22.238+0000 7f9d479dc640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-21T03:43:25.265351+0000 front 2024-09-21T03:43:25.265386+0000 (oldest deadline 2024-09-21T03:43:49.965504+0000) 2024-09-21T03:44:22.500 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 21 03:44:22 smithi110 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-10[57089]: 2024-09-21T03:44:22.238+0000 7f9d479dc640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.114:6804 osd.2 since back 2024-09-21T03:43:33.966726+0000 front 2024-09-21T03:43:33.966884+0000 (oldest deadline 2024-09-21T03:43:59.866510+0000) 2024-09-21T03:44:22.500 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 21 03:44:22 smithi110 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-10[57089]: 2024-09-21T03:44:22.238+0000 7f9d479dc640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.89:6814 osd.3 since back 2024-09-21T03:43:43.367532+0000 front 2024-09-21T03:43:43.367626+0000 (oldest deadline 2024-09-21T03:44:06.867392+0000) 2024-09-21T03:44:22.500 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 21 03:44:22 smithi110 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-10[57089]: 2024-09-21T03:44:22.238+0000 7f9d479dc640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.110:6812 osd.4 since back 2024-09-21T03:43:48.568210+0000 front 2024-09-21T03:43:48.568248+0000 (oldest deadline 2024-09-21T03:44:14.468315+0000) 2024-09-21T03:44:22.501 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 21 03:44:22 smithi110 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-10[57089]: 2024-09-21T03:44:22.238+0000 7f9d479dc640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.114:6812 osd.5 since back 2024-09-21T03:43:56.769315+0000 front 2024-09-21T03:43:56.769405+0000 (oldest deadline 2024-09-21T03:44:22.069274+0000) 2024-09-21T03:44:22.711 INFO:journalctl@ceph.osd.8.smithi114.stdout:Sep 21 03:44:22 smithi114 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-8[51702]: 2024-09-21T03:44:22.283+0000 7f3e66f25640 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-21T03:43:23.926943+0000 front 2024-09-21T03:43:23.926870+0000 (oldest deadline 2024-09-21T03:43:52.127434+0000) 2024-09-21T03:44:22.711 INFO:journalctl@ceph.osd.8.smithi114.stdout:Sep 21 03:44:22 smithi114 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-8[51702]: 2024-09-21T03:44:22.283+0000 7f3e66f25640 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.114:6804 osd.2 since back 2024-09-21T03:43:32.128066+0000 front 2024-09-21T03:43:32.127963+0000 (oldest deadline 2024-09-21T03:43:58.028025+0000) 2024-09-21T03:44:22.711 INFO:journalctl@ceph.osd.8.smithi114.stdout:Sep 21 03:44:22 smithi114 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-8[51702]: 2024-09-21T03:44:22.283+0000 7f3e66f25640 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.89:6814 osd.3 since back 2024-09-21T03:43:45.029373+0000 front 2024-09-21T03:43:45.029438+0000 (oldest deadline 2024-09-21T03:44:10.329391+0000) 2024-09-21T03:44:22.711 INFO:journalctl@ceph.osd.8.smithi114.stdout:Sep 21 03:44:22 smithi114 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-8[51702]: 2024-09-21T03:44:22.283+0000 7f3e66f25640 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.110:6812 osd.4 since back 2024-09-21T03:43:50.330091+0000 front 2024-09-21T03:43:50.330000+0000 (oldest deadline 2024-09-21T03:44:16.229845+0000) 2024-09-21T03:44:22.711 INFO:journalctl@ceph.osd.11.smithi114.stdout:Sep 21 03:44:22 smithi114 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-11[58313]: 2024-09-21T03:44:22.294+0000 7f7657ce9640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2024-09-21T03:43:19.721069+0000 front 2024-09-21T03:43:19.721119+0000 (oldest deadline 2024-09-21T03:43:47.321877+0000) 2024-09-21T03:44:22.712 INFO:journalctl@ceph.osd.11.smithi114.stdout:Sep 21 03:44:22 smithi114 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-11[58313]: 2024-09-21T03:44:22.294+0000 7f7657ce9640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-21T03:43:27.322375+0000 front 2024-09-21T03:43:27.322426+0000 (oldest deadline 2024-09-21T03:43:51.422473+0000) 2024-09-21T03:44:22.712 INFO:journalctl@ceph.osd.11.smithi114.stdout:Sep 21 03:44:22 smithi114 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-11[58313]: 2024-09-21T03:44:22.294+0000 7f7657ce9640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.114:6804 osd.2 since back 2024-09-21T03:43:37.224230+0000 front 2024-09-21T03:43:37.224148+0000 (oldest deadline 2024-09-21T03:43:58.924202+0000) 2024-09-21T03:44:22.712 INFO:journalctl@ceph.osd.11.smithi114.stdout:Sep 21 03:44:22 smithi114 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-11[58313]: 2024-09-21T03:44:22.294+0000 7f7657ce9640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.110:6812 osd.4 since back 2024-09-21T03:43:50.626287+0000 front 2024-09-21T03:43:50.626333+0000 (oldest deadline 2024-09-21T03:44:15.326365+0000) 2024-09-21T03:44:22.811 INFO:journalctl@ceph.osd.9.smithi089.stdout:Sep 21 03:44:22 smithi089 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-9[65404]: 2024-09-21T03:44:22.490+0000 7f6c4e7c5640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2024-09-21T03:43:18.845419+0000 front 2024-09-21T03:43:18.845455+0000 (oldest deadline 2024-09-21T03:43:44.745556+0000) 2024-09-21T03:44:22.811 INFO:journalctl@ceph.osd.9.smithi089.stdout:Sep 21 03:44:22 smithi089 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-9[65404]: 2024-09-21T03:44:22.490+0000 7f6c4e7c5640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-21T03:43:24.746023+0000 front 2024-09-21T03:43:24.745970+0000 (oldest deadline 2024-09-21T03:43:50.646080+0000) 2024-09-21T03:44:22.811 INFO:journalctl@ceph.osd.9.smithi089.stdout:Sep 21 03:44:22 smithi089 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-9[65404]: 2024-09-21T03:44:22.490+0000 7f6c4e7c5640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.114:6804 osd.2 since back 2024-09-21T03:43:34.147460+0000 front 2024-09-21T03:43:34.147364+0000 (oldest deadline 2024-09-21T03:43:58.247427+0000) 2024-09-21T03:44:22.811 INFO:journalctl@ceph.osd.9.smithi089.stdout:Sep 21 03:44:22 smithi089 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-9[65404]: 2024-09-21T03:44:22.490+0000 7f6c4e7c5640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.89:6814 osd.3 since back 2024-09-21T03:43:39.848901+0000 front 2024-09-21T03:43:39.848935+0000 (oldest deadline 2024-09-21T03:44:09.849669+0000) 2024-09-21T03:44:22.811 INFO:journalctl@ceph.osd.9.smithi089.stdout:Sep 21 03:44:22 smithi089 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-9[65404]: 2024-09-21T03:44:22.490+0000 7f6c4e7c5640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.110:6812 osd.4 since back 2024-09-21T03:43:49.850335+0000 front 2024-09-21T03:43:49.850307+0000 (oldest deadline 2024-09-21T03:44:13.950181+0000) 2024-09-21T03:44:23.711 INFO:journalctl@ceph.osd.8.smithi114.stdout:Sep 21 03:44:23 smithi114 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-8[51702]: 2024-09-21T03:44:23.278+0000 7f3e66f25640 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-21T03:43:23.926943+0000 front 2024-09-21T03:43:23.926870+0000 (oldest deadline 2024-09-21T03:43:52.127434+0000) 2024-09-21T03:44:23.711 INFO:journalctl@ceph.osd.8.smithi114.stdout:Sep 21 03:44:23 smithi114 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-8[51702]: 2024-09-21T03:44:23.278+0000 7f3e66f25640 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.114:6804 osd.2 since back 2024-09-21T03:43:32.128066+0000 front 2024-09-21T03:43:32.127963+0000 (oldest deadline 2024-09-21T03:43:58.028025+0000) 2024-09-21T03:44:23.711 INFO:journalctl@ceph.osd.8.smithi114.stdout:Sep 21 03:44:23 smithi114 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-8[51702]: 2024-09-21T03:44:23.278+0000 7f3e66f25640 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.89:6814 osd.3 since back 2024-09-21T03:43:45.029373+0000 front 2024-09-21T03:43:45.029438+0000 (oldest deadline 2024-09-21T03:44:10.329391+0000) 2024-09-21T03:44:23.711 INFO:journalctl@ceph.osd.8.smithi114.stdout:Sep 21 03:44:23 smithi114 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-8[51702]: 2024-09-21T03:44:23.278+0000 7f3e66f25640 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.110:6812 osd.4 since back 2024-09-21T03:43:50.330091+0000 front 2024-09-21T03:43:50.330000+0000 (oldest deadline 2024-09-21T03:44:16.229845+0000) 2024-09-21T03:44:23.711 INFO:journalctl@ceph.osd.11.smithi114.stdout:Sep 21 03:44:23 smithi114 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-11[58313]: 2024-09-21T03:44:23.299+0000 7f7657ce9640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2024-09-21T03:43:19.721069+0000 front 2024-09-21T03:43:19.721119+0000 (oldest deadline 2024-09-21T03:43:47.321877+0000) 2024-09-21T03:44:23.711 INFO:journalctl@ceph.osd.11.smithi114.stdout:Sep 21 03:44:23 smithi114 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-11[58313]: 2024-09-21T03:44:23.299+0000 7f7657ce9640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-21T03:43:27.322375+0000 front 2024-09-21T03:43:27.322426+0000 (oldest deadline 2024-09-21T03:43:51.422473+0000) 2024-09-21T03:44:23.711 INFO:journalctl@ceph.osd.11.smithi114.stdout:Sep 21 03:44:23 smithi114 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-11[58313]: 2024-09-21T03:44:23.299+0000 7f7657ce9640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.114:6804 osd.2 since back 2024-09-21T03:43:37.224230+0000 front 2024-09-21T03:43:37.224148+0000 (oldest deadline 2024-09-21T03:43:58.924202+0000) 2024-09-21T03:44:23.711 INFO:journalctl@ceph.osd.11.smithi114.stdout:Sep 21 03:44:23 smithi114 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-11[58313]: 2024-09-21T03:44:23.299+0000 7f7657ce9640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.110:6812 osd.4 since back 2024-09-21T03:43:50.626287+0000 front 2024-09-21T03:43:50.626333+0000 (oldest deadline 2024-09-21T03:44:15.326365+0000) 2024-09-21T03:44:23.750 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 21 03:44:23 smithi110 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-10[57089]: 2024-09-21T03:44:23.280+0000 7f9d479dc640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2024-09-21T03:43:21.264189+0000 front 2024-09-21T03:43:21.264118+0000 (oldest deadline 2024-09-21T03:43:43.564295+0000) 2024-09-21T03:44:23.750 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 21 03:44:23 smithi110 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-10[57089]: 2024-09-21T03:44:23.280+0000 7f9d479dc640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-21T03:43:25.265351+0000 front 2024-09-21T03:43:25.265386+0000 (oldest deadline 2024-09-21T03:43:49.965504+0000) 2024-09-21T03:44:23.750 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 21 03:44:23 smithi110 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-10[57089]: 2024-09-21T03:44:23.280+0000 7f9d479dc640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.114:6804 osd.2 since back 2024-09-21T03:43:33.966726+0000 front 2024-09-21T03:43:33.966884+0000 (oldest deadline 2024-09-21T03:43:59.866510+0000) 2024-09-21T03:44:23.750 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 21 03:44:23 smithi110 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-10[57089]: 2024-09-21T03:44:23.280+0000 7f9d479dc640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.89:6814 osd.3 since back 2024-09-21T03:43:43.367532+0000 front 2024-09-21T03:43:43.367626+0000 (oldest deadline 2024-09-21T03:44:06.867392+0000) 2024-09-21T03:44:23.750 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 21 03:44:23 smithi110 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-10[57089]: 2024-09-21T03:44:23.280+0000 7f9d479dc640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.110:6812 osd.4 since back 2024-09-21T03:43:48.568210+0000 front 2024-09-21T03:43:48.568248+0000 (oldest deadline 2024-09-21T03:44:14.468315+0000) 2024-09-21T03:44:23.750 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 21 03:44:23 smithi110 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-10[57089]: 2024-09-21T03:44:23.280+0000 7f9d479dc640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.114:6812 osd.5 since back 2024-09-21T03:43:56.769315+0000 front 2024-09-21T03:43:56.769405+0000 (oldest deadline 2024-09-21T03:44:22.069274+0000) 2024-09-21T03:44:23.811 INFO:journalctl@ceph.osd.9.smithi089.stdout:Sep 21 03:44:23 smithi089 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-9[65404]: 2024-09-21T03:44:23.529+0000 7f6c4e7c5640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2024-09-21T03:43:18.845419+0000 front 2024-09-21T03:43:18.845455+0000 (oldest deadline 2024-09-21T03:43:44.745556+0000) 2024-09-21T03:44:23.811 INFO:journalctl@ceph.osd.9.smithi089.stdout:Sep 21 03:44:23 smithi089 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-9[65404]: 2024-09-21T03:44:23.529+0000 7f6c4e7c5640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-21T03:43:24.746023+0000 front 2024-09-21T03:43:24.745970+0000 (oldest deadline 2024-09-21T03:43:50.646080+0000) 2024-09-21T03:44:23.811 INFO:journalctl@ceph.osd.9.smithi089.stdout:Sep 21 03:44:23 smithi089 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-9[65404]: 2024-09-21T03:44:23.529+0000 7f6c4e7c5640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.114:6804 osd.2 since back 2024-09-21T03:43:34.147460+0000 front 2024-09-21T03:43:34.147364+0000 (oldest deadline 2024-09-21T03:43:58.247427+0000) 2024-09-21T03:44:23.811 INFO:journalctl@ceph.osd.9.smithi089.stdout:Sep 21 03:44:23 smithi089 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-9[65404]: 2024-09-21T03:44:23.529+0000 7f6c4e7c5640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.89:6814 osd.3 since back 2024-09-21T03:43:39.848901+0000 front 2024-09-21T03:43:39.848935+0000 (oldest deadline 2024-09-21T03:44:09.849669+0000) 2024-09-21T03:44:23.811 INFO:journalctl@ceph.osd.9.smithi089.stdout:Sep 21 03:44:23 smithi089 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-9[65404]: 2024-09-21T03:44:23.529+0000 7f6c4e7c5640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.110:6812 osd.4 since back 2024-09-21T03:43:49.850335+0000 front 2024-09-21T03:43:49.850307+0000 (oldest deadline 2024-09-21T03:44:13.950181+0000) 2024-09-21T03:44:23.812 INFO:journalctl@ceph.osd.9.smithi089.stdout:Sep 21 03:44:23 smithi089 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-9[65404]: 2024-09-21T03:44:23.529+0000 7f6c4e7c5640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.114:6812 osd.5 since back 2024-09-21T03:43:58.551756+0000 front 2024-09-21T03:43:58.551778+0000 (oldest deadline 2024-09-21T03:44:23.251588+0000) 2024-09-21T03:44:24.500 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 21 03:44:24 smithi110 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-10[57089]: 2024-09-21T03:44:24.288+0000 7f9d479dc640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2024-09-21T03:43:21.264189+0000 front 2024-09-21T03:43:21.264118+0000 (oldest deadline 2024-09-21T03:43:43.564295+0000) 2024-09-21T03:44:24.500 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 21 03:44:24 smithi110 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-10[57089]: 2024-09-21T03:44:24.288+0000 7f9d479dc640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-21T03:43:25.265351+0000 front 2024-09-21T03:43:25.265386+0000 (oldest deadline 2024-09-21T03:43:49.965504+0000) 2024-09-21T03:44:24.500 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 21 03:44:24 smithi110 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-10[57089]: 2024-09-21T03:44:24.288+0000 7f9d479dc640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.114:6804 osd.2 since back 2024-09-21T03:43:33.966726+0000 front 2024-09-21T03:43:33.966884+0000 (oldest deadline 2024-09-21T03:43:59.866510+0000) 2024-09-21T03:44:24.501 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 21 03:44:24 smithi110 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-10[57089]: 2024-09-21T03:44:24.288+0000 7f9d479dc640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.89:6814 osd.3 since back 2024-09-21T03:43:43.367532+0000 front 2024-09-21T03:43:43.367626+0000 (oldest deadline 2024-09-21T03:44:06.867392+0000) 2024-09-21T03:44:24.501 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 21 03:44:24 smithi110 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-10[57089]: 2024-09-21T03:44:24.288+0000 7f9d479dc640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.110:6812 osd.4 since back 2024-09-21T03:43:48.568210+0000 front 2024-09-21T03:43:48.568248+0000 (oldest deadline 2024-09-21T03:44:14.468315+0000) 2024-09-21T03:44:24.501 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 21 03:44:24 smithi110 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-10[57089]: 2024-09-21T03:44:24.288+0000 7f9d479dc640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.114:6812 osd.5 since back 2024-09-21T03:43:56.769315+0000 front 2024-09-21T03:43:56.769405+0000 (oldest deadline 2024-09-21T03:44:22.069274+0000) 2024-09-21T03:44:24.711 INFO:journalctl@ceph.osd.8.smithi114.stdout:Sep 21 03:44:24 smithi114 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-8[51702]: 2024-09-21T03:44:24.240+0000 7f3e66f25640 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-21T03:43:23.926943+0000 front 2024-09-21T03:43:23.926870+0000 (oldest deadline 2024-09-21T03:43:52.127434+0000) 2024-09-21T03:44:24.711 INFO:journalctl@ceph.osd.8.smithi114.stdout:Sep 21 03:44:24 smithi114 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-8[51702]: 2024-09-21T03:44:24.240+0000 7f3e66f25640 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.114:6804 osd.2 since back 2024-09-21T03:43:32.128066+0000 front 2024-09-21T03:43:32.127963+0000 (oldest deadline 2024-09-21T03:43:58.028025+0000) 2024-09-21T03:44:24.711 INFO:journalctl@ceph.osd.8.smithi114.stdout:Sep 21 03:44:24 smithi114 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-8[51702]: 2024-09-21T03:44:24.240+0000 7f3e66f25640 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.89:6814 osd.3 since back 2024-09-21T03:43:45.029373+0000 front 2024-09-21T03:43:45.029438+0000 (oldest deadline 2024-09-21T03:44:10.329391+0000) 2024-09-21T03:44:24.711 INFO:journalctl@ceph.osd.8.smithi114.stdout:Sep 21 03:44:24 smithi114 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-8[51702]: 2024-09-21T03:44:24.240+0000 7f3e66f25640 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.110:6812 osd.4 since back 2024-09-21T03:43:50.330091+0000 front 2024-09-21T03:43:50.330000+0000 (oldest deadline 2024-09-21T03:44:16.229845+0000) 2024-09-21T03:44:24.711 INFO:journalctl@ceph.osd.11.smithi114.stdout:Sep 21 03:44:24 smithi114 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-11[58313]: 2024-09-21T03:44:24.295+0000 7f7657ce9640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2024-09-21T03:43:19.721069+0000 front 2024-09-21T03:43:19.721119+0000 (oldest deadline 2024-09-21T03:43:47.321877+0000) 2024-09-21T03:44:24.711 INFO:journalctl@ceph.osd.11.smithi114.stdout:Sep 21 03:44:24 smithi114 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-11[58313]: 2024-09-21T03:44:24.295+0000 7f7657ce9640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-21T03:43:27.322375+0000 front 2024-09-21T03:43:27.322426+0000 (oldest deadline 2024-09-21T03:43:51.422473+0000) 2024-09-21T03:44:24.711 INFO:journalctl@ceph.osd.11.smithi114.stdout:Sep 21 03:44:24 smithi114 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-11[58313]: 2024-09-21T03:44:24.295+0000 7f7657ce9640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.114:6804 osd.2 since back 2024-09-21T03:43:37.224230+0000 front 2024-09-21T03:43:37.224148+0000 (oldest deadline 2024-09-21T03:43:58.924202+0000) 2024-09-21T03:44:24.712 INFO:journalctl@ceph.osd.11.smithi114.stdout:Sep 21 03:44:24 smithi114 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-11[58313]: 2024-09-21T03:44:24.295+0000 7f7657ce9640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.110:6812 osd.4 since back 2024-09-21T03:43:50.626287+0000 front 2024-09-21T03:43:50.626333+0000 (oldest deadline 2024-09-21T03:44:15.326365+0000) 2024-09-21T03:44:25.061 INFO:journalctl@ceph.osd.9.smithi089.stdout:Sep 21 03:44:24 smithi089 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-9[65404]: 2024-09-21T03:44:24.562+0000 7f6c4e7c5640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2024-09-21T03:43:18.845419+0000 front 2024-09-21T03:43:18.845455+0000 (oldest deadline 2024-09-21T03:43:44.745556+0000) 2024-09-21T03:44:25.061 INFO:journalctl@ceph.osd.9.smithi089.stdout:Sep 21 03:44:24 smithi089 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-9[65404]: 2024-09-21T03:44:24.562+0000 7f6c4e7c5640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-21T03:43:24.746023+0000 front 2024-09-21T03:43:24.745970+0000 (oldest deadline 2024-09-21T03:43:50.646080+0000) 2024-09-21T03:44:25.061 INFO:journalctl@ceph.osd.9.smithi089.stdout:Sep 21 03:44:24 smithi089 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-9[65404]: 2024-09-21T03:44:24.562+0000 7f6c4e7c5640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.114:6804 osd.2 since back 2024-09-21T03:43:34.147460+0000 front 2024-09-21T03:43:34.147364+0000 (oldest deadline 2024-09-21T03:43:58.247427+0000) 2024-09-21T03:44:25.061 INFO:journalctl@ceph.osd.9.smithi089.stdout:Sep 21 03:44:24 smithi089 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-9[65404]: 2024-09-21T03:44:24.562+0000 7f6c4e7c5640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.89:6814 osd.3 since back 2024-09-21T03:43:39.848901+0000 front 2024-09-21T03:43:39.848935+0000 (oldest deadline 2024-09-21T03:44:09.849669+0000) 2024-09-21T03:44:25.062 INFO:journalctl@ceph.osd.9.smithi089.stdout:Sep 21 03:44:24 smithi089 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-9[65404]: 2024-09-21T03:44:24.562+0000 7f6c4e7c5640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.110:6812 osd.4 since back 2024-09-21T03:43:49.850335+0000 front 2024-09-21T03:43:49.850307+0000 (oldest deadline 2024-09-21T03:44:13.950181+0000) 2024-09-21T03:44:25.062 INFO:journalctl@ceph.osd.9.smithi089.stdout:Sep 21 03:44:24 smithi089 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-9[65404]: 2024-09-21T03:44:24.562+0000 7f6c4e7c5640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.114:6812 osd.5 since back 2024-09-21T03:43:58.551756+0000 front 2024-09-21T03:43:58.551778+0000 (oldest deadline 2024-09-21T03:44:23.251588+0000) 2024-09-21T03:44:25.500 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 21 03:44:25 smithi110 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-10[57089]: 2024-09-21T03:44:25.245+0000 7f9d479dc640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2024-09-21T03:43:21.264189+0000 front 2024-09-21T03:43:21.264118+0000 (oldest deadline 2024-09-21T03:43:43.564295+0000) 2024-09-21T03:44:25.500 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 21 03:44:25 smithi110 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-10[57089]: 2024-09-21T03:44:25.245+0000 7f9d479dc640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-21T03:43:25.265351+0000 front 2024-09-21T03:43:25.265386+0000 (oldest deadline 2024-09-21T03:43:49.965504+0000) 2024-09-21T03:44:25.500 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 21 03:44:25 smithi110 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-10[57089]: 2024-09-21T03:44:25.245+0000 7f9d479dc640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.114:6804 osd.2 since back 2024-09-21T03:43:33.966726+0000 front 2024-09-21T03:43:33.966884+0000 (oldest deadline 2024-09-21T03:43:59.866510+0000) 2024-09-21T03:44:25.500 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 21 03:44:25 smithi110 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-10[57089]: 2024-09-21T03:44:25.245+0000 7f9d479dc640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.89:6814 osd.3 since back 2024-09-21T03:43:43.367532+0000 front 2024-09-21T03:43:43.367626+0000 (oldest deadline 2024-09-21T03:44:06.867392+0000) 2024-09-21T03:44:25.500 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 21 03:44:25 smithi110 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-10[57089]: 2024-09-21T03:44:25.245+0000 7f9d479dc640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.110:6812 osd.4 since back 2024-09-21T03:43:48.568210+0000 front 2024-09-21T03:43:48.568248+0000 (oldest deadline 2024-09-21T03:44:14.468315+0000) 2024-09-21T03:44:25.500 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 21 03:44:25 smithi110 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-10[57089]: 2024-09-21T03:44:25.245+0000 7f9d479dc640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.114:6812 osd.5 since back 2024-09-21T03:43:56.769315+0000 front 2024-09-21T03:43:56.769405+0000 (oldest deadline 2024-09-21T03:44:22.069274+0000) 2024-09-21T03:44:25.711 INFO:journalctl@ceph.osd.8.smithi114.stdout:Sep 21 03:44:25 smithi114 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-8[51702]: 2024-09-21T03:44:25.290+0000 7f3e66f25640 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-21T03:43:23.926943+0000 front 2024-09-21T03:43:23.926870+0000 (oldest deadline 2024-09-21T03:43:52.127434+0000) 2024-09-21T03:44:25.711 INFO:journalctl@ceph.osd.8.smithi114.stdout:Sep 21 03:44:25 smithi114 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-8[51702]: 2024-09-21T03:44:25.290+0000 7f3e66f25640 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.114:6804 osd.2 since back 2024-09-21T03:43:32.128066+0000 front 2024-09-21T03:43:32.127963+0000 (oldest deadline 2024-09-21T03:43:58.028025+0000) 2024-09-21T03:44:25.711 INFO:journalctl@ceph.osd.8.smithi114.stdout:Sep 21 03:44:25 smithi114 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-8[51702]: 2024-09-21T03:44:25.290+0000 7f3e66f25640 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.89:6814 osd.3 since back 2024-09-21T03:43:45.029373+0000 front 2024-09-21T03:43:45.029438+0000 (oldest deadline 2024-09-21T03:44:10.329391+0000) 2024-09-21T03:44:25.711 INFO:journalctl@ceph.osd.8.smithi114.stdout:Sep 21 03:44:25 smithi114 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-8[51702]: 2024-09-21T03:44:25.290+0000 7f3e66f25640 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.110:6812 osd.4 since back 2024-09-21T03:43:50.330091+0000 front 2024-09-21T03:43:50.330000+0000 (oldest deadline 2024-09-21T03:44:16.229845+0000) 2024-09-21T03:44:25.711 INFO:journalctl@ceph.osd.11.smithi114.stdout:Sep 21 03:44:25 smithi114 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-11[58313]: 2024-09-21T03:44:25.281+0000 7f7657ce9640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2024-09-21T03:43:19.721069+0000 front 2024-09-21T03:43:19.721119+0000 (oldest deadline 2024-09-21T03:43:47.321877+0000) 2024-09-21T03:44:25.711 INFO:journalctl@ceph.osd.11.smithi114.stdout:Sep 21 03:44:25 smithi114 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-11[58313]: 2024-09-21T03:44:25.281+0000 7f7657ce9640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-21T03:43:27.322375+0000 front 2024-09-21T03:43:27.322426+0000 (oldest deadline 2024-09-21T03:43:51.422473+0000) 2024-09-21T03:44:25.712 INFO:journalctl@ceph.osd.11.smithi114.stdout:Sep 21 03:44:25 smithi114 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-11[58313]: 2024-09-21T03:44:25.281+0000 7f7657ce9640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.114:6804 osd.2 since back 2024-09-21T03:43:37.224230+0000 front 2024-09-21T03:43:37.224148+0000 (oldest deadline 2024-09-21T03:43:58.924202+0000) 2024-09-21T03:44:25.712 INFO:journalctl@ceph.osd.11.smithi114.stdout:Sep 21 03:44:25 smithi114 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-11[58313]: 2024-09-21T03:44:25.281+0000 7f7657ce9640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.110:6812 osd.4 since back 2024-09-21T03:43:50.626287+0000 front 2024-09-21T03:43:50.626333+0000 (oldest deadline 2024-09-21T03:44:15.326365+0000) 2024-09-21T03:44:25.811 INFO:journalctl@ceph.osd.9.smithi089.stdout:Sep 21 03:44:25 smithi089 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-9[65404]: 2024-09-21T03:44:25.557+0000 7f6c4e7c5640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2024-09-21T03:43:18.845419+0000 front 2024-09-21T03:43:18.845455+0000 (oldest deadline 2024-09-21T03:43:44.745556+0000) 2024-09-21T03:44:25.811 INFO:journalctl@ceph.osd.9.smithi089.stdout:Sep 21 03:44:25 smithi089 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-9[65404]: 2024-09-21T03:44:25.557+0000 7f6c4e7c5640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-21T03:43:24.746023+0000 front 2024-09-21T03:43:24.745970+0000 (oldest deadline 2024-09-21T03:43:50.646080+0000) 2024-09-21T03:44:25.811 INFO:journalctl@ceph.osd.9.smithi089.stdout:Sep 21 03:44:25 smithi089 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-9[65404]: 2024-09-21T03:44:25.557+0000 7f6c4e7c5640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.114:6804 osd.2 since back 2024-09-21T03:43:34.147460+0000 front 2024-09-21T03:43:34.147364+0000 (oldest deadline 2024-09-21T03:43:58.247427+0000) 2024-09-21T03:44:25.811 INFO:journalctl@ceph.osd.9.smithi089.stdout:Sep 21 03:44:25 smithi089 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-9[65404]: 2024-09-21T03:44:25.557+0000 7f6c4e7c5640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.89:6814 osd.3 since back 2024-09-21T03:43:39.848901+0000 front 2024-09-21T03:43:39.848935+0000 (oldest deadline 2024-09-21T03:44:09.849669+0000) 2024-09-21T03:44:25.812 INFO:journalctl@ceph.osd.9.smithi089.stdout:Sep 21 03:44:25 smithi089 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-9[65404]: 2024-09-21T03:44:25.557+0000 7f6c4e7c5640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.110:6812 osd.4 since back 2024-09-21T03:43:49.850335+0000 front 2024-09-21T03:43:49.850307+0000 (oldest deadline 2024-09-21T03:44:13.950181+0000) 2024-09-21T03:44:25.812 INFO:journalctl@ceph.osd.9.smithi089.stdout:Sep 21 03:44:25 smithi089 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-9[65404]: 2024-09-21T03:44:25.557+0000 7f6c4e7c5640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.114:6812 osd.5 since back 2024-09-21T03:43:58.551756+0000 front 2024-09-21T03:43:58.551778+0000 (oldest deadline 2024-09-21T03:44:23.251588+0000) 2024-09-21T03:44:25.990 INFO:journalctl@ceph.osd.8.smithi114.stdout:Sep 21 03:44:25 smithi114 podman[66379]: 2024-09-21 03:44:25.715878763 +0000 UTC m=+5.537599425 container died 275dd45675fc4d0438a241ba63aa6df139306cfef6eeb372b3db7f107e64b9c4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-8, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-21T03:44:26.249 INFO:journalctl@ceph.osd.8.smithi114.stdout:Sep 21 03:44:26 smithi114 podman[66379]: 2024-09-21 03:44:26.088278398 +0000 UTC m=+5.909999054 container cleanup 275dd45675fc4d0438a241ba63aa6df139306cfef6eeb372b3db7f107e64b9c4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-8, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-21T03:44:26.249 INFO:journalctl@ceph.osd.8.smithi114.stdout:Sep 21 03:44:26 smithi114 bash[66379]: ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-8 2024-09-21T03:44:26.702 INFO:journalctl@ceph.osd.8.smithi114.stdout:Sep 21 03:44:26 smithi114 podman[66395]: 2024-09-21 03:44:26.373121632 +0000 UTC m=+0.653442122 container remove 275dd45675fc4d0438a241ba63aa6df139306cfef6eeb372b3db7f107e64b9c4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-8, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-09-21T03:44:26.702 INFO:journalctl@ceph.osd.11.smithi114.stdout:Sep 21 03:44:26 smithi114 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-11[58313]: 2024-09-21T03:44:26.247+0000 7f7657ce9640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2024-09-21T03:43:19.721069+0000 front 2024-09-21T03:43:19.721119+0000 (oldest deadline 2024-09-21T03:43:47.321877+0000) 2024-09-21T03:44:26.703 INFO:journalctl@ceph.osd.11.smithi114.stdout:Sep 21 03:44:26 smithi114 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-11[58313]: 2024-09-21T03:44:26.247+0000 7f7657ce9640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-21T03:43:27.322375+0000 front 2024-09-21T03:43:27.322426+0000 (oldest deadline 2024-09-21T03:43:51.422473+0000) 2024-09-21T03:44:26.703 INFO:journalctl@ceph.osd.11.smithi114.stdout:Sep 21 03:44:26 smithi114 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-11[58313]: 2024-09-21T03:44:26.247+0000 7f7657ce9640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.114:6804 osd.2 since back 2024-09-21T03:43:37.224230+0000 front 2024-09-21T03:43:37.224148+0000 (oldest deadline 2024-09-21T03:43:58.924202+0000) 2024-09-21T03:44:26.703 INFO:journalctl@ceph.osd.11.smithi114.stdout:Sep 21 03:44:26 smithi114 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-11[58313]: 2024-09-21T03:44:26.247+0000 7f7657ce9640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.110:6812 osd.4 since back 2024-09-21T03:43:50.626287+0000 front 2024-09-21T03:43:50.626333+0000 (oldest deadline 2024-09-21T03:44:15.326365+0000) 2024-09-21T03:44:26.750 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 21 03:44:26 smithi110 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-10[57089]: 2024-09-21T03:44:26.282+0000 7f9d479dc640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2024-09-21T03:43:21.264189+0000 front 2024-09-21T03:43:21.264118+0000 (oldest deadline 2024-09-21T03:43:43.564295+0000) 2024-09-21T03:44:26.750 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 21 03:44:26 smithi110 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-10[57089]: 2024-09-21T03:44:26.282+0000 7f9d479dc640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-21T03:43:25.265351+0000 front 2024-09-21T03:43:25.265386+0000 (oldest deadline 2024-09-21T03:43:49.965504+0000) 2024-09-21T03:44:26.750 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 21 03:44:26 smithi110 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-10[57089]: 2024-09-21T03:44:26.282+0000 7f9d479dc640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.114:6804 osd.2 since back 2024-09-21T03:43:33.966726+0000 front 2024-09-21T03:43:33.966884+0000 (oldest deadline 2024-09-21T03:43:59.866510+0000) 2024-09-21T03:44:26.750 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 21 03:44:26 smithi110 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-10[57089]: 2024-09-21T03:44:26.282+0000 7f9d479dc640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.89:6814 osd.3 since back 2024-09-21T03:43:43.367532+0000 front 2024-09-21T03:43:43.367626+0000 (oldest deadline 2024-09-21T03:44:06.867392+0000) 2024-09-21T03:44:26.750 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 21 03:44:26 smithi110 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-10[57089]: 2024-09-21T03:44:26.282+0000 7f9d479dc640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.110:6812 osd.4 since back 2024-09-21T03:43:48.568210+0000 front 2024-09-21T03:43:48.568248+0000 (oldest deadline 2024-09-21T03:44:14.468315+0000) 2024-09-21T03:44:26.750 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 21 03:44:26 smithi110 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-10[57089]: 2024-09-21T03:44:26.282+0000 7f9d479dc640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.114:6812 osd.5 since back 2024-09-21T03:43:56.769315+0000 front 2024-09-21T03:43:56.769405+0000 (oldest deadline 2024-09-21T03:44:22.069274+0000) 2024-09-21T03:44:26.961 INFO:journalctl@ceph.osd.8.smithi114.stdout:Sep 21 03:44:26 smithi114 podman[66484]: 2024-09-21 03:44:26.603602587 +0000 UTC m=+0.015804853 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T03:44:26.961 INFO:journalctl@ceph.osd.8.smithi114.stdout:Sep 21 03:44:26 smithi114 podman[66484]: 2024-09-21 03:44:26.707158315 +0000 UTC m=+0.119360570 container create 20498ce416193bd437dfb8fcc830c1ea04ac03244649e2681e664a64e9b8fc30 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-8-deactivate, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-21T03:44:26.961 INFO:journalctl@ceph.osd.8.smithi114.stdout:Sep 21 03:44:26 smithi114 podman[66484]: 2024-09-21 03:44:26.926453297 +0000 UTC m=+0.338655557 container init 20498ce416193bd437dfb8fcc830c1ea04ac03244649e2681e664a64e9b8fc30 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-8-deactivate, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.37.2) 2024-09-21T03:44:26.961 INFO:journalctl@ceph.osd.8.smithi114.stdout:Sep 21 03:44:26 smithi114 podman[66484]: 2024-09-21 03:44:26.929706715 +0000 UTC m=+0.341908970 container start 20498ce416193bd437dfb8fcc830c1ea04ac03244649e2681e664a64e9b8fc30 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-8-deactivate, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-09-21T03:44:27.061 INFO:journalctl@ceph.osd.9.smithi089.stdout:Sep 21 03:44:26 smithi089 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-9[65404]: 2024-09-21T03:44:26.600+0000 7f6c4e7c5640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2024-09-21T03:43:18.845419+0000 front 2024-09-21T03:43:18.845455+0000 (oldest deadline 2024-09-21T03:43:44.745556+0000) 2024-09-21T03:44:27.061 INFO:journalctl@ceph.osd.9.smithi089.stdout:Sep 21 03:44:26 smithi089 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-9[65404]: 2024-09-21T03:44:26.600+0000 7f6c4e7c5640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-21T03:43:24.746023+0000 front 2024-09-21T03:43:24.745970+0000 (oldest deadline 2024-09-21T03:43:50.646080+0000) 2024-09-21T03:44:27.062 INFO:journalctl@ceph.osd.9.smithi089.stdout:Sep 21 03:44:26 smithi089 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-9[65404]: 2024-09-21T03:44:26.600+0000 7f6c4e7c5640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.114:6804 osd.2 since back 2024-09-21T03:43:34.147460+0000 front 2024-09-21T03:43:34.147364+0000 (oldest deadline 2024-09-21T03:43:58.247427+0000) 2024-09-21T03:44:27.062 INFO:journalctl@ceph.osd.9.smithi089.stdout:Sep 21 03:44:26 smithi089 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-9[65404]: 2024-09-21T03:44:26.600+0000 7f6c4e7c5640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.89:6814 osd.3 since back 2024-09-21T03:43:39.848901+0000 front 2024-09-21T03:43:39.848935+0000 (oldest deadline 2024-09-21T03:44:09.849669+0000) 2024-09-21T03:44:27.062 INFO:journalctl@ceph.osd.9.smithi089.stdout:Sep 21 03:44:26 smithi089 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-9[65404]: 2024-09-21T03:44:26.600+0000 7f6c4e7c5640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.110:6812 osd.4 since back 2024-09-21T03:43:49.850335+0000 front 2024-09-21T03:43:49.850307+0000 (oldest deadline 2024-09-21T03:44:13.950181+0000) 2024-09-21T03:44:27.062 INFO:journalctl@ceph.osd.9.smithi089.stdout:Sep 21 03:44:26 smithi089 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-9[65404]: 2024-09-21T03:44:26.600+0000 7f6c4e7c5640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.114:6812 osd.5 since back 2024-09-21T03:43:58.551756+0000 front 2024-09-21T03:43:58.551778+0000 (oldest deadline 2024-09-21T03:44:23.251588+0000) 2024-09-21T03:44:27.271 INFO:journalctl@ceph.osd.8.smithi114.stdout:Sep 21 03:44:27 smithi114 podman[66484]: 2024-09-21 03:44:27.001524921 +0000 UTC m=+0.413727181 container attach 20498ce416193bd437dfb8fcc830c1ea04ac03244649e2681e664a64e9b8fc30 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-8-deactivate, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221) 2024-09-21T03:44:27.271 INFO:journalctl@ceph.osd.8.smithi114.stdout:Sep 21 03:44:27 smithi114 podman[66484]: 2024-09-21 03:44:27.113363133 +0000 UTC m=+0.525565393 container died 20498ce416193bd437dfb8fcc830c1ea04ac03244649e2681e664a64e9b8fc30 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-8-deactivate, ceph=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD) 2024-09-21T03:44:27.500 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 21 03:44:27 smithi110 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-10[57089]: 2024-09-21T03:44:27.240+0000 7f9d479dc640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2024-09-21T03:43:21.264189+0000 front 2024-09-21T03:43:21.264118+0000 (oldest deadline 2024-09-21T03:43:43.564295+0000) 2024-09-21T03:44:27.500 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 21 03:44:27 smithi110 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-10[57089]: 2024-09-21T03:44:27.240+0000 7f9d479dc640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-21T03:43:25.265351+0000 front 2024-09-21T03:43:25.265386+0000 (oldest deadline 2024-09-21T03:43:49.965504+0000) 2024-09-21T03:44:27.500 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 21 03:44:27 smithi110 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-10[57089]: 2024-09-21T03:44:27.240+0000 7f9d479dc640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.114:6804 osd.2 since back 2024-09-21T03:43:33.966726+0000 front 2024-09-21T03:43:33.966884+0000 (oldest deadline 2024-09-21T03:43:59.866510+0000) 2024-09-21T03:44:27.500 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 21 03:44:27 smithi110 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-10[57089]: 2024-09-21T03:44:27.240+0000 7f9d479dc640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.89:6814 osd.3 since back 2024-09-21T03:43:43.367532+0000 front 2024-09-21T03:43:43.367626+0000 (oldest deadline 2024-09-21T03:44:06.867392+0000) 2024-09-21T03:44:27.500 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 21 03:44:27 smithi110 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-10[57089]: 2024-09-21T03:44:27.240+0000 7f9d479dc640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.110:6812 osd.4 since back 2024-09-21T03:43:48.568210+0000 front 2024-09-21T03:43:48.568248+0000 (oldest deadline 2024-09-21T03:44:14.468315+0000) 2024-09-21T03:44:27.500 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 21 03:44:27 smithi110 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-10[57089]: 2024-09-21T03:44:27.240+0000 7f9d479dc640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.114:6812 osd.5 since back 2024-09-21T03:43:56.769315+0000 front 2024-09-21T03:43:56.769405+0000 (oldest deadline 2024-09-21T03:44:22.069274+0000) 2024-09-21T03:44:27.711 INFO:journalctl@ceph.osd.11.smithi114.stdout:Sep 21 03:44:27 smithi114 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-11[58313]: 2024-09-21T03:44:27.269+0000 7f7657ce9640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2024-09-21T03:43:19.721069+0000 front 2024-09-21T03:43:19.721119+0000 (oldest deadline 2024-09-21T03:43:47.321877+0000) 2024-09-21T03:44:27.711 INFO:journalctl@ceph.osd.11.smithi114.stdout:Sep 21 03:44:27 smithi114 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-11[58313]: 2024-09-21T03:44:27.269+0000 7f7657ce9640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-21T03:43:27.322375+0000 front 2024-09-21T03:43:27.322426+0000 (oldest deadline 2024-09-21T03:43:51.422473+0000) 2024-09-21T03:44:27.711 INFO:journalctl@ceph.osd.11.smithi114.stdout:Sep 21 03:44:27 smithi114 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-11[58313]: 2024-09-21T03:44:27.269+0000 7f7657ce9640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.114:6804 osd.2 since back 2024-09-21T03:43:37.224230+0000 front 2024-09-21T03:43:37.224148+0000 (oldest deadline 2024-09-21T03:43:58.924202+0000) 2024-09-21T03:44:27.711 INFO:journalctl@ceph.osd.11.smithi114.stdout:Sep 21 03:44:27 smithi114 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-11[58313]: 2024-09-21T03:44:27.269+0000 7f7657ce9640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.110:6812 osd.4 since back 2024-09-21T03:43:50.626287+0000 front 2024-09-21T03:43:50.626333+0000 (oldest deadline 2024-09-21T03:44:15.326365+0000) 2024-09-21T03:44:27.711 INFO:journalctl@ceph.osd.11.smithi114.stdout:Sep 21 03:44:27 smithi114 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-11[58313]: 2024-09-21T03:44:27.269+0000 7f7657ce9640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.114:6812 osd.5 since back 2024-09-21T03:44:00.627730+0000 front 2024-09-21T03:44:00.627753+0000 (oldest deadline 2024-09-21T03:44:26.527718+0000) 2024-09-21T03:44:27.767 DEBUG:teuthology.orchestra.run.smithi114:> sudo pkill -f 'journalctl -f -n 0 -u ceph-7c374098-77c9-11ef-baf5-efdc52797490@osd.8.service' 2024-09-21T03:44:27.804 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T03:44:27.804 INFO:tasks.cephadm.osd.8:Stopped osd.8 2024-09-21T03:44:27.805 INFO:tasks.cephadm.osd.9:Stopping osd.9... 2024-09-21T03:44:27.805 DEBUG:teuthology.orchestra.run.smithi089:> sudo systemctl stop ceph-7c374098-77c9-11ef-baf5-efdc52797490@osd.9 2024-09-21T03:44:27.859 INFO:journalctl@ceph.osd.9.smithi089.stdout:Sep 21 03:44:27 smithi089 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-9[65404]: 2024-09-21T03:44:27.583+0000 7f6c4e7c5640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2024-09-21T03:43:18.845419+0000 front 2024-09-21T03:43:18.845455+0000 (oldest deadline 2024-09-21T03:43:44.745556+0000) 2024-09-21T03:44:27.859 INFO:journalctl@ceph.osd.9.smithi089.stdout:Sep 21 03:44:27 smithi089 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-9[65404]: 2024-09-21T03:44:27.583+0000 7f6c4e7c5640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-21T03:43:24.746023+0000 front 2024-09-21T03:43:24.745970+0000 (oldest deadline 2024-09-21T03:43:50.646080+0000) 2024-09-21T03:44:27.860 INFO:journalctl@ceph.osd.9.smithi089.stdout:Sep 21 03:44:27 smithi089 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-9[65404]: 2024-09-21T03:44:27.583+0000 7f6c4e7c5640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.114:6804 osd.2 since back 2024-09-21T03:43:34.147460+0000 front 2024-09-21T03:43:34.147364+0000 (oldest deadline 2024-09-21T03:43:58.247427+0000) 2024-09-21T03:44:27.860 INFO:journalctl@ceph.osd.9.smithi089.stdout:Sep 21 03:44:27 smithi089 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-9[65404]: 2024-09-21T03:44:27.583+0000 7f6c4e7c5640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.89:6814 osd.3 since back 2024-09-21T03:43:39.848901+0000 front 2024-09-21T03:43:39.848935+0000 (oldest deadline 2024-09-21T03:44:09.849669+0000) 2024-09-21T03:44:27.860 INFO:journalctl@ceph.osd.9.smithi089.stdout:Sep 21 03:44:27 smithi089 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-9[65404]: 2024-09-21T03:44:27.583+0000 7f6c4e7c5640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.110:6812 osd.4 since back 2024-09-21T03:43:49.850335+0000 front 2024-09-21T03:43:49.850307+0000 (oldest deadline 2024-09-21T03:44:13.950181+0000) 2024-09-21T03:44:27.860 INFO:journalctl@ceph.osd.9.smithi089.stdout:Sep 21 03:44:27 smithi089 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-9[65404]: 2024-09-21T03:44:27.583+0000 7f6c4e7c5640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.114:6812 osd.5 since back 2024-09-21T03:43:58.551756+0000 front 2024-09-21T03:43:58.551778+0000 (oldest deadline 2024-09-21T03:44:23.251588+0000) 2024-09-21T03:44:28.311 INFO:journalctl@ceph.osd.9.smithi089.stdout:Sep 21 03:44:27 smithi089 systemd[1]: Stopping Ceph osd.9 for 7c374098-77c9-11ef-baf5-efdc52797490... 2024-09-21T03:44:28.311 INFO:journalctl@ceph.osd.9.smithi089.stdout:Sep 21 03:44:28 smithi089 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-9[65404]: 2024-09-21T03:44:28.096+0000 7f6c521da640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.9 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T03:44:28.311 INFO:journalctl@ceph.osd.9.smithi089.stdout:Sep 21 03:44:28 smithi089 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-9[65404]: 2024-09-21T03:44:28.096+0000 7f6c521da640 -1 osd.9 88 *** Got signal Terminated *** 2024-09-21T03:44:28.311 INFO:journalctl@ceph.osd.9.smithi089.stdout:Sep 21 03:44:28 smithi089 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-9[65404]: 2024-09-21T03:44:28.096+0000 7f6c521da640 -1 osd.9 88 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-21T03:44:28.500 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 21 03:44:28 smithi110 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-10[57089]: 2024-09-21T03:44:28.209+0000 7f9d479dc640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2024-09-21T03:43:21.264189+0000 front 2024-09-21T03:43:21.264118+0000 (oldest deadline 2024-09-21T03:43:43.564295+0000) 2024-09-21T03:44:28.500 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 21 03:44:28 smithi110 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-10[57089]: 2024-09-21T03:44:28.209+0000 7f9d479dc640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-21T03:43:25.265351+0000 front 2024-09-21T03:43:25.265386+0000 (oldest deadline 2024-09-21T03:43:49.965504+0000) 2024-09-21T03:44:28.500 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 21 03:44:28 smithi110 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-10[57089]: 2024-09-21T03:44:28.209+0000 7f9d479dc640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.114:6804 osd.2 since back 2024-09-21T03:43:33.966726+0000 front 2024-09-21T03:43:33.966884+0000 (oldest deadline 2024-09-21T03:43:59.866510+0000) 2024-09-21T03:44:28.500 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 21 03:44:28 smithi110 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-10[57089]: 2024-09-21T03:44:28.209+0000 7f9d479dc640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.89:6814 osd.3 since back 2024-09-21T03:43:43.367532+0000 front 2024-09-21T03:43:43.367626+0000 (oldest deadline 2024-09-21T03:44:06.867392+0000) 2024-09-21T03:44:28.500 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 21 03:44:28 smithi110 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-10[57089]: 2024-09-21T03:44:28.209+0000 7f9d479dc640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.110:6812 osd.4 since back 2024-09-21T03:43:48.568210+0000 front 2024-09-21T03:43:48.568248+0000 (oldest deadline 2024-09-21T03:44:14.468315+0000) 2024-09-21T03:44:28.500 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 21 03:44:28 smithi110 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-10[57089]: 2024-09-21T03:44:28.209+0000 7f9d479dc640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.114:6812 osd.5 since back 2024-09-21T03:43:56.769315+0000 front 2024-09-21T03:43:56.769405+0000 (oldest deadline 2024-09-21T03:44:22.069274+0000) 2024-09-21T03:44:28.711 INFO:journalctl@ceph.osd.11.smithi114.stdout:Sep 21 03:44:28 smithi114 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-11[58313]: 2024-09-21T03:44:28.240+0000 7f7657ce9640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2024-09-21T03:43:19.721069+0000 front 2024-09-21T03:43:19.721119+0000 (oldest deadline 2024-09-21T03:43:47.321877+0000) 2024-09-21T03:44:28.711 INFO:journalctl@ceph.osd.11.smithi114.stdout:Sep 21 03:44:28 smithi114 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-11[58313]: 2024-09-21T03:44:28.240+0000 7f7657ce9640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-21T03:43:27.322375+0000 front 2024-09-21T03:43:27.322426+0000 (oldest deadline 2024-09-21T03:43:51.422473+0000) 2024-09-21T03:44:28.711 INFO:journalctl@ceph.osd.11.smithi114.stdout:Sep 21 03:44:28 smithi114 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-11[58313]: 2024-09-21T03:44:28.240+0000 7f7657ce9640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.114:6804 osd.2 since back 2024-09-21T03:43:37.224230+0000 front 2024-09-21T03:43:37.224148+0000 (oldest deadline 2024-09-21T03:43:58.924202+0000) 2024-09-21T03:44:28.711 INFO:journalctl@ceph.osd.11.smithi114.stdout:Sep 21 03:44:28 smithi114 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-11[58313]: 2024-09-21T03:44:28.240+0000 7f7657ce9640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.110:6812 osd.4 since back 2024-09-21T03:43:50.626287+0000 front 2024-09-21T03:43:50.626333+0000 (oldest deadline 2024-09-21T03:44:15.326365+0000) 2024-09-21T03:44:28.711 INFO:journalctl@ceph.osd.11.smithi114.stdout:Sep 21 03:44:28 smithi114 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-11[58313]: 2024-09-21T03:44:28.240+0000 7f7657ce9640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.114:6812 osd.5 since back 2024-09-21T03:44:00.627730+0000 front 2024-09-21T03:44:00.627753+0000 (oldest deadline 2024-09-21T03:44:26.527718+0000) 2024-09-21T03:44:28.811 INFO:journalctl@ceph.osd.9.smithi089.stdout:Sep 21 03:44:28 smithi089 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-9[65404]: 2024-09-21T03:44:28.554+0000 7f6c4e7c5640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2024-09-21T03:43:18.845419+0000 front 2024-09-21T03:43:18.845455+0000 (oldest deadline 2024-09-21T03:43:44.745556+0000) 2024-09-21T03:44:28.811 INFO:journalctl@ceph.osd.9.smithi089.stdout:Sep 21 03:44:28 smithi089 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-9[65404]: 2024-09-21T03:44:28.554+0000 7f6c4e7c5640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-21T03:43:24.746023+0000 front 2024-09-21T03:43:24.745970+0000 (oldest deadline 2024-09-21T03:43:50.646080+0000) 2024-09-21T03:44:28.811 INFO:journalctl@ceph.osd.9.smithi089.stdout:Sep 21 03:44:28 smithi089 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-9[65404]: 2024-09-21T03:44:28.554+0000 7f6c4e7c5640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.114:6804 osd.2 since back 2024-09-21T03:43:34.147460+0000 front 2024-09-21T03:43:34.147364+0000 (oldest deadline 2024-09-21T03:43:58.247427+0000) 2024-09-21T03:44:28.811 INFO:journalctl@ceph.osd.9.smithi089.stdout:Sep 21 03:44:28 smithi089 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-9[65404]: 2024-09-21T03:44:28.554+0000 7f6c4e7c5640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.89:6814 osd.3 since back 2024-09-21T03:43:39.848901+0000 front 2024-09-21T03:43:39.848935+0000 (oldest deadline 2024-09-21T03:44:09.849669+0000) 2024-09-21T03:44:28.811 INFO:journalctl@ceph.osd.9.smithi089.stdout:Sep 21 03:44:28 smithi089 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-9[65404]: 2024-09-21T03:44:28.554+0000 7f6c4e7c5640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.110:6812 osd.4 since back 2024-09-21T03:43:49.850335+0000 front 2024-09-21T03:43:49.850307+0000 (oldest deadline 2024-09-21T03:44:13.950181+0000) 2024-09-21T03:44:28.811 INFO:journalctl@ceph.osd.9.smithi089.stdout:Sep 21 03:44:28 smithi089 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-9[65404]: 2024-09-21T03:44:28.554+0000 7f6c4e7c5640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.114:6812 osd.5 since back 2024-09-21T03:43:58.551756+0000 front 2024-09-21T03:43:58.551778+0000 (oldest deadline 2024-09-21T03:44:23.251588+0000) 2024-09-21T03:44:29.500 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 21 03:44:29 smithi110 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-10[57089]: 2024-09-21T03:44:29.201+0000 7f9d479dc640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2024-09-21T03:43:21.264189+0000 front 2024-09-21T03:43:21.264118+0000 (oldest deadline 2024-09-21T03:43:43.564295+0000) 2024-09-21T03:44:29.500 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 21 03:44:29 smithi110 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-10[57089]: 2024-09-21T03:44:29.201+0000 7f9d479dc640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-21T03:43:25.265351+0000 front 2024-09-21T03:43:25.265386+0000 (oldest deadline 2024-09-21T03:43:49.965504+0000) 2024-09-21T03:44:29.500 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 21 03:44:29 smithi110 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-10[57089]: 2024-09-21T03:44:29.201+0000 7f9d479dc640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.114:6804 osd.2 since back 2024-09-21T03:43:33.966726+0000 front 2024-09-21T03:43:33.966884+0000 (oldest deadline 2024-09-21T03:43:59.866510+0000) 2024-09-21T03:44:29.500 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 21 03:44:29 smithi110 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-10[57089]: 2024-09-21T03:44:29.201+0000 7f9d479dc640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.89:6814 osd.3 since back 2024-09-21T03:43:43.367532+0000 front 2024-09-21T03:43:43.367626+0000 (oldest deadline 2024-09-21T03:44:06.867392+0000) 2024-09-21T03:44:29.500 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 21 03:44:29 smithi110 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-10[57089]: 2024-09-21T03:44:29.201+0000 7f9d479dc640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.110:6812 osd.4 since back 2024-09-21T03:43:48.568210+0000 front 2024-09-21T03:43:48.568248+0000 (oldest deadline 2024-09-21T03:44:14.468315+0000) 2024-09-21T03:44:29.501 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 21 03:44:29 smithi110 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-10[57089]: 2024-09-21T03:44:29.201+0000 7f9d479dc640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.114:6812 osd.5 since back 2024-09-21T03:43:56.769315+0000 front 2024-09-21T03:43:56.769405+0000 (oldest deadline 2024-09-21T03:44:22.069274+0000) 2024-09-21T03:44:29.711 INFO:journalctl@ceph.osd.11.smithi114.stdout:Sep 21 03:44:29 smithi114 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-11[58313]: 2024-09-21T03:44:29.290+0000 7f7657ce9640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2024-09-21T03:43:19.721069+0000 front 2024-09-21T03:43:19.721119+0000 (oldest deadline 2024-09-21T03:43:47.321877+0000) 2024-09-21T03:44:29.711 INFO:journalctl@ceph.osd.11.smithi114.stdout:Sep 21 03:44:29 smithi114 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-11[58313]: 2024-09-21T03:44:29.290+0000 7f7657ce9640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-21T03:43:27.322375+0000 front 2024-09-21T03:43:27.322426+0000 (oldest deadline 2024-09-21T03:43:51.422473+0000) 2024-09-21T03:44:29.711 INFO:journalctl@ceph.osd.11.smithi114.stdout:Sep 21 03:44:29 smithi114 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-11[58313]: 2024-09-21T03:44:29.290+0000 7f7657ce9640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.114:6804 osd.2 since back 2024-09-21T03:43:37.224230+0000 front 2024-09-21T03:43:37.224148+0000 (oldest deadline 2024-09-21T03:43:58.924202+0000) 2024-09-21T03:44:29.711 INFO:journalctl@ceph.osd.11.smithi114.stdout:Sep 21 03:44:29 smithi114 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-11[58313]: 2024-09-21T03:44:29.290+0000 7f7657ce9640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.110:6812 osd.4 since back 2024-09-21T03:43:50.626287+0000 front 2024-09-21T03:43:50.626333+0000 (oldest deadline 2024-09-21T03:44:15.326365+0000) 2024-09-21T03:44:29.711 INFO:journalctl@ceph.osd.11.smithi114.stdout:Sep 21 03:44:29 smithi114 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-11[58313]: 2024-09-21T03:44:29.290+0000 7f7657ce9640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.114:6812 osd.5 since back 2024-09-21T03:44:00.627730+0000 front 2024-09-21T03:44:00.627753+0000 (oldest deadline 2024-09-21T03:44:26.527718+0000) 2024-09-21T03:44:30.061 INFO:journalctl@ceph.osd.9.smithi089.stdout:Sep 21 03:44:29 smithi089 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-9[65404]: 2024-09-21T03:44:29.580+0000 7f6c4e7c5640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2024-09-21T03:43:18.845419+0000 front 2024-09-21T03:43:18.845455+0000 (oldest deadline 2024-09-21T03:43:44.745556+0000) 2024-09-21T03:44:30.061 INFO:journalctl@ceph.osd.9.smithi089.stdout:Sep 21 03:44:29 smithi089 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-9[65404]: 2024-09-21T03:44:29.580+0000 7f6c4e7c5640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-21T03:43:24.746023+0000 front 2024-09-21T03:43:24.745970+0000 (oldest deadline 2024-09-21T03:43:50.646080+0000) 2024-09-21T03:44:30.061 INFO:journalctl@ceph.osd.9.smithi089.stdout:Sep 21 03:44:29 smithi089 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-9[65404]: 2024-09-21T03:44:29.580+0000 7f6c4e7c5640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.114:6804 osd.2 since back 2024-09-21T03:43:34.147460+0000 front 2024-09-21T03:43:34.147364+0000 (oldest deadline 2024-09-21T03:43:58.247427+0000) 2024-09-21T03:44:30.061 INFO:journalctl@ceph.osd.9.smithi089.stdout:Sep 21 03:44:29 smithi089 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-9[65404]: 2024-09-21T03:44:29.580+0000 7f6c4e7c5640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.89:6814 osd.3 since back 2024-09-21T03:43:39.848901+0000 front 2024-09-21T03:43:39.848935+0000 (oldest deadline 2024-09-21T03:44:09.849669+0000) 2024-09-21T03:44:30.061 INFO:journalctl@ceph.osd.9.smithi089.stdout:Sep 21 03:44:29 smithi089 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-9[65404]: 2024-09-21T03:44:29.580+0000 7f6c4e7c5640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.110:6812 osd.4 since back 2024-09-21T03:43:49.850335+0000 front 2024-09-21T03:43:49.850307+0000 (oldest deadline 2024-09-21T03:44:13.950181+0000) 2024-09-21T03:44:30.062 INFO:journalctl@ceph.osd.9.smithi089.stdout:Sep 21 03:44:29 smithi089 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-9[65404]: 2024-09-21T03:44:29.580+0000 7f6c4e7c5640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.114:6812 osd.5 since back 2024-09-21T03:43:58.551756+0000 front 2024-09-21T03:43:58.551778+0000 (oldest deadline 2024-09-21T03:44:23.251588+0000) 2024-09-21T03:44:30.500 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 21 03:44:30 smithi110 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-10[57089]: 2024-09-21T03:44:30.162+0000 7f9d479dc640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2024-09-21T03:43:21.264189+0000 front 2024-09-21T03:43:21.264118+0000 (oldest deadline 2024-09-21T03:43:43.564295+0000) 2024-09-21T03:44:30.500 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 21 03:44:30 smithi110 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-10[57089]: 2024-09-21T03:44:30.162+0000 7f9d479dc640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-21T03:43:25.265351+0000 front 2024-09-21T03:43:25.265386+0000 (oldest deadline 2024-09-21T03:43:49.965504+0000) 2024-09-21T03:44:30.500 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 21 03:44:30 smithi110 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-10[57089]: 2024-09-21T03:44:30.162+0000 7f9d479dc640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.114:6804 osd.2 since back 2024-09-21T03:43:33.966726+0000 front 2024-09-21T03:43:33.966884+0000 (oldest deadline 2024-09-21T03:43:59.866510+0000) 2024-09-21T03:44:30.500 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 21 03:44:30 smithi110 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-10[57089]: 2024-09-21T03:44:30.162+0000 7f9d479dc640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.89:6814 osd.3 since back 2024-09-21T03:43:43.367532+0000 front 2024-09-21T03:43:43.367626+0000 (oldest deadline 2024-09-21T03:44:06.867392+0000) 2024-09-21T03:44:30.500 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 21 03:44:30 smithi110 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-10[57089]: 2024-09-21T03:44:30.162+0000 7f9d479dc640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.110:6812 osd.4 since back 2024-09-21T03:43:48.568210+0000 front 2024-09-21T03:43:48.568248+0000 (oldest deadline 2024-09-21T03:44:14.468315+0000) 2024-09-21T03:44:30.500 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 21 03:44:30 smithi110 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-10[57089]: 2024-09-21T03:44:30.162+0000 7f9d479dc640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.114:6812 osd.5 since back 2024-09-21T03:43:56.769315+0000 front 2024-09-21T03:43:56.769405+0000 (oldest deadline 2024-09-21T03:44:22.069274+0000) 2024-09-21T03:44:30.711 INFO:journalctl@ceph.osd.11.smithi114.stdout:Sep 21 03:44:30 smithi114 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-11[58313]: 2024-09-21T03:44:30.248+0000 7f7657ce9640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2024-09-21T03:43:19.721069+0000 front 2024-09-21T03:43:19.721119+0000 (oldest deadline 2024-09-21T03:43:47.321877+0000) 2024-09-21T03:44:30.711 INFO:journalctl@ceph.osd.11.smithi114.stdout:Sep 21 03:44:30 smithi114 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-11[58313]: 2024-09-21T03:44:30.248+0000 7f7657ce9640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-21T03:43:27.322375+0000 front 2024-09-21T03:43:27.322426+0000 (oldest deadline 2024-09-21T03:43:51.422473+0000) 2024-09-21T03:44:30.711 INFO:journalctl@ceph.osd.11.smithi114.stdout:Sep 21 03:44:30 smithi114 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-11[58313]: 2024-09-21T03:44:30.248+0000 7f7657ce9640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.114:6804 osd.2 since back 2024-09-21T03:43:37.224230+0000 front 2024-09-21T03:43:37.224148+0000 (oldest deadline 2024-09-21T03:43:58.924202+0000) 2024-09-21T03:44:30.711 INFO:journalctl@ceph.osd.11.smithi114.stdout:Sep 21 03:44:30 smithi114 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-11[58313]: 2024-09-21T03:44:30.248+0000 7f7657ce9640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.110:6812 osd.4 since back 2024-09-21T03:43:50.626287+0000 front 2024-09-21T03:43:50.626333+0000 (oldest deadline 2024-09-21T03:44:15.326365+0000) 2024-09-21T03:44:30.711 INFO:journalctl@ceph.osd.11.smithi114.stdout:Sep 21 03:44:30 smithi114 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-11[58313]: 2024-09-21T03:44:30.248+0000 7f7657ce9640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.114:6812 osd.5 since back 2024-09-21T03:44:00.627730+0000 front 2024-09-21T03:44:00.627753+0000 (oldest deadline 2024-09-21T03:44:26.527718+0000) 2024-09-21T03:44:30.711 INFO:journalctl@ceph.osd.11.smithi114.stdout:Sep 21 03:44:30 smithi114 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-11[58313]: 2024-09-21T03:44:30.248+0000 7f7657ce9640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.89:6822 osd.6 since back 2024-09-21T03:44:07.029032+0000 front 2024-09-21T03:44:07.029045+0000 (oldest deadline 2024-09-21T03:44:29.928840+0000) 2024-09-21T03:44:31.061 INFO:journalctl@ceph.osd.9.smithi089.stdout:Sep 21 03:44:30 smithi089 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-9[65404]: 2024-09-21T03:44:30.591+0000 7f6c4e7c5640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2024-09-21T03:43:18.845419+0000 front 2024-09-21T03:43:18.845455+0000 (oldest deadline 2024-09-21T03:43:44.745556+0000) 2024-09-21T03:44:31.062 INFO:journalctl@ceph.osd.9.smithi089.stdout:Sep 21 03:44:30 smithi089 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-9[65404]: 2024-09-21T03:44:30.591+0000 7f6c4e7c5640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-21T03:43:24.746023+0000 front 2024-09-21T03:43:24.745970+0000 (oldest deadline 2024-09-21T03:43:50.646080+0000) 2024-09-21T03:44:31.062 INFO:journalctl@ceph.osd.9.smithi089.stdout:Sep 21 03:44:30 smithi089 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-9[65404]: 2024-09-21T03:44:30.591+0000 7f6c4e7c5640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.114:6804 osd.2 since back 2024-09-21T03:43:34.147460+0000 front 2024-09-21T03:43:34.147364+0000 (oldest deadline 2024-09-21T03:43:58.247427+0000) 2024-09-21T03:44:31.062 INFO:journalctl@ceph.osd.9.smithi089.stdout:Sep 21 03:44:30 smithi089 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-9[65404]: 2024-09-21T03:44:30.591+0000 7f6c4e7c5640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.89:6814 osd.3 since back 2024-09-21T03:43:39.848901+0000 front 2024-09-21T03:43:39.848935+0000 (oldest deadline 2024-09-21T03:44:09.849669+0000) 2024-09-21T03:44:31.062 INFO:journalctl@ceph.osd.9.smithi089.stdout:Sep 21 03:44:30 smithi089 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-9[65404]: 2024-09-21T03:44:30.591+0000 7f6c4e7c5640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.110:6812 osd.4 since back 2024-09-21T03:43:49.850335+0000 front 2024-09-21T03:43:49.850307+0000 (oldest deadline 2024-09-21T03:44:13.950181+0000) 2024-09-21T03:44:31.062 INFO:journalctl@ceph.osd.9.smithi089.stdout:Sep 21 03:44:30 smithi089 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-9[65404]: 2024-09-21T03:44:30.591+0000 7f6c4e7c5640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.114:6812 osd.5 since back 2024-09-21T03:43:58.551756+0000 front 2024-09-21T03:43:58.551778+0000 (oldest deadline 2024-09-21T03:44:23.251588+0000) 2024-09-21T03:44:31.500 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 21 03:44:31 smithi110 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-10[57089]: 2024-09-21T03:44:31.175+0000 7f9d479dc640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2024-09-21T03:43:21.264189+0000 front 2024-09-21T03:43:21.264118+0000 (oldest deadline 2024-09-21T03:43:43.564295+0000) 2024-09-21T03:44:31.500 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 21 03:44:31 smithi110 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-10[57089]: 2024-09-21T03:44:31.175+0000 7f9d479dc640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-21T03:43:25.265351+0000 front 2024-09-21T03:43:25.265386+0000 (oldest deadline 2024-09-21T03:43:49.965504+0000) 2024-09-21T03:44:31.500 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 21 03:44:31 smithi110 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-10[57089]: 2024-09-21T03:44:31.175+0000 7f9d479dc640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.114:6804 osd.2 since back 2024-09-21T03:43:33.966726+0000 front 2024-09-21T03:43:33.966884+0000 (oldest deadline 2024-09-21T03:43:59.866510+0000) 2024-09-21T03:44:31.501 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 21 03:44:31 smithi110 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-10[57089]: 2024-09-21T03:44:31.175+0000 7f9d479dc640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.89:6814 osd.3 since back 2024-09-21T03:43:43.367532+0000 front 2024-09-21T03:43:43.367626+0000 (oldest deadline 2024-09-21T03:44:06.867392+0000) 2024-09-21T03:44:31.501 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 21 03:44:31 smithi110 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-10[57089]: 2024-09-21T03:44:31.175+0000 7f9d479dc640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.110:6812 osd.4 since back 2024-09-21T03:43:48.568210+0000 front 2024-09-21T03:43:48.568248+0000 (oldest deadline 2024-09-21T03:44:14.468315+0000) 2024-09-21T03:44:31.501 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 21 03:44:31 smithi110 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-10[57089]: 2024-09-21T03:44:31.175+0000 7f9d479dc640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.114:6812 osd.5 since back 2024-09-21T03:43:56.769315+0000 front 2024-09-21T03:43:56.769405+0000 (oldest deadline 2024-09-21T03:44:22.069274+0000) 2024-09-21T03:44:31.501 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 21 03:44:31 smithi110 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-10[57089]: 2024-09-21T03:44:31.175+0000 7f9d479dc640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.89:6822 osd.6 since back 2024-09-21T03:44:06.770305+0000 front 2024-09-21T03:44:06.770257+0000 (oldest deadline 2024-09-21T03:44:30.270157+0000) 2024-09-21T03:44:31.711 INFO:journalctl@ceph.osd.11.smithi114.stdout:Sep 21 03:44:31 smithi114 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-11[58313]: 2024-09-21T03:44:31.214+0000 7f7657ce9640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2024-09-21T03:43:19.721069+0000 front 2024-09-21T03:43:19.721119+0000 (oldest deadline 2024-09-21T03:43:47.321877+0000) 2024-09-21T03:44:31.711 INFO:journalctl@ceph.osd.11.smithi114.stdout:Sep 21 03:44:31 smithi114 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-11[58313]: 2024-09-21T03:44:31.214+0000 7f7657ce9640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-21T03:43:27.322375+0000 front 2024-09-21T03:43:27.322426+0000 (oldest deadline 2024-09-21T03:43:51.422473+0000) 2024-09-21T03:44:31.711 INFO:journalctl@ceph.osd.11.smithi114.stdout:Sep 21 03:44:31 smithi114 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-11[58313]: 2024-09-21T03:44:31.214+0000 7f7657ce9640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.114:6804 osd.2 since back 2024-09-21T03:43:37.224230+0000 front 2024-09-21T03:43:37.224148+0000 (oldest deadline 2024-09-21T03:43:58.924202+0000) 2024-09-21T03:44:31.711 INFO:journalctl@ceph.osd.11.smithi114.stdout:Sep 21 03:44:31 smithi114 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-11[58313]: 2024-09-21T03:44:31.214+0000 7f7657ce9640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.110:6812 osd.4 since back 2024-09-21T03:43:50.626287+0000 front 2024-09-21T03:43:50.626333+0000 (oldest deadline 2024-09-21T03:44:15.326365+0000) 2024-09-21T03:44:31.711 INFO:journalctl@ceph.osd.11.smithi114.stdout:Sep 21 03:44:31 smithi114 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-11[58313]: 2024-09-21T03:44:31.214+0000 7f7657ce9640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.114:6812 osd.5 since back 2024-09-21T03:44:00.627730+0000 front 2024-09-21T03:44:00.627753+0000 (oldest deadline 2024-09-21T03:44:26.527718+0000) 2024-09-21T03:44:31.711 INFO:journalctl@ceph.osd.11.smithi114.stdout:Sep 21 03:44:31 smithi114 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-11[58313]: 2024-09-21T03:44:31.214+0000 7f7657ce9640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.89:6822 osd.6 since back 2024-09-21T03:44:07.029032+0000 front 2024-09-21T03:44:07.029045+0000 (oldest deadline 2024-09-21T03:44:29.928840+0000) 2024-09-21T03:44:32.061 INFO:journalctl@ceph.osd.9.smithi089.stdout:Sep 21 03:44:31 smithi089 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-9[65404]: 2024-09-21T03:44:31.559+0000 7f6c4e7c5640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2024-09-21T03:43:18.845419+0000 front 2024-09-21T03:43:18.845455+0000 (oldest deadline 2024-09-21T03:43:44.745556+0000) 2024-09-21T03:44:32.061 INFO:journalctl@ceph.osd.9.smithi089.stdout:Sep 21 03:44:31 smithi089 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-9[65404]: 2024-09-21T03:44:31.559+0000 7f6c4e7c5640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-21T03:43:24.746023+0000 front 2024-09-21T03:43:24.745970+0000 (oldest deadline 2024-09-21T03:43:50.646080+0000) 2024-09-21T03:44:32.061 INFO:journalctl@ceph.osd.9.smithi089.stdout:Sep 21 03:44:31 smithi089 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-9[65404]: 2024-09-21T03:44:31.559+0000 7f6c4e7c5640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.114:6804 osd.2 since back 2024-09-21T03:43:34.147460+0000 front 2024-09-21T03:43:34.147364+0000 (oldest deadline 2024-09-21T03:43:58.247427+0000) 2024-09-21T03:44:32.061 INFO:journalctl@ceph.osd.9.smithi089.stdout:Sep 21 03:44:31 smithi089 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-9[65404]: 2024-09-21T03:44:31.559+0000 7f6c4e7c5640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.89:6814 osd.3 since back 2024-09-21T03:43:39.848901+0000 front 2024-09-21T03:43:39.848935+0000 (oldest deadline 2024-09-21T03:44:09.849669+0000) 2024-09-21T03:44:32.062 INFO:journalctl@ceph.osd.9.smithi089.stdout:Sep 21 03:44:31 smithi089 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-9[65404]: 2024-09-21T03:44:31.559+0000 7f6c4e7c5640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.110:6812 osd.4 since back 2024-09-21T03:43:49.850335+0000 front 2024-09-21T03:43:49.850307+0000 (oldest deadline 2024-09-21T03:44:13.950181+0000) 2024-09-21T03:44:32.062 INFO:journalctl@ceph.osd.9.smithi089.stdout:Sep 21 03:44:31 smithi089 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-9[65404]: 2024-09-21T03:44:31.559+0000 7f6c4e7c5640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.114:6812 osd.5 since back 2024-09-21T03:43:58.551756+0000 front 2024-09-21T03:43:58.551778+0000 (oldest deadline 2024-09-21T03:44:23.251588+0000) 2024-09-21T03:44:32.500 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 21 03:44:32 smithi110 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-10[57089]: 2024-09-21T03:44:32.147+0000 7f9d479dc640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2024-09-21T03:43:21.264189+0000 front 2024-09-21T03:43:21.264118+0000 (oldest deadline 2024-09-21T03:43:43.564295+0000) 2024-09-21T03:44:32.500 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 21 03:44:32 smithi110 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-10[57089]: 2024-09-21T03:44:32.147+0000 7f9d479dc640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-21T03:43:25.265351+0000 front 2024-09-21T03:43:25.265386+0000 (oldest deadline 2024-09-21T03:43:49.965504+0000) 2024-09-21T03:44:32.500 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 21 03:44:32 smithi110 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-10[57089]: 2024-09-21T03:44:32.147+0000 7f9d479dc640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.114:6804 osd.2 since back 2024-09-21T03:43:33.966726+0000 front 2024-09-21T03:43:33.966884+0000 (oldest deadline 2024-09-21T03:43:59.866510+0000) 2024-09-21T03:44:32.500 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 21 03:44:32 smithi110 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-10[57089]: 2024-09-21T03:44:32.147+0000 7f9d479dc640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.89:6814 osd.3 since back 2024-09-21T03:43:43.367532+0000 front 2024-09-21T03:43:43.367626+0000 (oldest deadline 2024-09-21T03:44:06.867392+0000) 2024-09-21T03:44:32.500 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 21 03:44:32 smithi110 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-10[57089]: 2024-09-21T03:44:32.147+0000 7f9d479dc640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.110:6812 osd.4 since back 2024-09-21T03:43:48.568210+0000 front 2024-09-21T03:43:48.568248+0000 (oldest deadline 2024-09-21T03:44:14.468315+0000) 2024-09-21T03:44:32.500 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 21 03:44:32 smithi110 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-10[57089]: 2024-09-21T03:44:32.147+0000 7f9d479dc640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.114:6812 osd.5 since back 2024-09-21T03:43:56.769315+0000 front 2024-09-21T03:43:56.769405+0000 (oldest deadline 2024-09-21T03:44:22.069274+0000) 2024-09-21T03:44:32.501 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 21 03:44:32 smithi110 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-10[57089]: 2024-09-21T03:44:32.147+0000 7f9d479dc640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.89:6822 osd.6 since back 2024-09-21T03:44:06.770305+0000 front 2024-09-21T03:44:06.770257+0000 (oldest deadline 2024-09-21T03:44:30.270157+0000) 2024-09-21T03:44:32.711 INFO:journalctl@ceph.osd.11.smithi114.stdout:Sep 21 03:44:32 smithi114 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-11[58313]: 2024-09-21T03:44:32.246+0000 7f7657ce9640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2024-09-21T03:43:19.721069+0000 front 2024-09-21T03:43:19.721119+0000 (oldest deadline 2024-09-21T03:43:47.321877+0000) 2024-09-21T03:44:32.711 INFO:journalctl@ceph.osd.11.smithi114.stdout:Sep 21 03:44:32 smithi114 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-11[58313]: 2024-09-21T03:44:32.246+0000 7f7657ce9640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-21T03:43:27.322375+0000 front 2024-09-21T03:43:27.322426+0000 (oldest deadline 2024-09-21T03:43:51.422473+0000) 2024-09-21T03:44:32.711 INFO:journalctl@ceph.osd.11.smithi114.stdout:Sep 21 03:44:32 smithi114 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-11[58313]: 2024-09-21T03:44:32.246+0000 7f7657ce9640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.114:6804 osd.2 since back 2024-09-21T03:43:37.224230+0000 front 2024-09-21T03:43:37.224148+0000 (oldest deadline 2024-09-21T03:43:58.924202+0000) 2024-09-21T03:44:32.711 INFO:journalctl@ceph.osd.11.smithi114.stdout:Sep 21 03:44:32 smithi114 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-11[58313]: 2024-09-21T03:44:32.246+0000 7f7657ce9640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.110:6812 osd.4 since back 2024-09-21T03:43:50.626287+0000 front 2024-09-21T03:43:50.626333+0000 (oldest deadline 2024-09-21T03:44:15.326365+0000) 2024-09-21T03:44:32.711 INFO:journalctl@ceph.osd.11.smithi114.stdout:Sep 21 03:44:32 smithi114 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-11[58313]: 2024-09-21T03:44:32.246+0000 7f7657ce9640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.114:6812 osd.5 since back 2024-09-21T03:44:00.627730+0000 front 2024-09-21T03:44:00.627753+0000 (oldest deadline 2024-09-21T03:44:26.527718+0000) 2024-09-21T03:44:32.711 INFO:journalctl@ceph.osd.11.smithi114.stdout:Sep 21 03:44:32 smithi114 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-11[58313]: 2024-09-21T03:44:32.246+0000 7f7657ce9640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.89:6822 osd.6 since back 2024-09-21T03:44:07.029032+0000 front 2024-09-21T03:44:07.029045+0000 (oldest deadline 2024-09-21T03:44:29.928840+0000) 2024-09-21T03:44:33.061 INFO:journalctl@ceph.osd.9.smithi089.stdout:Sep 21 03:44:32 smithi089 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-9[65404]: 2024-09-21T03:44:32.605+0000 7f6c4e7c5640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2024-09-21T03:43:18.845419+0000 front 2024-09-21T03:43:18.845455+0000 (oldest deadline 2024-09-21T03:43:44.745556+0000) 2024-09-21T03:44:33.062 INFO:journalctl@ceph.osd.9.smithi089.stdout:Sep 21 03:44:32 smithi089 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-9[65404]: 2024-09-21T03:44:32.605+0000 7f6c4e7c5640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-21T03:43:24.746023+0000 front 2024-09-21T03:43:24.745970+0000 (oldest deadline 2024-09-21T03:43:50.646080+0000) 2024-09-21T03:44:33.062 INFO:journalctl@ceph.osd.9.smithi089.stdout:Sep 21 03:44:32 smithi089 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-9[65404]: 2024-09-21T03:44:32.605+0000 7f6c4e7c5640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.114:6804 osd.2 since back 2024-09-21T03:43:34.147460+0000 front 2024-09-21T03:43:34.147364+0000 (oldest deadline 2024-09-21T03:43:58.247427+0000) 2024-09-21T03:44:33.062 INFO:journalctl@ceph.osd.9.smithi089.stdout:Sep 21 03:44:32 smithi089 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-9[65404]: 2024-09-21T03:44:32.605+0000 7f6c4e7c5640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.89:6814 osd.3 since back 2024-09-21T03:43:39.848901+0000 front 2024-09-21T03:43:39.848935+0000 (oldest deadline 2024-09-21T03:44:09.849669+0000) 2024-09-21T03:44:33.062 INFO:journalctl@ceph.osd.9.smithi089.stdout:Sep 21 03:44:32 smithi089 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-9[65404]: 2024-09-21T03:44:32.605+0000 7f6c4e7c5640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.110:6812 osd.4 since back 2024-09-21T03:43:49.850335+0000 front 2024-09-21T03:43:49.850307+0000 (oldest deadline 2024-09-21T03:44:13.950181+0000) 2024-09-21T03:44:33.062 INFO:journalctl@ceph.osd.9.smithi089.stdout:Sep 21 03:44:32 smithi089 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-9[65404]: 2024-09-21T03:44:32.605+0000 7f6c4e7c5640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.114:6812 osd.5 since back 2024-09-21T03:43:58.551756+0000 front 2024-09-21T03:43:58.551778+0000 (oldest deadline 2024-09-21T03:44:23.251588+0000) 2024-09-21T03:44:33.500 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 21 03:44:33 smithi110 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-10[57089]: 2024-09-21T03:44:33.146+0000 7f9d479dc640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2024-09-21T03:43:21.264189+0000 front 2024-09-21T03:43:21.264118+0000 (oldest deadline 2024-09-21T03:43:43.564295+0000) 2024-09-21T03:44:33.500 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 21 03:44:33 smithi110 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-10[57089]: 2024-09-21T03:44:33.146+0000 7f9d479dc640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-21T03:43:25.265351+0000 front 2024-09-21T03:43:25.265386+0000 (oldest deadline 2024-09-21T03:43:49.965504+0000) 2024-09-21T03:44:33.500 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 21 03:44:33 smithi110 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-10[57089]: 2024-09-21T03:44:33.146+0000 7f9d479dc640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.114:6804 osd.2 since back 2024-09-21T03:43:33.966726+0000 front 2024-09-21T03:43:33.966884+0000 (oldest deadline 2024-09-21T03:43:59.866510+0000) 2024-09-21T03:44:33.500 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 21 03:44:33 smithi110 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-10[57089]: 2024-09-21T03:44:33.146+0000 7f9d479dc640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.89:6814 osd.3 since back 2024-09-21T03:43:43.367532+0000 front 2024-09-21T03:43:43.367626+0000 (oldest deadline 2024-09-21T03:44:06.867392+0000) 2024-09-21T03:44:33.500 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 21 03:44:33 smithi110 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-10[57089]: 2024-09-21T03:44:33.146+0000 7f9d479dc640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.110:6812 osd.4 since back 2024-09-21T03:43:48.568210+0000 front 2024-09-21T03:43:48.568248+0000 (oldest deadline 2024-09-21T03:44:14.468315+0000) 2024-09-21T03:44:33.500 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 21 03:44:33 smithi110 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-10[57089]: 2024-09-21T03:44:33.146+0000 7f9d479dc640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.114:6812 osd.5 since back 2024-09-21T03:43:56.769315+0000 front 2024-09-21T03:43:56.769405+0000 (oldest deadline 2024-09-21T03:44:22.069274+0000) 2024-09-21T03:44:33.500 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 21 03:44:33 smithi110 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-10[57089]: 2024-09-21T03:44:33.146+0000 7f9d479dc640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.89:6822 osd.6 since back 2024-09-21T03:44:06.770305+0000 front 2024-09-21T03:44:06.770257+0000 (oldest deadline 2024-09-21T03:44:30.270157+0000) 2024-09-21T03:44:33.711 INFO:journalctl@ceph.osd.11.smithi114.stdout:Sep 21 03:44:33 smithi114 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-11[58313]: 2024-09-21T03:44:33.216+0000 7f7657ce9640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2024-09-21T03:43:19.721069+0000 front 2024-09-21T03:43:19.721119+0000 (oldest deadline 2024-09-21T03:43:47.321877+0000) 2024-09-21T03:44:33.711 INFO:journalctl@ceph.osd.11.smithi114.stdout:Sep 21 03:44:33 smithi114 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-11[58313]: 2024-09-21T03:44:33.216+0000 7f7657ce9640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-21T03:43:27.322375+0000 front 2024-09-21T03:43:27.322426+0000 (oldest deadline 2024-09-21T03:43:51.422473+0000) 2024-09-21T03:44:33.711 INFO:journalctl@ceph.osd.11.smithi114.stdout:Sep 21 03:44:33 smithi114 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-11[58313]: 2024-09-21T03:44:33.216+0000 7f7657ce9640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.114:6804 osd.2 since back 2024-09-21T03:43:37.224230+0000 front 2024-09-21T03:43:37.224148+0000 (oldest deadline 2024-09-21T03:43:58.924202+0000) 2024-09-21T03:44:33.711 INFO:journalctl@ceph.osd.11.smithi114.stdout:Sep 21 03:44:33 smithi114 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-11[58313]: 2024-09-21T03:44:33.216+0000 7f7657ce9640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.110:6812 osd.4 since back 2024-09-21T03:43:50.626287+0000 front 2024-09-21T03:43:50.626333+0000 (oldest deadline 2024-09-21T03:44:15.326365+0000) 2024-09-21T03:44:33.711 INFO:journalctl@ceph.osd.11.smithi114.stdout:Sep 21 03:44:33 smithi114 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-11[58313]: 2024-09-21T03:44:33.216+0000 7f7657ce9640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.114:6812 osd.5 since back 2024-09-21T03:44:00.627730+0000 front 2024-09-21T03:44:00.627753+0000 (oldest deadline 2024-09-21T03:44:26.527718+0000) 2024-09-21T03:44:33.711 INFO:journalctl@ceph.osd.11.smithi114.stdout:Sep 21 03:44:33 smithi114 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-11[58313]: 2024-09-21T03:44:33.216+0000 7f7657ce9640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.89:6822 osd.6 since back 2024-09-21T03:44:07.029032+0000 front 2024-09-21T03:44:07.029045+0000 (oldest deadline 2024-09-21T03:44:29.928840+0000) 2024-09-21T03:44:33.832 INFO:journalctl@ceph.osd.9.smithi089.stdout:Sep 21 03:44:33 smithi089 podman[92277]: 2024-09-21 03:44:33.565361289 +0000 UTC m=+5.561118680 container died 31b294330e445a3ab61234c20d7ca28afa8a5161e0b435070514c17bd338b304 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-9, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-21T03:44:34.183 INFO:journalctl@ceph.osd.9.smithi089.stdout:Sep 21 03:44:34 smithi089 podman[92277]: 2024-09-21 03:44:34.005992681 +0000 UTC m=+6.001750064 container cleanup 31b294330e445a3ab61234c20d7ca28afa8a5161e0b435070514c17bd338b304 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-9, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, org.label-schema.build-date=20240731, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=) 2024-09-21T03:44:34.184 INFO:journalctl@ceph.osd.9.smithi089.stdout:Sep 21 03:44:34 smithi089 bash[92277]: ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-9 2024-09-21T03:44:34.461 INFO:journalctl@ceph.osd.11.smithi114.stdout:Sep 21 03:44:34 smithi114 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-11[58313]: 2024-09-21T03:44:34.202+0000 7f7657ce9640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2024-09-21T03:43:19.721069+0000 front 2024-09-21T03:43:19.721119+0000 (oldest deadline 2024-09-21T03:43:47.321877+0000) 2024-09-21T03:44:34.461 INFO:journalctl@ceph.osd.11.smithi114.stdout:Sep 21 03:44:34 smithi114 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-11[58313]: 2024-09-21T03:44:34.202+0000 7f7657ce9640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-21T03:43:27.322375+0000 front 2024-09-21T03:43:27.322426+0000 (oldest deadline 2024-09-21T03:43:51.422473+0000) 2024-09-21T03:44:34.461 INFO:journalctl@ceph.osd.11.smithi114.stdout:Sep 21 03:44:34 smithi114 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-11[58313]: 2024-09-21T03:44:34.202+0000 7f7657ce9640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.114:6804 osd.2 since back 2024-09-21T03:43:37.224230+0000 front 2024-09-21T03:43:37.224148+0000 (oldest deadline 2024-09-21T03:43:58.924202+0000) 2024-09-21T03:44:34.461 INFO:journalctl@ceph.osd.11.smithi114.stdout:Sep 21 03:44:34 smithi114 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-11[58313]: 2024-09-21T03:44:34.202+0000 7f7657ce9640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.110:6812 osd.4 since back 2024-09-21T03:43:50.626287+0000 front 2024-09-21T03:43:50.626333+0000 (oldest deadline 2024-09-21T03:44:15.326365+0000) 2024-09-21T03:44:34.461 INFO:journalctl@ceph.osd.11.smithi114.stdout:Sep 21 03:44:34 smithi114 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-11[58313]: 2024-09-21T03:44:34.202+0000 7f7657ce9640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.114:6812 osd.5 since back 2024-09-21T03:44:00.627730+0000 front 2024-09-21T03:44:00.627753+0000 (oldest deadline 2024-09-21T03:44:26.527718+0000) 2024-09-21T03:44:34.461 INFO:journalctl@ceph.osd.11.smithi114.stdout:Sep 21 03:44:34 smithi114 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-11[58313]: 2024-09-21T03:44:34.202+0000 7f7657ce9640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.89:6822 osd.6 since back 2024-09-21T03:44:07.029032+0000 front 2024-09-21T03:44:07.029045+0000 (oldest deadline 2024-09-21T03:44:29.928840+0000) 2024-09-21T03:44:34.500 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 21 03:44:34 smithi110 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-10[57089]: 2024-09-21T03:44:34.194+0000 7f9d479dc640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2024-09-21T03:43:21.264189+0000 front 2024-09-21T03:43:21.264118+0000 (oldest deadline 2024-09-21T03:43:43.564295+0000) 2024-09-21T03:44:34.500 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 21 03:44:34 smithi110 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-10[57089]: 2024-09-21T03:44:34.194+0000 7f9d479dc640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-21T03:43:25.265351+0000 front 2024-09-21T03:43:25.265386+0000 (oldest deadline 2024-09-21T03:43:49.965504+0000) 2024-09-21T03:44:34.500 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 21 03:44:34 smithi110 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-10[57089]: 2024-09-21T03:44:34.194+0000 7f9d479dc640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.114:6804 osd.2 since back 2024-09-21T03:43:33.966726+0000 front 2024-09-21T03:43:33.966884+0000 (oldest deadline 2024-09-21T03:43:59.866510+0000) 2024-09-21T03:44:34.500 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 21 03:44:34 smithi110 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-10[57089]: 2024-09-21T03:44:34.194+0000 7f9d479dc640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.89:6814 osd.3 since back 2024-09-21T03:43:43.367532+0000 front 2024-09-21T03:43:43.367626+0000 (oldest deadline 2024-09-21T03:44:06.867392+0000) 2024-09-21T03:44:34.500 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 21 03:44:34 smithi110 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-10[57089]: 2024-09-21T03:44:34.194+0000 7f9d479dc640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.110:6812 osd.4 since back 2024-09-21T03:43:48.568210+0000 front 2024-09-21T03:43:48.568248+0000 (oldest deadline 2024-09-21T03:44:14.468315+0000) 2024-09-21T03:44:34.501 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 21 03:44:34 smithi110 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-10[57089]: 2024-09-21T03:44:34.194+0000 7f9d479dc640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.114:6812 osd.5 since back 2024-09-21T03:43:56.769315+0000 front 2024-09-21T03:43:56.769405+0000 (oldest deadline 2024-09-21T03:44:22.069274+0000) 2024-09-21T03:44:34.501 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 21 03:44:34 smithi110 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-10[57089]: 2024-09-21T03:44:34.194+0000 7f9d479dc640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.89:6822 osd.6 since back 2024-09-21T03:44:06.770305+0000 front 2024-09-21T03:44:06.770257+0000 (oldest deadline 2024-09-21T03:44:30.270157+0000) 2024-09-21T03:44:34.561 INFO:journalctl@ceph.osd.9.smithi089.stdout:Sep 21 03:44:34 smithi089 podman[92292]: 2024-09-21 03:44:34.182378794 +0000 UTC m=+0.616800233 container remove 31b294330e445a3ab61234c20d7ca28afa8a5161e0b435070514c17bd338b304 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-9, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731) 2024-09-21T03:44:34.946 INFO:journalctl@ceph.osd.9.smithi089.stdout:Sep 21 03:44:34 smithi089 podman[92378]: 2024-09-21 03:44:34.523098807 +0000 UTC m=+0.016010484 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T03:44:34.946 INFO:journalctl@ceph.osd.9.smithi089.stdout:Sep 21 03:44:34 smithi089 podman[92378]: 2024-09-21 03:44:34.650359848 +0000 UTC m=+0.143271559 container create 71fcbf442c87a56e100a9e076834086ee3df271212535392f5b873f32b1bbeae (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-9-deactivate, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS) 2024-09-21T03:44:34.946 INFO:journalctl@ceph.osd.9.smithi089.stdout:Sep 21 03:44:34 smithi089 podman[92378]: 2024-09-21 03:44:34.852669068 +0000 UTC m=+0.345580740 container init 71fcbf442c87a56e100a9e076834086ee3df271212535392f5b873f32b1bbeae (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-9-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221) 2024-09-21T03:44:34.946 INFO:journalctl@ceph.osd.9.smithi089.stdout:Sep 21 03:44:34 smithi089 podman[92378]: 2024-09-21 03:44:34.855845898 +0000 UTC m=+0.348757563 container start 71fcbf442c87a56e100a9e076834086ee3df271212535392f5b873f32b1bbeae (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-9-deactivate, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, ceph=True) 2024-09-21T03:44:34.946 INFO:journalctl@ceph.osd.9.smithi089.stdout:Sep 21 03:44:34 smithi089 podman[92378]: 2024-09-21 03:44:34.944709187 +0000 UTC m=+0.437620856 container attach 71fcbf442c87a56e100a9e076834086ee3df271212535392f5b873f32b1bbeae (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-9-deactivate, ceph=True, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2) 2024-09-21T03:44:35.311 INFO:journalctl@ceph.osd.9.smithi089.stdout:Sep 21 03:44:35 smithi089 podman[92378]: 2024-09-21 03:44:35.045157071 +0000 UTC m=+0.538068737 container died 71fcbf442c87a56e100a9e076834086ee3df271212535392f5b873f32b1bbeae (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-9-deactivate, org.label-schema.vendor=CentOS, ceph=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-21T03:44:35.461 INFO:journalctl@ceph.osd.11.smithi114.stdout:Sep 21 03:44:35 smithi114 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-11[58313]: 2024-09-21T03:44:35.155+0000 7f7657ce9640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2024-09-21T03:43:19.721069+0000 front 2024-09-21T03:43:19.721119+0000 (oldest deadline 2024-09-21T03:43:47.321877+0000) 2024-09-21T03:44:35.461 INFO:journalctl@ceph.osd.11.smithi114.stdout:Sep 21 03:44:35 smithi114 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-11[58313]: 2024-09-21T03:44:35.155+0000 7f7657ce9640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-21T03:43:27.322375+0000 front 2024-09-21T03:43:27.322426+0000 (oldest deadline 2024-09-21T03:43:51.422473+0000) 2024-09-21T03:44:35.461 INFO:journalctl@ceph.osd.11.smithi114.stdout:Sep 21 03:44:35 smithi114 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-11[58313]: 2024-09-21T03:44:35.155+0000 7f7657ce9640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.114:6804 osd.2 since back 2024-09-21T03:43:37.224230+0000 front 2024-09-21T03:43:37.224148+0000 (oldest deadline 2024-09-21T03:43:58.924202+0000) 2024-09-21T03:44:35.461 INFO:journalctl@ceph.osd.11.smithi114.stdout:Sep 21 03:44:35 smithi114 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-11[58313]: 2024-09-21T03:44:35.155+0000 7f7657ce9640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.110:6812 osd.4 since back 2024-09-21T03:43:50.626287+0000 front 2024-09-21T03:43:50.626333+0000 (oldest deadline 2024-09-21T03:44:15.326365+0000) 2024-09-21T03:44:35.461 INFO:journalctl@ceph.osd.11.smithi114.stdout:Sep 21 03:44:35 smithi114 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-11[58313]: 2024-09-21T03:44:35.155+0000 7f7657ce9640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.114:6812 osd.5 since back 2024-09-21T03:44:00.627730+0000 front 2024-09-21T03:44:00.627753+0000 (oldest deadline 2024-09-21T03:44:26.527718+0000) 2024-09-21T03:44:35.461 INFO:journalctl@ceph.osd.11.smithi114.stdout:Sep 21 03:44:35 smithi114 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-11[58313]: 2024-09-21T03:44:35.155+0000 7f7657ce9640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.89:6822 osd.6 since back 2024-09-21T03:44:07.029032+0000 front 2024-09-21T03:44:07.029045+0000 (oldest deadline 2024-09-21T03:44:29.928840+0000) 2024-09-21T03:44:35.500 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 21 03:44:35 smithi110 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-10[57089]: 2024-09-21T03:44:35.178+0000 7f9d479dc640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2024-09-21T03:43:21.264189+0000 front 2024-09-21T03:43:21.264118+0000 (oldest deadline 2024-09-21T03:43:43.564295+0000) 2024-09-21T03:44:35.500 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 21 03:44:35 smithi110 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-10[57089]: 2024-09-21T03:44:35.178+0000 7f9d479dc640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-21T03:43:25.265351+0000 front 2024-09-21T03:43:25.265386+0000 (oldest deadline 2024-09-21T03:43:49.965504+0000) 2024-09-21T03:44:35.501 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 21 03:44:35 smithi110 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-10[57089]: 2024-09-21T03:44:35.178+0000 7f9d479dc640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.114:6804 osd.2 since back 2024-09-21T03:43:33.966726+0000 front 2024-09-21T03:43:33.966884+0000 (oldest deadline 2024-09-21T03:43:59.866510+0000) 2024-09-21T03:44:35.501 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 21 03:44:35 smithi110 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-10[57089]: 2024-09-21T03:44:35.178+0000 7f9d479dc640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.89:6814 osd.3 since back 2024-09-21T03:43:43.367532+0000 front 2024-09-21T03:43:43.367626+0000 (oldest deadline 2024-09-21T03:44:06.867392+0000) 2024-09-21T03:44:35.501 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 21 03:44:35 smithi110 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-10[57089]: 2024-09-21T03:44:35.178+0000 7f9d479dc640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.110:6812 osd.4 since back 2024-09-21T03:43:48.568210+0000 front 2024-09-21T03:43:48.568248+0000 (oldest deadline 2024-09-21T03:44:14.468315+0000) 2024-09-21T03:44:35.501 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 21 03:44:35 smithi110 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-10[57089]: 2024-09-21T03:44:35.178+0000 7f9d479dc640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.114:6812 osd.5 since back 2024-09-21T03:43:56.769315+0000 front 2024-09-21T03:43:56.769405+0000 (oldest deadline 2024-09-21T03:44:22.069274+0000) 2024-09-21T03:44:35.501 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 21 03:44:35 smithi110 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-10[57089]: 2024-09-21T03:44:35.178+0000 7f9d479dc640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.89:6822 osd.6 since back 2024-09-21T03:44:06.770305+0000 front 2024-09-21T03:44:06.770257+0000 (oldest deadline 2024-09-21T03:44:30.270157+0000) 2024-09-21T03:44:35.695 DEBUG:teuthology.orchestra.run.smithi089:> sudo pkill -f 'journalctl -f -n 0 -u ceph-7c374098-77c9-11ef-baf5-efdc52797490@osd.9.service' 2024-09-21T03:44:35.732 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T03:44:35.733 INFO:tasks.cephadm.osd.9:Stopped osd.9 2024-09-21T03:44:35.733 INFO:tasks.cephadm.osd.10:Stopping osd.10... 2024-09-21T03:44:35.733 DEBUG:teuthology.orchestra.run.smithi110:> sudo systemctl stop ceph-7c374098-77c9-11ef-baf5-efdc52797490@osd.10 2024-09-21T03:44:36.014 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 21 03:44:35 smithi110 systemd[1]: Stopping Ceph osd.10 for 7c374098-77c9-11ef-baf5-efdc52797490... 2024-09-21T03:44:36.461 INFO:journalctl@ceph.osd.11.smithi114.stdout:Sep 21 03:44:36 smithi114 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-11[58313]: 2024-09-21T03:44:36.140+0000 7f7657ce9640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2024-09-21T03:43:19.721069+0000 front 2024-09-21T03:43:19.721119+0000 (oldest deadline 2024-09-21T03:43:47.321877+0000) 2024-09-21T03:44:36.461 INFO:journalctl@ceph.osd.11.smithi114.stdout:Sep 21 03:44:36 smithi114 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-11[58313]: 2024-09-21T03:44:36.140+0000 7f7657ce9640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-21T03:43:27.322375+0000 front 2024-09-21T03:43:27.322426+0000 (oldest deadline 2024-09-21T03:43:51.422473+0000) 2024-09-21T03:44:36.461 INFO:journalctl@ceph.osd.11.smithi114.stdout:Sep 21 03:44:36 smithi114 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-11[58313]: 2024-09-21T03:44:36.140+0000 7f7657ce9640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.114:6804 osd.2 since back 2024-09-21T03:43:37.224230+0000 front 2024-09-21T03:43:37.224148+0000 (oldest deadline 2024-09-21T03:43:58.924202+0000) 2024-09-21T03:44:36.461 INFO:journalctl@ceph.osd.11.smithi114.stdout:Sep 21 03:44:36 smithi114 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-11[58313]: 2024-09-21T03:44:36.140+0000 7f7657ce9640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.110:6812 osd.4 since back 2024-09-21T03:43:50.626287+0000 front 2024-09-21T03:43:50.626333+0000 (oldest deadline 2024-09-21T03:44:15.326365+0000) 2024-09-21T03:44:36.461 INFO:journalctl@ceph.osd.11.smithi114.stdout:Sep 21 03:44:36 smithi114 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-11[58313]: 2024-09-21T03:44:36.140+0000 7f7657ce9640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.114:6812 osd.5 since back 2024-09-21T03:44:00.627730+0000 front 2024-09-21T03:44:00.627753+0000 (oldest deadline 2024-09-21T03:44:26.527718+0000) 2024-09-21T03:44:36.461 INFO:journalctl@ceph.osd.11.smithi114.stdout:Sep 21 03:44:36 smithi114 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-11[58313]: 2024-09-21T03:44:36.140+0000 7f7657ce9640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.89:6822 osd.6 since back 2024-09-21T03:44:07.029032+0000 front 2024-09-21T03:44:07.029045+0000 (oldest deadline 2024-09-21T03:44:29.928840+0000) 2024-09-21T03:44:36.502 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 21 03:44:36 smithi110 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-10[57089]: 2024-09-21T03:44:36.012+0000 7f9d4f3f9640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.10 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T03:44:36.502 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 21 03:44:36 smithi110 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-10[57089]: 2024-09-21T03:44:36.012+0000 7f9d4f3f9640 -1 osd.10 88 *** Got signal Terminated *** 2024-09-21T03:44:36.502 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 21 03:44:36 smithi110 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-10[57089]: 2024-09-21T03:44:36.012+0000 7f9d4f3f9640 -1 osd.10 88 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-21T03:44:36.503 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 21 03:44:36 smithi110 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-10[57089]: 2024-09-21T03:44:36.154+0000 7f9d479dc640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2024-09-21T03:43:21.264189+0000 front 2024-09-21T03:43:21.264118+0000 (oldest deadline 2024-09-21T03:43:43.564295+0000) 2024-09-21T03:44:36.503 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 21 03:44:36 smithi110 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-10[57089]: 2024-09-21T03:44:36.154+0000 7f9d479dc640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-21T03:43:25.265351+0000 front 2024-09-21T03:43:25.265386+0000 (oldest deadline 2024-09-21T03:43:49.965504+0000) 2024-09-21T03:44:36.503 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 21 03:44:36 smithi110 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-10[57089]: 2024-09-21T03:44:36.154+0000 7f9d479dc640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.114:6804 osd.2 since back 2024-09-21T03:43:33.966726+0000 front 2024-09-21T03:43:33.966884+0000 (oldest deadline 2024-09-21T03:43:59.866510+0000) 2024-09-21T03:44:36.503 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 21 03:44:36 smithi110 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-10[57089]: 2024-09-21T03:44:36.154+0000 7f9d479dc640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.89:6814 osd.3 since back 2024-09-21T03:43:43.367532+0000 front 2024-09-21T03:43:43.367626+0000 (oldest deadline 2024-09-21T03:44:06.867392+0000) 2024-09-21T03:44:36.503 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 21 03:44:36 smithi110 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-10[57089]: 2024-09-21T03:44:36.154+0000 7f9d479dc640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.110:6812 osd.4 since back 2024-09-21T03:43:48.568210+0000 front 2024-09-21T03:43:48.568248+0000 (oldest deadline 2024-09-21T03:44:14.468315+0000) 2024-09-21T03:44:36.503 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 21 03:44:36 smithi110 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-10[57089]: 2024-09-21T03:44:36.154+0000 7f9d479dc640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.114:6812 osd.5 since back 2024-09-21T03:43:56.769315+0000 front 2024-09-21T03:43:56.769405+0000 (oldest deadline 2024-09-21T03:44:22.069274+0000) 2024-09-21T03:44:36.503 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 21 03:44:36 smithi110 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-10[57089]: 2024-09-21T03:44:36.154+0000 7f9d479dc640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.89:6822 osd.6 since back 2024-09-21T03:44:06.770305+0000 front 2024-09-21T03:44:06.770257+0000 (oldest deadline 2024-09-21T03:44:30.270157+0000) 2024-09-21T03:44:37.461 INFO:journalctl@ceph.osd.11.smithi114.stdout:Sep 21 03:44:37 smithi114 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-11[58313]: 2024-09-21T03:44:37.100+0000 7f7657ce9640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2024-09-21T03:43:19.721069+0000 front 2024-09-21T03:43:19.721119+0000 (oldest deadline 2024-09-21T03:43:47.321877+0000) 2024-09-21T03:44:37.461 INFO:journalctl@ceph.osd.11.smithi114.stdout:Sep 21 03:44:37 smithi114 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-11[58313]: 2024-09-21T03:44:37.100+0000 7f7657ce9640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-21T03:43:27.322375+0000 front 2024-09-21T03:43:27.322426+0000 (oldest deadline 2024-09-21T03:43:51.422473+0000) 2024-09-21T03:44:37.461 INFO:journalctl@ceph.osd.11.smithi114.stdout:Sep 21 03:44:37 smithi114 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-11[58313]: 2024-09-21T03:44:37.100+0000 7f7657ce9640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.114:6804 osd.2 since back 2024-09-21T03:43:37.224230+0000 front 2024-09-21T03:43:37.224148+0000 (oldest deadline 2024-09-21T03:43:58.924202+0000) 2024-09-21T03:44:37.461 INFO:journalctl@ceph.osd.11.smithi114.stdout:Sep 21 03:44:37 smithi114 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-11[58313]: 2024-09-21T03:44:37.100+0000 7f7657ce9640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.110:6812 osd.4 since back 2024-09-21T03:43:50.626287+0000 front 2024-09-21T03:43:50.626333+0000 (oldest deadline 2024-09-21T03:44:15.326365+0000) 2024-09-21T03:44:37.461 INFO:journalctl@ceph.osd.11.smithi114.stdout:Sep 21 03:44:37 smithi114 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-11[58313]: 2024-09-21T03:44:37.100+0000 7f7657ce9640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.114:6812 osd.5 since back 2024-09-21T03:44:00.627730+0000 front 2024-09-21T03:44:00.627753+0000 (oldest deadline 2024-09-21T03:44:26.527718+0000) 2024-09-21T03:44:37.462 INFO:journalctl@ceph.osd.11.smithi114.stdout:Sep 21 03:44:37 smithi114 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-11[58313]: 2024-09-21T03:44:37.100+0000 7f7657ce9640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.89:6822 osd.6 since back 2024-09-21T03:44:07.029032+0000 front 2024-09-21T03:44:07.029045+0000 (oldest deadline 2024-09-21T03:44:29.928840+0000) 2024-09-21T03:44:37.500 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 21 03:44:37 smithi110 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-10[57089]: 2024-09-21T03:44:37.190+0000 7f9d479dc640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2024-09-21T03:43:21.264189+0000 front 2024-09-21T03:43:21.264118+0000 (oldest deadline 2024-09-21T03:43:43.564295+0000) 2024-09-21T03:44:37.500 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 21 03:44:37 smithi110 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-10[57089]: 2024-09-21T03:44:37.190+0000 7f9d479dc640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-21T03:43:25.265351+0000 front 2024-09-21T03:43:25.265386+0000 (oldest deadline 2024-09-21T03:43:49.965504+0000) 2024-09-21T03:44:37.500 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 21 03:44:37 smithi110 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-10[57089]: 2024-09-21T03:44:37.190+0000 7f9d479dc640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.114:6804 osd.2 since back 2024-09-21T03:43:33.966726+0000 front 2024-09-21T03:43:33.966884+0000 (oldest deadline 2024-09-21T03:43:59.866510+0000) 2024-09-21T03:44:37.500 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 21 03:44:37 smithi110 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-10[57089]: 2024-09-21T03:44:37.190+0000 7f9d479dc640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.89:6814 osd.3 since back 2024-09-21T03:43:43.367532+0000 front 2024-09-21T03:43:43.367626+0000 (oldest deadline 2024-09-21T03:44:06.867392+0000) 2024-09-21T03:44:37.500 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 21 03:44:37 smithi110 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-10[57089]: 2024-09-21T03:44:37.190+0000 7f9d479dc640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.110:6812 osd.4 since back 2024-09-21T03:43:48.568210+0000 front 2024-09-21T03:43:48.568248+0000 (oldest deadline 2024-09-21T03:44:14.468315+0000) 2024-09-21T03:44:37.500 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 21 03:44:37 smithi110 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-10[57089]: 2024-09-21T03:44:37.190+0000 7f9d479dc640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.114:6812 osd.5 since back 2024-09-21T03:43:56.769315+0000 front 2024-09-21T03:43:56.769405+0000 (oldest deadline 2024-09-21T03:44:22.069274+0000) 2024-09-21T03:44:37.500 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 21 03:44:37 smithi110 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-10[57089]: 2024-09-21T03:44:37.190+0000 7f9d479dc640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.89:6822 osd.6 since back 2024-09-21T03:44:06.770305+0000 front 2024-09-21T03:44:06.770257+0000 (oldest deadline 2024-09-21T03:44:30.270157+0000) 2024-09-21T03:44:38.362 INFO:journalctl@ceph.osd.11.smithi114.stdout:Sep 21 03:44:38 smithi114 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-11[58313]: 2024-09-21T03:44:38.103+0000 7f7657ce9640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2024-09-21T03:43:19.721069+0000 front 2024-09-21T03:43:19.721119+0000 (oldest deadline 2024-09-21T03:43:47.321877+0000) 2024-09-21T03:44:38.362 INFO:journalctl@ceph.osd.11.smithi114.stdout:Sep 21 03:44:38 smithi114 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-11[58313]: 2024-09-21T03:44:38.103+0000 7f7657ce9640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-21T03:43:27.322375+0000 front 2024-09-21T03:43:27.322426+0000 (oldest deadline 2024-09-21T03:43:51.422473+0000) 2024-09-21T03:44:38.363 INFO:journalctl@ceph.osd.11.smithi114.stdout:Sep 21 03:44:38 smithi114 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-11[58313]: 2024-09-21T03:44:38.103+0000 7f7657ce9640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.114:6804 osd.2 since back 2024-09-21T03:43:37.224230+0000 front 2024-09-21T03:43:37.224148+0000 (oldest deadline 2024-09-21T03:43:58.924202+0000) 2024-09-21T03:44:38.363 INFO:journalctl@ceph.osd.11.smithi114.stdout:Sep 21 03:44:38 smithi114 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-11[58313]: 2024-09-21T03:44:38.103+0000 7f7657ce9640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.110:6812 osd.4 since back 2024-09-21T03:43:50.626287+0000 front 2024-09-21T03:43:50.626333+0000 (oldest deadline 2024-09-21T03:44:15.326365+0000) 2024-09-21T03:44:38.363 INFO:journalctl@ceph.osd.11.smithi114.stdout:Sep 21 03:44:38 smithi114 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-11[58313]: 2024-09-21T03:44:38.103+0000 7f7657ce9640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.114:6812 osd.5 since back 2024-09-21T03:44:00.627730+0000 front 2024-09-21T03:44:00.627753+0000 (oldest deadline 2024-09-21T03:44:26.527718+0000) 2024-09-21T03:44:38.363 INFO:journalctl@ceph.osd.11.smithi114.stdout:Sep 21 03:44:38 smithi114 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-11[58313]: 2024-09-21T03:44:38.103+0000 7f7657ce9640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.89:6822 osd.6 since back 2024-09-21T03:44:07.029032+0000 front 2024-09-21T03:44:07.029045+0000 (oldest deadline 2024-09-21T03:44:29.928840+0000) 2024-09-21T03:44:38.500 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 21 03:44:38 smithi110 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-10[57089]: 2024-09-21T03:44:38.193+0000 7f9d479dc640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2024-09-21T03:43:21.264189+0000 front 2024-09-21T03:43:21.264118+0000 (oldest deadline 2024-09-21T03:43:43.564295+0000) 2024-09-21T03:44:38.500 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 21 03:44:38 smithi110 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-10[57089]: 2024-09-21T03:44:38.193+0000 7f9d479dc640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-21T03:43:25.265351+0000 front 2024-09-21T03:43:25.265386+0000 (oldest deadline 2024-09-21T03:43:49.965504+0000) 2024-09-21T03:44:38.500 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 21 03:44:38 smithi110 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-10[57089]: 2024-09-21T03:44:38.193+0000 7f9d479dc640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.114:6804 osd.2 since back 2024-09-21T03:43:33.966726+0000 front 2024-09-21T03:43:33.966884+0000 (oldest deadline 2024-09-21T03:43:59.866510+0000) 2024-09-21T03:44:38.500 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 21 03:44:38 smithi110 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-10[57089]: 2024-09-21T03:44:38.193+0000 7f9d479dc640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.89:6814 osd.3 since back 2024-09-21T03:43:43.367532+0000 front 2024-09-21T03:43:43.367626+0000 (oldest deadline 2024-09-21T03:44:06.867392+0000) 2024-09-21T03:44:38.500 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 21 03:44:38 smithi110 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-10[57089]: 2024-09-21T03:44:38.193+0000 7f9d479dc640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.110:6812 osd.4 since back 2024-09-21T03:43:48.568210+0000 front 2024-09-21T03:43:48.568248+0000 (oldest deadline 2024-09-21T03:44:14.468315+0000) 2024-09-21T03:44:38.500 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 21 03:44:38 smithi110 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-10[57089]: 2024-09-21T03:44:38.193+0000 7f9d479dc640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.114:6812 osd.5 since back 2024-09-21T03:43:56.769315+0000 front 2024-09-21T03:43:56.769405+0000 (oldest deadline 2024-09-21T03:44:22.069274+0000) 2024-09-21T03:44:38.500 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 21 03:44:38 smithi110 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-10[57089]: 2024-09-21T03:44:38.193+0000 7f9d479dc640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.89:6822 osd.6 since back 2024-09-21T03:44:06.770305+0000 front 2024-09-21T03:44:06.770257+0000 (oldest deadline 2024-09-21T03:44:30.270157+0000) 2024-09-21T03:44:39.374 INFO:journalctl@ceph.osd.11.smithi114.stdout:Sep 21 03:44:39 smithi114 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-11[58313]: 2024-09-21T03:44:39.077+0000 7f7657ce9640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2024-09-21T03:43:19.721069+0000 front 2024-09-21T03:43:19.721119+0000 (oldest deadline 2024-09-21T03:43:47.321877+0000) 2024-09-21T03:44:39.374 INFO:journalctl@ceph.osd.11.smithi114.stdout:Sep 21 03:44:39 smithi114 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-11[58313]: 2024-09-21T03:44:39.077+0000 7f7657ce9640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-21T03:43:27.322375+0000 front 2024-09-21T03:43:27.322426+0000 (oldest deadline 2024-09-21T03:43:51.422473+0000) 2024-09-21T03:44:39.374 INFO:journalctl@ceph.osd.11.smithi114.stdout:Sep 21 03:44:39 smithi114 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-11[58313]: 2024-09-21T03:44:39.077+0000 7f7657ce9640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.114:6804 osd.2 since back 2024-09-21T03:43:37.224230+0000 front 2024-09-21T03:43:37.224148+0000 (oldest deadline 2024-09-21T03:43:58.924202+0000) 2024-09-21T03:44:39.374 INFO:journalctl@ceph.osd.11.smithi114.stdout:Sep 21 03:44:39 smithi114 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-11[58313]: 2024-09-21T03:44:39.077+0000 7f7657ce9640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.110:6812 osd.4 since back 2024-09-21T03:43:50.626287+0000 front 2024-09-21T03:43:50.626333+0000 (oldest deadline 2024-09-21T03:44:15.326365+0000) 2024-09-21T03:44:39.374 INFO:journalctl@ceph.osd.11.smithi114.stdout:Sep 21 03:44:39 smithi114 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-11[58313]: 2024-09-21T03:44:39.077+0000 7f7657ce9640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.114:6812 osd.5 since back 2024-09-21T03:44:00.627730+0000 front 2024-09-21T03:44:00.627753+0000 (oldest deadline 2024-09-21T03:44:26.527718+0000) 2024-09-21T03:44:39.374 INFO:journalctl@ceph.osd.11.smithi114.stdout:Sep 21 03:44:39 smithi114 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-11[58313]: 2024-09-21T03:44:39.077+0000 7f7657ce9640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.89:6822 osd.6 since back 2024-09-21T03:44:07.029032+0000 front 2024-09-21T03:44:07.029045+0000 (oldest deadline 2024-09-21T03:44:29.928840+0000) 2024-09-21T03:44:39.500 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 21 03:44:39 smithi110 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-10[57089]: 2024-09-21T03:44:39.188+0000 7f9d479dc640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2024-09-21T03:43:21.264189+0000 front 2024-09-21T03:43:21.264118+0000 (oldest deadline 2024-09-21T03:43:43.564295+0000) 2024-09-21T03:44:39.500 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 21 03:44:39 smithi110 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-10[57089]: 2024-09-21T03:44:39.188+0000 7f9d479dc640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-21T03:43:25.265351+0000 front 2024-09-21T03:43:25.265386+0000 (oldest deadline 2024-09-21T03:43:49.965504+0000) 2024-09-21T03:44:39.500 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 21 03:44:39 smithi110 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-10[57089]: 2024-09-21T03:44:39.188+0000 7f9d479dc640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.114:6804 osd.2 since back 2024-09-21T03:43:33.966726+0000 front 2024-09-21T03:43:33.966884+0000 (oldest deadline 2024-09-21T03:43:59.866510+0000) 2024-09-21T03:44:39.501 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 21 03:44:39 smithi110 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-10[57089]: 2024-09-21T03:44:39.188+0000 7f9d479dc640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.89:6814 osd.3 since back 2024-09-21T03:43:43.367532+0000 front 2024-09-21T03:43:43.367626+0000 (oldest deadline 2024-09-21T03:44:06.867392+0000) 2024-09-21T03:44:39.501 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 21 03:44:39 smithi110 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-10[57089]: 2024-09-21T03:44:39.188+0000 7f9d479dc640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.110:6812 osd.4 since back 2024-09-21T03:43:48.568210+0000 front 2024-09-21T03:43:48.568248+0000 (oldest deadline 2024-09-21T03:44:14.468315+0000) 2024-09-21T03:44:39.501 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 21 03:44:39 smithi110 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-10[57089]: 2024-09-21T03:44:39.188+0000 7f9d479dc640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.114:6812 osd.5 since back 2024-09-21T03:43:56.769315+0000 front 2024-09-21T03:43:56.769405+0000 (oldest deadline 2024-09-21T03:44:22.069274+0000) 2024-09-21T03:44:39.501 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 21 03:44:39 smithi110 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-10[57089]: 2024-09-21T03:44:39.188+0000 7f9d479dc640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.89:6822 osd.6 since back 2024-09-21T03:44:06.770305+0000 front 2024-09-21T03:44:06.770257+0000 (oldest deadline 2024-09-21T03:44:30.270157+0000) 2024-09-21T03:44:40.461 INFO:journalctl@ceph.osd.11.smithi114.stdout:Sep 21 03:44:40 smithi114 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-11[58313]: 2024-09-21T03:44:40.098+0000 7f7657ce9640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2024-09-21T03:43:19.721069+0000 front 2024-09-21T03:43:19.721119+0000 (oldest deadline 2024-09-21T03:43:47.321877+0000) 2024-09-21T03:44:40.461 INFO:journalctl@ceph.osd.11.smithi114.stdout:Sep 21 03:44:40 smithi114 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-11[58313]: 2024-09-21T03:44:40.098+0000 7f7657ce9640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-21T03:43:27.322375+0000 front 2024-09-21T03:43:27.322426+0000 (oldest deadline 2024-09-21T03:43:51.422473+0000) 2024-09-21T03:44:40.461 INFO:journalctl@ceph.osd.11.smithi114.stdout:Sep 21 03:44:40 smithi114 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-11[58313]: 2024-09-21T03:44:40.098+0000 7f7657ce9640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.114:6804 osd.2 since back 2024-09-21T03:43:37.224230+0000 front 2024-09-21T03:43:37.224148+0000 (oldest deadline 2024-09-21T03:43:58.924202+0000) 2024-09-21T03:44:40.461 INFO:journalctl@ceph.osd.11.smithi114.stdout:Sep 21 03:44:40 smithi114 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-11[58313]: 2024-09-21T03:44:40.098+0000 7f7657ce9640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.110:6812 osd.4 since back 2024-09-21T03:43:50.626287+0000 front 2024-09-21T03:43:50.626333+0000 (oldest deadline 2024-09-21T03:44:15.326365+0000) 2024-09-21T03:44:40.462 INFO:journalctl@ceph.osd.11.smithi114.stdout:Sep 21 03:44:40 smithi114 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-11[58313]: 2024-09-21T03:44:40.098+0000 7f7657ce9640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.114:6812 osd.5 since back 2024-09-21T03:44:00.627730+0000 front 2024-09-21T03:44:00.627753+0000 (oldest deadline 2024-09-21T03:44:26.527718+0000) 2024-09-21T03:44:40.462 INFO:journalctl@ceph.osd.11.smithi114.stdout:Sep 21 03:44:40 smithi114 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-11[58313]: 2024-09-21T03:44:40.098+0000 7f7657ce9640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.89:6822 osd.6 since back 2024-09-21T03:44:07.029032+0000 front 2024-09-21T03:44:07.029045+0000 (oldest deadline 2024-09-21T03:44:29.928840+0000) 2024-09-21T03:44:40.462 INFO:journalctl@ceph.osd.11.smithi114.stdout:Sep 21 03:44:40 smithi114 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-11[58313]: 2024-09-21T03:44:40.098+0000 7f7657ce9640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.110:6820 osd.7 since back 2024-09-21T03:44:16.830368+0000 front 2024-09-21T03:44:16.830322+0000 (oldest deadline 2024-09-21T03:44:39.130216+0000) 2024-09-21T03:44:40.500 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 21 03:44:40 smithi110 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-10[57089]: 2024-09-21T03:44:40.160+0000 7f9d479dc640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2024-09-21T03:43:21.264189+0000 front 2024-09-21T03:43:21.264118+0000 (oldest deadline 2024-09-21T03:43:43.564295+0000) 2024-09-21T03:44:40.500 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 21 03:44:40 smithi110 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-10[57089]: 2024-09-21T03:44:40.160+0000 7f9d479dc640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-21T03:43:25.265351+0000 front 2024-09-21T03:43:25.265386+0000 (oldest deadline 2024-09-21T03:43:49.965504+0000) 2024-09-21T03:44:40.500 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 21 03:44:40 smithi110 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-10[57089]: 2024-09-21T03:44:40.160+0000 7f9d479dc640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.114:6804 osd.2 since back 2024-09-21T03:43:33.966726+0000 front 2024-09-21T03:43:33.966884+0000 (oldest deadline 2024-09-21T03:43:59.866510+0000) 2024-09-21T03:44:40.500 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 21 03:44:40 smithi110 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-10[57089]: 2024-09-21T03:44:40.160+0000 7f9d479dc640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.89:6814 osd.3 since back 2024-09-21T03:43:43.367532+0000 front 2024-09-21T03:43:43.367626+0000 (oldest deadline 2024-09-21T03:44:06.867392+0000) 2024-09-21T03:44:40.500 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 21 03:44:40 smithi110 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-10[57089]: 2024-09-21T03:44:40.160+0000 7f9d479dc640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.110:6812 osd.4 since back 2024-09-21T03:43:48.568210+0000 front 2024-09-21T03:43:48.568248+0000 (oldest deadline 2024-09-21T03:44:14.468315+0000) 2024-09-21T03:44:40.500 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 21 03:44:40 smithi110 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-10[57089]: 2024-09-21T03:44:40.160+0000 7f9d479dc640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.114:6812 osd.5 since back 2024-09-21T03:43:56.769315+0000 front 2024-09-21T03:43:56.769405+0000 (oldest deadline 2024-09-21T03:44:22.069274+0000) 2024-09-21T03:44:40.501 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 21 03:44:40 smithi110 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-10[57089]: 2024-09-21T03:44:40.160+0000 7f9d479dc640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.89:6822 osd.6 since back 2024-09-21T03:44:06.770305+0000 front 2024-09-21T03:44:06.770257+0000 (oldest deadline 2024-09-21T03:44:30.270157+0000) 2024-09-21T03:44:41.461 INFO:journalctl@ceph.osd.11.smithi114.stdout:Sep 21 03:44:41 smithi114 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-11[58313]: 2024-09-21T03:44:41.119+0000 7f7657ce9640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2024-09-21T03:43:19.721069+0000 front 2024-09-21T03:43:19.721119+0000 (oldest deadline 2024-09-21T03:43:47.321877+0000) 2024-09-21T03:44:41.461 INFO:journalctl@ceph.osd.11.smithi114.stdout:Sep 21 03:44:41 smithi114 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-11[58313]: 2024-09-21T03:44:41.119+0000 7f7657ce9640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-21T03:43:27.322375+0000 front 2024-09-21T03:43:27.322426+0000 (oldest deadline 2024-09-21T03:43:51.422473+0000) 2024-09-21T03:44:41.461 INFO:journalctl@ceph.osd.11.smithi114.stdout:Sep 21 03:44:41 smithi114 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-11[58313]: 2024-09-21T03:44:41.119+0000 7f7657ce9640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.114:6804 osd.2 since back 2024-09-21T03:43:37.224230+0000 front 2024-09-21T03:43:37.224148+0000 (oldest deadline 2024-09-21T03:43:58.924202+0000) 2024-09-21T03:44:41.461 INFO:journalctl@ceph.osd.11.smithi114.stdout:Sep 21 03:44:41 smithi114 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-11[58313]: 2024-09-21T03:44:41.119+0000 7f7657ce9640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.110:6812 osd.4 since back 2024-09-21T03:43:50.626287+0000 front 2024-09-21T03:43:50.626333+0000 (oldest deadline 2024-09-21T03:44:15.326365+0000) 2024-09-21T03:44:41.461 INFO:journalctl@ceph.osd.11.smithi114.stdout:Sep 21 03:44:41 smithi114 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-11[58313]: 2024-09-21T03:44:41.119+0000 7f7657ce9640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.114:6812 osd.5 since back 2024-09-21T03:44:00.627730+0000 front 2024-09-21T03:44:00.627753+0000 (oldest deadline 2024-09-21T03:44:26.527718+0000) 2024-09-21T03:44:41.462 INFO:journalctl@ceph.osd.11.smithi114.stdout:Sep 21 03:44:41 smithi114 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-11[58313]: 2024-09-21T03:44:41.119+0000 7f7657ce9640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.89:6822 osd.6 since back 2024-09-21T03:44:07.029032+0000 front 2024-09-21T03:44:07.029045+0000 (oldest deadline 2024-09-21T03:44:29.928840+0000) 2024-09-21T03:44:41.462 INFO:journalctl@ceph.osd.11.smithi114.stdout:Sep 21 03:44:41 smithi114 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-11[58313]: 2024-09-21T03:44:41.119+0000 7f7657ce9640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.110:6820 osd.7 since back 2024-09-21T03:44:16.830368+0000 front 2024-09-21T03:44:16.830322+0000 (oldest deadline 2024-09-21T03:44:39.130216+0000) 2024-09-21T03:44:41.697 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 21 03:44:41 smithi110 podman[65373]: 2024-09-21 03:44:41.41747019 +0000 UTC m=+5.496830126 container died b0203de153a771916a52fa6b23bc6ed6c013b49b10e15186ed81d2c681b525b2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-10, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-21T03:44:42.000 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 21 03:44:41 smithi110 podman[65373]: 2024-09-21 03:44:41.812410841 +0000 UTC m=+5.891770772 container cleanup b0203de153a771916a52fa6b23bc6ed6c013b49b10e15186ed81d2c681b525b2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-10, ceph=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.37.2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-21T03:44:42.001 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 21 03:44:41 smithi110 bash[65373]: ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-10 2024-09-21T03:44:42.348 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 21 03:44:42 smithi110 podman[65388]: 2024-09-21 03:44:42.038379741 +0000 UTC m=+0.623014559 container remove b0203de153a771916a52fa6b23bc6ed6c013b49b10e15186ed81d2c681b525b2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-10, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_BRANCH=HEAD) 2024-09-21T03:44:42.461 INFO:journalctl@ceph.osd.11.smithi114.stdout:Sep 21 03:44:42 smithi114 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-11[58313]: 2024-09-21T03:44:42.111+0000 7f7657ce9640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2024-09-21T03:43:19.721069+0000 front 2024-09-21T03:43:19.721119+0000 (oldest deadline 2024-09-21T03:43:47.321877+0000) 2024-09-21T03:44:42.461 INFO:journalctl@ceph.osd.11.smithi114.stdout:Sep 21 03:44:42 smithi114 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-11[58313]: 2024-09-21T03:44:42.111+0000 7f7657ce9640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-21T03:43:27.322375+0000 front 2024-09-21T03:43:27.322426+0000 (oldest deadline 2024-09-21T03:43:51.422473+0000) 2024-09-21T03:44:42.461 INFO:journalctl@ceph.osd.11.smithi114.stdout:Sep 21 03:44:42 smithi114 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-11[58313]: 2024-09-21T03:44:42.111+0000 7f7657ce9640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.114:6804 osd.2 since back 2024-09-21T03:43:37.224230+0000 front 2024-09-21T03:43:37.224148+0000 (oldest deadline 2024-09-21T03:43:58.924202+0000) 2024-09-21T03:44:42.461 INFO:journalctl@ceph.osd.11.smithi114.stdout:Sep 21 03:44:42 smithi114 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-11[58313]: 2024-09-21T03:44:42.111+0000 7f7657ce9640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.110:6812 osd.4 since back 2024-09-21T03:43:50.626287+0000 front 2024-09-21T03:43:50.626333+0000 (oldest deadline 2024-09-21T03:44:15.326365+0000) 2024-09-21T03:44:42.461 INFO:journalctl@ceph.osd.11.smithi114.stdout:Sep 21 03:44:42 smithi114 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-11[58313]: 2024-09-21T03:44:42.111+0000 7f7657ce9640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.114:6812 osd.5 since back 2024-09-21T03:44:00.627730+0000 front 2024-09-21T03:44:00.627753+0000 (oldest deadline 2024-09-21T03:44:26.527718+0000) 2024-09-21T03:44:42.461 INFO:journalctl@ceph.osd.11.smithi114.stdout:Sep 21 03:44:42 smithi114 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-11[58313]: 2024-09-21T03:44:42.111+0000 7f7657ce9640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.89:6822 osd.6 since back 2024-09-21T03:44:07.029032+0000 front 2024-09-21T03:44:07.029045+0000 (oldest deadline 2024-09-21T03:44:29.928840+0000) 2024-09-21T03:44:42.461 INFO:journalctl@ceph.osd.11.smithi114.stdout:Sep 21 03:44:42 smithi114 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-11[58313]: 2024-09-21T03:44:42.111+0000 7f7657ce9640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.110:6820 osd.7 since back 2024-09-21T03:44:16.830368+0000 front 2024-09-21T03:44:16.830322+0000 (oldest deadline 2024-09-21T03:44:39.130216+0000) 2024-09-21T03:44:42.609 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 21 03:44:42 smithi110 podman[65478]: 2024-09-21 03:44:42.249227664 +0000 UTC m=+0.015627907 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T03:44:42.609 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 21 03:44:42 smithi110 podman[65478]: 2024-09-21 03:44:42.34746505 +0000 UTC m=+0.113865280 container create c360a14b01d90f3064cf57ad38b52b5adbba91eb29c8b4279372f5287f0868bf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-10-deactivate, io.buildah.version=1.37.2, ceph=True, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.license=GPLv2) 2024-09-21T03:44:42.609 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 21 03:44:42 smithi110 podman[65478]: 2024-09-21 03:44:42.533478626 +0000 UTC m=+0.299878856 container init c360a14b01d90f3064cf57ad38b52b5adbba91eb29c8b4279372f5287f0868bf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-10-deactivate, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_CLEAN=True) 2024-09-21T03:44:42.610 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 21 03:44:42 smithi110 podman[65478]: 2024-09-21 03:44:42.536752552 +0000 UTC m=+0.303152785 container start c360a14b01d90f3064cf57ad38b52b5adbba91eb29c8b4279372f5287f0868bf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-10-deactivate, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731) 2024-09-21T03:44:42.610 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 21 03:44:42 smithi110 podman[65478]: 2024-09-21 03:44:42.608174113 +0000 UTC m=+0.374574358 container attach c360a14b01d90f3064cf57ad38b52b5adbba91eb29c8b4279372f5287f0868bf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-10-deactivate, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.license=GPLv2) 2024-09-21T03:44:43.000 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 21 03:44:42 smithi110 podman[65478]: 2024-09-21 03:44:42.724177058 +0000 UTC m=+0.490577299 container died c360a14b01d90f3064cf57ad38b52b5adbba91eb29c8b4279372f5287f0868bf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-10-deactivate, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, ceph=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2) 2024-09-21T03:44:43.141 INFO:journalctl@ceph.osd.11.smithi114.stdout:Sep 21 03:44:43 smithi114 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-11[58313]: 2024-09-21T03:44:43.138+0000 7f7657ce9640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2024-09-21T03:43:19.721069+0000 front 2024-09-21T03:43:19.721119+0000 (oldest deadline 2024-09-21T03:43:47.321877+0000) 2024-09-21T03:44:43.141 INFO:journalctl@ceph.osd.11.smithi114.stdout:Sep 21 03:44:43 smithi114 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-11[58313]: 2024-09-21T03:44:43.138+0000 7f7657ce9640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-21T03:43:27.322375+0000 front 2024-09-21T03:43:27.322426+0000 (oldest deadline 2024-09-21T03:43:51.422473+0000) 2024-09-21T03:44:43.141 INFO:journalctl@ceph.osd.11.smithi114.stdout:Sep 21 03:44:43 smithi114 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-11[58313]: 2024-09-21T03:44:43.138+0000 7f7657ce9640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.114:6804 osd.2 since back 2024-09-21T03:43:37.224230+0000 front 2024-09-21T03:43:37.224148+0000 (oldest deadline 2024-09-21T03:43:58.924202+0000) 2024-09-21T03:44:43.142 INFO:journalctl@ceph.osd.11.smithi114.stdout:Sep 21 03:44:43 smithi114 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-11[58313]: 2024-09-21T03:44:43.138+0000 7f7657ce9640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.110:6812 osd.4 since back 2024-09-21T03:43:50.626287+0000 front 2024-09-21T03:43:50.626333+0000 (oldest deadline 2024-09-21T03:44:15.326365+0000) 2024-09-21T03:44:43.142 INFO:journalctl@ceph.osd.11.smithi114.stdout:Sep 21 03:44:43 smithi114 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-11[58313]: 2024-09-21T03:44:43.138+0000 7f7657ce9640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.114:6812 osd.5 since back 2024-09-21T03:44:00.627730+0000 front 2024-09-21T03:44:00.627753+0000 (oldest deadline 2024-09-21T03:44:26.527718+0000) 2024-09-21T03:44:43.142 INFO:journalctl@ceph.osd.11.smithi114.stdout:Sep 21 03:44:43 smithi114 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-11[58313]: 2024-09-21T03:44:43.138+0000 7f7657ce9640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.89:6822 osd.6 since back 2024-09-21T03:44:07.029032+0000 front 2024-09-21T03:44:07.029045+0000 (oldest deadline 2024-09-21T03:44:29.928840+0000) 2024-09-21T03:44:43.142 INFO:journalctl@ceph.osd.11.smithi114.stdout:Sep 21 03:44:43 smithi114 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-11[58313]: 2024-09-21T03:44:43.138+0000 7f7657ce9640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.110:6820 osd.7 since back 2024-09-21T03:44:16.830368+0000 front 2024-09-21T03:44:16.830322+0000 (oldest deadline 2024-09-21T03:44:39.130216+0000) 2024-09-21T03:44:43.270 DEBUG:teuthology.orchestra.run.smithi110:> sudo pkill -f 'journalctl -f -n 0 -u ceph-7c374098-77c9-11ef-baf5-efdc52797490@osd.10.service' 2024-09-21T03:44:43.293 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 21 03:44:43 smithi110 podman[65502]: 2024-09-21 03:44:43.253357903 +0000 UTC m=+0.523051902 container remove c360a14b01d90f3064cf57ad38b52b5adbba91eb29c8b4279372f5287f0868bf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-10-deactivate, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.build-date=20240731, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221) 2024-09-21T03:44:43.293 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 21 03:44:43 smithi110 systemd[1]: ceph-7c374098-77c9-11ef-baf5-efdc52797490@osd.10.service: Deactivated successfully. 2024-09-21T03:44:43.294 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 21 03:44:43 smithi110 systemd[1]: Stopped Ceph osd.10 for 7c374098-77c9-11ef-baf5-efdc52797490. 2024-09-21T03:44:43.294 INFO:journalctl@ceph.osd.10.smithi110.stdout:Sep 21 03:44:43 smithi110 systemd[1]: ceph-7c374098-77c9-11ef-baf5-efdc52797490@osd.10.service: Consumed 1min 4.128s CPU time. 2024-09-21T03:44:43.308 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T03:44:43.308 INFO:tasks.cephadm.osd.10:Stopped osd.10 2024-09-21T03:44:43.308 INFO:tasks.cephadm.osd.11:Stopping osd.11... 2024-09-21T03:44:43.308 DEBUG:teuthology.orchestra.run.smithi114:> sudo systemctl stop ceph-7c374098-77c9-11ef-baf5-efdc52797490@osd.11 2024-09-21T03:44:43.461 INFO:journalctl@ceph.osd.11.smithi114.stdout:Sep 21 03:44:43 smithi114 systemd[1]: Stopping Ceph osd.11 for 7c374098-77c9-11ef-baf5-efdc52797490... 2024-09-21T03:44:43.961 INFO:journalctl@ceph.osd.11.smithi114.stdout:Sep 21 03:44:43 smithi114 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-11[58313]: 2024-09-21T03:44:43.607+0000 7f765f706640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.11 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T03:44:43.961 INFO:journalctl@ceph.osd.11.smithi114.stdout:Sep 21 03:44:43 smithi114 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-11[58313]: 2024-09-21T03:44:43.607+0000 7f765f706640 -1 osd.11 88 *** Got signal Terminated *** 2024-09-21T03:44:43.961 INFO:journalctl@ceph.osd.11.smithi114.stdout:Sep 21 03:44:43 smithi114 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-11[58313]: 2024-09-21T03:44:43.607+0000 7f765f706640 -1 osd.11 88 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-21T03:44:44.461 INFO:journalctl@ceph.osd.11.smithi114.stdout:Sep 21 03:44:44 smithi114 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-11[58313]: 2024-09-21T03:44:44.168+0000 7f7657ce9640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2024-09-21T03:43:19.721069+0000 front 2024-09-21T03:43:19.721119+0000 (oldest deadline 2024-09-21T03:43:47.321877+0000) 2024-09-21T03:44:44.461 INFO:journalctl@ceph.osd.11.smithi114.stdout:Sep 21 03:44:44 smithi114 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-11[58313]: 2024-09-21T03:44:44.168+0000 7f7657ce9640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-21T03:43:27.322375+0000 front 2024-09-21T03:43:27.322426+0000 (oldest deadline 2024-09-21T03:43:51.422473+0000) 2024-09-21T03:44:44.461 INFO:journalctl@ceph.osd.11.smithi114.stdout:Sep 21 03:44:44 smithi114 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-11[58313]: 2024-09-21T03:44:44.168+0000 7f7657ce9640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.114:6804 osd.2 since back 2024-09-21T03:43:37.224230+0000 front 2024-09-21T03:43:37.224148+0000 (oldest deadline 2024-09-21T03:43:58.924202+0000) 2024-09-21T03:44:44.461 INFO:journalctl@ceph.osd.11.smithi114.stdout:Sep 21 03:44:44 smithi114 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-11[58313]: 2024-09-21T03:44:44.168+0000 7f7657ce9640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.110:6812 osd.4 since back 2024-09-21T03:43:50.626287+0000 front 2024-09-21T03:43:50.626333+0000 (oldest deadline 2024-09-21T03:44:15.326365+0000) 2024-09-21T03:44:44.461 INFO:journalctl@ceph.osd.11.smithi114.stdout:Sep 21 03:44:44 smithi114 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-11[58313]: 2024-09-21T03:44:44.168+0000 7f7657ce9640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.114:6812 osd.5 since back 2024-09-21T03:44:00.627730+0000 front 2024-09-21T03:44:00.627753+0000 (oldest deadline 2024-09-21T03:44:26.527718+0000) 2024-09-21T03:44:44.461 INFO:journalctl@ceph.osd.11.smithi114.stdout:Sep 21 03:44:44 smithi114 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-11[58313]: 2024-09-21T03:44:44.168+0000 7f7657ce9640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.89:6822 osd.6 since back 2024-09-21T03:44:07.029032+0000 front 2024-09-21T03:44:07.029045+0000 (oldest deadline 2024-09-21T03:44:29.928840+0000) 2024-09-21T03:44:44.462 INFO:journalctl@ceph.osd.11.smithi114.stdout:Sep 21 03:44:44 smithi114 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-11[58313]: 2024-09-21T03:44:44.168+0000 7f7657ce9640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.110:6820 osd.7 since back 2024-09-21T03:44:16.830368+0000 front 2024-09-21T03:44:16.830322+0000 (oldest deadline 2024-09-21T03:44:39.130216+0000) 2024-09-21T03:44:45.461 INFO:journalctl@ceph.osd.11.smithi114.stdout:Sep 21 03:44:45 smithi114 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-11[58313]: 2024-09-21T03:44:45.206+0000 7f7657ce9640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2024-09-21T03:43:19.721069+0000 front 2024-09-21T03:43:19.721119+0000 (oldest deadline 2024-09-21T03:43:47.321877+0000) 2024-09-21T03:44:45.461 INFO:journalctl@ceph.osd.11.smithi114.stdout:Sep 21 03:44:45 smithi114 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-11[58313]: 2024-09-21T03:44:45.206+0000 7f7657ce9640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-21T03:43:27.322375+0000 front 2024-09-21T03:43:27.322426+0000 (oldest deadline 2024-09-21T03:43:51.422473+0000) 2024-09-21T03:44:45.461 INFO:journalctl@ceph.osd.11.smithi114.stdout:Sep 21 03:44:45 smithi114 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-11[58313]: 2024-09-21T03:44:45.206+0000 7f7657ce9640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.114:6804 osd.2 since back 2024-09-21T03:43:37.224230+0000 front 2024-09-21T03:43:37.224148+0000 (oldest deadline 2024-09-21T03:43:58.924202+0000) 2024-09-21T03:44:45.461 INFO:journalctl@ceph.osd.11.smithi114.stdout:Sep 21 03:44:45 smithi114 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-11[58313]: 2024-09-21T03:44:45.206+0000 7f7657ce9640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.110:6812 osd.4 since back 2024-09-21T03:43:50.626287+0000 front 2024-09-21T03:43:50.626333+0000 (oldest deadline 2024-09-21T03:44:15.326365+0000) 2024-09-21T03:44:45.461 INFO:journalctl@ceph.osd.11.smithi114.stdout:Sep 21 03:44:45 smithi114 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-11[58313]: 2024-09-21T03:44:45.206+0000 7f7657ce9640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.114:6812 osd.5 since back 2024-09-21T03:44:00.627730+0000 front 2024-09-21T03:44:00.627753+0000 (oldest deadline 2024-09-21T03:44:26.527718+0000) 2024-09-21T03:44:45.461 INFO:journalctl@ceph.osd.11.smithi114.stdout:Sep 21 03:44:45 smithi114 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-11[58313]: 2024-09-21T03:44:45.206+0000 7f7657ce9640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.89:6822 osd.6 since back 2024-09-21T03:44:07.029032+0000 front 2024-09-21T03:44:07.029045+0000 (oldest deadline 2024-09-21T03:44:29.928840+0000) 2024-09-21T03:44:45.461 INFO:journalctl@ceph.osd.11.smithi114.stdout:Sep 21 03:44:45 smithi114 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-11[58313]: 2024-09-21T03:44:45.206+0000 7f7657ce9640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.110:6820 osd.7 since back 2024-09-21T03:44:16.830368+0000 front 2024-09-21T03:44:16.830322+0000 (oldest deadline 2024-09-21T03:44:39.130216+0000) 2024-09-21T03:44:46.711 INFO:journalctl@ceph.osd.11.smithi114.stdout:Sep 21 03:44:46 smithi114 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-11[58313]: 2024-09-21T03:44:46.222+0000 7f7657ce9640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2024-09-21T03:43:19.721069+0000 front 2024-09-21T03:43:19.721119+0000 (oldest deadline 2024-09-21T03:43:47.321877+0000) 2024-09-21T03:44:46.711 INFO:journalctl@ceph.osd.11.smithi114.stdout:Sep 21 03:44:46 smithi114 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-11[58313]: 2024-09-21T03:44:46.222+0000 7f7657ce9640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-21T03:43:27.322375+0000 front 2024-09-21T03:43:27.322426+0000 (oldest deadline 2024-09-21T03:43:51.422473+0000) 2024-09-21T03:44:46.712 INFO:journalctl@ceph.osd.11.smithi114.stdout:Sep 21 03:44:46 smithi114 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-11[58313]: 2024-09-21T03:44:46.222+0000 7f7657ce9640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.114:6804 osd.2 since back 2024-09-21T03:43:37.224230+0000 front 2024-09-21T03:43:37.224148+0000 (oldest deadline 2024-09-21T03:43:58.924202+0000) 2024-09-21T03:44:46.712 INFO:journalctl@ceph.osd.11.smithi114.stdout:Sep 21 03:44:46 smithi114 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-11[58313]: 2024-09-21T03:44:46.222+0000 7f7657ce9640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.110:6812 osd.4 since back 2024-09-21T03:43:50.626287+0000 front 2024-09-21T03:43:50.626333+0000 (oldest deadline 2024-09-21T03:44:15.326365+0000) 2024-09-21T03:44:46.712 INFO:journalctl@ceph.osd.11.smithi114.stdout:Sep 21 03:44:46 smithi114 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-11[58313]: 2024-09-21T03:44:46.222+0000 7f7657ce9640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.114:6812 osd.5 since back 2024-09-21T03:44:00.627730+0000 front 2024-09-21T03:44:00.627753+0000 (oldest deadline 2024-09-21T03:44:26.527718+0000) 2024-09-21T03:44:46.712 INFO:journalctl@ceph.osd.11.smithi114.stdout:Sep 21 03:44:46 smithi114 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-11[58313]: 2024-09-21T03:44:46.222+0000 7f7657ce9640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.89:6822 osd.6 since back 2024-09-21T03:44:07.029032+0000 front 2024-09-21T03:44:07.029045+0000 (oldest deadline 2024-09-21T03:44:29.928840+0000) 2024-09-21T03:44:46.712 INFO:journalctl@ceph.osd.11.smithi114.stdout:Sep 21 03:44:46 smithi114 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-11[58313]: 2024-09-21T03:44:46.222+0000 7f7657ce9640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.110:6820 osd.7 since back 2024-09-21T03:44:16.830368+0000 front 2024-09-21T03:44:16.830322+0000 (oldest deadline 2024-09-21T03:44:39.130216+0000) 2024-09-21T03:44:47.461 INFO:journalctl@ceph.osd.11.smithi114.stdout:Sep 21 03:44:47 smithi114 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-11[58313]: 2024-09-21T03:44:47.184+0000 7f7657ce9640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2024-09-21T03:43:19.721069+0000 front 2024-09-21T03:43:19.721119+0000 (oldest deadline 2024-09-21T03:43:47.321877+0000) 2024-09-21T03:44:47.461 INFO:journalctl@ceph.osd.11.smithi114.stdout:Sep 21 03:44:47 smithi114 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-11[58313]: 2024-09-21T03:44:47.184+0000 7f7657ce9640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-21T03:43:27.322375+0000 front 2024-09-21T03:43:27.322426+0000 (oldest deadline 2024-09-21T03:43:51.422473+0000) 2024-09-21T03:44:47.461 INFO:journalctl@ceph.osd.11.smithi114.stdout:Sep 21 03:44:47 smithi114 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-11[58313]: 2024-09-21T03:44:47.184+0000 7f7657ce9640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.114:6804 osd.2 since back 2024-09-21T03:43:37.224230+0000 front 2024-09-21T03:43:37.224148+0000 (oldest deadline 2024-09-21T03:43:58.924202+0000) 2024-09-21T03:44:47.461 INFO:journalctl@ceph.osd.11.smithi114.stdout:Sep 21 03:44:47 smithi114 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-11[58313]: 2024-09-21T03:44:47.184+0000 7f7657ce9640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.110:6812 osd.4 since back 2024-09-21T03:43:50.626287+0000 front 2024-09-21T03:43:50.626333+0000 (oldest deadline 2024-09-21T03:44:15.326365+0000) 2024-09-21T03:44:47.461 INFO:journalctl@ceph.osd.11.smithi114.stdout:Sep 21 03:44:47 smithi114 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-11[58313]: 2024-09-21T03:44:47.184+0000 7f7657ce9640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.114:6812 osd.5 since back 2024-09-21T03:44:00.627730+0000 front 2024-09-21T03:44:00.627753+0000 (oldest deadline 2024-09-21T03:44:26.527718+0000) 2024-09-21T03:44:47.461 INFO:journalctl@ceph.osd.11.smithi114.stdout:Sep 21 03:44:47 smithi114 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-11[58313]: 2024-09-21T03:44:47.184+0000 7f7657ce9640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.89:6822 osd.6 since back 2024-09-21T03:44:07.029032+0000 front 2024-09-21T03:44:07.029045+0000 (oldest deadline 2024-09-21T03:44:29.928840+0000) 2024-09-21T03:44:47.461 INFO:journalctl@ceph.osd.11.smithi114.stdout:Sep 21 03:44:47 smithi114 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-11[58313]: 2024-09-21T03:44:47.184+0000 7f7657ce9640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.110:6820 osd.7 since back 2024-09-21T03:44:16.830368+0000 front 2024-09-21T03:44:16.830322+0000 (oldest deadline 2024-09-21T03:44:39.130216+0000) 2024-09-21T03:44:47.461 INFO:journalctl@ceph.osd.11.smithi114.stdout:Sep 21 03:44:47 smithi114 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-11[58313]: 2024-09-21T03:44:47.184+0000 7f7657ce9640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.114:6820 osd.8 since back 2024-09-21T03:44:20.830973+0000 front 2024-09-21T03:44:20.831009+0000 (oldest deadline 2024-09-21T03:44:46.731011+0000) 2024-09-21T03:44:48.461 INFO:journalctl@ceph.osd.11.smithi114.stdout:Sep 21 03:44:48 smithi114 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-11[58313]: 2024-09-21T03:44:48.148+0000 7f7657ce9640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2024-09-21T03:43:19.721069+0000 front 2024-09-21T03:43:19.721119+0000 (oldest deadline 2024-09-21T03:43:47.321877+0000) 2024-09-21T03:44:48.461 INFO:journalctl@ceph.osd.11.smithi114.stdout:Sep 21 03:44:48 smithi114 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-11[58313]: 2024-09-21T03:44:48.148+0000 7f7657ce9640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2024-09-21T03:43:27.322375+0000 front 2024-09-21T03:43:27.322426+0000 (oldest deadline 2024-09-21T03:43:51.422473+0000) 2024-09-21T03:44:48.461 INFO:journalctl@ceph.osd.11.smithi114.stdout:Sep 21 03:44:48 smithi114 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-11[58313]: 2024-09-21T03:44:48.148+0000 7f7657ce9640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.114:6804 osd.2 since back 2024-09-21T03:43:37.224230+0000 front 2024-09-21T03:43:37.224148+0000 (oldest deadline 2024-09-21T03:43:58.924202+0000) 2024-09-21T03:44:48.461 INFO:journalctl@ceph.osd.11.smithi114.stdout:Sep 21 03:44:48 smithi114 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-11[58313]: 2024-09-21T03:44:48.148+0000 7f7657ce9640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.110:6812 osd.4 since back 2024-09-21T03:43:50.626287+0000 front 2024-09-21T03:43:50.626333+0000 (oldest deadline 2024-09-21T03:44:15.326365+0000) 2024-09-21T03:44:48.461 INFO:journalctl@ceph.osd.11.smithi114.stdout:Sep 21 03:44:48 smithi114 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-11[58313]: 2024-09-21T03:44:48.148+0000 7f7657ce9640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.114:6812 osd.5 since back 2024-09-21T03:44:00.627730+0000 front 2024-09-21T03:44:00.627753+0000 (oldest deadline 2024-09-21T03:44:26.527718+0000) 2024-09-21T03:44:48.461 INFO:journalctl@ceph.osd.11.smithi114.stdout:Sep 21 03:44:48 smithi114 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-11[58313]: 2024-09-21T03:44:48.148+0000 7f7657ce9640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.89:6822 osd.6 since back 2024-09-21T03:44:07.029032+0000 front 2024-09-21T03:44:07.029045+0000 (oldest deadline 2024-09-21T03:44:29.928840+0000) 2024-09-21T03:44:48.461 INFO:journalctl@ceph.osd.11.smithi114.stdout:Sep 21 03:44:48 smithi114 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-11[58313]: 2024-09-21T03:44:48.148+0000 7f7657ce9640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.110:6820 osd.7 since back 2024-09-21T03:44:16.830368+0000 front 2024-09-21T03:44:16.830322+0000 (oldest deadline 2024-09-21T03:44:39.130216+0000) 2024-09-21T03:44:48.461 INFO:journalctl@ceph.osd.11.smithi114.stdout:Sep 21 03:44:48 smithi114 ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-11[58313]: 2024-09-21T03:44:48.148+0000 7f7657ce9640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.114:6820 osd.8 since back 2024-09-21T03:44:20.830973+0000 front 2024-09-21T03:44:20.831009+0000 (oldest deadline 2024-09-21T03:44:46.731011+0000) 2024-09-21T03:44:49.418 INFO:journalctl@ceph.osd.11.smithi114.stdout:Sep 21 03:44:49 smithi114 podman[66597]: 2024-09-21 03:44:49.120413387 +0000 UTC m=+5.621119231 container died 1c802259437b56c6cfd4a0824b64835e4d2fd4d1b8f10f4de0c4c0e8fb451fb2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-11, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, ceph=True, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, GIT_CLEAN=True) 2024-09-21T03:44:49.711 INFO:journalctl@ceph.osd.11.smithi114.stdout:Sep 21 03:44:49 smithi114 podman[66597]: 2024-09-21 03:44:49.490946437 +0000 UTC m=+5.991652274 container cleanup 1c802259437b56c6cfd4a0824b64835e4d2fd4d1b8f10f4de0c4c0e8fb451fb2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-11, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-21T03:44:49.711 INFO:journalctl@ceph.osd.11.smithi114.stdout:Sep 21 03:44:49 smithi114 bash[66597]: ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-11 2024-09-21T03:44:50.087 INFO:journalctl@ceph.osd.11.smithi114.stdout:Sep 21 03:44:49 smithi114 podman[66612]: 2024-09-21 03:44:49.759076765 +0000 UTC m=+0.639456265 container remove 1c802259437b56c6cfd4a0824b64835e4d2fd4d1b8f10f4de0c4c0e8fb451fb2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-11, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , ceph=True, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0) 2024-09-21T03:44:50.405 INFO:journalctl@ceph.osd.11.smithi114.stdout:Sep 21 03:44:50 smithi114 podman[66701]: 2024-09-21 03:44:50.001055922 +0000 UTC m=+0.016020260 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T03:44:50.405 INFO:journalctl@ceph.osd.11.smithi114.stdout:Sep 21 03:44:50 smithi114 podman[66701]: 2024-09-21 03:44:50.126877958 +0000 UTC m=+0.141842280 container create 5bc6e1c2474fac11b09fc844f62aedf747a97cc6a4c58edc1662713951b2161e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-11-deactivate, ceph=True, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731) 2024-09-21T03:44:50.405 INFO:journalctl@ceph.osd.11.smithi114.stdout:Sep 21 03:44:50 smithi114 podman[66701]: 2024-09-21 03:44:50.328899958 +0000 UTC m=+0.343864288 container init 5bc6e1c2474fac11b09fc844f62aedf747a97cc6a4c58edc1662713951b2161e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-11-deactivate, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-09-21T03:44:50.405 INFO:journalctl@ceph.osd.11.smithi114.stdout:Sep 21 03:44:50 smithi114 podman[66701]: 2024-09-21 03:44:50.332081511 +0000 UTC m=+0.347045834 container start 5bc6e1c2474fac11b09fc844f62aedf747a97cc6a4c58edc1662713951b2161e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-11-deactivate, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_CLEAN=True) 2024-09-21T03:44:50.405 INFO:journalctl@ceph.osd.11.smithi114.stdout:Sep 21 03:44:50 smithi114 podman[66701]: 2024-09-21 03:44:50.403864185 +0000 UTC m=+0.418828521 container attach 5bc6e1c2474fac11b09fc844f62aedf747a97cc6a4c58edc1662713951b2161e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-11-deactivate, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2) 2024-09-21T03:44:50.711 INFO:journalctl@ceph.osd.11.smithi114.stdout:Sep 21 03:44:50 smithi114 podman[66701]: 2024-09-21 03:44:50.52402414 +0000 UTC m=+0.538988462 container died 5bc6e1c2474fac11b09fc844f62aedf747a97cc6a4c58edc1662713951b2161e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-11-deactivate, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2) 2024-09-21T03:44:51.077 INFO:journalctl@ceph.osd.11.smithi114.stdout:Sep 21 03:44:51 smithi114 podman[66724]: 2024-09-21 03:44:51.057312661 +0000 UTC m=+0.524401013 container remove 5bc6e1c2474fac11b09fc844f62aedf747a97cc6a4c58edc1662713951b2161e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-7c374098-77c9-11ef-baf5-efdc52797490-osd-11-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0) 2024-09-21T03:44:51.082 DEBUG:teuthology.orchestra.run.smithi114:> sudo pkill -f 'journalctl -f -n 0 -u ceph-7c374098-77c9-11ef-baf5-efdc52797490@osd.11.service' 2024-09-21T03:44:51.119 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T03:44:51.120 INFO:tasks.cephadm.osd.11:Stopped osd.11 2024-09-21T03:44:51.120 INFO:tasks.cephadm.mds.a:Stopping mds.a... 2024-09-21T03:44:51.120 DEBUG:teuthology.orchestra.run.smithi089:> sudo systemctl stop ceph-7c374098-77c9-11ef-baf5-efdc52797490@mds.a 2024-09-21T03:44:51.427 INFO:journalctl@ceph.mds.a.smithi089.stdout:Sep 21 03:44:51 smithi089 systemd[1]: Stopping Ceph mds.a for 7c374098-77c9-11ef-baf5-efdc52797490... 2024-09-21T03:44:51.811 INFO:journalctl@ceph.mds.a.smithi089.stdout:Sep 21 03:44:51 smithi089 ceph-7c374098-77c9-11ef-baf5-efdc52797490-mds-a[71915]: 2024-09-21T03:44:51.425+0000 7f3034fda640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T03:44:51.811 INFO:journalctl@ceph.mds.a.smithi089.stdout:Sep 21 03:44:51 smithi089 ceph-7c374098-77c9-11ef-baf5-efdc52797490-mds-a[71915]: 2024-09-21T03:44:51.425+0000 7f3034fda640 -1 mds.a *** got signal Terminated *** 2024-09-21T03:44:53.311 INFO:journalctl@ceph.mds.a.smithi089.stdout:Sep 21 03:44:52 smithi089 podman[92482]: 2024-09-21 03:44:52.880507474 +0000 UTC m=+1.568646539 container died a9b21a569206b02a008e784f6e1fc614a9001b8fc489f59d33710c72db5018a0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-7c374098-77c9-11ef-baf5-efdc52797490-mds-a, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux ) 2024-09-21T03:44:53.697 INFO:journalctl@ceph.mds.a.smithi089.stdout:Sep 21 03:44:53 smithi089 podman[92482]: 2024-09-21 03:44:53.519538643 +0000 UTC m=+2.207677752 container cleanup a9b21a569206b02a008e784f6e1fc614a9001b8fc489f59d33710c72db5018a0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-7c374098-77c9-11ef-baf5-efdc52797490-mds-a, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.build-date=20240731) 2024-09-21T03:44:53.697 INFO:journalctl@ceph.mds.a.smithi089.stdout:Sep 21 03:44:53 smithi089 bash[92482]: ceph-7c374098-77c9-11ef-baf5-efdc52797490-mds-a 2024-09-21T03:44:53.941 DEBUG:teuthology.orchestra.run.smithi089:> sudo pkill -f 'journalctl -f -n 0 -u ceph-7c374098-77c9-11ef-baf5-efdc52797490@mds.a.service' 2024-09-21T03:44:53.963 INFO:journalctl@ceph.mds.a.smithi089.stdout:Sep 21 03:44:53 smithi089 podman[92494]: 2024-09-21 03:44:53.695902401 +0000 UTC m=+0.817590548 container remove a9b21a569206b02a008e784f6e1fc614a9001b8fc489f59d33710c72db5018a0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-7c374098-77c9-11ef-baf5-efdc52797490-mds-a, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-21T03:44:53.963 INFO:journalctl@ceph.mds.a.smithi089.stdout:Sep 21 03:44:53 smithi089 systemd[1]: ceph-7c374098-77c9-11ef-baf5-efdc52797490@mds.a.service: Deactivated successfully. 2024-09-21T03:44:53.963 INFO:journalctl@ceph.mds.a.smithi089.stdout:Sep 21 03:44:53 smithi089 systemd[1]: Stopped Ceph mds.a for 7c374098-77c9-11ef-baf5-efdc52797490. 2024-09-21T03:44:53.976 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T03:44:53.976 INFO:tasks.cephadm.mds.a:Stopped mds.a 2024-09-21T03:44:53.976 INFO:tasks.cephadm.mds.d:Stopping mds.d... 2024-09-21T03:44:53.976 DEBUG:teuthology.orchestra.run.smithi089:> sudo systemctl stop ceph-7c374098-77c9-11ef-baf5-efdc52797490@mds.d 2024-09-21T03:44:54.308 INFO:journalctl@ceph.mds.d.smithi089.stdout:Sep 21 03:44:54 smithi089 systemd[1]: Stopping Ceph mds.d for 7c374098-77c9-11ef-baf5-efdc52797490... 2024-09-21T03:44:54.561 INFO:journalctl@ceph.mds.d.smithi089.stdout:Sep 21 03:44:54 smithi089 ceph-7c374098-77c9-11ef-baf5-efdc52797490-mds-d[70792]: 2024-09-21T03:44:54.305+0000 7efe9ee0b640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.d -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T03:44:54.561 INFO:journalctl@ceph.mds.d.smithi089.stdout:Sep 21 03:44:54 smithi089 ceph-7c374098-77c9-11ef-baf5-efdc52797490-mds-d[70792]: 2024-09-21T03:44:54.305+0000 7efe9ee0b640 -1 mds.d *** got signal Terminated *** 2024-09-21T03:44:58.311 INFO:journalctl@ceph.mds.d.smithi089.stdout:Sep 21 03:44:57 smithi089 podman[92602]: 2024-09-21 03:44:57.989383512 +0000 UTC m=+3.798210589 container died a3f28b104710192d937a35d3a5314607b574cf53521d7404b7e71bd89d953933 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-7c374098-77c9-11ef-baf5-efdc52797490-mds-d, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-21T03:44:58.809 INFO:journalctl@ceph.mds.d.smithi089.stdout:Sep 21 03:44:58 smithi089 podman[92602]: 2024-09-21 03:44:58.557030835 +0000 UTC m=+4.365857909 container cleanup a3f28b104710192d937a35d3a5314607b574cf53521d7404b7e71bd89d953933 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-7c374098-77c9-11ef-baf5-efdc52797490-mds-d, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-09-21T03:44:58.810 INFO:journalctl@ceph.mds.d.smithi089.stdout:Sep 21 03:44:58 smithi089 bash[92602]: ceph-7c374098-77c9-11ef-baf5-efdc52797490-mds-d 2024-09-21T03:44:58.988 DEBUG:teuthology.orchestra.run.smithi089:> sudo pkill -f 'journalctl -f -n 0 -u ceph-7c374098-77c9-11ef-baf5-efdc52797490@mds.d.service' 2024-09-21T03:44:59.023 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T03:44:59.023 INFO:tasks.cephadm.mds.d:Stopped mds.d 2024-09-21T03:44:59.023 INFO:tasks.cephadm.mds.g:Stopping mds.g... 2024-09-21T03:44:59.023 DEBUG:teuthology.orchestra.run.smithi089:> sudo systemctl stop ceph-7c374098-77c9-11ef-baf5-efdc52797490@mds.g 2024-09-21T03:44:59.347 INFO:journalctl@ceph.mds.g.smithi089.stdout:Sep 21 03:44:59 smithi089 systemd[1]: Stopping Ceph mds.g for 7c374098-77c9-11ef-baf5-efdc52797490... 2024-09-21T03:44:59.811 INFO:journalctl@ceph.mds.g.smithi089.stdout:Sep 21 03:44:59 smithi089 ceph-7c374098-77c9-11ef-baf5-efdc52797490-mds-g[71550]: 2024-09-21T03:44:59.344+0000 7ff3081b2640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.g -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T03:44:59.811 INFO:journalctl@ceph.mds.g.smithi089.stdout:Sep 21 03:44:59 smithi089 ceph-7c374098-77c9-11ef-baf5-efdc52797490-mds-g[71550]: 2024-09-21T03:44:59.344+0000 7ff3081b2640 -1 mds.g *** got signal Terminated *** 2024-09-21T03:45:01.697 INFO:journalctl@ceph.mds.g.smithi089.stdout:Sep 21 03:45:01 smithi089 podman[92737]: 2024-09-21 03:45:01.424096974 +0000 UTC m=+2.182799986 container died bdceadcf48c0f07544364d3e72d42a1271c87f0660dc7bd28a9310e2e2cb1573 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-7c374098-77c9-11ef-baf5-efdc52797490-mds-g, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True) 2024-09-21T03:45:02.048 INFO:journalctl@ceph.mds.g.smithi089.stdout:Sep 21 03:45:01 smithi089 podman[92737]: 2024-09-21 03:45:01.844733175 +0000 UTC m=+2.603436187 container cleanup bdceadcf48c0f07544364d3e72d42a1271c87f0660dc7bd28a9310e2e2cb1573 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-7c374098-77c9-11ef-baf5-efdc52797490-mds-g, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux ) 2024-09-21T03:45:02.048 INFO:journalctl@ceph.mds.g.smithi089.stdout:Sep 21 03:45:01 smithi089 bash[92737]: ceph-7c374098-77c9-11ef-baf5-efdc52797490-mds-g 2024-09-21T03:45:02.268 DEBUG:teuthology.orchestra.run.smithi089:> sudo pkill -f 'journalctl -f -n 0 -u ceph-7c374098-77c9-11ef-baf5-efdc52797490@mds.g.service' 2024-09-21T03:45:02.301 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T03:45:02.301 INFO:tasks.cephadm.mds.g:Stopped mds.g 2024-09-21T03:45:02.301 INFO:tasks.cephadm.mds.j:Stopping mds.j... 2024-09-21T03:45:02.301 DEBUG:teuthology.orchestra.run.smithi089:> sudo systemctl stop ceph-7c374098-77c9-11ef-baf5-efdc52797490@mds.j 2024-09-21T03:45:02.561 INFO:journalctl@ceph.mds.j.smithi089.stdout:Sep 21 03:45:02 smithi089 systemd[1]: Stopping Ceph mds.j for 7c374098-77c9-11ef-baf5-efdc52797490... 2024-09-21T03:45:03.061 INFO:journalctl@ceph.mds.j.smithi089.stdout:Sep 21 03:45:02 smithi089 ceph-7c374098-77c9-11ef-baf5-efdc52797490-mds-j[70422]: 2024-09-21T03:45:02.648+0000 7f2d5003e640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.j -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T03:45:03.061 INFO:journalctl@ceph.mds.j.smithi089.stdout:Sep 21 03:45:02 smithi089 ceph-7c374098-77c9-11ef-baf5-efdc52797490-mds-j[70422]: 2024-09-21T03:45:02.648+0000 7f2d5003e640 -1 mds.j *** got signal Terminated *** 2024-09-21T03:45:06.776 INFO:journalctl@ceph.mds.j.smithi089.stdout:Sep 21 03:45:06 smithi089 podman[92858]: 2024-09-21 03:45:06.529312363 +0000 UTC m=+4.003648193 container died 67a53e1dba194e1cd9fd98383e31a90d58308856dd632848b46e0a8466e678c6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-7c374098-77c9-11ef-baf5-efdc52797490-mds-j, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-21T03:45:07.043 INFO:journalctl@ceph.mds.j.smithi089.stdout:Sep 21 03:45:06 smithi089 podman[92858]: 2024-09-21 03:45:06.865495346 +0000 UTC m=+4.339831178 container cleanup 67a53e1dba194e1cd9fd98383e31a90d58308856dd632848b46e0a8466e678c6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-7c374098-77c9-11ef-baf5-efdc52797490-mds-j, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, org.label-schema.build-date=20240731, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-09-21T03:45:07.043 INFO:journalctl@ceph.mds.j.smithi089.stdout:Sep 21 03:45:06 smithi089 bash[92858]: ceph-7c374098-77c9-11ef-baf5-efdc52797490-mds-j 2024-09-21T03:45:07.285 DEBUG:teuthology.orchestra.run.smithi089:> sudo pkill -f 'journalctl -f -n 0 -u ceph-7c374098-77c9-11ef-baf5-efdc52797490@mds.j.service' 2024-09-21T03:45:07.306 INFO:journalctl@ceph.mds.j.smithi089.stdout:Sep 21 03:45:07 smithi089 podman[92870]: 2024-09-21 03:45:07.041531412 +0000 UTC m=+0.511677172 container remove 67a53e1dba194e1cd9fd98383e31a90d58308856dd632848b46e0a8466e678c6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-7c374098-77c9-11ef-baf5-efdc52797490-mds-j, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731) 2024-09-21T03:45:07.306 INFO:journalctl@ceph.mds.j.smithi089.stdout:Sep 21 03:45:07 smithi089 systemd[1]: ceph-7c374098-77c9-11ef-baf5-efdc52797490@mds.j.service: Deactivated successfully. 2024-09-21T03:45:07.307 INFO:journalctl@ceph.mds.j.smithi089.stdout:Sep 21 03:45:07 smithi089 systemd[1]: Stopped Ceph mds.j for 7c374098-77c9-11ef-baf5-efdc52797490. 2024-09-21T03:45:07.318 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T03:45:07.318 INFO:tasks.cephadm.mds.j:Stopped mds.j 2024-09-21T03:45:07.318 INFO:tasks.cephadm.mds.b:Stopping mds.b... 2024-09-21T03:45:07.318 DEBUG:teuthology.orchestra.run.smithi110:> sudo systemctl stop ceph-7c374098-77c9-11ef-baf5-efdc52797490@mds.b 2024-09-21T03:45:07.615 INFO:journalctl@ceph.mds.b.smithi110.stdout:Sep 21 03:45:07 smithi110 systemd[1]: Stopping Ceph mds.b for 7c374098-77c9-11ef-baf5-efdc52797490... 2024-09-21T03:45:08.000 INFO:journalctl@ceph.mds.b.smithi110.stdout:Sep 21 03:45:07 smithi110 ceph-7c374098-77c9-11ef-baf5-efdc52797490-mds-b[62090]: 2024-09-21T03:45:07.612+0000 7fb204658640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T03:45:08.000 INFO:journalctl@ceph.mds.b.smithi110.stdout:Sep 21 03:45:07 smithi110 ceph-7c374098-77c9-11ef-baf5-efdc52797490-mds-b[62090]: 2024-09-21T03:45:07.612+0000 7fb204658640 -1 mds.b *** got signal Terminated *** 2024-09-21T03:45:11.500 INFO:journalctl@ceph.mds.b.smithi110.stdout:Sep 21 03:45:11 smithi110 podman[65595]: 2024-09-21 03:45:11.223575954 +0000 UTC m=+3.698400002 container died 469eb07a1cfbb185c9852eb5cbec18f7b08cd2235178d172647d0ccdd7dc8616 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-7c374098-77c9-11ef-baf5-efdc52797490-mds-b, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.build-date=20240731, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.license=GPLv2) 2024-09-21T03:45:11.856 INFO:journalctl@ceph.mds.b.smithi110.stdout:Sep 21 03:45:11 smithi110 podman[65595]: 2024-09-21 03:45:11.653811818 +0000 UTC m=+4.128635856 container cleanup 469eb07a1cfbb185c9852eb5cbec18f7b08cd2235178d172647d0ccdd7dc8616 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-7c374098-77c9-11ef-baf5-efdc52797490-mds-b, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.build-date=20240731, ceph=True, maintainer=Guillaume Abrioux ) 2024-09-21T03:45:11.856 INFO:journalctl@ceph.mds.b.smithi110.stdout:Sep 21 03:45:11 smithi110 bash[65595]: ceph-7c374098-77c9-11ef-baf5-efdc52797490-mds-b 2024-09-21T03:45:12.071 DEBUG:teuthology.orchestra.run.smithi110:> sudo pkill -f 'journalctl -f -n 0 -u ceph-7c374098-77c9-11ef-baf5-efdc52797490@mds.b.service' 2024-09-21T03:45:12.105 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T03:45:12.106 INFO:tasks.cephadm.mds.b:Stopped mds.b 2024-09-21T03:45:12.106 INFO:tasks.cephadm.mds.e:Stopping mds.e... 2024-09-21T03:45:12.106 DEBUG:teuthology.orchestra.run.smithi110:> sudo systemctl stop ceph-7c374098-77c9-11ef-baf5-efdc52797490@mds.e 2024-09-21T03:45:12.426 INFO:journalctl@ceph.mds.e.smithi110.stdout:Sep 21 03:45:12 smithi110 systemd[1]: Stopping Ceph mds.e for 7c374098-77c9-11ef-baf5-efdc52797490... 2024-09-21T03:45:12.750 INFO:journalctl@ceph.mds.e.smithi110.stdout:Sep 21 03:45:12 smithi110 ceph-7c374098-77c9-11ef-baf5-efdc52797490-mds-e[61549]: 2024-09-21T03:45:12.424+0000 7f4ca23ab640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.e -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T03:45:12.750 INFO:journalctl@ceph.mds.e.smithi110.stdout:Sep 21 03:45:12 smithi110 ceph-7c374098-77c9-11ef-baf5-efdc52797490-mds-e[61549]: 2024-09-21T03:45:12.424+0000 7f4ca23ab640 -1 mds.e *** got signal Terminated *** 2024-09-21T03:45:15.500 INFO:journalctl@ceph.mds.e.smithi110.stdout:Sep 21 03:45:15 smithi110 podman[65718]: 2024-09-21 03:45:15.229741772 +0000 UTC m=+2.903554387 container died 368b2198fd454390a5266bdd0ef8475d40e70908480e3168995e79d8877c3ea5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-7c374098-77c9-11ef-baf5-efdc52797490-mds-e, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.vendor=CentOS, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, GIT_CLEAN=True) 2024-09-21T03:45:16.044 INFO:journalctl@ceph.mds.e.smithi110.stdout:Sep 21 03:45:15 smithi110 podman[65718]: 2024-09-21 03:45:15.858337204 +0000 UTC m=+3.532149819 container cleanup 368b2198fd454390a5266bdd0ef8475d40e70908480e3168995e79d8877c3ea5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-7c374098-77c9-11ef-baf5-efdc52797490-mds-e, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_CLEAN=True, io.buildah.version=1.37.2, ceph=True, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.vendor=CentOS) 2024-09-21T03:45:16.044 INFO:journalctl@ceph.mds.e.smithi110.stdout:Sep 21 03:45:15 smithi110 bash[65718]: ceph-7c374098-77c9-11ef-baf5-efdc52797490-mds-e 2024-09-21T03:45:16.290 DEBUG:teuthology.orchestra.run.smithi110:> sudo pkill -f 'journalctl -f -n 0 -u ceph-7c374098-77c9-11ef-baf5-efdc52797490@mds.e.service' 2024-09-21T03:45:16.311 INFO:journalctl@ceph.mds.e.smithi110.stdout:Sep 21 03:45:16 smithi110 podman[65729]: 2024-09-21 03:45:16.042631566 +0000 UTC m=+0.808613298 container remove 368b2198fd454390a5266bdd0ef8475d40e70908480e3168995e79d8877c3ea5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-7c374098-77c9-11ef-baf5-efdc52797490-mds-e, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux ) 2024-09-21T03:45:16.311 INFO:journalctl@ceph.mds.e.smithi110.stdout:Sep 21 03:45:16 smithi110 systemd[1]: ceph-7c374098-77c9-11ef-baf5-efdc52797490@mds.e.service: Deactivated successfully. 2024-09-21T03:45:16.311 INFO:journalctl@ceph.mds.e.smithi110.stdout:Sep 21 03:45:16 smithi110 systemd[1]: Stopped Ceph mds.e for 7c374098-77c9-11ef-baf5-efdc52797490. 2024-09-21T03:45:16.311 INFO:journalctl@ceph.mds.e.smithi110.stdout:Sep 21 03:45:16 smithi110 systemd[1]: ceph-7c374098-77c9-11ef-baf5-efdc52797490@mds.e.service: Consumed 1.497s CPU time. 2024-09-21T03:45:16.323 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T03:45:16.323 INFO:tasks.cephadm.mds.e:Stopped mds.e 2024-09-21T03:45:16.323 INFO:tasks.cephadm.mds.h:Stopping mds.h... 2024-09-21T03:45:16.323 DEBUG:teuthology.orchestra.run.smithi110:> sudo systemctl stop ceph-7c374098-77c9-11ef-baf5-efdc52797490@mds.h 2024-09-21T03:45:16.638 INFO:journalctl@ceph.mds.h.smithi110.stdout:Sep 21 03:45:16 smithi110 systemd[1]: Stopping Ceph mds.h for 7c374098-77c9-11ef-baf5-efdc52797490... 2024-09-21T03:45:17.000 INFO:journalctl@ceph.mds.h.smithi110.stdout:Sep 21 03:45:16 smithi110 ceph-7c374098-77c9-11ef-baf5-efdc52797490-mds-h[61819]: 2024-09-21T03:45:16.635+0000 7f98fabb1640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.h -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T03:45:17.000 INFO:journalctl@ceph.mds.h.smithi110.stdout:Sep 21 03:45:16 smithi110 ceph-7c374098-77c9-11ef-baf5-efdc52797490-mds-h[61819]: 2024-09-21T03:45:16.635+0000 7f98fabb1640 -1 mds.h *** got signal Terminated *** 2024-09-21T03:45:19.500 INFO:journalctl@ceph.mds.h.smithi110.stdout:Sep 21 03:45:19 smithi110 podman[65838]: 2024-09-21 03:45:19.227089879 +0000 UTC m=+2.685769899 container died 9ea35882772d0e1274196d02245f31d253ca3349e4fa33822481dfc227df39c0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-7c374098-77c9-11ef-baf5-efdc52797490-mds-h, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_CLEAN=True) 2024-09-21T03:45:19.954 INFO:journalctl@ceph.mds.h.smithi110.stdout:Sep 21 03:45:19 smithi110 podman[65838]: 2024-09-21 03:45:19.544943365 +0000 UTC m=+3.003623382 container cleanup 9ea35882772d0e1274196d02245f31d253ca3349e4fa33822481dfc227df39c0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-7c374098-77c9-11ef-baf5-efdc52797490-mds-h, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0) 2024-09-21T03:45:19.954 INFO:journalctl@ceph.mds.h.smithi110.stdout:Sep 21 03:45:19 smithi110 bash[65838]: ceph-7c374098-77c9-11ef-baf5-efdc52797490-mds-h 2024-09-21T03:45:19.954 INFO:journalctl@ceph.mds.h.smithi110.stdout:Sep 21 03:45:19 smithi110 podman[65850]: 2024-09-21 03:45:19.74609757 +0000 UTC m=+0.511999230 container remove 9ea35882772d0e1274196d02245f31d253ca3349e4fa33822481dfc227df39c0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-7c374098-77c9-11ef-baf5-efdc52797490-mds-h, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-21T03:45:19.976 DEBUG:teuthology.orchestra.run.smithi110:> sudo pkill -f 'journalctl -f -n 0 -u ceph-7c374098-77c9-11ef-baf5-efdc52797490@mds.h.service' 2024-09-21T03:45:20.008 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T03:45:20.008 INFO:tasks.cephadm.mds.h:Stopped mds.h 2024-09-21T03:45:20.008 INFO:tasks.cephadm.mds.k:Stopping mds.k... 2024-09-21T03:45:20.009 DEBUG:teuthology.orchestra.run.smithi110:> sudo systemctl stop ceph-7c374098-77c9-11ef-baf5-efdc52797490@mds.k 2024-09-21T03:45:20.250 INFO:journalctl@ceph.mds.k.smithi110.stdout:Sep 21 03:45:20 smithi110 systemd[1]: Stopping Ceph mds.k for 7c374098-77c9-11ef-baf5-efdc52797490... 2024-09-21T03:45:20.750 INFO:journalctl@ceph.mds.k.smithi110.stdout:Sep 21 03:45:20 smithi110 ceph-7c374098-77c9-11ef-baf5-efdc52797490-mds-k[61279]: 2024-09-21T03:45:20.339+0000 7f7669ee2640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.k -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T03:45:20.750 INFO:journalctl@ceph.mds.k.smithi110.stdout:Sep 21 03:45:20 smithi110 ceph-7c374098-77c9-11ef-baf5-efdc52797490-mds-k[61279]: 2024-09-21T03:45:20.339+0000 7f7669ee2640 -1 mds.k *** got signal Terminated *** 2024-09-21T03:45:23.642 INFO:journalctl@ceph.mds.k.smithi110.stdout:Sep 21 03:45:23 smithi110 podman[65959]: 2024-09-21 03:45:23.361719522 +0000 UTC m=+3.127790115 container died c0c7a27c8983c1475a15f7e3d2c47bf505f2b2a4808e0896bd0287331b6094fd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-7c374098-77c9-11ef-baf5-efdc52797490-mds-k, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-09-21T03:45:23.984 INFO:journalctl@ceph.mds.k.smithi110.stdout:Sep 21 03:45:23 smithi110 podman[65959]: 2024-09-21 03:45:23.723655364 +0000 UTC m=+3.489725957 container cleanup c0c7a27c8983c1475a15f7e3d2c47bf505f2b2a4808e0896bd0287331b6094fd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-7c374098-77c9-11ef-baf5-efdc52797490-mds-k, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, ceph=True, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True, CEPH_POINT_RELEASE=) 2024-09-21T03:45:23.984 INFO:journalctl@ceph.mds.k.smithi110.stdout:Sep 21 03:45:23 smithi110 bash[65959]: ceph-7c374098-77c9-11ef-baf5-efdc52797490-mds-k 2024-09-21T03:45:24.159 DEBUG:teuthology.orchestra.run.smithi110:> sudo pkill -f 'journalctl -f -n 0 -u ceph-7c374098-77c9-11ef-baf5-efdc52797490@mds.k.service' 2024-09-21T03:45:24.190 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T03:45:24.190 INFO:tasks.cephadm.mds.k:Stopped mds.k 2024-09-21T03:45:24.190 INFO:tasks.cephadm.mds.c:Stopping mds.c... 2024-09-21T03:45:24.190 DEBUG:teuthology.orchestra.run.smithi114:> sudo systemctl stop ceph-7c374098-77c9-11ef-baf5-efdc52797490@mds.c 2024-09-21T03:45:24.500 INFO:journalctl@ceph.mds.c.smithi114.stdout:Sep 21 03:45:24 smithi114 systemd[1]: Stopping Ceph mds.c for 7c374098-77c9-11ef-baf5-efdc52797490... 2024-09-21T03:45:24.766 INFO:journalctl@ceph.mds.c.smithi114.stdout:Sep 21 03:45:24 smithi114 ceph-7c374098-77c9-11ef-baf5-efdc52797490-mds-c[62970]: 2024-09-21T03:45:24.498+0000 7fcdced60640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T03:45:24.766 INFO:journalctl@ceph.mds.c.smithi114.stdout:Sep 21 03:45:24 smithi114 ceph-7c374098-77c9-11ef-baf5-efdc52797490-mds-c[62970]: 2024-09-21T03:45:24.498+0000 7fcdced60640 -1 mds.c *** got signal Terminated *** 2024-09-21T03:45:27.533 INFO:journalctl@ceph.mds.c.smithi114.stdout:Sep 21 03:45:27 smithi114 podman[66805]: 2024-09-21 03:45:27.231507523 +0000 UTC m=+2.847736087 container died bf3e0de614f42bd43da8e4e74d065a0be8fe272dd6cc6920e39a48b8abeb5749 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-7c374098-77c9-11ef-baf5-efdc52797490-mds-c, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, ceph=True, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-21T03:45:27.842 INFO:journalctl@ceph.mds.c.smithi114.stdout:Sep 21 03:45:27 smithi114 podman[66805]: 2024-09-21 03:45:27.614861816 +0000 UTC m=+3.231090381 container cleanup bf3e0de614f42bd43da8e4e74d065a0be8fe272dd6cc6920e39a48b8abeb5749 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-7c374098-77c9-11ef-baf5-efdc52797490-mds-c, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD) 2024-09-21T03:45:27.842 INFO:journalctl@ceph.mds.c.smithi114.stdout:Sep 21 03:45:27 smithi114 bash[66805]: ceph-7c374098-77c9-11ef-baf5-efdc52797490-mds-c 2024-09-21T03:45:27.973 DEBUG:teuthology.orchestra.run.smithi114:> sudo pkill -f 'journalctl -f -n 0 -u ceph-7c374098-77c9-11ef-baf5-efdc52797490@mds.c.service' 2024-09-21T03:45:28.009 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T03:45:28.010 INFO:tasks.cephadm.mds.c:Stopped mds.c 2024-09-21T03:45:28.010 INFO:tasks.cephadm.mds.f:Stopping mds.f... 2024-09-21T03:45:28.010 DEBUG:teuthology.orchestra.run.smithi114:> sudo systemctl stop ceph-7c374098-77c9-11ef-baf5-efdc52797490@mds.f 2024-09-21T03:45:28.461 INFO:journalctl@ceph.mds.f.smithi114.stdout:Sep 21 03:45:28 smithi114 systemd[1]: Stopping Ceph mds.f for 7c374098-77c9-11ef-baf5-efdc52797490... 2024-09-21T03:45:28.461 INFO:journalctl@ceph.mds.f.smithi114.stdout:Sep 21 03:45:28 smithi114 ceph-7c374098-77c9-11ef-baf5-efdc52797490-mds-f[62699]: 2024-09-21T03:45:28.310+0000 7f5f7fce7640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.f -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T03:45:28.461 INFO:journalctl@ceph.mds.f.smithi114.stdout:Sep 21 03:45:28 smithi114 ceph-7c374098-77c9-11ef-baf5-efdc52797490-mds-f[62699]: 2024-09-21T03:45:28.310+0000 7f5f7fce7640 -1 mds.f *** got signal Terminated *** 2024-09-21T03:45:31.529 INFO:journalctl@ceph.mds.f.smithi114.stdout:Sep 21 03:45:31 smithi114 podman[66934]: 2024-09-21 03:45:31.228828897 +0000 UTC m=+2.996980114 container died 843d5ceede5f3b8dfbcd705e6493553441aa76f90017c5511d1181aab6349a67 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-7c374098-77c9-11ef-baf5-efdc52797490-mds-f, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-09-21T03:45:31.863 INFO:journalctl@ceph.mds.f.smithi114.stdout:Sep 21 03:45:31 smithi114 podman[66934]: 2024-09-21 03:45:31.627226613 +0000 UTC m=+3.395377818 container cleanup 843d5ceede5f3b8dfbcd705e6493553441aa76f90017c5511d1181aab6349a67 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-7c374098-77c9-11ef-baf5-efdc52797490-mds-f, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731) 2024-09-21T03:45:31.863 INFO:journalctl@ceph.mds.f.smithi114.stdout:Sep 21 03:45:31 smithi114 bash[66934]: ceph-7c374098-77c9-11ef-baf5-efdc52797490-mds-f 2024-09-21T03:45:32.009 DEBUG:teuthology.orchestra.run.smithi114:> sudo pkill -f 'journalctl -f -n 0 -u ceph-7c374098-77c9-11ef-baf5-efdc52797490@mds.f.service' 2024-09-21T03:45:32.042 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T03:45:32.042 INFO:tasks.cephadm.mds.f:Stopped mds.f 2024-09-21T03:45:32.043 INFO:tasks.cephadm.mds.i:Stopping mds.i... 2024-09-21T03:45:32.043 DEBUG:teuthology.orchestra.run.smithi114:> sudo systemctl stop ceph-7c374098-77c9-11ef-baf5-efdc52797490@mds.i 2024-09-21T03:45:32.461 INFO:journalctl@ceph.mds.i.smithi114.stdout:Sep 21 03:45:32 smithi114 systemd[1]: Stopping Ceph mds.i for 7c374098-77c9-11ef-baf5-efdc52797490... 2024-09-21T03:45:32.461 INFO:journalctl@ceph.mds.i.smithi114.stdout:Sep 21 03:45:32 smithi114 ceph-7c374098-77c9-11ef-baf5-efdc52797490-mds-i[63238]: 2024-09-21T03:45:32.355+0000 7f1571f3a640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.i -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T03:45:32.461 INFO:journalctl@ceph.mds.i.smithi114.stdout:Sep 21 03:45:32 smithi114 ceph-7c374098-77c9-11ef-baf5-efdc52797490-mds-i[63238]: 2024-09-21T03:45:32.355+0000 7f1571f3a640 -1 mds.i *** got signal Terminated *** 2024-09-21T03:45:35.532 INFO:journalctl@ceph.mds.i.smithi114.stdout:Sep 21 03:45:35 smithi114 podman[67054]: 2024-09-21 03:45:35.222714119 +0000 UTC m=+2.968280945 container died 16eb77ff66450689697786890a8004a04670cc57d9d023916b0728df9c6bb951 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-7c374098-77c9-11ef-baf5-efdc52797490-mds-i, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-21T03:45:35.795 INFO:journalctl@ceph.mds.i.smithi114.stdout:Sep 21 03:45:35 smithi114 podman[67054]: 2024-09-21 03:45:35.622484138 +0000 UTC m=+3.368051010 container cleanup 16eb77ff66450689697786890a8004a04670cc57d9d023916b0728df9c6bb951 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-7c374098-77c9-11ef-baf5-efdc52797490-mds-i, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , ceph=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS) 2024-09-21T03:45:35.795 INFO:journalctl@ceph.mds.i.smithi114.stdout:Sep 21 03:45:35 smithi114 bash[67054]: ceph-7c374098-77c9-11ef-baf5-efdc52797490-mds-i 2024-09-21T03:45:35.965 DEBUG:teuthology.orchestra.run.smithi114:> sudo pkill -f 'journalctl -f -n 0 -u ceph-7c374098-77c9-11ef-baf5-efdc52797490@mds.i.service' 2024-09-21T03:45:35.998 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T03:45:35.998 INFO:tasks.cephadm.mds.i:Stopped mds.i 2024-09-21T03:45:35.999 INFO:tasks.cephadm.mds.l:Stopping mds.l... 2024-09-21T03:45:35.999 DEBUG:teuthology.orchestra.run.smithi114:> sudo systemctl stop ceph-7c374098-77c9-11ef-baf5-efdc52797490@mds.l 2024-09-21T03:45:36.328 INFO:journalctl@ceph.mds.l.smithi114.stdout:Sep 21 03:45:36 smithi114 systemd[1]: Stopping Ceph mds.l for 7c374098-77c9-11ef-baf5-efdc52797490... 2024-09-21T03:45:36.711 INFO:journalctl@ceph.mds.l.smithi114.stdout:Sep 21 03:45:36 smithi114 ceph-7c374098-77c9-11ef-baf5-efdc52797490-mds-l[62431]: 2024-09-21T03:45:36.326+0000 7f2353ace640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.l -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T03:45:36.711 INFO:journalctl@ceph.mds.l.smithi114.stdout:Sep 21 03:45:36 smithi114 ceph-7c374098-77c9-11ef-baf5-efdc52797490-mds-l[62431]: 2024-09-21T03:45:36.326+0000 7f2353ace640 -1 mds.l *** got signal Terminated *** 2024-09-21T03:45:40.711 INFO:journalctl@ceph.mds.l.smithi114.stdout:Sep 21 03:45:40 smithi114 podman[67174]: 2024-09-21 03:45:40.331827926 +0000 UTC m=+4.110858289 container died 6dbceef3df18ca4ec88a90127f37432ede0905d07c3c2fdade2d7015a094fb78 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-7c374098-77c9-11ef-baf5-efdc52797490-mds-l, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221) 2024-09-21T03:45:41.070 INFO:journalctl@ceph.mds.l.smithi114.stdout:Sep 21 03:45:40 smithi114 podman[67174]: 2024-09-21 03:45:40.792937604 +0000 UTC m=+4.571967957 container cleanup 6dbceef3df18ca4ec88a90127f37432ede0905d07c3c2fdade2d7015a094fb78 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-7c374098-77c9-11ef-baf5-efdc52797490-mds-l, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_CLEAN=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2) 2024-09-21T03:45:41.070 INFO:journalctl@ceph.mds.l.smithi114.stdout:Sep 21 03:45:40 smithi114 bash[67174]: ceph-7c374098-77c9-11ef-baf5-efdc52797490-mds-l 2024-09-21T03:45:41.211 DEBUG:teuthology.orchestra.run.smithi114:> sudo pkill -f 'journalctl -f -n 0 -u ceph-7c374098-77c9-11ef-baf5-efdc52797490@mds.l.service' 2024-09-21T03:45:41.243 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T03:45:41.244 INFO:tasks.cephadm.mds.l:Stopped mds.l 2024-09-21T03:45:41.244 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 7c374098-77c9-11ef-baf5-efdc52797490 --force --keep-logs 2024-09-21T03:45:41.445 INFO:teuthology.orchestra.run.smithi089.stdout:Deleting cluster with fsid: 7c374098-77c9-11ef-baf5-efdc52797490 2024-09-21T03:45:43.961 DEBUG:teuthology.orchestra.run.smithi110:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 7c374098-77c9-11ef-baf5-efdc52797490 --force --keep-logs 2024-09-21T03:45:44.160 INFO:teuthology.orchestra.run.smithi110.stdout:Deleting cluster with fsid: 7c374098-77c9-11ef-baf5-efdc52797490 2024-09-21T03:45:46.521 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 7c374098-77c9-11ef-baf5-efdc52797490 --force --keep-logs 2024-09-21T03:45:46.724 INFO:teuthology.orchestra.run.smithi114.stdout:Deleting cluster with fsid: 7c374098-77c9-11ef-baf5-efdc52797490 2024-09-21T03:45:49.076 DEBUG:teuthology.orchestra.run.smithi089:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-21T03:45:49.110 DEBUG:teuthology.orchestra.run.smithi110:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-21T03:45:49.138 DEBUG:teuthology.orchestra.run.smithi114:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-21T03:45:49.164 INFO:tasks.cephadm:Archiving crash dumps... 2024-09-21T03:45:49.167 DEBUG:teuthology.misc:Transferring archived files from smithi089:/var/lib/ceph/7c374098-77c9-11ef-baf5-efdc52797490/crash to /home/teuthworker/archive/pdonnell-2024-09-20_23:48:22-fs-wip-pdonnell-testing-20240920.212106-debug-distro-default-smithi/7913466/remote/smithi089/crash 2024-09-21T03:45:49.168 DEBUG:teuthology.orchestra.run.smithi089:> sudo tar c -f - -C /var/lib/ceph/7c374098-77c9-11ef-baf5-efdc52797490/crash -- . 2024-09-21T03:45:49.208 INFO:teuthology.orchestra.run.smithi089.stderr:tar: /var/lib/ceph/7c374098-77c9-11ef-baf5-efdc52797490/crash: Cannot open: No such file or directory 2024-09-21T03:45:49.208 INFO:teuthology.orchestra.run.smithi089.stderr:tar: Error is not recoverable: exiting now 2024-09-21T03:45:49.210 DEBUG:teuthology.misc:Transferring archived files from smithi110:/var/lib/ceph/7c374098-77c9-11ef-baf5-efdc52797490/crash to /home/teuthworker/archive/pdonnell-2024-09-20_23:48:22-fs-wip-pdonnell-testing-20240920.212106-debug-distro-default-smithi/7913466/remote/smithi110/crash 2024-09-21T03:45:49.211 DEBUG:teuthology.orchestra.run.smithi110:> sudo tar c -f - -C /var/lib/ceph/7c374098-77c9-11ef-baf5-efdc52797490/crash -- . 2024-09-21T03:45:49.252 INFO:teuthology.orchestra.run.smithi110.stderr:tar: /var/lib/ceph/7c374098-77c9-11ef-baf5-efdc52797490/crash: Cannot open: No such file or directory 2024-09-21T03:45:49.252 INFO:teuthology.orchestra.run.smithi110.stderr:tar: Error is not recoverable: exiting now 2024-09-21T03:45:49.254 DEBUG:teuthology.misc:Transferring archived files from smithi114:/var/lib/ceph/7c374098-77c9-11ef-baf5-efdc52797490/crash to /home/teuthworker/archive/pdonnell-2024-09-20_23:48:22-fs-wip-pdonnell-testing-20240920.212106-debug-distro-default-smithi/7913466/remote/smithi114/crash 2024-09-21T03:45:49.255 DEBUG:teuthology.orchestra.run.smithi114:> sudo tar c -f - -C /var/lib/ceph/7c374098-77c9-11ef-baf5-efdc52797490/crash -- . 2024-09-21T03:45:49.287 INFO:teuthology.orchestra.run.smithi114.stderr:tar: /var/lib/ceph/7c374098-77c9-11ef-baf5-efdc52797490/crash: Cannot open: No such file or directory 2024-09-21T03:45:49.287 INFO:teuthology.orchestra.run.smithi114.stderr:tar: Error is not recoverable: exiting now 2024-09-21T03:45:49.288 INFO:tasks.cephadm:Checking cluster log for badness... 2024-09-21T03:45:49.289 DEBUG:teuthology.orchestra.run.smithi089:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/7c374098-77c9-11ef-baf5-efdc52797490/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v 'fs.*is degraded' | egrep -v 'filesystem is degraded' | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v 'filesystem is offline' | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v 'insufficient standby MDS daemons available' | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'online, but wants' | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'do not have an application enabled' | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'Degraded data redundancy' | head -n 1 2024-09-21T03:45:49.327 INFO:tasks.cephadm:Compressing logs... 2024-09-21T03:45:49.327 DEBUG:teuthology.orchestra.run.smithi089:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-21T03:45:49.370 DEBUG:teuthology.orchestra.run.smithi110:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-21T03:45:49.372 DEBUG:teuthology.orchestra.run.smithi114:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-21T03:45:49.393 INFO:teuthology.orchestra.run.smithi089.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.log 2024-09-21T03:45:49.393 INFO:teuthology.orchestra.run.smithi089.stderr:: No such file or directory 2024-09-21T03:45:49.394 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/7c374098-77c9-11ef-baf5-efdc52797490/ceph-osd.0.log 2024-09-21T03:45:49.394 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/7c374098-77c9-11ef-baf5-efdc52797490/ceph-osd.9.log 2024-09-21T03:45:49.398 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.log: gzip -5 --verbose -- /var/log/ceph/7c374098-77c9-11ef-baf5-efdc52797490/ceph-volume.log 2024-09-21T03:45:49.398 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/7c374098-77c9-11ef-baf5-efdc52797490/ceph-osd.0.log: /var/log/ceph/7c374098-77c9-11ef-baf5-efdc52797490/ceph-osd.9.log: gzip -5 --verbose -- /var/log/ceph/7c374098-77c9-11ef-baf5-efdc52797490/ceph.audit.log 2024-09-21T03:45:49.398 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/7c374098-77c9-11ef-baf5-efdc52797490/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/7c374098-77c9-11ef-baf5-efdc52797490/ceph-mds.a.log 2024-09-21T03:45:49.398 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/7c374098-77c9-11ef-baf5-efdc52797490/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/7c374098-77c9-11ef-baf5-efdc52797490/ceph.cephadm.log 2024-09-21T03:45:49.398 INFO:teuthology.orchestra.run.smithi089.stderr: 92.5% -- replaced with /var/log/ceph/ceph-client.admin.log.gz 2024-09-21T03:45:49.398 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/7c374098-77c9-11ef-baf5-efdc52797490/ceph-mon.a.log 2024-09-21T03:45:49.398 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/7c374098-77c9-11ef-baf5-efdc52797490/ceph-mds.a.log: /var/log/ceph/7c374098-77c9-11ef-baf5-efdc52797490/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/7c374098-77c9-11ef-baf5-efdc52797490/ceph-mds.g.log 2024-09-21T03:45:49.398 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/7c374098-77c9-11ef-baf5-efdc52797490/ceph-mon.a.log: 82.3% -- replaced with /var/log/ceph/7c374098-77c9-11ef-baf5-efdc52797490/ceph.cephadm.log.gz 2024-09-21T03:45:49.398 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/7c374098-77c9-11ef-baf5-efdc52797490/ceph-osd.6.log 2024-09-21T03:45:49.398 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/7c374098-77c9-11ef-baf5-efdc52797490/ceph.log 2024-09-21T03:45:49.398 INFO:teuthology.orchestra.run.smithi110.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/7c374098-77c9-11ef-baf5-efdc52797490/ceph-osd.10.log 2024-09-21T03:45:49.399 INFO:teuthology.orchestra.run.smithi110.stderr:: No such file or directory 2024-09-21T03:45:49.399 INFO:teuthology.orchestra.run.smithi110.stderr:gzip -5 --verbose -- /var/log/ceph/7c374098-77c9-11ef-baf5-efdc52797490/ceph-osd.1.log 2024-09-21T03:45:49.399 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/7c374098-77c9-11ef-baf5-efdc52797490/ceph-osd.10.log: gzip -5 --verbose -- /var/log/ceph/7c374098-77c9-11ef-baf5-efdc52797490/ceph-osd.7.log 2024-09-21T03:45:49.399 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/7c374098-77c9-11ef-baf5-efdc52797490/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/7c374098-77c9-11ef-baf5-efdc52797490/ceph-volume.log 2024-09-21T03:45:49.399 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/7c374098-77c9-11ef-baf5-efdc52797490/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/7c374098-77c9-11ef-baf5-efdc52797490/ceph-mds.b.log 2024-09-21T03:45:49.399 INFO:teuthology.orchestra.run.smithi110.stderr:gzip -5 --verbose -- /var/log/ceph/7c374098-77c9-11ef-baf5-efdc52797490/ceph-mds.k.log 2024-09-21T03:45:49.399 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/7c374098-77c9-11ef-baf5-efdc52797490/ceph-volume.log: /var/log/ceph/7c374098-77c9-11ef-baf5-efdc52797490/ceph-mds.b.log: gzip -5 --verbose -- /var/log/ceph/7c374098-77c9-11ef-baf5-efdc52797490/ceph-osd.4.log 2024-09-21T03:45:49.399 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/7c374098-77c9-11ef-baf5-efdc52797490/ceph-mds.k.log: gzip -5 --verbose -- /var/log/ceph/7c374098-77c9-11ef-baf5-efdc52797490/ceph.audit.log 2024-09-21T03:45:49.402 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/7c374098-77c9-11ef-baf5-efdc52797490/ceph-mds.g.log: /var/log/ceph/7c374098-77c9-11ef-baf5-efdc52797490/ceph-osd.6.log: 90.0% -- replaced with /var/log/ceph/7c374098-77c9-11ef-baf5-efdc52797490/ceph.audit.log.gz 2024-09-21T03:45:49.402 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/7c374098-77c9-11ef-baf5-efdc52797490/ceph-mds.d.log 2024-09-21T03:45:49.403 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/7c374098-77c9-11ef-baf5-efdc52797490/ceph.log: 95.0% -- replaced with /var/log/ceph/7c374098-77c9-11ef-baf5-efdc52797490/ceph-mds.a.log.gz 2024-09-21T03:45:49.403 INFO:teuthology.orchestra.run.smithi089.stderr: 86.5% -- replaced with /var/log/ceph/7c374098-77c9-11ef-baf5-efdc52797490/ceph.log.gz 2024-09-21T03:45:49.403 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/7c374098-77c9-11ef-baf5-efdc52797490/ceph-mgr.x.log 2024-09-21T03:45:49.403 INFO:teuthology.orchestra.run.smithi089.stderr: 95.0% -- replaced with /var/log/ceph/7c374098-77c9-11ef-baf5-efdc52797490/ceph-mds.g.log.gz 2024-09-21T03:45:49.403 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/7c374098-77c9-11ef-baf5-efdc52797490/ceph-mds.d.log: gzip -5 --verbose -- /var/log/ceph/7c374098-77c9-11ef-baf5-efdc52797490/ceph-mds.j.log 2024-09-21T03:45:49.403 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/7c374098-77c9-11ef-baf5-efdc52797490/ceph-osd.4.log: /var/log/ceph/7c374098-77c9-11ef-baf5-efdc52797490/ceph.audit.log: 90.2% -- replaced with /var/log/ceph/7c374098-77c9-11ef-baf5-efdc52797490/ceph.audit.log.gz 2024-09-21T03:45:49.403 INFO:teuthology.orchestra.run.smithi110.stderr:gzip -5 --verbose -- /var/log/ceph/7c374098-77c9-11ef-baf5-efdc52797490/ceph.cephadm.log 2024-09-21T03:45:49.403 INFO:teuthology.orchestra.run.smithi110.stderr:gzip -5 --verbose -- /var/log/ceph/7c374098-77c9-11ef-baf5-efdc52797490/ceph-mgr.y.log 2024-09-21T03:45:49.403 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/7c374098-77c9-11ef-baf5-efdc52797490/ceph.cephadm.log: 83.0% -- replaced with /var/log/ceph/7c374098-77c9-11ef-baf5-efdc52797490/ceph.cephadm.log.gz 2024-09-21T03:45:49.404 INFO:teuthology.orchestra.run.smithi110.stderr: 95.0% -- replaced with /var/log/ceph/7c374098-77c9-11ef-baf5-efdc52797490/ceph-mds.k.log.gz 2024-09-21T03:45:49.404 INFO:teuthology.orchestra.run.smithi110.stderr:gzip -5 --verbose -- /var/log/ceph/7c374098-77c9-11ef-baf5-efdc52797490/ceph.log 2024-09-21T03:45:49.406 INFO:teuthology.orchestra.run.smithi114.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/7c374098-77c9-11ef-baf5-efdc52797490/ceph-volume.log 2024-09-21T03:45:49.406 INFO:teuthology.orchestra.run.smithi114.stderr:: No such file or directory 2024-09-21T03:45:49.406 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/7c374098-77c9-11ef-baf5-efdc52797490/ceph.audit.log 2024-09-21T03:45:49.406 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/7c374098-77c9-11ef-baf5-efdc52797490/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/7c374098-77c9-11ef-baf5-efdc52797490/ceph-mds.i.log 2024-09-21T03:45:49.406 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/7c374098-77c9-11ef-baf5-efdc52797490/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/7c374098-77c9-11ef-baf5-efdc52797490/ceph-mon.c.log 2024-09-21T03:45:49.406 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/7c374098-77c9-11ef-baf5-efdc52797490/ceph-mds.i.log: gzip -5 --verbose -- /var/log/ceph/7c374098-77c9-11ef-baf5-efdc52797490/ceph.cephadm.log 2024-09-21T03:45:49.406 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/7c374098-77c9-11ef-baf5-efdc52797490/ceph-osd.11.log 2024-09-21T03:45:49.407 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/7c374098-77c9-11ef-baf5-efdc52797490/ceph-mon.c.log: /var/log/ceph/7c374098-77c9-11ef-baf5-efdc52797490/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/7c374098-77c9-11ef-baf5-efdc52797490/ceph-osd.2.log 2024-09-21T03:45:49.407 INFO:teuthology.orchestra.run.smithi114.stderr: 82.9% -- replaced with /var/log/ceph/7c374098-77c9-11ef-baf5-efdc52797490/ceph.cephadm.log.gz 2024-09-21T03:45:49.407 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/7c374098-77c9-11ef-baf5-efdc52797490/ceph-osd.11.log: gzip -5 --verbose -- /var/log/ceph/7c374098-77c9-11ef-baf5-efdc52797490/ceph-osd.5.log 2024-09-21T03:45:49.407 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/7c374098-77c9-11ef-baf5-efdc52797490/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/7c374098-77c9-11ef-baf5-efdc52797490/ceph.log 2024-09-21T03:45:49.407 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/7c374098-77c9-11ef-baf5-efdc52797490/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/7c374098-77c9-11ef-baf5-efdc52797490/ceph-mds.l.log 2024-09-21T03:45:49.407 INFO:teuthology.orchestra.run.smithi114.stderr: 90.2% -- replaced with /var/log/ceph/7c374098-77c9-11ef-baf5-efdc52797490/ceph.audit.log.gz 2024-09-21T03:45:49.407 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/7c374098-77c9-11ef-baf5-efdc52797490/ceph.log: 86.5% -- replaced with /var/log/ceph/7c374098-77c9-11ef-baf5-efdc52797490/ceph.log.gz 2024-09-21T03:45:49.407 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/7c374098-77c9-11ef-baf5-efdc52797490/ceph-osd.8.log 2024-09-21T03:45:49.407 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/7c374098-77c9-11ef-baf5-efdc52797490/ceph-mds.l.log: gzip -5 --verbose -- /var/log/ceph/7c374098-77c9-11ef-baf5-efdc52797490/ceph-mgr.z.log 2024-09-21T03:45:49.408 INFO:teuthology.orchestra.run.smithi110.stderr:gzip -5 --verbose -- /var/log/ceph/7c374098-77c9-11ef-baf5-efdc52797490/ceph-mds.h.log 2024-09-21T03:45:49.408 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/7c374098-77c9-11ef-baf5-efdc52797490/ceph-mgr.y.log: /var/log/ceph/7c374098-77c9-11ef-baf5-efdc52797490/ceph.log: 86.5% -- replaced with /var/log/ceph/7c374098-77c9-11ef-baf5-efdc52797490/ceph.log.gz 2024-09-21T03:45:49.408 INFO:teuthology.orchestra.run.smithi110.stderr:gzip -5 --verbose -- /var/log/ceph/7c374098-77c9-11ef-baf5-efdc52797490/ceph-mon.b.log 2024-09-21T03:45:49.409 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/7c374098-77c9-11ef-baf5-efdc52797490/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/7c374098-77c9-11ef-baf5-efdc52797490/ceph-osd.3.log 2024-09-21T03:45:49.409 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/7c374098-77c9-11ef-baf5-efdc52797490/ceph-mds.j.log: 95.0% -- replaced with /var/log/ceph/7c374098-77c9-11ef-baf5-efdc52797490/ceph-mds.d.log.gz 2024-09-21T03:45:49.409 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.log 2024-09-21T03:45:49.409 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/7c374098-77c9-11ef-baf5-efdc52797490/ceph-osd.3.log: 95.0% -- replaced with /var/log/ceph/7c374098-77c9-11ef-baf5-efdc52797490/ceph-mds.j.log.gz 2024-09-21T03:45:49.409 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-09-21T03:45:49.409 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.0.log: 90.9% -- replaced with /var/log/ceph/7c374098-77c9-11ef-baf5-efdc52797490/ceph-volume.log.gz 2024-09-21T03:45:49.411 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/7c374098-77c9-11ef-baf5-efdc52797490/ceph-osd.8.log: 95.1% -- replaced with /var/log/ceph/7c374098-77c9-11ef-baf5-efdc52797490/ceph-mds.l.log.gz 2024-09-21T03:45:49.411 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/7c374098-77c9-11ef-baf5-efdc52797490/ceph-mds.c.log 2024-09-21T03:45:49.412 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/7c374098-77c9-11ef-baf5-efdc52797490/ceph-mgr.z.log: 90.9% -- replaced with /var/log/ceph/7c374098-77c9-11ef-baf5-efdc52797490/ceph-volume.log.gz 2024-09-21T03:45:49.412 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/7c374098-77c9-11ef-baf5-efdc52797490/ceph-mds.f.log 2024-09-21T03:45:49.412 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/7c374098-77c9-11ef-baf5-efdc52797490/ceph-mds.h.log: 91.8% -- replaced with /var/log/ceph/7c374098-77c9-11ef-baf5-efdc52797490/ceph-mgr.y.log.gz 2024-09-21T03:45:49.412 INFO:teuthology.orchestra.run.smithi110.stderr:gzip -5 --verbose -- /var/log/ceph/7c374098-77c9-11ef-baf5-efdc52797490/ceph-mds.e.log 2024-09-21T03:45:49.412 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/7c374098-77c9-11ef-baf5-efdc52797490/ceph-mon.b.log: 90.9% -- replaced with /var/log/ceph/7c374098-77c9-11ef-baf5-efdc52797490/ceph-volume.log.gz 2024-09-21T03:45:49.414 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/7c374098-77c9-11ef-baf5-efdc52797490/ceph-mds.c.log: 92.0% -- replaced with /var/log/ceph/7c374098-77c9-11ef-baf5-efdc52797490/ceph-mgr.z.log.gz 2024-09-21T03:45:49.414 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-09-21T03:45:49.423 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/cephadm.log: 93.2% -- replaced with /var/log/ceph/cephadm.log.gz 2024-09-21T03:45:49.424 INFO:teuthology.orchestra.run.smithi110.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-09-21T03:45:49.426 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/7c374098-77c9-11ef-baf5-efdc52797490/ceph-mds.e.log: /var/log/ceph/cephadm.log: 91.9% -- replaced with /var/log/ceph/cephadm.log.gz 2024-09-21T03:45:49.427 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/7c374098-77c9-11ef-baf5-efdc52797490/ceph-mds.f.log: /var/log/ceph/cephadm.log: 92.2% -- replaced with /var/log/ceph/cephadm.log.gz 2024-09-21T03:45:49.450 INFO:teuthology.orchestra.run.smithi114.stderr: 92.6% -- replaced with /var/log/ceph/7c374098-77c9-11ef-baf5-efdc52797490/ceph-mds.i.log.gz 2024-09-21T03:45:49.452 INFO:teuthology.orchestra.run.smithi110.stderr: 92.7% -- replaced with /var/log/ceph/7c374098-77c9-11ef-baf5-efdc52797490/ceph-mds.h.log.gz 2024-09-21T03:45:49.460 INFO:teuthology.orchestra.run.smithi114.stderr: 94.1% -- replaced with /var/log/ceph/7c374098-77c9-11ef-baf5-efdc52797490/ceph-mds.f.log.gz 2024-09-21T03:45:49.470 INFO:teuthology.orchestra.run.smithi110.stderr: 93.9% -- replaced with /var/log/ceph/7c374098-77c9-11ef-baf5-efdc52797490/ceph-mds.e.log.gz 2024-09-21T03:45:49.486 INFO:teuthology.orchestra.run.smithi114.stderr: 93.3% -- replaced with /var/log/ceph/7c374098-77c9-11ef-baf5-efdc52797490/ceph-mds.c.log.gz 2024-09-21T03:45:49.553 INFO:teuthology.orchestra.run.smithi089.stderr: 90.1% -- replaced with /var/log/ceph/7c374098-77c9-11ef-baf5-efdc52797490/ceph-mgr.x.log.gz 2024-09-21T03:45:49.632 INFO:teuthology.orchestra.run.smithi114.stderr: 91.7% -- replaced with /var/log/ceph/7c374098-77c9-11ef-baf5-efdc52797490/ceph-mon.c.log.gz 2024-09-21T03:45:49.716 INFO:teuthology.orchestra.run.smithi110.stderr: 91.8% -- replaced with /var/log/ceph/7c374098-77c9-11ef-baf5-efdc52797490/ceph-mon.b.log.gz 2024-09-21T03:45:49.718 INFO:teuthology.orchestra.run.smithi110.stderr: 92.8% -- replaced with /var/log/ceph/7c374098-77c9-11ef-baf5-efdc52797490/ceph-mds.b.log.gz 2024-09-21T03:45:50.030 INFO:teuthology.orchestra.run.smithi089.stderr: 90.6% -- replaced with /var/log/ceph/7c374098-77c9-11ef-baf5-efdc52797490/ceph-mon.a.log.gz 2024-09-21T03:45:59.661 INFO:teuthology.orchestra.run.smithi089.stderr: 96.1% -- replaced with /var/log/ceph/ceph-client.0.log.gz 2024-09-21T03:45:59.756 INFO:teuthology.orchestra.run.smithi114.stderr: 90.8% -- replaced with /var/log/ceph/7c374098-77c9-11ef-baf5-efdc52797490/ceph-osd.8.log.gz 2024-09-21T03:46:02.121 INFO:teuthology.orchestra.run.smithi110.stderr: 90.9% -- replaced with /var/log/ceph/7c374098-77c9-11ef-baf5-efdc52797490/ceph-osd.1.log.gz 2024-09-21T03:46:02.590 INFO:teuthology.orchestra.run.smithi089.stderr: 90.8% -- replaced with /var/log/ceph/7c374098-77c9-11ef-baf5-efdc52797490/ceph-osd.6.log.gz 2024-09-21T03:46:02.880 INFO:teuthology.orchestra.run.smithi110.stderr: 91.0% -- replaced with /var/log/ceph/7c374098-77c9-11ef-baf5-efdc52797490/ceph-osd.10.log.gz 2024-09-21T03:46:04.424 INFO:teuthology.orchestra.run.smithi114.stderr: 91.0% -- replaced with /var/log/ceph/7c374098-77c9-11ef-baf5-efdc52797490/ceph-osd.11.log.gz 2024-09-21T03:46:05.771 INFO:teuthology.orchestra.run.smithi089.stderr: 90.8% -- replaced with /var/log/ceph/7c374098-77c9-11ef-baf5-efdc52797490/ceph-osd.0.log.gz 2024-09-21T03:46:06.185 INFO:teuthology.orchestra.run.smithi110.stderr: 90.8% -- replaced with /var/log/ceph/7c374098-77c9-11ef-baf5-efdc52797490/ceph-osd.7.log.gz 2024-09-21T03:46:06.338 INFO:teuthology.orchestra.run.smithi114.stderr: 91.0% -- replaced with /var/log/ceph/7c374098-77c9-11ef-baf5-efdc52797490/ceph-osd.2.log.gz 2024-09-21T03:46:09.645 INFO:teuthology.orchestra.run.smithi110.stderr: 90.9% -- replaced with /var/log/ceph/7c374098-77c9-11ef-baf5-efdc52797490/ceph-osd.4.log.gz 2024-09-21T03:46:09.647 INFO:teuthology.orchestra.run.smithi110.stderr: 2024-09-21T03:46:09.647 INFO:teuthology.orchestra.run.smithi110.stderr:real 0m20.263s 2024-09-21T03:46:09.647 INFO:teuthology.orchestra.run.smithi110.stderr:user 1m1.926s 2024-09-21T03:46:09.647 INFO:teuthology.orchestra.run.smithi110.stderr:sys 0m1.988s 2024-09-21T03:46:10.195 INFO:teuthology.orchestra.run.smithi089.stderr: 90.5% -- replaced with /var/log/ceph/7c374098-77c9-11ef-baf5-efdc52797490/ceph-osd.3.log.gz 2024-09-21T03:46:13.078 INFO:teuthology.orchestra.run.smithi114.stderr: 90.8% -- replaced with /var/log/ceph/7c374098-77c9-11ef-baf5-efdc52797490/ceph-osd.5.log.gz 2024-09-21T03:46:13.079 INFO:teuthology.orchestra.run.smithi114.stderr: 2024-09-21T03:46:13.079 INFO:teuthology.orchestra.run.smithi114.stderr:real 0m23.693s 2024-09-21T03:46:13.079 INFO:teuthology.orchestra.run.smithi114.stderr:user 1m4.407s 2024-09-21T03:46:13.079 INFO:teuthology.orchestra.run.smithi114.stderr:sys 0m1.924s 2024-09-21T03:46:15.878 INFO:teuthology.orchestra.run.smithi089.stderr: 90.5% -- replaced with /var/log/ceph/7c374098-77c9-11ef-baf5-efdc52797490/ceph-osd.9.log.gz 2024-09-21T03:46:15.881 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-09-21T03:46:15.881 INFO:teuthology.orchestra.run.smithi089.stderr:real 0m26.498s 2024-09-21T03:46:15.881 INFO:teuthology.orchestra.run.smithi089.stderr:user 1m25.213s 2024-09-21T03:46:15.881 INFO:teuthology.orchestra.run.smithi089.stderr:sys 0m2.573s 2024-09-21T03:46:15.881 INFO:tasks.cephadm:Archiving logs... 2024-09-21T03:46:15.881 DEBUG:teuthology.misc:Transferring archived files from smithi089:/var/log/ceph to /home/teuthworker/archive/pdonnell-2024-09-20_23:48:22-fs-wip-pdonnell-testing-20240920.212106-debug-distro-default-smithi/7913466/remote/smithi089/log 2024-09-21T03:46:15.882 DEBUG:teuthology.orchestra.run.smithi089:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-21T03:46:29.150 DEBUG:teuthology.misc:Transferring archived files from smithi110:/var/log/ceph to /home/teuthworker/archive/pdonnell-2024-09-20_23:48:22-fs-wip-pdonnell-testing-20240920.212106-debug-distro-default-smithi/7913466/remote/smithi110/log 2024-09-21T03:46:29.151 DEBUG:teuthology.orchestra.run.smithi110:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-21T03:46:39.672 DEBUG:teuthology.misc:Transferring archived files from smithi114:/var/log/ceph to /home/teuthworker/archive/pdonnell-2024-09-20_23:48:22-fs-wip-pdonnell-testing-20240920.212106-debug-distro-default-smithi/7913466/remote/smithi114/log 2024-09-21T03:46:39.673 DEBUG:teuthology.orchestra.run.smithi114:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-21T03:46:51.987 INFO:tasks.cephadm:Removing cluster... 2024-09-21T03:46:51.988 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 7c374098-77c9-11ef-baf5-efdc52797490 --force 2024-09-21T03:46:52.192 INFO:teuthology.orchestra.run.smithi089.stdout:Deleting cluster with fsid: 7c374098-77c9-11ef-baf5-efdc52797490 2024-09-21T03:46:52.561 DEBUG:teuthology.orchestra.run.smithi110:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 7c374098-77c9-11ef-baf5-efdc52797490 --force 2024-09-21T03:46:52.764 INFO:teuthology.orchestra.run.smithi110.stdout:Deleting cluster with fsid: 7c374098-77c9-11ef-baf5-efdc52797490 2024-09-21T03:46:53.190 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 7c374098-77c9-11ef-baf5-efdc52797490 --force 2024-09-21T03:46:53.392 INFO:teuthology.orchestra.run.smithi114.stdout:Deleting cluster with fsid: 7c374098-77c9-11ef-baf5-efdc52797490 2024-09-21T03:46:53.752 INFO:tasks.cephadm:Removing cephadm ... 2024-09-21T03:46:53.753 DEBUG:teuthology.orchestra.run.smithi089:> rm -rf /home/ubuntu/cephtest/cephadm 2024-09-21T03:46:53.772 DEBUG:teuthology.orchestra.run.smithi110:> rm -rf /home/ubuntu/cephtest/cephadm 2024-09-21T03:46:53.789 DEBUG:teuthology.orchestra.run.smithi114:> rm -rf /home/ubuntu/cephtest/cephadm 2024-09-21T03:46:53.804 INFO:tasks.cephadm:Teardown complete 2024-09-21T03:46:53.804 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-09-21T03:46:53.821 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-09-21T03:46:53.821 DEBUG:teuthology.orchestra.run.smithi089:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-21T03:46:53.824 DEBUG:teuthology.orchestra.run.smithi110:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-21T03:46:53.831 DEBUG:teuthology.orchestra.run.smithi114:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-21T03:46:53.939 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-09-21T03:46:53.950 INFO:teuthology.task.clock:Checking final clock skew... 2024-09-21T03:46:53.950 DEBUG:teuthology.orchestra.run.smithi089:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-21T03:46:53.952 DEBUG:teuthology.orchestra.run.smithi110:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-21T03:46:53.955 DEBUG:teuthology.orchestra.run.smithi114:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-21T03:46:53.967 INFO:teuthology.orchestra.run.smithi110.stderr:bash: line 1: ntpq: command not found 2024-09-21T03:46:53.967 INFO:teuthology.orchestra.run.smithi089.stderr:bash: line 1: ntpq: command not found 2024-09-21T03:46:53.969 INFO:teuthology.orchestra.run.smithi114.stderr:bash: line 1: ntpq: command not found 2024-09-21T03:46:53.971 INFO:teuthology.orchestra.run.smithi110.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-21T03:46:53.971 INFO:teuthology.orchestra.run.smithi110.stdout:=============================================================================== 2024-09-21T03:46:53.971 INFO:teuthology.orchestra.run.smithi110.stdout:^* hv01.front.sepia.ceph.com 3 6 377 12 -492us[ -487us] +/- 60ms 2024-09-21T03:46:53.971 INFO:teuthology.orchestra.run.smithi110.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 10 +625us[ +625us] +/- 57ms 2024-09-21T03:46:53.971 INFO:teuthology.orchestra.run.smithi110.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 12 -272us[ -272us] +/- 59ms 2024-09-21T03:46:53.971 INFO:teuthology.orchestra.run.smithi110.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-09-21T03:46:53.972 INFO:teuthology.orchestra.run.smithi089.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-21T03:46:53.973 INFO:teuthology.orchestra.run.smithi089.stdout:=============================================================================== 2024-09-21T03:46:53.973 INFO:teuthology.orchestra.run.smithi089.stdout:^* hv01.front.sepia.ceph.com 3 6 377 10 -456us[ -473us] +/- 60ms 2024-09-21T03:46:53.973 INFO:teuthology.orchestra.run.smithi089.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 10 +675us[ +675us] +/- 57ms 2024-09-21T03:46:53.973 INFO:teuthology.orchestra.run.smithi089.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 12 -245us[ -262us] +/- 59ms 2024-09-21T03:46:53.973 INFO:teuthology.orchestra.run.smithi089.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-09-21T03:46:53.974 INFO:teuthology.orchestra.run.smithi114.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-21T03:46:53.974 INFO:teuthology.orchestra.run.smithi114.stdout:=============================================================================== 2024-09-21T03:46:53.974 INFO:teuthology.orchestra.run.smithi114.stdout:^* hv01.front.sepia.ceph.com 3 6 377 9 -395us[ -412us] +/- 60ms 2024-09-21T03:46:53.974 INFO:teuthology.orchestra.run.smithi114.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 13 +678us[ +661us] +/- 57ms 2024-09-21T03:46:53.974 INFO:teuthology.orchestra.run.smithi114.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 11 -225us[ -241us] +/- 59ms 2024-09-21T03:46:53.974 INFO:teuthology.orchestra.run.smithi114.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-09-21T03:46:53.974 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-09-21T03:46:53.985 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-09-21T03:46:53.985 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-09-21T03:46:54.024 DEBUG:teuthology.orchestra.run.smithi089:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-21T03:46:54.104 DEBUG:teuthology.orchestra.run.smithi110:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-21T03:46:54.169 DEBUG:teuthology.orchestra.run.smithi114:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-21T03:46:54.235 DEBUG:teuthology.orchestra.run.smithi089:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 2024-09-21T03:46:54.265 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T03:46:54.265 DEBUG:teuthology.orchestra.run.smithi110:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 2024-09-21T03:46:54.293 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T03:46:54.294 DEBUG:teuthology.orchestra.run.smithi114:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 2024-09-21T03:46:54.321 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T03:46:54.322 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-09-21T03:46:54.332 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-09-21T03:46:54.353 INFO:teuthology.task.internal:Duration was 1481.049148 seconds 2024-09-21T03:46:54.353 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-09-21T03:46:54.424 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-09-21T03:46:54.424 DEBUG:teuthology.orchestra.run.smithi089:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-21T03:46:54.427 DEBUG:teuthology.orchestra.run.smithi110:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-21T03:46:54.429 DEBUG:teuthology.orchestra.run.smithi114:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-21T03:46:54.466 INFO:teuthology.orchestra.run.smithi110.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-21T03:46:54.466 INFO:teuthology.orchestra.run.smithi089.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-21T03:46:54.468 INFO:teuthology.orchestra.run.smithi114.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-21T03:46:54.791 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-09-21T03:46:54.792 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi089.front.sepia.ceph.com 2024-09-21T03:46:54.792 DEBUG:teuthology.orchestra.run.smithi089:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-21T03:46:54.813 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi110.front.sepia.ceph.com 2024-09-21T03:46:54.813 DEBUG:teuthology.orchestra.run.smithi110:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-21T03:46:54.859 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi114.front.sepia.ceph.com 2024-09-21T03:46:54.860 DEBUG:teuthology.orchestra.run.smithi114:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-21T03:46:54.881 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-09-21T03:46:54.881 DEBUG:teuthology.orchestra.run.smithi089:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-21T03:46:54.884 DEBUG:teuthology.orchestra.run.smithi110:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-21T03:46:54.903 DEBUG:teuthology.orchestra.run.smithi114:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-21T03:46:54.948 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-09-21T03:46:54.948 DEBUG:teuthology.orchestra.run.smithi089:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-21T03:46:54.951 DEBUG:teuthology.orchestra.run.smithi110:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-21T03:46:54.974 DEBUG:teuthology.orchestra.run.smithi114:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-21T03:46:55.554 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-09-21T03:46:55.566 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-09-21T03:46:55.566 DEBUG:teuthology.orchestra.run.smithi089:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-21T03:46:55.591 DEBUG:teuthology.orchestra.run.smithi110:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-21T03:46:55.617 DEBUG:teuthology.orchestra.run.smithi114:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-21T03:46:55.647 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-09-21T03:46:55.698 DEBUG:teuthology.orchestra.run.smithi089:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-21T03:46:55.701 DEBUG:teuthology.orchestra.run.smithi110:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-21T03:46:55.705 DEBUG:teuthology.orchestra.run.smithi114:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-21T03:46:55.722 INFO:teuthology.orchestra.run.smithi089.stdout:kernel.core_pattern = core 2024-09-21T03:46:55.726 INFO:teuthology.orchestra.run.smithi110.stdout:kernel.core_pattern = core 2024-09-21T03:46:55.730 INFO:teuthology.orchestra.run.smithi114.stdout:kernel.core_pattern = core 2024-09-21T03:46:55.760 DEBUG:teuthology.orchestra.run.smithi089:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-21T03:46:55.788 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T03:46:55.789 DEBUG:teuthology.orchestra.run.smithi110:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-21T03:46:55.805 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T03:46:55.806 DEBUG:teuthology.orchestra.run.smithi114:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-21T03:46:55.823 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T03:46:55.824 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-09-21T03:46:55.839 INFO:teuthology.task.internal:Transferring archived files... 2024-09-21T03:46:55.840 DEBUG:teuthology.misc:Transferring archived files from smithi089:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/pdonnell-2024-09-20_23:48:22-fs-wip-pdonnell-testing-20240920.212106-debug-distro-default-smithi/7913466/remote/smithi089 2024-09-21T03:46:55.840 DEBUG:teuthology.orchestra.run.smithi089:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-21T03:46:55.906 DEBUG:teuthology.misc:Transferring archived files from smithi110:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/pdonnell-2024-09-20_23:48:22-fs-wip-pdonnell-testing-20240920.212106-debug-distro-default-smithi/7913466/remote/smithi110 2024-09-21T03:46:55.906 DEBUG:teuthology.orchestra.run.smithi110:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-21T03:46:55.949 DEBUG:teuthology.misc:Transferring archived files from smithi114:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/pdonnell-2024-09-20_23:48:22-fs-wip-pdonnell-testing-20240920.212106-debug-distro-default-smithi/7913466/remote/smithi114 2024-09-21T03:46:55.949 DEBUG:teuthology.orchestra.run.smithi114:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-21T03:46:55.989 INFO:teuthology.task.internal:Removing archive directory... 2024-09-21T03:46:55.989 DEBUG:teuthology.orchestra.run.smithi089:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-21T03:46:55.992 DEBUG:teuthology.orchestra.run.smithi110:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-21T03:46:55.995 DEBUG:teuthology.orchestra.run.smithi114:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-21T03:46:56.036 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-09-21T03:46:56.050 INFO:teuthology.task.internal:Not uploading archives. 2024-09-21T03:46:56.050 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-09-21T03:46:56.063 INFO:teuthology.task.internal:Tidying up after the test... 2024-09-21T03:46:56.063 DEBUG:teuthology.orchestra.run.smithi089:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-21T03:46:56.065 DEBUG:teuthology.orchestra.run.smithi110:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-21T03:46:56.067 DEBUG:teuthology.orchestra.run.smithi114:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-21T03:46:56.078 INFO:teuthology.orchestra.run.smithi089.stdout: 265340 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 21 03:46 /home/ubuntu/cephtest 2024-09-21T03:46:56.080 INFO:teuthology.orchestra.run.smithi110.stdout: 262169 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 21 03:46 /home/ubuntu/cephtest 2024-09-21T03:46:56.091 INFO:teuthology.orchestra.run.smithi114.stdout: 266063 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 21 03:46 /home/ubuntu/cephtest 2024-09-21T03:46:56.092 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-09-21T03:46:56.115 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-09-21T03:46:56.163 INFO:teuthology.run:Summary data: description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/fuse objectstore-ec/bluestore-comp-ec-root omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/random export-check n/3 replication/default} standby-replay tasks/{0-subvolume/{with-no-extra-options} 1-check-counter 2-scrub/no 3-snaps/no 4-flush/yes 5-quiesce/with-quiesce 6-workunit/suites/fsync-tester}} duration: 1481.049147605896 flavor: default owner: scheduled_pdonnell@teuthology success: true 2024-09-21T03:46:56.164 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-21T03:46:56.243 INFO:teuthology.run:pass